AR Grape Thinning Support

Abstract

Recent advancements in deep neural networks (DNN) and augmented reality (AR) have improved the efficiency and automation of agriculture. This study proposes an AR grape thinning support system to assist in grape thinning operations. The proposed system uses DNN to predict grape berries that need to be thinned and uses the optical see-through Head-Mounted Display (HMD) HoloLens 2 to superimpose contour information over the real target berry, making it easy for users to identify the berry to be thinned. Additionally, the hand-tracking function of HoloLens 2 is utilized to monitor the thinning operation in real-time and provide voice instructions to improve work efficiency and user experience. The evaluation experiment compared three interfaces: "image only", "image with contour overlay", and "image with contour overlay and voice instructions", evaluating using the metric of time taken to thin one grape cluster and the usability and user experience. The results showed that the image with contour overlay and voice instructions could significantly improve usability.