In-field multi-ripeness blackberry detection for soft robotic harvesting

Authors: THAYANANTHAN, THEVATHAYARAJH - University Of Georgia; ZHANG, XIN - University Of Georgia; HARJONO, JONATHAN - University Of Georgia; Huang, Yanbo; LIU, WENBO - Mississippi State University; MCWHIRT, AMANDA - University Of Arkansas; THRELFALL, RENEE - University Of Arkansas; CHEN, YUE - Georgia Institute Of Technology

Submitted to: Journal of the ASABE
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 10/13/2025
Publication Date: 11/18/2025
Citation: Thayananthan, T., Zhang, X., Mcwhrit, A.L., Threlfall, R.T., Lui, W., Huang, Y., Zhao, Y., Gunderma, A.L., Chen, Y. 2025. In-field multi-ripeness blackberry detection for soft robotic harvesting. Smart Agricultural Technology. 68(6):1073-1089. https://doi.org/10.13031/ja.16300.
DOI: https://doi.org/10.13031/ja.16300

Interpretive Summary: Blackberry harvesting encounters several major challenges in the U.S., including the shortage of agricultural labor and non-secured postharvest fruit quality. Developing a computer vision-enabled, robotic blackberry selective-picking system can provide an alternative solution to mitigate the issues and stresses, so that the profitability of fresh blackberry growers can be secured. Scientists of Mississippi State University, University of Arkansas, USDA ARS, Geogoria Tech University, and Emory University collectively assessed and compared the feasibility, accuracy, and efficiency of a series of state-of-the-art YOLO (You Only Look Once) deep learning artificial intelligence models in detecting multiripeness blackberries in the farm conditions. The results are informative to provide the optimal detection model for developing soft robotic harvesting for in-field blackberry detection and localization.

Technical Abstract: Blackberries are a type of high-value specialty berry crop composed of aggregated black/dark purple color drupelets. Their distinctive burst of sweet and tart flavors positions them as one of the most popular fruits within the realm of berry production in the world. Blackberry harvesting is a crucial step for the fresh market production, requiring multiple passes of hand-picking because the berries do not ripen simultaneously, even on a plant, during the harvesting season. The blackberry harvesting process encounters several major hurdles in the U.S., including the shortage of agricultural labor and postharvest fruit quality since the blackberries are highly delicate and sensitive. Developing a computer vision-enabled, robotic blackberry selective-picking system can provide an alternative solution to mitigate the issues and stresses, so that the profitability of fresh blackberry growers can be secured. In-field blackberry detection and localization are extremely challenging attributing to several driving-factors, such as small size of the target berries, multiple levels of berry ripeness, and great variation of outdoor lighting conditions. This study aims to assess and compare the feasibility, accuracy, and efficiency of a series of state-of-the-art YOLO (You Only Look Once) models in detecting multiripeness blackberries in the farm conditions. A total of 1,086 images were acquired from different blackberry farms in the U.S. states of Arkansas and Georgia, employing three different cameras across two years. Three different ripeness levels of blackberries were observed during the harvesting season, and were thus predefined during the annotation process, including the ripe (in black color), ripening (in pink color), and unripe berries (in green color). The computer vision pipeline developed in this study had the ability to detect and localize all berries at different ripeness levels, while detecting the ripe berries only was a particular focus. Overall, eight YOLO models (i.e., YOLOv5-x6, YOLOv6-l6, YOLOv7-base, YOLOv7-x, YOLOv7-e6e, YOLOv8-l, YOLOv8-x, and the fine-tuned version of YOLOv8-x) were trained and validated using randomly selected 809 (74%) and 193 images (18%), respectively. Among all, YOLOv7-x outperformed all other models and configurations on the test set, containing a totality of 84 images (8%). More specifically, YOLOv7-x achieved the mean Average Precision (mAP) of 92.6%, F1-score of 86.4%, and inference speed of 12.6 ms per image with 1,024x1,024 pixels across all classes of ripeness. In addition, its mAP on the ripe berries was 95.2%, making YOLOv7-x a reliable tool for near real-time, in-field blackberry detection for soft robotic selective harvesting.