Few-shot fruit segmentation via transfer learning

Authors: JORDAN, JAMES - University Of Texas At Arlington; MANCHING, HEATHER - North Carolina State University; Hulse-Kemp, Amanda; BEKSI, WILLIAM - University Of Texas At Arlington

Submitted to: IEEE International Conference on Robotics and Automation
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 3/10/2024
Publication Date: 5/13/2024
Citation: Jordan, J.A., Manching, H.K., Hulse-Kemp, A.M., Beksi, W.J. 2024. Few-shot fruit segmentation via transfer learning. IEEE International Conference on Robotics and Automation. https://doi.org/10.1109/ICRA57147.2024.10610003.
DOI: https://doi.org/10.1109/ICRA57147.2024.10610003

Interpretive Summary: Tools for determining characteristics of plants in an automated fashion are lacking in many important agricultural systems and the data necessary to build new automated tools is very costly and time consuming. In this study, we tested the ability to leverage a model built in one cropping system with limited amount of data into another cropping system, a process called transfer learning. Utilizing only a few images in the new cropping system (i.e. few-shot), we showed that it is possible to accomplish the model task developed in the original system, thus minimizing cost and time to develop a new model from scratch.

Technical Abstract: Advancements in machine learning, computer vision, and robotics have paved the way for transformative solutions in various domains, particularly in agriculture. For example, accurate identification and segmentation of fruits from field images plays a crucial role in automating jobs such as harvesting, disease detection, and yield estimation. However, achieving robust and precise infield fruit segmentation remains a challenging task since large amounts of labeled data are required to handle variations in fruit size, shape, color, and occlusion. In this paper, we develop a few-shot semantic segmentation framework for infield fruits using transfer learning. Concretely, our work is aimed at addressing agricultural domains that lack publicly available labeled data. Motivated by similar success in urban scene parsing, we propose specialized pre-training using a public benchmark dataset for fruit transfer learning. By leveraging pre-trained neural networks, accurate semantic segmentation of fruit in the field is achieved with only a few labeled images. Furthermore, we show that models with pre-training learn to distinguish between fruit still on the trees and fruit that have fallen on the ground, and they can effectively transfer the knowledge to the target fruit dataset.