Skip to main content
ARS Home » Northeast Area » Kearneysville, West Virginia » Appalachian Fruit Research Laboratory » Innovative Fruit Production, Improvement, and Protection » Research » Publications at this Location » Publication #353280
Title: Multispecies fruit flower detection using a refined semantic segmentation network

Author
item DIAS, PHILIPE - Marquette University
item Tabb, Amy
item MEDEIROS, HENRY - Marquette University

Submitted to: International of Electrical and Electronics Engineers (IEEE) Robotics and Automation Letters
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 6/10/2018
Publication Date: 6/22/2018
Citation: Dias, P., Tabb, A., Medeiros, H. 2018. Multispecies fruit flower detection using a refined semantic segmentation network. International of Electrical and Electronics Engineers (IEEE) Robotics and Automation Letters. 3(4):3003-3010. https://doi.org/10.1109/LRA.2018.2849498.
DOI: https://doi.org/10.1109/LRA.2018.2849498

Interpretive Summary: Fruit trees currently produce many more blossoms than are needed for optimal fruit production and as a result, an estimate of the number of blossoms is required to plan for adequate removal (called thinning). Currently, the method for estimating the number of blossoms is manual inspection. This paper presents an automatic method for estimating the number of blossoms in an image using convolutional neural networks. This work is novel in that coarse segmentations provided by convolutional neural networks are refined for more accurate pixel-wise segmentations, and the method is robust to new environments and settings. The future impact of this work is increased accuracy and speed for bloom estimation in orchard settings.

Technical Abstract: In fruit production, critical crop management decisions are guided by bloom intensity, i.e. the number of flowers present in an orchard. Despite its importance, bloom intensity is still typically estimated by means of human visual inspection. Existing automated computer vision systems for flower identification are based on hand-engineered techniques that work only under specific conditions and with limited performance. This work proposes an automated technique for flower identification that is robust to uncontrolled environments and applicable to different flower species. Our method relies on an end-to-end residual convolutional neural network (CNN) that represents the state-of-the-art in semantic segmentation. To enhance its sensitivity to flowers, we fine-tune this network using a single dataset of apple flower images. Since CNNs tend to produce coarse segmentations, we employ a refinement method to better distinguish between individual flower instances. Without any pre-processing or dataset-specific training, experimental results on images of apple, peach and pear flowers, acquired under different conditions, demonstrate the robustness and broad applicability of our method.