CottonSim: A vision-guided autonomous robotic system for cotton harvesting in Gazebo simulation

Authors: THAYANANTHAN, THEVATHAYARAJH - Mississippi State University; ZHANG, XIN - Mississippi State University; Huang, Yanbo; CHEN, JINGDAO - Mississippi State University; WIJEWARDANE, NUWAN - Mississippi State University; MARTINS, VITOR - Mississippi State University; CHESSER, GARY - North Carolina State University; GOODIN, CHRISTOPHER - Mississippi State University

Submitted to: Computers and Electronics in Agriculture
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 9/3/2025
Publication Date: 10/6/2025
Citation: Thayananthan, T., Zhang, X., Huang, Y., Chen, J., Wijewardane, N.K., Martins, V.S., Chesser, G.D., Goodin, C.T. 2025. CottonSim: A vision-guided autonomous robotic system for cotton harvesting in Gazebo simulation. Computers and Electronics in Agriculture. 235(2025)100963:1-16. https://doi.org/10.1016/j.compag.2025.110963.
DOI: https://doi.org/10.1016/j.compag.2025.110963

Interpretive Summary: Artificial intelligence is important for enhancing agricultural systems. This research was to develop an autonomous visual-guided robotic cotton-picking system in a software environment for simulating autonomous field navigation with a virtual cotton farm. The model achieved desired mean accuracy and precision with scene segmentation in the system testing. This study establishes a fundamental baseline of simulation for future agricultural robotics and autonomous vehicles in cotton farming and beyond.

Technical Abstract: Cotton is one of the primary cash crops of the United States, and the U.S.is the lead producer and exporter of cotton in the world market. Almost all cotton in the U.S. is produced in the Cotton Belt, containing the 17 cotton-producing states in the lower half region of the U.S. Harvesting is a crucial step in cotton farming, however, it comes with various challenges, both economically and environmentally. Many factors negatively impact the yield and quality of cotton production, including the costly and harmful defoliants, and heavy and expensive traditional cotton pickers. Consequently, cotton farmers face the challenges of the reduced profits due to yield loss and non-premium-quality cotton. In addition, the heavy machinery can compact the soil, threatening the sustainability of cotton farming, which is essential to meet the fiber demands of the growing population. To tackle with these challenges, a small-scale, lightweight, and visual-guided autonomous robotic cotton picker can provide a promising alternative solution. Testing the autonomous system and its algorithm in a simulation environment before actual field deployment is beneficial for identifying and eliminating unwanted technical issues. In this study, an autonomous visual-guided robotic cotton-picking system, built on a Clearpath’s Husky robot platform and the Cotton-Eye perception system, was developed in the Gazebo robotic simulator. Furthermore, a virtual cotton farm was designed and developed as a Robot Operating System (ROS 1) package to deploy the robotic cotton picker in the Gazebo environment for simulating autonomous field navigation. The navigation was assisted by the map coordinates and an RGB-depth camera, while the ROS navigation algorithm utilized a trained YOLOv8n-seg model for instance segmentation. The model achieved a desired mean Average Precision (mAP) of 85.2%, a recall of 88.9%, and a precision of 93.0% for scene segmentation with the test set. The developed ROS navigation packages enabled our robotic cotton-picking system to autonomously navigate through the cotton field using either map-based or GPS-based approach, visually aided by a deep learning-based perception system. The GPS-based navigation approach achieved a 100% completion rate with a threshold of (5 × 10-6 )', while the map-based navigation approach attained a 96.7% completion rate with a threshold of 0.25 m. This study establishes a fundamental baseline of simulation for future agricultural robotics and autonomous vehicles in cotton farming and beyond. CottonSim code and data are released to the research community via GitHub (https://github.com/imtheva/CottonSim).