Detecting invasive insects using uncrewed aerial vehicles and variational autoencoders

Authors: MEDEIROS, HENRY - University Of Florida; Tabb, Amy; STEWART, SCOTT - Marquette University; Leskey, Tracy

Submitted to: Computers and Electronics in Agriculture
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 3/27/2025
Publication Date: 4/26/2025
Citation: Medeiros, H., Tabb, A., Stewart, S., Leskey, T.C. 2025. Detecting invasive insects using uncrewed aerial vehicles and variational autoencoders. Computers and Electronics in Agriculture. 236. Article 110362. https://doi.org/10.1016/j.compag.2025.110362.
DOI: https://doi.org/10.1016/j.compag.2025.110362

Interpretive Summary: We investigated methods to detect and track insects using a camera and Uncrewed Aerial Vehicle (UAV). The insects are visible from the air because they are coated in fluorescent powder. We developed a method of insect detection in the video using a deep learning technique, Convolutional Variational Auto Encoder, and matched insects across multiple video frames using a technique, Multiple Hypothesis Tracking. The result is that using this type of UAV system and this level of experiment with dead insects, we can detect and track insects. Our method is an automated form of mark-release-recapture, which is used to understand insect behavior and is important for understanding new insects to an environment.

Technical Abstract: Invasive insect pests, such as the brown marmorated stink bug (BMSB), cause significant economic and environmental damage to agricultural crops. To mitigate damage, entomological research to determine insect behavior in the invaded region is needed; a component of this research is tracking insect movement with mark-release-recapture methods. In mark-release-recapture, a researcher marks insects with a fluorescent powder, releases the insects back into the wild, and searches for the insects using ultraviolet (UV) flashlights at suspected destination locations. However, this involves a significant amount of labor and has a low recapture rate. By automating the insect search step, the recapture rate can be improved, reducing the amount of labor required in the process and improving the quality of the data. We propose a new method of mark-release-recapture by using an uncrewed aerial vehicle (UAV) to collect video data of the area of interest. Our system uses a UV lighting array and a digital camera mounted on the bottom of the UAV, and convolutional neural networks (CNN) and multiple hypotheses tracking (MHT) techniques to process the video data. Specifically, we propose a novel computer vision method for insect detection using a Convolutional Variational AutoEncoder (CVAE). Additionally, we associate insect observations using MHT, improving detection results and accurately counting real-world insects. Our experimental results show that our system can detect BMSB with high precision and recall, outperforming the current state-of-the-art.