Computer-simulated virtual image datasets to train machine learning models for non-invasive fish detection in recirculating aquaculture

Authors: STEELE, SULLIVAN - Freshwater Institute; RANJAN, RAKESH - Freshwater Institute; SHARRER, KATA - Freshwater Institute; TSUKUDA, SCOTT - Freshwater Institute; GOOD, CHRISTOPHER - Freshwater Institute

Submitted to: Sensors
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 9/2/2024
Publication Date: 9/7/2024
Citation: Steele, S., Ranjan, R., Sharrer, K., Tsukuda, S., Good, C. 2024. Computer-simulated virtual image datasets to train machine learning models for non-invasive fish detection in recirculating aquaculture. Sensors. 24(17). Article 5816. https://doi.org/10.3390/s24175816.
DOI: https://doi.org/10.3390/s24175816

Interpretive Summary: Artificial intelligence and machine learning can enhance management of recirculating aquaculture systems by allowing for non-invasive fish detection to monitor, in real time, fish behavior, biomass, and health. The performance of machine learning models depends primarily on the quality of training data, and manual annotation of images for model training can be subjective, time-consuming, and labor-intensive. This study assessed the feasibility of computer simulation to generate virtual images and use these images for training in-tank fish detection models. The virtual model trained solely with simulated images did not perform satisfactorily in partial fish detection. However, replacing small numbers of virtual images from the training dataset with manually-annotated real images significantly improved performance of the model. This mixed model precisely detected whole as well as partial fish in an actual recirculating aquaculture system environment. Additionally, the automated annotation resulted in a seven-fold reduction in total training time cost compared to manual annotation of real images.

Technical Abstract: Artificial Intelligence (AI) and Machine Learning (ML) can assist producers to better manage recirculating aquaculture systems (RAS). ML however is a data-intensive process, and model performance primarily depends on the quality of training data. Relatively higher fish density and water turbidity in intensive RAS culture produce major challenges in acquiring high-quality underwater image data. Additionally, the manual image annotation involved in model training can be subjective, time-consuming, and labor-intensive. Therefore, the presented study aimed to simulate the fish schooling behavior for RAS conditions and investigate the feasibility of using computer-simulated virtual images to train a robust fish detection model. Additionally, to expedite the model training, a process flow was developed to automate the virtual image annotation. The ‘virtual model’ performances were compared with models trained with real-world images and combinations of real and virtual images. The results of the study indicate that the virtual model trained solely with computer-simulated images could not perform satisfactorily (mAP = 62.8%, F1 score = 0.61) to detect fish in the real RAS environment; however, replacing a small number of the virtual image with real images in the training dataset significantly improved the model performance. The M6 mixed model trained with 630 virtual and 70 real images (virtual to real image ratio: 90:10) achieved mAP and F1 scores of 91.8% and 0.87, respectively. Furthermore, the training time cost for the M6 model was seven times lower than the ‘real model’, respectively. Overall, the virtual simulation approach exhibited great promise in rapidly training a reliable fish detection model for RAS operations.