Research Project: Advancement of Sensing Technologies for Food Safety and Security Applications

Location: Environmental Microbial & Food Safety Laboratory

2024 Annual Report

Objectives
Objective 1: Develop and validate an autonomous unmanned aerial vehicle with multimode imaging technologies for preharvest inspection of produce fields for animal intrusion and fecal contamination, and for irrigation water quality monitoring. Objective 2: Advance the development of customized compact spectral sensing technologies for food inspection and sanitation assessment in food processing, and for controlled-environment produce production, with embedded automated detection results for non-expert end users. Sub-objective 2.A: Develop a handheld line-scan hyperspectral imaging device with enhanced capabilities for contamination and sanitation inspection in food processing environments. Sub-objective 2.B: Develop a compact automated hyperspectral imaging platform for food safety and plant health monitoring for controlled-environment produce production in NASA space missions. Objective 3: Develop innovative spectroscopic and optical methods to characterize food composition and nondestructively detect adulterants and contaminants, for screening and inspecting agricultural commodities and commercially prepared food materials. Sub-objective 3.A: Develop a transportable multimodal optical sensing system for rapid, automated, and intelligent biological and chemical food safety inspection. Sub-objective 3.B: Develop a novel apparatus enabling dual-modality concomitant detection, along with associated methods and procedures, for assuring food integrity.

Approach
The overall goal of this project is to develop and validate automated sensing tools and techniques to reduce food safety risks in food production and processing environments. Engineering-driven research will develop the next generation of rapid, intelligent, user-friendly sensing technologies for use in food production, processing, and other supply chain operations. Feedback from industrial and regulatory end users, and from stakeholders throughout the food supply chain, indicates that effective automated sensing and instrumentation systems require real-time data processing to provide non-expert users with a clear understanding and ability to make decisions based on the system output. Towards this end, we will develop unmanned aerial vehicles with multimodal remote sensing platforms and on-board data-processing capability to provide real-time detection and classification of animal intrusion and fecal contamination in farm fields and of irrigation water microbial quality. We will upgrade our existing handheld imaging device for contamination and sanitation inspection with multispectral imaging and embedded computing and artificial intelligence. We are also partnering with the NASA Kennedy Space Center to develop a novel, compact, automated hyperspectral platform for monitoring food safety and plant health of space crop production systems. Food safety and integrity requires identifying adulterants, foreign materials, and microbial contamination as well as authenticating ingredients. We will develop innovative multimodal optical sensing systems utilizing dual-band laser Raman, and Raman plus infrared, for simultaneous detection on a single sampling site. Spectroscopic and spectral imaging-based methodologies will be developed to enhance detection efficacy for liquid or powder samples. These systems will be supported with intuitive, intelligent sample-evaluation software and procedures for both biological and chemical contaminants.

Progress Report
Significant progress has been made on all Objectives of the project, which fall under Food Safety National Program 108. For Objective 1, USDA-ARS scientists in Beltsville, Maryland, began design, testing, and development of a small unmanned autonomous vehicle (sUAV) mounted with a multimodal imaging system (including hyperspectral, thermal, and 3D/color cameras), and continued development and testing of the field transportable multimodal imaging system. Data were collected from both undisturbed fields and fields with evidence of animal intrusion using the sUAV and the ground truth imaging systems. During the testing, the performance of the ground-based platform was evaluated, and improvements were made in both ease of use and data collection quality. This testing also led to the start of a collaborative project with an industry partner to develop a multimodal imaging system for the sUAV that fulfills the needs of this research, as no commercially available systems meet the requirements of this project. The new sUAV system has now been flown for both functionality and early design testing as well as data collection. Work continues for gathering data and making design improvements, as well as evaluating the potential of emerging sensor technology for application in the new system. For Objective 2A, USDA-ARS scientists in Beltsville, Maryland, engaged in collaborative efforts to advance the CSI-D device. This device incorporates patented handheld fluorescence imaging technology and is specifically engineered for detecting contamination on food contact surfaces. It utilizes ultraviolet (UV)-A fluorescence imaging to identify contaminants, employs UV-C illumination for disinfection purposes, and facilitates documentation of cleanliness levels. Throughout rigorous experiments, the device demonstrated robust performance in automatically detecting contaminants, achieving notable accuracies aided by artificial intelligence models. Furthermore, the research team developed a sophisticated bench-top system capable of emitting UV-B and UV-C light. This system was meticulously designed and evaluated for effectiveness in eradicating harmful bacteria, particularly within food processing environments and potential applications in plant settings. The system will be transferred to USDA plant scientists for collaboration on exploring its use and effects in plant germicidal applications. For Objective 2B, USDA-ARS scientists in Beltsville, Maryland, collaborated with NASA Kennedy Space Center (KSC) to advance ARS hyperspectral imaging technology tailored for monitoring plant health and ensuring food safety in fresh produce production systems for future spaceflight. The ARS and KSC team developed a new multimodal imaging system, featuring an automated imaging gantry platform created by ARS scientists to acquire a range of multimodal spectral and phenotype data essential for monitoring plant health in controlled environments. This system was installed in a plant growth chamber at NASA KSC for full-scale plant imaging experiments and includes two LED line lights providing broad visible to near-infrared illumination for reflectance, along with 365 nm ultraviolet-A excitation for fluorescence imaging. Python 3.0-based software was developed to control movements of the 3-axis gantry system and acquisition of multimodal spectral image data. NASA KSC utilized this platform to gather imaging data on pick-and-eat salad crop seedlings cultivated under well-watered and stressed conditions. ARS scientists developed chemometric and machine learning models to analyze the imaging data collected at NASA KSC. A machine learning method employing an optimized discriminant classifier based on combination spectra with VNIR reflectance and fluorescence achieved classification accuracies exceeding 90% for drought stress treatments. This approach demonstrates significant potential for early detection of drought stress on lettuce leaves, preempting visible symptoms and size differences. In addition, the ARS team and other collaborators developed a customized light source aimed at improving data quality. In collaboration with the University of Florida (UF), USDA-ARS scientists in Beltsville, Maryland, continued work to develop citrus disease detection and classification methods. Using a portable hyperspectral imaging system recently developed by the ARS team, UF collaborators collected hyperspectral reflectance and fluorescence images from the front and back sides of both health and diseased citrus leaves at the Citrus Research and Education Center in Lake Alfred, Florida. Based on the hyperspectral reflectance image data, UF collaborators developed a leaf disease classification method based on hyperspectral band selection and YOLOv8 network architectures. In addition, USDA-ARS scientists in Beltsville, Maryland, developed a leaf disease classification method based on machine learning models using combined reflectance and fluorescence spectral data extracted from the hyperspectral images. For Objective 3A, in collaboration with the National Agricultural Products Quality Management Service (NAQS), South Korea, USDA-ARS scientists in Beltsville, Maryland, completed integration, calibration, and optimization for a multimodal optical sensing system for automated and intelligent biological and chemical assessment in food safety applications. New white and ultraviolet-A LED spot lights were added to the system to improve color and fluorescence imaging for bacterial colonies grown in agar Petri dishes, with an oblique angle of illumination to minimize reflected glare from various agars in the Petri dishes. The LabVIEW system software was also modified for lighting control, image saving, and synchronization functions. To prepare for full-scale bacterial experiments, we replaced the black foam boards previously used in the system’s aluminum-framed enclosure with black PVC panels for easier cleaning and sanitation. In addition to the bacteria study, the system has also been used by a visiting scientist from NAQS to conduct experiments for detection of mycotoxin contamination in grains and animal feeds. In collaboration with an industry partner, USDA-ARS scientists in Beltsville, Maryland, continued work to develop a portable multimode spectroscopy device for industrial applications, such as detecting fungi and mycotoxins. This device can measure four types of spectral data, including fluorescence from both 365 nm and 405 nm excitations, reflectance in the visible region, and reflectance in the near-infrared region. ARS scientists developed the system's interface software in Python 3.0 to calibrate and optimize system parameters, as well as to acquire and visualize multimodal spectral data. Machine learning and chemometric models were developed to classify contaminated corn using various spectra collected by this system, and were demonstrated to be capable of 91% accuracy in detecting mycotoxins in corn. In collaboration with the University of North Dakota (UND), USDA-ARS scientists in Beltsville, Maryland, continued work to develop fish authentication methods to address issues of species mislabeling and fraud as well as freshness of fish fillets. Based on a multimode hyperspectral image dataset (i.e., fluorescence with VNIR and SWIR reflectance) collected using ARS in-house developed imaging systems in Beltsville, Maryland, UND developed a method for detecting mislabeling of fish fillets that was based on multimode spectroscopy data fusion and machine learning techniques and which achieved 89% accuracy in classifying fish species. The industry partner will use the results to design and develop portable smart spectroscopy-based sensing devices for industrial applications for on-site fish species and freshness inspection. For Objective 3B, USDA-ARS scientists in Beltsville, Maryland, conducted dual-modality IR and Raman measurements of three commercial plasticizers to (1) identify spectral wavenumbers in which the IR and Raman signals for the same vibrational mode are the most different in normalized relative intensity; (2) use this dual modality data as a marker to determine the most sensitive signal ratio which is specific to BPA, BPS, and BPF; (3) assign these wavenumbers to vibrational modes characteristic to individual compounds. This expands continuing research demonstrating the application of dual-modality techniques in confirming identity of specific food safety related products. A point-scan IR and Raman dual spectral imaging system is being developed to overcome the lack of instrumentation designed specifically for macro-scale sample measurement. Automated sample positioning, spectral acquisition, and synchronization functions are realized using in-house developed control software. System capabilities for food safety applications will be demonstrated by experiments and results for authenticating selected foods using the fusion of the IR and Raman data. A rapid spectral detection technique was also developed to analyze in-situ (as is) wheat-like products labeled as gluten-free. Three chemical standards, gliadin, gluten, and starch from wheat, and 62 different types of commercial flour products were scanned by FT-IR spectroscopy over the wavenumber range of 4000 and 400. The linear discriminant analysis models were successfully used to evaluate the data.

Accomplishments
1. Detection of aflatoxins in ground maize using Raman spectroscopy and machine learning. Aflatoxin contamination of maize is becoming an important issue in human food and animal feed supply. There is a need for an effective and efficient detection method that can be used for rapid onsite inspection of aflatoxin contamination. USDA-ARS scientists in Beltsville, Maryland, developed an aflatoxin detection method based on a custom developed compact and automated laser Raman spectroscopy system. Using a customized sample holder, Raman spectral data were automatically collected from ground maize samples naturally contaminated with aflatoxin. The data were analyzed using a machine learning method. A classification accuracy of 95.7% was achieved using a machine learning model based on linear discriminant analysis to differentiate aflatoxin levels in ground maize samples. The system and the detection method have the potential to be used at onsite processing locations to rapidly screen food and feed for aflatoxin and other hazardous substances affecting human and animal health. The technique would benefit the food industry and regulatory agencies in enforcing standards of safety and quality for maize-related food products.

2. Inspection of E. coli colonies using handheld fluorescence imaging and deep learning. Fruits and vegetables (e.g., citrus) can host bacterial pathogens (e.g., E. coli) that can cause severe health issues for the consumers. Detection of bacterial colonies on fruits and vegetables is important to reduce food safety risks and foodborne diseases. ARS scientists in Beltsville, Maryland, developed a method for inspecting E. coli colonies using a contamination, sanitization inspection and disinfection (CSI-D) handheld fluorescence imaging device, which was commercialized based on an ARS patented technology. Fluorescence images were collected from different concentrations of E. coli populations inoculated on black rubber slides. State-of-the-art deep learning algorithms (i.e., convolutional neural networks and generative adversarial networks) were used for image classifications. The best accuracy was achieved at 97% to classify four concentration levels of the E. coli colonies. The combination of the CSI-D handheld imaging and deep learning techniques would benefit the food industry and regulatory agencies in ensuring and enforcing food safety standards for products related to fruits and vegetables.

3. Identification of fish species using multi-mode spectroscopy and machine learning. Seafood mislabeling poses risks for consumers’ health and gives rise to economic and environmental hazards. Mixing less expensive species with more expensive species is a frequently recurring fraudulent practice in the seafood industry. ARS scientists in Beltsville, Maryland, developed a method based on multi-mode spectroscopy and machine learning techniques for detecting mislabeling of fish fillets. Three modes of spectral data, including fluorescence and reflectance in visible and near-infrared and short-wave infrared regions, were extracted from hyperspectral images collected from 216 fish fillet samples of 43 species. Algorithms were developed to create a hierarchical decision process for higher classification performance. Based on a classifier incorporating global and dispute models, a classification accuracy of 89% was achieved using the fusion of the three spectroscopic modes. The method developed in this study can be used to develop a rapid and cost-effective spectral sensing device for on-site inspection of the fish fillet mislabeling, which can be used for authentication of the fish fillets and other related food products by the seafood industry and regulatory agencies.

4. Classification of citrus fruit and leaf diseases using hyperspectral imaging and machine learning. Citrus black spot and canker are two significant diseases that pose quarantine threat, restrict market access, and cause economic losses for citrus growers. Early detection and management of groves infected with black spot or canker through fruit and leaf inspection can greatly benefit the citrus industry. ARS scientists in Beltsville, Maryland, developed an AI-based hyperspectral imaging and classification method for detection of fruits infected with black spot and leaves infected with canker. Hyperspectral reflectance images were collected in visible and near-infrared wavelength range from Valencia orange fruits and leaves with black spot, canker, and other common citrus diseases. Using convolutional neural network generated features and machine learning classifiers, classification accuracies were achieved at over 92% for classifying fruits with black spot and four other conditions and at over 93% for classifying leaves with canker and four other conditions. The method would benefit citrus industry and regulatory agencies in ensuring and enforcing the quality and safety standards for the citrus-related food and beverage products.

5. Non-destructive evaluation of soybean protein and lipid content using Raman chemical imaging. Soybean is an important crop that serves as a rich source of proteins and lipids, both of which are essential for human nutrition and are considered key parameters in determining soybean quality and market value. Traditional chemical analysis approaches (e.g., Soxhlet and Kjeldahl methods) for determining protein and lipid content of soybeans are destructive, time-consuming, and labor-intensive. ARS scientists in Beltsville, Maryland, developed a rapid and non-destructive method to evaluate soybean protein and lipid content based on macro-scale Raman hyperspectral imaging technique. A line-scan Raman hyperspectral imaging system was used to collect images of whole, intact soybean seeds. Partial least squares regression method was used to develop prediction models to correlate the Raman spectral data with the protein and lipid content of the soybean seeds. Chemical images were created to show the distributions and amounts of protein and lipid on single soybean seeds. The method developed in this study can be used for accurate and efficient estimation of protein and lipid content, which would benefit the soybean industry for quality control and breeding programs.

6. A rapid, accurate, and sensitive dual-modality IR and Raman spectroscopic technique for confirming the identity of plasticizers used in food containers. Bisphenol A (BPA) has previously been used as a common plasticizer in polycarbonate plastics for containers that store food and beverages. Regulatory measures have restricted the use of BPA in plastic formulations, especially for those which come in contact with food products. Rapid, accurate spectroscopic measurements are required for uniquely distinguishing BPA from other bis-phenols. ARS scientists in Beltsville, Maryland, developed a rapid, accurate, and sensitive dual-modality IR and Raman spectroscopic technique to distinguish BPA from other commercially used bis-phenols (BPS and BPF). Dual modality analysis enables addressing exactly the wavenumbers in which the IR and Raman spectra are the most different. Analysis found that the major phenolic ring vibrational mode intensities in BPA, BPS and BPF are significantly different, sufficiently so to identify them from each other despite their structural similarity. This approach provides practical information for the use of dual-modality technology for food detection, which will benefit researchers who have interest in developing and using the dual-modality techniques for safety and quality inspection of food products.

7. Rapid spectral identification of wheat-like products labeled gluten-free. Food allergies are major food and health concern. Celiac disease is a food allergy condition related to wheat protein gluten. Currently, the most commonly used methods for gluten testing of foods require complicated chemical procedures. ARS scientists in Beltsville, Maryland, developed a Fourier-transform infrared (FT-IR) in-situ (as is) spectroscopic technique for authentication of gluten-free flour products. Three chemical standards—gliadin, gluten, and starch from wheat—and 62 different types of flour products were scanned by FT-IR spectroscopy over the wavenumber range of 4000 and 400. Notable signal differences were observed between the chemical standards and wheat samples over the wavenumber range of 1800 to 450. The linear discriminant analysis models were successfully used to evaluate the data. This approach provides a practical method for evaluating commercial products labeled gluten-free for the presence of wheat in-situ (as is).

Review Publications
Guo, Q., Peng, Y., Chao, K., Qin, J., Chen, Y., Yin, T. 2023. A determination method for clenbuterol residue in pork based on optimal particle size gold colloid using SERS. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy. 302:1386-1425. https://doi.org/10.1016/j.saa.2023.123097.
Yadav, P., Burks, T.F., Dudhe, K., Frederick, Q., Qin, J., Kim, M.S., Ritenour, M.A. 2023. Classification of E. coli colony with generative adversarial networks, discrete wavelet transforms and VGG19. Veterinary Radiology and Ultrasound. 6(3):146-160.
Aulia, R., Amanah, H., Lee, H., Kim, M.S., Baek, I., Qin, J., Cho, B. 2023. Proteins and lipids content estimation in soybeans using Raman hyperspectral imaging. Frontiers in Plant Science. 14. Article e1167139. https://doi.org/10.3389/fpls.2023.1167139.
Gorji, H., Saeedi, M., Zadeh, H., Husairik, K., Mojtaba, S., Qin, J., Chan, D.E., Baaek, I., Kim, M.S., Akhbardeh, A., Mackinnon, N., Vasefi, F., Tavakolian, K. 2023. Federated learning for clients' data privacy assurance in food service industry. Applied Sciences. 13(16):9330. https://doi.org/10.3390/app13169330.
Sueker, M., Daghighi, A., Akhbardesh, A., Mackinnon, N., Bearman, G., Baek, I., Hwang, C., Qin, J., Tabb, A.M., Roungchun, J.B., Hellberg, R.S., Vasefi, F., Kim, M.S., Tavakolian, K., Kashani Zadeh, H. 2023. A novel machine learning framework based on a hierarchy of dispute models for the identification of fish species using multi-mode spectroscopy . Sensors. 23(22): Article e9062. https://doi.org/10.3390/s23229062.
Frederick, Q., Burks, T.F., Watson, A., Yadav, P., Qin, J., Kim, M.S., Ritenour, M.A. 2024. Selecting hyperspectral bands and extracting features with a custom shallow convolutional neural network to classify citrus peel defects. Smart Agricultural Technology. 6: Article e100365. https://doi.org/10.1016/j.atech.2023.100365.
Kim, Y., Qin, J., Baek, I., Lee, K., Kim, S., Kim, S., Chan, D.E., Herrman, T.J., Kim, N., Kim, M.S. 2023. Detection of aflatoxins in ground maize using a compact and automated Raman spectroscopy with machine learning. Current Research in Food Science. 7: Article e100647. https://doi.org/10.1016/j.crfs.2023.100647.
Yadav, P., Burks, T.F., Qin, J., Kim, M.S., Frederick, Q., Dewdney, M.M., Ritenour, M.A. 2024. Automated classification of citrus disease on fruits and leaves using convolutional neural network (CNN) generated features from hyperspectral images and machine learning classifiers. Journal of Applied Remote Sensing (JARS). 18 (1): Article e014512. https://doi.org/10.1117/1.JRS.18.014512.
Sanders, J., Alarcorn, V., Marquis, G., Tabb, A., Van Kessel, J.S., Sonnier, J.L., Haley, B.J., Baek, I., Qin, J., Kim, M.S., Vasefi, F., Sokolov, S., Hellberg, R. 2024. Disinfection of foodborne bacteria using the Contamination Sanitization Inspection and Disinfection (CSI-D) device bg. Food Microbiology. 10(9): Article e30490. https://doi.org/10.1016/j.heliyon.2024.e30490.
Guo, Q., Peng, Y., Qin, J., Chao, K., Zhao, Z., Yin, T. 2024. Advance in detection technique of lean meat powder residues in meat using SERS: a review. Molecules. 28(22). Article e7504. https://doi.org/10.3390/molecules28227504.
Kim, J., Kurniawan, H., Faqeerzada, M.A., Kim, G., Lee, H., Kim, M.S., Baek, I., Cho, B. 2023. Proximate content monitoring of black soldier fly larval (Hermetia illucens) dry matter for feed material using short-wave infrared hyperspectral imaging. Food Science of Animal Resources. 43(6):1150-1169. https://doi.org/10.5851/kosfa.2023.e33.
Chun, S., Song, D., Lee, K., Kim, M., Kim, M.S., Kyoung-Su, K., Mo, C. 2024. Deep learning algorithm development for early detection of Botrytis cinerea infected strawberry fruit using hyperspectral fluorescence imaging. Postharvest Biology and Technology. 214: Article e112918. https://doi.org/10.1016/j.postharvbio.2024.112918.
Kim, M., Yu, W., Song, D., Chun, S., Kim, M.S., Lee, A., Kim, G., Mo, C. 2024. Prediction of soluble-solid content in citrus fruit using visible–near infrared hyperspectral imaging based on machine learning and effective-wavelength selection algorithm. Sensors. 24, 1512. https://doi.org/10.3390/s24051512.
Kurniawana, H., Ariefa, M., Santosh, L., Kim, M.S., Baek, I., Cho, B. 2024. Dual imaging technique for a real-time inspection system of foreign object detection in fresh-cut vegetables. Current Research in Food Science. 9: article 100802. https://doi.org/10.1016/j.crfs.2024.100802.
Patel, A., Park, E., Lee, H., Priya, G., Kim, H., Joshi, R., Arief, M.A., Kim, M.S., Baek, I., Cho, B. 2023. Deep learning-based plant organ segmentation and phenotyping of sorghum plants using LiDAR point cloud. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 16, 8492. https://doi.org/10.1109/JSTARS.2023.3312815.
Liu, Z., Zhou, H., Huang, M., Zhu, Q., Qin, J., Kim, M.S. 2023. Packaged butter adulteration evaluation based on spatially offset Raman spectroscopy coupled with FastICA. Journal of Food Composition and Analysis. 117:105149. https://doi.org/10.1016/j.jfca.2023.105149.
Prabhukhot, G., Yin, H., Eggelton, C., Kim, M.S., Patel, J.R. 2024. Impact of surface topography and shear stress on single and dual species biofilm formation by Escherichia coli O157:H7 and Listeria monocytogenes in presence of promotor bacteria. BIOFOULING. 201: e116240. https://doi.org/10.1016/j.lwt.2024.116240.
Boyd, A., Luo, Y., Lunney, J.K., Kustas, B., Fukagawa, N.K., Mattoo, A.K., Crow, W.T., Pachepsky, Y.A., Kim, M.S., Lillehoj, H.S., Van Tassell, C.P., Zhang, H.Q., Blomberg, L., Dubey, J.P. 2023. Cross-cutting concepts to transform agricultural research. Frontiers in Sustainable Food Systems. 7. Article e1242665. https://doi.org/10.3389/fsufs.2023.1242665.
Kim, Y., Baek, I., Lee, K., Kim, G., Kim, S., Kim, S., Chan, D.E., Herrman, T., Kim, N., Kim, M.S. 2023. Hyperspectral imaging techniques for rapid detection of single- and co-contaminant aflatoxins and fumonisins in ground maize. Toxins. 15(7):472. https://doi.org/10.3390/toxins15070472.
Aline, U., Bhattacharya, T., Faqeerzada, M., Kim, M.S., Baek, I., Cho, B. 2023. Advancement of non-destructive spectral measurements for the quality of major tropical fruits and vegetables: a review. Frontiers in Plant Science. 14:1240361. https://doi.org/10.3389/fpls.2023.1240361.
Hong, S., Morgan, B.J., Stocker, M.D., Smith, J.E., Kim, M.S., Cho, K., Pachepsky, Y.A. 2024. Estimating concentrations of Escherichia coli across a farm pond from the sUAS-based RGB imagery and water quality variables with machine learning techniques. Water Research. 260: Article e121861. https://doi.org/10.1016/j.watres.2024.121861.