Machine Vision Sees the Food Contaminants We Can't See

Robotic cameras may one day stand between us and the danger of drinking fresh, unpasteurized juices contaminated with fecal bacteria. Scientists at the ARS Instrumentation and Sensing Laboratory in Beltsville, Maryland, are developing "machine-vision" systems that can detect contamination the human eye often can't see.

The issue of contaminated apple juice came to the forefront in recent years with major outbreaks of Escherichia coli O157:H7 infections in people who drank unpasteurized apple juice or cider. The E. coli presumably got into the drinks via the skins of apples contaminated with fecal matter. This can happen when apples drop to the ground in an orchard and land in deer droppings or livestock manure. Or a rainstorm can splash parts of cowpats or deer droppings onto low-hanging fruit. Various forms of E. coli are present in fecal matter—whether of cows, deer, or people—and some, like E. coli O157:H7, are harmful. About a million fecal bacteria—including Salmonella—can live in a gram of cow manure.

When the apples are mashed, the E. coli becomes part of the mash and the juice. Fortunately, pasteurization kills all E. coli. But 2 percent of the fruit and vegetable juices sold in this country are unpasteurized, and that 2 percent accounts for an estimated 16,000 to 48,000 people sickened each year from bacterial or viral infections.

To identify fecal contamination early and quickly, Yud-Ren Chen, an agricultural engineer, and colleagues Kuanglin Chao, an agricultural engineer; Moon Kim, a biophysicist; and Alan Lefcourt, a biomedical engineer, are building a prototype "multispectral imaging" apple-inspection system. It uses reflectance from apples illuminated by halogen lamps in the invisible near-infrared and visible color light bands, as well as fluorescence techniques. It also detects dirt, fly specks, fungi, rot, and other diseases, all of which can cause fruit to harbor more bacteria, besides creating obvious quality problems.

Such systems are called machine-vision systems. They are quicker and more accurate than the human eye and don't require anyone to handle the fruit or cut it up.

Chen leads the diverse team that specializes in developing machine-vision systems using visible and near-infrared light. A mechanical engineer, an electrical engineer, a computer scientist, and a USDA Food Safety Inspection Service (FSIS) industrial engineer are also in the group. The Instrumentation and Sensing Laboratory is known for using cutting-edge technology and developing unique tools and technology to design equipment for commercial use.

For their latest project, the well-oiled team is testing machine vision on a commercial apple-sorting line. They are using a new digital spectral camera that is really several cameras in one. It can take pictures at different wavelengths simultaneously, creating multiple images. This once required two or more cameras, each with its own light filter. Using a hyperspectral imager, the team can find the wavelengths best suited to spotting fecal contamination or cuts and bruises that can harbor bacteria. Some wavelengths are chosen because of their identifiable relationships to photosynthetic pigments in apples.

Hyperspectral Imaging

Biophysicist Kim came to ARS in 1999 from the National Aeronautics and Space Administration (NASA), where he used reflectance and fluorescence for remote sensing of vegetation from airplanes to check on the planet's environmental health. Using his NASA experience, Kim added a fluorescence capability to the Beltsville lab's existing hyperspectral imaging equipment. To detect fecal contamination, he is still sensing photosynthetic pigments from plants, but on a much smaller scale, now working barely 2 feet from his targets rather than several thousand.

Kim and Lefcourt, who came to Chen's lab in 2001, upgraded and modernized the lab's existing hyperspectral imaging equipment. The lab uses the hyperspectral system to design commercial inspection systems for poultry and produce.

The unique instrument, designed and hand-built by the Beltsville team using commercially available components, is called hyperspectral rather than multispectral because it can capture images at up to 256 different wavelengths. A multispectral system generally uses only 2 to 4 wavelengths.

Agricultural engineers Kuanglin
Chao (left) and Yud-Ren Chen
discuss poultry carcass images
taken with a multispectral
imaging system.
(K9936-1)

"In the research stage, we use over 100 images at many different wavelengths," Kim says. "But it takes several minutes to scan objects at that many wavelengths. So hyperspectral imaging wouldn't be practical for commercial operations. But it is valuable because it lets us visualize images across a range of the spectrum. We can then choose a few optimal spectral bands that will get the job done with enough speed and accuracy when used in multispectral imaging systems." A multispectral imaging system can scan a whole object in a fraction of a second and is more suitable for real-time use in processing plants, Kim says.

Their latest hyperspectral imaging system has the newest "scientific-grade" imaging spectrograph and halogen and fluorescent lamps, all packaged in one unit that sits above a motorized positioning table where the apple is placed. The imaging spectrograph is connected to a computer. For reflectance sensing, visible to near-infrared light comes from quartz halogen bulbs connected to the unit through fiber-optic lines, while fluorescence imaging uses fluorescent lamps. ARS-developed computer software analyzes the hyperspectral images.

The imaging spectrograph scans a moving apple hundreds of times, each time sensing a line across the apple's surface. The light on each point on the line is spread out like a rainbow by the spectrograph, creating a three-dimensional image.

"The hyperspectral imaging system is versatile and has many research applications besides food safety," Kim says. Chen agrees that the lab's hyperspectral imaging equipment can be used in many disciplines and with a variety of agricultural products. For example, Stephen Delwiche, an agricultural engineer on the team, uses the equipment to test for fungal contamination of wheat kernels.

Or, says Lefcourt, "A color change in leaves can signal a problem like serious nutrient deficiency. Machine vision can spot the problem when it's still minor and causing slight color changes not visible to the human eye. There's no need to destroy the leaf to diagnose the condition."

"One of our lab's strengths is that we can study biological things such as chickens or apples in detail, from the smallest spot all the way up to the whole object," Lefcourt says. We can obtain a picture of the object with spectral signature information for each spot of its surface. This allows us to start with a concept of a problem, set up lab equipment to test the concept, assemble a prototype system to test it in an in-house pilot plant, and finally test it in a commercial environment.

"We use machine vision to find common patterns in wholesome agricultural objects, so that any anomalies—diseases, defects, or contamination—stand out. Similar machine-vision technology can be applied to detect tumors in chickens or fecal contamination or bruises on apples, or a fungus on a kernel of grain. Since natural objects are not uniform, we can't compare one spot on an object to another spot, but we can find common features among objects in the same class. We take pictures of whole objects with spectral signatures at each spot on these objects to detect anomalies and then figure out what the anomalies are."

From Chickens to Apples

Chen first developed his machine-vision inspection techniques with chickens, which present more complex problems than apples. Apples are easier in part because they are more uniform in shape and surface texture than chickens. Still, there are uniformity problems, such as color differences from variety to variety and even within a single apple.

More than a decade ago, Chen and a team of engineers, working alongside FSIS scientists and veterinarians began developing a prototype of a four-camera multispectral imaging system and a near-infrared light probe for reflectance scans of chicken carcasses on the processing line. (See "Automated Chicken Inspection," Agricultural Research, May 1998, p. 4.)

"We use visible and near-infrared light bounced off various spots on a chicken carcass to find systemic problems," Chen says. The reflected light is analyzed by a computer using software and hardware combinations designed by Chen's team. Differences between light shining on the chicken and light reflected are due to variations in external skin color, texture, and chemical contents that are clues to problems.

"We use the multispectral imaging to view each carcass as a whole, so we can spot quality problems, such as undersize birds, as well as food safety-related problems, such as blood poisoning," Chen says. For chickens, he uses a green and a red filter to create an image. He found that these two light wavelengths were best for detecting physical and biological problems.

The equipment identifies definitely unwholesome carcasses for rejection and suspect carcasses requiring closer human inspection.

Automated Chicken Inspection Ready To Commercialize

The lab has a cooperative research and development agreement (CRADA) with Stork Gamco, Inc., of Gainesville, Georgia—one of the largest manufacturers of chicken-processing plant equipment in the world—to commercialize the system and move it into use among the nation's 300-plus poultry processing plants.

Chen's colleague Chao says that Stork Gamco will soon test the system in a chicken-processing plant under the most demanding situation—lines that move 140 birds a minute. Chen says the system could handle up to 180 birds a minute.

Chao says the new system will be contained in a box hung over the beginning of the processing line, right after the point where chickens are killed and defeathered. Its camera will send spectral images to a computer set up in another room.

Chao and colleagues, including Sukwon Kang, an agricultural engineer, updated the machine-vision system to its present user-friendly form, ready to leave the research bench for commercial development. The two redesigned the system to use the new camera, instead of multiple cameras that required additional mathematical adjustments to join separate images. They also changed the software from the DOS operating system to function in Windows, where users can easily navigate by clicking on graphic images.

"We recognize that the users—in this case the chicken processing plant employees—must be considered at every design stage," Chao says.

The Time Is Right

Chao says the new system is ready to market at just the right time, when everything is in place for its success. FSIS is looking at machine vision as a way to help implement its Hazard Analysis and Critical Control Points system, which shifts more inspection responsibility to the processing plant. "This would free up inspectors so they have time to take a close, careful look at the birds the machine-vision system judges suspect," Chao says.

Also, the processing industry is moving to high-speed lines in response to a rising demand for poultry. The industry wants the highest feasible speeds for maximum efficiency, and they see machine vision as the way to make it possible while also improving inspection efficacy. Chao says that the high-speed lines separate into two or three more lines after the birds are killed, and more inspectors are added to meet USDA's requirement of a maximum speed of 35 birds a minute for each inspector.

Robots Are Quick Learners

Chao stresses that machine vision requires occasional input from the inspectors to make assessments and adjustments. While the line is down for cleaning, the system can update itself based on this input. The system has to be more intelligent than one that inspects nonbiological products like bottle labels and caps. "The size and shape of each cap and the cap's label are all uniform, making variations easy to detect," Chao says. "Unfortunately, nature doesn't make chickens with anywhere near that much uniformity."

Chickens differ not only in size, but also in other key characteristics such as skin color and chemical composition. These change with each batch of chickens, depending on the feed they were raised on and the weather conditions they lived with.

Ironically, some uniformity for chickens may be found in their diseases. "Chickens are raised so closely together that a disease tends to spread through the flock," Chao says, "making it more likely that if one carcass has a disease, others in the batch may also."

The inspectors and plant veterinarians can teach the system based on their skills and experience and based on what they find as they inspect the birds rejected by the system. The system has another advantage: It keeps records on every chicken, and these records can later be used by the plant to alert poultry growers to conditions in particular batches of chickens.

Chen's system quickly diagnoses all physical or biological conditions that cause an inspector to remove a chicken from the processing line. It does not spot bacterial contamination, but the ARS Poultry Processing and Meat Quality Research Unit in Athens, Georgia, has signed a CRADA with Stork Gamco to use machine vision to spot contamination from "ingesta," partially digested food from the ruptured crops of chicken carcasses, and from fecal matter, both of which are associated with bacterial contamination.

The chicken plant of the very near future will likely have two new systems in place—one for the automated diagnosis of wholesomeness and one for fecal and ingesta contamination—combining the Beltsville and Athens machine-vision systems.

From Apples to All Produce

In a similar way, apple-packing plants will have two or more systems in place. Some plants currently have automated ways to sort out undersized apples. The system Chen and his team are working on, when commercialized, would likely be merged with that system, along with others in the pipeline, including one by Renfu Lu, an ARS agricultural engineer in East Lansing, Michigan. He is developing an automated way to sort apples by quality, including deep internal bruises, as well as taste and firmness. Lu once worked in Chen's lab, as did Bosoon Park, now an agricultural engineer in the Athens lab.

Chen applied the experience and technology developed from his chicken work to detecting fecal contamination on apples. "We had to develop new technology for this application as well, but the work evolved from our success with chicken inspection," Chen says.

"We want 100 percent of the apples to be inspected as each passes the continuously operating lights and camera," Chen says. "We expect this system to be adaptable for use with all fruits and other produce."—By Don Comis, Agricultural Research Service Information Staff.

This research is part of Food Safety (Animal & Plant Products), an ARS National Program (#108) described on the World Wide Web at http://www.nps.ars.usda.gov.

To reach scientists featured in this story, contact Don Comis, ARS Information Staff, 5601 Sunnyside Ave., Beltsville, MD 20705-5129; phone (301) 504-1625, fax (301 504-1641.