Machine Vision Sees the Food Contaminants We Can't See
Agricultural engineer Bosson Park,
formerly with the Instrumentation
and Sensing Laboratory, checks a
light probe, which scans the
chicken breast to determine
Robotic cameras may one day stand between us and the danger of drinking
fresh, unpasteurized juices contaminated with fecal bacteria. Scientists
at the ARS Instrumentation and
Sensing Laboratory in Beltsville, Maryland, are developing "machine-vision"
systems that can detect contamination the human eye often can't see.
The issue of contaminated apple juice came to the forefront in recent years with major outbreaks of Escherichia coli O157:H7 infections in people who drank unpasteurized apple juice or cider. The E. coli presumably got into the drinks via the skins of apples contaminated with fecal matter. This can happen when apples drop to the ground in an orchard and land in deer droppings or livestock manure. Or a rainstorm can splash parts of cowpats or deer droppings onto low-hanging fruit. Various forms of E. coli are present in fecal matterwhether of cows, deer, or peopleand some, like E. coli O157:H7, are harmful. About a million fecal bacteriaincluding Salmonellacan live in a gram of cow manure.
This newly upgraded
hyperspectral imaging system,
being used by biophysicist Moon
Kim, takes pictures at
different wavelengths simultaneously.
Three-dimensional images are
created from the process, and
researchers can then choose
the wavelengths best suited for
spotting fecal contamination or
cuts and bruises in agricultural
When the apples are mashed, the E. coli becomes part of the mash and the juice. Fortunately, pasteurization kills all E. coli. But 2 percent of the fruit and vegetable juices sold in this country are unpasteurized, and that 2 percent accounts for an estimated 16,000 to 48,000 people sickened each year from bacterial or viral infections.
To identify fecal contamination early and quickly, Yud-Ren Chen, an
agricultural engineer, and colleagues Kuanglin Chao, an agricultural
engineer; Moon Kim, a biophysicist; and Alan Lefcourt, a biomedical
engineer, are building a prototype "multispectral imaging"
apple-inspection system. It uses reflectance from apples illuminated
by halogen lamps in the invisible near-infrared and visible color light
bands, as well as fluorescence techniques. It also detects dirt, fly
specks, fungi, rot, and other diseases, all of which can cause fruit
to harbor more bacteria, besides creating obvious quality problems.
Such systems are called machine-vision systems. They are quicker and more accurate than the human eye and don't require anyone to handle the fruit or cut it up.
Agricultural engineer Stephen
Delwiche positions wheat
kernels on a custom-designed
tray in preparation for
hyperspectral imaging for
Chen leads the diverse team that specializes in developing machine-vision systems using visible and near-infrared light. A mechanical engineer, an electrical engineer, a computer scientist, and a USDA Food Safety Inspection Service (FSIS) industrial engineer are also in the group. The Instrumentation and Sensing Laboratory is known for using cutting-edge technology and developing unique tools and technology to design equipment for commercial use.
For their latest project, the well-oiled team is testing machine vision on a commercial apple-sorting line. They are using a new digital spectral camera that is really several cameras in one. It can take pictures at different wavelengths simultaneously, creating multiple images. This once required two or more cameras, each with its own light filter. Using a hyperspectral imager, the team can find the wavelengths best suited to spotting fecal contamination or cuts and bruises that can harbor bacteria. Some wavelengths are chosen because of their identifiable relationships to photosynthetic pigments in apples.
Mechanical engineer Talaya
James and agricultural engineer
Yud-Ren Chen use a common
aperture multispectral system
to detect apple bruises. They
are checking on apples to
evaluate the effectiveness
of the detection algorithms.
Biophysicist Kim came to ARS in 1999 from the National Aeronautics and Space Administration (NASA), where he used reflectance and fluorescence for remote sensing of vegetation from airplanes to check on the planet's environmental health. Using his NASA experience, Kim added a fluorescence capability to the Beltsville lab's existing hyperspectral imaging equipment. To detect fecal contamination, he is still sensing photosynthetic pigments from plants, but on a much smaller scale, now working barely 2 feet from his targets rather than several thousand.
Kim and Lefcourt, who came to Chen's lab in 2001, upgraded and modernized the lab's existing hyperspectral imaging equipment. The lab uses the hyperspectral system to design commercial inspection systems for poultry and produce.
The unique instrument, designed and hand-built by the Beltsville team using commercially available components, is called hyperspectral rather than multispectral because it can capture images at up to 256 different wavelengths. A multispectral system generally uses only 2 to 4 wavelengths.
"In the research stage, we use over 100 images at many different
wavelengths," Kim says. "But it takes several minutes to scan
objects at that many wavelengths. So hyperspectral imaging wouldn't
be practical for commercial operations. But it is valuable because it
lets us visualize images across a range of the spectrum. We can then
choose a few optimal spectral bands that will get the job done with
enough speed and accuracy when used in multispectral imaging systems."
A multispectral imaging system can scan a whole object in a fraction
of a second and is more suitable for real-time use in processing plants,
Their latest hyperspectral imaging system has the newest "scientific-grade" imaging spectrograph and halogen and fluorescent lamps, all packaged in one unit that sits above a motorized positioning table where the apple is placed. The imaging spectrograph is connected to a computer. For reflectance sensing, visible to near-infrared light comes from quartz halogen bulbs connected to the unit through fiber-optic lines, while fluorescence imaging uses fluorescent lamps. ARS-developed computer software analyzes the hyperspectral images.
The imaging spectrograph scans a moving apple hundreds of times, each time sensing a line across the apple's surface. The light on each point on the line is spread out like a rainbow by the spectrograph, creating a three-dimensional image.The positioning table lets the researchers run hundreds of scans of the apple surface, placing the apple in many different positions, while recording the exact position of the apple so a scan can be repeated later. Mathematical algorithms interpret the multiple images.
"The hyperspectral imaging system is versatile and has many research
applications besides food safety," Kim says. Chen agrees that the
lab's hyperspectral imaging equipment can be used in many disciplines
and with a variety of agricultural products. For example, Stephen Delwiche,
an agricultural engineer on the team, uses the equipment to test for
fungal contamination of wheat kernels.
Or, says Lefcourt, "A color change in leaves can signal a problem
like serious nutrient deficiency. Machine vision can spot the problem
when it's still minor and causing slight color changes not visible to
the human eye. There's no need to destroy the leaf to diagnose the condition."
"One of our lab's strengths is that we can study biological things
such as chickens or apples in detail, from the smallest spot all the
way up to the whole object," Lefcourt says. We can obtain a picture
of the object with spectral signature information for each spot of its
surface. This allows us to start with a concept of a problem, set up
lab equipment to test the concept, assemble a prototype system to test
it in an in-house pilot plant, and finally test it in a commercial environment.
"We use machine vision to find common patterns in wholesome agricultural
objects, so that any anomaliesdiseases, defects, or contaminationstand
out. Similar machine-vision technology can be applied to detect tumors
in chickens or fecal contamination or bruises on apples, or a fungus
on a kernel of grain. Since natural objects are not uniform, we can't
compare one spot on an object to another spot, but we can find common
features among objects in the same class. We take pictures of whole
objects with spectral signatures at each spot on these objects to detect
anomalies and then figure out what the anomalies are."
From Chickens to Apples
Chen first developed his machine-vision inspection techniques with
chickens, which present more complex problems than apples. Apples are
easier in part because they are more uniform in shape and surface texture
than chickens. Still, there are uniformity problems, such as color differences
from variety to variety and even within a single apple.
More than a decade ago, Chen and a team of engineers, working alongside
FSIS scientists and veterinarians began developing a prototype of a
four-camera multispectral imaging system and a near-infrared light probe
for reflectance scans of chicken carcasses on the processing line. (See
Chicken Inspection," Agricultural Research, May 1998,
"We use visible and near-infrared light bounced off various spots
on a chicken carcass to find systemic problems," Chen says. The
reflected light is analyzed by a computer using software and hardware
combinations designed by Chen's team. Differences between light shining
on the chicken and light reflected are due to variations in external
skin color, texture, and chemical contents that are clues to problems.
"We use the multispectral imaging to view each carcass as a whole,
so we can spot quality problems, such as undersize birds, as well as
food safety-related problems, such as blood poisoning," Chen says.
For chickens, he uses a green and a red filter to create an image. He
found that these two light wavelengths were best for detecting physical
and biological problems.
The equipment identifies definitely unwholesome carcasses for rejection and suspect carcasses requiring closer human inspection.
Automated Chicken Inspection Ready To Commercialize
The lab has a cooperative research and development agreement (CRADA)
with Stork Gamco, Inc., of Gainesville, Georgiaone of the largest
manufacturers of chicken-processing plant equipment in the worldto
commercialize the system and move it into use among the nation's 300-plus
poultry processing plants.
Chen's colleague Chao says that Stork Gamco will soon test the system
in a chicken-processing plant under the most demanding situationlines
that move 140 birds a minute. Chen says the system could handle up to
180 birds a minute.
Chao says the new system will be contained in a box hung over the beginning
of the processing line, right after the point where chickens are killed
and defeathered. Its camera will send spectral images to a computer
set up in another room.
Chao and colleagues, including Sukwon Kang, an agricultural engineer,
updated the machine-vision system to its present user-friendly form,
ready to leave the research bench for commercial development. The two
redesigned the system to use the new camera, instead of multiple cameras
that required additional mathematical adjustments to join separate images.
They also changed the software from the DOS operating system to function
in Windows, where users can easily navigate by clicking on graphic images.
"We recognize that the usersin this case the chicken processing plant employeesmust be considered at every design stage," Chao says.
The Time Is Right
Chao says the new system is ready to market at just the right time,
when everything is in place for its success. FSIS is looking at machine
vision as a way to help implement its Hazard Analysis and Critical Control
Points system, which shifts more inspection responsibility to the processing
plant. "This would free up inspectors so they have time to take
a close, careful look at the birds the machine-vision system judges
suspect," Chao says.
Also, the processing industry is moving to high-speed lines in response to a rising demand for poultry. The industry wants the highest feasible speeds for maximum efficiency, and they see machine vision as the way to make it possible while also improving inspection efficacy. Chao says that the high-speed lines separate into two or three more lines after the birds are killed, and more inspectors are added to meet USDA's requirement of a maximum speed of 35 birds a minute for each inspector.
Robots Are Quick Learners
Chao stresses that machine vision requires occasional input from the
inspectors to make assessments and adjustments. While the line is down
for cleaning, the system can update itself based on this input. The
system has to be more intelligent than one that inspects nonbiological
products like bottle labels and caps. "The size and shape of each
cap and the cap's label are all uniform, making variations easy to detect,"
Chao says. "Unfortunately, nature doesn't make chickens with anywhere
near that much uniformity."
Chickens differ not only in size, but also in other key characteristics
such as skin color and chemical composition. These change with each
batch of chickens, depending on the feed they were raised on and the
weather conditions they lived with.
Ironically, some uniformity for chickens may be found in their diseases.
"Chickens are raised so closely together that a disease tends to
spread through the flock," Chao says, "making it more likely
that if one carcass has a disease, others in the batch may also."
The inspectors and plant veterinarians can teach the system based on
their skills and experience and based on what they find as they inspect
the birds rejected by the system. The system has another advantage:
It keeps records on every chicken, and these records can later be used
by the plant to alert poultry growers to conditions in particular batches
Chen's system quickly diagnoses all physical or biological conditions
that cause an inspector to remove a chicken from the processing line.
It does not spot bacterial contamination, but the ARS Poultry Processing
and Meat Quality Research Unit in Athens, Georgia, has signed a CRADA
with Stork Gamco to use machine vision to spot contamination from "ingesta,"
partially digested food from the ruptured crops of chicken carcasses,
and from fecal matter, both of which are associated with bacterial contamination.
The chicken plant of the very near future will likely have two new systems in placeone for the automated diagnosis of wholesomeness and one for fecal and ingesta contaminationcombining the Beltsville and Athens machine-vision systems.
From Apples to All Produce
In a similar way, apple-packing plants will have two or more systems
in place. Some plants currently have automated ways to sort out undersized
apples. The system Chen and his team are working on, when commercialized,
would likely be merged with that system, along with others in the pipeline,
including one by Renfu Lu, an ARS agricultural engineer in East Lansing,
Michigan. He is developing an automated way to sort apples by quality,
including deep internal bruises, as well as taste and firmness. Lu once
worked in Chen's lab, as did Bosoon Park, now an agricultural engineer
in the Athens lab.
Chen applied the experience and technology developed from his chicken
work to detecting fecal contamination on apples. "We had to develop
new technology for this application as well, but the work evolved from
our success with chicken inspection," Chen says.
"We want 100 percent of the apples to be inspected as each passes
the continuously operating lights and camera," Chen says. "We
expect this system to be adaptable for use with all fruits and other
Comis, Agricultural Research Service Information Staff.
This research is part of Food Safety (Animal & Plant Products),
an ARS National Program (#108) described on the World Wide Web at http://www.nps.ars.usda.gov.
|"Machine Vision Sees the Food Contaminants We Can't See" was published in the August 2002 issue of Agricultural Research magazine.|