Validation study of the USDA’s Data Quality Evaluation System

Authors: Bhagwat, Seema; Patterson, Kristine; Holden, Joanne

Submitted to: Meeting Abstract
Publication Type: Abstract Only
Publication Acceptance Date: 3/12/2007
Publication Date: 10/24/2007
Citation: Bhagwat, S.A., Patterson, K.K., Holden, J.M. 2007. Validation study of the USDA’s Data Quality Evaluation System. 7th International Food Data Conference, Food Composition and Biodiversity, October 22-24, 2007, Sao Paulo, Brazil.

Interpretive Summary:

Technical Abstract: Introduction: The Nutrient Data Laboratory (NDL) of USDA has conducted a validation study of the USDA Data Quality Evaluation System. The system evaluates the quality of analytical data by rating important documentation concerning the analytical method, analytical quality control, number of samples, sampling plan and sample handling. A “Quality Index” and “Confidence Code” are created for each nutrient and food. Objectives: 1) To measure the variability of ratings assigned by evaluators, 2) To test the robustness of the rating scale and 3) To assess the objectivity of the system categories. Methods: Fifteen individuals who participated in the International Postgraduate Course for Food Composition offered by EuroFIR in Slovakia evaluated a research article containing analytical data on catechin in black grapes. The various rating scores assigned by the participants were analyzed to assess the success of above objectives. The maximum score for each category is 20. Results: Preliminary observations revealed the importance of documenting procuring and handling of food samples as well as analytical methods. Reasonable consistency was observed if the pertinent information was provided as reflected in the ratings for sample handling (12-17 range) and sampling plan (11-16 range. The ratings for the analytical method category ranged from 6.7 to 10.7. The rating for the number of samples category was 14 by all the participants, as reported. The authors did not report conducting analytical quality control although they did use quality control material for the validation of the analytical method. Therefore analytical quality control ratings were zero by all the participants. Conclusions: Clear documentation by authors will reduce the variability in responses. Some questions in the Data Quality Evaluation System about critical issues will require additional refinement and/or specificity, e.g., the difference between the reference material and the analytical quality control material. Future work will assess the assignment of equal rating points for all the categories. Impact: The USDA Data Quality Evaluation System represents one of the first efforts to standardize and harmonize the evaluation of analytical data quality across the international food composition network.