Identifying data anomalies in milk component measurements from partial-day milking records

Authors: WU, XIAO-LIN - Council On Dairy Cattle Breeding; CAPUTO, MALIA - Council On Dairy Cattle Breeding; DONATONE, CHIP - Council On Dairy Cattle Breeding; Miles, Asha; Baldwin, Ransom; SIEVERT, STEVEN - Collaborator; MATTISON, JAY - Collaborator; COLE, JOHN - Council On Dairy Cattle Breeding; BURCHARD, JAVIER - Council On Dairy Cattle Breeding; DURR, JOAO - Council On Dairy Cattle Breeding

Submitted to: Journal of Dairy Science Communications
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 11/14/2025
Publication Date: 12/13/2025
Citation: Wu, X., Caputo, M.J., Donatone, C., Miles, A.M., Baldwin, R.L., Sievert, S., Mattison, J., Cole, J.B., Burchard, J., Durr, J. 2025. Identifying data anomalies in milk component measurements from partial-day milking records. Journal of Dairy Science Communications. https://doi.org/10.3168/jdsc.2025-0825.
DOI: https://doi.org/10.3168/jdsc.2025-0825

Interpretive Summary: The quality of milk is based on the primary constituents (water, fat protein, lactose, minerals and other minor components. The collection and evaluation of milk component records and their correlation with quality are crucial for accurate genetic evaluation and effective daily herd management to produce high-quality milk. However, potential errors in the measurement or assessments of the individual constituents can compromise the validity of downstream management and genetic selection decisions. To evaluate the quality and consistency of records from a single milking from herds across different farms, we have used a statistical method called intraclass evaluation to identify farms where there are concerns about the quality milk component records. Beyond the herd, we also need to know the quality of records for each cow in the herd. In this study we determined that a new statistical metric, individual-level intraclass correlations (I-ICC) can be used. Using I-ICC was superior to other methods for identifying anomalies. This research will enhance the ability of dairy producers to identify suspect milk samples and improve milk component records at the individual cow level allowing dairy producers to make accurate choices in breeding and management strategies on farm.

Technical Abstract: High-quality milk and milk component data are crucial for accurate genetic evaluations and effective daily herd management. However, potential errors can compromise the validity of downstream decisions. In a recent study, we proposed using intraclass correlation (ICC) as a herd-level metric to assess the consistency of milk components from single milkings, thereby effectively identifying farms with potential data quality concerns. Yet a key challenge remains: can we effectively identify potentially erroneous records at the cow-day level? In the present study, we introduce a novel metric – individual-level intraclass correlations (I-ICC) – to assess data consistency at the cow-day level, and we evaluate its performance in comparison with three commonly used methods. We also introduce a two-step approach to estimate percentile thresholds for flagging outliers. The results demonstrate the superior performance of this new metric over the conventional univariate and multivariate methods in identifying anomalies in correlated partial daily milk component data. The Impact of data shuffling was also examined. Together, these methods provide robust and practical tools for detecting suspect milk component records at the individual cow-day level.