ARS | Publication request: Using R^2 to compare least-squares fit models: When it must fail

Submitted to: Chemometrics and Intelligent Laboratory Systems
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: January 5, 2011
Publication Date: January 13, 2011
Repository URL: http://handle.nal.usda.gov/10113/55052
Citation: Tellinghuisen, J., Bolster, C.H. 2011. Using R^2 to compare least-squares fit models: When it must fail. Chemometrics and Intelligent Laboratory Systems. 105:220-222.

Interpretive Summary: R2 is arguably the most frequently used, and abused, metric for judging goodness of fit. Indeed, R^2 is often used to select the best-fit model among competing models when using least-squares regression. In this study we show through the use of computer generated data how R^2 can be properly used when comparing different forms of the same model. Our results show that with proper weighting R^2 can be used correctly for model comparisons; however, R^2 comparisons then become equivalent to comparisons of the estimated fit variance s^2 in unweighted fitting, or of the reduced chi-square in weighted fitting with weights taken as inverse variances. The latter metrics are much more easily interpreted, and thus are better than R^2 for such purposes. When models are compared by fitting data that have been mathematically transformed in different ways, with proper weighting, s^2 and chi-square remain valid; but R2 fails miserably.

Technical Abstract: R^2 can be used correctly to select from among competing least-squares fit models when the data are fitted in common form and with common weighting. However, then R^2 comparisons become equivalent to comparisons of the estimated fit variance s^2 in unweighted fitting, or of the reduced chi-square in weighted fitting with weights taken as inverse variances. The latter metrics are arguably more easily interpreted, thus better than R^2 for such purposes. When models are compared by fitting data that have been mathematically transformed in different ways, with proper weighting, s^2 and chi-square remain valid; but R2 fails miserably.