Skip to main content
ARS Home » Midwest Area » Bowling Green, Kentucky » Food Animal Environmental Systems Research » Research » Publications at this Location » Publication #242156
Title: Least-Squares Analysis of Phosphorus Soil Sorption Data with Weighting from Variance Function Estimation: A Statistical Case for the Freundlich Isotherm

Author
item TELLINGHUISEN, JOEL - Vanderbilt University
item Bolster, Carl

Submitted to: Environmental Science and Technology
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: 5/28/2010
Publication Date: 7/1/2010
Citation: Tellinghuisen, J., Bolster, C.H. 2010. Least-Squares Analysis of Phosphorus Soil Sorption Data with Weighting from Variance Function Estimation: A Statistical Case for the Freundlich Isotherm. Environmental Science and Technology. 44(13):5029-5034.

Interpretive Summary: High replication phosphorus sorption data are collected for soil of the Pembroke classification so that the data error structure can be obtained through variance function estimation. The results permit the assignment of reliable weights for the analysis of the data in least-squares fits with common 2-, 3-, and 4-parameter models. Proper weighting of sorption data requires attention to a special problem often ignored in sorption work -- that the dependent variable is obtained in a simple calculation from the measured equilibrium concentration, which is commonly taken as the independent variable yet is subject to experimental error and therefore in conflict with a fundamental least-squares assumption. We handle this weighting problem through an effective variance treatment and also by using a method based on Deming's treatment of least-squares with any number of uncertain variables. The importance of weighting is illustrated by fitting the Langmuir isotherm and three of its linearized forms, which show good mutual agreement when weighted consistently. However, the properly weighted Langmuir fits display too-large chi-square values, indicating that the Langmuir model is inadequate. On the other hand, reasonable chi-square values are obtained when fitting with several 3- and 4-parameter models. On this basis these models are all judged statistically satisfactory.

Technical Abstract: Phosphorus sorption data for soil of the Pembroke classification are recorded at high replication — 10 experiments at each of 7 initial concentrations — for characterizing the data error structure through variance function estimation. The results permit the assignment of reliable weights for the subsequent analysis of the data in least-squares fits with common 2-, 3-, and 4-parameter isotherms as models. The importance of weighting is illustrated through fitting to the Langmuir isotherm and three of its linearized forms, which show good mutual agreement when weighted consistently. However, proper weighting in this case requires attention to a special problem often ignored in sorption work — that the dependent variable S is obtained in a simple calculation from the measured equilibrium concentration C, which is commonly taken as the independent variable, but which is subject to experimental error, in conflict with a fundamental least-squares assumption. The result is almost perfectly correlated error in S and C, which we handle through an effective variance treatment and also by using a method based on Deming's treatment of least-squares with any number of uncertain variables. The properly weighted Langmuir fits display too-large chi-square values, indicating that the Langmuir model is inadequate. On the other hand, reasonable chi-square values are obtained when fitting with several 3-parameter models — the Langmuir plus a constant, the Langmuir-Freundlich, the Redlich-Peterson, the Toth, and the Dubinin-Radushkevich — and also the 4-parameter two-surface Langmuir. On this basis these models are all judged statistically satisfactory.