ARS | Publication request: Invasive Plant Researchers Should Calculate Effect Sizes, Not P-Values

Submitted to: Journal of Invasive Plant Science and Management
Publication Type: Peer Reviewed Journal
Publication Acceptance Date: October 21, 2009
Publication Date: April 1, 2010
Citation: Rinella, M.J., James, J.J. 2010. Invasive Plant Researchers Should Calculate Effect Sizes, Not P-Values. Journal of Invasive Plant Science and Management. 3:106-112.

Interpretive Summary: The most common statistical technique in range is null hypothesis significance testing (NHST). NHST has a variety of major shortcomings. For one thing, NHST only allows for rejection, not acceptance, of null hypotheses. More importantly, what matters most to range scientists and managers is the magnitude of studied effects, and NHST provides no information about that. In this paper, we reanalyze four datasets (including two of our own) from the range science literature to illustrate problems with NHST. Through our reanalysis, we build the case for interval estimates (confidence and credibility intervals) as preferable alternatives to P-values. Confidence and credibility intervals indicate effect sizes, and in comparison to P-values, provide a more complete, intuitively appealing depiction of what data do/do not indicate.

Technical Abstract: Null hypothesis significance testing (NHST) is the default analytical procedure in range science, with over 90% of the recent articles in Rangeland Ecology and Management reporting some kind of NHST result (e.g. P-value, LSD). Unfortunately, NHST is less useful than its ubiquity implies. For one thing, P-values from NHST do not provide what range scientists need most; estimates of the magnitude and uncertainty of studied effects. And even if researchers can live with this shortcoming, P-values are wrought with other problems. Namely, P-values are hard to interpret, are regularly misinterpreted, and are not very informative even when they are interpreted correctly. In this paper, we reanalyze four datasets (including two of our own) from the range science literature to illustrate problems with P-values. Through our reanalysis, we build the case for interval estimates (confidence and credibility intervals) as preferable alternatives to P-values. Confidence intervals and credibility intervals indicate effect sizes, and in comparison to P-values, provide a more complete, intuitively appealing depiction of what data do/do not indicate.