VIEW ARTICLE

Special Topics

Factors that Influence the Value of the Coefficient of Determination in Simple Linear and Nonlinear Regression Models. J. A. Cornell, Statistics Department, University of Florida, Gainesville 32611; R. D. Berger, Plant Pathology Department, University of Florida, Gainesville 32611. Phytopathology 77:63-70. Accepted for publication 28 February 1986. Copyright 1987 The American Phytopathological Society. DOI: 10.1094/Phyto-77-63.

In the fitting of linear regression equations, the coefficient of determination (R²) is one of the most widely used statistics to assess the goodness-of-fit of the equation. Its value, however, is affected by several factors, some of which are associated more closely with the data collection scheme or the experimental design than with how close the regression equation actually fits the observations. These design factors are: the range of values of the independent variable (X), the arrangement of X values within the range, the number of replicate observations (Y), and the variation among the Y values at each value of X. Another little-known fact is the effect of R² of the ratio of the slope of the fitted equation to the estimated standard error of the observations. In nonlinear model fitting, the value of R² is best determined by calculating the proportion of the total variation in the observations that cannot be explained by the fitted model and subtracting this proportion from one. Several statistics that are analogous to the standard formula for R² in the linear regression case are given and determined to be inappropriate in the nonlinear case. The use of R² alone as a model-fitting criterion is often risky and other statistics should be used to assess the goodness of the model when responses from quantitative treatments are analyzed by regression techniques.