VIEW ARTICLE

Ecology and Epidemiology

Assessing the Accuracy, Intra-rater Repeatability, and Inter-rater Reliability of Disease Assessment Systems. F. W. Nutter, Jr., Associate professor, Department of Plant Pathology, 351 Bessey Hall, Iowa State University, Ames 50011; M. L. Gleason(2), J. H. Jenco(3), and N. C. Christians(4). (2)(3)associate professor and graduate research assistant, respectively, Department of Plant Pathology, 351 Bessey Hall, Iowa State University, Ames 50011; (4)professor, Department of Horticulture, 138 Horticulture Building, Iowa State University, Ames 50011. Phytopathology 83:806-812. Accepted for publication 30 March 1993. Copyright 1993 The American Phytopathological Society. DOI: 10.1094/Phyto-83-806.

Dollar spot of bentgrass, caused by Sclerotinia homoeocarpa, was chosen as a model pathosystem to evaluate the accuracy and precision of several disease assessment methods. Quadrats were assessed by visually estimating percent disease severity, by measuring percent reflectance of sunlight at 600 and 800 nm, and by image analysis of color photographic slides. Visual and reflectance assessments were performed by four raters and repeated 24-h later to obtain a measure of intra-rater repeatability. Linear regression of the original assessments (Y) versus the repeated assessments (X) revealed that intra-rater repeatability was highest for the 600-nm radiometric method as indicated by slope values not significantly different from 1.0 (P ≤ 0.01) and intercepts close to and not significantly different from zero (P ≤ 0.01). Coefficients of determination (R²) also were highest for this method, ranging from 98.7 to 99.6%, whereas R² values relating intra-rater visual assessments ranged from 83.4 to 93.1%. Inter-rater reliability was highest using the 600-nm radiometric method as determined from regression equations relating one rater’s visual and radiometric assessments to assessments performed by other raters. Slopes and intercepts among raters using the 600-nm radiometric method did not differ significantly from 1.0 and zero, respectively. However, two of six intercepts and six of six slopes measuring inter-rater reliability of the visual assessment method were significantly different. Slopes significantly different from one indicate the presence of systematic bias among raters, whereas intercepts significantly different from zero indicate the presence of constant sources of error among raters. Radiometric assessments also had a better relationship (R²) to the true level of dollar spot severity as determined with an acetate sheet-image analysis technique. The radiometric assessment method provided a fast, accurate method to measure dollar spot severity that was more precise than were visual assessments.