This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Keselman, H. J. Articles by Levin, J. R. Search for Related Content Social Bookmarking                   What's this?

Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses

University of Manitoba

Carl J. Huberty

University of Georgia

Lisa M. Lix

Private Scholar

Stephen Olejnik

University of Georgia

Robert A. Cribbie

University of Manitoba

Barbara Donahue

University of Georgia

Rhonda K. Kowalchuk

University of Manitoba

Laureen L. Lowman

University of Georgia

Martha D. Petoskey

University of Georgia

Joanne C. Keselman

University of Manitoba

Joel R. Levin

University of Wisconsin, Madison

Articles published in several prominent educational journals were examined to investigate the use of data analysis tools by researchers in four research paradigms: between-subjects univariate designs, between-subjects multivariate designs, repeated measures designs, and covariance designs. In addition to examining specific details pertaining to the research design (e.g., sample size, group size equality/inequality) and methods employed for data analysis, the authors also catalogued whether (a) validity assumptions were examined, (b) effect size indices were reported, (c) sample sizes were selected on the basis of power considerations, and (d) appropriate textbooks and/or articles were cited to communicate the nature of the analyses that were performed. The present analyses imply that researchers rarely verify that validity assumptions are satisfied and that, accordingly, they typically use analyses that are nonrobust to assumption violations. In addition, researchers rarely report effect size statistics, nor do they routinely perform power analyses to determine sample size requirements. Recommendations are offered to rectify these shortcomings.

Review of Educational Research, Vol. 68, No. 3, 350-386 (1998)
DOI: 10.3102/00346543068003350


CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				R. F. Dedrick, J. M. Ferron, M. R. Hess, K. Y. Hogarty, J. D. Kromrey, T. R. Lang, J. D. Niles, and R. S. Lee Multilevel Modeling: A Review of Methodological Issues and Applications Review of Educational Research, March 1, 2009; 79(1): 69 - 102. [Abstract] [Full Text] [PDF]

				R. I. Parker, S. Hagan-Burke, and K. Vannest Percentage of All Non-Overlapping Data (PAND): An Alternative to PND Journal of Special Education, February 1, 2007; 40(4): 194 - 204. [Abstract] [PDF]

				R. K. Henson Effect-Size Measures and Meta-Analytic Thinking in Counseling Psychology Research The Counseling Psychologist, September 1, 2006; 34(5): 601 - 629. [Abstract] [PDF]

				S. G. Aldana, W. R. Whitmer, R. Greenlaw, A. L. Avins, D. Thomas, A. Salberg, A. Greenwell, L. Lipsenthal, and G. W. Fellingham Effect of Intense Lifestyle Modification and Cardiac Rehabilitation on Psychosocial Cardiovascular Disease Risk Factors and Quality of Life Behav Modif, July 1, 2006; 30(4): 507 - 525. [Abstract] [PDF]

				G. V. Seco, M. C. Izquierdo, M. P. F. Garcia, and F. J. H. Diez A Comparison of the Bootstrap-F, Improved General Approximation, and Brown-Forsythe Multivariate Approaches in a Mixed Repeated Measures Design Educational and Psychological Measurement, February 1, 2006; 66(1): 35 - 62. [Abstract] [PDF]

				R. R. Wilcox An Approach to Ancova that Allows Multiple Covariates, Nonlinearity, and Heteroscedasticity Educational and Psychological Measurement, June 1, 2005; 65(3): 442 - 450. [Abstract] [PDF]

				R. K. Kowalchuk, H. J. Keselman, J. Algina, and R. D. Wolfinger The Analysis of Repeated Measurements with Mixed-Model Adjusted F Tests Educational and Psychological Measurement, April 1, 2004; 64(2): 224 - 242. [Abstract] [PDF]

				R. A. Cribbie and H. J. Keselman The Effects of Nonnormality on Parametric, Nonparametric, and Model Comparison Approaches to Pairwise Comparisons Educational and Psychological Measurement, August 1, 2003; 63(4): 615 - 635. [Abstract] [PDF]

				S. Finch, N. Thomason, and G. Cumming Past and Future American Psychological Association Guidelines for Statistical Practice Theory Psychology, December 1, 2002; 12(6): 825 - 853. [Abstract] [PDF]

				S. Guimond and M. Dambrun When Prosperity Breeds Intergroup Hostility: The Effects of Relative Deprivation and Relative Gratification on Prejudice Pers Soc Psychol Bull, July 1, 2002; 28(7): 900 - 912. [Abstract] [PDF]

				H. J. Keselman, R. A. Cribbie, and R. R. Wilcox Pairwise Multiple Comparison Tests when Data are Nonnormal Educational and Psychological Measurement, June 1, 2002; 62(3): 420 - 434. [Abstract] [PDF]

				S. R. Hooper, C. W. Swartz, M. B. Wakely, R. E. L. de Kruif, and J. W. Montgomery Executive Functions in Elementary School Children With and Without Problems in Written Expression J Learn Disabil, January 1, 2002; 35(1): 57 - 68. [Abstract] [PDF]

				J B. Feland, J W. Myrer, S. S Schulthies, G. W Fellingham, and G. W Measom The Effect of Duration of Stretching of the Hamstring Muscle Group for Increasing Range of Motion in People Aged 65 Years or Older Physical Therapy, May 1, 2001; 81(5): 1110 - 1117. [Abstract] [Full Text] [PDF]

				R. E. Kirk Promoting Good Statistical Practices: Some Suggestions Educational and Psychological Measurement, April 1, 2001; 61(2): 213 - 218. [PDF]

				H. J. Keselman, J. Algina, R. R. Wilcox, and R. K. Kowa Testing Repeated Measures Hypotheses When Covariance Matrices are Heterogeneous: Revisiting the Robustness of the Welch-James Test Again Educational and Psychological Measurement, December 1, 2000; 60(6): 925 - 938. [Abstract] [PDF]

				T. Vacha-Haase, J. E. Nilsson, D. R. Reetz, T. S. Lance, and B. Thompson Reporting Practices and APA Editorial Policies Regarding Statistical Significance and Effect Size Theory Psychology, June 1, 2000; 10(3): 413 - 425. [Abstract] [PDF]

				B. Thompson Statistical Significance Tests, Effect Size Reporting and the Vain Pursuit of Pseudo-Objectivity Theory Psychology, April 1, 1999; 9(2): 191 - 196. [Abstract] [PDF]