What's Wrong With Research Literatures? And How to Make Them Right

George S. Howard, Trey L. Hill, Scott E. Maxwell, Telmo Mourinho Baptista, Miguel H. Farias, Claudia Coelho, Marcie Coulter-Kern, Russell Coulter-Kern

Research output: Contribution to journalArticle

13 Citations (Scopus)

Abstract

Meta-analysis is now the accepted procedure for summarizing research literatures in areas of applied psychology. Because of the bias for publishing statistically significant findings, while usually rejecting nonsignificant results, our research literatures yield misleading answers to important quantitative questions (e.g., How much better is the average psychotherapy patient relative to a comparable group of untreated controls? How much more aggressive are children who watch a great deal of violent TV than children who watch little or no violence on TV?). While all such research literatures provide overly optimistic meta-analytic estimates, exactly how practically important are these overestimates? Three studies testing the literature on implementation intentions finds only slightly elevated effectiveness estimates. Conversely, in three studies another growing research literature (the efficacy of remote intercessory prayer) is found to be misleading and is in all likelihood not a real effect (i.e., our three studies suggest the literature likely consists of Type I errors). Rules of thumb to predict which research literatures are likely invalid are offered. Finally, revised publication and data analysis procedures to generate unbiased research literatures in the future are examined.

Original languageEnglish
Pages (from-to)146-166
Number of pages21
JournalReview of General Psychology
Volume13
Issue number2
DOIs
Publication statusPublished - Jun 2009
Externally publishedYes

ASJC Scopus subject areas

  • Psychology(all)

Fingerprint Dive into the research topics of 'What's Wrong With Research Literatures? And How to Make Them Right'. Together they form a unique fingerprint.

  • Cite this

    Howard, G. S., Hill, T. L., Maxwell, S. E., Baptista, T. M., Farias, M. H., Coelho, C., ... Coulter-Kern, R. (2009). What's Wrong With Research Literatures? And How to Make Them Right. Review of General Psychology, 13(2), 146-166. https://doi.org/10.1037/a0015319