The following post is from a recent issue of Nature, which highlights the importance of replication and open data for science. However, some of the examples might apply more to medicine or biology than population science. Lest, readers think that this issue doesn’t apply to demographers, here’s a tweet from Justin Wolfers, advertising a piece in Bloomberg Business on the importance of replication for the field of economics. His motivation is the recent dust-up due to an error in a famous paper by Reinhart and Rogoff [See PSC-Info], but the discussion is much broader than that example.
[Link to Stevenson/Wolfers Replication article]
INTRODUCTION TO SPECIAL NATURE ISSUE
No research paper can ever be considered to be the final word, and the replication and corroboration of research results is key to the scientific process. In studying complex entities, especially animals and human beings, the complexity of the system and of the techniques can all too easily lead to results that seem robust in the lab, and valid to editors and referees of journals, but which do not stand the test of further studies. Nature has published a series of articles about the worrying extent to which research results have been found wanting in this respect. The editors of Nature and the Nature life sciences research journals have also taken substantive steps to put our own houses in order, in improving the transparency and robustness of what we publish. Journals, research laboratories and institutions and funders all have an interest in tackling issues of irreproducibility. We hope that the articles contained in this collection will help.
Reducing our irreproducibility
(25 April ,2013)
Further confirmation needed
A new mechanism for independently replicating research findings is one of several changes required to improve the quality of the biomedical literature.
Nature Biotechnology 30, 806 ( 10 September 2012 )
Biologists must realize the pitfalls of work on massive amounts of data.
Nature 487, 406 ( 26 July 2012 )
Must Try Harder
Too many sloppy mistakes are creeping into scientific papers. Lab heads must look more rigorously at the data — and at themselves.
Nature 483, 509 ( 29 March 2012 )
NEWS AND ANALYSIS
Independent labs to verify high-profile papers
Nature News ( 14 August 2012 )
Power Failure: Why small sample size undermines the reliability of neuroscience
Katherine S. Button, John P. A. Ioannidis et al.
Nature Reviews Neuroscience 14, 365-376 ( 15 April 2013 )
Replication studies: Bad copy
Nature 485, 298-300 ( 17 May 2012 )
Reliability of ‘new drug target’ claims called into question
Nature Reviews Drug Discovery 10, 643-644 ( September 2011 )
If a job is worth doing, it is worth doing twice
Jonathan F. Russell
Nature 496, 7 ( 04 April 2013 )
Methods: Face up to false positives
Nature 487, 427-429 ( 26 July 2012 )
Drug development: Raise standards for preclinical cancer research
C. Glenn Begley & Lee M. Ellis
Nature 483, 531-533 ( 29 March 2012 )
Believe it or not: how much can we rely on published data on potential drug targets?
Florian Prinz, Thomas Schlange & Khusru Asadullah
Nature Reviews Drug Discovery 10, 712 ( September 2011 )
Tackling the widespread and critical impact of batch effects in high-throughput data
Jeffrey T. Leek, Robert B. Scharpf et al.
Nature Reviews Genetics 11, 733-739 ( October 2010 )
PERSPECTIVES AND REVIEWS
Research methods: know when your numbers are significant
David L. Vaux
Nature 492, 180-181 ( 13 December 2012 )
A call for transparent reporting to optimize the predictive value of preclinical research
Story C. Landis, Susan G. Amara et al.
Nature 490, 187-191 ( 11 October 2012 )
Next-generation sequencing data interpretation: enhancing reproducibility and accessibility
Anton Nekrutenko & James Taylor
Nature Reviews Genetics 13, 667-672 ( September 2012 )
The case for open computer programs
Darrel C. Ince, Leslie Hatton & John Graham-Cumming
Nature 482, 485-488 ( 23 February 2012 )
Reuse of public genome-wide gene expression data
ohan Rung & Alvis Brazma
Nature Reviews Genetics 14, 89-99 ( February 2013 )