This issue of Nature is a compilation of replication articles across several issues of Nature. They highlight the importance of replication and open data for science. However, some of the examples might apply more to medicine or biology than population science. Lest, readers think that this issue doesn’t apply to demographers, here’s a tweet from Justin Wolfers, advertising a piece in Bloomberg Business on the importance of replication for the field of economics. His motivation is the recent dust-up due to an error in a famous paper by Reinhart and Rogoff [See PSC-Info], but the discussion is much broader than that example.
[Link to Stevenson/Wolfers Replication article]
INTRODUCTION TO SPECIAL NATURE ISSUE
No research paper can ever be considered to be the final word, and the replication and corroboration of research results is key to the scientific process. In studying complex entities, especially animals and human beings, the complexity of the system and of the techniques can all too easily lead to results that seem robust in the lab, and valid to editors and referees of journals, but which do not stand the test of further studies. Nature has published a series of articles about the worrying extent to which research results have been found wanting in this respect. The editors of Nature and the Nature life sciences research journals have also taken substantive steps to put our own houses in order, in improving the transparency and robustness of what we publish. Journals, research laboratories and institutions and funders all have an interest in tackling issues of irreproducibility. We hope that the articles contained in this collection will help.
Reducing our irreproducibility
(April 25 , 2013)
Further confirmation needed
A new mechanism for independently replicating research findings is one of several changes required to improve the quality of the biomedical literature.
Nature Biotechnology 30, 806
(September 10, 2012)
Biologists must realize the pitfalls of work on massive amounts of data.
Nature 487, 406
(July 26, 2012)
Must Try Harder
Too many sloppy mistakes are creeping into scientific papers. Lab heads must look more rigorously at the data — and at themselves.
Nature 483, 509 x
(March 29, 2012)
NEWS AND ANALYSIS
Independent labs to verify high-profile papers
(August 14, 2012)
Power Failure: Why small sample size undermines the reliability of neuroscience
Katherine S. Button, John P. A. Ioannidis et al.
Nature Reviews Neuroscience 14, 365-376
(April 15, 2013)
Replication studies: Bad copy
Nature 485, 298-300
(May 17, 2012)
Reliability of ‘new drug target’ claims called into question
Nature Reviews Drug Discovery 10, 643-644
If a job is worth doing, it is worth doing twice
Jonathan F. Russell
Nature 496, 7
(April 4, 2013)
Methods: Face up to false positives )
Nature 487, 427-429 \
(July 26, 2012)
Drug development: Raise standards for preclinical cancer research )
C. Glenn Begley & Lee M. Ellis
Nature 483, 531-533
(March 29, 2012
Believe it or not: how much can we rely on published data on potential drug targets? )
Florian Prinz, Thomas Schlange & Khusru Asadullah
Nature Reviews Drug Discovery 10, 712
Tackling the widespread and critical impact of batch effects in high-throughput data
Jeffrey T. Leek, Robert B. Scharpf et al.
Nature Reviews Genetics 11, 733-739 )
PERSPECTIVES AND REVIEWS
Research methods: know when your numbers are significant
David L. Vaux
Nature 492, 180-181
(December 13, 2012)
A call for transparent reporting to optimize the predictive value of preclinical research
Story C. Landis, Susan G. Amara et al.
Nature 490, 187-191
(October 11, 2012)
Next-generation sequencing data interpretation: enhancing reproducibility and accessibility
Anton Nekrutenko & James Taylor
Nature Reviews Genetics 13, 667-672
The case for open computer programs
Darrel C. Ince, Leslie Hatton & John Graham-Cumming
Nature 482, 485-488
(February 23, 2012)
Reuse of public genome-wide gene expression data
ohan Rung & Alvis Brazma
Nature Reviews Genetics 14, 89-99