Can we trust traditional peer review? If it’s broken, how might we fix it?
These questions are put to a panel of experts in this Journal of Participatory Medicine podcast. The spirited discussion accompanies 2 articles on the subject in JPM’s inaugural issue: “In Search of an Optimal Peer Review System”, by former British Medical Journal Richard Smith, and “Reputation Systems: A New Vision for Publishing and Peer Review” by Medscape founder Peter Frishauf.
In addition to Smith and Frishauf, participating in the podcast are peer review researcher Liz Wager, health policy researcher Alex Jadad, and computer scientist Thomas (Bo) Adler. More complete bios may be found by clicking on the participants’ names at the top of this page. Click here to see a list of recommended resources compiled by the participants.
While there are many views on getting to a better form of peer review, the panelists agreed on a number of criteria that ideal peer review should include. Bo Adler and Frishauf led a discussion on specific improvements that might come about through an Internet-based reputation system that is now being beta tested in Wikipedia.
Chapters
Chapter 1: Defining the Problems and Issues with Peer Review Today [download]
[powerpress url=”/wp-content/uploads/2014/05/JOPM_Chap1.mp3″]
Chapter 2: Light Versus Heavy Peer Review [download]
[powerpress url=”/wp-content/uploads/2014/05/JOPM_Chap2.mp3″]
Chapter 3: Transparency in Peer Review [download]
[powerpress url=”/wp-content/uploads/2014/05/JOPM_Chap3.mp3″]
Chapter 4: Wikipedia-Style Peer Review…and Rating/Reputation Systems [download]
[powerpress url=”/wp-content/uploads/2014/05/JOPM_Chap4.mp3″]
Chapter 5: Crowdsourcing Research/Peer Review [download]
[powerpress url=”/wp-content/uploads/2014/05/JOPM_Chap5.mp3″]
Chapter 6: Building a Community [download]
[powerpress url=”/wp-content/uploads/2014/05/JOPM_Chap6.mp3″]
Full Discussion
I think it’s wonderful that the Journal of Participatory Medicine organized this discussion and made it readily available. Biomedical research scientists living and working within the current paradigm can all too easily become so focused on getting ahead within the current peer review-dominated career ladder (in which reputation and grants follow publication in prestigious journals) that they never think about how fundamentally flawed the peer review system actually is, and how unnecessary this 19th Century relic is in the Internet Age.
It is heartening to see journals like PLoS and Nature taking small steps to correct this, but I think that real change is going to come from the next generation of scientists, who will begin publishing in online journals with sophisticated quality and reputation-monitoring systems. Tomorrow’s top-tier journals don’t exist yet, but this discussion provides a foundation for how they’ll be developed.