Journal of Participatory Medicine
Search all of the Society for Participatory Medicine website:Search
The Journal for Participatory Medicine's website has moved. Please check out the new website for the latest articles.

Peer Review and Reputation Systems: Recommended Reading

The participants in “Peer Review and Reputation Systems: A Discussion” have each compiled a list of useful resources on this topic.

Liz Wager:

Bo Adler:

  • AcaWiki.org is like the community that I tried to build, which lets people write summaries about academic papers and gives a place for people to discuss them―sort of like a global reading group.
  • Mendeley.com is a bibliography tool for researchers, but allows them to “link” to other researchers to create a social network. They also collect statistics about research trends based on information from their users, which is like a reputation system for research areas.
  • Of course, the ultimate peer review system would be a combination of the features of AcaWiki, Mendeley, and PLoS, plus the ability to track reputations of authors/scientists.
  • Golbeck, JA. Computing and Applying Trust in Web-Based Social Networks. Guide to Computing Literature; 2005.
    Using a social networking aspect to temper the reputation system, as Golbeck does in her thesis can help reduce the perception of competition between researchers since there is no “absolute” reputation value, only perspectives from various points in the social network.
  • Facebook for Science? One group thinking about 21st century reputation problems is the Peer-to-Peer University project. I have spoken to them briefly about some ideas for students to provide feedback to each other, which could be parlayed into a reputation system and could be extended to a wider group of people. As the population continues to grow, being able to know “who’s who” in a field gets harder and harder, which I think is one reason why sites like LinkedIn and Plaxo (and social media in general) are growing in popularity.
  • The PLoS article metrics are fascinating for new ways of thinking about the “impact” of academic papers. An example article that has some comments is available at PLoS ONE.
  • You can learn more about the WikiTrust project at their website: http://wikitrust.soe.ucsc.edu/.

Richard Smith:

  • Godlee F, Jefferson T. Peer Review in Health Sciences. 2nd ed. London: BMJ Books; 2003. [Google Scholar]
  • Smith R. Peer review: A flawed process at the heart of science and journals. J R Soc Med 2006;99:178-182. [Google Scholar]
  • Smith R. The Trouble With Medical Journals. London: RSM Press; 2006. [Google Scholar]
  • Altman DG. Poor-quality medical research: What can journals do? JAMA. 2002;287:2765–2767.  [Google Scholar]
  • Altman DG. Statistics in medical journals. Stat Med. 1982;1:59–71.  [Google Scholar]
  • Andersen B. Methodological Errors in Medical Research. Oxford: Blackwell; 1990. [Google Scholar]
  • Altman DG. The scandal of poor medical research. BMJ. 1994;308:283–284. 
    [Google Scholar]
  • Jefferson T, Rudin M, Brodney Folse S, Davidoff F. Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database of Systematic Reviews 2007, Issue 1. Art. No.: MR000016. DOI: 10.1002/14651858.MR000016.pub3 [Google Scholar]
  • Demicheli V, Di Pietrantonj C. Peer review for improving the quality of grant applications. Cochrane Database of Systematic Reviews 2007, Issue 1. Art. No.: MR000003. DOI: 10.1002/14651858.MR000003.pub2 [Google Scholar]
  • Schroter S, Black N, Evans S, Godlee F, Osorio L, Smith R. What errors do peer reviewers detect, and does training improve their ability to detect them? J R Soc Med. 2008;101: 507-514. [Google Scholar]
  • McNutt RA, Evans AT, Fletcher RH, Fletcher SW. The effects of blinding on the quality of peer review. A randomized trial. JAMA. 1990;263:1371-1376.
    [Google Scholar]
  • Justice AC, Cho MK, Winker MA, Berlin JA, Rennie D. The PEER investigators. Does masking author identity improve peer review quality: A randomised controlled trial. JAMA. 1998;280:240-242. [Google Scholar]
  • van Rooyen S, Godlee F, Evans S, Smith R, Black N. Effect of blinding and unmasking on the quality of peer review: A randomized trial. JAMA 1998;280:234-237. [Google Scholar]
  • van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers’ recommendations: A randomised trial. BMJ. 1999;318:23-27. [Google Scholar]
  • Schroter S, Black N, Evans S, et al. Effects of training on the quality of peer review: A randomised controlled trial. BMJ 2004;328:657–658. [Google Scholar]
  • Surowiecki J. The Wisdom of Crowds: Why the Many Are Smarter Than the
    Few. London: Abacus, 2005. [Google Scholar]
  • Leadbeater C. We-think, Mass Innovation, Not Mass Production. London: Profile, 2008. [Google Scholar]
  • Shirky C. Here Comes Everybody: How Change Happens When People Come Together. London: Penguin, 2009. [Google Scholar]
  • Frishauf P. The end of peer review and traditional publishing as we know it. Available at: http://www.medscape.com/viewarticle/583316 Accessed October 3, 2009 (free registration required). [Google Scholar]
  • Adler BT, de Alfaro, L. A content driven reputation system for the Wikipedia. In WWW 2007, Proceedings of the 16th International World Wide Web Conference, ACM Press, 2007. Available at: http://users.soe.ucsc.edu/~luca/papers/07/wikiwww2007.html. Accessed October 14, 2009. [Google Scholar]

Peter Frishauf:

 

Donate