NOTE: We’re happy to welcome back Sarah Greene, one of the founding members in 2009 of SPM and its journal. She left a while ago for London, where she’s continued her work at the leading edge of thought about medical knowledge. 

Sarah is ahead of most of us. Only in the past six months did I begin to comprehend what she was talking about in her “revolution” post three years ago, cited below. So consider that what she says here will quite likely be accepted some years into the future, and we would do well to listen. (e-Patient Dave)

Here in London, ‘peerage’ has a special meaning: as the Free Dictionary puts it, “the rank, title, or jurisdiction of a peer or peeress; a duchy, marquisate, county, viscountcy, or barony.” So when I ran across a recent blog post about Peerage of Science (PoS) – a new peer review invention started in Finland by scientists, for scientists – I thought perhaps they were using the word naively, or tongue-in-cheek. But having now read the PoS-About page, it seems the service may represent a genuflection before that sovereignty which has ruled the research kingdom for the past century.

It was the subtitle of the Nature Network blog post by ecologist Mike Fowler Peerage of Science: A Publishing Revolution?  that caught my attention, as it hearkened back to a piece I wrote for e-Patients.net – Participatory Medicine as Revolution! Think Critically! Communicate! – some while ago. I’m on permanent watch for the “R” word, and though I’ve been on the sidelines of PM for a couple of years, I’m working with a true revolutionary of scientific publishing, Vitek Tracz, to help build his post-publication peer review (PPPR) service, Faculty of 1000. Arguably the originator of the open access publishing platform, Vitek believes there should be no gatekeepers for the deposition of research, and that all refereeing, debating, and revisions should be done transparently after publication. More about this in a minute.

What does the machinery of peer review have to do with participatory medicine (PM), and why am I disappointed in the newly announced PoS that some consider to be revolutionary?  To begin with, many argue that it’s not the concept of peer review that’s holding back scientific discovery, but rather its unrivaled status as the backbone and pre-requisite of journal publishing.  One of the best critiques of the current system was written for the Journal of Participatory Medicine, sister of this blog.  In its inaugural issue Richard Smith, MD, former editor of the British Medical Journal, summarized his observations:

After 30 years of practicing peer review and 15 years of studying it experimentally, I’m unconvinced of its value. Its downside is much more obvious to me than its upside, and the evidence we have on peer review tends to support that jaundiced view.

Not only does the current system delay valuable findings for months or even years, but it can prevent discoveries from ever seeing the light of day. Furthermore, studies have been conducted demonstrating that referees routinely miss serious errors. And in an editorial titled Is Peer Review Censorship?, Infection and Immunity editors Arturo Casadevall and Ferric C. Fang wrote:

If editors are overzealous in screening manuscripts for perceived newsworthiness or consistency with prevailing dogma, there is a danger of blurring the distinction between peer review and censorship. If a reviewer obstructs the publication of a manuscript because it competes with or questions his or her own work, there is an ethical dimension as well.

Yet worse than the delays, worse than the errors that creep through, and worse than censorship and obstruction of competition, is how pre-publication peer review limits what’s available to other researchers to build upon, and to all citizens, including patients who are on the front line, to broaden their perspectives and contribute to the armamentarium. Neuroscientist Bjoern Brembs states it vividly in his blog post Scientific Discoveries are Like Orgasms: You Can’t Have a Bad One. In today’s world of small-N clinical trials, genomic medicine, and a highly networked public, you can be sure that crowd-sourced knowledge will come from unexpected quarters, and will become critical to the discovery process.

Back to our “revolution”: Peerage of Science is assembling qualified peer reviewers and offering credits for their work, which can then be used to publish the reviewers’ own research. Articles are reviewed in a single round and participating journal editors can then track, accept, and publish the refereed content. I’m pleased that a ‘reputation system’ (explained helpfully by Peter Frishauf, founder of Medscape) for the work of peer reviewing is being tested.  But as stated on their website, PoS is (bold emphasis their own):

• not an open peer review service
• not a preprint server like arXiv.org or Nature Precedings – it is closed to the outside world
• not providing an alternative publishing model.

It’s a start, but is it disruptive?  Michael Nielsen, author of the recent powerhouse book, Reinventing Discovery: The New Era of Networked Science, gives a historical perspective in his essay The Future of Science:

The adoption and growth of the scientific journal system has created a body of shared knowledge for our civilization, a collective long-term memory which is the basis for much of human progress. This system has changed surprisingly little in the last 300 years. The internet offers us the first major opportunity to improve this collective long-term memory, and to create a collective short-term working memory, a conversational commons for the rapid collaborative development of ideas. The process of scientific discovery – how we do science – will change more over the next 20 years than in the past 300 years.

And his vision of the future:

 We should aim to create an open scientific culture where as much information as possible is moved out of people’s heads and labs, onto the network, and into tools which can help us structure and filter the information. This means everything – data, scientific opinions, questions, ideas, folk knowledge, workflows, and everything else – the works. Information not on the network can’t do any good.

Ideally, we’ll achieve a kind of extreme openness. This means: making many more types of content available than just scientific papers; allowing creative reuse and modification of existing work through more open licensing and community norms; making all information not just human readable but also machine readable; providing open APIs to enable the building of additional services on top of the scientific literature, and possibly even multiple layers of increasingly powerful services. Such extreme openness is the ultimate expression of the idea that others may build upon and extend the work of individual scientists in ways they themselves would never have conceived.

Recognizing that review and commenting is central to the discovery process, Nielsen bemoans the lack of incentives for sharing knowledge in the culture of science, and indeed the many disincentives:

The contrast between the science comment sites and the success of the amazon.com reviews is stark. To pick just one example, you’ll find ~1500 reviews of Pokemon products at amazon.com…Some people find this contrast curious or amusing; I believe it signifies something seriously amiss with science, something we need to understand and change.

Nielsen goes much further and I highly recommend his book. It’s encouraging that there’s now a vocal wave of enthusiasm from a growing enclave of publishers and scientists, to shed the vestigial processes of reporting research and embrace open collaboration as central to discovery (indeed, check out this brilliant Glowing Map of Science Collaboration created by Olivier Beauchesne at Science-Metrixm, described in David Dobbs’ blog at Wired.com).

Nature, Science, Cell, PLoS, Springer and others are testing the water, but a true revolution will take many more daring experiments; as Richard Smith has noted,

…experimentation inevitably means some failures. ‘In all science, error precedes the truth, and it is better it should go first than last,’ said Hugh Walpole…. It does, however, feel very bold for editors to abandon prepublication of peer review — like walking into the street naked. But if the emperor has no clothes, what’s to be lost? Nothing, but much is to be gained.

Meanwhile, this week we at Faculty of 1000 announce additional rebel fire in the guise of F1000 Research (@F1000Research).  In this open access repository for original research analysis and data, we’ll tackle all the “not’s” described above by Peerage of Science, with immediate publication followed by open review. Read more about it in the inaugural blog post penned by editor Rebecca Lawrence.

NOTE: Andrzej Krauze illustrated the Quixote drawing for the F1000 2012 calendar. His brilliant satirical art appears regularly in The Guardian, New Scientist, The Scientist and many other publications worldwide.