In medicine, to achieve the best you need the best information. So an essential question is, who gets to say what’s best?
That question took a sharp turn this week with the news that Medpedia is dead.
Medical librarian Laika Spoetnik has a strong post on the demise, including an interview with its founder, James Currier: Medpedia, the Medical Wikipedia, is Dead. And we Missed its Funeral. The bottom line is:
The crowd trumps credentials.
It’s another nail in the coffin of the outdated idea that authority with massive credentials – and investor money – will give you better information than a mob with no investors, governed only by community rules. Read Laika’s post for several analyses back then; here’s the short version of the concept as I saw it:
- In 2007 “Doc Tom” Ferguson’s white paper (see the right column of this site) documented how information in patient communities sometimes goes well beyond what their clinicians have seen.
- In my own 2007 kidney cancer case, all the treatment information on established medical websites was out of date, but my patient community had the latest. What’s up with that??
- So Medpedia seemed like a great idea: a medical Wikipedia, where patients and clinicians could collaborate on keeping things up to date.
- But, as I posted then in Medpedia: where are the patients:
… when I saw their home page it literally took my breath away: there are invitations for doctors to join, and organizations to contribute content, but nothing for patients – we can contribute ideas, not content.
A crowdsourced medical encyclopedia was fine in concept, but its implementation was unaware that reality has changed. (Ferguson saw it; their investors didn’t.) So among other things, Wikipedia – an uncredentialed, leaderless crowd – has often outperformed Medpedia, whose reliance on credentials was its entire raison d’être: if the premise fails, the enterprise flops. So it goes.
The problem was predicted by our Feb 2009 post Who gets to say what info is reliable? which drew 70 comments. The post asked, “Who will vet the vetters?” Comments included this, from SPM founder John Grohol:
… If you start with a closed, walled-off garden to begin with in terms of contributors, you’re already starting at a disadvantage.
Wikipedia showed that shared knowledge can work … but it takes the devotion of thousands of dedicated, altruistic people. Medpedia, although perhaps well-intentioned, is coming at health and medical knowledge from a distinctly 1.0 “docs know best” philosophy.
It’s fascinating to see that old-school philosophy be grafted on to a 2.0 tool and business model, in hopes of generating something new and different. I have my doubts, reading through the tripe listed currently for mental disorders (full of misinformation).
Laika’s post links to numerous comments by sharp observers, including internet mega-maven Clay Shirky, and ScienceRoll doc @berci, who said back then:
I believe elitism kills content. Only the power of masses controlled by well-designed editing guidelines can lead to a comprehensive encyclopaedia.
The very first medical conference I ever attended was Connected Health, October 2008, where Shirky said in a keynote:
The patients on ACOR don’t need our help, and they don’t need our permission.
Here’s to the visionaries – the people who understand what’s going on, below the surface, and can see the future, so we can build on it.
I encourage you to read Laika’s post, including all the links. She’s @LaikaS on Twitter.
Thanks, Dave and Laika! I have now read both posts and found this chestnut from 2009 to add to the pile:
Shared Kismet: Wikipedia and the NIH
http://pmedicine.org/epatients/archives/2009/07/shared-kismet-wikipedia-and-the-nih.html
The comments, as usual, are as interesting as the post, particularly the exchange between Judson, Gilles & I regarding what is a credible source, eligible for citation on Wikipedia.
That post is one example of many on this blog — flares on the horizon for people who would like to know what’s coming in health and health care.
After years of honing my observation skills, scanning the present for clues about the future, I am no longer surprised when a project like Medpedia fails. I am only surprised when people invest millions of dollars and thousands of hours in a project that is doomed.
Events like this, the shuttering of a massive mistake, inspire me to redouble my efforts to communicate more effectively about what I’m seeing in the data we collect at the Pew Research Center. I’ve adopted a phrase that makes me feel better when someone ignores our research or – yes, this happens – laughs at the implications: “I’ll wait. You’ll see.”
Susannah, I’m SO glad you posted this – four years ago this was way over my head. I read your post again; I urge all interested readers to let it sink in.
A question from the post –
> Frank Schulenberg, Wikimedia: Wikipedia does not work in theory, but only in practice.
Did he mean “Theoretically it can’t work, but in reality it does”?
Here’s a point that SO many scientists (including health policy people) forget:
> John Broughton, author of “Wikipedia – The Missing Manual”: what is a good article? Factually accurate, verifiable, neutral
*Verifiable*. As we’ve repeatedly pointed out here, despite medicine’s blind hammering on honoring only “the literature” / “the evidence,” most studies in the literature have never been verified by having an independent lab replicate the work. This is medical science’s Achilles’ Leg – way bigger than the heel.
And that means it’s a great big mistake to assert “Trust what’s published in a high-impact journal.” Real science is always open to independent verification.
And I say that has to include verification by people whose careers aren’t tied to the academic peer review system.
______
And btw, this exchange earns an Epic Foreshadowing badge:
> NCI q: wary of anonymous editors
> Vickers: Net is anon medium; we must scale, so onus is on cited sources, not personal credentials
To your question about the Schulenberg’s quote: I think you’re right that he meant that Wikipedia looks unsustainable from the outside, at first glance, but it works somehow — and very well, by most measures.
I spent a long time looking for a post that I thought I wrote around the same time, maybe a year later, about the HON Code. It’s also a long-time member of my list of “the world has moved on” ideas. If you remember it or see it, please let me know.
Re you writing about the HON code: boy, Google’s not helping – the best I can find is this comment of yours, 7/23/10, on your “Crowdsourcing a survey” post:
______
We asked our most in-depth questions related to trust & credibility of health information online in 2001. Here is the key section of our 2002 “Vital Decisions” report in which I compared two ways of asking the questions (and why researchers get quite different answers):
http://pewrsr.ch/csYH6o
I have been meaning to write a blog post (and at this point, maybe a whole report) about the questions I am now thinking about regarding “seals of approval.” It was a hot topic at Health 2.0 Paris: gov’t oversight of online health content, the HON Code, etc.
MedPedia is no more not only because it was a bad idea, but even more so because it represented itself as THE medical reference source, without having any of the characteristics of a contemporary reference site.
In the 21st century, any online medical resource not having direct and constant involvement from patients is a guaranteed failure in the making.
In a world of ever expanding knowledge, just saying you are the reference simply doesn’t work. People have become far too intelligent to just go along a marketing statement. Success is based only on providing a service that people find of value. And there is no value if the only people allowed to provide input are those defending their turf.
Too ambitious to really get off the ground, the site was based on the false premise that all medical information must be vetted by professionals from major institutions. A good dose of humility could have helped create a useful crowdsourced medical resource. But then, why recreate wikipedia? Creating any medical resource that remains constantly current and accurate is much more difficult that just getting some content from major medical institutions. Without free and regular conversations, it can’t work.
As Gilles has quite rightly pointed out about Medpedia:
“In the 21st century, any online medical resource not having direct and constant involvement from patients is a guaranteed failure in the making.”
The initial premise of the the site looked good, but the practicalities and basics of making it work were not set up.
I also love his “A good dose of humility could have helped”
John
I am one of the medpedia editors. The idea that some professional physician share and update professional medical information is useful and feasible in a specific mode, as pillole.org demontrates. A few devoted foolish professional physicians provide critical appraisal and short summary of some medical papers.
But we are few and we are foolish, none earns money and the cost of the project is 50 usd per year for the shared web server. The content providing system is old as the asp languages demonstrates. the layout is old 1.0 because we are not professional informatics we are physicians.
Of course the traffic generated by our visitors who are professionals mostly is low and a shared web server is sufficient. We have been spending our free time in this project since 1998! But if the traffic would be bigger also the shared server would become not sufficient to manage the traffic and the cost for a more strong informatic system would compel to find money and it means sponsors!! This would mean the end of our independence and the end of the reason itself for our existence.
Why a further medical-dedicated wiki should be if it were not physician driven? In which aspect should be different from others well known longstanding wiki general community driven? It is just the competence of real professional physicians that makes sense to a project that is interesting just because is different from others.
Unfortunately or not, depending from the point of view, not all are foolish as we are. I spent many nights to read, to summarize to comment the studies we review. This is the problem: the professionals are responsable for their contents and may provide the contents also on a professional basis it means much time and many resources. It is not possible to sustain the project of an independent site sharing regularly updated, professional content just on a voluntary basis. The role of not professional for such a site may be not for providing contents or editing but to check the availability, the indepency the readability of the contents.
Sorry for my awful English.
Luca Puccetti MD
pisa Italy
Luca, thank you very much for your thoughtful comment! I can see that this is important to you, and I’m grateful for the many hours you and your colleagues have contributed to providing good information for the public – since 1998!
Do you and your colleagues at Pillole also contribute on Wikipedia? I’m very interested in how we can make good use of your generous help.
(To help a little, I fixed a few spelling mistakes in your comment. Thank you again.)
Thanks Dave, No we do not contribute to wikipedia. We do not know who really controls the contents published in wikipedia that is the problem!
The editors in our project have been carefully selected on the basis of a longstanding, deep, personal knowledge of their professional skillness and real independency as well.
I appreciate your help in correcting the mistakes
regards
Luca Puccetti, MD
Pisa Italy