I’ve only been studying healthcare for two years – far less than most people on this blog – and I hesitate to be overly assertive. But I have, finally, reached the point where I feel confident in citing cases where people are simply being unscientific: ignoring evidence. That’s always hazardous, and it becomes insidious when it’s caused by a blind, unquestioning belief in our institutions.
Case in point:
Julie Thoren is an active practitioner of Participatory Medicine who’s seen first-hand the tremendous value informed e-patients share with each other. This morning she wrote to me about a post, How online patient communities make money from patient data, on the blog of KevinMD, a neighbor of mine in Nashua NH.
Kevin pointed out, correctly, that patients need to have their eyes open about what might be done with their data, etc. Commenter “Doc D” cited the chestnut “The plural of anecdote is not data,” and Julie responded about Participatory Medicine and the value she’s found in online communities.
Here’s my comment (with a few edits), reflecting on the realities I’ve personally seen and what others have written in JoPM (the Journal of Participatory Medicine):
_________________________
This is a complex and multidimensional subject. I couldn’t agree more with Kevin’s basic assertion that patients need to be on guard.
On the other hand, I know first-hand (and second-hand many times over) that Julie’s got her facts straight that many many many patients have suffered at length because they presumed the right/smart thing to do was whatever their physician told them, and who only learned of other options – sometimes life-altering – from an online patient community.
Patients and caregivers (and, I’d assert, physicians and insurers and policy people) simply need to wise up about the difficulty and skills and knowledge and risks that health and healthcare involve. In my view, only when someone realizes all of that, can they make truly informed decisions without blinders on or their head in the sand. JoPM editorial board member Trisha Torrey has written a well documented book about this: You Bet Your Life: Ten Mistakes Every Patient Makes / How to fix them to get the health care you deserve. Those are not imaginary stories – including her own, where the physicians, in whom she of course had faith, told her she had six months to live due to inoperable cancer, when in fact she didn’t have cancer at all, and she only found it out through her own research.
I couldn’t agree more with Doc D that the plural of anecdote is not data. Any patient community that doesn’t understand this is being unscientific. On the other hand, it’s equally unscientific to think that our established channels are reliable sources of information – especially to think they’re the only reliable source. I won’t take time here to go into that in depth, but I’ll quote this from the Journal of Participatory Medicine:
“…evidence on the upside of peer review is sparse, while evidence on the downside is abundant. We struggle to find convincing evidence of its benefit, but we know that it is slow, expensive, largely a lottery, poor at detecting error, ineffective at diagnosing fraud, biased, and prone to abuse … most of what appears in peer-reviewed journals is scientifically weak.”
Was that said by an angry disillusioned patient, mad at publications like the British Medical Journal? Why no, it was written by Richard Smith, after 25 years as editor of the British Medical Journal.
Then we have the recent news that the PatientsLikeMe community beat Lancet to the punch, by a long shot, regarding lithium, after a peer reviewed journal had published a ludicrously inept paper (contradicted by both PLM and Lancet).
No smart patient will unthinkingly trust a patient community (nor a physician), online or off. We all need to wise up and be smart consumers. And based on Smith’s writing, I assert that no smart physician or consumer will unthinkingly trust evidence from medical journals.
_______________
Thanks to Julie for bringing this stimulating topic to my attention. Kevin, I suspect I don’t see entirely eye-to-eye with you on this, but as I said, I think we share a common sense that people need to wise up.
(Another “wise up” tool, btw, is Gary Schwitzer’s Health News Review site, supported by the Foundation for Informed Medical Decision Making.)
I think this has got to be the consistent message in educating patients to help them empower themselves: exhaust all possible resources to gather information, but don’t blindly trust any one source, be that doctors, an online forum or anything else.
For two years we’ve been working on tools for e-patients and other consumers to gather, store and share the results of their searches for and analysis of online health information. We’ve offered this technology, with no strings attached, to ACOR.org and other e-patient groups but there’s been little or no interest. Yet all I continue to hear are rather vague complaints and admonitions about the quality of traditional medical information resources. Stop talking about it and do something practical, reusable and scalable.
Mark, are you complaining to me because I personally haven’t adopted your tools, or ACOR’s volunteer community hasn’t?
If so, among other things what does that have to do with the point of this post – trustworthiness of various sources of information?
Dave,
Based on your own story, I think you’d agree that a high-quality source of health information is the data collected by assertive and well-informed e-patients and then “pressure-tested” against other informed opinions. Resounding Health was founded to provide a system to create a crowd-sourced textbook of personalized medicine from the e-patient’s point of view. In the present environment, most hard-worn, e-patient knowledge is diluted or even lost among hundreds of web sites, discussion boards and other social media sites. What was needed, in my view, was a mechanism and set of tools to capture all of this information and organize it on a rigorous underlying ontology framework. Only then can synergies be found and the information effectively re-used beyond that patient’s group of twitter followers. I’ll address the current contents of the Resounding Health knowledgebase and provide some use cases in a separate write-up. I’m sorry if I haven’t communicated this information clearly to you in our past encounters.
Mark,
I am always interested in using technology that can help some, if it has no strings attached and doesn’t redirect users of the technology to any site, robot, API, or any other service that collects ANY information, including meta-data, about the users. If you are telling me that you are willing to donate to ACOR your code so that we can install on one of our machines I am very interested. Otherwise, it’s not exactly with no strings attached, is it?
I am not one who complains about the quality of our forums. I wish patients had conversations with their doctors that were always based on as-solid evidence as you can get on the good ACOR communities. Adding services won’t improve the quality of the exchanges but will provide better navigability, findability, etc.
Giles,
You and I spoke on the phone more than six months ago and I explained this all to you. I also offered to volunteer my personal time cleaning up all of the broken links and other errors on the ACOR site. You initially responded positively but then disappeared despite repeated attempts to contact you for follow up. So what you’re saying now is that you’ll take free stuff but do nothing with it.
Mark,
there is a great difference between talking about things and providing replicable proof that what you offer is of any value to our users. I don’t remember seeing any form of validated study showing that your site or technology offers services that benefit patients. I would love to see such a proof. Saying “”it is just so” multiple times doesn’t provide that proof.
The ACOR site is broken beyond simple repair. It requires a full redesign. But the remarkable aspect of it is that even with a broken website ACOR is helping about 8,000 more people than 6 months ago. And that is irrefutable evidence that the main services offered by our organization are of real value. How many users are using your site, BTW?
Resounding Health is supported by a Small Business Innovation Research Award from the National Institutes of Health. One of the goals of this research project is to prove (or not) whether we can organize high-quality, consumer health information in more effective ways. We will publish our results in a peer-review journal when the analysis is complete. Our success metrics for this Phase I grant don’t include user experiences at this point. Formal testing by Pew-like surveys and focus groups will come in Phase II. If you’d like to assess my background and qualifications for this research, please visit http://www.markboguski.net
Dave,
One reason I can’t wait for the video to be posted from Health 2.0 Paris (I understand there’s an issue with the recording – English audio overlaying the French) is b/c information quality was a significant part of the conversation. Who decides which information is worthy? Who is guiding people, and to which sources? Is there a need for regulation of health information?
It was such a dominant theme that I would like to reprise some survey questions from 2006 regarding how consumers decide which information to trust online. Of course I’d also love to field those questions in the EU to obtain a comparison sample.
See:
Online Health Search 2006
http://www.pewinternet.org/Reports/2006/Online-Health-Search-2006.aspx
Summary:
Most internet users start at a general search engine when researching health and medical advice online. Just 15% of health seekers say they “always” check the source and date of the health information they find online, while another 10% say they do so “most of the time.” Fully three-quarters of health seekers say they check the source and date “only sometimes,” “hardly ever,” or “never,” which translates to about 85 million Americans gathering health advice online without consistently examining the quality indicators of the information they find. Most health seekers are pleased about what they find online, but some are frustrated or confused.
Within the report I qualified that finding:
Health seekers might be forgiven if they give up what at times is a search for a needle in a haystack. A recent study commissioned by the U.S. Department of Health and Human Services (HHS) finds that a tiny percentage of health sites display the source and date of the information on their pages.
And now, of course, we know that “health sites” represent only part of the information stream coming into consumer’s search results and explorations. Advice is welcome on how to ask people how they tell good information from bad on social network sites, blogs, etc.
Susannah,
I remember reading that “needle in a haystack” report last year! I couldn’t agree more that it needs to be repeated, to see how things have shifted (or not) in the 4-5 years since that data was collected.
To the point of our headline: are e-patients wising up? If not, let’s get to work!
Ooo, another tasty Gilles Frydman phrase:
In a comment on the original post at KevinMD, he used the term “the age of exponential data.” Google says the phrase already existed, but I’ve never heard it applied to healthcare. It’s a great addition to the points in the e-patient white paper about the information explosion.
I’m adding this to the one he spouted at the ICSI / IHI Colloquium last month: “All knowledge is in constant beta.”
e-Patient Dave
Your “recent news” link gives a 404 error. Out of curiosity I Googled PatientsLikeMe or “patients like me” + lancet + lithium and could find no recent news PatientsLikeMe beating Lancet to detecting a flawed paper. The phrase “recent news” is a bit misleading as the link appears to be self referential within your blog.
Rick,
Apologies for the broken link, and thanks for letting us know. I fixed it.
Yes, if all I did was link to our own stuff it wouldn’t add much value. :–) As you’ll see, it links to a piece in MIT Technology Review.
I have to take exception to your lauding Trisha Torrey’s anecdote-rich book in one paragraph and then supporting the statement “The plural of anecdote is not data” in the next. Half of Trisha’s book is anecdotal. I could counter with the usual response that “the plural of datum is not proof,” but I think your Richard Smith quotation does that.
The statement “the plural of anecdote is not data” attemtps to dismiss anecdotal evidence as meaningless, which it most certainly is not. The plural of anecdote, I would argue, is evidence. Like all evidence, the anecdotal variety requires a degree of mitigation–not necessarily vetting, but that’s one means of mitigation.
As for Ms. Fox’s question about improvements in Internet resource-gathering over the past five years, I doubt you’d get any different polling data. Most readers are not trained to determine the validity of sources. Fortunately, a number of the larger available sources are getting more thorough at obtaining validation for us. Look at Wikipedia or WebMD, for example. Both have gotten much better in the past half decade at obtaining and dating their sources. Wikipedia even regularly annotates entries to point out where source data are needed.
We’re getting there. The Internet is not, on its own, a reliable source of information. But it’s getting there.
Training e-patients to check their sources is a good idea, but perhaps it would be easier to teach physicians to ask e-patients if they’ve verified their sources. I’ve had to train my doctors in this. Each of them initially wanted to discount information I brought to them from Internet sources. After I demonstrated that I’m careful to check my sources, though, they’ve each learned that saying, “I downloaded this” doesn’t automatically discount its validity.
Hi Dennis,
Thanks for pulling us (me?) in the direction of rigor.
The point of my citing the “plural of” saw was to acknowledge that I don’t think a set of anecdotes constitutes a rigorous finding. (Too often I hear from people who’ve read something on this blog and think all patients are idiots about reaching conclusions. Those people haven’t read much of the blog, but I try occasionally to head them off.)
And yes, when my physician (and Society VP) Dr. Danny Sands speaks to professionals, he counsels them to coach patients. The other day in Ontario he suggested simple steps: Ask the patient if they browse; ask where; and ask if they’ve found anything good. Because, he says, it’s not unusual for patients to show him something good that he hadn’t seen yet.
Trumping all of this, I think, is the vitally important awakening that’s needed on the part of patients, physicians, administrators and policy people: (a) healthcare isn’t nearly as reliable as we all have come to think; (b) patients have far more power to contribute usefully to this situation than most would imagine.
Hence the tagline in our blog’s title: “Because health professionals can’t do it alone.”
Hi Dave,
thank you for your post. I made some comments to it on my Blog http://patient-centered-it.com.
For me online communities should be approached like other traetments :)
Check out:
http://patient-centered-it.com/2010/08/01/online-communities-vs-treatments/
Love it! Thanks, Christian! Good to meet you.
Have you joined the Society for Participatory Medicine yet? It sounds like you’re practically a member. It’s cool – co-chairs are a patient and physician.