Michael Pollan’s answer to diet angst is to “Eat food. Not too much. Mostly plants.” Is there an equivalent maxim for information angst? If not, does someone out there want to make one up? Because a new study published in Cancer argues that e-patients can take a common-sense approach to online health research and do just fine.
I was able to obtain a full-text copy of the report, so here are a few lines you won’t read in the press release:
Few consumers consider the quality of online health information as they conduct their web searches. In light of our findings, perhaps this is not a bad thing.
Wow. Did anyone else hear the scream of a million librarians and accreditation executives?
This finding goes along with what other researchers have found about Google being a “good enough” diagnostic tool. It also goes along with what Tom Ferguson always said: “e-patients believe that they can find the good stuff online.”
Speaking of Tom, the second sentence in the Cancer journal article cites a meme that has taken on a life of its own: “every day, more patients seek health information online than visit a physician.” The Pew Internet Project gets the citation, but that is a vintage DocTom data point that he fed to us while we were writing the report.
But wait! There’s more. This phrase stopped me for a moment: “information toxicity,” or the harm that comes to people if they follow bad advice found online which, according to the study’s authors “may be underreported in the published medical literature.” Do you think there’s a chance that the opposite of information toxicity is also underreported? What should that be called?
One possibility is to call it “information therapy,” so I emailed Josh Seidman, the executive director of the Center for Information Therapy to get his views on this study. He wrote back that his own research “corroborates their findings that proxy measures of information quality do not bear much relationship to the actual accuracy and comprehensiveness of Internet health information” (PDF). Josh also pointed out that “inaccuracy can be fairly limiting as a gold-standard information quality marker, which is part of why I sought to determine both accuracy and comprehensiveness (as did RAND).”
By citing RAND, Josh is pointing to the gold standard (or “Journal Article Zero” if you don’t love the premise that doctors know best) in the online information-quality debate: “Evaluation of English and Spanish Health Information on the Internet” (Journal of the American Medical Association, Vol. 285, No. 20, May 23 2001, pp. 2612-2621). I compare all other studies to it and would love to see it replicated in 2008 so we can find out if we’ve actually made progress, as this much less ambitious study suggests.
In case you missed it, check out this Seven-Word Wisdom contest:
http://well.blogs.nytimes.com/2008/01/17/seven-word-wisdom-the-contest/
One of the problems of almost all evaluations of online medical information is their limitations. It is always based on search results & always web-centric. Therefore they completely miss many if not most of the online communities of care, since a respectable medical community system will protect the privacy of its members by blocking all search engine robots. This is so frustrating! I can tell you that every day on one or more of the ACOR lists someone will post information with a profound impact on the life of QOL of other list members. We know that in general teh quality of the information is as high (and often better) than what can gathered directly from a search engine result. Search engines do not put their results in perspective but community members DO!
Anyone visiting http://acor.org will see that we do not have the HON certified logo. There is a good reason for it. HON is so web-centric and so focused on static information that 1/2 of the questions asked during the certification process make ABSOLUTELY no sense for medical online communities.
I think there is a space in between “doctor knows best” and “leave the e-patient alone and he/she will find what is needed”, and that is a high quality, interactive partnership between physicians and their e patients. It is challenging, though, to describe these dimensions and, besides, one size doesn’t fit all anyway. Engaging large numbers of providers in the world of on line communication with and between patients about health issues seems to me to be the best way to craft a new model of higher quality medical care than either hewing to the old in-office model or encouraging the patients to forge ahead independent of their physicians’ input.
Goodness, they don’t give us patients much credit, do they?
I’ve been at this for three years and never had any “information toxicity” problems, personally — or from my readers.
See my story in Newsweek:
http://www.newsweek.com/id/104413
Thanks for the comments! I highly recommend reading the full-text of the article if you can (I wish it were publicly available instead of behind a $$ subscription wall). One of the lead authors, Funda Meric-Bernstam, also recorded an audio interview:
http://www.medpagetoday.com/HematologyOncology/BreastCancer/tb/8302
She comments on Wikipedia, the value of “community correction” in breast cancer forums, and her previous work which can be turned up with a search on Google Scholar or PubMed.
Well, there is a very real and documented phenomenon of information overload — that people can become so overwhelmed by the amount of information, they don’t do a very good job organizing it or necessarily paying as much attention as they should to where it comes from (e.g., not all information is equal).
Google (or whatever search engine eventually takes its place) will always trump everything else, because it’s where we go for all of our needs, regardless of what they are.
I think it’s always been kind of silly to say that if an article carries an author’s name and date, it’s likely to be more “accurate” in its content than an article that doesn’t. The two have always had a tenuous relationship at best.
So this study confirms what many of us have suspected was always the case — that there is no easy way to determine information accuracy. Period. Online or off. Nearly every scientific journal has published “inaccurate” research (some days its hard for me to find a journal accurate where its authors don’t cross over into hyperbole in their discussion section, jumping from what their data show to speculation about causative relationships that are nowhere to be found in the study).
The answer is what Tom recognized — that information is empowerment, and to consume information is to become educated and learn to differentiate the good from the less good information.
I think this focused learning process occurs with a lot more frequency for some concerns over others (chronic or life-threatening, versus short-term or non life-threatening). And that if we try and generalize about all “e-patients” or the like, we’re probably doing a disservice to most of them.
John just wrote: “The answer is what Tom recognized — that information is empowerment, and to consume information is to become educated and learn to differentiate the good from the less good information.”
One of the newest articles from JMIR, “Impact of Web Searching and Social Feedback on Consumer Decision Making: A Prospective Online Experiment” shows that as usual Tom foresight was multidimentional. Here is a small quote:”There was an improvement in the percentage of correct answers after searching and after feedback with other subjects’ answers”.
See http://www.jmir.org/2008/1/e2
Looks like this topic has stirred us up a bit. Thanks, Susannah, for starting.
I want to reiterate and expand the point Gilles made earlier in the discussion, that the hightest quality data is probably in more private communities, and not easily searched. In addition, it’s the conversations that is critical. A conversation is dynamic, not a static set of facts and it’s very hard to assess the quality of such interactions.
That said, I’d also like to encourage us all to keep an open mind about the idea of information overload and information toxicity. I’m married to a psychiatrist who uses technology as a tool with strick limiations. I’ve heard many times from her about the ill effects of information overload that each of us face every day. The juggling of multiple inputs and soruces of data, multi-tasking, and otherwise bathing ourselves ad lib in this sea of information probaby IS toxic if taken to an extreme. I’m not just talking about health information, but ALL informaiton. The ability to set personal filters is probably just as important for someone seeking knowledge and support for a medical condition, as for someone interested in fly fishing.
I hope that my post conveys my respect for the researchers’ project and conclusions (all the while raising questions about some of the points they make). Linda Stone, a Pew Internet Project board member and originator of the “continuous partial attention” concept, has some interesting/alarming theories related to this discussion.
You can read her thoughts here:
http://continuouspartialattention.jot.com/WikiHome
Or take a deep cleansing breath and click here (you’ll see why in a minute):
Diagnosis: Email Apnea?
http://radar.oreilly.com/lindas/
Linda Stone’s “continuous partial attention” concept and Dan’s post about Information Toxicity have to be associated with “identity crash”:
http://www.zephoria.org/quotes/2007/09/identity_crash.html
“Identity crash: Sudden and catastrophic collapse of an individual’s ability to keep all the threads of his or her online identity straight when the individual joins one too many social networks. Example: I was ok keeping up with Facebook, Flickr, and Myspace, but after throwing lawlink, Last.fm, and Orkut into the mix, I had a total identity crash and forgot what went where.” — Urban Dictionary (tx Becky)
Dude! (That’s the “dude” usage as exclamation, not a form of address…)
I’m going to think about this more (this is a quick workday note), but here’s a start:
Gilles hits the #1 nail on the head:
> Search engines do not put their results
> in perspective but community members DO!
There are multiple challenges associated with this. For one thing he’s right about resources that hide themselves from search engines.
Another is that many patients (“when the patient is [just] a googler”) don’t yet know that they need to install a junk filter in their research process. Note: THAT APPLIES TO PUBLISHED, PEER-REVIEWED JOURNAL ARTICLES AS MUCH AS IT DOES TO MyFlakyUncleSaysSo.com. MOST of the misleading information I found in researching my own cancer came from well respected sites – it was all out of date! (Yes I’m shouting.)
The defense for both is SMART USER COMMUNITIES. ACOR listserves are one example.
What we need is information that is peer-reviewed BY PEOPLE WHOSE BUTTS ARE ON THE LINE RIGHT NOW: other patients and their supporters.
I suggest that anyone who dares to write about how we should seek medical information should first have their @ss on the line – their very survival. And believe me, it helps to start with a sense of urgency – like the 5.5 month median survival time I read about in January 2007.
When you REALLY need a lifeline, that’s when you get a personal sense of what’s important in a rope.
And while “rope scientists” certainly have important information to offer, I say they’re bad scientists if they don’t go out into the field and see whether or not their recommendations are saving lives.
Btw, I can’t emphasize enough that reading the white paper had an important and unexpected effect on me: it gave me a greater sense of MY responsibility in the partnership with my primary.
Putting the finishing touches on my e-book (transcript of my CaringBridge journal), in the prolog I cited Norman Cousins, and discovered that the introduction to Anatomy of an Illness begins: “The basic theme of this book is that every person must accept a certain measure of responsibility for his or her own recovery from disease or disability.” I’m hijacking that, or extending it, to include personal responsibility for knowing how to filter junk out of what I find on the web.
I assert that what we’re all about is responsible, empowered partnership, and by donning the e-patient moniker, I’m accepting responsibility for whatever data I bring to the conversation.
In two posts Sunday I said:
‘E-patient means something specific, a set of values and behaviors worth knowing about, and worth distinguishing from “just googling.”‘
and
“If your physician is new to the idea, share, don’t clobber.”
One last thing – I left this blog and discovered that Paul Levy, CEO of Boston’s Beth Israel Deaconess (the first hospital CEO to blog), posted something extraordinary yesterday at
http://runningahospital.blogspot.com/2008/02/you-can-take-your-15-fee-and.html:
“Finally, I have had enough. I want to state this clearly and directly: When a respected medical journal issues a press release about a given article that has important public policy ramifications but does not make available the full text of the article, it is a bad thing. It inhibits full public understanding of the issue and makes us beholden to other people’s interpretation of the article. It is inconsistent with the general principle of academic discourse and also is counterproductive in facilitating an informed debate on issues.”
And when commenters focused on whether he should have access to articles, he replied:
“Folks, thanks for all your helpful advice, but this isn’t about me! It is about the hundreds and maybe thousands of normal people who don’t work in a hospital or medical school and might be interested in this topic and might want to be involved in the debate.”
Huzzah. Go read his post, and add your 2c. (He was psyched when he discovered and read our white paper recently.)
On my walk to work this morning, I came up with this Seven-Word Wisdom for information angst:
Go online. Use common sense. Don’t panic.
(Alternative ending: Ignore quacks.)
Or maybe:
Go online. Ask other e-patients. Filter appropriately.
On Line Dieting
Approach dieting holistically with ‘the last weight