Just before Thanksgiving, Microsoft released a study entitled, “Cyberchondria: Studies of the Escalation of Medical Concerns in Web Search.” Ryen White and Eric Horvitz took advantage of a data set that few people have access to (log files from Microsoft’s Live Search engine and MSN Health and Fitness) as well as a survey of 515 Microsoft employees. They also did a great service to those of us who have a problem with the term “cyberchondriac” since they define cyberchondria as “the unfounded escalation of concerns about common symptomatology, based on the review of search results and literature on the Web.” That does not describe most internet users and therefore, people might think about retiring the term from general usage.
There were 350 males and 165 females in their sample and the median age was 35 (plus or minus 8 years). I’d hazard a guess that they are all college educated. So one’s antennae might go up just on those grounds since that’s not even close to a representative sample of the current internet population.
But right away I was drawn in by their methods and their research questions. I think it’s a fantastic idea to harness the power of log files to investigate what people are actually doing online. That’s why I’m a big fan of Bill Tancer of Hitwise, who once told me that they have to remove all data related to porn from their data set in order to see anything else. “Adult” searches just swamp the rest of what’s happening online, apparently. That’s the beauty of log files over phone surveys: we can only get 15% of adults to tell our interviewers that they have visited adult websites.
But that’s also where my antennae went up again — the researchers use the word “may” a bit too often for me to feel comfortable with all of their assertions related to their very cool data set. For example:
Identifying the recurrence of concerns about a rare disorder — especially when the recurrence occurs during another search task — may indicate that earlier escalations extend over time, and that anxieties or heightened awareness continue to interrupt users’ online activities over prolonged time periods. Such findings may be proxies for the rise and persistence of deep concerns that may disrupt other aspects of daily life.
Or they may not!
John Markoff’s November 24 article in the New York Times, however, helped me put my finger on why I thought I’d heard something like this before: “Medical Web Searches and Escalating Fears: Microsoft Finds Top Results Lead to Anxiety.” Markoff described Eric Horvitz as an artificial intelligence researcher (bing! a bell goes off in my head) and concludes with the following:
In the 1990s, Microsoft researchers built a health advisory system for pregnancy and child care. Mr. Horvitz said that in the future it would be possible to create search engines that were able to detect medical queries and offer advice that did not automatically make Web searchers fear the worst.
Bing! Bing! Bing! Now I know where I have heard something like that before: Diana Forsythe in her classic article, “Old Wine in New Bottles,” described how artificial intelligence researchers, when designing a migraine information kiosk (yes, it was the 80s), didn’t ask patients what they wanted to know. They asked one doctor what he thought patients should know about their condition. And guess what? He didn’t include the answer to the number one question asked by migraine sufferers: Am I going to die from this?
No, the migraine patients weren’t going to die from the pain, but isn’t it their right to ask the question? In asking it, does it mean they want to “escalate” their anxieties? Who should be the trusted guide? Is it Microsoft?
I hope the conversation about this study continues since I am intrigued by the questions it raises. And here I’ll give the last word to John Naughton, a columnist for The Observer:
The moral? If you’re a hypochondriac, stay off the web. And remember that, as the man said, the best cure for your condition is to forget about your body and become interested in somebody else’s.
I think more than 15% of American adults would agree.
The issue of what can be learned from adult web sites (often touted as the leading edge on the web) is an interesting one. While I do not doubt that there is a huge discrepancy in the reported number vs. the actual number of people who view adult sites, there is one other huge factor to consider. Those who frequent porn sites probably do it a lot — sticky content at its best!
How does this apply to cyberchondriacs? I would venture to guess that those who lean that way spend a lot of time searching and reading, but not enough sleeping at night!
Could be most health searchers are far more balanced in their approach, but less exciting to watch.
C~
> asked one doctor what he thought patients should know
Thank you for naming a recurring elephant in the room. This is so wrong-headed. And, more to the point, it’s “doctosaurus” thinking, not patient-oriented. How dumb, when designing something for patients!
It’s quite as if a retailer designed its store based on what the engineers thought people would want to know about.
A year ago at the Kidney Cancer Association’s Boston “Patient Day,” as they called it, the entire agenda was designed that way. It was horrid! The #1 question on the mind of a new patient is just as in your migraine story: Am I going to die?? But they didn’t even touch on it.
Please please please, people in healthcare, ASK US! Ask the patients! It’s not hard!
When e-patient pioneer Doc Tom was asked once to present to first-graders about being a doctor, he did a radical thing: he got in touch with them in advance and asked them what they’d like to know about. He got some surprising answers: one little girl wanted to know what’s in blood, and several wanted to know “What’s this stethoscope thing all about??” So he brought stethoscopes, a microscope, and blood slides.
I know your academic training is to study in isolation and think what to present to others. Within HC that may be fine, but NOT TO PATIENTS! Arg!
And here’s the irony: I ultimately learned that for whatever reason, the KCA isn’t the least bit interested in what patients want to hear about. I outlined an alternate track of patient-centered topics, starting with:
1) Cancer is no longer a death sentence;
2) Understanding statistics;
3) Building a support community
… etc, 7 sessions for a day-long conference. I sent it to them via email twice, and wrote a paper letter, and never heard a thing back.
Especially ironic is that after the opening session, during Q&A I asked the speaker about the freshness of his data. He agreed that it was not relevant to anyone trying to understand their odds today, because all the data was collected before any of today’s treatments existed.
!!!!!!!!!!!!!!!
What are people thinking when they dispense (valid) data like that, at a so-called “Patient Day”?
You do have to be careful in interpreting data in a vacuum, or with a biased sample. They tried to take care of the vacuum with the survey, but then made the results of the survey virtually useless as something generalizable (unless you happen to be a white collar employee who works for Microsoft).
Second, they recognized the problem with analyzing only search queries versus the far richer navigational paths of users. A navigational path would provide more contextual information — information that would really confirm (or disconfirm) their hypotheses about “escalation.” Their failure to find a method to analyze this data is unfortunate.
Third, I believe it’s natural human curiosity to explore the “worst case scenario,” especially when it comes to our own or a loved one’s health. We want to prepare ourselves for the worst possibilities, so when we’re told it’s just hypertension, it’s a pleasant relief.
After all, when you investigate a new car, don’t most people nowadays check out the crash ratings of their car? Does that mean people are more inclined to imagine themselves getting into an accident? Or simply doing more research on the topic because tools like search engines make such research practically free (except for our time). The opportunity costs are the same, so this finding, if true, isn’t even surprising.
Very, very good insights, John! Thanks!
I’m coming late in the game but wanted to mention one more potential problem with the study that the great expert at the NY Times didn’t even mention: is there anywhere behind the study a corporate reason to position Healthvault as a better alternative to Dr. Google?
So, the study is conducted by Ryen W. White and Eric Horvitz, 2 full time employees of Microsoft. That may start to explain why they publish on the research.microsoft.com platform. We have been talking a lot about conflict of interest when relating to misuse of statistical data.
Why not look into the potential reasons for Microsoft to publish research that demonstrate why the 80% of medical searches done by the American public using their favorite search engine, Google, “could” be dangerous.
Why not look into the potential conflict of interest of the 2 researchers in promoting a better way to get “scientific and proven” diagnosis by entering your data into Microsoft Healthvault PHR.
Are we all so numb that we never question what seems to be the most obvious problems of corporation funded research? Researchers working hard to resurrect a moribund term usually do so for a very good reason. I wish the so-called expert journalist from the NY Times would have abandoned for a while his role of groupie of Eric Horvitz, the President of the Association for the Advancement of Artificial Intelligence, and started to behave like journalists used to behave, asking hard questions.
In any case, it’s time to retire cyberchondria for another 10 years. It is also time for John Markoff to retire for the same length of time. By then the New York Times will probably be in a similar situation to where GM is today, an old and beaten whale, unable to survive in its former shape, if at all.
Informed patients will, by then, have no patience for transparent abuse of power by the experts of yesterday.
I just Googled “cyberchondria” to see how far the term reaches (363k links) and of course it led me to the Wikipedia entry. The entry is interesting, but more for the meta commentary by MaxEnt:
I’m extremely dubious about mention of Harris in the lede. The first thing that went through my head was “this is Sut Jhally writ large”. How does the market Harris serves wish this term to function? Probably they want patients showing up at their doctor’s office requesting exotic medications for conditions of borderline concern. Does it help them if the negative, neutral, or confused interpretation prevails? It’s a non-trivial task to determine what is being spun and to whom.(endquote)
This goes toward the conflict of interest discussion we are having elsewhere.