We’ve often cited Gary Schwitzer’s Health News Review (@HealthNewsRevu on Twitter) as an invaluable e-patient resource. With a structured ten point evaluation process, the site’s many trained reviewers evaluate the reporting of health news. We reported on their first makeover two years ago, and they’ve just announced another.

I like it – especially the new name of their blog, Health News Watchdog. And for the first time, they’re accepting comments and enabling dialog with the reviewers. Well done!

Why it matters:

As HNR documents, mainstream health reporting is too often full of crap. Most doctors don’t even realize this, so we patients out here are told to stay off the internet and stick to mainstream news sources. What’s an e-patient to do??

Get educated, get engaged, learn specifics. Anyone can say “that story’s crap,” but this site gives a framework for our thoughts. When reading about a new treatment, ask:

1. How often do the claimed benefits occur? It’s not uncommon for a 1% or 2% response to get massive publicity. See their informative post last January about statins.
2. How often do harms occur? You can’t evaluate claimed benefits without knowing the risks, but too often health reporters don’t ask.
3. How strong is the evidence? Way too often, a reporter blindly accepts what a press release says.
4. Is the condition it treats exaggerated? The landmark books Selling Sickness and Overtreated established that diseases are often overblown for commercial purposes. No health reporter can overlook this issue and be considered competent.
5. Are there alternative options? Often a press release will, not surprisingly, tout the product’s advantages as if it were in a vacuum. It’s a health reporter’s job to report it in context. Many stories don’t.
6. Is this really a new approach? Surprisingly often, an existing product is re-packaged , or re-promoted as new, and an untrained reporter doesn’t notice.
7. Is it available to me?
8. What’s the total cost of the treatment being covered?
9. Who’s promoting it? If the story doesn’t mention where the information came from, readers can’t evaluate its objectivity.
10. Do they have a conflict of interest?

Based on these criteria, stories are rated from zero to five stars. Of the 1,653 articles reviewed so far, 39% have earned more than three stars, and 11% earned only one or zero. (Yes, 1 in 9 mainstream media health stories were almost useless!)

Rating not just stories, but organizations’ long-term records:

A new section on news organizations’ overall ratings (across many articles) is informative. Among the best ratings:

• LA Times (130 stories) got 4 or 5 stars 55% of the time, and 0-2 stars only 20%
• Associated Press (238 stories): 57%; 0-2, 18%.
• NY Times (114 stories): 4-5 52%, and 0-2 stars 21%
• NPR (39 stories): 4-5 stars 57%, and 0-2 21%
• USA Today (50): 4-5 52%; 0-2, 24%
• Washington Post (41): 4-5 51%. That’s good, but a surprising 36% earned 0-2 stars.

Among the lesser stellar:

• Newsweek (10 stories): two had 5 stars, four had 3, four had just 2
• Time (21): 4-5 stars 39%; 0-2 stars 34%.
• CNN (33): 4-5 stars 36%, 0-2 34%.
• US News (33): 4-5 19%; 0-2, 26%.

Most amazing to me is that WebMD, which is often recommended as reliable, earned 4-5 stars just 39% of the time, and was more likely (43%) to earn just 0-2 stars.

See why a health information seeker needs to know this? Someday perhaps your doctor’s office will have an “information coach,” but even then, Health News Review will still be more accessible. Besides, even the best of these outlets had 0-2 stars 18% of the time. You really do need to evaluate each story separately.

Long-term impact?

To me one great promise of the site has been that perhaps it will improve the quality of the reporting. So, being an analytics / trend-watching guy, I’d love to see if ratings have improved year to year! Subjectively, it sure seems like we’re seeing more high-quality articles than we used to.

Kudos to publisher Gary Schwitzer, and many thanks to the site’s 28 reviewers – including Carol Torgan, Ph.D., who’s a member of our Society. You rock (and enable) the e-patient world.