A prime benefit of individual membership ($30) in our Society for Participatory Medicine is the right to participate in our members-only listserv. It was pretty sleepy a year ago, but these days it’s a hotbed of juicy discussions.

Here’s something that arose Thursday. Following a discussion of a controversial clinic, Fred Trotter (one of our most senior members) noted a post on BoingBoing about it, and said this whole situation puts engaged patients “in a difficult position. How do we tell the difference between a healthcare crank and a real victim [of FDA persecution]? Is there a difference?”

In the old days of paternal medicine a question like this was none of our business – we were to just follow instructions. But today we know even the establishment has flaws too, and if we want to be responsible, we need to think for ourselves.

So the ability to diagnose a spitting contest is essential. Some principles, with links to past posts:

• A core principle of science is that others must be able to examine what you did, and if they try the same themselves, produce about the same result.
• When somebody attacks a paper with anger, not logic, it’s not a good sign. But not all anger is a sign of corruption.
  • Same applies when a published author uses anger to attack a questioner.
• A long list of “publications” (appearing in print) is not the same as a long list of others verifying your work.
  • Unfortunately, appearing in a peer reviewed journal isn’t the same, either

That last item shook my image of science (ever since grade school) to the core. Like it or not, as noted in our January post “The Decline Effect,” most peer reviewed studies are not ever replicated by another lab!

Heck, I couldn’t get a thing through high school science if I couldn’t reproduce the result.

But then I reflected that in the opening issue of our JoPM, 25 year BMJ editor Richard Smith said “most of what appears in peer-reviewed journals is scientifically weak.” And I remembered this, from Marcia Angell, quoted in our post A Quote I Won’t Soon Forget:

“It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine.”

Statements like hers, and Richard Smith’s, are vital for anyone “e” who wants to evaluate evidence – and they should be understood just as well by the clinicians who use such evidence to recommend treatments.

Yet, perversely, quotes like Smith’s and Angell’s are  used by crackpots to say “Don’t question MY logic – it’s as good as the establishment’s.” Oy. What’s an e-patient to do?

Return to the basics: Science is still science. Good research is transparent: you can see what the researcher did, you can see his/her data, you can try to reproduce it yourself.  You can build a future on it.  I shy away from anyone – establishment or not – who’s hostile to questions about their methods and thoughts.

In that regard I think science should be held to the same standard as engineering a bridge: if I follow your instructions, the damn thing better not fall apart when we test it over and over and over and over. And IMO, until something’s been tested over and over, it’s an educated guess.

(Please do read the Decline Effect post if this interests you, including its links.)

(And join the Society, if you’d like to participate in the back room. You can plow through our archives for free, but it’s much more fun to have it in your inbox … and outbox.)