A prime benefit of individual membership ($30) in our Society for Participatory Medicine is the right to participate in our members-only listserv. It was pretty sleepy a year ago, but these days it’s a hotbed of juicy discussions.
Here’s something that arose Thursday. Following a discussion of a controversial clinic, Fred Trotter (one of our most senior members) noted a post on BoingBoing about it, and said this whole situation puts engaged patients “in a difficult position. How do we tell the difference between a healthcare crank and a real victim [of FDA persecution]? Is there a difference?”
In the old days of paternal medicine a question like this was none of our business – we were to just follow instructions. But today we know even the establishment has flaws too, and if we want to be responsible, we need to think for ourselves.
So the ability to diagnose a spitting contest is essential. Some principles, with links to past posts:
- A core principle of science is that others must be able to examine what you did, and if they try the same themselves, produce about the same result.
- When somebody attacks a paper with anger, not logic, it’s not a good sign. But not all anger is a sign of corruption.
- Same applies when a published author uses anger to attack a questioner.
- A long list of “publications” (appearing in print) is not the same as a long list of others verifying your work.
- Unfortunately, appearing in a peer reviewed journal isn’t the same, either
That last item shook my image of science (ever since grade school) to the core. Like it or not, as noted in our January post “The Decline Effect,” most peer reviewed studies are not ever replicated by another lab!
Heck, I couldn’t get a thing through high school science if I couldn’t reproduce the result.
But then I reflected that in the opening issue of our JoPM, 25 year BMJ editor Richard Smith said “most of what appears in peer-reviewed journals is scientifically weak.” And I remembered this, from Marcia Angell, quoted in our post A Quote I Won’t Soon Forget:
“It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine.”
Statements like hers, and Richard Smith’s, are vital for anyone “e” who wants to evaluate evidence – and they should be understood just as well by the clinicians who use such evidence to recommend treatments.
Yet, perversely, quotes like Smith’s and Angell’s are used by crackpots to say “Don’t question MY logic – it’s as good as the establishment’s.” Oy. What’s an e-patient to do?
Return to the basics: Science is still science. Good research is transparent: you can see what the researcher did, you can see his/her data, you can try to reproduce it yourself. You can build a future on it. I shy away from anyone – establishment or not – who’s hostile to questions about their methods and thoughts.
In that regard I think science should be held to the same standard as engineering a bridge: if I follow your instructions, the damn thing better not fall apart when we test it over and over and over and over. And IMO, until something’s been tested over and over, it’s an educated guess.
(Please do read the Decline Effect post if this interests you, including its links.)
(And join the Society, if you’d like to participate in the back room. You can plow through our archives for free, but it’s much more fun to have it in your inbox … and outbox.)
Serendipity is an integral part of the Internet. It is funny that you posted this a few minutes after I wrote the following tweet: “Absolute certainty is not scientific” is a good corrolary to “knowledge is in constant beta” http://on.wsj.com/ry9Bug.
There are simply many, many sides to the limitations of the scientific method in medicine!
I had read the blog with the long string of threatening mail on Thursday and it had impressed me how people could so naive to trust non EBMed medicine…The fact that this clinic used these means to silence critical voices should raise worries its patients… Wonder if local authorities have checked into this facility to see what is going on with the patients.
Your post explains clearly what patients have to watch and I will add to serendipity as Gilles Fry said, by adding that at the Cochrane meeting tonight Prof. Cindy Farquhar and Ass.Prof. Evangelia Ntzani spoke about appraising studies and assessing the risk of bias. These lectures were meant to help physicians review critically various pieces of research for their clinical decision making. But these same principles apply too to patients who try to sort out information about their health. These lectures and the exercises we made afterwards (the audience was mostly gynaecologists, psychiatrists, oncologists and two patients)reminded me of the lectures of Gary Schwitzer and Prof. Gerd Gigenrenzer at Salzburg.