This is longer than it might be, because this point is essential. If this subject is familiar to you, skip to the heading “Today’s update.”
As we said in December, an e-patient essential is sorting out what writings to trust, whether we find them online or in print. There’s an important update on this in the New York Times today, an interview with Marcia Angell MD, an author we’ve quoted before, and her husband Dr. Arnold Relman – both long-time editors of one of the world’s top journals, the New England Journal of Medicine. It contains some eye-poppers.
Caveat: the Times makes clear that these two are controversial – some people think they’re “unrepentant scolds” who only see bad news, opposing all commercial interests in medicine (for-profit hospitals, insurance (which calls their payouts “medical loss ratio”), pharma). Whatever you think about them, it seems clear that there’s an important takeaway for all medical decision makers, patient and provider alike: have your eyes wide open about the weak science behind most journal articles.
This interview gives new reason to do so.
First, a little background for our newer readers.
In 2009, as I was just starting to get educated about healthcare, I posted A Quote I Won’t Soon Forget, which began:
It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine.
It was from Dr. Angell’s review Drug Companies & Doctors: A Story of Corruption. I continued:
Dana Blankenhorn of the ZDNet health blog called it “a bombshell.” I couldn’t agree more. And I must say, with all the smart people in this community, why on earth haven’t we heard more about this??
And how on earth are we supposed to be empowered participatory patients if we can’t trust the world’s leading journal?
It echoed Richard Smith, 25 year editor of another top-tier journal (the British Medical Journal), in the first issue of our Journal of Participatory Medicine:
…most of what appears in peer-reviewed journals is scientifically weak.
This is no small issue in the life of an engaged patient. e-Patients who bring googled articles to their office visits are often lambasted or subjected to eye-rolls by clinicians who say that we should only trust academic medical journals. But can we trust them??
We got a rude update on why published science is shaky in our January 2011 post The Decline Effect: most published studies are never replicated by another lab! That’s absurd – heck, in high school I couldn’t get a science experiment passed if it wasn’t reproducible, but our journals do that??
If a study isn’t replicated, how can we be confident of what will happen when we put the drugs in our bodies?
In that December post I said:
Good research is transparent: you can see what the researcher did, you can see his/her data, you can try to reproduce it yourself. You can build a future on it. I shy away from anyone – establishment or not – who’s hostile to questions about their methods and thoughts.
In that regard I think science should be held to the same standard as engineering a bridge: if I follow your instructions, the damn thing better not fall apart when we test it over and over and over and over. And IMO, until something’s been tested over and over, it’s an educated guess.
At best it’s an educated guess – it could be corruption.
Today’s update in the Times:
Dr. Angell, and her former mentor Dr. Relman (whom she married in 2009), are interviewed about their current views.
… the late 1980s, when manuscripts she edited for The New England Journal testified … to the “new power and influence of pharma” over studies validating its products. Instead of standing back while impartial scientists evaluated drugs, manufacturers were suddenly involved in every aspect of the process.
… she vetted manuscripts that omitted any mention of a drug’s side effects, and studies that were weighted to make a drug look good; she repeatedly heard about studies never submitted for publication because they made a drug look bad.
Indeed: see Negative Data on Seroquel Suppressed By Manufacturers: the manufacturer ran several clinical trials, most of which failed, and suppressed them, reporting only the favorable data! If that isn’t intellectual and moral corruption, I don’t know what is.
And this eye-popper … I’d had no idea that within my adult lifetime authors had been allowed to publish about drugs they have a stake in. Dr. Relman, today considered an “unrepentant pharmascold,” is reported to be the one who stopped that:
In 1984, Dr. Relman became the first editor of a medical journal to require authors to disclose financial ties to their subject matter and to publish those disclosures. He later came to suspect that simple disclosure was not enough, and his policy evolved to excluding all authors with financial interests from writing large educational reviews.
Note: Relman’s policy said “It’s not enough to say ‘By the way, I get money from this product I’m reviewing’ – you simply shouldn’t be writing about it.” Makes sense, right? But I had no idea that apparently it’s not easy to find untainted experts anymore, so Relman’s successor punted that rule:
That rule was reversed in 2002, after the journal’s current editor in chief, Dr. Jeffrey M. Drazen, took the job. Dr. Drazen and his colleagues reported that for some subjects, so few experts without financial ties could be found that the journal’s scope was becoming artificially curtailed. (Emphasis added)
What?? If we exclude tainted people, the NEJM won’t have enough authors??
Note: Drs. Relman and Angell are criticized as scolds, but their successor said that.
If you want to research the payments issue, one resource is this database:
“The issues of conflict of interest and integrity in medical research are vitally important for journalists and the public,” said Charles Ornstein, president of the Association of Health Care Journalists. An investigative reporter for ProPublica, Mr. Ornstein has collaborated on a searchable “Dollars for Docs” database that compiles the limited payment data currently publicly available.
As I say, the former editors have detractors, and I’m not in a position to assess the details. And of course I’m not saying the whole profession is morally bankrupt. Having spoken at many conferences and having met with many physicians (not to mention my own), my personal experience matches Dr. Relman’s quote in this article:
“I happen to believe that doctors are not saints, but not sinners either,” he said. “They are sensible, pragmatic, decent.”
Nonetheless, in every medical decision, engaged patients must have their eyes wide open about the potential for unscientific influence. Knowing what I now know about the publication process, my perspective is as I said above: until a study has been replicated by another lab, I view it as nothing more than a first datum – an educated guess, not validated science. And it seems any science-minded clinician must do the same.
See the full Times article, with 6 minute video interview of Relman and Angell.