This is longer than it might be, because this point is essential. If this subject is familiar to you, skip to the heading “Today’s update.”
As we said in December, an e-patient essential is sorting out what writings to trust, whether we find them online or in print. There’s an important update on this in the New York Times today, an interview with Marcia Angell MD, an author we’ve quoted before, and her husband Dr. Arnold Relman – both long-time editors of one of the world’s top journals, the New England Journal of Medicine. It contains some eye-poppers.
Caveat: the Times makes clear that these two are controversial – some people think they’re “unrepentant scolds” who only see bad news, opposing all commercial interests in medicine (for-profit hospitals, insurance (which calls their payouts “medical loss ratio”), pharma). Whatever you think about them, it seems clear that there’s an important takeaway for all medical decision makers, patient and provider alike: have your eyes wide open about the weak science behind most journal articles.
This interview gives new reason to do so.
___________________
First, a little background for our newer readers.
In 2009, as I was just starting to get educated about healthcare, I posted A Quote I Won’t Soon Forget, which began:
Marcia Angell MD is a well-known, respected physician, long-time editor of NEJM. So it was a bit of a shock today when Amy Romano, blogger for Lamaze International, sent me this quote:
It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine.
It was from Dr. Angell’s review Drug Companies & Doctors: A Story of Corruption. I continued:
Dana Blankenhorn of the ZDNet health blog called it “a bombshell.” I couldn’t agree more. And I must say, with all the smart people in this community, why on earth haven’t we heard more about this??
And how on earth are we supposed to be empowered participatory patients if we can’t trust the world’s leading journal?
It echoed Richard Smith, 25 year editor of another top-tier journal (the British Medical Journal), in the first issue of our Journal of Participatory Medicine:
…most of what appears in peer-reviewed journals is scientifically weak.
This is no small issue in the life of an engaged patient. e-Patients who bring googled articles to their office visits are often lambasted or subjected to eye-rolls by clinicians who say that we should only trust academic medical journals. But can we trust them??
We got a rude update on why published science is shaky in our January 2011 post The Decline Effect: most published studies are never replicated by another lab! That’s absurd – heck, in high school I couldn’t get a science experiment passed if it wasn’t reproducible, but our journals do that??
If a study isn’t replicated, how can we be confident of what will happen when we put the drugs in our bodies?
In that December post I said:
Good research is transparent: you can see what the researcher did, you can see his/her data, you can try to reproduce it yourself. You can build a future on it. I shy away from anyone – establishment or not – who’s hostile to questions about their methods and thoughts.
In that regard I think science should be held to the same standard as engineering a bridge: if I follow your instructions, the damn thing better not fall apart when we test it over and over and over and over. And IMO, until something’s been tested over and over, it’s an educated guess.
At best it’s an educated guess – it could be corruption.
Today’s update in the Times:
Dr. Angell, and her former mentor Dr. Relman (whom she married in 2009), are interviewed about their current views.
… the late 1980s, when manuscripts she edited for The New England Journal testified … to the “new power and influence of pharma” over studies validating its products. Instead of standing back while impartial scientists evaluated drugs, manufacturers were suddenly involved in every aspect of the process.
… she vetted manuscripts that omitted any mention of a drug’s side effects, and studies that were weighted to make a drug look good; she repeatedly heard about studies never submitted for publication because they made a drug look bad.
Indeed: see Negative Data on Seroquel Suppressed By Manufacturers: the manufacturer ran several clinical trials, most of which failed, and suppressed them, reporting only the favorable data! If that isn’t intellectual and moral corruption, I don’t know what is.
And this eye-popper … I’d had no idea that within my adult lifetime authors had been allowed to publish about drugs they have a stake in. Dr. Relman, today considered an “unrepentant pharmascold,” is reported to be the one who stopped that:
In 1984, Dr. Relman became the first editor of a medical journal to require authors to disclose financial ties to their subject matter and to publish those disclosures. He later came to suspect that simple disclosure was not enough, and his policy evolved to excluding all authors with financial interests from writing large educational reviews.
Note: Relman’s policy said “It’s not enough to say ‘By the way, I get money from this product I’m reviewing’ – you simply shouldn’t be writing about it.” Makes sense, right? But I had no idea that apparently it’s not easy to find untainted experts anymore, so Relman’s successor punted that rule:
That rule was reversed in 2002, after the journal’s current editor in chief, Dr. Jeffrey M. Drazen, took the job. Dr. Drazen and his colleagues reported that for some subjects, so few experts without financial ties could be found that the journal’s scope was becoming artificially curtailed. (Emphasis added)
What?? If we exclude tainted people, the NEJM won’t have enough authors??
Note: Drs. Relman and Angell are criticized as scolds, but their successor said that.
If you want to research the payments issue, one resource is this database:
“The issues of conflict of interest and integrity in medical research are vitally important for journalists and the public,” said Charles Ornstein, president of the Association of Health Care Journalists. An investigative reporter for ProPublica, Mr. Ornstein has collaborated on a searchable “Dollars for Docs” database that compiles the limited payment data currently publicly available.
As I say, the former editors have detractors, and I’m not in a position to assess the details. And of course I’m not saying the whole profession is morally bankrupt. Having spoken at many conferences and having met with many physicians (not to mention my own), my personal experience matches Dr. Relman’s quote in this article:
“I happen to believe that doctors are not saints, but not sinners either,” he said. “They are sensible, pragmatic, decent.”
Nonetheless, in every medical decision, engaged patients must have their eyes wide open about the potential for unscientific influence. Knowing what I now know about the publication process, my perspective is as I said above: until a study has been replicated by another lab, I view it as nothing more than a first datum – an educated guess, not validated science. And it seems any science-minded clinician must do the same.
See the full Times article, with 6 minute video interview of Relman and Angell.
I finally had a moment to read the NYT article and catch up on the latest installment of Relman/Angell saga. (Who will play them in the movie?)
A quick search for more articles about Relman yielded:
Medical Professionalism in a Commercialized Health Care Market, by Arnold S. Relman, MD; JAMA (2007)
http://jama.ama-assn.org/content/298/22/2668.extract
Which led to this thought:
Have Relman and Angell commented on the current debate about medical professionalism online?
In case you missed it:
Physician Violations of Online Professionalism and Disciplinary Actions: A National Survey of State Medical Boards, by S. Ryan Greysen, MD, MHS, MA; Katherine C. Chretien, MD; Terry Kind, MD, MPH; Aaron Young, PhD; Cary P. Gross, MD, MPH; JAMA (yesterday)
http://jama.ama-assn.org/content/307/11/1141.extract
I got an early look at the full text article, which sadly isn’t available online (irony alert!), so I lined up a series of tweets to inform the discussion:
Coverage: “Physician Misconduct Showing Up on the Internet: Survey” – @BrophyMarcus http://is.gd/b5rejB (quotes @SeattleMamaDoc)
Perspective: “Distilling the essence of medicine using Twitter” – @RonanTKavanagh http://is.gd/eXbpzO
Perspective: “6 things I never talk about on Twitter” – @Doctor_V http://bit.ly/GAvIuH
Perspective: “Are we experiencing a social health correction?” – @Doctor_V http://bit.ly/zPZJUs
Perspective: “How can medical students and residents use social media?” http://bit.ly/GD1CZT via @Berci
Then the community started helping me. @bethmazur introduced me to Yoni Freedhoff, an MD in Canada, who contributed:
Perspective: “Health care professionals, Twitter, and the abuse of public trust” – @YoniFreedhoff http://is.gd/SDyxvI
I added what the Pew Research Center always tries to inject into the public conversation: data.
Context: “What internet users do online” – @pewinternet http://pewrsr.ch/bOLpKF
Another community member sent me this gem:
History: 2008 JAMA article on unprofessional content posted online by med students http://bit.ly/FPxdY0 (PDF) via @k8lin
And at what I thought was just the right moment, after all this high-minded discussion:
Here’s what most Americans will read about that JAMA letter: “Docs ask out patients online; some get reported” http://apne.ws/GFysrH
I ended the theme with an invitation for people to do their own fieldwork:
In case you want to check out what some clinicians post on Twitter, here’s my list: https://twitter.com/#!/SusannahFox/clinicians
So, all of this is to say: Thanks, Dave! And yes, let’s all keep our eyes wide open.
Susannah,
A contributing factor to this dearth of high quality research may well be attributable to the incestuous ties between grant funding agencies and academics. Today, unless you are PhD or MD affiliated with a high profile university it is almost impossible to get grant funding to do any kind of research. Because most researchers and funding agencies are looking for the next big thing…of course there is no interest in replicating previously published study research.
Meanwhile those of us entrepreneurs who are willing and able to do such research can’t get the time of day from potential academic partners or the funding agencies.
Steve Wilkins, MPH
As a physician-researcher, I agree that medical research is often not very accurate and results are not necessarily applicable to all people. Bias is a real issue (although mostly subconscious- most of us are trying to do unbiased work). But I believe the most important thing to understand about this problem is that we are using incomplete scientific principles- those of reductionist science (appropriate for a static, linear world)- and trying to apply them to a complex world. There is no way to control all variables to do a randomized, “controlled” trial. That is just not reality in a complex world. The same treatment that helps one person can cause harm to another person. Complex systems science allows us to learn and improve and group people into clusters which will help improve the value of healthcare. I don’t think things will change until we understand this and apply it to healthcare. I think if we don’t change our thinking, we will continue to get more of what we have now in medical research.
Dave,
Thank you for adding the 6 minute video interview of Relman and Angell.
I found their intellectual banter refreshing and insightful. The ability to see them in action adds another layer of value that helps me understand why their work is so critical.
In the failed us empire, it was all about getting money. Hustlers, hucksters, and imperialists. Why would “health” “care” “research” be any different. It’s get that money, “publish” weak results, then on to the next grant…it’s a US hamster wheel of endless hu$tling.
As aptly stated by Mr. Steven Wilkins above, unless you’re well connected (politically, academic buddy system, economically, neponomics) and from a ‘prestigious’ us ‘university’ one will have a very difficult time getting a grant and/or collaborations.
Also, the NY times (all the news that fits our views) was (past tense intentional) a well known lap dog mouth piece for the us govt and their hustling/corrupt interests. It exists simply to sate the professional classes with everything is ok, on track , all right, nothing to see, we have a few bumps in the road, etc.etc. tell them what they want to hear. Yet more US corporate/imperialistic propaganda. Morris Berman’s book–Why America FAILED, gives much food for thought of the business enterprise called the US.
Moreover, many so called ‘non profit’ public policy organizations paint rosy picture stories with numerous white papers, issues briefs, academic “experts,” etc…They do. not. want nor desire ANY ‘negative’ info (AKA: truth/reality) that taint their ‘we’re on the right’, path/grass is green/minor ‘bumps in the road’ dominant propaganda narrative. They’re selling pretty little lies for their ‘living’. That us health policy. LOL.
As Noam Chomsky stated, the purpose of academics/”policy experts” is to serve the elite. They cannot accept reality because that would impede their grants, CONsultantships, and other honoraria, speaking engagements, etc….It’s another us hustle.
I am doing research on corruption in our medical system, and was actually surprised by old posts such as this one, since I found a lot of old blogs/articles/comments on the internet on corruption and ethics in the pharmaceutical industry and on medical stuff in general. Having worked in this industry as a researcher and quality/regulatory professional, I had become aware of problems but it wasn’t until the Covid plandemic and my cancer diagnosis, that I became totally aware that there is something terribly wrong with all these institutions that I used to praise and admire (NIH, FDA, CDC, our own government, etc). I am so glad to see all these articles and comments. Thank you for sharing.
“Plandemic”? Are you one of the conspiracy theorists who thinks COVID-19 was dreamed up by government and billionaires dating back to the Nazis?
Here’s my personal take on that on my own blog: “Plandemic” and Judy Mikovits are COVID-19 conspiracy-theory quackery
https://www.epatientdave.com/2020/05/07/plandemic-and-judy-mikovits-are-conspiracy-theory-quackery/
Medicine requires careful thinking. As that blog post (and its many many links) detail, the whole plandemic crowd falls way short on that. Tellingly, the dozens of comments complaining about my post ALL changed the subject, illustrating again that they’re unable to hold a single thought and thnik carefully about it.
People who act that way are not just bad for medicine; they’re bad for civilization itself. They will throw us back to the dark ages.