Please cite this post as “by Dave deBronkart, Marilyn Mann and Peter Elias MD” or, on Twitter, “@ePatientDave, @MarilynMann & @PHEski.” Our blog software only allows listing one author but they provided 2/3 of the content.
The medical news is abuzz – and your newspapers will be abuzz – with coverage of a study released Friday. Understanding it properly requires several things that we’ve blogged about here in the past, not least of which is understanding statistics, especially these key points:
- Avoid relative risk reduction (headlines about percentages)
- Look instead for actual (absolute) numbers of patients helped
- Figure out the NNT – the number needed to treat. It’s the number savvy people look for.
I’m not involved in cardiac issues, so I first heard about it on ABC’s evening news, where, sure as you’re born, all they talked about was relative risk: “lowers the risk of heart attack, stroke, etc by 15-20%.” That’s good news, of course, but you can’t tell how good it is unless they dish up the absolute numbers. (For example, a study of 1000 patients on a new drug could show a 20% reduction in relative risk from 5 to 4 patients or from 50 to 40. Both are 20% reductions, but there’s a tenfold difference in how many were helped!)
The graphic above is from the company’s press release. Surprise: relative risk! What if that’s all a news outlet reads? Will their coverage be what you need? (Plus, some news outlets also post the company’s press release as is, like this. Consumer beware!)
One reason this is a big problem is that if the public doesn’t understand this, doctors say they’re beset with people asking for a drug based on something they saw poorly reported on TV. Another reason was covered in the news I saw: insurance companies have been hesitant to pay for this drug, which costs $14,000/year
month. Are they scrooges?
How should patients and clinicians look at this drug? This post is edited by several members of our Society: Peter Elias MD (a retired physician and member-at-large of our board), Marilyn Mann, who moderates a community of familial hypercholesterolemia (FH) patients and their family members, and Casey Quinlan, cancer survivor and board member of TheNNT.com, a superb resource that analyzes literature on diseases and treatments, particularly turning relative risk into the real bottom-line number: how many patients need to take a drug, for one of them to benefit?
- The new drug is currently approved only for patients with FH, a genetic cholesterol disorder, and patients who already have heart disease, have had a stroke, or have peripheral artery disease.
- It’s a BIG study – 27,000 patients. That’s good: bigger studies are far less likely to give shaky results.
- Major heart problems or strokes happened to 11.3% of patients WITHOUT the new drug, and 9.8% of patients WITH the new drug. In other words, 1.5% of patients avoided a problem event, but 9.8% still experienced a problem event despite taking the drug.
- Since 1.5% of the population had this benefit, it means on average, 1 patient in 67 benefits from the drug. That’s the NNT – the number of patients to treat, for one to get any benefit.
- Note, though, that the drug saved no lives: the same percent died whether or not they got the drug. So it prevented 1.5% of these major cardiac events, but didn’t alter death rates – at least not during the time of this study.
- The drug costs $14,000/year, and these patients were watched a median of 2.2 years, so the cost was about $30,800 per patient.
- The 67:1 ratio means each prevented heart attack etc came at a cost of 67 x $30,800 = $2.06 million.
- No new side effects were discovered. That’s good – many new drugs bring new risks too. (But Marilyn points out what savvy patients know: this study was pretty short, so more news about side effects may come out later.)
What to make of it?
Obviously, neither the press release headline nor the ABC Evening News coverage gives the information that a patient and clinician would need to make an informed choice.
In the Facebook thread, Marilyn Mann’s perspective contributed the most resources:
- NPR’s Shots blog: Pricey New Cholesterol Drug’s Effect On Heart Disease Is More Modest Than Hoped by noted cardiologist Harlan Krumholz – a great balanced perspective
- Krumholz also writes from a different angle on Forbes: With Trial Results, Should Amgen Reconsider Its Pricing For Repatha?
- This one rounds down the numbers and comes at a similar estimate: “about $1.4 million for every event averted.”
- It also includes this great summary of the many aspects of the pricing problem: “Amgen has done heroic work to bring this drug to market and test it appropriately in a rigorous scientific trial. Perhaps the pricing would account, for some period of time, for that effort to break new ground and provide bonus pricing for a couple of years. Such an approach may represent an innovation bonus based on the novelty of the approach and what was required to produce it. We do need to figure out how to continue to reward risk and innovation in producing breakthroughs. But in the end the question is…how does what the drug cost square with what it provides?”
- Health editor Matthew Herper, also on Forbes: Amgen Drug Prevents Heart Attacks, Not Deaths, Disappointing Experts, a deep look at the complexities of assessing the drug’s trial, benefits and costs, including some human cases.
- CardioBrief: FOURIER Shows New Cholesterol Drugs Work, But Are They Worth It? Subhead: Doctors and patients now must wrestle with a modestly effective but expensive drug.
- ICER, a nonprofit that evaluates evidence on the value of medical tests and treatments: Institute for Clinical and Economic Review to Produce “New Evidence Update”, saying they’ll update their recommendations by mid-May.
- tctMD’s piece leads off with the absolute benefit (1.5%) and quotes the study’s lead author on an important point Herper noted: did the study stop too soon, in its haste to get published? “We’ve seen this for all the statin trials as well. It takes time for LDL lowering to translate into healthier arteries.” In other words, 3 years from now will we discover “Wait – in the long run it’s lots better than we thought back then”? There’s no way to know without waiting.
- A 2016 article in JAMA that concludes the price of such drugs would have to drop 70% to meet a common measure of value.
What can we learn? What’s the impact?
Did I hear someone saying patients should stay off the internet? This list of great resources was pulled together (on Facebook!), within a day after the study was published, by Marilyn, who has no medical degree: she’s a highly motivated participant from the patient perspective. The rules for who can contribute to this work have changed forever.
Healthcare works better when clinicians and e-patients are both well informed, and that means digging past the headlines, which are designed to catch attention rather than inform. (Plus, these days headlines are often written NOT by an article’s author, but by separate people who are trained to write for clicks, not for accuracy – example.) (Addition: my college friend Stephen Owades notes that this is no different from tabloid headline writers, whose job was to write in a way that grabbed the eye of newsstand customers.)
Peter adds a superb observation: Both patients and their clinicians should not be distracted by arguments in the media about whether or not a new treatment is worth it, because care must go on. “Worth it” is the wrong question and a distraction from the real issue: patients still have problems and need to decide what to do.
Medicine is full of challenges like this, he says – some more expensive and some less, some with big and some with small consequences. We need to stop thinking in terms of a global ‘this is good’ or ‘this is bad’ evaluation and think in terms of ‘how can we support patients who are making decisions in conditions of uncertainty?
Marilyn responded with a related note: this study included very high risk patients, and if your risk isn’t as bad, then the benefits of the drug would not be comparable. You’d be much less likely to benefit, so the NNT for patients like you would be much larger.
e-Patient takeaway: the details are tricky but the basics are manageable.
It takes knowledge and savvy to do this, but notice: even without Marilyn’s list of great articles, the bottom line issues (mostly that $2 million number) were figured out from knowing these e-patient basics:
- Ignore percentages (relative risk reduction, “RRR”) when making decisions. Look for the absolute numbers. (Problems were experienced by x% with the drug, and y% without.)
- In this study it was 9.8% vs 11.3%, so 1.5% of patients got a benefit during the length of this study.
- From that, figure the NNT. That’s 100 divided by the absolute number: in this case,100 / 1.5 = 67 people. So a user of the drug has a 1 in 67 chance that it will help (if they’re like the people in the study, as Marilyn says).
- Notice: from the headlines saying 15% reduction, you don’t have a clue what the NNT is.
- Look at the study’s endpoints (what they were monitoring) to answer an important question: “Specifically how might this help?” Marilyn and Peter both point out that while it’s important to reduce heart attacks etc, the overall chance of dying was not helped.
- Teach others to watch out for this too. Do not respect any coverage that only talks percent.
One last tip: these journalistic hazards and many more are taught on the great Health News Review site, which we’ve blogged about here for years. It’s an essential e-patient resource.
Great job summarizing the study and pulling together the resources to look at it from different angles.
Here’s another angle that I found annoying:
in which the authors suggest that the study “refutes Dr. Google” and then suggest that it answers a question (do statins make you stupid?) that it was NOT DESIGNED to answer.
Background for people reading this — in the preapproval trials for evolocumab and alirocumab there was a small apparent increase in neurocognitive events, and the FDA required the sponsors to study cognitive effects in a systematic way. Amgen did a substudy within the FOURIER trial that we’ve been discussing to study cognitive effects of evolocumab. The substudy is called EBBINGHAUS, and the results are described at the link in Terry’s comment. No adverse cognitive effects were found.
I agree with Terry that EBBINGHAUS was not designed to study cognitive effects of statins — it was designed to study cognitive effects of evolocumab.
Interestingly, there have been statin trials that studied cognitive effects of statins. One, called HOPE-3, was reported at the American Heart Association conference in November 2016. The main results of HOPE-3 have been published, but the cognitive study has not yet been published, so far as I know. However, I happen to have the slides that were presented at the AHA conference and you can review them at this link:
Cognitive function was also tested in the PROSPER trial, which was a trial of pravastatin vs. placebo. Neither HOPE-3 nor PROSPER found adverse cognitive effects of statins (HOPE-3 also studied cognitive effects of blood pressure drugs).
Of course, one cannot exclude the possibility of rare side effects that would not be detected in a clinical trial — that is always the case. The HOPE-3 investigators actually hoped to find beneficial effects on cognitive of rosuvastatin and/or blood pressure drugs. They did not find beneficial effects in the overall results. You can look at the slides for the discussion of subgroups.
Sorry, here’s the link to the PROSPER study:
Sorry, typo, I meant “beneficial effects on cognition.”
Here’s even more analysis by Harlan Krumholz:
Which is it, $14,000/month (4th paragraph) or $14,000 per year (The Specifics, 5th bullet)?
Nice write up though; DON’T trust the media to ever get it correct, especially after the manufacturer has put their spin on it.
Yikes, HUGE disconnect, Hawkeye! It’s $14k/year. Fixing it!
Delighted to see this piece today after taking part in the thread on your FB timeline, Bingo and bullseye all around, Dave, from the all-too common distortion of staistics to feed a sales pitch (or justify a cost) to the factors one needs to consider in scaling results back to your own N of one. I would love to see us at SPM keep on providing exactly this kind of counterpoint to major health headlines, as it is normally quite hard to find!
Great job, Dave. A point that was mentioned by Herper and TCTMD was this 2.2 year follow-up. In reality, no patient is prescribed a cholesterol drug for two years. Most are expected to take this drug for decades, not two years. It’s during those decades that we’ll ultimately see the side effects that emerge in the lived experience of patients. For years, most cardiologists have seemed dismissive of occasional patient reports of debilitating statin side effects(I’m talking severe side effects affecting quality of life, like leg pain so severe that climbing stairs was no longer possible). Many were advised simply to change the brand of statin they were taking. It was only with the recent emergence of non-statin cholesterol-lowering drugs that suddenly drug manufacturers have started bragging about meds for those “who cannot tolerate statins”.
Another interesting point about the FOURIER study findings presented at #ACC17 is that not only were 75% of the subjects male, but no sex-specific outcomes were presented either, which makes it impossible to extrapolate that Repatha works as Amgen says it will in women.
Carolyn, the sex-specific outcomes for the primary endpoint were presented in the appendix. If you look at Dave’s post on Facebook, I posted the table of subgroup effects in a comment. There was a benefit in both male and female subgroups.
Carolyn, here’s the link to Marilyn’s screen capture of that table. https://www.facebook.com/epatientdave/posts/10211540320401443?comment_id=10211549109221158&comment_tracking=%7B%22tn%22%3A%22R9%22%7D
VERY disappointed to see that Consumer Reports coverage of this study pretty much blew it – pretty much just commented on the press release, and didn’t get into any of the issues that Marilyn and Peter raised in our post.
What You Need to Know About the Heart Drug Repatha
Great post. Tx