Important update: it turns out the writer did get it right, and this was an editing error at the Boston Globe. See my comment August 17.
—–
As empowered, engaged patients we have a responsibility to evaluate the articles we read. A case in point is this week’s Associated Press article Any Spread Of Breast Cancer Raises Risk Of Return. It’s about a paper in this week’s New England Journal of Medicine that found a 50% increase in the chance of cancer recurring, and quotes an expert as saying
“I think it will influence treatment. If we’re considering treating the patient, we probably should.”
Can you spot the flaw in that reasoning, if all you have is that number? That’s all the article reported, and that’s a problem.
The problem is relative risk reduction. As we discussed last November in Making sense of health statistics, relative risk is an unsound basis for policy decisions, because you can’t tell the actual size of the impact. That affects you as a patient and you as a citizen thinking about policy.
In the study, did the 50% change mean represent a change from 200 patients per thousand to 300 per thousand, or from 2 per thousand to 3 per thousand? (Specifically, if we spend a zillion dollars based on this result, will we affect 1/1oth of the US population, or 1/1,000th?
Think about it for your own body. Your health provider suggests a treatment and says it’ll reduce your chance of a problem by 35%. (That’s the number commonly cited for statins like Lipitor.) Sounds impressive, right? But that’s a relative risk number. The raw data showed that among the general population, 1.59% had a coronary event, and among Lipitor users, 1.02% had one anyway. So for the $25 billion we spend on Lipitor in the US, we affect one patient in 200.
So now imagine your provider says “There’s a one in 200 chance this prescription will make a difference for you. Want it?”
I think there are two levels of concern for us as patients engaged in the new world of healthcare:
- For you personally, ask your providers for the absolute numbers. They may need to go digging, because they themselves may not have been given the numbers. But you have every right to know, eh?
- At a national level, the same thinking must be applied to policy decisions.
I wrote a note to the author of the AP article asking if she can get the absolute numbers. (As usual, the original article in NEJM isn’t open for us to evaluate.)
Related resources:
- Health News Review – a superb resource for understanding how to evaluate health news. (Thanks to Ted Eytan for the tip last year.)
- Evidence-based medicine – my post last November citing the Lipitor numbers, as reported in Andy Kessler’s book The End of Medicine.
Once again, you are right on spot!
Lodewijk
Apologies for the odd background formatting on this comment – something went weird when we upgraded recently. We’ll investigate.
————
Important update: AP writer Marilynn Marchione did get it right, reporting both absolute and relative numbers:
That’s perfect: in my question above (“does this mean a change from 200 per thousand to 300 per thousand, or from 2 to 3?”) the answer is:
To me that’s substantial. But whatever your view, the absolute numbers let you make an informed decision. But when the Globe edited the story, they cut out the good stuff and only printed the relative numbers. (Here’s one site that ran the fulll article as written: http://www.wtop.com/?nid=106&sid=1738698)
Kudos and brickbats:
Next assignment for e-patients in training: What would your next step be?
An excellent point! Many people fundamentally misunderstand statistics… Saying that a drug reduced by 35% a certain symptom in a study does NOT mean it will necessarily do so in any INDIVIDUAL.
I liken it to the coin toss experiment. Most patients have a sample size of one (themselves), and on a single coin toss the statistical chances of heads/tails is 50%/50% BY DEFINITION. You need to perform more experiments to see if the coin is weighted and even then you can only use your findings to speak about the group of experiments, not each individual one… :)
Update a week later: I did write to the Globe’s science & health editor with the same info I sent to the AP writer – haven’t heard anything back.