{"id":8385,"date":"2011-01-29T03:31:23","date_gmt":"2011-01-29T07:31:23","guid":{"rendered":"http:\/\/pmedicine.org\/epatients\/?p=8385"},"modified":"2011-01-29T11:26:23","modified_gmt":"2011-01-29T15:26:23","slug":"electronic-health-records-do-not-impact-the-quality-of-healthcare","status":"publish","type":"post","link":"https:\/\/participatorymedicine.org\/epatients\/2011\/01\/electronic-health-records-do-not-impact-the-quality-of-healthcare.html","title":{"rendered":"“Electronic Health Records Do Not Impact the Quality of Healthcare?”"},"content":{"rendered":"

We’ve recently been talking here about problems with poor study design in clinical trials. A health IT version of this problem raced through the newswires\u00a0this week while I was on the road. The news coverage was particularly na\u00efve, illustrating our point.<\/p>\n

I’ll say at the outset that I haven’t corresponded with the study’s authors, and I’d welcome constructive dialog. I find myself frustrated, and I’ll lay out my reasons, open to correction.<\/p>\n

Researchers at Stanford did a retrospective analysis of patient records 2005-2007 (article here<\/a>) and concluded that quality of care was no better in institutions that used electronic medical records. Journalists who don’t know how to assess study design – or even look beyond the abstract of an article – were sucked in and posted headlines like\u00a0Stanford researchers find EHRs don’t boost care quality<\/a>.<\/p>\n

Before we analyze, please reflect: what’s your impression upon reading those words? What does that headline lead you to think about the push to computerize healthcare?<\/p>\n

It’s hard to know where to start in cataloging the ways these headlines give an impression that’s way <\/em>off base. For starters, I wondered what they meant by “quality of healthcare.” What did they measure? Did they assess how well the patients turned out? How few complications they had, how few side effects? To me those are pretty important quality measures.<\/p>\n

Well, no, they only measured whether the doctors prescribed the right treatment <\/em>(or medication).<\/p>\n

So it seems to me the headline should say, “Stanford researchers find EHRs don’t change what doctors prescribe.” Very different impression.<\/p>\n

But what do I know? I do know that in healthcare a lot of people think the right prescription – the so-called “standard of care” – is the definition of quality. And indeed, that’s how this study defined quality. \u00a0(Did you know patients only receive the standard of care a bit more than half the time? Indeed, installing EHRs wouldn’t change that. Methinks that <\/em>problem has nothing to do with computers.)<\/p>\n

A deeper issue is the question of who gets to define what quality is. In every transformed industry, the customer does, but in this one, they measure whether one professional did what another one said to.<\/p>\n

But let’s stick to the issues of the research itself:\u00a0social media to the rescue. On Twitter that day, health IT guru Brian Ahier steered us to this terrific analysis<\/a> by Dr. Bill Hersh of Oregon Health & Science University. Bill catalogs the study’s limitations in clear English. A few excerpts: (Bill, I’m going to rip off chunks of your post, to reach lazy readers, in the hope that serious ones will click through for your whole great post.)<\/em><\/p>\n

Like almost all science that gets reported in the general media, there is more to this study than what is described in the headlines and news reports. …<\/p>\n

There is no reason to believe that the results obtained do not derive from the methods used….\u00a0However, there are serious limitations to this type of study and to the data resources used to answer the researchers’ question, which was whether ambulatory EHRs that include clinical decision support (CDS) lead to improved quality of medical care delivered.<\/p>\n

“The data resources used” resonates with our posts this week about whether a study actually measures what it set out \u00a0to measure.<\/p>\n

…it is important to understand some serious limitations in these types of studies and this one in particular. A first limitation is that the study looks at correlation, which does not mean causality. This was an observational and not an experimental study. … As with any correlational study, there may be confounders that cause the correlation or lack of it.<\/p>\n

The best study design to assess causality is an experimental randomized controlled trial. Indeed, such studies have been done and many have found that EHRs do lead to improvements in quality of care….<\/p>\n

See Bill’s post for more. Then:<\/p>\n

A second limitation of this study is the quality measures used. Quality measures are of two general types, process and outcome. Process measures look at what was done, such as ordering a certain test or prescribing a specific treatment. Outcome measures look at the actual clinical outcomes of the patient…<\/p>\n

[In healthcare today] most measures used… are process measures that may or may not result in improved patient outcomes.<\/p>\n

I guess that matches my initial gut reaction, above. (They looked at a process measure (doing the right thing) rather than how the patient turned out. Quite a different concept of quality.)<\/p>\n

A third limitation … is that we do not know whether the physicians … \u00a0had decision support [software] in place.<\/p>\n

If I read this right it means the researchers evaluated whether docs made the right decision, even though they didn’t have data on whether the systems in use had <\/em>decision support installed.<\/p>\n

Please do read Bill’s post, and the original article, to fully understand the situation. My concern here is twofold:<\/p>\n