I’d like your help preparing thoughts and testimony for a policy meeting I’ve been invited to attend in Washington next week.
For these meetings, one needs to submit prepared remarks in advance, for the committee to digest in advance. And from what I’ve learned so far about this, there’s a lot to chew on, and people of all stripes (that’s you) can probably provide valuable input. At very least you can express yourself.
[Update 1 pm ET Thursday: I haven’t been able to convert the recording mentioned below (which is in RealPlayer format) to display it here, but if you have RealPlayer installed you can play it yourself at http://real.welch.jhu.edu/ramgen/DHSI/Dec182009.rm. The slides are often out of sync with the audio but they catch up. Skip the first 9 minutes; the talk starts around 9:15 and goes 30 minutes, followed by 30 minutes of Q&A. The meat of it is in Dr. Koppel’s talk, but the Q&A has more juice.]
The meeting subject is Health IT Safety. This sounded like an odd topic – considering all the things that go wrong because data is NOT computerized, what are they talking about, re safety risks WITH the technology??
I know I’m in over my head, being “just a patient.” But the more I learned tonight, the more I thought maybe that’s a good thing.
Because tonight I learned there are truly massive problems with the workflow and flow of information in today’s systems. Later Thursday I hope to post a link to a 30 minute webcast with 30 minutes of Q&A from December, documenting what some horrifying failures in today’s big EMR systems. You’ll judge for yourself, but imagine…
- …if a system you had to use every day would often display unreliable information, while insisting you do things that make no sense.
- …that the maker of the system insists on a “hold harmless” clause, so when something goes wrong, it’s not their fault. Legally no consequence for failures.
- …that nobody involved in the malfunctions talks about them, and many people are not allowed to talk about them, much less collect a bug list.
Examples of software issues in these multi-million dollar systems, as documented in the webcast:
- The user interface (UI) may not highlight what needs the user’s attention. (Ever dealt with a web page where you can’t spot the info you need? That’s what these systems can be like.)
- Sometimes, values can’t be sorted in numerical order. The computer system cannot be programmed to do this, so workers have to hunt through a list to find the number they want. (I’m not making this up.) (Imagine a website where a list of states was unsorted and couldn’t be fixed.)
- Units of measure get intermingled: a patient’s weight that must be entered in kilograms on one screen might be displayed on another screen without the units showing, so sometimes it’s interpreted as pounds (because pounds are used elsewhere in the same system). Imagine the consequence on medication dosage.
- Workflows are sometimes set up to insist on certain actions (even if they’re wrong), so users have to create torturous workarounds. (Remember, “users” is the doctors and nurses who are trying to take care of your mother. Part of their attention is consumed by coping with a system that doesn’t work.)
- Attempts at quality control (for instance bar codes for prescriptions) can be thwarted by real-world circumstances that weren’t tested: babies who chew off their bar code bracelets, bar codes on prescription bottles that aren’t durable enough to withstand normal handling, etc. In this study, 4.2% of all bar code bracelets didn’t work! How’s that for a safety feature?
I’m not making this up; you’ll judge for yourself when the video is up. Pitfalls we wouldn’t tolerate in the simplest word processing program are commonplace in million dollar medical systems. Ending up with erroneous data is not unusual.
And we want to transmit this data? Yikes.
For the purpose of this post:
- Imagine that the system running your hospital might be full of crap. (Yes, that word appears in the webcast: a major hospital executive famously rejected a major EMR system and replaced it with another, which he described as “the cream of the crap.”) (He said about the system they’re moving to.)
- Given this situation, what do we ask our government to do?
Our answers will go to the specific people who will recommend national policy on this. What advice can we give them on how certify a system so that it qualifies for Federal stimulus money?
You software system people out there: what would you do? You don’t get to demand a different reality; we have to start where we’re at. What do we do?
You clinicians – the people who have to use the systems: what would you want the government to do?
_____________
Here’s one radical idea:
What if a system could only get certified (and thus get the stimulus money) if the people who use it say it basically works??
(Can a system be meaningful if the users say it doesn’t work?)
Guiding principle: ask the workers who are directly impacted if the system screws up.
_____________
Another idea: since we can’t wave a wand and fix everything instantly, prioritize collecting failure data so we can figure out what needs fixing, and we can prove that a fix has worked. (Software tools to do this are common in high tech.)
Two guiding principles here:
- Lives are at stake. I can imagine no valid excuse for interfering with this effort.
- Let doctors and nurses do their jobs. If a system interferes with my nurse practitioner, to me that’s a problem. We must stop systems from getting in the way; good systems don’t.
I imagine this has to be combined with amnesty for errors. If people get punished for reporting a mistake, it won’t happen. (I heard the FAA has such a policy, and it’s helped greatly in reducing causes of crashes.)
_____________
A third idea, which would need to be thought out: Allow a second set of eyes to check for obvious mistakes. An obvious resource here is the patient or family or advocate. But given that the systems can be awkward for professionals to use, I’m not sure how to approach this.
I do know, though, that no stakeholder is more motivated. And as cancer widow and 73 cents artist Regina Holliday made abundantly clear in December, you might be surprised what a motivated “just a high school graduate” can spot that’s a useful contribution. (And free.)
_____________
What else can we say about achieving safe, reliable data? What policies should they recommend, to cope with the cream of the crap? Comment please.
Some background “footnotey” details follow.
____________________________________________________
This workgroup’s position in the hierarchy:
- The Health IT Policy Committee is chaired by Dr. David Blumenthal, the National Coordinator for health IT. The committee is tasked with
- recommending a “policy framework for … a nationwide health information infrastructure, including standards for the exchange of patient medical information.” (That’s your medical data, your mother’s, etc. etc.)
- “recommendations on standards, implementation specifications, and certifications criteria in eight specific areas.” In other words, they get to say what’s acceptable and what’s not, when it comes to health IT.
- Inside that committee, this is a meeting of the Certification/Adoption Workgoup, which will make recommendations about “certified electronic health records that support meaningful use, including issues related to certification, health information extension centers and workforce training.”
- Got that? That’s “How do we certify that a given health IT system is reliable, so it contains – and transmits – accurate data?”
- They even get to recommend issues of workforce training. If you’ve ever implemented a new computer system in your workplace, you know how important that is.
This meeting’s agenda:
Adoption/Certification Workgroup Meeting
Omni Shoreham Hotel, 2500 Calvert Street, NW, Washington, DC
Thursday, February 25, 2010, 9 a.m. to 3:00 p.m./Eastern Time
9:00 a.m. Call to Order/Roll Call – Judy Sparrow, Office of the National Coordinator
9:05 a.m. Meeting Objectives and Outcomes: Health IT Safety
– Paul Egerman and Marc Probst, Co-Chairs
9:15 a.m. Identifying the Issues
- Ross Koppel, University of Pennsylvania
- David Classen, CSC
- Gil Kuperman, Columbia University
- Alan Morris, Intermountain Healthcare
10:45 a.m. Stakeholders
- Dave deBronkart, ePatientDave
- Justin Starren, Marshfield Clinic
- Jeanie Scott, Veterans Health Administration
- Susan Edgman-Levitin, National Patient Safety Foundation [invited]
- Gay Johannes, Cerner
- Carl Dvorak, Epic
12:15 p.m. LUNCH BREAK
1:00 p.m. Possible Approaches
- Jeff Shuren, FDA/medical devices
- William Munier, AHRQ
- James Walker, Geisinger
- Edward Shortliffe, AMIA
2:30 p.m. Summary Comments from the Workgroup
2:45 p.m. Public Comments
3:00 p.m. Adjourn
One word: Standards. Standards, standards, standards. We’ll never be able to get everyone to adopt the same exact software solutions, so long as these are products developed by private companies. I don’t see the government doing a good job developing specific software solutions any better than the private companies…
I will paraphrase a PhD friend of mine who was born in east germany:
“Everyone wants the government to ensure road safety – maintain signage, create a licensing system so that everyone has a reasonable expectation that drivers know what they’re doing, maintain the infrastructure, etc. But once the government gets into the business of making cars or telling you which cars to drive, you’re in trouble.”
And about your mention of the “simplest word-processing program” – this is taking things way out of context. I can’t tell you the number of times, as a software developer, that someone has asked for MS Outlook-like calendar functionality or MS Excel-like worksheet functionality in a web application. You don’t realize how much work has gone into something “simple” like MS word. Microsoft word is a software application that has been under development for close to THIRTY YEARS, has had probably THOUSAND of programmers working on it for millions of man hours. It’s easy to knock EMR systems until you realize how complicated and ill-defined the workflows are around them.
Just go to any hospital’s ER and ask someone to explain step by step how their triage process works, and you’ll get 150 different answers from 18 people. Then go try the same thing at another hospital.
Ben,
1. I agree about standards vs products. To legislate products would stifle innovation. (For several years I led the working group that developed PPML, an XML-based print language for digital print, now supported by all the major controllers for digital print: HP, Xerox, EFI etc.)
But if you’ve been following the debates this past year about HITSP (health IT standards), I don’t think there’s much useful that can be achieved with today’s products in the short term by standards. The PPML standard was developed from scratch precisely because the existing proprietary systems (with their proprietary data formats, object models, and workflows) were too far apart to try to converge on a harmonized model.
In my experience trying to solve reliability and concerns by developing a standard can be a good idea – though it still doesn’t address usability, to my knowledge. And besides, in my experience once you have a standard THEN you start developing products.
So I’m still left with the question: what do we do NOW, with this group that’s defining certification (or rejection) of systems in the next several years, before “start from scratch” systems AND WORKFLOWS are created & implemented? It’s a real-world question.
We do have a very complicated system and I agree with standards as well. The clinical area needs some uniformity, I had one doctor tell me he had to learn 5 different EHR systems to get through his residency, lucky he worked at 5 facilities with EHRs I would imagine.
What needs to be certified are the back end algorithms so information is calculated and presented accurately.
Now if the software vendors could get their acts together and at least make similar clinical viewing and input screens, huge time saver. There is a project called the Common User Interface which is on Code Plex, the open source Microsoft site and developers have used it and are building interfaces. What doctor would not love to have the same or similar screen at any hospital, no brainer there.
http://ducknetweb.blogspot.com/2009/09/codeplex-foundation-new-open-source-non.html
Here’s a doctor/developer in New Zealand I chat with who is working with it, no Silveright in there but the stuff is in the same format.
http://ducknetweb.blogspot.com/2008/10/common-user-interface-ehr-development.html
The rest of the system could remain intact as designed but not enough attention is paid to the front end for the doctors, nurses, etc. that use the systems. If we could get closer on an item like this, certification would become an easier process as well. Basically it’s time for the developers to “eat some of their own dogfood” and collaborate on this one area where it makes the difference in patient care, in other words some standards.
The common user interface has been around for a few years with many NHS developers contributing as well and it was somewhat born in the UK. I have had some informal communications with a couple developers who like it too, makes sense.
So we have many tools available today, but can we collaborate is the big question. Having had written a simple EMR years ago, I too tire at looking at all the choices (EHR vendors)we have today, the goal has not changed as far as creating records for better healthcare, but how we are getting there certainly has.
“…if a system you had to use every day would often display unreliable information, while insisting you do things that make no sense.
…that the maker of the system insists on a “hold harmless” clause, so when something goes wrong, it’s not their fault. Legally no consequence for failures.
…that nobody involved in the malfunctions talks about them, and many people are not allowed to talk about them, much less collect a bug list.”
Talk about a busted system.
I haven’t been able to convert the recording to display it here (it’s in RealPlayer’s proprietary format), but if you have RealPlayer installed you can play it yourself at http://real.welch.jhu.edu/ramgen/DHSI/Dec182009.rm.
The slides are often out of sync with the audio but they catch up.
Skip the first 9 minutes; the talk starts around 9:15 and goes 30 minutes, followed by 30 minutes of Q&A. The meat of it is in Dr. Koppel’s talk, but as usual the Q&A contains additional juice.
I disagree with everything. EMR people talk about interoperability but they have yet to achieve operability. CCHIT defines the UI and workflow for EMR systems in their standards based on the UI and workflow in systems that don’t get adopted. If I were ONCHIT, my single driving focus would be: don’t stifle innovation. Health IT is today in the “Altair” phase, and we’re adopting it as a standard. Imagine if the government set standards for computers that locked in 300-baud modems, the AT&T telephone network, and computers managed by 16 toggle switches on the front panel. That’s essentially where we are today with health IT.
For those of us with our feet in both camps — health IT and IT — we can see where health IT could go. We also recognize that safety is the paramount issue and so a generation of BSOD’s (like we had in tech) is not acceptable. Regulate quality of systems. Require documented use-cases. Make providers and software publishers accountable when errors occur in off-label (i.e., off use-case) use of HIT. Make software publishers and implementation consultants accountable for errors in standard function.
Do not, however, tell us what that standard function should be. Don’t lock us into your use cases. You believe in the system. We don’t.
The entire healthcare system is broken. Here’s how it works: physicians and facilities borrow enormous sums of money to purchase equipment. They need that equipment to be used to service the debt. The equipment gets used. Remember McGlynn’s 2003 NEJM article on the percentage of recommended care received by patients? Just because they average 55% of recommended care, doesn’t mean that they are getting less care. Patients today get care they don’t need and don’t get care they do. They also don’t learn how to avoid needing that care in the first place.
Healthcare today is not a system we want to codify into our software, and we don’t want to penalize systems that use technology to deliver care in a way that ONCHIT didn’t imagine. Innovation comes from a small group of people who come up with an idea so brilliant that no one recognizes its genius until they see the results. Healthcare needs a “germ theory of disease” type of revolution — a change in delivery paradigm so radical that every entrenched bureaucrat tears his or her hair out to even think of it. This revolution will not just integrate patient engagement, it will be based on patient engagement. Society will have to change. We have seen a change like this in our lifetimes: our society has re-aligned around a radical shift in access to information. We need to lead society to such a change in our approach to healthcare.
Tell ONC to stay out of our way.
I am co-Chair of the Adoption Certification Workgroup and I have a few comments.
Like all technologies, electronic health records have huge potential to be beneficial to patients, but also have potential to be the source of unintended consequences. Our Workgroup is holding public hearings with a goal of understanding the circumstances under which Patient Safety problems occur in EHR systems and, also, determining the best approach to insure that these IT systems do no harm to patients. For this hearing on Feb. 25, the Workgroup members will be listening with an open mind to many viewpoints. We asked Dave to participate to represent a patient (consumer) viewpoint.
In accordance with the transparency rules established by the Obama administration, all Workgroup hearings and discussions are open to the public. The schedule of all HIT events can be found at:
http://healthit.hhs.gov/portal/server.pt?open=512&objID=1269&&PageID=16497&mode=2&in_hi_userid=11113&cached=true
You are invited to listen to the Workgroup’s meetings and to participate in our work. Your comments and advice will influence our recommendations.
Paul Egerman
So great to see you here, Paul. Thanks for your informative call today.
I’m amazed that none of us has yet come up with other suggestions. Difficult questions!
Pete – you’re missing something major here. The purpose of most commercial EHR systems is NOT NOT NOT to create interfaces that are “intuitive” or model the workflows of their end-users. That you assume this is the case is a false assumption on your part. :)
Take a step back. Who is buying these EHR systems and why? For the most part, it’s either small private practices or it’s large hospitals. Small private practices cannot afford the super-tailored solutions. Hospitals are not looking for a solution which improves outcomes, enables patient-driven medicine, or are even easy to use. Your average hospital will put out an RFP for an electronic medical record system with the primary success metric being “will it increase our income and accuracy of our insurance billings.” That’s it. Most EHRs are primarily designed to facilitate and document BILLING OF MEDICARE and other patient insurance carriers. And for the most part, they do this very well…
Ben- I have built, sold, and implemented EHR systems and you are right. This is why government shouldn’t take standard practice and cloak it with the mantel of respectability. All of American healthcare is going to change in the next two decades or else America will sink under its weight. In 2002, McKinsey projected that within 5 years healthcare costs for the fortune 500 would pass profits at those companies and that this would constitute a crisis. We shrugged off that milestone, but eventually the crisis will come. It is inevitable.
The joint problem of financing a system with terribly misaligned incentives will have to be addressed. My hope is that ONCHIT does not make true reform more difficult by increasing the amount of our health infrastructure that is designed to support the old, bad way. Right now, a revolution in healthcare would required mostly financial restructuring — many hospitals would have to pass through bankruptcy as financial incentives get realigned to correlate high pay with successful outcomes. If health information — and the workflows for health information — also have to be scrapped and re-built to achieve this, we will be that much less likely to see true reform and more likely to see gradualism without disruption.
Thank you for offering the opportunity to provide suggestions as you prepare for Thursday’s meeting. There will be no one solution or strategy to unwind the many bizarre and unintended consequences embedded in the health care system, this is the first point that must be embraced. The second point to embrace is that the US health care industry does “work” for a large majority of stakeholders, otherwise it would not be so incredibly stable in the face of challenges. What has changed is our collective tolerance of unlimited funding for unknown benefits, but we’ve not yet fully translated that into clear and concise policies and performance feedback mechanisms and we’re ignoring the substantial (although insufficient) advances that have been made areas like chronic disease management. I’d suggest we keep in mind that the government is now payer of >50% of health care dollars and it doesn’t even place significant resources to make sure it is buying legitimate services, much less effective ones. Despite the fact that every year the OIG reports 5:1 return on investment in recovering fraudulent claims, largely from overtly criminal enterprises with no legitimate purposes, the government only slowly expands those efforts. When the biggest player in the industry cannot even assure it is buying real care, the need for sophisticated IT implementations begins to fade behind the simple need to operate like a responsible enterprise.
Here is why this is especially pertinent. The ONC’s own commissioned 2005 report on EHRs and fraud showed that EHRs would initially worsen the trustworthiness of health care information, and that this would be reflected in increased health care fraud. The follow-on report to the ONC, commissioned to provide recommendations to forestall or mitigate such damage, was delivered June 2007 and provided a list of key EHR functions to improve simple records validity assurance. That report has been buried. Those few that, by individual effort, made it into EHR Certification recommendations thru CCHIT were stripped out in 2008 and 2009 (I know because I served on the CCHIT Compliance workgroup, which now doesn’t even exist). The then-ONC, Dr. Kolodner, met with the 2007 report workgroup in January 2007 and, among other topics, had a lengthy interchange about the fact that neither HITSP (driving EHR interop) nor CCHIT (driving EHR certification) had basic business record validity and integrity record on their requirements list. Dr. Kolodner promised he would make it so and never did. Repeated inquiries to the ONC on this point have gone unanswered.
So, in a nutshell, this is a long way of saying that at the present time, EHRs are not even required to meet the same requirements that a record must uphold in all other business domains. If we do not expect health care’s primary source record to achieve the level of trustworthiness that your common everyday storefront retail store must uphold, why do we expect that these same non-standardized, minimally certified systems will necessarily improve patient safety, security, care effectiveness, etc.?
What is needed at this point is nothing more sophisticated that to point out to any and every entity looking at EHRs that they should:
1. Make sure the system is capable of supporting that organization’s policies and procedures for accurate, reliable medical records. (Sadly, even Certified EHRs exist that can’t pass this simple test, for example, is the indicated author of the record actually accurate.)
2. Make sure the system is trained and supported in a manner that aligns with the organization’s medical records policies and procedures.
A little due diligence by EHR purchasers would go a long way. A little emphasis on information integrity by the largest purchaser of health care services (i.e. HHS) would go a long way.
Lastly, every patient of a practice with an EHR should regularly request a printed or electronic version of their clinical record. The errors, omissions, over-documentation, and wrong author/provider events will stir some useful “feedback” as well.
The most important thing in my opinion is to quit giving the public the implicit message that some overarching “we” knows how this all should work. This is a matter of national urgency and critical importance for the future of the country, and reasonable expectations should be nurtured, as well as everyone’s engagement on behalf of national interest and, more importantly, the benefit of every person in the US.
Best of luck in your remarks this Thursday.
Reed D. Gelzer
Wallingford, CT
(Former member HL7 EHR-S Standards workgroup on Records Management and Evidentiary Support
Former member of CCHIT Ambulatory Cert. WG, Privacy and Compliance Expert Panel, and Privacy and Compliance WG, Former Member of AHIMA/RTI Study and EHRs and Data Qualty/Anti-Fraud.
Co-Author: How to Evaluate Electronic Health Records Systems.)
Good morning ePatient Dave, any follow-up?
Reed D. Gelzer, MD, MPH, CHCC
Advocates for Documentation Integrity and Compliance
Wallingford, CT
Hi, Reed. Thanks for prodding me to go back and post follow-up notes.
After this preliminary discussion, I posted the testimony document that I pre-submitted here:
http://pmedicine.org/epatients/archives/2010/02/testimony-submitted-to-the-adoptioncertification-workgroup-for-its-feb-25-meeting.html
Under “Government 2.0” transparency, complete meeting resources and transcripts are posted online. Here’s the archive of that meeting, including audio, everyone’s testimony documents, and transcripts of the actual discussions.
After the meeting there was email discussion among those who testified. I posted here about that later.
All of that was under the Adoption/Certification Workgroup, which looks at what the rules will be for providers to qualify for the incentive money.
You may be interested in a subsequent hearing of the Meaningful Use workgroup, which is writing the regulations on what qualifies as meaningful use. The April 20 hearing was on “What is the role of patient-generated data in meaningful use?”
I posted my testimony, linking to an earlier discussion here:
http://pmedicine.org/epatients/archives/2010/04/testimony-submitted-to-the-meaningful-use-workgroup-for-its-april-20-meeting.html
Here’s the archive of that meeting, including audio, everyone’s testimony documents, and full text transcripts of the meeting. Regina Holliday’s testimony was especially compelling.
Thanks again for prompting this update.
Good afternoon Dave,
Thank you for the updates. FWIW, I was part of the team that supported CMS’s review and analysis of public comments on the draft Meaningful Use rules, as well as responding to their requests for recommendations on the final rules for qualifying for incentives payments, so quite familiar with that process.
Interesting times indeed.
I am much more interested in your observations, reflections, and predictions on how well your concerns are being heard and acted on inside the Beltway.
Cheers!
Reed D. Gelzer, MD, MPH, CHCC
Advocates for Documentation Integrity and Compliance
Wallingford, CT