Raise Your Hand if You Think You Understand Usability
In my previous post, I didn’t address much of the industry discussion about Electronic Health Record (EHR) usability: What, exactly, is usability in the first place? How do you measure it? How do you improve it?
These questions are important for a couple of reasons:
- First, usability is a critical requirement for any software and should be a main focus of development.
- Second, the Office of the National Coordinator (ONC) is establishing policy to attempt to improve EHR usability, making it a regulatory issue as well.
Here’s some background on the second point for context. When the government passed ARRA to help fund Health IT adoption, a primary issue raised by the healthcare community was that Health IT systems have many usability issues. So – as many people posed – if the government was going to mandate EHR adoption, shouldn’t the government also ensure usability?
As a result, the ONC is under significant pressure to help improve EHR usability, but the complication that we have faced for decades is that usability is a compound construct. It is composed of three primary measures:
- efficiency (how quickly can I do something?),
- effectiveness (how accurately can I do something?), and
- satisfaction (how satisfied am I when doing something?)
But this presents a challenge: One person might identify that efficiency is the most important factor while another may decide satisfaction is more important. And what makes different people satisfied is, of course, different.
Considering this, the ONC (with AHRQ and NIST) decided that the only area it should regulate was safety (identified in the UX community as a component of effectiveness). As a result, there is a requirement in Stage 2 of the Meaningful Use EHR Incentive Program for a summative usability test. This test is comprised of having representative users walk through a series of tasks and measuring the amount of time to complete the task, how many and what type of errors occur, and satisfaction with the task interaction. But this measure does little to prove or ensure the safety of the system.
As an example, a workflow could automatically dismiss DUR warnings when prescribing a medication. By the letter of the Final Rule on certifying products for Stage 2, we could test this workflow where the user prescribes penicillin to a patient with a penicillin allergy. In this test, the workflow might show that we have improved the efficiency (and arguably the clinician satisfaction) by dismissing an “annoying” alert, and it may have even improved the effectiveness (depending on how we defined the goal), but it could possibly have a very serious impact on safety. However, this test would be satisfactory under the MU 2 guidelines.
If the goal is to improve the safety of the products, then we should consider models similar to those developed by the FDA.
The FDA has worked on improving safety in medical devices for years, including the application of a Human Factors technique called a hazard analysis. This is a process where all critical events are defined and categorized by a Human Factors professional. These critical events are then back propagated through the design to determine when, where and how they could occur, leading manufacturers to design mitigations that ensure the critical events are as unlikely as possible. It’s a process that has proven highly effective in improving the safety of these systems.
I’m not saying that what’s been developed by the FDA can be applied as is, because software development is different. But if the ONC is serious about improving the safety of these systems, then it needs to introduce measures and processes that will definitively improve the safety of these systems. And, the ONC needs to be explicit about that priority and the potential impact on the products (which could include some negative impact to efficiency).
The reality is that any government mandates into factors as esoteric as user satisfaction need to considered before implementation. These are the areas that market forces are expert in changing, and we should allow those forces to make an impact without government intervention. Usability is more complex than many people understand. My suggestion to the ONC: tread lightly or we may all find ourselves dealing with unintended consequences.
We at Allscripts are dedicated to the usability and safety of both our products, and the usability and safety of the industry as whole. We will continue to monitor these efforts to ensure a pragmatic approach that has positive benefits for patients, clients, and the Health IT industry.