Skip to content

Blog Posts

User-centered design process step 4: Usability testing

Editor’s Note: This blog series provides readers an inside look at how members of our user experience team approach new projects. As they go through the design process for a specific solution, they’ll share how they use science and a rigorous process to create a better experience for our users. This is the final post in a four-part series.

In my second blog post, I talked about creating measurable usability goals for the design of the product. These goals are used to measure the success of the design and ensure that the specific usability targets are met prior to shipping the product. This post will cover how we measure the goals through usability testing.

Review of goals

For Rx Writer we created three different types of goals:

  • Task-based goals: Can the user complete specific tasks required for prescribing?
  • Efficiency goals: Have we made searching for and creating a prescription faster?
  • Perception goals: Does the user feel like the new Rx Writer is safer, faster, helpful, more modern and easier to use?

We talked about how these goals will be measured in multiple ways, including asking the user to complete tasks, subjectively measuring before and after times with the old product and through the use of surveys and user feedback. We use formative user testing to measure these goals.

Testing methods 

During the project, we completed four rounds of formative user testing. Each round of testing focused on specific functionality and iterations to the design:

  • Round 1: General navigation, searching medications, completing basic medication forms
  • Round 2: Enhanced searching of medications, dose calculations and favoriting medications
  • Round 3: Current medication list functionality, improved form layout, adding free text to the medication instructions and the display of pricing information
  • Round 4: Accessing the PDMP, display of MME (morphine milligram equivalents) for patients with opioid medications, entering free text medications, performing bulk actions on medications and searching for groups of medications.

During each session, a moderator asked the participants to complete a series of tasks using a prototype of the Rx Writer design. Since there was no prior training or exposure to the prototype, we were able to see how challenging it would be for a new user to use the design. An example of a task given is “Add a prescription for amoxicillin to treat a patient’s UTI”.

The participants also completed a pre-survey to collect feedback on the existing Rx Writer in Allscripts SunriseTM (if they have used it) and a post survey about their experience with the prototype that they used during the session.

Usability goals measured

Throughout the rounds of testing, the goals created at the beginning of the project were measured during the tests. If a goal was not met, we iterated the design and then retested in a later test until the goal was met. Here are the some of the results of the measurable usability goals identified for this project.

 

Metric Goal Achievement
The user understands how to access each part of the application via the high-level navigation model

(search, Add/Edit, Submitting Rx, etc.)

100% of users can complete task by the second attempt Met Goal: First attempt completion rate was 97% with 100% completion by the second attempt
The user can Complete a new prescription 100% of users can complete task by the second attempt Met Goal: Average completion was 93%

Note: For the tasks that did not test well, the design was updated to improve the usability

The application helps the user prescribe medications safely 4 on 1 – 5 scale Met Goal: Average score was 4.0, which was higher than the existing score of 2.2.

Note: This improved between test 1 and 2 from a 3.6 to a 4.0

The application helps the user prescribe medications quickly 4 on 1 – 5 scale Met Goal: Average score was 4.4, which was higher than the existing score of 2.8.

Note: This improved between test 1 and 2 from a 4.0 to a 4.4

Users evaluate the application as easy to use Score of 70 (out of 100) Met Goal: The average SUS of both studies also met goal with an average SUS of 83 compared to the existing score of 49

 

 

We also learned other valuable information about the design of the product such as how users expect the form to be laid out and what their workflow is for changing a preferred pharmacy.

When we tested performing actions on multiple medications (Renew, Discontinue, etc.) the design was not really intuitive, and the users didn’t understand it. Their feedback prompted us to change the design and the changes tested really well in the next round of testing.

Testing also lets us know where the designs are good.  For example, when we tested the new dose calculator, one physician said, “It’s beautiful, really really nice, we’d use it every day.”  Another one told us that they “really enjoyed the dosing rounding.” While we refined the dose calculator design as a result of the testing, we made sure to keep the areas that the users really liked.

Design iterations

To achieve a usable design, iterations are essential. Like any science, as we collect the data, it is important to apply what we learned to the design to improve the overall usability of the product. We changed and retested problematic parts of the design to ensure we achieved the best outcomes for our users.

One example of this is the search screen. During our provider interviews, it became clear to us that our users wanted the search to be easy, help find common medications that they prescribe and provide one-click prescribing where possible.

In the first round of testing we provided this functionality, but the terminology we used was confusing to users. We changed the label to Favorites which was more recognizable to the users. Through observing our users, we learned that most of the providers tried searching using multiple keywords. For example, when trying to find amoxicillin capsules, the users typed in “amox 250 cap.” This was incorporated into the search design.

After multiple rounds of testing, our final design is a search paradigm that is intuitive, which enables the user to search in a way that makes sense to them and helps them find their common medications quickly for one-click prescribing.

Summary

Throughout this blog series we have gone through each step of the user-centered design process and learned how to use the information gained throughout the process. We learned who our users are and their current challenges with our existing system. We then set goals for the new design and tested the design to ensure the goals were met. We used science to create designs that reduce provider burden and help them focus on what is most important while prescribing. We revised the design based on user feedback and challenges during testing and then retested the changes. The end result is a new Rx Writer that has been proven to be more efficient, easier to use, safer and more satisfying to our users.

Our design process doesn’t end here. We will conduct a summative test pre-release and then evaluate the Rx Writer post release in use at our customer sites. We look forward to learning more about how we can further support providers as they prescribe medications.

View the previous posts in our series:

 

Comments 1

  1. David 07/21/2020

    Carin,
    This was a fascinating article and directly relates to my work here as a PM. I really enjoyed how you took seemingly subjective criteria and applied scientific metrics to make data objective and measurable. Great work – hope to hear more about similar projects and what you’ve learned!

Add a Comment


Scroll To Top