Skip to content

Blog Posts

Machine learning to optimize order set maintenance

Order sets have long been a workhorse within the electronic health record (EHR). They serve to organize and encapsulate evidence-based practice and clinical decision support, helping to reduce variation and standardize care for common conditions such as sepsis and heart failure. They also add convenience for common tasks such as routine AM lab ordering.

Well-designed order sets enable an efficient “one stop” experience for common workflows, such as admission to the hospital. However, order sets frequently do not capture unanticipated needs of end users who are addressing not only the condition driving admission but also frequently managing multiple comorbid conditions and polypharmacy. Ordering needs may change unexpectedly if the patient’s condition acutely changes, or as new important historical (e.g., from a family member) or diagnostic information (e.g., critical lab result) becomes available.

These factors increase the need to search for orders outside of the standard order set workflow, so called a la carte orders, which can contribute to increased clinician work burden. Maintaining order sets is labor intensive and requires specialized clinical and workflow knowledge.

To help reduce clinician order entry work burden, researchers (Zhang used a machine learning algorithm to identify commonly entered a la carte and order set orders soon after admission.1 This approach could potentially be incorporated within standard order sets, shifting to a more “one stop” ordering experience.

In a pilot study presented at 2018 AMIA annual symposium, researchers used EHR data to study two cohorts, patients receiving percutaneous coronary intervention (PCI) with no heart attack (n=109) and patients with pneumonia (n=80). Approximately 55% or orders for these conditions were a la carte. Using a technique called K-Means Clustering, frequent a la carte and order set orders placed within 12 hours of admission were identified.

Compared to existing production order sets, the machine learning modeled sets that incorporated frequently using a la carte orders would result in significantly fewer overall mouse clicks. Mouse click burden is expressed as Mouse Click Cost (MCC) in an optimization model.

A few caveats: While fewer click counts appear feasible over broad time intervals after admission, the distribution of formerly a la carte orders within an order set may lead to a single large order set or placement in many smaller order sets, which may have workflow consequences. In addition, frequently used a la carte orders do not always account for clinical appropriateness, and in some cases, might contribute to use of low value and wasteful testing or treatment. Actual implementations would require careful evaluations by clinical experts, staff, and IT team in each institution.

The challenge is to find processes that balance the efficiency opportunities uncovered by machine learning with potential unintentional consequences on workflow, clinical care processes, and ultimately outcomes. Human oversight is crucial, involving those with clinical and workflow expertise, ideally within an organizational framework, such as an Order Set Committee. Once order set changes are made, processes should also be in place to monitor workflow and care impact of any order set changes.

Machine learning is rapidly becoming a part of many aspects of our lives, and when combined with human oversight, have the potential to reduce clinician work burden, improve care processes and outcomes.

Zhang Y, Padman R, Levin JE. Paving the COWpath: data-driven design of pediatric order sets. Journal of the American Medical Informatics Association : JAMIA 2014;21:e304-11.

Yiye Zhang, Richard Trepp, Weiguang Wang, Jorge Luna, David K Vawdrey, Victoria Tiase; Developing and maintaining clinical decision support using clinical knowledge and machine learning: the case of order sets, Journal of the American Medical Informatics Association, Volume 25, Issue 11, 1 November 2018, Pages 1547–1551,

Comments 1

  1. Elmer Baquero 09/08/2019

    I blog often and I truly thank you for your content. This great article has truly peaked my interest. I will take a note of your website and keep checking for new information about once a week. I opted in for your RSS feed too.

Add a Comment

Scroll To Top