Optimizing Order Sets With a Large Language Model–Powered Multiagent System

What is the utility of a large language model–powered multiagent system in generating suggestions to optimize order sets compared with expert evaluation?In this cohort study including 735 suggestions for 71 order sets, 96 suggestions for 9 order sets were evaluated by 3 physicians, and 639 suggestions for 62 order sets were evaluated by 1 physician. The median number of useful suggestions per order set was 2 in both evaluations. Among the 96 suggestions, 54% were rated highly accurate (score ≥4), while 19% were rated highly useful, 16% feasible, and 12% as having a direct impact.Results of this study suggest that multiagent systems offer a scalable and effective approach to enhancing order set optimization.

Corresponding Author: Siru Liu, PhD, Department of Biomedical Informatics, Vanderbilt University Medical Center, 2525 West End Ave, #1475, Nashville, TN 37212 (siru.liu@vumc.org).

Link to the article in your story

We encourage you to link out to this article in your story using the link below. It includes an access token that will give free access to the article for your readers up to one year after publication. (The link will be live after the article publishes and embargo is lifted.)

Please see the article for additional information, including full author list, author contributions and affiliations, conflict of interest and financial disclosures, and funding and support.

Need more information? Contact us.