Performance Drift in a Nationally Deployed Population Health Risk Algorithm in the US Veterans Health Administration

What are the extent, mechanisms, and clinical consequences of performance changes in the Care Assessment Needs (CAN) algorithm, a nationally deployed population health-risk algorithm used by the Veterans Health Administration?In this retrospective cohort study of 27 787 152 observations among 7 215 711 unique veterans, with more than 5 million observations from this patient pool recorded annually, CAN algorithm positive predictive value decreased by 4.00% and false positive rate increased by 0.34% from 2016 to 2021, corresponding to 18 288 increased false positives. Demographic, health care utilization, and laboratory variables were significant contributors to drift.These findings suggest that clinical risk algorithms such as the CAN algorithm are susceptible to performance drift, particularly during shifts during time periods associated with the COVID-19 pandemic, contributing to decreased reliability of quality metrics; therefore, regular monitoring and adaptation of algorithms should be used to ensure continued clinical utility.

Corresponding Author: Ravi B. Parikh, MD, MPP, Emory University School of Medicine, Emory University, 1750 Haygood Dr, N525, Atlanta, GA 30307 (ravi.bharat.parikh@emory.edu).

Link to the article in your story

We encourage you to link out to this article in your story using the link below. It includes an access token that will give free access to the article for your readers up to one year after publication. (The link will be live after the article publishes and embargo is lifted.)

Please see the article for additional information, including full author list, author contributions and affiliations, conflict of interest and financial disclosures, and funding and support.

Need more information? Contact us.