A primary focus for healthcare organizations post-Covid is improving health equity for their communities. At HAS 2022, a duo of data scientists from ChristianaCare shared an analytics framework for measuring and managing equity disparities and described how they developed and applied a machine-learning-based algorithm to quantify differences in health outcomes.
The framework consists of a data layer, an analytics layer, and an interpretation and decision support layer. The data layer identifies and standardizes data structures for a list of health outcomes. The analytics layer calculates the defined metrics in an integrated, automated way. And the interpretation and decision support layer highlights how a user-centered design dashboard can support clinicians and leaders in understanding and interpreting outcomes.
Download
Editor’s Note: Editor’s Note: This article is based on the Achieving Health Equity: Measuring and Managing Disparities with Machine Learning presentation featured at the 2022 Healthcare Analytics Summit. The presentation was delivered by ChristianaCare Senior Data Scientist Wei Liu and ChristianaCare Data Scientist and Data Governance Lead Yuchen Zhan.
Data analytics and augmented intelligence can be crucial tools to advance health equity.
At the Healthcare Analytics Summit 2022, ChristianaCare Senior Data Scientist Wei Liu, along with ChristianaCare Data Scientist and Data Governance Lead Yuchen Zhang, outlined their success in developing and implementing a machine-learning-based data analytics framework that provided more robust insights for addressing health disparities at enterprise scale, helping clinicians and leaders identify opportunities for improved health outcomes.
Health organizations are continually challenged with determining which equity factors lead to health disparities in their communities. According to Liu and Zhang, the first step in advancing health equity is for an organization’s leaders to establish health equity as a strategic goal.
At ChristianaCare, Liu and Zhang developed a platform using machine-learning algorithms to quantify, visualize, and interpret the current state of health equity across the organization for multiple health outcomes. This platform provided data-informed insights in real-time that proved integral to the various initiatives ChristianaCare launched to close health equity care gaps.
ChristianaCare serves a diverse population. Health system leaders recognized that health disparities were evident in multiple health outcomes and sought to identify strategies for improving outcomes for these groups by establishing organization-wide initiatives that could quantify the influence of race, ethnicity, and SDOH status on health disparity across the entire organization.
Liu and Zhang recognized that conventional analytics were insufficient in addressing equity care gaps. To advance health equity, their framework needed to be updated to provide the insights necessary to determine which factors contributed to these inequities.
First, the team needed to establish system-wide metrics that would allow them to quantify health equity or health disparities in a meaningful way.
Liu and Zhang looked to microeconomics for inspiration, specifically the Gini Index, used for decades by the World Bank to establish metrics for income disparities in different countries worldwide.
Mathematically related to the Area Under the Receiver Operating Characteristic (AUROC) curve, a measure used to evaluate prediction models, Liu and Zhang identified the Gini index as a model framework for beginning to quantify how personal characteristics may be recognized as predictors of health outcomes.
By combining the Gini Index metrics with an already established understanding of how quality of care can vary according to factors such as age, ethnicity, gender, language preference, payer, and race, the team was able to leverage these tools and develop an analytic framework that analyzes and predicts the influence of personal characteristics on a variety of health outcomes.
The efficiency of this multilayered framework helped aggregate data, calculate core metrics, and efficiently interpret the results.
The three components of the analytic framework included:
The analytic framework implemented at ChristianaCare that aimed to advance health equity provided core metrics in four areas:
As the system’s broadest metric, the Gini Index provides the base understanding for analyzing and ranking health disparities across health outcomes. Seven health outcomes are analyzed in their model, including chronic obstructive pulmonary disease (COPD) patients with 30-day readmission, Covid-19 patients with hospitalization, diabetes patients, heart failure patients with 30-day readmission, hypertension, pre-term deliveries, and sepsis mortalities.
The Feature Importance establishes which equity factors are the driving force behind health disparities. The model includes the following factors: age, ethnicity, gender, language preference, payer, and race.
By analyzing health outcomes related to equity factors, the framework quantifies the level of disparities based on the proportion of patients with defined health outcomes in each equity factor group.
Calculated with the prediction score model and adjusted in relation to other factors in the model, the OE quantifies the difference between observed outcomes versus the target outcomes where no disparity exists.
Liu provided graphics of the dashboard they developed, which helps clinicians visualize and interpret the data.
These charts are augmented by the dashboard’s visual timelines that track the historical trends for both the Gini Index metrics and Feature Importance factors that allow clinicians and leaders to identify significant changes over time.
In their example, the Gini Index graphic ranks the selected health outcomes according to the level of health disparity. Covid-19 patients have the highest disparity, and COPD and sepsis patients ranking behind with the second and third-highest proportions of disparity.
Displayed directly underneath the Gini Index are the same seven health outcomes, segmented according to each of the seven Feature Importance factors with visual representations quantifying the degree to which each impacts health disparity.
The presenters provided data analysis examples observed at ChristianaCare:
The framework unveiled correlations that would otherwise have remained hidden. In the sepsis mortality example, the Gini Index score was the third highest of all the health outcomes. However, a closer look at the Feature Importance levels revealed age, not race, as the primary factor, putting it out of the range of equity disparities.
Through these findings, the presenters demonstrated how leveraging conventional statistical methods with machine-learning analysis allows organizations to develop informed strategies and efficiently allocate valuable resources.
Liu and Zhang emphasized that the clinical examples presented by ChristianaCare also demonstrate how the multiple analysis layers provide flexibility and can be scaled to address the unique circumstances of other organizations.
Each layer of this analytic framework is designed to provide the data and user-centered visualization and interpretation that will supply all members of an organization, from operational leaders and clinicians to health executives, with the actionable data necessary to understand and close equity gaps.
Through engaging executives, clinicians, and operational teams, machine-learning analytics can provide the decision support necessary for implementing organization-wide strategies that minimize health disparities, advancing health equity.
Identifying underlying causes of inequities in healthcare is complex. As evidenced in Liu and Zhang’s work at ChristianaCare, health equity first needs to be a strategic priority for healthcare organizations. Organizations then need to build a flexible framework that incorporates machine learning to complete advanced calculations, eventually identifying which of the numerous factors (age, race, and SDOHs) result in care inequities. Following ChristianaCare’s lead, other healthcare organizations can begin advancing health equity and enable data-informed improvements for the communities they serve.
