Healthcare has always been a complex environment, but with potential legislative appeals and unforeseen mandates on the horizon, the present state of uncertainty brings a new set of challenges. Success requires unprecedented agility in terms of acquiring and responding to meaningful, strategic information. Developing an effective strategy requires that a healthcare organization understand the reality, as distinct from the perceived reality, of its operational environment.1 This move to basing decisions on facts, as opposed to assumptions, sets the stage for hospitals and health systems to maximize benefits gained from the information they have, and to move to the next level of discoveries and insights available through analytics adoption.2
Mass General Brigham is a non-profit integrated healthcare delivery system that includes over 6,000 physicians. It’s composed of community hospitals, health centers, ambulatory sites, and two large academic medical centers. Through its network, Mass General Brigham offers a wide range of healthcare services, including psychiatric, rehabilitation, and homecare. Its mission is to address the need for fostering improvements in patient care through a focus on research, teaching, and community health.
To better manage the complexity inherent with a structure and mission of this scope, and to grasp the real nature of the environment in which it operates, Mass General Brigham needed to tap into the insights provided by its complex data streams. To this end, Mass General Brigham determined it needed to establish a more robust, data-rich analytic environment that would allow for many more data sources and greater integration of information. The health system implemented the Health Catalyst Analytics Platform, including the Late-Binding™ Data Warehouse and broad suite of analytics applications.
Recognizing that to gain full understanding of the complex healthcare environment it would be necessary to have many data sources available in a single environment, the health system began immediately using the installed EDW to demonstrate the value of the clinical care it provides. However, Mass General Brigham knew that getting a clearer picture of performance and installing an EDW would not be a solution in and of itself, regardless of the EDWs inherent flexibility and ease of data acquisition. To be successful, Mass General Brigham needed widespread adoption of the EDW to ensure information was meaningfully incorporated into strategic and operational decision making to support patient care.
The value of an EDW cannot be measured by its content nor by how often it is accessed. Instead, the value of an EDW is measured by what impact it has on clinical, financial, and operational outcomes. The size of that impact is dependent on the ability of users to interact with, and effectively use, the tools and information that the EDW provides.
With any analytic endeavor, establishing priorities is a must; it’s impossible to take on everything at once. A good prioritization model must take into consideration the highest needs, weighed against what is achievable with the resources and time available. While keeping a focus on high-level priorities, an organization must also make sure they don’t inadvertently limit the ability of a department to innovate and make the improvements necessary to function more efficiently and more effectively care for patients.
Ultimately, the Mass General Brigham team viewed successful adoption of the EDW as a two-fold challenge: First, how do you drive widespread adoption throughout the health system? Second, how do you ensure users have the knowledge, skills, and support necessary to use the EDW and analytics applications effectively?
Granting the right access to the right people. Mass General Brigham had three pre-existing data warehouses for claims, cost accounting, and clinical quality data, and there was a dedicated set of users in place who worked with each. As Mass General Brigham transitioned to the new EDW, these teams were able to bring with them specific content knowledge and analytic expertise, along with the wide range of tools they used to mine data in the warehouses. These sophisticated users had a long history of interaction with the Mass General Brigham EDW team.
However, the long-range goal was to expand the EDW user base to include users who needed to access information to support patient care, not just individuals versed in data management and analytics. To that end, Mass General Brigham’s executive leadership embraced a philosophy of granting expanded access to data stored in the new EDW, as long as there was legitimate need. The criteria for access would be based on a user’s responsibility and job function, regardless of their previous knowledge of analytics.
This decision opened up analytics to a whole new audience. Leadership at Mass General Brigham saw this as the most direct way to foster an expanded user base and drive rapid adoption, which in turn would encourage innovation and the strategic use of information. Leadership also adopted a “bring your own tools policy,” allowing the use of multiple business intelligence tools on top of the new analytics platform. This allowed the experienced users to quickly take advantage of the expanded information in the new EDW by using tools they were already familiar with.
The philosophy of expanded access brings with it some concerns, but Mass General Brigham has not had any problems involving a misuse of data. To protect against inappropriate data access, Mass General Brigham adheres to all requirements of the Health Insurance Portability and Accountability Act (HIPPA), the provisions of which are designed to safeguard medical information and ensure data privacy and security. As part of this, Mass General Brigham enforces a requirement that individuals can only access data that is needed as part of their job. This standard guidance includes all data stored in the EDW.
However, despite safeguards that ensure the right people have the right access to data, opportunities remain for improving the efficiency of data access for users. For example, there are likely some individuals who access data they don’t fully understand. Also, some analysts waste time by insisting on going back to the database level to calculate a measure that has already been defined and made available, simply because they don’t trust the data. These are issues that Mass General Brigham continues to work on to improve.
Assessing the impact of new users. The new users came in as a result of Mass General Brigham’s new robust analytic environment’s ability to aid the clinical, operational, and research components of the organization. However, they brought with them their own set of challenges and demands. In the beginning, new users were not well supported; there was no additional infrastructure to provide them with analytic help or business case analysis. There were also hundreds of new users, and as a group they generated a large demand for support due to their lack of knowledge and minimal analytics background.
Each new user, however, also provides a great source of information and insights into the needs of ongoing operations. To make things easier for these users to get up and running, the Mass General Brigham’s EDW team has provided a gateway for users to request support. Through this gateway, the EDW team can provide an evaluation of requests, suggest sources of data, refer the user to a department with resources that are proficient with that subject area (such as population health management), or grant access to applications. However, the EDW team cannot manage how the information will be used or where it will be presented. That responsibility falls to the enforcement of policies and the management structure of the individual departments.
Previously, to get access to the information they needed, users filled out a form to request a report. This led to challenges in keeping up with the demand, and the reports themselves frequently needed further refinement, leading to users making multiple requests for information. Now, more users have access to the EDW and can create the reports themselves. This has improved efficiency and freed up analysts to conduct more meaningful data analysis, as most users are well-suited to this self-service model. It is possible that some additional analytic support will be needed for users who don’t know how to write their own reports, but Mass General Brigham has seen the self-service model provide a more efficient gateway to data access and decision making.
Measuring use of the EDW after analytics adoption. In complying with the improvement adage that you can’t improve what you can’t measure, Mass General Brigham developed an application to help monitor system usage statistics. Currently, the EDW at Mass General Brigham has over 700 users.
Growth in user numbers is important, but meaningful use is more important. Use cases are just now starting to present themselves, driven by user needs. This is an indication that the adoption approach is working and that an increasing number of new users are effectively using the data in the system. Rather than counting sessions based on single access, Mass General Brigham has developed parameters to identify more meaningful interactions. As a rough rule of thumb, it assumes that queries involving three or more databases, or involving multiple different applications, are an indication that data is being used effectively.
These assumptions are the basis for the trends monitored in the EDW Utilization Application (see Figure 1), which tracks the number of unique users and sessions. Additionally, the application tracks the number of users based on the number of sessions, reports, and sessions by duration. Not only is the data used to understand EDW utilization, it is also used as a surveillance tool for compliance auditing and technical performance.
Looking at how data is being used. Another way of gauging the meaningful use of information by the user base is to look at what they are doing with the information in the EDW.
By focusing on the users that are dependent on analytics to support their job functions, the Mass General Brigham’s EDW team was able to build visible, stand-out capabilities of the system with departments such as population health management, quality safety and value, business planning, and contracting. The following are just a few examples of how they are using the EDW and leveraging data analytics:
By deliberately allowing less obvious capabilities of the system to develop and grow through Mass General Brigham’s expanded access policy, the organization was able to uncover and address additional use cases:
Some groups of new users are leveraging the EDW in unanticipated ways, providing additional fuel for innovation. Users have done analysis for operational and clinical improvement initiatives, such as a project to identify and follow up on high-risk patients who would have benefitted from additional care in the Emergency Department, and a project investigating differences in resident-covered and nonresident-covered inpatient care.
Providing User Support. Mass General Brigham’s strategy to provide access to a broader set of individuals that needed data grew the EDW user base. As more people started accessing the data directly, the need for user support began to grow and change, sometimes in unforeseen ways. When Mass General Brigham originally implemented the EDW, its goals were to:
At that time, the Mass General Brigham EDW team was largely focused on the data management needs of the analytics community, supporting them with a portfolio of analytic assets, including the EDW, the EMR, and the legacy data warehouse. The EDW team did not do training at that time because users from the three legacy data warehouses already had the analytic, technical, and content knowledge that was needed.
The strategy to build user adoption through expanded access required a different level of support, education, and training. With the advent of the new platform, Mass General Brigham did hundreds of presentations that were tailored to the audience, usually at a high level or specific to one aspect of the EDW. The content was a moving target, as the EDW and the needs of users evolved. Physician training was delivered to regional service organizations which are a collection of practices, by physician champions and included, presentations, and small group discussions.
For employees who were new, or at least new to the EDW, Mass General Brigham conducted “orientations” that were 60 to 90 minutes in length. This was enough time to introduce the concepts, but not enough time to train an individual to use the EDW system. For users that wanted to learn more and dig deeper into the data, the EDW team provided online guidebooks and support documents, intended to provide self-service guidance.
In addition, Mass General Brigham developed an ongoing network of communications resources including an optional email alert process that users can sign up for to receive more detailed updates on the various EDW resources. These emails include data updates and alerts about data issues. The EDW team also provides a monthly email that is sent to all users that includes more general information to notify the user community of new resources that are available, and those that are currently being worked on, as well as updates to important policies/procedures and training offerings. The goal of this network of communication resources is to get the right level of communication to the right audience.
In hindsight, Mass General Brigham believes it underestimated the amount and types of training that would be required to support the expanding user base. As user numbers continued to increase and mature, it became apparent that the needs of the user community were shifting. The original EDW training was focused on what data was available, the data organization, and strategies on navigating the data set. The nature of the questions being asked of the EDW team have become less about data acquisition and management, and more about data interpretation and content analysis. Mass General Brigham has determined that the knowledge that was maintained and transferred by the teams who managed the individual data sets prior to the EDW, now needs to be communicated through more formal and diverse training opportunities. It is felt that this approach will develop the different type of knowledge and skill needed by the expanding user community.
Mass General Brigham strongly believes that a significant investment in training is required to further the effective use of analytic investments it has made to date, and to further increase data literacy of users. Improving literacy in analytics users ensures Mass General Brigham becomes more data-driven as an organization. Before forming a plan for training, the Mass General Brigham’s EDW team wanted to ensure they had a full understanding of the educational needs across the organization related to data analytics.
The team conducted two surveys to better understand areas where the platform and team were performing well, as well as areas where there was opportunity for improvement. The feedback revealed a strong need to improve training and user-support capabilities. The most significant need identified by respondents was content training, with more than 90 percent of those surveyed ranking it as a beneficial or critical need. When asked to prioritize their content topic needs, revenue/expenses, payer claims, and EMR registry data were ranked at the top. The next six content topic rankings listed by those surveyed all centered on EMR data and workflows.
As far as the type of training needed, 75 percent indicated that training on how to effectively combine information across EDW data sources would be beneficial. Respondents identified that computer-based and dedicated in-person delivery were most desirable, and surveyed users also requested to have a combination of options available. Around 90 percent indicated it was “very” or “somewhat critical” to include Mass General Brigham’s data and use cases in the trainings.
Mass General Brigham’s approach to driving analytics adoption has resulted in hundreds of people having broad access to the most diverse and robust analytic dataset ever made available to its analytic community.
The success in user adoption was accomplished by focusing and delivering against specific, targeted analytic needs, as well as by providing access to a platform where a much broader audience can leverage self-service which helps drive innovation and analytic efficiency.
More users and more data creates new, unprecedented challenges for support and training.
Mass General Brigham, like many healthcare organizations, has had to come up with an innovative solution to meet this need. Mass General Brigham formed a new, more comprehensive Data and Analytics Organization to better meet the changing needs of the expanding EDW, data analytics, and support.
In 2016 (through June), the Mass General Brigham’s EDW Team fielded about 22 questions a week, and about 50 percent of them were content related questions across data sources and subject area data marts. Mass General Brigham used survey results to both quantify the success of their initial approach to support and training, and provide directional insight on how to move forward.
“The value of an EDW should not be measured by how much it is used, but rather by what impact it has on improving care or lowering costs. That said, you improve the chances for greater impact by achieving broader EDW adoption.”
Mike Noke, Associate Director, Enterprise Data Warehousing
The Mass General Brigham’s EDW team expects the user base to triple in the next year—propelled by a new phase of users at the frontline clinician level seeking access to two new analytics applications, Quality Insights and Physician Variation, providing these users with data they want and need and cannot get elsewhere. By leveraging the evolution of self-service tools within the analytics applications they expect to make it easier for users to serve themselves and interact with the data they need to do their work.
To further the culture of self-service, Mass General Brigham is planning to increase its level of technology-enabled support through further development of documentation on a collaborative sharing platform to include how-to guides, data models, frequently asked questions, and more.
Mass General Brigham is in the process of determining the next set of analytic priorities for the organization. The further development of the Mass General Brigham Data and Analytics Organization will shape some of the direction. The organization intends to add an additional analytics training program focused on workflows and data derivatives. This program will be based on a combination of online and classroom training that incorporates real Mass General Brigham’s use cases and data. To speed up the time to delivery, Mass General Brigham is looking for an outside partner to aid in the development of initial training content and materials.
