Guest Blog: Measuring Telehealth: Finding the Right Measure or Asking the Wrong Question?

Juhi Israni, MS, Adriane Lesser, MS, Michael Kurliand, RN, MHA, Zia Agha, MD, MPH, & Kelly J. Ko, PhD

A paradigm shift is slowly occurring in the healthcare field, moving from incentivizing the volume of services delivered to rewarding value-based care for patients.¹ With the significant shortage of healthcare providers, our nation’s profound demographic shift (i.e., growing senior population) and the increasing number of doctors retiring from practice, transitioning to value-based care is becoming, if not already, one of the most pressing issues in healthcare today.²Given these circumstances, alternative delivery models are being explored to more effectively utilize a shrinking supply of providers, while addressing the growing healthcare demands of our nation. One such delivery model is telehealth, which broadly defined, refers to the use of information technology and telecommunications to share information and provide training and clinical services from a distance. Although the potential for telehealth has been touted for years, adoption remains limited, with one of the most likely reasons being lack of reimbursement. While there are several factors for limited reimbursement, inconsistent measurement and mixed results regarding the effectiveness of telehealth certainly haven’t helped.

What does the data tell us today?

While there is a vast body of literature addressing the impact of telehealth, it is equally varied. Notably, a recent review of telehealth’s impact conducted by the Agency for Healthcare Research and Quality (AHRQ) reported mixed results after evaluating thousands of telehealth studies. However, the most robust evidence to show telehealth as generally clinically effective points towards its use in chronic condition management, remote patient monitoring, and delivering behavioral healthcare.³ Outside of these areas, there still remains an insufficient evidence base. While the volume of research is growing, so does the variability in how outcomes are measured, preventing firm conclusions as to where telehealth is most clinically and cost effective. For example, the literature focuses largely on specific clinical use cases of telehealth, outcomes which are captured inconsistently across institutions, or characteristics of the delivery model itself (e.g. modality), all of which are likely to vary and yield different results (see Figure 1.).

**Figure 1:** AHRQ literature map of existing telehealth studies

The literature map developed by AHRQ illustrates how telehealth is used for a variety of functions within multiple clinical focus areas. The studies that make up the existing knowledge base for telehealth typically focus on a limited number of research questions within a single context.

Given the heterogeneity of telehealth research, it comes as no surprise that measurement in telehealth is inconsistent, making it difficult to generalize across institutions or use cases. However, the most glaring knowledge gap identified within the AHRQ review, as well as other systematic reviews, is on the diverse and limited data regarding costs and utilization. The lack of consistent and available outcomes data on cost and utilization is most likely the primary barrier to reimbursement.⁴ While it’s important to note that outcomes measurement, especially cost and utilization, undoubtedly varies across organizations (i.e., based on coverage policies, operational differences, etc.) this is also what makes it difficult for payers and policymakers to assess the value of telehealth.⁴ Based on the current body of telehealth research, as well as a recent report by the National Quality Forum⁵, the opportunity appears to be in developing the right measures to help evaluate the effectiveness of telehealth.

Other measurement perspectives

While developing outcome measures specific to telehealth may be one approach, another perspective is to view measurement in telehealth from a broader perspective. In this view, outcomes are not focused on the delivery model or service itself, rather on the overall impact of a program. This perspective is perhaps best illustrated through the logic model framework which is used extensively within AHRQ as well as the Centers for Disease Control and Prevention (CDC) to evaluate the effectiveness of programs across a variety of settings.^6,7 Specifically, in a logic model, a program is characterized by inputs (e.g. resources, funding, staff), activities (e.g. events, training), and outputs (e.g. products, activities) and measured against overall goals or outcomes desired (see Figure 2). In this approach, the process (i.e., inputs, activities, outputs) are strictly means to the end goal, which is to evaluate outcomes.

Logic models show links in a chain outlining “what causes what” in the pathway toward desired outcomes, assessing continuously and multidimensionally from the planning stage to overall performance evaluation. Understanding the underlying logic of the process (e.g. care delivery model) from start to finish allows evaluators to select indicators to be used in implementation including measures of whether the activity provided sufficient resources, successfully delivered services as planned, and eventually attained overall outcomes of interest.

Using the logic model framework, the majority of telehealth research falls within identifying and understanding the process, as studies have generally focused on the delivery model itself (i.e., telehealth-related activities and telehealth-specific outputs) as opposed to the intended impact (i.e., outcomes). While understanding that process is important, there is a need to address the broader goals or outcomes of interest. For instance, the broader goal of a telehealth program may be to increase access, maintain quality of care, and demonstrate cost savings. These outcomes represent the overarching goals of a telehealth program, thus should be the focal point of outcomes measurement as opposed to telehealth-specific activities or outputs.

The topic of measurement in telehealth is timely given the inevitable shift in healthcare from volume to value, as healthcare systems are being tasked with providing the right care without increasing costs, while using fewer resources. Being asked to do more with less has steered health system leaders to explore alternative delivery models, and while telehealth is a promising approach, demonstrating value has been challenging and warrants a closer look at how we measure it.

The majority of telehealth research was done at a time when healthcare was still being measured in volume, hence corollary research was focused on the delivery model in addressing healthcare needs, and subsequently, volume. However, in a value-based environment, focus is shifting away from the delivery model, and instead considering its overall impact on health. As such, the need may not be in developing the measures specific to telehealth, but asking the right question; is telehealth improving health? Addressing this question requires not telehealth-specific outcomes, but simply “health” outcomes.

References

← Prev Next →

Guest Blog: Measuring Telehealth: Finding the Right Measure or Asking the Wrong Question?

Juhi Israni, MS, Adriane Lesser, MS, Michael Kurliand, RN, MHA, Zia Agha, MD, MPH, & Kelly J. Ko, PhD

ATA Webinars & Events