All studies

Does this really work? Best practices for using real-world healthcare data to evaluate medical interventions 

Existing guidance for the comparative assessment of medical interventions using real-world data is vast and complex. A new step-by-step guide from Carelon Research helps decision makers and researchers in the creation and evaluation of such studies. 


Domain(s): All therapeutic areas and cost/quality of care assessments

Summary

Background

While high-quality randomized controlled trials (RCTs) provide the best opportunity to assess whether a new medication is effective, conducting RCTs is often infeasible, and results may have limited generalizability.

Increasingly, data collected during routine healthcare encounters, such as claims and electronic health records (EHR), are used to investigate treatment effectiveness and safety. Such studies require relevant and reliable data as well as careful design, analysis, and interpretation.

In RCTs, patients are randomized to treatment vs. control. In contrast, treatment decisions in real-world contexts are influenced by many factors (e.g., comorbidities, patient/provider preferences, costs), that can confound the causal effect of a treatment on the outcome of interest. Some of these factors may not even be observable by the research team.

Improvements in data quantity and quality as well as methodology over the last decade allow us to more confidently extract underlying, causal relationships from observational data, and to augment the high-quality evidence base available to health-care decision-makers. However, existing methodological guidance is vast and complex.

At Carelon Research, we created a concise step-by-step guide to causal inference to assist decision-makers and researchers in the creation and evaluation of such real-world studies.
 

Methods

A multidisciplinary team (epidemiology, biostatistics, health economics) reviewed the literature related to causal inference and observational data to identify key concepts. We developed a visual guide to causal study design to concisely and clearly illustrate each key concept and how they are related to one another.


Results

  • The guide to causal study design integrates essential concepts from the literature into eight steps, anchored into groupings that align with the typical study design process.
  • As scientific inquiry and study design are iterative processes, the various steps may be completed in a different order than shown, and steps may be revisited. We recommend that all steps are considered and addressed before proceeding with data analysis.
  • The steps include defining the research question and the parameter to be estimated; creating a graph that describes the intervention, outcome, and other related variables; identifying biases and design and analytic techniques to mitigate their effects; and applying techniques to examine the robustness of findings.
  • Please see the Figure for details of the steps.
  • The following table demonstrates how each step could be completed using a previously published study, which examined the impact of beta-blockers on mortality after a heart attack.[1]

Causal Inference Guide Figure for Actionable Insights newsletter.png               Causal table enlarged2.png

Key Takeaways
  • Many US healthcare stakeholders, including FDA, CMS, and private payers, increasingly value evidence from real-world studies, but these studies need high-quality data and nuanced, transparent study designs.
  • Our step-by-step guide provides a concise, visual summary of key study design steps to assist in the development and critical evaluation of comparative, observational healthcare analyses. For example, when reviewing published literature, this guide can be used as part of a "checklist" to assess study quality. It can also assist in planning inhouse evaluations of the impact of care management interventions.
  • At Carelon Research, we're committed to providing high-quality, actionable real-world evidence through data-driven strategies and collaborative partnerships. We welcome feedback on this step-by-step guide and related study ideas.

Publications
This research has been published at the following venues:
  • The 2022 Annual Meeting of The Professional Society for Health Economics and Outcomes Research (ISPOR) held May 15-18, 2022, in Washington, DC (as a poster and workshop)
  • The 2022 Annual Meeting of the International Society for Pharmacoepidemiology (ISPE) held August 24-28, 2022, in Copenhagen, Denmark (as a poster)

Carelon Research project team: Sarah Hoffman, Nilesh Gangan, Xiaoxue Chen*, Joseph L. Smith*, Arlene Tave, Yiling Yang*, Michael Grabner
*Carelon Research associates at the time the study was conducted. 

For more information on a specific study or to connect with the Actionable Insights Committee,
contact us at [email protected].

Sponsor: Carelon Research, Inc.

Dissemination and sharing of the Newsletter is limited to Elevance Health and its subsidiaries, and included findings and implications are for Elevance Health and its affiliates’ internal use only.

[1] Dondo TB, Hall M, West RM, Jernberg T, Lindahl B, Bueno H, Danchin N, Deanfield JE, Hemingway H, Fox KAA, Timmis AD, Gale CP. β-Blockers and Mortality After Acute Myocardial Infarction in Patients Without Heart Failure or Ventricular Dysfunction. J Am Coll Cardiol. 2017 Jun 6;69(22):2710-2720. doi: 10.1016/j.jacc.2017.03.578.

Ready to get started? Sign up now!

Lorem ipsum dolor sit amet

2-Col, Left Image

Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut.

  • Many addon features
  • Fully responsive & adaptive
  • SEO optimized
  • Attractive with a modern touch
  • Full Support

Data analysis uncovers variability in vaccine safety results across algorithms

Recent findings offer hope for children with multisystem inflammatory syndrome (MIS-C)

eGFR monitoring crucial in managing chronic kidney disease progression

Consectetur adipiscing elit...

Joanna C.

"Et harum quidem rerum facilis est et expedita distinctio!"

Stanley T.

"Nam libero tempore, cum soluta nobis est eligendi."

Danielle W.

"Temporibus autem quibusdam et aut officiis debitis!"

Teams at Carelon Research