Measure the clinical, utilization, and financial impact of care management programs and population health interventions. Use when evaluating program outcomes, conducting pre/post analyses, comparing intervention cohorts, or reporting program ROI to stakeholders.
This skill evaluates the effectiveness of care management programs — disease management, care coordination, transitional care, remote patient monitoring, and behavioral health integration — by measuring outcomes against matched comparison groups. It applies quasi-experimental methods, difference-in-differences analysis, and propensity score matching to isolate program impact from secular trends.
When to Use
Evaluating whether a care management program achieved its intended outcomes
Conducting annual program effectiveness reviews for leadership or board reporting
Comparing intervention cohorts to matched controls for causal attribution
Quantifying cost avoidance and utilization reduction attributable to programs
Deciding whether to expand, modify, or sunset a care program
Required Inputs
Input
Description
Format
Program enrollment data
Related Skills
Enrollment dates, program type, engagement level
Program roster
Claims/encounter data
Pre- and post-enrollment claims for participants and comparison
Claims detail
Clinical outcomes
Lab values, vitals, functional assessments
Clinical data
Cost data
Allowed amounts, PMPM, total cost of care
Financial tables
Eligibility data
Continuous enrollment, LOB, demographics
Enrollment file
Program operational data
Touchpoints, interventions delivered, care plans
Program logs
Methodology
Step 1 — Define Program Cohort and Measurement Periods
Establish clear cohort boundaries:
Enrollment criteria: Minimum program engagement threshold (e.g., ≥ 2 care manager contacts)
Pre-period: 12 months before program enrollment (baseline measurement)
Post-period: 12 months after program enrollment (outcome measurement)
Wash-out period: Exclude first 30-60 days post-enrollment to allow intervention ramp-up
Continuous enrollment: Require continuous eligibility through pre and post periods
Exclusion criteria: Deceased during measurement, disenrolled, < minimum engagement
Step 2 — Construct Comparison Group
Build a matched comparison cohort using propensity score methods:
Matching variables: Age, sex, baseline RAF score, prior-year cost, prior-year utilization (IP admits, ED visits), chronic condition count, dual-eligible status
Gross savings: Total cost avoidance in intervention group vs. comparison
Program cost: Staffing, technology, overhead allocated to program
Net savings: Gross savings minus program cost
ROI: (Net savings / Program cost) × 100
Cost per quality-adjusted outcome: Program cost / number of members achieving target
Step 7 — Synthesize and Recommend
Produce an integrated assessment:
Summarize findings across all four outcome domains
Rate program effectiveness: highly effective, moderately effective, inconclusive, or ineffective
Identify sub-populations with strongest and weakest response
Recommend: expand, maintain, modify (specify modifications), or sunset
Define metrics and timeline for next evaluation cycle
Output Specification
Program Effectiveness Report:
├── Executive Summary (program description, key findings, recommendation)
├── Methodology Overview (study design, matching, measurement periods)
├── Cohort Description (enrollment, demographics, baseline equivalence)
├── Clinical Outcomes (pre/post, DID, significance, effect size)
├── Utilization Outcomes (admits, ED, readmissions — DID with CI)
├── Cost Outcomes (PMPM, total cost, cost avoidance)
├── Dose-Response Analysis (engagement tiers, response gradient)
├── ROI Calculation (gross savings, net savings, cost per outcome)
├── Sub-Group Analysis (by condition, acuity, demographics)
├── Limitations and Sensitivity Analyses
└── Recommendations and Next Steps
Analysis Framework
Effectiveness Rating Criteria
Rating
Clinical
Utilization
Cost
Highly effective
≥ 10% improvement, p < 0.01
≥ 15% reduction in IP/ED
ROI > 2:1
Moderately effective
5-10% improvement, p < 0.05
5-15% reduction
ROI 1:1-2:1
Inconclusive
< 5% change or p > 0.05
< 5% change
ROI < 1:1
Ineffective
No improvement or worsening
No reduction or increase
Net cost
Common Program Benchmarks
Disease management programs: 8-15% IP reduction, ROI 1.5:1-3:1
Transitional care: 20-30% readmission reduction in first 90 days
Remote patient monitoring: 15-25% ED reduction for enrolled CHF patients
Care coordination: 5-10% total cost reduction for high-risk members
Examples
Example 1 — CHF Care Management Program
Evaluate a CHF disease management program with 1,200 enrolled members over 18 months. Match 1:1 against 1,200 non-enrolled CHF patients. DID analysis shows: IP admissions −22% (p=0.003), ED visits −18% (p=0.01), 30-day readmissions −31% (p=0.001), total cost PMPM −$142 (p=0.008). Program cost $1.8M, gross savings $3.1M, net savings $1.3M, ROI 1.72:1. Recommend expansion to all CHF patients with EF < 40%.
Example 2 — Diabetes Prevention Program
Assess a DPP-modeled lifestyle intervention for 800 pre-diabetic members. After 12 months, 34% of participants achieved ≥ 5% weight loss vs. 12% in comparison (p < 0.001). Diabetes conversion rate 4.2% vs. 8.8% in comparison (p=0.02). Program cost per prevented diabetes case: $3,200 vs. estimated 5-year diabetes care cost of $48,000.
Pre-post analysis without a comparison group is insufficient for causal claims — label clearly as associational
Account for regression to the mean, particularly for programs targeting high-cost members
Report all outcomes including null or negative findings to avoid publication bias
Use intent-to-treat analysis as primary; per-protocol as sensitivity analysis
Validation Checklist
Comparison group is well-balanced on all matching variables (SMD < 0.1)
Pre-period trends are parallel between groups (parallel trends assumption)
Continuous enrollment requirement applied consistently to both groups
Program costs include all direct and allocated indirect costs
Statistical tests are appropriate for outcome data types
Sensitivity analyses confirm robustness of primary findings
Limitations are clearly documented
HIPAA Compliance
This skill processes Protected Health Information (PHI) for program evaluation, which constitutes health care operations under HIPAA (45 CFR §164.506). All outputs must comply with HIPAA Privacy and Security Rules. Apply minimum necessary standards to data access. De-identify comparison group outputs. Individual-level program data must be maintained in access-controlled systems. Report aggregate outcomes only in external communications, applying minimum cell-size suppression (n ≥ 11) for sub-group analyses.