Diagnostic Meta Analysis

The 2x2 Table:

                    Disease Status
                    +           -
Test Result  +     TP          FP      → PPV = TP/(TP+FP)
             -     FN          TN      → NPV = TN/(FN+TN)
                   ↓           ↓
              Sens=TP/(TP+FN)  Spec=TN/(FP+TN)

Key Measures:

Socratic Questions:

• "Why can't we simply average sensitivities across studies?"
• "What happens to sensitivity when you lower the test threshold?"
• "Why might PPV vary dramatically across settings with the same test?"

2. The Threshold Effect

Critical Concept: Sensitivity and specificity are inversely related through the diagnostic threshold.

Visualization:

Sensitivity
    1.0 ┃●
        ┃ ●
        ┃  ●
        ┃   ●●
        ┃     ●●
        ┃       ●●●
    0.0 ┗━━━━━━━━━━━━━
        0.0         1.0
           1 - Specificity

Each ● = one study
Curve = SROC (Summary ROC)

Why It Matters:

• Different studies may use different thresholds
• Pooling sens/spec separately ignores this correlation
• Need methods that account for threshold variation

3. SROC Curves

What is SROC?

• Summary Receiver Operating Characteristic curve
• Shows trade-off between sensitivity and specificity
• Each study is a point; curve summarizes relationship

Key SROC Elements:

• Summary point: Best estimate of sens/spec
• Confidence region: Uncertainty around summary point
• Prediction region: Where future studies might fall
• AUC: Area under SROC curve (overall accuracy)

Interpretation of AUC:

4. Statistical Models

Bivariate Model (Reitsma et al. 2005):

• Models logit(sens) and logit(spec) jointly
• Accounts for correlation between measures
• Estimates between-study heterogeneity for each
• Produces summary sensitivity and specificity

HSROC Model (Rutter & Gatsonis 2001):

• Hierarchical Summary ROC
• Models threshold and accuracy parameters
• Equivalent to bivariate under certain conditions
• Better for exploring threshold effects

When to Use Each:

5. Implementation in R

Using mada package:

library(mada)

# Prepare data (2x2 table counts)
data <- data.frame(
  study = c("Study1", "Study2", "Study3", "Study4", "Study5"),
  TP = c(45, 52, 38, 61, 44),
  FP = c(8, 12, 5, 15, 9),
  FN = c(5, 8, 7, 9, 6),
  TN = c(92, 78, 100, 65, 91)
)

# Bivariate model
fit <- reitsma(data)
summary(fit)

# Summary operating point
summary(fit)$coefficients

# SROC plot
plot(fit, sroclwd = 2,
     main = "SROC Curve for Diagnostic Test X")
points(fpr(data), sens(data), pch = 19)

# Add confidence and prediction regions
plot(fit, sroclwd = 2, predict = TRUE)

Forest Plots for Sens/Spec:

# Paired forest plot
forest(fit, type = "sens", main = "Sensitivity")
forest(fit, type = "spec", main = "Specificity")

# Or use madad for separate plots
madad(data)

Using metafor for DTA:

library(metafor)

# Calculate logit sens and spec
data$yi_sens <- log(data$TP / data$FN)  # logit sensitivity
data$yi_spec <- log(data$TN / data$FP)  # logit specificity

# Variance (approximate)
data$vi_sens <- 1/data$TP + 1/data$FN
data$vi_spec <- 1/data$TN + 1/data$FP

# Bivariate model using rma.mv
# (More complex setup required - see metafor documentation)

6. Assessing Quality with QUADAS-2

QUADAS-2 Domains:

Key Signaling Questions:

Patient Selection:

• Was a consecutive or random sample enrolled?
• Was a case-control design avoided?
• Did the study avoid inappropriate exclusions?

Index Test:

• Were results interpreted without knowledge of reference?
• Was threshold pre-specified?

Reference Standard:

• Is the reference standard likely to correctly classify?
• Were results interpreted without knowledge of index test?

Flow and Timing:

• Was there appropriate interval between tests?
• Did all patients receive the same reference standard?
• Were all patients included in analysis?

7. Handling Heterogeneity

Sources of Heterogeneity in DTA:

1. Threshold variation (expected)
2. Population differences (disease spectrum)
3. Test execution differences
4. Reference standard variation
5. Study quality differences

Investigating Heterogeneity:

# Meta-regression in bivariate model
fit_cov <- reitsma(data, 
                   formula = cbind(tsens, tfpr) ~ covariate)
summary(fit_cov)

# Compare models
anova(fit, fit_cov)

Visual Assessment:

# ROC space plot - look for clustering
ROCellipse(data, pch = 19)

# If studies cluster in different regions,
# investigate sources of heterogeneity

8. Reporting DTA Meta-Analysis

Essential Elements (PRISMA-DTA):

1. Summary sensitivity and specificity with 95% CI
2. SROC curve with confidence/prediction regions
3. Heterogeneity assessment (visual and statistical)
4. QUADAS-2 quality assessment results
5. Subgroup/sensitivity analyses if applicable

Example Results Section:

"The bivariate meta-analysis of 12 studies (N=2,450 patients) yielded a summary sensitivity of 0.85 (95% CI: 0.79-0.90) and specificity of 0.92 (95% CI: 0.87-0.95). The positive likelihood ratio was 10.6 (95% CI: 6.8-16.5) and negative likelihood ratio was 0.16 (95% CI: 0.11-0.24). The area under the SROC curve was 0.94, indicating excellent overall accuracy. Substantial heterogeneity was observed for sensitivity (I²=78%) but not specificity (I²=32%). QUADAS-2 assessment identified high risk of bias in patient selection for 4 studies due to case-control design."

Assessment Questions

1. Basic: "Why can't we simply pool sensitivities using standard meta-analysis methods?"

   • Correct: Sensitivity and specificity are correlated; threshold effects create dependence

2. Intermediate: "What does the prediction region on an SROC curve represent?"

   • Correct: The region where we expect 95% of future studies to fall, accounting for heterogeneity

3. Advanced: "A diagnostic MA shows high sensitivity (0.95) but moderate specificity (0.70). How would you advise using this test clinically?"

   • Guide: Good for ruling out (high NPV when negative); positive results need confirmation; consider two-stage testing

Common Misconceptions

1. "Higher AUC always means better test"

   • Reality: Clinical utility depends on where on the curve you operate; context matters

2. "We should only include studies with the same threshold"

   • Reality: Bivariate/HSROC models handle threshold variation; excluding studies loses information

3. "Sensitivity and specificity are fixed properties of a test"

   • Reality: They vary with threshold, population, and disease spectrum

Example Dialogue

User: "I have 8 studies evaluating a rapid antigen test for COVID-19. How do I combine the results?"

Response Framework:

1. Acknowledge DTA-specific methods needed
2. Ask about data format (2x2 tables available?)
3. Discuss bivariate model approach
4. Guide through SROC curve creation
5. Emphasize QUADAS-2 quality assessment
6. Discuss clinical interpretation of results

References

• Reitsma JB et al. Bivariate analysis of sensitivity and specificity. J Clin Epidemiol 2005
• Macaskill P et al. Cochrane Handbook Chapter on DTA reviews
• Whiting PF et al. QUADAS-2: A revised tool. Ann Intern Med 2011
• Doebler P. mada: Meta-Analysis of Diagnostic Accuracy. R package

Adaptation Guidelines

Glass (the teaching agent) MUST adapt this content to the learner:

1. Language Detection: Detect the user's language from their messages and respond naturally in that language
2. Cultural Context: Adapt examples to local healthcare systems and diagnostic practices when relevant
3. Technical Terms: Maintain standard English terms (e.g., "SROC", "sensitivity", "QUADAS-2") but explain them in the user's language
4. Level Adaptation: Adjust complexity based on user's demonstrated knowledge level
5. Socratic Method: Ask guiding questions in the detected language to promote deep understanding
6. Local Examples: When possible, reference diagnostic tests and guidelines familiar to the user's region

Example Adaptations:

• 🇧🇷 Portuguese: Reference Brazilian diagnostic guidelines (CONITEC) and local test validations
• 🇪🇸 Spanish: Include examples from PAHO diagnostic recommendations
• 🇨🇳 Chinese: Reference Chinese diagnostic accuracy studies and local guidelines

Related Skills

• meta-analysis-fundamentals - Basic concepts prerequisite
• heterogeneity-analysis - Understanding between-study variation
• data-extraction - Extracting 2x2 tables from studies
• grade-assessment - Rating certainty of DTA evidence