Meta Analysis Fundamentals

• Individual studies have limitations (small samples, specific populations)
• Combining studies increases statistical power
• Allows detection of smaller effects
• Improves generalizability of findings

Socratic Questions:

• "Why might a single study not give us the complete picture?"
• "What happens to our confidence when we have more data?"
• "Can you think of situations where combining studies might be problematic?"

2. Effect Sizes

Effect sizes quantify the magnitude of a treatment effect in a standardized way.

Teaching Approach:

1. First identify the outcome type (binary vs continuous)
2. Then consider whether scales are comparable
3. Guide user to appropriate effect size choice

3. Fixed vs Random Effects Models

Fixed-Effect Model:

• Assumes ONE true effect across all studies
• Differences between studies = sampling error only
• Use when: Studies are functionally identical

Random-Effects Model:

• Assumes true effects VARY between studies
• Accounts for both within-study and between-study variance
• Use when: Studies differ in populations, interventions, or settings
• Most common in medical research (DerSimonian-Laird method)

Decision Framework:

Are studies measuring the exact same thing 
in the exact same population?
    │
    ├── YES → Consider Fixed-Effect
    │
    └── NO → Use Random-Effects (default choice)

Assessment Questions

Use these to verify understanding:

1. Basic: "What is the main advantage of meta-analysis over a single study?"

   • Correct: Increased statistical power
   • Common misconception: "It's faster" or "It eliminates bias"

2. Intermediate: "When should you use a random-effects model?"

   • Correct: When true effects are expected to vary between studies
   • Common misconception: "When you have fewer studies"

3. Advanced: "An OR of 0.5 with 95% CI [0.3, 0.8] - is this statistically significant and clinically meaningful?"

   • Guide: CI doesn't cross 1 → significant; 50% odds reduction → likely meaningful

Common Misconceptions to Address

1. "Meta-analysis eliminates bias"

   • Reality: Can amplify biases if studies are biased
   • Teach: "Garbage in, garbage out"

2. "More studies = better meta-analysis"

   • Reality: Quality matters more than quantity
   • Teach: Risk of bias assessment is crucial

3. "The pooled effect is the 'true' effect"

   • Reality: It's an estimate with uncertainty
   • Teach: Always report confidence intervals

Example Dialogue

User: "I want to combine results from 5 studies on aspirin for heart disease. How do I start?"

Response Framework:

1. Acknowledge the goal
2. Ask about outcome type (heart attacks? deaths? continuous measure?)
3. Guide to appropriate effect size
4. Discuss model choice (likely random-effects given clinical heterogeneity)
5. Mention data requirements

References

See references/cochrane-handbook.md for detailed methodology. See references/effect-size-formulas.md for calculations.

Adaptation Guidelines

Glass (the teaching agent) MUST adapt this content to the learner:

1. Language Detection: Detect the user's language from their messages and respond naturally in that language
2. Cultural Context: Adapt examples to local healthcare systems and research contexts when relevant
3. Technical Terms: Maintain standard English terms (e.g., "forest plot", "effect size", "I²") but explain them in the user's language
4. Level Adaptation: Adjust complexity based on user's demonstrated knowledge level
5. Socratic Method: Ask guiding questions in the detected language to promote deep understanding
6. Local Examples: When possible, reference studies or guidelines familiar to the user's region

Example Adaptations:

• 🇧🇷 Portuguese: Use Brazilian health system examples (SUS, ANVISA guidelines)
• 🇪🇸 Spanish: Reference PAHO/OPS guidelines for Latin America
• 🇨🇳 Chinese: Include examples from Chinese medical literature

Related Skills

• forest-plot-creation - Visualizing meta-analysis results
• heterogeneity-analysis - Assessing between-study variation
• publication-bias-detection - Identifying missing studies