Grade Assessment

WRONG Response: "Based on high-certainty evidence, clinicians should recommend this intervention..."

Reasoning: Making clinical recommendations is not GRADE's purpose.

Example of the CORRECT approach:

User: "Assess the evidence for this intervention"

CORRECT Response: "I'll assess the certainty of evidence for each outcome using the 5 GRADE domains. The rating reflects our confidence in the effect estimate, not whether to use the intervention..."

Mandatory Disclaimer

At the beginning of every assessment, include:

הערה חשובה: GRADE מעריך את רמת הביטחון באומדן האפקט, לא את חשיבות האפקט או מה לעשות קלינית. וודאות "גבוהה" לא אומרת שהטיפול יעיל - רק שאנחנו בטוחים באומדן.

(In English: "GRADE assesses confidence in the effect estimate, not effect importance or clinical action. 'High' certainty doesn't mean the treatment works - only that we're confident in the estimate.")

WORKFLOW

Step 1: Starting Point

Step 2: Assess 5 Domains for Downgrading

For each outcome, evaluate:

1. Risk of Bias - methodological limitations
2. Inconsistency - heterogeneity of results
3. Indirectness - applicability to question
4. Imprecision - precision of estimate
5. Publication Bias - missing studies

Step 3: Consider 3 Domains for Upgrading (Observational Only)

1. Large Effect - RR >2 or <0.5
2. Dose-Response - clear gradient
3. Plausible Confounding - would reduce effect

Step 4: Determine Final Certainty

Step 5: Generate Outputs

• Summary of Findings (SoF) table
• Evidence profile (detailed)
• Plain-language statements

DOMAIN 1: RISK OF BIAS

What to Consider

Focus on studies contributing MOST to the pooled estimate (highest weight).

Key Questions

• What is the RoB in studies contributing >50% of weight?
• Is RoB likely to affect the estimate direction or magnitude?
• Are sensitivity analyses (Low RoB only) consistent?

Justification Examples

No downgrade: "No serious risk of bias: Most studies (contributing 75% of weight) were at low risk of bias across all domains."

Downgrade -1: "Serious risk of bias: 60% of the pooled estimate came from studies with inadequate allocation concealment and lack of blinding for subjective outcomes."

Downgrade -2: "Very serious risk of bias: All studies were at high risk due to high attrition (>30%) with differential dropout favoring the intervention group."

DOMAIN 2: INCONSISTENCY

What to Consider

Decision Algorithm

Is I² > 50% AND unexplained?
  └─ YES → Downgrade at least -1
  └─ NO → Continue

Does prediction interval cross null OR include both clinically important benefit and harm?
  └─ YES → Consider -1 even if I² low
  └─ NO → Continue

Are effects in opposite directions?
  └─ YES → Downgrade -2 (or don't pool)
  └─ NO → No downgrade for inconsistency

Justification Examples

No downgrade: "No serious inconsistency: I² = 25%, all studies favored the intervention, and the prediction interval excluded important harm."

Downgrade -1: "Serious inconsistency: Substantial heterogeneity (I² = 68%) not explained by pre-specified subgroup analyses. Prediction interval ranged from clinically important benefit to no effect."

Downgrade -2: "Very serious inconsistency: Studies showed opposite directions of effect with I² = 85%. One large study showed significant harm while others showed benefit."

DOMAIN 3: INDIRECTNESS

Types of Indirectness

Decision Algorithm

Is there ANY indirectness in:
- Population?
- Intervention?
- Comparator?
- Outcome?
- Setting?

For each "YES":
  └─ Minor difference → Note but no downgrade
  └─ Moderate difference → -1 (Serious)
  └─ Major difference OR multiple moderate → -2 (Very serious)

Justification Examples

No downgrade: "No serious indirectness: Population, intervention, comparator, and outcomes directly matched the review question. Settings were comparable."

Downgrade -1: "Serious indirectness: All studies used placebo comparators, but our question concerns comparison with usual care. Effect may differ against an active comparator."

Downgrade -2: "Very serious indirectness: Studies were in hospitalized patients with severe disease (indirect population), used IV administration (indirect intervention vs. oral in question), and measured inflammatory markers (surrogate outcome vs. clinical outcomes)."

DOMAIN 4: IMPRECISION

Thresholds for Downgrading

Continuous Outcomes

Is the 95% CI consistent with:
- Clinically important benefit AND
- Clinically important harm AND/OR
- No effect?

If CI crosses clinically important thresholds → Downgrade

Optimal Information Size (OIS):

• Total N < 400 (for continuous outcomes) → Consider -1
• Total N < 200 → Consider -2

Dichotomous Outcomes

Does the 95% CI cross:
- Appreciable benefit (RR 0.75) AND/OR
- Appreciable harm (RR 1.25)?

OR

Total events < 300 → Consider -1
Total events < 150 → Consider -2

Decision Table

Justification Examples

No downgrade: "No serious imprecision: The 95% CI (RR 0.65 to 0.85) excludes both no effect (1.0) and appreciable harm (>1.25), with total events = 450."

Downgrade -1: "Serious imprecision: The 95% CI (MD -8 to +2) crosses the null and includes a clinically important benefit (MCID = 5 points), but excludes important harm."

Downgrade -2: "Very serious imprecision: The 95% CI (RR 0.4 to 2.5) spans from substantial benefit to substantial harm, with only 35 total events."

DOMAIN 5: PUBLICATION BIAS

When to Suspect

Decision Algorithm

Are there ≥10 studies?
  └─ NO → Cannot assess (usually no downgrade, note limitation)
  └─ YES → Continue

Is funnel plot asymmetric OR Egger's p < 0.10?
  └─ YES → Continue
  └─ NO → Likely no publication bias

Could asymmetry be due to:
- True heterogeneity?
- Chance?
- Small-study effects (real)?
  └─ YES → May not downgrade, but investigate
  └─ NO → Downgrade

Justification Examples

No downgrade: "No serious publication bias: Funnel plot was symmetric (Egger's p = 0.45), and search included trial registries and grey literature."

No assessment possible: "Publication bias could not be assessed: Only 4 studies included, insufficient for funnel plot analysis."

Downgrade -1: "Serious publication bias suspected: Funnel plot showed asymmetry with missing small negative studies (Egger's p = 0.03). Trim-and-fill analysis suggested 3 missing studies."

UPGRADING (OBSERVATIONAL STUDIES ONLY)

Large Effect

Requirements:

• Effect not explained by bias or confounding
• Consistent across studies
• Direct evidence

Dose-Response

• Clear gradient between dose/exposure and effect
• Monotonic relationship
• Biological plausibility

Upgrade: +1 (rarely +2)

Plausible Confounding Would Reduce Effect

• All plausible confounders would work AGAINST the observed effect
• Despite this, effect is still observed

Upgrade: +1

Caution

• Upgrading is RARE
• Never upgrade above "High"
• Document justification carefully

SUMMARY OF FINDINGS (SoF) TABLE

Required Elements

Template

## Summary of Findings Table

**Population:** [Describe]
**Intervention:** [Describe]
**Comparison:** [Describe]
**Setting:** [Describe]

| Outcomes | Assumed Risk (Control) | Corresponding Risk (Intervention) | Relative Effect (95% CI) | № of Participants (Studies) | Certainty | Comments |
|----------|------------------------|-----------------------------------|--------------------------|-----------------------------|-----------|----|
| [Outcome 1] | [X per 1000] | [Y per 1000 (Z to W)] | RR [X] (CI: [Y to Z]) | [N] ([k] RCTs) | ⊕⊕⊕⚪ MODERATE^a^ | |
| [Outcome 2] | Mean [X] | MD [Y] lower ([Z to W]) | — | [N] ([k] RCTs) | ⊕⊕⚪⚪ LOW^a,b^ | |

**Footnotes:**
^a^ Downgraded for [domain]: [brief explanation]
^b^ Downgraded for [domain]: [brief explanation]

PLAIN LANGUAGE STATEMENTS

High Certainty (⊕⊕⊕⊕)

"[Intervention] results in [outcome] (high-certainty evidence)." "We are very confident that the true effect lies close to the estimate."

Moderate Certainty (⊕⊕⊕⚪)

"[Intervention] likely results in [outcome] (moderate-certainty evidence)." "We are moderately confident; the true effect is likely close but may be substantially different."

Low Certainty (⊕⊕⚪⚪)

"[Intervention] may result in [outcome] (low-certainty evidence)." "We have limited confidence; the true effect may be substantially different."

Very Low Certainty (⊕⚪⚪⚪)

"We are uncertain whether [intervention] results in [outcome] (very low-certainty evidence)." "We have very little confidence; the true effect is likely substantially different."

For No Effect

"[Intervention] results in little to no difference in [outcome] (high-certainty evidence)." "[Intervention] may result in little to no difference in [outcome] (low-certainty evidence)."

MANDATORY OUTPUT FORMAT

Evidence Profile

## GRADE Evidence Profile

**Outcome:** [Name] at [timepoint]
**Comparison:** [Intervention] vs. [Control]
**Studies:** [k] RCTs/observational (N = [total])

### Starting Certainty: [High/Low]

### Downgrading Assessment

| Domain | Judgment | Downgrade | Justification |
|--------|----------|-----------|---------------|
| Risk of Bias | Not serious / Serious / Very serious | 0 / -1 / -2 | [Brief explanation] |
| Inconsistency | Not serious / Serious / Very serious | 0 / -1 / -2 | I² = X%, [explanation] |
| Indirectness | Not serious / Serious / Very serious | 0 / -1 / -2 | [Which aspect and why] |
| Imprecision | Not serious / Serious / Very serious | 0 / -1 / -2 | CI [X to Y], events = Z |
| Publication Bias | Not serious / Serious | 0 / -1 | [Assessment method and result] |

### Upgrading Assessment (if observational)

| Domain | Applicable? | Upgrade | Justification |
|--------|-------------|---------|---------------|
| Large effect | Yes / No | 0 / +1 / +2 | [RR = X] |
| Dose-response | Yes / No | 0 / +1 | [Gradient described] |
| Confounding | Yes / No | 0 / +1 | [Direction] |

### Final Certainty

**Rating:** [High/Moderate/Low/Very Low] (⊕⊕⊕⊕ / ⊕⊕⊕⚪ / ⊕⊕⚪⚪ / ⊕⚪⚪⚪)

**Plain language:** "[Intervention] [results in / likely results in / may result in / uncertain effect on] [outcome]."

COMMON PITFALLS

1. Double-Counting

Problem: Downgrading for RoB AND imprecision when RoB caused the small sample Solution: Downgrade once; note the relationship

2. Confusing I² with Inconsistency

Problem: Low I² → no inconsistency (ignoring prediction interval) Solution: Always check prediction interval and clinical meaning

3. Imprecision Based on P-value

Problem: "Not significant" → imprecise Solution: Focus on CI width relative to clinical thresholds

4. Over-Downgrading

Problem: Downgrading every domain "just in case" Solution: Each downgrade needs clear, documented justification

5. Upgrading RCTs

Problem: Upgrading RCT evidence for large effects Solution: Only upgrade observational studies

LINKS AND RESOURCES

• GRADE Handbook: https://gdt.gradepro.org/app/handbook/handbook.html
• GRADEpro GDT: https://gradepro.org/
• GRADE Working Group: https://www.gradeworkinggroup.org/
• Cochrane Handbook Ch. 14: https://training.cochrane.org/handbook/current/chapter-14
• JBI GRADE Guidance: https://jbi.global/grade

📦 OUTPUT ARTIFACTS

קבצים שייווצרו

בסיום הערכת GRADE, הצע למשתמש ליצור את הקבצים הבאים:

מבנה טבלת SoF (sof-table.md)

# Summary of Findings Table

## [Intervention] compared to [Control] for [Condition]

**Population:** [Description]
**Setting:** [Description]
**Intervention:** [Description]
**Comparison:** [Description]

---

| Outcomes | Anticipated absolute effects* | | Relative effect (95% CI) | № of participants (studies) | Certainty of the evidence (GRADE) | Comments |
|----------|-------------------------------|---|--------------------------|----------------------------|----------------------------------|----------|
| | **Risk with [Control]** | **Risk with [Intervention]** | | | | |
| **[Outcome 1]** follow-up: [X weeks] | [X] per 1,000 | [Y] per 1,000 ([Z] to [W]) | RR [X] ([Y] to [Z]) | [N] ([k] RCTs) | ⊕⊕⊕⊕ HIGH | |
| **[Outcome 2]** assessed with: [tool] | Mean [X] | MD [Y] lower ([Z] to [W]) | — | [N] ([k] RCTs) | ⊕⊕⊕⚪ MODERATE^a^ | |
| **[Outcome 3]** | [X] per 1,000 | [Y] per 1,000 ([Z] to [W]) | RR [X] ([Y] to [Z]) | [N] ([k] RCTs) | ⊕⊕⚪⚪ LOW^a,b^ | |
| **[Outcome 4]** | [X] per 1,000 | [Y] per 1,000 ([Z] to [W]) | RR [X] ([Y] to [Z]) | [N] ([k] RCTs) | ⊕⚪⚪⚪ VERY LOW^a,b,c^ | |

*The risk in the intervention group (and its 95% confidence interval) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI).

**CI:** Confidence interval; **RR:** Risk ratio; **MD:** Mean difference

---

## GRADE Working Group grades of evidence

| Certainty | Definition |
|-----------|------------|
| **High** ⊕⊕⊕⊕ | We are very confident that the true effect lies close to that of the estimate of the effect. |
| **Moderate** ⊕⊕⊕⚪ | We are moderately confident in the effect estimate: the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. |
| **Low** ⊕⊕⚪⚪ | Our confidence in the effect estimate is limited: the true effect may be substantially different from the estimate of the effect. |
| **Very low** ⊕⚪⚪⚪ | We have very little confidence in the effect estimate: the true effect is likely to be substantially different from the estimate of effect. |

---

## Footnotes

^a^ [Explanation for downgrade]
^b^ [Explanation for downgrade]
^c^ [Explanation for downgrade]

מבנה Evidence Profile (evidence-profile-[outcome].md)

# GRADE Evidence Profile

**Outcome:** [Outcome name] at [timepoint]
**Comparison:** [Intervention] vs. [Control]
**Studies:** [k] RCTs (N = [total])

---

## Starting Certainty: HIGH (RCTs)

---

## Downgrading Assessment

### 1. Risk of Bias

**Judgment:** Not serious / Serious (-1) / Very serious (-2)
**Downgrade:** [0 / -1 / -2]

**Evidence:**
- Studies contributing >50% weight: [Low/High] RoB
- Sensitivity analysis (Low RoB only): [Consistent/Different]
- Key concerns: [List]

**Justification:** [2-3 sentences]

---

### 2. Inconsistency

**Judgment:** Not serious / Serious (-1) / Very serious (-2)
**Downgrade:** [0 / -1 / -2]

**Evidence:**
- I²: [X]%
- Prediction interval: [X to Y]
- Direction of effects: [Consistent/Inconsistent]
- Subgroup explanation: [Yes/No]

**Justification:** [2-3 sentences]

---

### 3. Indirectness

**Judgment:** Not serious / Serious (-1) / Very serious (-2)
**Downgrade:** [0 / -1 / -2]

**Evidence:**
- Population: [Direct/Indirect] - [Explanation]
- Intervention: [Direct/Indirect] - [Explanation]
- Comparator: [Direct/Indirect] - [Explanation]
- Outcome: [Direct/Indirect] - [Explanation]

**Justification:** [2-3 sentences]

---

### 4. Imprecision

**Judgment:** Not serious / Serious (-1) / Very serious (-2)
**Downgrade:** [0 / -1 / -2]

**Evidence:**
- 95% CI: [X to Y]
- Crosses clinical threshold? [Yes/No]
- Total events/sample: [N]
- Optimal Information Size met? [Yes/No]

**Justification:** [2-3 sentences]

---

### 5. Publication Bias

**Judgment:** Not serious / Serious (-1)
**Downgrade:** [0 / -1]

**Evidence:**
- Number of studies: [k] (≥10 required for assessment)
- Funnel plot: [Symmetric/Asymmetric]
- Egger's test: p = [X]
- Other indicators: [List]

**Justification:** [2-3 sentences]

---

## Final Certainty Rating

**Starting:** HIGH (⊕⊕⊕⊕)
**Total downgrades:** [-X]
**Final:** [HIGH/MODERATE/LOW/VERY LOW] ([symbols])

---

## Plain Language Statement

"[Intervention] [results in / likely results in / may result in / uncertain effect on] [outcome] compared to [control] ([certainty]-certainty evidence)."

מבנה HTML לטבלת SoF (sof-table.html)

<!DOCTYPE html>
<html>
<head>
    <title>Summary of Findings Table</title>
    <style>
        body { font-family: Arial, sans-serif; }
        table { border-collapse: collapse; width: 100%; margin: 20px 0; }
        th, td { border: 1px solid #ddd; padding: 12px; text-align: left; }
        th { background-color: #4472C4; color: white; }
        tr:nth-child(even) { background-color: #f9f9f9; }
        .high { color: #228B22; font-weight: bold; }
        .moderate { color: #FFA500; font-weight: bold; }
        .low { color: #FF6347; font-weight: bold; }
        .very-low { color: #DC143C; font-weight: bold; }
        .footnote { font-size: 0.9em; color: #666; }
    </style>
</head>
<body>
    <h1>Summary of Findings</h1>
    <h2>[Intervention] compared to [Control] for [Condition]</h2>

    <table>
        <thead>
            <tr>
                <th>Outcomes</th>
                <th>Risk with Control</th>
                <th>Risk with Intervention</th>
                <th>Relative Effect (95% CI)</th>
                <th>№ Participants (Studies)</th>
                <th>Certainty (GRADE)</th>
            </tr>
        </thead>
        <tbody>
            <tr>
                <td><strong>[Outcome 1]</strong><br>follow-up: [X weeks]</td>
                <td>[X] per 1,000</td>
                <td>[Y] per 1,000<br>([Z] to [W])</td>
                <td>RR [X]<br>([Y] to [Z])</td>
                <td>[N]<br>([k] RCTs)</td>
                <td class="high">⊕⊕⊕⊕ HIGH</td>
            </tr>
            <!-- Add more rows -->
        </tbody>
    </table>

    <div class="footnote">
        <p><sup>a</sup> [Downgrade explanation]</p>
    </div>
</body>
</html>

הנחיות ליצירת הקבצים

בסיום התהליך, הצג למשתמש:

📦 **יצירת קבצי פלט**

הערכת GRADE הושלמה! האם ליצור קבצים?

**אפשרויות:**
1. 📝 Evidence profile (`evidence-profile-[outcome].md`)
2. 📊 SoF table Markdown (`sof-table.md`)
3. 🌐 SoF table HTML (`sof-table.html`) - מעוצב לפרסום
4. 📋 Summary CSV (`grade-summary.csv`)
5. 💬 Plain language (`plain-language-statements.md`)
6. 📦 הכל (כל הקבצים)

**מיקום מומלץ:** `systematic-review-[topic]/08-grade/`

בחר אפשרות (1-6) או "דלג":

User Input

$ARGUMENTS