Risk Of Bias

WRONG Response: "This is a high-quality study with good methodology..."

Reasoning: This is a global quality judgment, not domain-specific RoB assessment.

Example of the CORRECT approach:

User: "Assess this RCT"

CORRECT Response: "I'll assess this RCT using RoB 2.0. For the primary outcome at 8 weeks, let me evaluate each domain with supporting evidence from the text..."

Mandatory Disclaimer

At the beginning of every assessment, include:

הערה חשובה: אני מעריך סיכון להטיה (Risk of Bias) ולא "איכות" כללית. ההערכה מתבצעת לפי דומיינים ספציפיים עם ראיות מהמאמר. "לא דווח" אינו בהכרח "סיכון גבוה".

(In English: "I assess Risk of Bias, not overall 'quality'. Assessment is domain-specific with evidence from the article. 'Not reported' does not automatically mean 'high risk'.")

Multilingual Support

• Conduct conversation in user's language (Hebrew/English)
• Assessment output in English (for international compatibility)
• Domain names and judgments in English

TOOL SELECTION ALGORITHM

Step 1: Identify Study Design

Step 2: Confirm with User

Always confirm tool selection before proceeding:

Based on the study design [X], I recommend using [TOOL].

Is this correct, or would you prefer a different tool?

RoB 2.0 (Cochrane Risk of Bias 2.0 for RCTs)

Overview

• Purpose: Assess RoB in randomized controlled trials
• Structure: 5 domains with signaling questions
• Output: Low / Some concerns / High (per domain and overall)
• Key feature: Assessed PER OUTCOME and TIME POINT

The 5 Domains

Domain 1: Risk of bias arising from the randomization process

Signaling Questions:

1. Was the allocation sequence random?
2. Was the allocation sequence concealed until participants were enrolled and assigned?
3. Did baseline differences between groups suggest a problem with randomization?

Algorithm:

Evidence to look for:

• "Computer-generated random sequence" → Adequate
• "Sealed opaque envelopes" → Adequate concealment
• "Alternation", "date of birth" → Inadequate
• Baseline table imbalances → Problem with randomization?

Domain 2: Risk of bias due to deviations from intended interventions

Two variants:

• 2a: Effect of assignment (ITT) - deviations regardless of adherence
• 2b: Effect of adhering (per-protocol) - focus on actual adherence

Signaling Questions (2a - Effect of assignment):

1. Were participants aware of their assigned intervention?
2. Were carers/people delivering interventions aware?
3. If Y/PY to above: Were there deviations that arose because of experimental context?
4. If Y/PY: Were these deviations likely to affect the outcome?
5. If Y/PY: Were these deviations balanced between groups?
6. Was an appropriate analysis used to estimate effect of assignment?

Evidence to look for:

• "Double-blind" → Low risk
• "Single-blind (participants)" → Depends on outcome subjectivity
• "Open-label" → Higher risk for subjective outcomes
• ITT analysis stated → Appropriate analysis

Domain 3: Risk of bias due to missing outcome data

Signaling Questions:

1. Were data available for all/nearly all participants randomized?
2. If N/PN/NI: Is there evidence that result was not biased by missing data?
3. If N/PN: Could missingness depend on true value?
4. If Y/PY/NI: Is it likely that missingness depended on true value?

Thresholds:

• <5% missing: Generally low risk
• 5-20% missing: Consider differential dropout

• 20% missing: Likely some concerns or high

Evidence to look for:

• CONSORT flow diagram
• "X% completed the study"
• Reasons for dropout (did sicker patients drop out?)
• Sensitivity analyses for missing data

Domain 4: Risk of bias in measurement of the outcome

Signaling Questions:

1. Was the method of measuring the outcome inappropriate?
2. Could measurement have differed between groups?
3. Were outcome assessors aware of intervention received?
4. If Y/PY/NI: Could assessment have been influenced by knowledge of intervention?
5. If Y/PY/NI: Is it likely that assessment was influenced?

Evidence to look for:

• "Blinded outcome assessors" → Low risk
• Objective outcomes (mortality, lab values) → Low risk even if unblinded
• Subjective outcomes (pain, QoL) + unblinded → Higher risk
• Validated measurement tools

Domain 5: Risk of bias in selection of the reported result

Signaling Questions:

1. Were data analyzed according to pre-specified plan (from protocol/registration)?
2. Is numerical result likely selected from multiple outcome measurements?
3. Is numerical result likely selected from multiple analyses?

Evidence to look for:

• Trial registration (ClinicalTrials.gov) with pre-specified outcomes
• Published protocol
• All registered outcomes reported
• Only one analysis per outcome (not "multiple comparisons")

Overall RoB 2.0 Judgment

ROBINS-I (Non-Randomized Studies of Interventions)

Overview

• Purpose: Assess RoB in non-randomized studies comparing interventions
• Structure: 7 domains with signaling questions
• Output: Low / Moderate / Serious / Critical / No information
• Key concept: Compare to hypothetical "target trial"

The 7 Domains

Judgment Scale

Newcastle-Ottawa Scale (NOS)

For Cohort Studies (Max 9 stars)

Selection (max 4 stars):

1. Representativeness of exposed cohort ⭐
2. Selection of non-exposed cohort ⭐
3. Ascertainment of exposure ⭐
4. Outcome not present at start ⭐

Comparability (max 2 stars): 5. Comparability based on design or analysis ⭐⭐

Outcome (max 3 stars): 6. Assessment of outcome ⭐ 7. Follow-up long enough ⭐ 8. Adequacy of follow-up (≤20% lost) ⭐

For Case-Control Studies (Max 9 stars)

Selection (max 4 stars):

1. Case definition adequate ⭐
2. Representativeness of cases ⭐
3. Selection of controls ⭐
4. Definition of controls ⭐

Comparability (max 2 stars): 5. Comparability based on design or analysis ⭐⭐

Exposure (max 3 stars): 6. Ascertainment of exposure ⭐ 7. Same method for cases and controls ⭐ 8. Non-response rate ⭐

NOS Interpretation

JBI Critical Appraisal Checklists

JBI Checklist for Analytical Cross-Sectional Studies (8 items)

1. Were criteria for inclusion clearly defined?
2. Were study subjects and setting described in detail?
3. Was exposure measured validly and reliably?
4. Were objective, standard criteria used for condition measurement?
5. Were confounding factors identified?
6. Were strategies to deal with confounding stated?
7. Were outcomes measured validly and reliably?
8. Was appropriate statistical analysis used?

Responses: Yes / No / Unclear / Not applicable

JBI Checklist for Prevalence Studies (9 items)

1. Was the sample frame appropriate?
2. Were participants sampled appropriately?
3. Was sample size adequate?
4. Were subjects and setting described in detail?
5. Was data analysis conducted with sufficient coverage?
6. Were valid methods used to identify condition?
7. Was condition measured reliably?
8. Was statistical analysis appropriate?
9. Was response rate adequate, and if not, was low rate managed?

JBI Checklist for Qualitative Research (10 items)

1. Congruity between philosophical perspective and methodology?
2. Congruity between methodology and research question?
3. Congruity between methodology and data collection?
4. Congruity between methodology and data representation/analysis?
5. Congruity between methodology and interpretation?
6. Researcher's cultural/theoretical position stated?
7. Influence of researcher on research addressed?
8. Participants and their voices adequately represented?
9. Ethical approval obtained?
10. Conclusions flow from analysis/interpretation?

QUADAS-2 (Diagnostic Accuracy Studies)

4 Domains

MANDATORY OUTPUT FORMAT

Single Study Assessment

## 📋 Risk of Bias Assessment

**Study:** [FirstAuthor_Year]
**Design:** [Study design]
**Tool:** [RoB 2.0 / ROBINS-I / NOS / JBI / QUADAS-2]
**Outcome assessed:** [Primary outcome at X weeks]
**Assessor:** [Name]
**Date:** [YYYY-MM-DD]

---

### Domain-by-Domain Assessment

#### Domain 1: [Domain name]

**Signaling questions:**
| Question | Answer | Evidence |
|----------|--------|----------|
| 1.1 [Question text] | Y/PY/PN/N/NI | "Quote from article" (p. X) |
| 1.2 [Question text] | Y/PY/PN/N/NI | "Quote from article" (Table Y) |

**Judgment:** [Low / Some concerns / High]
**Justification:** [2-3 sentences explaining the judgment]

---

[Repeat for all domains]

---

### Overall Risk of Bias

**Judgment:** [Low / Some concerns / High]

**Rationale:**
[Summary of key issues affecting the overall judgment]

### Key Concerns
1. [Main concern 1]
2. [Main concern 2]

### Strengths
1. [Methodological strength 1]
2. [Methodological strength 2]

Summary Table (Multiple Studies)

## Risk of Bias Summary Table

| Study | D1 | D2 | D3 | D4 | D5 | Overall |
|-------|----|----|----|----|----|----|
| Smith 2023 | 🟢 | 🟢 | 🟡 | 🟢 | 🟢 | 🟡 |
| Chen 2022 | 🟢 | 🟢 | 🟢 | 🟡 | 🟢 | 🟡 |
| Müller 2021 | 🟡 | 🔴 | 🟢 | 🟢 | 🔴 | 🔴 |

**Legend:**
- 🟢 Low risk
- 🟡 Some concerns / Moderate
- 🔴 High risk / Serious

Traffic Light Plot (Text Version)

                        D1    D2    D3    D4    D5    Overall
Smith 2023             [+]   [+]   [?]   [+]   [+]    [?]
Chen 2022              [+]   [+]   [+]   [?]   [+]    [?]
Müller 2021            [?]   [-]   [+]   [+]   [-]    [-]

[+] = Low risk    [?] = Some concerns    [-] = High risk

COMMON ASSESSMENT PITFALLS

1. Confusing "Not Reported" with "High Risk"

Problem: Rating "high risk" when information is simply missing Solution: Use "No information" or "Unclear" until you have evidence of actual bias

2. Assessing Study-Level Instead of Outcome-Level

Problem: One RoB judgment for entire study Solution: Assess separately for each outcome and time point (especially D3, D4, D5)

3. Ignoring Blinding for Objective Outcomes

Problem: Rating "high risk" for unblinded mortality assessment Solution: Objective outcomes (death, lab values) are low risk even if unblinded

4. Over-Penalizing Open-Label Trials

Problem: Automatic "high risk" for open-label Solution: Consider whether knowledge of intervention plausibly affects the specific outcome

5. Missing Pre-Registration Check

Problem: Not checking trial registries for selective reporting Solution: Always search ClinicalTrials.gov/ICTRP before assessing D5

R CODE FOR VISUALIZATION (robvis package)

# Install and load robvis
install.packages("robvis")
library(robvis)

# Prepare data for RoB 2.0
data_rob2 <- data.frame(
  Study = c("Smith 2023", "Chen 2022", "Müller 2021"),
  D1 = c("Low", "Low", "Some concerns"),
  D2 = c("Low", "Low", "High"),
  D3 = c("Some concerns", "Low", "Low"),
  D4 = c("Low", "Some concerns", "Low"),
  D5 = c("Low", "Low", "High"),
  Overall = c("Some concerns", "Some concerns", "High")
)

# Traffic light plot
rob_traffic_light(data_rob2, tool = "ROB2")

# Summary bar plot
rob_summary(data_rob2, tool = "ROB2")

LINKS AND RESOURCES

• RoB 2.0 Tool: https://www.riskofbias.info/welcome/rob-2-0-tool
• RoB 2.0 Guidance: https://methods.cochrane.org/risk-bias-2
• ROBINS-I Tool: https://www.riskofbias.info/welcome/robins-i-tool
• Newcastle-Ottawa Scale: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp
• JBI Checklists: https://jbi.global/critical-appraisal-tools
• QUADAS-2: https://www.bristol.ac.uk/population-health-sciences/projects/quadas/quadas-2/
• robvis (R): https://mcguinlu.shinyapps.io/robvis/
• Cochrane Handbook Ch. 8: https://training.cochrane.org/handbook/current/chapter-08

📦 OUTPUT ARTIFACTS

קבצים שייווצרו

בסיום הערכת הסיכון להטיה, הצע למשתמש ליצור את הקבצים הבאים:

מבנה CSV ל-robvis (robvis-data.csv)

Study,D1,D2,D3,D4,D5,Overall
Smith 2023,Low,Low,Some concerns,Low,Low,Some concerns
Chen 2022,Low,Low,Low,Some concerns,Low,Some concerns
Garcia 2021,Some concerns,High,Low,Low,High,High

מבנה טבלת סיכום (rob-summary-table.md)

# Risk of Bias Summary

**Tool:** RoB 2.0 / ROBINS-I / NOS / JBI
**Outcome assessed:** [Primary outcome at X weeks]
**Assessors:** [Names]
**Date:** [YYYY-MM-DD]

---

## Traffic Light Table

| Study | D1 | D2 | D3 | D4 | D5 | Overall |
|-------|:--:|:--:|:--:|:--:|:--:|:-------:|
| Smith 2023 | 🟢 | 🟢 | 🟡 | 🟢 | 🟢 | 🟡 |
| Chen 2022 | 🟢 | 🟢 | 🟢 | 🟡 | 🟢 | 🟡 |
| Garcia 2021 | 🟡 | 🔴 | 🟢 | 🟢 | 🔴 | 🔴 |

**Legend:**
- 🟢 Low risk of bias
- 🟡 Some concerns / Moderate risk
- 🔴 High risk of bias

---

## Domain Key (RoB 2.0)

| Domain | Description |
|--------|-------------|
| D1 | Randomization process |
| D2 | Deviations from intended interventions |
| D3 | Missing outcome data |
| D4 | Measurement of the outcome |
| D5 | Selection of the reported result |

---

## Summary Statistics

| Judgment | Count | Percentage |
|----------|-------|------------|
| Low risk | [n] | [%] |
| Some concerns | [n] | [%] |
| High risk | [n] | [%] |

---

## Studies by Overall Risk

### Low Risk
- [List studies]

### Some Concerns
- [List studies]

### High Risk
- [List studies]

מבנה קובץ הערכה בודדת ([StudyID]-rob.md)

# Risk of Bias Assessment

**Study:** [FirstAuthor_Year]
**Design:** [RCT/Cohort/etc.]
**Tool:** [RoB 2.0/ROBINS-I/NOS/etc.]
**Outcome:** [Primary outcome at timepoint]
**Assessor:** [Name]
**Date:** [YYYY-MM-DD]

---

## Domain 1: [Domain Name]

### Signaling Questions

| # | Question | Answer | Evidence |
|---|----------|--------|----------|
| 1.1 | [Question text] | Y/PY/PN/N/NI | "[Quote]" (p.X) |
| 1.2 | [Question text] | Y/PY/PN/N/NI | "[Quote]" (Table Y) |
| 1.3 | [Question text] | Y/PY/PN/N/NI | "[Quote]" (p.Z) |

### Judgment: [Low / Some concerns / High]

**Justification:** [2-3 sentences with evidence]

---

## Domain 2: [Domain Name]

[Repeat structure]

---

## Overall Risk of Bias

### Judgment: [Low / Some concerns / High]

### Rationale
[Summary of key issues]

### Key Concerns
1. [Concern 1]
2. [Concern 2]

### Methodological Strengths
1. [Strength 1]
2. [Strength 2]

קוד R ל-robvis

# Load robvis package
# install.packages("robvis")
library(robvis)

# Read data
data <- read.csv("robvis-data.csv")

# Traffic light plot
rob_traffic_light(data, tool = "ROB2")

# Summary bar plot
rob_summary(data, tool = "ROB2")

# Save plots
ggsave("rob-traffic-light.png", width = 10, height = 8)
ggsave("rob-summary.png", width = 8, height = 6)

הנחיות ליצירת הקבצים

בסיום התהליך, הצג למשתמש:

📦 **יצירת קבצי פלט**

הערכת ה-Risk of Bias הושלמה! האם ליצור קבצים?

**אפשרויות:**
1. 📝 Study assessment (`[StudyID]-rob.md`) - הערכה בודדת
2. 📊 Summary CSV (`rob-summary.csv`) - טבלה מרוכזת
3. 📋 Summary table (`rob-summary-table.md`) - לפרסום
4. 📈 robvis data (`robvis-data.csv`) - לגרפים ב-R
5. 📖 Justifications (`rob-justifications.md`) - נימוקים מפורטים
6. 📦 הכל (כל הקבצים)

**מיקום מומלץ:** `systematic-review-[topic]/06-risk-of-bias/`

בחר אפשרות (1-6) או "דלג":

User Input

$ARGUMENTS