Evidence Levels And Hierarchies

• A well-conducted cohort study may provide stronger evidence than a poorly conducted RCT
• RCTs are not always feasible or ethical (surgical procedures, rare diseases, long-term harms)
• The hierarchy addresses internal validity but not external validity or applicability
• Some questions are better answered by observational designs (harm, prognosis, diagnosis)
• N-of-1 trials and pragmatic designs do not fit neatly into the pyramid

Oxford Centre for Evidence-Based Medicine (CEBM) Levels

The Oxford CEBM system (2011 revision) is question-type specific, recognizing that the optimal study design depends on the clinical question.

Therapy / Prevention / Harm

Diagnosis

Prognosis

Grades of Recommendation (Oxford CEBM)

• A — Consistent Level 1 studies
• B — Consistent Level 2 or 3 studies, or extrapolation from Level 1
• C — Level 4 studies or extrapolation from Level 2 or 3
• D — Level 5 evidence or troublingly inconsistent/inconclusive studies at any level

"All-or-None" Studies (Level 1c)

A special category for dramatic effects observed without controlled trials:

• All: Before treatment, all patients died; after treatment, some survive (e.g., insulin for Type 1 diabetes)
• None: Before exposure, outcome never occurred; after exposure, it does (e.g., thalidomide and phocomelia)

These observations can provide Level 1c evidence despite being observational.

GRADE System (Grading of Recommendations, Assessment, Development, and Evaluation)

GRADE has become the dominant system for guideline development worldwide (WHO, Cochrane, NICE, AHA, ESC, and 100+ organizations).

Certainty of Evidence

GRADE uses four levels of certainty:

Starting Point

• RCTs start at High certainty
• Observational studies start at Low certainty

Downgrading Factors (each can reduce by 1-2 levels)

1. Risk of bias — Methodological limitations in component studies
2. Inconsistency — Unexplained heterogeneity in results across studies
3. Indirectness — Evidence does not directly address the clinical question (population, intervention, comparator, or outcome differences)
4. Imprecision — Wide confidence intervals, small sample size, few events
5. Publication bias — Suspicion that unfavorable results are missing

Upgrading Factors (observational studies only)

1. Large magnitude of effect — RR >2 or <0.5 with no plausible confounding (+1); RR >5 or <0.2 (+2)
2. Dose-response gradient — Clear dose-response relationship
3. Residual confounding — All plausible confounders would reduce the observed effect (strengthening causal inference)

GRADE Recommendation Strength

A strong recommendation can be based on low-certainty evidence (e.g., seatbelts — no RCTs, but strong recommendation based on mechanism and observational evidence).

Mapping Between Systems

This mapping is approximate. GRADE evaluates bodies of evidence holistically, while CEBM classifies individual studies.

Other Classification Systems

USPSTF Grades

• A: High certainty of substantial net benefit — recommend
• B: High certainty of moderate benefit or moderate certainty of substantial benefit — recommend
• C: Moderate certainty of small net benefit — offer selectively
• D: Moderate/high certainty of no net benefit or harms outweigh benefits — discourage
• I: Insufficient evidence to assess balance

Cochrane Risk of Bias

Not an evidence level system but a per-study quality assessment (covered in Bias Assessment skill).

Common Misunderstandings

1. "Level 1 evidence = good evidence" — Not necessarily. A poorly conducted systematic review of biased RCTs is still weak evidence despite being "Level 1a."
2. "Observational evidence is always weak" — Large, well-conducted observational studies with consistent results can provide moderate or even high certainty (GRADE upgrading).
3. "We need an RCT for everything" — Some questions cannot or should not be answered with RCTs (ethical constraints, rare outcomes, long time horizons).
4. "Expert opinion is worthless" — It occupies the lowest level because it is most susceptible to bias, but it is essential when higher-level evidence does not exist.
5. "Strong recommendation requires high-certainty evidence" — GRADE separates certainty of evidence from strength of recommendation. Context, values, and consequences all factor in.

Applying This Skill

When discussing evidence:

• Specify the evidence level/GRADE certainty for key claims
• Use the system appropriate to the context (GRADE for guideline evaluation, CEBM for individual study classification)
• Note limitations of the evidence, not just its level
• Distinguish between quality of evidence and strength of recommendation
• Flag when the available evidence does not match the question type (e.g., only RCTs for a prognostic question)