Sagan Standard

Bayesian Foundation: If P(claim) = 0.001%, evidence must shift likelihood ratio 100,000:1 to reach 50% confidence.

When to Use

• Evaluating novel scientific claims: Proportion skepticism to claim's implausibility
• Due diligence: Require stronger verification for too-good-to-be-true opportunities
• Debugging: Mundane explanations need less proof than exotic ones
• Hiring: Extraordinary resume claims demand extraordinary verification
• Product claims: Revolutionary benefits require revolutionary evidence
• Strategy validation: Paradigm-shifting plans need stronger evidence than incremental ones

Execution Steps

1. Calibrate Prior Probability

• Ask: "How much existing evidence contradicts this claim?"
• Quantify: Is this 1-in-10 unusual? 1-in-1000? 1-in-million?
• Consider base rates: How often do claims like this turn out true?
• Assess track record: Do similar extraordinary claims usually pan out?

2. Define "Extraordinary Evidence"

Not just "more evidence" but:

• Independent replication: Others reach same conclusion independently
• Mechanism explanation: Plausible causal story, not just correlation
• Controls for bias: Pre-registration, blinding, adversarial review
• Convergent evidence: Multiple independent lines pointing same direction
• Ruled out alternatives: Mundane explanations actively eliminated

3. Apply Proportional Skepticism

Evidence Quality Ladder (lowest to highest):
1. Anecdote/testimonial
2. Uncontrolled observation
3. Correlation study
4. Controlled experiment
5. Replicated controlled experiments
6. Meta-analysis of quality studies
7. Converging evidence across multiple methodologies

Match required level to claim extraordinariness.

4. Avoid Common Misapplications

Not about dismissing new ideas: Novel ≠ extraordinary

• Novel: "This new catalyst improves yield 10%"
• Extraordinary: "This catalyst violates thermodynamics"

Not about impossible burdens: Must be achievable evidence

• Bad: "Show me video of every atom behaving as quantum mechanics predicts"
• Good: "Show me repeatable experimental results inconsistent with alternatives"

Not about doubling standards: Apply consistently

• Apply same rigor to claims you like vs. dislike
• Don't accept weak evidence for preferred conclusions

5. Implement Evidence Thresholds in Decision-Making

Hiring Example:

• "5 years experience" → Check LinkedIn (ordinary)
• "Increased revenue 300%" → Verify with references, see documentation (extraordinary)
• "Invented algorithm 10x better than state-of-art" → Require proof-of-concept, technical deep-dive, independent review (very extraordinary)

Investment Example:

• "Growing 20% YoY" → Review financials (ordinary)
• "Discovered new market 10x current TAM" → Deep diligence, market evidence, customer validation (extraordinary)
• "Technology defies physical limits" → Probably fraud, require independent scientific validation (extremely extraordinary)

6. Update Beliefs Proportionally

• When extraordinary evidence appears: Update strongly
• When ordinary evidence appears for extraordinary claim: Remain skeptical
• Track claim's prior probability evolution as evidence accumulates
• Don't freeze beliefs; allow extraordinary evidence to shift extraordinary claims

Real-World Examples

Cold Fusion (1989)

• Claim: Room-temperature fusion in simple apparatus (contradicts massive physics evidence)
• Evidence Provided: Single experiment, press conference announcement
• Sagan Standard: Requires independent replication, mechanism explanation, peer review
• Outcome: Couldn't be replicated; mundane explanation (experimental error) prevailed

Helicobacter Pylori Causes Ulcers (1982)

• Claim: Bacteria cause ulcers (contradicted prevailing "stress and acid" theory)
• Evidence Provided: Initially weak, but Marshall self-experimented, then rigorous studies
• Sagan Standard: Unusual but not physics-violating; required good evidence, not impossible bar
• Outcome: Replicated studies convinced community; Nobel Prize 2005

Theranos (2003-2018)

• Claim: Comprehensive blood tests from finger prick (contradicted known technical limits)
• Evidence Provided: Charismatic founder, impressive board, media hype, ZERO peer review
• Sagan Standard Failure: Investors/partners didn't demand extraordinary evidence
• Outcome: Fraud; technology didn't work

Quantum Mechanics (1900-1930)

• Claim: Particles behave as waves, observer affects outcome (deeply counterintuitive)
• Evidence Provided: Repeatable experiments, mathematical framework, convergent evidence
• Sagan Standard: Extraordinary claim met with extraordinary evidence (replicated experiments worldwide)
• Outcome: Accepted as foundation of modern physics

Common Traps

Trap 1: Impossible Standards

• Demanding literally impossible evidence ("prove a negative")
• Sagan Standard requires proportional evidence, not infinite evidence

Trap 2: Dismissing All Novelty

• Conflating "new" with "extraordinary"
• New ideas contradicting little existing evidence need good evidence, not extraordinary

Trap 3: Motivated Reasoning

• Applying high bar to claims you dislike, low bar to claims you like
• Must calibrate to actual prior probability, not preference

Trap 4: Static Priors

• Refusing to update even when extraordinary evidence appears
• Prior probability should shift as evidence accumulates

Trap 5: Confusing Extraordinary with Unlikely

• "I won the lottery" is unlikely but not extraordinary (happens to someone)
• "I always win the lottery" is extraordinary (contradicts probability)

Relationship to Other Principles

Occam's Razor: Prefer simpler explanations

• Sagan Standard: Require stronger proof for complex explanations
• Complementary: Simple explanation needs less proof

Hitchens's Razor: "What asserted without evidence can be dismissed without evidence"

• More extreme: Demands some evidence for any claim
• Sagan Standard: Demands evidence proportional to extraordinariness

Hume's Principle: "No testimony is sufficient to establish a miracle unless..."

• Direct inspiration for Sagan Standard
• Focused on miracles; Sagan generalized to all extraordinary claims

Cross-Domain Applications

Product Management: "This feature will 10x retention" → Show A/B tests, not gut feel

Sales: "Our ROI is 50x" → Require customer case studies, not vendor white paper

Debugging: "Cosmic ray bit flip" → Eliminate prosaic bugs first (memory leak, race condition)

Medicine: "This supplement cures cancer" → Phase III RCTs or it's noise

Academia: "This overturns established theory" → Replication, peer review, alternative explanations ruled out

Further Reading

• Sagan, Carl (1979). Broca's Brain (original popularization)
• Hume, David (1748). "Of Miracles" in An Enquiry Concerning Human Understanding (philosophical foundation)
• Truzzi, Marcello (1987). "On Pseudo-Skepticism" (discusses misapplication)
• Pearl, Judea (2018). The Book of Why (Bayesian reasoning and evidence)
• Yudkowsky, Eliezer (2007). "How Much Evidence Does It Take?" (LessWrong formalization)

Source Domain: Military Strategy, Ancient Wisdom & Hidden Gems (07) Pattern Type: Epistemological Principle / Evidence Evaluation Practitioner Value: 9/10 | Clarity: 10/10 | ROI: 9/10 | Novelty: 7/10 | Cross-Domain: 10/10 Total Score: 45/50