Systematically verify documentation claims against codebase reality. Use when auditing docs, before releases, after refactors, or when documentation drift is suspected. Two-pass approach with pattern expansion ensures comprehensive detection of false claims, dead references, and gaps.
Systematically verify claims in documentation against the actual codebase using a two-pass approach.
Trigger phrases: "audit docs", "verify documentation", "check docs", "docs accurate", "documentation drift"
Low recall is worse than false positives — missed claims stay invisible and mislead users.
Two-pass process:
plans/, audits/)docs/audits/AUDIT_REPORT_YYYY-MM-DD.md| Type | Example | Verification |
|---|---|---|
file_ref | scripts/foo.py | File exists? |
config_default | "defaults to 'AI Radio'" | Check schema/code |
env_var | STATION_NAME | In .env.example + code? |
cli_command | --normalize flag | Script supports it? |
behavior | "runs every 2 minutes" | Check timers/code |
After Pass 1, analyze false claims and search for similar patterns:
Dead script found: diagnose_track_selection.py
→ Search: all script references → Found 8 more dead scripts
Wrong interval: "every 10 seconds"
→ Search: "every \d+ (seconds?|minutes?)" → Found 3 more
Wrong service name: ai-radio-break-gen.service
→ Search: service/timer names → Found naming inconsistencies
Always verify these patterns across all docs:
scripts/*.py referencesevery \d+ (seconds?|minutes?)ai-radio-*.service, *.timerRADIO_* environment variables--flag patterns in bash blocksGenerate docs/audits/AUDIT_REPORT_YYYY-MM-DD.md:
# Documentation Audit Report
Generated: YYYY-MM-DD | Commit: abc123
## Executive Summary
| Metric | Count |
|--------|-------|
| Documents scanned | 12 |
| Claims verified | ~180 |
| Verified TRUE | ~145 (81%) |
| **Verified FALSE** | **31 (17%)** |
## False Claims Requiring Fixes
### CONFIGURATION.md
| Line | Claim | Reality | Fix |
|------|-------|---------|-----|
| 135 | `claude-sonnet-4-5` | Actual: `claude-3-5-sonnet-latest` | Update |
## Pattern Summary
| Pattern | Count | Root Cause |
|---------|-------|------------|
| Dead scripts | 9 | Scripts deleted, docs not updated |
## Human Review Queue
- [ ] Line 436: behavior claim needs verification
This skill runs in Plan mode with a forked context to keep extraction artifacts separate from your main conversation:
| Include | Skip |
|---|---|
| User-facing README.md | docs/plans/ |
docs/ directory | docs/audits/ |
| Getting started guides | Design documents |
| Configuration docs | Historical artifacts |
| API documentation | Internal brainstorms |
| Trigger | Frequency |
|---|---|
| Before release | Every release |
| After refactor | As needed |
| User reports mismatch | Immediate |
| Periodic hygiene | Quarterly |
From a representative audit:
For comprehensive coverage of specific topics, see: