Rigorously audit scientific papers from arXiv or PDF. Runs 5-agent pipeline with adversarial review, mathematical verification, and code execution. Use when asked to review, audit, or analyze an academic paper.
name audit-paper description Rigorously audit scientific papers from arXiv or PDF. Runs 5-agent pipeline with adversarial review, mathematical verification, and code execution. Use when asked to review, audit, or analyze an academic paper. Paper Auditing Pipeline Overview You are orchestrating a rigorous 5-agent pipeline to audit a scientific/technical paper. Your goal is to produce a comprehensive, adversarial review that catches issues before publication. Architecture ┌─────────────────────────────────────────────────────────────┐ │ "Audit this paper" │ │ (YOU - Main Orchestrator) │ └──────────────────────────┬──────────────────────────────────┘ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Agent A: Deconstructor │ │ Input: arXiv ID or PDF path │ │ Output: Structured claim extraction (JSON) │ └──────────────────────────┬──────────────────────────────────┘ │ ┌─────────────────┼─────────────────┐ ▼ ▼ ▼ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Agent B │ │ Agent C │ │ Agent D │ │ Formalist │ │ SKEPTIC │ │ Verifier │ │ Math Audit │ │ ADVERSARY │ │ Code Exec │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │ │ └──────────────────┼──────────────────┘ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Agent E: Editor-in-Chief │ │ Synthesizes all reports → Final Decision Memo │ └─────────────────────────────────────────────────────────────┘ Input Parsing Parse the user's input to determine the paper source: arXiv ID formats: arXiv:2512.15605 2512.15605 Local PDF: ./path/to/paper.pdf /absolute/path/to/paper.pdf def parse_input ( arg ):
arxiv_patterns = [ r'arXiv:(\d{4}.\d{4,5})' , r'^(\d{4}.\d{4,5})$' , r'arxiv.org/abs/(\d{4}.\d{4,5})' ] for pattern in arxiv_patterns: match = re.search(pattern, arg) if match : return { 'type' : 'arxiv' , 'id' : match .group( 1 )}
if arg.endswith( '.pdf' ): return { 'type' : 'pdf' , 'path' : arg} raise ValueError( f"Cannot parse input: {arg} " ) Execution Flow Phase 1: Paper Acquisition For arXiv papers:
mkdir -p output/{paper_id}
Use WebFetch to get the paper from arXiv: Abstract page: https://arxiv.org/abs/{paper_id} PDF: https://arxiv.org/pdf/{paper_id}.pdf For local PDFs: Verify file exists Extract paper ID from filename or generate one Copy to output directory Phase 2: Agent A - Deconstruction Run Agent A to extract structured claims: Invoke skill: paper-audit:agent-a-deconstructor
Input: The paper content (PDF text or arXiv data) Output: output/{paper_id}/deconstruction.json Agent A deliverables: deconstruction.json with all claim categories Identified: theoretical_claims, empirical_claims, comparative_claims, novelty_claims Reproducibility artifacts catalogued Phase 3: Parallel Agent Execution (B, C, D) IMPORTANT: Agents B, C, and D can run in PARALLEL since they all depend only on Agent A's output. Launch these agents concurrently using the Task tool: Agent B: Formalist (Math Audit) Invoke skill: paper-audit:agent-b-formalist Condition: Only if deconstruction.json has non-empty theoretical_claims Output: output/{paper_id}/math_audit.md Agent C: Skeptic (Adversarial Review) Invoke skill: paper-audit:agent-c-skeptic CRITICAL: This is the core adversarial agent - ensure maximum skepticism Output:
title} arXiv: {paper id} | Date: {date} | Decision: {ACCEPT/MAJOR REVISION/REJECT}
| Priority | File | Description |
|---|---|---|
| 1️⃣ | ||
decision_memo.md | ||
| START HERE |
adversarial_review.md
| Critical issues and adversarial questions |
| 3️⃣ |
math_audit.md
| Mathematical rigor analysis |decision_memo.md
adversarial_review.md
math_audit.md
contradicting_papers.md
FULL_RESEARCH_PROPOSAL.md
literature_gaps.md
exploration_notebook.ipynb
deconstruction.json
verification/
main.py
results.json
plots/
Rebuttal Difficulty: {difficulty}/10 Phase 6: Final Report Display the final summary to the user and direct them to README.md : ═══════════════════════════════════════════════════════════════ AUDIT COMPLETE: {paper_id} ═══════════════════════════════════════════════════════════════ Paper: {title} Authors: {authors}
DECISION: {ACCEPT/MAJOR REVISION/REJECT}
Score Breakdown: ├─ Agent B (Math): {score}/10 (weight: 30%) ├─ Agent C (Skeptic): {score}/10 (weight: 40%) └─ Agent D (Verifier): {score}/10 (weight: 30%) ──────────────────────────────────── FINAL SCORE: {weighted_total}/10
Agent C Verdict: {SUSPICIOUS/QUESTIONABLE/DEFENSIBLE/ROBUST} Rebuttal Difficulty: {score}/10
Verification Status: {X}/{Y} claims verified
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 📂 START HERE: output/{paper_id}/README.md
The README provides a guided reading order for all generated files based on your goals (quick summary, deep dive, research). ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ═══════════════════════════════════════════════════════════════ IMPORTANT: Always end by pointing users to README.md as their entry point. Progress Reporting As orchestrator, report progress at each phase: [paper-audit] Starting audit of {paper_id}...
[Phase 1] Fetching paper... ✓ Paper downloaded: {title} ✓ Output directory: output/{paper_id}/
[Phase 2] Agent A: Deconstructing paper... ✓ Found {N} theoretical claims ✓ Found {N} empirical claims ✓ Found {N} comparative claims ✓ Found {N} novelty claims ✓ Saved: deconstruction.json
[Phase 3] Running parallel agents... [Agent B] Auditing mathematical rigor... [Agent C] Launching adversarial review... [Agent C] Searching for contradicting papers... [Agent D] Generating verification code... [Agent D] Executing verification...
✓ Agent B complete: {SOUND/QUESTIONABLE/N/A} ✓ Agent C complete: {verdict}, {N} questions generated ✓ Agent D complete: {X}/{Y} claims verified
[Phase 4] Agent E: Synthesizing reports... ✓ Decision memo generated ✓ Research proposal generated ✓ Exploration notebook generated
[Phase 5] Creating README index... ✓ README.md generated with reading guide
[Phase 6] Audit complete! 📂 Start here: output/{paper_id}/README.md Error Handling Paper Not Found ERROR: Could not fetch paper {paper_id}