Ar Ref Checker

Process

Step 1: Extract All References

Parse the paper to extract every reference from the References / Bibliography section. For each reference, extract:

• Author name(s)
• Year of publication
• Title of the work
• Venue (journal, publisher, report issuer)
• URL (if provided)

Also scan the body text for in-text citations (e.g., "Smith, 2023", "Smith & Jones, 2022") and ensure each has a corresponding entry in the reference list.

Step 2: Verify Each Reference

For each extracted reference, run a WebSearch to verify:

Search query: "{first_author_surname}" "{title}" {year}

Check:

1. Existence: Does this paper/report/article actually exist?
2. Author match: Does the author name match? Watch for:
   • Swapped first/last names
   • Misspelled author names
   • Hallucinated authors (real paper, wrong author attributed)
   • Combined authors from different papers
3. Title match: Is the title correct? Watch for:
   • Paraphrased titles (real paper, but title reworded)
   • Fabricated titles (no paper with this title exists)
   • Partial titles (title truncated or merged with another)
4. Year match: Is the publication year correct?
5. Venue match: Is the journal/publisher correct?

Step 3: Classify Each Reference

Assign a verification status to each reference:

• verified: the reference exists with matching author, title, and year
• corrected: the reference exists but had minor errors (e.g., misspelled name, wrong year) — include the correction
• mismatch: the reference partially matches a real work but has significant errors (wrong author for the title, or wrong title for the author)
• unverified: could not find evidence that this reference exists — may be hallucinated
• orphan: cited in-text but missing from the reference list

Step 4: Suggest Corrections

For mismatch and corrected references, provide:

• The correct author name(s)
• The correct title
• The correct year
• The correct venue
• A URL to the verified source

Step 5: Write Reference Check Report

Write {workspace}/reviews/ref_check_report.json:

{
  "total_references": 0,
  "verified": 0,
  "corrected": 0,
  "mismatch": 0,
  "unverified": 0,
  "orphan_citations": 0,
  "references": [
    {
      "original_author": "string",
      "original_title": "string",
      "original_year": 2024,
      "status": "verified|corrected|mismatch|unverified",
      "correct_author": "string or null",
      "correct_title": "string or null",
      "correct_year": "number or null",
      "correct_venue": "string or null",
      "verified_url": "string or null",
      "issue": "string or null"
    }
  ],
  "orphan_citations": [
    {
      "in_text_citation": "string",
      "location": "string"
    }
  ]
}

Step 6: Write Reference Check Summary

Write {workspace}/reviews/ref_check_summary.md:

A human-readable summary that lists:

• How many references were verified, corrected, mismatched, and unverified
• Each problematic reference with the specific issue and suggested correction
• Any orphan citations (in-text but no reference entry)

Quality Rules

• Every reference must be checked — do not skip any
• A mismatch or unverified reference is a blocker for the review stage — it must be corrected or removed
• Do not verify by guessing — only mark as verified if the WebSearch confirms the reference exists
• Be precise about what is wrong: "wrong author" is more useful than "mismatch"
• Use British English in all output text