Multi-Round Academic Paper Peer Review

Phase 0: Paper Intake & Model Assignment

Before reviewing, gather context and plan model allocation:

1. Read the full paper (all .tex files and .bib)
2. Identify target venue type:
   • Security conference (USENIX, CCS, NDSS, S&P) → emphasize threat model, real-world applicability
   • ML conference (NeurIPS, ICLR, ICML) → emphasize novelty, theoretical grounding, ablations
   • Journal (IEEE TDSC/TIFS, ACM TOPS) → emphasize completeness, reproducibility
   • Cross-domain (ML + Security) → apply BOTH sets of criteria
3. Detect paper domain: APT/intrusion detection, malware, vulnerability, privacy, etc.
4. Count pages and check against venue limit
5. Assign models to roles — see multi-model-strategy.md for recommended assignments. If only one model is available, see the single-model mitigation protocol in that file

Phase 1: Independent Reviews

Run 4 reviewer personas independently (parallel is preferred when sub-agents are available; sequential is acceptable as a fallback). Each reviewer MUST read the paper in full before reviewing. Each review MUST cite specific line numbers or section references.

Reviewer Panel & Model Assignment

For model assignment rationale, see multi-model-strategy.md. For detailed checklists per reviewer, see review-dimensions.md.

Review Output Format (per reviewer)

# Review — Reviewer [A/B/C/D]
Expertise: [role description]

## 1. Summary (3-5 sentences)
[What the paper does, core contribution, main finding]

## 2. Strengths
- **S1** [§X / Line Y]: [specific strength with evidence]
- **S2** [§X / Line Y]: ...
- **S3** [§X / Line Y]: ...

## 3. Weaknesses
- **W1** [§X / Line Y] ⬛ Critical: [specific issue, why it matters, suggested fix]
- **W2** [§X / Line Y] 🟧 Major: [specific issue, why it matters, suggested fix]
- **W3** [§X / Line Y] 🟨 Minor: [specific issue, suggested fix]

## 4. Questions for Authors
- **Q1**: [specific technical question requiring author response]
- **Q2**: ...

## 5. Missing References
[Papers the authors should cite and compare against]

## 6. Scores
| Dimension     | Score | Scale |
|---------------|-------|-------|
| Soundness     | X/4   | 1=poor, 2=fair, 3=good, 4=excellent |
| Presentation  | X/4   | 1=poor, 2=fair, 3=good, 4=excellent |
| Contribution  | X/4   | 1=poor, 2=fair, 3=good, 4=excellent |
| Overall       | X/10  | 1=reject ... 6=weak accept ... 10=award |
| Confidence    | X/5   | 1=guess ... 5=certain |

## 7. Verdict
[Strong Accept / Accept / Weak Accept / Borderline Accept / Borderline Reject / Reject]
One-sentence justification.

Verdict should be consistent with Overall score (recommended mapping: 8-10 = Strong Accept, 7 = Accept, 6 = Weak Accept, 5 = Borderline Accept, 4 = Borderline Reject, 1-3 = Reject).

Severity Definitions

Phase 2: Meta-Review Synthesis

After all 4 reviews, produce a meta-review as Area Chair:

# Meta-Review (Area Chair Synthesis)

## Consensus Strengths
[Points all/most reviewers agree are strong]

## Consensus Weaknesses
[Problems identified by 2+ reviewers — highest priority]

## Divergent Points
[Where reviewers disagree — needs discussion]

## Critical Path to Acceptance
Ordered list of must-fix items (from Critical → Major):
1. [Highest priority issue — which reviewer(s) raised it]
2. ...
3. ...

## Score Summary
| Reviewer | Soundness | Presentation | Contribution | Overall | Confidence |
|----------|-----------|--------------|--------------|---------|------------|
| A        |           |              |              |         |            |
| B        |           |              |              |         |            |
| C        |           |              |              |         |            |
| D        |           |              |              |         |            |
| **Mean** |           |              |              |         |            |

## Preliminary Decision
[Strong Accept / Accept / Weak Accept / Borderline Accept / Borderline Reject / Revise & Resubmit / Reject]

Phase 3: Author Response Guidance

For each weakness and question, draft a rebuttal strategy:

# Rebuttal Plan

## W1 [Reviewer X]: [issue title]
- **Response type**: [New experiment / Clarification / Rewrite / Acknowledge limitation]
- **Action**: [Specific modification to make]
- **Evidence**: [What to add — new table, figure, paragraph, citation]
- **Effort**: [Low / Medium / High]

## Q1 [Reviewer X]: [question]
- **Answer**: [Draft answer]
- **Revision needed**: [Yes — describe change / No — clarification only]

Prioritization rule: Address ALL Critical items first, then Major items that 2+ reviewers raised, then remaining Major items, then Minor items if space permits.

Phase 4: Revision Execution

For each planned revision:

1. Before editing: Re-read the relevant section in full
2. Make the change: Edit the .tex file with precise modifications
3. Track changes: Note what was modified (section, line range, nature of change)
4. Verify consistency: Check that the change doesn't contradict other sections (especially Abstract ↔ Results, Introduction claims ↔ Experiments)
5. Cross-check numbers: Any new numbers in text must match their source tables

Revision Log Format

# Revision Log

| # | Addresses | Section | Change Description | Lines |
|---|-----------|---------|-------------------|-------|
| 1 | W1 (Rev A) | §5.2 | Added statistical significance test results | 450-465 |
| 2 | Q2 (Rev B) | §4.1 | Clarified edge weight computation formula | 280-295 |
| ...| | | | |

Phase 5: Re-Review (Cross-Model Verification)

After revisions, re-review with cross-model verification — the model that re-reviews should differ from the model that wrote the original review when possible. This prevents the "self-satisfied rebuttal" problem where a model accepts its own fixes too easily.

For each reviewer role:

1. Check each addressed weakness: Was it adequately resolved?
2. Check for new issues: Did the revision introduce problems?
3. Update scores: Adjust scores up or down with justification
4. Final verdict: Accept / Still needs revision / Reject

Re-review output is a shorter version of the Phase 1 format, focusing on delta assessment.

Cross-Model Re-Review Matrix

Phase 6: Final Polish

After all reviews pass, hand off to the paper-submission-check skill (if available) for formatting, AI-style removal, citation, and LaTeX quality checks. This skill focuses on technical depth; paper-submission-check focuses on surface quality. Both are needed before submission.

Domain-Specific Review Modules

When the paper falls into a specific domain, load additional review criteria:

Scoring Rubric Reference

Overall Rating (1-10, NeurIPS scale)

Sub-Scores (1-4)

Additional Resources

• Multi-model strategy and single-model fallback: multi-model-strategy.md
• Detailed reviewer role definitions and persona prompts: reviewer-profiles.md
• Complete review dimension checklists per reviewer: review-dimensions.md
• ML-for-security 10 pitfalls audit guide: ml-security-pitfalls.md
• Example review reports (accept and reject cases): review-examples.md