Structured framework for writing peer review reports and paper critiques
Writing constructive peer reviews is a core academic skill. This framework provides a systematic approach to evaluating manuscripts — from initial read-through to the final referee report. It covers what reviewers should assess, how to structure feedback, and how to calibrate between different review outcomes (accept, revise, reject). Applicable to conference papers, journal articles, and internal lab reviews.
Read only these elements:
After Pass 1, answer:
□ What is the main claim?
□ What type of contribution? (empirical, theoretical, system, survey)
□ Is it within the venue's scope?
□ Does the abstract accurately represent the content?
□ Initial impression: novel or incremental?
Read the full paper. Annotate as you go:
Annotation symbols:
? = I don't understand this
! = This is interesting / strong point
X = I disagree / see a problem
→ = This needs more evidence or justification
≈ = This is similar to [existing work] — check novelty
Focus on:
For papers you're seriously evaluating:
## Summary (3-5 sentences)
[Describe what the paper does, the approach, and the main finding.
Demonstrate that you understood the paper.]
## Strengths (bulleted list)
- S1: [Specific strength with evidence from the paper]
- S2: [Another strength]
- S3: [Another strength]
## Weaknesses (bulleted list, ordered by severity)
- W1 (Major): [Specific weakness + why it matters + suggestion to fix]
- W2 (Major): [Another major weakness]
- W3 (Minor): [A less critical issue]
- W4 (Minor): [Another minor issue]
## Questions for Authors
- Q1: [Something you'd like clarified]
- Q2: [A concern that the authors might be able to address]
## Detailed Comments
[Page/line-specific comments, typos, suggestions]
## Overall Assessment
Recommendation: [Strong Accept / Accept / Weak Accept / Borderline /
Weak Reject / Reject / Strong Reject]
Confidence: [High / Medium / Low]
| Dimension | Questions to Ask | Weight |
|---|---|---|
| Novelty | Is the idea new? Is the contribution beyond incremental? | High |
| Significance | Would this matter to the community? Does it advance the field? | High |
| Soundness | Are the methods correct? Are conclusions supported? | High |
| Clarity | Is it well-written? Can it be understood and reproduced? | Medium |
| Completeness | Are related works covered? Are experiments thorough? | Medium |
| Reproducibility | Could someone replicate this? Code/data available? | Medium |
Strong Accept: Significant contribution, technically sound, well-written.
Would be a highlight of the venue.
Accept: Solid contribution with minor issues. Advances the field.
Worth publishing as-is or with minor revisions.
Weak Accept: Has merit but notable weaknesses. Contribution is real but modest.
Borderline for this venue; would be accepted at a less selective venue.
Borderline: Equal arguments for and against. Significant weaknesses offset
by some novelty. Depends on other reviews.
Weak Reject: Interesting direction but fundamental issues not addressed.
Major revisions needed that likely require a new submission cycle.
Reject: Significant problems in novelty, soundness, or relevance.
Not suitable for this venue even with revisions.
Strong Reject: Fundamental flaws. Clearly below threshold.
| Pitfall | Better Approach |
|---|---|
| "The writing needs improvement" (vague) | Give 2-3 specific examples with suggested fixes |
| Rejecting for not solving YOUR problem | Evaluate the paper on its own stated goals |
| Demanding impossible experiments | Suggest feasible improvements within scope |
| Ignoring supplementary material | Check appendix — authors may have addressed your concern |
| Being harsh without being constructive | Every weakness should include a suggestion for improvement |
| Reviewing too quickly | Block dedicated time; a rushed review harms both authors and science |
| Citing only your own work as "missing" | Only cite if genuinely relevant, not self-promotion |
During review, check:
□ Are human subjects involved? Was IRB/ethics approval obtained?
□ Are there potential harms from the technology described?
□ Is the data collection ethical? (Consent, privacy, bias)
□ Are dual-use concerns addressed? (Misuse potential)
□ Are limitations and societal implications discussed?