Pr Learning

Usage Modes

Single PR Mode

/pr-learning 12345
/pr-learning https://github.com/owner/repo/pull/12345

Analyzes one merged PR and appends learnings to both documents.

Batch Mode

/pr-learning last 5
/pr-learning --since 2026-01-01

Processes multiple recent PRs. Useful for catching up on learnings.

No Argument Mode

If invoked with no arguments, prompt the user:

What PR would you like to extract learnings from?

• A PR number or URL (e.g., /pr-learning 12345)
• Recent PRs (e.g., /pr-learning last 5)

Output Files

Every run updates two living documents:

1. Project-Specific Learnings

Location: ./learnings/pr-learnings.md (in the repository)

Contains learnings specific to this codebase:

• Project conventions and patterns
• Module-specific gotchas
• Team preferences and standards
• Architecture decisions

2. General Learnings

Location: ~/learnings/general-learnings.md (machine-wide)

Contains transferable knowledge:

• Language idioms and best practices
• Common review feedback patterns
• Security and performance principles
• Testing strategies

Workflow

Step 1: Validate PR State

# Get PR state
gh pr view <PR_NUMBER> --json state,merged,mergedAt,author

# Reject if not merged

If PR is not merged, respond:

This PR hasn't been merged yet. Learning extraction works best on completed PRs where we can see the full review cycle. Would you like me to do a standard PR review instead?

Step 2: Gather PR Lifecycle Data

Spawn 3 sub-agents in parallel to collect:

Agent 1: Initial State

# Get first commit(s) in PR
gh pr view <PR_NUMBER> --json commits

# Get the diff at first commit
git show <first_commit_sha> --stat
git show <first_commit_sha> --no-stat

Agent 2: Review Comments & Feedback

# Get all review comments
gh pr view <PR_NUMBER> --json reviews,comments

# Get review threads with context
gh api repos/{owner}/{repo}/pulls/{pr_number}/comments

Agent 3: Final State

# Get final merged diff
gh pr diff <PR_NUMBER>

# Get commit messages showing evolution
gh pr view <PR_NUMBER> --json commits --jq '.commits[].messageHeadline'

Step 3: Analyze the Delta

Compare initial submission vs final merge to identify:

1. Code Changes After Review

   • What code was added/removed/modified after review feedback?
   • Which review comments led to code changes?

2. Pattern Corrections

   • Where did reviewers suggest different patterns?
   • What conventions were enforced?

3. Blind Spots Exposed

   • What did the author miss that reviewers caught?
   • Security issues, edge cases, performance concerns?

4. Knowledge Gaps

   • What did the author not know about the codebase?
   • What framework features were suggested?

Step 4: Classify Learnings

For each learning, determine:

Step 5: Deduplicate Against Existing Learnings

Before adding a new learning, scan both documents for semantic duplicates:

# Pseudo-logic for deduplication
for existing_learning in document:
    if semantic_similarity(new_learning, existing_learning) > 0.8:
        # Enhance existing instead of duplicating
        enhance_with_new_evidence(existing_learning, new_learning)
        return
# No match - add as new entry
append_learning(document, new_learning)

Deduplication signals:

• Same code pattern discussed
• Same file/module referenced
• Same reviewer feedback theme
• Same fix applied

If a near-duplicate exists, enhance the existing entry with:

• Additional evidence from the new PR
• Updated "frequency" count
• Cross-reference to the new PR

Step 6: Write Learnings

Project-specific document (./learnings/pr-learnings.md):

# PR Learnings - [Project Name]

> Auto-generated knowledge base from merged PR reviews.
> Last updated: [timestamp]

## Key Patterns (Auto-consolidated)

[Recurring themes extracted from 10+ PRs appear here]

---

## Recent Learnings

### [YYYY-MM-DD] PR #123: [PR Title]

**Author:** @username | **Reviewers:** @reviewer1, @reviewer2

#### Learning: [Concise title]

**Category:** `pattern` | **Severity:** `important`

**What happened:**
[Brief description of what the author originally submitted]

**What changed:**
[What reviewers requested and why]

**The lesson:**
[Actionable takeaway in imperative form]

**Evidence:**
- Review comment by @reviewer: "[quote]"
- Changed: `src/module.py:42-58`

**Related PRs:** #100, #89 (similar feedback)

---

General learnings document (~/learnings/general-learnings.md):

# General Engineering Learnings

> Transferable knowledge extracted from PR reviews across all projects.
> Last updated: [timestamp]

## Patterns by Category

### Security
- [Learning entries...]

### Performance
- [Learning entries...]

### Testing
- [Learning entries...]

---

## Recent Additions

### [YYYY-MM-DD] From: owner/repo#123

**Learning:** [Title]

**Category:** `security` | **Confidence:** `high`

**Context:**
[What triggered this learning]

**The principle:**
[Generalized, transferable lesson]

**Example:**
```[language]
// Before (problematic)
[code]

// After (correct)
[code]

### Step 7: Smart Consolidation (Periodic)

Trigger consolidation when:
- Document exceeds 50KB
- 10+ new entries since last consolidation
- User explicitly requests: `/pr-learning consolidate`

**Consolidation process:**

1. **Extract recurring themes** from individual entries
2. **Create "Key Patterns" section** at document top
3. **Archive old entries** (move to bottom, collapse details)
4. **Update cross-references** between related learnings

### Step 8: Create Backup

Before any write operation:
```bash
# Backup existing files
cp ./learnings/pr-learnings.md "./learnings/pr-learnings_$(date +%Y%m%d_%H%M%S).md.bak"

Keep last 5 backups, delete older ones.

Step 9: Report Summary

After processing, output:

## PR Learning Extraction Complete

**PR:** #123 - [Title]
**Author:** @username
**Merged:** 2026-04-10

### Learnings Extracted: 3

| # | Learning | Category | Scope | Destination |
|---|----------|----------|-------|-------------|
| 1 | Use `pytest.raises` context manager | testing | general | ~/learnings/general-learnings.md |
| 2 | Always validate webhook signatures | security | general | ~/learnings/general-learnings.md |
| 3 | Service layer must not import from handlers | architecture | project | ./learnings/pr-learnings.md |

### Deduplication Actions: 1

- Enhanced existing learning "Prefer pytest fixtures over setUp" with new evidence from this PR

### Files Updated:
- `./learnings/pr-learnings.md` (2 new entries)
- `~/learnings/general-learnings.md` (2 new entries, 1 enhanced)

Quality Filters

Not all PRs have extractable learnings. Skip or flag when:

Example Session

User: /pr-learning 456

Agent gathers data, then outputs:

PR Learning Extraction Complete

PR: #456 - Add rate limiting to public API endpoints Author: @alice Merged: 2026-04-08 Reviewers: @bob, @carol

Analysis

Initial submission: Added rate limiting using a custom in-memory counter per endpoint.

Review feedback:

1. @bob: "We have a rate limiter middleware in src/middleware/rate_limit.py - please use that instead of rolling your own"
2. @carol: "In-memory counters won't work in our multi-pod deployment - need Redis-backed limiting"
3. @bob: "Missing tests for the 429 response case"

Final changes:

• Removed custom counter implementation
• Integrated existing RateLimiter middleware
• Added Redis backend configuration
• Added test for rate limit exceeded scenario

Learnings Extracted: 3

1. Check for existing infrastructure before implementing (Project)

Category: architecture | Severity: important

The lesson: Before implementing cross-cutting concerns (auth, rate limiting, caching, logging), search the codebase for existing solutions. This project has middleware in src/middleware/ for common concerns.

Files to check: src/middleware/, src/utils/, ARCHITECTURE.md

2. Stateless services need external state stores (General)

Category: architecture | Severity: critical

The lesson: In containerized/multi-instance deployments, in-memory state (counters, caches, sessions) won't work correctly. Use external stores (Redis, Memcached, database) for shared state.

Pattern:

# Wrong: In-memory (fails in multi-pod)
request_counts = {}  # Lost on restart, not shared across pods

# Right: External store
redis_client.incr(f"rate_limit:{user_id}")

3. Always test error responses (General)

Category: testing | Severity: important

The lesson: When adding features that can return error responses (rate limits, validation errors, auth failures), explicitly test those error paths including response codes and error message format.

Files Updated:

• ./learnings/pr-learnings.md (1 new entry)
• ~/learnings/general-learnings.md (2 new entries)

Important Rules

1. Only process merged PRs - Open PRs don't have final state
2. Require review feedback - No comments = no learnings
3. Preserve evidence - Always link to specific review comments
4. Deduplicate aggressively - Enhance existing over creating duplicates
5. Classify scope correctly - Project-specific stays in repo, general goes machine-wide
6. Back up before writing - Never lose existing learnings
7. Keep learnings actionable - "Do X" not "X is important"

File Initialization

If learning files don't exist, create them with headers:

./learnings/pr-learnings.md:

# PR Learnings - [Repository Name]

> Knowledge base auto-generated from merged PR reviews.
> Run `/pr-learning <PR>` to add learnings from any merged PR.

---

## Key Patterns

[Consolidated patterns will appear here after 10+ entries]

---

## Recent Learnings

[Individual PR learnings will be added below]

~/learnings/general-learnings.md:

# General Engineering Learnings

> Transferable knowledge extracted from PR reviews across all projects.
> This file grows across all repositories you work in.

---

## Patterns by Category

### Security

### Performance

### Testing

### Architecture

### Tooling

---

## Recent Additions

[Learnings will be added below]

Integration with PR Review

This skill complements /pr-review:

• pr-review: Reviews PRs before merge (prospective)
• pr-learning: Extracts lessons after merge (retrospective)

For maximum value, run /pr-learning on PRs that had significant review cycles.