Rlm Unlimited Context

Three Design Principles (from the paper)

1. Symbolic Handle to Prompt

Never load large inputs into your context window. The prompt is a file path — a variable in the environment. You interact with it through code.

# WRONG: Read the whole file into context
cat huge_file.txt

# RIGHT: Keep it external, probe with code
wc -l huge_file.txt
head -50 huge_file.txt
grep -c "error" huge_file.txt

2. Symbolic Recursion (sub-LM calls)

Use the Task tool as your lm_query() function. Each sub-agent processes a chunk and returns structured results to a file — not to your context.

lm_query(chunk, question) → Task tool sub-agent → writes result to file

3. Output Through Variables

Never try to hold all results in context. Write everything to files. Build the final answer by aggregating file contents programmatically.

# Sub-agent writes: results/chunk_0001.json
# Sub-agent writes: results/chunk_0002.json
# You aggregate with code, then read the small summary

When to Use

Don't use RLM for files < 50KB or tasks where a single grep answers the question.

The RLM Loop (Step by Step)

Step 1: INIT — Probe the Environment

Before writing any processing code, understand what you're working with.

# Size and structure
wc -l -c INPUT_PATH
file INPUT_PATH
head -100 INPUT_PATH

# For directories
find DIR -type f -name "*.EXT" | wc -l
find DIR -type f | xargs wc -c | sort -rn | head -20

# For structured data
python3 -c "import json; d=json.load(open('f.json')); print(type(d), len(d))"

Output: You now know size, format, structure. Set your strategy.

Step 2: FILTER — Reduce Before Processing

This is the biggest efficiency win. Use code to eliminate irrelevant content BEFORE any LLM processing. The paper found that model priors (keyword filtering) dramatically reduce cost.

# Fast regex pre-filter
python3 scripts/rlm_search.py INPUT "keyword1|keyword2" --context 5

# Structure-based filter
grep -rl "class.*Controller" ./src/
find ./repo -name "*.py" -path "*/api/*"

# Statistical filter
python3 -c "
import re
with open('INPUT') as f: text = f.read()
sections = re.split(r'\n## ', text)
relevant = [s for s in sections if 'TARGET_TOPIC' in s.lower()]
print(f'Filtered {len(sections)} → {len(relevant)} sections')
for i, s in enumerate(relevant):
    open(f'filtered_{i}.txt', 'w').write(s)
"

Step 3: CHUNK — Split into Processable Pieces

# Auto-detect best strategy
python3 scripts/rlm_chunker.py auto INPUT_PATH --output ./chunks

# Manual control
python3 scripts/rlm_chunker.py file large.txt --method lines --size 500
python3 scripts/rlm_chunker.py file doc.md --method separator --sep "\n## "
python3 scripts/rlm_chunker.py dir ./src --ext .py .js --size 60000

Target: 30K–80K chars per chunk for sub-agent calls. Check chunks/manifest.json for the plan.

Step 4: DELEGATE — Sub-LM Calls (Full RLM only)

This is lm_query() from Algorithm 1. Use the Task tool to process chunks in parallel.

Critical rules:

1. Each sub-agent gets ONE chunk + ONE clear question
2. Sub-agents write results to files (not your context)
3. Batch 3–10 small chunks per sub-agent call
4. If chunk count > 50, add more filtering or increase chunk size

Sub-agent prompt template:

Read the file at {chunk_path}.

TASK: {specific_question}

Write your answer as JSON to {result_path}:
{
  "chunk_index": N,
  "findings": ["finding1", "finding2"],
  "summary": "1-2 sentence summary",
  "relevant": true/false
}

If nothing relevant, set "relevant": false and "findings": [].

For RLM-lite (no sub-calls): Skip this step. Use rlm_search.py regex results directly.

Step 5: AGGREGATE — Build FINAL_ANSWER

# Collect all results
python3 scripts/rlm_aggregate.py ./results --output final_answer.json

# Or manually
python3 -c "
import json, glob
results = []
for p in sorted(glob.glob('./results/*.json')):
    results.append(json.load(open(p)))
relevant = [r for r in results if r.get('relevant')]
findings = [f for r in relevant for f in r.get('findings', [])]
json.dump(findings, open('aggregated.json', 'w'), indent=2)
print(f'{len(relevant)}/{len(results)} chunks relevant, {len(findings)} findings')
"

Then do a final synthesis (as sub-agent if aggregated results > 30K chars, or in your own context if small enough):

ORIGINAL QUERY: {query}
AGGREGATED FINDINGS (from {N} chunks):
{findings}
Synthesize a comprehensive answer.

Write the final answer to a file — this is FINAL_ANSWER from Algorithm 1.

Strategy Patterns

Pattern A: Map-Reduce — O(N) linear tasks

Probe → Chunk → Map (parallel sub-agents) → Reduce (aggregate) → FINAL_ANSWER

Best for: summarization, classification, extraction across all content.

Pattern B: Filter-Then-Process — O(1) needle tasks

Probe → Filter (regex/grep) → Process (sub-agent on filtered content) → FINAL_ANSWER

Best for: finding specific information. RLM-lite often sufficient here.

Pattern C: Classify-Then-Compute — O(N²) pairwise tasks

Probe → Chunk → Classify (sub-agents label each) → Compute pairs (code) → FINAL_ANSWER

Best for: cross-referencing, relationship finding, comparisons.

Pattern D: Hierarchical — 1M+ token inputs

Probe → Mega-chunk (~10 groups) → Sub-agent per mega-chunk (each further chunks internally) → Aggregate → FINAL_ANSWER

Best for: inputs so large that chunking alone produces 100+ pieces.

Cost Control (from paper's Observation 4)

RLM costs are comparable to base model at the median but have high variance at the tail due to long trajectories.

• Always filter first. An 80% filter = 80% cost reduction.
• Batch aggressively. 5 chunks × 1 call > 5 × 1 call.
• Code for deterministic work. Counting, sorting, dedup → code, not sub-agents.
• Sub-agents for semantic work only. Classification, summarization, reasoning → sub-agents.
• Cap sub-agent calls. If approaching 50+ calls, add filtering or increase batch size.
• Use RLM-lite first. Try no-sub-calls mode; escalate to full RLM only if needed.

Anti-Patterns

1. Don't load large files into context — they're variables in the environment
2. Don't hold sub-agent results in context — write to files
3. Don't use sub-agents for counting/sorting — use code
4. Don't skip the probe step — you need structure before strategy
5. Don't skip filtering — it's the biggest efficiency win
6. Don't launch 100+ sub-agents — batch more aggressively
7. Don't process when a grep suffices — RLM-lite exists for a reason

CLAUDE.md Template

Add this to any project to activate RLM:

## Large Input Processing (RLM)

When inputs exceed 50KB or tasks require processing all/most of a large file:

1. Prompt is a VARIABLE (file path), not in context
2. Probe first: wc, head, file, find
3. Filter with grep/regex before any LLM processing
4. Chunk into 30K-80K char pieces (scripts/rlm_chunker.py)
5. Delegate semantic work to sub-agents (Task tool = lm_query)
6. Store all intermediate results in files, never context
7. Aggregate programmatically → FINAL_ANSWER in a file
8. Code for deterministic work, sub-agents for semantic work only