Knowledge Compiler

Core promise: Passive mode (user provides files → agent compiles) only requires Python 3 + filesystem access, zero network dependencies. All network calls are optional enhancements, and the environment check on first use will inform the user of available capabilities.

Quick Start

User: /auto-wiki recall personal-pension
Agent: [Scan .wiki/personal-pension/ → Read index.md → Load data.db summary]
Agent: Entered recall mode. Current wiki: 22 pages / 8 data points / 2 contested.
       Subsequent questions will be answered by checking wiki first.

User: What are the reasons for low participation rates?
Agent: [Read wiki pages enrollment-friction, tax-incentive-effect, etc.]
Agent: According to 6 sources accumulated in wiki... (cite specific pages and data)
       ⚠️ Note: Tax incentive effects are contested (77.8% vs 25%), see [[participation-willingness]]

User: ingest this industry report into XX wiki
Agent: [Read report → Search existing wiki → Compare old vs new → Update 3 pages, create 1 page]
Agent: Ingested into XX wiki: updated 3 pages (list), created 1 page (list)

Core Philosophy

Agents help you research, write reports, and pull data every day—but then forget everything. The next time you ask about the same domain, it starts from scratch.

This skill solves one thing: giving agents a knowledge base that accumulates over time.

It's not RAG (retrieving from documents on each query), but compilation—after reading source files, the agent writes key information into existing wiki pages, comparing with old knowledge, merging, and flagging conflicts. Before executing any task next time, it reads the wiki and works from the accumulated foundation.

Four Modes

deep-dive is not a 5th independent mode—it's a combined pipeline of lint (Coverage) and ingest (with search tools). Requires search tools (active mode).

Recall mode vs query difference: query is one-time operation (ask one question, check wiki once). Recall mode is continuous state—after entering, every question in this conversation goes through wiki first.

Recall Mode

Entering

Triggered when user says /auto-wiki recall or /auto-wiki recall {topic}.

Agent executes:

1. Scan .wiki/ directory, list available wiki topics
2. If user specified topic → Load that wiki; If not specified → List optional topics for user to choose
3. Read index.md → Get all page list and structure
4. Read data.db summary → python references/store.py dump .wiki/{topic}/, get data point count, relation count, contested count
5. Report to user:

   Entered recall mode: {topic}
   - Pages: {N} (sources: X, entities: Y, concepts: Z)
   - Data points: {N} | Relations: {N} | Contested: {N}
   Subsequent questions will be answered by checking wiki first. Say "exit recall mode" to return to normal.
   

Response Flow

After entering recall mode, upon receiving each user question:

1. Extract keywords from question (entity names, concept names, metric names)
2. Match in index.md for relevant pages (title + description)
3. Query data.db for relevant data points:

   SELECT * FROM data_points WHERE field LIKE '%keyword%' OR page_slug LIKE '%keyword%'
   

4. Read matched wiki pages (usually 2-5), expand one level along wikilinks
5. Synthesize answer, must:
   • Cite specific pages: [[slug]]
   • Cite specific data: value + unit + period + source
   • If involving contested info, proactively label
   • If wiki info insufficient, clearly state "no accumulation in this aspect in wiki, suggest ingesting XX"
6. Do not fabricate information not in wiki. Better to say "don't know" than pretend wiki has it

Exit

Exit when user says exit recall, switches to other operations (ingest/lint), or starts a new topic.

Execution Flow

Phase 0: Identify Research Topic and Ontology Type

After receiving user input, determine three things: operation type, target wiki, ontology type.

Ontology type determines wiki page structure and collection strategy:

A wiki has only one type. If research spans both person and domain (e.g., "Munger's investment framework applied to enterprise annuity"), it belongs to two wikis, with cross-wiki query for synthesis. Don't mix cognitive and domain page structures in one wiki.

If wiki directory doesn't exist, first confirm creation location with user (default .wiki/{topic}/ under current repository root), then create initial structure per references/storage-spec.md (including meta.yaml, index.md template, log.md template). Suggest user add .wiki/ to .gitignore if not already present.

Domain seed: If the target domain has a corresponding seed file (seeds/{name}.md), declare seed: {name} in meta.yaml. Seeds provide standard terminology, relationship templates, and anti-confusion rules, letting wikis start from a normalized foundation. Domains without seeds grow freely — both paths work. Seeds are community-contributable plugins — anyone can write a markdown file for their vertical domain. See references/seed-ontologies.md.

On first use, perform environment check (see references/source-validation.md), informing user of available capabilities (passive mode vs active mode).

Reference Loading Strategy

Don't read all references at once. Load on-demand by operation type:

Not needed: scaling.md only relevant when page count > 500; ontology-types/ only when creating new wiki and need to determine type.

Phase 1: Ingest (Knowledge Compilation)

This is the core operation. Detailed protocol in references/ingest-protocol.md.

Brief flow:

1. Read source file, extract key information
2. Validate key data (if tools available)—see references/fact-check.md
3. Write source summary page (sources/{date}-{slug}.md)
4. Search wiki for existing related pages (read index.md, grep key entity names)
5. Compare old vs new page by page:
   • New info supports existing conclusion → add citation, boost confidence
   • New info overturns existing conclusion → write value to data.db (old value auto-enters history table), rewrite body analysis
   • New info conflicts and can't determine → present both views, confidence → contested
6. Create new pages (only for entities/concepts not yet in wiki)
7. Update index.md + append log.md
8. Schema validation—run python references/schema.py {page.md} on all created/modified pages to ensure frontmatter compliance. Fix immediately if failed

After ingest, report to user:

Ingested into {topic} wiki:
- Created: {N} pages (list)
- Updated: {N} pages (list + brief reason)
- Conflicts: {N} (list conflict points)
- Validation: {N} pages all passed / {M} pages have issues (list)

Phase 2: Query (Knowledge Retrieval)

Detailed protocol in references/query-protocol.md.

1. Read index.md, identify pages relevant to question
2. Read matched pages + follow wikilinks to related pages (one level)
3. Synthesize answer based on page content, cite source pages:

   Based on {N} source files accumulated in wiki:
   ... analysis content ...
   Sources: [[alpha-corp]], [[2026-policy-doc]]
   

4. If involving contested info, clearly label the conflict
5. If answer contains valuable new analysis, prompt user to archive

If wiki info insufficient, clearly state the gap:

Wiki information on XX is insufficient. Currently only 2 relevant source files.
Suggest ingesting more materials about XX.

Phase 3: Lint (Knowledge Governance)

Detailed protocol in references/lint-protocol.md (7 checks + health report format).

Lint has two levels:

1. Structural: Auto-scan all pages, fix format, broken links, orphans, staleness
2. Semantic (user triggered): Detect conflicts, duplicates, coverage gaps. Wiki < 50 pages = full scan, 50-200 pages = only last 30 days' ingest-touched pages, > 200 pages = user must specify scope
3. Report health:

Wiki Health Report: {topic}
- Total pages: 42 (sources: 12, entities: 15, concepts: 10, analyses: 5)
- Health: Good
- Structural fixes: Fixed 1 broken link, archived 1 stale page
- [Semantic] Pending human review: 2 conflicts (list)
- Suggestion: Few source files in XX domain (only 1), recommend supplement

Phase 4: Deep-Dive (Knowledge Completion Pipeline)

deep-dive = lint(Coverage) + ingest(search-fill). Not an independent mode, but a combined pipeline.

Prerequisite: Requires search tools (active mode). Without search tools, only outputs gap report without auto-filling. Prompts user to manually ingest.

Flow:

1. Run lint Coverage check (5-category gap detection)
   → Output structured Gap Report (see lint-protocol.md)

2. Present Gap Report, request user confirmation
   → User can: accept all / select subset / limit scope / cancel
   → This step cannot be skipped — prevents unsupervised batch writes

3. For each confirmed gap, execute from-lint ingest flow
   → Search → User confirms source → Standard ingest
   → See From-Lint section in ingest-protocol.md

4. Output completion report: filled / unable to fill / recommendations

Trigger words: deep-dive, deep dive, fill gaps, research gaps, level up

Example:

User: deep-dive treasury-futures
Agent: [Running Coverage lint...]
Agent: Found 6 knowledge gaps:
       1. [high] page_missing: stock-bond-correlation (referenced by 4 pages)
       2. [high] concept_missing: basis (mentioned in 5 entity pages)
       3. [medium] single_source: treasury-futures-basics (only 1 source)
       ...
       Which to fill? (all / select numbers / cancel)

User: 1, 2

Agent: [Searching "stock bond correlation treasury futures"...]
Agent: Found 2 candidate sources:
       - [authoritative-secondary] CICC Fixed Income "Stock-Bond Correlation Analysis" ← Recommended
       - [secondary] Blog post ← Skip (blacklisted channel)
       Confirm using CICC report?

User: confirm

Agent: [Execute standard ingest → Create concepts/stock-bond-correlation.md]
Agent: Completion done. Created 2 pages, updated 0 pages, 1 gap unable to fill (suggest providing materials manually).

Wiki Page Format

See references/wiki-format.md for details. Brief summary:

• Each page is markdown with frontmatter (title, type, created, updated, sources, confidence)
• 5 page types: source / entity / concept / analysis / mental-model
• Use [[slug]] for inter-page links
• index.md is the directory, log.md is the operation log

Ontology Type Reference

When research target is a person, see references/ontology-types/cognitive.md for collection strategy—page types focus on mental models, heuristics, value systems, expression styles.

When research target is a domain, see references/ontology-types/domain.md for collection strategy—page types focus on institutional entities, system concepts, quantitative metrics.

Both share the same wiki infrastructure (ingest/query/lint), differing only in page classification and collection emphasis.

Vertical Domain Adaptation

The skill core is a domain-agnostic compilation engine. Vertical domain expertise is injected through two plugin layers:

Without plugins, wikis grow freely — suitable for exploratory research. With plugins, wikis start from industry standards with normalized naming, clear relationship structures, and detectable logic gaps.

Community-contributable: Write a seed file (markdown) for your vertical domain — declare 20-50 core terms and anti-confusion rules, and wikis in that domain start from a normalized foundation.

Currently available:

• seeds/fibo-pensions.md — Enterprise annuity / pensions (based on FIBO standard)
• validators/fibo-mcp.md — FIBO SPARQL logical validation (627K inferred triples)

What This Skill Doesn't Do

• No vector retrieval. Small scale uses index + grep, large scale uses SQLite FTS5 + BM25 (see references/scaling.md). Vector retrieval is left to platform-level tools.
• No multi-user collaboration. Wiki directory is local files, one wiki per user.
• No replacement for professional data tools. Domain data acquisition uses corresponding MCP/tools; this skill only receives their outputs and compiles into wiki.

Relationship with Other Tools

This skill doesn't replace any professional tool, it connects them:

Any research tool produces analysis → ingest into corresponding wiki
Any data tool pulls data           → ingest into corresponding wiki
Domain seeds provide starting line  → standard terms + anti-confusion rules
External validators check logic     → lint checks knowledge structure completeness

Next time executing tasks, agent reads relevant wiki → works with accumulated knowledge