Index Note

Workflow

Step 0: Initialize

# Ensure output directories exist
mkdir -p ./IndexVault/_new ./IndexVault/_images ./IndexVault/_downloads

Step 1: Classify Input

Run the classification script to determine input type:

uv run python ./skills/index-note/scripts/classify_input.py --input "INPUT_STRING"

The script returns JSON with:

• type: one of idea, project, book, paper, webinfo, webnews
• confidence: 0.0-1.0
• needs_phase2: whether content analysis is needed for reclassification
• details: type-specific metadata

Phase 1 Classification Rules (handled by script):

1. Bare arXiv ID (e.g., 2401.12345) → paper
2. Local folder path (exists as directory) → project
3. Local file path (exists as file):
   • .docx/.doc/.txt/.epub → book
   • .pdf → tentative paper (needs Phase 2)
4. GitHub URL (github.com/owner/repo) → project
5. arXiv URL (arxiv.org/abs/... or arxiv.org/pdf/...) → paper
6. Academic domain (doi.org, openreview.net, etc.) → paper (needs Phase 2)
7. News domain (reuters.com, bbc.com, 36kr.com, etc.) → webnews
8. Other URL → webinfo (needs Phase 2)
9. Plain text (no URL, no path) → idea

Phase 2 Content-Based Reclassification

If needs_phase2 is true, perform content analysis:

For local PDF files:

• Read the first few pages of the PDF
• Check for academic paper indicators:
  • "Abstract" section near the beginning
  • "References" or "Bibliography" section
  • arXiv ID in the document
  • Citation patterns (e.g., [1], [Author et al., YYYY])
  • Author affiliations with universities/labs
  • Conference/journal headers (CVPR, NeurIPS, ICML, etc.)
• If >= 3 academic indicators found → paper
• Otherwise → book

For general web URLs:

• Use WebFetch to retrieve page content
• Check if the page is an academic paper:
  • URL contains: "paper", "proceedings", "journal", "doi.org", "scholar"
  • Page contains: "Abstract", "References", author affiliations
  • → Reclassify as paper
• Check if the page is a book:
  • URL or page contains: "book", "chapter", "ISBN", publisher names
  • → Reclassify as book
• Check if it's actually news (missed by domain list):
  • Page has: prominent publication date, journalist byline, news article structure
  • → Reclassify as webnews
• Otherwise: keep as webinfo

Step 2: Download Content (if needed)

Based on the classified type, download source material:

Project (GitHub URL):

mkdir -p ./IndexVault/_downloads/github
# Extract repo name from URL
REPO_NAME=$(echo "GITHUB_URL" | sed 's|.*/||' | sed 's|\.git$||')
git clone --depth 1 "GITHUB_URL" "./IndexVault/_downloads/github/$REPO_NAME"

Project (local folder):

No download needed. Read directly from the path.

Paper (arXiv URL or ID):

mkdir -p ./IndexVault/_downloads/arxiv
# Extract arXiv ID
# Download PDF
curl -L "https://arxiv.org/pdf/${ARXIV_ID}" -o "./IndexVault/_downloads/arxiv/${ARXIV_ID}.pdf"

Also fetch metadata from arXiv API:

curl -s "https://export.arxiv.org/api/query?id_list=${ARXIV_ID}" -o "./IndexVault/_downloads/arxiv/${ARXIV_ID}_meta.xml"

Paper (local PDF):

No download needed. Read from original path.

Book (local file):

No download needed. Read from original path.

Book / Web Info / Web News (web URL):

Use WebFetch tool to retrieve and analyze web content. The WebFetch tool returns processed content directly -- no file download needed for text content.

Step 2.5: Extract PDF Content

When the source material is a PDF file, call the extract-pdf skill to get structured Markdown (with tables, headings, lists preserved) and rendered page images. This provides much better content than reading the raw PDF directly.

Applies to:

• Paper type (arXiv PDF at ./IndexVault/_downloads/arxiv/${ARXIV_ID}.pdf, or local PDF path)
• Book type (when the source is a local PDF file)

Skip for: idea, project, webinfo, webnews, and non-PDF book sources (.docx, .txt, .epub).

Run the extractor:

# Derive a slug from the PDF filename (e.g., "2401.12345" or "thinking_fast_and_slow")
PDF_SLUG=$(basename "${PDF_PATH}" .pdf)

uv run --with pymupdf4llm --with pdfminer.six python ./skills/extract-pdf/scripts/extract_pdf.py \
  --input "${PDF_PATH}" \
  --output-dir ./IndexVault/_downloads/_pdf_extracts/${PDF_SLUG}/

Read the manifest to understand the PDF structure:

Read ./IndexVault/_downloads/_pdf_extracts/${PDF_SLUG}/manifest.json

Check total_pages_in_pdf, tables_detected, and per-page text_chars. Pages with very low text_chars are likely scanned/image-only — read the page image directly for those.

Extracted content is available at:

• full_text.md — structured Markdown with tables and headings (primary reading source)
• full_text_pdfminer.md — plain-text complement for cross-checking mangled passages
• pages/page_NNN.md — per-page Markdown (aligned with images by page number)
• images/page_NNN.png — rendered page images (for figures, math, scanned pages, complex tables)

Use this extracted content in Step 3 for analysis instead of reading the raw PDF.

Step 3: Read and Analyze Content

Read the source material thoroughly. The analysis depth and strategy depends on the type.

For PDF sources (paper/book): Read from the extract-pdf output generated in Step 2.5. Start with full_text.md for the main content. For pages containing figures, math, or complex tables, read the corresponding images/page_NNN.png directly — Claude can see and interpret the rendered page. Cross-check with full_text_pdfminer.md if any passage looks garbled or has suspicious gaps.

Cross-cutting principles (apply to ALL types):

Based on cognitive science research, every note should:

1. TL;DR first (Cognitive Load Theory): Start with a 1-3 sentence executive summary in a callout block. This reduces search time and supports progressive disclosure.
2. "In My Own Words" / self-explanation (Chi et al., 1989): Restate key ideas in your own words rather than copying. Self-explanation significantly improves comprehension and retention.
3. "So What?" prompt (Elaborative Interrogation): Every note must answer "how does this change what I do?" Knowledge without action connections has low retrieval value.
4. Typed connections (Zettelkasten): Don't just list related notes -- classify the relationship (supports, contradicts, extends, applies). This builds a navigable knowledge graph.
5. Retrieval cues (Testing Effect): End with specific questions and scenario triggers ("when would I come back to this?") rather than just summaries. Questions force deeper processing than re-reading.
6. Separate facts from interpretations: Always keep verifiable facts distinct from subjective analysis.

IDEA analysis strategy:

• State the core idea in plain language (Feynman Technique)
• Identify the trigger context (what sparked the idea)
• Challenge assumptions explicitly with a table (assumption / why it holds / what if wrong)
• Design a "minimum viable experiment" to test the idea
• Assign maturity level: seed (just captured) / sprout (developed) / tree (validated)
• Mark connection types: supports, contradicts, analogizes, extends

PROJECT analysis strategy:

• Top-down first (Brooks' program comprehension model): Start with purpose/problem, then architecture overview, then details
• Read README.md first, then directory structure, then key source files
• Focus on "when to use / when not to use" -- this has highest retrieval value
• Extract design decisions and their rationale (not just patterns observed)
• Identify entry points for understanding (which files to read first)
• Keep tech stack rationale (why chosen, not just what)

BOOK analysis strategy:

• Apply Adler's analytical reading: classify the book type, state the central thesis, identify the argument structure
• Extract key concepts using Feynman Technique (define, explain simply, identify application)
• Do NOT write chapter-by-chapter summaries (low retrieval value, high redundancy). Instead, trace the book's argument flow as a structure map
• Engage in dialogue with the author: agree, question, disagree (Adler's "Criticizing a Book Fairly")
• Identify the author's blind spots and unstated assumptions
• Must have at least one "immediate application" action item
• Assess the mental model shift: what do you think differently after reading?

PAPER analysis strategy:

• "In My Own Words" section is mandatory: Before filling technical details, write a plain-language explanation of what the paper does. This triggers self-explanation effect (Chi et al., 1989).
• Extract problem → motivation → gap → solution chain (not just "what they did")
• Focus on innovations: what's genuinely new vs. incremental engineering
• Identify what this paper "opens up" -- what new research questions become possible
• Position in field: classify related work as precursor, competitor, or successor
• Score with brief rationale per dimension (not just numbers)

WEB_INFO analysis strategy:

• Apply SIFT method (Stop, Investigate, Find better coverage, Trace claims) for credibility
• Estimate "information half-life": how quickly will this become outdated? (long/medium/short)
• Separate facts from opinions from predictions in a structured table
• Identify what's NOT said (missing perspectives, uncovered data)
• Must answer "so what?" -- what does this mean for my work specifically

WEB_NEWS analysis strategy:

• Apply 5W1H as a compact table (not 6 separate sections -- reduces cognitive load)
• Separate verifiable facts from media narrative
• Signal vs Noise assessment: what's genuinely important vs. media amplification
• Stakeholder analysis with explicit motivations
• Record a personal prediction (what will happen next + rationale + verification timeline)
• This builds calibration over time -- revisiting past predictions improves judgment

Step 3.5: Extract Images to Temp Directory

After analyzing content, extract images from the source material into a temporary directory. Only images actually embedded in the final note will be kept (see Step 4f).

Note: For PDF sources, Step 2.5 already produced rendered page images at ./IndexVault/_downloads/_pdf_extracts/${PDF_SLUG}/images/page_NNN.png. Those are full-page renders useful for reading content. This step extracts individual figures (diagrams, charts, result plots) for embedding in the final note — they serve different purposes. For arXiv papers, extract_images.py also attempts to download high-resolution figures from the LaTeX source, which are higher quality than anything extracted from the compiled PDF.

IMPORTANT: Run the image extraction script using uv run --with pymupdf to ensure PyMuPDF is available.

The temp directory is: ./IndexVault/_images/_tmp_{NOTE_ID}/

For PAPER type:

Uses 3-level priority system (arXiv source > PDF figures > PDF extraction):

uv run --with pymupdf --with requests python ./skills/index-note/scripts/extract_images.py \
  --type paper \
  --input "ARXIV_ID_OR_PDF_PATH" \
  --note-id "YYYY-MM-DD_paper_NNN" \
  --output-dir ./IndexVault/_images/_tmp_YYYY-MM-DD_paper_NNN/

If the input is an arXiv URL, extract the arXiv ID first (e.g., 2505.00949 from https://arxiv.org/abs/2505.00949).

For BOOK type (PDF only):

uv run --with pymupdf --with requests python ./skills/index-note/scripts/extract_images.py \
  --type book \
  --input "PDF_FILE_PATH" \
  --note-id "YYYY-MM-DD_book_NNN" \
  --output-dir ./IndexVault/_images/_tmp_YYYY-MM-DD_book_NNN/

For PROJECT type:

uv run --with pymupdf --with requests python ./skills/index-note/scripts/extract_images.py \
  --type project \
  --input "FOLDER_PATH" \
  --note-id "YYYY-MM-DD_project_NNN" \
  --output-dir ./IndexVault/_images/_tmp_YYYY-MM-DD_project_NNN/

For WEB_INFO / WEB_NEWS type:

For web pages, use the browser screenshot approach:

1. Use WebFetch to get the page content
2. If the page contains important diagrams or figures referenced in the article, note their descriptions in the note
3. Web images are generally not downloaded automatically -- focus on text content extraction

For IDEA type:

No image extraction needed.

Using extracted images in notes

The script outputs JSON with an images array. Each image has a filename field.

Embed images in the Obsidian note using wikilink format:

![[image_filename|600]]

Place images at appropriate locations within the note:

• Paper: After "Method Architecture" section for architecture diagrams, after "Experimental Results" for result figures
• Book: After relevant chapter summaries or concept explanations
• Project: After "System Architecture" section
• Web Info/News: After relevant content sections

Only embed the most informative images (typically 3-5 max). Skip redundant or low-quality images.

Step 4: Generate Note

Step 4a: Get next ID

uv run python ./skills/index-note/scripts/generate_id.py --type TYPE --vault-new-dir ./IndexVault/_new/ --vault-deep-dir ./IndexVault/deep/

This returns a 3-digit ID (e.g., 001).

Step 4b: Construct filename

Format: YYYY-MM-DD_TYPE_NNN.md

Type mapping:

• idea → idea
• project → project
• book → book
• paper → paper
• webinfo → webinfo
• webnews → webnews

Example: 2026-04-05_paper_001.md

Step 4c: Read the template

Read the appropriate template:

./IndexVault/_template/{type}_template.md

Step 4d: Fill the template

Replace all {{PLACEHOLDER}} values with actual data from the analysis. Fill every section with substantive content based on Step 3 analysis.

Image embedding: If Step 3.5 extracted images, embed the most relevant ones using ![[filename|600]]:

• Paper: after "Architecture" and "Results" sections
• Book: near relevant concept explanations
• Project: after "System Overview"
• Select 3-5 best images; skip redundant ones

CRITICAL: Content Quality Requirements

The filled note must match what a domain expert would produce:

1. TL;DR first: Every note starts with a callout summary block
2. Self-explanation: Restate ideas in your own words, don't just copy
3. Completeness without bloat: All important info, no filler. Prefer tables over long paragraphs for structured data
4. Insight over description: Go beyond "what" to "so what" and "now what"
5. Typed connections: Classify relationships (supports/contradicts/extends), not just list links
6. Actionability: Every note must answer "how does this change what I do?"
7. Retrieval-optimized: End with specific scenario triggers, not just summaries
8. Facts separated from interpretation: Keep verifiable facts distinct from analysis

Step 4e: Write the note

Write the completed note to:

./IndexVault/_new/YYYY-MM-DD_TYPE_NNN.md

IMPORTANT: Every note must end with a read-status checkbox (after a --- separator) so users can mark it as read in Obsidian. The <big><big> tags make it visually prominent:

---

- [ ] <big><big>已读</big></big>

Step 4f: Finalize images

After the note is written, run the finalize script to keep only images that are actually embedded in the note and organize them into a per-note subdirectory:

uv run python ./skills/index-note/scripts/finalize_images.py \
  --note-path ./IndexVault/_new/YYYY-MM-DD_TYPE_NNN.md \
  --temp-dir ./IndexVault/_images/_tmp_YYYY-MM-DD_TYPE_NNN/ \
  --images-dir ./IndexVault/_images/ \
  --note-id YYYY-MM-DD_TYPE_NNN

This script:

1. Parses the note for ![[filename|...]] image references
2. Moves referenced images from the temp dir to ./IndexVault/_images/{note_id}/
3. Discards unused images and removes the temp directory

Skip this step for types that had no image extraction (idea, webinfo, webnews).

Step 5: Report to User

After generating the note, display:

1. Detected type and confidence level
2. Output filename and path
3. Brief summary (3-5 bullet points of the most important content)
4. Any issues encountered during processing

Obsidian Formatting Rules

These rules MUST be followed in all generated notes:

1. Frontmatter: Valid YAML between --- markers at the top of the file
2. Tags: No spaces in tags, use hyphens (e.g., machine-learning not machine learning)
3. Formulas: Inline $...$ and block $$...$$ (NOT in code blocks)
4. Image embedding: Obsidian wikilink format ![[filename.png|600]] (NOT markdown ![](url))
5. Internal links: Use wikilinks with display alias [[File_Name|Display Title]]
6. Missing data: Use -- as placeholder (NOT --- which creates horizontal rule)
7. File names: Replace spaces with underscores in all generated filenames
8. Tables: Use standard markdown tables with | separators
9. Headings: Use ## for main sections, ### for subsections
10. Bilingual headers: Each section header includes both English and Chinese ## English (中文)

Error Handling

• If a URL is unreachable, report the error and create a minimal note with available information
• If a local file/folder does not exist, report the error and stop
• If classification is ambiguous, explain the ambiguity in the report and use the best guess
• If content is too large to analyze fully (e.g., very long book), analyze available content and note what was skipped
• If download fails (e.g., git clone fails, arXiv is down), attempt alternative approaches or create note with partial information

Examples

Example 1: Idea

/index-note "大语言模型的推理能力可能本质上是模式匹配而非真正的逻辑推理"

→ Type: idea → File: 2026-04-05_idea_001.md

Example 2: Project (GitHub)

/index-note https://github.com/anthropics/claude-code

→ Type: project → Downloads repo → File: 2026-04-05_project_001.md

Example 3: Paper (arXiv)

/index-note https://arxiv.org/abs/2401.12345

→ Type: paper → Downloads PDF + metadata → File: 2026-04-05_paper_001.md

Example 4: Web News

/index-note https://www.reuters.com/technology/ai-breakthrough-2026/

→ Type: webnews → Fetches content → File: 2026-04-05_webnews_001.md

Example 5: Book (local file)

/index-note C:\Users\Admin\Books\thinking_fast_and_slow.pdf

→ Type: book (Phase 2: no academic indicators) → File: 2026-04-05_book_001.md

Example 6: Web Info

/index-note https://huggingface.co/blog/open-llm-leaderboard

→ Type: webinfo → Fetches content → File: 2026-04-05_webinfo_001.md