Converting Books To Skills

Core Workflow

Step 0: Prerequisites

Before starting conversion:

# Navigate to repository root
ROOT="$(git rev-parse --show-superproject-working-tree --show-toplevel | head -1)" && cd "$ROOT"

# Verify book file exists (user provides the path)
ls {path-to-book-file}.md

Input required from user:

• Path to markdown file containing the book
• Desired skill name (kebab-case)

For EPUB files: First use converting-epub-to-markdown skill to extract markdown:

Read(".claude/skill-library/claude/skill-management/converting-epub-to-markdown/SKILL.md")

Cannot proceed without valid markdown file ✅

Step 1: Validate Book File

Verify the book markdown is properly formatted:

# Check file size (OCR books are typically 10K-30K lines)
wc -l {path-to-book-file}.md

# Preview first 200 lines to identify structure
head -200 {path-to-book-file}.md

Look for:

• Table of Contents section (usually in first 300 lines)
• Chapter markers (e.g., ## CHAPTER 8, ## 1 INFECTION VECTORS)
• Consistent header patterns for chapter detection

Real examples from today's conversion:

• Windows Internals Part 2: ## CHAPTER 8 System mechanisms (line 534)
• Mac Malware Vol 1: ## 1 INFECTION VECTORS (line 640)
• Mac Malware Vol 2: ## 1 EXAMINING PROCESSES (line 548)

Step 2: Extract Keywords from Table of Contents

The TOC typically contains the best keywords for skill description:

# Find TOC section (usually marked with "Table of Contents" or "Contents")
grep -n -i "table of contents\|^contents$\|^## BRIEF CONTENTS" {path-to-book-file}.md

# Extract chapter titles from TOC (adjust line range based on grep results)
sed -n '100,300p' {path-to-book-file}.md | grep -i "chapter"

Manual keyword selection:

1. Read extracted chapter titles from TOC
2. Identify 5-10 core technical topics
3. Use these for skill description

Real examples from today's conversion:

• Windows Internals Part 2: "system mechanisms, virtualization, VBS, VSM, registry, ETW, NTFS, boot"
• Mac Malware Vol 1: "infection vectors, persistence, capabilities, Mach-O, disassembly, anti-analysis"
• Mac Malware Vol 2: "Endpoint Security, code signing, network monitoring, tool development"

See: references/keyword-extraction-patterns.md for TOC parsing strategies.

Step 3: Detect Chapter Boundaries

Identify the markdown header pattern used for chapters:

# Search for common chapter patterns
grep -n "^##\s\+CHAPTER\|^##\s\+[0-9]\|^##\s\+[0-9][0-9]" {path-to-book-file}.md | head -20

Common patterns found in production:

• ## CHAPTER 8 System mechanisms (Windows Internals - H2 with "CHAPTER" keyword)
• ## 1 INFECTION VECTORS (Mac Malware - H2 with digit and double space)
• ## {digit} {TITLE} (most common - note the double space)

Record the pattern and line numbers - you'll use these exact boundaries for extraction.

CRITICAL: Verify chapter sequence

After detecting chapter markers, verify no chapters are missing:

# List all detected chapters in order
grep -n "^##\s\+CHAPTER\|^##\s\+[0-9]" {path-to-book-file}.md

# Check the sequence for gaps (e.g., 1,2,3,8,9,10 - missing 4-7)

Production note: In today's conversion, we successfully detected:

• Windows Part 2: Chapters 8-12 (no gaps)
• Mac Vol 1: Chapters 1-11 (sequential)
• Mac Vol 2: Chapters 1-14 (sequential, but some chapter numbers on separate lines from headers)

If the list looks incomplete or has gaps:

Check the table of contents to see which chapters should exist, then search for missing chapter numbers in the file.

See: references/ocr-gap-detection.md for gap detection techniques.

Step 4: Split Book into Chapter Files

🚨 CRITICAL REQUIREMENT:

SKILL.md files MUST ONLY reference chapter files in references/chapters/, NEVER the full Books/*.md files.

The Books/ files are too large to parse efficiently. All user-facing documentation in SKILL.md must point to chapter files only.

Chunking Requirements:

• Target: <25,000 tokens per chunk (Read tool limit)
• Token estimation: ~4 characters = 1 token, so max ~100,000 characters per file
• Minimum: 5+ chunks per book (ensures granular access)
• Primary strategy: Split by chapters
• Fallback: If book has <5 chapters, split by H2 sections within chapters
• MANDATORY: Extract ALL chapters from Books/ into chapter files, then update SKILL.md to reference only chapter files

Validation:

# After splitting, check chunk sizes (both lines and tokens)
wc -l references/chapters/*.md

# Check token counts (character count / 4 = approximate tokens)
for f in references/chapters/*.md; do
  chars=$(wc -c < "$f")
  tokens=$((chars / 4))
  echo "$f: $tokens tokens (approx)"
done

# Check for violations:
# - Any file >25,000 tokens (~100,000 characters)? → Need semantic splitting
# - Total count <5? → Need finer granularity (H2 sections)

Step 4.5: Split Oversized Chapters (if needed)

If any chapter exceeds 25,000 tokens (~100,000 characters):

1. Check token count: chars=$(wc -c < chapter-NN.md); echo "$((chars / 4)) tokens (approx)"
2. Find semantic split point: grep -n "^##[^#]" chapter-NN.md
3. Identify major topic shift around the midpoint
4. Split at that boundary using sed
5. Name as chapter-NN-part1.md and chapter-NN-part2.md
6. Delete the original unsplit chapter file
7. Re-validate: both parts should be <25,000 tokens (~100,000 characters)

Production example (Windows Part 2 Chapter 8 - 8,389 lines, ~33,556 tokens):

# Check token count first
chars=$(wc -c < chapter-08.md); echo "$((chars / 4)) tokens (approx)"

# Found semantic split at line 4300 (## Executive objects - major topic shift)
sed -n '1,4299p' chapter-08.md > chapter-08-part1.md    # 4,299 lines, ~17,196 tokens ✓
sed -n '4300,8389p' chapter-08.md > chapter-08-part2.md  # 4,090 lines, ~16,360 tokens ✓
rm chapter-08.md  # Remove unsplit version

# Verify both parts are under 25,000 token limit
for f in chapter-08-part*.md; do
  chars=$(wc -c < "$f")
  echo "$f: $((chars / 4)) tokens (approx)"
done

Production example (Chapter 10 - 5,687 lines, ~22,748 tokens):

# Check if splitting needed
chars=$(wc -c < chapter-10.md); echo "$((chars / 4)) tokens (approx)"

# Split at line 2668 (## Windows Management Instrumentation)
sed -n '1,2668p' chapter-10.md > chapter-10-part1.md    # 2,668 lines, ~10,672 tokens ✓
sed -n '2669,5687p' chapter-10.md > chapter-10-part2.md  # 3,019 lines, ~12,076 tokens ✓
rm chapter-10.md

Result: Oversized chapters (>25,000 tokens) split successfully, all parts now under token limit.

Manual sed extraction (recommended - precise control):

# Create chapters directory
mkdir -p .claude/skill-library/{category}/{skill-name}/references/chapters

# Extract each chapter using line boundaries from Step 3
# Format: sed -n '{start},{end}p' {source} > {output}

# Example from today's conversion (Windows Part 2):
sed -n '534,8922p' {path-to-book-file}.md > .claude/skill-library/{path}/references/chapters/chapter-08.md
sed -n '8923,12692p' {path-to-book-file}.md > .claude/skill-library/{path}/references/chapters/chapter-09.md
# ... repeat for each chapter

# Verify extraction
wc -l .claude/skill-library/{path}/references/chapters/*.md

Production evidence from today:

• Windows Part 2: Used exact line boundaries (534, 8923, 12693, 18380, 24803, 27497)
• Mac Vol 1: Used boundaries (640, 1124, 1888, 2526, etc.)
• Mac Vol 2: Used boundaries (548, 3347, 4923, etc.)

Result: All chapters extracted successfully with exact content boundaries.

Expected output:

• chapter-01.md through chapter-NN.md - Main chapters (numbered to match book)
• Each file <5,000 lines initially (will split oversized ones in next step)
• Total: 8-14 chapter files typical for technical books
• Chapter numbers match the book's numbering (e.g., Windows Part 2 has chapters 8-12, not 1-5)

See: references/chapter-splitting-techniques.md for handling complex book structures.

Step 5: Generate SKILL.md with Chapter Summaries

🚨 CRITICAL REQUIREMENT:

SKILL.md files MUST ONLY reference chapter files in references/chapters/, NEVER the source book file.

The source markdown files are too large to parse efficiently. All user-facing documentation in SKILL.md must point to chapter files only.

Required template:

---