/eat — Knowledge Extraction Skill

Workflow

Step 1: Identify source type and fetch content

YouTube video (youtube.com or youtu.be):

Run scripts/fetch_youtube.sh <url> — tries subtitle extraction first, falls back to Groq audio transcription. Outputs transcript to stdout.

If it fails: tell the user exactly what failed and stop.

Instagram / TikTok / X video (instagram.com, tiktok.com, x.com with video):

yt-dlp is configured with Brave cookies — authenticated access, no extra flags needed.

# 1. Metadata first (always start here)
yt-dlp --print title --print description --print duration --print uploader --skip-download "<url>"

# 2. Download to tmp
yt-dlp -o "/tmp/extract-%(id)s.%(ext)s" "<url>"

# 3. Transcribe audio with local Whisper
ffmpeg -i /tmp/extract-<id>.mp4 -vn -acodec pcm_s16le -ar 16000 -ac 1 /tmp/extract-<id>-audio.wav
whisper /tmp/extract-<id>-audio.wav --model base --language en --output_format txt --output_dir /tmp/

# 4. Extract key frames (one every ~10 seconds)
mkdir -p /tmp/extract-frames
ffmpeg -i /tmp/extract-<id>.mp4 -vf "fps=1/10" -q:v 2 /tmp/extract-frames/<id>-%02d.jpg

# 5. Read frames visually — look for on-screen text, diagrams, handwritten notes, visual content
# 6. Synthesize: transcript + visuals + caption
# 7. Trash all temp files when done

For visual content (cinematography, design, art): frame extraction is critical — the visuals ARE the knowledge. For talking-head content: transcript carries most of the value, frames are supplementary.

Podcast / direct audio (MP3/M4A URL, SoundCloud, podcast episode):

yt-dlp handles most audio URLs natively. Use Groq transcription (requires GROQ_API_KEY) or local Whisper as fallback. For RSS feeds: extract the episode <enclosure> URL first, then treat as direct audio.

X/Twitter thread (x.com or twitter.com):

X blocks all unauthenticated access. Requires X_BEARER_TOKEN in environment. X API uses pay-per-use credits — each call costs credits from your balance.

Run scripts/fetch_twitter.sh <url> — fetches the thread via X API v2, filters to author tweets, outputs in chronological order.

If X_BEARER_TOKEN is not set: ask the user to paste the thread text directly. "X requires a paid API for access. Paste the thread text and I'll extract from that."

If the thread is older than 7 days (search returns empty): ask the user to paste the thread text. "This thread is older than 7 days — X's search API can't reach it. Paste the thread text and I'll extract from that."

Reconstruct thread as sequential blockquotes before extracting.

Web article (any HTTP/HTTPS URL, not YouTube, X, or social): Prefer defuddle: defuddle parse <url> --md — strips clutter, returns clean markdown.

Fallback to WebFetch if defuddle fails or isn't installed.

If both return garbage (login wall, JS-only rendering): ask the user to paste the article text.

Paywalled content: Note it clearly at the top:

Warning: Paywalled — only the free preview was accessible. Extraction is based on partial content.

Then extract whatever is accessible. Do not fabricate beyond the paywall.

Local file / PDF: Use the Read tool directly.

Step 2: Assess content quality

Before extracting, scan the raw content:

• Shallow (listicle, hype, no real methodology) — say so in 1-2 lines. Don't manufacture depth.
• News/announcement-only — extract only if real methodology is buried in it.
• Substantial — proceed.

Step 3: Extract knowledge

Target depth by insight density, not raw length.

First, assess density:

• High density (every minute has new ideas): technical talks, dense essays, practitioner deep-dives
• Medium density (mixed signal/filler): most interviews, conference talks, long-form articles
• Low density (mostly filler, few real insights): casual podcasts, rambling discussions

These are floors. Dense content warrants more. Below the floor = you're under-extracting. Low density + short content may not be worth extracting at all — say so.

Extraction Framework

Use whichever categories are present. Skip empty ones.

Mental Models & Frameworks

Ways of thinking. Decision heuristics. How experts frame situations differently.

Systematic Methods & Processes

Step-by-step techniques. Playbooks. Include sequence and reasoning behind each step.

Specific Techniques & Tactics

Named techniques, scripts, templates, prompt structures. Concrete and immediately applicable.

Key Numbers & Benchmarks

Every specific statistic, threshold, ratio, percentage, timeframe, quantity. Never omit or round.

Use Cases & Applications

Concrete examples — situation, action, result. Include vivid anecdotes even if specific to one person.

Principles & Heuristics

Underlying truths. Rules of thumb. "Always X, never Y" guidance.

Contrarian & Non-Obvious Insights

Challenges conventional wisdom. Only include if genuinely non-obvious.

Include practitioner honesty: when the speaker admits their practice contradicts their advice, or acknowledges failure.

Predictions & Future Signals

Forward-looking bets, timeline estimates, emerging trends. Preserve the reasoning chain.

Tools & Resources

Named tools, books, people, communities with organic use-case context. Strip affiliate/sponsored mentions.

Filtering Rules

Always strip: sponsored segments, ad reads, CTAs, self-promotion, filler intros/outros.

Strip time-sensitive noise: "[Tool] just launched" (unless the build methodology is the insight), pricing, availability dates, capability comparisons that will expire.

Always preserve: reasoning behind tool choices, prediction reasoning chains, historical context, every specific number, practitioner admissions, vivid examples, organic tool recommendations with context.

Output Format

Source: [title] — [URL]

Then categories as markdown headers (###). Bullet points for discrete insights, numbered lists for processes, blockquotes for sharp direct quotes.

Output Destination

By default, print the extraction to the conversation.

If the user says "save this" or "write this", ask where to save. Default to ./YYYY-MM-DD-<slugified-title>.md in the current working directory.

Edge Cases

• Multiple topics: Extract all. Separate with ---.
• Interview format: Extract from ALL participants. Attribute when it matters.
• Tutorial/how-to: Full process. Do not skip steps.
• Non-English: Extract in English. Keep original terms in parentheses when needed.
• 2hr+ content: Do not compress. Match depth to density tier.

Setup & Dependencies

Required

Transcription (at least one required for video/audio)

Whisper base model is fast and good enough for most content. Use small or medium for noisy audio or accents. Groq is tried first when available, Whisper is the local fallback.

Optional

Social media video access (Instagram, TikTok, X)

These platforms block anonymous downloads. To access them, yt-dlp needs cookies from a browser where you're logged in.

One-time setup:

# 1. Create config directory
mkdir -p ~/.config/yt-dlp

# 2. Export cookies from your browser (replace 'brave' with chrome, firefox, etc.)
yt-dlp --cookies-from-browser brave --cookies ~/.config/yt-dlp/cookies.txt --skip-download "https://www.instagram.com/reel/ANYTHING/"

# 3. Set global config so yt-dlp always uses the cookies
echo "--cookies $HOME/.config/yt-dlp/cookies.txt" > ~/.config/yt-dlp/config

Supported browsers: brave, chrome, firefox, edge, safari, opera, vivaldi.

You must be logged into the platforms you want to access in that browser. Cookies expire — if downloads start failing after a few weeks/months, re-run step 2.

Without cookies: YouTube, podcasts, articles, PDFs, and public content all work fine. Only authenticated platform videos (Instagram reels, TikTok, X videos) require cookies.