Strip clutter from web pages before ingesting into the wiki. Removes ads, navigation, headers, footers, and boilerplate: leaving clean readable markdown that saves 40-60% tokens. Triggers on: defuddle, clean this page, strip this url, fetch and clean, clean web content before ingesting, strip ads, remove clutter, clean URL content, readable markdown from URL.
Defuddle extracts the meaningful content from a web page and drops everything else: ads, cookie banners, nav bars, related articles, footers, social sharing buttons. What remains is the article body as clean markdown.
Use this before any URL ingestion. It is optional but strongly recommended. It cuts token usage by 40-60% on typical web articles and produces cleaner wiki pages.
npm install -g defuddle-cli
Verify: defuddle --version
defuddle https://example.com/article
Outputs clean markdown to stdout.
defuddle https://example.com/article > .raw/articles/article-slug-$(date +%Y-%m-%d).md
After running defuddle, prepend the source URL and fetch date:
SLUG="article-slug-$(date +%Y-%m-%d)"
{ echo "---"; echo "source_url: https://example.com/article"; echo "fetched: $(date +%Y-%m-%d)"; echo "---"; echo ""; defuddle https://example.com/article; } > .raw/articles/$SLUG.md
defuddle page.html
Use defuddle when:
Skip defuddle when:
If defuddle is not installed, check:
which defuddle 2>/dev/null || echo "not installed"
If not installed: use WebFetch directly. The content will be less clean but still workable.
The /wiki-ingest skill checks for defuddle automatically when a URL is passed. You do not need to run defuddle manually before ingesting a URL. The ingest skill will call it if available.
To manually clean a page and save before ingesting:
ingest .raw/articles/[slug].md