Abx Dl

Quick Start

• Prefer the repo-local command when you are already in this checkout:

uv sync
uv run abx-dl 'https://example.com'

• If you only need the published CLI:

uvx --from abx-dl abx-dl 'https://example.com'

Installing Plugin Dependencies

• Check missing binaries:

abx-dl plugins

• Pre-install dependency hooks before a real run:

abx-dl install wget ytdlp chrome

• If you skip pre-install, abx-dl <url> will auto-install missing dependencies by default. Use --no-install to disable that behavior.
• Dependency preflight is driven entirely by each plugin's config.json > required_binaries.
• on_BinaryRequest__* hooks emit only Binary records.

Running Against URLs

• Minimal run:

abx-dl 'https://example.com'

• Restrict to a subset of plugins:

abx-dl --plugins=title,wget,screenshot 'https://example.com'

• Write into an explicit output directory:

abx-dl --dir=./runs/example 'https://example.com'

Output Behavior

• By default, output is written into the current working directory.
• Expect an index.jsonl file plus plugin-specific subdirectories such as ./title/, ./wget/, ./screenshot/, and ./pdf/.
• For clean automation, create a throwaway working directory first or always pass --dir=....
• on_CrawlSetup__* hooks do not emit stdout JSONL records.
• on_Snapshot__* hooks emit ArchiveResult and may also emit Snapshot and Tag.

Example:

mkdir -p /tmp/abx-run && cd /tmp/abx-run
uvx --from abx-dl abx-dl --plugins=title,wget 'https://example.com'
find . -maxdepth 2 -type f | sort

Tuning Behavior

• Useful CLI flags:

  • --plugins=title,wget,..., --output=video,text/html, --dir=DIR, --timeout=120, --no-install

• Useful env vars:

  • TIMEOUT, USER_AGENT, CHECK_SSL_VALIDITY
  • LIB_DIR, PERSONAS_DIR, TMP_DIR
  • {PLUGIN}_BINARY, {PLUGIN}_ENABLED, {PLUGIN}_TIMEOUT
  • ABX_PLUGINS_DIR

Examples:

TIMEOUT=120 USER_AGENT='Mozilla/5.0 (abx-dl test)' abx-dl 'https://example.com'
CHROME_BINARY=/usr/bin/chromium abx-dl --plugins=screenshot,pdf 'https://example.com'
LIB_DIR=./.abx/lib PERSONAS_DIR=./.abx/personas abx-dl 'https://example.com'

• User config is stored in ~/.config/abx/config.env; runtime-derived cache entries are stored in ~/.config/abx/derived.env.
• config.env is user-owned only.
• derived.env stores derived binary cache entries such as resolved *_BINARY paths and ABX_INSTALL_CACHE.
• MachineService keeps those layers separate during a run instead of merging them together.
• Use abx-dl config to inspect or save defaults:

abx-dl config
abx-dl config --get TIMEOUT
abx-dl config --set TIMEOUT=120
abx-dl config --set WGET_ENABLED=false

Recommended Agent Workflow

1. Run in a clean directory or pass --dir=....
2. Install the CLI with uv run or uvx.
3. Run abx-dl plugins or abx-dl install ... if dependency state matters.
4. Run abx-dl '<url>' with any needed --plugins, --timeout, or env vars.
5. Inspect index.jsonl and the plugin subdirectories to confirm what was produced.