Heavy File Ingestion

Process

1. Identify the file path, extension, and rough size.
2. Run the converter script instead of reading the original file directly:

uv run \
  --with pdfplumber \
  --with python-docx \
  --with python-pptx \
  --with openpyxl \
  python skills/heavy-file-ingestion/scripts/convert_heavy_file.py /absolute/path/to/file.ext

1. If you already have markitdown installed and want to prefer it for PDF or DOCX conversion, rerun with:

python skills/heavy-file-ingestion/scripts/convert_heavy_file.py /absolute/path/to/file.ext --prefer markitdown

1. Read the generated index.md first.
2. Only read the extracted markdown or CSV outputs that the index says are worth reading.
3. If the index flags weak extraction, use a cheap fallback:
   • Try an alternate deterministic converter
   • Use a small model to rebuild only the structure or outline from the extracted artifact
   • Escalate to a stronger model only when the cheaper passes still leave critical ambiguity

Output

The skill should leave behind:

• A deterministic artifact the agent can work from
• index.md with file counts, structure hints, preview lines, and a recommended next step
• index.json with the same information in machine-friendly form
• Warnings when the deterministic pass is not trustworthy enough for direct reasoning

Notes

• Prefer the bundled script over rewriting ad hoc conversion code each time.
• Do not treat "sub-agent" as the default answer to messy files. A cheap deterministic pass beats a cheap model when the task is conversion, counting, routing, or indexing.
• For scanned PDFs, image-heavy decks, or bizarre layouts, the deterministic pass is still useful because it tells you that a fallback is needed before you waste a stronger model on the original file.
• Use references/open-source-stack.md when you need to choose a better extractor or explain why one was picked.