Reduce token consumption by 30-50% using deduplication, compression, relevance filtering, and truncation. Composes with omni-link's existing token pruner for maximum efficiency. Use when context windows are tight or costs need optimization.
Use this skill when token budgets are tight or when you want to reduce costs without losing signal.
The TokenOptimizer applies four strategies in order:
Removes near-duplicate content. Two entries with ≥85% word overlap are collapsed to one.
Drops content that scores below the relevance threshold. Technical content (code, errors, APIs) scores higher; empty/whitespace-only content scores zero.
Content exceeding the maximum character length is truncated with an indicator.
The TokenOptimizer runs before omni-link's pruneToTokenBudget. The pipeline is:
raw content → TokenOptimizer (dedup, compress, filter) → omni-link pruner (priority-ranked section trimming)
This layered approach achieves higher savings than either system alone.