Track skill usage patterns to enable data-driven skill improvements. Opt-in tracking that logs minimal metadata to a central usage log for the skill-evolver to analyze.
Skill loaded → Is ~/.agents/skills/.tracking-disabled present?
├─ Yes → Skip silently
├─ No → Track AFTER execution completes
└─ Unclear → Ask user once, create .tracking-disabled if they decline
Outcome:
├─ Completed fully → "success"
├─ Failed (timeout, crash, permission) → "error:<code>"
└─ Partially completed (some steps done, some failed) → "partial"
After each skill invocation completes:
source ~/.agents/skills/skill-tracker/scripts/track.sh <skill> <outcome> [model] [notes]
Auto-detect skill name from the loaded skill's frontmatter. Fire and forget.
Disable tracking: touch ~/.agents/skills/.tracking-disabled
Each log entry is one compact JSON line:
{"skill":"plan-review-adversarial","ts":"2026-04-03T12:34:56Z","outcome":"success","model":"opus","notes":""}
Fields: skill (required, string), ts (required, ISO 8601 UTC), outcome (required, see below), model (optional, string), notes (optional, max 50 chars, escape quotes as \")
Error codes: timeout (exceeded time limit), permission (access denied), parse (input unparseable), collision (write conflict), other (unclassified)
Outcome definitions:
success — All steps completed, no errorspartial — Some steps completed, others skipped or failed non-critically (e.g., 3 of 5 sub-tasks done)error:<code> — Execution failed entirely (e.g., error:timeout)Example entries:
{"skill":"refactor","ts":"2026-04-03T14:22:01Z","outcome":"success","model":"sonnet"}
{"skill":"plan-review-adversarial","ts":"2026-04-03T14:30:15Z","outcome":"error:timeout","model":"opus","notes":"critic agent timed out after 120s"}
{"skill":"task-supervisor","ts":"2026-04-03T15:00:00Z","outcome":"partial","model":"sonnet","notes":"2 of 4 subtasks completed"}
# All analysis at once (preferred)
~/.agents/skills/skill-tracker/scripts/analyze.sh
# Or inline:
LOG=~/.agents/skills/.usage-log.jsonl
[ -s "$LOG" ] && { echo "=== Most Used ===" && awk -F'"' '{for(i=1;i<=NF;i++) if($i=="skill") print $(i+2)}' "$LOG" | sort | uniq -c | sort -rn; echo "=== Errors ===" && awk -F'"' '{for(i=1;i<=NF;i++) if($i=="skill") s=$(i+2); if($i=="outcome") o=$(i+2)} o~/^error/{print s}' "$LOG" | sort | uniq -c | sort -rn; echo "=== Outcomes ===" && awk -F'"' '{for(i=1;i<=NF;i++) if($i=="outcome") print $(i+2)}' "$LOG" | sort | uniq -c | sort -rn; } || echo "No entries"
# Validate integrity
[ -s "$LOG" ] && awk '/^\{.*"skill".*"ts".*"outcome".*\}$/' "$LOG" | wc -l && echo "valid entries"
Expected output:
=== Most Used ===
15 plan-review-adversarial
8 task-supervisor
=== Errors ===
2 plan-review-adversarial
=== Outcomes ===
24 success
2 error:timeout
1 partial
Minimum 10 entries per skill for reliable analysis. Sample first 20 entries if log >100 lines.
Skill execution → Determine outcome → track.sh → .usage-log.jsonl → skill-evolver reads → Prioritizes improvements
↑
.tracking-disabled (skip)
| Symptom | Cause | Fix |
|---|---|---|
| Log file not found | Never created | touch ~/.agents/skills/.usage-log.jsonl |
| Entries not appearing | No write permission | chmod u+w ~/.agents/skills/ |
| Malformed JSON | Unescaped quotes in notes | Script auto-escapes; or omit notes |
| Analysis returns nothing | Empty log or wrong path | Check $LOG path |
| Log file too large | Never rotated | See Control below |
| Tracking in tight loop | Bug in skill logic | Add dedup check, review skill |
Self-test: ~/.agents/skills/skill-tracker/scripts/track.sh --self-test
| Action | Command |
|---|---|
| Disable | touch ~/.agents/skills/.tracking-disabled |
| Enable | rm ~/.agents/skills/.tracking-disabled |
| Delete all data | rm ~/.agents/skills/.usage-log.jsonl |
| Rotate (>1MB) | mv ~/.agents/skills/.usage-log.jsonl ~/.agents/skills/.usage-log-$(date +%Y-%m).jsonl.gz && gzip ~/.agents/skills/.usage-log-*.jsonl 2>/dev/null; touch ~/.agents/skills/.usage-log.jsonl |
| Retention | Keep 3 months, then delete |
| Max entries | 10,000 (auto-truncates oldest) |
When skill-evolver runs Phase 0:
scripts/analyze.shscripts/analyze.sh for one-command analysisscripts/track.sh --self-test and --help