Persistent NDJSON event logging with rotation, compression, and retention policies. Use this skill when: 1. You need persistent storage of event streams 2. You need log rotation and compression for long-term retention 3. You need to audit event history with timestamps 4. You need to export events for external analysis Do NOT use this skill when: - You need real-time event processing (use pilot-event-bus instead) - You need short-term replay (use pilot-event-replay instead) - You need filtered logs (use pilot-event-filter first, then log)
Persistent NDJSON logging of Pilot Protocol event streams with rotation and compression.
pilotctl --json subscribe <source> <topic> --timeout <seconds> | jq -c '.data.events[]' >> <log-file.ndjson>
jq -c --arg start "2026-04-08T00:00:00Z" --arg end "2026-04-08T23:59:59Z" \
'select(.timestamp >= $start and .timestamp <= $end)' events.ndjson
jq -c 'select(.topic | startswith("alerts."))' events.ndjson
[ "$(du -m "$log_file" | cut -f1)" -ge "$MAX_SIZE_MB" ] && mv "$log_file" "$log_file.$(date +%s)" && gzip "$log_file.$(date +%s)" &
find /var/log/pilot-events -name "events-*.ndjson" -mtime +1 -exec gzip {} \;
find /var/log/pilot-events -name "events-*.ndjson.gz" -mtime +90 -delete
#!/bin/bash
# Production event logging
SOURCE="${1:-production-app}"
LOG_DIR="/var/log/pilot-events/$SOURCE"
MAX_SIZE_MB=500
mkdir -p "$LOG_DIR"
log_file="$LOG_DIR/current.ndjson"
while true; do
pilotctl --json subscribe "$SOURCE" "*" --timeout 600 | jq -c '.data.events[]? // empty' | \
while IFS= read -r event; do
echo "$event" | jq -c '. + {logged_at: (now | todate)}' >> "$log_file"
size_mb=$(du -m "$log_file" 2>/dev/null | cut -f1)
if [ "${size_mb:-0}" -ge "$MAX_SIZE_MB" ]; then
rotated="$LOG_DIR/events-$(date +%Y%m%d-%H%M%S).ndjson"
mv "$log_file" "$rotated" && gzip "$rotated" &
fi
done
sleep 5
done
Requires pilot-protocol skill, jq (1.6+), gzip, and running daemon.