Sfx Catalog

1. Not searchable - Generic filenames, no metadata
2. Not reusable - Can't remember which sounds work for which scenes
3. Not learnable - No pattern recognition across projects

Solution: The sfx-catalog system makes sound effects discoverable, reusable, and learnable.

Core Features

1. Audio Analysis

Extracts technical characteristics from MP3 files:

• Duration, frequency profile, envelope (ADSR)
• Loudness metrics, harmonic content
• Automatic categorization (impact, ambient, foley, etc.)

2. Semantic Search

Natural language queries powered by memory integration:

./run.sh search "deep ominous thunder"
# Returns ranked results with similarity scores

3. Memory First

Learns from prior usage to improve recommendations:

./run.sh recall-usage "tense apartment entrance"
# Returns SFX successfully used in similar scenes before

4. Usage Tracking

Records context for every SFX selection:

./run.sh record-usage \
    --sfx-id abc123 \
    --project "Dark Horizon" \
    --scene "INT. APARTMENT - Sarah enters cautiously" \
    --rationale "Adds tension to entrance"

5. On-Demand Generation

Creates missing sound effects via AI when library lacks options:

./run.sh generate "metallic door slam" --duration 2.5 --ingest

Integration Points

create-movie

Automatically selects and applies SFX during movie generation:

# In create-movie workflow
from sfx_catalog.query_engine import SFXQueryEngine

engine = SFXQueryEngine(scope="horus_lore")

# Memory First: Check for prior usage
sfx = engine.recall_usage(scene_description)

# Fallback: Semantic search
if not sfx:
    sfx = engine.search(query, categories, duration_range)

# Record for future learning
engine.record_usage(sfx_id, project, scene, timestamp, rationale)

create-storyboard

Suggests sound effects during storyboard planning:

# During storyboard phase
suggestions = suggest_sfx_for_shot(shot)
# Returns natural language recommendations with alternatives

Architecture

High-level system design:

┌─────────────────┐
│ 166 MP3 Files   │
└────────┬────────┘
         │
         ▼
┌─────────────────┐     ┌──────────────────┐
│ Audio Analyzer  │────►│ Content Class.   │
│ (librosa)       │     │ (rule-based)     │
└─────────────────┘     └────────┬─────────┘
                                 │
                                 ▼
                        ┌──────────────────┐
                        │ Metadata Gen.    │
                        │ (LLM-assisted)   │
                        └────────┬─────────┘
                                 │
                                 ▼
                        ┌──────────────────┐
                        │ JSON Manifest    │
                        └────────┬─────────┘
                                 │
                                 ▼
┌─────────────────────────────────────────┐
│          ArangoDB Memory                │
│  ┌────────────┐  ┌────────────────┐    │
│  │sfx_library │  │  sfx_usage     │    │
│  │(catalog)   │  │  (tracking)    │    │
│  └────────────┘  └────────────────┘    │
│  ┌────────────┐                         │
│  │sfx_generated                         │
│  │(cache)     │                         │
│  └────────────┘                         │
└─────────────────────────────────────────┘
         │
         ▼
┌─────────────────┐
│ Query Engine    │ ◄──── create-movie, create-storyboard
│ (multi-strategy)│
└─────────────────┘

Deep Dive: See ARCHITECTURE.md for detailed component specifications.

Memory Schema

SFX data is stored in ArangoDB with three main collections:

sfx_library - Sound Effect Catalog

{
  "_key": "sfx_abc123",
  "file_path": "/mnt/storage12tb/media/sfx/...",
  "description": "Deep, punchy impact with quick attack",
  "categories": ["impact", "low_frequency"],
  "audio_features": {
    "duration_seconds": 2.34,
    "envelope": {...},
    "frequency_profile": {...}
  },
  "embedding": [...],
  "usage_count": 5
}

sfx_usage - Usage History

{
  "sfx_id": "sfx_library/sfx_abc123",
  "project_name": "Dark Horizon",
  "scene_description": "INT. APARTMENT - Tense entrance",
  "timestamp_in_scene": 2.5,
  "rationale": "Adds atmosphere and tension"
}

sfx_generated - Generation Cache

{
  "prompt": "metallic door creak",
  "file_path": "/mnt/.../generated/door_creak.mp3",
  "reuse_count": 3,
  "user_approved": true
}

Deep Dive: See MEMORY_SCHEMA.md for complete schema, indices, and query patterns.

CLI Commands

Core Workflow

# 1. Catalog audio files
./run.sh catalog <directory> --output manifest.json

# 2. Ingest into memory
./run.sh ingest manifest.json

# 3. Search catalog
./run.sh search "explosion boom" --categories impact --duration 2-5

# 4. Check system status
./run.sh status

Advanced Operations

# Record usage after selection
./run.sh record-usage \
    --sfx-id abc123 \
    --project "Dark Horizon" \
    --scene "INT. WAREHOUSE - Explosion" \
    --timestamp 5.2

# Recall prior usage (Memory First)
./run.sh recall-usage "warehouse explosion scene"

# Find similar sounds
./run.sh similar sfx_abc123 --threshold 0.80

# Generate missing SFX
./run.sh generate "deep rumbling thunder" --duration 4.0 --ingest

# View usage statistics
./run.sh stats --type categories

Full Reference: See API.md for complete command documentation and Python API.

Python API

For programmatic integration:

from sfx_catalog import SFXQueryEngine

engine = SFXQueryEngine(scope="horus_lore")

# Memory First pattern
prior_usage = engine.recall_usage(
    scene_description="tense entrance scene",
    threshold=0.7
)

# Semantic search with filters
results = engine.search(
    query="door creak",
    categories=["foley"],
    duration_range=(1.0, 3.0),
    k=5
)

# Record usage for learning
engine.record_usage(
    sfx_id="sfx_abc123",
    project_name="Dark Horizon",
    scene_description="INT. APARTMENT - Sarah enters",
    timestamp_in_scene=2.5,
    rationale="Perfect tension builder"
)

# Generate if needed
generated = engine.generate_sfx(
    prompt="metallic door slam",
    duration=2.0,
    check_cache=True,  # Avoid duplicate generation
    ingest=True        # Add to catalog
)

Data Flow

1. Cataloging Workflow

User SFX files
    ↓
Audio Analysis (librosa)
    ↓
Content Classification (rule-based)
    ↓
Description Generation (LLM optional)
    ↓
JSON Manifest
    ↓
Memory Ingestion (ArangoDB)
    ↓
Searchable Catalog

2. Query Workflow

User/Agent Query
    ↓
Query Engine
    ├─► Strategy 1: Check memory (prior usage)
    ├─► Strategy 2: Semantic search (embeddings)
    └─► Strategy 3: Generate (if missing)
    ↓
Ranked Results
    ↓
User Selection
    ↓
Usage Recording (for learning)

3. Learning Workflow

Every SFX Use
    ↓
Record: project, scene, rationale
    ↓
Store in sfx_usage collection
    ↓
Build patterns over time
    ↓
Improve future recommendations

Technology Stack

Audio Processing

• librosa - Audio feature extraction
• soundfile - Audio I/O
• scipy - Signal processing
• numpy - Numerical operations

Memory & Search

• ArangoDB - Graph database (via memory skill)
• python-arango - Database client
• Embeddings - Via embedding skill

Generation (Optional)

• Stable Audio Open - Text-to-audio generation
• PyTorch - ML framework

CLI & UX

• typer - CLI framework
• rich - Terminal formatting
• task-monitor - Progress reporting

Performance

Cataloging

• Single file: 2-3 seconds (audio analysis + description)
• 166 files: 5-10 minutes (parallel processing, 4 workers)

Queries

• Memory search: <100ms (indexed ArangoDB)
• Semantic search: <200ms (pre-computed embeddings)
• Total query time: <500ms end-to-end

Generation

• Text-to-audio: 30-60 seconds per 3-second clip
• Cache lookup: <50ms (check before generating)

Storage Requirements

• Library files: ~166MB (existing MP3 files)
• Metadata: ~50MB (JSON + ArangoDB)
• Embeddings: ~2MB (166 × 384 dimensions)
• Generated SFX: ~1GB over time
• Total: ~1.2GB

Dependencies

Required

• Python 3.11+
• ArangoDB (existing memory system)
• librosa, soundfile, scipy, numpy
• memory skill
• embedding skill

Optional

• scillm skill (for LLM descriptions)
• Stable Audio Open (for generation)
• GPU with 8GB+ VRAM (accelerates generation)

Configuration

Environment variables (optional, has defaults):

# Memory system
export MEMORY_ROOT="$HOME/workspace/experiments/memory"
export ARANGO_HOST="127.0.0.1"
export ARANGO_PORT="8529"

# SFX catalog data
export SFX_DATA_DIR="$HOME/.pi/sfx-catalog"

# LLM for descriptions (optional)
export SFX_LLM_MODEL="qwen2.5-coder:7b"
export SFX_LLM_PROVIDER="ollama"

# Audio generation (optional)
export STABLE_AUDIO_DEVICE="cuda:0"  # or "cpu"

Installation

cd .pi/skills/sfx-catalog

# Install dependencies
uv sync

# Run sanity checks
./sanity/run_all.sh

# Verify memory connection
./run.sh status

Usage Examples

Example 1: Initial Library Cataloging

# Catalog the entire library
./run.sh catalog /mnt/storage12tb/media/sfx/ \
    --output library_manifest.json \
    --parallel 4

# Review manifest
cat library_manifest.json | jq '.items[0]'

# Ingest into memory
./run.sh ingest library_manifest.json

# Test search
./run.sh search "impact" --limit 3

Example 2: Finding SFX for a Scene

# Search for thunder sounds
./run.sh search "deep rumbling thunder" \
    --categories ambient \
    --duration 3-8

# Preview top result (if ffplay/mpv installed)
./run.sh search "thunder" --play-top

Example 3: Recording Usage

# After selecting SFX for a scene
./run.sh record-usage \
    --sfx-id sfx_abc123 \
    --project "Storm Chaser" \
    --scene "EXT. FIELD - Dark clouds gather, distant thunder" \
    --timestamp 12.5 \
    --rationale "Sets ominous mood, foreshadows the storm"

Example 4: Memory First Pattern

# Working on a new tense scene
./run.sh recall-usage "tense interior entrance quiet footsteps"

# Returns SFX used successfully in similar scenes:
# Result 1: footsteps_hardwood_slow.mp3 (used in "Dark Horizon" INT. APARTMENT)
#   Rationale: "Built tension perfectly during cautious entrance"
#   Score: 0.87

Example 5: Generating Missing SFX

# Library doesn't have "sci-fi computer beep"
./run.sh generate "futuristic computer beep, clean, short" \
    --duration 0.5 \
    --ingest

# Generated SFX is now searchable:
./run.sh search "computer beep"

Testing

# Run all sanity checks
./sanity/run_all.sh

# Individual checks
./sanity/test_audio_analysis.sh   # Verify librosa
./sanity/test_memory.sh           # Verify ArangoDB
./sanity/test_search.sh           # Verify queries work

# Unit tests
uv run pytest tests/

# Integration tests
uv run pytest tests/integration/

Troubleshooting

"ArangoDB connection failed"

# Check if ArangoDB is running
systemctl status arangodb3

# Verify memory skill works
cd ../memory
./run.sh status

"No module named 'librosa'"

# Reinstall dependencies
uv sync --reinstall

"Search returns no results"

# Check if catalog was ingested
./run.sh status

# Re-ingest if needed
./run.sh ingest library_manifest.json

"Generation is too slow"

# Check if using GPU
./run.sh status --verbose

# Use shorter duration or lower quality
./run.sh generate "thunder" --duration 2.0 --steps 50

Roadmap

Phase 1: Foundation (MVP) - Weeks 1-3

• ✅ Audio analysis with librosa
• ✅ Rule-based classification
• ✅ Memory integration
• ✅ CLI interface

Phase 2: Integration - Weeks 4-5

• ⏳ Usage tracking
• ⏳ Query engine (multi-strategy)
• ⏳ create-movie integration
• ⏳ create-storyboard integration

Phase 3: Generation - Weeks 6-7

• ⏳ Stable Audio integration
• ⏳ Generation caching
• ⏳ Similarity graph

Phase 4: Enhancement - Weeks 8-9

• ⏳ Advanced classification (ML-based)
• ⏳ Usage analytics
• ⏳ Performance optimization

Full Plan: See ROADMAP.md for detailed implementation timeline.

Contributing

This skill follows the conventions in ../CONVENTIONS.md:

• Code: Stored in .pi/skills/sfx-catalog/
• Data: Stored in ~/.pi/sfx-catalog/ (persistent across syncs)
• Progress: Reported to task-monitor for long operations
• Memory: Uses horus_lore scope via memory skill

Documentation

• ARCHITECTURE.md - System design and component specs
• MEMORY_SCHEMA.md - ArangoDB collections and queries
• API.md - Complete CLI and Python API reference
• ROADMAP.md - Implementation plan and timeline

License

Part of Horus's filmmaking toolkit. For private use.

Status: Design phase complete, ready for implementation.

Next Step: Begin Phase 1 implementation (audio analysis + cataloging).