Video Frame Reader

Linux/Mac — venv:

cd {baseDir}/scripts
python3 -m venv venv
source venv/bin/activate
pip install Pillow numpy --quiet

3. Extract Keyframes

Windows (conda):

conda run -n "media tools" python "{baseDir}/scripts/extract_keyframes.py" "<video_path>" --method scene --ensure-last

Linux/Mac (venv):

source {baseDir}/scripts/venv/bin/activate
# Fast default (scene detection) with optional last-frame inclusion
python3 {baseDir}/scripts/extract_keyframes.py "<video_path>" --method scene --ensure-last

4. Analyze Frames with Mistral Vision (Parallel)

After user approval, run analyze_frames_mistral.py with all frame paths. The script batches every 4 consecutive frames into one Mistral API call and fires all batches in parallel (e.g. 40 frames → 10 simultaneous API calls).

Setup (first time only):

# Linux/Mac
source {baseDir}/scripts/venv/bin/activate
pip install mistralai --quiet

# Windows (conda)
conda install -n "media tools" mistralai --quiet -y

Run analysis:

# Linux/Mac
source {baseDir}/scripts/venv/bin/activate
python3 {baseDir}/scripts/analyze_frames_mistral.py \
  frame_001.jpg frame_002.jpg ... frame_040.jpg \
  --prompt "User intent: {Intent from Step 1}. Describe what is happening across these consecutive video frames." \
  --format json \
  --output analysis.json

# Windows (conda)
conda run -n "media tools" python "{baseDir}/scripts/analyze_frames_mistral.py" ^
  frame_001.jpg frame_002.jpg ... frame_040.jpg ^
  --prompt "User intent: {Intent from Step 1}. Describe what is happening across these consecutive video frames." ^
  --format json ^
  --output analysis.json

Key options:

• --model — default mistral-large-latest (Mistral Large 3); use pixtral-large-latest or pixtral-12b-2409 for vision-only alternatives
• --workers N — cap parallel calls (default: all batches at once)
• --format json|text — JSON for structured pickup, text for readability
• MISTRAL_API_KEY — read from env or --api-key flag

The script reads the JSON output and you merge the results[].description fields into a final chronological report.

Benefits of this approach:

• All batches fire simultaneously — 40 frames analyzed as fast as 4
• mistral-large-latest sees exactly 4 consecutive frames per call for temporal context
• Pure Python + mistralai SDK, no Claude subagents needed
• Results arrive in original frame order regardless of completion order

Tip: Pass --env /path/to/.env.local if your project stores MISTRAL_API_KEY in .env.local instead of .env.

Options

Token Reduction Example

Windows (conda):

# More aggressive reduction
conda run -n "media tools" python "{baseDir}/scripts/extract_keyframes.py" video.mp4 --method scene -t 0.2 -q 20 -s 0.2 --ensure-last

# Similarity method
conda run -n "media tools" python "{baseDir}/scripts/extract_keyframes.py" video.mp4 --method similarity -t 0.85 -q 30 -s 0.3 -w 6

Linux/Mac (venv):

# More aggressive reduction (lower threshold, quality, and size)
python3 {baseDir}/scripts/extract_keyframes.py video.mp4 --method scene -t 0.2 -q 20 -s 0.2 --ensure-last

# Similarity method (slower, more precise)
python3 {baseDir}/scripts/extract_keyframes.py video.mp4 --method similarity -t 0.85 -q 30 -s 0.3 -w 6