Video Production — Strategy Map

Kolbo MCP Tools (Active When kolbo auth login Is Done)

These are available as MCP tools — use them directly without any Python/API key setup:

Visual Analysis Workflow — MANDATORY for all video analysis

Step 1 is NOT optional. You cannot skip upload_media or construct the URL yourself.

Step 1: upload_media({ source: "/absolute/path/to/video.mp4" })
  → Returns: { url, thumbnail_url, ... }
  → Save the "url" field — this is the CDN URL you will pass to Gemini
  → NEVER use thumbnail_url (it's a JPG preview, not the video)

Step 2: chat_send_message({
  message: "Describe this video in detail. What is shown?",
  media_urls: ["<url from step 1>"]   ← must be an array, must be the "url" field
})
→ returns: { content: "..." }

❌ Common mistakes that break video analysis:

• Skipping upload_media and passing a local file path to chat_send_message — local paths don't work
• Using the transcription .txt URL as the media_urls value — Gemini needs the actual video CDN URL
• Using thumbnail_url instead of url from the upload_media response
• Calling transcribe_audio first then passing its output URL as the video — transcription gives text, not video

Omit model — Smart Select detects video/audio and auto-routes to Gemini. Sessions do NOT remember media between messages. On retry: reuse the same CDN url from step 1 (no re-upload needed) but always pass media_urls again.

Batch analysis (many videos): Pass model: "gemini-3.1-flash-lite-preview" explicitly for cheaper bulk runs.

For YouTube videos — download first with yt-dlp (see below), then follow steps 1–2 above.

Pipeline

Input: local video / YouTube URL / uploaded file

→ [DEFAULT] Visual Analysis: upload_media → chat_send_message (Gemini)
→ [EXPLICIT REQUEST] Transcription: transcribe_audio → SRT / text
→ [EDITING] FFmpeg: cut, silence removal, 9:16 conversion
→ [MOTION GRAPHICS] Remotion: compositions, captions, B-roll
→ Output: Premiere XML / DaVinci EDL / MP4s / SRT

APIs & Capabilities

Kolbo MCP tools need no API keys — auth is handled by kolbo auth login. FFmpeg/yt-dlp need to be installed locally on the machine.

YouTube / Social Media Download (yt-dlp)

Download video from YouTube, TikTok, Instagram, Twitter, etc.:

# Best quality MP4
yt-dlp -f "bestvideo[height<=1080][ext=mp4]+bestaudio/best" \
  --merge-output-format mp4 \
  -o "%(id)s.%(ext)s" <url>

# With subtitles
yt-dlp -f "bestvideo[height<=1080][ext=mp4]+bestaudio/best" \
  --write-auto-sub --sub-lang en --convert-subs srt \
  --merge-output-format mp4 \
  -o "%(id)s.%(ext)s" <url>

# Audio only (for transcription)
yt-dlp -f "bestaudio" --extract-audio --audio-format mp3 -o "%(id)s.%(ext)s" <url>

After download → upload to Kolbo CDN with upload_media → analyze visually with chat_send_message.

Key Rules

• FFmpeg on Windows: always copy inputs to tempfile.mkdtemp() first (handles spaces in paths)
• 9:16 shorts: blurred background + centered content — never crop the original
• Hebrew / RTL subtitles: ASS format, Heebo Bold font, \pos() for RTL rendering
• Video quality standard: -c:v libx264 -crf 18 -c:a aac -b:a 128k
• Image generation: prefer fal MCP server over Python scripts when available
• Silence removal: silencedetect -35dB:d=0.4 → trim+concat → atempo=1.14

Transcription

Use ElevenLabs Scribe for word-level SRT with speaker diarization:

import requests

def transcribe(audio_path, api_key, language="he"):
    with open(audio_path, "rb") as f:
        response = requests.post(
            "https://api.elevenlabs.io/v1/speech-to-text",
            headers={"xi-api-key": api_key},
            files={"file": f},
            data={"model_id": "scribe_v1", "language_code": language,
                  "timestamps_granularity": "word", "diarize": True}
        )
    return response.json()

9:16 Shorts — Blurred Background

filter_complex = (
    "[0:v]split[bg][fg];"
    "[bg]scale=1080:1920:force_original_aspect_ratio=increase,"
    "crop=1080:1920,gblur=sigma=40[blurred];"
    "[fg]scale=1080:1920:force_original_aspect_ratio=decrease,"
    "pad=1080:1920:(ow-iw)/2:(oh-ih)/2:color=black@0[front];"
    "[blurred][front]overlay=0:0"
)

Silence Removal

import subprocess, json

def detect_silence(video_path, noise_db=-35, duration=0.4):
    result = subprocess.run([
        "ffmpeg", "-i", video_path,
        "-af", f"silencedetect=noise={noise_db}dB:d={duration}",
        "-f", "null", "-"
    ], capture_output=True, text=True)
    # Parse silence_start/silence_end from stderr
    ...

RTL (Hebrew/Arabic) Subtitles

For comprehensive RTL subtitle handling, load the subtitle-production skill — it contains full patterns for:

• Simple SRT burn-in with Heebo font + Encoding=177
• ASS per-word positioning for karaoke (with PIL ~0.74 scale factor)
• Remotion RTL captions with CSS direction: rtl and all the flip rules
• RTL progress bar with FFmpeg geq filter

CRITICAL: Any inline ASS tag (\c, \K, \1c, etc.) between RTL words breaks Unicode bidi in libass — words render LTR. Use separate Dialogue lines per word instead.

For Remotion RTL layout rules (padding flips, transform-origin, gradient direction), load the typography-video skill.

Remotion Motion Graphics

For motion graphics rendering, use the remotion-best-practices skill for detailed Remotion patterns.

For cloud rendering via Remotion Lambda:

npx remotion lambda render <serve-url> <composition-id> --out output.mp4

Premiere Pro XML Timeline

def generate_premiere_xml(clips, output_path, fps=30):
    # Generate FCP7 XML compatible with Premiere Pro
    ...

Output Structure

Organize outputs per project:

<project>/
├── raw/          # original footage
├── transcripts/  # SRT, word-level JSON
├── clips/        # cut segments
├── shorts/       # 9:16 vertical versions
├── b-roll/       # generated B-roll images/videos
├── motion/       # Remotion compositions
└── export/       # final deliverables + XML timelines

Check Before Writing New Scripts

Before writing a new script, ask the user if they already have one for the task — they may have existing tools for clipping, silence removal, or subtitle burning.