ASCII Video Production Pipeline

Stack

Single self-contained Python script per project. No GPU.

Pipeline Architecture (v2)

Every mode follows the same 6-stage pipeline. See references/architecture.md for implementation details, references/scenes.md for scene protocol, and references/composition.md for multi-grid composition and tonemap.

┌─────────┐   ┌──────────┐   ┌───────────┐   ┌──────────┐   ┌─────────┐   ┌────────┐
│ 1.INPUT  │→│ 2.ANALYZE │→│ 3.SCENE_FN │→│ 4.TONEMAP │→│ 5.SHADE  │→│ 6.ENCODE│
│ load src │  │ features  │  │ → canvas   │  │ normalize │  │ post-fx  │  │ → video │
└─────────┘   └──────────┘   └───────────┘   └──────────┘   └─────────┘   └────────┘

1. INPUT — Load/decode source material (video frames, audio samples, images, or nothing)
2. ANALYZE — Extract per-frame features (audio bands, video luminance/edges, motion vectors)
3. SCENE_FN — Scene function renders directly to pixel canvas (uint8 H,W,3). May internally compose multiple character grids via _render_vf() + pixel blend modes. See references/composition.md
4. TONEMAP — Percentile-based adaptive brightness normalization with per-scene gamma. Replaces linear brightness multipliers. See references/composition.md § Adaptive Tonemap
5. SHADE — Apply post-processing ShaderChain + FeedbackBuffer. See references/shaders.md
6. ENCODE — Pipe raw RGB frames to ffmpeg for H.264/GIF encoding

Creative Direction

Every project should look and feel different. The references provide a vocabulary of building blocks — don't copy them verbatim. Combine, modify, and invent.

Aesthetic Dimensions to Vary

Per-Section Variation

Never use the same config for the entire video. For each section/scene/quote:

• Choose a different background effect (or compose 2-3)
• Choose a different character palette (match the mood)
• Choose a different color strategy (or at minimum a different hue)
• Vary shader intensity (more bloom during peaks, more grain during quiet)
• Use different particle types if particles are active

Project-Specific Invention

For every project, invent at least one of:

• A custom character palette matching the theme
• A custom background effect (combine/modify existing ones)
• A custom color palette (discrete RGB set matching the brand/mood)
• A custom particle character set

Workflow

Step 1: Determine Mode and Gather Requirements

Establish with user:

• Input source — file path, format, duration
• Mode — which of the 6 modes above
• Sections — time-mapped style changes (timestamps → effect names)
• Resolution — landscape 1920x1080 (default), portrait 1080x1920, square 1080x1080 @ 24fps; GIFs typically 640x360 @ 15fps
• Style direction — dense/sparse, bright/dark, chaotic/minimal, color palette
• Text/branding — easter eggs, overlays, credits, themed character sets
• Output format — MP4 (default), GIF, PNG sequence
• Aspect ratio — landscape (16:9), portrait (9:16 for TikTok/Reels/Stories), square (1:1 for IG feed)

Step 2: Detect Hardware and Set Quality

Before building the script, detect the user's hardware and set appropriate defaults. See references/optimization.md § Hardware Detection.

hw = detect_hardware()
profile = quality_profile(hw, target_duration, user_quality_pref)
log(f"Hardware: {hw['cpu_count']} cores, {hw['mem_gb']:.1f}GB RAM")
log(f"Render: {profile['vw']}x{profile['vh']} @{profile['fps']}fps, {profile['workers']} workers")

Never hardcode worker counts, resolution, or CRF. Always detect and adapt.

Step 3: Build the Script

Write as a single Python file. Major components:

1. Hardware detection + quality profile — see references/optimization.md
2. Input loader — mode-dependent; see references/inputs.md
3. Feature analyzer — audio FFT, video luminance, or pass-through
4. Grid + renderer — multi-density character grids with bitmap cache; _render_vf() helper for value/hue field → canvas
5. Character palettes — multiple palettes chosen per project theme; see references/architecture.md
6. Color system — HSV + discrete RGB palettes as needed; see references/architecture.md
7. Scene functions — each returns canvas (uint8 H,W,3) directly. May compose multiple grids internally via pixel blend modes. See references/scenes.md + references/composition.md
8. Tonemap — adaptive brightness normalization with per-scene gamma; see references/composition.md
9. Shader pipeline — ShaderChain + FeedbackBuffer per-section config; see references/shaders.md
10. Scene table + dispatcher — maps time ranges to scene functions + shader/feedback configs; see references/scenes.md
11. Parallel encoder — N-worker batch clip rendering with ffmpeg pipes
12. Main — orchestrate full pipeline

Step 4: Handle Critical Bugs

Font Cell Height (macOS Pillow)

textbbox() returns wrong height. Use font.getmetrics():

ascent, descent = font.getmetrics()
cell_height = ascent + descent  # correct

ffmpeg Pipe Deadlock

Never use stderr=subprocess.PIPE with long-running ffmpeg. Redirect to file:

stderr_fh = open(err_path, "w")
pipe = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL, stderr=stderr_fh)

Brightness — Use tonemap(), Not Linear Multipliers

ASCII on black is inherently dark. This is the #1 visual issue. Do NOT use linear * N brightness multipliers — they clip highlights and wash out the image. Instead, use the adaptive tonemap function from references/composition.md:

def tonemap(canvas, gamma=0.75):
    """Percentile-based adaptive normalization + gamma. Replaces all brightness multipliers."""
    f = canvas.astype(np.float32)
    lo = np.percentile(f, 1)          # black point (1st percentile)
    hi = np.percentile(f, 99.5)       # white point (99.5th percentile)
    if hi - lo < 1: hi = lo + 1
    f = (f - lo) / (hi - lo)
    f = np.clip(f, 0, 1) ** gamma     # gamma < 1 = brighter mids
    return (f * 255).astype(np.uint8)

Pipeline ordering: scene_fn() → tonemap() → FeedbackBuffer → ShaderChain → ffmpeg

Per-scene gamma overrides for destructive effects:

• Default: gamma=0.75
• Solarize scenes: gamma=0.55 (solarize darkens above-threshold pixels)
• Posterize scenes: gamma=0.50 (quantization loses brightness range)
• Already-bright scenes: gamma=0.85

Additional brightness best practices:

• Dense animated backgrounds — never flat black, always fill the grid
• Vignette minimum clamped to 0.15 (not 0.12)
• Bloom threshold lowered to 130 (not 170) so more pixels contribute to glow
• Use screen blend mode (not overlay) when compositing dark ASCII layers — overlay squares dark values: 2 * 0.12 * 0.12 = 0.03

Font Compatibility

Not all Unicode characters render in all fonts. Validate palettes at init:

for c in palette:
    img = Image.new("L", (20, 20), 0)
    ImageDraw.Draw(img).text((0, 0), c, fill=255, font=font)
    if np.array(img).max() == 0:
        log(f"WARNING: char '{c}' (U+{ord(c):04X}) not in font, removing from palette")

Step 4b: Per-Clip Architecture (for segmented videos)

When the video has discrete segments (quotes, scenes, chapters), render each as a separate clip file. This enables:

• Re-rendering individual clips without touching the rest (--clip q05)
• Faster iteration on specific sections
• Easy reordering or trimming in post

segments = [
    {"id": "intro", "start": 0.0, "end": 5.0, "type": "intro"},
    {"id": "q00", "start": 5.0, "end": 12.0, "type": "quote", "qi": 0, ...},
    {"id": "t00", "start": 12.0, "end": 13.5, "type": "transition", ...},
    {"id": "outro", "start": 208.0, "end": 211.6, "type": "outro"},
]

from concurrent.futures import ProcessPoolExecutor, as_completed
with ProcessPoolExecutor(max_workers=hw["workers"]) as pool:
    futures = {pool.submit(render_clip, seg, features, path): seg["id"]
               for seg, path in clip_args}
    for fut in as_completed(futures):
        fut.result()

CLI: --clip q00 t00 q01 to re-render specific clips, --list to show segments, --skip-render to re-stitch only.

Step 5: Render and Iterate

Performance targets per frame:

Fast iteration: render single test frames to check brightness/layout before full render:

canvas = render_single_frame(frame_index, features, renderer)
Image.fromarray(canvas).save("test.png")

Brightness verification: sample 5-10 frames across video, check mean > 8 for ASCII content.

References