AI Multimodal

GEMINI_API_KEY=key1
GEMINI_API_KEY_2=key2
GEMINI_API_KEY_3=key3

Features:

• Auto-rotates on rate limit (429/RESOURCE_EXHAUSTED) errors
• 60-second cooldown per key after rate limit
• Logs rotation events with --verbose flag
• Backward compatible: single key still works
• Works without external helper modules; centralized resolver support is optional

Quick Start

Verify setup: python scripts/check_setup.py Analyze media: python scripts/gemini_batch_process.py --files <file> --task <analyze|transcribe|extract> Generate content: python scripts/gemini_batch_process.py --task <generate|generate-video> --prompt "description" Convert docs to Markdown: python scripts/document_converter.py --input <file> Preflight media: python scripts/media_optimizer.py --input <file> --output <optimized-file>

Stdin support: The standalone CLI supports piping supported binary files through stdin (PNG/JPG/PDF/WAV/MP3/WEBP).

• cat image.png | python scripts/gemini_batch_process.py --task analyze --prompt "Describe this"
• python scripts/gemini_batch_process.py --files image.png --task analyze (traditional)

Models

• Image generation: imagen-4.0-generate-001 (standard), imagen-4.0-ultra-generate-001 (quality), imagen-4.0-fast-generate-001 (speed)
• Video generation: veo-3.1-generate-preview (8s clips with audio)
• Analysis: gemini-2.5-flash (recommended), gemini-2.5-pro (advanced)

Scripts

• gemini_batch_process.py: CLI orchestrator for transcribe|analyze|extract|generate|generate-video that auto-resolves API keys, picks sensible default models per task, streams files inline vs File API, and saves structured outputs (text/JSON/CSV/markdown plus generated assets) for Imagen 4 + Veo workflows.
• media_optimizer.py: ffmpeg/Pillow-based preflight tool that compresses/resizes/converts audio, image, and video inputs, enforces target sizes/bitrates, splits long clips into hour chunks, and batch-processes directories so media stays within Gemini limits.
• document_converter.py: Gemini-powered converter that uploads PDFs/images/Office docs, applies a markdown-preserving prompt, batches multiple files, auto-names outputs under docs/assets, and exposes CLI flags for model, prompt, auto-file naming, and verbose logging.
• check_setup.py: Interactive readiness checker that verifies directory layout, centralized env resolver, required Python deps, and GEMINI_API_KEY availability/format, then performs a live Gemini API call and prints remediation instructions if anything fails.

Use --help for options.

Runtime Surfaces

Keep these surfaces distinct:

• Bundled CLI
  • gemini_batch_process.py
  • document_converter.py
  • media_optimizer.py
  • check_setup.py
• Reference-only SDK features
  • TTS and native audio generation
  • image editing and multi-turn refinement
  • search-grounded image generation
  • YouTube analysis, video clipping, custom FPS, multi-video comparison
  • structured JSON schema output
  • live music generation with Lyria

Do not assume every capability documented in references/ is already exposed by the CLI.

References

Load for detailed guidance:

Limits

Formats: Audio (WAV/MP3/AAC, 9.5h), Images (PNG/JPEG/WEBP, 3.6k), Video (MP4/MOV, 6h), PDF (1k pages) Size: 20MB inline, 2GB File API Important:

• If you are going to generate a transcript of the audio, and the audio length is longer than 15 minutes, the transcript often gets truncated due to output token limits in the Gemini API response. To get the full transcript, you need to split the audio into smaller chunks (max 15 minutes per chunk) and transcribe each segment for a complete transcript.
• If you are going to generate a transcript of the video and the video length is longer than 15 minutes, use ffmpeg to extract the audio from the video, truncate the audio to 15 minutes, transcribe all audio segments, and then combine the transcripts into a single transcript.
• Image and video generation require billing; free tier has zero quota for those models.
• The current video generation CLI uses the first reference image as the opening frame and the second as the closing frame.
• --num-images is meaningful on the Imagen path; do not assume the same behavior for every Gemini image model. Transcription Output Requirements:
• Format: Markdown
• Metadata: Duration, file size, generated date, description, file name, topics covered, etc.
• Parts: from-to (e.g., 00:00-00:15), audio chunk name, transcript, status, etc.
• Transcript format:

  [HH:MM:SS -> HH:MM:SS] transcript content
  [HH:MM:SS -> HH:MM:SS] transcript content
  ...
  

Resources

• API Docs
• Pricing

Integration with Pulse

During pulse:executing, invoke this skill when beads require multimodal asset processing, media analysis, or content generation. Feed outputs back into the bead's canonical verification evidence at history/<feature>/verification/<bead-id>.md.

Write/Edit tools are available for extending or creating processing scripts, not for modifying reference documentation.