Stable Diffusion Image Generation

Before choosing, ask:

• What VRAM is available? (SD 1.5: 4GB, SDXL: 8GB, Flux: 24GB, or use enable_model_cpu_offload() to halve)
• Do they need a specific LoRA? (Check which base model it targets — using a LoRA on the wrong base silently produces garbage)
• Is speed or quality the priority?

Scheduler selection (the non-obvious part)

Claude knows the scheduler list. Here's what it doesn't know:

• DPMSolverMultistep is the default safe choice (20 steps, excellent quality). But it fails with LCM — you MUST use LCMScheduler with LCM LoRAs.
• EulerAncestral adds stochasticity — better for creative exploration but non-deterministic even with fixed seed.
• DDIM is the only scheduler that supports meaningful interpolation between latents (for morphing/animation).
• UniPC matches DPMSolver quality in fewer steps but has compatibility issues with some custom models.

Guidance scale sweet spots (not in docs)

NEVER do

• NEVER use guidance_scale > 2 with LCM — it will produce noise/artifacts. LCM was trained with guidance_scale=1.0.
• NEVER mix LoRA base models — a LoRA trained on SD 1.5 loaded into SDXL produces corrupted output with no error message.
• NEVER set height/width that aren't multiples of 8 — silent misalignment causes border artifacts.
• NEVER skip pipe.vae = pipe.vae.to(dtype=torch.float32) when getting black images — VAE is numerically unstable in fp16 on many models (especially SDXL).
• NEVER use num_inference_steps < 15 without LCM/Turbo — standard schedulers need minimum ~20 steps for coherent output.
• NEVER forget to .fuse_lora() or set scale after loading a LoRA — without this, the LoRA has zero effect and you'll debug for hours.
• NEVER use enable_sequential_cpu_offload() in production — it's ~3x slower than enable_model_cpu_offload(). Use it only when the latter still OOMs.
• NEVER exceed 77 tokens in a prompt for SD 1.5/SDXL — tokens beyond 77 are silently truncated. Use prompt weighting (word:1.5) instead of repetition.
• NEVER assume safety_checker=None is safe for production — disabling it removes NSFW filtering. Keep it on for user-facing apps.

ControlNet: which preprocessor for which task

img2img strength calibration

• 0.3-0.4: Color/style shift, structure preserved (touch-up)
• 0.5-0.7: Significant transformation, composition kept (style transfer)
• 0.8-1.0: Near-complete regeneration (only vague shapes kept)
• The common mistake: setting strength too high for edits. Start at 0.3 and increase.

Memory optimization decision order

Apply in this order, stop when VRAM fits:

1. torch_dtype=torch.float16 (always do this — 50% savings, no quality loss)
2. pipe.enable_model_cpu_offload() (moderate slowdown, large savings)
3. pipe.enable_vae_slicing() + pipe.enable_vae_tiling() (needed for >1024px)
4. pipe.enable_attention_slicing() (small extra savings)
5. pipe.enable_xformers_memory_efficient_attention() (if xformers available)
6. pipe.enable_sequential_cpu_offload() (last resort — very slow)

References

Load these only when needed for the specific task:

• Advanced Usage — MANDATORY before: building custom pipelines from components, writing custom denoising loops, fine-tuning (DreamBooth/LoRA/Textual Inversion), IP-Adapter, SDXL Refiner, T2I-Adapter, quantization, production deployment, callbacks, multi-GPU. Do NOT load for standard generation, img2img, inpainting, or ControlNet — the SKILL.md body is sufficient.

• Troubleshooting — MANDATORY when: encountering errors (CUDA OOM, black images, package conflicts, LoRA loading failures, ControlNet not conditioning, hub download issues). Do NOT load proactively — only when a specific error needs debugging.