Litellm

Standard workflow

1. Confirm objective:

• sdk: Python client usage and provider-specific model calls.
• proxy: gateway deployment, routing, auth, budgets, and observability.

2. Start from a minimal proxy config:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: azure/my_azure_deployment
      api_base: os.environ/AZURE_API_BASE
      api_key: os.environ/AZURE_API_KEY
      api_version: "2025-01-01-preview"
general_settings:
  master_key: sk-1234

3. Run and validate:

litellm --config /path/to/config.yaml --detailed_debug
curl -sS http://127.0.0.1:4000/health/liveliness
curl -sS http://127.0.0.1:4000/health/readiness
curl -sS -H "Authorization: Bearer sk-1234" http://127.0.0.1:4000/health

4. Validate request path compatibility:

curl -sS -X POST http://127.0.0.1:4000/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-1234' \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"ping"}]}'

5. Expand only as needed:

• Routing/load balancing/fallbacks.
• Virtual keys, budgets/rate limits, and model access controls.
• Logging/alerting/metrics integrations.

Outputs

When applying this skill, return:

• Exact commands run.
• Config changes with rationale.
• Verification evidence (health, readiness/liveliness, one real model request).
• Follow-up actions for production hardening (if requested).

References

• references/official-docs.md