Promptfoo Init

Generate a complete promptfooconfig.yaml configuration from a task description. The output is a ready-to-run config with system prompt, dataset, assertions, and provider setup. Also supports creating targeted evals from user requests (e.g., "create an eval that checks if the model refuses harmful requests").

Workflow

1. Gather requirements

Ask the user (concisely, in one message):

• Task: What does the LLM need to do? (e.g., "classify support tickets", "extract entities from text", "translate English to Spanish")
• Input: What does the user input look like? (e.g., "a customer email", "a JSON object with name and address")
• Expected output: What should the LLM produce? (e.g., "a category label", "JSON with extracted fields", "translated text")
• Provider: Which LLM provider? (e.g., OpenAI, Anthropic)
• Any extra requirements: Output format constraints, tone, special instructions

If the user already provided some of this context, skip those questions.

2. Generate the config