Guidance for creating, running, fixing, and promoting behavioral evaluations. Use when verifying agent decision logic, debugging failures, debugging prompt steering, or adding workspace regression tests.
Behavioral evaluations (evals) are tests that validate the agent's decision-making (e.g., tool choice) rather than pure functionality. They are critical for verifying prompt changes, debugging steerability, and preventing regressions.
[!NOTE] Single Source of Truth: For core concepts, policies, running tests, and general best practices, always refer to evals/README.md.
appEvalTest (). See .AppRigevalTest (TestRig). See creating.md.USUALLY_PASSES.ALWAYS_PASSES (locks in regression).Seed the workspace with necessary files using the files object to simulate a realistic scenario (e.g., NodeJS project with package.json).
Audit agent decisions using rig.setBreakpoint() (AppRig only) or index verification on rig.readToolLogs().
Run single tests locally with Vitest. Confirm stability locally before relying on CI workflows.
Detailed procedural guides: