Eval | Skills Pool

技能檔案

Eval

Run the eval framework to measure agent output quality.

eysenfalk0 星標2026年2月13日

職業: 軟件質量保證分析師同測試人員
分類: LLM 同 AI

技能內容

/eval — Run Agent Quality Evals

Execute eval scenarios to measure agent output quality mechanically.

What It Does

Reads scenario definitions from evals/scenarios/
For each scenario: sets up workspace, runs agent, checks output
Scores: files exist, content matches, forbidden patterns absent, commands pass
Saves results to evals/results/ as timestamped JSON
Prints summary scorecard

Usage

Run all scenarios:

./scripts/eval.sh

Run a specific scenario:

./scripts/eval.sh evals/scenarios/$ARGUMENTS

After Running

相關技能