Builds tests, evaluation oracles, case matrices, and validation scripts for assignments, labs, exams, and projects in computing. Use when a pedagogical artifact must be corrected reliably and reproducibly.
Produce a complete, reproducible test suite that allows a grader (human or automated) to evaluate student submissions without ambiguity. The suite must cover nominal cases, edge cases, and error cases.
Read all of the following before generating any output:
mission.json — level, constraints, evaluation_modestatement.md — the exam or TP subject that students will receiverubric.md — grading criteria; tests must be traceable to rubric criteriareference/<solution> — the reference implementation produced by reference-code-engineerProduce the following artifacts, placed under tests/:
| File | Description |
|---|---|
tests/test-matrix.md | Human-readable case matrix: one row per test case with input, expected output, rubric criterion ID, and pass/fail oracle |
tests/run_tests.sh (or .py) | Executable script that runs all cases against a student binary and reports results |
tests/cases/ | Directory with one subdirectory per test case: input/, expected_output/, README.md |
test-matrix.md format| ID | Description | Input | Expected output | Criterion | Notes |
|----|-------------|-------|-----------------|-----------|-------|
| TC-01 | Nominal: single child process | ... | exit 0, log line present | CR-FORK | ... |
| TC-02 | Edge: empty input | ... | exit 1, error on stderr | CR-ERRHANDLING | ... |
tests/PLATFORM.md.mission.json → constraints.tests/results.json).ctest as the execution entrypoint for automated validation.cmake -S reference -B reference/buildcmake --build reference/buildctest --test-dir reference/build --output-on-failuretests/test-matrix.md.javac -d reference/build reference/src/Main.javajava -cp reference/build Main <sample-input>tests/test-matrix.md.cargo build --manifest-path reference/Cargo.tomlcargo test --manifest-path reference/Cargo.tomltests/test-matrix.md.