Co Scientist Adaptive Experiments

Deliverables

• report.md: concise method, results, interpretation, and file inventory in the user's language.
• results/: structured outputs, metrics, model artifacts, or extracted findings.
• figures/: English-only charts, diagrams, or panels when visual output is needed.
• data/: processed or derived datasets when transformation occurs.

Quality Gates

• The selected method matches the scientific question and stated assumptions.
• Outputs are reproducible, saved to files, and traceable from inputs to conclusions.
• Missing data, uncertainty, bias, and hard limits are made explicit.
• report.md and logs/process-log.jsonl reference the generated artifacts.
• No essential result remains chat-only.

If any gate fails: identify the specific failing check, fix the issue, and re-validate before proceeding.

Gotchas

• Experimental protocols must include negative controls. Results without controls are not interpretable
• Data format validation should occur at ingestion, not after analysis. Fail fast on schema violations
• Pipeline intermediate outputs must be saved to files. Rerunning the entire pipeline for debugging is costly

Validation Loop

1. Execute analysis and generate outputs
2. Check:
   • Method selection matches the research question and stated assumptions
   • All outputs are saved to files (no chat-only results)
   • Limitations and uncertainty are explicitly stated
   • logs/process-log.jsonl is updated with execution trace
3. If any check fails:
   • Identify the failing gate
   • Fix the specific issue
   • Re-run validation
4. Proceed only after all gates pass