Co Scientist Ensemble Methods

Deliverables

• report.md: concise method, results, interpretation, and file inventory in the user's language.
• results/: structured outputs, metrics, model artifacts, or extracted findings.
• figures/: English-only charts, diagrams, or panels when visual output is needed.
• data/: processed or derived datasets when transformation occurs.

Quality Gates

• The selected method matches the scientific question and stated assumptions.
• Outputs are reproducible, saved to files, and traceable from inputs to conclusions.
• Missing data, uncertainty, bias, and hard limits are made explicit.
• report.md and logs/process-log.jsonl reference the generated artifacts.
• No essential result remains chat-only.

If any gate fails: identify the specific failing check, fix the issue, and re-validate before proceeding.

Gotchas

• Data leakage between train/test splits invalidates all metrics. Verify no leakage before reporting results
• Random seeds must be set for numpy, random, and framework-specific RNGs separately (torch, tf)
• Hyperparameter tuning needs held-out test data never seen during tuning. Three-way split is minimum

Validation Loop

1. Execute analysis and generate outputs
2. Check:
   • Method selection matches the research question and stated assumptions
   • All outputs are saved to files (no chat-only results)
   • Limitations and uncertainty are explicitly stated
   • logs/process-log.jsonl is updated with execution trace
3. If any check fails:
   • Identify the failing gate
   • Fix the specific issue
   • Re-run validation
4. Proceed only after all gates pass