Research Assistant

Workflow: Scientific Iteration

Each iteration follows this structure:

Phase 1: Hypothesis Generation

hypothesis_init = generate_hypothesis(context, user_steering?)

1. Load framework-context.md and experiment-state.md
2. Review recent results (if any)
3. Propose 1-3 testable hypotheses
4. Present to user for approval/refinement

Phase 2: Literature Grounding (if NotebookLM configured)

literature = query_notebooklm(context, hypothesis_init)
hypothesis_final = refine_hypothesis(literature, hypothesis_init)

1. Load notebooklm skill
2. Query configured notebooks with hypothesis context
3. Extract relevant citations and insights
4. Refine hypothesis based on literature
5. Present refined hypothesis to user

Phase 3: Experiment Execution

1. Generate experiment configurations
2. Present batch to user for approval
3. Execute approved experiments (track in MLflow)
4. Monitor progress, handle failures
5. Update experiment-state.md with run IDs

STOP HERE — Do not interpret results from terminal output.
Execution outputs are for monitoring only, not analysis.

Phase 4: Result Interpretation

See references/result-interpretation.md for the complete workflow.

MANDATORY FIRST STEP: Query MLflow before any analysis.

# REQUIRED: Fetch data from MLflow
runs_df = mlflow.search_runs(experiment_ids=[...], filter_string="tags.hypothesis_id = 'H1'")

# Only THEN perform analysis
results = aggregate_metrics(runs_df)
interpretation = analyze(results, hypothesis, context)

1. Query MLflow using mlflow.search_runs() — see reference file for patterns
2. Load metrics into DataFrame — this is the authoritative data source
3. Compute statistics (mean, std, CI) from the DataFrame
4. Run statistical tests (t-test, effect size)
5. Generate interpretation with:
   • Support/refute/inconclusive verdict
   • Key observations from the queried data
   • Suggested next directions
6. Update research-log.md with full record

DO NOT:

• Parse terminal output, script return values, or print statements
• Load querying-mlflow-metrics skill — that is for GenAI trace metrics (tokens, latency), not ML experiment metrics like best_fitness

Phase 5: Context Update

context_{i+1} = update_context(results, hypothesis, interpretation)

1. Check if git diff shows framework changes
2. If stale, prompt user to re-run framework discovery
3. Update experiment-state.md with:
   • Completed hypotheses
   • New observations
   • Updated priors

Supporting Skills

Load these as needed:

For ML experiment metrics: Use mlflow.search_runs() directly — see references/result-interpretation.md | notebooklm | Literature grounding |

Detailed Workflow References

For in-depth guidance on each phase, see:

Core Scientific Loop

Operations & Integration

State File Schemas

config.yaml