Agent Self Improvement

2. Workflow: Isolated Execution

• Trigger: This workflow can be triggered by a new capability to test, a reported bug in agent behavior, or as a regular benchmark using existing CLs (Changelists).
• Setup Isolation:
  • Create a separate worktree for the subagent using standard workspace tools.
  • Example (Reimposition from CL): If starting from an existing CL:
    • Check out the parent commit of the CL (without the fix).
    • Extract the regression test or reproduction script from the CL.
    • Place the test in the isolated worktree.
• Launch Subagent:
  • Spawn a subagent with the specific task (e.g., "Fix the crash reproducer in test/mjsunit/...").
  • The subagent should follow the standard workflow_debugging or relevant skill.
  • The subagent operates independently and reports back its final result (code changes, analysis) when complete.

3. Workflow: Evaluation

• Trigger: Triggered automatically after the execution subagent completes its task.
• Launch Evaluation Agent:
  • Spawn an evaluation agent (or the main agent takes this role).
  • Provide the evaluation agent with:
    • The execution subagent's logs and final result.
    • The reference solution (e.g., the actual good fix from the original CL).
• Process & Result Assessment:
  • The evaluation agent compares the subagent's result with the reference solution.
  • Divergence Analysis:
    • Did the subagent find the same root cause?
    • Is the proposed fix equivalent, better, or worse?
    • Did the subagent take unnecessary shortcuts or hallucinate complexity?
  • Layered Improvement Analysis: The evaluation agent must analyze how the process could be improved in the various layers to get to a better solution:
    • Tool Layer: Did the agent lack necessary tools or misuse existing ones?
    • Skill Layer: Did the agent fail to follow skill instructions, or are the skills missing critical advice?
    • Prompt/Orchestration Layer: Was the initial prompt unclear, or did the orchestrator fail to coordinate effectively?

4. Actionable Outcomes

• Skill Updates: Propose specific diffs to existing skills (e.g., workflow_debugging/SKILL.md, orchestrator/SKILL.md) based on the evaluation findings.
• Process Refinement: Document new best practices or anti-patterns discovered during the evaluation.
• Report: Send a synthesized report to the main agent detailing the execution, evaluation, and proposed improvements.