Revise Calibration

Calibration Iteration Cap — HARD LIMIT

• Track calibration iteration count: CALIBRATION_ITER:N in Beads notes
• After 3 failed iterations: STOP tweaking parameters. Write a root-cause diagnosis that explains WHY the calibration is failing (not just what failed). Common root causes:
  • LLMs use training priors, not statistics — varying noise/SNR may have no effect if the model relies on semantic priors rather than quantitative features in the input. Solution: scale the signal (e.g., coefficients, feature values), not just the noise.
  • Effect sizes too small relative to LLM output variance
  • Encoding/prompt design doesn't expose the manipulation to the model
  • Wrong metric — measuring something orthogonal to the manipulation
• After 5 failed iterations: dispatch Methodologist for post-mortem. Flag CALIBRATION_ESCALATED in Beads. Do NOT continue tweaking.
• NEVER make >5 calibration attempts on the same design. Either the Methodologist redesigns the experiment or you report the calibration failure as a finding (it IS informative data).