Gpd Compare Experiment

The principle: Agreement between theory and experiment must be quantified. "Looks about right" is not physics. The comparison must state: (1) what was predicted, (2) what was measured, (3) what the uncertainties are on both sides, (4) whether the agreement is statistically significant, and (5) if not, what the discrepancy tells us.

For contract-backed work, the comparison must also state which decisive output or contract target is being tested and emit an explicit verdict ledger keyed by subject_id / reference_id, not just a prose comparison. </objective>

Interpretation:

• If a prediction name: compare that specific theoretical prediction with data
• If a dataset path: compare theoretical model against that dataset
• If a phase number: compare all predictions from that phase with available data
• If empty: prompt for comparison target

Load theoretical predictions:

cat .gpd/research-map/ARCHITECTURE.md 2>/dev/null | grep -A 20 "Predictions"
find artifacts/ results/ data/ figures/ simulations/ paper/ -maxdepth 4 \
  \( -name "*.json" -o -name "*.csv" -o -name "*.dat" -o -name "*.h5" \) 2>/dev/null | \
  grep -i "result\|predict\|spectrum\|observable" | head -20

Treat .gpd/** as internal provenance only. Discover predictions and reusable comparison inputs from stable workspace directories such as artifacts/, results/, data/, figures/, simulations/, or paper/.

<execution_context>

Called from $gpd-compare-experiment command. Produces COMPARISON.md with quantified agreement metrics.

Agreement between theory and experiment must be quantified. "Looks about right" is not physics. The comparison must state: (1) what decisive output or contract target was predicted, (2) what was measured, (3) what the uncertainties are on both sides, (4) whether the agreement is statistically significant, and (5) if not, what the discrepancy tells us. </purpose>

<required_reading> Read these files using the read_file tool:

• ./.codex/get-physics-done/templates/paper/experimental-comparison.md -- Template for systematic theory-experiment comparison (data source metadata, unit conversion checklist, pull calculation, discrepancy classification, root cause hierarchy) </required_reading>

0. Load Project Context

Load project state and conventions to ensure correct unit systems and sign conventions:

• Run:

INIT=$(/home/qol/.gpd/venv/bin/python -m gpd.runtime_cli --runtime codex --config-dir ./.codex --install-scope local init progress --include state)
if [ $? -ne 0 ]; then
  echo "ERROR: gpd initialization failed: $INIT"
  # STOP — display the error to the user and do not proceed.
fi

• Parse JSON for: commit_docs, state_exists, project_exists, current_phase
• If state_exists is true: Read .gpd/state.json to extract convention_lock for unit system, metric signature, and Fourier conventions. Extract active approximations and their validity ranges from state. Load intermediate_results from state for any previously computed quantities.
• If state_exists is false (standalone usage): Proceed with explicit convention declarations required from user via ask_user (unit system, sign conventions, normalization)

Convention context is critical for theory-experiment comparison: unit mismatches and convention mismatches are the two most common sources of discrepancy.

Convention verification (if project exists):

CONV_CHECK=$(/home/qol/.gpd/venv/bin/python -m gpd.runtime_cli --runtime codex --config-dir ./.codex --install-scope local --raw convention check 2>/dev/null)
if [ $? -ne 0 ]; then
  echo "WARNING: Convention verification failed — unit mismatches between theory and experiment are the #1 source of false discrepancies"
  echo "$CONV_CHECK"
fi

1. Identify What to Compare

If the project is contract-backed, first resolve the comparison target against the approved contract:

• subject_id
• subject_kind (claim, deliverable, acceptance_test, or artifact)
• subject_role (decisive, supporting, supplemental)
• reference_id for the benchmark / prior-work / data anchor
• the pass condition or tolerance that makes this comparison decisive

Do not write a generic comparison report without this mapping when a decisive comparison target exists.

1a. Theoretical predictions

For each prediction, establish:

• Quantity predicted: Name, symbol, definition
• Predicted value or functional form: Expression or numerical table
• Theoretical uncertainties: From truncation, approximation, numerical precision
• Conventions: Units, normalization, sign conventions
• Regime of validity: Where the prediction is expected to hold

1b. Experimental data

For each measurement, establish:

• Quantity measured: Name, definition, what detector/apparatus
• Measured values: Central values with error bars
• Statistical uncertainties: From measurement statistics
• Systematic uncertainties: From calibration, background, modeling
• Conventions: Units, normalization, binning, acceptance corrections
• Conditions: Temperature, pressure, energy, other experimental parameters

Ask the user once using a single compact prompt block if data source is ambiguous:

1. Data source -- Published paper | Our own measurements | Database (PDG, NIST, etc.) | Simulation (lattice, MD, etc.)
2. Data format -- CSV/table | Digitized from plot | API/database query | File path
3. Uncertainty treatment -- Statistical only | Statistical + systematic (separate) | Combined | Correlated uncertainties

2. Unit Conversion and Convention Matching

This step catches the most common theory-experiment comparison errors.

2a. Unit alignment

Ensure theory and experiment use the same units:

# Common unit conversion pitfalls in physics:
# - eV vs Kelvin: 1 eV = 11604.5 K
# - eV vs cm^{-1}: 1 eV = 8065.54 cm^{-1}
# - Gaussian vs SI electromagnetism: factor of 4*pi*epsilon_0
# - Natural units to SI: restore factors of hbar, c, k_B
# - Energy vs frequency: E = hbar * omega (not h * nu !)
# - Cross-section: barn (1e-24 cm^2) vs fm^2 (1e-26 cm^2) vs GeV^{-2}
# - Decay width vs lifetime: Gamma = hbar / tau

Document every conversion applied.

2b. Convention matching

Check that theory and experiment use the same conventions:

2c. Acceptance and efficiency corrections

Experimental data often includes detector acceptance and efficiency:

• Is the data already corrected? (Unfolded to particle level)
• If not: apply theory-side smearing/cuts to match detector level
• Document what corrections are applied on which side

3. Perform the Comparison

3a. Point-by-point comparison

import numpy as np

# For each data point:
for i in range(len(data)):
    x_exp = data[i]['x']
    y_exp = data[i]['y']
    dy_exp = data[i]['uncertainty']  # total: stat + syst combined
    y_theory = theory_prediction(x_exp)
    dy_theory = theory_uncertainty(x_exp)

    # Pull: (theory - experiment) / combined_uncertainty
    dy_combined = np.sqrt(dy_exp**2 + dy_theory**2)
    pull = (y_theory - y_exp) / dy_combined

    print(f"x={x_exp:.4f}  exp={y_exp:.6f}+/-{dy_exp:.6f}  "
          f"theory={y_theory:.6f}+/-{dy_theory:.6f}  pull={pull:.2f}")

3b. Global fit quality

# Chi-squared test
chi2 = 0
for i in range(len(data)):
    dy_combined = np.sqrt(data[i]['uncertainty']**2 + theory_uncertainty(data[i]['x'])**2)
    chi2 += ((theory_prediction(data[i]['x']) - data[i]['y']) / dy_combined)**2

ndof = len(data) - n_free_params
chi2_reduced = chi2 / ndof
p_value = 1 - scipy.stats.chi2.cdf(chi2, ndof)

print(f"chi2/ndof = {chi2:.1f}/{ndof} = {chi2_reduced:.2f}")
print(f"p-value = {p_value:.4f}")

3c. Statistical interpretation

3d. Residual analysis

# Check for systematic patterns in residuals:
# - Trending residuals suggest missing physics (wrong functional form)
# - Oscillating residuals suggest discretization artifacts
# - Constant offset suggests normalization error
# - Growing residuals suggest wrong scaling / missing terms

4. Discrepancy Analysis

If the comparison shows significant disagreement:

4a. Classify the discrepancy

4b. Root cause investigation

1. Check units again -- Most theory-experiment discrepancies are unit errors
2. Check conventions -- Second most common: different normalization or phase convention
3. Check data processing -- Binning, acceptance, background subtraction
4. Check approximations -- Is the theoretical prediction valid in this regime?
5. Check for missing physics -- Higher-order corrections, finite-size effects, interactions neglected

4c. Quantify the tension

# Number of sigma tension:
tension_sigma = abs(theory_central - exp_central) / np.sqrt(theory_unc**2 + exp_unc**2)

# Interpretation:
# < 1 sigma: consistent
# 1-2 sigma: mild tension, not significant
# 2-3 sigma: interesting tension, warrants investigation
# 3-5 sigma: significant discrepancy
# > 5 sigma: discovery-level discrepancy

5. Generate Comparison Report

Write COMPARISON.md:

---