Research Statistician

B. Project-Specific Statistical Conventions

This project uses these conventions consistently. Always follow them unless the user explicitly requests otherwise.

Default Tests (Already Established)

When to Use Parametric Alternatives

Parametric tests may be mentioned as secondary/sensitivity checks when:

• Sample sizes are large (n > 30) and distributions appear reasonably symmetric
• The user asks for parametric equivalents
• A reviewer requests it

Even then, report the non-parametric result as primary for this project.

C. Effect Size Reporting

Always compute and report effect sizes alongside p-values. This is a current gap in the project that this skill should fill.

Implementation

from scipy.stats import mannwhitneyu, wilcoxon, kruskal, chi2_contingency, spearmanr

def effect_size_mannwhitney(x, y):
    """Rank-biserial r for Mann-Whitney U."""
    U, p = mannwhitneyu(x, y, alternative='two-sided')
    r = 1 - 2 * U / (len(x) * len(y))
    return r, U, p

def effect_size_kruskal(*groups):
    """Epsilon-squared (η²_H) for Kruskal-Wallis."""
    H, p = kruskal(*groups)
    n = sum(len(g) for g in groups)
    k = len(groups)
    eta_sq = (H - k + 1) / (n - k)
    return eta_sq, H, p

def cramers_v(contingency_table):
    """Cramér's V from a contingency table."""
    chi2, p, dof, expected = chi2_contingency(contingency_table)
    n = contingency_table.sum().sum()
    min_dim = min(contingency_table.shape) - 1
    V = np.sqrt(chi2 / (n * min_dim))
    return V, chi2, p

D. Multiple Comparisons Management

Within-Figure Corrections

When a single figure/analysis contains multiple related tests (e.g., pairwise comparisons after a significant omnibus test):

1. Omnibus test first — Run Kruskal-Wallis (or equivalent) to test for any group difference.
2. Post-hoc only if omnibus is significant (p < 0.05).
3. Post-hoc correction — Apply Bonferroni or Holm-Bonferroni to pairwise comparisons.
4. Report both — Adjusted and unadjusted p-values.

from itertools import combinations
from scipy.stats import mannwhitneyu

def posthoc_mannwhitney(groups_dict, alpha=0.05):
    """Pairwise Mann-Whitney with Holm-Bonferroni correction."""
    pairs = list(combinations(groups_dict.keys(), 2))
    results = []
    for g1, g2 in pairs:
        U, p = mannwhitneyu(groups_dict[g1], groups_dict[g2], alternative='two-sided')
        r = 1 - 2 * U / (len(groups_dict[g1]) * len(groups_dict[g2]))
        results.append({'group1': g1, 'group2': g2, 'U': U, 'p': p, 'r_rb': r})
    
    # Holm-Bonferroni correction
    results = sorted(results, key=lambda x: x['p'])
    m = len(results)
    for i, res in enumerate(results):
        res['p_adjusted'] = min(res['p'] * (m - i), 1.0)
        res['significant'] = res['p_adjusted'] < alpha
    return results

Across-Figure Corrections (Per-Unit Mass Testing)

Use Benjamini-Hochberg FDR (already in utils.fdr_correct()) for:

• Responsiveness screening across hundreds of units
• Per-unit selectivity testing
• Any test applied to every unit in the dataset

Do NOT apply FDR across different figures/analyses — each analysis addresses a distinct scientific question.

E. Results Summary Format

Standard Table Format

For every set of statistical tests, produce a summary table. Use this format:

┌─────────────────────────────────┬────────┬─────────────┬─────────┬──────────────┬───────────────────────┐
│ Test                            │ Stat   │ Value       │ p-value │ Effect size  │ Interpretation        │
├─────────────────────────────────┼────────┼─────────────┼─────────┼──────────────┼───────────────────────┤
│ d′ trend across sessions        │ ρ      │ 0.769       │ <0.001  │ ρ=0.77       │ Strong positive trend │
│ d′ by stage (L vs E)           │ H      │ 15.52       │ <0.001  │ η²=0.63      │ Large stage effect    │
│ Hit rate L vs E                 │ U      │ 23.0        │ 0.003   │ r=0.61       │ Large difference      │
│ Responsive fraction by stage    │ χ²     │ 8.21        │ 0.016   │ V=0.22       │ Small-medium effect   │
└─────────────────────────────────┴────────┴─────────────┴─────────┴──────────────┴───────────────────────┘

CSV Output Format

When saving to CSV (matching project convention), include these columns:

test,statistic_name,statistic_value,p_value,effect_size_name,effect_size_value,n,n_per_group,interpretation,notes
d_prime_trend_sessions,rho,0.769,1.82e-05,rho,0.769,23,,Strong positive monotonic trend,Spearman rank correlation
d_prime_by_stage,H,15.52,8.18e-05,eta_sq_H,0.63,23,L:14|E:9,Large stage effect,Kruskal-Wallis; post-hoc: L<E (U=12 p=0.001 r=0.72)

Inline Reporting Format (for text/methods)

Use APA-style inline reporting:

• Spearman: ρ(21) = 0.77, p < .001
• Mann-Whitney: U = 23.0, p = .003, r_rb = 0.61
• Kruskal-Wallis: H(1) = 15.52, p < .001, η² = 0.63
• Wilcoxon: W = 45.0, p = .012, r = 0.38
• Chi-squared: χ²(2) = 8.21, p = .016, V = 0.22
• Fisher's exact: OR = 3.2, p = .041

Domain-Specific Statistical Knowledge

Signal Detection Theory (SDT)

• d′ = z(hit_rate) − z(fa_rate), where rates are clipped to [1/(2n), 1−1/(2n)] to avoid ±∞.
• Criterion (c) = −0.5 × [z(hit_rate) + z(fa_rate)]. Negative = liberal (bias toward responding).
• Hit rate computed on go trials only (change_size > 1.01).
• FA rate computed on catch trials only (change_size ≤ 1.01).

Neural Responsiveness

• Compare baseline FR (−400 to −50 ms) vs response FR (0 to 250 ms) around Change_ON.
• Use permutation test (500 shuffles) for single-unit significance.
• Population-level: report fraction responsive with binomial CI.

TF Pulse Analysis

• Z-score: (post_mean − pre_mean) / pre_std, where pre = (−0.4, 0) s and post = (0, 0.5) s relative to pulse onset.
• Responsive if |z| ≥ 3.0 for either fast or slow pulses.
• Fast pulse: log₂(TF) > 0.25; Slow pulse: log₂(TF) < −0.25.

Decoding

• Logistic regression (L2, C=1.0) with 5-fold stratified cross-validation.
• StandardScaler applied per fold (fit on train, transform test).
• Chance level: permutation (20–200 label shuffles). Report as mean ± 2SD of null.
• Significance: Wilcoxon signed-rank of fold accuracies vs chance (0.5 for binary, 1/k for k-class).

Modulation Indices

• Modulation Index (MI) = (A − B) / (|A| + |B| + ε), bounded ∈ (−1, 1).
• Selectivity Index (SI) = (preferred − non-preferred) / (preferred + non-preferred + ε).
• Test against zero with Wilcoxon signed-rank.

Quality Checklist

Before finalizing any statistical output, verify:

• Appropriate test: Non-parametric for neural data unless justified otherwise.
• Two-sided: All tests two-sided unless biological hypothesis is strongly directional.
• Effect size: Reported alongside every p-value.
• Sample sizes: Reported for every test (total n and per-group n).
• Multiple comparisons: Addressed within each analysis (post-hoc correction or FDR).
• Assumptions checked: Independence, sufficient n per group (≥5), no ties dominating rank tests.
• Exact p-values: Report exact values (p = 0.0034), not just thresholds (p < 0.05), except for very small p (<0.001 is acceptable shorthand).
• Confidence intervals: Bootstrap CI (1000 resamples) for key estimates when requested or when point estimates alone are insufficient.
• Reproducibility: All random seeds set (seed=42 for bootstrap/permutation).

Optotagging (D1/D2 Identification)

• SALT test: Stimulus-Associated spike Latency Test — Jensen-Shannon divergence of latency histograms vs jittered baseline (500 jitters).
• Responsive criteria: SALT p < 0.01, latency < 8 ms, jitter < 3.5 ms, reliability >= 10%.
• Two fiber targets per session: GPe (block 1, D2 pathway) and SNr (block 2, D1 pathway).
• Dual-responsive units (responding to both fibers) require careful interpretation — may indicate non-specific activation or fibers-of-passage.

Consistency Verification (Do Before Every Statistical Analysis)

Before running any statistical test, verify:

1. Are trial outcomes filtered correctly? Change_ON alignment must exclude FA/abort (no change was presented). Check EVENT_VALID_OUTCOMES.
2. Is the trial-type classification correct? Go vs catch is from change_size > 1.0, NOT from the trialoutcome label. The behavioral fa label is NOT an SDT false alarm.
3. Are constants from the canonical source? Thresholds, windows, bin sizes must match visdetect/analysis/constants.py. No hardcoded values.
4. Is the session filter consistent? All tests should use the same session set. Use load_staging_manifest().
5. Are sample sizes adequate? Check n per group for every test. Flag if any group has n < 5.
6. Is this test already implemented? Check analysis_suite/utils.py for bootstrap_ci(), permutation_test(), fdr_correct(), compute_auroc() before reimplementing.

Decision Flow

When asked to perform statistical analysis:

1. Understand the question — What comparison? What is the null hypothesis?
2. Check data structure — Paired? Independent? How many groups? Sample sizes?
3. Select primary test — Justify the choice.
4. Identify secondary/tertiary tests — For robustness or reviewer requests.
5. Implement — Use project utility functions when available (utils.py).
6. Compute effect sizes — Always, for every test.
7. Format results — Table + CSV + inline text.
8. Hand off — Provide results to Research Visualizer (for annotation) and Research Notes Summarizer (for documentation).