Stat Compare Multiple Groups

1. Read data; validate that group column has three or more levels.
2. Run normality tests (Shapiro-Wilk) within each group and run one global Levene's test across groups for homogeneity of variance.
3. If all groups are normal and variances are approximately equal: one-way ANOVA.
   • If second factor provided: two-way ANOVA with interaction term.
4. If normality or homogeneity of variance is violated: Kruskal-Wallis test.
5. If omnibus test is significant (p < 0.05): run post-hoc pairwise comparisons.
   • ANOVA: Tukey HSD (controls family-wise error rate).
   • Kruskal-Wallis: Dunn test with Benjamini-Hochberg correction.
6. Compute effect size: eta-squared (η²) for ANOVA, epsilon-squared for Kruskal-Wallis.
7. Generate a box plot with all groups and pairwise significance brackets.
8. Save the executable analysis script (analysis_code.py or analysis_code.R) and software version manifest (session_info.txt) used to produce results.
9. Write results and plot (PDF) to output directory.
10. Report: omnibus test result, effect size, significant pairwise comparisons.

Output Contract

• Omnibus test result (TSV): test, statistic, df, p_value, effect_size, effect_size_type
• Post-hoc comparisons table (TSV): group1, group2, statistic, p_value, adj_p_value, significant
• Box plot with significance brackets (PDF)
• Decision log explaining method selection (printed to stdout)
• Reproducibility artifacts: analysis_code.py or analysis_code.R, plus session_info.txt

Limits

• Two-way ANOVA assumes balanced or near-balanced design; unbalanced designs require Type III sums of squares.
• Does not support repeated-measures ANOVA; use specialized tools for within-subject designs.
• Tukey HSD assumes equal group sizes; Games-Howell is preferred for unequal variances.
• Common failure: too few observations per group (n < 5) makes tests unreliable.