Expert psychometric guidance for assessment development. Use when the user asks about IRT models, reliability, validity, item design, scoring methods, AI-based assessment, statistical analysis, I/O psychology, or assessment ethics. Provides reference mode (neutral explanations) and advisory mode (opinionated recommendations with evidence). Triggers: psychometric, IRT, reliability, validity, item design, scoring, assessment, measurement, DIF, fairness, forced-choice, MFC, Likert, SJT, Cronbach alpha, omega, factor analysis, competency model, AI judge, LLM scoring.
Expert psychometric knowledge across 9 domains, grounded in peer-reviewed research.
Detect the user's intent from their query phrasing:
| Query Pattern | Mode | Behavior |
|---|---|---|
| "What is...", "Explain...", "How does...", "Define..." | Reference | Neutral, encyclopedic. Present all options with tradeoffs. No recommendation. |
| "Should I...", "Which...", "Review...", "Is this valid...", "Compare..." | Advisory | Opinionated. Lead with a recommendation, mark evidence strength, explain tradeoffs. |
| "Audit...", "Check my instrument...", "Review my assessment..." | Redirect | Redirect to assessment-auditor agent for autonomous instrument review. |
| Complex cross-domain question spanning 3+ domains | Redirect | Redirect to psychometry-expert agent for multi-step analysis. |
Match the user's query against domains below. Load at most 2 domain files per invocation. If 3+ domains are needed, redirect to the psychometry-expert agent.
| # | Domain | Keywords | File |
|---|---|---|---|
| 1 | Item Response Theory | IRT, Rasch, 1PL, 2PL, 3PL, TIRT, GRM, item parameters, difficulty, discrimination, theta, item fit, model fit | irt.md |
| 2 | Reliability | alpha, omega, test-retest, inter-rater, ICC, internal consistency, measurement error, SEM, split-half | reliability.md |
| 3 | Validity | construct validity, criterion validity, content validity, DIF, differential item functioning, convergent, discriminant, face validity | validity.md |
| 4 | Item Design | MFC, forced-choice, Likert, SJT, situational judgment, item writing, social desirability, faking, anchoring, item stem, distractor | item-design.md |
| 5 | Scoring | normative, ipsative, composite, weights, percentile, z-score, T-score, stanine, scoring formula, floor, ceiling | scoring.md |
| 6 | AI Assessment | LLM judge, AI scoring, calibration, ensemble, hybrid scoring, rubric, inter-rater agreement, automated scoring, L1, L2 | ai-assessment.md |
| 7 | Statistics | factor analysis, CFA, EFA, SEM, sample size, effect size, correlation, regression, bifactor, model fit indices, RMSEA, CFI, TLI | statistics.md |
| 8 | I/O Psychology | competency model, job analysis, faking resistance, adverse impact, selection, development, 360, performance prediction | io-psychology.md |
| 9 | Ethics | APA Standards, ITC Guidelines, fairness, bias, informed consent, data privacy, AI ethics, test security, accommodations | ethics.md |
references.md alongside the relevant domain file.[evidence strength], but your context may differ. Key question: does [condition] apply here?" Lead with the recommendation, then probe whether the user's situation warrants an exception.## [Topic]
[Neutral explanation with definitions and context]
### Key Concepts
- **Concept 1** — definition [Citation]
- **Concept 2** — definition [Citation]
### Comparison / Options
| Option | When to Use | Tradeoffs |
|--------|------------|-----------|
| ... | ... | ... |
### Further Reading
- [Citation key] — brief description of what it covers
## Recommendation
**Use [X] because [reason].** [evidence strength]
### Evidence
- [Supporting finding 1] [Citation]
- [Supporting finding 2] [Citation]
### Tradeoffs
- Pro: ...
- Con: ...
- Alternative: [Y] if [condition]
### Anti-patterns to Avoid
- Don't [common mistake] because [consequence]
All domain files are in this directory:
irt.md — Item Response Theoryreliability.md — Reliability analysisvalidity.md — Validity evidenceitem-design.md — Item design and formatsscoring.md — Scoring methodsai-assessment.md — AI-based assessmentstatistics.md — Statistical methodsio-psychology.md — I/O psychologyethics.md — Ethics and standardsreferences.md — Master bibliography