Guide competency framework development and operation. Use when building training that produces capability, when existing training doesn't produce competence, when structuring knowledge for multiple audiences, or when setting up feedback loops to surface gaps.
Build and operate competency frameworks that produce capability—not just completion. Diagnose where competency development is stuck and guide the next step.
Competencies are observable capabilities, not knowledge states. If you can't watch someone demonstrate it, it's not a competency.
Symptoms: Have training content but no competency structure. People complete training but can't apply it. Same questions keep getting asked.
Test:
Intervention: Start with failure modes. List mistakes you've seen, questions that shouldn't need asking, things that take too long. Each failure mode suggests a competency that would prevent it.
Symptoms: Started by listing all the information people need to know. Training is comprehensive but competence is low. "We trained on that" but mistakes continue.
Test:
Intervention: Reframe each content chunk as "what decision/action does this enable?" Kill orphan content that doesn't support a competency. Work backward from actions to required knowledge.
Symptoms: Competencies are knowledge states ("understands X") not capabilities ("can evaluate X against Y"). Can't tell if someone has the competency or not.
Test:
Intervention: Rewrite each competency as observable behavior. Transform:
Symptoms: Competencies defined but no way to test them. Assessment is knowledge recall (quizzes, multiple choice). People pass but fail in real situations.
Test:
Intervention: For each core competency, create a scenario that:
Create variants: interview (generic), assessment (org-specific), ongoing (real situations).
Symptoms: Scenarios exist but have artificial clarity. All information needed is provided. There's an obvious "right answer." People pass but fail in messy real situations.
Test:
Intervention: Add ambiguity. Remove artificial clarity. Include information that might be relevant but isn't, and omit information that would make the answer obvious. Test with real people—if everyone gets the same answer immediately, it's too simple.
Symptoms: Everyone gets the same training. Specialists are bored by basics. Generalists are overwhelmed by detail. One-size-fits-none.
Test:
Intervention: Define audience layers (typically General / Practitioner / Specialist). Map competencies to audiences. Layer content by depth:
Symptoms: Competencies exist but no clear order. Prerequisites unclear. No skip logic. Everyone follows the same path regardless of prior knowledge.
Test:
Intervention: Map dependencies. Build progression tree:
Foundation (everyone)
├── Prerequisite competencies
├─► Intermediate (builds on foundation)
└─► Role-specific branches (parallel tracks)
Define skip logic: what evidence allows skipping which modules?
Symptoms: Assessment exists but doesn't gate anything. People skip or game it. No consequence for demonstrating vs. not demonstrating competency.
Test:
Intervention: Connect each verification to a decision:
If verification doesn't connect to a decision, question whether it's worth doing.
Symptoms: Framework built once and never updated. Questions keep arising that weren't anticipated. No visibility into what's not working.
Test:
Intervention: Implement feedback loop:
Symptoms: Framework was built months/years ago. Reality has changed but framework hasn't. Questions reveal framework doesn't match current state.
Test:
Intervention: Define:
Symptoms: Competencies observable, scenarios tested, progression mapped, verification meaningful, feedback loop active, maintenance owned.
Indicators:
When someone presents a competency development need:
| Pattern | Problem | Fix |
|---|---|---|
| Document Dump | Converting existing documentation into "training" without restructuring | Identify decisions documentation supports. Build backward from decisions to content. |
| Quiz Fallacy | Assessing competency with knowledge recall questions | Replace with scenarios requiring judgment. Can't answer by ctrl+F. |
| Universal Training | One training for all audiences | Layer content. Define minimum viable competency per role. |
| Orphan Scenario | Scenario doesn't map to any defined competency | Either add the competency it tests, or cut the scenario. |
| Orphan Content | Content doesn't support any competency | Either identify the competency it serves, or cut the content. |
| Checkbox Completion | "Completed training" without demonstrated competency | Tie completion to demonstrated competency, not time spent. |
| Perfect on Paper | Framework exists but isn't used; training continues as before | Pilot with real people. Get feedback. Iterate. |
| Build-Once | Framework created, never updated | Define triggers, owners, cadence for maintenance. |
## [Cluster Name] Competencies
| ID | Competency | Description |
|----|------------|-------------|
| [PREFIX]-1 | [Action verb phrase] | [Observable capability starting with "Can..."] |
### Scenario: [Name]
**Core decision structure:** [What judgment is being tested]
**Interview variant:**
> [Generic situation]
**Assessment variant:**
> [Organization-specific situation]
**Competencies assessed:** [IDs]
**What good looks like:**
- [Consideration]
**Red flags:**
- [Weak response indicator]
Foundation (Role: Everyone)
├── [COMP-1]: [Name]
└── [COMP-2]: [Name]
├─► Intermediate (Role: [Role])
│ ├── [COMP-3]: [Name] (requires: COMP-1)
│ └── [COMP-4]: [Name] (requires: COMP-2)
└─► Specialist (Role: [Role])
└── [COMP-5]: [Name] (requires: COMP-3, COMP-4)
## Feedback Loop Design
**Observation mechanism:**
- How questions are logged
- What context is captured
- How they're tagged to competencies
**Analysis cadence:** [frequency]
**Pattern categories:**
- Training gap: [who handles]
- Framework gap: [who handles]
- Process gap: [who handles]
- Tooling gap: [who handles]
**Change tracking:**
- How changes are documented
- How effectiveness is measured
If starting small:
Expand based on what you learn from using it.
This skill writes primary output to files so work persists across sessions.
Before doing any other work:
context/output-config.md in the projectexplorations/competency/ or a sensible location for this projectcontext/output-config.md if context network exists.competency-builder-output.md at project root otherwiseFor this skill, persist:
| Goes to File | Stays in Conversation |
|---|---|
| State diagnosis | Clarifying questions |
| Competency definitions | Discussion of failure modes |
| Scenario templates | Iteration on structure |
| Framework architecture | Real-time feedback |
Pattern: {domain}-competency-{date}.md
Example: ai-literacy-competency-2025-01-15.md
During competency framework development, ask:
| Skill | Connection |
|---|---|
| research | Use when building L3 content that requires domain expertise |
| framework-development | Related but distinct: frameworks capture knowledge; competency frameworks build capability |
| framework-to-mastra | Competency framework + feedback loop = deployable agent |
User: "We have a 40-page security policy. Everyone 'completes' the training but keeps making mistakes."
Diagnosis: CF1 (Content-First Trap)
Questions to ask:
Guidance: "Each mistake suggests a competency gap. Let's work backward: if someone incorrectly handles sensitive data, the missing competency might be 'Can classify data according to organizational categories.' Once we have 3-5 competencies from failure modes, we'll design scenarios that test whether someone can actually apply the knowledge—not just recall it."
Derived from: references/competency-framework-development.md