Content Auditor

• Slide decks (Slidev, PowerPoint, Keynote)
• Facilitator scripts and guides
• Workshop plans and constitutions
• Specifications and task lists
• Student materials and handouts
• Assessment instruments

2. Criteria-Based Evaluation

Support multiple evaluation criteria formats:

• YAML-based rubrics (primary format)
• JSON-based criteria
• Markdown checklists
• Custom evaluation frameworks

3. Evidence Mapping

For each criterion:

• Search content for evidence of coverage
• Extract specific quotes/references (with file:line citations)
• Assess depth of coverage (Awareness / Working Knowledge / Mastery)
• Identify partial vs. complete coverage
• Flag gaps and missing elements

4. Gap Analysis & Recommendations

For missing or partial coverage:

• Suggest specific content to add
• Reference similar patterns from existing content
• Recommend artifacts to create (slides, exercises, assessments)
• Prioritize gaps (Critical / Important / Nice-to-have)
• Estimate effort to close gaps

5. Annotated Report Generation

Generate comprehensive audit reports:

• Original criteria with where_the_course_covers field populated
• Evidence citations with file paths and line numbers
• Coverage percentages and statistics
• Gap analysis summary
• Actionable recommendations
• YAML output format (preserves original structure)

Workflow

Stage 1: Gather Inputs

Ask clarifying questions to understand the audit scope:

1. Evaluation Criteria Source

   • "Where is your evaluation criteria file? (YAML, JSON, or other format)"
   • "What framework are you evaluating against? (e.g., ICP-BAF, ICP-ACC, custom)"

2. Content to Audit

   • "What content should I audit? (specific files, directories, or patterns)"
   • "Should I include all artifacts (slides, scripts, plans, etc.) or specific types?"

3. Audit Depth

   • "What level of rigor? (Quick scan / Standard / Deep dive)"
     • Quick scan: Check for presence/absence only
     • Standard: Assess depth + provide quotes
     • Deep dive: Full analysis + gap recommendations

4. Output Format

   • "Do you want the annotated YAML file, or also a summary report?"
   • "Should I include recommendations for closing gaps?"

Stage 2: Load and Parse Criteria

Action: Read the evaluation criteria file

YAML Criteria Format (Standard):

- evaluation:
    expectation: "What participants should be able to do"
    purpose: "Why this learning outcome matters"
    context: "Background/setting for the learning outcome"
    learning_outcome: "1.1.1. Specific LO Number and Name"
    where_the_course_covers: ""  # <-- FILL THIS IN

Parsing Strategy:

• Extract all evaluation items into a list
• For each item, note:
  • learning_outcome (the criterion ID/name)
  • expectation (what to look for in content)
  • where_the_course_covers (initially empty—to be populated)

Stage 3: Discover Content Artifacts

Action: Identify all content files to audit

Search Patterns (use Glob tool):

• Slide decks: **/*.md (Slidev), **/*.pptx, **/*.key
• Scripts: **/facilitator-script.md, **/module*-script.md
• Plans: **/plan.md, **/specification.md, **/constitution.md
• Tasks: **/tasks.md
• Other: **/README.md, **/handouts/*.md

Cataloging:

• Create a list of all discovered files
• For each file, note:
  • File path
  • File type (slide deck, script, plan, etc.)
  • Estimated purpose (Module 1 slides, Module 2 script, etc.)

Stage 4: Content Analysis (Criterion-by-Criterion)

For each evaluation criterion:

4.1 Extract Search Keywords

From the expectation field, extract key concepts to search for:

Example: