Auditing Bdd Tests

Step 1: Discovery & Auto-Inference

Target: {argument OR cwd}

Auto-detect (no user input needed):

Output immediately:

Target: {path}
Stack: {stack} (auto-detected)
Size: {small/medium/large} ({N} features, {M} scenarios)
History: {yes/no} | CI: {yes/no} | Artifacts: {configured/missing}

See modules/discovery.md for detailed detection rules.

Step 2: Sampling (Medium/Large repos only)

Skip for small repos — analyze all scenarios.

For medium/large repos, use stratified sampling. See modules/sampling.md.

Progress Indicator (Medium/Large repos)

For repositories with 50+ scenarios, show progress during analysis:

Analyzing BDD Test Solution...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0%

[■■■■■■■■■■░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░] 25%
✓ Discovery complete (playwright-bdd detected)
→ Analyzing features/auth/*.feature (8 scenarios)

[■■■■■■■■■■■■■■■■■■■■░░░░░░░░░░░░░░░░░░░░] 50%
→ Analyzing features/checkout/*.feature (12 scenarios)

[■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■░░░░░░░░░░] 75%
→ Scoring aspects...

[■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■] 100%
✓ Analysis complete

Progress stages:

1. Discovery (10%)
2. Feature file analysis (10-70%, proportional to file count)
3. Step definition analysis (70-85%)
4. Scoring (85-95%)
5. Report generation (95-100%)

Update progress after each feature file or major step.

Step 3: Score Aspects

Score each aspect using rubrics from criteria/aspects.md.

Aspects and weights:

Scoring: 0 (bad) / 5 (partial) / 10 (good) per criterion.

See modules/scoring.md for calculation formulas.

Step 4: Report

4.1 Terminal Output (Always)

Print ASCII dashboard with scores and issues. See modules/output-formats.md.

4.2 Issues by Severity

Classify using reference/severity.md:

• 🔴 CRITICAL — blocks reliable execution
• 🟡 WARNING — hinders speed/maintainability
• 🔵 INFO — optimizations

Every issue MUST have:

• Evidence (file path, pattern, or code snippet)
• Impact (why it matters)
• Effort estimate (Low/Medium/High)

4.3 Save Reports

Save to .bddready/history/reports/:

• {REPORT_ID}.json — machine-readable
• {REPORT_ID}.md — human-readable

Update .bddready/history/index.json for delta tracking.

4.4 HTML Report (Offer to User)

After showing terminal output, ask:

Would you like me to generate an interactive HTML report?

If yes, run:

node scripts/render-html.mjs .bddready/history/reports/{REPORT_ID}.json .bddready/history/reports/{REPORT_ID}.html

Interactive Fix Mode

After showing issues, offer to fix quick wins immediately.

Trigger Conditions

Offer interactive fixes when:

• At least 1 CRITICAL issue with Effort: Low
• Issue has clear, automatable fix pattern

Flow

╔══════════════════════════════════════════════════════════════════╗
║                     QUICK FIX AVAILABLE                          ║
╠══════════════════════════════════════════════════════════════════╣
║  [C1] Flake Resistance: Found 7 arbitrary sleeps                 ║
║       Fix: Replace `wait X seconds` with condition waits         ║
║       Effort: Low | Files: 3                                     ║
║                                                                  ║
║  → Fix C1 now? [y/n/skip all]                                    ║
╚══════════════════════════════════════════════════════════════════╝

Response Handling

Fixable Patterns

After Each Fix

✓ Fixed C1: Replaced 7 sleeps with condition waits
  Modified: features/checkout.feature, features/auth.feature
  
→ Fix C2 now? [y/n/skip all]

Post-Fix Summary

╔══════════════════════════════════════════════════════════════════╗
║                     FIX SUMMARY                                  ║
╠══════════════════════════════════════════════════════════════════╣
║  ✓ C1: Fixed (7 sleeps → condition waits)                        ║
║  ✓ C3: Fixed (added trace-on-failure)                            ║
║  ✗ C2: Skipped (requires manual review)                          ║
║                                                                  ║
║  Files modified: 5                                               ║
║  New score estimate: 68 → 74 (+6)                                ║
╚══════════════════════════════════════════════════════════════════╝

Step 5: Roadmap (Medium/Large repos only)

Skip for small repos — provide inline recommendations instead.

User Questions

Auto-Inference First

Before asking, try to infer from codebase:

Minimal Question Set

Ask ONLY what cannot be inferred:

Small repos (1 question):

What is your priority: stability, speed, or AI-agent readability?

Medium repos (2 questions):

1. What is your priority: stability, speed, or AI-agent readability?
2. How deep can changes go: quick fixes only, or can we refactor?

Large repos (3 questions):

1. What is your priority: stability, speed, or AI-agent readability?
2. How deep can changes go: quick fixes only, medium refactor, or deep restructuring?
3. Are there CI/environment constraints? (e.g., worker limits, no mocks, staging only)

Dynamic Questions (Only if triggered)

Ask ONLY if specific issues found:

Semantic/A11y Refactoring Proposal

If Aspect 4 (Selector Strategy) or Aspect 8 (AI-Agent Operability) scores below 60, propose:

╔══════════════════════════════════════════════════════════════════╗
║           SEMANTIC/A11Y REFACTORING PROPOSAL                     ║
╠══════════════════════════════════════════════════════════════════╣
║  Your locators would be more stable with semantic HTML.          ║
║                                                                  ║
║  Would you like me to help refactor:                             ║
║  [ ] Component markup (replace div onclick → button, add ARIA)   ║
║  [ ] Test locators (migrate CSS → getByRole)                     ║
╚══════════════════════════════════════════════════════════════════╝

Ask only if user has access to modify application source code.

Reference Files

Quick Reference: Workflow by Size

Small Repo (≤20 scenarios)

1. Discover (auto-detect stack, size)
2. Ask 1 question (priority)
3. Analyze all scenarios
4. Score aspects (simplified)
5. Print terminal report + issues
6. Interactive fix mode (if Low-effort CRITICAL issues)
7. Offer HTML report
8. Provide inline recommendations

Medium Repo (21–100 scenarios)

1. Discover (auto-detect)
2. Ask 2 questions
3. Sample 30–50%
4. Show progress (50+ scenarios)
5. Full aspect scoring
6. Terminal + saved reports
7. Interactive fix mode
8. Offer HTML report
9. Phased roadmap (3 phases)

Large Repo (100+ scenarios)

1. Discover (auto-detect)
2. Ask 3 questions
3. Stratified sampling
4. Show progress (with stage updates)
5. Full aspect scoring
6. All report formats
7. Interactive fix mode
8. HTML report
9. Detailed phased roadmap
10. Propose a11y refactoring if applicable