Comprehensive QA gate with multi-model debugging chamber and self-improving learning loop. Run before commits, PRs, and releases. Invoked by master-orchestrator after work streams complete, or explicitly via /bb-qa-gate. Covers Next.js, Supabase RLS, ACID transactions, state management, AI pipeline, Microsoft Graph, and all FINTECH_ABSOLUTES rules.
Version: 4.2 Last Updated: 2026-03-18 Purpose: Comprehensive QA framework — 29 automated pre-commit checks (+ --deep self-simulation mode) + 13 guide modules + multi-model debugging chamber (GPT/Gemini/Claude self-sim fallback) + self-improving learning loop + build phase→plugin mapping.
This skill operates in three modes:
Run the automated checks and apply the relevant guide module.
bash skills/bb-qa-gate/scripts/bb-qa-gate.sh
When a bug escapes to production, run the full 5-phase chamber process.
Read references/chamber-process.md for the complete protocol.
After every significant bug, update this skill so the class of bug never escapes again.
Read references/learning-loop.md for the update protocol.
The golden rule: An incident cannot be closed until either the QA skill is updated, or the chamber explicitly states why no reusable QA update is warranted.
| Input Type | Primary Module | Secondary Module |
|---|---|---|
| PR diff | Regression Review (§11) + Release Readiness (§1) | — |
| Feature spec | Feature Test Plan (§2) | — |
| Bug report | Bug Triage (§10) → Debugging Chamber | — |
| API route file | API Contract QA (§5) | Auth/DB QA (§3) |
| Component file | Frontend State QA (§6) | Realtime QA (§4) |
| Incident summary | Production Incident QA (§12) → Debugging Chamber | — |
| Migration file | Database/Auth QA (§3) | FINTECH_ABSOLUTES |
| Post-incident review | Learning Loop | — |
Output format: Every run produces one or both of:
The debugging chamber uses three models with distinct responsibilities:
Claude (orchestrator)
GPT (diagnostician)
Gemini (challenger)
When a production bug arrives, Claude runs scripts/chamber.py which handles all 5 phases automatically — calling GPT and Gemini APIs, collecting diagnoses, running rebuttals, and producing the converged verdict. See references/chamber-process.md for the full protocol details.
Quick start: Just tell Claude about the bug. Claude assembles the incident JSON and runs the chamber script. Keys are in .env.chamber.local.
Every debugging run produces two artifacts:
{
"root_cause": "string — what actually broke and why",
"affected_subsystem": "auth|permissions|api_contract|realtime|react_state|ssr_boundary|graph|ai_pipeline|env_config|regression",
"severity": "P0|P1|P2|P3",
"patch_plan": ["step 1", "step 2"],
"verification_steps": ["how to confirm the fix works"],
"rollback_needed": false
}
{
"qa_skill_delta": {
"gap_type": "missing_test|missing_heuristic|missing_regression_rule|missing_architecture_check|unclear_guidance",
"why_missed": "string — why the current QA deck did not catch this",
"new_rule": "string — concrete QA guidance to add, written so future runs catch this",
"new_test_cases": ["string — specific test scenarios to add"],
"target_section": "§1-§12 or new subsystem bucket",
"update_priority": "high|medium|low",
"reusable_for_future": true
}
}
Each build phase loads ONLY the plugins relevant to its domain. This prevents context pollution and ensures integrators get targeted coverage.
Risk profile: Data integrity, tenant isolation, financial precision Plugins:
finance:journal-entry-prep — validates double-entry GL patterns in migrationsfinance:reconciliation — checks balance constraints and FK integrityfinance:audit-support — SOX control testing on financial tablesdata:data-validation — methodology checks on schema designdata:sql-queries — validates PostgreSQL syntax, index strategy, query performanceengineering:architecture — ADR review for schema decisionslegal:compliance-check — NJ DCA, Fair Housing, PCI compliance on data modelFINTECH_ABSOLUTES enforced: ABS-001 (NUMERIC(14,2)), ABS-005 (SERIALIZABLE), ABS-011 (balanced GL), ABS-015 (RLS), ABS-021 (TIMESTAMPTZ), ABS-030 (updated_at trigger), ABS-032 (no hard DELETE)
Risk profile: Security, ACID compliance, input validation, error handling Plugins:
engineering:code-review — security, performance, correctness on route handlersengineering:testing-strategy — test plan for API contractsdata:sql-queries — validates Supabase RPC function SQLdata:data-validation — verifies API response data accuracyFINTECH_ABSOLUTES enforced: ABS-004 (RPC procedures), ABS-008 (idempotency), ABS-010 (optimistic locking), ABS-013 (no card data), ABS-016 (SECURITY INVOKER), ABS-018 (PostgREST filters), ABS-027 (no service_role in client), ABS-031 (no SELECT *)
Risk profile: Accessibility, state bugs, SSR boundary errors, UX quality Plugins:
design:design-critique — usability, visual hierarchy, consistency reviewdesign:accessibility-review — WCAG 2.1 AA compliance auditdesign:ux-writing — microcopy, error messages, empty states, CTAsdesign:design-system-management — component consistency, design token compliancedesign:user-research — test plan for user flowsengineering:code-review — React patterns, hook usage, state managementFINTECH_ABSOLUTES enforced: ABS-002 (no JS financial math), ABS-031 (explicit column selection in stores)
Risk profile: OAuth, webhook reliability, partial outage handling Plugins:
engineering:system-design — integration architecture reviewengineering:incident-response — failure mode analysis for external servicesengineering:testing-strategy — mock provider test strategylegal:compliance-check — OAuth scope review, PCI compliance for payment integrationsFINTECH_ABSOLUTES enforced: ABS-013 (PCI tokenization), ABS-014 (payment state machine), ABS-019 (MIME validation on inbound), ABS-027 (key management)
Risk profile: Escaped defects, deployment failures, rollback safety Plugins:
engineering:deploy-checklist — pre-deployment verificationengineering:testing-strategy — regression test coverageengineering:code-review — final PR reviewdata:data-validation — production data integrity verificationfinance:audit-support — SOX control verification before financial features shipFINTECH_ABSOLUTES enforced: ALL 34 rules verified via bb-qa-gate.sh
When an integrator is assigned to a wave, their task prompt specifies their phase:
## PHASE: DB/Schema
Load plugins: finance:journal-entry-prep, data:sql-queries, engineering:architecture
FINTECH checks: ABS-001, ABS-005, ABS-011, ABS-015, ABS-021, ABS-030, ABS-032
The orchestrator includes this in the integrator task package. The integrator reads it in PRE-FLIGHT and loads only the specified plugins.
Every bug and every QA rule maps into one of these 10 subsystem buckets. This gives the deck a stable structure instead of random growth.
| Bucket | Covers | Guide Module | Build Phase |
|---|---|---|---|
| auth/session | Login, SSO, token refresh, session expiry, multi-tab | §3 | API/Backend |
| permissions/RLS | RLS policies, checkPermission, role mapping, cross-tenant | §3 | DB/Schema |
| api_contract | Input validation, response shape, error codes, rate limits | §5 | API/Backend |
| realtime_sync | Subscriptions, dedup, reconnect, optimistic updates | §4 | Integration |
| react_state | Zustand, React Query, cache invalidation, stale data | §6 | UI/UX |
| ssr_boundary | 'use client', Server/Client split, dynamic imports, hydration | §1 | UI/UX |
| graph_integration | OAuth tokens, calendar sync, email, consent revocation | §9 | Integration |
| ai_pipeline | Prompts, fallback, hallucination, schema adherence, cost | §7 | |
| env_config | Secrets, .env drift, deployment config, feature flags | §8 | |
| regression | Shared component changes, migration side-effects, store changes | §11 |
When adding a new rule from a bug, tag it to the correct bucket. If a bug spans multiple buckets, add the rule to the primary bucket and cross-reference the secondary.
The learning loop uses tiered auto-updates based on priority:
| Priority | Action | Rationale |
|---|---|---|
| Low | Auto-update the skill immediately | Minor additions that follow established patterns |
| Medium | Auto-update the skill immediately | Important coverage gaps, but the fix is clear |
| High | Propose delta for Vince's review before writing | Structural changes, new subsystem buckets, or rule removals |
Convergence filter for skill improvement — do not blindly merge all suggestions from the chamber. Apply these filters:
The deck covers all three phases, weighted by where bugs most commonly escape:
This is the primary defense. Most bugs should be caught here.
bb-qa-gate.sh (23+ checks)Integration-level issues that only surface in a full environment.
Final go/no-go before production deploy.
Before any deployment, verify against the full stack:
Promise<> typed paramsRecord<string, unknown> casts in TSX (use as any)'use client' in first 15 linesconst X: any = dynamic(... as any)middleware.ts warning acknowledged (Next.js 16 deprecation)@ts-expect-error without explanation commentSECURITY DEFINER functions (use SECURITY INVOKER).from() table names exist in schemacheckPermission() module values exist in Module typesupabase/edge-functions in exclude array)https://esm.sh/, https://deno.land/)Deno.env.get()| Risk Level | Criteria | Action |
|---|---|---|
| SHIP | 0 failures, 0 high warnings, QA gate PASS | Deploy |
| SHIP WITH MONITORING | 0 failures, warnings < 5, no financial code changed | Deploy + watch logs for 1 hour |
| HOLD | Any failure, or financial code changed without ACID review | Fix first |
| ROLLBACK | Production incident, data integrity risk | Revert immediately |
Given a feature description, generate:
For each user story, write the expected flow:
GIVEN [precondition]
WHEN [action]
THEN [expected result]
AND [side effects: DB writes, notifications, audit log]
-- Verify RLS is enabled
SELECT tablename, rowsecurity FROM pg_tables WHERE schemaname = 'public';
-- Verify policies exist (need all 4: SELECT, INSERT, UPDATE, DELETE)
SELECT tablename, policyname, cmd FROM pg_policies WHERE schemaname = 'public';
-- Test cross-tenant isolation
-- As User A (org_id = 'aaa'): SELECT * FROM payments; → should only see org_aaa data
-- As User B (org_id = 'bbb'): SELECT * FROM payments; → should only see org_bbb data
-- As Anon (no auth): SELECT * FROM payments; → should return 0 rows
checkPermission()?supabase.channel().on('postgres_changes', ...) fires on INSERT/UPDATE/DELETEsupabase.auth.getUser()checkPermission(module, action) for protected operations{ error: string } shape{ data, count, page, pageSize }'use client' directive.env.local in .gitignore**Title:** [One-line description]
**Severity:** P0 (data loss) | P1 (broken feature) | P2 (degraded UX) | P3 (cosmetic)
**Subsystem:** [Auth | Finance | Leasing | Maintenance | AI | Graph | UI]
**Steps to Reproduce:**
1. [exact step]
2. [exact step]
3. [exact step]
**Expected:** [what should happen]
**Actual:** [what actually happens]
**Evidence:** [screenshot, error log, network trace]
**Environment:** [browser, OS, user role, property context]
**First Occurrence:** [date/time]
**Frequency:** [always | intermittent | once]
| Severity | Criteria | Response Time |
|---|---|---|
| P0 | Data loss, financial error, auth bypass | Immediate |
| P1 | Feature broken, blocks workflow | Same day |
| P2 | Degraded UX, workaround exists | This sprint |
| P3 | Cosmetic, minor text issue | Backlog |
After triage, if P0 or P1: Immediately launch the Debugging Chamber (references/chamber-process.md).
Given a PR diff, spec, or release note, generate a regression checklist:
For each changed file, identify:
## Regression Checklist for [PR/Feature]
### Changed Files
- [ ] file1.ts — [what changed] — consumers: [list]
- [ ] file2.tsx — [what changed] — consumers: [list]
### Database
- [ ] Migration applied successfully in staging?
- [ ] Rollback tested?
- [ ] RLS policies still protect affected tables?
- [ ] Existing data not corrupted by ALTER TABLE?
### Auth
- [ ] All roles still have correct access?
- [ ] Login flow still works?
- [ ] Permission checks on affected routes verified?
### Financial
- [ ] GL balance integrity maintained?
- [ ] Payment flows still atomic?
- [ ] Idempotency keys still enforced?
### UI
- [ ] Affected pages still render?
- [ ] Loading/error states still work?
- [ ] Mobile responsive on affected pages?
### Integration
- [ ] Graph API calls still work?
- [ ] Realtime subscriptions still fire?
- [ ] AI pipeline still responds?
| Affected System | Severity Escalation |
|---|---|
| Auth (login broken) | P0 — all users affected |
| Financial (wrong amounts) | P0 — data integrity risk |
| GL (unbalanced) | P0 — accounting error |
| Payments (double charge) | P0 — legal liability |
| Realtime (stale data) | P1 — UX degraded |
| AI pipeline (wrong output) | P1 — user confusion |
| Graph (email/calendar) | P2 — feature unavailable |
| UI (rendering) | P2-P3 depending on scope |
After resolving any P0/P1 incident, the incident cannot be closed until:
This is the self-improvement engine that turns production bugs into QA training data. Read references/learning-loop.md for the full protocol.
After every significant bug, Claude answers these five questions:
The bash script runs 29 automated checks (+ Check 29 in --deep mode). See the script for full implementation.
Standard mode: bash skills/bb-qa-gate/scripts/bb-qa-gate.sh (28 checks, no API calls)
Deep mode: bash skills/bb-qa-gate/scripts/bb-qa-gate.sh --deep (28 checks + Anthropic self-simulation on diff)
Current checks:
as any[]Rule: Every CI failure becomes a new automated check. The skill learns from each failure.
Batch fix rule (from Chamber 2026-03-18): After any batch regex fix across multiple files, verify the output parses correctly by checking at least 3 sample files for syntax errors before committing. Never trust regex on function signatures without validating the output.
Build phase plugin loading: Each integrator loads ONLY the plugins relevant to their build phase (DB/Schema, API/Backend, UI/UX, Integration, QA/Release). See BUILD PHASE → PLUGIN MAPPING section above.
This section tracks bugs that escaped to production and the QA rules added in response. It serves as a changelog for the learning loop and helps identify patterns over time.
const params = await paramsPromise; between the last function parameter and the closing ) {, instead of inside the function body after {.Promise<> but didn't verify the await was syntactically correct. No "verify output parses" step existed for batch fixes.Referenced by:
reference/INTEGRATOR_SKILL.md — §7.1 Pre-flight checklistreference/FINTECH_ABSOLUTES.md — ENFORCEMENT section.github/workflows/ci.yml — architecture validation jobtemplates/ORCHESTRATOR_SKILL_TEMPLATE.md — §3.1a QA skill docstemplates/ORCHESTRATOR_DASHBOARD.html — Skills & Templates sectionBlueBeast PMS — QA Gate Skill v4.2 29 automated checks + --deep self-simulation + 13 guide modules + debugging chamber (4-tier fallback) + learning loop + build phase→plugin mapping Every CI failure becomes a new check. Every production bug teaches the system.