Speckit Product Forge Test Plan

⚠️ Phase 7 (verify-full) must be completed before test planning. Run: /speckit.product-forge.verify-full

Set TESTING_DIR = {FEATURE_DIR}/testing/ Set BUGS_DIR = {FEATURE_DIR}/bugs/

Step 2: Auto-detect Project Setup

Before asking the user, scan the codebase to collect environment info:

Scanning project for test configuration...

Detect:

1. Test framework: Vitest / Jest / Mocha / pytest / RSpec / other
   • Look for: vitest.config.*, jest.config.*, package.json scripts.test, .mocharc.*
2. E2E framework: Playwright / Cypress / Selenium
   • Look for: playwright.config.*, cypress.json, cypress.config.*
3. Frontend entry point: Guess from config files
   • Look for: vite.config.*, next.config.*, nuxt.config.*
   • Extract server.port or dev script port
4. Backend entry point:
   • Look for: nest-cli.json, package.json main script, .env PORT variable
5. Existing test files:
   • Count *.spec.*, *.test.*, e2e/**, tests/**
6. Docker / CI config:
   • Look for: docker-compose.yml, .github/workflows/, Dockerfile
7. Environment files:
   • Look for: .env.example, .env.test, .env.local
   • Extract public variable names (NEVER log values)

Report findings:

🔍 Auto-detected:
  Test framework:    Vitest (vitest.config.ts found)
  E2E framework:     Playwright (playwright.config.ts found)
  Frontend:          http://localhost:5173 (vite.config.ts port: 5173)
  Backend:           http://localhost:3000 (.env PORT=3000)
  Existing tests:    47 spec files, 3 e2e files
  Docker:            docker-compose.yml found (services: app, db)
  Env template:      .env.example (12 variables)

Step 3: Environment Configuration Interview

Ask the user in ONE message, pre-filling any auto-detected values:

Test environment setup — please confirm or correct auto-detected values:

1. **Frontend URL:** {auto-detected or "?"} — where should Playwright navigate?
   (e.g., http://localhost:5173 or https://staging.myapp.com)

2. **Backend/API URL:** {auto-detected or "?"} — base URL for API tests
   (e.g., http://localhost:3000/api)

3. **Test user credentials:** Do tests require login?
   If yes — provide test account: email + password (will be stored only in testing/env.md, never in code)

4. **Additional env vars needed:** Any other credentials, API keys, or config for tests?
   (List names only, I'll ask for values if needed. e.g., "STRIPE_TEST_KEY, GOOGLE_CLIENT_ID")

5. **Test types to generate:**
   Select all that apply:
   - [ ] Smoke tests (critical path, runs first — ~5 min)
   - [ ] E2E Playwright tests (full user journeys — ~15-30 min)
   - [ ] API/Integration tests (endpoint contracts — ~5-10 min)
   - [ ] Regression tests (existing features shouldn't break — ~10-20 min)
   - [ ] Custom: ________________

6. **Test scope:** How thorough?
   - Minimal — happy path only + 2-3 critical edge cases
   - Standard — happy paths + main edge cases + error states
   - Full — all user stories + all acceptance criteria + all edge cases

7. **Browser/Platform targets for Playwright:**
   - [ ] Chrome/Chromium (default)
   - [ ] Firefox
   - [ ] Safari/WebKit
   - [ ] Mobile viewport (375px — iPhone)
   - [ ] Tablet viewport (768px)

Store: FRONTEND_URL, API_URL, TEST_TYPES, TEST_SCOPE, BROWSERS, HAS_AUTH.

Step 4: Create Environment Config

Create {TESTING_DIR}/env.md — stores test environment config (NOT a .env file — never commit credentials):

# Test Environment Config: {Feature Name}

> ⚠️ This file contains test credentials. Do NOT commit to version control.
> Add `testing/env.md` to .gitignore

## Environment

| Variable | Value | Source |
|----------|-------|--------|
| FRONTEND_URL | {url} | User input |
| API_URL | {url} | User input / auto-detected |
| TEST_SCOPE | {scope} | User selection |

## Auth Credentials (test account)
{If auth required — stored here for reference during test execution}
Test email: {email}
Test password: {password}

## Additional Variables
{list of additional env vars with values if provided}

## Browser Targets
{list of selected browsers}

## Notes
{Any special setup steps: seed data, feature flags to enable, etc.}

Also update .gitignore to add testing/env.md if not already present.

Step 5: Extract Test Cases from Feature Artifacts

Read and synthesize:

1. product-spec/product-spec.md → Must Have user stories + acceptance criteria
2. product-spec/user-journey*.md → All user flows, steps, decision branches
3. spec.md → Acceptance criteria (may be more detailed than product-spec)
4. research/ux-patterns.md → Edge cases and state inventory

Build a structured test case matrix:

5A: Smoke Test Cases (always first)

Derive 4–8 critical-path scenarios that answer: "does the feature basically work?"

## Smoke Tests (TC-SMK-NNN)

| ID | Title | Steps | Expected | Priority |
|----|-------|-------|----------|----------|
| TC-SMK-001 | Feature loads without error | 1. Navigate to {URL} 2. Feature renders | No JS errors, content visible | P0 |
| TC-SMK-002 | Primary action works | 1. Perform {main action} | {expected outcome} | P0 |
[...]

5B: E2E Test Cases (per user journey)

For each user journey file, create test cases for:

• Primary happy path (all steps complete)
• Each alternative path
• Each error scenario from the journey

## E2E Tests: {Journey Name} (TC-E2E-NNN)

| ID | Journey | Scenario | Preconditions | Steps | Expected | Story |
|----|---------|----------|--------------|-------|----------|-------|
| TC-E2E-001 | {journey} | Happy path | {preconditions} | {numbered steps} | {outcome} | US-001 |
| TC-E2E-002 | {journey} | Empty state | No data | Navigate to feature | Empty state UI shown | US-001 |
[...]

5C: API Test Cases (if API tests selected)

For each API endpoint identified in the plan:

• Happy path (200/201)
• Invalid input (400)
• Unauthorized (401)
• Not found (404)
• Edge cases

## API Tests (TC-API-NNN)

| ID | Endpoint | Method | Input | Expected Status | Expected Body |
|----|----------|--------|-------|----------------|---------------|
| TC-API-001 | /api/feature | GET | valid token | 200 | {schema} |
| TC-API-002 | /api/feature | GET | no token | 401 | error message |
[...]

5D: Regression Test Cases (if selected)

Identify existing features that could be affected by this change:

• Look at integration points from research/codebase-analysis.md
• Look at shared components identified in plan.md

## Regression Tests (TC-REG-NNN)

| ID | Existing Feature | Risk | Test Scenario | Expected |
|----|----------------|------|--------------|----------|
| TC-REG-001 | {feature name} | {how new feature could break it} | {test} | {expected} |
[...]

Step 6: Generate Playwright Test Files

If E2E Playwright tests are selected, generate actual .spec.js / .spec.ts test files.

Create {TESTING_DIR}/playwright-tests/ folder.

For each E2E test group:

// {TESTING_DIR}/playwright-tests/{feature-slug}-{journey-name}.spec.ts
import { test, expect } from '@playwright/test';

// Feature: {Feature Name}
// Journey: {Journey Name}
// Generated by Product Forge Phase 8A
// Stories covered: {US-NNN list}

test.describe('{Feature Name} — {Journey Name}', () => {

  test.beforeEach(async ({ page }) => {
    // Setup: navigate and authenticate if needed
    await page.goto('{FRONTEND_URL}');
    {if HAS_AUTH:}
    // Login with test credentials
    await page.fill('[data-testid="email"]', process.env.TEST_EMAIL || '{test-email}');
    await page.fill('[data-testid="password"]', process.env.TEST_PASSWORD || '{test-password}');
    await page.click('[data-testid="login-submit"]');
    await page.waitForURL('**/dashboard');
  });

  // TC-E2E-001: Happy Path — {description}
  // Covers: US-001 acceptance criteria: {AC text}
  test('should {primary user outcome}', async ({ page }) => {
    // Arrange
    await page.goto('{feature URL}');

    // Act — Step 1: {action}
    await page.click('{selector}');

    // Act — Step 2: {action}
    await page.fill('{selector}', '{value}');

    // Act — Step 3: {action}
    await page.click('[data-testid="submit"]');

    // Assert
    await expect(page.locator('{result selector}')).toBeVisible();
    await expect(page.locator('{result selector}')).toContainText('{expected text}');
  });

  // TC-E2E-002: Empty State
  test('should show empty state when no data exists', async ({ page }) => {
    // Navigate to feature with no data
    await page.goto('{feature URL}?empty=true');
    await expect(page.locator('[data-testid="empty-state"]')).toBeVisible();
  });

  // TC-E2E-003: Error State
  test('should handle error gracefully', async ({ page }) => {
    // Simulate error condition
    await page.route('**/api/**', route => route.fulfill({ status: 500 }));
    await page.goto('{feature URL}');
    await expect(page.locator('[data-testid="error-message"]')).toBeVisible();
  });

});

Important notes in generated tests:

• Use data-testid selectors by default (most stable)
• Add comments mapping each test to US-NNN and AC text
• Use process.env for credentials (never hardcode)
• Include setup/teardown for test isolation

Create {TESTING_DIR}/playwright-tests/playwright.config.ts if not exists:

// Product Forge generated Playwright config for: {feature-slug}
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './playwright-tests',
  timeout: 30000,
  retries: 1,
  reporter: [['html', { outputFolder: '../testing/playwright-report' }]],
  use: {
    baseURL: process.env.FRONTEND_URL || '{FRONTEND_URL}',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
    trace: 'retain-on-failure',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    {FIREFOX_if_selected: name: 'firefox', use: { ...devices['Desktop Firefox'] }},
    {WEBKIT_if_selected: name: 'webkit', use: { ...devices['Desktop Safari'] }},
    {MOBILE_if_selected: name: 'mobile-chrome', use: { ...devices['Pixel 5'] }},
  ],
});

Step 7: Generate Test Plan Master Document

Create {TESTING_DIR}/test-plan.md:

# Test Plan: {Feature Name}

> Created: {date} | Phase: 8A
> Feature: `{feature-slug}`
> Environment: {FRONTEND_URL}

## Scope

### In Scope
{Features and flows being tested}

### Out of Scope
{What is explicitly NOT being tested — and why}

## Test Types & Estimated Duration

| Type | Count | Est. Duration | Files |
|------|-------|--------------|-------|
| Smoke | {N} | ~5 min | playwright-tests/{slug}-smoke.spec.ts |
| E2E Playwright | {N} | ~{N*2} min | playwright-tests/{slug}-*.spec.ts |
| API/Integration | {N} | ~{N} min | — |
| Regression | {N} | ~{N*3} min | playwright-tests/{slug}-regression.spec.ts |

**Total estimated:** ~{N} minutes

## Environment
- Frontend: {FRONTEND_URL}
- API: {API_URL}
- Browsers: {list}
- Auth required: {yes/no}

## Test Case Summary

### Coverage Matrix

| User Story | Smoke | E2E | API | Regression | Coverage |
|------------|-------|-----|-----|-----------|---------|
| US-001: {title} | TC-SMK-001 | TC-E2E-001,002 | TC-API-001 | — | ✅ Full |
| US-002: {title} | — | TC-E2E-005 | TC-API-003,004 | TC-REG-001 | ✅ Full |

### Complete Test Case Index
{Link to test-cases.md}

## Entry Criteria (before testing starts)
- [ ] All Phase 7 verify-full CRITICAL issues resolved
- [ ] Feature deployed to test environment
- [ ] Test data seeded / reset
- [ ] Playwright installed: `npx playwright install`
- [ ] Credentials configured in `testing/env.md`

## Exit Criteria (testing complete when)
- [ ] All P0 smoke tests PASS
- [ ] All E2E happy paths PASS
- [ ] ≥80% of all test cases PASS
- [ ] Zero P0/P1 open bugs
- [ ] All P2 bugs documented with workarounds

## Bug Severity Definition
| Severity | Definition | Examples |
|----------|-----------|---------|
| P0 Blocker | Cannot proceed with testing | App crashes, auth broken |
| P1 Critical | Core user journey broken | Primary action fails |
| P2 High | Important feature broken | Edge case fails, UX degraded |
| P3 Medium | Minor issue | Wrong text, small layout issue |
| P4 Low | Cosmetic | Typo, pixel misalignment |

## How to Run Tests
\`\`\`bash
# Smoke tests (run first)
npx playwright test --grep @smoke

# All E2E tests for this feature
npx playwright test testing/playwright-tests/{slug}-*.spec.ts

# Regression tests
npx playwright test --grep @regression

# Single test by ID
npx playwright test --grep "TC-E2E-001"

# Run with UI mode (debug)
npx playwright test --ui
\`\`\`

Step 8: Create Test Cases Document

Create {TESTING_DIR}/test-cases.md — all test cases in one searchable document.

Include all TC-SMK, TC-E2E, TC-API, TC-REG cases in full detail with:

• Preconditions
• Step-by-step instructions
• Expected result
• Linked story (US-NNN)
• Linked AC

Step 9: Create Bugs Folder

Initialize {BUGS_DIR}/README.md:

# Bug Tracker: {Feature Name}

> Feature: `{feature-slug}` | Testing started: {date}

## Dashboard

| Severity | Open | Fixed | Retested ✅ | Won't Fix |
|----------|------|-------|------------|-----------|
| P0 Blocker | 0 | 0 | 0 | 0 |
| P1 Critical | 0 | 0 | 0 | 0 |
| P2 High | 0 | 0 | 0 | 0 |
| P3 Medium | 0 | 0 | 0 | 0 |
| P4 Low | 0 | 0 | 0 | 0 |
| **Total** | **0** | **0** | **0** | **0** |

## Bug List

| ID | Title | Severity | Status | Test Case | Assigned |
|----|-------|----------|--------|-----------|---------|

## Status Legend
🔴 Open · 🟡 In Progress · 🟢 Fixed · ✅ Verified · ❌ Won't Fix

Step 10: Update Status