Test Data Management Strategy

Phase 1 — Setup

**Environments**: [list]
**Data classes**: [PII + financial + health + proprietary + non-sensitive]
**Compliance**: [GDPR / HIPAA / PCI / sector]
**Test levels**: [unit / component / integration / E2E / perf / UAT]
**Scale**: [baseline + perf sizing]
**Existing tooling**: [factories / seeds / snapshots / masking tools]
**Current pain**: [flaky due to data / prod data leaking / slow seeds]

Disclaimer: Not legal advice. Compliance decisions require counsel and DPO.

Ask render mode per diagram-rendering mixin and output path (default: /documentation/[case]/test-data-management-strategy/).

Phase 2 — Data source strategies

A. Synthetic (preferred default for lower envs)

• Generated via factories / builders / faker libraries with domain constraints
• No PII risk; deterministic
• Suited for: unit, component, integration, most E2E
• Scales cheaply; reproducible via seeds

B. Masked / pseudonymized

• Redact + tokenize sensitive fields; keep schema + distribution
• Suited for: realistic data distributions (analytics, search relevance, large-joins)
• Risk: re-identification via quasi-identifiers (ZIP + DOB + gender)
• Use k-anonymity / l-diversity where possible
• Requires DPA + policy sign-off

C. Subset from production (curated slice)

• Cherry-picked + masked subset to limit volume
• Suited for: upgrade testing, perf at approximated scale
• Risk: accidental selection of sensitive edge cases; subset drift over time

D. Frozen golden datasets

• Versioned + immutable; source-of-truth for regression assertions
• Suited for: ML eval sets, compiler tests, algorithm tests, billing invariants
• Risk: staleness; coupling between tests + data

E. Live / sandbox third-party data

• Vendor sandbox (e.g., Stripe test keys) with fake cards
• Suited for: external-integration tests
• Risk: sandbox outages + rate limits + behavioral drift

Choose per environment × test level — not one-size-fits-all.

Phase 3 — Per-environment data matrix

Prod data never copied raw to any env.

Phase 4 — Masking techniques

Select per field per need:

Phase 5 — Re-identification risk

Quasi-identifiers combined can re-identify:

• Age + ZIP + gender uniquely identifies a large % of US residents
• Timestamp + location → likely identity
• Long chains of actions → behavior fingerprint

Mitigations:

• k-anonymity (≥ k identical records on quasi-identifiers)
• l-diversity (≥ l distinct sensitive values per equivalence class)
• Differential privacy for aggregate shares
• Avoid joining masked data with other sources in lower envs

Evaluate risk per dataset. Document in a DPIA attachment when required.

Phase 6 — Lifecycle

Generation

• Factories produce valid domain aggregates: OrderFactory.build() with sensible defaults
• Composable overrides: OrderFactory.build(status: 'refunded')
• Libraries: factory_bot (Ruby) / factory-boy (Python) / fishery (JS) / Bogus (.NET) / Faker in all

Seeding

• Deterministic seeds per test
• Idempotent seed scripts (re-runnable)
• Transactional where possible (rollback post-test)
• Testcontainers: spin a fresh DB per suite run or per worker

Reset

• Truncate + reseed between tests to avoid cross-contamination
• Snapshot/restore patterns (e.g., pg-restore, BEGIN; ... ROLLBACK;)
• Avoid sleep-based waits — use event / polling

Refresh

• Staging refresh cadence (weekly, monthly)
• Refresh pipeline: extract subset → mask → load → smoke test
• Promote only after masking validated

Phase 7 — Scale needs (perf testing)

• Generate sized data deterministically with known distributions
• Seed = config + commit-sha for reproducibility
• Use parallel generators for speed
• Store in a performance-env database separate from staging
• Warm caches before measurement

Hand off perf scenarios to non-functional-test-planning.

Phase 8 — Reference / master data

Some data isn't test-specific — reference tables (currencies, countries, SKU catalog). Policy:

• Source of truth: repo + migration seeds
• Versioned with the app
• Kept minimal in lower envs unless needed for integration

Phase 9 — Third-party sandboxes

• Document vendor-sandbox requirements per integration
• Rotate credentials; never use prod vendor keys in lower envs
• Record fixtures from sandbox where flakiness or rate limits hurt
• Decision: live sandbox vs recorded fixtures per test reliability needs

Phase 10 — Data tooling

Phase 11 — Governance

• Owner of test-data strategy (QA lead / data engineer)
• Review cadence (quarterly)
• Incident: if prod data detected in lower env → remove + investigate + DPO
• Audit trail for refresh pipelines

Phase 12 — Anti-patterns

Phase 13 — Metrics

• Seed time per test / per suite
• Data-caused flake count
• PII incidents in lower envs (target: 0)
• Refresh pipeline success + duration
• Re-identification risk score per masked dataset

Phase 14 — Diagrams

Data flow across environments

flowchart LR
    Prod[(Prod)] -->|never raw| X[blocked]
    Prod -->|extract + mask| Masker[Mask pipeline]
    Masker --> Staging[(Staging)]
    Masker --> Perf[(Perf)]
    Factories[Factories] --> Dev[(Dev)]
    Factories --> CI[(CI)]
    Factories --> Integration[(Integration)]
    Golden[(Golden datasets)] --> Integration
    Golden --> UAT[(UAT)]

Masking technique selection

flowchart TD
    F[Field]
    F --> Q{PII class?}
    Q -->|direct PII| D[Pseudonymize / tokenize]
    Q -->|quasi-id| G[Generalize]
    Q -->|regulated: card/SSN| T[Tokenize via vault]
    Q -->|not sensitive| N[Keep as-is]

Phase 15 — Diagram rendering

Per diagram-rendering mixin.

Phase 16 — Report assembly and approval

# Test Data Management Strategy: [Product]

**Date**: [date]
**Environments**: [...]
**Data classes**: [...]
**Compliance**: [...]

> Disclaimer: Not legal advice.

## Scope
## Data Source Strategies
## Per-Environment Data Matrix
## Masking Techniques
## Re-Identification Risk
## Lifecycle (Generation / Seeding / Reset / Refresh)
## Scale Needs
## Reference / Master Data
## Third-Party Sandboxes
## Tooling
## Governance
## Anti-Patterns to Avoid
## Metrics
## Diagrams
## Hand-offs
## Assumptions & Limitations

Present for user approval. Save only after confirmation.

Assessment + planning rules

• Synthetic-first for lower envs
• Never raw prod PII below staging
• Determinism via seeds
• Re-identification risk evaluated
• Reset between tests
• Disclaimer present
• No fabricated compliance claims

Failure behavior

Self-check

[] Synthetic-first per env
[] Never raw prod PII in lower envs
[] Masking technique per field class
[] Re-identification risk addressed
[] Deterministic seeds
[] Reset strategy per test
[] Refresh pipeline (if masked / subset)
[] Third-party sandbox policy
[] Tooling chosen
[] Governance ownership named
[] Metrics + incidents-0 target
[] Disclaimer present
[] Diagrams valid
[] No fabricated claims
[] Report follows output contract