Synthdata Prompt Builder

Key rule: if a table requires reading generated data and performing aggregation, grouping, scoring, or cross-table joins — it's derived, and synthdata-generate cannot produce it. Use synthdata-compute instead.

Step 3: Match to templates

Check if any of the 12 built-in templates cover the raw tables. If so, start there. If not, describe a custom schema for synthdata-generate's interview flow.

Step 4: Output a sequenced plan

Format the plan as numbered steps with the specific command or prompt for each:

Step 1: [synthdata-generate] Generate raw tables from template X
Step 2: [synthdata-compute]  Derive monthly rollups from raw events
Step 3: [synthdata-generate] Generate peer benchmark dataset (custom schema)
Step 4: [synthdata-compute]  Compute segment summaries from peer data
Step 5: [synthdata-extract]  Extract everything to JSON for the downstream tool

Offer to execute Step 1 immediately.

Template Catalog

The 12 built-in templates and what they cover. Use --list-templates to confirm:

What the Engine CAN Generate Directly

synthdata-generate produces raw tabular data with:

• Column types: id, faker (100+ Faker methods), choice (weighted), int/float (7 distributions: uniform, normal, lognormal, exponential, poisson, gamma, pareto), bool, date, timestamp, constant, formula (same-row only), ref (copy parent column to child)
• Relationships: parent → child foreign keys with zipfian or uniform distribution
• Profiles: behavioral archetypes (e.g., high_risk/baseline/champion) with weighted assignment and jittered numeric overrides that condition child-table column values
• Temporal patterns: uniform, business-hours, diurnal; weekday-only option
• Effort scaling: quick (~100 rows) / medium (~1K) / thorough (~5K+)
• Output formats: xlsx, csv, json, sql, parquet

What the Engine CANNOT Generate

These require synthdata-compute or LLM-driven generation:

• Cross-table aggregations — e.g., "count of threat_events per user per month" (requires reading generated threat_events and grouping)
• Composite scores — e.g., weighted risk scores from multiple sub-scores derived from different event tables
• Percentile ranks — relative positioning across a population
• Rolling windows / trends — month-over-month change, moving averages
• Multi-table joins + aggregation — e.g., monthly_risk derived from 5 event tables joined on user_id and grouped by month
• Correlated peer benchmarks — 200 organizations with cross-metric correlations (e.g., high maturity → low click rate → high MFA adoption). These need LLM reasoning to produce realistic covariance structure
• Summary statistics — mean/median/p25/p75 across segments

The formula column type works only within a single row of a single table. It cannot reference other tables, aggregate child rows, or perform lookups.

Common Derived-Data Patterns

Reference recipes for synthdata-compute. Include the relevant pattern in the plan description.

Monthly event rollup

Group events by (entity_id, month), count/sum event columns. Used for time-series dashboards, trend analysis, scoring.

Composite score from weighted sub-scores

Compute sub-scores per dimension (e.g., threat=35%, sim=25%, dlp=20%, training=10%, reporting=10%), then weighted sum → composite (0–100). Apply tier thresholds.

Risk tier assignment

Map numeric scores to categorical tiers via thresholds (e.g., Critical >=75, High 50-74, Medium 25-49, Low <25).

Percentile rank

df['percentile'] = df['score'].rank(pct=True) * 100 — useful for "where does this user/org stand relative to peers."

Segment summary

Group a population by segment flags, compute mean/median/p25/p75 for each metric. Common for peer benchmarking.

Department / team breakdown

Join user dimension to event/score tables, group by department, compute averages and counts.

Worked Example: Healthcare Human Risk Management

User says: "I need healthcare risk data for a security product demo — user risk scores, threat events, peer benchmarks, department summaries."

Plan:

Step 1: [synthdata-generate]
  Generate raw event tables from the healthcare-hrm-security template at medium effort.
  > synthdata-generate --template healthcare-hrm-security --effort medium --seed 42
  Produces: users (1K), threat_events (~90K), phishing_sims (12K), training (5K),
  dlp_events (~850), abuse_mailbox (~22K)

Step 2: [synthdata-compute]
  Derive monthly_risk from the raw event tables:
  - Group threat_events by (user_id, month) → count events, sum clicks
  - Group phishing_sims by user_id → sim click rate, report rate
  - Group dlp_events by user_id → count violations
  - Group training by user_id → completion rate, avg score
  - Group abuse_mailbox by user_id → report count, confirmed-malicious ratio
  - Compute sub-scores (0-100 each), weighted composite (Threat 35%, Sim 25%,
    DLP 20%, Training 10%, Reporting 10%), assign risk tiers

Step 3: [synthdata-generate or LLM-driven]
  Generate 200 peer organizations with correlated metrics.
  This is best done via a prompted generation (not a template) because the cross-metric
  correlations (mature orgs have lower click rates, higher MFA adoption, etc.) require
  reasoning that the schema engine can't express.
  > "Generate a peer benchmarks dataset: 200 healthcare-adjacent organizations with
  >  company info, security maturity, sim/threat/DLP/training metrics, composite risk
  >  scores, and segment flags for healthcare/geography/size. Include realistic
  >  cross-metric correlations."

Step 4: [synthdata-compute]
  Derive segment_summaries from the peer data:
  - Group peers by each segment flag
  - Compute mean/median/p25/p75 for each metric
  - Add the user's organization as a comparison row

Step 5: [synthdata-extract]
  Extract the final xlsx workbooks to JSON for the downstream product:
  > synthdata-extract --input <company_data>.xlsx --output json/
  > synthdata-extract --input <peer_data>.xlsx --output json/

Worked Example: E-Commerce Fraud Detection

User says: "I need transaction data to test a fraud detection model — transactions, user profiles, and fraud labels with known fraud rings."

Plan:

Step 1: [synthdata-generate]
  Generate raw tables from the financial-transactions template at thorough effort.
  > synthdata-generate --template financial-transactions --effort thorough --seed 42
  Produces: customers (~5K), accounts (~8K), transactions (~100K+) with fraud_ring profiles

Step 2: [synthdata-compute]
  Derive fraud_labels table:
  - Flag transactions from fraud_ring profile users with amount > p95 as "suspicious"
  - Flag transactions with velocity > 3 per hour from same account as "velocity_alert"
  - Compute per-customer risk features (avg transaction amount, transaction frequency,
    unique merchant count, max single transaction)

Step 3: [synthdata-compute]
  Derive daily_summary:
  - Group transactions by date → total volume, total amount, fraud count, fraud rate
  - Compute 7-day rolling averages for trend analysis

Worked Example: IoT Monitoring Dashboard

User says: "I need sensor data for a monitoring dashboard — device readings, alerts, and health summaries."

Plan:

Step 1: [synthdata-generate]
  Generate from iot-sensors template at medium effort.
  > synthdata-generate --template iot-sensors --effort medium --seed 42
  Produces: devices, readings, events

Step 2: [synthdata-compute]
  Derive hourly_averages:
  - Group readings by (device_id, hour) → mean/min/max for each metric
  - Flag anomalies where value > 3 standard deviations from device mean

Step 3: [synthdata-compute]
  Derive device_health_summary:
  - Per device: uptime %, alert count, last reading timestamp, current status

Interaction Pattern

1. Listen — let the user describe their domain and needs
2. Classify — identify raw tables, derived tables, and independent reference datasets
3. Match — check templates for raw tables; note what needs computation or LLM generation
4. Plan — output a sequenced, numbered plan with specific commands/prompts
5. Offer — "Want me to start with Step 1?"

If the user's needs are simple (single template, no derived tables), skip the planning and suggest going directly to synthdata-generate.