Clickwrap → Salesforce (Tray) Sync Triage

When to use / when not to use

Use when:

• CLICKWRAP / clickthrough contract + Salesforce sync-back / KP fields
• Empty Tray logs for contract id filter
• Discussion of solution instance disabled, re-enable, retrigger webhooks

Do not use for:

• 400 on clickwrap submit API → clickwrap-400-submission-triage
• download_link 404 / ContractVersionNotFound right after create → contract-lookup (CLICKWRAP + Tray race)
• HubSpot / SFDC questionnaire intake → other skills

Investigation workflow

Step 1: Confirm contract shape

1. {contract_id} — open in app + Django admin.
2. Confirm kind = CLICKWRAP / clickthrough path.
3. Check ExternalIntegrationContractDetail (or equivalent) — salesforce_id may be present without KP sync having run.

Step 2: Tray solution instance status

1. In Tray.io, open the workspace workflow used for reverse sync / KP Created (customer-specific; get link from CS or integration docs).
2. Confirm solution instance is enabled and not paused — note who disabled and when (GCP audit / Tray audit if available).

From incident thread (example timeline — use as pattern only):

• Solution disabled → sync stops immediately for new contracts.
• Solution re-enabled → new contracts may sync; old ones need retrigger.

Step 3: Native vs Tray

Ask integrations / IE: Is Native Salesforce reverse sync enabled for this customer?

• If Native is on for clickthrough path and Tray was disabled, behavior can be confusing (Native may not support clickthrough KP sync the same way).
• Mitigation pattern from incident: Re-enable Tray for reverse sync and disable Native reverse sync until CI-2100-class work ties clickthrough audits to a workflow.

Step 4: BigQuery — Tray table (optional)

Use contract-lookup skill — Prod USA Tray table:

SELECT id, solution_instance_id, is_enabled, is_installed, created, modified
FROM `spotdraft-prod.prod_usa_db.public_tray_integrations_traysolutioninstance`
WHERE created_by_workspace_id = {workspace_id}
  AND is_enabled = TRUE
  AND is_installed = TRUE

Column is created_by_workspace_id, not workspace_id — see contract-lookup.

Step 5: GCP logs

Incident referenced Cloud Logging short links for Tray disable/enable events — use similar filters for audit who toggled integration flags (exact log names depend on current infra; ask #integrations-oncall if unsure).

Mitigation: bulk retrigger KP Created (clickthrough)

After Tray is enabled and Native conflict is resolved:

1. Resolve numeric {workspace_id} (e.g. support may say US_147206 → 147206 — verify in admin / BQ).
2. Define date range of affected contracts (example from incident: 2026-02-04 through 2026-02-26).
3. Run management command dry run first (command ships in django-rest-api — verify current name/flags in repo):

# Dry run (preview only)
python manage.py retrigger_kp_created_webhook_for_clickthrough_contracts \
    --workspace_id={workspace_id} \
    --start_date=YYYY-MM-DD \
    --end_date=YYYY-MM-DD \
    --dry_run

# Execute
python manage.py retrigger_kp_created_webhook_for_clickthrough_contracts \
    --workspace_id={workspace_id} \
    --start_date=YYYY-MM-DD \
    --end_date=YYYY-MM-DD

Operational notes from incident:

• Script/command may require auth user and org user ids aligned with clickwrap creator from MergeBase (MB) — confirm with author of the PR before prod run.
• Code review / merge: e.g. django-rest-api PR #12411 (comment thread referenced in Slack); initial implementation in PR #12269.
• Internal doc (if present in repo): docs/incidents/ABNORMAL_SECURITY_RETRIGGER_KP_WEBHOOK_MITIGATION.md

4. Verify: Tray logs for sample contract_ids + Salesforce fields populated.

Long-term fix

Until then, document Tray dependency for this customer’s clickthrough → SFDC KP sync.

Confirm / disprove

Escalation triggers

• Many workspaces affected after a platform change → integrations platform team.
• Single customer, confirmed Tray gap → runbook + CS; IE for Tray/Native toggles.
• Command fails or wrong user context → django-rest-api owners + dry run only until fixed.

Related incidents & Jira

Connections to other skills

• contract-lookup — workspace_id, CLICKWRAP + Tray download_link race (different symptom).
• clickwrap-400-submission-triage — Contract never created / API 400.
• incident-lookup — Row for Tray download_link race on WSID 147206.

Operational note

Run BigQuery queries one at a time (repo README — parallel BQ MCP calls may 500).