Triage clickthrough (clickwrap) contracts that exist in SpotDraft with salesforce_id but Salesforce fields stay blank — Tray shows no logs for KP Created / reverse sync. Covers Tray solution disabled, Native vs Tray conflict for clickthrough, and bulk retrigger of KP Created webhooks. Use when: clickwrap not syncing to Salesforce, Abnormal-style Tray integration, empty Tray logs, reverse sync clickthrough, KP created webhook, US_147206 pattern. Trigger phrases: clickthrough Salesforce blank fields, Tray solution disabled, retrigger KP created, native integration clickwrap conflict.
Post-execution issues: the clickwrap contract is created in SpotDraft (often with salesforce_id / external link), but mapped Key Pointer (KP) fields on the Salesforce Account or Opportunity stay empty. Customer may report no entries in Tray logs for that contract_id.
Common root cause (production-confirmed pattern): The Tray.io solution instance that listens for KP Created (or equivalent reverse-sync webhooks) was disabled for a date range. Contracts created while disabled never triggered a successful Tray run — enabling Tray again does not retroactively send webhooks.
Architectural constraint: Native reverse sync expects contracts tied to a workflow. Clickthrough audits are often tied to a contract type only, not a workflow — so Native reverse sync can fail for clickthrough while custom sync works. for the same flow without explicit integration-team approval (incident: mutual exclusion required — for mitigation).
Use when:
Do not use for:
ContractVersionNotFound right after create → contract-lookup (CLICKWRAP + Tray race){contract_id} — open in app + Django admin.salesforce_id may be present without KP sync having run.From incident thread (example timeline — use as pattern only):
Ask integrations / IE: Is Native Salesforce reverse sync enabled for this customer?
Use contract-lookup skill — Prod USA Tray table:
SELECT id, solution_instance_id, is_enabled, is_installed, created, modified
FROM `spotdraft-prod.prod_usa_db.public_tray_integrations_traysolutioninstance`
WHERE created_by_workspace_id = {workspace_id}
AND is_enabled = TRUE
AND is_installed = TRUE
Column is
created_by_workspace_id, notworkspace_id— see contract-lookup.
Incident referenced Cloud Logging short links for Tray disable/enable events — use similar filters for audit who toggled integration flags (exact log names depend on current infra; ask #integrations-oncall if unsure).
After Tray is enabled and Native conflict is resolved:
{workspace_id} (e.g. support may say US_147206 → 147206 — verify in admin / BQ).# Dry run (preview only)
python manage.py retrigger_kp_created_webhook_for_clickthrough_contracts \
--workspace_id={workspace_id} \
--start_date=YYYY-MM-DD \
--end_date=YYYY-MM-DD \
--dry_run
# Execute
python manage.py retrigger_kp_created_webhook_for_clickthrough_contracts \
--workspace_id={workspace_id} \
--start_date=YYYY-MM-DD \
--end_date=YYYY-MM-DD
Operational notes from incident:
docs/incidents/ABNORMAL_SECURITY_RETRIGGER_KP_WEBHOOK_MITIGATION.mdcontract_ids + Salesforce fields populated.| Tracker | Purpose |
|---|---|
| CI-2100 | Associate clickthrough audits with a workflow so Native integration can replace Tray for reverse sync (Jira) |
Until then, document Tray dependency for this customer’s clickthrough → SFDC KP sync.
| Evidence | Conclusion |
|---|---|
| Tray disabled in window; no logs for contract | Missing webhooks — retrigger after fix |
| Tray enabled; logs show 4xx/5xx | Payload / auth — different triage |
| Native on + Tray off; clickthrough only | Wrong integration path for KP sync |
| download_link 404 at create time | Race — contract-lookup CLICKWRAP section |
| Reference | Notes |
|---|---|
| SPD-42002 | Task / incident ticket linked from Rootly |
| Rootly #2954 | Abnormal Security — retrigger KP Created |
| Slack | #incident-20260305-medium-abnormal-security-retrigger-kp-created-webhook-for-clic (C0AJQH11XHT) |
| Example WS | 147206 (US) — verify every time |
| Example contract | 1390887 (from integrations thread — example only) |
workspace_id, CLICKWRAP + Tray download_link race (different symptom).Run BigQuery queries one at a time (repo README — parallel BQ MCP calls may 500).