Third Party Api Evaluation

Phase 1 — Setup

**API / vendor**: [name + product]
**Use case**: [what problem it solves for us]
**Volume forecast**: [req/month, data volume]
**Alternatives**: [list or "open"]
**Must-have features**: [functional non-negotiables]
**Regulatory context**: [GDPR, HIPAA, PCI, sector]
**Decision timeframe**: [when]
**Budget envelope**: [ballpark monthly]

Ask render mode per diagram-rendering mixin and output path (default: /documentation/[case]/third-party-api-evaluation/).

Phase 2 — Evaluation dimensions

1. Functional coverage

• Does it do what we need? Feature-by-feature against must-have list
• Gaps: workarounds, missing, roadmap-promised
• Versioning — stable vs beta endpoints

2. Contract stability

• Versioning policy published?
• Breaking-change history — changelog evidence
• Deprecation notice period
• Sandbox / test environment provided

3. Reliability

• SLA — uptime %, credits, scope
• Public status page — incident frequency + RCA quality
• Regional outage patterns
• MTTR public data

4. Security

• Auth: OAuth2 / API key / mTLS / JWT flows
• TLS minimums
• Certifications: SOC 2 Type II, ISO 27001, PCI DSS, HIPAA BAA, GDPR DPA
• Pentest reports — available under NDA?
• Incident response SLA

5. Rate limits + quotas

• Per-minute / per-day / per-account
• Burst capacity
• 429 response + Retry-After adherence
• Enterprise / custom limits available

6. Pricing + cost scaling

• Pricing model: per-request / per-seat / per-GB / flat tier
• Cost at projected volume
• Cost curve — linear, tiered, exponential
• Hidden costs: egress, support tier, premium features
• Overage behavior

7. Data residency + GDPR

• Region of processing
• Data processing agreement
• Sub-processors list
• Right to erasure / export / rectification mechanisms
• Data retention defaults

8. Support + docs

• Docs quality — examples, SDKs, changelogs
• SDK coverage for our languages
• Response SLA on support tickets
• Community — forums, Discord/Slack, Stack Overflow answers
• Enterprise contacts available

9. Lock-in risk

• Data portability — export available in open formats?
• Proprietary features vs standards
• SDK-specific idioms baked into our code
• Contract-level lock-in (annual, multi-year)

10. Exit strategy

• How long to migrate off?
• Alternatives viable?
• Abstraction layer possible?
• Data export fidelity

11. Operational fit

• Timezone overlap with our support hours
• Communication channels + escalation path
• Change-management cadence alignment
• Roadmap transparency

Phase 3 — Scoring matrix

Score 1–5 per dimension for each candidate (including "build in-house" if relevant).

Compute weighted score. Use as an input to decision, not the decision itself.

Phase 4 — Risk register

Phase 5 — Proof-of-concept plan

• Scope: smallest integration that proves real value
• Duration: 1–4 weeks typical
• Success criteria: measurable (latency, correctness, cost)
• Exit criteria: when we stop and decide
• Cost cap: sandbox + trial budget

Phase 6 — Build vs buy handoff

If build-in-house is a serious option, hand off to build-vs-buy-analysis for deeper TCO modeling.

Phase 7 — Recommendation

One paragraph:

• Recommended option
• Why: top 2–3 dimensions
• Trade-offs accepted
• Top risks + mitigations
• PoC plan (if decision not final)
• Exit strategy

Phase 8 — Diagrams

Dimension radar

xychart-beta
    title "Candidates across dimensions (1-5)"
    x-axis ["Func", "Stab", "Rel", "Sec", "Rate", "Price", "Res", "Supp", "Lock", "Exit"]
    y-axis "Score" 0 --> 5
    bar [5, 4, 4, 5, 3, 3, 4, 4, 3, 3]
    bar [4, 5, 3, 4, 4, 4, 3, 4, 4, 4]

Bars = Vendor A / Vendor B.

Cost at forecast

xychart-beta
    title "Monthly cost at projected volume"
    x-axis ["10k/mo", "100k/mo", "1M/mo", "10M/mo"]
    y-axis "USD" 0 --> 10000
    line [50, 300, 2000, 9000]
    line [100, 400, 1500, 7000]

Phase 9 — Diagram rendering

Per diagram-rendering mixin.

Phase 10 — Report assembly and approval

# Third-Party API Evaluation: [Vendor(s)]

**Date**: [date]
**Use case**: [...]
**Recommended**: [vendor + "subject to PoC" if applicable]

> Not legal or contractual advice. This is an engineering/procurement input artifact.

## Scope
[Use case, volume, alternatives, regulatory context, timeframe, budget]

## Evaluation Dimensions
[All 11 dimensions filled]

## Scoring Matrix
[Weighted scores per candidate]

## Risk Register
[Risks + mitigations]

## Proof-of-Concept Plan
[Scope + duration + success criteria + exit]

## Recommendation
[Choice + rationale + trade-offs + top risks + exit strategy]

## Diagrams
[Radar + cost curve]

## Assumptions & Limitations
[What's [unknown]; vendor claims vs verified]

Present for user approval. Save only after confirmation.

Assessment + planning rules

• Use case drives evaluation
• Every dimension scored or marked [unknown]
• Lock-in + exit explicit
• Risks + mitigations listed
• PoC plan if decision pending
• Not legal / contractual advice
• No fabricated features / pricing

Failure behavior

Self-check

[] Use case + volume declared
[] All 11 dimensions covered or [unknown]
[] Scoring matrix with weights
[] Risk register with mitigations
[] PoC plan if applicable
[] Exit strategy explicit
[] Disclaimer (not legal advice)
[] Diagrams valid
[] No fabricated features / pricing
[] Report follows output contract