Add Hosted Key

Step 1: Register the BYOK Provider ID

Add the new provider to the BYOKProviderId union in tools/types.ts:

export type BYOKProviderId =
  | 'openai'
  | 'anthropic'
  // ...existing providers
  | 'your_service'

Then add it to VALID_PROVIDERS in app/api/workspaces/[id]/byok-keys/route.ts:

const VALID_PROVIDERS = ['openai', 'anthropic', 'google', 'mistral', 'your_service'] as const

Step 2: Research the API's Pricing Model and Rate Limits

Before writing any getCost or rateLimit code, look up the service's official documentation for both pricing and rate limits. You need to understand:

Pricing

1. How the API charges — per request, per credit, per token, per step, per minute, etc.
2. Whether the API reports cost in its response — look for fields like creditsUsed, costDollars, tokensUsed, or similar in the response body or headers
3. Whether cost varies by endpoint/options — some APIs charge more for certain features (e.g., Firecrawl charges 1 credit/page base but +4 for JSON format, +4 for enhanced mode)
4. The dollar-per-unit rate — what each credit/token/unit costs in dollars on our plan

Rate Limits

1. What rate limits the API enforces — requests per minute/second, tokens per minute, concurrent requests, etc.
2. Whether limits vary by plan tier — free vs paid vs enterprise often have different ceilings
3. Whether limits are per-key or per-account — determines whether adding more hosted keys actually increases total throughput
4. What the API returns when rate limited — HTTP 429, Retry-After header, error body format, etc.
5. Whether there are multiple dimensions — some APIs limit both requests/min AND tokens/min independently

Search the API's docs/pricing page (use WebSearch/WebFetch). Capture the pricing model as a comment in getCost so future maintainers know the source of truth.

Setting Our Rate Limits

Our rate limiter (lib/core/rate-limiter/hosted-key/) uses a token-bucket algorithm applied per billing actor (workspace). It supports two modes:

• per_request — simple; just requestsPerMinute. Good when the API charges flat per-request or cost doesn't vary much.
• custom — requestsPerMinute plus additional dimensions (e.g., tokens, search_units). Each dimension has its own limitPerMinute and an extractUsage function that reads actual usage from the response. Use when the API charges on a variable metric (tokens, credits) and you want to cap that metric too.

When choosing values for requestsPerMinute and any dimension limits:

• Stay well below the API's per-key limit — our keys are shared across all workspaces. If the API allows 60 RPM per key and we have 3 keys, the global ceiling is ~180 RPM. Set the per-workspace limit low enough (e.g., 20-60 RPM) that many workspaces can coexist without collectively hitting the API's ceiling.
• Account for key pooling — our round-robin distributes requests across N hosted keys, so the effective API-side rate per key is (total requests) / N. But per-workspace limits are enforced before key selection, so they apply regardless of key count.
• Prefer conservative defaults — it's easy to raise limits later but hard to claw back after users depend on high throughput.

Step 3: Add hosting Config to the Tool

Add a hosting object to the tool's ToolConfig. This tells the execution layer how to acquire hosted keys, calculate cost, and rate-limit.