Errors & rate limits

Every error, on every endpoint, uses one envelope:

{
  "error": {
    "code": 401,
    "message": "Missing or invalid API key",
    "metadata": { "error_type": "authentication" }
  }
}

code repeats the HTTP status. metadata.error_type is a stable machine-readable classifier — branch on it, not on message, which is written for humans and may change. Some errors add further metadata members, listed below where they exist. Monetary members among them (required, spendable, limit, used) are integer strings in Heimdallm's internal fixed-point unit (10⁻²⁴ USD) — compare them to each other, not to dollar figures.

Statuses#

Retrying. 408, 429, 502, 503 and 504 are worth retrying with exponential backoff — routing re-runs on every attempt, so the retry may land on a different endpoint entirely. A restart of the marketplace is the case to wait out rather than give up on: it answers 503 with Retry-After: 5 and X-Should-Retry: true, and a client that keeps retrying is served by the replacement within seconds. 4xx responses other than those will fail the same way until the request changes. During a routed request Heimdallm already fails over between eligible endpoints on its own; errors that no other endpoint could fix (an invalid body, an oversized payload, a client abort) are returned immediately instead.

Provider failures carry evidence. When the terminal error came from a provider, metadata includes provider_status (their HTTP status) and usually provider_message (a short excerpt of their error text), plus provider_code where one was given. A provider rejecting our upstream credential is reported as a plain 502 — it is not your authentication problem.

For a failure attributed to the provider, metadata.provider_name names the provider whose terminal attempt failed. metadata.raw, when a response body was received, contains its original text: the HTTP body or the failing SSE event's data payload, including its whitespace. It is capped at 4,096 UTF-8 bytes without splitting a character, so a truncated JSON body may not parse. No response body is invented for a disconnect or timeout. These members are carried identically in HTTP errors and mid-stream error chunks and are omitted for caller-caused errors, blocked addresses and marketplace failures. They add no stored response-body history.

Rate limits#

Two request-rate buckets protect the public API, both over a 60-second sliding window:

A client address is an IPv4 address, or the /64 prefix of an IPv6 address, which is the block one subscriber is assigned. A HEAD request counts against the same bucket as the GET it mirrors.

Every counted response carries the state of your bucket:

X-RateLimit-Limit: 600
X-RateLimit-Remaining: 597
X-RateLimit-Reset: 1755300060

X-RateLimit-Reset is Unix seconds. When the bucket is exhausted the request is refused with 429 and a Retry-After header (seconds); back off until then. Rate limits count requests, not tokens — batch where the API allows it (embeddings input arrays, longer conversations per call) rather than multiplying round-trips.

{
  "error": {
    "code": 429,
    "message": "Rate limit exceeded",
    "metadata": { "error_type": "rate_limit_exceeded" }
  }
}

Spending ceilings are a separate mechanism — a breached spend limit is a 402, described in Pricing & billing.

Errors while streaming#

Once a streaming response has started, the HTTP status can no longer change. A failure then arrives in-band as a final chunk with an error member — the same envelope shape — and finish_reason: "error", followed by data: [DONE]:

data: {"id":"gen-9f2c7e1a4b6d","object":"chat.completion.chunk","created":1755300000,"model":"deepseek/deepseek-v4-flash-0731-abliterated","provider":"ImpCC","error":{"code":502,"message":"Provider disconnected","metadata":{"error_type":"provider_error"}},"choices":[{"index":0,"delta":{"content":""},"finish_reason":"error"}]}

data: [DONE]

Treat such a chunk as terminal for that request. A provider that closes its stream after part of the answer, with no finish reason, no usage report and no [DONE], fails the same way, with a 502 whose message is The provider closed the stream before the answer finished. A 503 chunk is the marketplace restarting under a stream: its message says to try again in 5 seconds, and repeating the request is correct. On /v1/messages the same failure is overloaded_error, and on /v1/responses it is server_error, the codes those clients retry. Output that streamed before the failure is billed for the tokens actually generated; a stream that failed before producing anything has its token charges waived (see Pricing & billing).

metadata may carry additional keys beyond error_type — operator notes about where a failure came from. The set grows over time, so ignore keys you do not recognise; the status, the code, the message and error_type are the contract.

Correlating failures#

Every inference response — errors included — carries an X-Generation-Id header, assigned before anything can fail. Quote it when reporting a problem, and use GET /generation to inspect what was recorded and charged for a request that went wrong.