How Heimdallm works

Heimdallm is an inference marketplace: independent providers serve models behind one OpenAI-compatible API. You bring an SDK you already use, point it at a new base URL, and pay per token at the price each provider lists. This page explains the moving parts once, so the rest of the documentation can assume them.

The four nouns#

When several providers serve the same model id, that is one model with several endpoints. There is no "the" version of a model to be loyal to: each endpoint is a separate offer of the same weights, at its own price and with its own behaviour. A quantized rebuild of a model is a different model with its own slug under the original author — and its own endpoints.

The marketplace also uses two words about numbers, and the difference matters:

• Declared — stated by a provider for its own endpoints: prices, context window, quantization, supported parameters, compliance and data policies. A provider declares; it is not believed on performance.
• Measured — computed by Heimdallm from real traffic and periodic probes: uptime, time to first token, response time, throughput, tool-call success, benchmark scores. A figure is never inferred from what a provider says about itself.

A value nobody can state honestly is shown as absent, never guessed. When an endpoint's time-to-first-token reads null, that is not zero and not bad — it means the window does not hold enough samples yet to earn a number.

The life of a request#

1. Authentication. The request carries a bearer API key. The key lives in a workspace; the workspace belongs to an account; the account's balance pays.
2. Routing. Heimdallm picks one of the model's live endpoints: provider-reported capacity first (endpoints with free slots before saturated ones — deprioritised, never excluded), then health, then a price-weighted balance. You can steer all of it — suffixes like :floor, fallback model lists, and the provider object — see Models & routing.
3. The credit hold. Before the request runs, a hold is placed for a conservative estimate (prompt plus completion ceiling at the endpoint's prices). A request whose estimate exceeds the spendable balance is refused with 402 before any provider is contacted.
4. Generation. The request is forwarded to the chosen endpoint, byte-compatible with the OpenAI dialect. If that endpoint fails, Heimdallm fails over to the next eligible one on its own. Every response carries the serving provider and the exact usage.cost.
5. Settlement. The hold settles to the actual cost and the movement lands in the account's ledger. The generation's full billing record stays queryable by its X-Generation-Id.

Billing is prepaid: deposits credit the balance, every request draws it down, and three optional spend limits (key, workspace, account) can each cap it.

The marketplace pages#

Everything the API serves is also browsable:

• Models — the catalogue grouped by family, with each model page showing every endpoint's prices, measured performance, benchmarks, uptime history and recent activity.
• Benchmarks — named benchmark scores for current provider and precision variants, with quality, cost, time and answer-length comparisons. Provisional scores show exact coverage after five scored questions; unsupported evaluators remain unscored. The public API provides the same evidence through GET /benchmarks and GET /benchmarks/variants/{endpointId} under /api/v1.
• Providers — the directory, with each provider page showing its availability history, where it serves from, and its declared policies.
• Activity — models by tokens processed over the last 30 days, and what was served each day. The platform's own paid probe traffic counts, like any other billed request.
• Activity — marketplace-wide totals and daily series.
• Status — a live health board of every endpoint.
• Chat playground — try any model from the browser, on an API key you hold.
• Coding playground — build software with an AI agent on a machine of your own.

Where to go next#

Buying inference:

• Quickstart — first request in three steps.
• Choosing a model — how to read prices and measurements.
• Switching to Heimdallm — from OpenAI, OpenRouter or any compatible client.

Selling inference:

• Listing your models — the complete provider contract.
• The provider dashboard — the surface you operate from.