Responses

POST https://api-staging.heimdallm.com/api/v1/responses

Send an OpenAI Responses-shaped request to a marketplace model. Requires a Heimdallm bearer API key and a model id from the catalogue. The gateway translates the conversation to chat inference and returns a response object or response.* events.

Send the full conversation in input every turn. Responses are not retained server-side, and an earlier response id cannot recover its content. See coding harness integration.

Request body#

A message item carries role and content. Content may be a string or an array of input_text / output_text blocks; user input_image blocks carry image_url for a model that reads images, and input_file blocks carry file_data, file_url or file_id for a model that reads files. A file_id names a file stored with the Files API, which is read under your account before routing and sent to the endpoint as a data URL; an id your account cannot read is answered 404. See Documents & files. system and developer messages become system content.

A returned function_call has call_id, name and JSON-string arguments. Your client executes the function and appends a function_call_output with the same call_id and an output string, or an array of input_text, input_image and input_file blocks. A returned image or file reaches the model as an image or file part in the user turn after the results, labelled with its call_id, so the request is routed only to an endpoint that reads that input; a model with none answers HTTP 503 naming it. A block of any other type is named in the result's text and its payload left out. Resend the prior messages, the call and its result together. The marketplace does not execute tools.

Structured output#

text.format asks for the shape of the answer. {"type":"json_schema","name":"…","schema":{…},"strict":true} is sent to the endpoint as a json_schema response_format and routed only to endpoints that declare structured_outputs, so a schema is never handed to an engine that would answer in prose. When no endpoint for the model declares it, the request is refused with HTTP 503 before inference. {"type":"json_object"} is sent as JSON mode and routed like any other request. {"type":"text"}, the default, asks for nothing. Any other format.type is refused with HTTP 400.

Example#

Replace the example model with a listed model. This reply excerpt uses illustrative ids and usage; the full object also carries compatibility fields described below.

curl https://api-staging.heimdallm.com/api/v1/responses \
-H "Authorization: Bearer $HEIMDALL_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "deepseek/deepseek-v4-flash-0731-abliterated",
  "input": [{ "type": "message", "role": "user", "content": [{ "type": "input_text", "text": "Say hello." }] }],
  "max_output_tokens": 64,
  "store": false
}'

{
  "id": "resp_gen-example",
  "object": "response",
  "status": "completed",
  "model": "deepseek/deepseek-v4-flash-0731-abliterated",
  "output": [
    {
      "id": "resp_gen-example-msg-0",
      "type": "message",
      "status": "completed",
      "role": "assistant",
      "content": [{ "type": "output_text", "text": "Hello.", "annotations": [] }]
    }
  ],
  "usage": { "input_tokens": 9, "output_tokens": 3, "total_tokens": 12 },
  "incomplete_details": null,
  "error": null
}

Media a model generates rides in the assistant message as further content parts, each at its own content_index. A picture is an output_image part whose image_url holds it — a data URL wherever the endpoint handed back bytes, its own URL where it answered with one. Generated speech is an output_audio part carrying data (base64), transcript and the endpoint's own id and expires_at. A reply whose whole answer is a picture still returns a message item.

output contains assistant messages and/or function calls. When the output ceiling ends a reply, status is incomplete and incomplete_details.reason is max_output_tokens. Usage contains token counts, without usage.cost or cache/reasoning breakdowns.

This dialect's usage object states all three counts, and a client parsing it rejects an object with a member missing, so a count the endpoint serving the reply never reported has no member to leave out. It is sent as 0 and named in usage.heimdall.unreported, a marketplace extension listing every member whose zero nothing measured — total_tokens among them wherever one of its terms is missing, a total short a term being unknown rather than smaller:

"usage": {
  "input_tokens": 0,
  "output_tokens": 3,
  "total_tokens": 3,
  "heimdall": { "unreported": ["input_tokens", "total_tokens"] }
}

A count not named there is the endpoint's own figure, so a zero without the marker is a measured zero. The marker is absent from a reply whose counts were all reported, and a reply the endpoint sent no usage document for names all three. Clients that do not know the extension read the object as before.

Compatibility fields are placeholders, not a record of the request: parallel_tool_calls is true, tool_choice is auto, tools is empty and metadata is empty. instructions, max_output_tokens, previous_response_id, reasoning, temperature and top_p are null.

Streaming#

Add "stream": true to the request and use curl -N. The text/event-stream starts with response.created and response.in_progress. Each JSON event has a matching type and an increasing sequence_number. Text uses output-item and content-part boundaries around deltas; for example, after those opening events:

event: response.output_text.delta
data: {"type":"response.output_text.delta","sequence_number":4,"item_id":"resp_gen-example-msg-0","output_index":0,"content_index":0,"delta":"Hello."}

event: response.output_text.done
data: {"type":"response.output_text.done","sequence_number":5,"item_id":"resp_gen-example-msg-0","output_index":0,"content_index":0,"text":"Hello."}

The text sequence includes response.output_item.added, response.content_part.added, response.output_text.delta, response.output_text.done, response.content_part.done and response.output_item.done. Function calls use response.function_call_arguments.delta and response.function_call_arguments.done between output-item boundaries; collect argument fragments before parsing JSON.

A picture arrives whole, so its response.content_part.added already carries the finished output_image part and response.content_part.done repeats it. Speech accumulates through response.output_audio.delta and response.output_audio.transcript.delta; the joined bytes and transcript arrive on response.output_audio.done and response.output_audio.transcript.done before the part's response.content_part.done.

The terminal event is response.completed, whose response holds the assembled output and final usage, under the same rule as a non-streamed reply: usage.heimdall.unreported names whichever members the trailing chunk left unstated, and all three where no chunk carried a count at all. Inspect its response.status: this event also carries an incomplete response when the output ceiling is reached. There is no [DONE] sentinel. SSE keep-alive comments may appear.

The stream commits on its first event, or after one keep-alive period when the model is slower than that to start. A rejection before then, such as an unknown model, no eligible endpoint or an engine refusing a request member, is an HTTP error status. Its body is {"error":{"message":"…","type":"invalid_request_error","code":null,"metadata":{…}}}, where metadata is the public error envelope's and carries the engine's own diagnostic as provider_message when one came back.

After the stream commits, a failure arrives as event: error with {"type":"error","code":null,"message":"…","param":null} and then response.failed, whose response.status is failed and whose response.error holds a code and the same message. The message ends with the engine's diagnostic when one came back. The code is invalid_prompt for a rejected request, rate_limit_exceeded, context_length_exceeded, or server_error for a failure worth retrying. The connection then closes. A connection ending without response.completed or response.failed is not proof of completion.

Heimdallm differences and errors#

X-Generation-Id identifies the generation; a successful non-streaming reply also carries X-Provider-Name. The body has neither provider nor usage.cost. Use the generation id with GET /generation for the account's recorded cost.

Stored-response chaining and background execution are refused before inference, even when stream: true. The HTTP 400 uses the public error envelope: error.code is 400, error.metadata.error_type is invalid_request, and error.message names previous_response_id or background. Omitted/null previous_response_id and omitted/false/null background select stateless foreground execution. Because a non-null previous_response_id is refused rather than served, a request that reaches inference never carries a usable conversation id on the Responses wire itself; send X-Session-Id to take part in session affinity.

Function tools inside a namespace are expanded for the chat backend and returned calls carry their namespace. Built-in tool types such as web_search and local_shell are dropped; they do not run on this endpoint, so a reply never carries a web_search_call item or a search citation. Unrecognized input-item types, reasoning effort values and extra body fields are not forwarded. models, provider and route carry the shared routing controls; key routing defaults apply below explicit request preferences. This route does not implement the full OpenAI Responses feature set.