OpenAI-compatible inference

/v1/chat/completions, /v1/embeddings and /v1/models in OpenAI's format: an existing SDK changes only its base URL.

Product side: AI and inference; operations side: Serving inference.

Calling

from openai import OpenAI
client = OpenAI(base_url="https://api.example.com/v1", api_key=token)  # an OIDC token, sent as Bearer
r = client.chat.completions.create(model="default", messages=[{"role": "user", "content": "Hello"}])

A Flexway API key goes in Authorization: ApiKey, so through a default header of the SDK (default_headers={"Authorization": f"ApiKey {key}"}). The model actually called is chosen by the operator's policy among those it declares; stream: true relays events as they come. Images as input and embeddings are served when the chosen backend can do them.

What you can get

Every response of /v1/*, gateway refusals included, is in OpenAI's envelope ({"error": {"type", "code", "message"}}).

Status	code	When
401	tenant_required	No identity, or an identity without a tenant
429	budget_exceeded	The tenant's budget, in tokens or euros, would be exceeded - before any call
403	sensitive_no_internal_backend	A request marked sensitive with no internal backend
502	backend_unavailable	Backend and fallbacks down; during a stream, a last error event
400	capability_unavailable	The chosen backend does not serve an image or embeddings

Response header	Meaning
Flexway-Budget: soft	Your tenant passed the soft threshold: the response comes from a cheaper backend
Flexway-Degraded: backend-fallback	Served by a fallback backend

A sensitive request is marked by the header the operator declares (for example X-Sensitive: true): it never leaves an internal backend.