AI and inference

Product guide: what the capability brings, for whom, its limits and the version that delivered it - no configuration syntax.

What it does	Serves an OpenAI-compatible surface (completions, model list, embeddings, images as input) through the same data plane as the APIs. Picks the backend by policy, pins a sensitive request to an internal backend, falls back to another backend on an outage, and holds a budget in tokens and euros per tenant
Why it matters	A requests-per-second limit says nothing about the cost of a model call; an agent in a loop is an invoice, not a rate incident. An existing client (an OpenAI SDK) changes only its base URL. Prompt content is never logged: the audit keeps metadata
Use cases	Offer models to several teams with a monthly budget each; move to a cheaper model past 80 % of the budget; keep sensitive data on an in-house model; meter at a provider or behind another Flexway; meter a specialised model called over gRPC
Limits	Not built yet: routing by declared task (a small model for a simple task), choice by hosting, region, cost or latency, model canary and allowed models per caller, data protection (classes, residency, detection and masking, input and output guardrails), inference-aware routing and the semantic cache come in 1.9.0; budgets per user and per use case, then per application, team, project and cost centre, with thresholds and a chargeback export, in 1.8.0
Since	0.1.0 (inference on the data plane, budgets, fallback), 1.5.0 (several backends, images, embeddings, specialised models)
Configure	Serving inference