Problem 1
One chatty client starves the others
A caller, a script or a looping agent takes all the throughput.
Flexway's answer: Token bucket per caller, never per pod; beyond it, 429 with Retry-After and RateLimit headers.
Traffic
Rate per caller, tenant quotas per day and month, shedding under saturation, timeouts per operation, bounded retries and circuit breakers: every excess gets a clear answer, the upstream stays protected.
14 of the 14 shown delivered · since v0.1.0
What you expect from this domain


Problem 1
A caller, a script or a looping agent takes all the throughput.
Flexway's answer: Token bucket per caller, never per pod; beyond it, 429 with Retry-After and RateLimit headers.
Problem 2
Requests pile up behind a service that no longer answers.
Flexway's answer: In-flight cap per upstream, circuit breaker with half-open state, timeout per operation.
Problem 3
A saturated replica is killed and drags the others.
Flexway's answer: Adaptive shedding and a global cap: an overloaded replica refuses the excess instead of being killed.
Every step is a decision readable in the trace and in the console.
Traffic - Domain diagram: who calls, the decision steps, what is called, and where a request is refused.
beyond the rate: 429
The applications that call your APIs: each request goes through Flexway, which checks the contract and the identity before reaching your systems.
Routes through here
AI agents: they call MCP tools and models through Flexway, under the same identity, quota and audit as an API.
Routes through here
The partner companies that consume your APIs: identified by a client certificate or an API key, under a quota per tenant.
Routes through here
Beyond the in-flight cap: 503, before routing; under saturation, adaptive shedding refuses non-critical traffic first, before identity.
Routes through here
Flexway identifies the caller and its rate class, declared per method or per operation.
Routes through here
A token bucket per caller limits its rate; beyond it: 429 with Retry-After and the RateLimit headers.
Routes through here
The tenant's daily and monthly quota applies across all its routes; beyond it: 429 quota_exceeded.
Routes through here
Towards the upstream: deadline per operation, bulkhead, circuit breaker; any degradation is announced, never silent.
Routes through here
Your REST and gRPC services: they receive only the requests Flexway let through, under deadline and circuit breaker.
Routes through here
Your internal models, called through the OpenAI-compatible surface; a sensitive request is pinned to them.
Routes through here
The essentials of each topic, with its status in the published release; unfold a card for the full text.
Rate is counted per verified caller, never per replica, and going over answers 429 with Retry-After and the RateLimit headers.
The rate is counted per verified caller - the identity's tenant and subject - never per replica or pod address: two replicas do not double a client's allowance. Rate classes are declared per method, and an operation can have its own.
Beyond it, the answer is an RFC 9457 429 with Retry-After and the RateLimit headers: a well-written client knows when to come back.
Each tenant has a quota per day or month, with counters shared across replicas and reconciled with no blocking call.
Beyond the instant rate, each tenant has a quota over a long window, per day or month. Across replicas, counters are shared and reconciled in the background, with no blocking call on the request path.
Under saturation, the gateway refuses the excess with an explicit 503 rather than building an endless queue.
Under saturation, the gateway sheds load: it refuses the excess with an explicit 503 instead of building an endless queue. A global in-flight cap protects each replica: overloaded, it refuses rather than being killed for lack of memory.
Each operation has its timeout budget, retries are kept to idempotent methods and a circuit breaker lets a sick upstream recover.
Each operation has its timeout budget, never a global timeout. Retries are limited to idempotent methods. An in-flight cap per upstream isolates services from each other, and a half-open circuit breaker lets a sick upstream recover.
A degradation is always announced, in a header and in the trace, never silent.
When a replica stops, requests and streams in flight are drained: a deployment cuts neither a call nor a stream.
When a replica stops, requests and streams in flight are drained before closing: a deployment cuts neither a call nor an SSE or WebSocket stream.
The rate-limits example, as it ships with Flexway v1.6.3.
# A token bucket per caller - the token's subject, else the client's address, read from
# X-Forwarded-For only behind a declared edge.
version: 1
contracts:
openapi: ./openapi.yaml
upstreams:
default:
url: http://search.internal:8080
server:
trusted_proxies: [10.0.0.0/8] # the edge; nobody else chooses their address
policies:
definitions:
per-caller:
type: rate-limit
config:
key: caller
classes:
read: { rps: 50, burst: 100 }
write: { rps: 10, burst: 20 }
expensive: { rps: 1, burst: 5 }
chains:
global: [per-caller]# Layered budgets, connection pools, a bulkhead and a circuit breaker per upstream. Retries are
# built in: once, for an idempotent method or an Idempotency-Key, within a retry budget.
version: 1
contracts:
openapi: ./openapi.yaml
upstreams:
default:
url: http://billing.internal:8080
timeout: 3s # total budget of an exchange with this upstream
pool: { max_idle: 100, max_conns: 200, idle_timeout: 90s }
max_in_flight: 500 # bulkhead: past it, 503 at once
circuit_breaker: { failure_ratio: 0.5, min_requests: 20, window: 10s, cooldown: 5s }
exports:
url: http://exports.internal:8080
max_in_flight: 20 # a slow upstream uses up its own slots, never everyone's
server:
timeouts:
connect: 2s
read: 60s # silence between two bytes - what governs a stream
total: 30s
limits:
max_body: 2MiB # a body read for validation; past it, 413
shutdown:
drain: 20sWhat is delivered in the published release v1.6.3, and what is coming; a feature is never shown as delivered before its release.
| Feature | Status |
|---|---|
| Rate per caller (token bucket) | Delivered · v0.1.0 |
| 429 with Retry-After and RateLimit headers | Delivered · v0.1.0 |
| Rate classes per method, per-operation override | Delivered · v0.1.0 |
| Quota per tenant per day and month | Delivered · v0.1.0 |
| Adaptive load shedding under saturation | Delivered · v0.1.0 |
| Timeout budget per operation | Delivered · v0.1.0 |
| In-flight request cap per upstream (bulkhead) | Delivered · v0.1.0 |
| Retries limited to idempotent methods | Delivered · v0.1.0 |
| Circuit breaker with half-open state | Delivered · v0.1.0 |
| Degradation announced (header and trace), never silent | Delivered · v0.1.0 |
| Graceful shutdown: streams in flight are drained | Delivered · v0.1.0 |
| Global in-flight cap: an overloaded replica sheds instead of being killed | Delivered · v0.2.0 |
| Traffic per operation, upstream and tenant | Delivered · v0.1.0 |
| Service-level objectives per operation: remaining error budget, burn rate over several windows, built-in alert rule | Delivered · v1.3.0 |
The other expectations of this domain
429 refusal and the tenant's soft-cap switch; graduated actions for each budget level are missingPartly · next in 1.8Flexwiz is a young company: you talk directly with the person who founds it. Tell us what you want to run through Flexway. We talk about your infrastructure and your digital sovereignty requirements.
Online booking is coming soon; until then, an e-mail is enough.