Reading with the console
GuideOperationsSince version 0.1.0
Operations guide: configure, deploy and observe the gateway. What the capability does, product side: product guide; calling it: developer guide.
Goal: operate without reading files - what is served, what applies, what goes through, what is used.
# With your backends: the console reads your Prometheus, links to your traces,
# receives nothing and stores nothing
helm install console oci://ghcr.io/flexwiz/charts/flexway-console --version <version> \
--set gateway.url=http://orders-flexway-admin:9090 --set metrics.url=http://victoriametrics:8428 \
--set traces.url='https://grafana.example/explore?traceId={trace_id}'
# Without any backend: the gateway pushes metrics and traces to the console (OTLP, TLS, mutual TLS
# by default), which keeps the history on a persistent volume per replica (8 GiB by default)
helm install console oci://ghcr.io/flexwiz/charts/flexway-console --version <version> \
--set gateway.url=http://orders-flexway-admin:9090 --set persistence.size=8Gi \
--set otlp.tls.issuerRef.name=flexway-ca --set 'otlp.tls.clientNames={orders-flexway.observability.svc}'Without a backend, the gateway announces itself by pushing (tracesToConsole, below): each replica
carries its pod name (service.instance.id), and a replica silent for 45 s turns unknown. With
several console replicas, the one that receives an export relays it to the others (bounded queue,
retries): each keeps a full copy; a replica back asks another for the last two hours. The /events
and /errors journals are still read on every gateway replica through its headless Service
(orders-flexway-admin-headless); its admin port is open to the console's pods (same namespace by
default, admin.networkPolicy.console.podLabels). Switching mode = uninstall then install again: the
state Secret and its accounts are kept.
Retention (without a backend): 35 days of detail (retention.detail, adjustable), daily
aggregates - requests, errors, tokens, euros - over 13 months (retention.daily) to compare month
over month and year over year; a persistent volume is required (the render is refused with
persistence.enabled: false), filled to 80 % at most, the oldest data going first. Past the
retention, a figure is --, never 0. A strip at the top of Overview, Traffic, Upstreams, Catalogue
and Usage says the source and the retention; its "Details" panel shows the replicas sending, the last
export received and the volume (to admins). Past 35 days, plan 10 GiB at least; further still, your Prometheus.
First run, as the NOTES say it: read the first admin's generated password from the state Secret
(kubectl get secret <release>-flexway-console-state -o jsonpath='{.data.initial-admin-password}',
base64-decoded), sign in as admin, choose a password, enrol a TOTP app and keep the ten recovery
codes. Then Users -> Invite: a single-use invitation link (72 h by default, 24 h or 7 days at your choice) to pass on - the console sends no
e-mail. Lost access: kubectl exec deploy/<release>-flexway-console -- flexway-console users reset admin (sts/ instead of deploy/ without a backend);
a locked account: ... users unlock <login>.
Security - Whoever can
execinto the console's pod is its administrator: keeppods/execon this Deployment to its operators (every action of the command is audited). Do not expose the console publicly. Behind an ingress, declare its network intrustedProxies: each client is then counted under its own address - five failures lock the attacker's address, never the owner's, who waits a minute at most.
To add an identity provider (Entra ID, Okta, Google, Auth0, Keycloak, Authentik...):
# values.yaml of the services/console/deploy/helm chart
gateway: { url: "http://orders-flexway-admin:9090" } # the admin surface
metrics: { url: "http://victoriametrics:8428" } # a Prometheus API
# Grafana Explore on Tempo; any trace tool reachable by URL works:
# {operation}, {client} (the agent), {start_us}/{end_us} (microseconds), {start_ms}/{end_ms} (milliseconds)
traces: { url: "https://grafana.example/explore?schemaVersion=1&orgId=1&panes=%7B%22a%22%3A%7B%22datasource%22%3A%22tempo%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22datasource%22%3A%7B%22type%22%3A%22tempo%22%2C%22uid%22%3A%22tempo%22%7D%2C%22queryType%22%3A%22traceql%22%2C%22query%22%3A%22%7Bresource.service.name%3D%5C%22flexway%5C%22%20%26%26%20span.operation_id%3D%5C%22{operation}%5C%22%7D%22%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22{start_ms}%22%2C%22to%22%3A%22{end_ms}%22%7D%7D%7D" }
oidc:
issuer: https://login.example.com/realms/staff
clientId: flexway-console
redirectUrl: https://console.example.com/auth/callback
displayName: Keycloak # the "Sign in with Keycloak" button
accountUrl: https://login.example.com/realms/staff/account # optional: the link of My account
rolesClaim: realm_access.roles # where the role is read
tenantFrom: claim # or role: flexway-viewer:acme gives the tenant acme -
# roles users cannot create themselves (no self-service groups)
tenantClaim: tenant # where a viewer's tenant is read
existingSecret: console-secrets # key client-secret, mounted as a file
auth:
local:
mode: breakglass # accounts (default), breakglass (local admins, 1 h) or offCheck:
| Screen | What it shows |
|---|---|
| Overview | The home screen: the gateway's health (status, replicas and version, in flight against the ceiling, shedding, last reload), seven indicators with a curve and the change against the previous window, the map clients -> gateway -> upstreams (a click opens an upstream's summary; the gateway's replicas as nodes under it, each with its health heart and label), the slowest and most failing operations and the biggest variations (rate, 5xx, p95, rises and falls), the latest events. A viewer reads their tenant, without the map nor the cross-tenant measures |
| Traffic | Six indicators against the previous window (requests, 5xx, 4xx, p95, 429 refusals, open streams); the evolution in tabs Rate (the 5 busiest operations), Errors, Latency; the biggest variations; hour by day of the week (last 4 weeks, UTC); a breakdown by operation - never by URL -, by upstream and by tenant (admins), filter, sort and facets; a click opens the summary, then the Operation screen; a trace link next to each failing operation; over 1 h to 7 d |
| Operation | One operation in detail, from the Catalogue, Traffic or the palette, over 1 h to 90 d: rate, 4xx, 5xx against the previous window; every tenant's p50, p95, p99 and their distribution; errors by the gateway's code (upstream_status for an upstream's answer relayed), a code never seen over the 7 previous days marked "new"; share by tenant (admins); streams: open, length, events relayed; the chain in short; recent errors and traces. A viewer reads their tenant; errors by code and streams, without a tenant dimension, are not shown to them |
| Upstreams | A card per upstream - status, breaker on its three states and next trial, 24-hour availability in 30-minute slices, rate, 4xx, 5xx, p50, p95, p99, in flight against the bulkhead - or a table; "Compare" lays a window B, dashed, on each p95 curve; search (/), filters, order; a click opens the detail (breaker history, latencies, errors by code, operations, the policies that call it, configuration) - admins only |
| Catalogue | The contracts' operations: a summary that filters; search (/) and faceted filters with counts (method, protocol, security, attributes, upstream, tag, host); a row per operation - owner, withdrawal date and days left, kinds of access, rate class, budget, the chain in short by category, curve, rate, 5xx, p95; a click opens it in the side panel (parameters and responses in short, extensions as YAML, effective chain towards Policies, traffic); the federated MCP tools apart; CSV and JSON export of the filtered list; the whole view in the URL. Compliance tab: six rules (no owner, deprecated without a date, no summary, public, write without a declared class, deprecated and still called), the operations that break them and what to write in the contract; two rules guaranteed at load. A viewer sees the whole catalogue, their tenant's traffic and every tenant's p95, said on screen |
| Policies | An operation's effective chain as a pipeline by phase (request, upstream, response, error), each step where the request path runs it: its configuration (a secret shows as ***), its level of attachment, its condition, what was removed or replaced; the gap with the global chain; the chain copied as YAML; for an admin, each step's counters (applied, skipped by its condition, 429 refusals), read every 30 s. Keyboard search: /, arrows, Enter, Esc |
| Agents | The MCP surface (path, audience, admitted clients, delegation bounds, tools served), indicators against the previous window; one agent per declared client (supervised or not, calls, refusals by cause, cost, scopes used, last call); an agent's panel; the delegation chain (structure, last refusals with depth and age against the bounds); the contract's and the federated tools, federated servers for admins. A viewer reads their tenant |
| Usage | An admin without a chosen tenant: every tenant ranked against its budget (status, share of the cap, trend, date reached), and the period day by day; a click opens a tenant. A tenant: its gauges (request quotas, tokens of the day and the month, euros of the month) over the gateway's UTC windows, with trend, band, date reached and status; the cumulative against the cap; by backend (price, input and output, calls), declared backends, by capability, metered upstream, switches, agent and operations. Period: this month, last month, 7 days or custom; CSV export of the period; on this month, the previous month's cumulative at the same day. History tab: 13 months, month over month and year over year at equal period, a table per tenant, CSV export |
| Users | The local accounts and the OIDC identities seen (read only): a summary by state that filters, the access hygiene (admins without a second factor, expired invitations, dormant accounts), a search (/) and facets; a click opens an account's panel - details and creator, actions under confirmation, sign-in history, activity - or an identity's ("Forget"); the accounts' activity and the policy in force - admins only |
| My account | Profile, password, second factor and change of app, recovery codes, session and "End every other session", recent sign-ins, language and theme; an OIDC session finds its profile, its preferences and the link to its account at the provider |
| Diagnostics -> Errors | The recent errors of each gateway replica, without reading any log: by kind (upstream, refused, internal), operation, upstream, stage, cause (admins), duration, replica, and the trace (kept by the console, or opened in your tool); a partial list is said when a replica does not answer; past 20 errors per second and per kind, the part not detailed is counted (admins); for an admin, a code never seen on the operation over the 7 previous days is marked "new". A viewer reads their tenant, without the cause |
| Diagnostics -> Traces | Without a trace backend: the slow or failed requests the console keeps (500, 24 h, 64 MiB per replica), filters, a trace as a waterfall with each span's kept attributes, an OTLP JSON download (replayable in Jaeger or Tempo). With a backend: the screen says where your traces are |
| Diagnostics -> Alerts | Admins only, in both modes: each declared or built-in rule, its state (ok, pending, firing, resolved, unknown), its value against the threshold, its curve over 6 h and its last transitions; "Silence" (duration, reason) and lifting it; the webhooks' deliveries (result, status, cause); the periodic summaries; the replica that notifies; a refused file, with its cause. The Overview carries a badge of the alerts firing |
A viewer sees their tenant, enforced on the server: another tenant is an explicit refusal, never an empty page - whether they come from the provider or are a local account. A provider's account without a recognised role has no access and sees the roles its token carried. The console keeps no token in the browser and writes only its own accounts: its failure stops no traffic. Figures come from the backend, or from the gateway's exports to the console (15 s); about 30 s late on VictoriaMetrics; the configuration version and the reading time sit at the top of every screen.
Each replica's health (admins) - every replica of the gateway has its status: the status rule on
its own signals (in flight against its ceiling, shedding, share shed, its last reload rejected -
the 24-hour count stays shown, as information), after the
freshness of its last report. A replica pushes every 15 s: no report for 45 s, “No report”; for 2 min
without a stop announced, “Degraded” (and the rule replica-silent notifies); forgotten after 1 h. A clean
stop (drain) is said by flexway_server_draining: the replica is “Retired”, never degraded - after a rollout,
the old pods stay so for an hour. A replica that is not well makes the gateway “Watch”, never “Degraded”: the
others serve. A click on a replica of the map opens its summary, with a link to its errors
(/errors?replica=). With your own backend, the collector must keep service.instance.id in the instance
label (Alloy and Prometheus' OTLP write do by default): without it, a single “replica” is seen.
Asynchronous plane - a broker (kafka, amqp in 0-9-1 or 1.0, mqtt, nats) and a webhook target are
upstreams: the same status rule, on their own signals - failed publications or failed deliveries,
503 overloaded and dead letters counted as refusals; their latency is the broker's acknowledgement
and their gauge the bounded buffer (max_in_flight). The Overview's map draws them with their protocol,
published, delivered and subscribers; the Catalogue filters doors by “Publication” and “Subscription”;
Traffic has an “Asynchronous plane” panel by broker and channel; a door's Operation screen shows its
outcomes, the acknowledgement, the subscribers open and its webhooks; Errors names buffer_full and
rejected. A viewer reads the published, delivered and 503 of their tenant, the acknowledgement of
every tenant (said on screen), never the subscribers open (no tenant dimension); the Upstreams screen stays
the admins'. Three built-in rules: broker-unavailable (1 min), broker-backpressure (5 min),
webhook-dead-letter (at once).
History, markers and links - the Usage History tab shows 13 UTC months, the current
one partial: it compares with the previous month and the same month a year before at equal
period (from the 1st to today), never with a whole month; an admin without a tenant reads one row
per tenant; the export button gives month,tenant,measure,value,partial as CSV. On every curve, a
line marks each configuration reload, applied or refused, of every replica: time, version and result
on hover or focus. "Copy the link" keeps the screen, the window, the tenant and the filters; the link
grants no right - a recipient outside that scope gets a refusal.
Compare, variations, hour by day - "Compare" (Traffic, Operation, Upstreams) lays a window B
of the same length, chosen by its start, dashed on the same curves; it stays in the link. The biggest
variations rank the operations by their move against the previous window - rate in percent, 5xx in
points, p95 in milliseconds -, from 50 requests on. The hour by day map averages each UTC hour of each
day over 4 weeks. Out of retention nothing is drawn: --, never 0.
Traces without Tempo - in the integrated mode, the console is also the destination of
the gateway's traces and keeps the failed requests itself (5xx, refusals written after admission except 401
and 404) and the slow ones (over 1 s, or over half the operation's budget); the rest is dropped.
With traces.url (and metrics.url), its receiver stays off.
# The console's values: TLS is required, mutual TLS by default (cert-manager, a CA Issuer)
otlp:
tls:
issuerRef: { name: flexway-ca } # a Certificate is rendered for the console
clientNames: [orders-flexway.observability.svc] # the name of the gateway's certificate
# meshEncrypted: true # only where a mesh already encrypts (Linkerd, strict Istio)
# The gateway's values:
tracesToConsole:
enabled: true
endpoint: https://console-flexway-console-otlp.observability:4317
tls: { issuerRef: { name: flexway-ca } } # renders its client certificate orders-flexway.observability.svcThe gateway's admin port is closed by default (
admin.networkPolicy): the console gets in; a tool that still reads/metricsmust be added toadmin.networkPolicy.from.
What is kept: a whitelist's attributes (operation_id, upstream, tenant, policies, status, the
contract's method and route), everything else dropped at reception - an address, a header or a body
cannot get in. What is not: a request the gateway's head sampling (observability.sampling.max_per_second)
did not retain never arrives; every error stays in the error journal. The gateway traces the request, its
stages - routing, identity, policies -, its external-calls and each attempt to the
upstream, with retry_count and circuit_state; the stages cost nothing to the requests not sampled.
Alerts - the console evaluates its rules every 15 s, in both modes, and notifies through a signed webhook. Nothing is typed into the console: rules, webhooks and summaries are declared in the chart's values, read again without a restart; an invalid version is refused and said on the Alerts screen, the previous one keeps applying.
# The console's values
alerts:
rules: # closed kinds: error_rate, p95, breaker_open, slo_burn, new_error_code
- { name: reports-5xx, kind: error_rate, upstream: reports, threshold: 5%, for: 1m, notify: [ops] }
- { name: orders-p95, kind: p95, operation: createOrder, threshold: 300ms, for: 2m }
webhooks: # https required (allowHttp lifts it); secretFile: whsec_<base64>, one secret per line
- { name: ops, url: "https://hooks.example/flexway", secretFile: /etc/flexway-console-alerts/ops }
- { name: chat, urlFile: /etc/flexway-console-alerts/chat-url, secretFile: /etc/flexway-console-alerts/chat, allowPrivate: true }
summaries:
- { name: weekly, every: weekly, at: "07:00", weekday: monday, webhook: chat } # UTC
existingSecret: console-alerts # secrets and URLs carrying a token, mounted at /etc/flexway-console-alerts
slos:
- { operation: createOrder, availability: 99.9%, latency: { p95: 300ms, target: 99% }, window: 30d }
- { name: orders-api, upstream: orders-svc, availability: 99.5% } # a group: see SLO objectives on groups of operations| Topic | What to know |
|---|---|
| Built-in rules | Always on: breaker-open (an open breaker), upstream-degraded and gateway-degraded (1 min), new-error-code (a code never seen on the operation in 7 days - it reads flexway_errors_total), slo-burn (as soon as an objective is declared, an operation's or a group's), replica-silent (a gateway replica without a report for 2 min and no stop announced), broker-unavailable (a broker's breaker open or more than half its publications failing, 1 min), broker-backpressure (more than 1 % of 503 overloaded, 5 min), webhook-dead-letter (a delivery in the dead letter over 15 min) |
| Judgement | error_rate = share of 5xx over 1 min, from 20 answers; p95 over 5 min. Silent metrics: no state changes, nothing is sent nor withdrawn, the screen says unknown. A target missing from the configuration makes the rule invalid, never notified |
| Who sends | Every replica evaluates; one notifies, the holder of the chart's Lease. The webhook-id is the same from one replica to another: deduplicate on it (at least once) |
| Webhook | https, the address checked when the connection is made: loopback, link-local (169.254.169.254) and multicast always refused, a private network only with allowPrivate: true (an internal target); no redirection followed; 5 s timeout; retries 5 s, 30 s, 2 min, 10 min, 30 min, then an abandon shown on the screen; a 4xx (but 408, 429) is a final refusal |
| Silence | Alerts screen, 24 h at most, a reason required: it suspends a rule's notifications, never its evaluation; when it ends, what still fires is notified |
| Summaries | daily or weekly at a UTC hour: requests against the previous period, 5xx and 4xx shares, new codes, each SLO's budget, the five largest changes, alerts - never a tenant's name |
| SLOs | Availability (non-5xx answers) and latency (share of the minutes whose p95 holds the target) over the window; remaining budget, burn rate over 1 h, 6 h, 3 days; slo-burn fires above 14.4 over 1 h and 5 min, or 6 over 6 h and 30 min. On an operation or, since 1.6.0, a group of operations - an upstream, a tag, a list, every one (SLO objectives on groups of operations). Traffic's Objectives (SLO) tab, the Operation panel, the global objective on the Overview; "Create an alert" on the charts and the objectives writes the YAML to paste in these values |
| Tuning the built-in rules | alerts.builtins: the name of a built-in rule, for (0 to 1 h) on every one; threshold (a ratio) and minRequests only on broker-unavailable (half, 20) and broker-backpressure (1 %, 20); excludeCauses only on a rule counting closed causes (the native Kafka proxy's rules, since 1.6.0). An unknown name, a rule tuned twice or a knob the rule does not carry refuses the file, as the rest does; without the key, the defaults. The Alerts screen shows the tuned threshold, GET /api/alerts the values in force and where they come from (tuning) |
Tune a built-in rule rather than silence it - a few cases:
alerts:
builtins:
# Tighten: a small publication traffic fires too early; 5 % refused, from 100 publications.
- { name: broker-backpressure, threshold: 5%, minRequests: 100 }
# Delay: a broker failing over in less than 3 min is not an incident.
- { name: broker-unavailable, for: 3m }
# Sooner: an open breaker already fires at once; a degraded gateway, after 30 s.
- { name: gateway-degraded, for: 30s }With the native Kafka proxy (since 1.6.0), the kafka-proxy-refusals rule counts closed causes and
accepts excludeCauses (Native Kafka clients: a virtual cluster per tenant):
alerts:
builtins:
- { name: kafka-proxy-refusals, threshold: 10%, excludeCauses: [quota] } # do not fire on a quota reachedThe body: {"type": "flexway.alert.firing", "timestamp": ..., "data": {"rule", "kind", "state", "since", "target", "value", "unit", "threshold", "subjects"}} (flexway.alert.resolved, flexway.summary.daily,
flexway.summary.weekly) - never a tenant. Verify the signature on the receiving side (Standard
Webhooks) before anything else:
import base64, hashlib, hmac, time
def verify(secret: str, headers, body: bytes) -> bool:
key = base64.b64decode(secret.removeprefix("whsec_"))
msg_id, ts = headers["webhook-id"], headers["webhook-timestamp"]
if abs(time.time() - int(ts)) > 300: # replay: 5 minutes of tolerance
return False
mac = hmac.new(key, f"{msg_id}.{ts}.".encode() + body, hashlib.sha256).digest()
want = base64.b64encode(mac).decode()
# two signatures during a rotation, separated by a space: one must match
return any(hmac.compare_digest(sig.removeprefix("v1,"), want)
for sig in headers["webhook-signature"].split())Rotation: put the new secret on the file's second line on the console's side, deploy it on the receiving side, then remove the old one - the console signs with both meanwhile.
Try a request (1.2.0) - under the pipeline of the Policies screen: method, path, host, a
described identity (subject, tenant, scopes, roles) and headers go through the gateway's
/simulate, as a dry run. The verdict shows with the request the upstream would have received
(secrets ***), and each step carries its result: applied, skipped, not simulated (an
external-call is never called), passed. A viewer simulates within their tenant: naming another
is refused.
Go to (Ctrl+K, Cmd+K on a Mac, or the top bar's field) - from any screen: a screen, an
operation (opens its chain), an upstream (opens its detail, admins), an agent (opens its panel; those
of their tenant for a viewer), a tool or a tenant (filters the console; their own for a viewer).
Arrows, Enter, Esc.
Accounts - what the Users screen does, and what it keeps:
| Need | Where | What happens |
|---|---|---|
| A shorter or longer link | Invite, or "Reset the password" on an account | 24 h, 72 h (default) or 7 days; single use, shown once, replaced by any reset |
| Require a second factor of a viewer | The account's panel -> "Require a second factor" | Their sessions without one close; they enrol an app at their next sign-in |
| A new phone | My account -> "Change app" | Password and a code of the current app (or a recovery code); the old app works until confirmed, which hands out ten new codes and ends the other sessions |
| A sign-in you do not recognise | My account -> "End every other session" | Every other session closes within 5 s; this one goes on |
| A locked account | The account's panel -> "Unlock now" | Every lock of the account lifts: each address's, the progressive delay, the second factor's |
| A provider's identity to take off the list | The identity's panel -> "Forget" | It comes back at its next sign-in; its rights stay those of its token. Removing the access is done at the provider |
What the console keeps - a bounded view of the accounts' activity (500 events, 64 KiB, 90 days: masked network /24 or /48, browser family, never an address nor the
User-Agent) and the OIDC identities seen (a fingerprint of issuer and subject, never the subject; login and name; forgotten 90 days after the last sign-in), in keys of their own of the stateSecret, read by admins only. The console's log stays the full audit trail. The gaugeflexway_console_store_bytes{key}shows each key's weight: their sum stays under 900 KiB, the activity then the registry giving way before an account. Language and theme stay in the browser.
Statuses and data - a status follows one rule: degraded when the
breaker is open or over 5 % of answers are 5xx, warning when half open, over 1 %, in flight at 80 %
of the cap, or refusals; the 24-hour availability is the share of answers that were not 5xx, the
gateway's own refusals included. With the metrics source unreachable (the backend down, or the embedded source not open yet) the screens stay readable: figures --,
status "Unknown"; with the gateway unreachable, the map and the list of upstreams are not shown
rather than stale.
Usage - two clocks. The gauges (requests per quota policy, tokens
of the day and the month, euros of the month) follow the gateway's windows, calendar day and
month in UTC: the ones that decide a 429. The breakdowns (per day, per backend, per agent,
operations, export) follow the chosen period - this month, last month, 7 days, or from-to (93
days at most) - in days of the console's time zone (FLEXWAY_CONSOLE_TIMEZONE). Each gauge carries
its trend (the pace of the last 24 hours), a band (the slowest and fastest of the last 7 days), the
date the cap is reached and a status: degraded when the cap is reached or will be before the
window ends, warning when the soft cap is crossed or the trend reaches 90 %, healthy
otherwise; no status without a declared cap. With the metrics backend down the caps stay readable and
the consumption is --; with the gateway silent the consumption stays readable, the caps --, no
status. An admin reads every tenant (GET /api/usage); the export GET /api/usage/export gives the
period as CSV (date, tenant, backend, direction, tokens, euros, requests), a viewer's own tenant
only.
On a phone or a tablet - the same screens, the same data and the same rights as on a workstation, from 360 to 1,024 px, portrait and landscape; at a workstation, with a mouse, nothing changes.
| To | Do | What happens |
|---|---|---|
| Change screens on a phone | The bottom bar: Overview, Alerts (and how many rules fire), Errors, Search, More | "More" opens the drawer: every screen of the role, the session and the freshness; the browser's Back closes it. A reader has Traces in place of Alerts |
| Change screens on a tablet | The rail of labelled icons on the left (under a finger, from 761 to 1,180 px) | The screens of an incident, "More" for the others; with a mouse, the workstation's side navigation stays |
| Read a curve | Touch or drag on the curve; with a keyboard, the arrows | The value is written in the reading line under the curve and stays there; a reload marker touched writes its version, time and result there. The same with a mouse, at a workstation too |
| Read a heat map, an upstream's 24-hour band | Touch the cell or the slice | Its day, hour and value in the reading line |
| Read a hidden column of a table | The row's "Detail" button | Every hidden column, named, with its value |
| Silence a rule | Alerts -> the rule (a full-screen sheet) -> a duration in buttons, the reason -> "Silence" at the foot of the sheet | The same request as at a workstation; the browser's Back closes the sheet |
| Filter the errors | Errors -> "Filters" | The four lists in a sheet; each filter on is a chip that removes it |
Targets of 44 px at least under a finger, fields written at 16 px (Safari does not zoom the page), no value shown on hover alone.
Who meters - totals, days, breakdowns, the ranking's totals, export and history count
what this gateway meters only (metered_by!="upstream"), with a metrics backend as with the
embedded source, whose daily aggregates keep metered_by. The gauges (and their trend, band,
date reached, status) count what the local cap counts, which applies to relayed calls too: every
series - tokens from the usage returned, euros at a declared price only -, and the gauge says so
("includes the relayed calls, which the local cap counts too"). A backend with metering: upstream relays to another Flexway that
counts the call itself: its tokens and calls are shown apart, under "Metered upstream", never
added to the totals - the upstream gateway is authoritative for the cost.
The breakdown by capability (chat, vision, embeddings, model) counts a specialised gRPC
model in calls, not tokens. The Declared backends panel says, from /config, each backend's kind
(openai, relay, grpc), capabilities, who meters it and where it is; an admin also reads the
authentication scheme (none, bearer, header, token_file) - never a key or its reference.
Available since 0.1.0; local accounts and an optional provider in 1.0.0; the Policies screen as a
pipeline and the simulation, the Overview, the redesigned Upstreams, the command palette, Users and My
account redesigned, the accounts' activity, the OIDC identities seen and configuration proposals
(Proposing a change from the console) in 1.2.0. Autonomous observability in
1.3.0: without a backend, the console is the gateway's OTLP collector - the embedded metrics source, the
history, reload markers, shareable links; Errors and Traces, kept traces; the Operation screen, Traffic
redesigned, streams measured, errors by code, biggest variations, two windows, hour by day, new code, the
traces' stages; Alerts - declared and built-in rules, a signed webhook notified by the holder of a Lease,
silences, SLOs, periodic summaries. Phone and tablet in 1.4.0: a bottom bar, a rail under a finger and a
drawer, the reading line of the curves, priority columns, full-screen sheets - the workstation unchanged at
the mouse. In 1.5.0, usage metered where it is paid, by capability, declared backends, and the
asynchronous plane: brokers and webhook targets as upstreams, doors in the Catalogue, panels of Traffic and
Operation, three built-in rules. In 1.6.0, SLO objectives on groups of operations (objectives by level and
the objective's panel in Traffic, the global objective on the Overview, "Member of" on Operation), Kafka
clients and clusters, tuning of the built-in rules.