flexway
FRENTalk to an expert

Native Kafka clients and field encryption

ConceptProductSince version 1.6.0

Markdown

Product guide: what the capability brings, for whom, its limits and the version that delivered it - no configuration syntax.

What it does A Kafka producer, consumer or admin tool, of any library, connects to the gateway with no code change. Each tenant sees its virtual cluster: its topics, groups and transactions under short names, rewritten into its namespace on the fly. Quotas per client and per tenant the way Kafka does them (throttling, never a cut). Fields declared sensitive are encrypted by the gateway under your KMS (Vault transit, AWS KMS, Google Cloud KMS) and decrypted for one scope only
Why it matters The broker's credentials stay the gateway's; a client authenticates with the same token, key or certificate as over HTTP. The broker, its backups and any reader without the scope never see the clear value. Destroying a tenant's key makes its fields unreadable for good (crypto-shredding), leaving the others untouched
Use cases Open a shared Kafka cluster to several customers without any of them seeing another's data; apply a rate per client; protect an email or an IBAN in a topic; answer a GDPR erasure request by destroying a key
Limits Tested end to end against Apache Kafka, Redpanda and Vault with franz-go, librdkafka and the Java client; performance benchmarks written, not run yet - no figure is promised; quotas not complete yet: a tenant's daily or monthly budget does not count reads yet, and two defaults (throttle cap, new connections per tenant) are provisional; KIP-848's group protocol is not served; administration (creating a topic) is refused unless an admin scope is declared; a native NATS or AMQP proxy is not planned
Since 1.6.0
Configure Native Kafka clients: a virtual cluster per tenant, Field encryption

Next steps