Native Kafka clients and field encryption
ConceptProductSince version 1.6.0
Product guide: what the capability brings, for whom, its limits and the version that delivered it - no configuration syntax.
| What it does | A Kafka producer, consumer or admin tool, of any library, connects to the gateway with no code change. Each tenant sees its virtual cluster: its topics, groups and transactions under short names, rewritten into its namespace on the fly. Quotas per client and per tenant the way Kafka does them (throttling, never a cut). Fields declared sensitive are encrypted by the gateway under your KMS (Vault transit, AWS KMS, Google Cloud KMS) and decrypted for one scope only |
| Why it matters | The broker's credentials stay the gateway's; a client authenticates with the same token, key or certificate as over HTTP. The broker, its backups and any reader without the scope never see the clear value. Destroying a tenant's key makes its fields unreadable for good (crypto-shredding), leaving the others untouched |
| Use cases | Open a shared Kafka cluster to several customers without any of them seeing another's data; apply a rate per client; protect an email or an IBAN in a topic; answer a GDPR erasure request by destroying a key |
| Limits | Tested end to end against Apache Kafka, Redpanda and Vault with franz-go, librdkafka and the Java client; performance benchmarks written, not run yet - no figure is promised; quotas not complete yet: a tenant's daily or monthly budget does not count reads yet, and two defaults (throttle cap, new connections per tenant) are provisional; KIP-848's group protocol is not served; administration (creating a topic) is refused unless an admin scope is declared; a native NATS or AMQP proxy is not planned |
| Since | 1.6.0 |
| Configure | Native Kafka clients: a virtual cluster per tenant, Field encryption |