kravii created KAFKA-20709:
------------------------------
Summary: Add built-in SMT for header-based topic routing in Kafka
Connect / MirrorMaker 2 with configurable priority across multiple headers
Key: KAFKA-20709
URL: https://issues.apache.org/jira/browse/KAFKA-20709
Project: Kafka
Issue Type: New Feature
Components: connect, mirrormaker
Affects Versions: 3.7.2
Reporter: kravii
Routing messages from a single source topic to multiple destination topics
based on configurable Kafka record headers, with priority ordering when
multiple headers are present, is not available natively in Apache Kafka today.
Plugin required in the mirror maker to handle this.
*Requirement:*
A built in SMT, one that reads a record header's value and uses it to determine
the destination topic.
*Motivation:*
Modern data platforms increasingly need to route records based on metadata
carried on the record itself, rather than on topic names or timestamps alone.
Required capabilities -
* Multi-tenant Kafka platforms that need per-tenant topic isolation driven by
tenant-id headers
* Data protection regulations, requiring data segregation by classification,
sensitivity, or residency.
* Content classification pipelines where an upstream classifier writes its
decision into a header and downstream routing must honor it.
* Tiered processing pipelines that route records to different topics based on
priority or SLA metadata.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
