[
https://issues.apache.org/jira/browse/KAFKA-15597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Schmitz updated KAFKA-15597:
----------------------------------
Description:
In many use cases you might not only want to drop a few specific Kafka headers
but a set of headers whose names can also dynamically change (e.g. when used
with some end-to-end-encryption libraries, tracing etc.). To prevent those
headers following a special pattern (which may not comply with downstream
system format) to be further forwarded/processed downstream (e.g.header
forwarding in Http Sinks), I suggest to add regexp matching to the *apply*
method instead of a set-based {*}contains{*}. Link to the relevant code:
[https://github.com/apache/kafka/blob/7b5d640cc656443a078bda096d01910b3edfdb37/connect/transforms/src/main/java/org/apache/kafka/connect/transforms/DropHeaders.java#L54
|http://example.com/]
One common example is forwarding of encryption-headers that contain byte arrays
to a HTTP server via HTTP Sinks which will not be accepted by the HTTP server.
Example implementation done - link to PR:
[https://github.com/apache/kafka/pull/14536]
was:
In many use cases you might not only want to drop a few specific Kafka headers
but a set of headers whose names can also dynamically change (e.g. when used
with some end-to-end-encryption libraries, tracing etc.). To prevent those
headers to be further forwarded/processed downstream (e.g. for header
forwarding in like Http Sinks), I suggest to add regexp matching to the *apply*
method instead of a set-based {*}contains{*}. Link to the relevant code:
[https://github.com/apache/kafka/blob/7b5d640cc656443a078bda096d01910b3edfdb37/connect/transforms/src/main/java/org/apache/kafka/connect/transforms/DropHeaders.java#L54
|http://example.com/]
Example implementation done - link to PR:
[https://github.com/apache/kafka/pull/14536]
> Enable Connect DropHeaders SMT to drop headers on a wildcard/regexp-basis
> -------------------------------------------------------------------------
>
> Key: KAFKA-15597
> URL: https://issues.apache.org/jira/browse/KAFKA-15597
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Roman Schmitz
> Assignee: Roman Schmitz
> Priority: Minor
> Labels: connect-transformation
>
> In many use cases you might not only want to drop a few specific Kafka
> headers but a set of headers whose names can also dynamically change (e.g.
> when used with some end-to-end-encryption libraries, tracing etc.). To
> prevent those headers following a special pattern (which may not comply with
> downstream system format) to be further forwarded/processed downstream
> (e.g.header forwarding in Http Sinks), I suggest to add regexp matching to
> the *apply* method instead of a set-based {*}contains{*}. Link to the
> relevant code:
> [https://github.com/apache/kafka/blob/7b5d640cc656443a078bda096d01910b3edfdb37/connect/transforms/src/main/java/org/apache/kafka/connect/transforms/DropHeaders.java#L54
> |http://example.com/]
> One common example is forwarding of encryption-headers that contain byte
> arrays to a HTTP server via HTTP Sinks which will not be accepted by the HTTP
> server.
> Example implementation done - link to PR:
> [https://github.com/apache/kafka/pull/14536]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
