[
https://issues.apache.org/jira/browse/KAFKA-21008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sanghyeok An updated KAFKA-21008:
---------------------------------
Labels: needs-kip streams (was: streams)
> Consider allowing record headers to be filtered or masked before persistence
> in Kafka Streams state stores
> ----------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-21008
> URL: https://issues.apache.org/jira/browse/KAFKA-21008
> Project: Kafka
> Issue Type: Improvement
> Reporter: sanghyeok An
> Assignee: sanghyeok An
> Priority: Minor
> Labels: needs-kip, streams
>
> Kafka Streams header-aware state stores introduced by KIP-1271 can persist
> record headers together with state values and propagate them through state
> store changelogs.
>
> Record headers may contain application-level sensitive information, such as
> credentials, tokens, personally identifiable information, or request metadata
> that should not be retained at rest.
>
> Currently, Kafka Streams does not provide a built-in mechanism to filter,
> remove, or mask selected headers before they are written to a header-aware
> state store. Applications must sanitize headers manually before every
> stateful operator, which is error-prone and may also modify the headers
> observed by downstream processors.
>
> Kafka Streams should provide an opt-in mechanism to filter or transform the
> copy of the headers that is persisted by header-aware state stores.
>
> The mechanism should:
> * Allow selected headers to be removed or their values to be masked.
> * Apply consistently to both local state and state-store changelog records.
> * Avoid mutating the headers of the in-flight input record.
> * Preserve the current behavior when no filtering or masking policy is
> configured.
> * Ensure that restored state contains the same sanitized headers that were
> written to the local store.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
