[
https://issues.apache.org/jira/browse/KAFKA-19506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kaushik Raina updated KAFKA-19506:
----------------------------------
Description:
Kafka clients currently select the first compression type offered by the broker
for telemetry push, often leading to errors if the required compression library
(e.g., zstd, lz4, snappy) is not present in the client runtime. This causes
telemetry push failures, while produce requests continue to work normally. Gzip
is generally reliable as it is always present in the JDK, but other algorithms
can trigger fatal exceptions, stopping further telemetry pushes.
This ticket proposes enhancing the Kafka client to dynamically select the next
available compression type at runtime. If the initial compression attempt fails
(e.g., due to a missing dependency), the client should automatically retry
using the next broker-supported compression type. If none are available, it
must gracefully fall back to sending uncompressed telemetry data, ensuring
telemetry push always succeeds regardless of available libraries.
was:
Kafka clients currently select the first compression type offered by the broker
for telemetry push, often leading to errors if the required compression library
(e.g., zstd, lz4, snappy) is not present in the client runtime. This causes
telemetry push failures, while produce requests continue to work normally. Gzip
is generally reliable as it is always present in the JDK, but other algorithms
can trigger fatal exceptions, stopping further telemetry pushes.
This ticket proposes enhancing the Kafka client to dynamically select the best
available compression type at runtime. If the initial compression attempt fails
(e.g., due to a missing dependency), the client should automatically retry
using the next broker-supported compression type. If none are available, it
must gracefully fall back to sending uncompressed telemetry data, ensuring
telemetry push always succeeds regardless of deployment environment or
available libraries.
> Implement dynamic compression type selection and fallback for client telemetry
> ------------------------------------------------------------------------------
>
> Key: KAFKA-19506
> URL: https://issues.apache.org/jira/browse/KAFKA-19506
> Project: Kafka
> Issue Type: Bug
> Reporter: Kaushik Raina
> Assignee: Kaushik Raina
> Priority: Major
> Fix For: 4.2.0
>
>
> Kafka clients currently select the first compression type offered by the
> broker for telemetry push, often leading to errors if the required
> compression library (e.g., zstd, lz4, snappy) is not present in the client
> runtime. This causes telemetry push failures, while produce requests continue
> to work normally. Gzip is generally reliable as it is always present in the
> JDK, but other algorithms can trigger fatal exceptions, stopping further
> telemetry pushes.
> This ticket proposes enhancing the Kafka client to dynamically select the
> next available compression type at runtime. If the initial compression
> attempt fails (e.g., due to a missing dependency), the client should
> automatically retry using the next broker-supported compression type. If none
> are available, it must gracefully fall back to sending uncompressed telemetry
> data, ensuring telemetry push always succeeds regardless of available
> libraries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
