Timur created KAFKA-17043:
-----------------------------
Summary: Strictly Uniform Sticky partition strategy leads to slow
handling of command events
Key: KAFKA-17043
URL: https://issues.apache.org/jira/browse/KAFKA-17043
Project: Kafka
Issue Type: Bug
Reporter: Timur
h1. Summary
Performance degradation in command scenario due to implementation of KIP-794
and the making of Strictly Uniform Sticky partition strategy default.
h1. Problem Description
Following KIP-794, the Strictly Uniform Sticky partition strategy has been
introduced and set as the default, resulting in the deprecation of the
{{{}DefaultPartitioner{}}}.
The new partitioning strategy has led to significant performance degradation
within our "command" scenario. Our workflow orchestrator dispatches command
topics to consumers via Kafka. These commands require high loads on the
consumers, and to manage this, we use multiple partitions to distribute the
workload across a consumer group.
However, with the Strictly Uniform Sticky partitioning, the majority of these
commands- which often do not even reach 100 bytes - are being directed to the
same partition. The new strategy employs a 16KB limit as a cutoff for partition
distribution, effectively disabling concurrency for our use case.
h1. Impact
- Severe performance degradation due to poor load distribution and increased
latency in command scenario processing.
h1. Notes
"Un"deprecating of {{DefaultPartitioner}} could be a working option for us.
We could send command messages with some random keys but it looks to us as a
workaround.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
