zhaobo created KAFKA-8543:
-----------------------------
Summary: Optimize Fetch thread allocation strategy to ensure full
utilization of Fetch thread resources
Key: KAFKA-8543
URL: https://issues.apache.org/jira/browse/KAFKA-8543
Project: Kafka
Issue Type: Improvement
Components: core
Affects Versions: 2.2.0
Reporter: zhaobo
The Follow TP on the Broker needs to fetch data from the Leader broker. The
original fetch thread allocation strategy has two areas that can be optimized:
（1）The configuration file has a parameter "num.consumer.fetchers" which limits
the number of threads fetching data from a source broker. In fact, Fetch
resources are not fully used, and only a part of the threads will be used with
a high probability. When the traffic of the Leader TP increases, the fetch
capability may be bottlenecked.
（2）The number of TPs responsible for each Fetch thread may be uneven.
Improve: We propose a polling allocation method to ensure that Fetch thread
resources can be fully utilized. And the number of TPs responsible for each
fetch thread is balanced.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
