Philip Nee created KAFKA-18139:
----------------------------------
Summary: AsyncKafkaConsumer's ConsumerNetworkThread loop requires
further optimization
Key: KAFKA-18139
URL: https://issues.apache.org/jira/browse/KAFKA-18139
Project: Kafka
Issue Type: Bug
Components: consumer
Reporter: Philip Nee
We've noticed the AsyncKafkaConsumer polls the NetworkClient at a much higher
cadence than the ClassicKafkaConsumer. Using the
`java-producer-consumer-demo.sh` example, the ClassicKafkaConsumer polls
networkClients almost twice as much as the old consumer and also results in
some side effects such as higher CPU usage (>10% more) and potentially
affecting the request latency. For example, using the example, we measured
ClassicKafkaConsumer's average latency is 10.5ms and AsyncKafkaConsumer is
~12ms.
We suspect the added time delay is mostly caused by the inefficient
NetworkClient poll.
During the demo run produce and consuming 100,000,000 records results in ~400k
network client poll calls for the classic consumer and ~800k for the async
consumer using the default client configuration provided by the example.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
