[
https://issues.apache.org/jira/browse/KAFKA-15402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17916037#comment-17916037
]
Benoit Delbosc edited comment on KAFKA-15402 at 1/22/25 12:18 PM:
------------------------------------------------------------------
Hi David, thanks for your response,
I have just tested a 3.9.0, here is my limited understanding:
The {{ClassicKafkaConsumer}} is closing the {{Fetcher}}, in
{{Fetcher#MaybecloseFetchSession}} a {{FetchRequest}} is prepared with
sessionId and the {{epoch=FINAL}} metadata to release the session.
The request is sent during the first call to {{ConsumerNetworkClient#poll}}
which returns nothing, the Fetcher performs a second poll, it takes ~500ms,
this is reported in the {{ClientResponse(latencyMs=508)}}, the time is spent in
{{org.apache.kafka.common.network.Selector#select}} which is invoking the
{{nioSelect}}.
On the broker side I see that the {{FetchRequest}} is processed only in few
milliseconds in {{kafka.server.KafkaApis#handleFetchRequest}} and the response
is immediately received by the consumer.
To me it looks like a network latency, it's like the {{FetchRequest}} is sent
by the consumer but it is not flushed or not readable immediately by the broker
but I am not able to prove it.
Let me know if you have any ideas for moving forward.
Note that Kafka is deployed with Zookeeper and consumer properties contain a
{{group.id}} and {{enable.auto.commit=false}}.
was (Author: bdelbosc):
Hi David, thanks for your response,
I have just tested a 3.9.0, here is my limited understanding:
The {{ClassicKafkaConsumer}} is closing the {{Fetcher}}, in
{{Fetcher#MaybecloseFetchSession}} a {{FetchRequest}} is prepared with
sessionId and the {{epoch=FINAL}} metadata to release the session.
The request is sent during the first call to {{ConsumerNetworkClient#poll}}
which returns nothing, the Fetcher performs second poll, it takes ~500ms, this
is reported in the {{ClientResponse(latencyMs=508)}}, the time is spent in
{{org.apache.kafka.common.network.Selector#select}} which is invoking the
{{nioSelect}}.
On the broker side I see that the {{FetchRequest}} is processed only in few
milliseconds in {{kafka.server.KafkaApis#handleFetchRequest}} and the response
is immediately received by the consumer.
So it's like the {{FetchRequest}} is sent by the consumer but it is not flushed
or not readable immediately by the broker but I am not able to prove it.
Let me know if you have any ideas for moving forward.
Note that Kafka is deployed with Zookeeper and consumer properties contain a
{{group.id}} and {{enable.auto.commit=false}}.
> Performance regression on close consumer after upgrading to 3.5.0
> -----------------------------------------------------------------
>
> Key: KAFKA-15402
> URL: https://issues.apache.org/jira/browse/KAFKA-15402
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Affects Versions: 3.5.0, 3.6.0, 3.5.1
> Reporter: Benoit Delbosc
> Assignee: Kirk True
> Priority: Major
> Fix For: 4.1.0
>
> Attachments: image-2023-08-24-18-51-21-720.png,
> image-2023-08-24-18-51-57-435.png, image-2023-08-25-10-50-28-079.png
>
>
> Hi,
> After upgrading to Kafka client version 3.5.0, we have observed a significant
> increase in the duration of our Java unit tests. These unit tests heavily
> rely on the Kafka Admin, Producer, and Consumer API.
> When using Kafka server version 3.4.1, the duration of the unit tests
> increased from 8 seconds (with Kafka client 3.4.1) to 18 seconds (with Kafka
> client 3.5.0).
> Upgrading the Kafka server to 3.5.1 show similar results.
> I have come across the issue KAFKA-15178, which could be the culprit. I will
> attempt to test the proposed patch.
> In the meantime, if you have any ideas that could help identify and address
> the regression, please let me know.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
