[
https://issues.apache.org/jira/browse/KAFKA-13597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17477785#comment-17477785
]
Luke Chen edited comment on KAFKA-13597 at 1/18/22, 1:12 PM:
-------------------------------------------------------------
Also, if removing the `ack` config doesn't fix the issue (which I hope it can,
because it means this is only a bad configuration issue), then, could you
please try to disable the idempotence config?
To summarize:
# please try to remove the `ack=1` config and add `enable.idempotence=false`,
and try if it fixes the issue.
# please try to add `enable.idempotence=false` producer config to disable the
idempotence (you can either keep ack=1 or remove it) and try if it fixes the
issue. If that fixes the issue, then it said the issue is in the
transactionManager or some other component related to idempotence.
Please help to test to narrow down the scope. (or you can teach me how to
reproduce it)
Thank you.
was (Author: showuon):
Also, if removing the `ack` config doesn't fix the issue (which I hope it can,
because it means this is only a bad configuration issue), then, could you
please try to disable the idempotence config?
To summarize:
# please try to remove the `ack=1` config and try if it fixes the issue.
# please try to add `enable.idempotence=false` producer config to disable the
idempotence (you can either keep ack=1 or remove it) and try if it fixes the
issue. If that fixes the issue, then it said the issue is in the
transactionManager or some other component related to idempotence.
Please help to test to narrow down the scope. (or you can teach me how to
reproduce it)
Thank you.
> Memory leak with kafka-clients 3.0.0
> ------------------------------------
>
> Key: KAFKA-13597
> URL: https://issues.apache.org/jira/browse/KAFKA-13597
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 3.0.0
> Reporter: Willian Dallastella
> Priority: Major
> Attachments: image-2022-01-17-12-59-43-411.png,
> image-2022-01-17-13-01-43-418.png
>
>
> I'm having this issue reported here:
> [https://github.com/spring-projects/spring-kafka/issues/2056]
>
> It is a Spring Boot 2.5.7 application that started failing (out of memory)
> when updated to Spring Boot 2.6.1.
> This updated *kafka-clients {color:#00875a}2.7.1{color}* to
> *{color:#ff0000}3.0.0{color}* and started causing the memory leak.
>
> The service is using a standard spring *katkaTemplate* to send messages.
> {code:java}
> kafkaTemplate.send(
> topic,
> id,
> data
> ).addCallback(kafkaLoggerCallback) {code}
> This producer is quite heavy (~2.5k messages/s).
> My kafka config:
> {code:java}
> spring:
> kafka:
> bootstrap-servers:
> - my_kafka_host:9092
> properties:
> client.id: my_client_id
> ssl.endpoint.identification.algorithm: https
> request.timeout.ms: 20000
> retry.backoff.ms: 500
> sasl.jaas.config:
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="my_user" password="my_password";
> security.protocol: SASL_SSL
> sasl.mechanism: PLAIN
> interceptor.classes:
> io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor
> confluent.monitoring.interceptor.sasl.mechanism: PLAIN
> confluent.monitoring.interceptor.security.protocol: SASL_SSL
> confluent.monitoring.interceptor.sasl.jaas.config:
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="my_user" password="my_password";
> schema.registry.basic.auth.user.info: sr_ccloud_key:sr_ccloud_key
> schema.registry.url: https://my_schema_registry
> ack: 1
> producer:
> key-serializer: io.confluent.kafka.serializers.KafkaAvroSerializer
> value-serializer: io.confluent.kafka.serializers.KafkaAvroSerializer
> {code}
>
> *Downgrading* just the lib *kafka-clients* to *{color:#00875a}2.7.1{color}*
> or *{color:#00875a}2.8.1{color}* solves the issue.
> h2. Memory leak sample
> * The green line around 16:40 is the service using kafka-clients 2.7.1
> * The pink line around 16:55 is without downgrading, that means
> kafka-clients 3.0.0
> !image-2022-01-17-12-59-43-411.png|width=1017,height=303!
>
> h2. Profiling sample
>
> !image-2022-01-17-13-01-43-418.png!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
