[
https://issues.apache.org/jira/browse/KAFKA-8448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16852473#comment-16852473
]
ASF GitHub Bot commented on KAFKA-8448:
---------------------------------------
jolshan commented on pull request #6847: KAFKA-8448: Too many kafka.log.Log
instances (Memory Leak)
URL: https://github.com/apache/kafka/pull/6847
Added a method to delete the PeriodicProducerExpirationCheck task that kept
adding to the log.
Includes a test to show that the task exists and then is removed when the
log is closed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Too many kafka.log.Log instances (Memory Leak)
> ----------------------------------------------
>
> Key: KAFKA-8448
> URL: https://issues.apache.org/jira/browse/KAFKA-8448
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 2.2.0
> Environment: Red Hat 4.4.7-16, java version "1.8.0_152",
> kafka_2.12-2.2.0
> Reporter: Juan Olivares
> Assignee: Justine Olshan
> Priority: Major
>
> We have a custom Kafka health check which creates a topic, add some ACLs
> (read/write topic and group), produce & consume a single message and then
> quickly remove it and all the related ACLs created. We close the consumer
> involved, but no the producer.
> We have observed that # of instances of {{kafka.log.Log}} keep growing, while
> there's no evidence of topics being leaked, neither running
> {{/opt/kafka/bin/kafka-topics.sh --zookeeper localhost:2181 --describe}} ,
> nor looking at the disk directory where topics are stored.
> After looking at the heapdump we've observed the following
> - None of the {{kafka.log.Log}} references ({{currentLogs}},
> {{logsToBeDeleted }} and {{logsToBeDeleted}}) in {{kafka.log.LogManager}} is
> holding the big amount of {{kafka.log.Log}} instances.
> - The only reference preventing {{kafka.log.Log}} to be Garbage collected
> seems to be
> {{java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue}} which
> contains schedule tasks created with the name
> {{PeriodicProducerExpirationCheck}}.
> I can see in the code that for every {{kafka.log.Log}} a task with this name
> is scheduled.
> {code:java}
> scheduler.schedule(name = "PeriodicProducerExpirationCheck", fun = () => {
> lock synchronized {
> producerStateManager.removeExpiredProducers(time.milliseconds)
> }
> }, period = producerIdExpirationCheckIntervalMs, delay =
> producerIdExpirationCheckIntervalMs, unit = TimeUnit.MILLISECONDS)
> {code}
> However it seems those tasks are never unscheduled/cancelled
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
