[ https://issues.apache.org/jira/browse/FLINK-31914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martijn Visser updated FLINK-31914: ----------------------------------- Component/s: Connectors / Kafka > Failing to close FlinkKafkaInternalProducer created in KafkaWriter with > exactly-once semantic results in memory leak > -------------------------------------------------------------------------------------------------------------------- > > Key: FLINK-31914 > URL: https://issues.apache.org/jira/browse/FLINK-31914 > Project: Flink > Issue Type: Bug > Components: Connectors / Kafka > Affects Versions: 1.15.0, 1.17.0 > Reporter: datariver > Priority: Major > Attachments: image-2023-04-24-16-11-22-251.png > > > Hi [~arvid] , If Exactly-Once writing is enabled, Kafka's transactional > writing will be used. KafkaWriter will create FlinkKafkaInternalProducer in > the initialization and snapshotState methods, but there is no place to close > it. As Checkpoints increase, Producers will continue to accumulate. Each > Producer maintains a Buffer, which will cause memory leaks and Job OOM. > By dumping an in-memory instance of Task Manager, you can see that there are > a lot of Producers: > !image-2023-04-24-16-11-22-251.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)