[
https://issues.apache.org/jira/browse/FLINK-31914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
datariver updated FLINK-31914:
------------------------------
Attachment: (was: image-2023-04-24-16-11-22-251.png)
> Failing to close FlinkKafkaInternalProducer created in KafkaWriter with
> exactly-once semantic results in memory leak
> --------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-31914
> URL: https://issues.apache.org/jira/browse/FLINK-31914
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: 1.15.0, 1.16.0, 1.17.0
> Reporter: datariver
> Priority: Major
> Attachments: image-2023-04-25-13-47-25-703.png
>
>
> Hi [~arvid] , If Exactly-Once writing is enabled, Kafka's transactional
> writing will be used. KafkaWriter will create FlinkKafkaInternalProducer in
> the initialization and snapshotState methods, but there is no place to close
> it. As Checkpoints increase, Producers will continue to accumulate. Each
> Producer maintains a Buffer, which will cause memory leaks and Job OOM.
> By dumping an in-memory instance of Task Manager, you can see that there are
> a lot of Producers:
> !image-2023-04-25-13-47-25-703.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
