[ 
https://issues.apache.org/jira/browse/FLINK-31914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martijn Visser updated FLINK-31914:
-----------------------------------
    Component/s: Connectors / Kafka

> Failing to close FlinkKafkaInternalProducer created in KafkaWriter with 
> exactly-once semantic results in memory leak
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-31914
>                 URL: https://issues.apache.org/jira/browse/FLINK-31914
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Kafka
>    Affects Versions: 1.15.0, 1.17.0
>            Reporter: datariver
>            Priority: Major
>         Attachments: image-2023-04-24-16-11-22-251.png
>
>
> Hi [~arvid] , If Exactly-Once writing is enabled, Kafka's transactional 
> writing will be used. KafkaWriter will create FlinkKafkaInternalProducer in 
> the initialization and snapshotState methods, but there is no place to close 
> it. As Checkpoints increase, Producers will continue to accumulate. Each 
> Producer maintains a Buffer, which will cause memory leaks and Job OOM.
> By dumping an in-memory instance of Task Manager, you can see that there are 
> a lot of Producers:
> !image-2023-04-24-16-11-22-251.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to