Sunyeop Lee created FLINK-33201:
-----------------------------------
Summary: Memory leak in CachingTopicSelector
Key: FLINK-33201
URL: https://issues.apache.org/jira/browse/FLINK-33201
Project: Flink
Issue Type: Bug
Components: Connectors / Kafka
Environment: I am out of office now, so this is what I remember (the
flink version may not be correct). Because I already identified the cause, this
should not matter anyway.
EKS 1.24, x86_64, Bottlerocket OS, flink 1.14, scala 2.12
Reporter: Sunyeop Lee
Attachments: 273084767-29bc0d8a-7445-4a74-a6e1-7c836775c7b1.png
Pull Request available at:
https://github.com/apache/flink-connector-kafka/pull/55
In the CachingTopicSelector, a memory leak may occur when the internal logic
fails to check the cache size due to a race condition.
([https://github.com/apache/flink-connector-kafka/blob/d89a082180232bb79e3c764228c4e7dbb9eb6b8b/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/sink/KafkaRecordSerializationSchemaBuilder.java#L287-L289)]
By analyzing a Java heap dump, I identified a memory leak in the
CachingTopicSelector. As in the screenshot, cache has 47,769 elements. If the
internal logic were functioning correctly, the number of elements should be
less than or equal to CACHE_RESET_SIZE (which is 5).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
