[
https://issues.apache.org/jira/browse/KAFKA-19763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kamal Chandraprakash updated KAFKA-19763:
-----------------------------------------
Description:
Broker heap memory gets filled up and throws OOM error when remote reads are
triggered for multiple partitions within a FETCH request.
Steps to reproduce:
1. Start a one node broker and configure LocalTieredStorage as remote storage.
2. Create a topic with 5 partitions.
3. Produce message and ensure that few segments are uploaded to remote.
4. Start a consumer to read from those 5 partitions. Seek the offset to
beginning for 4 partitions and to end for 1 partitions. This is to simulate the
FETCH request read from both remote-log and local-log.
5. The broker crashes with the OOM error.
6. The DelayedRemoteFetch / RemoteLogReadResult references are being held by
the purgatory, so the broker crashes.
was:
Broker heap memory gets filled up and throws OOM error when remote reads are
triggered for multiple partitions within a FETCH request.
Steps to reproduce:
1. Start a one node broker and configure LocalTieredStorage as remote storage.
2. Create a topic with 5 partitions.
3. Produce message and ensure that few segments are uploaded to remote.
4. Start a consumer to read from those 5 partitions. Seek the offset to
beginning for 4 partitions and to end for 1 partitions. This is to simulate the
FETCH request read from both remote-log and local-log.
5. The broker crashes with the OOM error.
6. The DelayedRemoteFetch / RemoteLogReadResult references are being held by
the purgatory, so the broker crashes with OOM.
> Parallel remote reads causes memory leak in broker
> --------------------------------------------------
>
> Key: KAFKA-19763
> URL: https://issues.apache.org/jira/browse/KAFKA-19763
> Project: Kafka
> Issue Type: Task
> Reporter: Kamal Chandraprakash
> Assignee: Kamal Chandraprakash
> Priority: Blocker
> Fix For: 4.2.0
>
> Attachments: Screenshot 2025-10-07 at 8.25.45 PM.png
>
>
> Broker heap memory gets filled up and throws OOM error when remote reads are
> triggered for multiple partitions within a FETCH request.
> Steps to reproduce:
> 1. Start a one node broker and configure LocalTieredStorage as remote
> storage.
> 2. Create a topic with 5 partitions.
> 3. Produce message and ensure that few segments are uploaded to remote.
> 4. Start a consumer to read from those 5 partitions. Seek the offset to
> beginning for 4 partitions and to end for 1 partitions. This is to simulate
> the FETCH request read from both remote-log and local-log.
> 5. The broker crashes with the OOM error.
> 6. The DelayedRemoteFetch / RemoteLogReadResult references are being held by
> the purgatory, so the broker crashes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
