[
https://issues.apache.org/jira/browse/KAFKA-19472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dyingjiecai updated KAFKA-19472:
--------------------------------
Description:
RemoteLogReader throws a BufferOverflowException when FetchApi version <= 2 and
firstBatchSize exceeds max.partition.fetch.bytes
Exception Message
{code:java}
[2025-07-04 16:11:42,910] ERROR Error occurred while reading the remote data
for remote-storage-perf-5-0 (kafka.log.remote.RemoteLogReader)
java.nio.BufferOverflowException
at java.base/java.nio.ByteBuffer.put(ByteBuffer.java:1007)
at java.base/java.nio.HeapByteBuffer.put(HeapByteBuffer.java:243)
at
org.apache.kafka.common.record.DefaultRecordBatch.writeTo(DefaultRecordBatch.java:236)
at kafka.log.remote.RemoteLogManager.read(RemoteLogManager.java:1647)
at
kafka.log.remote.RemoteLogReader.lambda$call$0(RemoteLogReader.java:66)
at com.yammer.metrics.core.Timer.time(Timer.java:91)
at kafka.log.remote.RemoteLogReader.call(RemoteLogReader.java:66)
at kafka.log.remote.RemoteLogReader.call(RemoteLogReader.java:36)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:842)
{code}
firstBatch.writeTo(buffer) will throw BufferOverflowException when firstBatch
size is larger than the size of buffer passed to the method.
The fix below appears to resolve this issue, however, the potential side
effects remain unknown.
!screenshot-1.png!
was:
RemoteLogReader throws a BufferOverflowException when FetchApi version <= 2 and
firstBatchSize exceeds max.partition.fetch.bytes
Exception Message
{code:java}
[2025-07-04 16:11:42,910] ERROR Error occurred while reading the remote data
for remote-storage-perf-5-0 (kafka.log.remote.RemoteLogReader)
java.nio.BufferOverflowException
at java.base/java.nio.ByteBuffer.put(ByteBuffer.java:1007)
at java.base/java.nio.HeapByteBuffer.put(HeapByteBuffer.java:243)
at
org.apache.kafka.common.record.DefaultRecordBatch.writeTo(DefaultRecordBatch.java:236)
at kafka.log.remote.RemoteLogManager.read(RemoteLogManager.java:1647)
at
kafka.log.remote.RemoteLogReader.lambda$call$0(RemoteLogReader.java:66)
at com.yammer.metrics.core.Timer.time(Timer.java:91)
at kafka.log.remote.RemoteLogReader.call(RemoteLogReader.java:66)
at kafka.log.remote.RemoteLogReader.call(RemoteLogReader.java:36)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:842)
{code}
firstBatch.writeTo(buffer) will throw BufferOverflowException when firstBatch
size is larger than the size of buffer passed to the method
!screenshot-1.png!
> A BufferOverflowException is thrown by RemoteLogManager when FetchApiVersion
> <= 2
> ---------------------------------------------------------------------------------
>
> Key: KAFKA-19472
> URL: https://issues.apache.org/jira/browse/KAFKA-19472
> Project: Kafka
> Issue Type: Bug
> Components: log
> Affects Versions: 3.9.1
> Environment: BrokerVersion 3.9.1
> ClientVersion 0.10.0.0
> Client max.partitioin.fetch.bytes > record size
> Reporter: dyingjiecai
> Priority: Major
> Attachments: 1280X1280 (1).PNG, screenshot-1.png
>
>
> RemoteLogReader throws a BufferOverflowException when FetchApi version <= 2
> and firstBatchSize exceeds max.partition.fetch.bytes
> Exception Message
> {code:java}
> [2025-07-04 16:11:42,910] ERROR Error occurred while reading the remote data
> for remote-storage-perf-5-0 (kafka.log.remote.RemoteLogReader)
> java.nio.BufferOverflowException
> at java.base/java.nio.ByteBuffer.put(ByteBuffer.java:1007)
> at java.base/java.nio.HeapByteBuffer.put(HeapByteBuffer.java:243)
> at
> org.apache.kafka.common.record.DefaultRecordBatch.writeTo(DefaultRecordBatch.java:236)
> at kafka.log.remote.RemoteLogManager.read(RemoteLogManager.java:1647)
> at
> kafka.log.remote.RemoteLogReader.lambda$call$0(RemoteLogReader.java:66)
> at com.yammer.metrics.core.Timer.time(Timer.java:91)
> at kafka.log.remote.RemoteLogReader.call(RemoteLogReader.java:66)
> at kafka.log.remote.RemoteLogReader.call(RemoteLogReader.java:36)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:842)
> {code}
> firstBatch.writeTo(buffer) will throw BufferOverflowException when firstBatch
> size is larger than the size of buffer passed to the method.
> The fix below appears to resolve this issue, however, the potential side
> effects remain unknown.
> !screenshot-1.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
