[
https://issues.apache.org/jira/browse/HDDS-15359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-15359:
------------------------------
Description:
`BufferUtils.assignByteBuffers` allocates heap `ByteBuffer`s for chunk reads.
Reading from a `FileChannel` into a heap buffer forces the JDK to first read
into a thread-local `DirectByteBuffer` (in `sun.nio.ch.IOUtil`) and then memcpy
the chunk-sized payload into the user's heap buffer. So every default-config
chunk read paid one extra user-space chunk-sized memcpy plus one chunk-sized
heap allocation.
Two prior PRs added opt-in direct-buffer paths to skip exactly this copy:
* HDDS-7188 (Tsz-Wo Nicholas Sze, Jan 2025):
`ChunkUtils.readDataNettyChunkedNioFile` via Netty
`PooledByteBufAllocator.DEFAULT`
* HDDS-10488 (Sammi Chen, Sep 2024): `MappedBufferManager` mmap reads with
semaphore-tracked quota
was:BufferUtils.assignByteBuffers allocated heap ByteBuffers
> Read chunks into pooled direct buffers
> --------------------------------------
>
> Key: HDDS-15359
> URL: https://issues.apache.org/jira/browse/HDDS-15359
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Datanode
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
>
> `BufferUtils.assignByteBuffers` allocates heap `ByteBuffer`s for chunk reads.
> Reading from a `FileChannel` into a heap buffer forces the JDK to first read
> into a thread-local `DirectByteBuffer` (in `sun.nio.ch.IOUtil`) and then
> memcpy the chunk-sized payload into the user's heap buffer. So every
> default-config chunk read paid one extra user-space chunk-sized memcpy plus
> one chunk-sized heap allocation.
> Two prior PRs added opt-in direct-buffer paths to skip exactly this copy:
> * HDDS-7188 (Tsz-Wo Nicholas Sze, Jan 2025):
> `ChunkUtils.readDataNettyChunkedNioFile` via Netty
> `PooledByteBufAllocator.DEFAULT`
> * HDDS-10488 (Sammi Chen, Sep 2024): `MappedBufferManager` mmap reads with
> semaphore-tracked quota
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
