hanishakoneru opened a new pull request #1685:
URL: https://github.com/apache/ozone/pull/1685
## What changes were proposed in this pull request?
When a ReadChunk operation is performed, all the data to be read from one
chunk is read into a single ByteBuffer.
```
#ChunkUtils#readData()
public static void readData(File file, ByteBuffer buf,
long offset, long len, VolumeIOStats volumeIOStats)
throws StorageContainerException {
.....
try {
bytesRead = processFileExclusively(path, () -> {
try (FileChannel channel = open(path, READ_OPTIONS, NO_ATTRIBUTES);
FileLock ignored = channel.lock(offset, len, true)) {
return channel.read(buf, offset);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
});
} catch (UncheckedIOException e) {
throw wrapInStorageContainerException(e.getCause());
}
.....
.....
```
This Jira proposes to read the data from the channel and put it into an
array of ByteBuffers each with a set capacity. This capacity can be
configurable by client.
This would help with optimizing Ozone InputStreams in terms of cached
memory. Currently, data in ChunkInputStream is cached till either the stream is
closed or the chunk EOF is reached. This sometimes leads to upto 4MB (default
ChunkSize) of data being cached in memory per ChunkInputStream.
After the proposed change, we can optimize ChunkInputStream to release a
ByteBuffer as soon as that ByteBuffer is read instead of waiting to read the
whole chunk (HDDS-4553). Read I/O performance will not be affected as the read
from DN still returns the requested length of data at one go. Only difference
would be that the data would be returned in an array of ByteBuffer instead of a
single ByteBuffer.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4552
## How was this patch tested?
Added integration test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
