chungen0126 commented on PR #9402: URL: https://github.com/apache/ozone/pull/9402#issuecomment-4976356671
Hi all, I've submitted a [PR](https://github.com/apache/ozone/pull/10765) to implement positional reads in `BlockInputStream`. Please take a look and review it when you are available. Along with the implementation, I've written a microbenchmark for readVectored on this [branch](https://github.com/chungen0126/ozone/tree/HDDS-13660) to verify the performance gains on the client side from reducing lock contention. ## Benchmark Setup: Platform: Local macOS File Configuration: FILE_SIZE = 100 * 1024 * 1024 (100 MB) BLOCK_SIZE = 20 * 1024 * 1024 (20 MB / 5 blocks total) CHUNK_SIZE = 4 * 1024 * 1024 (4 MB / 5 chunks per block) ## Test Scenarios: Consecutive range pattern: 20 ranges, 1 MB each Random range pattern: 20 ranges, 1 MB each, spread randomly Sparse range pattern: 5 ranges, 128 KB each, widely separated ## Results: Plaintext --- Pattern: Consecutive Ranges --- Vectored Read (Positional Enabled=true) : Mean: 90.54 ms | Min: 42.02 ms | Max: 146.75 ms | Throughput: 220.89 MB/s Vectored Read (Positional Enabled=false) : Mean: 97.63 ms | Min: 48.74 ms | Max: 175.95 ms | Throughput: 204.86 MB/s --- Pattern: Random Ranges --- Number of ranges: 4 Total read size : 4.00 MB Vectored Read (Positional Enabled=true) : Mean: 59.60 ms | Min: 36.05 ms | Max: 74.04 ms | Throughput: 67.12 MB/s Vectored Read (Positional Enabled=false) : Mean: 120.47 ms | Min: 96.82 ms | Max: 150.69 ms | Throughput: 33.20 MB/s --- Pattern: Sparse Ranges --- Number of ranges: 5 Total read size : 0.63 MB Vectored Read (Positional Enabled=true) : Mean: 164.00 ms | Min: 96.81 ms | Max: 245.98 ms | Throughput: 3.81 MB/s Vectored Read (Positional Enabled=false) : Mean: 164.61 ms | Min: 101.27 ms | Max: 240.58 ms | Throughput: 3.80 MB/s ## Current Progress & Next Steps: Currently, this branch only implements positional read for general reads. While working on BlockStreamRead, I ran into a few bugs. Therefore, I will follow up with another PR to implement positional read for BlockStreamRead later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
