guyinyou opened a new issue, #10015: URL: https://github.com/apache/rocketmq/issues/10015
### Before Creating the Enhancement Request - [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature. ### Summary Optimize write performance in writeWithoutMmap mode by aligning write operations to page boundaries, avoiding expensive read-modify-write operations when pagecache is not available. ### Motivation When warmMappedFile is enabled, files are preloaded into pagecache. However, after system restarts or when pagecache is evicted, the file may not be in pagecache. In writeWithoutMmap mode, append writes become overwrite operations via FileChannel.write(). When write end positions are not aligned to page boundaries, the OS must perform read-modify-write operations: 1. Read the entire page containing the write position 2. Merge the new data with existing page data 3. Write back the entire merged page This is inefficient compared to direct page-aligned writes. By aligning writes to page boundaries, we can achieve pure write operations without read-modify-write overhead, significantly improving write performance in scenarios where pagecache is unavailable. ### Describe the Solution You'd Like Implement page alignment for writes in writeWithoutMmap mode: 1. Calculate the write end position: `endpos = currentPos + msgLen` 2. Calculate extra bytes needed for alignment: `extraAppendSize = UNSAFE_PAGE_SIZE - endpos % UNSAFE_PAGE_SIZE` 3. Calculate aligned write size: `actualAppendSize = msgLen + extraAppendSize` 4. If buffer capacity allows, write the aligned size; otherwise, write only the original message length Implementation location: `DefaultMappedFile.appendMessagesInner()` method, specifically in the `writeWithoutMmap` path when using `sharedByteBuffer`. The solution should: - Align write end positions to `UNSAFE_PAGE_SIZE` boundaries - Handle cases where buffer capacity is insufficient for full alignment - Maintain backward compatibility (commitlog can contain dirty data at the end, which is acceptable) ### Describe Alternatives You've Considered 1. **Always keep files in pagecache**: This is not always possible due to system memory pressure and OS pagecache eviction policies. 2. **Use mmap for all writes**: The writeWithoutMmap mode exists for specific use cases where mmap is not desired, so this alternative doesn't address the requirement. 3. **Pre-allocate aligned buffers**: While this could help, it doesn't solve the fundamental issue of unaligned write positions causing read-modify-write operations. 4. **Ignore alignment and accept read-modify-write overhead**: This maintains current behavior but sacrifices performance, which is not ideal for high-throughput scenarios. The proposed page alignment solution is the most effective approach as it directly addresses the root cause (unaligned writes) with minimal overhead and maintains compatibility with existing behavior. ### Additional Context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
