geniusjoe commented on code in PR #25274: URL: https://github.com/apache/pulsar/pull/25274#discussion_r2881607558
########## bin/bookkeeper: ########## @@ -209,6 +209,24 @@ OPTS="-Djava.net.preferIPv4Stack=true $OPTS" if [[ $JAVA_MAJOR_VERSION -ge 23 ]]; then OPTS="--sun-misc-unsafe-memory-access=allow $OPTS" fi +# Netty tuning +# These settings are primarily used to modify the Netty allocator configuration, +# improving memory utilization and reducing the frequency of requesting off-heap memory from the OS +# +# Based on the netty source code, the allocator's default chunk size is calculated as: +# io.netty.allocator.pageSize(default: 8192) << io.netty.allocator.maxOrder(default: 9 after Netty 4.1.76.Final version). +# This equals 8192 * 2^9 = 4 MB： +# https://github.com/netty/netty/blob/4.1/buffer/src/main/java/io/netty/buffer/PooledByteBufAllocator.java#L105 +# +# Allocations that are larger than chunk size are considered huge allocations and don't use the pool: +# https://github.com/netty/netty/blob/4.1/buffer/src/main/java/io/netty/buffer/PoolArena.java#L141-L142 +# +# Currently, Pulsar defaults to a maximum single message size of 5 MB. +# Therefore, when frequently producing messages whose size exceeds the chunk size, +# Netty cannot utilize resources from the memory pool and must frequently allocate native memory. +# This can lead to increased physical memory fragmentation and higher reclamation costs. +# Thus, increasing io.netty.allocator.maxOrder to 10 to ensure that a single message is larger than chunk size(8MB) and can reuse Netty's memory pool. Review Comment: Fixed. Line breaks have been added to single-line comments that were of excessive length. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
