hachikuji opened a new pull request #9619: URL: https://github.com/apache/kafka/pull/9619
In #9401, I observed some strange latency behavior when testing locally (on macos). The behavior seemed to be caused by slightly different write behavior as a result of the new `SendBuilder` class. After the change, we were making several calls down to both `GatheringByteChannel.write(ByteBuffer[])` as well as `WritableByteChannel.write` whereas previously there would be just one path through the former. The difference is clearly seen in the flame graphs posted here: https://github.com/apache/kafka/pull/9401#issuecomment-729258346. This patch change the grouping of `Send` objects created by `SendBuilder` to try and restore the previous write behavior. Rather than creating a single `ByteBufferSend` for each `ByteBuffer`, we attempt to group consecutive buffers into a single `ByteBufferSend`. I confirmed that did indeed address the latency issue I observed using the same test setup with a single broker that was discussed in #9401. ``` bin/kafka-topics.sh --create --topic foo --replication-factor 1 --partitions 10 --bootstrap-server $BROKER bin/kafka-producer-perf-test.sh --topic foo --num-records 250000000 --throughput -1 --record-size 256 --producer-props bootstrap.servers=$BROKER ``` Here are the results: ``` Before KAFKA-9628: 250000000 records sent, 1398546.630342 records/sec (341.44 MB/sec), 39.71 ms avg latency, 424.00 ms max latency, 16 ms 50th, 137 ms 95th, 194 ms 99th, 270 ms 99.9th. Trunk: 250000000 records sent, 1413619.374502 records/sec (345.12 MB/sec), 87.72 ms avg latency, 2058.00 ms max latency, 64 ms 50th, 247 ms 95th, 429 ms 99th, 790 ms 99.9th. This patch: 250000000 records sent, 1469723.691946 records/sec (358.82 MB/sec), 12.72 ms avg latency, 320.00 ms max latency, 2 ms 50th, 83 ms 95th, 162 ms 99th, 256 ms 99.9th. ``` ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
