frankvicky opened a new pull request, #18917: URL: https://github.com/apache/kafka/pull/18917
https://github.com/apache/kafka/pull/18783#issuecomment-2646059254 This benchmark compares the performance of the new implementation using records and that of the original implementation of `TopicIdPartition`. ``` # JMH version: 1.37 # VM version: JDK 21.0.4, OpenJDK 64-Bit Server VM, 21.0.4+7-LTS # VM invoker: /home/frankvicky/.sdkman/candidates/java/21.0.4-zulu/bin/java # VM options: <none> # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable) # Warmup: 5 iterations, 10 s each # Measurement: 15 iterations, 10 s each # Timeout: 10 min per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Average time, time/op # Benchmark: org.apache.kafka.jmh.common.TopicIdPartitionBenchmark.oldImplBenchmark # Run progress: 0.00% complete, ETA 00:06:40 # Fork: 1 of 1 # Warmup Iteration 1: 13378553.352 ns/op # Warmup Iteration 2: 13143933.967 ns/op # Warmup Iteration 3: 13789562.764 ns/op # Warmup Iteration 4: 14100194.587 ns/op # Warmup Iteration 5: 13455289.745 ns/op Iteration 1: 12864633.009 ns/op Iteration 2: 12752273.632 ns/op Iteration 3: 14119982.567 ns/op Iteration 4: 14096648.766 ns/op Iteration 5: 14238380.744 ns/op Iteration 6: 13990067.980 ns/op Iteration 7: 12631045.376 ns/op Iteration 8: 12917548.382 ns/op Iteration 9: 12690788.812 ns/op Iteration 10: 12860821.365 ns/op Iteration 11: 12794588.477 ns/op Iteration 12: 12659599.461 ns/op Iteration 13: 13168017.536 ns/op Iteration 14: 12951730.841 ns/op Iteration 15: 12979521.488 ns/op Result "org.apache.kafka.jmh.common.TopicIdPartitionBenchmark.oldImplBenchmark": 13181043.229 ±(99.9%) 639257.496 ns/op [Average] (min, avg, max) = (12631045.376, 13181043.229, 14238380.744), stdev = 597961.859 CI (99.9%): [12541785.733, 13820300.725] (assumes normal distribution) # JMH version: 1.37 # VM version: JDK 21.0.4, OpenJDK 64-Bit Server VM, 21.0.4+7-LTS # VM invoker: /home/frankvicky/.sdkman/candidates/java/21.0.4-zulu/bin/java # VM options: <none> # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable) # Warmup: 5 iterations, 10 s each # Measurement: 15 iterations, 10 s each # Timeout: 10 min per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Average time, time/op # Benchmark: org.apache.kafka.jmh.common.TopicIdPartitionBenchmark.recordBenchmark # Run progress: 50.00% complete, ETA 00:03:21 # Fork: 1 of 1 # Warmup Iteration 1: 11317503.784 ns/op # Warmup Iteration 2: 11222028.274 ns/op # Warmup Iteration 3: 11193265.445 ns/op # Warmup Iteration 4: 11482521.968 ns/op # Warmup Iteration 5: 11185707.489 ns/op Iteration 1: 11548208.823 ns/op Iteration 2: 11049864.919 ns/op Iteration 3: 10936808.624 ns/op Iteration 4: 10900101.174 ns/op Iteration 5: 11074759.463 ns/op Iteration 6: 11078497.916 ns/op Iteration 7: 11216136.820 ns/op Iteration 8: 11286822.822 ns/op Iteration 9: 11220933.487 ns/op Iteration 10: 11171010.641 ns/op Iteration 11: 11086594.281 ns/op Iteration 12: 11174884.708 ns/op Iteration 13: 10899437.063 ns/op Iteration 14: 10966079.005 ns/op Iteration 15: 11117782.846 ns/op Result "org.apache.kafka.jmh.common.TopicIdPartitionBenchmark.recordBenchmark": 11115194.839 ±(99.9%) 180693.785 ns/op [Average] (min, avg, max) = (10899437.063, 11115194.839, 11548208.823), stdev = 169021.079 CI (99.9%): [10934501.054, 11295888.625] (assumes normal distribution) # Run complete. Total time: 00:06:43 REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial experiments, perform baseline and negative tests that provide experimental control, make sure the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts. Do not assume the numbers tell you what you want them to tell. NOTE: Current JVM experimentally supports Compiler Blackholes, and they are in use. Please exercise extra caution when trusting the results, look into the generated code to check the benchmark still works, and factor in a small probability of new VM bugs. Additionally, while comparisons between different JVMs are already problematic, the performance difference caused by different Blackhole modes can be very significant. Please make sure you use the consistent Blackhole mode for comparisons. Benchmark Mode Cnt Score Error Units TopicIdPartitionBenchmark.oldImplBenchmark avgt 15 13181043.229 ± 639257.496 ns/op TopicIdPartitionBenchmark.recordBenchmark avgt 15 11115194.839 ± 180693.785 ns/op JMH benchmarks done ``` ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
