lbradstreet commented on pull request #9008: URL: https://github.com/apache/kafka/pull/9008#issuecomment-659673793
> Updated the benchmarks with @lbradstreet's suggestions. Here are the results for 3 partitions, 10 topics. GC profiles included. > > On this branch: > > ``` > Benchmark (partitionCount) (topicCount) Mode Cnt Score Error Units > FetchRequestBenchmark.testFetchRequestForConsumer 3 10 avgt 15 2110.741 ± 27.935 ns/op > FetchRequestBenchmark.testFetchRequestForReplica 3 10 avgt 15 2021.114 ± 7.816 ns/op > FetchRequestBenchmark.testSerializeFetchRequestForConsumer 3 10 avgt 15 3452.799 ± 16.013 ns/op > FetchRequestBenchmark.testSerializeFetchRequestForReplica 3 10 avgt 15 3691.157 ± 60.260 ns/op > > GC Profile (partitionCount) (topicCount) Mode Cnt Score Error Units > FetchRequestBenchmark.testFetchRequestForConsumer:·gc.alloc.rate 3 10 avgt 15 4295.532 ± 56.061 MB/sec > FetchRequestBenchmark.testFetchRequestForConsumer:·gc.alloc.rate.norm 3 10 avgt 15 9984.000 ± 0.001 B/op > FetchRequestBenchmark.testFetchRequestForConsumer:·gc.churn.PS_Eden_Space 3 10 avgt 15 4292.525 ± 56.341 MB/sec > FetchRequestBenchmark.testFetchRequestForConsumer:·gc.churn.PS_Eden_Space.norm 3 10 avgt 15 9977.037 ± 28.311 B/op > FetchRequestBenchmark.testFetchRequestForConsumer:·gc.churn.PS_Survivor_Space 3 10 avgt 15 0.187 ± 0.027 MB/sec > FetchRequestBenchmark.testFetchRequestForConsumer:·gc.churn.PS_Survivor_Space.norm 3 10 avgt 15 0.435 ± 0.060 B/op > FetchRequestBenchmark.testFetchRequestForConsumer:·gc.count 3 10 avgt 15 2335.000 counts > FetchRequestBenchmark.testFetchRequestForConsumer:·gc.time 3 10 avgt 15 1375.000 ms > FetchRequestBenchmark.testFetchRequestForReplica:·gc.alloc.rate 3 10 avgt 15 4416.855 ± 16.429 MB/sec > FetchRequestBenchmark.testFetchRequestForReplica:·gc.alloc.rate.norm 3 10 avgt 15 9832.000 ± 0.001 B/op > FetchRequestBenchmark.testFetchRequestForReplica:·gc.churn.PS_Eden_Space 3 10 avgt 15 4417.032 ± 24.858 MB/sec > FetchRequestBenchmark.testFetchRequestForReplica:·gc.churn.PS_Eden_Space.norm 3 10 avgt 15 9832.358 ± 28.932 B/op > FetchRequestBenchmark.testFetchRequestForReplica:·gc.churn.PS_Survivor_Space 3 10 avgt 15 0.186 ± 0.015 MB/sec > FetchRequestBenchmark.testFetchRequestForReplica:·gc.churn.PS_Survivor_Space.norm 3 10 avgt 15 0.415 ± 0.033 B/op > FetchRequestBenchmark.testFetchRequestForReplica:·gc.count 3 10 avgt 15 2280.000 counts > FetchRequestBenchmark.testFetchRequestForReplica:·gc.time 3 10 avgt 15 1376.000 ms > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.alloc.rate 3 10 avgt 15 3256.172 ± 15.524 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.alloc.rate.norm 3 10 avgt 15 12384.000 ± 0.001 B/op > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.churn.PS_Eden_Space 3 10 avgt 15 3255.019 ± 21.484 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.churn.PS_Eden_Space.norm 3 10 avgt 15 12379.587 ± 49.161 B/op > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.churn.PS_Survivor_Space 3 10 avgt 15 0.122 ± 0.022 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.churn.PS_Survivor_Space.norm 3 10 avgt 15 0.462 ± 0.084 B/op > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.count 3 10 avgt 15 2054.000 counts > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.time 3 10 avgt 15 1389.000 ms > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.alloc.rate 3 10 avgt 15 3319.965 ± 53.427 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.alloc.rate.norm 3 10 avgt 15 13496.000 ± 0.001 B/op > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.churn.PS_Eden_Space 3 10 avgt 15 3320.125 ± 52.812 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.churn.PS_Eden_Space.norm 3 10 avgt 15 13496.813 ± 64.774 B/op > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.churn.PS_Survivor_Space 3 10 avgt 15 0.126 ± 0.021 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.churn.PS_Survivor_Space.norm 3 10 avgt 15 0.512 ± 0.085 B/op > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.count 3 10 avgt 15 2122.000 counts > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.time 3 10 avgt 15 1395.000 ms > ``` > > On trunk: > > ``` > Benchmark (partitionCount) (topicCount) Mode Cnt Score Error Units > FetchRequestBenchmark.testFetchRequestForConsumer 3 10 avgt 15 3.457 ± 0.016 ns/op > FetchRequestBenchmark.testFetchRequestForReplica 3 10 avgt 15 3.453 ± 0.035 ns/op > FetchRequestBenchmark.testSerializeFetchRequestForConsumer 3 10 avgt 15 13214.306 ± 61.158 ns/op > FetchRequestBenchmark.testSerializeFetchRequestForReplica 3 10 avgt 15 13147.870 ± 52.318 ns/op > > GC Profile (partitionCount) (topicCount) Mode Cnt Score Error Units > FetchRequestBenchmark.testFetchRequestForConsumer:·gc.alloc.rate 3 10 avgt 15 ≈ 10⁻⁴ MB/sec > FetchRequestBenchmark.testFetchRequestForConsumer:·gc.alloc.rate.norm 3 10 avgt 15 ≈ 10⁻⁶ B/op > FetchRequestBenchmark.testFetchRequestForConsumer:·gc.count 3 10 avgt 15 ≈ 0 counts > FetchRequestBenchmark.testFetchRequestForReplica:·gc.alloc.rate 3 10 avgt 15 ≈ 10⁻⁴ MB/sec > FetchRequestBenchmark.testFetchRequestForReplica:·gc.alloc.rate.norm 3 10 avgt 15 ≈ 10⁻⁶ B/op > FetchRequestBenchmark.testFetchRequestForReplica:·gc.count 3 10 avgt 15 ≈ 0 counts > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.alloc.rate 3 10 avgt 15 1795.576 ± 8.351 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.alloc.rate.norm 3 10 avgt 15 26136.002 ± 0.005 B/op > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.churn.PS_Eden_Space 3 10 avgt 15 1796.108 ± 11.527 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.churn.PS_Eden_Space.norm 3 10 avgt 15 26143.702 ± 100.832 B/op > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.churn.PS_Survivor_Space 3 10 avgt 15 0.163 ± 0.019 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.churn.PS_Survivor_Space.norm 3 10 avgt 15 2.366 ± 0.270 B/op > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.count 3 10 avgt 15 2134.000 counts > FetchRequestBenchmark.testSerializeFetchRequestForConsumer:·gc.time 3 10 avgt 15 1412.000 ms > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.alloc.rate 3 10 avgt 15 1804.695 ± 7.193 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.alloc.rate.norm 3 10 avgt 15 26136.002 ± 0.005 B/op > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.churn.PS_Eden_Space 3 10 avgt 15 1805.666 ± 7.990 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.churn.PS_Eden_Space.norm 3 10 avgt 15 26150.127 ± 86.455 B/op > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.churn.PS_Survivor_Space 3 10 avgt 15 0.166 ± 0.016 MB/sec > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.churn.PS_Survivor_Space.norm 3 10 avgt 15 2.406 ± 0.238 B/op > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.count 3 10 avgt 15 2097.000 counts > FetchRequestBenchmark.testSerializeFetchRequestForReplica:·gc.time 3 10 avgt 15 1395.000 ms > ``` Nice, so roughly for the replica fetch: ``` 2021.114 + 3691.157 = 5712.271 ns vs 0.035 + 13147.870 = 13147.905 ns ``` 57% reduction in CPU time. Alloc rate normalized comparison: ``` 9984.000 + 9832.000 = 19816 B/op vs 26136.002 B/op ``` 24.18% reduction in garbage generation. I think the garbage generation will massively improve once we can get rid of toPartitionDataMap later. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org