vamossagar12 commented on pull request #10798: URL: https://github.com/apache/kafka/pull/10798#issuecomment-864174533
@cadonna , @guozhangwang I ran some jmh benchmarks on this. I ran the 2 tests with G1GC and prof gc=> 1 for putAll and the other one for range query. Here are the results for putAll: ``` Original AK codebase Benchmark Mode Cnt Score Error Units StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance thrpt 15 60.489 ? 2.154 ops/s StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.alloc.rate thrpt 15 158.731 ? 5.651 MB/sec StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.alloc.rate.norm thrpt 15 2889264.628 ? 26.474 B/op StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.PS_Eden_Space thrpt 15 160.811 ? 9.073 MB/sec StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.PS_Eden_Space.norm thrpt 15 2927507.100 ? 141145.743 B/op StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.PS_Survivor_Space thrpt 15 0.251 ? 0.145 MB/sec StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.PS_Survivor_Space.norm thrpt 15 4536.878 ? 2486.338 B/op StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.count thrpt 15 161.000 counts StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.time ``` ``` DirectByteBuffer Benchmark Mode Cnt Score Error Units StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance thrpt 15 97.908 ? 1.820 ops/s StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.alloc.rate thrpt 15 256.946 ? 4.777 MB/sec StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.alloc.rate.norm thrpt 15 2889553.977 ? 4.331 B/op StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Eden_Space thrpt 15 256.202 ? 51.550 MB/sec StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Eden_Space.norm thrpt 15 2879662.678 ? 563793.832 B/op StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Old_Gen thrpt 15 0.030 ? 0.039 MB/sec StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Old_Gen.norm thrpt 15 336.245 ? 437.382 B/op StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Survivor_Space thrpt 15 0.025 ? 0.105 MB/sec StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Survivor_Space.norm thrpt 15 285.618 ? 1182.589 B/op StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.count thrpt 15 33.000 counts StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.time thrpt 15 308.000 ms ``` And here are the results for range: ``` Original AK code Benchmark Mode Cnt Score Error Units StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance thrpt 15 117.454 ? 2.620 ops/s StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.alloc.rate thrpt 15 290.118 ? 6.471 MB/sec StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.alloc.rate.norm thrpt 15 2719700.009 ? 32.687 B/op StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Eden_Space thrpt 15 295.066 ? 64.171 MB/sec StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Eden_Space.norm thrpt 15 2768668.326 ? 613598.308 B/op StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Old_Gen thrpt 15 0.002 ? 0.005 MB/sec StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Old_Gen.norm thrpt 15 16.068 ? 47.296 B/op StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Survivor_Space thrpt 15 0.051 ? 0.210 MB/sec StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Survivor_Space.norm thrpt 15 466.034 ? 1929.591 B/op StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.count thrpt 15 38.000 counts StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.time thrpt 15 431.000 ms ``` ``` DirectBytBuffer Benchmark Mode Cnt Score Error Units StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance thrpt 15 114.871 ? 4.080 ops/s StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.alloc.rate thrpt 15 284.127 ? 10.090 MB/sec StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.alloc.rate.norm thrpt 15 2723428.297 ? 33.503 B/op StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Eden_Space thrpt 15 286.413 ? 64.054 MB/sec StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Eden_Space.norm thrpt 15 2743440.651 ? 593666.103 B/op StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Old_Gen thrpt 15 0.007 ? 0.024 MB/sec StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Old_Gen.norm thrpt 15 67.053 ? 224.086 B/op StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Survivor_Space thrpt 15 0.025 ? 0.105 MB/sec StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Survivor_Space.norm thrpt 15 239.812 ? 992.929 B/op StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.count thrpt 15 37.000 counts StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.time thrpt 15 515.000 ms ``` Throughput wise, there was a jump of 30+ ops/s for the putAll case. range performed slightly worse in the original codebase. I can change the benchmark mode to AverageTime for range query. One thing I am noticing is that the DirectByteBuffer tests are continuously clocking higher gc allocation /gc count and gc time. You can find the benchmark related code here: https://github.com/apache/kafka/pull/10842/files. One of my guesses is that for this to actually show some gains, the application shouldn't be making use of byte[] and allocate directly. That doesn't seem possible with the way the State stores are exposed via the DSL because finally, everything is serialised into a byte[]. What do you guys think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org