vamossagar12 commented on pull request #10798:
URL: https://github.com/apache/kafka/pull/10798#issuecomment-864174533
@cadonna , @guozhangwang I ran some jmh benchmarks on this. I ran the 2
tests with G1GC and prof gc=> 1 for putAll and the other one for range query.
Here are the results for putAll:
```
Original AK codebase
Benchmark
Mode Cnt Score Error Units
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance
thrpt 15 60.489 ? 2.154 ops/s
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.alloc.rate
thrpt 15 158.731 ? 5.651 MB/sec
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.alloc.rate.norm
thrpt 15 2889264.628 ? 26.474 B/op
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.PS_Eden_Space
thrpt 15 160.811 ? 9.073 MB/sec
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.PS_Eden_Space.norm
thrpt 15 2927507.100 ? 141145.743 B/op
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.PS_Survivor_Space
thrpt 15 0.251 ? 0.145 MB/sec
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.PS_Survivor_Space.norm
thrpt 15 4536.878 ? 2486.338 B/op
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.count
thrpt 15 161.000 counts
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.time
```
```
DirectByteBuffer
Benchmark
Mode Cnt Score Error Units
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance
thrpt 15 97.908 ? 1.820 ops/s
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.alloc.rate
thrpt 15 256.946 ? 4.777 MB/sec
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.alloc.rate.norm
thrpt 15 2889553.977 ? 4.331 B/op
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Eden_Space
thrpt 15 256.202 ? 51.550 MB/sec
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Eden_Space.norm
thrpt 15 2879662.678 ? 563793.832 B/op
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Old_Gen
thrpt 15 0.030 ? 0.039 MB/sec
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Old_Gen.norm
thrpt 15 336.245 ? 437.382 B/op
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Survivor_Space
thrpt 15 0.025 ? 0.105 MB/sec
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.churn.G1_Survivor_Space.norm
thrpt 15 285.618 ? 1182.589 B/op
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.count
thrpt 15 33.000 counts
StreamsPersistentStoreBenchmark.testPersistentPutAllPerformance:?gc.time
thrpt 15 308.000 ms
```
And here are the results for range:
```
Original AK code
Benchmark
Mode Cnt Score Error Units
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance
thrpt 15 117.454 ? 2.620 ops/s
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.alloc.rate
thrpt 15 290.118 ? 6.471 MB/sec
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.alloc.rate.norm
thrpt 15 2719700.009 ? 32.687 B/op
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Eden_Space
thrpt 15 295.066 ? 64.171 MB/sec
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Eden_Space.norm
thrpt 15 2768668.326 ? 613598.308 B/op
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Old_Gen
thrpt 15 0.002 ? 0.005 MB/sec
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Old_Gen.norm
thrpt 15 16.068 ? 47.296 B/op
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Survivor_Space
thrpt 15 0.051 ? 0.210 MB/sec
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Survivor_Space.norm
thrpt 15 466.034 ? 1929.591 B/op
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.count
thrpt 15 38.000 counts
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.time
thrpt 15 431.000 ms
```
```
DirectBytBuffer
Benchmark
Mode Cnt Score Error Units
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance
thrpt 15 114.871 ? 4.080 ops/s
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.alloc.rate
thrpt 15 284.127 ? 10.090 MB/sec
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.alloc.rate.norm
thrpt 15 2723428.297 ? 33.503 B/op
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Eden_Space
thrpt 15 286.413 ? 64.054 MB/sec
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Eden_Space.norm
thrpt 15 2743440.651 ? 593666.103 B/op
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Old_Gen
thrpt 15 0.007 ? 0.024 MB/sec
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Old_Gen.norm
thrpt 15 67.053 ? 224.086 B/op
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Survivor_Space
thrpt 15 0.025 ? 0.105 MB/sec
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.churn.G1_Survivor_Space.norm
thrpt 15 239.812 ? 992.929 B/op
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.count
thrpt 15 37.000 counts
StreamsPersistentStoreBenchmark.testPersistentRangeQueryPerformance:?gc.time
thrpt 15 515.000 ms
```
Throughput wise, there was a jump of 30+ ops/s for the putAll case. range
performed slightly worse in the original codebase. I can change the benchmark
mode to AverageTime for range query.
One thing I am noticing is that the DirectByteBuffer tests are continuously
clocking higher gc allocation /gc count and gc time. You can find the benchmark
related code here: https://github.com/apache/kafka/pull/10842/files.
One of my guesses is that for this to actually show some gains, the
application shouldn't be making use of byte[] and allocate directly. That
doesn't seem possible with the way the State stores are exposed via the DSL
because finally, everything is serialised into a byte[]. What do you guys think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
