gortiz commented on PR #12354: URL: https://github.com/apache/pinot/pull/12354#issuecomment-1964228389
I've created another benchmark where the null distribution is not constant but random while keeping a single null value per given number of rows (aka interval). Also in order to mess up the compilation I've changed the benchmark to do not always run with or without null handling but changing the calls in the same JVM by providing a null percentage. Every trial will toss a coin to check whether nulls will be used or not. In order to have enough samples, I've increased the number of trials and reduced the trial length. I've also sorted the params to first group resutls by null interval, then percentage and finally implementation Results are: ``` Benchmark (_aNullInterval) (_nullHandlingEnabledPerCent) (_zImpl) Mode Cnt Score Error Units BenchmarkSumAggregation3.test 2 100 normal thrpt 50 111.205 ± 1.330 ops/ms BenchmarkSumAggregation3.test 2 100 foldDouble thrpt 50 31.637 ± 0.562 ops/ms BenchmarkSumAggregation3.test 2 50 normal thrpt 50 125.432 ± 11.036 ops/ms BenchmarkSumAggregation3.test 2 50 foldDouble thrpt 50 60.079 ± 17.668 ops/ms BenchmarkSumAggregation3.test 2 0 normal thrpt 50 142.717 ± 1.249 ops/ms BenchmarkSumAggregation3.test 2 0 foldDouble thrpt 50 140.213 ± 0.543 ops/ms BenchmarkSumAggregation3.test 4 100 normal thrpt 50 4.304 ± 0.104 ops/ms BenchmarkSumAggregation3.test 4 100 foldDouble thrpt 50 52.099 ± 0.310 ops/ms BenchmarkSumAggregation3.test 4 50 normal thrpt 50 81.429 ± 34.278 ops/ms BenchmarkSumAggregation3.test 4 50 foldDouble thrpt 50 108.721 ± 19.590 ops/ms BenchmarkSumAggregation3.test 4 0 normal thrpt 50 144.474 ± 0.680 ops/ms BenchmarkSumAggregation3.test 4 0 foldDouble thrpt 50 143.460 ± 1.093 ops/ms BenchmarkSumAggregation3.test 8 100 normal thrpt 50 4.801 ± 0.018 ops/ms BenchmarkSumAggregation3.test 8 100 foldDouble thrpt 50 85.509 ± 0.574 ops/ms BenchmarkSumAggregation3.test 8 50 normal thrpt 50 81.977 ± 34.200 ops/ms BenchmarkSumAggregation3.test 8 50 foldDouble thrpt 50 134.188 ± 3.026 ops/ms BenchmarkSumAggregation3.test 8 0 normal thrpt 50 141.820 ± 0.859 ops/ms BenchmarkSumAggregation3.test 8 0 foldDouble thrpt 50 144.061 ± 0.633 ops/ms BenchmarkSumAggregation3.test 16 100 normal thrpt 50 6.259 ± 0.044 ops/ms BenchmarkSumAggregation3.test 16 100 foldDouble thrpt 50 119.675 ± 0.945 ops/ms BenchmarkSumAggregation3.test 16 50 normal thrpt 50 84.264 ± 34.529 ops/ms BenchmarkSumAggregation3.test 16 50 foldDouble thrpt 50 158.657 ± 7.734 ops/ms BenchmarkSumAggregation3.test 16 0 normal thrpt 50 145.898 ± 0.665 ops/ms BenchmarkSumAggregation3.test 16 0 foldDouble thrpt 50 144.811 ± 0.740 ops/ms BenchmarkSumAggregation3.test 32 100 normal thrpt 50 7.463 ± 0.034 ops/ms BenchmarkSumAggregation3.test 32 100 foldDouble thrpt 50 148.919 ± 1.352 ops/ms BenchmarkSumAggregation3.test 32 50 normal thrpt 50 84.360 ± 34.162 ops/ms BenchmarkSumAggregation3.test 32 50 foldDouble thrpt 50 167.707 ± 12.256 ops/ms BenchmarkSumAggregation3.test 32 0 normal thrpt 50 146.683 ± 0.838 ops/ms BenchmarkSumAggregation3.test 32 0 foldDouble thrpt 50 144.886 ± 0.741 ops/ms BenchmarkSumAggregation3.test 64 100 normal thrpt 50 7.122 ± 0.125 ops/ms BenchmarkSumAggregation3.test 64 100 foldDouble thrpt 50 160.228 ± 0.353 ops/ms BenchmarkSumAggregation3.test 64 50 normal thrpt 50 82.165 ± 33.204 ops/ms BenchmarkSumAggregation3.test 64 50 foldDouble thrpt 50 161.608 ± 12.202 ops/ms BenchmarkSumAggregation3.test 64 0 normal thrpt 50 146.049 ± 0.583 ops/ms BenchmarkSumAggregation3.test 64 0 foldDouble thrpt 50 140.151 ± 0.255 ops/ms BenchmarkSumAggregation3.test 128 100 normal thrpt 50 10.146 ± 0.124 ops/ms BenchmarkSumAggregation3.test 128 100 foldDouble thrpt 50 175.641 ± 1.888 ops/ms BenchmarkSumAggregation3.test 128 50 normal thrpt 50 84.023 ± 32.846 ops/ms BenchmarkSumAggregation3.test 128 50 foldDouble thrpt 50 153.464 ± 7.716 ops/ms BenchmarkSumAggregation3.test 128 0 normal thrpt 50 145.157 ± 0.685 ops/ms BenchmarkSumAggregation3.test 128 0 foldDouble thrpt 50 144.998 ± 0.738 ops/ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
