liming30 commented on PR #833: URL: https://github.com/apache/incubator-paimon/pull/833#issuecomment-1498718896
@JingsongLi hi, I changed the key in the [benchmark](https://github.com/liming30/paimon-benchmark/blob/master/src/main/java/com/bytedance/MergeReaderBenchmark.java) to a `String` type with a length of 128 bytes. Now `loserTree` has a significant performance improvement compared to `minHeap`(30%~50%). With the increase of the number of `RecordReader` and the amount of data, the performance improvement is more obvious. ``` Benchmark (readersNum) (recordNum) Mode Cnt Score Error Units MergeReaderBenchmark.group:loserTree 2 1000 avgt 10 0.495 ± 0.282 ms/op MergeReaderBenchmark.group:minHeap 2 1000 avgt 10 0.777 ± 0.462 ms/op MergeReaderBenchmark.group:loserTree 2 10000 avgt 10 5.461 ± 1.191 ms/op MergeReaderBenchmark.group:minHeap 2 10000 avgt 10 8.405 ± 1.735 ms/op MergeReaderBenchmark.group:loserTree 2 100000 avgt 10 60.266 ± 4.047 ms/op MergeReaderBenchmark.group:minHeap 2 100000 avgt 10 91.507 ± 5.479 ms/op MergeReaderBenchmark.group:loserTree 5 1000 avgt 10 1.752 ± 0.239 ms/op MergeReaderBenchmark.group:minHeap 5 1000 avgt 10 2.502 ± 0.329 ms/op MergeReaderBenchmark.group:loserTree 5 10000 avgt 10 19.769 ± 3.273 ms/op MergeReaderBenchmark.group:minHeap 5 10000 avgt 10 28.785 ± 4.636 ms/op MergeReaderBenchmark.group:loserTree 5 100000 avgt 10 216.575 ± 33.923 ms/op MergeReaderBenchmark.group:minHeap 5 100000 avgt 10 378.799 ± 58.567 ms/op MergeReaderBenchmark.group:loserTree 10 1000 avgt 10 5.360 ± 0.293 ms/op MergeReaderBenchmark.group:minHeap 10 1000 avgt 10 8.919 ± 0.470 ms/op MergeReaderBenchmark.group:loserTree 10 10000 avgt 10 54.107 ± 5.948 ms/op MergeReaderBenchmark.group:minHeap 10 10000 avgt 10 87.113 ± 9.001 ms/op MergeReaderBenchmark.group:loserTree 10 100000 avgt 10 495.890 ± 20.231 ms/op MergeReaderBenchmark.group:minHeap 10 100000 avgt 10 792.621 ± 34.431 ms/op MergeReaderBenchmark.group:loserTree 20 1000 avgt 10 18.246 ± 9.235 ms/op MergeReaderBenchmark.group:minHeap 20 1000 avgt 10 29.491 ± 15.529 ms/op MergeReaderBenchmark.group:loserTree 20 10000 avgt 10 252.389 ± 63.124 ms/op MergeReaderBenchmark.group:minHeap 20 10000 avgt 10 410.268 ± 106.181 ms/op MergeReaderBenchmark.group:loserTree 20 100000 avgt 10 1797.648 ± 598.206 ms/op MergeReaderBenchmark.group:minHeap 20 100000 avgt 10 2802.025 ± 742.304 ms/op MergeReaderBenchmark.group:loserTree 50 1000 avgt 10 33.583 ± 1.122 ms/op MergeReaderBenchmark.group:minHeap 50 1000 avgt 10 64.716 ± 2.202 ms/op MergeReaderBenchmark.group:loserTree 50 10000 avgt 10 351.078 ± 36.385 ms/op MergeReaderBenchmark.group:minHeap 50 10000 avgt 10 707.821 ± 77.199 ms/op ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
