Github user ggevay commented on the pull request:
https://github.com/apache/flink/pull/1517#issuecomment-186244959
@fhueske:
So far, I have only run benchmarks on my work laptop and home laptop.
I have used the newly created `ReducePerformance`, with varying the
`(number of elements / number of different keys)` ratio, and also the memory
usage (between 250 MB - 6 GB). The hash-based strategy was always faster, up to
3 times, and mostly by a factor of 1.5 - 2.5.
I have also used `ReduceHashTableTest.testWithIntPair`. To make this a
benchmark test, the lines that deal with the `reference` hash table for
checking the correct output should be commented out, and the time measurement
commented in. (This only tests the hash table, not end-to-end; and it can't be
directly compared with the sort based reduce.)
I have also used `HashTablePerformanceComparison` to compare with the other
hash tables. This has the disadvantage that the workload here is not
characteristic of a reduce (eg. no processRecordWithReduce calls, because the
other hash tables don't have that).
I have also benchmarked using my game theory stuff (which originally
prompted me to start working on this), and the hash-based combiner was faster
here as well.
> Did you check the combine rate (input / output ratio) compared to the
sort-based strategy?
I didn't measure it directly, but `ReducePerformance` shows the difference
nicely when the number of input elements is large enough that the sort-based
strategy has to emit often, but the number of different keys is small enough
that the hash-based strategy can fit all the keys in memory. In this case, the
speedup of the final reduce phase is striking.
> Have you checked heap memory consumption / GC activity compared to the
sort-based strategy?
I haven't actually checked it, but it should be the same or better as with
the sort-based strategy:
- enabled object reuse: no allocations at all
- disabled object reuse: one allocation per input element, and one
allocation per reduce step
The sort-based strategy might additionally have allocations inside maybe
the sorter and/or MergeIterator.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
