Github user ueshin commented on the pull request:
https://github.com/apache/spark/pull/283#issuecomment-39417683
@rxin Thank you for your reply.
There are some case to use merge join for optimization:
1. If data to be joined are already sorted by join keys, merge join would
be done more efficiently than hash join. In my test case, both algorithms were
almost same speed, but merge join was scalable.
2. Merge join for sorted data by the same keys would be pipelined (each
output can be produced immediately for arrived input tuples) even if N-way join
(N>2). Hash join blocks due to building a hash-table before output are produced.
I think it is useful for users to choose ways to optimize their processing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
