GitHub user ueshin opened a pull request:
https://github.com/apache/spark/pull/825
[SPARK-1880] [SQL] Eliminate unnecessary job executions.
There are unnecessary job executions in `BroadcastNestedLoopJoin`.
When `Innner` or `LeftOuter` join, preparation of `rightOuterMatches` for
`RightOuter` or `FullOuter` join is not neccessary.
And when `RightOuter` or `FullOuter`, it should use not `count` and then
`reduce` but `fold`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ueshin/apache-spark issues/SPARK-1880
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/825.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #825
----
commit 7bff4d978fa0cb64ef6d4e935aaa8c394a6fa4ae
Author: Takuya UESHIN <ues...@happy-camper.st>
Date: 2014-05-19T07:47:39Z
Move preparing of rightOuterMatches to after checking joinType not to
execute unnecessary job when joinType is Inner or LeftOuter.
commit 63cacbfd41090185de64c27586cacc5265c3c0d7
Author: Takuya UESHIN <ues...@happy-camper.st>
Date: 2014-05-19T07:52:33Z
Use fold instead of count and then reduce to eliminate unnecessary job
executions.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
