Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50857406
Thank you @marmbrus I've updated the code as suggested.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50857544
QA tests have started for PR 1147. This patch merges cleanly. brView
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17663/consoleFull
---
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50863187
QA results for PR 1147:br- This patch PASSES unit tests.br- This patch
merges cleanlybr- This patch adds the following public classes
(experimental):brcase class
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15710615
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -137,6 +137,185 @@ trait HashJoin {
}
/**
+ *
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15710690
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -137,6 +137,185 @@ trait HashJoin {
}
/**
+ *
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15710803
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -137,6 +137,185 @@ trait HashJoin {
}
/**
+ *
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15710817
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -137,6 +137,185 @@ trait HashJoin {
}
/**
+ *
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15710919
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -137,6 +137,185 @@ trait HashJoin {
}
/**
+ *
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50918377
I'm going to go ahead and merge this so we can have it in 1.1. It would be
great if you could address the final readability concerns in a follow up PR.
Thanks
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/1147
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15725454
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -137,6 +137,185 @@ trait HashJoin {
}
/**
+ *
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50948662
Thank you very much @marmbrus , I will create follow up for the
improvement. :)
---
If your project is set up for it, you can reply to this email and have your
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15725487
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -137,6 +137,185 @@ trait HashJoin {
}
/**
+ *
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50716934
QA tests have started for PR 1147. This patch merges cleanly. brView
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17560/consoleFull
---
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50725881
QA results for PR 1147:br- This patch PASSES unit tests.br- This patch
merges cleanlybr- This patch adds the following public classes
(experimental):brtrait
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15675477
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15675495
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15675665
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15676544
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15676602
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15676609
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15677804
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15677845
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15678005
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15678029
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15678572
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15678780
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -138,6 +267,67 @@ trait HashJoin {
/**
* ::
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15678840
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15678978
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679079
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala ---
@@ -106,38 +183,131 @@ class JoinSuite extends QueryTest {
}
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679103
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala ---
@@ -17,15 +17,42 @@
package org.apache.spark.sql
+import
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679444
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679474
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679535
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679602
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679650
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679744
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala ---
@@ -17,15 +17,42 @@
package org.apache.spark.sql
+import
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679804
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679832
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50841534
I also really like how isolated this change is and the inclusion of the
benchmark :)
---
If your project is set up for it, you can reply to this email and have your
Github user marmbrus commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50841494
Thanks a lot for working on this! I agree that it would be great to merge
this in before 1.1. I'm a little worried about how much memory this is going
to require.
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679983
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15679993
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15680608
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user chenghao-intel commented on a diff in the pull request:
https://github.com/apache/spark/pull/1147#discussion_r15680598
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala ---
@@ -37,6 +37,135 @@ case object BuildLeft extends BuildSide
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50711862
QA tests have started for PR 1147. This patch merges cleanly. brView
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17551/consoleFull
---
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50712207
I've updated the code by rebasing the latest master, this PR will greatly
improve the outer join performance for big tables. See the local benchmark in
the
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-50715224
QA results for PR 1147:br- This patch FAILED unit tests.br- This patch
merges cleanlybr- This patch adds the following public classes
(experimental):brtrait
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-47307221
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-47307224
All automated tests passed.
Refer to this link for build results:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16189/
---
If your
Github user chenghao-intel commented on the pull request:
https://github.com/apache/spark/pull/1147#issuecomment-47308393
Thanks @marmbrus merged some of the dependent PRs. I've also updated both
code and description for this PR accordingly, some of the issues were listed in
the PR
51 matches
Mail list logo
