[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-20 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220672478 @mengxr Disable this test in master and 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-20 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220652565 @davies @rxin It seems that this PR caused OOO in master builds. ~~~ *** RUN ABORTED *** java.lang.OutOfMemoryError: Java heap space at org.apac

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13182 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-19 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220416710 Merging this into master and 2.0, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220413819 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220413814 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220413524 **[Test build #58877 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58877/consoleFull)** for PR 13182 at commit [`3ab5c13`](https://g

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220392230 **[Test build #58877 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58877/consoleFull)** for PR 13182 at commit [`3ab5c13`](https://gi

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13182#discussion_r63822575 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -410,9 +410,10 @@ private[execution] final class LongToUns

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13182#discussion_r63822450 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -410,9 +410,10 @@ private[execution] final class LongToUns

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13182#discussion_r63822349 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -410,9 +410,10 @@ private[execution] final class LongToUns

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/13182#discussion_r63815948 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala --- @@ -72,9 +72,18 @@ case class BroadcastExchangeExec

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread sameeragarwal
Github user sameeragarwal commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220204711 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/13182#discussion_r63810701 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala --- @@ -72,9 +72,18 @@ case class BroadcastExcha

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220198917 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220198916 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220198799 **[Test build #58824 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58824/consoleFull)** for PR 13182 at commit [`8714022`](https://g

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220186165 **[Test build #58824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58824/consoleFull)** for PR 13182 at commit [`8714022`](https://gi

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220183818 **[Test build #58820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58820/consoleFull)** for PR 13182 at commit [`1b5c8e1`](https://g

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220183824 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220183821 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220183697 cc @sameeragarwal --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220183556 **[Test build #58820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58820/consoleFull)** for PR 13182 at commit [`1b5c8e1`](https://gi

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/13182#discussion_r63798134 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala --- @@ -72,9 +72,18 @@ case class BroadcastExchangeExec(

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220181917 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220181911 **[Test build #58819 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58819/consoleFull)** for PR 13182 at commit [`07d64c1`](https://g

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220181919 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13182#issuecomment-220181647 **[Test build #58819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58819/consoleFull)** for PR 13182 at commit [`07d64c1`](https://gi

[GitHub] spark pull request: [SPARK-15390] fix broadcast with 100 millions ...

2016-05-18 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/13182 [SPARK-15390] fix broadcast with 100 millions rows ## What changes were proposed in this pull request? When broadcast a table with more than 100 millions rows (should not ideally), the size