[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19394 Thanks for reviewing, @gatorsmile! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19394 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82558/ Test PASSed. ---

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82558 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82558/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82558 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82558/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82555/ Test PASSed. ---

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82555 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82555/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82556/ Test FAILed. ---

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82556 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82556/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19394 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82556 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82556/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82555 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82555/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82532/ Test PASSed. ---

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82532/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82530/ Test FAILed. ---

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82530/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82532/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82530/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19394 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19394 Anyone have a clue what the python error could be? It doesn't look related. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82524/ Test FAILed. ---

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82524/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82525/ Test FAILed. ---

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82525/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82525/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82524/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-05 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/19394 What's the other value? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-05 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19394 Here's the error message: TestFailedException: 347.5272 was not greater than 1000 --- - To unsubscribe, e-mail:

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-05 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/19394 Not sure - maybe print the chi-value of the test and see if they make sense. If they do, we can change the threshold. --- - To

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82486/ Test FAILed. ---

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82486 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82486/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-05 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19394 Yes, we've been running this in production for a few weeks now. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19394 Is the fix being used in the Netflix production environment? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19394 This is a pretty cool fix --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-05 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19394 @rxin, any idea why this would fail ConfigBehaviorSuite? I don't think the failure is related because that test doesn't use a broadcast join. Should I rebase on master? ---

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82486/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-10-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19394 cc @liufengdb --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-09-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19394 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82338/ Test FAILed. ---

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-09-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82338/testReport)** for PR 19394 at commit

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-09-29 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19394 Ideally, this would also use a TaskMemoryManager so the driver can spill results to disk instead of dying with an OOM. Is there any plan to add a memory manager for the driver? ---

[GitHub] spark issue #19394: [SPARK-22170][SQL] Reduce memory consumption in broadcas...

2017-09-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19394 **[Test build #82338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82338/testReport)** for PR 19394 at commit