[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22847 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98386/ Test PASSed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98386/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98386 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98386/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-02 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22847 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98384/ Test FAILed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98384 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98384/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-01 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98384 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98384/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-01 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22847 @cloud-fan @rednaxelafx I missed that! Please help review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-01 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22847 did you address https://github.com/apache/spark/pull/22847#issuecomment-434836278 ? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-01 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22847 @cloud-fan @gatorsmile How about merging this PR first? And then we can dissuss those performance issue in other PR? 1. One PR to improve WideTableBenchmark #22823 WIP. 2. One PR to add more

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-11-01 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22847 I used the WideTableBenchmark to test this configuration. 4 scenarioes are tested, `2048` is always better than `1024`, overall it is also good and looks more safe to avoid hitting 8KB limitaion.

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/22847 Just in case people wonder, the following is the hack patch that I used for stress testing code splitting before this PR: ```diff ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22847 @rednaxelafx ah good point! It's hardcoded as 1024 too, and it's also doing method splitting. Let's apply the config there too. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/22847 Can / should we apply the same threshold conf to `Expression.reduceCodeSize()`? cc @yucai @cloud-fan @kiszk --- - To

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/22847 Using source code length will have to be a very coarse-grained, "fuzzy" heuristic. It's not meant to be accurate. So just pick some number that makes sense. 2048 might be a good enough

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98301/ Test PASSed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98301 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98301/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98298/ Test PASSed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98298 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98298/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98301/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98298 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98298/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22847 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98295/ Test FAILed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98295 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98295/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98295/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98288/ Test PASSed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98288 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98288/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-30 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22847 @cloud-fan @dongjoon-hyun @kiszk I just add a negative check, maybe we need another PR to figure better value later if it is not easy to decide now. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98288/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98190/ Test PASSed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98190 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98190/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98190 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98190/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-29 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22847 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98182/ Test FAILed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98182 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98182/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98182/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98132/ Test PASSed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98132 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98132/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98130/ Test PASSed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98130 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98130/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98129/ Test PASSed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98129 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98129/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98132/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98130 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98130/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98129 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98129/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98080/ Test PASSed. ---

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98080 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98080/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-26 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22847 @cloud-fan @dongjoon-hyun @gengliangwang Kindly help review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22847 **[Test build #98080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98080/testReport)** for PR 22847 at commit

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22847: [SPARK-25850][SQL] Make the split threshold for the code...

2018-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22847 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional