[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-26 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20372 please see https://spark.apache.org/contributing.html open a JIRA and update this PR? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-26 Thread vgankidi
Github user vgankidi commented on the issue: https://github.com/apache/spark/pull/20372 I agree with @ash211. Applications shouldn't rely on the order of the files within a partition. This optimization looks good to me. ---

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-24 Thread ash211
Github user ash211 commented on the issue: https://github.com/apache/spark/pull/20372 Tagging folks who have touched this code recently: @vgankidi @ericl @davies This seems to provide a more compact packing in every scenario, which should improve execution times. One risk

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20372 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20372 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86591/ Test PASSed. ---

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20372 **[Test build #86591 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86591/testReport)** for PR 20372 at commit

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20372 **[Test build #86591 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86591/testReport)** for PR 20372 at commit

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-24 Thread glentakahashi
Github user glentakahashi commented on the issue: https://github.com/apache/spark/pull/20372 The large non-splittable files is already tested by

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread ash211
Github user ash211 commented on the issue: https://github.com/apache/spark/pull/20372 Please fix the scala style checks -- ``` Running Scala style checks Scalastyle checks failed at following

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20372 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20372 **[Test build #86567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86567/testReport)** for PR 20372 at commit

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20372 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86567/ Test FAILed. ---

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20372 **[Test build #86567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86567/testReport)** for PR 20372 at commit

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread ash211
Github user ash211 commented on the issue: https://github.com/apache/spark/pull/20372 Jenkins, this is ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20372 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20372 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional