[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19458 thanks, merging to master/2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19458 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19458 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83048/ Test PASSed. ---

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19458 **[Test build #83048 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83048/testReport)** for PR 19458 at commit

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19458 **[Test build #83048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83048/testReport)** for PR 19458 at commit

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19458 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19458 **[Test build #83043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83043/testReport)** for PR 19458 at commit

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19458 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83043/ Test FAILed. ---

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19458 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19458 **[Test build #83043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83043/testReport)** for PR 19458 at commit

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19458 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread superbobry
Github user superbobry commented on the issue: https://github.com/apache/spark/pull/19458 @jerryshao the error looks a bit unrelated. ``` sbt.ForkMain$ForkError: org.scalatest.exceptions.TestFailedException: null ``` The suite passes locally for me. ---

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19458 **[Test build #83034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83034/testReport)** for PR 19458 at commit

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19458 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19458 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83034/ Test FAILed. ---

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19458 There's a UT failure (https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83014/testReport/junit/org.apache.spark.storage/BlockIdSuite/test_bad_deserialization/). @superbobry

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19458 **[Test build #83034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83034/testReport)** for PR 19458 at commit

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19458 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19458 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83014/ Test FAILed. ---

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19458 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19458 **[Test build #83014 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83014/testReport)** for PR 19458 at commit

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19458 **[Test build #83014 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83014/testReport)** for PR 19458 at commit

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19458 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-19 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19458 ping @superbobry --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-12 Thread superbobry
Github user superbobry commented on the issue: https://github.com/apache/spark/pull/19458 On second thought, the documentation of `DiskBlockManager.getAllFiles` states that it returns all files in the directories managed by `DiskBlockManager`. Filtering anything would violate this

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-12 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19458 Yes, I agree in any case it should not throw an exception. But in this PR you filtered out temp shuffle/local blocks, do you think this block is valid or not, are they blocks? So I'd

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-12 Thread superbobry
Github user superbobry commented on the issue: https://github.com/apache/spark/pull/19458 > Instead of filtering out temp blocks, why not adding parsing rule for `TempLocalBlockId` and `TempShuffleBlockId`? Because this fixes the issue out 2 out of 3 possible temp files. The

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19458 Instead of filtering out temp blocks, why not adding parsing rule for `TempLocalBlockId` and `TempShuffleBlockId`? That could also solve the problem. Since `DiskBlockManager#getAllFiles` doesn't

[GitHub] spark issue #19458: [SPARK-22227][CORE] DiskBlockManager.getAllBlocks now to...

2017-10-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19458 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional