[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/20667 Thanks @cloud-fan @jiangxb1987 @kiszk @Ngone51 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20667 thanks, merging to master/2.3/2.2! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20667 LGTM if we need caching. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20667 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87761/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20667 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20667 **[Test build #87761 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87761/testReport)** for PR 20667 at commit [`bf79f4d`](https://github.com/apache/spark/commit/bf79f4d5c83c364c7f1fc05f158753d282409330). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20667 **[Test build #87744 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87744/testReport)** for PR 20667 at commit [`3379899`](https://github.com/apache/spark/commit/337989945b0757dfc6a069315c4e7828afe77d00). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20667 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20667 **[Test build #87761 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87761/testReport)** for PR 20667 at commit [`bf79f4d`](https://github.com/apache/spark/commit/bf79f4d5c83c364c7f1fc05f158753d282409330). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20667 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20667 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87744/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20667 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20667 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87748/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20667 **[Test build #87748 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87748/testReport)** for PR 20667 at commit [`bf79f4d`](https://github.com/apache/spark/commit/bf79f4d5c83c364c7f1fc05f158753d282409330). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20667 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20667 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87746/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20667 **[Test build #87746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87746/testReport)** for PR 20667 at commit [`3379899`](https://github.com/apache/spark/commit/337989945b0757dfc6a069315c4e7828afe77d00). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/20667 Hi, @jiangxb1987 , thanks for your kindly explanation. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20667 In case the same `BlockManagerId` being created multiple times, this cache will ensure we always use the first one that is created, which make it possible for the rest `BlockManagerId` instances being recycled shortly. The downside is we have to persist all the distinct `BlockManagerId` created. Since the code is added long times ago, and it's actually hard to examine the performance with/without the cache, we'd like to keep it for now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/20667 Hi, @caneGuy , sorry for my previous comment as I mixed up ```BlockId``` with ```BlockManagerId```, and leave some wrong comments. And thanks for your reply. Back to now, I have the same question with @cloud-fan , > Why we need this cache? though, we have a better cache way(guava cache) now. My confusions: - It is weird that we need to create a ```BlockManagerId ``` before we get a same one from the cache. - And on executor side, when ```BlockManagerId ``` registered to master and return with an updated ```BlockManagerId ``` , the new ```BlockManagerId ``` does not be updated to ```blockManagerIdCache```. So, it seems executor side's ```BlockManagerId``` has little relevance with ```blockManagerIdCache```. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20667 **[Test build #87748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87748/testReport)** for PR 20667 at commit [`bf79f4d`](https://github.com/apache/spark/commit/bf79f4d5c83c364c7f1fc05f158753d282409330). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20667 **[Test build #87746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87746/testReport)** for PR 20667 at commit [`3379899`](https://github.com/apache/spark/commit/337989945b0757dfc6a069315c4e7828afe77d00). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20667 **[Test build #87744 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87744/testReport)** for PR 20667 at commit [`3379899`](https://github.com/apache/spark/commit/337989945b0757dfc6a069315c4e7828afe77d00). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20667 add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20667 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20667 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20667: [SPARK-23508][CORE] Fix BlockmanagerId in case blockMana...
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/20667 Update @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
