[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-30 Thread maropu
Github user maropu closed the pull request at: https://github.com/apache/spark/pull/9478 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-30 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160842618 @andrewor14 Okay and thanks. Also, can you close SPARK-6521? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-30 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160807059 Can you close this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-30 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160806983 @maropu thanks for running the benchmarks. It seems that the gains are not really significant enough to warrant all the complexity this patch adds. This is to a certa

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160117761 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160117763 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160117706 **[Test build #46806 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46806/consoleFull)** for PR 9478 at commit [`303abcd`](https://git

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-26 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160057949 @andrewor14 ISTM that this pr has a little effect on performance even in case of many partitions involved in shuffle. Test settings: - 10 test runs except

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160056489 **[Test build #46806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46806/consoleFull)** for PR 9478 at commit [`303abcd`](https://gith

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r46021085 --- Diff: core/src/test/scala/org/apache/spark/storage/ShuffleBlockFetcherIteratorSuite.scala --- @@ -27,16 +27,24 @@ import org.mockito.Matchers.{any, eq =>

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r46020888 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -47,12 +47,35 @@ private[spark] class IndexShuffleBlockResolver(c

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r46020880 --- Diff: core/src/main/scala/org/apache/spark/network/BlockDataManager.scala --- @@ -30,6 +30,13 @@ trait BlockDataManager { def getBlockData(blockId:

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r46020431 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -58,6 +58,17 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r46020157 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -47,12 +47,35 @@ private[spark] class IndexShuffleBlockResolver(c

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160034859 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160034858 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160034840 **[Test build #46794 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46794/consoleFull)** for PR 9478 at commit [`ba94687`](https://git

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-160023236 **[Test build #46794 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46794/consoleFull)** for PR 9478 at commit [`ba94687`](https://gith

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-159839408 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-159839409 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-159839358 **[Test build #46741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46741/consoleFull)** for PR 9478 at commit [`4ca2d72`](https://git

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-159806939 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-159806935 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-159806689 **[Test build #46739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46739/consoleFull)** for PR 9478 at commit [`1653691`](https://git

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-159794374 **[Test build #46741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46741/consoleFull)** for PR 9478 at commit [`4ca2d72`](https://gith

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-159793574 **[Test build #46739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46739/consoleFull)** for PR 9478 at commit [`1653691`](https://gith

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-25 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-159781902 @andrewor14 I tried quick benchmarks though, I saw little difference, so I'm taking various tests on it. I'll get the result in a day. --- If your project is set up

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-19 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-158169482 @maropu any updates? Did you have a chance to do the benchmarks? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-155672972 **[Test build #45591 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45591/consoleFull)** for PR 9478 at commit [`6b8f7bf`](https://git

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-155673033 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-155673034 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-155655819 **[Test build #45591 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45591/consoleFull)** for PR 9478 at commit [`6b8f7bf`](https://gith

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-155655274 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-155655286 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-155652857 Thx for your reviews. I'll fix it and also do benchmarks . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-155615442 By the way, have you done some benchmarking to measure how much this actually saves? I wonder if the gains are actually all that significant. Maybe it actually matter

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-155614988 @maropu Thanks for taking this over. I took a pass and it seems that the abstractions can be simplified a little. Please also fix the style issues that I pointed out.

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44488520 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -299,7 +310,11 @@ class BlockManagerMasterEndpoint(

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44488463 --- Diff: core/src/test/scala/org/apache/spark/storage/ShuffleBlockFetcherIteratorSuite.scala --- @@ -27,16 +27,24 @@ import org.mockito.Matchers.{any, eq

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44488357 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -233,19 +250,28 @@ final class ShuffleBlockFetcherIterator

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44488406 --- Diff: core/src/test/scala/org/apache/spark/storage/DiskBlockManagerSuite.scala --- @@ -17,21 +17,21 @@ package org.apache.spark.storage

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44488245 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -58,6 +58,17 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44488213 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerId.scala --- @@ -65,6 +65,10 @@ class BlockManagerId private ( executorId ==

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44488201 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -302,6 +303,12 @@ private[spark] class BlockManager( } }

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44488161 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -47,12 +47,35 @@ private[spark] class IndexShuffleBlockResolv

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44488058 --- Diff: core/src/main/scala/org/apache/spark/shuffle/FileShuffleBlockResolver.scala --- @@ -107,8 +107,13 @@ private[spark] class FileShuffleBlockResolve

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44487716 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -240,6 +242,15 @@ class BlockManagerMasterEndpoint(

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44487704 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala --- @@ -19,17 +19,16 @@ package org.apache.spark.storage

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44487656 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -302,6 +303,12 @@ private[spark] class BlockManager( } }

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44487614 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -47,12 +47,35 @@ private[spark] class IndexShuffleBlockResolv

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44487535 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -47,12 +47,35 @@ private[spark] class IndexShuffleBlockResolv

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44487505 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -47,12 +47,35 @@ private[spark] class IndexShuffleBlockResolv

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44487407 --- Diff: core/src/main/scala/org/apache/spark/network/BlockDataManager.scala --- @@ -30,6 +30,13 @@ trait BlockDataManager { def getBlockData(block

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154270938 @andrewor14 Could you review this and give some suggestions? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154265383 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154265380 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154265172 **[Test build #45170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45170/consoleFull)** for PR 9478 at commit [`cebc895`](https://git

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154261775 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154261778 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154261668 **[Test build #45166 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45166/consoleFull)** for PR 9478 at commit [`211f0b9`](https://git

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154257146 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154257149 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154257041 **[Test build #45163 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45163/consoleFull)** for PR 9478 at commit [`8d0a2f0`](https://git

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154240184 **[Test build #45170 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45170/consoleFull)** for PR 9478 at commit [`cebc895`](https://gith

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154238607 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154238591 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154234656 **[Test build #45166 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45166/consoleFull)** for PR 9478 at commit [`211f0b9`](https://gith

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154234475 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154234463 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154233268 **[Test build #45163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45163/consoleFull)** for PR 9478 at commit [`8d0a2f0`](https://gith

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154232445 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154232458 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154231492 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread maropu
Github user maropu commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154231203 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44070440 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManagerMaster.scala --- @@ -17,14 +17,14 @@ package org.apache.spark.storage -

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44070245 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -95,10 +118,12 @@ private[spark] class IndexShuffleBlockResolver(

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/9478#discussion_r44070286 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -38,10 +30,16 @@ import org.apache.spark.network.netty.SparkTransportConf

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154130913 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154130916 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154130815 **[Test build #45117 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45117/consoleFull)** for PR 9478 at commit [`8d0a2f0`](https://git

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154098687 **[Test build #45117 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45117/consoleFull)** for PR 9478 at commit [`8d0a2f0`](https://gith

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154095960 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154095987 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154009898 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154009895 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-154009796 **[Test build #45106 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45106/consoleFull)** for PR 9478 at commit [`5c00a5a`](https://git

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-153982455 **[Test build #45106 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45106/consoleFull)** for PR 9478 at commit [`5c00a5a`](https://gith

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-153982287 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-153982306 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-153934112 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-153934111 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-153934042 **[Test build #45071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45071/consoleFull)** for PR 9478 at commit [`849eaa7`](https://git

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-04 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-153917270 **[Test build #45071 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45071/consoleFull)** for PR 9478 at commit [`849eaa7`](https://gith

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-153916097 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9478#issuecomment-153916074 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-6521][Core] Bypass unnecessary network ...

2015-11-04 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/9478 [SPARK-6521][Core] Bypass unnecessary network access if block managers share an identical host Refactored #5178 and added unit tests. You can merge this pull request into a Git repository by running