[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2016-04-12 Thread yaooqinn
Github user yaooqinn commented on the pull request: https://github.com/apache/spark/pull/3438#issuecomment-208769628 Is there any progress about this mechanism study? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-09-01 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3438#issuecomment-136918372 @jerryshao unfortunately the shuffle code has changed significantly since this patch was last updated and it will unlikely be merged. Would you mind closing this

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-09-01 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/3438#issuecomment-136939951 OK, thanks a lot. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-09-01 Thread jerryshao
Github user jerryshao closed the pull request at: https://github.com/apache/spark/pull/3438 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-05-13 Thread jeanlyn
Github user jeanlyn commented on a diff in the pull request: https://github.com/apache/spark/pull/3438#discussion_r30295775 --- Diff: core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleReader.scala --- @@ -0,0 +1,337 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3438#issuecomment-92231768 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/3438#discussion_r28222570 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -726,8 +725,8 @@ class RDDSuite extends FunSuite with SharedSparkContext {

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3438#issuecomment-92231763 [Test build #30143 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30143/consoleFull) for PR 3438 at commit

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/3438#discussion_r28221127 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -726,8 +725,8 @@ class RDDSuite extends FunSuite with SharedSparkContext {

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-13 Thread chenghao-intel
Github user chenghao-intel commented on a diff in the pull request: https://github.com/apache/spark/pull/3438#discussion_r28223056 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -726,8 +725,8 @@ class RDDSuite extends FunSuite with SharedSparkContext {

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-13 Thread adrian-wang
Github user adrian-wang commented on a diff in the pull request: https://github.com/apache/spark/pull/3438#discussion_r28220021 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -726,8 +725,8 @@ class RDDSuite extends FunSuite with SharedSparkContext {

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/3438#discussion_r28223321 --- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala --- @@ -726,8 +725,8 @@ class RDDSuite extends FunSuite with SharedSparkContext {

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-12 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/3438#discussion_r28210241 --- Diff: core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleManager.scala --- @@ -48,7 +47,7 @@ private[spark] class SortShuffleManager(conf:

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-12 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3438#issuecomment-92214906 [Test build #30143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/30143/consoleFull) for PR 3438 at commit

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-12 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/3438#discussion_r28210315 --- Diff: core/src/main/scala/org/apache/spark/util/collection/TieredDiskMerger.scala --- @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-10 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/3438#discussion_r28191449 --- Diff: core/src/main/scala/org/apache/spark/util/collection/TieredDiskMerger.scala --- @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-04-10 Thread lianhuiwang
Github user lianhuiwang commented on a diff in the pull request: https://github.com/apache/spark/pull/3438#discussion_r28191464 --- Diff: core/src/main/scala/org/apache/spark/util/collection/TieredDiskMerger.scala --- @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-02-22 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/3438#issuecomment-75448758 Hi @andrewor14 , I've rebased this code against the latest master, would you please help to review this code, thanks a lot. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-02-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3438#issuecomment-75421923 [Test build #27831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27831/consoleFull) for PR 3438 at commit

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-02-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3438#issuecomment-75423996 [Test build #27831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27831/consoleFull) for PR 3438 at commit

[GitHub] spark pull request: [SPARK-2926][Shuffle]Add MR style sort-merge s...

2015-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3438#issuecomment-75423999 Test PASSed. Refer to this link for build results (access rights to CI server needed):