GitHub user mridulm opened a pull request:
https://github.com/apache/spark/pull/1609
[SPARK-2532] WIP Consolidated shuffle fixes
Status of the PR
- [X] Cherry pick and merge changes from internal branch to spark master
- [X] Remove WIP comments and 2G branch references.
-
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50267547
QA results for PR 1609:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds no public classesFor more
information see test
ouptut:https://amplab.c
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50267540
QA tests have started for PR 1609. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17242/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50274622
QA results for PR 1609:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds no public classesFor more
information see test
ouptut:https://amplab.c
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50274618
QA tests have started for PR 1609. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17243/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50277674
QA tests have started for PR 1609. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17244/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50278406
QA results for PR 1609:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds no public classesFor more
information see test
ouptut:https://amplab.c
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15442565
--- Diff:
core/src/test/scala/org/apache/spark/storage/DiskBlockObjectWriterSuite.scala
---
@@ -0,0 +1,296 @@
+/*
+ * Licensed to the Apache Software
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50289340
@adav @andrewor14 would be good if you two take a look at this when it's
merging correctly.
---
If your project is set up for it, you can reply to this email and have you
Github user mridulm commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15442779
--- Diff:
core/src/test/scala/org/apache/spark/storage/DiskBlockObjectWriterSuite.scala
---
@@ -0,0 +1,296 @@
+/*
+ * Licensed to the Apache Software
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50290037
QA tests have started for PR 1609. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17256/consoleFull
---
If
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50290580
Did you mean @aarondav?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have t
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50291239
QA results for PR 1609:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds no public classesFor more
information see test
ouptut:https://amplab.c
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15446910
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter(
pri
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15446921
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -176,10 +406,17 @@ private[spark] class DiskBlockObjectWriter(
if
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447384
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter(
pri
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447402
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter(
pri
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447397
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter(
pri
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447401
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter(
pri
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447436
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter(
pri
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447430
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter(
pri
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447449
--- Diff:
core/src/main/scala/org/apache/spark/storage/BlockObjectWriter.scala ---
@@ -107,68 +109,296 @@ private[spark] class DiskBlockObjectWriter(
pri
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447510
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -935,15 +941,22 @@ private[spark] object Utils extends Logging {
* Currently, this
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447541
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -353,26 +368,53 @@ class ExternalAppendOnlyMap[K, V, C](
Github user witgo commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15447535
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala
---
@@ -353,26 +368,53 @@ class ExternalAppendOnlyMap[K, V, C](
Github user aarondav commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50305416
@mridulm Thanks for submitting this! I would like to dig a little deeper
into understanding the specific issues you found, in order to understand the
solutions you have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50306070
QA tests have started for PR 1609. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17276/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50306230
QA results for PR 1609:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds no public classesFor more
information see test
ouptut:https://amplab.c
Github user mridulm commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15448803
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -935,15 +941,22 @@ private[spark] object Utils extends Logging {
* Currently, th
Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50306648
Accidental close, apologies !
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not hav
Github user mridulm closed the pull request at:
https://github.com/apache/spark/pull/1609
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is ena
Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50306633
@witgo I did not understand the space issue : stylecheck seems to run fine.
Regarding the actual issues : the JIRA lists some of them - unfortunately
it is not ex
GitHub user mridulm reopened a pull request:
https://github.com/apache/spark/pull/1609
[SPARK-2532] WIP Consolidated shuffle fixes
Status of the PR
- [X] Cherry pick and merge changes from internal branch to spark master
- [X] Remove WIP comments and 2G branch references.
Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50307155
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have th
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50307247
QA tests have started for PR 1609. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17277/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50307411
QA results for PR 1609:- This patch FAILED unit tests.- This patch
merges cleanly- This patch adds no public classesFor more
information see test
ouptut:https://amplab.c
Github user mridulm commented on a diff in the pull request:
https://github.com/apache/spark/pull/1609#discussion_r15449433
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -947,6 +958,34 @@ private[spark] object Utils extends Logging {
}
fals
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50308229
QA tests have started for PR 1609. This patch merges cleanly. View
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17279/consoleFull
---
If
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50311354
QA results for PR 1609:- This patch PASSES unit tests.- This patch
merges cleanly- This patch adds no public classesFor more
information see test
ouptut:https://amplab.c
Github user aarondav commented on the pull request:
https://github.com/apache/spark/pull/1609#issuecomment-50363390
@mridulm Thanks for the clarifications! Those make sense and are some
tricky edge cases. I will begin reviewing the code as soon as possible.
---
If your project is set
40 matches
Mail list logo
