Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-120274059
:v:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled an
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/4984
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-120135865
As discussed, this implementation currently doesn't clean up shuffle files,
but this will be addressed separately by @tnachen in SPARK-8873. The dynamic
allocation si
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119924279
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119924187
[Test build #36921 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36921/console)
for PR 4984 at commit
[`39df8cd`](https://github.
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119906804
I've tested this on a small Mesos cluster, things look as before.
I uploaded a pre-packaged binary
[here](http://downloads.typesafe.com/tmp/spark-prs/spark-1.5.0-
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119899316
Build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does n
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119899281
[Test build #36914 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36914/console)
for PR 4984 at commit
[`4f30803`](https://github.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119889198
[Test build #36921 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36921/consoleFull)
for PR 4984 at commit
[`39df8cd`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119887954
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119888016
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user tnachen commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119881649
SPARK-8873 created for cleaning up shuffle files
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119870500
[Test build #36914 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36914/consoleFull)
for PR 4984 at commit
[`4f30803`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119869196
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119869176
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have thi
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119686334
In case others are following, @dragos @tnachen and I discussed offline
about a way forward. To merge this feature sooner, we decided to split this
patch into two sepa
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119664895
[Test build #36802 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36802/console)
for PR 4984 at commit
[`cf97666`](https://github.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119664995
Build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does n
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119635596
[Test build #36802 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36802/consoleFull)
for PR 4984 at commit
[`cf97666`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119634837
Build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this f
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119634811
Build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have thi
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119634626
But you can still have a look and give it a go in the meantime.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-119634567
I made the changes we agreed. I still have to rebase, that will take some
time. Conflicts are not trivial, I hope to have it tomorrow.
---
If your project is set up for
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33918664
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -285,4 +336,38 @@ private[spark] class CoarseM
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118338860
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118338785
[Test build #36496 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36496/console)
for PR 4984 at commit
[`9d88756`](https://github.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118311327
[Test build #36496 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36496/consoleFull)
for PR 4984 at commit
[`9d88756`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118310127
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118310171
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118257649
@andrewor14 the part about cleaning up files is indeed the part that
deserves some discussion. I'm very open to alternative solutions, and you
summarised the challenges v
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33843874
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -152,6 +164,15 @@ private[spark] class CoarseM
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118184097
@dragos I think this is pretty close. The one outstanding issue is still
who cleans up the shuffle files.
This patch proposes to have the driver send clean up
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118184103
Lastly, I suggested this before but I would actually recommend that we
split this patch into two separate issues. The first, which is Mesos dynamic
allocation (SPARK-
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33826106
--- Diff:
core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala ---
@@ -124,10 +124,16 @@ private[spark] class DiskBlockManager(blockManager:
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33825379
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -286,4 +324,40 @@ private[spark] class Coa
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33825396
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -111,12 +113,19 @@ private[spark] class BlockManager(
// Clie
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33825210
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -286,4 +324,40 @@ private[spark] class Coa
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33825030
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -264,17 +286,33 @@ private[spark] class Co
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33824866
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -169,14 +190,15 @@ private[spark] class Co
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33824449
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -152,6 +164,15 @@ private[spark] class Coa
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118135755
[Test build #36411 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36411/console)
for PR 4984 at commit
[`1673c8b`](https://github.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118135860
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118094831
I rebased on master and implemented the latest round of reviews. Please let
me know what you think, @andrewor14 /cc @tnachen.
I'm not sure how much more time I ca
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118094314
[Test build #36411 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/36411/consoleFull)
for PR 4984 at commit
[`1673c8b`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118093559
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-118093588
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33796897
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -59,9 +62,24 @@ private[spark] class CoarseMes
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33796575
--- Diff:
network/shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleClient.java
---
@@ -138,6 +139,26 @@ public void registerWithShuffleSe
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33795413
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -111,12 +113,19 @@ private[spark] class BlockManager(
// Client t
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33795284
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -286,4 +324,40 @@ private[spark] class CoarseM
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33793099
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -264,17 +286,33 @@ private[spark] class Coarse
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33792492
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -264,17 +286,33 @@ private[spark] class Coarse
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33792212
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -206,8 +228,7 @@ private[spark] class CoarseMe
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33791859
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -169,14 +190,15 @@ private[spark] class Coarse
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33790229
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -152,6 +164,15 @@ private[spark] class CoarseM
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33755309
--- Diff:
core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala ---
@@ -124,10 +124,16 @@ private[spark] class DiskBlockManager(blockManager:
Bl
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33726149
--- Diff:
core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala ---
@@ -124,10 +124,16 @@ private[spark] class DiskBlockManager(blockManager:
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33723361
--- Diff:
core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala ---
@@ -124,10 +124,16 @@ private[spark] class DiskBlockManager(blockManager:
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33719412
--- Diff:
network/shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleClient.java
---
@@ -138,6 +139,26 @@ public void registerWithShuffleSe
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-117812183
@andrewor14 thanks for you thorough review. I'll address your comments
tomorrow (my timezone).
---
If your project is set up for it, you can reply to this email and have
Github user dragos commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33718904
--- Diff:
core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala ---
@@ -124,10 +124,16 @@ private[spark] class DiskBlockManager(blockManager:
Bl
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-117802931
@dragos Thanks for all your work and patience! I took a pass over it and I
think the high level approach is reasonable. Most of my comments are relatively
minor, but
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33715021
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -1241,8 +1253,37 @@ private[spark] class BlockManager(
futureExec
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33714930
--- Diff:
core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala ---
@@ -124,10 +124,16 @@ private[spark] class DiskBlockManager(blockManager:
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33714772
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/mesos/CoarseMesosSchedulerBackendSuite.scala
---
@@ -0,0 +1,187 @@
+/*
+ * Licensed to th
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33714710
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/mesos/CoarseMesosSchedulerBackendSuite.scala
---
@@ -0,0 +1,187 @@
+/*
+ * Licensed to th
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33714625
--- Diff:
network/shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleClient.java
---
@@ -138,6 +139,26 @@ public void registerWithShuff
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33714143
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -1223,6 +1232,9 @@ private[spark] class BlockManager(
}
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33714016
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala
---
@@ -111,12 +113,19 @@ private[spark] class BlockManager(
// Clie
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33711743
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -286,4 +324,40 @@ private[spark] class Coa
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33710870
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -286,4 +324,40 @@ private[spark] class Coa
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33709366
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -264,17 +286,33 @@ private[spark] class Co
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33709279
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -264,17 +286,33 @@ private[spark] class Co
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33709017
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -264,17 +286,33 @@ private[spark] class Co
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33708031
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -264,17 +286,33 @@ private[spark] class Co
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33707899
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -142,8 +154,8 @@ private[spark] class Coar
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33707814
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -286,4 +324,40 @@ private[spark] class Coa
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33707677
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -264,17 +286,33 @@ private[spark] class Co
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33707474
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -206,8 +228,7 @@ private[spark] class Coar
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33706782
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -169,14 +190,15 @@ private[spark] class Co
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33706732
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -169,14 +190,15 @@ private[spark] class Co
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33706378
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -152,6 +164,15 @@ private[spark] class Coa
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33705759
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -59,9 +62,24 @@ private[spark] class Coars
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33705725
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -59,9 +62,24 @@ private[spark] class Coars
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33705634
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -59,9 +62,24 @@ private[spark] class Coars
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33705062
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -286,4 +324,40 @@ private[spark] class Coa
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33704850
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -59,9 +62,24 @@ private[spark] class Coars
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33704772
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -59,9 +62,24 @@ private[spark] class Coars
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33704710
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
---
@@ -59,9 +62,24 @@ private[spark] class Coars
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33642003
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1355,10 +1354,14 @@ class SparkContext(config: SparkConf) extends
Logging with Ex
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4984#discussion_r33641981
--- Diff:
core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala ---
@@ -268,8 +268,10 @@ private[spark] class ExecutorAllocationManager(
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-117351436
No worries. When I chatted to @tnachen there were a few misunderstandings
w.r.t to who deletes the shuffle files. Here's the story in a nutshell:
There are three
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-116926897
Hi @dragos I will take another pass over this within the next day or two.
Sorry for slipping.
---
If your project is set up for it, you can reply to this email and h
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-115552267
@andrewor14, I had a great chat with @tnachen yesterday and went through
the code of this PR. There are some subtle interactions. I'd be very happy to
jump on a hangout w
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-105263365
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-105263362
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-105263307
[Test build #33470 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33470/consoleFull)
for PR 4984 at commit
[`d50249a`](https://git
Github user dragos commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-105246967
For easier testing, I uploaded a [pre-built binary
here](http://downloads.typesafe.com/tmp/spark-prs/spark-1.4.0-SNAPSHOT-bin-dynalloc013-fdbbc32.tgz)
The gist is
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-105245996
[Test build #33470 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/33470/consoleFull)
for PR 4984 at commit
[`d50249a`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4984#issuecomment-105245461
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
1 - 100 of 176 matches
Mail list logo
