Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/3711
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user ksakellis commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-69804537
@pwendell @nitin2goyal I had previously thought of adding it to the
DAGScheduler - this would avoid the duplicated code between the
CoarseGrainedSchedulerBackend and
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22777939
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala
---
@@ -344,12 +354,20 @@ private[spark] class
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-69538284
LGTM - just had a minor comment that can also be addressed on merge.
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-69543920
[Test build #25406 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25406/consoleFull)
for PR 3711 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-69543929
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-69538514
[Test build #25406 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25406/consoleFull)
for PR 3711 at commit
Github user nitin2goyal commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22779931
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -213,6 +216,7 @@ class
Github user tnachen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22832694
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala
---
@@ -344,12 +354,20 @@ private[spark] class
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-69623494
Actually, why not add these in the DAG Scheduler per @nitin2goyal's
suggestion.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22811962
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -213,6 +216,7 @@ class
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-69216849
Ok, latest changes look fine to me. Any other comments @pwendell @JoshRosen?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user ksakellis commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22681441
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala
---
@@ -344,12 +354,20 @@ private[spark] class
Github user ksakellis commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22680915
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/EventLoggingListenerSuite.scala
---
@@ -160,7 +160,7 @@ class EventLoggingListenerSuite extends
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22679008
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/EventLoggingListenerSuite.scala
---
@@ -160,7 +160,7 @@ class EventLoggingListenerSuite extends
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22679665
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala
---
@@ -344,12 +354,20 @@ private[spark] class
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22633647
--- Diff: core/src/main/java/org/apache/spark/SparkListenerAdapter.java ---
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22624112
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/ExecutorDetails.scala ---
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22624324
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala ---
@@ -84,6 +85,14 @@ case class SparkListenerBlockManagerRemoved(time:
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22624199
--- Diff: core/src/main/java/org/apache/spark/SparkListenerAdapter.java ---
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22624632
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -362,6 +378,10 @@ private[spark] object JsonProtocol {
(Disk Size -
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22624645
--- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala ---
@@ -745,6 +780,11 @@ private[spark] object JsonProtocol {
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-69110079
Left a few relatively minor comments. Otherwise this LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user ksakellis commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22624775
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala ---
@@ -84,6 +85,14 @@ case class SparkListenerBlockManagerRemoved(time: Long,
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22627136
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala ---
@@ -84,6 +85,14 @@ case class SparkListenerBlockManagerRemoved(time:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-69120909
[Test build #25188 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25188/consoleFull)
for PR 3711 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-69126629
[Test build #25188 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25188/consoleFull)
for PR 3711 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-69126633
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68787824
This looks like a legitimate test failure. Ther AMPLab webserver is having
some issues today, so here's a different link to reach the same test result:
Github user ksakellis commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68788875
Yes it is. Not sure why I changed the #of cores between the two commits in
the unit test - weird. Anyways. it has been fixed.
---
If your project is set up for it,
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68789356
[Test build #25065 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25065/consoleFull)
for PR 3711 at commit
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22495998
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/ExecutorDetails.scala ---
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68798860
[Test build #25065 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25065/consoleFull)
for PR 3711 at commit
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68796368
I had some minor comments around naming, but overall this looks good.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68798867
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22495242
--- Diff: core/src/main/java/org/apache/spark/SparkListenerAdapter.java ---
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22495870
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -106,6 +108,7 @@ class
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68787217
[Test build #25060 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25060/consoleFull)
for PR 3711 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68787229
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68790202
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68790195
[Test build #25061 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25061/consoleFull)
for PR 3711 at commit
Github user ksakellis commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68778757
@JoshRosen I actually couldn't make ExecutorInfo (now ExecutorDetails) a
trait because the json protocol stuff requires a concrete object to deserialize
events. The
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68779123
[Test build #25060 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25060/consoleFull)
for PR 3711 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-68781542
[Test build #25061 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25061/consoleFull)
for PR 3711 at commit
Github user ksakellis commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22366699
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/ExecutorInfo.scala ---
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software
Github user ksakellis commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22366788
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala ---
@@ -84,6 +86,14 @@ case class SparkListenerBlockManagerRemoved(time: Long,
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22231207
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala ---
@@ -19,6 +19,8 @@ package org.apache.spark.scheduler
import
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22231341
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala ---
@@ -84,6 +86,14 @@ case class SparkListenerBlockManagerRemoved(time: Long,
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22231382
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala ---
@@ -109,7 +119,8 @@ private[spark] case object SparkListenerShutdown
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22231531
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/ExecutorInfo.scala ---
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22237608
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala ---
@@ -84,6 +86,14 @@ case class SparkListenerBlockManagerRemoved(time: Long,
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22237639
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/SparkListenerWithClusterSuite.scala
---
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22237848
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/ExecutorInfo.scala ---
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software
Github user ksakellis commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22195578
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/ExecutorInfo.scala ---
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3711#discussion_r22139204
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/ExecutorInfo.scala ---
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software
GitHub user ksakellis opened a pull request:
https://github.com/apache/spark/pull/3711
[SPARK-4857] [CORE] Adds Executor membership events to SparkListener
Adds onExecutorAdded and onExecutorRemoved events to the SparkListener.
This will allow a client to get notified when an
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-67130479
[Test build #24494 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24494/consoleFull)
for PR 3711 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-67130482
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-67130472
[Test build #24494 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24494/consoleFull)
for PR 3711 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-67133566
[Test build #24496 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24496/consoleFull)
for PR 3711 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-67143159
[Test build #24496 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24496/consoleFull)
for PR 3711 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3711#issuecomment-67143165
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
62 matches
Mail list logo
