Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/2696
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2696#issuecomment-60474144
Thanks for all of the review feedback! I'm going to merge this now so that
we can begin using it in other PRs and features, such as progress bars in the
Spark shells.
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/2696#issuecomment-60460535
LGTM. Thanks for adding the tests!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2696#issuecomment-60459934
[Test build #22171 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22171/consoleFull)
for PR 2696 at commit
[`e6aa78d`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2696#issuecomment-60459939
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2696#issuecomment-60455677
I added a couple of simple tests. One subtlety here is the fact that this
API is sort of "eventually consistent" in the sense that we might know about a
job before we
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2696#issuecomment-60455158
[Test build #22171 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22171/consoleFull)
for PR 2696 at commit
[`e6aa78d`](https://githu
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19363966
--- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
@@ -116,4 +94,60 @@ private[spark] object SparkUI {
def getUIPort(conf: SparkConf): I
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19363621
--- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
@@ -116,4 +94,60 @@ private[spark] object SparkUI {
def getUIPort(conf: SparkConf
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19362398
--- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
@@ -116,4 +94,60 @@ private[spark] object SparkUI {
def getUIPort(conf: SparkConf)
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19362366
--- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
@@ -116,4 +94,60 @@ private[spark] object SparkUI {
def getUIPort(conf: SparkConf): I
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19361954
--- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
@@ -116,4 +94,60 @@ private[spark] object SparkUI {
def getUIPort(conf: SparkConf)
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19361746
--- Diff: core/src/main/java/org/apache/spark/JobExecutionStatus.java ---
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) u
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19361659
--- Diff: core/src/main/java/org/apache/spark/SparkStageInfo.java ---
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/2696#issuecomment-60136663
@JoshRosen lgtm; I don't really like the listener changes in `SparkUI`
(aside from the job one which is necessary), but can't really think of a much
better approach (other
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19235882
--- Diff: core/src/main/scala/org/apache/spark/ui/SparkUI.scala ---
@@ -116,4 +94,60 @@ private[spark] object SparkUI {
def getUIPort(conf: SparkConf): I
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19235533
--- Diff: core/src/main/scala/org/apache/spark/SparkStatusAPI.scala ---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19235559
--- Diff: core/src/main/scala/org/apache/spark/SparkStatusAPI.scala ---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19235394
--- Diff: core/src/main/scala/org/apache/spark/SparkStatusAPI.scala ---
@@ -0,0 +1,142 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19235280
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -51,6 +50,7 @@ import
org.apache.spark.scheduler.cluster.mesos.{CoarseMesosSchedulerBa
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19235115
--- Diff: core/src/main/java/org/apache/spark/JobExecutionStatus.java ---
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) unde
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2696#issuecomment-60019531
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22013/consoleFull)
for PR 2696 at commit
[`2707f98`](https://github.com/a
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2696#issuecomment-60019534
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2696#issuecomment-60013190
Alright, I've updated this based on the latest round of feedback. I've
removed the WIP, since I think this is enough for a basic first-cut at this
API; we can always e
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2696#discussion_r19185681
--- Diff:
core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala ---
@@ -40,17 +40,25 @@ class JobProgressListener(conf: SparkConf) extends
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2696#issuecomment-60013181
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22013/consoleFull)
for PR 2696 at commit
[`2707f98`](https://github.com/ap
26 matches
Mail list logo
