Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-52655499
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18849/consoleFull)
for PR 2001 at commit
[`caa9c50`](https://github.com/ap
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-52663412
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/18849/consoleFull)
for PR 2001 at commit
[`caa9c50`](https://github.com/a
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-53881604
[QA tests have
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19475/consoleFull)
for PR 2001 at commit
[`0029d2b`](https://github.com/ap
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-53889964
Is this still needed after https://github.com/apache/spark/pull/1180? It
added a call to sc.stop in the YarnClientSchedulerBackend when the application
is finished/fai
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-53889418
[QA tests have
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/19475/consoleFull)
for PR 2001 at commit
[`0029d2b`](https://github.com/a
Github user sarutak commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-53890682
Thank you for your comment, @tgravescs .
I think, it's better that sc.stop is invoked in shutdown hook as well as
ApplicationMaster, running Driver program in the
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/2001#discussion_r16906649
--- Diff:
yarn/common/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientClusterScheduler.scala
---
@@ -36,4 +36,18 @@ private[spark] class YarnClien
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-53911884
sc.stop in client mode isn't being called on the applicationMaster in
YarnClientSchedulerBackend, its called from the driver on the client/gateway
box.
The shu
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-61994202
ping, @sarutak can you answer previous question or close this out please?
---
If your project is set up for it, you can reply to this email and have your
reply appear o
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-62041396
[Test build #23013 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23013/consoleFull)
for PR 2001 at commit
[`c560c6a`](https://githu
Github user sarutak commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-62041692
@tgravescs I meant that in YARN Client mode, user apps runs on the client,
and SparkContext should be stopped by users explicitly.
In other hand, in YARN Cluster mode,
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-62046346
So the yarnClientSchedulerBackend already has an asyncMonitor to check is
the yarn side got shutdown and if it does it does an sc.stop. If the yarn
client mode driver
Github user sarutak commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-62048632
Ah, I see, now monitor thread call "sc.stop" at the end of an application.
Actually, because sc.stop was not called in an user app, cleanups for
submitted jars on HDFS
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-62053848
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-62053837
[Test build #23013 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23013/consoleFull)
for PR 2001 at commit
[`c560c6a`](https://gith
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/2001#issuecomment-63078694
@sarutak can we close this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user sarutak closed the pull request at:
https://github.com/apache/spark/pull/2001
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is ena
17 matches
Mail list logo
