[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-218263708 Seems like @nraychaudhuri is busy, I'll take this PR and update it myself. We definitely need this to be merged as it's quite useful for testing. --- If your project

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user srowen commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-217506772 @nraychaudhuri can you update or close this PR then? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-212173748 @andrewor14 @nraychaudhuri @dragos Sorry I'm not suggesting we close this PR, we still need the flag since we want to be able to either failover automatically or not.

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-203657688 @nraychaudhuri can you close this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user dragos commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-203394614 Sounds good. Who can close this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-202640932 OK, let's not add a flag if it's a bug in Mesos. In the mean time before they fix it downstream we can use the workaround @tnachen suggested. --- If your project

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-192042971 I just found out that this is actually a bug in Mesos, where we cannot store a duration that's larger than int64_t. I filed a Mesos jira for this

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-191015322 I've tested this myself and is indeed now doing the correct behavior when not adding the flag in. I'll need to dig more, @nraychaudhuri have you tried this as well?

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user dragos commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-183929182 > On 14 feb. 2016, at 10:01, Timothy Chen wrote: > > @dragos you mean the framework no longer shows up in the UI? the console

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-183855606 @dragos you mean the framework no longer shows up in the UI? the console output doesn't seem to suggest it's gone. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-174975882 **[Test build #50096 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50096/consoleFull)** for PR 10701 at commit

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-174946300 **[Test build #50096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50096/consoleFull)** for PR 10701 at commit

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user dragos commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-174942139 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user dragos commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-174944748 I confirm that the framework deregisters from Mesos. However, I don't see the old behavior anymore, where the framework stays even after stopping it, with or without

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/10701#discussion_r49760002 --- Diff: core/src/main/scala/org/apache/spark/deploy/mesos/MesosClusterDispatcherArguments.scala --- @@ -97,6 +102,7 @@ private[mesos] class

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-171723201 Besides what @skyluc and my comments I think this patch LGTM. Have you tested this btw? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user tnachen commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-171723091 jenkins please test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-171725744 Yes. I have tested this and it seems to work. I will make the necessary changes --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user nraychaudhuri commented on a diff in the pull request: https://github.com/apache/spark/pull/10701#discussion_r49776110 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -318,7 +319,7 @@ private[spark] class

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user skyluc commented on a diff in the pull request: https://github.com/apache/spark/pull/10701#discussion_r49733021 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -318,7 +319,7 @@ private[spark] class

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-171302887 @dragos @tnachen @skyluc Could you please take a look at this one? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

GitHub user nraychaudhuri opened a pull request: https://github.com/apache/spark/pull/10701 [SPARK-7831][Mesos] Added flag to shutdown driver when mesos dispatch… Fix for SPARK-7831 You can merge this pull request into a Git repository by running: $ git pull

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-170668469 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your