[jira] [Commented] (SPARK-7831) Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
[ https://issues.apache.org/jira/browse/SPARK-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092554#comment-15092554 ] Apache Spark commented on SPARK-7831: - User 'nraychaudhuri' has created a pull request for this issue: https://github.com/apache/spark/pull/10701 > Mesos dispatcher doesn't deregister as a framework from Mesos when stopped > -- > > Key: SPARK-7831 > URL: https://issues.apache.org/jira/browse/SPARK-7831 > Project: Spark > Issue Type: Bug > Components: Mesos >Affects Versions: 1.4.0 > Environment: Spark 1.4.0-rc1, Mesos 0.2.2 (compiled from source) >Reporter: Luc Bourlier > > To run Spark on Mesos in cluster mode, a Spark Mesos dispatcher has to be > running. > It is launched using {{sbin/start-mesos-dispatcher.sh}}. The Mesos dispatcher > registers as a framework in the Mesos cluster. > After using {{sbin/stop-mesos-dispatcher.sh}} to stop the dispatcher, the > application is correctly terminated locally, but the framework is still > listed as {{active}} in the Mesos dashboard. > I would expect the framework to be de-registered when the dispatcher is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7831) Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
[ https://issues.apache.org/jira/browse/SPARK-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083142#comment-15083142 ] Nilanjan Raychaudhuri commented on SPARK-7831: -- I am working on a possible fix for this. I will submit a pull request soon > Mesos dispatcher doesn't deregister as a framework from Mesos when stopped > -- > > Key: SPARK-7831 > URL: https://issues.apache.org/jira/browse/SPARK-7831 > Project: Spark > Issue Type: Bug > Components: Mesos >Affects Versions: 1.4.0 > Environment: Spark 1.4.0-rc1, Mesos 0.2.2 (compiled from source) >Reporter: Luc Bourlier > > To run Spark on Mesos in cluster mode, a Spark Mesos dispatcher has to be > running. > It is launched using {{sbin/start-mesos-dispatcher.sh}}. The Mesos dispatcher > registers as a framework in the Mesos cluster. > After using {{sbin/stop-mesos-dispatcher.sh}} to stop the dispatcher, the > application is correctly terminated locally, but the framework is still > listed as {{active}} in the Mesos dashboard. > I would expect the framework to be de-registered when the dispatcher is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7831) Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
[ https://issues.apache.org/jira/browse/SPARK-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009070#comment-15009070 ] Zapi Dev commented on SPARK-7831: - Well and what's the state of this issue, do you have any plan to implement it? > Mesos dispatcher doesn't deregister as a framework from Mesos when stopped > -- > > Key: SPARK-7831 > URL: https://issues.apache.org/jira/browse/SPARK-7831 > Project: Spark > Issue Type: Bug > Components: Mesos >Affects Versions: 1.4.0 > Environment: Spark 1.4.0-rc1, Mesos 0.2.2 (compiled from source) >Reporter: Luc Bourlier > > To run Spark on Mesos in cluster mode, a Spark Mesos dispatcher has to be > running. > It is launched using {{sbin/start-mesos-dispatcher.sh}}. The Mesos dispatcher > registers as a framework in the Mesos cluster. > After using {{sbin/stop-mesos-dispatcher.sh}} to stop the dispatcher, the > application is correctly terminated locally, but the framework is still > listed as {{active}} in the Mesos dashboard. > I would expect the framework to be de-registered when the dispatcher is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7831) Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
[ https://issues.apache.org/jira/browse/SPARK-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14559168#comment-14559168 ] Luc Bourlier commented on SPARK-7831: - Ok. Yes it makes sense. It is not because the dispatcher died that the the jobs in progress should be terminated. I think it would be good to have an option on the shutdown script, to be able to terminate the Mesos framework. Also, is there already a way to reconnect a dispatcher to a running framework? > Mesos dispatcher doesn't deregister as a framework from Mesos when stopped > -- > > Key: SPARK-7831 > URL: https://issues.apache.org/jira/browse/SPARK-7831 > Project: Spark > Issue Type: Bug > Components: Mesos >Affects Versions: 1.4.0 > Environment: Spark 1.4.0-rc1, Mesos 0.2.2 (compiled from source) >Reporter: Luc Bourlier > > To run Spark on Mesos in cluster mode, a Spark Mesos dispatcher has to be > running. > It is launched using {{sbin/start-mesos-dispatcher.sh}}. The Mesos dispatcher > registers as a framework in the Mesos cluster. > After using {{sbin/stop-mesos-dispatcher.sh}} to stop the dispatcher, the > application is correctly terminated locally, but the framework is still > listed as {{active}} in the Mesos dashboard. > I would expect the framework to be de-registered when the dispatcher is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-7831) Mesos dispatcher doesn't deregister as a framework from Mesos when stopped
[ https://issues.apache.org/jira/browse/SPARK-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14556382#comment-14556382 ] Timothy Chen commented on SPARK-7831: - Hi Luc, actually that's the default behavior that I put into the dispatcher framework, since I expect the dispatcher to be a long running framework and even when it goes away it's expected to be resumed and all the tasks should be running. To really shut down the framework a user can ask Mesos to terminate the framework via the shutdown REST api call. I think what we should do here is to make a configuration flag to trigger this behavior or not, and default I think should be what its currently like. What you think? > Mesos dispatcher doesn't deregister as a framework from Mesos when stopped > -- > > Key: SPARK-7831 > URL: https://issues.apache.org/jira/browse/SPARK-7831 > Project: Spark > Issue Type: Bug > Components: Mesos >Affects Versions: 1.4.0 > Environment: Spark 1.4.0-rc1, Mesos 0.2.2 (compiled from source) >Reporter: Luc Bourlier > > To run Spark on Mesos in cluster mode, a Spark Mesos dispatcher has to be > running. > It is launched using {{sbin/start-mesos-dispatcher.sh}}. The Mesos dispatcher > registers as a framework in the Mesos cluster. > After using {{sbin/stop-mesos-dispatcher.sh}} to stop the dispatcher, the > application is correctly terminated locally, but the framework is still > listed as {{active}} in the Mesos dashboard. > I would expect the framework to be de-registered when the dispatcher is > stopped. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org