[
https://issues.apache.org/jira/browse/MESOS-3059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nitin updated MESOS-3059:
-------------------------
Description:
This is well understood that - changing the attributes dynamically is not safe
without a restart because slave itself may not know which old framework tasks
are running on it that were dependent on previous attributes.
However, total restart makes lot of other history to delete. We need to ensure
a dynamic attribute changes with a soft restart.
It will be good to expose a rest endpoint either at slave or mesos-master which
directly changes the state in zookeeper.
USE-CASE
We use slave attributes/roles to direct the framework scheduling to use
specific slave as per it's requirements. Mesos scheduler only creates the offer
on the basis of some resources.
In our use case, we have some categorization of our spark frameworks or jobs
with framework(like marathon) based on multiple factors. We want job or
frameworks belonging to one category be running into their specific cluster of
resources. We want to dynamically manage the slaves into these logical
sub-clusters.
Since number of jobs that will be submitted or when it will be submitted is
very dynamic, it make sense to be able to dynamically assign roles or
attributes to slaves. It is not possible to gauge the requirements at time of
cluster provisioning. Static role or attribute assignment leads to sub-optimal
use of the cluster.
was:
This is well understood that - changing the attributes dynamically is not safe
without a restart because slave itself may not know which old framework tasks
are running on it that were dependent on previous attributes.
However, total restart makes lot of other history to delete. We need to ensure
a dynamic attribute changes with a soft restart.
It will be good to expose a rest endpoint either at slave or mesos-master which
directly changes the state in zookeeper.
USE-CASE
We use slave attributes/roles to direct the framework scheduling to use
specific slave as per it's requirements. Mesos scheduler only creates the offer
on the basis of some resources.
In our use case, we have some categorization of our spark frameworks or jobs
with framework(like marathon) based on multiple factors. We want job or
frameworks belonging to one category be running into their specific cluster of
resources. We want to dynamically manage the slaves into these logical
sub-clusters.
> Allow http endpoint to dynamically change the slave attributes
> --------------------------------------------------------------
>
> Key: MESOS-3059
> URL: https://issues.apache.org/jira/browse/MESOS-3059
> Project: Mesos
> Issue Type: Wish
> Reporter: Nitin
>
> This is well understood that - changing the attributes dynamically is not
> safe without a restart because slave itself may not know which old framework
> tasks are running on it that were dependent on previous attributes.
> However, total restart makes lot of other history to delete. We need to
> ensure a dynamic attribute changes with a soft restart.
> It will be good to expose a rest endpoint either at slave or mesos-master
> which directly changes the state in zookeeper.
> USE-CASE
> We use slave attributes/roles to direct the framework scheduling to use
> specific slave as per it's requirements. Mesos scheduler only creates the
> offer on the basis of some resources.
> In our use case, we have some categorization of our spark frameworks or jobs
> with framework(like marathon) based on multiple factors. We want job or
> frameworks belonging to one category be running into their specific cluster
> of resources. We want to dynamically manage the slaves into these logical
> sub-clusters.
> Since number of jobs that will be submitted or when it will be submitted is
> very dynamic, it make sense to be able to dynamically assign roles or
> attributes to slaves. It is not possible to gauge the requirements at time of
> cluster provisioning. Static role or attribute assignment leads to
> sub-optimal use of the cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
