Autoscaling in an IaaS environment

Hi, I am currently working for some projects using Mesos at Atos Toulouse and we are using it on top of a classical IaaS. After playing with Mesos and looking at some code it appears to me that there is no elasticity mechanism in place. I opened an issue in Jira some months ago here, which con

Autoscaling in an IaaS environment

Hi, I am currently working for some projects using Mesos at Atos Toulouse and we are using it on top of a classical IaaS. After playing with Mesos and looking at some code it appears to me that there is no elasticity mechanism in place. I opened an issue in Jira some months ago here, which con

Re: Autoscaling in an IaaS environment

With the number of IaaS providers out there, and the fact that Mesos doesn't really concern itself with where it's running (IaaS, bare-metal, on-prem, in the cloud), this sounds more like an operations problem than a feature that should be in Mesos core. By any chance, have you had a chance to loo

Re: Autoscaling in an IaaS environment

Thanks for pinging again Mathieu! I think auto-scaling of a Mesos cluster is a nifty feature to have. The only question in my mind (and likely others) is whether this functionality should reside in Mesos, or a framework or an operator. As you mentioned, Netflix took the framework way but it doesn'

Re: Autoscaling in an IaaS environment

I agree with Vinod that the Master accumulates a lot of statistics that can be used for smarter decisions about cluster scaling. However, I'm not sure this feature should reside in Mesos. I would rather expose statistics and / or recommendations and let external tooling or an operator do the job. O

Re: Autoscaling in an IaaS environment

my want: write a daemon to query mesos framework api, get the statistics from mesos api. then invoke the IaaS's API to scale the cluster size. 2015-08-02 22:32 GMT+08:00 Alex Rukletsov : > I agree with Vinod that the Master accumulates a lot of statistics that can > be used for smarter decisions

Re: Autoscaling in an IaaS environment

I prefer an auto-scaler outside mesos as well. As long as Mesos exports enough statistics, an outside auto-scaler should be able to make the auto-scale decision as smart as Mesos itself. It will also help to douple the resource scheduilng from resource infrastructure management. Mesos just need foc

Re: Autoscaling in an IaaS environment

With auto-scaling, shrinking is not as easy as growing. For example, we may need to "defragment" the cluster in order to shrink the number of slaves, and mesos seems to be in the best position to orchestrate such a process if you want do this based on framework's SLA constraints (would re-use inver

Re: Autoscaling in an IaaS environment

Hi Benjamin Right, Mesos has to orchestrate shrink, for example notify framework to gracefully terminate workload, or even make the schedule decision which host will be closed and reclaimed. However it does not mean Mesos has to be built with policy to trigger the auto-scale. The policy of auto-s

RE: Autoscaling in an IaaS environment

packing since most of our workflow is stateless and I think we will not allow stateful services in volatile slaves anyway. Regards, Mathieu -Original Message- From: Yong Feng [mailto:fengyong...@gmail.com] Sent: Tuesday, August 04, 2015 5:59 AM To: dev@mesos.apache.org Subject: R

Re: Autoscaling in an IaaS environment

Message----- > From: Yong Feng [mailto:fengyong...@gmail.com] > Sent: Tuesday, August 04, 2015 5:59 AM > To: dev@mesos.apache.org > Subject: Re: Autoscaling in an IaaS environment > > Hi Benjamin > > Right, Mesos has to orchestrate shrink, for example notify framework to >