Hey Everyone, Excited to see discussions of this. Something I started playing around with just as the external containerizer was coming to life!
Diptanu – A few responses to your notes... > a. Mesos understanding docker metrics which should be straightforward because > docker writes all its metrics in the following fashin for cpu, blkio, memory > etc - /sys/fs/cgroup/cpu/docker/<conntainerid> Are these paths not operating system dependent? I'm not too familiar with cross-platform cgroups, so I am no doubt wrong here. These cgroup metrics are also the ones Mesos currently uses (both for usage statistics and for memory/cpu limits) so they can be pulled out much the same. > b. Easier way to map tasks to docker containers, probably the external > containerizer takes care of it to a large extent. It would be helpful if > there was a blog about it's API and internals by the core committers > explaining the design. Even a simple example in the mesos codebase using the > external containerizer would help. That's an interesting one, a blog post would be awesome. The containerizers right now currently use the "ContainerId" string provided to them from Mesos (I believe this is just the TaskID but i'm not certain of that). This helps ensure consistency of how containerizers are implemented, and makes them much simpler. > c. stdout and stderr of docker containers in the Mesos task stdout and stderr > logs. Does the external containerizer already takes care of it? I had to > write a service which runs on every slave for exposing the container logs to > an user. The external containerizer itself doesn't help you with this. The logs from the containerizer calls are dumped into the sandbox, however it's up to the containerizer (e.g Deimos) to redirect the logs from the container it launches. Deimos does take care of this already (as seen here https://github.com/mesosphere/deimos/blob/master/deimos/containerizer/docker.py#L132). > e. Translate all task constraints to docker run flags. This is probably the > easiest and I know it's super easy to implement with the external > containerizer. The current Docker containerizer implementations both do this, they support CPU, Memory and Ports. Docker currently doesn't support changing these limits on a running container, so you have to go behind docker and write to the cgroup limits yourself. There's also no way to change the port mappings of a container that I know of. Hope that answers some of your questions! Tom. On 21 Jun 2014, at 00:20, Diptanu Choudhury <dipta...@gmail.com> wrote: > Great timing for this thread! > > I have been working on this for the past few months and here is what I am > doing and would be nice if docker was supported straight way in Mesos. So > here goes the features that I would personally love to see in Mesos Core from > the perspective of an user which I had to implement on my own - > > a. Mesos understanding docker metrics which should be straightforward because > docker writes all its metrics in the following fashin for cpu, blkio, memory > etc - /sys/fs/cgroup/cpu/docker/<conntainerid> > I am sending all these metrics right now as a framework message back to my > framework/scheduler but it would be cool if Mesos took care of them. > > b. Easier way to map tasks to docker containers, probably the external > containerizer takes care of it to a large extent. It would be helpful if > there was a blog about it's API and internals by the core committers > explaining the design. Even a simple example in the mesos codebase using the > external containerizer would help. > > c. stdout and stderr of docker containers in the Mesos task stdout and stderr > logs. Does the external containerizer already takes care of it? I had to > write a service which runs on every slave for exposing the container logs to > an user. > > d. Mesos GC of tasks taking care of cleaning up docker containers which have > terminated. Right now the way I implemented this is that the service which > exposes the logs of a container also listens to docker events and when a > container exits, it knows that this has to be cleaned up and so removes it > after a fixed amount of time[configurable through a Rest API/config file] > > e. Translate all task constraints to docker run flags. This is probably the > easiest and I know it's super easy to implement with the external > containerizer. > > > On Fri, Jun 20, 2014 at 3:40 PM, Tobias Knaup <t...@knaup.me> wrote: > Hi all, > > We've got a lot of feedback from folks who use Mesos to run Dockers at scale > via Deimos, and the main wish was to make Docker a first class citizen in > Mesos, instead of a plugin that needs to be installed separately. Mesosphere > wants to contribute this and I already chatted with Ben H about what an > implementation could look like. > > I'd love for folks on here that are working with Docker to chime in! > I created a JIRA here: https://issues.apache.org/jira/browse/MESOS-1524 > > Cheers, > > Tobi > > > > -- > Thanks, > Diptanu Choudhury > Web - www.linkedin.com/in/diptanu > Twitter - @diptanu
