Re: Mesos 0.20.0 with Docker registry availability

2014-09-05 Thread Tim Chen
Hi Tom, It's definitely a tradeoff between able to automatically pick up latest changes, and able to have more control of what your cluster is running. The downside of just pulling latest with the latest is that you might not know what's exactly running since someone might override latest and the

Re: Mesos 0.20.0 with Docker registry availability

2014-09-05 Thread Tom Arnfeld
> You can tag each image with your commit hash that way Mesos will always have to do a docker pull and you don't lose the fast iteration cycle in development. I mentioned this on one of the review requests the other day. The problem here is that, say I want to iterate quickly on installing things

Re: Mesos 0.20.0 with Docker registry availability

2014-09-05 Thread Steve Domin
Hi Ryan, You can tag each image with your commit hash that way Mesos will always have to do a docker pull and you don't lose the fast iteration cycle in development. Steve On Friday, September 5, 2014, craig mcmillan wrote: > hey ryan, > > there are two deployment use-cases i generally have :

Re: Mesos 0.20.0 with Docker registry availability

2014-09-05 Thread craig mcmillan
hey ryan, there are two deployment use-cases i generally have : - production : i want to consider carefully what i deploy, and refer to a specific image. a versioned tag works well here - development : i want to iterate quickly and something like a branch (movable

Re: Mesos 0.20.0 with Docker registry availability

2014-09-05 Thread Ryan Thomas
Whilst this is somewhat unrelated to the mesos implementation, I think it is generally good practice to have immutable tags on the images, this is something I dislike about docker :) Whist the gc of old images will eventually become a problem, it will really only be the layer delta that is consume

Re: Mesos 0.20.0 with Docker registry availability

2014-09-05 Thread mccraig mccraig
ah, so i will have to use a different tag to update an app one immediate problem i can see is that it makes garbage collecting old docker images from slaves harder : currently i update the image associated with a tag and restart tasks to update the running app, then occasionally a cron job to remo

Re: Mesos 0.20.0 with Docker registry availability

2014-09-05 Thread Ryan Thomas
Hey Craig, docker run will attempt a pull of the image if it cannot find a matching image and tag in its local repository. So it should only pull on the first run of a given tag. ryan On 5 Sep 2014 17:41, "mccraig mccraig" wrote: > hi tim, > > if it doesn't pull on every run, when will it pull

Re: Mesos 0.20.0 with Docker registry availability

2014-09-05 Thread mccraig mccraig
hi tim, if it doesn't pull on every run, when will it pull ? :craig On 5 Sep 2014, at 07:05, Tim Chen wrote: Hi Maxime, It is a very valid concern and that's why I've added a patch that should go out in 0.20.1 to not do a docker pull on every run anymore. Mesos will still try to docker pull

Re: Mesos 0.20.0 with Docker registry availability

2014-09-04 Thread Tim Chen
It is an option that I'm considering adding, but not sure about the failure to pull part. Can you create a jira for this? We can then discuss there. Tim On Thu, Sep 4, 2014 at 11:19 PM, Steven Schlansker < sschlans...@opentable.com> wrote: > Would it be possible to have a mode where it tries to

Re: Mesos 0.20.0 with Docker registry availability

2014-09-04 Thread Steven Schlansker
Would it be possible to have a mode where it tries to pull, but then does not fail solely due to the fail of a pull? In particular, we use tags to indicate which build should be deployed e.g. “foo-server:production” tag vs “foo-server:staging” tags. On Sep 4, 2014, at 11:05 PM, Tim Chen wrote

Re: Mesos 0.20.0 with Docker registry availability

2014-09-04 Thread Tim Chen
Hi Maxime, It is a very valid concern and that's why I've added a patch that should go out in 0.20.1 to not do a docker pull on every run anymore. Mesos will still try to docker pull when the image isn't available locally (via docker inspect), but only once. The downside ofcourse is that you're

Mesos 0.20.0 with Docker registry availability

2014-09-04 Thread Maxime Brugidou
Hi, The current Docker integration in 0.20 does a "docker pull" from the registry before running any task. This means that your entire Mesos cluster becomes unusable if the registry goes down. The docs allow you to configure a custom .dockercfg for your tasks to point to a private docker registry