I'm running a Zookeeper, Master, 3 x Slave + Marathon cluster in Docker
successfully without using --net=host (using container linking instead) and
without modifying privileges. I'll try and post some Gists shortly but
here is how it is launched.
docker run -d --hostname=zookeeper --name=zookeeper -p 2181:2181 -p
2888:2888 -p 3888:3888 moderation/zookeeper
ZK_IP="$(docker inspect -f '{{ .NetworkSettings.IPAddress }}' zookeeper)"
docker run -d --hostname=mesosmaster --name=mesosmaster
--link=zookeeper:zookeeper -e MESOS_LOG_DIR=/var/log -e
MESOS_WORK_DIR=/opt/mesos-work-dir -e MESOS_CLUSTER=moderation -e
MESOS_ZK=zk://$ZK_IP:2181/mesos -e MESOS_QUORUM=1 -p 5050:5050
moderation/mesos-master
JOIN_IP="$(docker inspect -f '{{ .NetworkSettings.IPAddress }}'
mesosmaster)"
for i in 1 2 3; do
docker run -d --hostname=mesosslave$i --name=mesosslave$i
--link=mesosmaster:mesosmaster -e MESOS_LOG_DIR=/var/log -e
MESOS_WORK_DIR=/opt/mesos-work-dir -e MESOS_MASTER=$JOIN_IP:5050 -p
505$i:5051 moderation/mesos-slave
done
docker run -d --hostname=mesosmarath --name=mesosmarath
--link=mesosmaster:mesosmaster -p 8080:8080 moderation/marathon --master
zk://$ZK_IP:2181/mesos --zk zk://$ZK_IP:2181/marathon
On Wed, Sep 24, 2014 at 9:39 AM, Tim Chen <t...@mesosphere.io> wrote:
> Actually we don't really have any option for user specified name, so in
> our code we simply have mesos-<container-id> as the whole container name.
>
> I think the only problem is what Grzegorz had where he tried to prepend
> mesos-* as well in his own containers.
>
> I think what I'll do is to update the docker-containerization
> documentation to specifiy that Mesos treats all containers with mesos-
> prefix as managed by Mesos itself.
>
> Tim
>
> On Wed, Sep 24, 2014 at 9:33 AM, Tim Chen <t...@mesosphere.io> wrote:
>
>> Hi Grzegorz,
>>
>> Yes we assumed every container with mesos-* prefix was launched by us.
>>
>> James that sounds like a good idea, I'll create a ticket for it. Thanks
>> for the suggestion!
>>
>> Tim
>>
>>
>> On Wed, Sep 24, 2014 at 4:33 AM, James DeFelice <james.defel...@gmail.com
>> > wrote:
>>
>>> Kubernetes prefixes all of its Docker containers with "k8s--".
>>> User-specified container names are also escaped so that it's possible to
>>> decode the Docker container name back into its constituent parts:
>>>
>>>
>>> https://github.com/GoogleCloudPlatform/kubernetes/blob/88fdb659bc44cf2d1895c03f8838d36f4d890796/pkg/kubelet/dockertools/docker.go#L275
>>>
>>> https://github.com/GoogleCloudPlatform/kubernetes/blob/88fdb659bc44cf2d1895c03f8838d36f4d890796/pkg/kubelet/dockertools/docker.go#L297
>>>
>>> Might it make sense to take a similar approach with naming for
>>> mesos-managed Docker containers?
>>>
>>> -James
>>>
>>>
>>> On Wed, Sep 24, 2014 at 3:53 AM, Grzegorz Graczyk <gregor...@gmail.com>
>>> wrote:
>>>
>>>> I did enable verbose logging and it turns out mesos is prepending names
>>>> of it's docker containers with "mesos-". My mesos slave container also had
>>>> this prefix ("mesos-slave"), so when docker containerizer inside mesos
>>>> slave was starting it checks which containers managed by mesos should be
>>>> started. Of course mesos-slave wasn't managed by mesos, so docker
>>>> containerizer decided to kill that container, terminating itself.
>>>> When I change container name everything seems to be working just fine.
>>>>
>>>> That's the command I'm running mesos slave right now:
>>>> docker run -d --name slave --privileged --net=host -v
>>>> /var/run/docker.sock:/var/run/docker.sock -v
>>>> /usr/local/bin/docker:/usr/local/bin/docker -v
>>>> /sys/fs/cgroup:/sys/fs/cgroup gregory90/mesos-slave
>>>> --containerizers=docker,mesos --master=zk://localhost:2181/mesos
>>>> --ip=127.0.0.1
>>>>
>>>> Thanks for your help, really appreciate it!
>>>>
>>>> On 23 September 2014 18:35, Tim Chen <t...@mesosphere.io> wrote:
>>>>
>>>>> Hi Grzegorz,
>>>>>
>>>>> The log snippet you've pasted seems quite normal, can you enable
>>>>> verbose logging (add GLOG_v=1 before mesos-slave) and share it?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Tim
>>>>>
>>>>> On Tue, Sep 23, 2014 at 3:10 AM, Grzegorz Graczyk <gregor...@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> Thanks for your response!
>>>>>>
>>>>>> Mounting /sys did the job, cgroups are working, but now mesos-slave
>>>>>> is just crushing after detecting new master or so (there's nothing useful
>>>>>> in the logs - is there a way to make them more verbose?)
>>>>>>
>>>>>> Last lines of logs from mesos-slave:
>>>>>> I0923 10:03:24.079859 10 detector.cpp:426] A new leading master
>>>>>> (UPID=master@127.0.0.1:5050) is detected
>>>>>> I0923 10:03:26.076053 9 slave.cpp:3195] Finished recovery
>>>>>> I0923 10:03:26.076505 9 slave.cpp:589] New master detected at
>>>>>> master@127.0.0.1:5050
>>>>>> I0923 10:03:26.076732 9 slave.cpp:625] No credentials provided.
>>>>>> Attempting to register without authentication
>>>>>> I0923 10:03:26.076812 9 slave.cpp:636] Detecting new master
>>>>>> I0923 10:03:26.076864 9 status_update_manager.cpp:167] New master
>>>>>> detected at master@127.0.0.1:5050
>>>>>>
>>>>>> There's no problem in running mesos-master in the container(at least
>>>>>> there wasn't any in my case, for now)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 23 September 2014 09:41, Tim Chen <t...@mesosphere.io> wrote:
>>>>>>
>>>>>>> Hi Grzegorz,
>>>>>>>
>>>>>>> To run Mesos master|slave in a docker container is not straight
>>>>>>> forward because we utilize kernel features therefore you need to
>>>>>>> explicitly
>>>>>>> test out the features you like to use with Mesos with slave/master in
>>>>>>> Docker.
>>>>>>>
>>>>>>> Gabriel during the Mesosphere hackathon has got master and slave
>>>>>>> running in docker containers, and he can probably share his Dockerfile
>>>>>>> and
>>>>>>> run command.
>>>>>>>
>>>>>>> I believe one work around to get cgroups working with Docker run is
>>>>>>> to mount /sys into the container (mount -v /sys:/sys).
>>>>>>>
>>>>>>> Gabriel do you still have the command you used to run slave/master
>>>>>>> with Docker?
>>>>>>>
>>>>>>> Tim
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Sep 23, 2014 at 12:24 AM, Grzegorz Graczyk <
>>>>>>> gregor...@gmail.com> wrote:
>>>>>>>
>>>>>>>> I'm trying to run mesos-slave inside Docker container, but it can't
>>>>>>>> start due to problem with mounting cgroups.
>>>>>>>>
>>>>>>>> I'm using:
>>>>>>>> Kernel Version: 3.13.0-32-generic
>>>>>>>> Operating System: Ubuntu 14.04.1 LTS
>>>>>>>> Docker: 1.2.0(commit fa7b24f)
>>>>>>>> Mesos: 0.20.0
>>>>>>>>
>>>>>>>> Following error appears:
>>>>>>>> I0923 07:11:20.921475 19 main.cpp:126] Build: 2014-08-22
>>>>>>>> 05:04:26 by root
>>>>>>>> I0923 07:11:20.921608 19 main.cpp:128] Version: 0.20.0
>>>>>>>> I0923 07:11:20.921620 19 main.cpp:131] Git tag: 0.20.0
>>>>>>>> I0923 07:11:20.921628 19 main.cpp:135] Git SHA:
>>>>>>>> f421ffdf8d32a8834b3a6ee483b5b59f65956497
>>>>>>>> Failed to create a containerizer: Could not create
>>>>>>>> DockerContainerizer: Failed to find a mounted cgroups hierarchy for the
>>>>>>>> 'cpu' subsystem; you probably need to mount cgroups manually!
>>>>>>>>
>>>>>>>> I'm running docker container with command:
>>>>>>>> docker run --name mesos-slave --privileged --net=host -v
>>>>>>>> /var/run/docker.sock:/var/run/docker.sock -v
>>>>>>>> /var/lib/docker:/var/lib/docker -v
>>>>>>>> /usr/local/bin/docker:/usr/local/bin/docker gregory90/mesos-slave
>>>>>>>> --containerizers=docker,mesos --master=zk://localhost:2181/mesos
>>>>>>>> --ip=127.0.0.1
>>>>>>>>
>>>>>>>> Everything is running on single machine.
>>>>>>>> Everything works as expected when mesos-slave is run outside docker
>>>>>>>> container.
>>>>>>>>
>>>>>>>> I'd appreciate some help.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> James DeFelice
>>> 585.241.9488 (voice)
>>> 650.649.6071 (fax)
>>>
>>
>>
>
