OK, the problem I spotted is related to the usage of the flag —docker_mesos_image that allows the executor to
--docker_mesos_image=VALUE The docker image used to launch this mesos
slave instance. If an image is specified, the docker containerizer assumes the
slave is running in a docker container, and launches executors with docker
containers in order to recover them when the slave restarts and recovers.
Has anyone used this flag and tested the behavior of the fetcher?
Thank you
Marica
Il giorno 18/dic/2015, alle ore 10:38, tommy xiao <xia...@gmail.com> ha scritto:
> no docker_mesos_image flag in my docker run, and the docker image is build
> by myself.
>
>
>
> 2015-12-18 17:20 GMT+08:00 Marica Antonacci <marica.antona...@ba.infn.it>:
> Yes, I did check inside the container and the csv file was not downloaded as
> shown also by the app details (see the screenshot below).
>
> Are you running your slave with the --docker_mesos_image flag? Can you please
> provide me the docker run command you are using to run your dockerized slave?
>
>
> Thank you very much
> Marica
>
>
> <Schermata 2015-12-18 alle 10.15.58.png>
>
>
> Il giorno 18/dic/2015, alle ore 10:00, tommy xiao <xia...@gmail.com> ha
> scritto:
>
>> Hi Marica,
>>
>> use your test-app json, i can run it correctly, the csv is truely download
>> by mesos slave. please check mesos-master:5050 to check the task detail
>> download files.
>>
>> you describe the app container why not found the csv, because the csv is
>> download in slave container's folder, not in app container. so if you run
>>
>> cd $MESOS_SANDBOX;
>>
>> the folder in app container is default value:
>> MESOS_SANDBOX=/mnt/mesos/sandbox
>>
>> but in real world, the sandbox is in slave container, not in app container.
>>
>>
>>
>> 2015-12-18 16:11 GMT+08:00 Marica Antonacci <marica.antona...@ba.infn.it>:
>> Thank you very much,
>>
>> I’m using a sample application definition file, just for testing purpose:
>>
>> {
>> "id": "test-app",
>> "container": {
>> "type": "DOCKER",
>> "docker": {
>> "image": "libmesos/ubuntu"
>> }
>> },
>> "cpus": 1,
>> "mem": 512,
>> "uris": [
>> "http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv";
>> ],
>> "cmd": "cd $MESOS_SANDBOX; ls -latr; while sleep 10; do date -u +%T; done"
>> }
>>
>> Here is the docker run command line:
>>
>> # docker run -d -e MESOS_HOSTNAME=<SLAVE_IP> -e MESOS_IP=<SLAVE_IP> -e
>> MESOS_MASTER=zk://<node-1>:2181,<node-2>:2181,<node-3>:2181/mesos -e
>> MESOS_CONTAINERIZERS=docker,mesos \
>> -e MESOS_EXECUTOR_REGISTRATION_TIMEOUT=5mins -e
>> MESOS_LOG_DIR=/var/log -e MESOS_docker_mesos_image=mesos-slave
>> -v /sys/fs/cgroup:/sys/fs/cgroup -v
>> /var/run/docker.sock:/var/run/docker.sock --name slave --net host
>> --privileged --pid host mesos-slave
>>
>>
>> As already mentioned, if I remove the environment variable
>> MESOS_docker_mesos_image the fetcher works fine and I can see the file
>> snoqualmie.csv inside the sandbox.
>>
>> Thank you again! I’m looking forward to hearing about your outcomes.
>> Best regards,
>> Marica
>>
>>
>> Il giorno 18/dic/2015, alle ore 04:51, tommy xiao <xia...@gmail.com> ha
>> scritto:
>>
>>> Hi Marica,
>>>
>>> Could you please give a sample marathon json, i can test it asap.
>>>
>>> 2015-12-18 5:11 GMT+08:00 Marica Antonacci <marica.antona...@ba.infn.it>:
>>> No, using the socket:
>>>
>>> -v /var/run/docker.sock:/var/run/docker.sock
>>>
>>>
>>> Il giorno 17/dic/2015, alle ore 18:07, tommy xiao <xia...@gmail.com> ha
>>> scritto:
>>>
>>>> docker in docker mode?
>>>>
>>>> 2015-12-17 19:08 GMT+08:00 Marica Antonacci <marica.antona...@ba.infn.it>:
>>>> Dear all,
>>>>
>>>> I'm testing the URIs fetching mechanism for both Marathon applications and
>>>> Chronos jobs and I have found that if the slave is running inside a docker
>>>> container (using docker_mesos_image startup flag) and you submit the
>>>> deployment of a dockerized application or job the fetcher step is not
>>>> performed. On the other hand, if I request the deployment of a
>>>> non-dockerized application, the URIs are correctly fetched. Moreover, if I
>>>> don’t provide the docker_mesos_image flag, the fetcher works fine again
>>>> for both dockerized and non-dockerized applications.
>>>>
>>>> Therefore, it seems that the information about the URIs gets lost when the
>>>> dockerized mesos slave spawns the executor docker container that in turn
>>>> launches the application docker container…Has anyone seen this problem
>>>> before? I would like to know if there is a workaround or a fixing.
>>>>
>>>> Thanks a lot in advance for you help
>>>> Best Regards,
>>>> Marica
>>>>
>>>>
>>>> ----------------------------------------------------------
>>>> Marica ANTONACCI
>>>> INFN - National Institute of Nuclear Physics
>>>> Via Orabona 4
>>>> 70126 Bari - ITALY
>>>> Phone +39 080 5443244
>>>> Skype: marica.antonacci
>>>> e-mail marica.antona...@ba.infn.it
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Deshi Xiao
>>>> Twitter: xds2000
>>>> E-mail: xiaods(AT)gmail.com
>>>
>>> ----------------------------------------------------------
>>> Marica ANTONACCI
>>> INFN - National Institute of Nuclear Physics
>>> Via Orabona 4
>>> 70126 Bari - ITALY
>>> Phone +39 080 5443244
>>> Skype: marica.antonacci
>>> e-mail marica.antona...@ba.infn.it
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Deshi Xiao
>>> Twitter: xds2000
>>> E-mail: xiaods(AT)gmail.com
>>
>> ----------------------------------------------------------
>> Marica ANTONACCI
>> INFN - National Institute of Nuclear Physics
>> Via Orabona 4
>> 70126 Bari - ITALY
>> Phone +39 080 5443244
>> Skype: marica.antonacci
>> e-mail marica.antona...@ba.infn.it
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>> Deshi Xiao
>> Twitter: xds2000
>> E-mail: xiaods(AT)gmail.com
>> <屏幕快照 2015-12-18 下午4.55.16.png><屏幕快照 2015-12-18 下午4.55.30.png><屏幕快照
>> 2015-12-18 下午4.55.38.png>
>
> ----------------------------------------------------------
> Marica ANTONACCI
> INFN - National Institute of Nuclear Physics
> Via Orabona 4
> 70126 Bari - ITALY
> Phone +39 080 5443244
> Skype: marica.antonacci
> e-mail marica.antona...@ba.infn.it
>
>
>
>
>
>
>
>
>
>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
----------------------------------------------------------
Marica ANTONACCI
INFN - National Institute of Nuclear Physics
Via Orabona 4
70126 Bari - ITALY
Phone +39 080 5443244
Skype: marica.antonacci
e-mail marica.antona...@ba.infn.it
smime.p7s
Description: S/MIME cryptographic signature
