I saw MESOS-4249 have already fixed in 0.28? On Wed, Mar 2, 2016 at 4:31 PM, Marica Antonacci < marica.antona...@ba.infn.it> wrote:
> Dear Shuai and Tim, > > any news about the patch for MESOS-4743? > Can I ask you when it will be officially released? > > Thank you in advance > Best regards, > Marica > > Il giorno 24/feb/2016, alle ore 11:23, Shuai Lin <linshuai2...@gmail.com> > ha scritto: > > ping @Tim, I think this bug also affects > https://issues.apache.org/jira/browse/MESOS-4743 . > > > > On Wed, Jan 20, 2016 at 10:20 PM, Shuai Lin <linshuai2...@gmail.com> > wrote: > >> The testing of this case requires to build a docker image for >> mesos-slave, so it seems not practical to add a test case for it in the >> mesos tests. >> >> Anyway, here is the scripts I use for testing this issue: >> https://gist.github.com/lins05/14455e92f37e91fd46ff >> >> On Wed, Jan 20, 2016 at 10:30 AM, Shuai Lin <linshuai2...@gmail.com> >> wrote: >> >>> Hi Tim, >>> >>> The review is here: https://reviews.apache.org/r/42390/ , would you >>> please take a look? >>> >>> Regards, >>> Shuai >>> >>> On Sat, Jan 9, 2016 at 9:42 AM, Shuai Lin <linshuai2...@gmail.com> >>> wrote: >>> >>>> Hi Maria and Tim, >>>> >>>> I'm setting up a test case for this scenario that would fail, after >>>> which I'll begin on fixing it. >>>> >>>> Is it feasible to include the fixing in the new release? >>>> >>>> >>>> I'm not sure, does 0.27 have an estimated release date now? >>>> >>>> >>>> Regards, >>>> Shuai >>>> >>>> On Sat, Jan 9, 2016 at 1:11 AM, Timothy Chen <t...@mesosphere.io> wrote: >>>> >>>>> I can shepherd no problem. >>>>> >>>>> Tim >>>>> >>>>> On Dec 25, 2015, at 4:32 PM, Shuai Lin <linshuai2...@gmail.com> wrote: >>>>> >>>>> I'll work on it. @Tim could you shepherd it? >>>>> >>>>> On Sat, Dec 26, 2015 at 2:49 AM, Marica Antonacci < >>>>> marica.antona...@ba.infn.it> wrote: >>>>> >>>>>> Hi Tim and Shuai, >>>>>> >>>>>> thank you very much for your reply. I have opened a JIRA issue on >>>>>> this: https://issues.apache.org/jira/browse/MESOS-4249 >>>>>> I hope it will be patched soon :) >>>>>> >>>>>> Best regards, >>>>>> Marica >>>>>> >>>>>> >>>>>> Il giorno 24/dic/2015, alle ore 17:54, Tim Chen <t...@mesosphere.io> >>>>>> ha scritto: >>>>>> >>>>>> Hi Marica/Shuai, >>>>>> >>>>>> Sorry haven't been able to spend the time to repro, but looks like >>>>>> Shuai confirmed it. >>>>>> >>>>>> Can one of you file a JIRA? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> Tim >>>>>> >>>>>> On Thu, Dec 24, 2015 at 6:16 AM, Shuai Lin <linshuai2...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Marica, >>>>>>> >>>>>>> I can reproduce the problem exactly as you described in the first >>>>>>> email of this thread. Without `MESOS_DOCKER_MESOS_IMAGE` environment >>>>>>> variable set, the fetcher works just fine; With it, the fetcher steps >>>>>>> seems >>>>>>> skipped. This looks like a bug to me. >>>>>>> >>>>>>> Regards, >>>>>>> Shuai >>>>>>> >>>>>>> On Tue, Dec 22, 2015 at 7:41 PM, Marica Antonacci < >>>>>>> marica.antona...@ba.infn.it> wrote: >>>>>>> >>>>>>>> Dear all, >>>>>>>> >>>>>>>> I have not solved this issue yet. Please, can anyone run the same >>>>>>>> test and let me know if the fetcher is correctly invoked? >>>>>>>> The test is really simple, just try to start a dockerized app (see >>>>>>>> json definition file below) through marathon on a mesos slave running >>>>>>>> in a >>>>>>>> docker container started with the option —docker_mesos_image=<mesos >>>>>>>> slave >>>>>>>> image>. >>>>>>>> I would appreciate very much any feedback. >>>>>>>> >>>>>>>> Sample Marathon app: >>>>>>>> { >>>>>>>> "id": "test-app", >>>>>>>> "container": { >>>>>>>> "type": "DOCKER", >>>>>>>> "docker": { >>>>>>>> "image": "libmesos/ubuntu" >>>>>>>> } >>>>>>>> }, >>>>>>>> "cpus": 1, >>>>>>>> "mem": 512, >>>>>>>> "uris": [ " >>>>>>>> http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv"; >>>>>>>> ], >>>>>>>> "cmd": "cd $MESOS_SANDBOX; ls -latr; while sleep 10; do date -u >>>>>>>> +%T; done" >>>>>>>> } >>>>>>>> >>>>>>>> Docker run command to start dockerized mesos slave: >>>>>>>> >>>>>>>> # docker run -d MESOS_HOSTNAME=<SLAVE_IP> -e MESOS_IP=<SLAVE_IP> -e >>>>>>>> MESOS_MASTER=zk://<node-1>:2181,<node-2>:2181,<node-3>:2181/mesos -e >>>>>>>> MESOS_CONTAINERIZERS=docker,mesos >>>>>>>> -e MESOS_EXECUTOR_REGISTRATION_TIMEOUT=5mins -e MESOS_LOG_DIR=/var/log >>>>>>>> -e >>>>>>>> MESOS_docker_mesos_image=mesos-slave -v /sys/fs/cgroup:/sys/fs/cgroup >>>>>>>> -v >>>>>>>> /var/run/docker.sock:/var/run/docker.sock -v /tmp/mesos:/tmp/mesos >>>>>>>> --name slave --net host --privileged --pid host mesos-slave >>>>>>>> >>>>>>>> Thank you very much in advance! >>>>>>>> Best regards, >>>>>>>> Marica >>>>>>>> >>>>>>>> Il giorno 19/dic/2015, alle ore 19:32, Marica Antonacci < >>>>>>>> marica.antona...@ba.infn.it> ha scritto: >>>>>>>> >>>>>>>> Dear Tim, >>>>>>>> >>>>>>>> I have collected some information from my test environment, >>>>>>>> starting the slave container with and without the —docker_mesos_image >>>>>>>> startup flag. Please let me know if you need further input. Thank you >>>>>>>> very >>>>>>>> much for your support! >>>>>>>> >>>>>>>> Using the flag —docker_mesos_image: >>>>>>>> >>>>>>>> root@mesos-slave:~# docker ps >>>>>>>> CONTAINER ID IMAGE COMMAND >>>>>>>> CREATED STATUS PORTS NAMES >>>>>>>> b30cea22a07c libmesos/ubuntu "/bin/sh -c 'cd $MESO" 2 >>>>>>>> minutes ago Up 2 minutes >>>>>>>> mesos-db70e09f-f39d-491c-8480-73d9858c140b-S0.d965f59b-cc1a-4081-95d2-f3370214c84d >>>>>>>> da9c78ec5727 mesos-slave "/bin/sh -c '/usr/lib" 2 >>>>>>>> minutes ago Up 2 minutes >>>>>>>> mesos-db70e09f-f39d-491c-8480-73d9858c140b-S0.d965f59b-cc1a-4081-95d2-f3370214c84d.executor >>>>>>>> 150f78fbf327 mesos-slave "/entrypoint.sh /usr/" 3 >>>>>>>> minutes ago Up 3 minutes slave >>>>>>>> >>>>>>>> *root@mesos-slave:~# docker logs slave* >>>>>>>> I1219 18:03:38.308544 19476 slave.cpp:1294] Got assigned task >>>>>>>> test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 for framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 >>>>>>>> I1219 18:03:38.314268 19476 slave.cpp:1410] Launching task >>>>>>>> test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 for framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 >>>>>>>> I1219 18:03:38.316261 19476 paths.cpp:436] Trying to chown >>>>>>>> '/tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000/executors/test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0/runs/d965f59b-cc1a-4081-95d2-f3370214c84d' >>>>>>>> to user 'root' >>>>>>>> I1219 18:03:38.327221 19476 slave.cpp:4999] Launching executor >>>>>>>> test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 of framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 with resources cpus(*):0.1; >>>>>>>> mem(*):32 in work directory >>>>>>>> '/tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000/executors/test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0/runs/d965f59b-cc1a-4081-95d2-f3370214c84d' >>>>>>>> I1219 18:03:38.330817 19476 slave.cpp:1628] Queuing task >>>>>>>> 'test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0' for executor >>>>>>>> 'test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0' of framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 >>>>>>>> I1219 18:03:38.335907 19480 docker.cpp:762] Starting container >>>>>>>> 'd965f59b-cc1a-4081-95d2-f3370214c84d' for task >>>>>>>> 'test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0' (and executor >>>>>>>> 'test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0') of framework >>>>>>>> '246b272b-d649-47c0-88ca-6b1ff35f437a-0000' >>>>>>>> I1219 18:03:40.084027 19482 docker.cpp:386] Checkpointing pid 19590 >>>>>>>> to >>>>>>>> '/tmp/mesos/meta/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000/executors/test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0/runs/d965f59b-cc1a-4081-95d2-f3370214c84d/pids/forked.pid' >>>>>>>> I1219 18:03:40.189172 19479 slave.cpp:2405] Got registration for >>>>>>>> executor 'test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0' of framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 from executor(1)@ >>>>>>>> 90.147.170.246:39381 >>>>>>>> I1219 18:03:40.191141 19479 docker.cpp:1012] Ignoring updating >>>>>>>> container 'd965f59b-cc1a-4081-95d2-f3370214c84d' with resources passed >>>>>>>> to >>>>>>>> update is identical to existing resources >>>>>>>> I1219 18:03:40.192512 19479 slave.cpp:1793] Sending queued task >>>>>>>> 'test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0' to executor >>>>>>>> 'test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0' of framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 at executor(1)@ >>>>>>>> 90.147.170.246:39381 >>>>>>>> I1219 18:03:42.425606 19483 slave.cpp:2762] Handling status update >>>>>>>> TASK_RUNNING (UUID: 8348c86f-f89a-4d45-afcb-0f5ea58ea552) for task >>>>>>>> test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 of framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 from executor(1)@ >>>>>>>> 90.147.170.246:39381 >>>>>>>> I1219 18:03:42.426415 19483 status_update_manager.cpp:322] Received >>>>>>>> status update TASK_RUNNING (UUID: >>>>>>>> 8348c86f-f89a-4d45-afcb-0f5ea58ea552) for >>>>>>>> task test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 of framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 >>>>>>>> I1219 18:03:42.427276 19483 status_update_manager.cpp:826] >>>>>>>> Checkpointing UPDATE for status update TASK_RUNNING (UUID: >>>>>>>> 8348c86f-f89a-4d45-afcb-0f5ea58ea552) for task >>>>>>>> test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 of framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 >>>>>>>> I1219 18:03:42.521395 19483 slave.cpp:3087] Forwarding the update >>>>>>>> TASK_RUNNING (UUID: 8348c86f-f89a-4d45-afcb-0f5ea58ea552) for task >>>>>>>> test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 of framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 to master@90.147.102.209 >>>>>>>> :5050 >>>>>>>> I1219 18:03:42.521852 19483 slave.cpp:3011] Sending acknowledgement >>>>>>>> for status update TASK_RUNNING (UUID: >>>>>>>> 8348c86f-f89a-4d45-afcb-0f5ea58ea552) >>>>>>>> for task test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 of framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 to executor(1)@ >>>>>>>> 90.147.170.246:39381 >>>>>>>> I1219 18:03:42.576373 19481 status_update_manager.cpp:394] Received >>>>>>>> status update acknowledgement (UUID: >>>>>>>> 8348c86f-f89a-4d45-afcb-0f5ea58ea552) >>>>>>>> for task test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 of framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 >>>>>>>> I1219 18:03:42.577263 19481 status_update_manager.cpp:826] >>>>>>>> Checkpointing ACK for status update TASK_RUNNING (UUID: >>>>>>>> 8348c86f-f89a-4d45-afcb-0f5ea58ea552) for task >>>>>>>> test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 of framework >>>>>>>> 246b272b-d649-47c0-88ca-6b1ff35f437a-0000 >>>>>>>> >>>>>>>> root@mesos-slave:~# *docker logs >>>>>>>> mesos-db70e09f-f39d-491c-8480-73d9858c140b-S0.d965f59b-cc1a-4081-95d2-f3370214c84d.executor* >>>>>>>> --container="mesos-db70e09f-f39d-491c-8480-73d9858c140b-S0.d965f59b-cc1a-4081-95d2-f3370214c84d" >>>>>>>> --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" >>>>>>>> --initialize_driver_logging="true" --launcher_dir="/usr/libexec/mesos" >>>>>>>> --logbufsecs="0" --logging_level="INFO" >>>>>>>> --mapped_directory="/mnt/mesos/sandbox" --quiet="false" >>>>>>>> --sandbox_directory="/tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000/executors/test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0/runs/d965f59b-cc1a-4081-95d2-f3370214c84d" >>>>>>>> --stop_timeout="0ns" >>>>>>>> --container="mesos-db70e09f-f39d-491c-8480-73d9858c140b-S0.d965f59b-cc1a-4081-95d2-f3370214c84d" >>>>>>>> --docker="docker" --docker_socket="/var/run/docker.sock" --help="false" >>>>>>>> --initialize_driver_logging="true" --launcher_dir="/usr/libexec/mesos" >>>>>>>> --logbufsecs="0" --logging_level="INFO" >>>>>>>> --mapped_directory="/mnt/mesos/sandbox" --quiet="false" >>>>>>>> --sandbox_directory="/tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000/executors/test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0/runs/d965f59b-cc1a-4081-95d2-f3370214c84d" >>>>>>>> --stop_timeout="0ns" >>>>>>>> I1219 18:03:40.177598 6 exec.cpp:136] Version: 0.26.0 >>>>>>>> I1219 18:03:40.192060 10 exec.cpp:210] Executor registered on >>>>>>>> slave db70e09f-f39d-491c-8480-73d9858c140b-S0 >>>>>>>> Registered docker executor on 90.147.170.246 >>>>>>>> Starting task test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 >>>>>>>> >>>>>>>> root@mesos-slave:~# *docker exec -it >>>>>>>> mesos-db70e09f-f39d-491c-8480-73d9858c140b-S0.d965f59b-cc1a-4081-95d2-f3370214c84d.executor >>>>>>>> bash* >>>>>>>> root@mesos-slave:/# ls -R /tmp/ >>>>>>>> hsperfdata_root/ mesos/ >>>>>>>> root@mesos-slave:/# ls -R /tmp/mesos/ >>>>>>>> /tmp/mesos/: >>>>>>>> *slaves* >>>>>>>> >>>>>>>> /tmp/mesos/slaves: >>>>>>>> *db70e09f-f39d-491c-8480-73d9858c140b-S0* >>>>>>>> >>>>>>>> /tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0: >>>>>>>> *frameworks* >>>>>>>> >>>>>>>> >>>>>>>> /tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0/frameworks: >>>>>>>> *246b272b-d649-47c0-88ca-6b1ff35f437a-0000* >>>>>>>> >>>>>>>> >>>>>>>> /tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000: >>>>>>>> *executors* >>>>>>>> >>>>>>>> >>>>>>>> /tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000/executors: >>>>>>>> *test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0* >>>>>>>> >>>>>>>> >>>>>>>> /tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000/executors/test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0: >>>>>>>> *runs* >>>>>>>> >>>>>>>> >>>>>>>> /tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000/executors/test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0/runs: >>>>>>>> *d965f59b-cc1a-4081-95d2-f3370214c84d* >>>>>>>> >>>>>>>> >>>>>>>> /tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S0/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000/executors/test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0/runs/d965f59b-cc1a-4081-95d2-f3370214c84d: >>>>>>>> stderr stdout >>>>>>>> >>>>>>>> >>>>>>>> root@mesos-slave:~# *docker exec -it >>>>>>>> mesos-db70e09f-f39d-491c-8480-73d9858c140b-S0.d965f59b-cc1a-4081-95d2-f3370214c84d >>>>>>>> bash* >>>>>>>> root@mesos-slave:/# env >>>>>>>> HOSTNAME=mesos-slave >>>>>>>> HOST=90.147.170.246 >>>>>>>> PORT0=31220 >>>>>>>> PORT_10000=31220 >>>>>>>> MESOS_TASK_ID=test-app.d4398af9-a67a-11e5-b1cf-fa163e920cd0 >>>>>>>> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin >>>>>>>> PWD=/ >>>>>>>> PORTS=31220 >>>>>>>> >>>>>>>> MESOS_CONTAINER_NAME=mesos-db70e09f-f39d-491c-8480-73d9858c140b-S0.d965f59b-cc1a-4081-95d2-f3370214c84d >>>>>>>> SHLVL=1 >>>>>>>> HOME=/ >>>>>>>> MARATHON_APP_ID=/test-app >>>>>>>> MARATHON_APP_DOCKER_IMAGE=libmesos/ubuntu >>>>>>>> MARATHON_APP_VERSION=2015-12-19T18:03:37.542Z >>>>>>>> PORT=31220 >>>>>>>> MESOS_SANDBOX=/mnt/mesos/sandbox >>>>>>>> _=/usr/bin/env >>>>>>>> root@mesos-slave:/# ls -R $MESOS_SANDBOX >>>>>>>> /mnt/mesos/sandbox: >>>>>>>> stderr stdout >>>>>>>> >>>>>>>> >>>>>>>> root@mesos-slave:/# ls /var/log/mesos* >>>>>>>> */var/log/mesos-slave.INFO* >>>>>>>> /var/log/mesos-slave.mesos-slave.invalid-user.log.INFO.20151219-182512.20647 >>>>>>>> >>>>>>>> /var/log/mesos: >>>>>>>> >>>>>>>> >>>>>>>> Disabling the flag —docker_mesos_image the fetcher is called, log >>>>>>>> is created, the file is downloaded in the sandbox: >>>>>>>> >>>>>>>> root@mesos-slave:~# docker exec -it slave bash >>>>>>>> root@mesos-slave:/# >>>>>>>> root@mesos-slave:/# >>>>>>>> root@mesos-slave:/# >>>>>>>> root@mesos-slave:/# ls /var/log/mesos* >>>>>>>> */var/log/mesos-fetcher.INFO* >>>>>>>> /var/log/mesos-fetcher.mesos-slave.invalid-user.log.INFO.20151219-181621.20157 >>>>>>>> */var/log/mesos-slave.INFO* >>>>>>>> /var/log/mesos-slave.mesos-slave.invalid-user.log.INFO.20151219-181612.20124 >>>>>>>> >>>>>>>> /var/log/mesos: >>>>>>>> root@mesos-slave:/# >>>>>>>> root@mesos-slave:/# cat /var/log/mesos-fetcher.INFO >>>>>>>> Log file created at: 2015/12/19 18:16:21 >>>>>>>> Running on machine: mesos-slave >>>>>>>> Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg >>>>>>>> I1219 18:16:21.127075 20157 logging.cpp:172] INFO level logging >>>>>>>> started! >>>>>>>> I1219 18:16:21.127499 20157 fetcher.cpp:422] Fetcher Info: >>>>>>>> {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/db70e09f-f39d-491c-8480-73d9858c140b-S1","items":[{"action":"BYPASS_CACHE","uri":{"extract":false,"value":"http:\/\/ >>>>>>>> www.stat.cmu.edu >>>>>>>> \/~cshalizi\/402\/lectures\/16-glm-practicals\/snoqualmie.csv"}}],"sandbox_directory":"\/tmp\/mesos\/slaves\/db70e09f-f39d-491c-8480-73d9858c140b-S1\/frameworks\/246b272b-d649-47c0-88ca-6b1ff35f437a-0000\/executors\/test-app.9aee927a-a67c-11e5-b1cf-fa163e920cd0\/runs\/1862ced9-3659-4429-8bc6-3f47f58970b4"} >>>>>>>> I1219 18:16:21.129547 20157 fetcher.cpp:377] Fetching URI ' >>>>>>>> http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv' >>>>>>>> I1219 18:16:21.129576 20157 fetcher.cpp:248] Fetching directly into >>>>>>>> the sandbox directory >>>>>>>> I1219 18:16:21.129607 20157 fetcher.cpp:185] Fetching URI ' >>>>>>>> http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv' >>>>>>>> I1219 18:16:21.129636 20157 fetcher.cpp:132] Downloading resource >>>>>>>> from ' >>>>>>>> http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv' >>>>>>>> to >>>>>>>> '/tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S1/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000/executors/test-app.9aee927a-a67c-11e5-b1cf-fa163e920cd0/runs/1862ced9-3659-4429-8bc6-3f47f58970b4/snoqualmie.csv' >>>>>>>> I1219 18:16:23.221889 20157 fetcher.cpp:454] Fetched ' >>>>>>>> http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv' >>>>>>>> to >>>>>>>> '/tmp/mesos/slaves/db70e09f-f39d-491c-8480-73d9858c140b-S1/frameworks/246b272b-d649-47c0-88ca-6b1ff35f437a-0000/executors/test-app.9aee927a-a67c-11e5-b1cf-fa163e920cd0/runs/1862ced9-3659-4429-8bc6-3f47f58970b4/snoqualmie.csv' >>>>>>>> root@mesos-slave:/# >>>>>>>> >>>>>>>> root@mesos-slave:~# docker exec -it >>>>>>>> mesos-db70e09f-f39d-491c-8480-73d9858c140b-S1.1862ced9-3659-4429-8bc6-3f47f58970b4 >>>>>>>> bash >>>>>>>> root@mesos-slave:/# >>>>>>>> root@mesos-slave:/# >>>>>>>> root@mesos-slave:/# ls /mnt/mesos/sandbox/ >>>>>>>> snoqualmie.csv stderr stdout >>>>>>>> root@mesos-slave:/# >>>>>>>> >>>>>>>> >>>>>>>> Best Regards, >>>>>>>> Marica >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Il giorno 19/dic/2015, alle ore 02:12, Tim Chen <t...@mesosphere.io> >>>>>>>> ha scritto: >>>>>>>> >>>>>>>> HI Marica, >>>>>>>> >>>>>>>> Did you see the fetcher invoked at all from the Slave logs? Doesn't >>>>>>>> seem possible we don't pass down the URI flags and if the fetcher >>>>>>>> failed >>>>>>>> the container launch should have failed too. >>>>>>>> >>>>>>>> Also another possible situation is that web UI not really showing >>>>>>>> the exact content of the sandbox, can you actually go into the >>>>>>>> directory >>>>>>>> and see if the files are there? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Tim >>>>>>>> >>>>>>>> On Fri, Dec 18, 2015 at 4:23 PM, Marica Antonacci < >>>>>>>> marica.antona...@ba.infn.it> wrote: >>>>>>>> >>>>>>>>> Hi Tim, >>>>>>>>> >>>>>>>>> looking at the sandbox I can see only the stderr and stout file >>>>>>>>> (see the attached screenshot). If I remove —docker_mesos_image (and >>>>>>>>> therefore the executor is run inside the slave container) the file >>>>>>>>> specified in the URI field is shown in the sandbox. >>>>>>>>> Did you verify that the fetcher is called when using the >>>>>>>>> —docker_mesos_image flag ? >>>>>>>>> >>>>>>>>> Thanks a lot for your feedback >>>>>>>>> Marica >>>>>>>>> >>>>>>>>> <Schermata 2015-12-18 alle 10.15.58.png> >>>>>>>>> >>>>>>>>> Il giorno 19/dic/2015, alle ore 00:25, Tim Chen <t...@mesosphere.io> >>>>>>>>> ha scritto: >>>>>>>>> >>>>>>>>> Hi Marica, >>>>>>>>> >>>>>>>>> It should work as we fetch all the files before we launch the >>>>>>>>> executor and place them in the sandbox, and we mount the sandbox into >>>>>>>>> that >>>>>>>>> container as well. >>>>>>>>> >>>>>>>>> How did you verify that the file is not downloaded? >>>>>>>>> >>>>>>>>> Tim >>>>>>>>> >>>>>>>>> On Fri, Dec 18, 2015 at 5:26 AM, Marica Antonacci < >>>>>>>>> marica.antona...@ba.infn.it> wrote: >>>>>>>>> >>>>>>>>>> Hi Grzegorz, >>>>>>>>>> >>>>>>>>>> I’m using this command line for docker run >>>>>>>>>> >>>>>>>>>> # docker run -d MESOS_HOSTNAME=<SLAVE_IP> -e MESOS_IP=<SLAVE_IP> >>>>>>>>>> -e MESOS_MASTER=zk://<node-1>:2181,<node-2>:2181,<node-3>:2181/mesos >>>>>>>>>> -e MESOS_CONTAINERIZERS=docker,mesos -e >>>>>>>>>> MESOS_EXECUTOR_REGISTRATION_TIMEOUT=5mins -e MESOS_LOG_DIR=/var/log >>>>>>>>>> -e >>>>>>>>>> MESOS_docker_mesos_image=mesos-slave -v >>>>>>>>>> /sys/fs/cgroup:/sys/fs/cgroup -v >>>>>>>>>> /var/run/docker.sock:/var/run/docker.sock -v /tmp/mesos:/tmp/mesos >>>>>>>>>> --name >>>>>>>>>> slave --net host --privileged --pid host mesos-slave >>>>>>>>>> >>>>>>>>>> where mesos-slave is the image built from the docker file in this >>>>>>>>>> repo https://github.com/maricaantonacci/mesos-slave-dev >>>>>>>>>> >>>>>>>>>> I have tested successfully the deployment of dockerized >>>>>>>>>> applications through Marathon and dockerized jobs through Chronos >>>>>>>>>> and also >>>>>>>>>> the recovery seems to work fine with the flag docker_mesos_image. >>>>>>>>>> What is >>>>>>>>>> not working for me is the fetcher: it seems that when the executor is >>>>>>>>>> launched as separate container (thanks to the flag >>>>>>>>>> docker_mesos_image) the >>>>>>>>>> information about the URIs to be downloaded is lost…I hope someone >>>>>>>>>> can help >>>>>>>>>> to understand if this a bug or I’ m missing something. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Marica >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Il giorno 18/dic/2015, alle ore 12:11, Grzegorz Graczyk < >>>>>>>>>> gregor...@gmail.com> ha scritto: >>>>>>>>>> >>>>>>>>>> I've tried to use this flag, but cannot really run any container >>>>>>>>>> when this flag is set. >>>>>>>>>> I've raised this issue here: >>>>>>>>>> https://www.mail-archive.com/user@mesos.apache.org/msg04975.html and >>>>>>>>>> here: >>>>>>>>>> https://github.com/mesosphere/docker-containers/issues/6#issuecomment-155364351 >>>>>>>>>> but >>>>>>>>>> sadly no one was able to help me... >>>>>>>>>> >>>>>>>>>> pt., 18.12.2015 o 11:33 użytkownik Marica Antonacci < >>>>>>>>>> marica.antona...@ba.infn.it> napisał: >>>>>>>>>> >>>>>>>>>>> OK, the problem I spotted is related to the usage of the >>>>>>>>>>> flag —docker_mesos_image that allows the executor to >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> --docker_mesos_image=VALUE The docker image used to launch this >>>>>>>>>>> mesos slave instance. If an image is specified, the docker >>>>>>>>>>> containerizer >>>>>>>>>>> assumes the slave is running in a docker container, and launches >>>>>>>>>>> executors >>>>>>>>>>> with docker containers in order to recover them when the slave >>>>>>>>>>> restarts and >>>>>>>>>>> recovers. >>>>>>>>>>> Has anyone used this flag and tested the behavior of the fetcher? >>>>>>>>>>> >>>>>>>>>>> Thank you >>>>>>>>>>> Marica >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Il giorno 18/dic/2015, alle ore 10:38, tommy xiao < >>>>>>>>>>> xia...@gmail.com> ha scritto: >>>>>>>>>>> >>>>>>>>>>> no docker_mesos_image flag in my docker run, and the docker >>>>>>>>>>> image is build by myself. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 2015-12-18 17:20 GMT+08:00 Marica Antonacci < >>>>>>>>>>> marica.antona...@ba.infn.it>: >>>>>>>>>>> >>>>>>>>>>> Yes, I did check inside the container and the csv file was not >>>>>>>>>>>> downloaded as shown also by the app details (see the screenshot >>>>>>>>>>>> below). >>>>>>>>>>>> >>>>>>>>>>>> Are you running your slave with the --docker_mesos_image flag? >>>>>>>>>>>> Can you please provide me the docker run command you are using to >>>>>>>>>>>> run your >>>>>>>>>>>> dockerized slave? >>>>>>>>>>>> >>>>>>>>>>>> Thank you very much >>>>>>>>>>>> >>>>>>>>>>> Marica >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> <Schermata 2015-12-18 alle 10.15.58.png> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Il giorno 18/dic/2015, alle ore 10:00, tommy xiao < >>>>>>>>>>>> xia...@gmail.com> ha scritto: >>>>>>>>>>>> >>>>>>>>>>>> Hi Marica, >>>>>>>>>>>> >>>>>>>>>>>> use your test-app json, i can run it correctly, the csv is >>>>>>>>>>>> truely download by mesos slave. please check mesos-master:5050 to >>>>>>>>>>>> check the >>>>>>>>>>>> task detail download files. >>>>>>>>>>>> >>>>>>>>>>>> you describe the app container why not found the csv, because >>>>>>>>>>>> the csv is download in slave container's folder, not in app >>>>>>>>>>>> container. so >>>>>>>>>>>> if you run >>>>>>>>>>>> >>>>>>>>>>>> cd $MESOS_SANDBOX; >>>>>>>>>>>> >>>>>>>>>>>> the folder in app container is default value: >>>>>>>>>>>> >>>>>>>>>>>> MESOS_SANDBOX=/mnt/mesos/sandbox >>>>>>>>>>>> but in real world, the sandbox is in slave container, not in >>>>>>>>>>>> app container. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> 2015-12-18 16:11 GMT+08:00 Marica Antonacci < >>>>>>>>>>>> marica.antona...@ba.infn.it>: >>>>>>>>>>>> >>>>>>>>>>>>> Thank you very much, >>>>>>>>>>>>> >>>>>>>>>>>>> I’m using a sample application definition file, just for >>>>>>>>>>>>> testing purpose: >>>>>>>>>>>>> >>>>>>>>>>>>> { >>>>>>>>>>>>> "id": "test-app", >>>>>>>>>>>>> "container": { >>>>>>>>>>>>> "type": "DOCKER", >>>>>>>>>>>>> "docker": { >>>>>>>>>>>>> "image": "libmesos/ubuntu" >>>>>>>>>>>>> } >>>>>>>>>>>>> }, >>>>>>>>>>>>> "cpus": 1, >>>>>>>>>>>>> "mem": 512, >>>>>>>>>>>>> *"uris": [ >>>>>>>>>>>>> "http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv >>>>>>>>>>>>> <http://www.stat.cmu.edu/~cshalizi/402/lectures/16-glm-practicals/snoqualmie.csv>" >>>>>>>>>>>>> ],* >>>>>>>>>>>>> "cmd": "cd $MESOS_SANDBOX; ls -latr; while sleep 10; do date >>>>>>>>>>>>> -u +%T; done" >>>>>>>>>>>>> } >>>>>>>>>>>>> >>>>>>>>>>>>> Here is the docker run command line: >>>>>>>>>>>>> >>>>>>>>>>>>> # docker run -d -e MESOS_HOSTNAME=<SLAVE_IP> -e >>>>>>>>>>>>> MESOS_IP=<SLAVE_IP> -e >>>>>>>>>>>>> MESOS_MASTER=zk://<node-1>:2181,<node-2>:2181,<node-3>:2181/mesos >>>>>>>>>>>>> -e MESOS_CONTAINERIZERS=docker,mesos \ >>>>>>>>>>>>> -e >>>>>>>>>>>>> MESOS_EXECUTOR_REGISTRATION_TIMEOUT=5mins -e >>>>>>>>>>>>> MESOS_LOG_DIR=/var/log -e >>>>>>>>>>>>> MESOS_docker_mesos_image=mesos-slave >>>>>>>>>>>>> -v /sys/fs/cgroup:/sys/fs/cgroup -v >>>>>>>>>>>>> /var/run/docker.sock:/var/run/docker.sock --name slave --net host >>>>>>>>>>>>> --privileged --pid host mesos-slave >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> As already mentioned, if I remove the environment variable >>>>>>>>>>>>> MESOS_docker_mesos_image the fetcher works fine and I can see the >>>>>>>>>>>>> file >>>>>>>>>>>>> snoqualmie.csv inside the sandbox. >>>>>>>>>>>>> >>>>>>>>>>>>> Thank you again! I’m looking forward to hearing about your >>>>>>>>>>>>> outcomes. >>>>>>>>>>>>> Best regards, >>>>>>>>>>>>> Marica >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Il giorno 18/dic/2015, alle ore 04:51, tommy xiao < >>>>>>>>>>>>> xia...@gmail.com> ha scritto: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Marica, >>>>>>>>>>>>> >>>>>>>>>>>>> Could you please give a sample marathon json, i can test it >>>>>>>>>>>>> asap. >>>>>>>>>>>>> >>>>>>>>>>>>> 2015-12-18 5:11 GMT+08:00 Marica Antonacci < >>>>>>>>>>>>> marica.antona...@ba.infn.it>: >>>>>>>>>>>>> >>>>>>>>>>>>>> No, using the socket: >>>>>>>>>>>>>> >>>>>>>>>>>>>> -v /var/run/docker.sock:/var/run/docker.sock >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Il giorno 17/dic/2015, alle ore 18:07, tommy xiao < >>>>>>>>>>>>>> xia...@gmail.com> ha scritto: >>>>>>>>>>>>>> >>>>>>>>>>>>>> docker in docker mode? >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2015-12-17 19:08 GMT+08:00 Marica Antonacci < >>>>>>>>>>>>>> marica.antona...@ba.infn.it>: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Dear all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm testing the URIs fetching mechanism for both Marathon >>>>>>>>>>>>>>> applications and Chronos jobs and I have found that if the >>>>>>>>>>>>>>> slave is running >>>>>>>>>>>>>>> inside a docker container (using *docker_mesos_image* >>>>>>>>>>>>>>> startup flag) and you submit the deployment of a dockerized >>>>>>>>>>>>>>> application or >>>>>>>>>>>>>>> job the fetcher step is not performed. On the other hand, if I >>>>>>>>>>>>>>> request the >>>>>>>>>>>>>>> deployment of a non-dockerized application, the URIs are >>>>>>>>>>>>>>> correctly fetched. >>>>>>>>>>>>>>> Moreover, if I don’t provide the docker_mesos_image flag, the >>>>>>>>>>>>>>> fetcher works >>>>>>>>>>>>>>> fine again for both dockerized and non-dockerized applications. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Therefore, it seems that the information about the URIs gets >>>>>>>>>>>>>>> lost when the dockerized mesos slave spawns the executor docker >>>>>>>>>>>>>>> container >>>>>>>>>>>>>>> that in turn launches the application docker container…Has >>>>>>>>>>>>>>> anyone seen this >>>>>>>>>>>>>>> problem before? I would like to know if there is a workaround >>>>>>>>>>>>>>> or a fixing. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks a lot in advance for you help >>>>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>>>> Marica >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> ---------------------------------------------------------- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Marica ANTONACCI >>>>>>>>>>>>>>> INFN - National Institute of Nuclear Physics >>>>>>>>>>>>>>> Via Orabona 4 >>>>>>>>>>>>>>> 70126 Bari - ITALY >>>>>>>>>>>>>>> Phone +39 080 5443244 >>>>>>>>>>>>>>> Skype: marica.antonacci >>>>>>>>>>>>>>> e-mail marica.antona...@ba.infn.it >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Deshi Xiao >>>>>>>>>>>>>> Twitter: xds2000 >>>>>>>>>>>>>> E-mail: xiaods(AT)gmail.com >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> ---------------------------------------------------------- >>>>>>>>>>>>>> >>>>>>>>>>>>>> Marica ANTONACCI >>>>>>>>>>>>>> INFN - National Institute of Nuclear Physics >>>>>>>>>>>>>> Via Orabona 4 >>>>>>>>>>>>>> 70126 Bari - ITALY >>>>>>>>>>>>>> Phone +39 080 5443244 >>>>>>>>>>>>>> Skype: marica.antonacci >>>>>>>>>>>>>> e-mail marica.antona...@ba.infn.it >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Deshi Xiao >>>>>>>>>>>>> Twitter: xds2000 >>>>>>>>>>>>> E-mail: xiaods(AT)gmail.com >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> ---------------------------------------------------------- >>>>>>>>>>>>> >>>>>>>>>>>>> Marica ANTONACCI >>>>>>>>>>>>> INFN - National Institute of Nuclear Physics >>>>>>>>>>>>> Via Orabona 4 >>>>>>>>>>>>> 70126 Bari - ITALY >>>>>>>>>>>>> Phone +39 080 5443244 >>>>>>>>>>>>> Skype: marica.antonacci >>>>>>>>>>>>> e-mail marica.antona...@ba.infn.it >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Deshi Xiao >>>>>>>>>>>> Twitter: xds2000 >>>>>>>>>>>> E-mail: xiaods(AT)gmail.com >>>>>>>>>>>> <屏幕快照 2015-12-18 下午4.55.16.png><屏幕快照 2015-12-18 下午4.55.30.png><屏幕快照 >>>>>>>>>>>> 2015-12-18 下午4.55.38.png> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ---------------------------------------------------------- >>>>>>>>>>>> >>>>>>>>>>>> Marica ANTONACCI >>>>>>>>>>>> INFN - National Institute of Nuclear Physics >>>>>>>>>>>> Via Orabona 4 >>>>>>>>>>>> 70126 Bari - ITALY >>>>>>>>>>>> Phone +39 080 5443244 >>>>>>>>>>>> Skype: marica.antonacci >>>>>>>>>>>> e-mail marica.antona...@ba.infn.it >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Deshi Xiao >>>>>>>>>>> Twitter: xds2000 >>>>>>>>>>> E-mail: xiaods(AT)gmail.com >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ---------------------------------------------------------- >>>>>>>>>>> >>>>>>>>>>> Marica ANTONACCI >>>>>>>>>>> INFN - National Institute of Nuclear Physics >>>>>>>>>>> Via Orabona 4 >>>>>>>>>>> 70126 Bari - ITALY >>>>>>>>>>> Phone +39 080 5443244 >>>>>>>>>>> Skype: marica.antonacci >>>>>>>>>>> e-mail marica.antona...@ba.infn.it >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> ---------------------------------------------------------- >>>>>>>>>> >>>>>>>>>> Marica ANTONACCI >>>>>>>>>> INFN - National Institute of Nuclear Physics >>>>>>>>>> Via Orabona 4 >>>>>>>>>> 70126 Bari - ITALY >>>>>>>>>> Phone +39 080 5443244 >>>>>>>>>> Skype: marica.antonacci >>>>>>>>>> e-mail marica.antona...@ba.infn.it >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> ---------------------------------------------------------- >>>>>>>>> >>>>>>>>> Marica ANTONACCI >>>>>>>>> INFN - National Institute of Nuclear Physics >>>>>>>>> Via Orabona 4 >>>>>>>>> 70126 Bari - ITALY >>>>>>>>> Phone +39 080 5443244 >>>>>>>>> Skype: marica.antonacci >>>>>>>>> e-mail marica.antona...@ba.infn.it >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> ---------------------------------------------------------- >>>>>>>> >>>>>>>> Marica ANTONACCI >>>>>>>> INFN - National Institute of Nuclear Physics >>>>>>>> Via Orabona 4 >>>>>>>> 70126 Bari - ITALY >>>>>>>> Phone +39 080 5443244 >>>>>>>> Skype: marica.antonacci >>>>>>>> e-mail marica.antona...@ba.infn.it >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ---------------------------------------------------------- >>>>>>>> >>>>>>>> Marica ANTONACCI >>>>>>>> INFN - National Institute of Nuclear Physics >>>>>>>> Via Orabona 4 >>>>>>>> 70126 Bari - ITALY >>>>>>>> Phone +39 080 5443244 >>>>>>>> Skype: marica.antonacci >>>>>>>> e-mail marica.antona...@ba.infn.it >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> ---------------------------------------------------------- >>>>>> >>>>>> Marica ANTONACCI >>>>>> INFN - National Institute of Nuclear Physics >>>>>> Via Orabona 4 >>>>>> 70126 Bari - ITALY >>>>>> Phone +39 080 5443244 >>>>>> Skype: marica.antonacci >>>>>> e-mail marica.antona...@ba.infn.it >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > > ---------------------------------------------------------- > > Marica ANTONACCI > INFN - National Institute of Nuclear Physics > Via Orabona 4 > 70126 Bari - ITALY > Phone +39 080 5443244 > Skype: marica.antonacci > e-mail marica.antona...@ba.infn.it > > > > > > > > > > > -- Best Regards, Haosdent Huang
