Arslan Qadeer created MESOS-6132: ------------------------------------ Summary: Mesos 1.0.1 marathon 1.1.1 deployment fails no such image, container or task Key: MESOS-6132 URL: https://issues.apache.org/jira/browse/MESOS-6132 Project: Mesos Issue Type: Bug Affects Versions: 1.0.1 Reporter: Arslan Qadeer Priority: Blocker
Trying to launch docker containers with marathon on bridged network but they fail. I have tried to pull containers manually but it did not work too. Here are some logs from Mesos Salve/Master and Marathon. Mesos Slave Logs: "sudo ./bin/mesos-slave.sh --ip=<IP OF SLAVE> --master=zk://<IP OF ZOOKEEPER>:2181/mesos --advertise_ip=<IP OF SLAVE> --docker_socket=/var/run/docker.sock --work_dir=$/tmp/mesos-slave --containerizers=docker --executor_registration_timeout=5mins --isolation=docker/runtime" I0907 06:02:58.206800 5820 slave.cpp:1495] Got assigned task helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545 for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 [23/9246] I0907 06:02:58.208921 5820 slave.cpp:1614] Launching task helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545 for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 [22/9246] I0907 06:02:58.210595 5820 paths.cpp:528] Trying to chown '$/tmp/mesos-slave/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185ee[21/9246] xecutors/helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545/runs/0a52cc6d-499d-487d-983a-d3d8e62213e2' to user 'aio' [20/9246] I0907 06:02:58.214004 5820 slave.cpp:5674] Launching executor helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 with[19/9246] s cpus(*):0.1; mem(*):32 in work directory '$/tmp/mesos-slave/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000/executor[18/9246] rld.43c00fb5-74e2-11e6-80f0-a2e45809a545/runs/0a52cc6d-499d-487d-983a-d3d8e62213e2' [17/9246] I0907 06:02:58.216660 5820 slave.cpp:1840] Queuing task 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' for executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' [16/9246] ork d034dd41-d32c-4e81-a168-070c2185eefe-0000 [15/9246] I0907 06:02:58.217999 5817 docker.cpp:1042] Starting container '0a52cc6d-499d-487d-983a-d3d8e62213e2' for task 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' (and exe[14/9246] I0907 06:02:58.217999 5817 docker.cpp:1042] Starting container '0a52cc6d-499d-487d-983a-d3d8e62213e2' for task 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' (and exe[14/9246$ lloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545') of framework 'd034dd41-d32c-4e81-a168-070c2185eefe-0000' I0907 06:03:02.005821 5820 docker.cpp:658] Checkpointing pid 5853 to '$/tmp/mesos-slave/meta/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-07$ c2185eefe-0000/executors/helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545/runs/0a52cc6d-499d-487d-983a-d3d8e62213e2/pids/forked.pid' I0907 06:03:02.057451 5814 slave.cpp:2828] Got registration for executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 from executor(1)@192.168.50.14:59541 I0907 06:03:02.059079 5814 docker.cpp:1443] Ignoring updating container '0a52cc6d-499d-487d-983a-d3d8e62213e2' with resources passed to update is identical to existing resources I0907 06:03:02.059430 5814 slave.cpp:2005] Sending queued task 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' to executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 at executor(1)@192.168.50.14:59541 I0907 06:03:07.500459 5815 slave.cpp:3211] Handling status update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a5$ 5 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 from executor(1)@192.168.50.14:59541 E0907 06:03:07.539857 5820 slave.cpp:3456] Failed to update resources for container 0a52cc6d-499d-487d-983a-d3d8e62213e2 of executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809$ 545' running task helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545 on status update for terminal task, destroying container: Failed to run 'docker -H unix:///var/run/docker.sock i$ spect mesos-9b5505a5-71d9-47cd-a810-898bb24be347-S0.0a52cc6d-499d-487d-983a-d3d8e62213e2': exited with status 1; stderr='Error: No such image, container or task: mesos-9b5505a5-7$d9-47cd-a810-898bb24be347-S0.0a52cc6d-499d-487d-983a-d3d8e62213e2 ' I0907 06:03:07.540410 5820 docker.cpp:1852] Destroying container '0a52cc6d-499d-487d-983a-d3d8e62213e2' I0907 06:03:07.540549 5820 docker.cpp:1938] Sending SIGTERM to executor with pid: 5853 I0907 06:03:07.540467 5814 status_update_manager.cpp:320] Received status update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00fb5-74e2-11e6-$0f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 I0907 06:03:07.543401 5814 status_update_manager.cpp:825] Checkpointing UPDATE for status update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c$0fb5-74e2-11e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 I0907 06:03:07.555461 5820 docker.cpp:1980] Running docker stop on container '0a52cc6d-499d-487d-983a-d3d8e62213e2' I0907 06:03:07.557958 5816 slave.cpp:3732] executor(1)@192.168.50.14:59541 exited I0907 06:03:07.642285 5821 docker.cpp:2132] Executor for container '0a52cc6d-499d-487d-983a-d3d8e62213e2' has exited I0907 06:03:07.644544 5821 slave.cpp:4089] Executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 terminated with sign$l Terminated I0907 06:03:07.703641 5818 slave.cpp:3604] Forwarding the update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a54$ of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 to master@192.168.50.10:5050 I0907 06:03:07.704087 5818 slave.cpp:3514] Sending acknowledgement for status update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00fb5-74e2-1$ e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 to executor(1)@192.168.50.14:59541 E0907 06:03:07.704627 5822 process.cpp:2105] Failed to shutdown socket with fd 11: Transport endpoint is not connected I0907 06:03:07.743113 5814 status_update_manager.cpp:392] Received status update acknowledgement (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00fb5-74e2-1$ e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 I0907 06:03:07.743348 5814 status_update_manager.cpp:825] Checkpointing ACK for status update TASK_FAILED (UUID: ed6f0a0c-0687-4039-93e7-c14be4a9ec8c) for task helloworld.43c00f$ 5-74e2-11e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 I0907 06:03:07.804318 5814 slave.cpp:4193] Cleaning up executor 'helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 at execu$ or(1)@192.168.50.14:59541 I0907 06:03:07.805060 5814 slave.cpp:4281] Cleaning up framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 I0907 06:03:07.805099 5819 gc.cpp:55] Scheduling '$/tmp/mesos-slave/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000/executors$ helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545/runs/0a52cc6d-499d-487d-983a-d3d8e62213e2' for gc 6.99999068706074days in the future I0907 06:03:07.805243 5819 gc.cpp:55] Scheduling '$/tmp/mesos-slave/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000/executors$ helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' for gc 6.99999068475852days in the future I0907 06:03:07.805344 5819 gc.cpp:55] Scheduling '$/tmp/mesos-slave/meta/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000/exec$ tors/helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545/runs/0a52cc6d-499d-487d-983a-d3d8e62213e2' for gc 6.99999068390222days in the future I0907 06:03:07.805438 5819 gc.cpp:55] Scheduling '$/tmp/mesos-slave/meta/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000/exec$ tors/helloworld.43c00fb5-74e2-11e6-80f0-a2e45809a545' for gc 6.99999068312593days in the future I0907 06:03:07.805673 5820 status_update_manager.cpp:282] Closing status update streams for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 I0907 06:03:07.805923 5817 gc.cpp:55] Scheduling '$/tmp/mesos-slave/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000' for gc 6$ 99999067363259days in the future I0907 06:03:07.806212 5818 gc.cpp:55] Scheduling '$/tmp/mesos-slave/meta/slaves/9b5505a5-71d9-47cd-a810-898bb24be347-S0/frameworks/d034dd41-d32c-4e81-a168-070c2185eefe-0000' for gc 6.99999066995259days in the future Mesos Master Logs: "./bin/mesos-master.sh --advertise_ip=192.168.50.10 --zk=zk://192.168.50.10:2181/mesos --port=5050 --quorum=1 --registry=in_memory --work_dir=$/tmp/mesos_master" I0907 06:06:39.715235 419 master.cpp:4046] Processing REVIVE call for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e88 5e56e@127.0.1.1:41351 I0907 06:06:39.715349 419 hierarchical.cpp:1022] Removed offer filters for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 I0907 06:06:39.716269 419 master.cpp:5725] Sending 1 offers to framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@1 27.0.1.1:41351 I0907 06:06:39.770603 416 master.cpp:3342] Processing ACCEPT call for offers: [ 9b5505a5-71d9-47cd-a810-898bb24be347-O54 ] on agent 9b5505a5-71d9-47cd-a810-898bb24be347-S0 at sl ave(1)@192.168.50.14:5051 (edge2.plumgrid.com) for framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@127.0.1.1:41351 I0907 06:06:39.772132 416 master.cpp:7447] Adding task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 with resources cpus(*):1; mem(*):128 on agent 9b5505a5-71d9-47cd-a810-898b b24be347-S0 (edge2.plumgrid.com) I0907 06:06:39.772290 416 master.cpp:3831] Launching task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at sc heduler-91481722-8a3f-4297-97b7-a8c5e885e56e@127.0.1.1:41351 with resources cpus(*):1; mem(*):128 on agent 9b5505a5-71d9-47cd-a810-898bb24be347-S0 at slave(1)@192.168.50.14:5051 ( edge2.plumgrid.com) I0907 06:06:40.471755 418 master.cpp:5725] Sending 1 offers to framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@1 27.0.1.1:41351 I0907 06:06:40.475045 415 master.cpp:3951] Processing DECLINE call for offers: [ 9b5505a5-71d9-47cd-a810-898bb24be347-O55 ] for framework d034dd41-d32c-4e81-a168-070c2185eefe-00 00 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@127.0.1.1:41351 I0907 06:06:47.039434 420 http.cpp:381] HTTP GET for /master/state from 192.168.50.1:33520 with User-Agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (K HTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36' I0907 06:06:48.750298 418 master.cpp:5137] Status update TASK_FAILED (UUID: 7d401455-72c5-42a0-90b6-40ab849f6075) for task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 of fra mework d034dd41-d32c-4e81-a168-070c2185eefe-0000 from agent 9b5505a5-71d9-47cd-a810-898bb24be347-S0 at slave(1)@192.168.50.14:5051 (edge2.plumgrid.com) I0907 06:06:48.750366 418 master.cpp:5199] Forwarding status update TASK_FAILED (UUID: 7d401455-72c5-42a0-90b6-40ab849f6075) for task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809 a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 I0907 06:06:48.750640 418 master.cpp:6841] Updating the state of task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (lat est state: TASK_FAILED, status update state: TASK_FAILED) I0907 06:06:48.811859 419 master.cpp:4262] Processing ACKNOWLEDGE call 7d401455-72c5-42a0-90b6-40ab849f6075 for task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 of framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@127.0.1.1:41351 on agent 9b5505a5-71d9-47cd-a810-898bb24be347-S0 I0907 06:06:48.811954 419 master.cpp:6907] Removing task helloworld.c4f4b0f5-74e2-11e6-80f0-a2e45809a545 with resources cpus(*):1; mem(*):128 of framework d034dd41-d32c-4e81-a16 8-070c2185eefe-0000 on agent 9b5505a5-71d9-47cd-a810-898bb24be347-S0 at slave(1)@192.168.50.14:5051 (edge2.plumgrid.com) I0907 06:06:49.482637 415 master.cpp:5725] Sending 1 offers to framework d034dd41-d32c-4e81-a168-070c2185eefe-0000 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@1 27.0.1.1:41351 I0907 06:06:49.485888 421 master.cpp:3951] Processing DECLINE call for offers: [ 9b5505a5-71d9-47cd-a810-898bb24be347-O56 ] for framework d034dd41-d32c-4e81-a168-070c2185eefe-00 00 (marathon) at scheduler-91481722-8a3f-4297-97b7-a8c5e885e56e@127.0.1.1:41351 Marathon Logs: "MESOS_NATIVE_JAVA_LIBRARY=/home/aio/mesos-1.0.1/build/src/.libs/libmesos.so bin/start --master zk://192.168.50.10:2181/mesos --zk zk://192.168.50.10:2181/marathon --event_subscriber http_callback --task_launch_timeout 300000" [2016-09-07 06:11:42,148] INFO [/helloworld]: new app detected (mesosphere.marathon.upgrade.GroupVersioningUtil$:ForkJoinPool-2-worker-13) [141/10000] [2016-09-07 06:11:42,150] INFO Computed new deployment plan: [140/10000] DeploymentPlan 2016-09-07T10:11:42.148Z [139/10000] step 1: [138/10000] * Start(App(/helloworld, image="tutum/hello-world")), instances=0) [137/10000] step 2: [136/10000] * Scale(App(/helloworld, image="tutum/hello-world")), instances=1) [135/10000] (mesosphere.marathon.state.GroupManager:ForkJoinPool-2-worker-13) [134/10000] [2016-09-07 06:11:42,150] INFO Deploy plan with force=false: [133/10000] DeploymentPlan 2016-09-07T10:11:42.148Z [132/10000] step 1: [131/10000] * Start(App(/helloworld, image="tutum/hello-world")), instances=0) [130/10000] step 2: [129/10000] * Scale(App(/helloworld, image="tutum/hello-world")), instances=1) [128/10000] (mesosphere.marathon.MarathonSchedulerService$$EnhancerByGuice$$7b7c577c:ForkJoinPool-2-worker-13) [127/10000] [2016-09-07 06:11:42,150] INFO [/helloworld] storing new app version 2016-09-07T10:11:42.148Z (mesosphere.marathon.state.GroupManager:ForkJoinPool-2-worker-13) [126/10000] [2016-09-07 06:11:42,217] INFO Starting app /helloworld (mesosphere.marathon.SchedulerActions:marathon-akka.actor.default-dispatcher-30) [125/10000] [2016-09-07 06:11:42,217] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-53) [124/10000] [2016-09-07 06:11:42,217] INFO Already running 0 instances of /helloworld. Not scaling. (mesosphere.marathon.SchedulerActions:marathon-akka.actor.default-dispatcher-30)[123/10000] [2016-09-07 06:11:42,217] INFO Successfully started 0 instances of /helloworld (mesosphere.marathon.upgrade.AppStartActor:marathon-akka.actor.default-dispatcher-55) [122/10000] [2016-09-07 06:11:42,217] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-41) [121/10000] [2016-09-07 06:11:42,217] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-30) [120/10000] [2016-09-07 06:11:42,220] INFO Started appTaskLaunchActor for /helloworld version 2016-09-07T10:11:42.148Z with initial count 1 (mesosphere.marathon.core.launchqueue.im[119/10000] auncherActor:marathon-akka.actor.default-dispatcher-56) [118/10000] [2016-09-07 06:11:42,220] INFO activating matcher ActorOfferMatcher(Actor[akka://marathon/user/launchQueue/1/2-helloworld#-434864973]). (mesosphere.marathon.core.matche[117/10000] mpl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-18) [116/10000] [2016-09-07 06:11:42,220] INFO Received offers WANTED notification (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-18) [115/10000] [2016-09-07 06:11:42,220] INFO => revive offers NOW, canceling any scheduled revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.default-d[114/10000] 8) [113/10000] [2016-09-07 06:11:42,220] INFO 2 further revives still needed. Repeating reviveOffers according to --revive_offers_repetitions 3 (mesosphere.marathon.core.flow.impl.Rev[112/10000] tor:marathon-akka.actor.default-dispatcher-18) [111/10000] [2016-09-07 06:11:42,221] INFO => Schedule next revive at 2016-09-07T10:11:47.220Z in 5000 milliseconds, adhering to --min_revive_offers_interval 5000 (ms) (mesosphere.[110/10000] re.flow.impl.ReviveOffersActor:marathon-akka.actor.default-dispatcher-45) [109/10000] [2016-09-07 06:11:42,227] INFO Processing LaunchEphemeral(LaunchedEphemeral(task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545],AgentInfo(edge2.plumgrid.com,Some(9b5[108/10000] 47cd-a810-898bb24be347-S0),Buffer()),2016-09-07T10:11:42.148Z,Status(2016-09-07T10:11:42.226Z,None,None),List())) for task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a[107/10000] phere.marathon.core.launcher.impl.OfferProcessorImpl:ForkJoinPool-2-worker-13) [106/10000] [2016-09-07 06:11:42,227] INFO Request Launch for task 'helloworld.794398bd-74e3-11e6-80f0-a2e45809a545', version '2016-09-07T10:11:42.148Z'. 1 tasksToLaunch, 0 in flig[105/10000] rmed. not backing off (mesosphere.marathon.core.launchqueue.impl.AppTaskLauncherActor:marathon-akka.actor.default-dispatcher-56) [104/10000] [2016-09-07 06:11:42,227] INFO No tasks left to launch. Stop receiving offers for /helloworld, 2016-09-07T10:11:42.148Z (mesosphere.marathon.core.launchqueue.impl.AppTa[103/10000] ctor:marathon-akka.actor.default-dispatcher-56) [102/10000] [2016-09-07 06:11:42,227] INFO removing matcher ActorOfferMatcher(Actor[akka://marathon/user/launchQueue/1/2-helloworld#-434864973]) (mesosphere.marathon.core.matcher.m[101/10000] .OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-56) [100/10000] [2016-09-07 06:11:42,228] INFO Received offers NOT WANTED notification, canceling 2 revives (mesosphere.marathon.core.flow.impl.ReviveOffersActor:marathon-akka.actor.def[99/10000] tcher-56) [98/10000] [2016-09-07 06:11:42,228] INFO Finished processing 9b5505a5-71d9-47cd-a810-898bb24be347-O126. Matched 1 ops after 2 passes. cpus(*) 3.0; mem(*) 6832.0; disk(*) 79158.0; [97/10000] 1000->32000 left. (mesosphere.marathon.core.matcher.manager.impl.OfferMatcherManagerActor:marathon-akka.actor.default-dispatcher-56) [96/10000] [2016-09-07 06:11:42,253] INFO receiveTaskUpdate: updating status of task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545] (mesosphere.marathon.core.launchqueue.impl.Ap[95/10000] herActor:marathon-akka.actor.default-dispatcher-18) [94/10000] [2016-09-07 06:11:42,255] INFO Task launch for 'task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545]' was accepted. 0 tasksToLaunch, 0 in flight, 1 confirmed. not bac[93/10000] mesosphere.marathon.core.launchqueue.impl.AppTaskLauncherActor:marathon-akka.actor.default-dispatcher-55) [92/10000] [2016-09-07 06:11:42,277] INFO Updated groups/apps according to deployment plan fb9a07c1-c317-4bcf-8e59-d1783683fa71 (mesosphere.marathon.state.GroupManager:ForkJoinPool[91/10000] 3) [90/10000] [2016-09-07 06:11:42,277] INFO Deployment acknowledged. Waiting to get processed: [89/10000] DeploymentPlan 2016-09-07T10:11:42.148Z [88/10000] step 1: [87/10000] * Start(App(/helloworld, image="tutum/hello-world")), instances=0) [86/10000] step 2: [85/10000] * Scale(App(/helloworld, image="tutum/hello-world")), instances=1) [84/10000] (mesosphere.marathon.state.GroupManager:ForkJoinPool-2-worker-3) [83/10000] [2016-09-07 06:11:42,277] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-55) [82/10000] [2016-09-07 06:11:42,277] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-18) [81/10000] [2016-09-07 06:11:42,278] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "POST //192.168.50.10:8080/v2/apps HTTP/1.1" 201 880 "http://192.168.50.10:8080/ui/" "Mozill[80/10000] intosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" 13 (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a[79/10000] 1010311355-37) [78/10000] [2016-09-07 06:11:42,320] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "GET //192.168.50.10:8080/ui/img/icons/icon-small-app.svg HTTP/1.1" 200 1519 "http://192.168[77/10000] 0/ui/main.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosR[76/10000] $EnhancerByGuice$$7a66064c:qtp1010311355-36) [75/10000] [2016-09-07 06:11:42,320] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "GET //192.168.50.10:8080/ui/img/icons/icon-mini-ellipse-white.svg HTTP/1.1" 200 976 "http:/[74/10000] 0.10:8080/ui/main.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.htt[73/10000] uestLog$$EnhancerByGuice$$7a66064c:qtp1010311355-40) [72/10000] [2016-09-07 06:11:42,277] INFO Updated groups/apps according to deployment plan fb9a07c1-c317-4bcf-8e59-d1783683fa71 (mesosphere.marathon.state.GroupManager:ForkJoinPool[91/10000$ 3) [2016-09-07 06:11:42,277] INFO Deployment acknowledged. Waiting to get processed: DeploymentPlan 2016-09-07T10:11:42.148Z step 1: * Start(App(/helloworld, image="tutum/hello-world")), instances=0) step 2: * Scale(App(/helloworld, image="tutum/hello-world")), instances=1) (mesosphere.marathon.state.GroupManager:ForkJoinPool-2-worker-3) [2016-09-07 06:11:42,277] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-55) [2016-09-07 06:11:42,277] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-18) [2016-09-07 06:11:42,278] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "POST //192.168.50.10:8080/v2/apps HTTP/1.1" 201 880 "http://192.168.50.10:8080/ui/" "Mozilla/5.0 (Mac intosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" 13 (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064c:qtp 1010311355-37) [2016-09-07 06:11:42,320] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "GET //192.168.50.10:8080/ui/img/icons/icon-small-app.svg HTTP/1.1" 200 1519 "http://192.168.50.10:808 0/ui/main.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$ $EnhancerByGuice$$7a66064c:qtp1010311355-36) [2016-09-07 06:11:42,320] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "GET //192.168.50.10:8080/ui/img/icons/icon-mini-ellipse-white.svg HTTP/1.1" 200 976 "http://192.168.5 0.10:8080/ui/main.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosReq uestLog$$EnhancerByGuice$$7a66064c:qtp1010311355-40) [2016-09-07 06:11:42,320] INFO 192.168.50.1 - - [07/Sep/2016:10:11:42 +0000] "GET //192.168.50.10:8080/ui/img/icons/icon-mini-deploying-color.png HTTP/1.1" 200 1144 "http://192.16 8.50.10:8080/ui/main.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.Chao$ RequestLog$$EnhancerByGuice$$7a66064c:qtp1010311355-34) [2016-09-07 06:11:46,053] INFO 192.168.50.1 - - [07/Sep/2016:10:11:46 +0000] "GET //192.168.50.10:8080/v2/deployments HTTP/1.1" 200 376 "http://192.168.50.10:8080/ui/" "Mozilla/5$ 0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064$ :qtp1010311355-68) [2016-09-07 06:11:46,053] INFO 192.168.50.1 - - [07/Sep/2016:10:11:46 +0000] "GET //192.168.50.10:8080/v2/queue HTTP/1.1" 200 12 "http://192.168.50.10:8080/ui/" "Mozilla/5.0 (Mac$ ntosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064c:qtp10$ 0311355-66) [2016-09-07 06:11:46,055] INFO 192.168.50.1 - - [07/Sep/2016:10:11:46 +0000] "GET //192.168.50.10:8080/v2/groups?embed=group.groups&embed=group.apps&embed=group.apps.deployments&$ mbed=group.apps.counts&embed=group.apps.readiness HTTP/1.1" 200 1090 "http://192.168.50.10:8080/ui/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, l$ ke Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064c:qtp1010311355-33) [2016-09-07 06:11:51,052] INFO 192.168.50.1 - - [07/Sep/2016:10:11:51 +0000] "GET //192.168.50.10:8080/v2/deployments HTTP/1.1" 200 376 "http://192.168.50.10:8080/ui/" "Mozilla/5$ 0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064$ :qtp1010311355-40) [2016-09-07 06:11:51,053] INFO 192.168.50.1 - - [07/Sep/2016:10:11:51 +0000] "GET //192.168.50.10:8080/v2/queue HTTP/1.1" 200 12 "http://192.168.50.10:8080/ui/" "Mozilla/5.0 (Mac$ ntosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064c:qtp10$ 0311355-37) [2016-09-07 06:11:51,058] INFO 192.168.50.1 - - [07/Sep/2016:10:11:51 +0000] "GET //192.168.50.10:8080/v2/groups?embed=group.groups&embed=group.apps&embed=group.apps.deployments&$ mbed=group.apps.counts&embed=group.apps.readiness HTTP/1.1" 200 1090 "http://192.168.50.10:8080/ui/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, l$ ke Gecko) Chrome/52.0.2743.116 Safari/537.36" (mesosphere.chaos.http.ChaosRequestLog$$EnhancerByGuice$$7a66064c:qtp1010311355-36) [2016-09-07 06:11:51,492] INFO Received status update for task helloworld.794398bd-74e3-11e6-80f0-a2e45809a545: TASK_FAILED (Container exited with status 125) (mesosphere.maratho$ .MarathonScheduler$$EnhancerByGuice$$3a82e0a2:Thread-189) [2016-09-07 06:11:51,509] INFO Removed app [/helloworld] from tracker (mesosphere.marathon.core.task.tracker.TaskTracker$TasksByApp$:marathon-akka.actor.default-dispatcher-53) [2016-09-07 06:11:51,510] INFO Increasing delay. Task launch delay for [/helloworld] changed from [0 milliseconds] to [1 seconds]. (mesosphere.marathon.core.launchqueue.impl.Rate$ imiter$:marathon-akka.actor.default-dispatcher-55) [2016-09-07 06:11:51,510] INFO receiveTaskUpdate: task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545] finished (mesosphere.marathon.core.launchqueue.impl.AppTaskLauncherActor:$ arathon-akka.actor.default-dispatcher-53) [2016-09-07 06:11:51,511] INFO Sending event notification for task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545] of app [/helloworld]: TASK_FAILED (mesosphere.marathon.core.t$ sk.update.impl.steps.PostToEventStreamStepImpl$$EnhancerByGuice$$8b5bf639:marathon-akka.actor.default-dispatcher-52) [2016-09-07 06:11:51,511] INFO POSTing to all endpoints. (mesosphere.marathon.event.http.HttpEventActor:marathon-akka.actor.default-dispatcher-30) [2016-09-07 06:11:51,512] WARN New task [task [helloworld.794398bd-74e3-11e6-80f0-a2e45809a545]] failed during app /helloworld scaling, queueing another task (mesosphere.marathon$ upgrade.TaskStartActor:marathon-akka.actor.default-dispatcher-30) Following JSON was used to launch containers: { "id": "helloworld", "cpus": 1, "mem": 128, "instances": 1, "ipAddress": {}, "container": { "type": "DOCKER", "docker": { "image": "tutum/hello-world", "forcePullImage": true, "network": "BRIDGE" } } } -- This message was sent by Atlassian JIRA (v6.3.4#6332)