Tomasz Janiszewski created MESOS-5536:
-----------------------------------------
Summary: Completed executors presented as alive
Key: MESOS-5536
URL: https://issues.apache.org/jira/browse/MESOS-5536
Project: Mesos
Issue Type: Bug
Affects Versions: 0.28.0
Environment: Ubuntu 14.04.3 LTS
Reporter: Tomasz Janiszewski
I'm running Mesos 0.28.0. Mesos slave(1)/state endpoint returns some completed
executors not in frameworks.completed_executors but in frameworks.executors.
{code:JavaScript:title=slave(1)/state}
{
"attributes": {...},
"completed_frameworks": [],
"flags": {...},
"frameworks": [
{
"checkpoint": true,
"completed_executors": [...],
"executors": [
{
"queued_tasks": [],
"tasks": [],
"completed_tasks": [
{
"discovery": {...},
"executor_id": "",
"framework_id":
"f65b163c-0faf-441f-ac14-91739fa4394c-0000",
"id": "service.a3b609b8-27ec-11e6-8044-02c89eb9127e",
"labels": [...],
"name": "service",
"resources": {...},
"slave_id":
"ef232fd9-5114-4d8f-adc3-1669c1e6fdc5-S13",
"state": "TASK_KILLED",
"statuses": []
}
],
"container": "ead42e63-ac92-4ad0-a99c-4af9c3fa5e31",
"directory": "...",
"id": "service.a3b609b8-27ec-11e6-8044-02c89eb9127e",
"name": "Command Executor (Task:
service.a3b609b8-27ec-11e6-8044-02c89eb9127e) (Command: sh -c 'cd
service...')",
"resources": {...},
"source": "service.a3b609b8-27ec-11e6-8044-02c89eb9127e"
},
...
],
}
],
"git_sha": "961edbd82e691a619a4c171a7aadc9c32957fa73",
"git_tag": "0.28.0",
"version": "0.28.0",
...
}
{code}
{code:title="var/log/mesos/mesos-slave.INFO"}
13:33:19.479182 [slave.cpp:1361] Got assigned task
service.a3b609b8-27ec-11e6-8044-02c89eb9127e for framework
f65b163c-0faf-441f-ac14-91739fa4394c-0000
13:33:19.482566 [slave.cpp:1480] Launching task
service.a3b609b8-27ec-11e6-8044-02c89eb9127e for framework
f65b163c-0faf-441f-ac14-91739fa4394c-0000
13:33:19.483921 [paths.cpp:528] Trying to chown
'/tmp/mesos/slaves/ef232fd9-5114-4d8f-adc3-1669c1e6fdc5-S13/frameworks/f65b163c-0faf-441f-ac14-91739fa4394c-0000/executors/service.a3b609b8-27ec-11e6-8044-02c89eb9127e/runs/ead42e63-ac92-4ad0-a99c-4af9c3fa5e31'
to user 'mesosuser'
13:33:19.504173 [slave.cpp:5367] Launching executor
service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
f65b163c-0faf-441f-ac14-91739fa4394c-0000 with resources cpus(*):0.1; mem(*):32
in work directory
'/tmp/mesos/slaves/ef232fd9-5114-4d8f-adc3-1669c1e6fdc5-S13/frameworks/f65b163c-0faf-441f-ac14-91739fa4394c-0000/executors/service.a3b609b8-27ec-11e6-8044-02c89eb9127e/runs/ead42e63-ac92-4ad0-a99c-4af9c3fa5e31'
13:33:19.505537 [containerizer.cpp:666] Starting container
'ead42e63-ac92-4ad0-a99c-4af9c3fa5e31' for executor
'service.a3b609b8-27ec-11e6-8044-02c89eb9127e' of framework
'f65b163c-0faf-441f-ac14-91739fa4394c-0000'
13:33:19.505734 [slave.cpp:1698] Queuing task
'service.a3b609b8-27ec-11e6-8044-02c89eb9127e' for executor
'service.a3b609b8-27ec-11e6-8044-02c89eb9127e' of framework
f65b163c-0faf-441f-ac14-91739fa4394c-0000
...
13:33:19.977483 [containerizer.cpp:1118] Checkpointing executor's forked pid
25576 to
'/tmp/mesos/meta/slaves/ef232fd9-5114-4d8f-adc3-1669c1e6fdc5-S13/frameworks/f65b163c-0faf-441f-ac14-91739fa4394c-0000/executors/service.a3b609b8-27ec-11e6-8044-02c89eb9127e/runs/ead42e63-ac92-4ad0-a99c-4af9c3fa5e31/pids/forked.pid'
13:33:35.775195 [slave.cpp:1891] Asked to kill task
service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
f65b163c-0faf-441f-ac14-91739fa4394c-0000
13:33:35.775645 [slave.cpp:3002] Handling status update TASK_KILLED (UUID:
eba64915-7df2-483d-8982-a9a46a48a81b) for task
service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
f65b163c-0faf-441f-ac14-91739fa4394c-0000 f
rom @0.0.0.0:0
13:33:35.778105 [cpushare.cpp:389] Updated 'cpu.shares' to 102 (cpus 0.1) for
container ead42e63-ac92-4ad0-a99c-4af9c3fa5e31
13:33:35.778488 [disk.cpp:169] Updating the disk resources for container
ead42e63-ac92-4ad0-a99c-4af9c3fa5e31 to cpus(*):0.1; mem(*):32
13:33:35.780349 [mem.cpp:353] Updated 'memory.soft_limit_in_bytes' to 32MB for
container ead42e63-ac92-4ad0-a99c-4af9c3fa5e3
1
13:33:35.782573 [status_update_manager.cpp:320] Received status update
TASK_KILLED (UUID: eba64915-7df2-483d-8982-a9a46a48a8
1b) for task service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
f65b163c-0faf-441f-ac14-9173
9fa4394c-0000
13:33:35.783860 [status_update_manager.cpp:824] Checkpointing UPDATE for
status update TASK_KILLED (UUID: eba64915-7df2-483d-8982-a9a46a48a81b) for task
service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
f65b163c-0faf-441f-ac14-91739fa4394c-0000
13:33:35.788767 [slave.cpp:3400] Forwarding the update TASK_KILLED (UUID:
eba64915-7df2-483d-8982-a9a46a48a81b) for task
service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
f65b163c-0faf-441f-ac14-91739fa4394c-0000 to [email protected]:5050
13:33:35.917932 [status_update_manager.cpp:392] Received status update
acknowledgement (UUID: eba64915-7df2-483d-8982-a9a46a48a81b) for task
service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
f65b163c-0faf-441f-ac14-91739fa4394c-0000
13:33:35.918143 [status_update_manager.cpp:824] Checkpointing ACK for status
update TASK_KILLED (UUID: eba64915-7df2-483d-8982-a9a46a48a81b) for task
service.a3b609b8-27ec-11e6-8044-02c89eb9127e of framework
f65b163c-0faf-441f-ac14-91739fa4394c-0000
...
13:33:39.031054 [slave.cpp:2643] Got registration for executor
'service.a3b609b8-27ec-11e6-8044-02c89eb9127e' of framework
f65b163c-0faf-441f-ac14-91739fa4394c-0000 from executor(1)@127.0.0.1:60083
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
