You'd better attaching whole file of supervisor log and worker logs if you really don't mind. (Double check on personal or credential informations.)
- Jungtaek Lim (HeartSaVioR) 2017년 3월 2일 (목) 오후 6:42, shawn.du <shawn...@neulion.com.cn>님이 작성: > Hi Jungtaek Lim, > > This is the log: > 2017-03-02 09:38:11.607 SLOT_6700 o.a.s.d.s.Slot [WARN] SLOT 6700: > Assignment Changed from LocalAssignment(topology_id:test-24-1488446190, > executors:[ExecutorInfo(task_start:8, task_end:8), > ExecutorInfo(task_start:12, task_end:12), ExecutorInfo(task_start:2, > task_end:2), ExecutorInfo(task_start:18, task_end:18), > ExecutorInfo(task_start:6, task_end:6), ExecutorInfo(task_start:16, > task_end:16), ExecutorInfo(task_start:10, task_end:10), > ExecutorInfo(task_start:14, task_end:14), ExecutorInfo(task_start:4, > task_end:4), ExecutorInfo(task_start:9, task_end:9), > ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:3, > task_end:3), ExecutorInfo(task_start:1, task_end:1), > ExecutorInfo(task_start:11, task_end:11), ExecutorInfo(task_start:13, > task_end:13), ExecutorInfo(task_start:5, task_end:5), > ExecutorInfo(task_start:15, task_end:15), ExecutorInfo(task_start:17, > task_end:17)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, > cpu:0.0)) to null > 2017-03-02 09:38:11.607 SLOT_6700 o.a.s.d.s.Container [INFO] Killing > 7c3845aa-4b43-4a43-b351-f55d64e2c11b:06658ce5-c083-40ce-a212-97f0d298b479 > 2017-03-02 09:38:12.990 Thread-76 o.a.s.d.s.BasicContainer [INFO] Worker > Process 06658ce5-c083-40ce-a212-97f0d298b479 exited with code: 20 > 2017-03-02 09:38:14.610 SLOT_6700 o.a.s.d.s.Slot [INFO] STATE RUNNING > msInState: 1296286 topo:test-24-1488446190 > worker:06658ce5-c083-40ce-a212-97f0d298b479 -> KILL msInState: 0 > topo:test-24-1488446190 worker:06658ce5-c083-40ce-a212-97f0d298b479 > 2017-03-02 09:38:14.610 SLOT_6700 o.a.s.d.s.Container [INFO] GET > worker-user for 06658ce5-c083-40ce-a212-97f0d298b479 > 2017-03-02 09:38:14.613 SLOT_6700 o.a.s.d.s.Slot [WARN] SLOT 6700 all > processes are dead... > 2017-03-02 09:38:14.613 SLOT_6700 o.a.s.d.s.Container [INFO] Cleaning up > 7c3845aa-4b43-4a43-b351-f55d64e2c11b:06658ce5-c083-40ce-a212-97f0d298b479 > 2017-03-02 09:38:14.614 SLOT_6700 o.a.s.d.s.Container [INFO] GET > worker-user for 06658ce5-c083-40ce-a212-97f0d298b479 > 2017-03-02 09:38:14.614 SLOT_6700 o.a.s.d.s.AdvancedFSOps [INFO] Deleting > path > /home/ec2-user/var/storm-local/workers/06658ce5-c083-40ce-a212-97f0d298b479/pids/2913 > 2017-03-02 09:38:14.614 SLOT_6700 o.a.s.d.s.AdvancedFSOps [INFO] Deleting > path > /home/ec2-user/var/storm-local/workers/06658ce5-c083-40ce-a212-97f0d298b479/heartbeats > 2017-03-02 09:38:14.617 SLOT_6700 o.a.s.d.s.AdvancedFSOps [INFO] Deleting > path > /home/ec2-user/var/storm-local/workers/06658ce5-c083-40ce-a212-97f0d298b479/pids > 2017-03-02 09:38:14.617 SLOT_6700 o.a.s.d.s.AdvancedFSOps [INFO] Deleting > path > /home/ec2-user/var/storm-local/workers/06658ce5-c083-40ce-a212-97f0d298b479/tmp > 2017-03-02 09:38:14.617 SLOT_6700 o.a.s.d.s.AdvancedFSOps [INFO] Deleting > path > /home/ec2-user/var/storm-local/workers/06658ce5-c083-40ce-a212-97f0d298b479 > 2017-03-02 09:38:14.617 SLOT_6700 o.a.s.d.s.Container [INFO] REMOVE > worker-user 06658ce5-c083-40ce-a212-97f0d298b479 > 2017-03-02 09:38:14.618 SLOT_6700 o.a.s.d.s.AdvancedFSOps [INFO] Deleting > path > /home/ec2-user/var/storm-local/workers-users/06658ce5-c083-40ce-a212-97f0d298b479 > 2017-03-02 09:38:14.618 SLOT_6700 o.a.s.d.s.BasicContainer [INFO] Removed > Worker ID 06658ce5-c083-40ce-a212-97f0d298b479 > 2017-03-02 09:38:14.618 SLOT_6700 o.a.s.l.AsyncLocalizer [INFO] Released > blob reference test-24-1488446190 6700 Cleaning up BLOB references... > 2017-03-02 09:38:14.619 SLOT_6700 o.a.s.l.AsyncLocalizer [INFO] Released > blob reference test-24-1488446190 6700 Cleaning up basic files... > 2017-03-02 09:38:14.619 SLOT_6700 o.a.s.d.s.AdvancedFSOps [INFO] Deleting > path /home/ec2-user/var/storm-local/supervisor/stormdist/test-24-1488446190 > 2017-03-02 09:38:14.629 SLOT_6700 o.a.s.d.s.Slot [INFO] STATE KILL > msInState: 19 topo:test-24-1488446190 worker:null -> EMPTY msInState: 0 > 2017-03-02 09:38:14.629 SLOT_6700 o.a.s.d.s.Slot [INFO] SLOT 6700: > Changing current assignment from > LocalAssignment(topology_id:test-24-1488446190, > executors:[ExecutorInfo(task_start:8, task_end:8), > ExecutorInfo(task_start:12, task_end:12), ExecutorInfo(task_start:2, > task_end:2), ExecutorInfo(task_start:18, task_end:18), > ExecutorInfo(task_start:6, task_end:6), ExecutorInfo(task_start:16, > task_end:16), ExecutorInfo(task_start:10, task_end:10), > ExecutorInfo(task_start:14, task_end:14), ExecutorInfo(task_start:4, > task_end:4), ExecutorInfo(task_start:9, task_end:9), > ExecutorInfo(task_start:7, task_end:7), ExecutorInfo(task_start:3, > task_end:3), ExecutorInfo(task_start:1, task_end:1), > ExecutorInfo(task_start:11, task_end:11), ExecutorInfo(task_start:13, > task_end:13), ExecutorInfo(task_start:5, task_end:5), > ExecutorInfo(task_start:15, task_end:15), ExecutorInfo(task_start:17, > task_end:17)], resources:WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, > cpu:0.0)) to null > > Thanks > Shawn > On 03/2/2017 16:45，Jungtaek Lim<kabh...@gmail.com> <kabh...@gmail.com> > wrote： > > Could you attach tail of your worker log and small part of supervisor log > for killing that worker? > > Thanks, > Jungtaek Lim (HeartSaVioR) > > 2017년 3월 2일 (목) 오후 2:10, shawn.du <shawn...@neulion.com.cn>님이 작성: > > Hi, > > https://issues.apache.org/jira/browse/STORM-2176 > I am testing this with storm 1.0.3. but it seems that hook shutdown > method is never called when I killed the topology. please confirm. > > Thanks > Shawn > >
