Hi,
We have 4 hosts on Version 4.2.1.7-1.el7.centos, but we have got an issue
with vdsm on our SPM which is causing some issues and we cannot migrate any
VM's from that host.
If I run 'vdsm-client Host getAllTasksStatuses' on the host, it returns 4
tasks, message: running job 1 of 1, code:0, ........ taskState:running for
all 4 tasks.
However we have no operations in the engine that should be running (not a
hosted engine), just a stuck task that logs every minute or so 'Clearing
asynchronous task Unknown that started at Thu Dec 06 14:42:03 GMT 2018'.
If I try to manually stop the task, 'vdsm-client Task stop taskID=' we get
'code=411, message=Task is aborted: u'task guid' - code 411'. So then If I
try and clean the task it can't because its in a running state: vdsm-client:
Command Task.clear with args {'taskID': 'task guid'} failed: (code=410,
message=Operation is not allowed in this task state: ("can't clean in state
running",))
Attached is part of the (sanitised) VDSM log that is recurring. VDSM
version is: vdsm-client-4.20.17-1.el7.centos.noarch
Does anyone have any ideas other than shutting down all the VM's on that
host and restarting it? Not sure if I can restart the VDSM service with
VM's running?
Thanks,
Nick
2018-12-10 09:20:03,136+0000 INFO (jsonrpc/4) [storage.TaskManager.Task]
(Task='00000000-0000-0000-0000-000000000000') aborting: Task is aborted:
u'Operation is not allowed in this task state: ("can\'t clean in state
running",)' - code 100 (task:1181)
2018-12-10 09:20:03,136+0000 ERROR (jsonrpc/4) [storage.Dispatcher] FINISH
clearTask error=Operation is not allowed in this task state: ("can't clean in
state running",) (dispatcher:82)
2018-12-10 09:20:03,137+0000 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call
Task.clear failed (error 410) in 0.00 seconds (__init__:573)
2018-12-10 09:20:03,143+0000 INFO (jsonrpc/3) [vdsm.api] START
clearTask(taskID=u'00000000-0000-0000-0000-000000000000', spUUID=None,
options=None) from=::ffff:ovirt-engine-ip,42176,
task_id=00000000-0000-0000-0000-000000000000 (api:46)
2018-12-10 09:20:03,144+0000 INFO (jsonrpc/3) [vdsm.api] FINISH clearTask
error=Operation is not allowed in this task state: ("can't clean in state
running",) from=::ffff:ovirt-engine-ip,42176,
task_id=00000000-0000-0000-0000-000000000000 (api:50)
2018-12-10 09:20:03,144+0000 ERROR (jsonrpc/3) [storage.TaskManager.Task]
(Task='00000000-0000-0000-0000-000000000000') Unexpected error (task:875)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in
_run
return fn(*args, **kargs)
File "<string>", line 2, in clearTask
File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method
ret = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2280, in
clearTask
return self.taskMng.clearTask(taskID=taskID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/taskManager.py", line
175, in clearTask
t.clean()
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1053, in
clean
raise se.TaskStateError("can't clean in state %s" % self.state)
TaskStateError: Operation is not allowed in this task state: ("can't clean in
state running",)
2018-12-10 09:20:03,144+0000 INFO (jsonrpc/3) [storage.TaskManager.Task]
(Task='00000000-0000-0000-0000-000000000000') aborting: Task is aborted:
u'Operation is not allowed in this task state: ("can\'t clean in state
running",)' - code 100 (task:1181)
2018-12-10 09:20:03,144+0000 ERROR (jsonrpc/3) [storage.Dispatcher] FINISH
clearTask error=Operation is not allowed in this task state: ("can't clean in
state running",) (dispatcher:82)
2018-12-10 09:20:03,145+0000 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call
Task.clear failed (error 410) in 0.00 seconds (__init__:573)
2018-12-10 09:20:03,151+0000 INFO (jsonrpc/0) [vdsm.api] START
clearTask(taskID=u'00000000-0000-0000-0000-000000000000', spUUID=None,
options=None) from=::ffff:ovirt-engine-ip,42176,
task_id=00000000-0000-0000-0000-000000000000 (api:46)
2018-12-10 09:20:03,152+0000 INFO (jsonrpc/0) [vdsm.api] FINISH clearTask
error=Operation is not allowed in this task state: ("can't clean in state
running",) from=::ffff:ovirt-engine-ip,42176,
task_id=00000000-0000-0000-0000-000000000000 (api:50)
2018-12-10 09:20:03,152+0000 ERROR (jsonrpc/0) [storage.TaskManager.Task]
(Task='00000000-0000-0000-0000-000000000000') Unexpected error (task:875)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in
_run
return fn(*args, **kargs)
File "<string>", line 2, in clearTask
File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method
ret = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2280, in
clearTask
return self.taskMng.clearTask(taskID=taskID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/taskManager.py", line
175, in clearTask
t.clean()
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1053, in
clean
raise se.TaskStateError("can't clean in state %s" % self.state)
TaskStateError: Operation is not allowed in this task state: ("can't clean in
state running",)
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/Z55XFXGGLVE37SIK6DXKZIR3M4SNS6RK/
