Hi,
I re-run engine-setup and it was cleared zombie status of this VM.
On 8/8/22 16:27, Benny Zlotnik wrote:
you can do:
select command_id,root_command_id from command_entities where
root_command_id = '2f8b32d8-fd3c-46c9-90e9-4863d63c0530’;
On Mon, Aug 8, 2022 at 4:19 PM Andrei Verovski <[email protected]> wrote:
Hi, Benny,
select * from command_entities where root_command_id =
'ed816f9d-e25c-4b58-8c8f-fd0393abda2f’;
-> 0 rows
select * from command_entities where root_command_id =
'2f8b32d8-fd3c-46c9-90e9-4863d63c0530’;
-> Huge page of smth, have to page out with “more”, is it possible to truncate
it to meaningful value, e.g. just number of rows?
I can restore this VM from backup copy, bit since it is unmanageable, I can’t
even remove it.
On 8 Aug 2022, at 13:07, Benny Zlotnik <[email protected]> wrote:
you can look up the relevant command by command_id in the
command_entities table, in your case it would be
2f8b32d8-fd3c-46c9-90e9-4863d63c0530 and
ed816f9d-e25c-4b58-8c8f-fd0393abda2f, there might be more as the log
is trimmed, so I suggest to look it up with
select *
from command_entities
where root_command_id = '2f8b32d8-fd3c-46c9-90e9-4863d63c0530';
Then delete the relevant entries and restart ovirt-engine (as they
might still be present in the cache)
But before doing that, is the command still running? Async commands
like create snapshot are failed automatically after 50 hours and I
believe it has already passed
Also, this manual operation is very intrusive and might have
unexpected consequences so make sure you have backups.
On Mon, Aug 8, 2022 at 9:33 AM Andrei Verovski <[email protected]> wrote:
HI,
OK, how to properly remove this lock? Right now VM is locked and is
unmanageable at all in any way.
I suppose its with some SQL commands in Postgres.
Thanks.
On 5 Aug 2022, at 11:34, Benny Zlotnik <[email protected]> wrote:
So based on your logs the lock you are seeing is a memory lock,
unlock_entity.sh can't really help with these.
Also, the job table is used mainly for presentation so removing an
entry from will not help.
Do you have the logs from when this snapshot operation started, you
can use the correlation id (28353fa0-5e36-4fe8-8609-e74cd1da6d36) to
search? Also, do you have the vdsm logs (SPM at the time and the host
running the VM), same correlation id can be used for this as well
The table that's used to coordinate this is command_entities, so in
theory removing the entries with this correlation id can help, but I'd
like to see what led to this first
On Fri, Aug 5, 2022 at 8:37 AM Andrei Verovski <[email protected]> wrote:
Hi, Benny,
I have sent log on your mailbox, its too big to post here on mailing list.
Looks like ghost task is still running, anything else need to be removed from
Postgres DB?
BTW, frozen dead snapshot is in invalid state, is there any way to get rid of
it? I think it's actually exists, but due to invalid state its not possible to
do anything with it.
——
Log file still shows zombie task:
2022-08-04 20:57:30,145+03 INFO
[org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-24)
[28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' (id:
'2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command id:
'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' type:'CreateLiveSnapshotForVm' to
complete
2022-08-04 20:57:40,176+03 INFO
[org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-85)
[28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' (id:
'2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command id:
'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' type:'CreateLiveSnapshotForVm' to
complete
2022-08-04 20:57:50,252+03 INFO
[org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback]
(EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66)
[28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' (id:
'2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command id:
'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' type:'CreateLiveSnapshotForVm' to
complete
On 4 Aug 2022, at 19:06, Benny Zlotnik <[email protected]> wrote:
can you share the logs after restarting ovirt-engine?
On Thu, Aug 4, 2022 at 4:58 PM Andrei Verovski <[email protected]> wrote:
Hi,
Creating snapshot of one of the VM vailed, and zombie tasks was killed with:
su postgres
psql -d engine -U postgres
select * from job order by start_time desc;
select DeleteJob('UUID_FROZEN_TASK_ID’);
However, VM remains in locked state (with lock sign left-below red “DOWN” arrow
in status column of web interface.
I run:
/usr/share/ovirt-engine/setup/dbutils/unlock_entity.sh -t all
then rebooted engine VM, still no luck. Can’t do anything with that VM.
Please advise how to fix.
Thanks in advance.
_______________________________________________
Users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/[email protected]/message/P2TVMLHC53JWCLDJNK6UXLZ7ZAUOSYFJ/
_______________________________________________
Users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/[email protected]/message/4AR2U3UY7JY4EMLPOYUZWAD64ZUXG2EG/