you can do: select command_id,root_command_id from command_entities where root_command_id = '2f8b32d8-fd3c-46c9-90e9-4863d63c0530’;
On Mon, Aug 8, 2022 at 4:19 PM Andrei Verovski <[email protected]> wrote: > > Hi, Benny, > > > select * from command_entities where root_command_id = > 'ed816f9d-e25c-4b58-8c8f-fd0393abda2f’; > -> 0 rows > > select * from command_entities where root_command_id = > '2f8b32d8-fd3c-46c9-90e9-4863d63c0530’; > -> Huge page of smth, have to page out with “more”, is it possible to > truncate it to meaningful value, e.g. just number of rows? > > I can restore this VM from backup copy, bit since it is unmanageable, I can’t > even remove it. > > > > On 8 Aug 2022, at 13:07, Benny Zlotnik <[email protected]> wrote: > > > > you can look up the relevant command by command_id in the > > command_entities table, in your case it would be > > 2f8b32d8-fd3c-46c9-90e9-4863d63c0530 and > > ed816f9d-e25c-4b58-8c8f-fd0393abda2f, there might be more as the log > > is trimmed, so I suggest to look it up with > > select * > > from command_entities > > where root_command_id = '2f8b32d8-fd3c-46c9-90e9-4863d63c0530'; > > > > Then delete the relevant entries and restart ovirt-engine (as they > > might still be present in the cache) > > > > But before doing that, is the command still running? Async commands > > like create snapshot are failed automatically after 50 hours and I > > believe it has already passed > > Also, this manual operation is very intrusive and might have > > unexpected consequences so make sure you have backups. > > > > > > > > > > > > On Mon, Aug 8, 2022 at 9:33 AM Andrei Verovski <[email protected]> wrote: > >> > >> HI, > >> > >> OK, how to properly remove this lock? Right now VM is locked and is > >> unmanageable at all in any way. > >> I suppose its with some SQL commands in Postgres. > >> > >> Thanks. > >> > >> > >>> On 5 Aug 2022, at 11:34, Benny Zlotnik <[email protected]> wrote: > >>> > >>> So based on your logs the lock you are seeing is a memory lock, > >>> unlock_entity.sh can't really help with these. > >>> Also, the job table is used mainly for presentation so removing an > >>> entry from will not help. > >>> > >>> Do you have the logs from when this snapshot operation started, you > >>> can use the correlation id (28353fa0-5e36-4fe8-8609-e74cd1da6d36) to > >>> search? Also, do you have the vdsm logs (SPM at the time and the host > >>> running the VM), same correlation id can be used for this as well > >>> > >>> The table that's used to coordinate this is command_entities, so in > >>> theory removing the entries with this correlation id can help, but I'd > >>> like to see what led to this first > >>> > >>> On Fri, Aug 5, 2022 at 8:37 AM Andrei Verovski <[email protected]> > >>> wrote: > >>>> > >>>> Hi, Benny, > >>>> > >>>> I have sent log on your mailbox, its too big to post here on mailing > >>>> list. > >>>> > >>>> Looks like ghost task is still running, anything else need to be removed > >>>> from Postgres DB? > >>>> > >>>> BTW, frozen dead snapshot is in invalid state, is there any way to get > >>>> rid of it? I think it's actually exists, but due to invalid state its > >>>> not possible to do anything with it. > >>>> > >>>> > >>>> —— > >>>> > >>>> Log file still shows zombie task: > >>>> > >>>> 2022-08-04 20:57:30,145+03 INFO > >>>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > >>>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-24) > >>>> [28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' > >>>> (id: '2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command > >>>> id: 'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' > >>>> type:'CreateLiveSnapshotForVm' to complete > >>>> 2022-08-04 20:57:40,176+03 INFO > >>>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > >>>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-85) > >>>> [28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' > >>>> (id: '2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command > >>>> id: 'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' > >>>> type:'CreateLiveSnapshotForVm' to complete > >>>> 2022-08-04 20:57:50,252+03 INFO > >>>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > >>>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66) > >>>> [28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' > >>>> (id: '2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command > >>>> id: 'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' > >>>> type:'CreateLiveSnapshotForVm' to complete > >>>> > >>>> > >>>>> On 4 Aug 2022, at 19:06, Benny Zlotnik <[email protected]> wrote: > >>>>> > >>>>> can you share the logs after restarting ovirt-engine? > >>>>> > >>>>> On Thu, Aug 4, 2022 at 4:58 PM Andrei Verovski <[email protected]> > >>>>> wrote: > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> > >>>>>> Creating snapshot of one of the VM vailed, and zombie tasks was killed > >>>>>> with: > >>>>>> > >>>>>> su postgres > >>>>>> psql -d engine -U postgres > >>>>>> select * from job order by start_time desc; > >>>>>> > >>>>>> select DeleteJob('UUID_FROZEN_TASK_ID’); > >>>>>> > >>>>>> > >>>>>> However, VM remains in locked state (with lock sign left-below red > >>>>>> “DOWN” arrow in status column of web interface. > >>>>>> > >>>>>> I run: > >>>>>> /usr/share/ovirt-engine/setup/dbutils/unlock_entity.sh -t all > >>>>>> > >>>>>> then rebooted engine VM, still no luck. Can’t do anything with that VM. > >>>>>> > >>>>>> Please advise how to fix. > >>>>>> Thanks in advance. > >>>>>> _______________________________________________ > >>>>>> Users mailing list -- [email protected] > >>>>>> To unsubscribe send an email to [email protected] > >>>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html > >>>>>> oVirt Code of Conduct: > >>>>>> https://www.ovirt.org/community/about/community-guidelines/ > >>>>>> List Archives: > >>>>>> https://lists.ovirt.org/archives/list/[email protected]/message/P2TVMLHC53JWCLDJNK6UXLZ7ZAUOSYFJ/ > >>>>> > >>>> > >>> > >> > > > _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/3CFKATATBWPLVLPI4GVCB4UGX5E5EKBW/

