no, DeleteJob deletes from the job table, you can use DeleteCommandEntity(uuid)[1]
[1] https://github.com/oVirt/ovirt-engine/blob/fbd5851b9de889fb88df6f10310ea9051c75beb9/packaging/dbscripts/command_entities_sp.sql#L247 On Mon, Aug 8, 2022 at 4:58 PM Andrei Verovski <[email protected]> wrote: > > Hi, Benny, > > 3 records found. > > engine=# select command_id,root_command_id from command_entities where > root_command_id = '2f8b32d8-fd3c-46c9-90e9-4863d63c0530'; > command_id | root_command_id > --------------------------------------+-------------------------------------- > 2f8b32d8-fd3c-46c9-90e9-4863d63c0530 | 2f8b32d8-fd3c-46c9-90e9-4863d63c0530 > 3c09e1ce-1e49-4d03-82dd-2844fb9dc39f | 2f8b32d8-fd3c-46c9-90e9-4863d63c0530 > ed816f9d-e25c-4b58-8c8f-fd0393abda2f | 2f8b32d8-fd3c-46c9-90e9-4863d63c0530 > > > Now this ? > select DeleteJob('2f8b32d8-fd3c-46c9-90e9-4863d63c0530'); > > > > On 8 Aug 2022, at 16:27, Benny Zlotnik <[email protected]> wrote: > > > > you can do: > > select command_id,root_command_id from command_entities where > > root_command_id = '2f8b32d8-fd3c-46c9-90e9-4863d63c0530’; > > > > On Mon, Aug 8, 2022 at 4:19 PM Andrei Verovski <[email protected]> wrote: > >> > >> Hi, Benny, > >> > >> > >> select * from command_entities where root_command_id = > >> 'ed816f9d-e25c-4b58-8c8f-fd0393abda2f’; > >> -> 0 rows > >> > >> select * from command_entities where root_command_id = > >> '2f8b32d8-fd3c-46c9-90e9-4863d63c0530’; > >> -> Huge page of smth, have to page out with “more”, is it possible to > >> truncate it to meaningful value, e.g. just number of rows? > >> > >> I can restore this VM from backup copy, bit since it is unmanageable, I > >> can’t even remove it. > >> > >> > >>> On 8 Aug 2022, at 13:07, Benny Zlotnik <[email protected]> wrote: > >>> > >>> you can look up the relevant command by command_id in the > >>> command_entities table, in your case it would be > >>> 2f8b32d8-fd3c-46c9-90e9-4863d63c0530 and > >>> ed816f9d-e25c-4b58-8c8f-fd0393abda2f, there might be more as the log > >>> is trimmed, so I suggest to look it up with > >>> select * > >>> from command_entities > >>> where root_command_id = '2f8b32d8-fd3c-46c9-90e9-4863d63c0530'; > >>> > >>> Then delete the relevant entries and restart ovirt-engine (as they > >>> might still be present in the cache) > >>> > >>> But before doing that, is the command still running? Async commands > >>> like create snapshot are failed automatically after 50 hours and I > >>> believe it has already passed > >>> Also, this manual operation is very intrusive and might have > >>> unexpected consequences so make sure you have backups. > >>> > >>> > >>> > >>> > >>> > >>> On Mon, Aug 8, 2022 at 9:33 AM Andrei Verovski <[email protected]> > >>> wrote: > >>>> > >>>> HI, > >>>> > >>>> OK, how to properly remove this lock? Right now VM is locked and is > >>>> unmanageable at all in any way. > >>>> I suppose its with some SQL commands in Postgres. > >>>> > >>>> Thanks. > >>>> > >>>> > >>>>> On 5 Aug 2022, at 11:34, Benny Zlotnik <[email protected]> wrote: > >>>>> > >>>>> So based on your logs the lock you are seeing is a memory lock, > >>>>> unlock_entity.sh can't really help with these. > >>>>> Also, the job table is used mainly for presentation so removing an > >>>>> entry from will not help. > >>>>> > >>>>> Do you have the logs from when this snapshot operation started, you > >>>>> can use the correlation id (28353fa0-5e36-4fe8-8609-e74cd1da6d36) to > >>>>> search? Also, do you have the vdsm logs (SPM at the time and the host > >>>>> running the VM), same correlation id can be used for this as well > >>>>> > >>>>> The table that's used to coordinate this is command_entities, so in > >>>>> theory removing the entries with this correlation id can help, but I'd > >>>>> like to see what led to this first > >>>>> > >>>>> On Fri, Aug 5, 2022 at 8:37 AM Andrei Verovski <[email protected]> > >>>>> wrote: > >>>>>> > >>>>>> Hi, Benny, > >>>>>> > >>>>>> I have sent log on your mailbox, its too big to post here on mailing > >>>>>> list. > >>>>>> > >>>>>> Looks like ghost task is still running, anything else need to be > >>>>>> removed from Postgres DB? > >>>>>> > >>>>>> BTW, frozen dead snapshot is in invalid state, is there any way to get > >>>>>> rid of it? I think it's actually exists, but due to invalid state its > >>>>>> not possible to do anything with it. > >>>>>> > >>>>>> > >>>>>> —— > >>>>>> > >>>>>> Log file still shows zombie task: > >>>>>> > >>>>>> 2022-08-04 20:57:30,145+03 INFO > >>>>>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > >>>>>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-24) > >>>>>> [28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' > >>>>>> (id: '2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command > >>>>>> id: 'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' > >>>>>> type:'CreateLiveSnapshotForVm' to complete > >>>>>> 2022-08-04 20:57:40,176+03 INFO > >>>>>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > >>>>>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-85) > >>>>>> [28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' > >>>>>> (id: '2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command > >>>>>> id: 'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' > >>>>>> type:'CreateLiveSnapshotForVm' to complete > >>>>>> 2022-08-04 20:57:50,252+03 INFO > >>>>>> [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] > >>>>>> (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-66) > >>>>>> [28353fa0-5e36-4fe8-8609-e74cd1da6d36] Command 'CreateSnapshotForVm' > >>>>>> (id: '2f8b32d8-fd3c-46c9-90e9-4863d63c0530') waiting on child command > >>>>>> id: 'ed816f9d-e25c-4b58-8c8f-fd0393abda2f' > >>>>>> type:'CreateLiveSnapshotForVm' to complete > >>>>>> > >>>>>> > >>>>>>> On 4 Aug 2022, at 19:06, Benny Zlotnik <[email protected]> wrote: > >>>>>>> > >>>>>>> can you share the logs after restarting ovirt-engine? > >>>>>>> > >>>>>>> On Thu, Aug 4, 2022 at 4:58 PM Andrei Verovski <[email protected]> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> Hi, > >>>>>>>> > >>>>>>>> > >>>>>>>> Creating snapshot of one of the VM vailed, and zombie tasks was > >>>>>>>> killed with: > >>>>>>>> > >>>>>>>> su postgres > >>>>>>>> psql -d engine -U postgres > >>>>>>>> select * from job order by start_time desc; > >>>>>>>> > >>>>>>>> select DeleteJob('UUID_FROZEN_TASK_ID’); > >>>>>>>> > >>>>>>>> > >>>>>>>> However, VM remains in locked state (with lock sign left-below red > >>>>>>>> “DOWN” arrow in status column of web interface. > >>>>>>>> > >>>>>>>> I run: > >>>>>>>> /usr/share/ovirt-engine/setup/dbutils/unlock_entity.sh -t all > >>>>>>>> > >>>>>>>> then rebooted engine VM, still no luck. Can’t do anything with that > >>>>>>>> VM. > >>>>>>>> > >>>>>>>> Please advise how to fix. > >>>>>>>> Thanks in advance. > >>>>>>>> _______________________________________________ > >>>>>>>> Users mailing list -- [email protected] > >>>>>>>> To unsubscribe send an email to [email protected] > >>>>>>>> Privacy Statement: https://www.ovirt.org/privacy-policy.html > >>>>>>>> oVirt Code of Conduct: > >>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ > >>>>>>>> List Archives: > >>>>>>>> https://lists.ovirt.org/archives/list/[email protected]/message/P2TVMLHC53JWCLDJNK6UXLZ7ZAUOSYFJ/ > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > > > _______________________________________________ Users mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/XVR7AFNQZCHEJKUVZHIISR7XMRUZ7IEF/

