On 08/10/2018 10:52 AM, Raz Tamir wrote: > > > On Fri, Aug 10, 2018 at 11:30 AM, Jiří Sléžka <jiri.sle...@slu.cz > <mailto:jiri.sle...@slu.cz>> wrote: > > Hi, > > On 08/09/2018 11:25 PM, Raz Tamir wrote: > > You can try using vdsm-client to check if there are running tasks on the > > host using this command: > > > > vdsm-client Host getAllTasksInfo > > thanks for support, I tried this on affected host > > [root@blade01 ~]# vdsm-client Host getAllTasksInfo > vdsm-client: Command Host.getAllTasksInfo with args {} failed: > (code=654, message=Not SPM: ()) > > so I moved into SPM host (blade03 at this time) > > [root@blade03 ~]# vdsm-client Host getAllTasksInfo > {} > > I tried also > > [root@blade01 ~]# vdsm-client Host getAllTasks > {} > [root@blade01 ~]# vdsm-client Host getJobs > {} > > [root@blade03 ~]# vdsm-client Host getAllTasks > {} > [root@blade03 ~]# vdsm-client Host getJobs > {} > > It looks like there are no tasks/jobs running on hosts. It looks like > something is stucked in engine db. > > At the beginning of this issue, there were some race condition where > blade01 had two running tasks/jobs which cannot be finished (network > configuration and activating of host). I deleted this jobs manually in > db via > > SELECT * FROM job ORDER BY start_time DESC; > SELECT DeleteJob('...job_id...'); > SELECT DeleteJob('...job_id...'); > > Maybe it was not wise... but what can I do now to solve this? :-) > > Yes it can be bad to mess with the DB. > My best suugestion at the momet is to try and re-add this host.
I would like to but I cannot even remove this host (Cannot remove Host. Related operation is currently in progress. Please try again later.) :-( But maybe I can try to add it as brand new one? But I am curious what checks do manager and why we cannot see the tasks/jobs/locks/anything in db... Cheers, Jiri > > > > Cheers, Jiri > > > > > > If there are running tasks, you can stop them: > > > > vdsm-client Task stop taskID=xxx-yyy > > > > If there are finished tasks, you can clear them: > > > > vdsm-client Task clear taskID=xxx-yyy > > > > > > On Thu, Aug 9, 2018, 16:15 Jiří Sléžka <jiri.sle...@slu.cz > <mailto:jiri.sle...@slu.cz> > > <mailto:jiri.sle...@slu.cz <mailto:jiri.sle...@slu.cz>>> wrote: > > > > Hello, > > > > still no luck with solving this issue. > > > > I cannot even remove this host. > > > > engine log is now spammed with this messages > > > > 2018-08-09 15:03:08,410+02 INFO > > [org.ovirt.engine.core.bll.lock.InMemoryLockManager] > > (EE-ManagedThreadFactory-engine-Thread-1057) > > [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock > and wait > > lock > > > > 'HostEngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS_INIT]', > > sharedLocks=''}' > > 2018-08-09 15:03:08,446+02 INFO > > [org.ovirt.engine.core.bll.lock.InMemoryLockManager] > > (EE-ManagedThreadFactory-engine-Thread-1057) > > [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock > and wait > > lock > > > > 'HostEngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS_INIT]', > > sharedLocks=''}' > > 2018-08-09 15:03:08,451+02 INFO > > [org.ovirt.engine.core.bll.lock.InMemoryLockManager] > > (EE-ManagedThreadFactory-engine-Thread-1057) > > [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock > and wait > > lock > > > > 'HostEngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS_INIT]', > > sharedLocks=''}' > > 2018-08-09 15:03:08,486+02 INFO > > [org.ovirt.engine.core.bll.lock.InMemoryLockManager] > > (EE-ManagedThreadFactory-engine-Thread-1057) > > [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock > and wait > > lock > > > > 'HostEngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS_INIT]', > > sharedLocks=''}' > > > > btw. oVirt 4.2.5.2-1.el7/CentOS7 > > > > Any tips how to resolve this and at least remove this host > from ovirt? > > > > Cheers, Jiri > > > > > > On 08/06/2018 12:52 PM, Gobinda Das wrote: > > > Can you please post vdsm log which is inside > /var/log/vdsm/vdsm.log ? > > > > > > On Mon, Aug 6, 2018 at 3:51 PM, Jiří Sléžka > <jiri.sle...@slu.cz <mailto:jiri.sle...@slu.cz> > > <mailto:jiri.sle...@slu.cz <mailto:jiri.sle...@slu.cz>> > > > <mailto:jiri.sle...@slu.cz <mailto:jiri.sle...@slu.cz> > <mailto:jiri.sle...@slu.cz <mailto:jiri.sle...@slu.cz>>>> wrote: > > > > > > Hi, > > > > > > no one can help? > > > > > > I still cannot activate this host - error is "Cannot > activate > > Host. > > > Related operation is currently in progress. Please try again > > later." > > > > > > I believe relevant log entrieas are > > > > > > 2018-08-06 12:15:50,398+02 INFO > > > [org.ovirt.engine.core.bll.ActivateVdsCommand] (default > task-426) > > > [9387077a-8276-4a3f-a087-584a10a09b08] Failed to Acquire > Lock > > to object > > > > > > > 'EngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS]', > > > sharedLocks=''}' > > > 2018-08-06 12:15:50,398+02 WARN > > > [org.ovirt.engine.core.bll.ActivateVdsCommand] (default > task-426) > > > [9387077a-8276-4a3f-a087-584a10a09b08] Validation of action > > > 'ActivateVds' failed for user ***my_username***. Reasons: > > > > > > VAR__ACTION__ACTIVATE,VAR__TYPE__HOST,ACTION_TYPE_FAILED_OBJECT_LOCKED > > > > > > > > > when I tried to "reinstall host" I got entry in Events > "Failed > > to update > > > Host blade01" and correlation id. Relevant log entries are > > > > > > > > > cat /var/log/ovirt-engine/engine.log | grep > > > d2b332da-9050-419c-8188-8fdda5e9d807 > > > > > > 2018-08-06 12:08:57,451+02 INFO > > > [org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand] > (default > > > task-425) [d2b332da-9050-419c-8188-8fdda5e9d807] Running > command: > > > InstallVdsCommand internal: false. Entities affected : ID: > > > 786646cd-c9ef-49a8-8aea-56e858dcf202 Type: VDSAction group > > > EDIT_HOST_CONFIGURATION with role type ADMIN > > > 2018-08-06 12:08:57,476+02 INFO > > > > [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] > > > (default task-425) [d2b332da-9050-419c-8188-8fdda5e9d807] > > Failed to > > > Acquire Lock to object > > > > > > > 'EngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS]', > > > sharedLocks=''}' > > > 2018-08-06 12:08:57,476+02 WARN > > > > [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] > > > (default task-425) [d2b332da-9050-419c-8188-8fdda5e9d807] > > Validation of > > > action 'InstallVdsInternal' failed for user > ***my_username***. > > Reasons: > > > ACTION_TYPE_FAILED_OBJECT_LOCKED > > > 2018-08-06 12:08:57,479+02 ERROR > > > [org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand] > (default > > > task-425) [d2b332da-9050-419c-8188-8fdda5e9d807] > > Installation/upgrade of > > > Host '786646cd-c9ef-49a8-8aea-56e858dcf202', 'blade01' > failed: > > Cannot > > > ${action} ${type}. Related operation is currently in > progress. > > Please > > > try again later. > > > 2018-08-06 12:08:57,482+02 INFO > > > [org.ovirt.engine.core.bll.CommandCompensator] (default > task-425) > > > [d2b332da-9050-419c-8188-8fdda5e9d807] Command > > > [id=b74adfbb-c043-4fa9-a163-0a6ea7fba9dc]: Compensating > > > DELETED_OR_UPDATED_ENTITY of > > > org.ovirt.engine.core.common.businessentities.VdsStatic; > snapshot: > > > id=786646cd-c9ef-49a8-8aea-56e858dcf202. > > > 2018-08-06 12:08:57,486+02 ERROR > > > [org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand] > (default > > > task-425) [d2b332da-9050-419c-8188-8fdda5e9d807] Transaction > > rolled-back > > > for command > > 'org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand'. > > > 2018-08-06 12:08:57,491+02 ERROR > > > > > > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > > > (default task-425) [d2b332da-9050-419c-8188-8fdda5e9d807] > > EVENT_ID: > > > USER_FAILED_UPDATE_VDS(105), Failed to update Host > blade01 (User: > > > ***my_username***). > > > > > > How can I unlock this entries? > > > > > > Thanks in advance, > > > > > > Jiri Slezka > > > > > > > > > > > > On 08/03/2018 02:07 PM, Jiří Sléžka wrote: > > > > Hi, > > > > > > > > I have one interesting issue. I was asked by colleague to > > repair his > > > > cluster which is part of our oVirt installation. He > tried to > > > upgrade his > > > > host to gain 4.2 compatibility level but it fails in some > > cases. Never > > > > mind, I checked it, repaired, installed upgrades and > > activate his > > > hosts > > > > with one exception. > > > > > > > > One of hosts was stucked in finishing two tasks. One was > > changing > > > > network configuration, the second was activating of > this host. I > > > have to > > > > manually clear this tasks in database by > > > > > > > > SELECT * FROM job ORDER BY start_time DESC; > > > > SELECT DeleteJob('...job_id...'); > > > > SELECT DeleteJob('...job_id...'); > > > > > > > > Now this tasks are gone but still cannot activate this > host. > > > > > > > > > > > > Error while executing action: > > > > > > > > blade01: > > > > > > > > Cannot activate Host. Related operation is > currently in > > progress. > > > > Please try again later. > > > > > > > > Host is updatet (latest 4.2 repo), rebooted, looks fine... > > > > > > > > In tasks in UI are no tasks found (the same in db)... > > > > > > > > What can I do now? > > > > > > > > Cheers, Jiri > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > Users mailing list -- users@ovirt.org > <mailto:users@ovirt.org> > > <mailto:users@ovirt.org <mailto:users@ovirt.org>> > <mailto:users@ovirt.org <mailto:users@ovirt.org> > > <mailto:users@ovirt.org <mailto:users@ovirt.org>>> > > > > To unsubscribe send an email to users-le...@ovirt.org > <mailto:users-le...@ovirt.org> > > <mailto:users-le...@ovirt.org <mailto:users-le...@ovirt.org>> > > > <mailto:users-le...@ovirt.org > <mailto:users-le...@ovirt.org> <mailto:users-le...@ovirt.org > <mailto:users-le...@ovirt.org>>> > > > > Privacy Statement: > https://www.ovirt.org/site/privacy-policy/ > <https://www.ovirt.org/site/privacy-policy/> > > > <https://www.ovirt.org/site/privacy-policy/ > <https://www.ovirt.org/site/privacy-policy/>> > > > > oVirt Code of Conduct: > > > https://www.ovirt.org/community/about/community-guidelines/ > <https://www.ovirt.org/community/about/community-guidelines/> > > > <https://www.ovirt.org/community/about/community-guidelines/ > <https://www.ovirt.org/community/about/community-guidelines/>> > > > > List Archives: > > > > > > https://lists.ovirt.org/archives/list/users@ovirt.org/message/C2TWKXGOEESXCDJUZ7IBKYDE2APFN4ZB/ > > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/C2TWKXGOEESXCDJUZ7IBKYDE2APFN4ZB/> > > > > > > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/C2TWKXGOEESXCDJUZ7IBKYDE2APFN4ZB/ > > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/C2TWKXGOEESXCDJUZ7IBKYDE2APFN4ZB/>> > > > > > > > > > > > > > > > > _______________________________________________ > > > Users mailing list -- users@ovirt.org > <mailto:users@ovirt.org> <mailto:users@ovirt.org > <mailto:users@ovirt.org>> > > <mailto:users@ovirt.org <mailto:users@ovirt.org> > <mailto:users@ovirt.org <mailto:users@ovirt.org>>> > > > To unsubscribe send an email to users-le...@ovirt.org > <mailto:users-le...@ovirt.org> > > <mailto:users-le...@ovirt.org <mailto:users-le...@ovirt.org>> > > > <mailto:users-le...@ovirt.org > <mailto:users-le...@ovirt.org> <mailto:users-le...@ovirt.org > <mailto:users-le...@ovirt.org>>> > > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > <https://www.ovirt.org/site/privacy-policy/> > > > <https://www.ovirt.org/site/privacy-policy/ > <https://www.ovirt.org/site/privacy-policy/>> > > > oVirt Code of Conduct: > > > https://www.ovirt.org/community/about/community-guidelines/ > <https://www.ovirt.org/community/about/community-guidelines/> > > > <https://www.ovirt.org/community/about/community-guidelines/ > <https://www.ovirt.org/community/about/community-guidelines/>> > > > List Archives: > > > > > > https://lists.ovirt.org/archives/list/users@ovirt.org/message/6H4QYVTTV37IEIA2MTABJ67P43LQ5AGI/ > > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/6H4QYVTTV37IEIA2MTABJ67P43LQ5AGI/> > > > > > > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/6H4QYVTTV37IEIA2MTABJ67P43LQ5AGI/ > > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/6H4QYVTTV37IEIA2MTABJ67P43LQ5AGI/>> > > > > > > > > > > > > > > > -- > > > Thanks, > > > Gobinda > > > > > > _______________________________________________ > > Users mailing list -- users@ovirt.org <mailto:users@ovirt.org> > <mailto:users@ovirt.org > <mailto:users@ovirt.org>> > > To unsubscribe send an email to users-le...@ovirt.org > <mailto:users-le...@ovirt.org> > > <mailto:users-le...@ovirt.org <mailto:users-le...@ovirt.org>> > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > <https://www.ovirt.org/site/privacy-policy/> > > oVirt Code of Conduct: > > https://www.ovirt.org/community/about/community-guidelines/ > <https://www.ovirt.org/community/about/community-guidelines/> > > List Archives: > > > > https://lists.ovirt.org/archives/list/users@ovirt.org/message/BP7SG5BQKZIVR75A6NSCUMGQQBXYL3FJ/ > > <https://lists.ovirt.org/archives/list/users@ovirt.org/message/BP7SG5BQKZIVR75A6NSCUMGQQBXYL3FJ/> > > > > > > > > -- > > > Raz Tamir > Manager, RHV QE
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/HGS5KF22YQD3XSNXHSQARGZHUVDXXZAQ/
