Here is the vdsm log of my ovirt-node01:
fda6e0ee-33e9-4eb2-b724-34f7a5492e83::ERROR::2014-11-12 16:13:20,071::sp::330::Storage.StoragePool::(startSpm) failed: Storage domain does not exist: ('6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1',) fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,071::sp::336::Storage.StoragePool::(_shutDownUpgrade) Shutting down upgrade process fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,071::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d`ReqID=`7ec0dd55-0b56-4d8a-bc21-5aa6fe2ec373`::Request was made in '/usr/share/vdsm/storage/sp.py' line '338' at '_shutDownUpgrade' fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,071::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d' for lock type 'exclusive' fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,072::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d' is free. Now locking as 'exclusive' (1 active user) fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,072::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d`ReqID=`7ec0dd55-0b56-4d8a-bc21-5aa6fe2ec373`::Granted request fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,072::resourceManager::198::ResourceManager.Request::(__init__) ResName=`Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1`ReqID=`a6bd57b0-5ac0-459a-a4c2-2a5a58c4b1ea`::Request was made in '/usr/share/vdsm/storage/sp.py' line '358' at '_shutDownUpgrade' fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,073::resourceManager::542::ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1' for lock type 'exclusive' fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,073::resourceManager::601::ResourceManager::(registerResource) Resource 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1' is free. Now locking as 'exclusive' (1 active user) fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,073::resourceManager::238::ResourceManager.Request::(grant) ResName=`Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1`ReqID=`a6bd57b0-5ac0-459a-a4c2-2a5a58c4b1ea`::Granted request fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,073::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1' fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,073::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1' (0 active users) fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,074::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1' is free, finding out if anyone is waiting for it. fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,074::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1', Clearing records. fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,074::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d' fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,074::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d' (0 active users) fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,075::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d' is free, finding out if anyone is waiting for it. fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,075::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_b384b3da-02a6-44f3-a3f6-56751ce8c26d', Clearing records. fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,075::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,075::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,076::persistentDict::299::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=RaidVolBGluster', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=HP_Proliant_DL18 0G6', 'POOL_DOMAINS=6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1:Active,abc51e26-7175-4b38-b3a8-95c6928fbc2b:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=0', 'POOL_UUID=b384b3da-02a6-44f3-a3f6-56751ce8c26d', 'REMOTE_PATH=127.0.0.1:/RaidVolB', 'ROLE=Master', 'SDUUID=abc51e26-7175-4b38-b3a8-95c6928fbc2b', 'TYPE=GLUSTERFS', 'VERSION=3', '_SHA_CKSUM=9b444340971e2506b55bfe1d4 a662fde62adbeaa'] fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,082::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction fda6e0ee-33e9-4eb2-b724-34f7a5492e83::INFO::2014-11-12 16:13:20,082::clusterlock::279::SANLock::(release) Releasing cluster lock for domain abc51e26-7175-4b38-b3a8-95c6928fbc2b Thread-28::DEBUG::2014-11-12 16:13:20,270::BindingXMLRPC::1067::vds::(wrapper) client [192.168.150.8]::call volumesList with () {} flowID [58a6ac1e] Thread-28::DEBUG::2014-11-12 16:13:20,403::BindingXMLRPC::1074::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'RaidVolB': {'transportType': ['TCP'], 'uuid': 'd46619e9-9368-4e82-bf3a-a2377b6e85e4', 'bricks': ['ovirt-node01.foobar.net:/raidvol/volb', 'ovirt-node02.foobar.net:/raidvol/volb'], 'volume Name': 'RaidVolB', 'volumeType': 'REPLICATE', 'replicaCount': '2', 'brickCount': '2', 'distCount': '2', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [{'name': 'ovirt-node01.foobar.net:/raidvol/volb', 'hostUuid': 'de2a515f-c728-449d-b91c-d80cabe0539f'}, {'name': 'ovirt-node02.foobar.net:/raidvol/volb', 'hostUuid': '7540f5c0-c4ba-4 520-bdf1-3115c10d0eea'}], 'options': {'user.cifs': 'disable', 'storage.owner-gid': '36', 'storage.owner-uid': '36', 'nfs.disable': 'on', 'auth.allow': '*'}}}} fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,460::clusterlock::289::SANLock::(release) Cluster lock for domain abc51e26-7175-4b38-b3a8-95c6928fbc2b successfully released fda6e0ee-33e9-4eb2-b724-34f7a5492e83::ERROR::2014-11-12 16:13:20,460::task::866::TaskManager.Task::(_setError) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::Unexpected error Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 873, in _run return fn(*args, **kargs) File "/usr/share/vdsm/storage/task.py", line 334, in run return self.cmd(*self.argslist, **self.argsdict) File "/usr/share/vdsm/storage/sp.py", line 296, in startSpm self._updateDomainsRole() File "/usr/share/vdsm/storage/securable.py", line 75, in wrapper return method(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 205, in _updateDomainsRole domain = sdCache.produce(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 98, in produce domain.getRealDomain() File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce domain = self._findDomain(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain dom = findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: ('6d882c77-cdbc-48ef-ae21-1a6d45e7f8a1',) fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,461::task::885::TaskManager.Task::(_run) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::Task._run: fda6e0ee-33e9-4eb2-b724-34f7a5492e83 () {} failed - stopping task fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,461::task::1211::TaskManager.Task::(stop) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::stopping in state running (force False) fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,461::task::990::TaskManager.Task::(_decref) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::ref 1 aborting True fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,461::task::916::TaskManager.Task::(_runJobs) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::aborting: Task is aborted: 'Storage domain does not exist' - code 358 fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,462::task::990::TaskManager.Task::(_decref) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::ref 0 aborting True fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,462::task::925::TaskManager.Task::(_doAbort) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::Task._doAbort: force False fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,462::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,462::task::595::TaskManager.Task::(_updateState) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::moving from state running -> state aborting fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,462::task::550::TaskManager.Task::(__state_aborting) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::_aborting: recover policy auto fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,463::task::595::TaskManager.Task::(_updateState) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::moving from state aborting -> state racquiring fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,463::task::595::TaskManager.Task::(_updateState) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::moving from state racquiring -> state recovering fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,463::task::798::TaskManager.Task::(_recover) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::_recover fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,463::task::805::TaskManager.Task::(_recover) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::running recovery None fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,463::task::786::TaskManager.Task::(_recoverDone) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::Recover Done: state recovering fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,463::task::595::TaskManager.Task::(_updateState) Task=`fda6e0ee-33e9-4eb2-b724-34f7a5492e83`::moving from state recovering -> state recovered fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,464::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d': < ResourceRef 'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d', isValid: 'True' obj: 'None'>} fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,464::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,464::resourceManager::616::ResourceManager::(releaseResource) Trying to release resource 'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d' fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,464::resourceManager::635::ResourceManager::(releaseResource) Released resource 'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d' (0 active users) fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,465::resourceManager::641::ResourceManager::(releaseResource) Resource 'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d' is free, finding out if anyone is waiting for it. fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,465::resourceManager::649::ResourceManager::(releaseResource) No one is waiting for resource 'Storage.b384b3da-02a6-44f3-a3f6-56751ce8c26d', Clearing records. fda6e0ee-33e9-4eb2-b724-34f7a5492e83::DEBUG::2014-11-12 16:13:20,465::threadPool::57::Misc.ThreadPool::(setRunningTask) Number of running tasks: 0 Thread-28::DEBUG::2014-11-12 16:13:20,940::BindingXMLRPC::251::vds::(wrapper) client [192.168.150.8] flowID [4c2997b8] Thread-28::DEBUG::2014-11-12 16:13:20,941::task::595::TaskManager.Task::(_updateState) Task=`60c56406-16d3-4dcd-986f-41f2bc1f78cb`::moving from state init -> state preparing Thread-28::INFO::2014-11-12 16:13:20,941::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='fda6e0ee-33e9-4eb2-b724-34f7a5492e83', spUUID=None, options=None) Thread-28::DEBUG::2014-11-12 16:13:20,941::taskManager::93::TaskManager::(getTaskStatus) Entry. taskID: fda6e0ee-33e9-4eb2-b724-34f7a5492e83 Thread-28::DEBUG::2014-11-12 16:13:20,941::taskManager::96::TaskManager::(getTaskStatus) Return. Response: {'code': 358, 'message': 'Storage domain does not exist', 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': 'fda6e0ee-33e9-4eb2-b724-34f7a5492e83'} Thread-28::INFO::2014-11-12 16:13:20,941::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 358, 'message': 'Storage domain does not exist', 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': 'fda6e0ee-33e9-4eb2-b724-34f7a5492e83'}} Thread-28::DEBUG::2014-11-12 16:13:20,942::task::1185::TaskManager.Task::(prepare) Task=`60c56406-16d3-4dcd-986f-41f2bc1f78cb`::finished: {'taskStatus': {'code': 358, 'message': 'Storage domain does not exist', 'taskState': 'finished', 'taskResult': 'cleanSuccess', 'taskID': 'fda6e0ee-33e9-4eb2-b724-34f7a5492e83'}} Thread-28::DEBUG::2014-11-12 16:13:20,942::task::595::TaskManager.Task::(_updateState) Task=`60c56406-16d3-4dcd-986f-41f2bc1f78cb`::moving from state preparing -> state finished Thread-28::DEBUG::2014-11-12 16:13:20,942::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-28::DEBUG::2014-11-12 16:13:20,942::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-28::DEBUG::2014-11-12 16:13:20,942::task::990::TaskManager.Task::(_decref) Task=`60c56406-16d3-4dcd-986f-41f2bc1f78cb`::ref 0 aborting False Thread-28::DEBUG::2014-11-12 16:13:20,951::BindingXMLRPC::251::vds::(wrapper) client [192.168.150.8] flowID [4c2997b8] Thread-28::DEBUG::2014-11-12 16:13:20,952::task::595::TaskManager.Task::(_updateState) Task=`a421f847-c259-4bdf-929a-b2df3568e881`::moving from state init -> state preparing Thread-28::INFO::2014-11-12 16:13:20,952::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='b384b3da-02a6-44f3-a3f6-56751ce8c26d', options=None) Thread-28::INFO::2014-11-12 16:13:20,956::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} Thread-28::DEBUG::2014-11-12 16:13:20,957::task::1185::TaskManager.Task::(prepare) Task=`a421f847-c259-4bdf-929a-b2df3568e881`::finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} Thread-28::DEBUG::2014-11-12 16:13:20,957::task::595::TaskManager.Task::(_updateState) Task=`a421f847-c259-4bdf-929a-b2df3568e881`::moving from state preparing -> state finished Thread-28::DEBUG::2014-11-12 16:13:20,957::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-28::DEBUG::2014-11-12 16:13:20,957::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-28::DEBUG::2014-11-12 16:13:20,957::task::990::TaskManager.Task::(_decref) Task=`a421f847-c259-4bdf-929a-b2df3568e881`::ref 0 aborting False Thread-28::DEBUG::2014-11-12 16:13:21,006::BindingXMLRPC::251::vds::(wrapper) client [192.168.150.8] flowID [4c2997b8] Thread-28::DEBUG::2014-11-12 16:13:21,006::task::595::TaskManager.Task::(_updateState) Task=`8c502838-deb0-41a6-a981-8b34acdb71c9`::moving from state init -> state preparing Thread-28::INFO::2014-11-12 16:13:21,006::logUtils::44::dispatcher::(wrapper) Run and protect: clearTask(taskID='fda6e0ee-33e9-4eb2-b724-34f7a5492e83', spUUID=None, options=None) Thread-28::DEBUG::2014-11-12 16:13:21,007::taskManager::161::TaskManager::(clearTask) Entry. taskID: fda6e0ee-33e9-4eb2-b724-34f7a5492e83 Thread-28::DEBUG::2014-11-12 16:13:21,007::taskManager::166::TaskManager::(clearTask) Return. Thread-28::INFO::2014-11-12 16:13:21,007::logUtils::47::dispatcher::(wrapper) Run and protect: clearTask, Return response: None Thread-28::DEBUG::2014-11-12 16:13:21,007::task::1185::TaskManager.Task::(prepare) Task=`8c502838-deb0-41a6-a981-8b34acdb71c9`::finished: None Thread-28::DEBUG::2014-11-12 16:13:21,007::task::595::TaskManager.Task::(_updateState) Task=`8c502838-deb0-41a6-a981-8b34acdb71c9`::moving from state preparing -> state finished Thread-28::DEBUG::2014-11-12 16:13:21,007::resourceManager::940::ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} Thread-28::DEBUG::2014-11-12 16:13:21,008::resourceManager::977::ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} Thread-28::DEBUG::2014-11-12 16:13:21,008::task::990::TaskManager.Task::(_decref) Task=`8c502838-deb0-41a6-a981-8b34acdb71c9`::ref 0 aborting False Again: i only replaced my ovirt-engine host by a backup restore. What could cause this problem? Thanks, Mario On Wed, Nov 12, 2014 at 2:16 PM, Sandro Bonazzola <sbona...@redhat.com> wrote: > Il 12/11/2014 14:06, Ml Ml ha scritto: >> Anyone? :-( > > Dan, Nir, can you take a look? > > >> >> On Tue, Nov 11, 2014 at 6:39 PM, Ml Ml <mliebher...@googlemail.com> wrote: >>> I dunno why this is all so simple for you. >>> >>> I just replaced the ovirt-engine like described in the docs. >>> >>> I ejected the CD ISOs on every vm so i was able to delete the ISO_DOMAIN. >>> >>> But i have still problems with my storage. Its a replicated glusterfs. >>> It looks healthy on the nodes itself. But somehow my ovirt-engine gets >>> confused. Can someone explain me what the actual error is? >>> >>> Note: i only replaced the ovirt-engine host and delete the ISO_DOMAIN: >>> >>> 2014-11-11 18:32:37,832 ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] Failed in >>> HSMGetTaskStatusVDS method >>> 2014-11-11 18:32:37,833 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] spmStart polling ended: >>> taskId = 8c5fae2c-0ddb-41cd-ac54-c404c943e00f task status = finished >>> 2014-11-11 18:32:37,834 ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] Start SPM Task failed - >>> result: cleanSuccess, message: VDSGenericException: VDSErrorException: >>> Failed to HSMGetTaskStatusVDS, error = Storage domain does not exist, >>> code = 358 >>> 2014-11-11 18:32:37,888 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] spmStart polling ended, >>> spm status: Free >>> 2014-11-11 18:32:37,889 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] START, >>> HSMClearTaskVDSCommand(HostName = ovirt-node01.foobar.net, HostId = >>> 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c, >>> taskId=8c5fae2c-0ddb-41cd-ac54-c404c943e00f), log id: 547e26fd >>> 2014-11-11 18:32:37,937 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] FINISH, >>> HSMClearTaskVDSCommand, log id: 547e26fd >>> 2014-11-11 18:32:37,938 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [71891fe3] FINISH, >>> SpmStartVDSCommand, return: >>> org.ovirt.engine.core.common.businessentities.SpmStatusResult@5027ed97, >>> log id: 461eb5b5 >>> 2014-11-11 18:32:37,941 INFO >>> [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] Running command: >>> SetStoragePoolStatusCommand internal: true. Entities affected : ID: >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d Type: StoragePool >>> 2014-11-11 18:32:37,948 ERROR >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] >>> IrsBroker::Failed::ActivateStorageDomainVDS due to: >>> IrsSpmStartFailedException: IRSGenericException: IRSErrorException: >>> SpmStart failed >>> 2014-11-11 18:32:38,006 INFO >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] Irs placed on server >>> 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c failed. Proceed Failover >>> 2014-11-11 18:32:38,044 INFO >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] >>> (DefaultQuartzScheduler_Worker-29) START, >>> GlusterVolumesListVDSCommand(HostName = ovirt-node01.foobar.net, >>> HostId = 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c), log id: 7a110756 >>> 2014-11-11 18:32:38,045 INFO >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] >>> hostFromVds::selectedVds - ovirt-node02.foobar.net, spmStatus Free, >>> storage pool HP_Proliant_DL180G6 >>> 2014-11-11 18:32:38,048 INFO >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] starting spm on vds >>> ovirt-node02.foobar.net, storage pool HP_Proliant_DL180G6, prevId -1, >>> LVER -1 >>> 2014-11-11 18:32:38,050 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] START, >>> SpmStartVDSCommand(HostName = ovirt-node02.foobar.net, HostId = >>> 6948da12-0b8a-4b6d-a9af-162e6c25dad3, storagePoolId = >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d, prevId=-1, prevLVER=-1, >>> storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log >>> id: 1a6ccb9c >>> 2014-11-11 18:32:38,108 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] spmStart polling >>> started: taskId = 78d31638-70a5-46aa-89e7-1d1e8126bdba >>> 2014-11-11 18:32:38,193 INFO >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] >>> (DefaultQuartzScheduler_Worker-29) FINISH, >>> GlusterVolumesListVDSCommand, return: >>> {d46619e9-9368-4e82-bf3a-a2377b6e85e4=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9746ef53}, >>> log id: 7a110756 >>> 2014-11-11 18:32:38,352 INFO >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] >>> (DefaultQuartzScheduler_Worker-29) START, >>> GlusterVolumesListVDSCommand(HostName = ovirt-node04.foobar.net, >>> HostId = 073c24e1-003f-412a-be56-0c41a435829a), log id: 2f25d56e >>> 2014-11-11 18:32:38,433 INFO >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] >>> (DefaultQuartzScheduler_Worker-29) FINISH, >>> GlusterVolumesListVDSCommand, return: >>> {660ca9ef-46fc-47b0-9b6b-61ccfd74016c=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cd3b51c4}, >>> log id: 2f25d56e >>> 2014-11-11 18:32:39,117 ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] Failed in >>> HSMGetTaskStatusVDS method >>> 2014-11-11 18:32:39,118 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] spmStart polling ended: >>> taskId = 78d31638-70a5-46aa-89e7-1d1e8126bdba task status = finished >>> 2014-11-11 18:32:39,119 ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] Start SPM Task failed - >>> result: cleanSuccess, message: VDSGenericException: VDSErrorException: >>> Failed to HSMGetTaskStatusVDS, error = Storage domain does not exist, >>> code = 358 >>> 2014-11-11 18:32:39,171 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] spmStart polling ended, >>> spm status: Free >>> 2014-11-11 18:32:39,173 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] START, >>> HSMClearTaskVDSCommand(HostName = ovirt-node02.foobar.net, HostId = >>> 6948da12-0b8a-4b6d-a9af-162e6c25dad3, >>> taskId=78d31638-70a5-46aa-89e7-1d1e8126bdba), log id: 46abf4a0 >>> 2014-11-11 18:32:39,220 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] FINISH, >>> HSMClearTaskVDSCommand, log id: 46abf4a0 >>> 2014-11-11 18:32:39,221 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [6d5f7d9d] FINISH, >>> SpmStartVDSCommand, return: >>> org.ovirt.engine.core.common.businessentities.SpmStatusResult@7d3782f7, >>> log id: 1a6ccb9c >>> 2014-11-11 18:32:39,224 INFO >>> [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] >>> (org.ovirt.thread.pool-6-thread-39) [4777665a] Running command: >>> SetStoragePoolStatusCommand internal: true. Entities affected : ID: >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d Type: StoragePool >>> 2014-11-11 18:32:39,232 ERROR >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (org.ovirt.thread.pool-6-thread-39) [4777665a] >>> IrsBroker::Failed::ActivateStorageDomainVDS due to: >>> IrsSpmStartFailedException: IRSGenericException: IRSErrorException: >>> SpmStart failed >>> 2014-11-11 18:32:39,235 INFO >>> [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] >>> (org.ovirt.thread.pool-6-thread-39) [4777665a] FINISH, >>> ActivateStorageDomainVDSCommand, log id: 75877740 >>> 2014-11-11 18:32:39,236 ERROR >>> [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] >>> (org.ovirt.thread.pool-6-thread-39) [4777665a] Command >>> org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw >>> Vdc Bll exception. With error message VdcBLLException: >>> org.ovirt.engine.core.vdsbroker.irsbroker.IrsSpmStartFailedException: >>> IRSGenericException: IRSErrorException: SpmStart failed (Failed with >>> error ENGINE and code 5001) >>> 2014-11-11 18:32:39,239 INFO >>> [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] >>> (org.ovirt.thread.pool-6-thread-39) [4777665a] Command >>> [id=c5315de2-0817-4da2-a13e-50c8cfa93a6a]: Compensating >>> CHANGED_STATUS_ONLY of >>> org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; >>> snapshot: EntityStatusSnapshot [id=storagePoolId = >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d, storageId = >>> abc51e26-7175-4b38-b3a8-95c6928fbc2b, status=Unknown]. >>> 2014-11-11 18:32:39,243 INFO >>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >>> (org.ovirt.thread.pool-6-thread-39) [4777665a] Correlation ID: >>> 71891fe3, Job ID: 239d4ac0-aa7d-486a-a70f-55a9d1b910f4, Call Stack: >>> null, Custom Event ID: -1, Message: Failed to activate Storage Domain >>> RaidVolBGluster (Data Center HP_Proliant_DL180G6) by admin >>> 2014-11-11 18:32:40,566 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] Command >>> org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand >>> return value >>> >>> TaskStatusListReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=654, >>> mMessage=Not SPM]] >>> >>> 2014-11-11 18:32:40,569 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] HostName = >>> ovirt-node02.foobar.net >>> 2014-11-11 18:32:40,570 ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] Command >>> HSMGetAllTasksStatusesVDSCommand(HostName = ovirt-node02.foobar.net, >>> HostId = 6948da12-0b8a-4b6d-a9af-162e6c25dad3) execution failed. >>> Exception: IRSNonOperationalException: IRSGenericException: >>> IRSErrorException: IRSNonOperationalException: Not SPM >>> 2014-11-11 18:32:40,625 INFO >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] hostFromVds::selectedVds >>> - ovirt-node02.foobar.net, spmStatus Free, storage pool >>> HP_Proliant_DL180G6 >>> 2014-11-11 18:32:40,628 INFO >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] starting spm on vds >>> ovirt-node02.foobar.net, storage pool HP_Proliant_DL180G6, prevId -1, >>> LVER -1 >>> 2014-11-11 18:32:40,630 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] START, >>> SpmStartVDSCommand(HostName = ovirt-node02.foobar.net, HostId = >>> 6948da12-0b8a-4b6d-a9af-162e6c25dad3, storagePoolId = >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d, prevId=-1, prevLVER=-1, >>> storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log >>> id: 1f3ac280 >>> 2014-11-11 18:32:40,687 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] spmStart polling >>> started: taskId = 50ab033e-76cd-44d5-b661-a1c2b8c312ef >>> 2014-11-11 18:32:41,735 ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] Failed in >>> HSMGetTaskStatusVDS method >>> 2014-11-11 18:32:41,736 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] spmStart polling ended: >>> taskId = 50ab033e-76cd-44d5-b661-a1c2b8c312ef task status = finished >>> 2014-11-11 18:32:41,737 ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] Start SPM Task failed - >>> result: cleanSuccess, message: VDSGenericException: VDSErrorException: >>> Failed to HSMGetTaskStatusVDS, error = Storage domain does not exist, >>> code = 358 >>> 2014-11-11 18:32:41,790 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] spmStart polling ended, >>> spm status: Free >>> 2014-11-11 18:32:41,791 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] START, >>> HSMClearTaskVDSCommand(HostName = ovirt-node02.foobar.net, HostId = >>> 6948da12-0b8a-4b6d-a9af-162e6c25dad3, >>> taskId=50ab033e-76cd-44d5-b661-a1c2b8c312ef), log id: 852d287 >>> 2014-11-11 18:32:41,839 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] FINISH, >>> HSMClearTaskVDSCommand, log id: 852d287 >>> 2014-11-11 18:32:41,840 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [47871083] FINISH, >>> SpmStartVDSCommand, return: >>> org.ovirt.engine.core.common.businessentities.SpmStatusResult@32b92b73, >>> log id: 1f3ac280 >>> 2014-11-11 18:32:41,843 INFO >>> [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] Running command: >>> SetStoragePoolStatusCommand internal: true. Entities affected : ID: >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d Type: StoragePool >>> 2014-11-11 18:32:41,851 ERROR >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] >>> IrsBroker::Failed::GetStoragePoolInfoVDS due to: >>> IrsSpmStartFailedException: IRSGenericException: IRSErrorException: >>> SpmStart failed >>> 2014-11-11 18:32:41,909 INFO >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] Irs placed on server >>> 6948da12-0b8a-4b6d-a9af-162e6c25dad3 failed. Proceed Failover >>> 2014-11-11 18:32:41,928 INFO >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] hostFromVds::selectedVds >>> - ovirt-node01.foobar.net, spmStatus Free, storage pool >>> HP_Proliant_DL180G6 >>> 2014-11-11 18:32:41,930 INFO >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] starting spm on vds >>> ovirt-node01.foobar.net, storage pool HP_Proliant_DL180G6, prevId -1, >>> LVER -1 >>> 2014-11-11 18:32:41,932 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] START, >>> SpmStartVDSCommand(HostName = ovirt-node01.foobar.net, HostId = >>> 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c, storagePoolId = >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d, prevId=-1, prevLVER=-1, >>> storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log >>> id: 56dfcc3c >>> 2014-11-11 18:32:41,984 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] spmStart polling >>> started: taskId = 84ac9f17-d5ec-4e43-8fcc-8ca9065a8492 >>> 2014-11-11 18:32:42,993 ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetTaskStatusVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] Failed in >>> HSMGetTaskStatusVDS method >>> 2014-11-11 18:32:42,994 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] spmStart polling ended: >>> taskId = 84ac9f17-d5ec-4e43-8fcc-8ca9065a8492 task status = finished >>> 2014-11-11 18:32:42,995 ERROR >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] Start SPM Task failed - >>> result: cleanSuccess, message: VDSGenericException: VDSErrorException: >>> Failed to HSMGetTaskStatusVDS, error = Storage domain does not exist, >>> code = 358 >>> 2014-11-11 18:32:43,048 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] spmStart polling ended, >>> spm status: Free >>> 2014-11-11 18:32:43,049 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] START, >>> HSMClearTaskVDSCommand(HostName = ovirt-node01.foobar.net, HostId = >>> 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c, >>> taskId=84ac9f17-d5ec-4e43-8fcc-8ca9065a8492), log id: 5abaa4ce >>> 2014-11-11 18:32:43,098 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] FINISH, >>> HSMClearTaskVDSCommand, log id: 5abaa4ce >>> 2014-11-11 18:32:43,098 INFO >>> [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] >>> (DefaultQuartzScheduler_Worker-28) [1ad3a509] FINISH, >>> SpmStartVDSCommand, return: >>> org.ovirt.engine.core.common.businessentities.SpmStatusResult@7d9b9905, >>> log id: 56dfcc3c >>> 2014-11-11 18:32:43,101 INFO >>> [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] >>> (DefaultQuartzScheduler_Worker-28) [725b57af] Running command: >>> SetStoragePoolStatusCommand internal: true. Entities affected : ID: >>> b384b3da-02a6-44f3-a3f6-56751ce8c26d Type: StoragePool >>> 2014-11-11 18:32:43,108 ERROR >>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] >>> (DefaultQuartzScheduler_Worker-28) [725b57af] >>> IrsBroker::Failed::GetStoragePoolInfoVDS due to: >>> IrsSpmStartFailedException: IRSGenericException: IRSErrorException: >>> SpmStart failed >>> 2014-11-11 18:32:43,444 INFO >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] >>> (DefaultQuartzScheduler_Worker-31) [7e2ba3a3] START, >>> GlusterVolumesListVDSCommand(HostName = ovirt-node01.foobar.net, >>> HostId = 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c), log id: 12ae9c47 >>> 2014-11-11 18:32:43,585 INFO >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] >>> (DefaultQuartzScheduler_Worker-31) [7e2ba3a3] FINISH, >>> GlusterVolumesListVDSCommand, return: >>> {d46619e9-9368-4e82-bf3a-a2377b6e85e4=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a5d949dc}, >>> log id: 12ae9c47 >>> 2014-11-11 18:32:43,745 INFO >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] >>> (DefaultQuartzScheduler_Worker-31) [7e2ba3a3] START, >>> GlusterVolumesListVDSCommand(HostName = ovirt-node04.foobar.net, >>> HostId = 073c24e1-003f-412a-be56-0c41a435829a), log id: 4b994fd9 >>> 2014-11-11 18:32:43,826 INFO >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] >>> (DefaultQuartzScheduler_Worker-31) [7e2ba3a3] FINISH, >>> GlusterVolumesListVDSCommand, return: >>> {660ca9ef-46fc-47b0-9b6b-61ccfd74016c=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@10521f1b}, >>> log id: 4b994fd9 >>> 2014-11-11 18:32:48,838 INFO >>> [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] >>> (DefaultQuartzScheduler_Worker-71) START, >>> GlusterVolumesListVDSCommand(HostName = ovirt-node01.foobar.net, >>> HostId = 2e8cec66-23d7-4a5c-b6f3-9758d1d87f5c), log id: 3b036a37 >>> >>> >>> >>> Thanks, >>> Mario >>> >>> On Fri, Nov 7, 2014 at 11:49 PM, Matt . <yamakasi....@gmail.com> wrote: >>>> Hi, >>>> >>>> Actually it's very simple as described in the docs. >>>> >>>> Just stop the engine, make a backup, copy it over, place it back and >>>> start it. You can do this in a several of ways. >>>> >>>> ISO domains is which I would remove and recreate again. ISO domains >>>> are actually dumb domains, so nothing can go wrong. >>>> >>>> Did it some time ago because I needed more performance. >>>> >>>> VDSM can run without the engine, it doesn't need it as the egine >>>> monitors and does the commands, so when it's not there... VM's just >>>> run (until you make them die yourself :)) >>>> >>>> I would give it 15-30 min/ >>>> >>>> Cheers, >>>> >>>> Matt >>>> >>>> >>>> 2014-11-07 18:36 GMT+01:00 Daniel Helgenberger >>>> <daniel.helgenber...@m-box.de>: >>>>> >>>>> Daniel Helgenberger >>>>> m box bewegtbild GmbH >>>>> >>>>> ACKERSTR. 19 P: +49/30/2408781-22 >>>>> D-10115 BERLIN F: +49/30/2408781-10 >>>>> >>>>> www.m-box.de >>>>> www.monkeymen.tv >>>>> >>>>> Geschäftsführer: Martin Retschitzegger / Michaela Göllner >>>>> Handeslregister: Amtsgericht Charlottenburg / HRB 112767 >>>>> On 07.11.2014, at 15:24, Koen Vanoppen <vanoppen.k...@gmail.com> wrote: >>>>> >>>>> Hi, >>>>> >>>>> We had a consulting partner who did the same for our company. This is his >>>>> procedure and worked great: >>>>> >>>>> How to migrate ovirt management engine >>>>> Packages >>>>> Ensure you have the same packages & versions installed on the destination >>>>> hostas on the source, using 'rpm -qa | grep ovirt'. Make sure versions are >>>>> 100%identical. >>>>> Default setup >>>>> >>>>> Run 'engine-setup' on the destination host after installing the packages. >>>>> Use >>>>> the following configuration: >>>>> 1. Backup existing configuration >>>>> 2. On the source host, do: >>>>> >>>>> You might want your consultant take a look on [1]... >>>>> Steps a-3d: >>>>> engine-backup mode=backup --file=~/ovirt-engine-source --log=backup.log >>>>> >>>>> a. service ovirt-engine stop >>>>> b. service ovirt-engine-dwhd stop >>>>> c. mkdir ~/backup >>>>> d. tar -C /etc/pki/ovirt-engine -czpf ~/backup/ovirt-engine-pki.tar.gz >>>>> . >>>>> e. tar -C /etc/ovirt-engine -czpf ~/backup/ovirt-engine-conf.tar.gz . >>>>> f. cd /usr/share/ovirt-engine/dbscripts >>>>> g. ./backup.sh >>>>> h. mv engine_*.sql ~/backup/engine.sql >>>>> 3. You may also want to backup dwh & reports: >>>>> a. cd /usr/share/ovirt-engine/bin/ >>>>> b. ./engine-backup.sh --mode=backup --scope=db --db-user=engine >>>>> --db-password=XXX --file=/usr/tmp/rhevm-backups/engine-backup >>>>> --log=/tmp/engine-backup.log >>>>> c. ./engine-backup.sh --mode=backup --scope=dwhdb --db-user=engine >>>>> --db-password=XXX --file=/usr/tmp/rhevm-backups/dwh-backup >>>>> --log=/tmp/engine-backup.log >>>>> d. ./engine-backup.sh --mode=backup --scope=reportsdb --db-user=engine >>>>> --db-password=XXX --file=/usr/tmp/rhevm-backups/reports-backup >>>>> --log=/tmp/engine-backup.log >>>>> 4. Download these backup files, and copy them to the destination host. >>>>> Restore configuration >>>>> 1. On the destination host, do: >>>>> >>>>> Again, steps a-h, basically >>>>> engine-setup >>>>> engine-cleanup >>>>> engine-backup mode=restore --file=~/ovirt-engine-source --log=backup.log >>>>> >>>>> also, I would run a second >>>>> engine-setup >>>>> After that, you should be good to go.. >>>>> >>>>> Of course, depending on your previous engine setup this could be a little >>>>> more complicated. Still, quite strait forward. >>>>> [1] http://www.ovirt.org/Ovirt-engine-backup >>>>> >>>>> a. service ovirt-engine stop >>>>> b. service ovirt-engine-dwhd stop >>>>> c. cd backup >>>>> d. tar -C /etc/pki/ovirt-engine -xzpf ovirt-engine-pki.tar.gz >>>>> e. tar -C /etc/ovirt-engine -xzpf ovirt-engine-conf.tar.gz >>>>> f. tar -xvjf engine-backup >>>>> g. tar -xvjf dwh-backup >>>>> h. tar -xvjf reports-backup >>>>> >>>>> Restore Database >>>>> 1. On the destination host do: >>>>> a. su - postgres -c "psql -d template1 -c 'drop database engine;'" >>>>> b. su - postgres -c "psql -d template1 -c 'create database engine >>>>> owner >>>>> engine;'" >>>>> c. su - postgres >>>>> d. psql >>>>> e. \c engine >>>>> f. \i /path/to/backup/engine.sql >>>>> NOTE: in case you have issues logging in to the database, add the >>>>> following >>>>> line to the pg_hba.conf file: >>>>> >>>>> host all engine 127.0.0.1/32 trust >>>>> >>>>> 2. Fix engine password: >>>>> a. su - postgres >>>>> b. psql >>>>> c. alter user engine with password 'XXXXXXX'; >>>>> Change ovirt hostname >>>>> On the destination host, run: >>>>> >>>>> /usr/share/ovirt-engine/setup/bin/ovirt-engine-rename >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> NB: >>>>> Restoring the dwh/reports database is similar to steps 5-7, but omitted >>>>> from >>>>> this document due to problems starting the reporting service. >>>>> >>>>> >>>>> 2014-11-07 10:28 GMT+01:00 Sven Kieske <s.kie...@mittwald.de>: >>>>>> >>>>>> >>>>>> >>>>>> On 07/11/14 10:10, Ml Ml wrote: >>>>>>> anyone? :) >>>>>>> >>>>>>> Or are you only doing backups, no restore? :-P >>>>>> >>>>>> gladly I just had to test disaster recovery and not actually >>>>>> perform it (yet) :D >>>>>> >>>>>> To be honest: I never have restored ovirt-engine with running vdsm >>>>>> hosts connected to it, sounds like a lot of fun, I see if I can >>>>>> grab some time and try this out myself :) >>>>>> >>>>>> By your description I guess you have nfs/iso domain on your engine host? >>>>>> why don't you just seperate it, so no need for remounts >>>>>> if your engine is destroyed. >>>>>> >>>>>> HTH >>>>>> >>>>>> -- >>>>>> Mit freundlichen Grüßen / Regards >>>>>> >>>>>> Sven Kieske >>>>>> >>>>>> Systemadministrator >>>>>> Mittwald CM Service GmbH & Co. KG >>>>>> Königsberger Straße 6 >>>>>> 32339 Espelkamp >>>>>> T: +49-5772-293-100 >>>>>> F: +49-5772-293-333 >>>>>> https://www.mittwald.de >>>>>> Geschäftsführer: Robert Meyer >>>>>> St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad >>>>>> Oeynhausen >>>>>> Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad >>>>>> Oeynhausen >>>>>> _______________________________________________ >>>>>> Users mailing list >>>>>> Users@ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>> >>>>> >>>>> _______________________________________________ >>>>> Users mailing list >>>>> Users@ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>> >>>>> >>>>> _______________________________________________ >>>>> Users mailing list >>>>> Users@ovirt.org >>>>> http://lists.ovirt.org/mailman/listinfo/users >>>>> >>>> _______________________________________________ >>>> Users mailing list >>>> Users@ovirt.org >>>> http://lists.ovirt.org/mailman/listinfo/users >> _______________________________________________ >> Users mailing list >> Users@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users >> > > > -- > Sandro Bonazzola > Better technology. Faster innovation. Powered by community collaboration. > See how it works at redhat.com _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users