Re: [Users] SD Again
Il 10/06/2013 11:54, Dafna Ron ha scritto: I did see issues with your storage in the logs previously to the detach issue but they seem to have been resolved so I do not think that they are related. what I can see in the logs is that the detach fails because the connectStorageServer command, which suppose to mount the storage, fails: Mount of `172.16.0.5:/home/external/migration` at `/rhev/data-center/mnt/172.16.0.5:_home_external_migration` does not exist so unless you had a momentary problem in your storage it actually looks like a vdsm bug to me. adding Eduardo, perhaps he will see something I missed, but I think that its possibly related to this issue: https://bugzilla.redhat.com/show_bug.cgi?id=971291 Thank you very much I confirm that to solve the problem I had to: -manually mkdir /rhev/data-center/mnt/172.16.0.5:_home_external_migration - give it properly permissions - manually mount the storage of Export Domain in the right place - detach the SD using oVirt GUI I looks similar to the ug you point Thank you for your assistance Ovirt looks terribly good but it sometimes makes my road longer without any apparent reason Best regards Alessandro Bianchi -- SkyNet SRL Via Maggiate 67/a - 28021 Borgomanero (NO) - tel. +39 0322-836487/834765 - fax +39 0322-836608 http://www.skynet.it Autorizzazione Ministeriale n.197 Le informazioni contenute in questo messaggio sono riservate e confidenziali ed è vietata la diffusione in qualunque modo eseguita. Qualora Lei non fosse la persona a cui il presente messaggio è destinato, La invitiamo ad eliminarlo ed a distruggerlo non divulgandolo, dandocene gentilmente comunicazione. Per qualsiasi informazione si prega di contattare i...@skynet.it (e-mail dell'azienda). Rif. D.L. 196/2003 ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] SD Again
Il 07/06/2013 05:05, Shu Ming ha
scritto:
From
engine log:
2013-06-06 12:36:58,999 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(pool-3-thread-47) [7345dde6] START,
ConnectStorageServerVDSCommand(HostName = nodo1, HostId =
3156bdac-ebfb-44cf-bea6-53d668b74a10, storagePoolId =
----, storageType = NFS,
connectionList = [{ id: 45085cbf-da10-4852-9d85-754707d20a92,
connection: 172.16.0.5:/home/external/migration, iqn: null,
vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans:
null, nfsTimeo: null };]), log id: 54464e6b2013-06-06
12:36:59,149 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(pool-3-thread-47) [7345dde6] FINISH,
ConnectStorageServerVDSCommand, return:
{45085cbf-da10-4852-9d85-754707d20a92=100}, log id: 54464e6b
2013-06-06 12:36:59,150 ERROR
[org.ovirt.engine.core.bll.storage.NFSStorageHelper]
(pool-3-thread-47) [7345dde6] The connection with details
172.16.0.5:/home/external/migration failed because of error code
100 and error message is: generalexception
2013-06-06 12:36:59,175 ERROR
[org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand]
(pool-3-thread-47) [7345dde6] Transaction rolled-back for
command:
org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand.
*It seems that 45085cbf-da10-4852-9d85-754707d20a92 was
disconnected at 12:36:49 from vdsm.log, just before this
ConnectStorageServerVDSCommand() in engine.log at 12:36:58. What
does that mean? Did it connect the Storage server after the VDSM
disconnected the StorageServer in a very short time interval. **
***
From vdsm log:
"Thread-281877::INFO::2013-06-06
12:36:49,933::logUtils::41::dispatcher::(wrapper) Run and
protect: disconnectStorageServer(domType=1,
spUUID='----',
conList=[{'connection': '172.16.0.5:/home/external/migration',
'iqn': '', 'portal': '', 'user': '', 'password': '**', 'id':
'45085cbf-da10-4852-9d85-754707d20a92', 'port': ''}],
options=None)
*Thread-281877::DEBUG::2013-06-06
12:36:49,933::misc::84::Storage.Misc.excCmd::(lambda)
'/usr/bin/sudo -n /usr/bin/umount -f -l
/rhev/data-center/mnt/172.16.0.5:_home_external_migration' (cwd
None)*"
"Thread-281877::DEBUG::2013-06-06
12:36:52,190::task::1151::TaskManager.Task::(prepare)
Task=`c0d1c115-cb3d-4f39-8c9b-448401097921`::finished:
{'statuslist': [{'status': 0, 'id':
'45085cbf-da10-4852-9d85-754707d20a92'}]}"
I suppose something went wrong in the flow of deactivating the SD
and then detach it
As soon as I saw the red mark showing the down status of the
Export domain I placed in Maint, I clicked the "detach" button and
kaboom!
IMO it took the "right time" before the mark went red and SD was
marked as "Maintenance" so I didn't raise any suspect, but
immediately after it failed to detach and was NOT mounted on the
node.
Did the GUI say SD was down while it wasn't yet?
After I remounted the Export SD on the node by hand I was able to
detach the Export Domain using the GUI.
It looks like some sort of "asynchronous" execution of commands on
the node isn't it?
Thanks and bye
Alessandro
--
SkyNet SRL
Via Maggiate 67/a - 28021 Borgomanero (NO)
- tel.
+39 0322-836487/834765 - fax +39 0322-836608
http://www.skynet.it
Autorizzazione Ministeriale n.197
Le informazioni contenute in questo messaggio
sono
riservate e confidenziali ed è vietata la diffusione in
qualunque
modo eseguita.
Qualora Lei non fosse la persona a cui il presente
messaggio è destinato, La invitiamo ad eliminarlo ed a
distruggerlo
non divulgandolo, dandocene gentilmente comunicazione.
Per
qualsiasi informazione si prega di contattare i...@skynet.it
(e-mail dell'azienda). Rif. D.L. 196/2003
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
[Users] SD again ...
Hi all
Another problem with SD (those SD drive me crazy !)
I've attached the Export Domain to a Local domain and succesfully
exported a VM
Then I've placed the Export SD in Maintenance
Then I asked oVirt to Detach the Domain and that seems impossible
I see the mount point has been unomunted on the node but oVirt
doesn't mark the SD as unattached ...
I tried to reacttivate it with non luck (it's not mounted at the
moment)
Andy advice about how to get Export domain available again?
I'm on Fedora 18
Here is the log
2013-06-06 13:03:12,676 INFO
[org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand]
(pool-3-thread-50) [59909ba8] Running command:
DetachStorageDomainFromPoolCommand internal: false. Entities
affected : ID: e79cd423-ae17-4f8b-9f53-28d851cc9822 Type: Storage
2013-06-06 13:03:12,676 INFO
[org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand]
(pool-3-thread-50) [59909ba8] Start detach storage domain
2013-06-06 13:03:12,684 INFO
[org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand]
(pool-3-thread-50) [59909ba8] Detach storage domain: before
connect
2013-06-06 13:03:12,689 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand]
(pool-3-thread-47) [69a827a7] START,
ValidateStorageServerConnectionVDSCommand(HostName = nodo1, HostId
= 3156bdac-ebfb-44cf-bea6-53d668b74a10, storagePoolId =
----, storageType = NFS,
connectionList = [{ id: 45085cbf-da10-4852-9d85-754707d20a92,
connection: 172.16.0.5:/home/external/migration, iqn: null,
vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans:
null, nfsTimeo: null };]), log id: 1b41b932
2013-06-06 13:03:12,695 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand]
(pool-3-thread-47) [69a827a7] FINISH,
ValidateStorageServerConnectionVDSCommand, return:
{45085cbf-da10-4852-9d85-754707d20a92=0}, log id: 1b41b932
2013-06-06 13:03:12,695 INFO
[org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand]
(pool-3-thread-47) [69a827a7] Running command:
ConnectStorageToVdsCommand internal: true. Entities affected :
ID: aaa0----123456789aaa Type: System
2013-06-06 13:03:12,697 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(pool-3-thread-47) [69a827a7] START,
ConnectStorageServerVDSCommand(HostName = nodo1, HostId =
3156bdac-ebfb-44cf-bea6-53d668b74a10, storagePoolId =
----, storageType = NFS,
connectionList = [{ id: 45085cbf-da10-4852-9d85-754707d20a92,
connection: 172.16.0.5:/home/external/migration, iqn: null,
vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans:
null, nfsTimeo: null };]), log id: 3f7041da
2013-06-06 13:03:12,738 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand]
(pool-3-thread-47) [69a827a7] FINISH,
ConnectStorageServerVDSCommand, return:
{45085cbf-da10-4852-9d85-754707d20a92=100}, log id: 3f7041da
2013-06-06 13:03:12,739 ERROR
[org.ovirt.engine.core.bll.storage.NFSStorageHelper]
(pool-3-thread-47) [69a827a7] The connection with details
172.16.0.5:/home/external/migration failed because of error code
100 and error message is: generalexception
2013-06-06 13:03:12,741 ERROR
[org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand]
(pool-3-thread-47) [69a827a7] Transaction rolled-back for command:
org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand.
2013-06-06 13:03:12,741 INFO
[org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand]
(pool-3-thread-50) [59909ba8] Detach storage domain: after
connect
2013-06-06 13:03:12,742 INFO
[org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand]
(pool-3-thread-50) [59909ba8] START,
DetachStorageDomainVDSCommand( storagePoolId =
d76c9edf-34cb-48eb-b53b-32d27bedc26a, ignoreFailoverLimit = false,
compatabilityVersion = null, storageDomainId =
e79cd423-ae17-4f8b-9f53-28d851cc9822, masterDomainId =
----, masterVersion = 1, force =
false), log id: 6e692e6d
2013-06-06 13:03:14,327 INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(DefaultQuartzScheduler_Worker-3) No string for UNASSIGNED type.
Use default Log
2013-06-06 13:03:14,922 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
(pool-3-thread-50) [59909ba8] Failed in
Re: [Users] SD Again
Hi guys I solved manually remounting the Expord SD after restarting the NFS server and then detaching the export domain I wonder if someone knows why this rhings happen ... Best regards ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] SD Again
Il 06/06/2013 13:56, Maor Lipchuk ha scritto: Hi Alessandro, Is there a possibility some one else is using the export domain in another DC or setup? Regards, Maor On 06/06/2013 02:25 PM, Alessandro Bianchi wrote: Hi guys I solved manually remounting the Expord SD after restarting the NFS server and then detaching the export domain I wonder if someone knows why this rhings happen ... Best regards ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users Hi and Thank you for your answer No: it's not possible since I'm the only admin of the ovirt nodes (3 nodes holding about 14 VMs) and I attached the Export SD, then exported the VM and got the lock on export domain. It's really funny that sometimes I keep in trouble with SD: they don't come up or don't go down (or they simply don't like me :-) All my nodes and the engine host are F 18 with all latest updates applied (engine is also checked daily for updates) except for kernel due to this nasty bug https://bugzilla.redhat.com/show_bug.cgi?id=902012 Am I the only one fighting with SD's? If yes it may be some sort of network/nfs/"x-file" problem, otherwise I don't know what to investigate ... Thanks again Best regards Alessandro Bianchi -- SkyNet SRL Via Maggiate 67/a - 28021 Borgomanero (NO) - tel. +39 0322-836487/834765 - fax +39 0322-836608 http://www.skynet.it Autorizzazione Ministeriale n.197 Le informazioni contenute in questo messaggio sono riservate e confidenziali ed vietata la diffusione in qualunque modo eseguita. Qualora Lei non fosse la persona a cui il presente messaggio destinato, La invitiamo ad eliminarlo ed a distruggerlo non divulgandolo, dandocene gentilmente comunicazione. Per qualsiasi informazione si prega di contattare i...@skynet.it (e-mail dell'azienda). Rif. D.L. 196/2003 ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] SD Again
Il 06/06/2013 15:55,
users-requ...@ovirt.org ha scritto:
Message: 5
Date: Thu, 06 Jun 2013 15:38:48 +0300
From: Dafna Ron d...@redhat.com
To: users@ovirt.org
Subject: Re: [Users] SD Again
Message-ID: 51b082d8.5040...@redhat.com
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
You can attach full engine and vdsm logs and I'll try to debug it (I
need to see the first umount).
On 06/06/2013 02:25 PM, Alessandro Bianchi wrote:
Hi guys
I solved manually remounting the Expord SD after restarting the NFS
server and then detaching the export domain
I wonder if someone knows why this rhings happen ...
Best regards
Hope I'm providing the right stuff
I may send you all the file but these are the relevat infos I
suppose
vdsm.log
Thread-281877::DEBUG::2013-06-06
12:36:49,932::BindingXMLRPC::161::vds::(wrapper) [172.16.0.5]
Thread-281877::DEBUG::2013-06-06
12:36:49,932::task::568::TaskManager.Task::(_updateState)
Task=`c0d1c115-cb3d-4f39-8c9b-448401097921`::moving from state
init - state preparing
Thread-281877::INFO::2013-06-06
12:36:49,933::logUtils::41::dispatcher::(wrapper) Run and protect:
disconnectStorageServer(domType=1,
spUUID='----',
conList=[{'connection': '172.16.0.5:/home/external/migration',
'iqn': '', 'portal': '', 'user': '', 'password': '**', 'id':
'45085cbf-da10-4852-9d85-754707d20a92', 'port': ''}],
options=None)
Thread-281877::DEBUG::2013-06-06
12:36:49,933::misc::84::Storage.Misc.excCmd::(lambda)
'/usr/bin/sudo -n /usr/bin/umount -f -l
/rhev/data-center/mnt/172.16.0.5:_home_external_migration' (cwd
None)
Thread-281877::DEBUG::2013-06-06
12:36:50,012::misc::1054::SamplingMethod::(__call__) Trying to
enter sampling method (storage.sdc.refreshStorage)
Thread-281877::DEBUG::2013-06-06
12:36:50,012::misc::1056::SamplingMethod::(__call__) Got in to
sampling method
Thread-281877::DEBUG::2013-06-06
12:36:50,012::misc::1054::SamplingMethod::(__call__) Trying to
enter sampling method (storage.iscsi.rescan)
Thread-281877::DEBUG::2013-06-06
12:36:50,013::misc::1056::SamplingMethod::(__call__) Got in to
sampling method
Thread-281877::DEBUG::2013-06-06
12:36:50,013::misc::84::Storage.Misc.excCmd::(lambda)
'/usr/bin/sudo -n /sbin/iscsiadm -m session -R' (cwd None)
Thread-281877::DEBUG::2013-06-06
12:36:50,021::misc::84::Storage.Misc.excCmd::(lambda)
FAILED: err = 'iscsiadm: No session found.\n'; rc
= 21
Thread-281877::DEBUG::2013-06-06
12:36:50,021::misc::1064::SamplingMethod::(__call__) Returning
last result
Thread-281877::DEBUG::2013-06-06
12:36:52,029::misc::84::Storage.Misc.excCmd::(lambda)
'/usr/bin/sudo -n /sbin/multipath' (cwd None)
Thread-281877::DEBUG::2013-06-06
12:36:52,188::misc::84::Storage.Misc.excCmd::(lambda)
SUCCESS: err = ''; rc = 0
Thread-281877::DEBUG::2013-06-06
12:36:52,188::lvm::477::OperationMutex::(_invalidateAllPvs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-281877::DEBUG::2013-06-06
12:36:52,189::lvm::479::OperationMutex::(_invalidateAllPvs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-281877::DEBUG::2013-06-06
12:36:52,189::lvm::488::OperationMutex::(_invalidateAllVgs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-281877::DEBUG::2013-06-06
12:36:52,189::lvm::490::OperationMutex::(_invalidateAllVgs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-281877::DEBUG::2013-06-06
12:36:52,189::lvm::508::OperationMutex::(_invalidateAllLvs)
Operation 'lvm invalidate operation' got the operation mutex
Thread-281877::DEBUG::2013-06-06
12:36:52,189::lvm::510::OperationMutex::(_invalidateAllLvs)
Operation 'lvm invalidate operation' released the operation mutex
Thread-281877::DEBUG::2013-06-06
12:36:52,189::misc::1064::SamplingMethod::(__call__) Returning
last result
Thread-281877::INFO::2013-06-06
12:36:52,189::logUtils::44::dispatcher::(wrapper) Run and protect:
disconnectStorageServer, Return response: {'statuslist':
[{'status': 0, 'id': '45085cbf-da10-4852-9d85-754707d20a92'}]}
Thread-281877::DEBUG::2013-06-06
12:36:52,190::task::1151::TaskManager.Task::(prepare)
Task=`c0d1c115-cb3d-4f39-8c9b-448401097921`::finished:
{'statuslist': [{'status': 0, 'id':
'45085cbf-da10-4852-9d85-754707d20a92'}]}
Thread-281877::DEBUG::2013-06-06
12:36:52,190::task::568::TaskManager.Task::(_updateState)
Task=`c0d1c115-cb3d
