Nir Soffer has uploaded a new change for review.
Change subject: hsm: wait until lvm bootstrap is done before connecting to pool
......................................................................
hsm: wait until lvm bootstrap is done before connecting to pool
When hsm is created, a bootsrap thread is started, initializing lvm
cache and other parts of the system. This leads to races between
different threads when accessing lvm module when connecting to storage
pool. The races may lead to two threads trying to rebuild the cache in
the same time and possibly corrupt the cache.
This patch adds a synchronization point after lvm bootstrap is done. If
engine attempt to connect to storage pool while lvm cache is not ready
yet, it will wait until it is ready.
This patch will add a delay until a host can connect to the storage
pool. This delay is unavoidable if we want to have correct code.
Change-Id: I22e851dca4c2063d19446f34897a7b208b9cace4
Signed-off-by: Nir Soffer <nsof...@redhat.com>
---
M lib/vdsm/config.py.in
M vdsm/storage/hsm.py
2 files changed, 19 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/88/21388/1
diff --git a/lib/vdsm/config.py.in b/lib/vdsm/config.py.in
index 9668772..8e075fd 100644
--- a/lib/vdsm/config.py.in
+++ b/lib/vdsm/config.py.in
@@ -267,6 +267,10 @@
('use_volume_leases', 'false',
'Whether to use the volume leases or not.'),
+
+ ('lvm_bootstrap_timeout', '120',
+ 'Time in seconds to wait until lvm bootstrap is done when'
+ ' connecting to storage pool.')
]),
# Section: [addresses]
diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py
index 5e53f3a..725279f 100644
--- a/vdsm/storage/hsm.py
+++ b/vdsm/storage/hsm.py
@@ -373,8 +373,16 @@
except Exception:
self.log.warn("Failed to clean Storage Repository.", exc_info=True)
+ self._bootstrapDone = threading.Event()
+
def storageRefresh():
- lvm._lvminfo.bootstrap()
+ # This may take more then a minute when having lot of lvs. Until it
+ # finish, we should not use the lvm module.
+ try:
+ lvm._lvminfo.bootstrap()
+ finally:
+ self._bootstrapDone.set()
+
sdCache.refreshStorage()
fileUtils.createdir(self.tasksDir)
@@ -1008,6 +1016,12 @@
"spUUID=%s, msdUUID=%s, masterVersion=%s, hostID=%s, "
"scsiKey=%s" % (spUUID, msdUUID, masterVersion,
hostID, scsiKey)))
+
+ timeout = config.getint('vars', 'lvm_bootstrap_timeout')
+ self._bootstrapDone.wait(timeout)
+ if not self._bootstrapDone.is_set():
+ raise se.StoragePoolConnectionError("Bootstrap is not done yet")
+
with rmanager.acquireResource(STORAGE, HSM_DOM_MON_LOCK,
rm.LockType.exclusive):
return self._connectStoragePool(spUUID, hostID, scsiKey, msdUUID,
--
To view, visit http://gerrit.ovirt.org/21388
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I22e851dca4c2063d19446f34897a7b208b9cace4
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsof...@redhat.com>
_______________________________________________
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
