bhouse-nexthop commented on code in PR #14189:
URL: https://github.com/apache/cloudstack/pull/14189#discussion_r4168795660
##########
plugins/hypervisors/kvm/src/main/java/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java:
##########
@@ -700,40 +708,45 @@ public KVMPhysicalDisk getPhysicalDisk(String volumeUuid,
KVMStoragePool pool) {
* adjust refcount
*/
private int adjustStoragePoolRefCount(String uuid, int adjustment) {
- final String mutexKey = storagePoolRefCounts.keySet().stream()
- .filter(k -> k.equals(uuid))
- .findFirst()
- .orElse(uuid);
- synchronized (mutexKey) {
- // some access on the storagePoolRefCounts.key(mutexKey) element
- int refCount = storagePoolRefCounts.computeIfAbsent(mutexKey, k ->
0);
- refCount += adjustment;
- if (refCount < 1) {
- storagePoolRefCounts.remove(mutexKey);
- } else {
- storagePoolRefCounts.put(mutexKey, refCount);
- }
- return refCount;
- }
+ /*
+ * compute() is atomic for the key, so concurrent callers cannot lose
an
+ * update. Returning null from the remapping function removes the
entry,
+ * which keeps the map free of pools that are no longer in use.
+ */
+ Integer refCount = storagePoolRefCounts.compute(uuid, (key, count) -> {
+ int adjusted = (count == null ? 0 : count) + adjustment;
+ return adjusted < 1 ? null : adjusted;
+ });
+ return refCount == null ? 0 : refCount;
}
/**
* Thread-safe increment storage pool usage refcount
* @param uuid UUID of the storage pool to increment the count
*/
- private void incStoragePoolRefCount(String uuid) {
+ protected void incStoragePoolRefCount(String uuid) {
adjustStoragePoolRefCount(uuid, 1);
}
/**
* Thread-safe decrement storage pool usage refcount for the given uuid
and return if storage pool still in use.
* @param uuid UUID of the storage pool to decrement the count
* @return true if the storage pool is still used, else false.
*/
- private boolean decStoragePoolRefCount(String uuid) {
+ protected boolean decStoragePoolRefCount(String uuid) {
return adjustStoragePoolRefCount(uuid, -1) > 0;
}
+ private static Object getStoragePoolLock(String uuid) {
+ return storagePoolLocks.computeIfAbsent(uuid, k -> new Object());
+ }
+
@Override
public KVMStoragePool createStoragePool(String name, String host, int
port, String path, String userInfo, StoragePoolType type, Map<String, String>
details, boolean isPrimaryStorage) {
+ synchronized (getStoragePoolLock(name)) {
Review Comment:
Yes, it could. I got that wrong in my last reply: the "other pools aren't
held up" part only held at the adaptor level.
`KVMStoragePoolManager.createStoragePool()` is `synchronized` on the manager,
and the pool lock was taken inside it. So a create of a pool being torn down
waited for the teardown while holding the manager-wide lock. The umount retry
runs with no timeout (`runSimpleBashScript(cmd, 0)`), so a umount hung on dead
storage would have blocked creates of every pool on the host.
Fixed in 7e093df62e by reversing the lock order. The manager now takes the
pool's lock first and only then its own monitor, so a create waiting for a
teardown holds nothing other pools need. The locks moved to a small
`KVMStoragePoolLocks` class that the manager and adaptor share. The adaptor
still takes the same lock: reentrantly on the create path, and on its own for
deletes, which covers `LibvirtStoragePool.delete()` going around the manager.
New test in `KVMStoragePoolManagerTest`: while a create of pool X is blocked
on X's lock, it asserts that another thread can still take the manager lock. It
fails with the old order.
A create of the *same* pool still waits for a hung umount of that pool,
which I think is correct, since remounting it mid-teardown isn't safe. Putting
a timeout on that umount would be a reasonable follow-up, but I've kept it out
of this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
