Yaniv Bronhaim has posted comments on this change.
Change subject: Adding threads limitation to misc.tmap
......................................................................
Patch Set 2: (4 inline comments)
....................................................
Commit Message
Line 8:
Line 9: Using threads queue to keep the threads order and limit threads count
Line 10:
Line 11: I moved the function code next to itmap for readability.
Line 12: Currently noone calls to tmap function, so I don't care to change and
itmap is different and not depended on tmap
Line 13: fix it now to avoid future bugs.
Line 14:
Line 15: Exceptions that related to the queue usage will be raised as part of
Line 16: the function's exceptions.
....................................................
File vdsm/storage/misc.py
Line 1227: if maxthreads < 1 and maxthreads != UNLIMITED_THREADS:
Line 1228: raise ValueError("Wrong input passed to function tmap: %s",
maxthreads)
Line 1229:
Line 1230: resultsDict = {}
Line 1231: error = [None]
I didn't implement this, i just added limitation of threads to this scope. I
agree that there is no need for a list here, the operation on it is only
assignment and it is thread safe. I'll change it.
Line 1232:
Line 1233: def wrapper(f, arg, index):
Line 1234: try:
Line 1235: resultsDict[index] = f(arg)
Line 1252: if threadsQueue.full():
Line 1253: # Wait for the first unfinished thread in list to finish
if we
Line 1254: # have already initiate all possible thread's slots
(maxthreads)
Line 1255: if threadsQueue.empty():
Line 1256: raise RuntimeError("Queue is empty")
agreed.. It's a double check that we can avoid, mainly because I check that
maxthreads can't be less than 1
Line 1257: else:
Line 1258: threadsQueue.get_nowait().join()
Line 1259: t = threading.Thread(target=wrapper, args=(func, arg, i))
Line 1260: threadsQueue.put_nowait(t)
Line 1267: for i, result in resultsDict.iteritems():
Line 1268: results[i] = result
Line 1269:
Line 1270: if error[0] is not None:
Line 1271: raise error[0]
Also agree, still I didn't want to change this implementation much because i
wanted to make a small patch for 'maxthreads limitation', but again I see that
we haven't used this function yet so i'll change it.
Thank you.
Line 1272:
Line 1273: return tuple(results)
Line 1274:
Line 1275:
--
To view, visit http://gerrit.ovirt.org/8858
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I07845bfd78b9215e8994ac2ebe46a7ff78c85625
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yaniv Bronhaim <[email protected]>
Gerrit-Reviewer: Saggi Mizrahi <[email protected]>
Gerrit-Reviewer: ShaoHe Feng <[email protected]>
Gerrit-Reviewer: Shu Ming <[email protected]>
Gerrit-Reviewer: Yaniv Bronhaim <[email protected]>
Gerrit-Reviewer: Zhou Zheng Sheng <[email protected]>
_______________________________________________
vdsm-patches mailing list
[email protected]
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
