Yaniv Bronhaim has posted comments on this change.
Change subject: Adding threads limitation to misc.tmap ......................................................................
Patch Set 2: (4 inline comments)
.................................................... Commit Message Line 8: Line 9: Using threads queue to keep the threads order and limit threads count Line 10: Line 11: I moved the function code next to itmap for readability. Line 12: Currently noone calls to tmap function, so I don't care to change and itmap is different and not depended on tmap Line 13: fix it now to avoid future bugs. Line 14: Line 15: Exceptions that related to the queue usage will be raised as part of Line 16: the function's exceptions.
.................................................... File vdsm/storage/misc.py Line 1227: if maxthreads < 1 and maxthreads != UNLIMITED_THREADS: Line 1228: raise ValueError("Wrong input passed to function tmap: %s", maxthreads) Line 1229: Line 1230: resultsDict = {} Line 1231: error = [None] I didn't implement this, i just added limitation of threads to this scope. I agree that there is no need for a list here, the operation on it is only assignment and it is thread safe. I'll change it. Line 1232: Line 1233: def wrapper(f, arg, index): Line 1234: try: Line 1235: resultsDict[index] = f(arg)
Line 1252: if threadsQueue.full(): Line 1253: # Wait for the first unfinished thread in list to finish if we Line 1254: # have already initiate all possible thread's slots (maxthreads) Line 1255: if threadsQueue.empty(): Line 1256: raise RuntimeError("Queue is empty") agreed.. It's a double check that we can avoid, mainly because I check that maxthreads can't be less than 1 Line 1257: else: Line 1258: threadsQueue.get_nowait().join() Line 1259: t = threading.Thread(target=wrapper, args=(func, arg, i)) Line 1260: threadsQueue.put_nowait(t)
Line 1267: for i, result in resultsDict.iteritems(): Line 1268: results[i] = result Line 1269: Line 1270: if error[0] is not None: Line 1271: raise error[0] Also agree, still I didn't want to change this implementation much because i wanted to make a small patch for 'maxthreads limitation', but again I see that we haven't used this function yet so i'll change it. Thank you. Line 1272: Line 1273: return tuple(results) Line 1274: Line 1275:
-- To view, visit http://gerrit.ovirt.org/8858 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I07845bfd78b9215e8994ac2ebe46a7ff78c85625 Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim ybronhei@redhat.com Gerrit-Reviewer: Saggi Mizrahi smizrahi@redhat.com Gerrit-Reviewer: ShaoHe Feng shaohef@linux.vnet.ibm.com Gerrit-Reviewer: Shu Ming shuming@linux.vnet.ibm.com Gerrit-Reviewer: Yaniv Bronhaim ybronhei@redhat.com Gerrit-Reviewer: Zhou Zheng Sheng zhshzhou@linux.vnet.ibm.com