Alban Hertroys wrote:
Hello
all,
I need your wisdom again. I'm working on a multi-threaded application
that handles multiple data sources in small batches each time. The idea
is that there are 3 threads that run simultaneously, each read a fixed
number of records, and then they wait for eachother. After that the
main thread does some processing, and the threads are allowed to
continue reading data.
I summarized this part of the application in the attached python
script, which locks up rather early, for reasons that I don't
understand (I don't have a computer science education), and I'm pretty
sure the problem is related to what I'm trying to fix in my
application. Can anybody explain what's happening (Or maybe even show
me a better way of doing this)?
Regards,
Alban Hertroys,
MAG Productions.
import sys
import threading
class AThread(threading.Thread):
def __init__(self, name, mainCond, allowedCond):
self.counter = 0
self.name = name
self.mainCond = mainCond
self.condAllowed = allowedCond
self.waitUntilRunning = threading.Condition()
threading.Thread.__init__(self, None, None, name, [])
def start(self):
threading.Thread.start(self)
# Let the main thread wait until this thread is ready to accept Notify
# events.
self.waitUntilRunning.acquire()
self.waitUntilRunning.wait()
self.waitUntilRunning.release()
def run(self):
threading.Thread.run(self)
# Print numbers 1 - 25
while self.counter < 25:
self.condAllowed.acquire()
# Tell the main thread that we're ready to receive Notifies
self.waitUntilRunning.acquire()
self.waitUntilRunning.notify()
print "Running"
self.waitUntilRunning.release()
# Wait for a Notify from the main thread
print "Wait"
self.condAllowed.wait()
self.condAllowed.release()
self.counter += 1
print "Thread %s: counter = %d" % (self.name, self.counter)
# Tell the main thread that a thread has reached the end of the loop
self.mainCond.acquire()
self.mainCond.notify()
self.mainCond.release()
class Main(object):
def __init__(self):
self.condWait = threading.Condition()
self.condAllowed = threading.Condition()
self.threads = [
AThread('A', self.condWait, self.condAllowed),
AThread('B', self.condWait, self.condAllowed),
AThread('C', self.condWait, self.condAllowed),
]
# Start the threads
for thread in self.threads:
thread.start()
while True:
# Allow the threads to run another iteration
self.condAllowed.acquire()
print "Notify"
self.condAllowed.notifyAll()
self.condAllowed.release()
# Wait until all threads reached the end of their loop
for thread in self.threads:
self.condWait.acquire()
self.condWait.wait()
self.condWait.release()
main = Main()
You've got a deadlock. I modified your script to add a print "T-%s" %
self.name before an acquire and after a release in the threads you spun
off (not in the main thread). Here is the output:
[EMAIL PROTECTED] threading]$ python tt.py
T-A: acquiring condAllowed
T-A: acquiring waitUntilRunning
T-A: Running
T-A: released waitUntilRunning
T-A: Wait
T-B: acquiring condAllowed
T-B: acquiring waitUntilRunning
T-B: Running
T-B: released waitUntilRunning
T-B: Wait
T-C: acquiring condAllowed
T-C: acquiring waitUntilRunning
T-C: Running
T-C: released waitUntilRunning
T-C: Wait
Notify
T-A: released condAllowed
T-A: counter = 1
T-A: acquiring mainCond
T-A: released mainCond
T-A: acquiring condAllowed
T-A: acquiring waitUntilRunning
T-A: Running
T-A: released waitUntilRunning
T-A: Wait
T-C: released condAllowed
T-C: counter = 1
T-C: acquiring mainCond
T-C: released mainCond
T-C: acquiring condAllowed
T-C: acquiring waitUntilRunning
T-C: Running
T-C: released waitUntilRunning
T-C: Wait
T-B: released condAllowed
T-B: counter = 1
T-B: acquiring mainCond
T-B: released mainCond
T-B: acquiring condAllowed
Notify <---------Here is
your problem
T-A: released condAllowed
T-A: counter = 2
T-A: acquiring mainCond
T-A: released mainCond
T-A: acquiring condAllowed
T-A: acquiring waitUntilRunning
T-A: Running
T-A: released waitUntilRunning
T-A: Wait
T-B: acquiring waitUntilRunning
T-B: Running
T-B: released waitUntilRunning
T-B: Wait
T-C: released condAllowed
T-C: counter = 2
T-C: acquiring mainCond
T-C: released mainCond
T-C: acquiring condAllowed
T-C: acquiring waitUntilRunning
T-C: Running
T-C: released waitUntilRunning
T-C: Wait
Notify is called before thread B (in this case) hits the
condAllowed.wait() piece of code. So, it sits at that wait() for
forever (because it doesn't get notified, because the notification
already happened), waiting to be notified from the main thread, and the
main thread is waiting on thread B (again, in this case) to call
mainCond.notify(). This approach is a deadlock just wanting to happen
(not waiting, because it already did happen). What is it exactly that
you are trying to accomplish? I'm sure there is a better approach.
Jeremy Jones
|
