First of all, let me say thank you to all of you. I have asked many questions (some of them are dump questions), and you have kindly helped me. I am not going to reply every message to say thank-you since that would be annoying for such group with such high daily traffics. Thank you very much.
Let's get back to topic of this message. Here's how I have implemented it so far, and I am taking the queue of work load items approach. In my child thread, I will keep checking for available work load item until a duration is reached. #inside the child# while endTime > time.time(): try: item = self.q.get(True, 3) except Queue.Empty: #what's wrong? AttributeError: class Queue has no attribute 'Empty' print 'cant find any work load item, so lets wait and try again later' time.sleep(1) #wait and then check again continue except: print "Unexpected error:", sys.exc_info()[0] raise #do the real work with load item In my parent thread, I will initialize X (depending on a cfg file) child threads and keep adding load items to a shared q until the duration is reached. #inside the parent# callCounter = 0 workers = [] #a list of child threads totalWorkers = 250 endTime = time.time() + duration for i in range(totalWorkers): w = Worker(q, duration, i) w.start() #worker, do your job now! workers.append(w) while endTime > time.time(): time.sleep(1) q.put(getWorkloadItem()) #add workload itmes callCounter += 1 #actually can we guarantee that the call will be sent?? #should we ask each child to report the number of calls they make? for i in range(totalWorkers): workers[i].join() # Wait for the child threads to finish Overall, it seems to be working now. Though, I still have a couple of problems to resolve. 1. I got the following error for the codes that attempt to catch Empty Queue exception. What's the right way to use it? except Queue.Empty: AttributeError: class Queue has no attribute 'Empty' 2. What's the best way to have each child thread to report the number of requests they send when they are done? To add the numbers to another queue? 3. I will need to do some logging for response time as well as some response contents. I have two choices, one big log file for all threads (both child and parent), and one log file for each thread. Given the fact that I may have to log tons of data, I think opening and maintaining a bunch of smaller logs may be better than dealing with a big one (it may grow very fast). Is there any best prastice for logging in Python? If I change my mind and go with one big log file (pass it to each thread), is there anything I should be aware of for multi-thread access (writting) to the same log file? Again, thank you. -- http://mail.python.org/mailman/listinfo/python-list