> There isn't a thread pool module in the standard library, but I'm sure
> many have been written by people in the python community.
> Anyone have a favorite? Is there one particular implementation that's
> recommended?
>
> Not looking for anything fancy, just something that lets me queue up
> tasks to be performed by a pool of threads and then retrieve results
> when the tasks complete.
I wrote this one the other day, it doesn't save the results though.
You could probably just add to an output queue at the end of
threadFunction(). It was designed to handle short-running HTTP requests,
not larger jobs, this is reflected in the probablyIdle().
Use it like this:
pool.addJob(<function>,<arg1>,<arg2>,...,<argN>)
e.g.:
def greet(times):
for i in range(times):
print "Hello World"
pool = ThreadPool(64)
poll.addJob(greet,3)
...
# -*- coding: iso-8859-1 -*-
import Queue
import thread
class ThreadPool:
"A pool of worker threads, efficiently waiting for a task"
def __init__(self,max=0):
self.queue = Queue.Queue()
self.threads = []
self.stopping = False
self.max_threads = 16
if (max > 0):
self.max_threads = max
### Start the threads up, waiting on the empty Queue
for i in range(self.max_threads):
self.threads.append(thread.start_new_thread(self.threadFunction,()))
def addJob(self,function,*args,**kwargs):
self.queue.put((function,args,kwargs),True)
def threadFunction(self):
while (not self.stopping):
task = self.queue.get(True)
if (task != None):
function,args,kwargs = task
function(*args,**kwargs)
def probablyIdle(self):
return self.queue.empty()
def getApproximateQueueSize(self):
return self.queue.qsize()
Please consider our environment before printing this email.
WARNING - This email and any attachments may be confidential. If received in
error, please delete and inform us by return email. Because emails and
attachments may be interfered with, may contain computer viruses or other
defects and may not be successfully replicated on other systems, you must be
cautious. Westpac cannot guarantee that what you receive is what we sent. If
you have any doubts about the authenticity of an email by Westpac, please
contact us immediately.
It is also important to check for viruses and defects before opening or using
attachments. Westpac's liability is limited to resupplying any affected
attachments.
This email and its attachments are not intended to constitute any form of
financial advice or recommendation of, or an offer to buy or offer to sell, any
security or other financial product. We recommend that you seek your own
independent legal or financial advice before proceeding with any investment
decision.
Westpac Institutional Bank is a division of Westpac Banking Corporation, a
company registered in New South Wales in Australia under the Corporations Act
2001 (Cth). Westpac is authorised and regulated in the United Kingdom by the
Financial Services Authority and is registered at Cardiff in the United Kingdom
as Branch No. BR 106. Westpac operates in the United States of America as a
federally chartered branch, regulated by the Office of the Comptroller of the
Currency.
Westpac Banking Corporation ABN 33 007 457 141.
--
http://mail.python.org/mailman/listinfo/python-list
