Ritesh Raj Sarraf wrote: > I'm planning to implement threads in my application so that multiple > items can be downloaded concurrently. I want the thread option to be > user-defined. > > Looking at the documentation of threads (Core Python Programming), I've > noticed that all threads are executed a once. Depending upon what they > are doing, some finish early and some later. > > But I want to implement something like: > > for item in list_items: > for num in thread_args: > thread[num].start() > thread[num].start() > > Is this the correct way of threading applications ? > This is the first time I'd be doing threading. So was looking for > comments and suggestions. >
What you want is to use a pool of threads so that you can configure how many requests are issued at a time (you don't want to try to issue 100 requests all in parallel). You can communicate with the threads through a Queue. So if the code for a thread looks like: def run(request, response): while 1: item = request.get() if item is None: break response.put(download_from_web(item)) # your main loop can be something like: requestQueue = Queue() responseQueue = Queue() thread_pool = [ Thread(target=run, args=(requestQueue, responseQueue) for i in range(numthreads)] for t in thread_pool: t.start() for item in list_items: requestQueue.put(item) for i in range(len(list_items)): response = responseQueue.get() handle_response(response) # and then to shut down the threads when you've finished: for t in thread_pool: requestQueue.put(None) for t in thread_pool: t.join() -- http://mail.python.org/mailman/listinfo/python-list