Duncan Booth on Thursday 27 Jul 2006 17:17 wrote: > What you want is to use a pool of threads so that you can configure how > many requests are issued at a time (you don't want to try to issue 100 > requests all in parallel). You can communicate with the threads through a > Queue. >
Thanks to both Duncan and Simon for your motivating replies. I had forgotten the truth that the lists are a medium to learn and educate. I hope you won't be very frustrated by my dummy questions. > So if the code for a thread looks like: > > def run(request, response): > while 1: > item = request.get() > if item is None: > break > response.put(download_from_web(item)) > This is where I'm most confused. I have a list, lRawData which has around, say, 100 items which are read from a file. Currently (without threads), I iterate upon each item in the list, split the item and then manipulate and pass it to download_from_web() to do the real work. I'm mainly confused in your run(). The line response.put(download_from_web(item)) is there. But if I use it, how am I going to iterate on my list lRawData. I'm not able to see where and how, run() or the line item = request.get() goes into my list lRawData to get the items. Can you please explain the functioning of the above code block. > # your main loop can be something like: > > requestQueue = Queue() > responseQueue = Queue() > thread_pool = [ > Thread(target=run, args=(requestQueue, responseQueue) > for i in range(numthreads)] > for t in thread_pool: t.start() This part seems understandable. > > for item in list_items: > requestQueue.put(item) > > for i in range(len(list_items)): > response = responseQueue.get() > handle_response(response) > I guess these would be clear once I'm clear on the first problem. > # and then to shut down the threads when you've finished: > for t in thread_pool: > requestQueue.put(None) > for t in thread_pool: > t.join() Thanks a lot for your replies. Ritesh -- Ritesh Raj Sarraf RESEARCHUT - http://www.researchut.com "Necessity is the mother of invention." "Stealing logic from one person is plagiarism, stealing from many is research." "The great are those who achieve the impossible, the petty are those who cannot - rrs" -- http://mail.python.org/mailman/listinfo/python-list
