En Mon, 06 Oct 2008 11:24:51 -0300, <[EMAIL PROTECTED]> escribió:
On 6 Ott, 15:24, oyster <[EMAIL PROTECTED]> wrote:
my code is not right, can sb give me a hand? thanx
for example, I have 1000 urls to be downloaded, but only 5 thread at
one time
I would restructure my code with someting like this ( WARNING: the
following code is
ABSOLUTELY UNTESTED and shall be considered only as pseudo-code to
express my idea of
the algorithm (which, also, could be wrong:-) ):
Your code creates one thread per url (but never more than MAX_THREADS
alive at the same time). Usually it's more efficient to create all the
MAX_THREADS at once, and continuously feed them with tasks to be done. A
Queue object is the way to synchronize them; from the documentation:
<code>
from Queue import Queue
from threading import Thread
num_worker_threads = 3
list_of_urls = ["http://foo.com";, "http://bar.com";,
"http://baz.com";, "http://spam.com";,
"http://egg.com";,
]
def do_work(url):
from time import sleep
from random import randrange
from threading import currentThread
print "%s downloading %s" % (currentThread().getName(), url)
sleep(randrange(5))
print "%s done" % currentThread().getName()
# from this point on, copied almost verbatim from the Queue example
# at the end of http://docs.python.org/library/queue.html
def worker():
while True:
item = q.get()
do_work(item)
q.task_done()
q = Queue()
for i in range(num_worker_threads):
t = Thread(target=worker)
t.setDaemon(True)
t.start()
for item in list_of_urls:
q.put(item)
q.join() # block until all tasks are done
print "Finished"
</code>
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
