Manlio Perillo ha scritto:
> [...]
> Here is a quick example, ABSOLUTELY NOT TESTED:
>
> class DownloadQueue(object):
> SIZE = 50
>
> def init(self):
> self.requests = [] # queued requests
> self.deferreds = [] # waiting requests
>
> def addRequest(self, url, timeout
rzimerman ha scritto:
> I'm hoping to write a program that will read any number of urls from
> stdin (1 per line), download them, and process them. So far my script
> (below) works well for small numbers of urls. However, it does not
> scale to more than 200 urls or so, because it issues HTTP reque
"rzimerman" wrote:
> Is Twisted the best library for me to be using? I do like Twisted, but
> it seems more suited to batch mode operations. Is there some way that I
> could continue registering url requests while the reactor is running?
> Is there a way to specify a time out per page request, rat
rzimerman wrote:
> I'm hoping to write a program that will read any number of urls from
> stdin (1 per line), download them, and process them. So far my script
> (below) works well for small numbers of urls. However, it does not
> scale to more than 200 urls or so, because it issues HTTP requests f
I'm hoping to write a program that will read any number of urls from
stdin (1 per line), download them, and process them. So far my script
(below) works well for small numbers of urls. However, it does not
scale to more than 200 urls or so, because it issues HTTP requests for
all of the urls simult
