On Mon, Jan 4, 2016 at 4:38 PM, Steven D'Aprano <st...@pearwood.info> wrote: > On Tue, 5 Jan 2016 07:50 am, livems...@gmail.com wrote: > >> So what is the fastest way to make 400 HTTP requests using "requests" >> library and also using tor proxy? > > > Since this will be I/O bound, not CPU bound, probably use separate threads. > > Push the 400 requests into a queue, then create N threads, where N will need > to be determined by experiment, but will probably be something like 4 or 8, > and let each thread pop a request from the queue as needed. > > Are you experienced with threads? Do you need further information about > using threads and queues?
Also see the concurrent.futures module in the standard library, which makes this sort of setup very simple to implement. -- https://mail.python.org/mailman/listinfo/python-list
