Found out that I'm not the only one - Google's usenet
search turned up a bunch of people that have had this problem, and the
resolution is to make use of a great utility on the Vaults, called
timeoutsocket.py. Import the code, set a timeout of say 20 seconds, and
urllib won't hang indefinitely anymore.
-
Rick
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of Richard Kemp
Sent: Friday, May 10, 2002 4:21 AM
To: [EMAIL PROTECTED]
Subject: urllib seems to die or hangI'm using urllib lib in a fairly intensive spider/load test application. After some time, though, my script just seems to hang, and I've traced it to waiting on the read() call on the file handle returned from urlopen(). It doesn't matter whether I attempt to read in small chunks, or reall all bytes. Although the program was designed multithreaded, I also built a single-thread version to see if it made a difference - it did not.Anyone have a similar experience with urllib? Is urllib2 better?Thanks,Rick
