Re: A problem while using urllib

2005-10-13 Thread Johnny Lee
Steve Holden 写道： > Good catch, John, I suspect this is a possibility so I've added the > following note: > > """The Windows 2.4.1 build doesn't show this error, but the Cygwin 2.4.1 > build does still have uncollectable objects after a urllib2.urlopen(), > so there may be a platform dependency her

Re: A problem while using urllib

2005-10-13 Thread Steve Holden
John J. Lee wrote: > Steve Holden <[EMAIL PROTECTED]> writes: > [...] > >> File "/usr/lib/python2.4/urllib2.py", line 996, in do_open >> raise URLError(err) >>urllib2.URLError: >> >>Looking at that part of the course of urrllib2 we see: >> >> headers["Connection"] = "close" >>

Re: A problem while using urllib

2005-10-13 Thread John J. Lee
Steve Holden <[EMAIL PROTECTED]> writes: [...] >File "/usr/lib/python2.4/urllib2.py", line 996, in do_open > raise URLError(err) > urllib2.URLError: > > Looking at that part of the course of urrllib2 we see: > > headers["Connection"] = "close" > try: > h.r

Re: A problem while using urllib

2005-10-12 Thread Steve Holden
Johnny Lee wrote: > Steve Holden wrote: > >>Steve Holden wrote: >> >>>Johnny Lee wrote: [...] >> >>So my conclusion is that there's something in the Cygwin socket module >>that causes problems not seen under other platforms. >> >>I couldn't find any obviously-related error in the Python bug tracke

Re: A problem while using urllib

2005-10-12 Thread Johnny Lee
Steve Holden wrote: > Steve Holden wrote: > > Johnny Lee wrote: > > [...] > > > >>I've sent the source, thanks for your help. > >> > > > > [...] > > Preliminary result, in case this rings bells with people who use urllib2 > > quite a lot. I modified the error case to report the actual message > >

Re: A problem while using urllib

2005-10-12 Thread Steve Holden
Steve Holden wrote: > Johnny Lee wrote: > [...] > >>I've sent the source, thanks for your help. >> > > [...] > Preliminary result, in case this rings bells with people who use urllib2 > quite a lot. I modified the error case to report the actual message > returned with the exception and I'm see

Re: A problem while using urllib

2005-10-12 Thread Steve Holden
Johnny Lee wrote: > Steve Holden wrote: > >>Johnny Lee wrote: >> >>>Alex Martelli wrote: >>> >>> Johnny Lee <[EMAIL PROTECTED]> wrote: ... > try: > webPage = urllib2.urlopen(url) > except urllib2.URLError: ... > webPage.close() >

Re: A problem while using urllib

2005-10-12 Thread Johnny Lee
Steve Holden wrote: > Johnny Lee wrote: > > Alex Martelli wrote: > > > >>Johnny Lee <[EMAIL PROTECTED]> wrote: > >> ... > >> > >>> try: > >>> webPage = urllib2.urlopen(url) > >>> except urllib2.URLError: > >> > >> ... > >> > >>> webPage.close() > >>> return True > >>>-

Re: A problem while using urllib

2005-10-12 Thread Steve Holden
Johnny Lee wrote: > Alex Martelli wrote: > >>Johnny Lee <[EMAIL PROTECTED]> wrote: >> ... >> >>> try: >>> webPage = urllib2.urlopen(url) >>> except urllib2.URLError: >> >> ... >> >>> webPage.close() >>> return True >>> >>> >>>

Re: A problem while using urllib

2005-10-11 Thread Johnny Lee
Alex Martelli wrote: > Johnny Lee <[EMAIL PROTECTED]> wrote: >... > >try: > > webPage = urllib2.urlopen(url) > >except urllib2.URLError: >... > >webPage.close() > >return True > > > > > >But every time when I ra

Re: A problem while using urllib

2005-10-11 Thread Alex Martelli
Johnny Lee <[EMAIL PROTECTED]> wrote: ... >try: > webPage = urllib2.urlopen(url) >except urllib2.URLError: ... >webPage.close() >return True > > >But every time when I ran to the 70 to 75 urls (that means 70-75 >

A problem while using urllib

2005-10-11 Thread Johnny Lee
Hi, I was using urllib to grab urls from web. here is the work flow of my program: 1. Get base url and max number of urls from user 2. Call filter to validate the base url 3. Read the source of the base url and grab all the urls from "href" property of "a" tag 4. Call filter to validate every u