Bugs item #1519816, was opened at 2006-07-10 10:29 Message generated for change (Comment added) made by jerrykhan You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1519816&group_id=5470
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michal Niklas (mniklas) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2 proxy does not work in 2.4.3 Initial Comment: My python app had to retrieve some web pages and while our network environment requires proxy it uses urllib2 opener (source is in attachment). It worked very well on older Python interpreters: ActivePython 2.4.2 Build 248 (ActiveState Corp.) based on Python 2.4.2 (#67, Oct 30 2005, 16:11:18) [MSC v.1310 32 bit (Intel)] on win32 It works on linux with 2.3 and 2.4.1: Python 2.4.1 (#2, May 5 2005, 11:32:06) [GCC 3.3.5 (Debian 1:3.3.5-12)] on linux2 But it does not work with newest 2.4.3 on Linux: Python 2.4.3 (#1, Jul 10 2006, 09:57:52) [GCC 3.3.5 (Debian 1:3.3.5-13)] on linux2 Desired result: isof-mark:~# python2.3 proxy_bug.py trying http://www.python.org ... OK. We have reply from http://www.python.org. Size: 13757 [b] <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/ xhtml1-transitional.dtd"> <html lang="en" xml:lang="en" xmlns="http://www.w3.org/ 1999/xhtml"> <head> <me ... Yorkshire">design by pollenation</a> </div> Copyright Š 1990-2006, <a href="psf">Python Software Foundation</a><br /> <a href="about/legal">Legal Statements</a> </div> </div> </div> </body> </html> isof-mark:~# /usr/local/bin/python proxy_bug.py trying http://www.python.org ... Traceback (most recent call last): File "proxy_bug.py", line 37, in ? get_page() File "proxy_bug.py", line 27, in get_page f = urllib2.urlopen(request) File "/usr/local/lib/python2.4/urllib2.py", line 130, in urlopen return _opener.open(url, data) File "/usr/local/lib/python2.4/urllib2.py", line 364, in open response = meth(req, response) File "/usr/local/lib/python2.4/urllib2.py", line 471, in http_response response = self.parent.error( File "/usr/local/lib/python2.4/urllib2.py", line 402, in error return self._call_chain(*args) File "/usr/local/lib/python2.4/urllib2.py", line 337, in _call_chain result = func(*args) File "/usr/local/lib/python2.4/urllib2.py", line 480, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 407: Proxy Authentication Required I have raported it on ActiveState bug list (http:// bugs.activestate.com/show_bug.cgi?id=47018) while I first spot this bug on their destribution but it seems that bug is in others distributions too. Regards, Michal Niklas ---------------------------------------------------------------------- Comment By: JerryKhan (jerrykhan) Date: 2006-11-28 16:17 Message: Logged In: YES user_id=867168 Originator: NO Hello, In my sens in a general manner there is something wrong in the urllib2 http code: But this may depends on the environment (I am not an expert in urllib) Here are my tests : using python 2.4.2 on Windows XP These simple codes failed with a 407 http error : Example E1: import urllib2 as URL a=URL.urlopen("http://lan_apache_url";) print a.read() OR example E2: import urllib2 as URL r=URL.Request("http://lan_apache_url";) a=URL.urlopen(r) print a.read() But succeed with urllib example E3 import urllib a=urllib.urlopen("http://lan_apache_url";) print a.read() Notice that different code lines are minimal E1 and E3 are close: Notice also that I'm try to access a lan apache server which is not behind a Proxy. And I don't want to access to any Proxy (like exclusion string in IExplorer) But I found also that If I try to access to a protected link with HTTPS ... on the LAN, there is not problem. The issue is really on the HTTP interpreter or during the configuration of the URL opener. In the same time, some of my programs are able to access to Internet servers using the current Proxy server without any problem. For that, I use: import urllib2 as URL URL.install_opener(URL.build_opener( s.https_handler, s.proxy_auth_handler, s.cookie_handler)) Well, I developed a workaround in my programs ... to use urllib instead of urllib2 in the case where I try to access the LAN (in fact when I don't want to configure the Proxy server, or when the URL match my own proxy exclusion list.) I espect this will help python urllib2 experts to find the issue. Jérôme Vacher alias jerrykhan the foolish dracomorpheus of the emerald dragon dynasty. ---------------------------------------------------------------------- Comment By: Michal Niklas (mniklas) Date: 2006-07-21 09:59 Message: Logged In: YES user_id=226518 I have just installed new virtual machine with Python 2.5b2 and my program works. It seems that only 2.4.3 is broken. Regards, Michal ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-07-20 20:09 Message: Logged In: YES user_id=261020 You're sure you didn't copy over the urllib2.py from 2.5b2 also? That might make the bug appear to go away, when really it's still there. The way to be sure is to try it on a different machine. Thanks for your report. ---------------------------------------------------------------------- Comment By: Michal Niklas (mniklas) Date: 2006-07-19 13:12 Message: Logged In: YES user_id=226518 I have checked that the last wersion my script works with is 2.4.2 and copied that version urllib2.py to 2.4.3 Lib directory. It works again. The only change in urllib2.py is in retry_http_basic_auth(), line 723: 2.4.2 user,pw = self.passwd.find_user_password(realm, host) 2.4.3 user, pw = self.passwd.find_user_password(realm, req.get_full_url()) So "host" is replaced by "req.get_full_url()". Checked again with 2.5b2 and it works! Probably I destroyed my test environment and tested it wrong way :( So the problem is only with 2.4.3. Previous versions and 2.5b works well. Regards, Michal Niklas ---------------------------------------------------------------------- Comment By: Michal Niklas (mniklas) Date: 2006-07-13 12:09 Message: Logged In: YES user_id=226518 2.5b2 does not work any better: Python 2.5b2 (r25b2:50512, Jul 11 2006, 10:16:14) [MSC v.1310 32 bit (Intel)] on win32 Result is the same as in 2.5b1 :( ---------------------------------------------------------------------- Comment By: Michal Niklas (mniklas) Date: 2006-07-11 08:27 Message: Logged In: YES user_id=226518 Tried it with 2.5 beta 1 and it is not better :( c:\tools\pyscripts\scripts>c:\python25\python2.5 Python 2.5b1 (r25b1:47027, Jun 20 2006, 09:31:33) [MSC v.1310 32 bit (Intel)] on win32 c:\tools\pyscripts\scripts>c:\python25\python2.5 proxy_bug.py trying http://www.python.org ... Traceback (most recent call last): File "proxy_bug.py", line 37, in <module> get_page() File "proxy_bug.py", line 27, in get_page f = urllib2.urlopen(request) File "c:\python25\lib\urllib2.py", line 121, in urlopen return _opener.open(url, data) File "c:\python25\lib\urllib2.py", line 380, in open response = meth(req, response) File "c:\python25\lib\urllib2.py", line 491, in http_response 'http', request, response, code, msg, hdrs) File "c:\python25\lib\urllib2.py", line 412, in error result = self._call_chain(*args) File "c:\python25\lib\urllib2.py", line 353, in _call_chain result = func(*args) File "c:\python25\lib\urllib2.py", line 831, in http_error_407 authority, req, headers) File "c:\python25\lib\urllib2.py", line 795, in http_error_auth_reqed return self.retry_http_basic_auth(host, req, realm) File "c:\python25\lib\urllib2.py", line 805, in retry_http_basic_auth return self.parent.open(req) File "c:\python25\lib\urllib2.py", line 380, in open response = meth(req, response) File "c:\python25\lib\urllib2.py", line 491, in http_response 'http', request, response, code, msg, hdrs) File "c:\python25\lib\urllib2.py", line 418, in error return self._call_chain(*args) File "c:\python25\lib\urllib2.py", line 353, in _call_chain result = func(*args) File "c:\python25\lib\urllib2.py", line 499, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 407: Proxy Authentication Required ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-07-10 14:49 Message: Logged In: YES user_id=849994 Can you please try with 2.5b1? A lot of urllib2 related bugs have been fixed before this release. ---------------------------------------------------------------------- Comment By: Michal Niklas (mniklas) Date: 2006-07-10 10:41 Message: Logged In: YES user_id=226518 Cannot add attachment via upload so I put it here: #!/usr/bin/python # -*- coding: cp1250 -*- import urllib import urllib2 def get_page(): url = 'http://www.python.org' print "trying %s ..." % (url) # Setup proxy & authentication proxy = "poczta.heuthes:8080" usr1 = "USER" pass1 = "PASSWD" proxy_handler = urllib2.ProxyHandler({"http" : "http:/ /" + proxy}) pass_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm() pass_mgr.add_password(None, "http://"; + proxy, usr1, pass1) pass_mgr.add_password(None, proxy, usr1, pass1) auth_handler = urllib2.HTTPBasicAuthHandler(pass_mgr) proxy_auth_handler = urllib2.ProxyBasicAuthHandler(pass_mgr) # Now build a new URL opener and install it opener = urllib2.build_opener(proxy_handler, proxy_auth_handler, auth_handler, urllib2.HTTPHandler) urllib2.install_opener(opener) request = urllib2.Request(url) f = urllib2.urlopen(request) data = f.read() print "OK. We have reply from %s.\nSize: %d [b]" % (url, len(data)) if len(data) < 400: print data else: print data[:200] print "..." print data[-200:] get_page() ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1519816&group_id=5470 _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
