[issue11563] test_urllibnet is triggering a ResourceWarning
Martin Panter added the comment: I think the fix for this bug only works if it gets the server to respond with a “Connection: close” header itself. I opened Issue 19524 because I was seeing keep-alive responses using chunked encoding that still trigger a socket leak. -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Roundup Robot devnull@devnull added the comment: New changeset edc3d3b07435 by Senthil Kumaran in branch '3.2': Closes issue11563 - test_urllibnet ResourceWarning. Patch by Jeff McNeil. http://hg.python.org/cpython/rev/edc3d3b07435 New changeset dfceb98767c0 by Senthil Kumaran in branch 'default': Closes issue11563 test_urllibnet is triggering a ResourceWarning. Patch by Jeff McNeil. http://hg.python.org/cpython/rev/dfceb98767c0 -- nosy: +python-dev resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
STINNER Victor victor.stin...@haypocalc.com added the comment: @Senthil Kumaran: Because your patch touchs not only the test, can you document your change in Misc/NEWS? Sending a new HTTP header should be documented. Is there an issue to support persistent connections in AbstractHTTPHandler.do_open()? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Nadeem Vawda nadeem.va...@gmail.com added the comment: urlopen_HttpTests.test_willclose() fails to call unfakehttp(), which breaks subsequent runs of test_urllib2_localnet, test_urllib2net and test_urllibnet. Fix attached. -- Added file: http://bugs.python.org/file21292/unfakehttp.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Senthil Kumaran orsent...@gmail.com added the comment: Victor - Issue9740 and Issue3566 talks about the need to have persistent connection. 3.3 would be a good target to have this feature in. Shall add the NEWS entry. Nadeen Wadwa - Thanks for your patch. I committed the fix as part of another bug (Issue3566). I shall mention the credits in an update to the log. thanks. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Changes by Senthil Kumaran orsent...@gmail.com: -- Removed message: http://bugs.python.org/msg131456 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Senthil Kumaran orsent...@gmail.com added the comment: Victor - Issue9740 and Issue3566 talks about the need to have persistent connection. 3.3 would be a good target to have this feature in. Shall add the NEWS entry. Nadeem Vawda - Thanks for your patch. I committed the fix as part of another bug (Issue3566). I shall mention the credits in an update to the log. thanks. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Roundup Robot devnull@devnull added the comment: New changeset 53c8f2bd0316 by Senthil Kumaran in branch '3.2': Add NEWS for Issue #11563. http://hg.python.org/cpython/rev/53c8f2bd0316 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Nadeem Vawda nadeem.va...@gmail.com added the comment: issue10883 is related; test_urllib2net also leaves sockets open in several places. -- nosy: +haypo, nvawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Jeff McNeil j...@jmcneil.net added the comment: So, it turned out to be more complicated than that. The HTTPConnection object returns an HTTPResponse, but never closes the underlying socket after calling makesock. Since persistent connections aren't supported, nothing actually closes the socket itself, it's just set to None. Explicitly calling a close turns out not to be correct either. I went down the same path as AbstractHTTPHandler and added a Connection: close header. That ensures that the remote host will close the underlying connection (more importantly, setting the HTTP Response object's will_close to True). That ensures HTTPConnection performs in a fire and forget mode, causing everything to close out as it should. I contemplated changing urlretrieve to use build_opener as urlopen does, but I figure that would have been done by now if it was a trivial operation. I'd be happy to take a whack at it if it's just a matter of getting around to it. -- keywords: +patch Added file: http://bugs.python.org/file21273/11562.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Senthil Kumaran orsent...@gmail.com added the comment: On Thu, Mar 17, 2011 at 06:30:51PM +, Jeff McNeil wrote: I went down the same path as AbstractHTTPHandler and added a Connection: close header. This is fine for the moment, tough I wish that the TODO pending in urllib.request with HTTP1.1 persistent connection be removed soon and will require changes it other places too. I contemplated changing urlretrieve to use build_opener as urlopen There is bug opened for this. All that would be required is output behavior of urlretrieve remain same, the implementation details (using build_opener or using urlopen itself!) may not be relevant and I think, it can be implemented using urlopen instead going through the handlers and OpenerDirector, BTW, urlretrive is a convenience function of some sort. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Jeff McNeil j...@jmcneil.net added the comment: Sounds good. I'll look at doing that, too. -- versions: +Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Changes by Jeff McNeil j...@jmcneil.net: -- nosy: +mcjeff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Jeff McNeil j...@jmcneil.net added the comment: So, I've been meaning to get more into contributing back to Python and I found this one somewhat interesting. As it turns out, even the following simple script raises the same warning: [jeff@martian cpython]$ ./python -c 'import urllib.request; urllib.request.urlretrieve(http://www.python.org;)' /home/jeff/cpython/Lib/socket.py:340: ResourceWarning: unclosed socket.socket object, fd=3, family=2, type=1, proto=6 self._sock = None [64388 refs] [jeff@martian cpython]$ The close method of Socket.SocketIO simply sets the underlying socket object to None, which causes that warning. Explicitly calling the close method on the underlying socket clears that up (and it's protected by that reference counter). The _decref_socketios just drops the internal ref count and never actually closes -- it won't unless self.__closed is True. So, when self._sock is set to None, that error bubbles up. As SocketIO is the foundation used in socket.makefile, I think just adding that close call ought to be correct. I can do the simple patch if you agree. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
Senthil Kumaran orsent...@gmail.com added the comment: Yes, go ahead with the patch. -- nosy: +orsenthil ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11563] test_urllibnet is triggering a ResourceWarning
New submission from Brett Cannon br...@python.org: test.test_urllibnet.urlopenNetworkTests.test_getcode() is leaving a socket open. My guess is that the error condition being triggered is somehow leaving the socket open but I can't find where. -- components: Library (Lib) messages: 131059 nosy: brett.cannon priority: normal severity: normal stage: needs patch status: open title: test_urllibnet is triggering a ResourceWarning ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
