Re: Socket exceptions aren't in the standard exception hierarchy (M2Crypto issue)
On a related note, M2Crypto's exceptions are direct children of Exception. If we add NetworkError, there will be a better place to put them. Currently, you have to catch, at least, M2Crypto.SSL.Checker.SSLVerificationError M2Crypto.SSL.SSLError both of which can be raised by socket operations if M2Crypto is installed. I'm not sure what errors the stock SSL module raises. (I'm running stress tests on a dedicated machine in a colocation facility. It's examining 11,000 known spam and malware sites right now. This exercises the error handling, forcing many unusual cases and logging the problems. That's why I'm discovering all these library issues.) John Nagle Steve Holden wrote: John Nagle wrote: Steve Holden wrote: John Nagle wrote: Steve Holden wrote: John Nagle wrote: [socket.error bug report] All these notes should be included in the bug report, as I suspect the module would benefit from additional clarity. Done. See [ 1706815 ] socket.error exceptions not subclass of StandardError Also see [ 805194 ] Inappropriate error received using socket timeout [ 1019808 ] wrong socket error returned [ 1571878 ] Improvements to socket module exceptions [ 708927 ] socket timeouts produce wrong errors in win32 for related but not identical problems in that area. Thanks. At least this is less likely to be overlooked now. regards Steve -- http://mail.python.org/mailman/listinfo/python-list
Re: Socket exceptions aren't in the standard exception hierarchy
John Nagle wrote: Steve Holden wrote: John Nagle wrote: Steve Holden wrote: John Nagle wrote: [socket.error bug report] All these notes should be included in the bug report, as I suspect the module would benefit from additional clarity. Done. See [ 1706815 ] socket.error exceptions not subclass of StandardError Also see [ 805194 ] Inappropriate error received using socket timeout [ 1019808 ] wrong socket error returned [ 1571878 ] Improvements to socket module exceptions [ 708927 ] socket timeouts produce wrong errors in win32 for related but not identical problems in that area. Thanks. At least this is less likely to be overlooked now. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Recent Ramblings http://holdenweb.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Socket exceptions aren't in the standard exception hierarchy
Steve Holden wrote: John Nagle wrote: [socket.error bug report] Where did you get this information? If true it would certainly need to be logged as a bug, but under Windows on 2,4 I see issubclass(socket.gaierror, Exception) True and the same under Cygwin 2.5. I am presuming most other users will see the same thing. regards Steve Ah. socket.error is a subclass of Exception, but not of StandardError. issubclass(socket.error,StandardError) is False. Right, so this *is* a bug, as long as Brett Cannon's upcoming (further) reorganization of the standard exception hierarchy doesn't stamp on it. It probably *was* overlooked in the reorganization of the hierarchy, and this implies there may be other extensions that also make the same error. It should be logged as a bug in the tracker - the fix is probably pretty simple, but it'll need some consideration of the forward compatibility considerations. regards Steve The bug tracker shows quite a number of problems related to socket exceptions. Searching for socket AND exception returns 45 results. These seem to be the most relevant ones: [ 805194 ] Inappropriate error received using socket timeout [ 1019808 ] wrong socket error returned [ 1571878 ] Improvements to socket module exceptions [ 708927 ] socket timeouts produce wrong errors in win32 None of them quite cover this new issue, but it's clear that the area needs some work. Should this new issue be added as a new bug or as an added comment to one of the above? Just figuring out what exceptions can be raised from the socket module is tough. I've seen exceptions derived from socket.error, exceptions from IOError, and exceptions from the SSL layer, which patches the sockets module when loaded. These are non-bug exceptions; that is, the problem is out in the network, external to the program. I'm still not sure I have all the possibilities covered. Retrying on unknown exceptions isn't the answer; that leads to exception loops if there's a program bug. The most important distinction with sockets is external network problem vs. local program program. I'd like to see a NetworkException in the exception hierarchy, with all the things that can go wrong due to conditions external to the local machine under that exception. I'd suggest the following: 1. Add NetworkError under IOError in the exception hierarchy. 2. Put the existing socket.error under NetworkError. Since socket.error needs to be reparented anyway (it's currently a direct descendant of Exception) this provides a good place for it. 3. Find any places where the socket module can raise IOError or OSError due to an external network condition, and make them raise something under NetworkError instead. Code that catches IOError will still work. 4. Put all errors in the various SSL modules (SSLError, etc.) which can be raised due to external network conditions under NetworkError. 5. Move urllib2.URLError, which is currently under IOError, to be under NetworkError. 6. Move the misc. errors from urllib, like ContentTooShortError, which are currently under IOError, down a level under NetworkError. Then, programs that catch NetworkError could be sure of catching all network trouble conditions, but not local code bugs. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: Socket exceptions aren't in the standard exception hierarchy
John Nagle wrote: Steve Holden wrote: John Nagle wrote: [socket.error bug report] Where did you get this information? If true it would certainly need to be logged as a bug, but under Windows on 2,4 I see issubclass(socket.gaierror, Exception) True and the same under Cygwin 2.5. I am presuming most other users will see the same thing. regards Steve Ah. socket.error is a subclass of Exception, but not of StandardError. issubclass(socket.error,StandardError) is False. Right, so this *is* a bug, as long as Brett Cannon's upcoming (further) reorganization of the standard exception hierarchy doesn't stamp on it. It probably *was* overlooked in the reorganization of the hierarchy, and this implies there may be other extensions that also make the same error. It should be logged as a bug in the tracker - the fix is probably pretty simple, but it'll need some consideration of the forward compatibility considerations. regards Steve The bug tracker shows quite a number of problems related to socket exceptions. Searching for socket AND exception returns 45 results. These seem to be the most relevant ones: [ 805194 ] Inappropriate error received using socket timeout [ 1019808 ] wrong socket error returned [ 1571878 ] Improvements to socket module exceptions [ 708927 ] socket timeouts produce wrong errors in win32 None of them quite cover this new issue, but it's clear that the area needs some work. Should this new issue be added as a new bug or as an added comment to one of the above? Just figuring out what exceptions can be raised from the socket module is tough. I've seen exceptions derived from socket.error, exceptions from IOError, and exceptions from the SSL layer, which patches the sockets module when loaded. These are non-bug exceptions; that is, the problem is out in the network, external to the program. I'm still not sure I have all the possibilities covered. Retrying on unknown exceptions isn't the answer; that leads to exception loops if there's a program bug. The most important distinction with sockets is external network problem vs. local program program. I'd like to see a NetworkException in the exception hierarchy, with all the things that can go wrong due to conditions external to the local machine under that exception. I'd suggest the following: 1. Add NetworkError under IOError in the exception hierarchy. 2. Put the existing socket.error under NetworkError. Since socket.error needs to be reparented anyway (it's currently a direct descendant of Exception) this provides a good place for it. 3. Find any places where the socket module can raise IOError or OSError due to an external network condition, and make them raise something under NetworkError instead. Code that catches IOError will still work. 4. Put all errors in the various SSL modules (SSLError, etc.) which can be raised due to external network conditions under NetworkError. 5. Move urllib2.URLError, which is currently under IOError, to be under NetworkError. 6. Move the misc. errors from urllib, like ContentTooShortError, which are currently under IOError, down a level under NetworkError. Then, programs that catch NetworkError could be sure of catching all network trouble conditions, but not local code bugs. John Nagle All these notes should be included in the bug report, as I suspect the module would benefit from additional clarity. I believe socket just grew up as a clever way of making as much use of the platform socket library as possible, and that the approach's deficiencies have become more obvious as time has gone by and Python has matured. The work you describe is not insignificant, so it should be fully described if it's to stand any chance of being implemented. I *do* think it would rationalize things considerably, but that's only me ... regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Recent Ramblings http://holdenweb.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Socket exceptions aren't in the standard exception hierarchy
Steve Holden wrote: John Nagle wrote: Steve Holden wrote: John Nagle wrote: [socket.error bug report] All these notes should be included in the bug report, as I suspect the module would benefit from additional clarity. Done. See [ 1706815 ] socket.error exceptions not subclass of StandardError Also see [ 805194 ] Inappropriate error received using socket timeout [ 1019808 ] wrong socket error returned [ 1571878 ] Improvements to socket module exceptions [ 708927 ] socket timeouts produce wrong errors in win32 for related but not identical problems in that area. John Nagle -- http://mail.python.org/mailman/listinfo/python-list
Re: Socket exceptions aren't in the standard exception hierarchy
Dennis Lee Bieber wrote:
On Sun, 22 Apr 2007 23:20:25 -0700, John Nagle [EMAIL PROTECTED]
declaimed the following in comp.lang.python:
2. File D:\Python24\lib\socket.py, line 295, in read
data = self._sock.recv(recv_size)
error: (10054, 'Connection reset by peer')
That looks like M$ Windows version of UNIX/Linux error number 54
(pretty much all Windows socket errors are UNIX number+1)
Errors coming from Windows may not be mapped to specific Python
exceptions, but rather to some general error conditions. {hypothesis} As
such, the Windows errors may not match what UNIX/Linux report.
Actually, that's not what's happening. The socket module is
explicitly raising socket.error in C code. It's not an OSError or
a WindowsError, although the text makes it look like one.
The problem is not that socket errors aren't entirely portable.
It's that they're not even in the Python exception hierarchy.
See http://docs.python.org/lib/module-exceptions.html;.
They have their own hierarchy, which starts at socket.error.
All built-in exceptions were supposed to be converted to the
standard exception hierarchy back before Python 2.0, but these
weren't.
Either they should be under IOError, or there should be
NetworkError under EnvironmentError, and they should be under
that. NetworkError, alongside IOError in the hierarchy,
would be useful. All the things that go wrong in networking
belong under there. Socket-level errors, SSL/TLS-level errors,
and HTTP/FTP/etc. level errors all belong under NetworkError.
This has to be fixed before PEP 352, when the exception
hierarchy is enforced, or the socket exceptions won't even work
right.
John Nagle
--
http://mail.python.org/mailman/listinfo/python-list
Re: Socket exceptions aren't in the standard exception hierarchy
John Nagle wrote:
Dennis Lee Bieber wrote:
On Sun, 22 Apr 2007 23:20:25 -0700, John Nagle [EMAIL PROTECTED]
declaimed the following in comp.lang.python:
2. File D:\Python24\lib\socket.py, line 295, in read
data = self._sock.recv(recv_size)
error: (10054, 'Connection reset by peer')
That looks like M$ Windows version of UNIX/Linux error number 54
(pretty much all Windows socket errors are UNIX number+1)
Errors coming from Windows may not be mapped to specific Python
exceptions, but rather to some general error conditions. {hypothesis} As
such, the Windows errors may not match what UNIX/Linux report.
Actually, that's not what's happening. The socket module is
explicitly raising socket.error in C code. It's not an OSError or
a WindowsError, although the text makes it look like one.
The problem is not that socket errors aren't entirely portable.
It's that they're not even in the Python exception hierarchy.
See http://docs.python.org/lib/module-exceptions.html;.
They have their own hierarchy, which starts at socket.error.
All built-in exceptions were supposed to be converted to the
standard exception hierarchy back before Python 2.0, but these
weren't.
Either they should be under IOError, or there should be
NetworkError under EnvironmentError, and they should be under
that. NetworkError, alongside IOError in the hierarchy,
would be useful. All the things that go wrong in networking
belong under there. Socket-level errors, SSL/TLS-level errors,
and HTTP/FTP/etc. level errors all belong under NetworkError.
This has to be fixed before PEP 352, when the exception
hierarchy is enforced, or the socket exceptions won't even work
right.
John:
Where did you get this information? If true it would certainly need to
be logged as a bug, but under Windows on 2,4 I see
issubclass(socket.gaierror, Exception)
True
and the same under Cygwin 2.5. I am presuming most other users will see
the same thing.
regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Socket exceptions aren't in the standard exception hierarchy
Steve Holden wrote:
John Nagle wrote:
Dennis Lee Bieber wrote:
On Sun, 22 Apr 2007 23:20:25 -0700, John Nagle [EMAIL PROTECTED]
declaimed the following in comp.lang.python:
2. File D:\Python24\lib\socket.py, line 295, in read
data = self._sock.recv(recv_size)
error: (10054, 'Connection reset by peer')
That looks like M$ Windows version of UNIX/Linux error number 54
(pretty much all Windows socket errors are UNIX number+1)
Errors coming from Windows may not be mapped to specific Python
exceptions, but rather to some general error conditions. {hypothesis} As
such, the Windows errors may not match what UNIX/Linux report.
Actually, that's not what's happening. The socket module is
explicitly raising socket.error in C code. It's not an OSError or
a WindowsError, although the text makes it look like one.
The problem is not that socket errors aren't entirely portable.
It's that they're not even in the Python exception hierarchy.
See http://docs.python.org/lib/module-exceptions.html;.
They have their own hierarchy, which starts at socket.error.
All built-in exceptions were supposed to be converted to the
standard exception hierarchy back before Python 2.0, but these
weren't.
Either they should be under IOError, or there should be
NetworkError under EnvironmentError, and they should be under
that. NetworkError, alongside IOError in the hierarchy,
would be useful. All the things that go wrong in networking
belong under there. Socket-level errors, SSL/TLS-level errors,
and HTTP/FTP/etc. level errors all belong under NetworkError.
This has to be fixed before PEP 352, when the exception
hierarchy is enforced, or the socket exceptions won't even work
right.
John:
Where did you get this information? If true it would certainly need to
be logged as a bug, but under Windows on 2,4 I see
issubclass(socket.gaierror, Exception)
True
and the same under Cygwin 2.5. I am presuming most other users will see
the same thing.
regards
Steve
Ah. socket.error is a subclass of Exception, but not
of StandardError.
issubclass(socket.error,StandardError)
is False.
John Nagle
--
http://mail.python.org/mailman/listinfo/python-list
Re: Socket exceptions aren't in the standard exception hierarchy
2. File D:\Python24\lib\socket.py, line 295, in read
data = self._sock.recv(recv_size)
error: (10054, 'Connection reset by peer')
That looks like M$ Windows version of UNIX/Linux error number 54
(pretty much all Windows socket errors are UNIX number+1)
Errors coming from Windows may not be mapped to specific Python
exceptions, but rather to some general error conditions. {hypothesis} As
such, the Windows errors may not match what UNIX/Linux report.
Actually, that's not what's happening. The socket module is
explicitly raising socket.error in C code. It's not an OSError or
a WindowsError, although the text makes it look like one.
The problem is not that socket errors aren't entirely portable.
It's that they're not even in the Python exception hierarchy.
See http://docs.python.org/lib/module-exceptions.html;.
They have their own hierarchy, which starts at socket.error.
All built-in exceptions were supposed to be converted to the
standard exception hierarchy back before Python 2.0, but these
weren't.
Either they should be under IOError, or there should be
NetworkError under EnvironmentError, and they should be under
that. NetworkError, alongside IOError in the hierarchy,
would be useful. All the things that go wrong in networking
belong under there. Socket-level errors, SSL/TLS-level errors,
and HTTP/FTP/etc. level errors all belong under NetworkError.
This has to be fixed before PEP 352, when the exception
hierarchy is enforced, or the socket exceptions won't even work
right.
John:
Where did you get this information? If true it would certainly need to
be logged as a bug, but under Windows on 2,4 I see
issubclass(socket.gaierror, Exception)
True
and the same under Cygwin 2.5. I am presuming most other users will see
the same thing.
regards
Steve
Ah. socket.error is a subclass of Exception, but not
of StandardError.
issubclass(socket.error,StandardError)
is False.
On linux, python 2.5:
import socket
issubclass(socket.error,Exception)
True
--
http://mail.python.org/mailman/listinfo/python-list
Re: Socket exceptions aren't in the standard exception hierarchy
John Nagle wrote: [socket.error bug report] Where did you get this information? If true it would certainly need to be logged as a bug, but under Windows on 2,4 I see issubclass(socket.gaierror, Exception) True and the same under Cygwin 2.5. I am presuming most other users will see the same thing. regards Steve Ah. socket.error is a subclass of Exception, but not of StandardError. issubclass(socket.error,StandardError) is False. Right, so this *is* a big, as long as Brett Cannon's upcoming (further) reorganization of the standard exception hierarchy doesn't stamp on it. It probably *was* overlooked in the reorganization of the hierarchy, and this implies there may be other extensions that also make the same error. It should be logged as a bug in the tracker - the fix is probably pretty simple, but it'll need some consideration of the forward compatibility considerations. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://del.icio.us/steve.holden Recent Ramblings http://holdenweb.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
