[issue7995] On Mac / BSD sockets returned by accept inherit the parent's FD flags
Justin Cappos added the comment: >> The Python implementation sets timeout=None (which implies that the >> underlying socket is blocking). > >No, it doesn't. A socket may be non-blocking without having a timeout; > that's the socket API (on all systems, not just BSD). Sure and this happens when the timeout is 0, but None has a different meaning than 0. >> The problem is that it has. It has created a new Python socket >> object with a specific value for timeout (None), but the underlying >> socket is nonblocking. >> >> The docs state that timeout = None makes the socket blocking. > > What specific wording are you looking at that makes you believe so? Here is the last part of the description of settimeout: s.settimeout(None) is equivalent to s.setblocking(1) if you look at setblocking: Set blocking or non-blocking mode of the socket: if flag is 0, the socket is set to non-blocking, else to blocking mode. This seems to imply that timeout = None -> blocking. -- ___ Python tracker <http://bugs.python.org/issue7995> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7995] On Mac / BSD sockets returned by accept inherit the parent's FD flags
Justin Cappos added the comment: > > Well, I don't think setting a timeout on a listening socket and then > > expecting the socket received through accept() to be non-blocking (but > > only on BSD) is a legitimate application. > > > Right. But setting the server socket to nonblocking, and then > expecting the connection socket to also be nonblocking might be. Okay sure. This is fine. That is why I suggested that if you don't like my patch, one might instead change new Python sockets to inherit the timeout / blocking setting on BSD. However, I hope we can all agree that having the Python socket object in a different blocking / non-blocking state than the OS socket descriptor is wrong. This is what needs to be fixed. -- ___ Python tracker <http://bugs.python.org/issue7995> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7995] On Mac / BSD sockets returned by accept inherit the parent's FD flags
Justin Cappos added the comment: >> This implementation assumes that the OS sets any socket it receives >> via accept to nonblocking. (this is a false assumption on BSD) > > Not true. It doesn't assume that (it doesn't assume the reverse, > either). The Python implementation sets timeout=None (which implies that the underlying socket is blocking). >> The end result is that the OS has a nonblocking socket and the Python >> object thinks it is blocking. This is why the socket object in >> Python has timeout=None yet calling fcntl shows the socket is >> nonblocking. > > That conclusion is flawed. Python has not associated a timeout with > the socket. It makes no claims as to whether the socket is blocking > or not. So you have created a non-blocking socket without timeout. The problem is that it has. It has created a new Python socket object with a specific value for timeout (None), but the underlying socket is nonblocking. The docs state that timeout = None makes the socket blocking. -- ___ Python tracker <http://bugs.python.org/issue7995> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7995] On Mac / BSD sockets returned by accept inherit the parent's FD flags
Justin Cappos added the comment: > Apparently, the designers of BSD thought differently. Remember that > it is them who defined the socket API in the first place, so they > have the right that their design decisions are considered. I think there is a bit of confusion here. The 'bug' isn't with different socket semantics on different OSes. The bug is that the programmer who wrote the wrapper for sockets on Python assumed the OS semantics weren't the BSD style. Here is the issue (put plainly): Python sockets support a notion of timeout (note this notion is not reflected in the OS socket API). The python socket implementation of timeouts uses the underlying OS / socket API to provide this by setting the socket to nonblocking and setting a timeout value in a Python object that holds socket info. This implementation assumes that the OS sets any socket it receives via accept to nonblocking. (this is a false assumption on BSD) The end result is that the OS has a nonblocking socket and the Python object thinks it is blocking. This is why the socket object in Python has timeout=None yet calling fcntl shows the socket is nonblocking. Calling code paths that handle timeouts and expect the socket to block causes bugs like I described in my code. This behavior is clearly wrong under any interpretation! You can debate whether the right patch is to use what I proposed or instead change new Python sockets to inherit the timeout / blocking setting on BSD. However, what is implemented now is clearly broken. -- ___ Python tracker <http://bugs.python.org/issue7995> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7850] platform.system() should be "macosx" instead of "Darwin" on OSX
Justin Cappos added the comment: Perhaps the right way to fix the problem without breaking code would be to propose a new function for platform which would return a 'newbie readable' string of the system type? -- nosy: +Justin.Cappos ___ Python tracker <http://bugs.python.org/issue7850> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7995] On Mac / BSD sockets returned by accept inherit the parent's FD flags
New submission from Justin Cappos : Suppose there is a program that has a listening socket that calls accept to obtain new sockets for client connections. socketmodule.c assumes that these client sockets have timeouts / blocking in the default state for new sockets (which on most systems means the sockets will block). However, socketmodule.c does not verify the state of the socket object that is returned by the system call accept. >From http://linux.die.net/man/2/accept : On Linux, the new socket returned by accept() does not inherit file status flags such as O_NONBLOCK and O_ASYNC from the listening socket. This behaviour differs from the canonical BSD sockets implementation. Portable programs should not rely on inheritance or non-inheritance of file status flags and always explicitly set all required flags on the socket returned from accept(). socketmodule.c does not explicitly set or check these flags for sockets returned by accept. The attached program will print the following on Linux regardless of whether the settimeout line for s exists or not: a has timeout: None O_NONBLOCK is set: False received: hi On Mac / BSD, the program will produce the following output when the timeout is set on the listening socket: a has timeout: None O_NONBLOCK is set: True Traceback (most recent call last): File "python-nonportable.py", line 39, in message = a.recv(1024) socket.error: (35, 'Resource temporarily unavailable') When the timeout is removed, the behavior is the same as linux: a has timeout: None O_NONBLOCK is set: False received: hi Note that the file descriptor problem crops up in odd ways on Mac systems. It's possible that issue 5154 may be due to this bug. I am aware of other problems with the socketmodule on Mac and will report them in other tickets. I believe that this problem can be easily mitigated by always calling fcntl to unset the O_NONBLOCK flag after accept (O_ASYNC should be unset too, for correctness). I would recommend adding the below code snippet at line 1653 in socketmodule.c (r78335). The resulting code would look something like this (with '+' in front of the added lines): ''' #ifdef MS_WINDOWS if (newfd == INVALID_SOCKET) #else if (newfd < 0) #endif return s->errorhandler(); +#if defined(__APPLE__) || defined(__OpenBSD__) || defined(__FreeBSD__) +int starting_flag; +// Unset O_NONBLOCK an O_ASYNC if they are inherited. +starting_flag = fcntl(newfd, F_GETFL, 0); +starting_flag &= ~(O_NONBLOCK | O_ASYNC); +fcntl(newfd, F_SETFL, starting_flag); +#endif /* Create the new object with unspecified family, to avoid calls to bind() etc. on it. */ sock = (PyObject *) new_sockobject(newfd, s->sock_family, s->sock_type, s->sock_proto); ''' I've tested this patch on my Mac and Linux systems and it seems to work fine. I haven't had a chance to test on BSD. Also, I did not test for this problem in Python 3, but I assume it exists there as well and the same fix should be applied. -- assignee: ronaldoussoren components: Library (Lib), Macintosh files: python-nonportable.py messages: 99852 nosy: Justin.Cappos, bbangert, giampaolo.rodola, loewis, nicdumz, ronaldoussoren severity: normal status: open title: On Mac / BSD sockets returned by accept inherit the parent's FD flags type: behavior versions: Python 2.5, Python 2.6, Python 2.7 Added file: http://bugs.python.org/file16325/python-nonportable.py ___ Python tracker <http://bugs.python.org/issue7995> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3144] popen / popen[234] inconsistent fd behavior
New submission from Justin Cappos <[EMAIL PROTECTED]>:
The behavior of popen vs popen[2-4] differs with respect to open file
descriptors (at least on the Linux implementation of popen). popen
does not close file descriptors, thus processes retain open file
descriptors from their parent. This is likely not desirable for
security and stability reasons.
If this isn't fixed, at a minimum it would be a good thing to document.
Here is an example that demonstrates the issue:
<<< start of open_and_popen.py>>>
# This will not be printed if popen closes file descriptors
import os
myfd = os.open("open_and_popen.py",os.O_RDONLY)
readfo = os.popen("python print_from_fd.py "+str(myfd),"r")
print "os.popen results in:"
print readfo.read()
# it will print the first line of the file here
readfo.close()
(junkinfo, readfo) = os.popen2("python print_from_fd.py "+str(myfd),"r")
junkinfo.close()
print "os.popen2 results in:"
print readfo.read()
# the child got an error, so this is just the error text
readfo.close()
os.close(myfd)
<<< end of open_and_popen.py>>>
<<< start of print_from_fd.py>>>
import os
import sys
print os.read(int(sys.argv[1]),60)
<<< end of print_from_fd.py>>>
--
components: Library (Lib)
messages: 68416
nosy: justincappos
severity: normal
status: open
title: popen / popen[234] inconsistent fd behavior
type: security
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3144>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
