Re: [Python-Dev] Pervasive socket failures on Windows
Neal Norwitz wrote: On 2/11/06, Tim Peters [EMAIL PROTECTED] wrote: [Tim telling how I broke pyuthon] [Martin fixing it] Sorry for the breakage (I didn't know about the Windows issues). Thank you Martin for fixing it. I agree with the solution. I was away from mail, ahem, working. yeah, right, at your off-site boondoggle south of the border. we know. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC www.holdenweb.com PyCon TX 2006 www.python.org/pycon/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
On 10-feb-2006, at 23:49, Martin v. Löwis wrote: Tim Peters wrote: I don't know. Of course it misses similar new tests added to _ssl.c (see the msg that started this thread), so it spreads beyond just this. Does it do the right thing for Windows variants like Cygwin, and OS/2? Don't know. I see. How does Py_SOCKET_FD_CAN_BE_GE_FD_SETSIZE help here? Does defining it in PC/pyconfig.h do the right thing? I guess I'm primarily opposed to the visual ugliness of the define. Why does it spell out can be, but abbreviates greater than or equal to? What about Py_CHECK_FD_SETSIZE? If I understand this discussion correctly that code that would be conditionalized using this define is the IS_SELECTABLE macro in selectmodule.c and very simular code in other modules. I'd say that calling the test _Py_IS_SELECTABLE and putting it into pyport.h as Tim mentioned in an aside seems to be a good solution. At the very least it is a lot nicer than defining a very long name in pyconfig.h and then having very simular code in several #if blocks. Ronald Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ ronaldoussoren%40mac.com smime.p7s Description: S/MIME cryptographic signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Ronald Oussoren wrote: If I understand this discussion correctly that code that would be conditionalized using this define is the IS_SELECTABLE macro in selectmodule.c and very simular code in other modules. I'd say that calling the test _Py_IS_SELECTABLE and putting it into pyport.h as Tim mentioned in an aside seems to be a good solution. At the very least it is a lot nicer than defining a very long name in pyconfig.h and then having very simular code in several #if blocks. For the moment, I have committed Tim's original proposal. Moving the macro into pyport.h could be done in addition. That should be done only if selectmodule is also adjusted; this currently tests for _MSC_VER. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
[Martin v. Löwis] For the moment, I have committed Tim's original proposal. Thank you! I checked, and that fixed all the test failures I was seeing on Windows. Moving the macro into pyport.h could be done in addition. That should be done only if selectmodule is also adjusted; this currently tests for _MSC_VER. It's a nice illustration of why platform-dependent code sprayed across modules sucks, too. Why _MSC_VER instead of MS_WINDOWS? What's the difference, exactly? Who knows? I see that selectmodule.c has this comment near the top: Under BeOS, we suffer the same dichotomy as Win32; sockets can be anything = 0. but there doesn't appear to be any _code_ matching that comment in that module -- unless on BeOS _MSC_VER is defined. Beats me whether it is, but doubt it. The code in selectmodule when _MSC_VER is _not_ defined complains if a socket fd is = FD_SETSIZE _or_ is 0. But the new code in socketmodule on non-Windows boxes is happy with negative fds, saying fine whenever fd FD_SETSIZE. Is that right or wrong? The answer isn't so important to me as that this kind of crap always happens when platform-specific logic ends up getting defined in multiple modules. Much better to define macros to hide this junk, exactly once; pyport.h is the natural place for it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Tim Peters wrote: The code in selectmodule when _MSC_VER is _not_ defined complains if a socket fd is = FD_SETSIZE _or_ is 0. But the new code in socketmodule on non-Windows boxes is happy with negative fds, saying fine whenever fd FD_SETSIZE. Is that right or wrong? I think it is right: the code just knows that negative values cannot happen. The socket handles originate from system calls (socket(2), accept(2)), and a negative value returned there is an error. However, the system might (and did) return handles larger than FD_SETSIZE (as the kernel often won't know what value FD_SETSIZE has). The answer isn't so important to me as that this kind of crap always happens when platform-specific logic ends up getting defined in multiple modules. Much better to define macros to hide this junk, exactly once; pyport.h is the natural place for it. That must be done carefully, though. For example, how should the line max = 0; /* not used for Win32 */ be treated? Should we introduce a #define Py_SELECT_NUMBER_OF_FDS_PARAMETER_IS_IRRELEVANT? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
[Tim]
The code in selectmodule when _MSC_VER is _not_ defined complains if a
socket fd is = FD_SETSIZE _or_ is 0. But the new code in
socketmodule on non-Windows boxes is happy with negative fds, saying
fine whenever fd FD_SETSIZE. Is that right or wrong?
[Martin]
I think it is right: the code just knows that negative values
cannot happen. The socket handles originate from system calls
(socket(2), accept(2)), and a negative value returned there is
an error. However, the system might (and did) return handles
larger than FD_SETSIZE (as the kernel often won't know what
value FD_SETSIZE has).
Since the new code was just added, you can remember that now. No
comments record the reasoning, though, and over time it's likely to
become another mass of micro-optimized mystery code. If it's true
that negative values can't happen (and I believe that), then it
doesn't hurt to verify that they're = 0 either (except from a
micro-efficiency view), and it would simplify the code do to so.
The answer isn't so important to me as that this kind of crap always
happens when platform-specific logic ends up getting defined in
multiple modules. Much better to define macros to hide this junk,
exactly once; pyport.h is the natural place for it.
That must be done carefully, though. For example, how should
the line
max = 0; /* not used for Win32 */
be treated? Should we introduce a
#define Py_SELECT_NUMBER_OF_FDS_PARAMETER_IS_IRRELEVANT?
I wouldn't: I'd simply throw away the current confusing avoidance of
computing max on Windows. That's another case where
platform-specific micro-efficiency seems the only justification
(select() on Windows ignores its first argument; there's nothing
special about 0 here, despite that the code currently makes 0 _look_
special on Windows somehow).
So fine by me if the current:
#if defined(_MSC_VER)
max = 0; /* not used for Win32 */
#else /* !_MSC_VER */
if (v 0 || v = FD_SETSIZE) {
PyErr_SetString(PyExc_ValueError,
filedescriptor out of range in select());
goto finally;
}
if (v max)
max = v;
#endif /* _MSC_VER */
block got replaced by, e.g.,:
max = 0;
if (! Py_IS_SOCKET_FD_OK(v)) {
PyErr_SetString(PyExc_ValueError,
filedescriptor out of range in select());
goto finally;
}
if (v max)
max = v;
Unlike the current code, that would, for example, also allow for the
_possibility_ of checking that v != INVALID_SOCKET on Windows, by
fiddling the Windows expansion of Py_IS_SOCKET_FD_OK (and of course
all users of that macro would grow the same new smarts).
I'm not really a macro fan: I'm a fan of centralizing portability
hacks in config header files, and hiding them under abstractions. C
macros are usually strong enough to support this, and are all the
concession to micro-efficiency I'm eager ;-) to make.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
On 2/11/06, Tim Peters [EMAIL PROTECTED] wrote: [Tim telling how I broke pyuthon] [Martin fixing it] Sorry for the breakage (I didn't know about the Windows issues). Thank you Martin for fixing it. I agree with the solution. I was away from mail, ahem, working. n ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
On Fri, Feb 10, 2006 at 12:36:09AM -0500, Tim Peters wrote: [Tim] ... FD_SETSIZE is the maximum number of distinct fd's an fdset can hold, and the numerical magnitude of any specific fd has nothing to do with that in general (they may be related in fact on Unix systems that implement an fdset as a big bit vector -- but Windows doesn't work that way, and neither do all Unix systems, and nothing in socket specs requires an implementation to work that way). Hmm. Looks like POSIX _does_ require that. Can't work on Windows, though. I have a distinct memory of a 64-bit Unix that didn't work that way either, but while that memory is younger than I am, it's too old for me to recall more than just that ;-). Perhaps the memory you have is of select-lookalikes, like poll(), or maybe of vendor-specific (and POSIX-breaking) extensions to select(). select() performs pretty poorly on large fdsets with holes in, and has the fixed size fdset problem, so poll() was added to fix that (by Linux and later by XPG4, IIRC.) poll() takes an array of structs containing the fd, the operations to watch for and an output parameter with seen events. Does that jar your memory? :) (The socketmodule has support for poll(), on systems that have it, by the way.) -- Thomas Wouters [EMAIL PROTECTED] Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
[Thomas Wouters] Perhaps the memory you have is of select-lookalikes, like poll(), No, it was definitely select(), and on a 64-bit Unix (probably _not_ Linux) that allowed for an enormous number of sockets. or maybe of vendor-specific (and POSIX-breaking) extensions to select(). Yes, it must have been non-POSIX. select() performs pretty poorly on large fdsets with holes in, and has the fixed size fdset problem, so poll() was added to fix that (by Linux and later by XPG4, IIRC.) poll() takes an array of structs containing the fd, the operations to watch for and an output parameter with seen events. Does that jar your memory? :) No more than it had been jarred ;-) Well, a bit more: it was possible to pass a first argument to select() that was larger than FD_SETSIZE. In effect, FD_SETSIZE had no meaning. (The socketmodule has support for poll(), on systems that have it, by the way.) Yup. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Tim Peters wrote: No more than it had been jarred ;-) Well, a bit more: it was possible to pass a first argument to select() that was larger than FD_SETSIZE. In effect, FD_SETSIZE had no meaning. This begs the question then whether the check that is implemented has any relevance to any platform other than Linux. I am no portability guru, but I have to think there are other platforms where this patch will cause problems. For now at least, can we at least do some preprocessing magic to not use this code with Windows? -- Scott Dial [EMAIL PROTECTED] [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
On Fri, Feb 10, 2006 at 03:24:28PM -0500, Scott Dial wrote: Tim Peters wrote: No more than it had been jarred ;-) Well, a bit more: it was possible to pass a first argument to select() that was larger than FD_SETSIZE. In effect, FD_SETSIZE had no meaning. any relevance to any platform other than Linux. I am no portability guru, but I have to think there are other platforms where this patch will cause problems. For now at least, can we at least do some preprocessing magic to not use this code with Windows? I doubt it will have problems on other platforms. As Tim said, FD_SETSIZE is mandated by POSIX. Perhaps some platforms do allow larger sizes, by replacing the FD_* macros with functions that dynamically grow whatever magic is the 'fdset' datatype. I sincerely doubt it's a common approach, though, and for them to be POSIX they would need to have FD_SETSIZE set to some semi-sane value. So at worst, on those platforms (if any), we're reducing the number of sockets you can actually select() on, from some undefined platform maximum to whatever the platform *claims* is the maximum. -- Thomas Wouters [EMAIL PROTECTED] Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Scott Dial wrote: This begs the question then whether the check that is implemented has any relevance to any platform other than Linux. I am no portability guru, but I have to think there are other platforms where this patch will cause problems. The patch is right on all platforms conforming to the POSIX standard. POSIX says that FD_ISSET and friends have undefined behaviour if the file descriptor is larger than FD_SETSIZE. For platforms not conforming to the POSIX standard, the patch errs on the conservative side: it refuses to do something that POSIX says has undefined behaviour, yet may be well-defined on that platform. Disabling this for Windows is fine with me; I also think there should be some kind of documentation that quickly shows the potential cause of the exception Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Thomas Wouters wrote: I doubt it will have problems on other platforms. As Tim said, FD_SETSIZE is mandated by POSIX. Perhaps some platforms do allow larger sizes, by replacing the FD_* macros with functions that dynamically grow whatever magic is the 'fdset' datatype. I sincerely doubt it's a common approach, though, and for them to be POSIX they would need to have FD_SETSIZE set to some semi-sane value. So at worst, on those platforms (if any), we're reducing the number of sockets you can actually select() on, from some undefined platform maximum to whatever the platform *claims* is the maximum. I think the Windows interpretation is actually well-designed: FD_SETSIZE shouldn't be the number of the largest descriptor, but instead be the maximum size of the set. So FD_SETSIZE is 64 on Windows, but you still can have much larger file descriptor numbers. The implementation strategy of Windows is to use an array of integers, rather than the bit mask, and an index telling you how many slots have already been filled. With FD_SETSIZE being 64, the fd_set requires 256 bytes. This strategy has a number of interesting implications: - a naive implementation of FD_SET is not idempotent; old winsock implementations where so naive. So you might fill the set by setting the same descriptor 64 times. Current implementations use a linear search to make the operation idempotent. - FD_CLR needs to perform a linear scan for the descriptor, and then shift all subsequent entries by one (it could actually just move the very last entry to the deleted slot, but doesn't) In any case, POSIX makes it undefined what FD_SET does when the socket is larger than FD_SETSIZE, and apparently clearly expects an fd_set to be a bit mask. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
[Scott Dial] This begs the question then whether the check that is implemented has any relevance to any platform other than Linux. I am no portability guru, but I have to think there are other platforms where this patch will cause problems. For now at least, can we at least do some preprocessing magic to not use this code with Windows? We _have_ to gut this patch on Windows, because Python code using sockets on Windows no longer works. That can't stand. Indeed, I'm half tempted to revert the checkin right now since Python's test suite fails or hangs on Windows in test after test now. This at least blocks me from doing work I wanted to do (instead I spent the time allocated for that staring at test failures). I suggest skipping the new crud conditionalized on a symbol like Py_SOCKET_FD_CAN_BE_GE_FD_SETSIZE The Windows pyconfig.h can #define that, and other platforms can ignore its possible existence. If it applies to some Unix variant too, fine, that variant can also #define it. No idea here what the story is on, e.g., Cygwin or OS2. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Tim Peters wrote: I suggest skipping the new crud conditionalized on a symbol like Py_SOCKET_FD_CAN_BE_GE_FD_SETSIZE Hmm... How about this patch: Index: Modules/socketmodule.c === --- Modules/socketmodule.c (Revision 42308) +++ Modules/socketmodule.c (Arbeitskopie) @@ -396,7 +396,14 @@ static PyTypeObject sock_type; /* Can we call select() with this socket without a buffer overrun? */ +#ifdef MS_WINDOWS +/* Everything is selectable on Windows */ +#define IS_SELECTABLE(s) 1 +#else +/* POSIX says selecting descriptors above FD_SETSIZE is undefined + behaviour. */ #define IS_SELECTABLE(s) ((s)-sock_fd FD_SETSIZE) +#endif static PyObject* select_error(void) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
[Martin v. Löwis] I think the Windows interpretation is actually well-designed: FD_SETSIZE shouldn't be the number of the largest descriptor, but instead be the maximum size of the set. It's more that the fdset macros were well designed: correct code using FD_SET() etc is portable across Windows and Linux, and that's so because the macros define an interface rather than an implementation. BTW, note that the first argument to select() is ignored on Windows. So FD_SETSIZE is 64 on Windows, In Python FD_SETSIZE is 512 on Windows (see the top of selectmodule.c). but you still can have much larger file descriptor numbers. Which is the _source_ of the problem on Windows: Windows socket handles aren't file descriptors (if they were, they'd be little integers ;-)). ... In any case, POSIX makes it undefined what FD_SET does when the socket is larger than FD_SETSIZE, and apparently clearly expects an fd_set to be a bit mask. Yup -- although the people who designed the fdset macros to begin with didn't appear to have this assumption. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Martin v. Löwis wrote: Tim Peters wrote: I suggest skipping the new crud conditionalized on a symbol like Py_SOCKET_FD_CAN_BE_GE_FD_SETSIZE Hmm... How about this patch: Index: Modules/socketmodule.c === --- Modules/socketmodule.c (Revision 42308) +++ Modules/socketmodule.c (Arbeitskopie) @@ -396,7 +396,14 @@ static PyTypeObject sock_type; /* Can we call select() with this socket without a buffer overrun? */ +#ifdef MS_WINDOWS +/* Everything is selectable on Windows */ +#define IS_SELECTABLE(s) 1 +#else +/* POSIX says selecting descriptors above FD_SETSIZE is undefined + behaviour. */ #define IS_SELECTABLE(s) ((s)-sock_fd FD_SETSIZE) +#endif static PyObject* select_error(void) Regards, Martin That is the exact patch I applied, but you also need to patch _ssl.c --- C:/python-trunk/Modules/_ssl.c (revision 42305) +++ C:/python-trunk/Modules/_ssl.c (working copy) @@ -376,9 +376,11 @@ if (s-sock_fd 0) return SOCKET_HAS_BEEN_CLOSED; +#ifndef MS_WINDOWS /* Guard against socket too large for select*/ if (s-sock_fd = FD_SETSIZE) return SOCKET_INVALID; +#endif /* Construct the arguments to select */ tv.tv_sec = (int)s-sock_timeout; But then that leaves whether to go with the Py_SOCKET_FD_CAN_BE_GE_FD_SETSIZE symbol instead of MS_WINDOWS. -- Scott Dial [EMAIL PROTECTED] [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Tim Peters wrote: [Martin v. Löwis] So FD_SETSIZE is 64 on Windows, In Python FD_SETSIZE is 512 on Windows (see the top of selectmodule.c). Although I agree, in terms of the socketmodule, there was no such define overriding the default FD_SETSIZE, so you are both right. -- Scott Dial [EMAIL PROTECTED] [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Tim Peters wrote: I don't know. Of course it misses similar new tests added to _ssl.c (see the msg that started this thread), so it spreads beyond just this. Does it do the right thing for Windows variants like Cygwin, and OS/2? Don't know. I see. How does Py_SOCKET_FD_CAN_BE_GE_FD_SETSIZE help here? Does defining it in PC/pyconfig.h do the right thing? I guess I'm primarily opposed to the visual ugliness of the define. Why does it spell out can be, but abbreviates greater than or equal to? What about Py_CHECK_FD_SETSIZE? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
[Martin v. Löwis] I see. How does Py_SOCKET_FD_CAN_BE_GE_FD_SETSIZE help here? By naming a logical condition as opposed to a list of platform-specific symbols that aren't documented anywhere. For example, I have no idea exactly which compiler+OS combinations define MS_WINDOWS, so #ifdef MS_WINDOWS is always something of a mystery. I don't want to see mystery-symbols inside modules -- to the extent that they must be used, I want to hide them in .h files clearly dedicated to wrestling with portability headaches (like pyconfig.h and pyport.h). Does defining it in PC/pyconfig.h do the right thing? That much would stop the test failures _I_ see, which is what I need to get unstuck. If POSIX systems simply ignore it, it would do the right thing for them too. Documentation in pyport.h would serve to guide others (in the Config #defines referenced here: comments near the top of that file). I don't know what other systems need, so assuming we have to do something _at all_ here, the best I can do is provide documented macros and config symbols to deal with it. I think the relationship between SIGNED_RIGHT_SHIFT_ZERO_FILLS and pyport.h's Py_ARITHMETIC_RIGHT_SHIFT macro is a good analogy here. Almost everyone ignores SIGNED_RIGHT_SHIFT_ZERO_FILLS, and that's fine, because almost all C compilers generate code to do sign-extending right shifts. If someone has a box that doesn't, fine, it's up to them to get SIGNED_RIGHT_SHIFT_ZERO_FILLS #define'd in their pyconfig.h, and everything else just works for them then. All other platforms can remain blissfully ignorant. I guess I'm primarily opposed to the visual ugliness of the define. I don't much care how it's spelled. Why does it spell out can be, but abbreviates greater than or equal to? Don't care. I don't know of a common abbrevation for can be, but GE same-as = is in my Fortran-trained blood :-) What about Py_CHECK_FD_SETSIZE? That's fine, except I think it would be pragmatically better to make it Py_DONT_CHECK_FD_SETSIZE, since most platforms want to check it. The platforms that don't want this check (like Windows) are the oddballs, so it's better to default to checking, making the oddballs explicitly do something to stop such checking. It's no problem to add a #define to PC/pyconfig.h, since that particular config file is 100% hand-written (and always will be). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Tim Peters wrote: ? Sorrry, don't know what you're talking about here. Python's selectmodule.c #defines FD_SETSIZE before it includes winsock.h on Windows, so Microsoft's default is irrelevant to Python. The reason selectmodule.c uses !defined(FD_SETSIZE) in its Not that this is really that important, but if we are talking about as the code stands right now, IS_SELECTABLE uses FD_SETSIZE with no such define ever appearing. That is what I meant, and I am pretty sure that is where Martin came up with saying it was 64. But like I say.. it's not that important. Sorry for the noise. -- Scott Dial [EMAIL PROTECTED] [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Tim Peters wrote: Does it do the right thing for Windows variants like Cygwin, and OS/2? I can at least say that the Cygwin implements a full POSIX facade in front of Windows sockets, so it would be important that the code in question is used to protect it as well. Also, MS_WINDOWS is not defined for a Cygwin compile, so it is fine to be using that. But I realize there is a whole 'nother discussion about that. -- Scott Dial [EMAIL PROTECTED] [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
Tim Peters wrote: I _suspect_ that rev 42253 introduced these problems. For example, that added: + /* Guard against socket too large for select*/ + if (s-sock_fd = FD_SETSIZE) + return SOCKET_INVALID; to _ssl.c, and added +/* Can we call select() with this socket without a buffer overrun? */ +#define IS_SELECTABLE(s) ((s)-sock_fd FD_SETSIZE) to socketmodule.c, but those appear to make no sense. FD_SETSIZE is the maximum number of distinct fd's an fdset can hold, and the numerical magnitude of any specific fd has nothing to do with that in general (they may be related in fact on Unix systems that implement an fdset as a big bit vector -- but Windows doesn't work that way, and neither do all Unix systems, and nothing in socket specs requires an implementation to work that way). Neal checked these changes in to address bug #876637 Random stack corruption from socketmodule.c But the Windows implementation of select is entirely different than other platforms, in so far as windows uses an internal counter to assign fds to an fd_set, so the fd number itself has no relevance to where they are placed in an fd_set. This stack corruption bug then does not exist on Windows, and so the code should not be used with Windows either. -- Scott Dial [EMAIL PROTECTED] [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pervasive socket failures on Windows
[Tim] ... FD_SETSIZE is the maximum number of distinct fd's an fdset can hold, and the numerical magnitude of any specific fd has nothing to do with that in general (they may be related in fact on Unix systems that implement an fdset as a big bit vector -- but Windows doesn't work that way, and neither do all Unix systems, and nothing in socket specs requires an implementation to work that way). Hmm. Looks like POSIX _does_ require that. Can't work on Windows, though. I have a distinct memory of a 64-bit Unix that didn't work that way either, but while that memory is younger than I am, it's too old for me to recall more than just that ;-). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
