[issue28708] Low FD_SETSIZE limit on Windows

Change by Zachary Ware : -- Removed message: https://bugs.python.org/msg393447 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue28708] Low FD_SETSIZE limit on Windows

Nicole Allen added the comment: Thanks for putting that all together. Besides for amazing digital marketing plans visit us now https://www.seobrisk.com/locations/seo-detroit/ -- nosy: +nicoleallen ___ Python tracker

[issue28708] Low FD_SETSIZE limit on Windows

Change by Nathaniel Smith : -- Removed message: https://bugs.python.org/msg390813 ___ Python tracker ___ ___ Python-bugs-list

[issue28708] Low FD_SETSIZE limit on Windows

Change by Nathaniel Smith : -- Removed message: https://bugs.python.org/msg392595 ___ Python tracker ___ ___ Python-bugs-list

[issue28708] Low FD_SETSIZE limit on Windows

Change by Nathaniel Smith : -- Removed message: https://bugs.python.org/msg392594 ___ Python tracker ___ ___ Python-bugs-list

[issue28708] Low FD_SETSIZE limit on Windows

Molly Eskam added the comment: Is FD_SETSIZE configurable at runtime? https://webcare360.com/ -- ___ Python tracker ___ ___

[issue28708] Low FD_SETSIZE limit on Windows

Molly Eskam added the comment: That's a ton of content on a single page. https://offshorededi.com/ -- nosy: +MollyEskam ___ Python tracker ___

[issue28708] Low FD_SETSIZE limit on Windows

Alan added the comment: The big innovation with poll() is that it takes an array of descriptors like a normal function. https://www.geometrydash.me/ -- nosy: +alanpreston ___ Python tracker

[issue28708] Low FD_SETSIZE limit on Windows

Change by Eryk Sun : -- components: +Extension Modules -Build versions: +Python 3.10 -Python 3.6, Python 3.7 ___ Python tracker ___

[issue28708] Low FD_SETSIZE limit on Windows

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue28708] Low FD_SETSIZE limit on Windows

Change by Eryk Sun : -- Removed message: https://bugs.python.org/msg387680 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue28708] Low FD_SETSIZE limit on Windows

Anastasia Stefanyuk added the comment: Hello! This piece of code is kind of a pain to solve, I guess you can somehow use the information here: https://mobilunity.com/blog/hire-sap-full-stack-developer/ - this will definitely help in some way or another. -- nosy: +stefanyuk.ana

[issue28708] Low FD_SETSIZE limit on Windows

Andrei Zene added the comment: That's actually a great explanation Nathaniel. Thanks for putting that all together. -- ___ Python tracker ___

[issue28708] Low FD_SETSIZE limit on Windows

Nathaniel Smith added the comment: Traditionally on Unix, sockets are represented by file descriptors. File descriptors are small integers. (POSIX actually mandates the "small" part: whenever a new fd is created, the OS has to assign it the smallest integer value that's not already being

[issue28708] Low FD_SETSIZE limit on Windows

Andrei Zene added the comment: I have updated PR 13842(https://github.com/python/cpython/pull/13842) to dynamically alocate the fd_sets on windows. I did some testing on windows with the following two paterns: 1. 1 transient clients: (open connection, send message, close connection) 2.

[issue28708] Low FD_SETSIZE limit on Windows

Eryk Sun added the comment: > Those fd_sets are currently allocated on stack and the array they > contain is determined currently by FD_SETSIZE. We'll basically need > a similar structure that we can allocate dynamically, and cast it to > fd_set when passing it to the select call. This is

[issue28708] Low FD_SETSIZE limit on Windows

Andrei Zene added the comment: Awesome work Victor! Thanks! I'll give it a try. One thing I see mising though (if I understand well) is the allocation of three fd_sets (I posted a comment on github regarding that). Those fd_sets are currently allocated on stack and the array they contain is

[issue28708] Low FD_SETSIZE limit on Windows

Steve Dower added the comment: > So it's really FD_SETSIZE that matters, and it's a compile-time constant. The only way this constant can matter is if the array has to end with an empty slot, which doesn't appear to be the case. The pre-compiled function can't tell how you compiled the code

[issue28708] Low FD_SETSIZE limit on Windows

Change by Antoine Pitrou : -- nosy: +eryksun ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue28708] Low FD_SETSIZE limit on Windows

Antoine Pitrou added the comment: > The actual number of descriptors is passed in Yes, but: """The nfds parameter is included only for compatibility with Berkeley sockets.""" ... according to https://docs.microsoft.com/en-us/windows/desktop/api/Winsock2/nf-winsock2-select So it's really

[issue28708] Low FD_SETSIZE limit on Windows

Eryk Sun added the comment: > It doesn't have to. The actual number of descriptors is passed in, and To clarify, this is the fd_count in each fd_set, i.e. Winsock uses a counted SOCKET array instead of the bitmask that's found on other platforms. Winsock select() ignores nfds [1]. If

[issue28708] Low FD_SETSIZE limit on Windows

Andrew Svetlov added the comment: >From my understanding Steve is right. -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue28708] Low FD_SETSIZE limit on Windows

Steve Dower added the comment: > How does select() know the FD_SETSIZE? It doesn't have to. The actual number of descriptors is passed in, and the API presumably assumes that FD_SETSIZE is bigger than that (or else the caller would have crashed writing to invalid memory). I don't see why

[issue28708] Low FD_SETSIZE limit on Windows

Change by STINNER Victor : -- pull_requests: +13722 pull_request: https://github.com/python/cpython/pull/13845 ___ Python tracker ___

[issue28708] Low FD_SETSIZE limit on Windows

STINNER Victor added the comment: I wrote a proof-of-concept: PR 13845. ... but now I'm confused. How does select() know the FD_SETSIZE? Is it hardcoded in select() ABI? Does this PR work? Or does it make no sense? -- ___ Python tracker

[issue28708] Low FD_SETSIZE limit on Windows

Andrew Svetlov added the comment: I believe the number still can be configurable but requires more work. The idea is: FD_* macroses should be replaced with custom fd_set manipulation functions. on select.select() call we can calculate the number of the highest used socket and use this number

[issue28708] Low FD_SETSIZE limit on Windows

Andrei Zene added the comment: > Would it be possible to make FD_SETSIZE configurable at runtime, at least on > Windows? IMHO it would be a better approach. That would be awesome, but it doesn't look like it would be possible. As I have already pointed out in my previous comment: > I'm not

[issue28708] Low FD_SETSIZE limit on Windows

STINNER Victor added the comment: Would it be possible to make FD_SETSIZE configurable at runtime, at least on Windows? IMHO it would be a better approach. -- ___ Python tracker

[issue28708] Low FD_SETSIZE limit on Windows

Andrei Zene added the comment: We would also need this limit to be raised. We are using a server that was implemented using tornado, and tornado uses select.select. > It looks like Modules/selectmodule.c already has the handling for when this > gets large (and we really want to stop

[issue28708] Low FD_SETSIZE limit on Windows

Change by Andrei Zene : -- keywords: +patch pull_requests: +13719 stage: -> patch review pull_request: https://github.com/python/cpython/pull/13842 ___ Python tracker ___

[issue28708] Low FD_SETSIZE limit on Windows

desbma added the comment: I just want to say that I strongly support either bumping the value of FD_SETSIZE to something a lot higher than 512, or making it configurable from Python code. I am the author of a program that makes heavy use of asyncio. Some Windows users

[issue28708] Low FD_SETSIZE limit on Windows

Antoine Pitrou added the comment: > To implement an efficient event loop, IOCP is the way to do: the asyncio > module supports it using ProactorEventLoop. Not everyone uses asyncio, though, so it would still be nice to see the select() limit raised anyway. Implementing your own event loop

[issue28708] Low FD_SETSIZE limit on Windows

Steve Dower added the comment: It looks like Modules/selectmodule.c already has the handling for when this gets large (and we really want to stop allocating the static array on the stack), which is a plus, but I'd want to see the performance cost to small selects if suddenly we're allocating

[issue28708] Low FD_SETSIZE limit on Windows

STINNER Victor added the comment: To implement an efficient event loop, IOCP is the way to do: the asyncio module supports it using ProactorEventLoop. -- nosy: +haypo ___ Python tracker

[issue28708] Low FD_SETSIZE limit on Windows

Changes by Zachary Ware : -- components: +Build, Windows nosy: +paul.moore, steve.dower, tim.golden, zach.ware ___ Python tracker

[issue28708] Low FD_SETSIZE limit on Windows

New submission from David Hirschfeld: Back in 1999 the compile-time constant FD_SETSIZE was raised from (the default on Windows) 64 to 512 open sockets to support *serious async servers* http://bugs.python.org/issue210843