Andrei Zene <andrei.z...@outlook.com> added the comment:
We would also need this limit to be raised. We are using a server that was implemented using tornado, and tornado uses select.select. > It looks like Modules/selectmodule.c already has the handling for when this > gets large (and we really want to stop allocating the static array on the > stack), which is a plus, but I'd want to see the performance cost to small > selects if suddenly we're allocating 100s of KB on the heap rather than small > and cheap stack allocations. I gave this a try. The code I used for the server and the client for taken from this page: https://steelkiwi.com/blog/working-tcp-sockets/. I just added some code to measure the time it stays in the "select" call. To measure the time I used time.process_time(). I then connected/disconnected only 1 client 10.000 times in a row, discarding the measurements of the first 1000 iterations (warm-up). Here are the results: FD_SETSIZE = 16384 bytes Time elapsed (avg ns): 20029.566224207087 Time elapsed (max ns): 15625000 Time elapsed (min ns): 0 FD_SETSIZE = 512 bytes Time elapsed (avg ns): 14671.361502347418 Time elapsed (max ns): 15625000 Time elapsed (min ns): 0 > However, I see no reason why this couldn't be totally dynamic, at least on > Windows. Redefining that value just changes a static array definition, but > the array size never gets passed in to the OS AFAICT, so there's no reason we > couldn't stack allocate for small select()s and heap allocate for big > select()s. That's considerably more work than anyone had in mind, I'd wager. I'm not sure this can be totally dynamic on Windows. The problem is the fd_set struct which is defined in Winsock2.h typedef struct fd_set { u_int fd_count; /* how many are SET? */ SOCKET fd_array[FD_SETSIZE]; /* an array of SOCKETs */ } fd_set; The heap-allocated pylists are converted into fd_set structs which are always allocated on the stack btw. The problem is there's no way to create a fd_set of a desired FD_SETSIZE at runtime. I have also submitted a PR with the change for convenience: https://github.com/python/cpython/pull/13842. ---------- nosy: +andzn _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue28708> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
