Nathaniel Smith <n...@pobox.com> added the comment: There's two separate issues here: the warning spew because asyncio's internal signal handling code starts losing signals when they arrive too quickly, and the way the child reaping loop polls all the children on every SIGCHLD, which makes reaping N children an O(N**2) operation.
The warning spew is inherent in asyncio's current signal handling design. (BTW, Victor, here's an answer to your question in https://bugs.python.org/issue30050#msg291533 about whether this overflow can happen in real life :-).) It's not *terribly* harmful – losing some SIGCHLDs among thousands doesn't matter. It does mean that you could lose other signals if they happen to arrive at the same time, e.g. SIGINT and SIGTERM are probably ignored while this is happening. For the O(N**2) issue, I think you can work around it by using some incantation involving set_child_watcher and the FastChildWatcher class. These aren't documented (maybe someone should document them!) and I don't know the exact details off the top of my head, but it is a public interface for making your child reaping O(N). (This uses the wait(-1, ...) trick. Unfortunately there are arcane technical limitations with signalfd and si_pid that mean they can't solve this problem. Something like CLONE_FD would help, but that patch was never merged into Linux.) You might also consider switching to something like uvloop, which has a more robust event loop implementation underneath. I haven't checked, but I'm pretty sure it'd fix the signal buffer overflow issue, and I doubt they use an O(N**2) child reaping algorithm. ---------- nosy: +njs, vstinner _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32776> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
