[issue21998] asyncio: a new self-pipe should be created in the child process after fork

STINNER Victor added the comment: Updated patch: - drop PollSelector._at_fork(): PollSelector is not shared with the parent process - _at_fork() of BaseEventLoop and SelectorEventLoop now do nothing by default: only _UnixSelectorEventLoop._at_fork() handle the fork, nothing is needed on Window

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

STINNER Victor added the comment: close_self_pipe_after_selector.patch only fixes test2.py, it doesn't fix the general case: run the "same" event loop in two different event loops. -- ___ Python tracker __

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

STINNER Victor added the comment: Attached at_fork.patch: detect fork and handle fork. * Add _at_fork() method to asyncio.BaseEventLoop * Add _detect_fork() method to asyncio.BaseEventLoop * Add _at_fork() method to selectors.BaseSelector I tried to minimize the number of calls to _detect_fork(

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

STINNER Victor added the comment: I suggest to split this issue: create a new issue focus on selectors.EpollSelector, it doesn't behave well with forking. If I understood correctly, you can workaround this specific issue by forcing the selector to selectors.SelectSelector for example, right?

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

Martin Richard added the comment: Currently, this is what I do in the child after the fork: >>> selector = loop._selector >>> parent_class = selector.__class__.__bases__[0] >>> selector.unregister = lambda fd: parent_class.unregister(selector, fd) It replaces unregister() by _BaseSelectorImpl.u

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

Guido van Rossum added the comment: How about not doing anything in the parent but in the child, closing the selector and then the event loop? On Mon, Dec 1, 2014 at 1:29 AM, Martin Richard wrote: > > Martin Richard added the comment: > > I said something wrong in my previous comment: removing

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

Martin Richard added the comment: I said something wrong in my previous comment: removing and re-adding the reader callback right after the fork() is obviously subject to a race condition. I'll go for the monkey patching. -- ___ Python tracker

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

Martin Richard added the comment: Guido, Currently in my program, I manually remove and then re-adds the reader to the loop in the parent process right after the fork(). I also considered a dirty monkey-patching of remove_reader() and remove_writer() which would act as the original versions b

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

Guido van Rossum added the comment: Martin, what is the magic call to make in the child (or in the parent pre-fork???) to disable the epoll object in the child without disabling it in the parent? (Perhaps just closing the selector and letting the unregister calls fail would work?) --

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

Martin Richard added the comment: Hi, Actually, closing and creating a new loop in the child doesn't work either, at least on Linux. When, in the child, we call loop.close(), it performs: self.remove_reader(self._ssock) (in selector_events.py, _close_self_pipe() around line 85) Both the paren

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

Guido van Rossum added the comment: That sounds about right -- it's a doc issue. Let me propose a paragraph: """ NOTE: It is not safe to share an asyncio event loop between processes that are related by os.fork(). If an event loop exists in a process, and that process is forked, the only safe

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

STINNER Victor added the comment: > Is there a use case for sharing an event loop across forking? I don't know if asyncio must have a builtin support for this use case. The minimum is to document the behaviour, or maybe even suggest a recipe to support it. For example, an event loop of asynci

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

Yury Selivanov added the comment: > Is there a use case for sharing an event loop across forking? I don't think so. I use forking mainly for the following two use-cases: - Socket sharing for web servers. Solution: if you want to have a shared sockets between multiple child processes, just ope

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

Guido van Rossum added the comment: Actually I expect that if you share an event loop across different processes via form, everything's a mess -- whenever a FD becomes ready, won't both the parent and the child be woken up? Then both would attempt to read from it. One would probably get EWOU

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

Yury Selivanov added the comment: It's not that it doesn't work after fork, right? Should we add a recipe with pid monitoring a self-pipe re-initialization? -- ___ Python tracker __

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

STINNER Victor added the comment: A first step would be to document the issue in the "developer" section of asyncio documentation. Mention that the event loop should be closed and a new event loop should be created. -- ___ Python tracker

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

STINNER Victor added the comment: Oh, I wanted to use the atfork module for that, but then I saw that it does not exist :-( -- ___ Python tracker ___ ___

[issue21998] asyncio: a new self-pipe should be created in the child process after fork

New submission from STINNER Victor: It looks like asyncio does not suppport fork() currently: on fork(), the event loop of the parent and the child process share the same "self pipe". Example: --- import asyncio, os loop = asyncio.get_event_loop() pid = os.fork() if pid: print("parent", loo