[issue37373] Configuration of windows event loop for libraries

Ben Darnell added the comment: > It's even slightly easier for tornado, which can reasonably set the > proactor-wrapper policy at IOLoop start time, which means > `asyncio.get_event_loop()` returns a loop with add_reader. But pyzmq doesn't > get invoked until an event loop is already

[issue37373] Configuration of windows event loop for libraries

Min RK added the comment: A hiccup to using uvloop is that it doesn't support Windows yet (https://github.com/MagicStack/uvloop/issues/14), so it can't be used in the affected environment. I'm exploring this again for pyzmq / Jupyter, and currently investigating relying on tornado's

[issue37373] Configuration of windows event loop for libraries

Change by jack1142 : -- nosy: +jack1142 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue37373] Configuration of windows event loop for libraries

Change by Chris Meyer : -- nosy: +cmeyer ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue37373] Configuration of windows event loop for libraries

Ben Darnell added the comment: > Would it be acceptable for you to *require* use of uvloop when Tornado is > used with AsyncIO? How, exactly? Adding the dependency is no problem, but AFAIK I'd still be stuck with an import-time side effect to set the event loop policy (or a .pth file hack,

[issue37373] Configuration of windows event loop for libraries

Łukasz Langa added the comment: I'd be +1 to bringing uvloop into the stdlib, it would solve many things while introducing an acceptable dependency in the form of libuv. However, uvloop itself is written in Cython which makes it impossible for us to directly merge it. So that option is

[issue37373] Configuration of windows event loop for libraries

Ben Darnell added the comment: I considered using the `selectors` module directly, but it's not as simple as it sounds. Using the low-level interface means you need to also use a self-waker-pipe (or socket on windows) and manage a queue analogous to that used by `call_soon_threadsafe`. We

[issue37373] Configuration of windows event loop for libraries

Nathaniel Smith added the comment: > was Tornado the only project experiencing this pain At least twisted and anyio are still broken on 3.8+Windows because of this change: https://twistedmatrix.com/trac/ticket/9766 https://github.com/agronholm/anyio/issues/77 --

[issue37373] Configuration of windows event loop for libraries

Andrew Svetlov added the comment: >From my understanding, the issue is still desirable but we have a lack of >resources. Regarding the patch for tornado, using selectors.SelectSelector for implementing add_reader()/remove_reader() and add_writer()/remove_writer() in ProactorEventLoop looks

[issue37373] Configuration of windows event loop for libraries

Ben Darnell added the comment: I have an implementation of the selector-in-another-thread solution in https://github.com/tornadoweb/tornado/pull/2815. Is something like this worth considering for Python 3.9, or was Tornado the only project experiencing this pain and a tornado-specific

[issue37373] Configuration of windows event loop for libraries

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue37373] Configuration of windows event loop for libraries

Change by Michael Hall : -- nosy: +mikeshardmind ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue37373] Configuration of windows event loop for libraries

Change by Steve Dower : -- nosy: +steve.dower ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue37373] Configuration of windows event loop for libraries

Change by Carlton Gibson : -- nosy: +carltongibson ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue37373] Configuration of windows event loop for libraries

STINNER Victor added the comment: > Is it be possible to backport this inside the standard ProactorEventLoop of > Python-3.8.1 ? As things are currently broken, no kitten would be armed > https://github.com/python-trio/trio/pull/1269 No, we don't add features to minor releases. If you need

[issue37373] Configuration of windows event loop for libraries

Nathaniel Smith added the comment: Yeah, that's the question. I've dug into the AFD_POLL stuff more now, and... There's a lot of edge cases to think about. It certainly *can* be done, because this is the primitive that 'select' and friends are built on, so obviously it can do anything they

[issue37373] Configuration of windows event loop for libraries

Andrew Svetlov added the comment: IOCTL_AFD_POLL looks interesting. I wonder if it is 100% reliable; we can miss some edge case easily. -- ___ Python tracker ___

[issue37373] Configuration of windows event loop for libraries

Big Stone added the comment: Is it be possible to backport this inside the standard ProactorEventLoop of Python-3.8.1 ? As things are currently broken, no kitten would be armed https://github.com/python-trio/trio/pull/1269 -- ___ Python tracker

[issue37373] Configuration of windows event loop for libraries

Nathaniel Smith added the comment: It's also possible to implement 'select' on top of IOCP using undocumented private APIs. That's what libuv does (and how 'select' works internally). There's a bunch of information on this collected here: https://github.com/python-trio/trio/issues/52

[issue37373] Configuration of windows event loop for libraries

STINNER Victor added the comment: Ben: > Concretely, this is a concern for Tornado (which requires add_reader()) and > applications in the scientific python community (including Jupyter) which > depend on it. If you need add_reader/add_writer in Python 3.8, you can switch the default event

[issue37373] Configuration of windows event loop for libraries

Andrew Svetlov added the comment: I don't like to have not required parts like thread pool, time handlers, and the exception handler executed in the auxiliary thread. What's about an alternative proposal: embed into ProactorEventLoop not entire SelectorEventLoop but selectors.SelectSelector

[issue37373] Configuration of windows event loop for libraries

Yury Selivanov added the comment: +1 to what Ben said. Andrew, > The proposal looks like a very dirty hack and smells as it is. I wonder why are you feeling like that about this idea. I don't think this is a hack at all. There's no native API, so we're forced to use another approach. >

[issue37373] Configuration of windows event loop for libraries

Ben Darnell added the comment: Yeah, it's definitely a hack. The argument for it, at best, is "practicality beats purity". The solution is two simple lines, but those lines need to be repeated in every project that depends on Tornado and cares about Windows, now or in the future. How many

[issue37373] Configuration of windows event loop for libraries

Andrew Svetlov added the comment: I doubt if we should split a proactor loop into two objects in two threads. I see no strong objections but have a feeling that we'll get other problems in this way. asyncio is just not designed for this mode. The proposal looks like a very dirty hack and

[issue37373] Configuration of windows event loop for libraries

Yury Selivanov added the comment: Andrew, > Sorry, no. There is no way IIUC. Well, in this case we should do that via a thread+select approach as I and Ben suggested. I can write some PoC code; will you have some time to polish it and commit it? (I'm OOO and don't have a windows

[issue37373] Configuration of windows event loop for libraries

Ben Darnell added the comment: > From my understanding, there is no issue for Tornado itself. If Jupiter > Notebook needs Tornado, Tornado needs selector event loop on Windows -- > Jupiter can install the proper loop. Right. I'm just advocating for something that would make the transition

[issue37373] Configuration of windows event loop for libraries

Andrew Svetlov added the comment: > Is it possible to implement add_reader/writer in ProactorEventLoop? Sorry, no. There is no way IIUC. Regarding a request for selecting event loop by a library. 1. You can call asyncio.set_event_loop_policy() already. 2. From my understanding, there is no

[issue37373] Configuration of windows event loop for libraries

Big Stone added the comment: Windows users would certainly prefer to have an emulated non-performant emulation of the old API, rather than a breakage of their beloved Jupyter/Tornado ecosystem. -- nosy: +Big Stone ___ Python tracker

[issue37373] Configuration of windows event loop for libraries

Yury Selivanov added the comment: Victor, Andrew, I'm not an expert in IOCP, but is it possible to implement add_reader/writer in ProactorEventLoop? If there's no native API for that, I guess we can spawn a thread with a 'select()' call to emulate this API? Lukasz, Another question: if

[issue37373] Configuration of windows event loop for libraries

New submission from Ben Darnell : On Windows there are two event loop implementions with different interfaces: The proactor event loop is missing the file descriptor family of methods (add_reader()), while the selector event loop has other limitations including missing support for pipes and