[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: I'm very happy that the issue is finally resolved. But a bit offended that it took Andrew only 4 days to push it :) > On 07 Nov 2016, at 17:04, Yury Selivanov wrote: > > > Yury Selivanov added the comment: > > See https://github.com/python/asyncio/pull/452. I

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: See https://github.com/python/asyncio/pull/452. I believe that the issue is not resolved (almost). What's left is documenting how to use get_event_loop and when to use explicit/implicit loop passing. Closing this issue. Will open a new one for the documentat

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Vincent Michel added the comment: I agree with Yury's ideas about the implementation of this feature. However, it is a bit confusing to have `asyncio.get_event_loop` defined as: def get_event_loop(): policy = get_event_loop_policy() return policy.get_running_loop() or policy.get_event_l

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: Yury, as you suggested posted to python-ideas (https://groups.google.com/forum/#!topic/python-ideas/ABOe22Mib44) -- ___ Python tracker ___ __

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: > If I understood it correctly, get_event_loop() would never return "default" > event loop (in terms of current implementation) for a running task, because > it always be overridden with "running" event loop. If it's so, are you sure > it won't raise any probl

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: Yury, > `get_event_loop()` will then try to use the `running_loop` object first, and > if nothing is there, fall back to its current implementation. Do you think hiding "default" event loop completly from a currently executing coroutine is safe? -- _

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: Yury, > Not sure I understand the question. If I understood it correctly, get_event_loop() would never return "default" event loop (in terms of current implementation) for a running task, because it always be overridden with "running" event loop. If it's so, ar

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: > Do you think hiding "default" event loop completly from a currently executing > coroutine is safe? Not sure I understand the question. Anyways, let's see what Guido thinks about this all. -- ___ Python tracker

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: >> I now think that we don't need a new function for getting the currently >> running event loop. > May I ask you to elaborate on this? In my last message I propose to modify 'get_event_loop' to return the running loop reliably. To me this is to me the most

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: Yury, > I now think that we don't need a new function for getting the currently > running event loop. May I ask you to elaborate on this? Asynchronous API I'm aware of (including other languages) typically allows to get "main" (which in asyncio is lazily crea

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: > TBH, I don't fully understand Ilya's case with threads, synchronous > coroutines, possible deadlocks etc. I feel sorry for sharing that example. It didn't help and made my points regarding original issue unclear, hidden behind example's complexity. --

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: > Yury, what do you think of this yourself? Maybe you can explain it better than Ilya? TBH, I don't fully understand Ilya's case with threads, synchronous coroutines, possible deadlocks etc. So I'll just explain what I would want out of this. I don't like a f

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Guido van Rossum added the comment: As long as there are still things you find confusing about asyncio's model, you should probably consider yourself unqualified to start proposing new features... -- ___ Python tracker

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: Yury, we're building our own CPython anyway (and we just updated to 3.5.1). I'd be glad to test the patch during the next iteration. Guido, I think my use case mixes up other things I find confusing about asyncio: e.g. inablitity to synchronously perform code wr

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Guido van Rossum added the comment: Yury, what do you think of this yourself? Maybe you can explain it better than Ilya? -- ___ Python tracker ___ ___

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: Ilya, at this point I'd suggest you to craft a very clear and simple to digest email to python-ideas describing 'get_running_loop()', how it can be used, your case, other cases, etc. -- ___ Python tracker

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Guido van Rossum added the comment: Still no green light. Ilya just seems to want something misguided. -- ___ Python tracker ___ ___ P

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: > Yury, could you submit a patch implements this feature? Yes, the patch part is easy. The hard part is to get a green light from Guido on adding the 'get_running_loop()' function... -- ___ Python tracker

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: Yury, could you submit a patch implements this feature? -- ___ Python tracker ___ ___ Python-bugs-list

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: > If you're worried about blocking the event loop just to acquire the threading > lock that's part of threading.Event, you can use run_in_executor() I'm more worried to delay invocation of a "synchronous" report because of outstanding tasks in event loop's queue

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Guido van Rossum added the comment: Instead of starting a new event loop, you should figure out a way to wait for an event in the existing loop. IIUC that loop runs in a different thread -- I think you can solve this by using a threading.Event that you set from a wrapper coroutine running in your

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: > You don't know what else that coroutine is waiting for, and it may be waiting > for some I/O whose socket is registered with the other event loop. Since the > other event loop won't make progress, you'd be deadlocked. As an end user I know what my coroutines m

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Guido van Rossum added the comment: No, that specifically sounds like asking for trouble to me. You don't know what else that coroutine is waiting for, and it may be waiting for some I/O whose socket is registered with the other event loop. Since the other event loop won't make progress, you'd be

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Guido van Rossum added the comment: PS. If you have something you sometimes want to run synchronously and sometimes as a coroutine, you probably need to refactor it somehow. -- ___ Python tracker __

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: Guido, Probably a legitimate example of having multiple event loops in a single thread: you want to run certain coroutine synchronously without running thread's own event loop. -- ___ Python tracker

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Guido van Rossum added the comment: FWIW asyncio's own test suite makes sure that the loop is indeed passed everywhere by setting the default loop to None. If a library chooses to pass the loop around like this it should structure its tests the same way. -- _

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: >> Update some places in asyncio where we currently use "get_event_loop()", >> such as Future constructor, Task.current_task, etc. > Yury, do you have an idea how it could be done? I think I do. To keep the backwards compatibility, we'll need to update the "

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

R. David Murray added the comment: The way I avoid the errors of failing to always pass the loop through in my application library, currently, is to run my tests in a thread with no event loop set up, so that get_event_loop will raise an error. Take that for what it is worth, since I don't cl

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: > Update some places in asyncio where we currently use "get_event_loop()", such > as Future constructor, Task.current_task, etc. Yury, do you have an idea how it could be done? -- ___ Python tracker

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: Yury, that would do it. Guido, that's indeed might be an anti-pattern. But it looks like passing event loop around is just a worse version of it. -- ___ Python tracker

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: > Honestly I think it's pretty crazy and out there to have multiple event loops in the same thread. That feels like an anti-pattern inspired by some other event loop APIs (in other languages) that encourage this. But asyncio was not designed for that. I agree.

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Guido van Rossum added the comment: Honestly I think it's pretty crazy and out there to have multiple event loops in the same thread. That feels like an anti-pattern inspired by some other event loop APIs (in other languages) that encourage this. But asyncio was not designed for that. --

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: > Thread may have multiple event loops, but only one, explicitly associated, is > default. And it's not necessary one which is currently running. Ah, I got it. You know what, this actually is starting to make sense. Guido, what do you think about this? Essen

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: > In this case I'm not sure how this is different from the current > "get_event_loop" Thread may have multiple event loops, but only one, explicitly associated, is default. And it's not necessary one which is currently running. I think what I propose here can b

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: To clarify: the problem lies in "return currently running event loop when called from within a coroutine" -- what if there is no GIL and we have a few event loops running? You'd need to use a threadlocal storage. In this case I'm not sure how this is differe

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: > Probably it would make sense to modify behavior of the default policy to make > get_event_loop to return Thread-local event loop when called from > out-of-event-loop and return currently running event loop when called from > within a coroutine. The problem

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Ilya Kulakov added the comment: > Why does it have to be a standard part of asyncio? I've only seen few libraries that deal with asyncio so far (aiohttp, pyzmq), but my general impression is that this is a generic problem. With asyncio code should be (ideally) written as a set of coroutines wh

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

R. David Murray added the comment: Because, as indicated by the OP, I'm not using the default event loop of the thread. Thinking about my code, I now remember that the reason I went down this path was because I wanted to make sure that my functions, like asyncio functions, accepted the option

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Yury Selivanov added the comment: > Currently if one needs lazily resolve event loop depending on where awaitable > is being awaited have to pass loop everywhere explicitly. That quickly > becomes an unnecessary noise in interfaces of callables. Why don't you just use 'loop = asyncio.get_even

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

R. David Murray added the comment: Can we at least make it part of the documentation? It's not obvious to me, at least, how to do it. (Maybe I just haven't thought about it carefully enough.) For that matter, it wasn't obvious to me it could even be done, so I've been passing loop everywher

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

Guido van Rossum added the comment: Can't you easily write such a policy yourself? Why does it have to be a standard part of asyncio? -- ___ Python tracker ___ __

[issue26969] ascynio should provide a policy to address pass-loop-everywhere problem

New submission from Ilya Kulakov: Currently if one needs lazily resolve event loop depending on where awaitable is being awaited have to pass loop everywhere explicitly. That quickly becomes an unnecessary noise in interfaces of callables. Consider an example where a coroutine which construct