[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Andrew Svetlov added the comment: Implemented by #32591 -- resolution: -> duplicate stage: patch review -> resolved status: open -> closed superseder: -> Deprecate sys.set_coroutine_wrapper and replace it with more focused API(s) ___ Python

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Guido van Rossum added the comment: Serhiy, that’s brilliant! Let’s do it. -- nosy: +Guido.van.Rossum ___ Python tracker ___ ___

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Serhiy Storchaka added the comment: The most common error is missing keyword "await" in function call. "f()" instead of "await f()". There is a way to detect this error at runtime with minimal false positive and with minimal overhead. We can add a new opcode which checks if the value on the

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Change by Nicholas Smith : -- nosy: +nzsmith ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Change by Nikolay Bryskin : -- nosy: +nikicat ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Andrew Svetlov added the comment: Let's keep it open if you don't give up the issue entirely. -- ___ Python tracker ___ ___

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Nathaniel Smith added the comment: It's something I'm still interested in, but I'm not actively working on it (as you noticed :-)), and there are some other CPython changes that I'll probably prioritize first. Do you want to close this and I can re-open it when I do get back to it, or...?

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Andrew Svetlov added the comment: This is a new feature (and not a blocker). Shift to 3.9. Nathaniel, the PR is outdated. Have you an intention to land it or the issue can be closed by the lack of interest? -- nosy: +asvetlov versions: +Python 3.9 -Python 3.8

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Change by Ryan Hiebert : -- nosy: +ryanhiebert ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Cheryl Sabella added the comment: Hi Nathaniel, Was this something you were still targeting for 3.8? Thanks! -- nosy: +cheryl.sabella ___ Python tracker ___

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Change by Xavier G. Domingo : -- nosy: +xgdomingo ___ Python tracker ___ ___

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Nathaniel Smith added the comment: Okay, more context for future archaelogists / me in 6 months, phrased in the form of a timeline: This discussion started back in March, when I made an issue on the trio tracker [1] with notes on rough edges in CPython that it might make

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Nathaniel Smith added the comment: > So let's retarget this to 3.8 as it's too close to 3.7 feature freeze to get > seriously considered/merged. I *think* I have a better idea than this for 3.8, but it's definitely PEP-sized. The unawaited coroutine tracking proposal here was

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Yury Selivanov added the comment: So let's retarget this to 3.8 as it's too close to 3.7 feature freeze to get seriously considered/merged. -- versions: +Python 3.8 -Python 3.7 ___ Python tracker

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Guido van Rossum added the comment: No. I just mailed Yury with an explanation. -- ___ Python tracker ___

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Nathaniel Smith added the comment: Thanks for the summary, Yury! One quick note: > Effectively, the previously merged origin-tracking API (the one with which we > replaced set_coroutine_wrapper) achieves the same goal. I would say the two features are complementary. This

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Change by Yury Selivanov : -- nosy: +gvanrossum ___ Python tracker ___ ___

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Yury Selivanov added the comment: Guido, This is another feature for native coroutines that Nathaniel proposes for 3.7. Here's a summary (this issue has too many messages): 1. It adds a new mechanism to detect un-awaited coroutines (i.e. when a user forgets to use

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Nathaniel Smith added the comment: Yury also asked me to try running a generator/coroutine microbenchmark from PEP 492 (https://www.python.org/dev/peps/pep-0492/#async-await). I'm attaching the actual script for that as well (pep492bench.py), since I had to add a few lines to

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Nathaniel Smith added the comment: Please insert the usual caveats around how reliable benchmarking is impossible. (Last month when I tried this with a previous version of the patch, the interpreter that had the patch applied -- and thus was doing slightly *more* work -- was

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Yury Selivanov added the comment: I'd like to see benchmarks of coroutine creation time and awaits with and without the proposed patch. -- ___ Python tracker

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Change by Nathaniel Smith : -- keywords: +patch pull_requests: +5123 stage: -> patch review ___ Python tracker ___

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Nathaniel Smith added the comment: I have a patch for this, but it will be simplest if we can merge https://github.com/python/cpython/pull/5250 (for bpo-32591) first, because they touch a lot of overlapping code. -- ___ Python

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Nathaniel Smith added the comment: > In any case, in my opinion, it doesn't matter. `sys.set_coroutine_wrapper` > controls a single thread-local setting, > `sys.set_unawaited_coroutine_tracking` *also* controls a single thread-local > setting. Both of them have the exact

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Yury Selivanov added the comment: > So first... that's not how nursery.start_soon works :-). It actually takes an > async function, not a coroutine object. [..] Interesting, and I think I like it. I definitely understand the motivation to not tell users the difference

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Nathaniel Smith added the comment: > 1. How will trio handle situations like: > > c = coro() > await ... > nursery.start_soon(c) > > ? [...] > Maybe creating a coroutine and not immediately passing it to 'start_soon' or > similar API is an anti-pattern in Trio,

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Yury Selivanov added the comment: Matthias, Thanks a lot for such a detailed write-up! I now better understand how you want to use the proposed API in Trio, and why the weakref approach isn't going to work (at least not in a straightforward way). >> 2. What if another

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Matthias Bussonnier added the comment: Let me try to explain better, I'm pretty sure there is just a misunderstanding from some of use in the vocabulary or presupposition we start from. I have the impression that you feel like the API will automatically make

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Yury Selivanov added the comment: > send_ping() # don't care about result, forgot await > # get collected > await something_that_will_trigger_check_coros_weakreaf() # oh no ! I don't understand what you are trying to say with this example. What is

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Matthias Bussonnier added the comment: Your last description is about exactly what https://github.com/python-trio/trio/pull/176 is about (which I need to resurect) There are some issue with weakref some that I don't remember, but one of them is (IIRC): what if a

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Yury Selivanov added the comment: First, I've no questions about the proposed implementation. It shouldn't have performance impact when unawaited coroutine tracking is off, which is the default. It will cause minimal overhead when the tracking is on, which is fine.

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Nathaniel Smith added the comment: Update! I've been experimenting with this some more, and here's a more detailed proposal, that I'd ideally like to get into 3.7. I don't *think* this is big enough to need a PEP? I dunno, thoughts on that welcome. Motivation: It's easy to

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Changes by Chris Jerdonek : -- nosy: +chris.jerdonek ___ Python tracker ___ ___

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

Changes by Matthias Bussonnier : -- nosy: +mbussonn ___ Python tracker ___ ___

[issue30491] Add a lightweight mechanism for detecting un-awaited coroutine objects

New submission from Nathaniel Smith: A common problem when working with async functions is to attempt to call them but forget the 'await', which eventually leads to a 'Warning: coroutine ... was never awaited' (possibly buried in the middle of a bunch of traceback shrapnel caused by follow-on