Reading into these flags more, I wonder if modifications to the
inspect.iscoroutine* functions as well as to genobject.c's finalizer are
justified. Specifically to check for both CO_ITERABLE_COROUTINE
and CO_COROUTINE in any place where a coroutine is expected. Note that
this is an inversion of my last email's statements about setting both flags
in types.coroutine. Here is some discovery to help in the discussion...
The "New Coroutine Declaration Syntax" section of PEP 482 states, *"Internally,
two new code object flags were introduced: CO_COROUTINE is used to mark
native coroutines (defined with new syntax). CO_ITERABLE_COROUTINE is used
to make generator-based coroutines compatible with native coroutines (set
by types.coroutine() function)."* Along with, *"When a coroutine is garbage
collected, a RuntimeWarning is raised if it was never awaited on (see also
Debugging Features )."*
The last comment does not delineate CO_ITERABLE_COROUTINE
from CO_COROUTINE, which makes me think that the genobject.c finalizer code
should check for un-awaited coroutines of both types. There are a few
other places affected by this including the repr output of objects as seen
by before and after repr() calls to asyncio.sleep from my previous
experiment in setting both COROUTINE flags...
#stock behavior..
>>> repr(asyncio.sleep(1))
'<generator object sleep at 0x7fa5ac7ee0a0>'
# hacked behavior setting both CO_ITERABLE_COROUTINE and CO_COROUTINE...
>>> asyncio.sleep(1)
__main__:1: RuntimeWarning: coroutine 'sleep' was never awaited
<coroutine object sleep at 0x7f8aac4c6360>
I posit that this last behavior is desirable but should be accomplished by
finding all places that check for CO_COROUTINE and have them check for
CO_ITERABLE_COROUTINE too.
Lastly, the docstring from inspect.iscoroutinefunction says *"Return true
if the object is a coroutine function. Coroutine functions are defined
with "async def" syntax, or generators decorated with `types.coroutine`."
*What I find problematic here is that types.coroutine never produces a
function that is compatible with inspect.iscoroutinefunction.
My suggestion is that we use a mask of something like CO_COROUTINE_ANY
defined as (CO_ITERABLE_COROUTINE & CO_COROUTINE), which can be used
anywhere coroutine tests need to be performed. I can think of only a few
places where you would need to know if it is native or not and for those
cases I imagine an internal check of the co_flags would suffice. If not,
perhaps a new function for inspect could cover this, such as
inspect.isnativecoroutinefunction().
Justin
On Tue, Nov 8, 2016 at 12:01 AM Justin Mayfield <too...@gmail.com> wrote:
Comments inline..
>From types.coroutine (which is also called by asyncio.coroutine)...
211 def coroutine(func):
212 """Convert regular generator function to a coroutine."""
213
214 if not callable(func):
215 raise TypeError('types.coroutine() expects a callable')
216
217 if (func.__class__ is FunctionType and
218 getattr(func, '__code__', None).__class__ is CodeType):
219
220 co_flags = func.__code__.co_flags
221
222 # Check if 'func' is a coroutine function.
223 # (0x180 == CO_COROUTINE | CO_ITERABLE_COROUTINE)
224 if co_flags & 0x180:
225 return func
226
227 # Check if 'func' is a generator function.
228 # (0x20 == CO_GENERATOR)
229 if co_flags & 0x20:
230 # TODO: Implement this in C.
231 co = func.__code__
232 func.__code__ = CodeType(
233 co.co_argcount, co.co_kwonlyargcount, co.co_nlocals,
234 co.co_stacksize,
*235 co.co_flags | 0x100, # 0x100 == CO_ITERABLE_COROUTINE*
236 co.co_code,
237 co.co_consts, co.co_names, co.co_varnames,
co.co_filename,
238 co.co_name, co.co_firstlineno, co.co_lnotab,
co.co_freevars,
239 co.co_cellvars)
240 return func
Line 235: Setting CO_ITERABLE_COROUTINE but not CO_ITERABLE makes
inspect.awaitable(corofunc()) pass but prevents
inspect.iscoroutinefunction(forofunc) from passing. Setting the flags to
(CO_ITERABLE_COROUTINE & CO_ITERABLE) in this same context actually
produces passing behavior for inspect.iscoroutinefuncion.
In addition, it triggers some other diagnostic code from genobject so you
get some handy messages like, "RuntimeWarning: coroutine 'sleep' was never
awaited".
See (from genobject.c):
62 /* If `gen` is a coroutine, and if it was never awaited on,
63 issue a RuntimeWarning. */
64 if (gen->gi_code != NULL &&
65 ((PyCodeObject *)gen->gi_code)->co_flags & CO_COROUTINE &&
66 gen->gi_frame->f_lasti == -1) {
67 if (!error_value) {
68 PyErr_WarnFormat(PyExc_RuntimeWarning, 1,
69 "coroutine '%.50S' was never awaited",
70 gen->gi_qualname);
71 }
72 }
My primary question here is about the intended difference
between CO_ITERABLE_COROUTINE and CO_ITERABLE; How perverse is a change
that naively modifies line 235 of types.coroutine to set both flags? In my
limited testing it produced better behavior than stock, but I honestly
can't figure out the intended differences between those flags.
On Mon, Nov 7, 2016 at 10:58 PM Yury Selivanov <yselivanov...@gmail.com>
wrote:
On 2016-11-08 12:47 AM, Justin Mayfield wrote:
> That's quite surprising to me too, BTW. I was focused on generator vs
> `async def ...` coros and hadn't even tried the legacy decorator.
>
> Would it be possible for asyncio code to be altered in a way that produces
> awaitables which conform with inspect's protocol? Ie. through code flags
> or some other flag that inspect looks for? I assume a similar stance
could
> then be taken by other async libraries.
There is no protocol here. The inspect module is simply tailored for
introspection of builtin types. If something mimics a builtin type (a
coroutine, a generator, or a simple function) inspect utils won't
recognize it.
> If it's just not in the cards may I humbly suggest the docs for the
inspect
> module include some language to help users avoid some of these
> idiosyncrasies?
Yes, I think we can improve docs: better clarify what is a
coroutine/coroutine function, reference asyncio functions, add an
example. Feel free to submit a patch!
Yury
