Re: [Python-Dev] PEP 492: async/await in Python; v3
On 28 Apr 2015, at 5:07, Yury Selivanov wrote: Hi python-dev, Another round of updates. Reference implementation has been updated: https://github.com/1st1/cpython/tree/await (includes all things from the below summary of updates + tests). [...] New Coroutine Declaration Syntax The following new syntax is used to declare a coroutine:: async def read_data(db): pass Key properties of coroutines: * ``async def`` functions are always coroutines, even if they do not contain ``await`` expressions. * It is a ``SyntaxError`` to have ``yield`` or ``yield from`` expressions in an ``async`` function. Does this mean it's not possible to implement an async version of os.walk() if we had an async version of os.listdir()? I.e. for async code we're back to implementing iterators by hand instead of using generators for it. [...] Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: No new syntax is required
On 28/04/15 20:24, Paul Sokolovsky wrote: Hello, [snip] Based on all this passage, my guess is that you miss difference between C and Python functions. This is rather patronising, almost to the point of being insulting. Please keep the debate civil. [snip] Cheers, Mark. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with PEP 482 (1)
On 28/04/15 20:39, Paul Sokolovsky wrote: Hello, On Tue, 28 Apr 2015 19:44:53 +0100 Mark Shannon m...@hotpy.org wrote: [] A coroutine without a yield statement can be defined simply and concisely, thus: @coroutine def f(): return 1 [] A pure-python definition of the coroutine decorator is given below. [] from types import FunctionType, CodeType CO_COROUTINE = 0x0080 CO_GENERATOR = 0x0020 def coroutine(f): 'Converts a function to a generator function' old_code = f.__code__ new_code = CodeType( old_code.co_argcount, old_code.co_kwonlyargcount, This is joke right? Well it was partly for entertainment value, although it works on PyPy. The point is that something that can be done with a decorator, whether in pure Python or as builtin, does not require new syntax. Cheers, Mark. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: No new syntax is required
On Tue, Apr 28, 2015 at 11:51 AM, Stefan Behnel stefan...@behnel.de wrote: Mark Shannon schrieb am 27.04.2015 um 09:48: On 27/04/15 00:13, Guido van Rossum wrote: Currently this means looking for yield [from]; PEP 492 just adds looking for await and async [for|with]. Making await() a function defeats the purpose because now aliasing can hide its presence, and we're back in the land of gevent or stackless (where *anything* can potentially suspend the current task). I don't want to live in that land. I don't think I was clear enough. I said that await *is* a function, not that is should be disguised as one. Reading the code, GetAwaitableIter would be a better name for that element of the implementation. It is a straightforward non-blocking function. 1) it's not like people commonly alias repr() or len(), so why would they alias an await() builtin ? Unless, obviously, there's an actual reason to do so, in which case having it as a functions comes in handy. :) We had the same line of reasoning with print() back in the days of Py3k. 2) an await() builtin function that calls an __await__() special method on its input object sounds very pythonic. This sounds confused. The await expression must be recognized by the parser so it can generate different code for it (the code to suspend the stack). A builtin function cannot generate different code -- to the compiler all functions look the same. I know we could change that rule, but that' would be a really a big deviation from Python's philosophy: Currently the code generator never needs to know the type of any variables -- and a builtin function 'await' would just be another variable to the code generator. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with PEP 482 (1)
On Tue, Apr 28, 2015 at 11:44 AM, Mark Shannon m...@hotpy.org wrote: Hi, I still think that there are several issues that need addressing with PEP 492. This time, one issue at a time :) async The Rationale and Goals of PEP 492 states that PEP 380 has 3 shortcomings. The second of which is: It is not possible to natively define a coroutine which has no yield or yield from statements. This is incorrect, although what is meant by 'natively' is unclear. A coroutine without a yield statement can be defined simply and concisely, thus: @coroutine def f(): return 1 This is only a few character longer than the proposed new syntax, perfectly explicit and requires no modification the language whatsoever. A pure-python definition of the coroutine decorator is given below. So could the Rationale and Goals be correctly accordingly, please. Also, either the async def syntax should be dropped, or a new justification is required. So here's *my* motivation for this. I don't want the code generator to have to understand decorators. To the code generator, a decorator is just an expression, and it shouldn't be required to understand decorators in sufficient detail to know that *this* particular decorator means to generate different code. And it's not just generating different code -- it's also the desire to issue static errors (SyntaxError) when await (or async for/with) is used outside a coroutine, or when yield [from] is use inside one. The motivation is clear enough to me (and AFAIR I'm the BDFL for this PEP :-). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492 vs. PEP 3152, new round
I don't care for await await x.
On Apr 28, 2015 6:53 PM, Yury Selivanov yselivanov...@gmail.com wrote:
Looking at the grammar -- the only downside of the current approach is that
you can't do 'await await fut'. I still think that it reads better with
parens. If we put 'await' to 'factor' terminal we would allow
await -fut # await (-fut)
I think I something like
power: atom_expr ['**' factor]
atom_expr: [AWAIT] atom_expr | atom_trailer
atom_trailer: atom trailer*
will fix 'await await' situation, but does it really need to be fixed?
Yury
On 2015-04-27 9:44 AM, Yury Selivanov wrote:
Hi Greg,
I don't want this: await a() * b() to be parsed, it's not meaningful.
Likely you'll see await await a() only once in your life, so I'm fine
to use parens for it (moreover, I think it reads better with parens)
Yury
On 2015-04-27 8:52 AM, Greg Ewing wrote:
Yury Selivanov wrote:
I've done some experiments with grammar, and it looks like
we indeed can parse await quite differently from yield. Three
different options:
You don't seem to have tried what I suggested, which is
to make 'await' a unary operator with the same precedence
as '-', i.e. replace
factor: ('+'|'-'|'~') factor | power
with
factor: ('+'|'-'|'~'|'await') factor | power
That would allow
await a()
res = await a() + await b()
res = await await a()
if await a(): pass
return await a()
print(await a())
func(arg=await a())
await a() * b()
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: async/await in Python; v3
Hi Guido,
Thank you for a very detailed review. Comments below:
On 2015-04-28 5:49 PM, Guido van Rossum wrote:
Inline comments below...
On Mon, Apr 27, 2015 at 8:07 PM, Yury Selivanov yselivanov...@gmail.com
wrote:
Hi python-dev,
Another round of updates. Reference implementation
has been updated: https://github.com/1st1/cpython/tree/await
(includes all things from the below summary of updates +
tests).
Summary:
1. PyTypeObject.tp_await slot. Replaces tp_reserved.
This is to enable implementation of Futures with C API.
Must return an iterator if implemented.
That's fine (though I didn't follow this closely).
My main question here is it OK to reuse 'tp_reserved'
(former tp_compare)?
I had to remove this check:
https://github.com/1st1/cpython/commit/4be6d0a77688b63b917ad88f09d446ac3b7e2ce9#diff-c3cf251f16d5a03a9e7d4639f2d6f998L4906
On the other hand I think that it's a slightly better
solution than adding a new slot.
2. New grammar for await expressions, see
'Syntax of await expression' section
I like it.
Great!
The current grammar requires parentheses for consequent
await expressions:
await (await coro())
I can change this (in theory), but I kind of like the parens
in this case -- better readability. And it'll be a very
rare case.
3. inspect.iscoroutine() and inspect.iscoroutineobjects()
functions.
What's the use case for these? I wonder if it makes more sense to have a
check for a generalized awaitable rather than specifically a coroutine.
It's important to at least have 'iscoroutine' -- to check that
the object is a coroutine function. A typical use-case would be
a web framework that lets you to bind coroutines to specific
http methods/paths:
@http.get('/spam')
async def handle_spam(request):
...
'http.get' decorator will need a way to raise an error if it's
applied to a regular function (while the code is being imported,
not in runtime).
The idea here is to cover all kinds of python objects in
inspect module, it's Python's reflection API.
The other thing is that it's easy to implement this function
for CPython: just check for CO_COROUTINE flag. For other
Python implementations it might be a different story.
(More arguments for isawaitable() below)
4. Full separation of coroutines and generators.
This is a big one; let's discuss.
a) Coroutine objects raise TypeError (is NotImplementedError
better?) in their __iter__ and __next__. Therefore it's
not not possible to pass them to iter(), tuple(), next() and
other similar functions that work with iterables.
I think it should be TypeError -- what you *really* want is not to define
these methods at all but given the implementation tactic for coroutines
that may not be possible, so the nearest approximation is TypeError. (Also,
NotImplementedError is typically to indicate that a subclass should
implement it.)
Agree.
b) Because of (a), for..in iteration also does not work
on coroutines anymore.
Sounds good.
c) 'yield from' only accept coroutine objects from
generators decorated with 'types.coroutine'. That means
that existing asyncio generator-based coroutines will
happily yield from both coroutines and generators.
*But* every generator-based coroutine *must* be
decorated with `asyncio.coroutine()`. This is
potentially a backwards incompatible change.
See below. I worry about backward compatibility. A lot. Are you saying
that asycio-based code that doesn't use @coroutine will break in 3.5?
I'll experiment with replacing (c) with a warning.
We can disable __iter__ and __next__ for coroutines, but
allow to use 'yield from' on them. Would it be a better
approach?
d) inspect.isgenerator() and inspect.isgeneratorfunction()
return `False` for coroutine objects coroutine functions.
Makes sense.
(d) can also break something (hypothetically). I'm not
sure why would someone use isgenerator() and
isgeneratorfunction() on generator-based coroutines in
code based on asyncio, but there is a chance that
someone did (it should be trivial to fix the code).
Same for iter() and next(). The chance is slim, but we
may break some obscure code.
Are you OK with this?
e) Should we add a coroutine ABC (for cython etc)?
I, personally, think this is highly necessary. First,
separation of coroutines from generators is extremely
important. One day there won't be generator-based
coroutines, and we want to avoid any kind of confusion.
Second, we only can do this in 3.5. This kind of
semantics change won't be ever possible.
Sounds like Stefan agrees. Are you aware of
http://bugs.python.org/issue24018 (Generator ABC)?
Yes, I saw the issue. I'll review it in more detail
before thinking about Coroutine ABC for the next PEP
update.
asyncio recommends using @coroutine decorator, and most
projects that I've seen do use it. Also there is no
reason for people to use iter() and next() functions
on coroutines when writing asyncio code. I doubt that
this will cause serious backwards
Re: [Python-Dev] PEP 492: async/await in Python; v3
On Tue, Apr 28, 2015 at 4:55 PM, Ethan Furman et...@stoneleaf.us wrote: On 04/28, Yury Selivanov wrote: This limitation will go away as soon as ``async`` and ``await`` ate proper keywords. Or if it's decided to use a future import for this PEP. `async` and `await` need to be proper keywords, and __future__ imports is how we do that (see, e.g., PEP 355 and and PEP 343) You could at least provide an explanation about how the current proposal falls short. What code will break? There's a cost to __future__ imports too. The current proposal is a pretty clever hack -- and we've done similar hacks in the past (last I remember when import ... as ... was introduced but we didn't want to make 'as' a keyword right away). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492 vs. PEP 3152, new round
I prefer option #3.
On Mon, Apr 27, 2015 at 4:44 PM, Yury Selivanov yselivanov...@gmail.com wrote:
Hi Greg,
I don't want this: await a() * b() to be parsed, it's not meaningful.
Likely you'll see await await a() only once in your life, so I'm fine to
use parens for it (moreover, I think it reads better with parens)
Yury
On 2015-04-27 8:52 AM, Greg Ewing wrote:
Yury Selivanov wrote:
I've done some experiments with grammar, and it looks like
we indeed can parse await quite differently from yield. Three
different options:
You don't seem to have tried what I suggested, which is
to make 'await' a unary operator with the same precedence
as '-', i.e. replace
factor: ('+'|'-'|'~') factor | power
with
factor: ('+'|'-'|'~'|'await') factor | power
That would allow
await a()
res = await a() + await b()
res = await await a()
if await a(): pass
return await a()
print(await a())
func(arg=await a())
await a() * b()
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
--
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: async/await in Python; v3
On 04/28, Yury Selivanov wrote: This limitation will go away as soon as ``async`` and ``await`` ate proper keywords. Or if it's decided to use a future import for this PEP. `async` and `await` need to be proper keywords, and __future__ imports is how we do that (see, e.g., PEP 355 and and PEP 343) -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492 vs. PEP 3152, new round
Looking at the grammar -- the only downside of the current approach is that
you can't do 'await await fut'. I still think that it reads better with
parens. If we put 'await' to 'factor' terminal we would allow
await -fut # await (-fut)
I think I something like
power: atom_expr ['**' factor]
atom_expr: [AWAIT] atom_expr | atom_trailer
atom_trailer: atom trailer*
will fix 'await await' situation, but does it really need to be fixed?
Yury
On 2015-04-27 9:44 AM, Yury Selivanov wrote:
Hi Greg,
I don't want this: await a() * b() to be parsed, it's not meaningful.
Likely you'll see await await a() only once in your life, so I'm
fine to use parens for it (moreover, I think it reads better with parens)
Yury
On 2015-04-27 8:52 AM, Greg Ewing wrote:
Yury Selivanov wrote:
I've done some experiments with grammar, and it looks like
we indeed can parse await quite differently from yield. Three
different options:
You don't seem to have tried what I suggested, which is
to make 'await' a unary operator with the same precedence
as '-', i.e. replace
factor: ('+'|'-'|'~') factor | power
with
factor: ('+'|'-'|'~'|'await') factor | power
That would allow
await a()
res = await a() + await b()
res = await await a()
if await a(): pass
return await a()
print(await a())
func(arg=await a())
await a() * b()
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A macro for easier rich comparisons
On Apr 28, 2015, at 11:13 AM, Victor Stinner wrote: It would be nice to have a six module for C extensions. I'm quite sure that many projects are already full of #ifdef PYTHON3 ... #else ... #endif macros. Maybe encapsulating some of the recommendations here: https://wiki.python.org/moin/PortingToPy3k/BilingualQuickRef#Python_extension_modules (We really need to collect all this information in on place.) #define Py_RETURN_RICHCOMPARE(val1, val2, op) I think this macro would make a nice addition to the C API. It might read better as `Py_RETURN_RICHCOMPARE(val1, op, val2)`. Cheers, -Barry ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: async/await in Python; v3
On 04/28, Yury Selivanov wrote: On 2015-04-28 1:43 AM, Stefan Behnel wrote: Should a Generator then inherit from both Iterator and Coroutine, or would that counter your intention to separate coroutines from generators as a concept? I mean, they do share the same interface ... Them sharing the same interface depends on how the discussion goes :) But all in all, I think that it should be totally separate classes, even if they share some methods. For those of us who like to meddle, would it be possible to create an object that supports __iter__, __next__, __aiter__, and __anext__? -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: async/await in Python; v3
Hi Stefan, On 2015-04-28 1:43 AM, Stefan Behnel wrote: Should a Generator then inherit from both Iterator and Coroutine, or would that counter your intention to separate coroutines from generators as a concept? I mean, they do share the same interface ... Them sharing the same interface depends on how the discussion goes :) But all in all, I think that it should be totally separate classes, even if they share some methods. Thanks, Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: async/await in Python; v3
Hi Walter, On 2015-04-28 10:23 AM, Walter Dörwald wrote: Key properties of coroutines: * ``async def`` functions are always coroutines, even if they do not contain ``await`` expressions. * It is a ``SyntaxError`` to have ``yield`` or ``yield from`` expressions in an ``async`` function. Does this mean it's not possible to implement an async version of os.walk() if we had an async version of os.listdir()? I.e. for async code we're back to implementing iterators by hand instead of using generators for it. For now yes. Unfortunately, we don't have time to implement coroutine-generators properly in 3.5. Thanks, Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: async/await in Python; v3
Guido,
I found a solution how to disable 'yield from',
iter()/tuple() and 'for..in' on native coroutines
with 100% backwards compatibility.
The idea is to add one more code object flag:
CO_NATIVE_COROUTINE, which will be applied, along with
CO_COROUTINE to all 'async def' functions.
This way:
1. old generator-based coroutines from asyncio
are awaitable, because of CO_COROUTINE flag (that
asyncio.coroutine decorator will set with
'types.coroutine').
2. new 'async def' functions are awaitable
because of CO_COROUTINE flag.
3. GenObject __iter__ and __next__ raise error
*only* if it has CO_NATIVE_COROUTINE flag. So
iter(), next(), for..in aren't supported only
for 'async def' functions (but will work ok
on asyncio generator-based coroutines)
4. 'yield from' *only* raises an error if it yields a
*coroutine with a CO_NATIVE_COROUTINE*
from a regular generator.
Thanks,
Yury
On 2015-04-28 7:26 PM, Yury Selivanov wrote:
Hi Guido,
Thank you for a very detailed review. Comments below:
On 2015-04-28 5:49 PM, Guido van Rossum wrote:
Inline comments below...
On Mon, Apr 27, 2015 at 8:07 PM, Yury Selivanov
yselivanov...@gmail.com
wrote:
Hi python-dev,
Another round of updates. Reference implementation
has been updated: https://github.com/1st1/cpython/tree/await
(includes all things from the below summary of updates +
tests).
Summary:
1. PyTypeObject.tp_await slot. Replaces tp_reserved.
This is to enable implementation of Futures with C API.
Must return an iterator if implemented.
That's fine (though I didn't follow this closely).
My main question here is it OK to reuse 'tp_reserved'
(former tp_compare)?
I had to remove this check:
https://github.com/1st1/cpython/commit/4be6d0a77688b63b917ad88f09d446ac3b7e2ce9#diff-c3cf251f16d5a03a9e7d4639f2d6f998L4906
On the other hand I think that it's a slightly better
solution than adding a new slot.
2. New grammar for await expressions, see
'Syntax of await expression' section
I like it.
Great!
The current grammar requires parentheses for consequent
await expressions:
await (await coro())
I can change this (in theory), but I kind of like the parens
in this case -- better readability. And it'll be a very
rare case.
3. inspect.iscoroutine() and inspect.iscoroutineobjects()
functions.
What's the use case for these? I wonder if it makes more sense to
have a
check for a generalized awaitable rather than specifically a coroutine.
It's important to at least have 'iscoroutine' -- to check that
the object is a coroutine function. A typical use-case would be
a web framework that lets you to bind coroutines to specific
http methods/paths:
@http.get('/spam')
async def handle_spam(request):
...
'http.get' decorator will need a way to raise an error if it's
applied to a regular function (while the code is being imported,
not in runtime).
The idea here is to cover all kinds of python objects in
inspect module, it's Python's reflection API.
The other thing is that it's easy to implement this function
for CPython: just check for CO_COROUTINE flag. For other
Python implementations it might be a different story.
(More arguments for isawaitable() below)
4. Full separation of coroutines and generators.
This is a big one; let's discuss.
a) Coroutine objects raise TypeError (is NotImplementedError
better?) in their __iter__ and __next__. Therefore it's
not not possible to pass them to iter(), tuple(), next() and
other similar functions that work with iterables.
I think it should be TypeError -- what you *really* want is not to
define
these methods at all but given the implementation tactic for coroutines
that may not be possible, so the nearest approximation is TypeError.
(Also,
NotImplementedError is typically to indicate that a subclass should
implement it.)
Agree.
b) Because of (a), for..in iteration also does not work
on coroutines anymore.
Sounds good.
c) 'yield from' only accept coroutine objects from
generators decorated with 'types.coroutine'. That means
that existing asyncio generator-based coroutines will
happily yield from both coroutines and generators.
*But* every generator-based coroutine *must* be
decorated with `asyncio.coroutine()`. This is
potentially a backwards incompatible change.
See below. I worry about backward compatibility. A lot. Are you saying
that asycio-based code that doesn't use @coroutine will break in 3.5?
I'll experiment with replacing (c) with a warning.
We can disable __iter__ and __next__ for coroutines, but
allow to use 'yield from' on them. Would it be a better
approach?
d) inspect.isgenerator() and inspect.isgeneratorfunction()
return `False` for coroutine objects coroutine functions.
Makes sense.
(d) can also break something (hypothetically). I'm not
sure why would someone use isgenerator() and
isgeneratorfunction() on generator-based coroutines in
code based on asyncio, but there is a chance that
someone did (it should be trivial to fix the
Re: [Python-Dev] PEP 492: async/await in Python; v3
On 29/04/2015 9:49 a.m., Guido van Rossum wrote: c) 'yield from' only accept coroutine objects from generators decorated with 'types.coroutine'. That means that existing asyncio generator-based coroutines will happily yield from both coroutines and generators. *But* every generator-based coroutine *must* be decorated with `asyncio.coroutine()`. This is potentially a backwards incompatible change. See below. I worry about backward compatibility. A lot. Are you saying that asycio-based code that doesn't use @coroutine will break in 3.5? That seems unavoidable if the goal is for 'await' to only work on generators that are intended to implement coroutines, and not on generators that are intended to implement iterators. Because there's no way to tell them apart without marking them in some way. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: async/await in Python; v3
On 04/28, Guido van Rossum wrote: On Tue, Apr 28, 2015 at 4:55 PM, Ethan Furman et...@stoneleaf.us wrote: On 04/28, Yury Selivanov wrote: This limitation will go away as soon as ``async`` and ``await`` ate proper keywords. Or if it's decided to use a future import for this PEP. `async` and `await` need to be proper keywords, and __future__ imports is how we do that (see, e.g., PEP 355 and and PEP 343) You could at least provide an explanation about how the current proposal falls short. What code will break? There's a cost to __future__ imports too. The current proposal is a pretty clever hack -- and we've done similar hacks in the past (last I remember when import ... as ... was introduced but we didn't want to make 'as' a keyword right away). My apologies, I was unaware we had done psuedo-keywords before. -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: async/await in Python; v3
On 2015-04-28 11:59 PM, Greg wrote: On 29/04/2015 9:49 a.m., Guido van Rossum wrote: c) 'yield from' only accept coroutine objects from generators decorated with 'types.coroutine'. That means that existing asyncio generator-based coroutines will happily yield from both coroutines and generators. *But* every generator-based coroutine *must* be decorated with `asyncio.coroutine()`. This is potentially a backwards incompatible change. See below. I worry about backward compatibility. A lot. Are you saying that asycio-based code that doesn't use @coroutine will break in 3.5? That seems unavoidable if the goal is for 'await' to only work on generators that are intended to implement coroutines, and not on generators that are intended to implement iterators. Because there's no way to tell them apart without marking them in some way. Not sure what you mean by unavoidable. Before the last revision of the PEP it was perfectly fine to use generators in 'yield from' in generator-based coroutines: @asyncio.coroutine def foo(): yield from gen() and yet you couldn't do the same with 'await' (as it has a special opcode instead of GET_ITER that can validate what you're awaiting). With the new version of the PEP - 'yield from' in foo() would raise a TypeError. If we change it to a RuntimeWarning then we're safe in terms of backwards compatibility. I just want to see how exactly warnings will work (i.e. will they occur multiple times at the same 'yield from' expression, etc) Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492 vs. PEP 3152, new round
Yury Selivanov wrote:
I don't want this: await a() * b() to be parsed, it's not meaningful.
Why not? If a() is a coroutine that returns a number,
why shouldn't I be able to multiply it by something?
I don't think your currently proposed grammar prevents
that anyway. We can have
expr -- factor '*' factor
-- power '*' power
-- atom_expr '*' atom_expr
-- 'await' atom trailer* '*' atom trailer*
-- 'await' 'a' '(' ')' '*' 'b' '(' ')'
It does, on the other hand, seem to prevent
x = - await a()
which looks perfectly sensible to me.
I don't like the idea of introducing another level
of precedence. Python already has too many of those
to keep in my brain. Being able to tell people it's
just like unary minus makes it easy to explain (and
therefore possibly a good idea!).
--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492 vs. PEP 3152, new round
Greg,
On 2015-04-29 1:40 AM, Greg Ewing wrote:
Yury Selivanov wrote:
I don't want this: await a() * b() to be parsed, it's not meaningful.
Why not? If a() is a coroutine that returns a number,
why shouldn't I be able to multiply it by something?
Sorry, I thought you meant parsing await a()*b() as
await (a() * b()).
I don't think your currently proposed grammar prevents
that anyway. We can have
expr -- factor '*' factor
-- power '*' power
-- atom_expr '*' atom_expr
-- 'await' atom trailer* '*' atom trailer*
-- 'await' 'a' '(' ')' '*' 'b' '(' ')'
It does, on the other hand, seem to prevent
x = - await a()
This works just fine:
https://github.com/1st1/cpython/commit/33b3cd052243cd71c064eb385c7a557eec3ced4b
Current grammar prevents this: await -fut, and this:
await fut ** 2 being parsed as await (fut ** 2)
which looks perfectly sensible to me.
I don't like the idea of introducing another level
of precedence. Python already has too many of those
to keep in my brain. Being able to tell people it's
just like unary minus makes it easy to explain (and
therefore possibly a good idea!).
It's just like unary minus ;)
Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 492: async/await in Python; v3
Literary critic here. In section Specification It is strongly suggested that the reader understands how coroutines are implemented in Python (PEP 342 and PEP 380). It is also recommended to read PEP 3156 (asyncio framework) and PEP 3152 (Cofunctions). The usual phrasing of strongly suggested in specifications is presumes knowledge. Some people think strongly suggest doing is presumptuous and condescending, YMMV. Also, the relationship to PEP 3152 should be mentioned IMO. I propose: This specification presumes knowledge of the implementation of coroutines in Python (PEP 342 and PEP 380). Motivation for the syntax changes proposed here comes from the asyncio framework (PEP 3156) and the Cofunctions proposal (PEP 3152, now rejected in favor of this specification). I'm not entirely happy with my phrasing, because there are at least four more or less different concepts that might claim the bare word coroutine: - this specification - the implementation of this specification - the syntax used to define coroutines via PEPs 342 and 380 - the semantics of PEP 342/380 coroutines In both your original and my rephrasing, the use of coroutine violates your convention that it refers to the PEP's proposed syntax for coroutines. Instead it refers to the semantics of coroutines implemented via PEP 342/380. This is probably the same concern that motivated Guido's suggestion to use native coroutines for the PEP 492 syntax (but I'm not Dutch, so maybe they're not the same :-). I feel this is a real hindrance to understanding for someone coming to the PEP for the first time. You know which meaning of coroutine you mean, but the new reader needs to think hard enough to disambiguate every time the word occurs. If people agree with me, I could go through the PEP and revise mentions of coroutine in disambiguated style. In section Comprehensions: For the sake of restricting the broadness of this PEP there is no new syntax for asynchronous comprehensions. This should be considered in a separate PEP, if there is a strong demand for this feature. Don't invite trouble.wink / How about: Syntax for asynchronous comprehensions could be provided, but this construct is outside of the scope of this PEP. In section Async lambdas Lambda coroutines are not part of this proposal. In this proposal they would look like ``async lambda(parameters): expression``. Unless there is a strong demand to have them as part of this proposal, it is recommended to consider them later in a separate PEP. Same recommendation as for Comprehensions. I wouldn't mention the tentative syntax, it is both obvious and inviting to trouble. Acknowledgments === I thank Guido van Rossum, Victor Stinner, Elvis Pranskevichus, Andrew Svetlov, and Łukasz Langa for their initial feedback. A partial list of commentators I've found to be notable, YMMV: Greg Ewing for PEP 3152 and his Loyal Opposition to this PEP. Mark Shannon's comments have led to substantial clarifications of motivation for syntax, at least in my mind. Paul Sokolovsky for information about the MicroPython implementation. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: async/await in Python; v3
Hi Stephen, Thanks a lot for the feedback and suggestions. I'll apply them to the PEP. On 2015-04-28 11:03 PM, Stephen J. Turnbull wrote: Literary critic here. In section Specification It is strongly suggested that the reader understands how coroutines are implemented in Python (PEP 342 and PEP 380). It is also recommended to read PEP 3156 (asyncio framework) and PEP 3152 (Cofunctions). The usual phrasing of strongly suggested in specifications is presumes knowledge. Some people think strongly suggest doing is presumptuous and condescending, YMMV. Also, the relationship to PEP 3152 should be mentioned IMO. I propose: This specification presumes knowledge of the implementation of coroutines in Python (PEP 342 and PEP 380). Motivation for the syntax changes proposed here comes from the asyncio framework (PEP 3156) and the Cofunctions proposal (PEP 3152, now rejected in favor of this specification). Your wording is 100% better and it's time to mention PEP 3152 too. I'm not entirely happy with my phrasing, because there are at least four more or less different concepts that might claim the bare word coroutine: - this specification - the implementation of this specification - the syntax used to define coroutines via PEPs 342 and 380 - the semantics of PEP 342/380 coroutines In both your original and my rephrasing, the use of coroutine violates your convention that it refers to the PEP's proposed syntax for coroutines. Instead it refers to the semantics of coroutines implemented via PEP 342/380. This is probably the same concern that motivated Guido's suggestion to use native coroutines for the PEP 492 syntax (but I'm not Dutch, so maybe they're not the same :-). I feel this is a real hindrance to understanding for someone coming to the PEP for the first time. You know which meaning of coroutine you mean, but the new reader needs to think hard enough to disambiguate every time the word occurs. If people agree with me, I could go through the PEP and revise mentions of coroutine in disambiguated style. I also like Guido's suggestion to use native coroutine term. I'll update the PEP (I have several branches of it in the repo that I need to merge before the rename). In section Comprehensions: For the sake of restricting the broadness of this PEP there is no new syntax for asynchronous comprehensions. This should be considered in a separate PEP, if there is a strong demand for this feature. Don't invite trouble.wink / How about: Syntax for asynchronous comprehensions could be provided, but this construct is outside of the scope of this PEP. In section Async lambdas Lambda coroutines are not part of this proposal. In this proposal they would look like ``async lambda(parameters): expression``. Unless there is a strong demand to have them as part of this proposal, it is recommended to consider them later in a separate PEP. Same recommendation as for Comprehensions. I wouldn't mention the tentative syntax, it is both obvious and inviting to trouble. Agree. Do you think it'd be better to combine comprehensions and async lambdas in one section? Acknowledgments === I thank Guido van Rossum, Victor Stinner, Elvis Pranskevichus, Andrew Svetlov, and Łukasz Langa for their initial feedback. A partial list of commentators I've found to be notable, YMMV: Sure! I was going to add everybody after the PEP is accepted/rejected/postponed. Greg Ewing for PEP 3152 and his Loyal Opposition to this PEP. Mark Shannon's comments have led to substantial clarifications of motivation for syntax, at least in my mind. Paul Sokolovsky for information about the MicroPython implementation. Thanks! Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: No new syntax is required
On 4/26/2015 4:32 PM, Paul Sokolovsky wrote: Then, is the only logic for proposing __aenter__ is to reinsure against a situation that someone starts to write async context manager, forgets that they write async context manager, and make an __enter__ method there. Then your implementation will announce that async context manager lacks __aenter__, whereas my approach would announce Async's manager __enter__ did not return awaitable value. Again, is that the distinction you're shooting for, or do I miss something? Seems like the missing __aenter__ can easily be detected by the interpreter at compile time, but the wrong type returned would be at run time, or after a complex type-analysis done at compile time (unlikely to be practical). So I think you've nailed the distinction... but I'm not the expert. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Unicode literals in Python 2.7
Hello, is it possible to somehow tell Python 2.7 to compile a code entered in the interactive session with the flag PyCF_SOURCE_IS_UTF8 set? I'm considering adding support for Python 2 in my package ( https://github.com/Drekin/win-unicode-console) and I have run into the fact that when uα is entered in the interactive session, it results in u\xce\xb1 rather than u\u03b1. As this seems to be a highly specialized question, I'm asking it here. Regards, Drekin ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with PEP 482 (1)
On Tue, Apr 28, 2015 at 1:22 PM, Mark Shannon m...@hotpy.org wrote: On 28/04/15 21:06, Guido van Rossum wrote: On Tue, Apr 28, 2015 at 11:44 AM, Mark Shannon m...@hotpy.org mailto:m...@hotpy.org wrote: Hi, I still think that there are several issues that need addressing with PEP 492. This time, one issue at a time :) async The Rationale and Goals of PEP 492 states that PEP 380 has 3 shortcomings. The second of which is: It is not possible to natively define a coroutine which has no yield or yield from statements. This is incorrect, although what is meant by 'natively' is unclear. A coroutine without a yield statement can be defined simply and concisely, thus: @coroutine def f(): return 1 This is only a few character longer than the proposed new syntax, perfectly explicit and requires no modification the language whatsoever. A pure-python definition of the coroutine decorator is given below. So could the Rationale and Goals be correctly accordingly, please. Also, either the async def syntax should be dropped, or a new justification is required. So here's *my* motivation for this. I don't want the code generator to have to understand decorators. To the code generator, a decorator is just an expression, and it shouldn't be required to understand decorators in sufficient detail to know that *this* particular decorator means to generate different code. The code generator knows nothing about it. The generated bytecode is identical, only the flags are changed. The decorator can just return a copy of the function with modified co_flags. The situation may be different for other Python implementations though. The minimal changes to the code object are an implementation tactic -- the syntactic marking of coroutines is fundamental (like in the past the choice to recognize generators syntactically, albeit in that case by the presence of yield in their body). And it's not just generating different code -- it's also the desire to issue static errors (SyntaxError) when await (or async for/with) is used outside a coroutine, or when yield [from] is use inside one. Would raising a TypeError at runtime be sufficient to catch the sort of errors that you are worried about? No. The motivation is clear enough to me (and AFAIR I'm the BDFL for this PEP :-). Can't argue with that. Cheers, Mark. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: async/await in Python; v3
Inline comments below... On Mon, Apr 27, 2015 at 8:07 PM, Yury Selivanov yselivanov...@gmail.com wrote: Hi python-dev, Another round of updates. Reference implementation has been updated: https://github.com/1st1/cpython/tree/await (includes all things from the below summary of updates + tests). Summary: 1. PyTypeObject.tp_await slot. Replaces tp_reserved. This is to enable implementation of Futures with C API. Must return an iterator if implemented. That's fine (though I didn't follow this closely). 2. New grammar for await expressions, see 'Syntax of await expression' section I like it. 3. inspect.iscoroutine() and inspect.iscoroutineobjects() functions. What's the use case for these? I wonder if it makes more sense to have a check for a generalized awaitable rather than specifically a coroutine. 4. Full separation of coroutines and generators. This is a big one; let's discuss. a) Coroutine objects raise TypeError (is NotImplementedError better?) in their __iter__ and __next__. Therefore it's not not possible to pass them to iter(), tuple(), next() and other similar functions that work with iterables. I think it should be TypeError -- what you *really* want is not to define these methods at all but given the implementation tactic for coroutines that may not be possible, so the nearest approximation is TypeError. (Also, NotImplementedError is typically to indicate that a subclass should implement it.) b) Because of (a), for..in iteration also does not work on coroutines anymore. Sounds good. c) 'yield from' only accept coroutine objects from generators decorated with 'types.coroutine'. That means that existing asyncio generator-based coroutines will happily yield from both coroutines and generators. *But* every generator-based coroutine *must* be decorated with `asyncio.coroutine()`. This is potentially a backwards incompatible change. See below. I worry about backward compatibility. A lot. Are you saying that asycio-based code that doesn't use @coroutine will break in 3.5? d) inspect.isgenerator() and inspect.isgeneratorfunction() return `False` for coroutine objects coroutine functions. Makes sense. e) Should we add a coroutine ABC (for cython etc)? I, personally, think this is highly necessary. First, separation of coroutines from generators is extremely important. One day there won't be generator-based coroutines, and we want to avoid any kind of confusion. Second, we only can do this in 3.5. This kind of semantics change won't be ever possible. Sounds like Stefan agrees. Are you aware of http://bugs.python.org/issue24018 (Generator ABC)? asyncio recommends using @coroutine decorator, and most projects that I've seen do use it. Also there is no reason for people to use iter() and next() functions on coroutines when writing asyncio code. I doubt that this will cause serious backwards compatibility problems (asyncio also has provisional status). I wouldn't count too much on asyncio's provisional status. What are the consequences for code that is written to work with asyncio but doesn't use @coroutine? Such code will work with 3.4 and (despite the provisional status and the recommendation to use @coroutine) I don't want that code to break in 3.5 (though maybe a warning would be fine). I also hope that if someone has their own (renamed) copy of asyncio that works with 3.4, it will all still work with 3.5. Even if asyncio itself is provisional, none of the primitives (e.g. yield from) that it is built upon are provisional, so there should be no reason for it to break in 3.5. Thank you, Yury Some more inline comments directly on the PEP below. PEP: 492 Title: Coroutines with async and await syntax Version: $Revision$ Last-Modified: $Date$ Author: Yury Selivanov yseliva...@sprymix.com Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 09-Apr-2015 Python-Version: 3.5 Post-History: 17-Apr-2015, 21-Apr-2015, 27-Apr-2015 Abstract This PEP introduces new syntax for coroutines, asynchronous ``with`` statements and ``for`` loops. The main motivation behind this proposal is to streamline writing and maintaining asynchronous code, as well as to simplify previously hard to implement code patterns. Rationale and Goals === Current Python supports implementing coroutines via generators (PEP 342), further enhanced by the ``yield from`` syntax introduced in PEP 380. This approach has a number of shortcomings: * it is easy to confuse coroutines with regular generators, since they share the same syntax; async libraries often attempt to alleviate this by using decorators (e.g. ``@asyncio.coroutine`` [1]_); * it is not possible to natively define a coroutine which has no ``yield`` or ``yield from`` statements, again requiring the use of decorators to fix potential refactoring issues; (I have to agree with Mark that this point is
Re: [Python-Dev] Issues with PEP 482 (1)
On 28/04/15 21:06, Guido van Rossum wrote: On Tue, Apr 28, 2015 at 11:44 AM, Mark Shannon m...@hotpy.org mailto:m...@hotpy.org wrote: Hi, I still think that there are several issues that need addressing with PEP 492. This time, one issue at a time :) async The Rationale and Goals of PEP 492 states that PEP 380 has 3 shortcomings. The second of which is: It is not possible to natively define a coroutine which has no yield or yield from statements. This is incorrect, although what is meant by 'natively' is unclear. A coroutine without a yield statement can be defined simply and concisely, thus: @coroutine def f(): return 1 This is only a few character longer than the proposed new syntax, perfectly explicit and requires no modification the language whatsoever. A pure-python definition of the coroutine decorator is given below. So could the Rationale and Goals be correctly accordingly, please. Also, either the async def syntax should be dropped, or a new justification is required. So here's *my* motivation for this. I don't want the code generator to have to understand decorators. To the code generator, a decorator is just an expression, and it shouldn't be required to understand decorators in sufficient detail to know that *this* particular decorator means to generate different code. The code generator knows nothing about it. The generated bytecode is identical, only the flags are changed. The decorator can just return a copy of the function with modified co_flags. And it's not just generating different code -- it's also the desire to issue static errors (SyntaxError) when await (or async for/with) is used outside a coroutine, or when yield [from] is use inside one. Would raising a TypeError at runtime be sufficient to catch the sort of errors that you are worried about? The motivation is clear enough to me (and AFAIR I'm the BDFL for this PEP :-). Can't argue with that. Cheers, Mark. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] async/await in Python; v2
On 25 April 2015 at 22:02, Yury Selivanov yselivanov...@gmail.com wrote: [...] On 2015-04-25 4:47 PM, Arnaud Delobelle wrote: [...] 1. About the 'async for' construct. Each iteration will create a new coroutine object (the one returned by Cursor.__anext__()) and it seems to me that it can be wasteful. In the example given of an 'aiterable' Cursor class, probably a large number of rows will fill the cursor buffer in one call of cursor._prefetch(). However each row that is iterated over will involve the creation execution of a new coroutine object. It seems to me that what is desirable in that case is that all the buffered rows will be iterated over as in a plain for loop. I agree that creating a new coroutine object is a little bit wasteful. However, the proposed iteration protocol was designed to: 1. Resemble already existing __iter__/__next__/StopIteration protocol; 2. Pave the road to introduce coroutine-generators in the future. Do you mean that __aiter__() would return a 'coroutine-generator'? I'm not sure what such an object is but if it is a suspendable generator in the same way that a coroutine is a suspendable function, then this is a strong argument to make the __aiter__() magic method a coroutine rather than a plain function. I.e. __aiter__() would return either an 'aiterator' or a 'coroutine generator object'. I think this could be mentioned in the section 'Why __aiter__ is a coroutine' [1]. We could, in theory, design the protocol to make __anext__ awaitable return a regular iterators (and raise StopAsyncIteration at the end) to make things more efficient, but that would complicate the protocol tremendously, and make it very hard to program and debug. My opinion is that this has to be addressed in 3.6 with coroutine-generators if there is enough interest from Python users. True, but to me this is bound to happen. I feel like the semantics of __anext__() is tied to the behaviour of this yet to be defined coroutine generator object and that if it turns out that the natural bevaviour of a coroutine generator is not consistent with the semantics of __anext__() then it would be a shame. I must say I have no evidence that this will happen! 2. I think the semantics of the new coroutine objects could be defined more clearly in the PEP. Of course they are pretty obvious when you know that the coroutines are meant to replace asyncio.coroutine as described in [1]. I understand that this PEP is mostly for the benefit of asyncio, hence mainly of interest of people who know it. However I think it would be good for it to be more self-contained. I have often read a PEP as an introduction to a new feature of Python. I feel that if I was not familiar with yield from and asyncio I would not be able to understand this PEP, even though potentially one could use the new constructs without knowing anything about them. I agree. I plan to update the PEP with some new semantics (prohibit passing coroutine-objects to iter(), tuple() and other builtins, as well as using them in 'for .. in coro()' loops). I'll add a section with a more detailed explanation of coroutine-objects. Great! Thanks, -- Arnaud PS: there's a slight asymmetry in the terminology between coroutines and generators. 'Generator functions' are to 'generators' what 'coroutines' are to 'coroutine objects', which makes it difficult to what one is talking about when referring to a 'coroutine generator'. [1] https://www.python.org/dev/peps/pep-0492/#id52 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: No new syntax is required
Hello, On Sun, 26 Apr 2015 16:40:03 -0400 Yury Selivanov yselivanov...@gmail.com wrote: Hi Mark, On 2015-04-26 4:21 PM, Mark Shannon wrote: Hi, I was looking at PEP 492 and it seems to me that no new syntax is required. Mark, all your points are explained in the PEP in a great detail: Indeed, they are. It is the very basic counter-argument against this PEP is that everything it proposes is already doable. But the PEP makes very clear that the only reason it exists is to make coroutine programming easier and less error-prone. However, given that there're questions even like that and you're great at keeping up discussion, I'd appreciate additional argumentation on: 2. Support a parallel set of special methods starting with 'a' or 'async'. Why not just use the current set of special methods? Because you can't reuse them. https://www.python.org/dev/peps/pep-0492/#why-not-reuse-existing-magic-names Ok, so here're 3 points this link gives, with my concerns/questions: An alternative idea about new asynchronous iterators and context managers was to reuse existing magic methods, by adding an async keyword to their declarations: [But:] - it would not be possible to create an object that works in both with and async with statements; Yes, and I would say, for good. Behavior of sync and async code is different enough to warrant separate classes (if not libraries!) to implement each paradigm. What if, in otherwise async code, someone will miss async in async with and call sync version of context manager? So, losing ability stated above isn't big practical loss. - it would look confusing Sorry, async def __enter__ doesn't look more confusing than __aenter__ (vs __enter__). and would require some implicit magic behind the scenes in the interpreter; Interpreter already does a lot of implicit magic. Can you please elaborate what exactly would need to be done? one of the main points of this proposal is to make coroutines as simple and foolproof as possible. And it's possible to agree that to separate notions, create a dichotomy is a simple principle on its own. But just taking bunch of stuff - special methods, exceptions - and duplicating it is a bloat a violates another principle - DRY. You argue that this will make coroutine writing simple. But coroutines are part of the language, and duplicating notions makes language more complex/complicated. Can you please argue that in this case it's worth duplicating hierarchies instead of reusing existing lower-level concepts, given that higher-level concepts are already distinguished well enough (async and await keywords separate old and new things very visibly). -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: No new syntax is required
Hello, On Sun, 26 Apr 2015 18:49:43 -0400 Yury Selivanov yselivanov...@gmail.com wrote: [] - it would look confusing Sorry, async def __enter__ doesn't look more confusing than __aenter__ (vs __enter__). I'll update the PEP. The argument shouldn't be that it's confusing, the argument is that __aenter__ returns an 'awaitable', which is either a coroutine-object or a future. You can't reuse __enter__, because you'd break backwards compatibility -- it's perfectly normal for context managers in python to return any object from their __enter__. If we assign some special meaning to futures -- we'll break existing code. So, again to make sure I (and hopefully other folks) understand it right. You say it's perfectly normal for context managers in python to return any object from their __enter__. That's true, but we talk about async context managers. There're no such at all, they yet need to be written. And whoever writes them, would need to return from __enter__ awaitable, because that's the requirement for an async context manager, and it is error to return something else. Then, is the only logic for proposing __aenter__ is to reinsure against a situation that someone starts to write async context manager, forgets that they write async context manager, and make an __enter__ method there. Then your implementation will announce that async context manager lacks __aenter__, whereas my approach would announce Async's manager __enter__ did not return awaitable value. Again, is that the distinction you're shooting for, or do I miss something? [] Anyways, I really doubt that you can convince anyone to reuse existing dunder methods for async stuff. Yeah, but it would be nice to understand why everyone and so easily agrees to them, after pretty thorough discussion of other aspects. Thanks, Yury -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: No new syntax is required
Hello, On Sun, 26 Apr 2015 16:13:43 -0700 Guido van Rossum gu...@python.org wrote: But new syntax is the whole point of the PEP. I want to be able to *syntactically* tell where the suspension points are in coroutines. Currently this means looking for yield [from]; PEP 492 just adds looking for await and async [for|with]. Making await() a function defeats the purpose because now aliasing can hide its presence, and we're back in the land of gevent or stackless (where *anything* can potentially suspend the current task). I don't want to live in that land. And someone in this thread should have link in a signature to the page on why there's not wanting to live in that land. I don't have it handy, so may offer only reminiscence: if you don't know where there're suspension points, you essentially should assume that any function call can be a suspension point. And then you dropped almost as low as when using threads - you can lose control anytime, data may change anytime, you need to extensively use locks, etc. Oh, and btw, there was related claim earlier that some things are better done in thread pools, so async context managers can be done away with. That's another faulty point - if asyncio is to be feature-complete, it should work [well] with own functionality, without crutches of foreign paradigms. Otherwise you may be just happy to use threads for everything at all, like everyone did 5 years ago. (And yeah, there's even usages for that, for example, MicroPython is proudly GIL-free, translating to not supporting those thread thingies and relying on native Python concurrency primitives). On Sun, Apr 26, 2015 at 1:21 PM, Mark Shannon m...@hotpy.org wrote: Hi, I was looking at PEP 492 and it seems to me that no new syntax is required. Looking at the code, it does four things; all of which, or a functional equivalent, could be done with no new syntax. 1. Make a normal function into a generator or coroutine. This can be done with a decorator. 2. Support a parallel set of special methods starting with 'a' or 'async'. Why not just use the current set of special methods? 3. await. await is an operator that takes one argument and produces a single result, without altering flow control and can thus be replaced by an function. 4. Asynchronous with statement. The PEP lists the equivalent as with (yield from xxx) which doesn't seem so bad. Please don't add unnecessary new syntax. Cheers, Mark. P.S. I'm not objecting to any of the other new features proposed, just the new syntax. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: No new syntax is required
Hello, On Sun, 26 Apr 2015 19:45:30 -0400 Yury Selivanov yselivanov...@gmail.com wrote: [] Then, is the only logic for proposing __aenter__ is to reinsure against a situation that someone starts to write async context manager, forgets that they write async context manager, and make an __enter__ method there. It's to make sure that it's impossible to accidentally use existing regular context manager that returns a future object from its __enter__ / __exit__ (nobody prohibits you to return a future object from __exit__, although it's pointless) in an 'async with' block. I see, so it's just to close the final loophole, unlikely to be hit in real life (unless you can say that there're cases of doing just that in existing asyncio libs). Well, given that Greg Ewing wanted even stricter error-proofness, and you rejected it as such strict as to disallow useful behavior, I've just got to trust you that in this case, you're as strict as needed. I really don't understand the desire to reuse existing magic methods. Even if it was decided to reuse them, it wouldn't even simplify the implementation in CPython; the code there is already DRY (I don't re-implement opcodes for 'with' statement; I reuse them). Well, there're 3 levels of this stuff: 1. How mere people write their code - everyone would use async def and await, this should be bullet- (and fool-) proof. 2. How library code is written - async iterators won't be written by everyone, and only few will write async context managers; it's fair to expect that people doing these know what they do and don't do stupid mistakes. 3. How it all is coded in particular Python implementation. It's clear that __enter__ vs __aenter__ distinction is 1st kind of issue in your list. As for 3rd point, I'd like to remind that CPython is only one Python implementation. And with my MicroPython hat on, I'd like to know if (some of) these new features are bloat or worthy for the space constraints we have. I appreciate the answers you gave on all accounts! Thanks! Yury -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] A macro for easier rich comparisons
It seems the discussion on python-ideas, and also the patch review,
died down. So I'm posting to python-dev.
A macro like this would reduce boilerplate in stdlib and third-party C
extensions. It would ease porting C extensions to Python 3, where rich
comparison is mandatory.
#define Py_RETURN_RICHCOMPARE(val1, val2, op) \
do {\
switch (op) { \
case Py_EQ: if ((val1) == (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_NE: if ((val1) != (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_LT: if ((val1) (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_GT: if ((val1) (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_LE: if ((val1) = (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_GE: if ((val1) = (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
} \
Py_RETURN_NOTIMPLEMENTED; \
} while (0)
Is any of the core devs interested in this macro? Anything I can do to
help get it in?
http://bugs.python.org/issue23699
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: No new syntax is required
Hello, On Sun, 26 Apr 2015 20:39:55 -0400 Yury Selivanov yselivanov...@gmail.com wrote: [] As for 3rd point, I'd like to remind that CPython is only one Python implementation. And with my MicroPython hat on, I'd like to know if (some of) these new features are bloat or worthy for the space constraints we have. OT: MicroPython is an amazing project. Kudos for doing it. I really hope that addition of few new magic methods won't make it too hard for you guys to implement PEP 492 in MicroPython one day. Thanks! Damien George, MicroPython's author, actually already made a basic implementation of async def/await: https://github.com/micropython/micropython/commit/81afa7e098634605c04597d34a51ca2e59a87d7c So we surely do hope this PEP will be accepted, and soonish! ;-) -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A macro for easier rich comparisons
On Tue, 2015-04-28 at 10:50 -0700, Glenn Linderman wrote:
On 4/28/2015 2:13 AM, Victor Stinner wrote:
#define Py_RETURN_RICHCOMPARE(val1, val2, op)
\
do {
\
switch (op) {
\
case Py_EQ: if ((val1) == (val2)) Py_RETURN_TRUE;
Py_RETURN_FALSE; \
case Py_NE: if ((val1) != (val2)) Py_RETURN_TRUE;
Py_RETURN_FALSE; \
case Py_LT: if ((val1) (val2)) Py_RETURN_TRUE;
Py_RETURN_FALSE; \
case Py_GT: if ((val1) (val2)) Py_RETURN_TRUE;
Py_RETURN_FALSE; \
case Py_LE: if ((val1) = (val2)) Py_RETURN_TRUE;
Py_RETURN_FALSE; \
case Py_GE: if ((val1) = (val2)) Py_RETURN_TRUE;
Py_RETURN_FALSE; \
}
\
Py_RETURN_NOTIMPLEMENTED;
\
} while (0)
I would prefer a function for that:
PyObject *Py_RichCompare(long val1, long2, int op);
Why would you prefer a function? As a macro, when the op is a
constant, most of the code would be optimized away by a decent
compiler.
I suppose when the op is not a constant, then a function would save
code space.
So I suppose it depends on the predominant use cases.
There's also the possibility of wrapping C++ code that uses overloaded
operators: having it as a macro could allow those C++ operators to be be
mapped into Python.
Hope this is constructive
Dave
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A macro for easier rich comparisons
On 4/28/2015 2:13 AM, Victor Stinner wrote:
#define Py_RETURN_RICHCOMPARE(val1, val2, op) \
do {\
switch (op) { \
case Py_EQ: if ((val1) == (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_NE: if ((val1) != (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_LT: if ((val1) (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_GT: if ((val1) (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_LE: if ((val1) = (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_GE: if ((val1) = (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
} \
Py_RETURN_NOTIMPLEMENTED; \
} while (0)
I would prefer a function for that:
PyObject *Py_RichCompare(long val1, long2, int op);
Why would you prefer a function? As a macro, when the op is a constant,
most of the code would be optimized away by a decent compiler.
I suppose when the op is not a constant, then a function would save code
space.
So I suppose it depends on the predominant use cases.
You should also handle invalid operator. PyUnicode_RichCompare() calls
PyErr_BadArgument() in this case.
One can quibble over the correct error return, but the above code does
handle invalid operators after the switch.
Glenn
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Issues with PEP 482 (1)
Hi, I still think that there are several issues that need addressing with PEP 492. This time, one issue at a time :) async The Rationale and Goals of PEP 492 states that PEP 380 has 3 shortcomings. The second of which is: It is not possible to natively define a coroutine which has no yield or yield from statements. This is incorrect, although what is meant by 'natively' is unclear. A coroutine without a yield statement can be defined simply and concisely, thus: @coroutine def f(): return 1 This is only a few character longer than the proposed new syntax, perfectly explicit and requires no modification the language whatsoever. A pure-python definition of the coroutine decorator is given below. So could the Rationale and Goals be correctly accordingly, please. Also, either the async def syntax should be dropped, or a new justification is required. Cheers, Mark. #coroutine.py from types import FunctionType, CodeType CO_COROUTINE = 0x0080 CO_GENERATOR = 0x0020 def coroutine(f): 'Converts a function to a generator function' old_code = f.__code__ new_code = CodeType( old_code.co_argcount, old_code.co_kwonlyargcount, old_code.co_nlocals, old_code.co_stacksize, old_code.co_flags | CO_GENERATOR | CO_COROUTINE, old_code.co_code, old_code.co_consts, old_code.co_names, old_code.co_varnames, old_code.co_filename, old_code.co_name, old_code.co_firstlineno, old_code.co_lnotab, old_code.co_freevars, old_code.co_cellvars) return FunctionType(new_code, f.__globals__) P.S. The reverse of this decorator, which unsets the flags, converts a generator function into a normal function. :? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: No new syntax is required
Mark Shannon schrieb am 27.04.2015 um 09:48: On 27/04/15 00:13, Guido van Rossum wrote: Currently this means looking for yield [from]; PEP 492 just adds looking for await and async [for|with]. Making await() a function defeats the purpose because now aliasing can hide its presence, and we're back in the land of gevent or stackless (where *anything* can potentially suspend the current task). I don't want to live in that land. I don't think I was clear enough. I said that await *is* a function, not that is should be disguised as one. Reading the code, GetAwaitableIter would be a better name for that element of the implementation. It is a straightforward non-blocking function. 1) it's not like people commonly alias repr() or len(), so why would they alias an await() builtin ? Unless, obviously, there's an actual reason to do so, in which case having it as a functions comes in handy. :) We had the same line of reasoning with print() back in the days of Py3k. 2) an await() builtin function that calls an __await__() special method on its input object sounds very pythonic. Stefan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Looking for someone to audit PEP 3156 (asyncio) against current version
PEP 3156 (Asynchronous IO Support Rebooted: the asyncio Module) was accepted 18 months ago, and since then we've aggressively test-driven it in the 3.4 stdlib. I think that in some cases we've changed or added APIs or changed semantics, and I would like to update the PEP to match reality (so that people reading the PEP as a specification aren't going to be confused). If you're good at reading and editing technical documentation I'd love for you to help me with this! (Another form of help I could use might be to check the docs on docs.python.org against the PEP and the implementation.) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A macro for easier rich comparisons
Hi,
2015-04-27 16:02 GMT+02:00 Petr Viktorin encu...@gmail.com:
A macro like this would reduce boilerplate in stdlib and third-party C
extensions. It would ease porting C extensions to Python 3, where rich
comparison is mandatory.
It would be nice to have a six module for C extensions. I'm quite sure
that many projects are already full of #ifdef PYTHON3 ... #else ...
#endif macros.
#define Py_RETURN_RICHCOMPARE(val1, val2, op) \
do {\
switch (op) { \
case Py_EQ: if ((val1) == (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_NE: if ((val1) != (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_LT: if ((val1) (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_GT: if ((val1) (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_LE: if ((val1) = (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
case Py_GE: if ((val1) = (val2)) Py_RETURN_TRUE; Py_RETURN_FALSE; \
} \
Py_RETURN_NOTIMPLEMENTED; \
} while (0)
I would prefer a function for that:
PyObject *Py_RichCompare(long val1, long2, int op);
You should also handle invalid operator. PyUnicode_RichCompare() calls
PyErr_BadArgument() in this case.
Anyway, please open an issue for this idea.
Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Issues with PEP 482 (1)
Mark, I'm sorry but you have to view the proposal as a whole. Discussing it point by point in isolation doesn't make any sense, as with any complex subject. Thanks, Yury On 2015-04-28 2:44 PM, Mark Shannon wrote: Hi, I still think that there are several issues that need addressing with PEP 492. This time, one issue at a time :) async The Rationale and Goals of PEP 492 states that PEP 380 has 3 shortcomings. The second of which is: It is not possible to natively define a coroutine which has no yield or yield from statements. This is incorrect, although what is meant by 'natively' is unclear. A coroutine without a yield statement can be defined simply and concisely, thus: @coroutine def f(): return 1 This is only a few character longer than the proposed new syntax, perfectly explicit and requires no modification the language whatsoever. A pure-python definition of the coroutine decorator is given below. So could the Rationale and Goals be correctly accordingly, please. Also, either the async def syntax should be dropped, or a new justification is required. Cheers, Mark. #coroutine.py from types import FunctionType, CodeType CO_COROUTINE = 0x0080 CO_GENERATOR = 0x0020 def coroutine(f): 'Converts a function to a generator function' old_code = f.__code__ new_code = CodeType( old_code.co_argcount, old_code.co_kwonlyargcount, old_code.co_nlocals, old_code.co_stacksize, old_code.co_flags | CO_GENERATOR | CO_COROUTINE, old_code.co_code, old_code.co_consts, old_code.co_names, old_code.co_varnames, old_code.co_filename, old_code.co_name, old_code.co_firstlineno, old_code.co_lnotab, old_code.co_freevars, old_code.co_cellvars) return FunctionType(new_code, f.__globals__) P.S. The reverse of this decorator, which unsets the flags, converts a generator function into a normal function. :? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 492: async/await in Python; v3
Ethan, On 2015-04-28 11:29 AM, Ethan Furman wrote: On 04/28, Yury Selivanov wrote: On 2015-04-28 1:43 AM, Stefan Behnel wrote: Should a Generator then inherit from both Iterator and Coroutine, or would that counter your intention to separate coroutines from generators as a concept? I mean, they do share the same interface ... Them sharing the same interface depends on how the discussion goes :) But all in all, I think that it should be totally separate classes, even if they share some methods. For those of us who like to meddle, would it be possible to create an object that supports __iter__, __next__, __aiter__, and __anext__? Sure, there is nothing that could prevent you from doing that. Thanks, Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
