Re: [Python-ideas] PEP 560 (second post)

2017-11-15 Thread Koos Zevenhoven 
On Tue, Nov 14, 2017 at 4:49 AM, Nick Coghlan  wrote:

> On 14 November 2017 at 09:41, Guido van Rossum  wrote:
> >
> > Thanks, I am happy now with the PEP, except for one detail: maybe
> > `__mro_entry__` should always return a tuple and then maybe renamed to
> > `__mro_entries__`. (See debate at
> > https://github.com/python/peps/pull/460#issuecomment-343969528 .)
>
> I like that - very nice refinement.
>
>
​I hope the order in which multiple __mro_entries__ will appear in the mro
will be documented clearly, regardless of how obvious it might feel.​ It
might take a while, before anyone notices that something weird happens
because they did it the wrong way around.

​Out of curiosity, what kind of cases would benefit from __mro__entries__
being able to return two
​or more ​
entries?

Also, I'm still wondering about __bases__ and __orig_bases__. Could we call
these __concrete_bases__ and __bases__ instead (respectively)?

For an explanation of why I think this might be a good idea, see this new
thread:

https://mail.python.org/pipermail/python-ideas/2017-November/047896.html

​––Koos
​


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-11-13 Thread Nick Coghlan 
On 14 November 2017 at 09:41, Guido van Rossum  wrote:
> On Fri, Nov 10, 2017 at 8:54 AM, Ivan Levkivskyi 
> wrote:
>>
>> On 10 November 2017 at 17:43, Guido van Rossum  wrote:
>>>
>>> There seem to be some action items from this thread that I haven't seen
>>> reflected in the PEP source code yet.
>>> [...snip...]
>>> Then the next step I propose is a PR with a full implementation. After
>>> that I'll likely approve the PEP (or we'll have minor feedback based on
>>> trying the implementation).
>>
>>
>> Yes, sorry, I wanted to make updates to the PEP and reference
>> implementation,
>> but last two weeks were very busy.
>> Hopefully, I will work on it this weekend.
>
> Thanks, I am happy now with the PEP, except for one detail: maybe
> `__mro_entry__` should always return a tuple and then maybe renamed to
> `__mro_entries__`. (See debate at
> https://github.com/python/peps/pull/460#issuecomment-343969528 .)

I like that - very nice refinement.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-11-13 Thread Guido van Rossum 
On Fri, Nov 10, 2017 at 8:54 AM, Ivan Levkivskyi 
wrote:

> On 10 November 2017 at 17:43, Guido van Rossum  wrote:
>
>> There seem to be some action items from this thread that I haven't seen
>> reflected in the PEP source code yet.
>> [...snip...]
>> Then the next step I propose is a PR with a full implementation. After
>> that I'll likely approve the PEP (or we'll have minor feedback based on
>> trying the implementation).
>>
>
> Yes, sorry, I wanted to make updates to the PEP and reference
> implementation,
> but last two weeks were very busy.
> Hopefully, I will work on it this weekend.
>

Thanks, I am happy now with the PEP, except for one detail: maybe
`__mro_entry__` should always return a tuple and then maybe renamed to
`__mro_entries__`. (See debate at
https://github.com/python/peps/pull/460#issuecomment-343969528 .)

Other than that I think we just need to satisfy a process nit: let's post
the final PEP (after that issue is resolved) to python-dev.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-11-10 Thread Ivan Levkivskyi 
On 10 November 2017 at 21:19, Koos Zevenhoven  wrote:

> On Fri, Nov 10, 2017 at 8:33 PM, Ivan Levkivskyi 
> wrote:
>
>> On 10 November 2017 at 18:39, Koos Zevenhoven  wrote:
>>
>>> On Wed, Sep 27, 2017 at 12:28 PM, Ivan Levkivskyi 
>>> wrote:
>>>
 ​

>>> ​
 After creating the class,
 the original bases are saved in ``__orig_bases__`` (currently this is
 also
 done by the metaclass).


>>> ​Those are *still* bases, right, even if they are not in the mro?​ I'm
>>> not sure if this is a naming thing or something even more.
>>>
>>
>> The objects that have __subclass_base__ method (proposed to rename to
>> __mro_entry__)
>> are removed from __bases__ attributed of the newly created class.
>> Otherwise they may cause a metaclass conflict.
>> One can however still call them syntactic (or static?) bases. For example
>> this is how it is going to be used by typing:
>>
>> from typing import List
>>
>> class Tokens(List[int]):
>> ...
>>
>> assert Tokens.__bases__ == (list,)
>>
>
> ​Why is List[int] not allowed to be the base? Neither method-lookup
> performance nor the metaclass conflict issue seem to depend on whether
> List[int] is in __bases__.
>
>
The situation is actually quite opposite. Interestingly, the whole
discussion started from Mark Shannon pointing to these problems with
List[int] at the Language Summit.
The original discussion on typing tracker is referenced in the PEP draft.

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-11-10 Thread Koos Zevenhoven 
On Fri, Nov 10, 2017 at 8:33 PM, Ivan Levkivskyi 
wrote:

> On 10 November 2017 at 18:39, Koos Zevenhoven  wrote:
>
>> On Wed, Sep 27, 2017 at 12:28 PM, Ivan Levkivskyi 
>> wrote:
>>
>>> ​
>>>
>> ​
>>> After creating the class,
>>> the original bases are saved in ``__orig_bases__`` (currently this is
>>> also
>>> done by the metaclass).
>>>
>>>
>> ​Those are *still* bases, right, even if they are not in the mro?​ I'm
>> not sure if this is a naming thing or something even more.
>>
>
> The objects that have __subclass_base__ method (proposed to rename to
> __mro_entry__)
> are removed from __bases__ attributed of the newly created class.
> Otherwise they may cause a metaclass conflict.
> One can however still call them syntactic (or static?) bases. For example
> this is how it is going to be used by typing:
>
> from typing import List
>
> class Tokens(List[int]):
> ...
>
> assert Tokens.__bases__ == (list,)
>

​Why is List[int] not allowed to be the base? Neither method-lookup
performance nor the metaclass conflict issue seem to depend on whether
List[int] is in __bases__.



>
>
>> NOTE: These two method names are reserved for exclusive use by
>>> the ``typing`` module and the generic types machinery, and any other use
>>> is
>>> strongly discouraged.
>>>
>>
>> ​Given the situation, that may be a good thing. But will it really work?
>> I think it is also strongly discouraged to invent your own dunder method
>> names, but people still do it.​
>>
>
> Terry had a similar comment. I will "soften" this formulation in the next
> revision of the PEP.
>
>
​
Right, I assume you mean the one where he pointed out that implicitly
turning the methods into staticmethods based on their names makes those
names reserved words.
​
​-- Koos

-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-11-10 Thread Ivan Levkivskyi 
On 10 November 2017 at 18:39, Koos Zevenhoven  wrote:

> On Wed, Sep 27, 2017 at 12:28 PM, Ivan Levkivskyi 
> wrote:
>
>> ​
>>
> ​
>> After creating the class,
>> the original bases are saved in ``__orig_bases__`` (currently this is also
>> done by the metaclass).
>>
>>
> ​Those are *still* bases, right, even if they are not in the mro?​ I'm not
> sure if this is a naming thing or something even more.
>

The objects that have __subclass_base__ method (proposed to rename to
__mro_entry__)
are removed from __bases__ attributed of the newly created class. Otherwise
they may cause a metaclass conflict.
One can however still call them syntactic (or static?) bases. For example
this is how it is going to be used by typing:

from typing import List

class Tokens(List[int]):
...

assert Tokens.__bases__ == (list,)


> NOTE: These two method names are reserved for exclusive use by
>> the ``typing`` module and the generic types machinery, and any other use
>> is
>> strongly discouraged.
>>
>
> ​Given the situation, that may be a good thing. But will it really work? I
> think it is also strongly discouraged to invent your own dunder method
> names, but people still do it.​
>

Terry had a similar comment. I will "soften" this formulation in the next
revision of the PEP.

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-11-10 Thread Ivan Levkivskyi 
On 10 November 2017 at 17:43, Guido van Rossum  wrote:

> Hey Ivan,
>
> There seem to be some action items from this thread that I haven't seen
> reflected in the PEP source code yet.
> [...snip...]
> Then the next step I propose is a PR with a full implementation. After
> that I'll likely approve the PEP (or we'll have minor feedback based on
> trying the implementation).
>
>
Yes, sorry, I wanted to make updates to the PEP and reference
implementation,
but last two weeks were very busy.
Hopefully, I will work on it this weekend.

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-11-10 Thread Guido van Rossum 
Hey Ivan,

There seem to be some action items from this thread that I haven't seen
reflected in the PEP source code yet.
- Change the title (I like "Core support for typing module and generic
types" but maybe it can be shortened to "Core support for generics in the
typing module" -- if you like that, go for it)
- Change __subclass_base__ to __mro_entry__
- Add types.resolve_bases and related changes
- Maybe some clarifications summarizing the feedback from this thread (esp.
Nick's)?

Then the next step I propose is a PR with a full implementation. After that
I'll likely approve the PEP (or we'll have minor feedback based on trying
the implementation).

I don't require a new typing.py that uses these features to be part of the
"full implementation" but it would be nice to make a start with that as
well -- if we can make the case that it will speed up "import typing" a lot
then that would be a powerful argument for the PEP.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-09-29 Thread Ivan Levkivskyi 
On 29 September 2017 at 08:57, Nick Coghlan  wrote:

> On 29 September 2017 at 08:04, Ivan Levkivskyi 
> wrote:
> >> How would you feel about calling it "__mro_entry__", as a mnemonic for
> >> "the substitute entry to use instead of this object when calculating a
> >> subclass MRO"?
> >
> > I don't have any preferences for the name, __mro_entry__ sounds equally
> OK
> > to me.
>
> I'd propose changing it then, as searching for "Python mro entry" is
> likely to get people to the right place faster than searching for
> "Python subclass base".
>
>
OK, will do.


> > I propose to update
> > ``type.__new__`` to just give
> > a better error message explaining this.
>
> +1 from me, since that avoids ever resolving the list of bases twice.
>
>
OK, I will update the reference implementation.


> > 3) Do we need to update types.new_class and types.prepare_class?
> > Here I am not sure. These functions are rather utility functions and are
> > designed to
> > mimic in Python what __build_class__ does in C. I think we might add
> > types._update_bases
> > that does the same as its C counterpart. Then we can update
> types.new_class
> > and types.prepare_class
> > like you proposed, this will preserve their current API while
> > types.new_class will match behaviour of __build_class__
>
> Your suggestion for `types.__new__` gave me a different idea: what if
> `types.prepare_class` *also* just raised an error when given a
> non-class as a nominal base class?
>
> Then if we added `types.resolve_bases` as a public API, a full
> reimplementation of `types.new_class` would now look like:
>
> resolved_bases = types.resolve_bases(bases)
> mcl, ns, updated_kwds = types.prepare_class(name, resolved_bases, kwds)
> exec_body(ns)
> ns["__orig_bases__"] = bases
> mcl(name, resolved_bases, ns, **updated_kwds)
>
> That way, `types.new_class` would transparently switch to the new
> behaviour, while clients of any other dynamic type creation API could
> do their own base class resolution, even if the type creation API they
> were using didn't implicitly support MRO entry resolution.
>
>
Yes, makes sense. I no one is against adding new public
``types.resolve_bases``
then I will add this to the PEP.

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-09-29 Thread Ivan Levkivskyi 
On 29 September 2017 at 10:14, Victor Stinner 
wrote:

> > Title: Core support for generic types
>
> Would it be possible to mention "typing" somewhere in the title? If
> you don't know the context, it's hard to understand that the PEP is
> related to type annotation and type checks. At least just from the
> title.
>

What do you think about "Core support for typing module and generic types"?
Another option is "Runtime mechanism to improve generics and typing module".

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-09-29 Thread Victor Stinner 
> Title: Core support for generic types

Would it be possible to mention "typing" somewhere in the title? If
you don't know the context, it's hard to understand that the PEP is
related to type annotation and type checks. At least just from the
title.

Victor
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-09-29 Thread Nick Coghlan 
On 29 September 2017 at 08:04, Ivan Levkivskyi  wrote:
> On 28 September 2017 at 08:27, Nick Coghlan  wrote:
>>
>> On 27 September 2017 at 19:28, Ivan Levkivskyi 
>> wrote:
>> > If an object that is not a class object appears in the bases of a class
>> > definition, the ``__subclass_base__`` is searched on it. If found,
>> > it is called with the original tuple of bases as an argument. If the
>> > result
>> > of the call is not ``None``, then it is substituted instead of this
>> > object.
>> > Otherwise (if the result is ``None``), the base is just removed. This is
>> > necessary to avoid inconsistent MRO errors, that are currently prevented
>> > by
>> > manipulations in ``GenericMeta.__new__``. After creating the class,
>> > the original bases are saved in ``__orig_bases__`` (currently this is
>> > also
>> > done by the metaclass).
>>
>> How would you feel about calling it "__mro_entry__", as a mnemonic for
>> "the substitute entry to use instead of this object when calculating a
>> subclass MRO"?
>
> I don't have any preferences for the name, __mro_entry__ sounds equally OK
> to me.

I'd propose changing it then, as searching for "Python mro entry" is
likely to get people to the right place faster than searching for
"Python subclass base".

>> I think the other thing that needs to be clarified is whether or not
>> the actual metaclass can expect to receive an already-resolved
>> sequence of MRO entries as its list of bases, or if it will need to
>> repeat the base resolution process executed while figuring out the
>> metaclass.
>>
>
> There are three points for discussion here:
>
> 1) It is necessary to make the bases resolution soon, before the metaclass
> is calculated. This is why I do this at the beginning of __build_class__ in
> the
> reference implementation.

Indeed.

> 2) Do we need to update type.__new__ to be able to accept non-classes as
> bases?
> I think no. One might be a bit surprised that
>
> class C(Iterable[int]):
> pass
>
> works, but
>
> type('C', (Iterable[int],), {})
>
> fails with a metaclass conflict, but I think it is natural that static
> typing and dynamic
> class creation should not be used together. I propose to update
> ``type.__new__`` to just give
> a better error message explaining this.

+1 from me, since that avoids ever resolving the list of bases twice.

> 3) Do we need to update types.new_class and types.prepare_class?
> Here I am not sure. These functions are rather utility functions and are
> designed to
> mimic in Python what __build_class__ does in C. I think we might add
> types._update_bases
> that does the same as its C counterpart. Then we can update types.new_class
> and types.prepare_class
> like you proposed, this will preserve their current API while
> types.new_class will match behaviour of __build_class__

Your suggestion for `types.__new__` gave me a different idea: what if
`types.prepare_class` *also* just raised an error when given a
non-class as a nominal base class?

Then if we added `types.resolve_bases` as a public API, a full
reimplementation of `types.new_class` would now look like:

resolved_bases = types.resolve_bases(bases)
mcl, ns, updated_kwds = types.prepare_class(name, resolved_bases, kwds)
exec_body(ns)
ns["__orig_bases__"] = bases
mcl(name, resolved_bases, ns, **updated_kwds)

That way, `types.new_class` would transparently switch to the new
behaviour, while clients of any other dynamic type creation API could
do their own base class resolution, even if the type creation API they
were using didn't implicitly support MRO entry resolution.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-09-28 Thread Ivan Levkivskyi 
On 28 September 2017 at 08:27, Nick Coghlan  wrote:

> On 27 September 2017 at 19:28, Ivan Levkivskyi 
> wrote:
> > If an object that is not a class object appears in the bases of a class
> > definition, the ``__subclass_base__`` is searched on it. If found,
> > it is called with the original tuple of bases as an argument. If the
> result
> > of the call is not ``None``, then it is substituted instead of this
> object.
> > Otherwise (if the result is ``None``), the base is just removed. This is
> > necessary to avoid inconsistent MRO errors, that are currently prevented
> by
> > manipulations in ``GenericMeta.__new__``. After creating the class,
> > the original bases are saved in ``__orig_bases__`` (currently this is
> also
> > done by the metaclass).
>
> How would you feel about calling it "__mro_entry__", as a mnemonic for
> "the substitute entry to use instead of this object when calculating a
> subclass MRO"?
>
>
I don't have any preferences for the name, __mro_entry__ sounds equally OK
to me.

I think the other thing that needs to be clarified is whether or not
> the actual metaclass can expect to receive an already-resolved
> sequence of MRO entries as its list of bases, or if it will need to
> repeat the base resolution process executed while figuring out the
> metaclass.
>
>
There are three points for discussion here:

1) It is necessary to make the bases resolution soon, before the metaclass
is calculated. This is why I do this at the beginning of __build_class__ in
the
reference implementation.

2) Do we need to update type.__new__ to be able to accept non-classes as
bases?
I think no. One might be a bit surprised that

class C(Iterable[int]):
pass

works, but

type('C', (Iterable[int],), {})

fails with a metaclass conflict, but I think it is natural that static
typing and dynamic
class creation should not be used together. I propose to update
``type.__new__`` to just give
a better error message explaining this.

3) Do we need to update types.new_class and types.prepare_class?
Here I am not sure. These functions are rather utility functions and are
designed to
mimic in Python what __build_class__ does in C. I think we might add
types._update_bases
that does the same as its C counterpart. Then we can update types.new_class
and types.prepare_class
like you proposed, this will preserve their current API while
types.new_class will match behaviour of __build_class__

If you and others agree with this, then I will update the PEP text and the
reference implementation.

Thanks for comments!

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-09-27 Thread Ivan Levkivskyi 
On 27 September 2017 at 18:08, Terry Reedy  wrote:

> On 9/27/2017 5:28 AM, Ivan Levkivskyi wrote:
>
> It is proposed to add two special methods ``__class_getitem__`` and
>> ``__subclass_base__`` to the core CPython for better support of
>> generic types.
>>
>
> I would not be concerned about anyone (mis)using reserved words.
>
>
These methods are quite specific (especially __subclass_base__) so there
are two points:

* I would like to say that there are less backwards compatibility
guarantees. Only the documented use
  is guaranteed to be backwards compatible, which is in this case
Iterable[int] etc.

* I don't want to "advertise" these methods. I could imagine someone will
be unpleasantly surprised when
  finding these while reading someone other's code.


> If the new methods were for general use, I would question making them
> automatically class methods.  Having __new__ automatically being a static
> method is convenient, but occasionally throws people off.  But if they were
> only used for typing, perhaps it is ok.  On the other hand, I expect that
> others will use __class_getitem__ for the same purpose -- to avoid defining
> a metaclass just to make class[something] work.  So I question defining
> that as 'typing only'.
>
>
I think we would rather want to limit the number of use cases for
SomeClass[int], so that one doesn't need to guess if it is a generic class
or something else.


> Without rereading the PEP, the use case for __subclass_base__ is not clear
> to me.  So I don't know if there are other uses for it.
>

The __subclass_base__ method is needed to avoid making the result of
Iterable[int] a class object. Creating new class objects on
every subscription is too expensive. However, these objects must be
subclassable, so that the only way is to introduce this new method.
For example:

class Iterable:
def __class_getitem__(cls, item):
return GenericAlias(cls, item)

class GenericAlias:
def __init__(self, origin, item):
self.origin = origin
self.item = item
def __subclass_base__(self, bases):
return self.origin

class MyIterable(Iterable[int]):
...

Real code will be more complex, but this illustrates the idea. I don't know
other use cases where one
would want to allow non-classes in base classes list.

Thanks for comments!

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

2017-09-27 Thread Terry Reedy 

On 9/27/2017 5:28 AM, Ivan Levkivskyi wrote:


Abstract


Initially PEP 484 was designed in such way that it would not introduce
*any* changes to the core CPython interpreter. Now type hints and
the ``typing`` module are extensively used by the community, e.g. PEP 526
and PEP 557 extend the usage of type hints, and the backport of ``typing``
on PyPI has 1M downloads/month. Therefore, this restriction can be removed.


It seem sensible to me that you waited awhile to discover what would be 
needed.



It is proposed to add two special methods ``__class_getitem__`` and
``__subclass_base__`` to the core CPython for better support of
generic types.


I would not be concerned about anyone (mis)using reserved words.

If the new methods were for general use, I would question making them 
automatically class methods.  Having __new__ automatically being a 
static method is convenient, but occasionally throws people off.  But if 
they were only used for typing, perhaps it is ok.  On the other hand, I 
expect that others will use __class_getitem__ for the same purpose -- to 
avoid defining a metaclass just to make class[something] work.  So I 
question defining that as 'typing only'.


Without rereading the PEP, the use case for __subclass_base__ is not 
clear to me.  So I don't know if there are other uses for it.


--
Terry Jan Reedy

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/