subject:"\[Python\-Dev\] Re\: PEP\: Deferred Evaluation Of Annotations Using Descriptors"



On 1/15/21 3:29 PM, Greg Ewing wrote:

On 16/01/21 7:56 am, Larry Hastings wrote:


As mentioned previously in this thread, typing.get_type_hints() is 
opinionated in ways that users of annotations may not want.


This brings us back to my idea of introducing a new
annotations() function to hide the details. It wouldn't
be the same as get_type_hints(), since it wouldn't make
any assumptions about what the annotations mean.



I think it's simpler and nicer for the user to preserve the existing 
interface, so I'm sticking with that approach.  If you feel strongly 
about this, I encourage you to write your own competing PEP.



//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZRFIAWZ772NOWAY4VE4WBNFPHNLIMX5V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-15 Thread Paul Bryan via Python-Dev

Would annotations() just access the dunder, like other builtins (and
then result in the descriptor resolving __co_annotations__ as
proposed), or would calling it be required to actually resolve
__co_annotations__? I think it should probably be the former.

On Sat, 2021-01-16 at 12:29 +1300, Greg Ewing wrote:
> On 16/01/21 7:56 am, Larry Hastings wrote:
> > 
> > As mentioned previously in this 
> > thread, typing.get_type_hints() is opinionated in ways that users
> > of 
> > annotations may not want.
> 
> This brings us back to my idea of introducing a new
> annotations() function to hide the details. It wouldn't
> be the same as get_type_hints(), since it wouldn't make
> any assumptions about what the annotations mean.
> 

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/457EYGAJAE4BQHUWH32MRZNSLFGNR6MM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-15 Thread Greg Ewing


On 16/01/21 7:56 am, Larry Hastings wrote:


As mentioned previously in this 
thread, typing.get_type_hints() is opinionated in ways that users of 
annotations may not want.


This brings us back to my idea of introducing a new
annotations() function to hide the details. It wouldn't
be the same as get_type_hints(), since it wouldn't make
any assumptions about what the annotations mean.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OHMQLQXGJDZLYTACJBGZIPJV45H2LF5H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-15 Thread Paul Bryan via Python-Dev

I knew I was missing something. Agree, annotations are not necessarily
type hints.

On Fri, 2021-01-15 at 10:56 -0800, Larry Hastings wrote:
> 
> On 1/15/21 10:12 AM, Paul Bryan wrote:
> 
> > I wonder if then the __co_annotations__ call and overwriting of
> > __annotations__ should be explicitly caused by a to get_type_hints
> > instead of (mysteriously) occurring on an attempt to getattr
> > __annotations__.
> 
> I would say: absolutely not.  While all "type hints" are annotations,
> not all annotations are "type hints".  As mentioned previously in
> this thread, typing.get_type_hints() is opinionated in ways that
> users of annotations may not want.  And personally I bristle at the
> idea of gating a language feature behind a library function.
> Besides, most users will never know or care about
> __co_annotations__.  If you're not even aware that it exists, it's
> not mysterious ;-)
> 
> Cheers,
> 
> /arry

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/C3HINI65UIFCA4EON55WMR6YR6THXQCQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-15 Thread Guido van Rossum

On Fri, Jan 15, 2021 at 10:53 AM Larry Hastings  wrote:

>
> Sorry it took me 3+ days to reply--I had a lot to think about here.  But I
> have good things to report!
>
>
> On 1/11/21 8:42 PM, Guido van Rossum wrote:
>
> On Mon, Jan 11, 2021 at 1:20 PM Larry Hastings  wrote:
>
>> PEP 563 states:
>>
>> For code that uses type hints, the typing.get_type_hints(obj,
>> globalns=None, localns=None) function correctly evaluates expressions back
>> from its string form.
>>
>> So, if you are passing in a localns argument that isn't None, okay, but
>> you're not using them "correctly" according to the language.  Also, this
>> usage won't be compatible with static type checkers.
>>
> I think you're misreading PEP 563 here. The mention of globalns=None,
> localns=None refers to the fact that these parameters have defaults, not
> that you must pass None. Note that the next paragraph in that PEP mentions
> eval(ann, globals, locals) -- it doesn't say eval(ann, {}, {}).
>
> I think that's misleading, then.  The passage is telling you how to
> "correctly evaluate[s] expressions", and how I read it was, it's telling me
> I have to supply globalns=None and localns=None for it to work
> correctly--which, I had to discover on my own, were the default values.  I
> don't understand why PEP 563 feels compelled to define a function that it's
> not introducing, and in fact had already shipped with Python two versions
> ago.
>

I suppose PEP 563 is ambiguous because on the one hand global symbols are
the only things that work out of the box, on the other hand you can make
other things work by passing the right scope (and there's lots of code now
that does so), and on the third hand, it claims that get_type_hints() adds
the class scope, which nobody noticed or implemented until this week
(there's a PR, can't recall the number).

But I think all this is irrelevant given what comes below.

>
> Later in that same section, PEP 563 points out a problem with annotations
> that reference class-scoped variables, and claims that the implementation
> would run into problems because methods can't "see" the class scope. This
> is indeed a problem for PEP 563, but *you* can easily generate correct
> code, assuming the containing class exists in the global scope (and your
> solution requires that anyway). So in this case
> ```
> class Outer:
> class Inner:
>...
> def method(self, a: Inner, b: Outer) -> None:
> ...
> ```
> The generated code for the `__annotations__` property could just have a
> reference to `Outer.Inner` for such cases:
> ```
> def __annotations__():
> return {"a": Outer.Inner, "b": Outer, "return": None}
> ```
>
> This suggestion was a revelation for me.  Previously, a combination of bad
> experiences early on when hacking on compile and symtable, and my
> misunderstanding of exactly what was being asserted in the November 2017
> thread, led me to believe that all I could support was globals.  But I've
> been turning this over in my head for several days now, and I suspect I can
> support... just about anything.
>
>
> I can name five name resolution scenarios I might encounter.  I'll discuss
> them below, in increasing order of difficulty.
>
>
> *First* is references to globals / builtins.  That's already working,
> it's obvious how it works, and I need not elaborate further.
>

Yup.

>
> *Second* is local variables in an enclosing function scope:
>
> def outer_fn():
> class C: pass
> def inner_fn(a:C=None): pass
> return inner_fn
>
> As you pointed out elsewhere in un-quoted text, I could make the
> annotation a closure, so it could retain a reference to the value of (what
> is from its perspective) the free variable "C".
>

Yup.

>
> *Third* is local variables in an enclosing class scope, as you describe
> above:
>
> class OuterCls:
> class InnerCls:
> def method(a:InnerCls=None): pass
>
> If I understand what you're suggesting, I could notice inside the compiler
> that Inner is being defined in a class scope, walk up the enclosing scopes
> until I hit the outermost class, then reconstruct the chain of pulling out
> attributes until it resolves globally.  Thus I'd rewrite this example to:
>
> class OuterCls:
> class InnerCls:
> def method(a:OuterCls.InnerCls=None): pass
>
> We've turned the local reference into a global reference, and we already
> know globals work fine.
>

I think this is going too far. A static method defined in InnerCls does not
see InnerCls (even after the class definitions are complete). E.g.
```
class Outer:
class Inner:
@staticmethod
def foo(): return Inner
```
If you then call Outer.Inner.foo() you get "NameError: name 'Inner' is not
defined".


>
> *Fourth* is local variables in an enclosing class scope, which are
> themselves local variables in an enclosing function scope:
>
> def outerfn():
> class OuterCls:
> class InnerCls:
> def method(a:InnerCls=None): pass
> return OuterCls.InnerCls
>

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/15/21 10:12 AM, Paul Bryan wrote:
I wonder if then the __co_annotations__ call and overwriting of 
__annotations__ should be explicitly caused by a to get_type_hints 
instead of (mysteriously) occurring on an attempt to getattr 
__annotations__.



I would say: absolutely not.  While all "type hints" are annotations, 
not all annotations are "type hints".  As mentioned previously in this 
thread, typing.get_type_hints() is opinionated in ways that users of 
annotations may not want.  And personally I bristle at the idea of 
gating a language feature behind a library function.


Besides, most users will never know or care about __co_annotations__.  
If you're not even aware that it exists, it's not mysterious ;-)



Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JOBGFRXKFBB5TYEVGJZ7INEAEC4ZQDBY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



Sorry it took me 3+ days to reply--I had a lot to think about here.  But 
I have good things to report!



On 1/11/21 8:42 PM, Guido van Rossum wrote:
On Mon, Jan 11, 2021 at 1:20 PM Larry Hastings > wrote:


PEP 563 states:

For code that uses type hints, the typing.get_type_hints(obj,
globalns=None, localns=None) function correctly evaluates
expressions back from its string form.

So, if you are passing in a localns argument that isn't None,
okay, but you're not using them "correctly" according to the
language.  Also, this usage won't be compatible with static type
checkers.

I think you're misreading PEP 563 here. The mention of globalns=None, 
localns=None refers to the fact that these parameters have defaults, 
not that you must pass None. Note that the next paragraph in that PEP 
mentions eval(ann, globals, locals) -- it doesn't say eval(ann, {}, {}).


I think that's misleading, then.  The passage is telling you how to 
"correctly evaluate[s] expressions", and how I read it was, it's telling 
me I have to supply globalns=None and localns=None for it to work 
correctly--which, I had to discover on my own, were the default values.  
I don't understand why PEP 563 feels compelled to define a function that 
it's not introducing, and in fact had already shipped with Python two 
versions ago.



Later in that same section, PEP 563 points out a problem with 
annotations that reference class-scoped variables, and claims that the 
implementation would run into problems because methods can't "see" the 
class scope. This is indeed a problem for PEP 563, but *you* can 
easily generate correct code, assuming the containing class exists in 
the global scope (and your solution requires that anyway). So in this case

```
class Outer:
    class Inner:
   ...
    def method(self, a: Inner, b: Outer) -> None:
    ...
```
The generated code for the `__annotations__` property could just have 
a reference to `Outer.Inner` for such cases:

```
def __annotations__():
    return {"a": Outer.Inner, "b": Outer, "return": None}
```


This suggestion was a revelation for me.  Previously, a combination of 
bad experiences early on when hacking on compile and symtable, and my 
misunderstanding of exactly what was being asserted in the November 2017 
thread, led me to believe that all I could support was globals.  But 
I've been turning this over in my head for several days now, and I 
suspect I can support... just about anything.



I can name five name resolution scenarios I might encounter. I'll 
discuss them below, in increasing order of difficulty.



*First* is references to globals / builtins.  That's already working, 
it's obvious how it works, and I need not elaborate further.



*Second* is local variables in an enclosing function scope:

   def outer_fn():
    class C: pass
    def inner_fn(a:C=None): pass
    return inner_fn

As you pointed out elsewhere in un-quoted text, I could make the 
annotation a closure, so it could retain a reference to the value of 
(what is from its perspective) the free variable "C".



*Third* is local variables in an enclosing class scope, as you describe 
above:


   class OuterCls:
    class InnerCls:
        def method(a:InnerCls=None): pass

If I understand what you're suggesting, I could notice inside the 
compiler that Inner is being defined in a class scope, walk up the 
enclosing scopes until I hit the outermost class, then reconstruct the 
chain of pulling out attributes until it resolves globally. Thus I'd 
rewrite this example to:


   class OuterCls:
    class InnerCls:
        def method(a:OuterCls.InnerCls=None): pass

We've turned the local reference into a global reference, and we already 
know globals work fine.



*Fourth* is local variables in an enclosing class scope, which are 
themselves local variables in an enclosing function scope:


   def outerfn():
    class OuterCls:
    class InnerCls:
        def method(a:InnerCls=None): pass
    return OuterCls.InnerCls

Even this is solvable, I just need to combine the "second" and "third" 
approaches above.  I walk up the enclosing scopes to find the outermost 
class scope, and if that's a function scope, I create a closure and 
retain a reference to /that/ free variable.  Thus this would turn into


   def outerfn():
    class OuterCls:
    class InnerCls:
        def method(a:OuterCls.InnerCls=None): pass

and method.__co_annotations__ would reference the free variable 
"OuterCls" defined in outerfn.



*Fifth* is the nasty one.  Note that so far every definition we've 
referred to in an annotation has been /before/ the definition of the 
annotation.  What if we want to refer to something defined /after/ the 
annotation?


   def outerfn():
    class OuterCls:
    class InnerCls:
        def method(a:zebra=None): pass
    ...

We haven't

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-15 Thread Paul Bryan via Python-Dev

OK. Makes sense to think of __annotations__ as being the location of
the final, stable, "affixed" type hints.

I wonder if then the __co_annotations__ call and overwriting of
__annotations__ should be explicitly caused by a to get_type_hints
instead of (mysteriously) occurring on an attempt to getattr
__annotations__. I know this changes the descriptor behavior you
documented, but at least it would occur explicitly in a function call
and may be easier for developers to reason about?  It would also
address my other question of trying to access __annotations__, only to
be confronted with an exception raised within __co_annotations__.


On Fri, 2021-01-15 at 09:47 -0800, Larry Hastings wrote:
> 
> On 1/11/21 6:34 PM, Paul Bryan wrote:
> 
> On Mon, 2021-01-11 at 17:56 -0800, Larry Hastings wrote:
> 
> 
> > On 1/11/21 5:02 PM, Paul Bryan wrote:
> > 
> 
> > 
> > > I'm probably naive, but is there a reason that one could not just
> > > store a callable in __annotations__, and use the descriptor to
> > > resolve it to a dictionary and store it when it is accessed? It
> > > would be one less dunder in the Python data model.
> > That would work, but I think the API is a bit of a code smell. 
> > __annotations__ would no longer be stable:
> > 
> > > a.__annotations__ = o
> > > assert a.__annotations__ == o
> > Would that assert fail?  It depends on what type(o) is, which is
> > surprising.
> 
> Equally surprising?:
> 
> a.__co_annotations__ = o
> a.__annotations__
> assert a.__co_annotations__ == o
> 
> I've ruminated about this a bit over the past few days, and I finally
> realized exactly why, yes, I think behavior is more surprising.  It's
> because __annotations__ is now 12 years old (!), and never in that
> entire time has it silently changed its value.  It's always been
> completely stable, and we have twelve years' worth of installed base
> that may rely on that assumption.  In comparison, __co_annotations__
> is a new attribute.  While it's also surprising that
> __co_annotations__ can be automatically unset, at least this would be
> a documented part of its behavior from day 1.
> Relatedly, __co_annotations__ is behaving somewhat like a cached
> value, in that cached values get deleted when they're out-of-date. 
> (An observation that may provide some guidance if we decide to rename
> __co_annotations__.)  This idiom may be familiar to the user--unlike
> your proposed semantics, which I don't recall ever seeing used in an
> API.
> I admit it's only a small difference between what you proposed and
> what I propose, but in the end I definitely prefer my approach.
> Cheers,
> 
> /arry

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AV6JRZZDDN4GAPE5MHRFEBFUY3TEO7VZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/11/21 6:34 PM, Paul Bryan wrote:

On Mon, 2021-01-11 at 17:56 -0800, Larry Hastings wrote:


On 1/11/21 5:02 PM, Paul Bryan wrote:

I'm probably naive, but is there a reason that one could not just 
store a callable in __annotations__, and use the descriptor to 
resolve it to a dictionary and store it when it is accessed? It 
would be one less dunder in the Python data model.


That would work, but I think the API is a bit of a code smell.  
__annotations__ would no longer be stable:



a.__annotations__ = o
assert a.__annotations__ == o

Would that assert fail?  It depends on what type(o) is, which is 
surprising.




Equally surprising?:

a.__co_annotations__ = o
a.__annotations__
assert a.__co_annotations__ == o



I've ruminated about this a bit over the past few days, and I finally 
realized exactly why, yes, I think behavior is more surprising.  It's 
because __annotations__ is now 12 years old (!), and never in that 
entire time has it silently changed its value. It's always been 
completely stable, and we have twelve years' worth of installed base 
that may rely on that assumption.  In comparison, __co_annotations__ is 
a new attribute.  While it's also surprising that __co_annotations__ can 
be automatically unset, at least this would be a documented part of its 
behavior from day 1.


Relatedly, __co_annotations__ is behaving somewhat like a cached value, 
in that cached values get deleted when they're out-of-date.  (An 
observation that may provide some guidance if we decide to rename 
__co_annotations__.)  This idiom may be familiar to the user--unlike 
your proposed semantics, which I don't recall ever seeing used in an API.


I admit it's only a small difference between what you proposed and what 
I propose, but in the end I definitely prefer my approach.


Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HVIVCQ6AONKRZYWNY4VYHAOWT3WTEJAF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Paul Sokolovsky

Hello,

On Wed, 13 Jan 2021 05:04:36 -
"Jim J. Jewett"  wrote:

> Paul Sokolovsky wrote:
> > Ok, let's take "module attribute" as an example. Why do you think
> > there's anything wrong with this code:
> > ==
> > import config
> > from .types import *
> > if config.SUPPORT_BIGINT:
> > var: bigint = 1
> > else:
> > var: int64 = 1  
> 
> "Wrong" is too strong, but it would be better as
> 
> mybigint = bigint if config.SUPPORT_BIGINT else int64
> ...
> var:mybigint = 1

What's the explanation of why the above is better?

It seems following is ok with PEP649:

if config.LAYOUT_INT:
@dataclass
class MyData:
val: int
else:
@dataclass
class MyData:
val: float

So, how to explain to people that using the normal "if" is ok when
defining classes/dataclasses, but suddenly not normal when defining just
variables, and people should switch to the "if" expression?

> so asking people to rewrite it that way over the course of a major
> release is probably an acceptable price.

But why haste to ask people to rewrite their code? Why not start with
saying that PEP649 is not backward compatible, and ask it to explain
why it has pretty arbitrary limitations and discrepancies like above?
Then ask it how it can achieve backward compatibility? And that way is
obvious - the smart code objects which PEP649 creates, they should store
annotations just like PEP563 does, in a serialized form. Then those
smart code objects would deserialize and evaluate them. They may even
cache the end result.

But wait, PEP563 already has all that! It provides public API to get
annotations, typing.get_type_hints(), which already does all the
deserialization (maybe it doesn't do caching - *yet*), and effectively
treats __annotations__ as implementation detail. Because clearly, the
format of information stored there already depends on a particular
CPython version, and if you believe a thread running in parallel, going
to change going forward.

Seen like that, PEP649 is just a quest for making __annotations__ be
the "public API", instead of the already defined public API, and which
__annotations__ already can't be, as its format already varies widely
(and likely will keep varying going forward). And while questing for
that elusive goal, it even adds arbitrary restrictions for usage of
annotations which never were there before, truly breaking backward
compatibility and some annotation usages.

-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PWMEB3LWM6WMEEA5ZTZUPA3JHRLDSF5R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Jim J. Jewett

Paul Sokolovsky wrote:
> Ok, let's take "module attribute" as an example. Why do you think
> there's anything wrong with this code:
> ==
> import config
> from .types import *
> if config.SUPPORT_BIGINT:
> var: bigint = 1
> else:
> var: int64 = 1

"Wrong" is too strong, but it would be better as

mybigint = bigint if config.SUPPORT_BIGINT else int64
...
var:mybigint = 1

so asking people to rewrite it that way over the course of a major release is 
probably an acceptable price.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FGZ4YK63MIZ6XVLQ4OMVJU7HUJPG73CD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Inada Naoki

On Wed, Jan 13, 2021 at 1:47 AM Larry Hastings  wrote:
>
> On 1/11/21 5:33 PM, Inada Naoki wrote:
>
> Note that PEP 563 semantics allows more efficient implementation.
> Annotation is just a single constant tuple, not a dict.
> We already have the efficient implementation for Python 3.10.
>
> The efficient implementation in 3.10 can share tuples. If there are
> hundreds of methods with the same signature, annotation is just a
> single tuple, not hundreds of tuples. This is very efficient for auto
> generated codebase. I think this PEP can share the code objects for
> same signature by removing co_firstlineno information too.
>
> That's very clever!  My co_annotations repo was branched from before this 
> feature was added, and I haven't pulled and merged recently.  So I hadn't 
> seen it.
>

Please see this pull request too. It merges co_code and co_consts. It
will save more RAM and importing time of your implementation.
https://github.com/python/cpython/pull/23056

>
> Additionally, we should include the cost for loading annotations from
> PYC files, because most annotations are "load once, set once".
> Loading "simple code object" from pyc files is not so cheap. It may
> affect importing time of large annotated codebase and memory
> footprints.
>
> I did some analysis in a separate message.  The summary is, the code object 
> for a single annotation costs us 232 bytes; that includes the code object 
> itself, the bytestring for the bytecode, and the bytestring for the lnotab.  
> This grows slowly as you add new parameters; the code object for ten 
> parameters is 360 bytes.
>
> It seems possible to create a hybrid of these two approaches!  Here's my 
> idea: instead of the compiler storing a code object as the annotations 
> argument to MAKE_FUNCTION, store a tuple containing the fields you'd need to 
> recreate the code object at runtime--bytecode, lnotab, names, consts, etc. 
> func_get_annotations would create the code object from that, bind it to a 
> function object, call it, and return the result.  These code-object-tuples 
> would then be automatically shared in the .pyc file and at runtime the same 
> way that 3.10 shares the tuples of stringized annotations today.

It may be good idea if we can strip most code object members, like
argcount, kwonlyargcount, nlocals, flags, freevars, cellvars,
filename, name, firstlineno, linetable.
It can be smaller than Python 3.9.

>
> That said, I suggest PEP 649's memory consumption isn't an urgent 
> consideration in choosing to accept or reject it.  PEP 649 is competitive in 
> terms of startup time and memory usage with PEP 563, and PEP 563 was accepted 
> and shipped with several versions of Python.
>

I still want a real-world application/library with heavy annotation.
My goal is to use annotations in the stdlib without caring about
resource usage or importtime.
But I agree with you if PEP 649 will be smaller than Python 3.9.

Regards,
-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CNXCFW2PXUCZ75OBFZTUS3TVKI3IEKZH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/12/21 3:53 PM, Greg Ewing wrote:

On 13/01/21 5:47 am, Larry Hastings wrote:


instead of the compiler storing a code object as the annotations 
argument to MAKE_FUNCTION, store a tuple containing the fields you'd 
need to /recreate/ the code object at runtime--bytecode, lnotab, 
names, consts, etc.


Would that tuple be significantly smaller than the corresponding
code object, though?


It would only be slightly smaller.  The point of doing it would be to 
boil out fields that change per-object (e.g. co_name) so that functions 
with identical signatures would share the same tuple both in the .pyc 
and at runtime.  This idea is predicated on Inada-san's assertion that 
this is an important memory optimization, that there are large 
heavily-annotated projects with lots of functions/methods with identical 
signatures where this memory savings is significant.



//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QCARB4FO3DYT46U2R5VX5AQYZ4OW7K3I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Greg Ewing


On 13/01/21 5:47 am, Larry Hastings wrote:


instead of the compiler storing a code object as the annotations 
argument to MAKE_FUNCTION, store a tuple containing the fields you'd 
need to /recreate/ the code object at runtime--bytecode, lnotab, names, 
consts, etc.


Would that tuple be significantly smaller than the corresponding
code object, though?

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2NHFIGK55XN4U77V6V2KTBRGJ7TSGAAB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Greg Ewing


On 13/01/21 6:54 am, Larry Hastings wrote:
Note that this only works in the current version of the PEP / prototype, 
where annotations are strictly evaluated in module scope.  If we start 
supporting closures, those need "cell" objects, which IIUC can't be 
marshalled.


The cells belong to the enclosing scope, not the function that uses
them. I don't think they would interfere with sharing parts of code
objects between identical annotations.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QKMYWGBSQKNFUNBHXXWFJZ7WIRHGTEWI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Guido van Rossum

On Tue, Jan 12, 2021 at 11:31 AM Jim J. Jewett  wrote:

> Larry Hastings wrote:
> > The control-flow exclusion is for /module//attribute/ or /class
> > attribute/ annotations:
> > class C:
> >if random.random() > 0.5:
> >  my_attr:int=3
> >else:
> >  my_attr2:float=3.5
>
> That very example would be helpful in the FAQ, though I understand if
> you're concerned about making a minor sub-section seem too long.
>

This elucidates a crucial point to me: Larry's proposal looks at the source
code of the annotations.


> If I understand correctly, the problem is that you can't store multiple
> alternative annotations on my_attr.  Therefore:
>
> class C:
> my_attr:(int if random.random > 0.5 else float)
>
> should be OK, because there is only a single annotation.
>

Does that mean that the generated function would contain the entire
expression `(int if random.random > 0.5 else float)`? I guess that's what
it has to mean. But the PEP only uses such simple examples that it's easy
to miss this.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CQYPHXLTKOI3ZWUOFTX5VAOUIXLVA3OP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/12/21 12:16 PM, Paul Bryan wrote:

On Tue, 2021-01-12 at 09:54 -0800, Larry Hastings wrote:


Note that this only works in the current version of the PEP / 
prototype, where annotations are strictly evaluated in module scope.  
If we start supporting closures, those need "cell" objects, which 
IIUC can't be marshalled.


Since the __co_annotations__ function will get globals from the 
function it annotates, doesn't it get more than just module scope?



I'm not sure what you're asking.  And, uh, what's the difference between 
"globals" and "module scope"?



//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OTIQH6QOZ7TTUPARANPSF3NNC275LGJS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Paul Sokolovsky

Hello,

On Mon, 11 Jan 2021 13:44:45 -0800
Larry Hastings  wrote:

> The control-flow exclusion is for /module//attribute/ or /class 
> attribute/ annotations:
> 
> class C:
>    if random.random() > 0.5:
>      my_attr:int=3
>    else:
>      my_attr2:float=3.5

Ok, let's take "module attribute" as an example. Why do you think
there's anything wrong with this code:

==
import config
from .types import *

if config.SUPPORT_BIGINT:
var: bigint = 1
else:
var: int64 = 1
==

> Your example doesn't define any module attributes or class attributes 
> inside flow control statements, so that code should work fine.  
> (Defining functions/methods inside flow control statements isn't a
> problem.)

PEP649 criticizes PEP563's approach with big words like "It requires
Python implementations to stringize their annotations. This is
surprising behavior — unprecedented for a language-level feature." But
itself devolves to clauses like:

>>>  It seems reasonable to declare that both are at the very least
>>> unsupported, and their use results in undefined behavior. It might
>>> be worth making a small effort to explicitly prohibit them with
>>> compile-time checks.

Isn't the fact that PEP563 doesn't have problems with annotations in
conditionals is a sign of PEP563 technical superiority? And its
"unprecedented" measure of storing annotations as strings is actually a
clever technical feat - it should have stored annotations as AST trees,
but such trees would take quite a lot of memory. So, PEP563 smartly
went to store those ASTs in a serialized format. So, those strings
aren't strings, but serialized ASTs.

Overall 2 comments/questions:

1. Was there an attempt to devise how to make PEP649 deal with existing
Python language features (like conditionals)?
2. As a general comment, PEP649, by placing arbitrary restrictions on
where annotations can appear, tries to dig under the foundations of
Python as a dynamic language. Which is absolutely great, it just the
PEP should be viewed as such - undermining Python's dynamic nature, as
if there's something wrong with it.

-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2GLVNLJ4WZMQ5FPMNAFZFU7MSVK5GOTJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Paul Bryan via Python-Dev

On Tue, 2021-01-12 at 09:54 -0800, Larry Hastings wrote:
> Note that this only works in the current version of the PEP /
> prototype, where annotations are strictly evaluated in module scope. 
> If we start supporting closures, those need "cell" objects, which
> IIUC can't be marshalled.
Since the __co_annotations__ function will get globals from the
function it annotates, doesn't it get more than just module scope?

Paul

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VWY42HWKA5LQ7BZ7VROCEDWVYHWELSBI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/12/21 11:26 AM, Jim J. Jewett wrote:

If I understand correctly, the problem is that you can't store multiple 
alternative annotations on my_attr.  Therefore:

 class C:
 my_attr:(int if random.random > 0.5 else float)

should be OK, because there is only a single annotation.


Sure, that works fine.  Any expression (except "yield" and ":=") is okay 
in an annotation.




What about optional attributes, like:

  class C:
     if random.random() > 0.5:
       my_attr:int=3

Also, would (conditionally defined) function variable attributes become a 
problem if they were actually stored?  (Take Larry's class example, and make if 
a def instead of a class statement.)


You mean attributions on function locals?

   def foo():
  if random.random() > 0.5:
    x:int=3
  else:
    x:float=3.5

As I mentioned in my PEP, attributions on function locals have no effect 
at runtime.  If they did, this would cause the same problem that doing 
it in classes has.



Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RXNT6SUAIBOEUBGQLZ4TCW7CJVWAFY7I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Jim J. Jewett

Larry Hastings wrote:
> The control-flow exclusion is for /module//attribute/ or /class 
> attribute/ annotations:
> class C:
>    if random.random() > 0.5:
>      my_attr:int=3
>    else:
>      my_attr2:float=3.5

That very example would be helpful in the FAQ, though I understand if you're 
concerned about making a minor sub-section seem too long.

If I understand correctly, the problem is that you can't store multiple 
alternative annotations on my_attr.  Therefore:

class C:
my_attr:(int if random.random > 0.5 else float)

should be OK, because there is only a single annotation.

What about optional attributes, like:

 class C:
    if random.random() > 0.5:
      my_attr:int=3

Also, would (conditionally defined) function variable attributes become a 
problem if they were actually stored?  (Take Larry's class example, and make if 
a def instead of a class statement.)

My (weakly held, personal) opinion is that these restrictions would be 
reasonable, and a single release of deprecation would be enough, but it would 
be better if that code could trigger a deprecation warning during that release, 
even for code that hasn't done the future import.  It would also be OK to just 
say "implementation-defined behavior; CPython 3.x ignores the annotation" 
instead of banning them.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TRX23XWZ3NGROVQRC6DXLIU7NPPNEZRB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Guido van Rossum

Given all the effort that get_type_hints() puts into evaluating those
strings it seems important to spell out explicitly that they're not
special. (IIRC they *are* special in PEP 563.)

On Tue, Jan 12, 2021 at 8:56 AM Larry Hastings  wrote:

>
> Yes, PEP 649 is completely agnostic about what values you put in as
> annotations.  You can put in strings, complex objects,
> expressions--whatever you put in, you get back out later.
>
> I'm happy to add some text to the PEP if this needs clarifying; I just
> thought it was obvious.
>
>
> Cheers,
>
>
> */arry*
> On 1/11/21 9:11 PM, Guido van Rossum wrote:
>
>  Another thought about this PEP (hopefully my last one tonight).
>
> The section on backwards compatibility doesn't mention what should happen
> with annotations that are stringified by the user (as is needed for forward
> references in code that hasn't been PEP-563-ified yet).
>
> That's a PEP 484 feature. Should we start deprecating that at the same
> time? Static checkers support it but don't need it (for example, stubs in
> typeshed don't use it since their code is never evaluated).
>
> At the very least I think your PEP should mention what happens for these
> -- presumably `__annotations__` will just contain the string literal, so
> get_type_hints() would be needed to evaluate these.
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
>
>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MFPOC45A5IWY6HZU6FQQQV2ZYIRVRWUP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/12/21 8:47 AM, Larry Hastings wrote:
It seems possible to create a hybrid of these two approaches! Here's 
my idea: instead of the compiler storing a code object as the 
annotations argument to MAKE_FUNCTION, store a tuple containing the 
fields you'd need to /recreate/ the code object at runtime--bytecode, 
lnotab, names, consts, etc. func_get_annotations would create the code 
object from that, bind it to a function object, call it, and return 
the result.  These code-object-tuples would then be automatically 
shared in the .pyc file and at runtime the same way that 3.10 shares 
the tuples of stringized annotations today.


Note that this only works in the current version of the PEP / prototype, 
where annotations are strictly evaluated in module scope.  If we start 
supporting closures, those need "cell" objects, which IIUC can't be 
marshalled.



//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I5TKGDL4KIRQIL5G56ZTCVG7O2PKAPBI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



Yes, PEP 649 is completely agnostic about what values you put in as 
annotations.  You can put in strings, complex objects, 
expressions--whatever you put in, you get back out later.


I'm happy to add some text to the PEP if this needs clarifying; I just 
thought it was obvious.



Cheers,


//arry/

On 1/11/21 9:11 PM, Guido van Rossum wrote:

 Another thought about this PEP (hopefully my last one tonight).

The section on backwards compatibility doesn't mention what should 
happen with annotations that are stringified by the user (as is needed 
for forward references in code that hasn't been PEP-563-ified yet).


That's a PEP 484 feature. Should we start deprecating that at the same 
time? Static checkers support it but don't need it (for example, stubs 
in typeshed don't use it since their code is never evaluated).


At the very least I think your PEP should mention what happens for 
these -- presumably `__annotations__` will just contain the string 
literal, so get_type_hints() would be needed to evaluate these.


--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him //(why is my pronoun here?)/ 

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7Q3SHNMRKLROBESEB2U4NNMZAFS2XFRU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 1/11/21 5:33 PM, Inada Naoki wrote:

Note that PEP 563 semantics allows more efficient implementation.
Annotation is just a single constant tuple, not a dict.
We already have the efficient implementation for Python 3.10.

The efficient implementation in 3.10 can share tuples. If there are
hundreds of methods with the same signature, annotation is just a
single tuple, not hundreds of tuples. This is very efficient for auto
generated codebase. I think this PEP can share the code objects for
same signature by removing co_firstlineno information too.

That's very clever! My co_annotations repo was branched from before
this feature was added, and I haven't pulled and merged recently. So I
hadn't seen it.

Additionally, we should include the cost for loading annotations from
PYC files, because most annotations are "load once, set once".
Loading "simple code object" from pyc files is not so cheap. It may
affect importing time of large annotated codebase and memory
footprints.

I did some analysis in a separate message. The summary is, the code
object for a single annotation costs us 232 bytes; that includes the
code object itself, the bytestring for the bytecode, and the bytestring
for the lnotab. This grows slowly as you add new parameters; the code
object for ten parameters is 360 bytes.

It seems possible to create a hybrid of these two approaches! Here's my
idea: instead of the compiler storing a code object as the annotations
argument to MAKE_FUNCTION, store a tuple containing the fields you'd
need to /recreate/ the code object at runtime--bytecode, lnotab, names,
consts, etc. func_get_annotations would create the code object from
that, bind it to a function object, call it, and return the result.
These code-object-tuples would then be automatically shared in the .pyc
file and at runtime the same way that 3.10 shares the tuples of
stringized annotations today.

That said, I suggest PEP 649's memory consumption isn't an urgent
consideration in choosing to accept or reject it. PEP 649 is
competitive in terms of startup time and memory usage with PEP 563, and
PEP 563 was accepted and shipped with several versions of Python.

Cheers,

//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/375OTKI42EUKMB5VQAAIBXMQWQTXPYHH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/11/21 5:37 PM, Larry Hastings wrote:
I'll have to let people with large code bases speak up about this, but 
my understanding is that most people would prefer Python to use less 
memory.  On my 64-bit Linux machine, a code object is 136 bytes, its 
empty __dict__ is 64 bytes, [...]



Oops: where I said "a code object is 136 bytes", I meant, a /function/ 
object is 136 bytes.  We regret the error.



//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XPEY2HJLVBW23YWQ7SK4YBNKMNW53GFT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-11 Thread Guido van Rossum

 Another thought about this PEP (hopefully my last one tonight).

The section on backwards compatibility doesn't mention what should happen
with annotations that are stringified by the user (as is needed for forward
references in code that hasn't been PEP-563-ified yet).

That's a PEP 484 feature. Should we start deprecating that at the same
time? Static checkers support it but don't need it (for example, stubs in
typeshed don't use it since their code is never evaluated).

At the very least I think your PEP should mention what happens for these --
presumably `__annotations__` will just contain the string literal, so
get_type_hints() would be needed to evaluate these.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VDLEMVDGBHKJEQR4JYZUWOPQMYYTCH5J/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

Will do.

On Mon, 2021-01-11 at 20:55 -0800, Guido van Rossum wrote:
> On Mon, Jan 11, 2021 at 3:51 PM Larry Hastings 
> wrote:
> > [...]
> > > This passage in PEP 563 appears not true in Python 3.9 with
> > > __future__ annotations, emphasis mine:
> > > 
> > > > The get_type_hints() function automatically resolves the
> > > > correct value of globalns for functions and classes. It also
> > > > automatically provides the correct localns for classes.
> > > 
> > > If this passage was true, I believe the issue that resulted in my
> > > affixing type hints could have been averted.
> > As you've discovered, this is one of the places where PEP 563 seems
> > to be out-of-date with respect to its implementation.  I sifted
> > through the source code to typing.get_type_hints() twice, and near
> > as I can figure out, localns is literally only ever set to None
> > unless you override it with the parameter.
> > 
> 
> 
> This seems to be a bug in get_type_hints() for which someone should
> file a bug on bpo, please!

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LS36RQ2UPLFKIOJAXF7EAGEMKBOCEIKW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-11 Thread Guido van Rossum

On Mon, Jan 11, 2021 at 3:51 PM Larry Hastings  wrote:

> [...]
>
> This passage in PEP 563 appears not true in Python 3.9 with __future__
> annotations, emphasis mine:
>
> The get_type_hints() function automatically resolves the correct value of
> globalns for functions and classes. *It also automatically provides the
> correct localns for classes.*
>
>
> If this passage was true, I believe the issue that resulted in my affixing
> type hints could have been averted.
>
> As you've discovered, this is one of the places where PEP 563 seems to be
> out-of-date with respect to its implementation.  I sifted through the
> source code to typing.get_type_hints() twice, and near as I can figure out,
> localns is literally only ever set to None unless you override it with the
> parameter.
>

This seems to be a bug in get_type_hints() for which someone should file a
bug on bpo, please!
-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FP5FMEHEA3FXOURIP5SSNDZMRBM7MHWU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-11 Thread Guido van Rossum

On Mon, Jan 11, 2021 at 1:20 PM Larry Hastings  wrote:

> PEP 563 states:
>
> For code that uses type hints, the typing.get_type_hints(obj,
> globalns=None, localns=None) function correctly evaluates expressions back
> from its string form.
>
> So, if you are passing in a localns argument that isn't None, okay, but
> you're not using them "correctly" according to the language.  Also, this
> usage won't be compatible with static type checkers.
>
I think you're misreading PEP 563 here. The mention of globalns=None,
localns=None refers to the fact that these parameters have defaults, not
that you must pass None. Note that the next paragraph in that PEP mentions
eval(ann, globals, locals) -- it doesn't say eval(ann, {}, {}). There is
considerable discussion below in that same section, but it doesn't come
closer than stating "using local state in annotations is no longer possible
in general", and it states that get_type_hints() correctly provides the
right localns for classes.

Apart from how PEP 563 should be interpreted (maybe we should ask Lukasz) I
would say that basically all feedback I've seen over the changed semantics
of annotations in 3.10 is about this problem, and when I first saw your
proposal I thought that this would solve those issues. I'd be much less
enthusiastic if we're still going to force annotations to only reference
globals.

My read on PEP 563 is that it *primarily* solves the issue of having to put
forward references in string quotes (the most common case probably being
methods that accept or return an instance of the class in which they are
being defined). You have come up with a better solution for that -- the
solution that eluded us all when we were debating PEP 563. But I think that
you should try for maximal backward compatibility with the state of the
world *before* PEP 563.

In particular, static type checkers have no problem with annotations
referencing types defined in the local scope. This has always worked (hence
the complaints about PEP 563) and it would actually be extra work to forbid
this in certain situations but not others.

[...]

> *2. __co_annotations__ scope?*
>
> I'm wondering why __co_annotations__ function could not be scoped (within
> a closure?) such that it can access the values where the function, method,
> class or module are being declared? I acknowledge that I'm railing against
> PEP 563 again, trying to reclaim lost ground.
>
> This is addressed in PEP 563, when it rejected the idea of using "function
> local state when defining annotations":
>
> This would be prohibitively expensive for highly annotated code as the
> frames would keep all their objects alive. That includes predominantly
> objects that won't ever be accessed again.
>
>
> https://www.python.org/dev/peps/pep-0563/#keeping-the-ability-to-use-function-local-state-when-defining-annotations
>
>
But that's a strawman for your PEP. In PEP 563, they would have to keep the
whole frame alive. But with your PEP only the cells containing objects that
are *actually* referenced by annotations, i.e. exactly the objects that the
user wants to see preserved in `__annotations__`. (Well, at Instagram they
probably don't want to see any annotations survive the compilation stage,
but they can hack their own Python compiler -- we know they do that anyway.
:-)

Later in that same section, PEP 563 points out a problem with annotations
that reference class-scoped variables, and claims that the implementation
would run into problems because methods can't "see" the class scope. This
is indeed a problem for PEP 563, but *you* can easily generate correct
code, assuming the containing class exists in the global scope (and your
solution requires that anyway). So in this case
```
class Outer:
class Inner:
   ...
def method(self, a: Inner, b: Outer) -> None:
...
```
The generated code for the `__annotations__` property could just have a
reference to `Outer.Inner` for such cases:
```
def __annotations__():
return {"a": Outer.Inner, "b": Outer, "return": None}
```

(Note that for *function* locals you don't have to do anything, they just
work, because of closures -- you may not understand them, but they are
important here. :-)

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IC2FDBJLHZA6HC7X5Q52Y46S66WTKPRS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/11/21 6:34 PM, Paul Bryan wrote:


a.__annotations__ = o
assert a.__annotations__ == o

Would that assert fail?  It depends on what type(o) is, which is 
surprising.




Equally surprising?:

a.__co_annotations__ = o
a.__annotations__
assert a.__co_annotations__ == o



Well, since you asked, no.  It's a good point, but I think my example is 
a touch more "surprising".  In my example, a.__annotations__ has two 
different True values; in yours, a.__co_annotations__ goes from True to 
False.  It's acting more like a cache.


Also consider: if you set o.__annotations__ to a function, what if you 
want to examine the function later?  What if you want to examine the 
function built by Python?


   def enhance_annotations(co_annotations):
    def enhance():
    d = co_annotations()
    d['extra'] = ...
    return d
   o.__co_annotations__ = enhance_annotations(o.__co_annotations__)

Finally, it's just a code smell to have one attribute support such a 
bewildering variety of types.  Of course, this /works,/ but it's bad 
hygiene to store such violently different types in what is ostensibly 
the same attribute.



Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5OSZHRZUXD2DZG54ROUYQ3IQW42DFUAK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/11/21 6:29 PM, Greg Ewing wrote:

On 12/01/21 10:41 am, Larry Hastings wrote:


So: if you're using annotations for something besides "type hints",


Didn't Guido decree that using annotations for anything other than
type hints is no longer supported?



It was never an official decree.  And, for what it's worth, Guido mostly 
walked that back, last month:


   
https://mail.python.org/archives/list/python-dev@python.org/message/OSA7VKZSAHQTTVEJDU2MZXESQKTDNKTB/


Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7HTD33PE7OLQ35ZH2G4U5463HY3GX3TN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors


On 12/01/21 2:56 pm, Larry Hastings wrote:


In PEP 649 I think every reference of "binding" is talking 
about binding a code object to a globals dict to produce a function 
object.   The process of binding a function to an object instance to 
make a method is conceptually very similar, but distinct.


Maybe "bare code object" would be a less confusing term?

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EMVOMYUEQN3ZV5Y2P5B4NEZEJ2KMJTLM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On Mon, 2021-01-11 at 17:56 -0800, Larry Hastings wrote:

> On 1/11/21 5:02 PM, Paul Bryan wrote:
> 
> > 
> > Some more questions...
> > 
> > "Binding"," bound" and "unbound" code objects:
> > Is your use of "binding" terminology in the PEP identical to the
> > binding of a function to an object instance as a method during
> > object creation?
> I'm not.  In PEP 649 I think every reference of "binding" is talking
> about binding a code object to a globals dict to produce a function
> object.   The process of binding a function to an object instance to
> make a method is conceptually very similar, but distinct.
> (and btw, functions aren't bound to their object to make methods
> during object creation, it's done lazily at the time you ask for it--
> that's what the "descriptor protocol" is all about!)
I think it'd be worth briefly describing binding, to avoid confusion.
I'm now more educated on function/method binding lifecycle though, so
bonus! 🙂

> > Function Annotations:
> > 
> > > When binding an unbound annotation code object, a function will
> > > use its own __globals__ as the new function's globals.
> > I'm having trouble parsing this. Does this mean the newly bound
> > __co_annotations__ function will inherit __globals__ from the
> > function it's annotating?
> Yes.  Though I wouldn't use "inherit", I'd just say it "uses" the
> __globals__ from the function.

Then I think this solves my particular scoping problem with 3.10
string-evaluated annotations!

> > Exceptions:
> > It's quite possible for a __co_annotation__ function call to raise
> > an exception (e.g. NameError). When accessing __annotations__, if
> > such an exception is raised during the call to __co_annotations__,
> > what is the expected behavior?
> If the function fails for any reason--throws an exception, or just
> doesn't return an acceptable value--then the getter immediately
> exits, and the internal state of the object is unchanged.  If you
> wanted to, you could catch the exception, fix the error, and get
> __annotations__ again, and it'd work.
OK.

> > s/__co_//?:
> > I'm probably naive, but is there a reason that one could not just
> > store a callable in __annotations__, and use the descriptor to
> > resolve it to a dictionary and store it when it is accessed? It
> > would be one less dunder in the Python data model.
> That would work, but I think the API is a bit of a code smell. 
> __annotations__ would no longer be stable:
> > a.__annotations__ = o
> > assert a.__annotations__ == o
> Would that assert fail?  It depends on what type(o) is, which is
> surprising.

Equally surprising?:

a.__co_annotations__ = o
a.__annotations__
assert a.__co_annotations__ == o

Paul

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z7SFYJZQHUQKZC77L4EMNZIYAQUXK3GJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors


On 12/01/21 11:32 am, Łukasz Langa wrote:


EdgeDB uses stringified annotations exclusively which 
minimizes runtime memory usage of annotations because those strings are 
pretty much all ASCII and many can be interned.


946 -> s_schema.Schema
362 -> str
298 -> sd.CommandContext


Seems to me the code objects for those would be identical
wherever they're used, and so could be cached and re-used
the same way as interned strings.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MYI2JXMBR52HGVL46DRM2KW5T34GC7OP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors


On 12/01/21 10:41 am, Larry Hastings wrote:


So: if you're using annotations for something besides "type hints",


Didn't Guido decree that using annotations for anything other than
type hints is no longer supported?

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GP5PHEAGY4USPDLLMWUG52CGQQ3K4AKI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/11/21 5:02 PM, Paul Bryan wrote:

Some more questions...

"Binding"," bound" and "unbound" code objects:
Is your use of "binding" terminology in the PEP identical to the 
binding of a function to an object instance as a method during object 
creation?


I'm not.  In PEP 649 I think every reference of "binding" is talking 
about binding a code object to a globals dict to produce a function 
object.   The process of binding a function to an object instance to 
make a method is conceptually very similar, but distinct.


(and btw, functions aren't bound to their object to make methods during 
object creation, it's done lazily at the time you ask for it--that's 
what the "descriptor protocol" is all about!)





Function Annotations:
When binding an unbound annotation code object, a function will use 
its own __globals__ as the new function's globals.
I'm having trouble parsing this. Does this mean the newly bound 
__co_annotations__ function will inherit __globals__ from the function 
it's annotating?


Yes.  Though I wouldn't use "inherit", I'd just say it "uses" the 
__globals__ from the function.




Exceptions:
It's quite possible for a __co_annotation__ function call to raise an 
exception (e.g. NameError). When accessing __annotations__, if such an 
exception is raised during the call to __co_annotations__, what is the 
expected behavior?


If the function fails for any reason--throws an exception, or just 
doesn't return an acceptable value--then the getter immediately exits, 
and the internal state of the object is unchanged.  If you wanted to, 
you could catch the exception, fix the error, and get __annotations__ 
again, and it'd work.




s/__co_//?:
I'm probably naive, but is there a reason that one could not just 
store a callable in __annotations__, and use the descriptor to resolve 
it to a dictionary and store it when it is accessed? It would be one 
less dunder in the Python data model.


That would work, but I think the API is a bit of a code smell. 
__annotations__ would no longer be stable:


   a.__annotations__ = o
   assert a.__annotations__ == o

Would that assert fail?  It depends on what type(o) is, which is surprising.


Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QTA2DT5G2U7PIQIABOW2FWRQS73EPDOR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/11/21 4:55 PM, Greg Ewing wrote:

On 12/01/21 6:21 am, Larry Hastings wrote:


Unbound code objects


...The "annotation code object" is then stored *unbound*
as the internal value of ``__co_annotations__``; it is then
bound on demand when the user asks for ``__annotations__``.


This seems like premature optimisation. Function objects are
tiny compared to the code object, which is already a fairly
complicated thing composed of a number of sub-objects.



I'll have to let people with large code bases speak up about this, but 
my understanding is that most people would prefer Python to use less 
memory.  On my 64-bit Linux machine, a code object is 136 bytes, its 
empty __dict__ is 64 bytes, and the other stuff you get for free.  So 
that's 200 bytes even.  Multiply that by 1000 and the back of my 
envelope says you've wasted 200k.  Is that a big deal?  I dunno.


On the other hand, the code to support dynamically binding the code 
object on demand wasn't a big deal.



Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VZVUIOMI4ECP5H56PZBYIR3RYS2FGFXK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-11 Thread Inada Naoki

> Performance
> ---
>
> Performance with this PEP should be favorable.  In general,
> resources are only consumed on demand—"you only pay for what you use".
>

Nice!

> There are three scenarios to consider:
>
> * the runtime cost when annotations aren't defined,
> * the runtime cost when annotations are defined but *not* referenced, and
> * the runtime cost when annotations are defined *and* referenced.
>

Note: The first two cases are major. Many codes doesn't have
annotations. Many codes use annotations just only for documentation or
static checker.
In the second scenario, the annotations must be very cheap. Its cost
must be comparable with docstrings.
Otherwise, people can not use annotation freely in a large codebase.
Or we must provide an option like -OO to strip annotations.


> We'll examine each of these scenarios in the context of all three
> semantics for annotations: stock, PEP 563, and this PEP.
>
> When there are no annotations, all three semantics have the same
> runtime cost: zero. No annotations dict is created and no code is
> generated for it.  This requires no runtime processor time and
> consumes no memory.
>
> When annotations are defined but not referenced, the runtime cost
> of Python with this PEP should be slightly faster than either
> original Python semantics or PEP 563 semantics.  With those, the
> annotations dicts are built but never examined; with this PEP,
> the annotations dicts won't even be built.  All that happens at
> runtime is the loading of a single constant (a simple code
> object) which is then set as an attribute on an object.  Since
> the annotations are never referenced, the code object is never
> bound to a function, the code to create the dict is never
> executed, and the dict is never constructed.
>

Note that PEP 563 semantics allows more efficient implementation.
Annotation is just a single constant tuple, not a dict.
We already have the efficient implementation for Python 3.10.

The efficient implementation in 3.10 can share tuples. If there are
hundreds of methods with the same signature, annotation is just a
single tuple, not hundreds of tuples. This is very efficient for auto
generated codebase. I think this PEP can share the code objects for
same signature by removing co_firstlineno information too.

Additionally, we should include the cost for loading annotations from
PYC files, because most annotations are "load once, set once".
Loading "simple code object" from pyc files is not so cheap. It may
affect importing time of large annotated codebase and memory
footprints.

I think we need a reference application that has a large codebase and
highly annotated. But we need to beware even if the large application
is 100% annotated, libraries used are not 100% annotated.
Many libraries are dropping Python 2 support and start annotating. The
cost of the annotations will become much more important in next
several years.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Q5MSBHXD3VPCVQODSSO3FOB3DRQS4SVG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors


On 12/01/21 10:16 am, Larry Hastings wrote:
This is addressed in PEP 563, when it rejected the idea of using 
"function local state when defining annotations":


This would be prohibitively expensive for highly annotated code as
the frames would keep all their objects alive. That includes
predominantly objects that won't ever be accessed again.


I'm not sure what that's supposed to mean.

Firstly, functions that reference nonlocal names don't keep whole
frames alive, only the particular objects they reference.

Secondly, if an annotation references something at module level,
that something will also be kept alive unless it is explicitly
removed from the module -- which could also be done at a local
level if you didn't want to keep those things around.

So I don't really see any difference between global and local
state when it comes to things being kept alive by annotations.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GEO6SQL2ZN4KR5A72RK3OGEN4SZQM4TK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/11/21 2:32 PM, Łukasz Langa wrote:
1. What do you anticipate the memory usage will look like for your 
solution compared to PEP 563?


It depends on the scenario.  I talk about three runtime scenarios in PEP 
649.  But the most relevant scenario is "annotations are defined but 
never examined", because it's by far the most common for people using 
annotations.  So for now let's just talk about that.  In this scenario, 
I expect PEP 649 to be on par with PEP 563.


PEP 563 will define a small dict that nobody looks at; PEP 649 will 
define a simple code object that nobody runs.  These objects consume 
pretty similar amounts of memory.


A quick experiment: on my 64-bit Linux laptop, with a function that had 
three annotated parameters, sys.sizeof() of the resulting annotation 
dict was 232 bytes.  PEP 649 generated a 176 byte code object--but we 
should also factor in its bytecode (45 bytes) and lnotab (33 bytes), 
giving us 257 bytes.  (The other fields of the code object are redundant 
references to stuff we already had lying around.)


In that case PEP 649 is slightly bigger.  But if we change it to twelve 
annotated parameters, PEP 649 becomes a big win.  The dict is now 640 
bytes (!), but the code object only totals 280 bytes. It seems to flip 
at about five parameters; less than that, and the dict wins a little, 
greater than that and the code object starts winning by more and more.



2. What is your expected startup performance of an annotated Python 
application using co_annotations?


Again, the most relevant scenario is "annotations are defined but not 
referenced" so we'll stick with that.


On balance it should be roughly equivalent to "PEP 563" semantics, and 
perhaps a teeny-tiny bit faster.


With PEP 563 semantics, defining a function / class / module with 
annotations must build the annotations dict, then store it on the 
object.  But all the keys and values are strings, so the bytecode isn't 
much--for functions, it's just a bunch of LOAD_CONSTs then a 
BUILD_CONST_KEY_MAP.  For classes and modules it's a bit wordier, but if 
the bytecode performance was important here, we could probably convert 
it to use BUILD_CONST_KEY_MAP too.


With my PEP, defining a function / class / module with annotations means 
you LOAD_CONST the code object, then store it on the object--and that's 
it.  (Right now there's the whole __globals__ thing but I expect to get 
rid of that eventually).  Of course, the code object isn't free, it has 
to be unmarshalled--but as code objects go these are pretty simple 
ones.  Annotations code objects tend to have two custom bytestrings and 
a non-"small" int, and all the other attributes we get for free.


"stock" Python semantics is a bit slower than either, because it also 
evaluates all the annotations at the time the function / class / module 
is bound.



I'd love to hear real-world results from someone with a large annotated 
code base.  Unfortunately, I'm pretty sure typing.py is broken in the 
prototype right now, so it's a bit early yet.  (I honestly don't think 
it'll be that hard to get it working again, it was just one of a million 
things and I didn't want to hold up releasing this stuff to the world 
any longer.)



The stringification process which your PEP describes as costly only 
happens during compilation of a .py file to .pyc. Since pip-installing 
pre-compiles modules for the user at installation time, there is very 
little runtime penalty for a fully annotated application.


I never intended to suggest that the stringification process /itself/ is 
costly at runtime--and I don't think I did.  Because, as you point out, 
it isn't.  PEP 563 semantics writ large are costly at runtime only when 
annotations are examined, because you have to call eval(), and calling 
eval() is expensive.


If the PEP does say that stringification is itself expensive at runtime, 
please point it out, and I'll fix it.



Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/K6MNCB5Z7PR4VPJUODI4DB4QPCQDFR2M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

Some more questions...

"Binding"," bound" and "unbound" code objects:
Is your use of "binding" terminology in the PEP identical to the
binding of a function to an object instance as a method during object
creation?

Function Annotations:
> When binding an unbound annotation code object, a function will use
> its own __globals__ as the new function's globals.
I'm having trouble parsing this. Does this mean the newly bound
__co_annotations__ function will inherit __globals__ from the function
it's annotating?

Exceptions:
It's quite possible for a __co_annotation__ function call to raise an
exception (e.g. NameError). When accessing __annotations__, if such an
exception is raised during the call to __co_annotations__, what is the
expected behavior?

s/__co_//?:
I'm probably naive, but is there a reason that one could not just store
a callable in __annotations__, and use the descriptor to resolve it to
a dictionary and store it when it is accessed? It would be one less
dunder in the Python data model.


On Mon, 2021-01-11 at 15:46 -0800, Larry Hastings wrote:
> 
> On 1/11/21 3:02 PM, Paul Bryan wrote:
> 
> > PEP 563 does go on to state:
> > 
> > > For code which uses annotations for other purposes, a regular
> > > eval(ann, globals, locals) call is enough to resolve the
> > > annotation.
> > 
> > And I believe this would no longer be true under PEP 649;
> > further, localns (and maybe globalns) parameters in get_type_hints
> > would become meaningless.
> > 
> > [...]
> > And, would get_type_hints continue evaluate [string] annotations in
> > that case?
> I don't work on typing.get_type_hints() so someone else will have to
> answer this question.  All I can say is, with PEP 649 semantics, if
> you set an annotation to a string, you'll get a string back.  And in
> 3.9 (and my out-of-date 3.10) I observe that typing.get_type_hints()
> will eval() string annotations for you, and localns is significant.
> 
> 
> > This passage in PEP 563 appears not true in Python 3.9 with
> > __future__ annotations, emphasis mine:
> > 
> > > The get_type_hints() function automatically resolves the correct
> > > value of globalns for functions and classes. It also
> > > automatically provides the correct localns for classes.
> > 
> > If this passage was true, I believe the issue that resulted in my
> > affixing type hints could have been averted.
> As you've discovered, this is one of the places where PEP 563 seems
> to be out-of-date with respect to its implementation.  I sifted
> through the source code to typing.get_type_hints() twice, and near as
> I can figure out, localns is literally only ever set to None unless
> you override it with the parameter.
> 
> 
> > OK, would string representations of type hints continue be
> > supported under PEP 649 if strings are used as annotations?
> PEP 649 is itself totally agnostic as to what value you use as an
> annotation.  It disallows a couple funky things (yield, walrus
> operator), but beyond that it doesn't care.  Any Python expression or
> value is fine.
> 
> 
> > 
> > > 
> > > > 2. __co_annotations__ scope?
> > > > 
> > > > I'm wondering why __co_annotations__ function could not be
> > > > scoped (within a closure?) such that it can access the values
> > > > where the function, method, class or module are being declared?
> > > > I acknowledge that I'm railing against PEP 563 again, trying to
> > > > reclaim lost ground.
> > > This is addressed in PEP 563, when it rejected the idea of using
> > > "function local state when defining annotations":
> > 
> > I wasn't thinking the function local state of that being annotated
> > (agree, this would be prohibitive), but rather the scope in which
> > the annotated function, class, module, etc. are being defined.
> That's what PEP 563 is referring to.  If you read the thread from
> November 2017 where the idea was discussed, they were talking about
> referring to e.g. "class-level definitions", as in, things defined
> inside class scope.  Which is prohibitive.
> (If I understand you correctly, you thought it was referring to the
> scope inside the function when it runs?  Because I can't imagine how
> that would ever work.  What if the function hasn't been called yet? 
> What if it's been called a thousand times?  What if it's running
> right now in various stages of completeness in five threads and you
> inspect the annotation from a sixth thread?)
> 
> Cheers,
> 
> /arry

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4EB5ZQQ65KLH3LDRLM4N4BQMVWGPKAAW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors


On 12/01/21 6:21 am, Larry Hastings wrote:


Unbound code objects


...The "annotation code object" is then stored *unbound*
as the internal value of ``__co_annotations__``; it is then
bound on demand when the user asks for ``__annotations__``.


This seems like premature optimisation. Function objects are
tiny compared to the code object, which is already a fairly
complicated thing composed of a number of sub-objects.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H7VGQG5YS6LUJGRQH5RHTC7HUUMH64NG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/11/21 3:02 PM, Paul Bryan wrote:

PEP 563 does go on to state:
For code which uses annotations for other purposes, a regular 
eval(ann, globals, locals) call is enough to resolve the annotation.


And I believe this would no longer be true under PEP 649; further, 
localns (and maybe globalns) parameters in get_type_hints would become 
meaningless.


[...]
And, would get_type_hints continue evaluate [string] annotations in 
that case?


I don't work on typing.get_type_hints() so someone else will have to 
answer this question.  All I can say is, with PEP 649 semantics, if you 
set an annotation to a string, you'll get a string back.  And in 3.9 
(and my out-of-date 3.10) I observe that typing.get_type_hints() will 
eval() string annotations for you, and localns is significant.



This passage in PEP 563 appears not true in Python 3.9 with __future__ 
annotations, emphasis mine:
The get_type_hints() function automatically resolves the correct 
value of globalns for functions and classes. *It also automatically 
provides the correct localns for classes.*


If this passage was true, I believe the issue that resulted in my 
affixing type hints could have been averted.


As you've discovered, this is one of the places where PEP 563 seems to 
be out-of-date with respect to its implementation.  I sifted through the 
source code to typing.get_type_hints() twice, and near as I can figure 
out, localns is literally only ever set to None unless you override it 
with the parameter.



OK, would string representations of type hints continue be supported 
under PEP 649 if strings are used as annotations?


PEP 649 is itself totally agnostic as to what value you use as an 
annotation.  It disallows a couple funky things (yield, walrus 
operator), but beyond that it doesn't care.  Any Python expression or 
value is fine.




*2. __co_annotations__ scope?*

I'm wondering why __co_annotations__ function could not be scoped 
(within a closure?) such that it can access the values where the 
function, method, class or module are being declared? I acknowledge 
that I'm railing against PEP 563 again, trying to reclaim lost ground.


This is addressed in PEP 563, when it rejected the idea of using 
"function local state when defining annotations":




I wasn't thinking the function local state of that being annotated 
(agree, this would be prohibitive), but rather the scope in which the 
annotated function, class, module, etc. are being defined.


That's what PEP 563 is referring to.  If you read the thread from 
November 2017 where the idea was discussed, they were talking about 
referring to e.g. "class-level definitions", as in, things defined 
inside class scope.  Which is prohibitive.


(If I understand you correctly, you thought it was referring to the 
scope inside the function when it runs?  Because I can't imagine how 
that would ever work.  What if the function hasn't been called yet?  
What if it's been called a thousand times?  What if it's running right 
now in various stages of completeness in five threads and you inspect 
the annotation from a sixth thread?)



Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UZOL6QXEZI5PXEHYMRNBEXY4W4N5OTFA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-11 Thread Neil Schemenauer

On 2021-01-11, Łukasz Langa wrote:
> The stringification process which your PEP describes as costly
> only happens during compilation of a .py file to .pyc. Since
> pip-installing pre-compiles modules for the user at installation
> time, there is very little runtime penalty for a fully annotated
> application.

It should be possible to make Larry's approach cheap as well.  I
have an old experiment stashed away[1] where I made the code object
for functions to be lazily created.  I.e. when a module is first
loaded, functions are not fully loaded until they are first
executed.  My goal was to reduce startup time.  It didn't show a
significant gain so I didn't pursue it further.

In my experiment, I deferred the unmarshal of the code object.
However, it occurs to me you could go a bit further and have the
function object be mostly skeletal until someone runs it or tries to
inspect it.  The skeleton would really be nothing but a file offset
(or memory offset, if using mmap) into the .pyc file.

Of course this would be some work to implement but then all Python
functions would benefit and likely Python startup time would be
reduced.  I think memory use would be reduced too since typically
you import a lot of modules but only use some of the functions in
them.

I like the idea of Larry's PEP.  I understand why the string-based
annotations was done (I use the __future__ import for my own code).
Using eval() is ugly though and Larry's idea seems like a nice way
to remove the need to call eval().

[1] https://github.com/nascheme/cpython/commits/lazy_codeobject
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/64DP2LFFRA5NO53PN3G46YZ7V3OD3RT2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On Mon, 2021-01-11 at 13:16 -0800, Larry Hastings wrote:
> 
> Thanks for your feedback!  I'll reply piecemeal.
> 
> On 1/11/21 12:32 PM, Paul Bryan wrote:
> 
> > 1. Backwards Compatibility
> > 
> > 
> > > PEP 563 changed the semantics of annotations. When its semantics
> > > are active, annotations must assume they will be evaluated in
> > > module-level scope. They may no longer refer directly to local
> > > variables or class attributes. 
> > 
> > Given get_type_hints can be provided localns argument, this
> > statement is not exactly true.
> PEP 563 states:
> 
> > For code that uses type hints, the typing.get_type_hints(obj,
> > globalns=None, localns=None) function correctly evaluates
> > expressions back from its string form.
> So, if you are passing in a localns argument that isn't None, okay,
> but you're not using them "correctly" according to the language. 
> Also, this usage won't be compatible with static type checkers.
I acknowledge that this will not fly with static type checkers. I also
want to make sure that annotations can continue to serve runtime type
validation.

PEP 563 does go on to state:
> For code which uses annotations for other purposes, a
> regulareval(ann, globals, locals) call is enough to resolve the
> annotation.

And I believe this would no longer be true under PEP 649;
further, localns (and maybe globalns) parameters in get_type_hints
would become meaningless.

This passage in PEP 563 appears not true in Python 3.9 with __future__
annotations, emphasis mine:
> The get_type_hints() function automatically resolves the correct
> value of globalns for functions and classes. It also automatically
> provides the correct localns for classes.

If this passage was true, I believe the issue that resulted in my
affixing type hints could have been averted. 

> > Under PEP 649, when __co_annotations__ is called (presumably by
> > calling get_type_hints), would localns effectively be ignored?
> Yes.  You can experiment with this in Python 3.9--just turn off
> annotation stringizing.  It seems that you can still use strings as
> annotations and typing.get_type_hints() will evaluate them--and I
> assume it'll use localns at that point, just as it does today.
OK, would string representations of type hints continue be supported
under PEP 649 if strings are used as annotations? And, would
get_type_hints continue evaluate the annotations in that case? 

> > 2. __co_annotations__ scope?
> > 
> > I'm wondering why __co_annotations__ function could not be scoped
> > (within a closure?) such that it can access the values where the
> > function, method, class or module are being declared? I acknowledge
> > that I'm railing against PEP 563 again, trying to reclaim lost
> > ground.
> This is addressed in PEP 563, when it rejected the idea of using
> "function local state when defining annotations":

I wasn't thinking the function local state of that being annotated
(agree, this would be prohibitive), but rather the scope in which the
annotated function, class, module, etc. are being defined.

> > This would be prohibitively expensive for highly annotated code as
> > the frames would keep all their objects alive. That includes
> > predominantly objects that won't ever be accessed again.
> > 
> > https://www.python.org/dev/peps/pep-0563/#keeping-the-ability-to-use-function-local-state-when-defining-annotations
> Doing this automatically would indeed incur a sizeable runtime cost,
> for a feature that is already rarely used at runtime.  I guess it
> would be remotely possible? to add this as an optional feature?  But
> this gets crazy quickly--what if it's defined inside a function
> inside another function inside a class inside another function?--and
> the use cases seem few, and TOOWTDI.

I think this exactly the case for closures today.

> I've never understood how closures work in Python, so I'm not the guy
> to ask how possible / hard this would be.  Then again, the
> implementation of closures is obscure enough that I've never been
> able to understand them, so that seems to establish at least a base
> level of difficulty.
> Anyway, one of the concepts my PEP is built on is that "annotations
> are always evaluated at module-level scope". I'd be against changing
> that unless it could be achieved without runtime cost--which AFAIK is
> impossible.
> 
> Cheers,
> 
> /arry

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZOEQFZ4RPZJW7MWO7US5OH23357QQLNV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-11 Thread Łukasz Langa

> On 11 Jan 2021, at 18:21, Larry Hastings wrote:
>
> I've written a new PEP. Please find it below. Happy reading!
>

Interesting! I like the clever lazy-evaluation of the __annotations__ using a
pre-set code object. My only real reservation is that the transition process
will be weird but I don't have much to offer in terms of how to smooth it out.
I have two questions though:

1. What do you anticipate the memory usage will look like for your solution
compared to PEP 563?

To give you an example, EdgeDB is a sizeable application with 100k SLOC of
Python. It's got around 3,500 typed functions, all in all >71% type coverage.
EdgeDB uses stringified annotations exclusively which minimizes runtime memory
usage of annotations because those strings are pretty much all ASCII and many
can be interned. Does it matter? It does, actually. Let's look at 20 most
popular annotations in the codebase and how often they appear:

946 -> s_schema.Schema
362 -> str
298 -> sd.CommandContext
118 -> base.PLBlock
107 -> irast.Set
99 -> CommandContext
95 -> Any
86 -> qlast.DDLOperation
85 -> s_types.Type
71 -> bool
70 -> irast.PathId
67 -> int
54 -> context.Environment
46 -> so.Object
45 -> pgast.Query
42 -> uuid.UUID
38 -> irast.Base
38 -> sn.Name
37 -> pgast.SelectStmt
33 -> context.ReplContext
(this list tapers of with a long tail after)

Turns out most annotations are simple and predictable. (As a side note: we
could make interning even stronger for this purpose if we allowed periods and
square brackets for interning.)

2. What is your expected startup performance of an annotated Python application
using co_annotations?

The stringification process which your PEP describes as costly only happens
during compilation of a .py file to .pyc. Since pip-installing pre-compiles
modules for the user at installation time, there is very little runtime penalty
for a fully annotated application.

Cheers,
Ł___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/AC6YLF4ZKXVXXXDVD357WU6I5B7P2WX2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/11/21 1:16 PM, Larry Hastings wrote:

On 1/11/21 12:32 PM, Paul Bryan wrote:

*1. Backwards Compatibility*

PEP 563  changed the 
semantics of annotations. When its semantics are active, annotations 
must assume they will be evaluated in /module-level/ scope. They may 
no longer refer directly to local variables or class attributes.


Given get_type_hints can be provided localns argument, this statement 
is not exactly true.


PEP 563 states:

For code that uses type hints, the typing.get_type_hints(obj,
globalns=None, localns=None) function correctly evaluates
expressions back from its string form.

So, if you are passing in a localns argument that isn't None, okay, 
but you're not using them "correctly" according to the language.  
Also, this usage won't be compatible with static type checkers.




Whoops!  Let me walk that back a little.  I'd been assuming that PEP 563 
used the terms "annotations" and "type hints" to mean the exact same 
thing.  But careful reading of PEP 484 suggests that they're distinct 
concepts; all "type hints" are annotations, but not all annotations are 
"type hints".


So: if you're using annotations for something besides "type hints", such 
that you have a use for a non-None localns, I guess you have two options 
with my PEP: either


a) use strings for your annotations where you need localns to work for 
you, or


b) skip using annotations syntax and instead write your own custom 
__co_annotations__ function.  Or, you could mix and match, using 
annotations syntax where it was convenient, and overriding only the 
troublesome spots in your custom __co_annotations__ function:


   def foo(a:int=3, b):
    ...

   foo_static_annotations = foo.__annotations__

   def foo_dynamic_closure_annotations():
    annotations = dict(foo_static_annotations)
    annotations['b'] = MyExcitingLocallyDefinedClosureTypeThing
    return annotations

   foo.__co_annotations = foo_dynamic_closure_annotations


Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GAFTOI2RGL7YVL4K7WKGZQHZRFABDP4N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



The control-flow exclusion is for /module//attribute/ or /class 
attribute/ annotations:


   class C:
  if random.random() > 0.5:
    my_attr:int=3
  else:
    my_attr2:float=3.5

Your example doesn't define any module attributes or class attributes 
inside flow control statements, so that code should work fine.  
(Defining functions/methods inside flow control statements isn't a problem.)



Cheers,


//arry/

On 1/11/21 1:39 PM, Jim J. Jewett wrote:

Could you be more explicit about what is banned by the control-flow exclusion?

I'm assuming that:

 class A:
 bar=float
 if FOO:
 bar=int
 def a(x:int, y:int)->int   # function defined with annotations 
inside control flow
 return x+y

 def b(x:bar)  # function annotated with value that depends on control 
flow

is OK, and you're just talking about direct access to (the unfinished class or 
module).__annotations__ but I'm not certain.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VMPMQWCGWR7LRFCEK57VJTQVV6TCQOQN/
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I2B6N34MBCD5PVNVWNCOZAI3FGHM5M4K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-11 Thread Jim J. Jewett

Could you be more explicit about what is banned by the control-flow exclusion?

I'm assuming that:

class A:
bar=float
if FOO:
bar=int
def a(x:int, y:int)->int   # function defined with annotations 
inside control flow
return x+y

def b(x:bar)  # function annotated with value that depends on control 
flow

is OK, and you're just talking about direct access to (the unfinished class or 
module).__annotations__ but I'm not certain.

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VMPMQWCGWR7LRFCEK57VJTQVV6TCQOQN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



Thanks for your feedback!  I'll reply piecemeal.


On 1/11/21 12:32 PM, Paul Bryan wrote:

*1. Backwards Compatibility*

PEP 563  changed the 
semantics of annotations. When its semantics are active, annotations 
must assume they will be evaluated in /module-level/ scope. They may 
no longer refer directly to local variables or class attributes.


Given get_type_hints can be provided localns argument, this statement 
is not exactly true.


PEP 563 states:

   For code that uses type hints, the typing.get_type_hints(obj,
   globalns=None, localns=None) function correctly evaluates
   expressions back from its string form.

So, if you are passing in a localns argument that isn't None, okay, but 
you're not using them "correctly" according to the language.  Also, this 
usage won't be compatible with static type checkers.



Under PEP 649, when __co_annotations__ is called (presumably by 
calling get_type_hints), would localns effectively be ignored?


Yes.  You can experiment with this in Python 3.9--just turn off 
annotation stringizing.  It seems that you can still use strings as 
annotations and typing.get_type_hints() will evaluate them--and I assume 
it'll use localns at that point, just as it does today.




*2. __co_annotations__ scope?*

I'm wondering why __co_annotations__ function could not be scoped 
(within a closure?) such that it can access the values where the 
function, method, class or module are being declared? I acknowledge 
that I'm railing against PEP 563 again, trying to reclaim lost ground.


This is addressed in PEP 563, when it rejected the idea of using 
"function local state when defining annotations":


   This would be prohibitively expensive for highly annotated code as
   the frames would keep all their objects alive. That includes
   predominantly objects that won't ever be accessed again.

   
https://www.python.org/dev/peps/pep-0563/#keeping-the-ability-to-use-function-local-state-when-defining-annotations

Doing this automatically would indeed incur a sizeable runtime cost, for 
a feature that is already rarely used at runtime.  I guess it would be 
remotely possible? to add this as an optional feature?  But this gets 
crazy quickly--what if it's defined inside a function inside another 
function inside a class inside another function?--and the use cases seem 
few, and TOOWTDI.


I've never understood how closures work in Python, so I'm not the guy to 
ask how possible / hard this would be.  Then again, the implementation 
of closures is obscure enough that I've never been able to understand 
them, so that seems to establish at least a base level of difficulty.


Anyway, one of the concepts my PEP is built on is that "annotations are 
always evaluated at module-level scope". I'd be against changing that 
unless it could be achieved without runtime cost--which AFAIK is impossible.



Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2Y5MYSPHKCYDVEQN5APBFGBE6ZZDTKHH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

I'm very much in favour of the this concept. A few points come to mind
right away:

1. Backwards Compatibility

> PEP 563 changed the semantics of annotations. When its semantics are
> active, annotations must assume they will be evaluated inmodule-level
> scope. They may no longer refer directly to local variables or class
> attributes. 

Given get_type_hints can be provided localns argument, this statement
is not exactly true.

Using localns is how I currently address scoping issues when affixing
type hints. Maybe it could be argued that I'm abusing this feature, but
then I would ask what the intent of localns is if not to provide
additional (missing) scope during type hint evaluation?

Under PEP 649, when __co_annotations__ is called (presumably by calling
get_type_hints), would localns effectively be ignored?

2. __co_annotations__ scope?

I'm wondering why __co_annotations__ function could not be scoped
(within a closure?) such that it can access the values where the
function, method, class or module are being declared? I acknowledge
that I'm railing against PEP 563 again, trying to reclaim lost ground. 

On Mon, 2021-01-11 at 10:27 -0800, Larry Hastings wrote:
> 
> On 1/11/21 10:16 AM, Chris Angelico wrote:
> 
> > Number allocation is pretty informal. Go ahead and grab PEP 649;
> It's now checked in as PEP 649, with a modern header, modern
> copyright, and I went ahead and grabbed the formatting stanza from
> the end too.
> Welcome to the world, baby 649!
> 
> /arry
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/OKOQEAEPKYDX6AVEFD7DDPBKOHGXB4GB/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5GLDASRAWNEJXYG3Y7NDUBPPNR6TTIGX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-11 Thread Eric V. Smith


On 1/11/2021 1:10 PM, Larry Hastings wrote:



Certainly.  I'm just another victim in the copy-and-paste wars.

I actually have write access to the PEPs repo (I'm a former release 
manager) so I'd be happy to check it in myself once it gets a number, 
however that happens.  Before I do so I'll study PEP 12 as if it was 
gonna be on tomorrow's midterms.


It gets a number by you renaming your file to the next available number 
and checking it in. There's a race condition, so act fast!


Eric



//arry/

On 1/11/21 9:59 AM, Chris Angelico wrote:

On Tue, Jan 12, 2021 at 4:22 AM Larry Hastings  wrote:

I've written a new PEP.  Please find it below.  Happy reading!


Can this get added to the PEPs repo and assigned a number and such?

BTW, the currently preferred wording for the copyright blurb is
slightly different. If you're the sole author of this text, can you
please consider the license terms shown in PEP 12?

ChrisA
PEP editor - if you need a hand, I'm here to help


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AHFBMKWOP6AOSDKFIFUFGGIO56I3UEPX/
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KMD6UQIHAPE7OL66PX4VUKTORY7RKN4M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/11/21 10:29 AM, Barry Warsaw wrote:

Given that PEP 563 is now the default in unreleased Python 3.10, does it make 
sense to introduce yet another __future__ import?  What would happen if you 
just piggybacked your idea onto that change?


Part of my proposal is to deprecate PEP 563's semantics.  If -> PEP 649 
<- was accepted, we'd undo making PEP 563 the default behavior in 3.10; 
the behavior would instead remain gated behind the "from __future__ 
import annotations".  It'd then go through a standard deprecation cycle 
(which is, what, three versions?) before finally being removed.


(If you look at the revision history of my repo, you'll see that my 
first checkin was to reverse Batuhan's checkin from October 6, restoring 
the "from __future__" gate for annotations.  Sorry, Batuhan!)


Frankly I'd be astonished if -> PEP 649 <- received such unanimous 
acceptance that it become the new default Python semantics without a 
"from __future__" introductory period.  You'd need a bigger brain than I 
have to think through all the ramifications of that sort of radical 
decision!  But if the steering committee requested it, I don't expect 
I'd put a fight.



Cheers,


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/D6BZKYWOTC5NQOVHEVLM53CINJWWPMBQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-11 Thread Barry Warsaw

Thanks for this detailed PEP and analysis, and for the interesting discussion 
in your separate thread.  I’m glad to see this work that we chatted about all 
that time before has coalesced into a PEP.

FYI: For those with write access to the PEPs repo, PEP number assignments are 
self-serve.  Just grab the next available one and manage any push race 
conditions accordingly.

Question:

> On Jan 11, 2021, at 09:21, Larry Hastings  wrote:
> 
> from __future__ import co_annotations
> -
> 
> In the prototype, the semantics presented in this PEP are gated with:
> 
> from __future__ import co_annotations

Given that PEP 563 is now the default in unreleased Python 3.10, does it make 
sense to introduce yet another __future__ import?  What would happen if you 
just piggybacked your idea onto that change?

-Barry

signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FPXDS2D5RSSPDWWMIXJWNWUDAUI4QN3K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors



On 1/11/21 10:16 AM, Chris Angelico wrote:

Number allocation is pretty informal. Go ahead and grab PEP 649;


It's now checked in as PEP 649, with a modern header, modern copyright, 
and I went ahead and grabbed the formatting stanza from the end too.


Welcome to the world, baby 649!


//arry/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OKOQEAEPKYDX6AVEFD7DDPBKOHGXB4GB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-11 Thread Chris Angelico

On Tue, Jan 12, 2021 at 5:10 AM Larry Hastings  wrote:
>
>
> Certainly.  I'm just another victim in the copy-and-paste wars.
>

Ah yes, the Battle of the Clipboard. Iconic, epic, such a glorious
engagement! But the casualties were steep. Fortunately we can rebuild.

> I actually have write access to the PEPs repo (I'm a former release manager) 
> so I'd be happy to check it in myself once it gets a number, however that 
> happens.  Before I do so I'll study PEP 12 as if it was gonna be on 
> tomorrow's midterms.
>

Number allocation is pretty informal. Go ahead and grab PEP 649; in
the unlikely event that someone else pushes a commit creating that PEP
before you get to it, grab PEP 650 instead :)

I'm happy to help out if you need me, but it sounds like you got this!

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/F65N4HBJZFTZL2VNFV77ZTJN4RVPCHWU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors