Re: [Python-Dev] Question about PEP 484

[Rebecca Chen via Python-Dev]

Hi Sebastian,

Both Teddy (cc'd) and I would like to volunteer to help. We're excited
about the prospect of an informational pyi PEP.

Best,
Rebecca

On Tue, Jul 17, 2018 at 10:42 AM 'Adam Cataldo' via pytype <
pyt...@googlegroups.com> wrote:

> Hi Sebastian,
>
> Of course, we'd be happy to work with you on this!  We just need to figure
> out which of us will drive this on our end (most likely Rebecca or Teddy).
> I'll huddle with the team and get back to you with an answer on who later
> today.
>
>
>
> On Tue, Jul 17, 2018 at 9:58 AM Sebastian Rittau 
> wrote:
>
>> On 17.07.2018 17:05, Guido van Rossum wrote:
>> > This is a good point. I presume specifying this unambiguously would be
>> > a huge amount of work, and it would mostly codify mypy's current
>> > behavior. I don't think that's within the scope of PEP 484, but it
>> > could well be done as a separate PEP (perhaps an informational one?).
>> > I hope you understand that I am not volunteering.
>> An informational PEP sounds about right to me. Such a PEP could also
>> include style recommendations like those from typeshed's CONTRIBUTING
>> file (https://github.com/python/typeshed/blob/master/CONTRIBUTING.md).
>>
>> I guess I just volunteered to help with such a PEP, although I feel that
>> someone from mypy's core team should take the lead on that. And if I
>> understood this thread correctly, the pytype team is also willing to
>> help out?
>>
>>   - Sebastian
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/acataldo%40google.com
>>
> --
> You received this message because you are subscribed to the Google Groups
> "pytype" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pytype+unsubscr...@googlegroups.com.
> To post to this group, send email to pyt...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pytype/CAKTwdc4Teidod9SnUy6Dp7BBBU21WTu%3DMQ7iPa%2BMvgLDshHcNg%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Adam Cataldo via Python-Dev]

Great Jelle! We look forward to working with you to.

On Tue, Jul 17, 2018 at 10:41 AM Jelle Zijlstra 
wrote:

>
>
> 2018-07-17 9:55 GMT-07:00 Sebastian Rittau :
>
>> On 17.07.2018 17:05, Guido van Rossum wrote:
>>
>>> This is a good point. I presume specifying this unambiguously would be a
>>> huge amount of work, and it would mostly codify mypy's current behavior. I
>>> don't think that's within the scope of PEP 484, but it could well be done
>>> as a separate PEP (perhaps an informational one?). I hope you understand
>>> that I am not volunteering.
>>>
>> An informational PEP sounds about right to me. Such a PEP could also
>> include style recommendations like those from typeshed's CONTRIBUTING file (
>> https://github.com/python/typeshed/blob/master/CONTRIBUTING.md).
>>
>> I guess I just volunteered to help with such a PEP, although I feel that
>> someone from mypy's core team should take the lead on that. And if I
>> understood this thread correctly, the pytype team is also willing to help
>> out?
>
> I can also help out.
>
>>
>>
>>  - Sebastian
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/jelle.zijlstra%40gmail.com
>>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/acataldo%40google.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Adam Cataldo via Python-Dev]

Hi Sebastian,

Of course, we'd be happy to work with you on this!  We just need to figure
out which of us will drive this on our end (most likely Rebecca or Teddy).
I'll huddle with the team and get back to you with an answer on who later
today.



On Tue, Jul 17, 2018 at 9:58 AM Sebastian Rittau  wrote:

> On 17.07.2018 17:05, Guido van Rossum wrote:
> > This is a good point. I presume specifying this unambiguously would be
> > a huge amount of work, and it would mostly codify mypy's current
> > behavior. I don't think that's within the scope of PEP 484, but it
> > could well be done as a separate PEP (perhaps an informational one?).
> > I hope you understand that I am not volunteering.
> An informational PEP sounds about right to me. Such a PEP could also
> include style recommendations like those from typeshed's CONTRIBUTING
> file (https://github.com/python/typeshed/blob/master/CONTRIBUTING.md).
>
> I guess I just volunteered to help with such a PEP, although I feel that
> someone from mypy's core team should take the lead on that. And if I
> understood this thread correctly, the pytype team is also willing to
> help out?
>
>   - Sebastian
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/acataldo%40google.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Jelle Zijlstra]

2018-07-17 9:55 GMT-07:00 Sebastian Rittau :

> On 17.07.2018 17:05, Guido van Rossum wrote:
>
>> This is a good point. I presume specifying this unambiguously would be a
>> huge amount of work, and it would mostly codify mypy's current behavior. I
>> don't think that's within the scope of PEP 484, but it could well be done
>> as a separate PEP (perhaps an informational one?). I hope you understand
>> that I am not volunteering.
>>
> An informational PEP sounds about right to me. Such a PEP could also
> include style recommendations like those from typeshed's CONTRIBUTING file (
> https://github.com/python/typeshed/blob/master/CONTRIBUTING.md).
>
> I guess I just volunteered to help with such a PEP, although I feel that
> someone from mypy's core team should take the lead on that. And if I
> understood this thread correctly, the pytype team is also willing to help
> out?

I can also help out.

>
>
>  - Sebastian
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/jelle.
> zijlstra%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Sebastian Rittau]

On 17.07.2018 17:05, Guido van Rossum wrote:
This is a good point. I presume specifying this unambiguously would be 
a huge amount of work, and it would mostly codify mypy's current 
behavior. I don't think that's within the scope of PEP 484, but it 
could well be done as a separate PEP (perhaps an informational one?). 
I hope you understand that I am not volunteering.
An informational PEP sounds about right to me. Such a PEP could also 
include style recommendations like those from typeshed's CONTRIBUTING 
file (https://github.com/python/typeshed/blob/master/CONTRIBUTING.md).


I guess I just volunteered to help with such a PEP, although I feel that 
someone from mypy's core team should take the lead on that. And if I 
understood this thread correctly, the pytype team is also willing to 
help out?


 - Sebastian

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Guido van Rossum]

This is a good point. I presume specifying this unambiguously would be a
huge amount of work, and it would mostly codify mypy's current behavior. I
don't think that's within the scope of PEP 484, but it could well be done
as a separate PEP (perhaps an informational one?). I hope you understand
that I am not volunteering.

On Tue, Jul 17, 2018 at 1:51 AM, Sebastian Rittau 
wrote:

> On 16.07.2018 19:21, Adam Cataldo via Python-Dev wrote:
>
>
>
>
> * - One thing we care about in particular, given the implementation of
> pytype, is the detailed definition of what goes in a .pyi file. Do folks
> think this makes sense to include as part of PEP 484, or would this be
> better in a separate PEP? We’d love to get your thoughts. *
>
> It would be useful to define - on a semantic, not syntactic level - what
> constructs a type checker is expected to understand. Not only for
> implementers, but more importantly for stub authors. Sometimes it's hard to
> judge, what constructs type checkers will understand. And while by now I
> think I personally have a solid understanding of what mypy understands, I
> have no idea whether that also applies to pytype, PyCharm, or other type
> checkers.
>
> For example, in one of my first pull requests for typeshed, I tried to use
> the following construct, expecting type checkers to understand it:
>
> class Foo:
> def bar(self) -> None:
> raise NotImplementedError()
>
> It seems they don't, but mypy understands:
>
> class Foo:
> @abstractmethod
> def bar(self) -> None: ...
>
> But do I need to import abstractmethod? Would @abc.abstractmethod also
> work? Can I have an assignment "_am = abc.abstractmethod" and then @_am
> would work? Can I alias functions by using assignments in stubs or should I
> use a second function definition? How complex can Python version checks be?
> There are many more of those questions.
>
> If these expectations would be documents, implementers of type checkers
> can still decide not to support certain constructs (or not support them
> yet), or even to support more constructs. But at least such a deviation
> could be documented, so users know what to expect. On the other hand, stub
> authors will know what constructs will likely not work and should be
> avoided.
>
>  - Sebastian
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Sebastian Rittau]

On 16.07.2018 19:21, Adam Cataldo via Python-Dev wrote:

*

 *

One thing we care about in particular, given the implementation of
pytype, is the detailed definition of what goes in a .pyi file. Do
folks think this makes sense to include as part of PEP 484, or
would this be better in a separate PEP? We’d love to get your
thoughts.

*
It would be useful to define - on a semantic, not syntactic level - what 
constructs a type checker is expected to understand. Not only for 
implementers, but more importantly for stub authors. Sometimes it's hard 
to judge, what constructs type checkers will understand. And while by 
now I think I personally have a solid understanding of what mypy 
understands, I have no idea whether that also applies to pytype, 
PyCharm, or other type checkers.


For example, in one of my first pull requests for typeshed, I tried to 
use the following construct, expecting type checkers to understand it:


    class Foo:
    def bar(self) -> None:
    raise NotImplementedError()

It seems they don't, but mypy understands:

    class Foo:
    @abstractmethod
    def bar(self) -> None: ...

But do I need to import abstractmethod? Would @abc.abstractmethod also 
work? Can I have an assignment "_am = abc.abstractmethod" and then @_am 
would work? Can I alias functions by using assignments in stubs or 
should I use a second function definition? How complex can Python 
version checks be? There are many more of those questions.


If these expectations would be documents, implementers of type checkers 
can still decide not to support certain constructs (or not support them 
yet), or even to support more constructs. But at least such a deviation 
could be documented, so users know what to expect. On the other hand, 
stub authors will know what constructs will likely not work and should 
be avoided.


 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Guido van Rossum]

Thanks, Rebecca! So the status of PEP 484 is that we don't want to add new
features to it -- those require their own PEP. But we do accept PRs for
clarifications, assuming the clarifications are about issues where the
intention is clear but the text is not (usually because we thought there
was only one possible interpretation). If PEP 484 had intended to only
allow a specific subset of Python *syntax* we surely would have specified
that subset -- so I think the intention is clear (enough).

Unfortunately a lot of PEP 484 is written in the form of examples, and in
many cases it would be possible to quibble about edge cases not covered by
the examples. (An example that just came up: is `#type: foo` a valid type
comment? What about `#   type:foo`? Or `# < type: foo`? The intention was
`#\s*type:\s*` but all the PEP examples use exactly one space.)


On Mon, Jul 16, 2018 at 4:28 PM, Rebecca Chen  wrote:

> Hi Guido and all,
>
> Let me try to clarify our (pytype team's) intentions a bit. Our
> overall goal is to see PEP 484 finalized, in the interest of having a
> definitive type-checking reference to point to and work off of. We're
> willing and eager to help with this, if we get some guidance on what
> the authors are still trying to finalize and how we can be useful. As
> a conversation starter, we offered two issues in which pytype has some
> personal stake.
>
> Best,
> Rebecca
>
> P.S. Yes, Adam manages Python at Google.
>
> On Mon, Jul 16, 2018 at 3:15 PM Guido van Rossum  wrote:
> >
> > On Mon, Jul 16, 2018 at 2:56 PM, Gregory P. Smith 
> wrote:
> >>
> >>
> >> On Mon, Jul 16, 2018 at 1:44 PM Guido van Rossum 
> wrote:
> >>>
> >>> As one of the authors of PEP 484, *I* never thought there was an
> ambiguity here. The intention was for stub files to conform to the same
> grammar as regular .py files, but with a different interpretation.
> >>>
> >>> > "Have the same syntax as regular Python modules" and "are files
> containing type hints" are at odds with each other.
> >>>
> >>> That depends. *same syntax as regular Python* is normative while
> *containing type hints* is an informal description of intent.
> >>>
> >>> I happen to be somewhat familiar with the situation that lead to this
> question -- pytype has its own parser for stub files that cannot parse all
> Python constructs. But claiming that PEP 484 is ambiguous feels wrong, and
> if we really need to clarify it the only way to go is to make "same syntax
> as regular Python" more clearly normative. Type checkers should of course
> feel free ignore everything they don't care about.
> >>
> >>
> >> I feel like the "same syntax as regular Python" is too broad a
> statement.  That effectively requires a version specific Python interpreter
> to execute the files.  With at least four different Python static analyzers
> in existence today, keeping the behavior of all of them consistent is
> important.  Otherwise pyi files will be (are being) created that are
> analyzer specific and break other type checkers when distributed.
> >
> >
> > it doesn't require an interpreter, just a parser. Is saying it should be
> syntactically valid Python 3.6 (though emphatically not executable!) still
> too much?
> >
> >>
> >> ex: We're encountering pyi files with conditional logic in them.  I
> believe we've encountered pyi files with del statements in them?  Both of
> these are a slippery slope towards being turing complete in something that
> isn't supposed to be code.  I don't like this.  Interface declarations
> should not contain logic.  If we allow conditions, we need to explicitly
> define what we do allow in the PEP.  (if+else and del?  what inputs are
> allowed for the expression in if statements?).  Otherwise at some point
> someone is going to create a pyi file containing loops, function calls, and
> generator expressions and expect it to _do_ something.  The horror!  Lets
> avoid that.
> >
> >
> > Syntactically, conditional logic *is* part of the spec because of
> `sys.version_info` and `sys.platform` checks. These are mentioned in PEP
> 484 (though IIRC they were added at some point).
> >
> > Feel free to ignore `del` statements, but they are valid syntax.
> >
> >>
> >> PEP-484 does contain the text, "This also reinforces the notion that no
> runtime behavior should be expected of stub files." But reinforcing a
> notion is not what I read as a concrete statement.
> >>
> >> I'd rather see that say something like, "There must not be any runtime
> behavior from a stub file. They will be parsed for information, not
> executed."  Wordsmith that all you want, I'm not pedantic enough. :)
> >
> >
> > I don't see that as any more precise as what the PEP says. :-)
> >
> > I'd be happy to claim that if the ast module can't parse a .pyi file,
> it's invalid, otherwise it's valid, except that `# type:` comments are
> significant and the ast module doesn't preserve those.
> >
> >>
> >> I expect someone pedantic to happily point out that a def or class or
> assignment to ... 

Re: [Python-Dev] Question about PEP 484

[Rebecca Chen via Python-Dev]

Hi Guido and all,

Let me try to clarify our (pytype team's) intentions a bit. Our
overall goal is to see PEP 484 finalized, in the interest of having a
definitive type-checking reference to point to and work off of. We're
willing and eager to help with this, if we get some guidance on what
the authors are still trying to finalize and how we can be useful. As
a conversation starter, we offered two issues in which pytype has some
personal stake.

Best,
Rebecca

P.S. Yes, Adam manages Python at Google.

On Mon, Jul 16, 2018 at 3:15 PM Guido van Rossum  wrote:
>
> On Mon, Jul 16, 2018 at 2:56 PM, Gregory P. Smith  wrote:
>>
>>
>> On Mon, Jul 16, 2018 at 1:44 PM Guido van Rossum  wrote:
>>>
>>> As one of the authors of PEP 484, *I* never thought there was an ambiguity 
>>> here. The intention was for stub files to conform to the same grammar as 
>>> regular .py files, but with a different interpretation.
>>>
>>> > "Have the same syntax as regular Python modules" and "are files 
>>> > containing type hints" are at odds with each other.
>>>
>>> That depends. *same syntax as regular Python* is normative while 
>>> *containing type hints* is an informal description of intent.
>>>
>>> I happen to be somewhat familiar with the situation that lead to this 
>>> question -- pytype has its own parser for stub files that cannot parse all 
>>> Python constructs. But claiming that PEP 484 is ambiguous feels wrong, and 
>>> if we really need to clarify it the only way to go is to make "same syntax 
>>> as regular Python" more clearly normative. Type checkers should of course 
>>> feel free ignore everything they don't care about.
>>
>>
>> I feel like the "same syntax as regular Python" is too broad a statement.  
>> That effectively requires a version specific Python interpreter to execute 
>> the files.  With at least four different Python static analyzers in 
>> existence today, keeping the behavior of all of them consistent is 
>> important.  Otherwise pyi files will be (are being) created that are 
>> analyzer specific and break other type checkers when distributed.
>
>
> it doesn't require an interpreter, just a parser. Is saying it should be 
> syntactically valid Python 3.6 (though emphatically not executable!) still 
> too much?
>
>>
>> ex: We're encountering pyi files with conditional logic in them.  I believe 
>> we've encountered pyi files with del statements in them?  Both of these are 
>> a slippery slope towards being turing complete in something that isn't 
>> supposed to be code.  I don't like this.  Interface declarations should not 
>> contain logic.  If we allow conditions, we need to explicitly define what we 
>> do allow in the PEP.  (if+else and del?  what inputs are allowed for the 
>> expression in if statements?).  Otherwise at some point someone is going to 
>> create a pyi file containing loops, function calls, and generator 
>> expressions and expect it to _do_ something.  The horror!  Lets avoid that.
>
>
> Syntactically, conditional logic *is* part of the spec because of 
> `sys.version_info` and `sys.platform` checks. These are mentioned in PEP 484 
> (though IIRC they were added at some point).
>
> Feel free to ignore `del` statements, but they are valid syntax.
>
>>
>> PEP-484 does contain the text, "This also reinforces the notion that no 
>> runtime behavior should be expected of stub files." But reinforcing a notion 
>> is not what I read as a concrete statement.
>>
>> I'd rather see that say something like, "There must not be any runtime 
>> behavior from a stub file. They will be parsed for information, not 
>> executed."  Wordsmith that all you want, I'm not pedantic enough. :)
>
>
> I don't see that as any more precise as what the PEP says. :-)
>
> I'd be happy to claim that if the ast module can't parse a .pyi file, it's 
> invalid, otherwise it's valid, except that `# type:` comments are significant 
> and the ast module doesn't preserve those.
>
>>
>> I expect someone pedantic to happily point out that a def or class or 
>> assignment to ... with an annotation is runtime behavior... technically 
>> correct, but that isn't how people like me think of them in this context.
>
>
> And mypy agrees. :-)
>
>>
>> We use a Pythonic syntax for stubs to be consistent with the language, that 
>> doesn't mean they are code.
>
>
> Define "code". To some people HTML is code.
>
>>
>> I wrote more than I thought I would here, I'll stop now. :)
>
>
> It would be nice if the pytype team could articulate the problems they're 
> trying to solve, rather than offering to "help finalize PEP 484". My guess is 
> that their parser for .pyi files only accepts a subset of Python and they're 
> (you're? do you report to Adam?) reluctant to sink more time in that parser 
> -- but I don't think that's my problem. :-)
>
> My intention for stubs was exactly what I've been saying in this thread: they 
> must be syntactically valid Python, they are not meant to be executed, and 
> type checkers are free to ignore 

Re: [Python-Dev] Question about PEP 484

[Guido van Rossum]

On Mon, Jul 16, 2018 at 2:56 PM, Gregory P. Smith  wrote:

>
> On Mon, Jul 16, 2018 at 1:44 PM Guido van Rossum  wrote:
>
>> As one of the authors of PEP 484, *I* never thought there was an
>> ambiguity here. The intention was for stub files to conform to the same
>> grammar as regular .py files, but with a different interpretation.
>>
>> > "Have the same syntax as regular Python modules" and "are files
>> containing type hints" are at odds with each other.
>>
>> That depends. *same syntax as regular Python* is normative while
>> *containing type hints* is an informal description of intent.
>>
>> I happen to be somewhat familiar with the situation that lead to this
>> question -- pytype has its own parser for stub files that cannot parse all
>> Python constructs. But claiming that PEP 484 is ambiguous feels wrong, and
>> if we really need to clarify it the only way to go is to make "same syntax
>> as regular Python" more clearly normative. Type checkers should of course
>> feel free ignore everything they don't care about.
>>
>
> I feel like the "same syntax as regular Python" is too broad a statement.
> That effectively requires a version specific Python interpreter to execute
> the files.  With at least four different Python static analyzers in
> existence today, keeping the behavior of all of them consistent is
> important.  Otherwise pyi files will be (are being) created that are
> analyzer specific and break other type checkers when distributed.
>

it doesn't require an interpreter, just a parser. Is saying it should be
syntactically valid Python 3.6 (though emphatically not executable!) still
too much?


> ex: We're encountering pyi files with conditional logic in them.  I
> believe we've encountered pyi files with del statements in them?  Both of
> these are a slippery slope towards being turing complete in something that
> isn't supposed to be code.  I don't like this.  Interface declarations
> should not contain logic.  If we allow conditions, we need to explicitly
> define what we do allow in the PEP.  (if+else and del?  what inputs are
> allowed for the expression in if statements?).  Otherwise at some point
> someone is going to create a pyi file containing loops, function calls, and
> generator expressions and expect it to _do_ something.  The horror!  Lets
> avoid that.
>

Syntactically, conditional logic *is* part of the spec because of
`sys.version_info` and `sys.platform` checks. These are mentioned in PEP
484 (though IIRC they were added at some point).

Feel free to ignore `del` statements, but they are valid syntax.


> PEP-484 does contain the text, "This also reinforces the notion that no
> runtime behavior should be expected of stub files." But reinforcing a
> notion is not what I read as a concrete statement.
>
> I'd rather see that say something like, "There must not be any runtime
> behavior from a stub file. They will be parsed for information, not
> executed."  Wordsmith that all you want, I'm not pedantic enough. :)
>

I don't see that as any more precise as what the PEP says. :-)

I'd be happy to claim that if the ast module can't parse a .pyi file, it's
invalid, otherwise it's valid, except that `# type:` comments are
significant and the ast module doesn't preserve those.


> I expect someone pedantic to happily point out that a def or class or
> assignment to ... with an annotation is runtime behavior... technically
> correct, but that isn't how people like me think of them in this context.
>

And mypy agrees. :-)


> We use a Pythonic syntax for stubs to be consistent with the language,
> that doesn't mean they are code.
>

Define "code". To some people HTML is code.


> I wrote more than I thought I would here, I'll stop now. :)
>

It would be nice if the pytype team could articulate the problems they're
trying to solve, rather than offering to "help finalize PEP 484". My guess
is that their parser for .pyi files only accepts a subset of Python and
they're (you're? do you report to Adam?) reluctant to sink more time in
that parser -- but I don't think that's my problem. :-)

My intention for stubs was exactly what I've been saying in this thread:
they must be syntactically valid Python, they are not meant to be executed,
and type checkers are free to ignore things they don't need. If that's not
sufficiently clear in the way the PEP is currently written, I welcome a PR
to the peps repo. If the pytype team wants a different interpretation, the
bar is much higher.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Gregory P. Smith]

On Mon, Jul 16, 2018 at 1:44 PM Guido van Rossum  wrote:

> As one of the authors of PEP 484, *I* never thought there was an ambiguity
> here. The intention was for stub files to conform to the same grammar as
> regular .py files, but with a different interpretation.
>
> > "Have the same syntax as regular Python modules" and "are files
> containing type hints" are at odds with each other.
>
> That depends. *same syntax as regular Python* is normative while
> *containing type hints* is an informal description of intent.
>
> I happen to be somewhat familiar with the situation that lead to this
> question -- pytype has its own parser for stub files that cannot parse all
> Python constructs. But claiming that PEP 484 is ambiguous feels wrong, and
> if we really need to clarify it the only way to go is to make "same syntax
> as regular Python" more clearly normative. Type checkers should of course
> feel free ignore everything they don't care about.
>

I feel like the "same syntax as regular Python" is too broad a statement.
That effectively requires a version specific Python interpreter to execute
the files.  With at least four different Python static analyzers in
existence today, keeping the behavior of all of them consistent is
important.  Otherwise pyi files will be (are being) created that are
analyzer specific and break other type checkers when distributed.

ex: We're encountering pyi files with conditional logic in them.  I believe
we've encountered pyi files with del statements in them?  Both of these are
a slippery slope towards being turing complete in something that isn't
supposed to be code.  I don't like this.  Interface declarations should not
contain logic.  If we allow conditions, we need to explicitly define what
we do allow in the PEP.  (if+else and del?  what inputs are allowed for the
expression in if statements?).  Otherwise at some point someone is going to
create a pyi file containing loops, function calls, and generator
expressions and expect it to _do_ something.  The horror!  Lets avoid that.

PEP-484 does contain the text, "This also reinforces the notion that no
runtime behavior should be expected of stub files." But reinforcing a
notion is not what I read as a concrete statement.

I'd rather see that say something like, "There must not be any runtime
behavior from a stub file. They will be parsed for information, not
executed."  Wordsmith that all you want, I'm not pedantic enough. :)
I expect someone pedantic to happily point out that a def or class or
assignment to ... with an annotation is runtime behavior... technically
correct, but that isn't how people like me think of them in this context.

We use a Pythonic syntax for stubs to be consistent with the language, that
doesn't mean they are code.

I wrote more than I thought I would here, I'll stop now. :)

-gps


>
> Regarding the unicode issue, that is indeed unfortunate, and there's a
> long but inconclusive discussion at
> https://github.com/python/typing/issues/208. (If you want a longer
> discussion here please start a new subject.)
>
> On Mon, Jul 16, 2018 at 1:02 PM, Adam Cataldo via Python-Dev <
> python-dev@python.org> wrote:
>
>> Thanks Brett and Teddy,
>>
>> Just so it doesn't get lost in the shuffle as folks dive into details,
>> I'll re-ask my earlier question about stub files. Assuming there is
>> consensus that there is ambiguity to resolve in the current definition, is
>> updating the section on stub files the preferred option? The only
>> alternative I can think of is to pull this out into a separate PEP. I
>> frankly have no opinion on what the best way to capture this is. We're
>> happy to help out either way.
>>
>> On Mon, Jul 16, 2018 at 12:48 PM Teddy Sudol  wrote:
>>
>>> Hi, my name is Teddy Sudol. I work with Adam and Rebecca on pytype.
>>>
>>> The explanation of stub files is unclear. The section you linked starts
>>> with, "Stub files are files containing type hints that are only for use by
>>> the type checker, not at runtime." According to
>>> https://www.python.org/dev/peps/pep-0484/#acceptable-type-hints, type
>>> hints may be classes, abstract base classes, types defined in the `types`
>>> and `typing` modules, type variables, type aliases and None. Further in the
>>> section you linked, PEP 484 also states, "Stub files have the same syntax
>>> as regular Python modules," and, "no runtime behavior should be expected of
>>> stub files."
>>>
>>> "Have the same syntax as regular Python modules" and "are files
>>> containing type hints" are at odds with each other. This has led to
>>> compatibility issues between Mypy and pytype. For example, `b''` is not a
>>> valid type annotation, but until a month ago, `codecs.pyi` in typeshed used
>>> exactly that:
>>> https://github.com/python/typeshed/commit/6bbf3d89eb9b6c3fd5b0c0f632b2ad9258cecf15#diff-5f6f48c425bc0c283784cf5277880c0cL95.
>>> If statements can be useful for things like version checks, but on the
>>> other hand, pyi files aren't supposed to 

Re: [Python-Dev] Question about PEP 484

[Jelle Zijlstra]

2018-07-16 10:21 GMT-07:00 Adam Cataldo via Python-Dev <
python-dev@python.org>:

>
>
>
>
>
> *Hi Folks,Cc: Rebecca, pytypeThis is Adam Cataldo; I’m the engineering
> manager for the Python team at Google. Rebecca Chen, our lead pytype
>  contributor, and I are interested in
> helping finalize PEP 484 if possible. To that end, we wanted to find out
> what technical issues the PEP 484 authors feel they still need to finalize.
> We also wanted to know how we can help.We have a large Python code base at
> Google, and may be able to use this to help resolve current incomplete
> definitions, by collecting data on how types are used. We also have a
> couple ambiguities that we’d love to get closure on: - One thing we care
> about in particular, given the implementation of pytype, is the detailed
> definition of what goes in a .pyi file. Do folks think this makes sense to
> include as part of PEP 484, or would this be better in a separate PEP? We’d
> love to get your thoughts.*
>

I would be happy to contribute to this if that would be useful for type
checkers like pytype, although like Guido I don't think the current text of
the PEP is ambiguous. Typeshed already has a set of lint rules that limit
what can be put in stub files, but it could be useful to communicate the
exact set of allowed constructs.


>
> * - The relationship between unicode and typing.Text in Python 2 has been
> a recurring source of confusion for our users. Especially since we
> contributed  to the current state
> of affairs, we’d like to figure out how to establish clarity here.*
>
There has already been a long discussion on the typing issue tracker that
resulted in no consensus for any change to the current recommendation.
Perhaps there is something we can do, but it's not clear to me what that
would be.

>
>
> *Thoughts?Thanks,Adam*
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/jelle.
> zijlstra%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Guido van Rossum]

As one of the authors of PEP 484, *I* never thought there was an ambiguity
here. The intention was for stub files to conform to the same grammar as
regular .py files, but with a different interpretation.

> "Have the same syntax as regular Python modules" and "are files
containing type hints" are at odds with each other.

That depends. *same syntax as regular Python* is normative while
*containing type hints* is an informal description of intent.

I happen to be somewhat familiar with the situation that lead to this
question -- pytype has its own parser for stub files that cannot parse all
Python constructs. But claiming that PEP 484 is ambiguous feels wrong, and
if we really need to clarify it the only way to go is to make "same syntax
as regular Python" more clearly normative. Type checkers should of course
feel free ignore everything they don't care about.

Regarding the unicode issue, that is indeed unfortunate, and there's a long
but inconclusive discussion at https://github.com/python/typing/issues/208.
(If you want a longer discussion here please start a new subject.)

On Mon, Jul 16, 2018 at 1:02 PM, Adam Cataldo via Python-Dev <
python-dev@python.org> wrote:

> Thanks Brett and Teddy,
>
> Just so it doesn't get lost in the shuffle as folks dive into details,
> I'll re-ask my earlier question about stub files. Assuming there is
> consensus that there is ambiguity to resolve in the current definition, is
> updating the section on stub files the preferred option? The only
> alternative I can think of is to pull this out into a separate PEP. I
> frankly have no opinion on what the best way to capture this is. We're
> happy to help out either way.
>
> On Mon, Jul 16, 2018 at 12:48 PM Teddy Sudol  wrote:
>
>> Hi, my name is Teddy Sudol. I work with Adam and Rebecca on pytype.
>>
>> The explanation of stub files is unclear. The section you linked starts
>> with, "Stub files are files containing type hints that are only for use by
>> the type checker, not at runtime." According to
>> https://www.python.org/dev/peps/pep-0484/#acceptable-type-hints, type
>> hints may be classes, abstract base classes, types defined in the `types`
>> and `typing` modules, type variables, type aliases and None. Further in the
>> section you linked, PEP 484 also states, "Stub files have the same syntax
>> as regular Python modules," and, "no runtime behavior should be expected of
>> stub files."
>>
>> "Have the same syntax as regular Python modules" and "are files
>> containing type hints" are at odds with each other. This has led to
>> compatibility issues between Mypy and pytype. For example, `b''` is not a
>> valid type annotation, but until a month ago, `codecs.pyi` in typeshed used
>> exactly that: https://github.com/python/typeshed/commit/
>> 6bbf3d89eb9b6c3fd5b0c0f632b2ad9258cecf15#diff-
>> 5f6f48c425bc0c283784cf5277880c0cL95. If statements can be useful for
>> things like version checks, but on the other hand, pyi files aren't
>> supposed to have any runtime behavior. Additionally, codifying the syntax
>> for pyi files would settle questions like whether constants should be typed
>> using "x: " or "x = ...  # type: ".
>>
>> We would like to see a clear statement about the syntax of stub files.
>> Personally, I think they should be a subset of Python, but I'd also be
>> happy with an EBNF grammar for them.
>>
>> -- Teddy
>>
>>
>> On Mon, Jul 16, 2018 at 11:05 AM Brett Cannon  wrote:
>>
>>>
>>>
>>> On Mon, 16 Jul 2018 at 10:32 Adam Cataldo via Python-Dev <
>>> python-dev@python.org> wrote:
>>>





 *Hi Folks,Cc: Rebecca, pytypeThis is Adam Cataldo; I’m the engineering
 manager for the Python team at Google. Rebecca Chen, our lead pytype
  contributor, and I are interested in
 helping finalize PEP 484 if possible. To that end, we wanted to find out
 what technical issues the PEP 484 authors feel they still need to finalize.
 We also wanted to know how we can help.We have a large Python code base at
 Google, and may be able to use this to help resolve current incomplete
 definitions, by collecting data on how types are used. We also have a
 couple ambiguities that we’d love to get closure on: - One thing we care
 about in particular, given the implementation of pytype, is the detailed
 definition of what goes in a .pyi file. Do folks think this makes sense to
 include as part of PEP 484, or would this be better in a separate PEP? We’d
 love to get your thoughts.*

>>>
>>> What specifically do you want beyond https://www.python.org/dev/
>>> peps/pep-0484/#stub-files?
>>>
>>>



 * - The relationship between unicode and typing.Text in Python 2 has
 been a recurring source of confusion for our users. Especially since we
 contributed  to the current state
 of affairs, we’d like to figure out how to establish clarity 
 here.Thoughts?*

>>>

Re: [Python-Dev] Question about PEP 484

[Adam Cataldo via Python-Dev]

Thanks Brett and Teddy,

Just so it doesn't get lost in the shuffle as folks dive into details, I'll
re-ask my earlier question about stub files. Assuming there is consensus
that there is ambiguity to resolve in the current definition, is updating
the section on stub files the preferred option? The only alternative I can
think of is to pull this out into a separate PEP. I frankly have no opinion
on what the best way to capture this is. We're happy to help out either way.

On Mon, Jul 16, 2018 at 12:48 PM Teddy Sudol  wrote:

> Hi, my name is Teddy Sudol. I work with Adam and Rebecca on pytype.
>
> The explanation of stub files is unclear. The section you linked starts
> with, "Stub files are files containing type hints that are only for use by
> the type checker, not at runtime." According to
> https://www.python.org/dev/peps/pep-0484/#acceptable-type-hints, type
> hints may be classes, abstract base classes, types defined in the `types`
> and `typing` modules, type variables, type aliases and None. Further in the
> section you linked, PEP 484 also states, "Stub files have the same syntax
> as regular Python modules," and, "no runtime behavior should be expected of
> stub files."
>
> "Have the same syntax as regular Python modules" and "are files containing
> type hints" are at odds with each other. This has led to compatibility
> issues between Mypy and pytype. For example, `b''` is not a valid type
> annotation, but until a month ago, `codecs.pyi` in typeshed used exactly
> that:
> https://github.com/python/typeshed/commit/6bbf3d89eb9b6c3fd5b0c0f632b2ad9258cecf15#diff-5f6f48c425bc0c283784cf5277880c0cL95.
> If statements can be useful for things like version checks, but on the
> other hand, pyi files aren't supposed to have any runtime behavior.
> Additionally, codifying the syntax for pyi files would settle questions
> like whether constants should be typed using "x: " or "x = ...
> # type: ".
>
> We would like to see a clear statement about the syntax of stub files.
> Personally, I think they should be a subset of Python, but I'd also be
> happy with an EBNF grammar for them.
>
> -- Teddy
>
>
> On Mon, Jul 16, 2018 at 11:05 AM Brett Cannon  wrote:
>
>>
>>
>> On Mon, 16 Jul 2018 at 10:32 Adam Cataldo via Python-Dev <
>> python-dev@python.org> wrote:
>>
>>>
>>>
>>>
>>>
>>>
>>> *Hi Folks,Cc: Rebecca, pytypeThis is Adam Cataldo; I’m the engineering
>>> manager for the Python team at Google. Rebecca Chen, our lead pytype
>>>  contributor, and I are interested in
>>> helping finalize PEP 484 if possible. To that end, we wanted to find out
>>> what technical issues the PEP 484 authors feel they still need to finalize.
>>> We also wanted to know how we can help.We have a large Python code base at
>>> Google, and may be able to use this to help resolve current incomplete
>>> definitions, by collecting data on how types are used. We also have a
>>> couple ambiguities that we’d love to get closure on: - One thing we care
>>> about in particular, given the implementation of pytype, is the detailed
>>> definition of what goes in a .pyi file. Do folks think this makes sense to
>>> include as part of PEP 484, or would this be better in a separate PEP? We’d
>>> love to get your thoughts.*
>>>
>>
>> What specifically do you want beyond
>> https://www.python.org/dev/peps/pep-0484/#stub-files?
>>
>>
>>>
>>>
>>>
>>> * - The relationship between unicode and typing.Text in Python 2 has
>>> been a recurring source of confusion for our users. Especially since we
>>> contributed  to the current state
>>> of affairs, we’d like to figure out how to establish clarity here.Thoughts?*
>>>
>>
>> Do be aware, Adam, that due to Guido's retirement last week people might
>> be a bit preoccupied and so a little slow in responding. But then again
>> Guido just got a bit more free time so he might chime in on this one. ;)
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "pytype" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to pytype+unsubscr...@googlegroups.com.
>> To post to this group, send email to pyt...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/pytype/CAP1%3D2W4NxcsSdsiMrh55KhjkwgD0PGRcZJF_Azq3g6QFQ2oiAw%40mail.gmail.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Teddy Sudol via Python-Dev]

Hi, my name is Teddy Sudol. I work with Adam and Rebecca on pytype.

The explanation of stub files is unclear. The section you linked starts
with, "Stub files are files containing type hints that are only for use by
the type checker, not at runtime." According to
https://www.python.org/dev/peps/pep-0484/#acceptable-type-hints, type hints
may be classes, abstract base classes, types defined in the `types` and
`typing` modules, type variables, type aliases and None. Further in the
section you linked, PEP 484 also states, "Stub files have the same syntax
as regular Python modules," and, "no runtime behavior should be expected of
stub files."

"Have the same syntax as regular Python modules" and "are files containing
type hints" are at odds with each other. This has led to compatibility
issues between Mypy and pytype. For example, `b''` is not a valid type
annotation, but until a month ago, `codecs.pyi` in typeshed used exactly
that:
https://github.com/python/typeshed/commit/6bbf3d89eb9b6c3fd5b0c0f632b2ad9258cecf15#diff-5f6f48c425bc0c283784cf5277880c0cL95.
If statements can be useful for things like version checks, but on the
other hand, pyi files aren't supposed to have any runtime behavior.
Additionally, codifying the syntax for pyi files would settle questions
like whether constants should be typed using "x: " or "x = ...
# type: ".

We would like to see a clear statement about the syntax of stub files.
Personally, I think they should be a subset of Python, but I'd also be
happy with an EBNF grammar for them.

-- Teddy


On Mon, Jul 16, 2018 at 11:05 AM Brett Cannon  wrote:

>
>
> On Mon, 16 Jul 2018 at 10:32 Adam Cataldo via Python-Dev <
> python-dev@python.org> wrote:
>
>>
>>
>>
>>
>>
>> *Hi Folks,Cc: Rebecca, pytypeThis is Adam Cataldo; I’m the engineering
>> manager for the Python team at Google. Rebecca Chen, our lead pytype
>>  contributor, and I are interested in
>> helping finalize PEP 484 if possible. To that end, we wanted to find out
>> what technical issues the PEP 484 authors feel they still need to finalize.
>> We also wanted to know how we can help.We have a large Python code base at
>> Google, and may be able to use this to help resolve current incomplete
>> definitions, by collecting data on how types are used. We also have a
>> couple ambiguities that we’d love to get closure on: - One thing we care
>> about in particular, given the implementation of pytype, is the detailed
>> definition of what goes in a .pyi file. Do folks think this makes sense to
>> include as part of PEP 484, or would this be better in a separate PEP? We’d
>> love to get your thoughts.*
>>
>
> What specifically do you want beyond
> https://www.python.org/dev/peps/pep-0484/#stub-files?
>
>
>>
>>
>>
>> * - The relationship between unicode and typing.Text in Python 2 has been
>> a recurring source of confusion for our users. Especially since we
>> contributed  to the current state
>> of affairs, we’d like to figure out how to establish clarity here.Thoughts?*
>>
>
> Do be aware, Adam, that due to Guido's retirement last week people might
> be a bit preoccupied and so a little slow in responding. But then again
> Guido just got a bit more free time so he might chime in on this one. ;)
>
> --
> You received this message because you are subscribed to the Google Groups
> "pytype" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pytype+unsubscr...@googlegroups.com.
> To post to this group, send email to pyt...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pytype/CAP1%3D2W4NxcsSdsiMrh55KhjkwgD0PGRcZJF_Azq3g6QFQ2oiAw%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Question about PEP 484

[Brett Cannon]

On Mon, 16 Jul 2018 at 10:32 Adam Cataldo via Python-Dev <
python-dev@python.org> wrote:

>
>
>
>
>
> *Hi Folks,Cc: Rebecca, pytypeThis is Adam Cataldo; I’m the engineering
> manager for the Python team at Google. Rebecca Chen, our lead pytype
>  contributor, and I are interested in
> helping finalize PEP 484 if possible. To that end, we wanted to find out
> what technical issues the PEP 484 authors feel they still need to finalize.
> We also wanted to know how we can help.We have a large Python code base at
> Google, and may be able to use this to help resolve current incomplete
> definitions, by collecting data on how types are used. We also have a
> couple ambiguities that we’d love to get closure on: - One thing we care
> about in particular, given the implementation of pytype, is the detailed
> definition of what goes in a .pyi file. Do folks think this makes sense to
> include as part of PEP 484, or would this be better in a separate PEP? We’d
> love to get your thoughts.*
>

What specifically do you want beyond
https://www.python.org/dev/peps/pep-0484/#stub-files?


>
>
>
> * - The relationship between unicode and typing.Text in Python 2 has been
> a recurring source of confusion for our users. Especially since we
> contributed  to the current state
> of affairs, we’d like to figure out how to establish clarity here.Thoughts?*
>

Do be aware, Adam, that due to Guido's retirement last week people might be
a bit preoccupied and so a little slow in responding. But then again Guido
just got a bit more free time so he might chime in on this one. ;)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com