Re: [Python-Dev] Remove typing from the stdlib

[Guido van Rossum]

Let me just cut this short. typing.py will stay, and it will stay
provisional.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Lukasz Langa]

> On Nov 7, 2017, at 2:33 PM, Nick Coghlan  wrote:
> 
> On 8 November 2017 at 04:41, Lukasz Langa  wrote:
>> 4. How do we even version this library then? Probably like this: 3.7.0.0,
>> 3.7.0.1, 3.7.1.0, and so on. But that depends on answers to the other
>> questions above.
> 
> Something you may want to consider is switching to CalVer for typing
> itself, such that we end up saying something like "Python 3.7.0
> includes typing 2017.12.1".

You don't need to sell me on CalVer, all my private packages use this 
versioning scheme (just with the shorthand 17) :-)

And yes, this is a good suggestion.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Nick Coghlan]

On 8 November 2017 at 04:41, Lukasz Langa  wrote:
> 4. How do we even version this library then? Probably like this: 3.7.0.0,
> 3.7.0.1, 3.7.1.0, and so on. But that depends on answers to the other
> questions above.

Something you may want to consider is switching to CalVer for typing
itself, such that we end up saying something like "Python 3.7.0
includes typing 2017.12.1".

My experience has been that CalVer is just *better* for
interoperability specifications, since it inherently conveys
information about the age of the specification. Saying "We target
typing 2017.12.1" in 2018 immediately lets people know they're going
to need some pretty up to date software to run a project that has that
caveat on it. By contrast, saying the same thing in 2021 means most
things released in the past 3 years should be able to handle it.

Such an approach also avoids future confusion if the final version of
Python 3.7 were to start bundling Python 3.8's version of typing.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Barry Warsaw]

On Nov 7, 2017, at 10:41, Lukasz Langa  wrote:

> 3. Does that mean that Debian is going to rip it out and make people install 
> a `python-typing` .deb? Sadly, probably yes. We need to figure out what that 
> means for us.

Maybe.  Full disclosure, I’ve recently scaled back my contributions to Debian, 
so I won’t be doing this work, but if I was, I’d probably handle it very 
similarly to other replaceable external dependencies (e.g. pip).

There is a small loophole in policy to allow for the building and use of wheels 
for just this *limited* purpose.  So roughly I would propose packaging the 
external python-typing package as a separately installable deb, but also to 
build this into a wheel that we can pull in at python3.7 interpreter package 
build time.

It’s fairly easy to do, and all the infrastructure is already there.  What 
would be useful is for upstream CPython to make it easy to import an externally 
built and installed wheel, from some location outside of its own installed tree 
(/usr/share/python-wheels).

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Lukasz Langa]

> On Nov 7, 2017, at 9:39 AM, Brett Cannon  wrote:
> 
> On Tue, 7 Nov 2017 at 03:34 Paul Moore  > wrote:
> 
> Because type annotations are a development-time feature, they should
> *not* require a dependency in the final deployment (apart from Python
> itself). However, because they are a language syntax feature they are
> of necessity written in the application source. And type specification
> of anything more complex than basic types (for example, List[int])
> requires classes defined in the typing module. Therefore, typing must
> be in the stdlib so that use of type annotations by the developer
> doesn't impose a runtime dependency on the end user.
> 
> That's not necessarily true if Lukasz's PEP lands and annotations become 
> strings. The dependency would only need to be installed if someone chose to 
> introspect the annotations and then "instantiate" them into actual objects. 
> And that only comes up if someone does it from outside by a 3rd-party, who 
> would then need to install the type annotation dependencies themselves.

PEP 563 is the first step there but it's not enough. For this idea to work, 
`typing.TYPE_CHECKING` would need to be moved to the Python runtime, so that 
you can do

if TYPE_CHECKING:
from typing import *

More importantly, you would put type aliases, type variables and new-types in 
this `if` block, too.

But even if all this is true, there's still two remaining features that require 
runtime availability:

1. `cast()`; and
2. creating generic classes by subclassing `Generic[T]` or any of its 
subclasses. And soon enough, Protocols.

I hope I'm not forgetting any other cases. For `cast()` I have an idea how to 
move it to a variable annotation. For generic classes, there's no magic, you 
need `typing` at runtime. Fortunately, that latter case is (I hope?) relatively 
unlikely.

All the above is summarized here:
https://github.com/python/typing/issues/496 


- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Lukasz Langa]

> On Nov 6, 2017, at 8:01 PM, Nick Coghlan  wrote:
> 
> As I indicated in my comment there, I'm now wondering if there might
> be an opportunity here whereby we could use the *dataclasses* module
> to define a stable non-provisional syntactically compatible subset of
> the typing module, and require folks to start explicitly depending on
> the typing module (as a regular PyPI dependency) if they want anything
> more sophisticated than that (Unions, Generics, TypeVars, etc).

I have an issue open about essentially this idea:
https://github.com/python/typing/issues/496


The consensus is that this is too expensive to do this in time for Python 3.7. 
On the other hand, not doing anything is terrible in terms of usability: now 
users will be forced to use both `typing` (a frozen non-provisional version in 
the stdlib) and `typing_extensions` (new features that we couldn't add directly 
to `typing`).


This is the reason why Guido asked about moving `typing` out of the standard 
library. Currently it seems that the best thing that we can do is to ship an 
upgradeable version of `typing` with Python 3.7.0. There are many details to be 
figured out here:

1. Does that mean `typing` is going to be provisional forever? The answer is 
surprisingly "yes" since it makes sense to keep updating the bundled `typing` 
in 3.7.1, 3.7.2, etc. We need to update PEP 411 to talk about this new case.

2. Does that mean `typing` can now become backwards incompatible randomly? The 
answer should be "no" and already is "no". If you look at typing.py, it already 
ships in Python 3.6 with some crazy dances that makes it work even on Python 
3.3.

3. Does that mean that Debian is going to rip it out and make people install a 
`python-typing` .deb? Sadly, probably yes. We need to figure out what that 
means for us.

4. How do we even version this library then? Probably like this: 3.7.0.0, 
3.7.0.1, 3.7.1.0, and so on. But that depends on answers to the other questions 
above.

5. What will happen to typing_extensions and mypy_extensions? They can probably 
be folded into typing 3.7.0.0. This is a huge win for usability.

6. Where will we track issues for it? Which repo will be the source of truth? 
Fortunately, we already have https://github.com/python/typing/ 



There's more things to figure out for sure. And the clock is ticking. But this 
is a great case study to figure out since more packages in the standard library 
could use a release model like this, and maybe in the future this will finally 
lead to unbundling of the standard library from the runtime. Yeah, it's a long 
shot and the dependency graph between libraries in the standard library is 
rather extensive. But this is the first step.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Brett Cannon]

On Tue, 7 Nov 2017 at 03:34 Paul Moore  wrote:

> On 7 November 2017 at 10:18, Steve Holden  wrote:
> > On Tue, Nov 7, 2017 at 12:35 AM, Donald Stufft  wrote:
> > [..]
> >
> >>
> >> Maybe we just need to fully flesh out the idea of a "Python Core" (What
> >> exists now as “Python”) and a “Python Platform” (Python Core + A select
> set
> >> of preinstalled libraries). Then typing can just be part of the Python
> >> Platform, and gets installed as part of your typical installation, but
> is
> >> otherwise an independent piece of code.
> >>
> > Given that (type and other) annotations have been promoted as an optional
> > feature of the language it seems unfair and perhaps unwise to add a
> > dependency specifically to support
> > them
> >  to the stdlib and therefore the Python core.
> > Since type annotations are, as Paul pointed out, development-time
> features,
> > it would appear to behoove those wishing to use them to separate them in
> > such a way that the software can be installed without annotations, and
> > therefore without the need for the typing module. Assuming they would
> like
> > to see the widest possible distribution, of course. For selected
> audiences I
> > am sure typing will be de rigeur,
>
> From my point of view, I take the same premise and come to the
> opposite conclusion.
>
> Because type annotations are a development-time feature, they should
> *not* require a dependency in the final deployment (apart from Python
> itself). However, because they are a language syntax feature they are
> of necessity written in the application source. And type specification
> of anything more complex than basic types (for example, List[int])
> requires classes defined in the typing module. Therefore, typing must
> be in the stdlib so that use of type annotations by the developer
> doesn't impose a runtime dependency on the end user.
>

That's not *necessarily* true if Lukasz's PEP lands and annotations become
strings. The dependency would only need to be installed if someone chose to
introspect the annotations and then "instantiate" them into actual objects.
And that only comes up if someone does it from outside by a 3rd-party, who
would then need to install the type annotation dependencies themselves.

I fully admit that could get messy if introspection from 3rd-parties
happens at all regularly and trying to manage that kind of situation. In
this instance I would argue that any code that is to facilitate creating an
object from an annotation exist outside of the stdlib if 'typing' gets
removed to prevent this sort of situation without the user of such code
being fully aware of what they are up against.

-Brett


>
> If there were a way of including type annotations that had no runtime
> effect on the final deployed program, things would be different. But
> the decision to make annotations part of the language syntax precludes
> that. In particular, "it would appear to behoove those wishing to use
> them to separate them" - there's no way of doing that *precisely*
> because they are a language syntax feature.
>
> Paul
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Paul Moore]

On 7 November 2017 at 10:18, Steve Holden  wrote:
> On Tue, Nov 7, 2017 at 12:35 AM, Donald Stufft  wrote:
> [..]
>
>>
>> Maybe we just need to fully flesh out the idea of a "Python Core" (What
>> exists now as “Python”) and a “Python Platform” (Python Core + A select set
>> of preinstalled libraries). Then typing can just be part of the Python
>> Platform, and gets installed as part of your typical installation, but is
>> otherwise an independent piece of code.
>>
> Given that (type and other) annotations have been promoted as an optional
> feature of the language it seems unfair and perhaps unwise to add a
> dependency specifically to support
> them
>  to the stdlib and therefore the Python core.
> Since type annotations are, as Paul pointed out, development-time features,
> it would appear to behoove those wishing to use them to separate them in
> such a way that the software can be installed without annotations, and
> therefore without the need for the typing module. Assuming they would like
> to see the widest possible distribution, of course. For selected audiences I
> am sure typing will be de rigeur,

From my point of view, I take the same premise and come to the
opposite conclusion.

Because type annotations are a development-time feature, they should
*not* require a dependency in the final deployment (apart from Python
itself). However, because they are a language syntax feature they are
of necessity written in the application source. And type specification
of anything more complex than basic types (for example, List[int])
requires classes defined in the typing module. Therefore, typing must
be in the stdlib so that use of type annotations by the developer
doesn't impose a runtime dependency on the end user.

If there were a way of including type annotations that had no runtime
effect on the final deployed program, things would be different. But
the decision to make annotations part of the language syntax precludes
that. In particular, "it would appear to behoove those wishing to use
them to separate them" - there's no way of doing that *precisely*
because they are a language syntax feature.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Steve Holden]

On Tue, Nov 7, 2017 at 12:35 AM, Donald Stufft  wrote:
[..]


> Maybe we just need to fully flesh out the idea of a "Python Core" (What
> exists now as “Python”) and a “Python Platform” (Python Core + A select set
> of preinstalled libraries). Then typing can just be part of the Python
> Platform, and gets installed as part of your typical installation, but is
> otherwise an independent piece of code.
>
> Given that (type and other) annotations have been promoted as an optional
feature of the language it seems unfair and perhaps unwise to add a
dependency specifically to support
​them
 to the stdlib and therefore the Python core.
​ Since type annotations are, as Paul pointed out, development-time
features, it​ would appear to behoove those wishing to use them to separate
them in such a way that the software can be installed without annotations,
and therefore without the need for the typing module. Assuming they would
like to see the widest possible distribution, of course. For selected
audiences I am sure typing will be *de rigeur*,


​In this scenario, surely the most "typical" installation would be a
virtualenv. ​Do all virtualenvs rely on the same Platform? Who decides
which additional libraries are required for the Platform? Doesn't this just
add another "type of installation" distinction to confuse the unwary? How
do I maintain the Platform separately from the Core? Does my sysadmin
maintain either or both? I'd like to see a little more clarity about the
benefits such a schism would offer.

regards
 Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Paul Moore]

On 7 November 2017 at 05:20, Ethan Smith  wrote:
> I'm not so keen on this because I think some things in typing (such as
> NamedTuple) probably deserve to be in the collections module. And some of
> the ABCs could probably also be merged with collections.abc but doing this
> correctly and not all at once would be quite difficult.
> I do think the typing concepts should be better integrated into the standard
> library. However, a fair amount of the clases you list (such as NamedTuple)
> are in of themselves dependent on parts of typing.

This is a good point. To what extent is it true that the stdlib
*already* uses the typing module internally, it's just that the usage
is hidden by the fact that the examples of this are in the typing
module - not because they "should" be there but simply because it
isolates the use of typing to that one module?

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Paul Moore]

On 7 November 2017 at 00:35, Donald Stufft  wrote:
> Maybe we just need to fully flesh out the idea of a "Python Core" (What
> exists now as “Python”) and a “Python Platform” (Python Core + A select set
> of preinstalled libraries). Then typing can just be part of the Python
> Platform, and gets installed as part of your typical installation, but is
> otherwise an independent piece of code.

I quite like this idea, but at a minimum it would mean that the
Windows installers for Python should include "core" and "platform"
builds, with "platform" being the version that new users are directed
to. And I'm not sure python-dev would be comfortable with agreeing and
distributing what would essentially be a curated subset of PyPI
packages. The most recent discussion on "a recommended set of
packages" didn't get much further than 3 or 4 candidates, which hardly
constitutes a "platform".

Conversely, something like Anaconda could be considered as the "Python
Platform" (assuming they bundle typing...), but I'm not really
comfortable with python-dev aligning themselves that strongly with a
single distributor. (We're talking about a recommendation along the
lines of "people wanting to gain, for example, corporate approval for
"Python" should request approval for the Anaconda distribution rather
than the python.org distribution").

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Ethan Smith]

> Beyond the API already proposed in PEP 557, this would mean adding:
>
> * dataclasses.ClassVar (as proposed above)
> * dataclasses.Any (probably just set to the literal string
> "dataclasses.Any")
> * dataclasses.NamedTuple (as a replacement for typing.NamedTuple)
> * potentially dataclasses.is_class_var (instead of dataclasses
> implicitly snooping in sys.modules looking for a "typing" module)
>
> In effect, where we now have "typing" and "typing_extensions", we'd
> instead have "dataclasses" in the standard library (with the subset of
> annotations needed to define data record classes), and "typing" as an
> independently versioned module on PyPI.
>
>
I'm not so keen on this because I think some things in typing (such as
NamedTuple) probably deserve to be in the collections module. And some of
the ABCs could probably also be merged with collections.abc but doing this
correctly and not all at once would be quite difficult.
I do think the typing concepts should be better integrated into the
standard library. However, a fair amount of the clases you list (such as
NamedTuple) are in of themselves dependent on parts of typing.

Cheers,
Ethan

I think such an approach could address a few problems:
>
> - the useful-at-runtime concepts (in dataclasses) would be clearly
> separated from the static-type-analysis enablers (in typing)
> - type hints would still have a clear presence in the regular standard
> library, but the more complex parts would be opt-in (similar to
> statistics vs SciPy, secrets vs cryptography, etc)
> - as various concepts in typing matured and became genuinely stable,
> they could potentially "graduate" into the standard library's
> dataclasses module
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> ethan%40ethanhs.me
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Nick Coghlan]

On 6 November 2017 at 12:18, Nick Coghlan  wrote:
> That particular dependency could also be avoided by defining an
> "is_class_var(annotation)" generic function and a "ClassVar" helper
> object in the dataclasses module. For example:
>
> class _ClassVar:
> def __init__(self, annotation):
> self.annotation = annotation
>
> class _MakeClassVar:
> def __getitem__(self, key):
> return _ClassVar(key)
>
> ClassVar = _MakeClassVar()
>
> @functools.singledispatch
> def is_class_var(annotation):
> return isinstance(annotation, _ClassVar)
>
> It would put the burden on static analysers and the typing module to
> understand that `dataclasses.ClassVar` meant the same thing
> conceptually as `typing.ClassVar`, but I think that's OK.

Eric filed a new issue for this idea here:
https://github.com/ericvsmith/dataclasses/issues/61

As I indicated in my comment there, I'm now wondering if there might
be an opportunity here whereby we could use the *dataclasses* module
to define a stable non-provisional syntactically compatible subset of
the typing module, and require folks to start explicitly depending on
the typing module (as a regular PyPI dependency) if they want anything
more sophisticated than that (Unions, Generics, TypeVars, etc).

Unlike the typing module, dataclasses *wouldn't* attempt to give
annotations any meaningful runtime semantics, they'd just be ordinary
class instances (hence no complex metaclasses required, hence a
relatively fast import time).

Beyond the API already proposed in PEP 557, this would mean adding:

* dataclasses.ClassVar (as proposed above)
* dataclasses.Any (probably just set to the literal string "dataclasses.Any")
* dataclasses.NamedTuple (as a replacement for typing.NamedTuple)
* potentially dataclasses.is_class_var (instead of dataclasses
implicitly snooping in sys.modules looking for a "typing" module)

In effect, where we now have "typing" and "typing_extensions", we'd
instead have "dataclasses" in the standard library (with the subset of
annotations needed to define data record classes), and "typing" as an
independently versioned module on PyPI.

I think such an approach could address a few problems:

- the useful-at-runtime concepts (in dataclasses) would be clearly
separated from the static-type-analysis enablers (in typing)
- type hints would still have a clear presence in the regular standard
library, but the more complex parts would be opt-in (similar to
statistics vs SciPy, secrets vs cryptography, etc)
- as various concepts in typing matured and became genuinely stable,
they could potentially "graduate" into the standard library's
dataclasses module

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Donald Stufft]

> On Nov 6, 2017, at 4:35 AM, Paul Moore  wrote:
> 
> 1. Without typing available, some programs using type annotations
> won't run. That is, using type annotations (a
> test-time/development-time feature) introduces a runtime dependency on
> typing, and hence introduces an extra deployment concern (unlike other
> development-type features like test frameworks).
> 2. For some people, if something isn't in the Python standard library
> (technically, in a standard install), it's not available (without
> significant effort, or possibly not at all). For those people, a
> runtime dependency on a non-stdlib typing module means "no use of type
> annotations allowed".
> 3. Virtual environments typically only include the stdlib, and "use
> system site-packages" has affects more than just a single module, so
> bundling still has issues for virtualenvs - and in the packaging
> tutorials, we're actively encouraging people to use virtualenvs. We
> (python-dev) can work around this issue for venv by auto-installing
> typing, but that still leaves virtualenv (which is critically short of
> resources, and needs to support older versions of Python, so a major
> change like bundling typing is going to be a struggle to get
> implemented).


Maybe we just need to fully flesh out the idea of a "Python Core" (What exists 
now as “Python”) and a “Python Platform” (Python Core + A select set of 
preinstalled libraries). Then typing can just be part of the Python Platform, 
and gets installed as part of your typical installation, but is otherwise an 
independent piece of code.___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Paul Moore]

On 6 November 2017 at 03:41, Lukasz Langa  wrote:
>
>> On 4 Nov, 2017, at 3:39 AM, Paul Moore  wrote:
>>
>> Lukasz Langa said:
>>> So, the difference is in perceived usability. It's psychological.
>>
>> Please, let's not start the "not in the stdlib isn't an issue" debate
>> again. If I concede it's a psychological issue, will you concede that
>> the fact that it's psychological doesn't mean that it's not a real,
>> difficult to solve, problem for some people? I'm also willing to
>> concede that it's a *minority* problem, if that helps. But can we stop
>> dismissing it as a non-existent problem?
>
> Paul, if you read the words I wrote in my e-mail verbatim, you will note that 
> I am not saying it's not real or it's not important. Quite the opposite. Can 
> you elaborate what made you think that my assertion that the issue is 
> psychological made you think I'm being dismissive? To me it looks like you're 
> aggressively agreeing with me, so I'd like to understand what caused your 
> reaction.

Apologies. On rereading your email, I can see how you meant that.
However, it didn't come across that way to me on first reading. There
have been a few other threads on various lists I've been involved in
recently where it's been really hard to get anyone to see that there's
any practical issues for people if a module is on PyPI rather than
being in the stdlib. So I'm afraid I'd got pretty tired of arguing the
same points over and over again, and over-reacted to what I thought
you said.

To explain my actual point a little more clearly:

1. Without typing available, some programs using type annotations
won't run. That is, using type annotations (a
test-time/development-time feature) introduces a runtime dependency on
typing, and hence introduces an extra deployment concern (unlike other
development-type features like test frameworks).
2. For some people, if something isn't in the Python standard library
(technically, in a standard install), it's not available (without
significant effort, or possibly not at all). For those people, a
runtime dependency on a non-stdlib typing module means "no use of type
annotations allowed".
3. Virtual environments typically only include the stdlib, and "use
system site-packages" has affects more than just a single module, so
bundling still has issues for virtualenvs - and in the packaging
tutorials, we're actively encouraging people to use virtualenvs. We
(python-dev) can work around this issue for venv by auto-installing
typing, but that still leaves virtualenv (which is critically short of
resources, and needs to support older versions of Python, so a major
change like bundling typing is going to be a struggle to get
implemented).

None of these problems are really addressed by simply saying "pip
install typing". That's not for psychological reasons, there are
genuine barriers to having that work. However, it's not at all clear
how many people are affected by these issues. My personal feeling is
that the group of people participating in open source development is
biased towards environments where it's not a problem, but that's more
gut feeling than hard facts. The place where barriers are
perceived/psychological is when we try to discuss workarounds or
solutions - because there's a lot of guesswork about what people's
environments are like, there's a tendency to assume scenarios that
support our preferred solution, rather than those that don't.

Personally, I like to think that my arguments are "giving a voice to
people in constrained corporate environments like mine, who are
under-represented in open source environments". But it's very easy for
a view like that to turn into "banging on about a minor problem as if
it were a crisis", and I'm probably guilty of doing that. Worse still,
that results in me getting frustrated and then over-reacting further -
which is where we came in.

So again, apologies. I misunderstood what you were saying, but more as
a result of my personal biases than because you weren't sufficiently
clear.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Lukasz Langa]

> On 4 Nov, 2017, at 3:39 AM, Paul Moore  wrote:
> 
> Lukasz Langa said:
>> So, the difference is in perceived usability. It's psychological.
> 
> Please, let's not start the "not in the stdlib isn't an issue" debate
> again. If I concede it's a psychological issue, will you concede that
> the fact that it's psychological doesn't mean that it's not a real,
> difficult to solve, problem for some people? I'm also willing to
> concede that it's a *minority* problem, if that helps. But can we stop
> dismissing it as a non-existent problem?

Paul, if you read the words I wrote in my e-mail verbatim, you will note that I 
am not saying it's not real or it's not important. Quite the opposite. Can you 
elaborate what made you think that my assertion that the issue is psychological 
made you think I'm being dismissive? To me it looks like you're aggressively 
agreeing with me, so I'd like to understand what caused your reaction.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Nick Coghlan]

On 4 November 2017 at 02:46, Eric V. Smith  wrote:
> On 11/3/2017 12:15 PM, Victor Stinner wrote:
>>
>> Hi,
>>
>> 2017-11-03 15:36 GMT+01:00 Guido van Rossum :
>>>
>>> Maybe we should remove typing from the stdlib?
>>> https://github.com/python/typing/issues/495
>
>
>> The typing module is not used yet in the stdlib, so there is no
>> technically reason to keep typing part of the stdlib. IMHO it's
>> perfectly fine to keep typing and annotations out of the stdlib, since
>> the venv & pip tooling is now rock solid ;-)
>
>
> I'm planning on using it for PEP 557:
> https://www.python.org/dev/peps/pep-0557/#class-variables
>
> The way the code currently checks for this should still work if typing is
> not in the stdlib, although of course it's assuming that the name "typing"
> really is the "official" typing library.
>
> # If typing has not been imported, then it's impossible for
> #  any annotation to be a ClassVar. So, only look for ClassVar
> #  if typing has been imported.
> typing = sys.modules.get('typing')
> if typing is not None:
> # This test uses a typing internal class, but it's the best
> #  way to test if this is a ClassVar.
> if type(a_type) is typing._ClassVar:
> # This field is a ClassVar. Ignore it.
> continue
>
> See also https://github.com/ericvsmith/dataclasses/issues/14

That particular dependency could also be avoided by defining an
"is_class_var(annotation)" generic function and a "ClassVar" helper
object in the dataclasses module. For example:

class _ClassVar:
def __init__(self, annotation):
self.annotation = annotation

class _MakeClassVar:
def __getitem__(self, key):
return _ClassVar(key)

ClassVar = _MakeClassVar()

@functools.singledispatch
def is_class_var(annotation):
return isinstance(annotation, _ClassVar)

It would put the burden on static analysers and the typing module to
understand that `dataclasses.ClassVar` meant the same thing
conceptually as `typing.ClassVar`, but I think that's OK.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Raymond Hettinger]

> On Nov 3, 2017, at 9:15 AM, Victor Stinner  wrote:
> 
> 2017-11-03 15:36 GMT+01:00 Guido van Rossum :
>> Maybe we should remove typing from the stdlib?
>> https://github.com/python/typing/issues/495
> 
> I'm strongly in favor on such move.
> 
> My experience with asyncio in the stdlib is that users expect changes
> faster than the very slow release process of the stdlib (a release
> every 18 months in average).
> 
> I saw many PEPs and discussion on the typing design (meta-classes vs
> regular classes), as if the typing is not stable enough to be part of
> the stdlib.
> 
> The typing module is not used yet in the stdlib, so there is no
> technically reason to keep typing part of the stdlib. IMHO it's
> perfectly fine to keep typing and annotations out of the stdlib, since
> the venv & pip tooling is now rock solid ;-)

I concur with Victor on every point.  In particular, many of the good reasons 
that typeshed is external to the standard library will also apply to typing.py. 
 

It would also be nice to not have typing.py vary with each version of CPython's 
release cycle.  Not only would typing benefit from more frequent updates, it 
would be nice to have updates that aren't tied to a specific version of CPython 
-- that would help folks who have to maintain code that works across multiple 
CPython versions (i.e. the same benefit that we get by always installing the 
most up-to-date versions of requests, typeshed, jinja2, etc).

Already, we've have updates to typing.py in the point releases of Python 
because those updates were considered so useful and important.


Raymond 





___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Brett Cannon]

On Fri, 3 Nov 2017 at 17:59 Lukasz Langa  wrote:

>
> > On 3 Nov, 2017, at 4:01 PM, Victor Stinner 
> wrote:
> >
> > The question is if you would only need  or  > pip install typing>.
> >
> > If typing is removed from the stdlib, you can still use it in your
> > application. It's "just" another dependency no?
>
>
> The ideal situation is that something is built-in and just works,
> examples: dicts, lists, sorted().
>
> So, if you have to import it to use it, it's still great but less
> seamless, current example: regular expressions. Let's say Guido suggests we
> should import sorted, dict, and list before use. Not a big deal, right? I
> mean, how many applications do you know that don't use any other imports?
>
> Finally, if you have to find a third-party package, add it to
> requirements.txt and manage the dependency forward, that's even less
> seamless. The standard library has a pretty conservative approach to
> backwards compatibility. On the other hand, third-party libraries often
> don't. Sure, there's noble exceptions but the general feel is that you need
> to be more careful with dependencies from PyPI. If somebody suggested that
> regular expressions or dictionaries should be moved to PyPI, in my book
> that would suggest strange things will start happening in the future.
>
> So, the difference is in perceived usability. It's psychological.
>

I think another psychological side-effect of removing 'typing' is how
strongly people will view the guidance that annotations are generally
expected to be type hints. With 'typing' in the stdlib, it made that
assumption a bit strong, especially if the stdlib ever started to use type
hints itself. But by saying "we expect annotations will be used for type
hints, but we actually don't do that in the stdlib nor provide a mechanism
for actually specifying type hints", that weakens the message. It starts to
feel like a "do as I say, not as I do" bit of guidance (which to me is
different than the worry that the usage of type hints will decrease).

But that might be fine; I personally don't know. But my suspicion is we
will see an uptick of alternative annotation uses that aren't type hints if
we take 'typing' out, and that's something we should be aware of and okay
with.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Paul Moore]

On 4 November 2017 at 03:53, Nick Coghlan  wrote:
> If I understand correctly, a lot of the complexity in the current
> typing.py implementation is there to make isinstance and issubclass do
> something "useful" at runtime, and to allow generics to be used as
> base classes.
>
> If it wasn't for those design goals, then "typing.List[int]" could
> just return a lightweight instance of a regular class rather than a
> usable Python class definition.

+1 to this.

> If I'm right about that, then PEP 560's proposal to allow types to
> implement a hook that says "Replace me with this other object for
> runtime subclassing purposes" may be enough to let you delete most of
> the current code in the typing module - you'd just need to have
> isinstance and issubclass respect that new hook as well, and define
> the hooks as returning the relevant ABCs.

That would seem ideal to me.

Lukasz Langa said:
> So, the difference is in perceived usability. It's psychological.

Please, let's not start the "not in the stdlib isn't an issue" debate
again. If I concede it's a psychological issue, will you concede that
the fact that it's psychological doesn't mean that it's not a real,
difficult to solve, problem for some people? I'm also willing to
concede that it's a *minority* problem, if that helps. But can we stop
dismissing it as a non-existent problem?

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Nick Coghlan]

On 4 November 2017 at 04:24, Guido van Rossum  wrote:
> A side note (I'm reading all responses but staying out of the discussion):
>
> No static checker should depend on the *contents* of typing.py, since it's
> just a bunch of runtime gymnastics to allow types to be evaluated at runtime
> without errors, with a secondary goal of making them introspectable (some
> folks don't even agree with the latter, e.g. Mark Shannon).
>
> Static analyzers should be able to make strong *assumptions* about what
> things defined there mean -- in mypy such assumptions are all over the
> place, based on the full name of things -- it never reads typing.py. (It
> reads typing.pyi from typeshed, but what's there is ignored in many cases
> too.)

If I understand correctly, a lot of the complexity in the current
typing.py implementation is there to make isinstance and issubclass do
something "useful" at runtime, and to allow generics to be used as
base classes.

If it wasn't for those design goals, then "typing.List[int]" could
just return a lightweight instance of a regular class rather than a
usable Python class definition.

If I'm right about that, then PEP 560's proposal to allow types to
implement a hook that says "Replace me with this other object for
runtime subclassing purposes" may be enough to let you delete most of
the current code in the typing module - you'd just need to have
isinstance and issubclass respect that new hook as well, and define
the hooks as returning the relevant ABCs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Lukasz Langa]

> On 3 Nov, 2017, at 4:01 PM, Victor Stinner  wrote:
> 
> The question is if you would only need  or  pip install typing>.
> 
> If typing is removed from the stdlib, you can still use it in your
> application. It's "just" another dependency no?


The ideal situation is that something is built-in and just works, examples: 
dicts, lists, sorted().

So, if you have to import it to use it, it's still great but less seamless, 
current example: regular expressions. Let's say Guido suggests we should import 
sorted, dict, and list before use. Not a big deal, right? I mean, how many 
applications do you know that don't use any other imports?

Finally, if you have to find a third-party package, add it to requirements.txt 
and manage the dependency forward, that's even less seamless. The standard 
library has a pretty conservative approach to backwards compatibility. On the 
other hand, third-party libraries often don't. Sure, there's noble exceptions 
but the general feel is that you need to be more careful with dependencies from 
PyPI. If somebody suggested that regular expressions or dictionaries should be 
moved to PyPI, in my book that would suggest strange things will start 
happening in the future.

So, the difference is in perceived usability. It's psychological.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Victor Stinner]

2017-11-03 18:00 GMT+01:00 Steve Dower :
> If not having typing installed means you can't use "Any" or "Optional" in an
> annotation, that basically kills the whole thing. Some primitives need to be
> there.

I'm not sure that I understand you.

The question is if you would only need  or .

If typing is removed from the stdlib, you can still use it in your
application. It's "just" another dependency no? Which major (non
trivial) application or Python module has zero external dependency
nowaday?

The only drawback is that we cannot use typing "anymore" in the stdlib
itself, since we don't allow external dependency in the stdlib for
practical reasons. But as I wrote, we don't use typing currently in
stdlib. I'm perfectly fine with the current status of having
annotations on the stdlib in a third party project.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Sven R. Kunze]

On 03.11.2017 21:34, Paul Moore wrote:

Consider someone who's downloaded Python and PyCharm (or Visual
Studio). They want to get the benefit of the IDE code completion
facilities, so they declare their argument as List[int], following the
information they've found on how to declare lists of integers.


The PyCharm I know is capable of detecting such simple types on its own, 
without type hints.



I for one like the idea of a faster evolution of typing.py.

Cheers,
Sven


PS: pip is pretty standard these days, so I don't think it's much of an 
issue for guys who really needs it installed.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Paul Moore]

On 3 November 2017 at 17:00, Steve Dower  wrote:
>
> I'm torn.
>
> If not having typing installed means you can't use "Any" or "Optional" in an
> annotation, that basically kills the whole thing. Some primitives need to be
> there.

Thinking some more about this, I think that it's now established that
the annotation syntax is for types - any debate over other uses for
them is now past. As a result, though, I think it's important that the
language (and/or the standard library) should include support for
*expressing* those types. The typing module is what allows users to
express types like "list of integers", or "optional string", or
"iterable", and if we move typing out of the stdlib, we make it
impossible for people who want to use the language feature to do so
from within the core language.

Consider someone who's downloaded Python and PyCharm (or Visual
Studio). They want to get the benefit of the IDE code completion
facilities, so they declare their argument as List[int], following the
information they've found on how to declare lists of integers. And now
their code won't run, until they install typing from PyPI. And there's
no workaround, because you can't express List[int] in the core
language/stdlib. That's not a very good start for a newcomer to
Python.

I'm fine with the "advanced" bits of typing being removed from the
stdlib, but I think we need to include in the stdlib at least enough
to express the basic types of the language (including common
combinations such as Optional and Union).

Paul

PS Apologies if I've misunderstood any of the technical aspects of
typing - I'm happy to be corrected. As I said in another email, I've
not actually used type annotations in my own code yet, although I'm
thinking of starting to - precisely because I've been using PyCharm
recently and the IDE support when you declare types is quite nice.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Barry Warsaw]

On Nov 3, 2017, at 10:00, Steve Dower  wrote:
> 
> On 03Nov2017 0915, Victor Stinner wrote:
>> Hi,
>> 2017-11-03 15:36 GMT+01:00 Guido van Rossum :
>>> Maybe we should remove typing from the stdlib?
>>> https://github.com/python/typing/issues/495
>> I'm strongly in favor on such move.
> 
> I'm torn.

Me too.  We’re seeing much greater adoption of type annotations, and it’s 
becoming one of the killer features for adopting Python 3.  I’d be hesitant to 
accept anything that slows that adoption down.  While it’s been technically 
provisional, a lot of code is beginning to depend on it and it would be a shame 
to break that code as we also start to adopt Python 3.7.  But I can appreciate 
the need to iterate on its API faster.

I don’t know if a middle ground is feasible.  What core functionality and 
stable-enough APIs can be kept in stdlib typing, and can we provide an 
extension or override mechanism if you want the latest and greatest?

Cheers,
-Barry



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Stéfane Fermigier]

On Fri, Nov 3, 2017 at 6:47 PM, Antoine Pitrou  wrote:

> On Fri, 3 Nov 2017 12:46:33 -0400
> "Eric V. Smith"  wrote:
> > On 11/3/2017 12:15 PM, Victor Stinner wrote:
> > > Hi,
> > >
> > > 2017-11-03 15:36 GMT+01:00 Guido van Rossum :
> > >> Maybe we should remove typing from the stdlib?
> > >> https://github.com/python/typing/issues/495
> >
> > > The typing module is not used yet in the stdlib, so there is no
> > > technically reason to keep typing part of the stdlib. IMHO it's
> > > perfectly fine to keep typing and annotations out of the stdlib, since
> > > the venv & pip tooling is now rock solid ;-)
> >
> > I'm planning on using it for PEP 557:
> > https://www.python.org/dev/peps/pep-0557/#class-variables
> >
> > The way the code currently checks for this should still work if typing
> > is not in the stdlib, although of course it's assuming that the name
> > "typing" really is the "official" typing library.
>
> I don't think other modules should start relying on the typing module at
> runtime.
>

They already do, as I've noted in my message about dependency injection,
and I find this quite elegant.

Previously, similar projects used decorators, or less elegant constructs.

  S.

-- 
Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier -
http://linkedin.com/in/sfermigier
Founder & CEO, Abilian - Enterprise Social Software -
http://www.abilian.com/
Chairman, Free&OSS Group / Systematic Cluster -
http://www.gt-logiciel-libre.org/
Co-Chairman, National Council for Free & Open Source Software (CNLL) -
http://cnll.fr/
Founder & Organiser, PyData Paris - http://pydata.fr/
---
“You never change things by fighting the existing reality. To change
something, build a new model that makes the existing model obsolete.” — R.
Buckminster Fuller
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Steve Dower]

On 03Nov2017 1124, Guido van Rossum wrote:

A side note (I'm reading all responses but staying out of the discussion):

No static checker should depend on the *contents* of typing.py, since 
it's just a bunch of runtime gymnastics to allow types to be evaluated 
at runtime without errors, with a secondary goal of making them 
introspectable (some folks don't even agree with the latter, e.g. Mark 
Shannon).


Static analyzers should be able to make strong *assumptions* about what 
things defined there mean -- in mypy such assumptions are all over the 
place, based on the full name of things -- it never reads typing.py. (It 
reads typing.pyi from typeshed, but what's there is ignored in many 
cases too.)


Thank you. Very glad to hear I understood it correctly.

Cheers,
Steve


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Antoine Pitrou]

Le 03/11/2017 à 19:34, Stéfane Fermigier a écrit :
> 
> On Fri, Nov 3, 2017 at 6:47 PM, Antoine Pitrou  > wrote:
> 
> I don't think other modules should start relying on the typing module at
> runtime.
> 
> They already do, as I've noted in my message about dependency injection,
> and I find this quite elegant.

Third-party libraries do what they want, but we are talking about the
stdlib here.

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Guido van Rossum]

A side note (I'm reading all responses but staying out of the discussion):

No static checker should depend on the *contents* of typing.py, since it's
just a bunch of runtime gymnastics to allow types to be evaluated at
runtime without errors, with a secondary goal of making them introspectable
(some folks don't even agree with the latter, e.g. Mark Shannon).

Static analyzers should be able to make strong *assumptions* about what
things defined there mean -- in mypy such assumptions are all over the
place, based on the full name of things -- it never reads typing.py. (It
reads typing.pyi from typeshed, but what's there is ignored in many cases
too.)



On Fri, Nov 3, 2017 at 10:00 AM, Steve Dower  wrote:

> On 03Nov2017 0915, Victor Stinner wrote:
>
>> Hi,
>>
>> 2017-11-03 15:36 GMT+01:00 Guido van Rossum :
>>
>>> Maybe we should remove typing from the stdlib?
>>> https://github.com/python/typing/issues/495
>>>
>>
>> I'm strongly in favor on such move.
>>
>
> I'm torn.
>
> If not having typing installed means you can't use "Any" or "Optional" in
> an annotation, that basically kills the whole thing. Some primitives need
> to be there.
>
> If annotations become glorified strings (which IMHO they should) and
> typing gains a function to parse those into type hints (which IMHO it
> should), then I'm in favour of splitting typing out. (Personally, if making
> every type hint a literal 'string' meant that I could avoid importing
> typing then I'd do it.)
>
> However, if typing is split out then its API (specifically, the contents
> of typing.__all__ and their semantic meaning (e.g. "Iterable" means
> something with an "__iter__" method)) needs to stay in PEPs.
>
> Static analysers using type hints encode much more information about these
> types than can be inferred statically from typing.py, which means the
> definitions should not change faster than Python x.*y*. Ideally, they would
> not change at all once released.
>
> For example, my static analyser has an existing object representing
> iterables, since we've been inferring iterables for years. When I parse a
> type annotation and see "typing.Iterable", I'm going to just substitute my
> own implementation - the definition in typing.py isn't going to be used (or
> be useful). And because it has to map across languages, it has to be a
> hard-coded mapping that can't rely on typing.py at all.
>
> Since the contents of typing.py will likely be completely ignored by my
> analyser, which means that I can't treat "whatever version of typing is
> installed" as ground truth. It needs to move slower or be purely additive.
> Being in the standard library is a nice easy way to ensure this - moving it
> out is a risk.
>
> That said, because I don't care about the contents of the file, all the
> heavy execution stuff can totally be moved out. If typing in the stdlib
> became very trivial definitions or just a set of names to support
> searching/editors/forward-refs, and typing out of the stdlib had the
> ability to convert annotations into an object model that provides rich
> runtime introspection, I'd also be fine. At least then the interface is
> highly stable, even if the implementation (for those who use it) changes.
>
> Cheers,
> Steve
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
> 40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Antoine Pitrou]

Also, as for dataclasses specifically, since it may become as pervasive
as namedtuples, we probably want it to be as light-weight as possible.
Which will be harder to achieve if its *API* depends on the typing
machinery.

Regards

Antoine.


Le 03/11/2017 à 19:04, Paul Moore a écrit :
> On 3 November 2017 at 17:47, Antoine Pitrou  wrote:
>> On Fri, 3 Nov 2017 12:46:33 -0400
>> "Eric V. Smith"  wrote:
>>> On 11/3/2017 12:15 PM, Victor Stinner wrote:
 Hi,

 2017-11-03 15:36 GMT+01:00 Guido van Rossum :
> Maybe we should remove typing from the stdlib?
> https://github.com/python/typing/issues/495
>>>
 The typing module is not used yet in the stdlib, so there is no
 technically reason to keep typing part of the stdlib. IMHO it's
 perfectly fine to keep typing and annotations out of the stdlib, since
 the venv & pip tooling is now rock solid ;-)
>>>
>>> I'm planning on using it for PEP 557:
>>> https://www.python.org/dev/peps/pep-0557/#class-variables
>>>
>>> The way the code currently checks for this should still work if typing
>>> is not in the stdlib, although of course it's assuming that the name
>>> "typing" really is the "official" typing library.
>>
>> I don't think other modules should start relying on the typing module at
>> runtime.
>> The dataclasses module can define its own "ClassVar" thing and then I
>> suspect it's easy to map it to typing._ClassVar.  It seems we should be
>> careful not to blur the distinction between declarations that have an
>> effect on actual code, and typing declarations which only affect
>> type-checking tools.
> 
> I'm looking forward to the dataclasses module, and I'm perfectly OK
> with the way that it uses type annotations to declare attributes. I
> also don't have a problem with it relying on the typing module - but
> *only* if the typing module is in the stdlib. I don't think it's good
> if a standard feature needs an external library for some of its
> functionality.
> 
> So I guess the point is, if we're considering moving typing out of the
> stdlib, then what's the impact on PEP 557?
> 
> Personally, I don't use type annotations myself yet, but I've used
> code that does and I'm considering looking into them - for a variety
> of reasons, documentation, IDE support, and the ability to type check
> my code via mypy. If typing moves out of the stdlib, I'd be much less
> inclined to do so - adding a runtime dependency is a non-trivial cost
> in terms of admin for deployment, handling within my (peculiar, if you
> want to debate workflow) development workflow, etc. Working out how to
> add type annotations *without* them being a runtime dependency (just
> at test-time) is too much work. So I am concerned that if we move
> typing out of the stdlib, it'll reduce adoption rates.
> 
> Paul
> 
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Steve Dower]

On 03Nov2017 0915, Victor Stinner wrote:

Hi,

2017-11-03 15:36 GMT+01:00 Guido van Rossum :

Maybe we should remove typing from the stdlib?
https://github.com/python/typing/issues/495


I'm strongly in favor on such move.


I'm torn.

If not having typing installed means you can't use "Any" or "Optional" 
in an annotation, that basically kills the whole thing. Some primitives 
need to be there.


If annotations become glorified strings (which IMHO they should) and 
typing gains a function to parse those into type hints (which IMHO it 
should), then I'm in favour of splitting typing out. (Personally, if 
making every type hint a literal 'string' meant that I could avoid 
importing typing then I'd do it.)


However, if typing is split out then its API (specifically, the contents 
of typing.__all__ and their semantic meaning (e.g. "Iterable" means 
something with an "__iter__" method)) needs to stay in PEPs.


Static analysers using type hints encode much more information about 
these types than can be inferred statically from typing.py, which means 
the definitions should not change faster than Python x.*y*. Ideally, 
they would not change at all once released.


For example, my static analyser has an existing object representing 
iterables, since we've been inferring iterables for years. When I parse 
a type annotation and see "typing.Iterable", I'm going to just 
substitute my own implementation - the definition in typing.py isn't 
going to be used (or be useful). And because it has to map across 
languages, it has to be a hard-coded mapping that can't rely on 
typing.py at all.


Since the contents of typing.py will likely be completely ignored by my 
analyser, which means that I can't treat "whatever version of typing is 
installed" as ground truth. It needs to move slower or be purely 
additive. Being in the standard library is a nice easy way to ensure 
this - moving it out is a risk.


That said, because I don't care about the contents of the file, all the 
heavy execution stuff can totally be moved out. If typing in the stdlib 
became very trivial definitions or just a set of names to support 
searching/editors/forward-refs, and typing out of the stdlib had the 
ability to convert annotations into an object model that provides rich 
runtime introspection, I'd also be fine. At least then the interface is 
highly stable, even if the implementation (for those who use it) changes.


Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib (was: Reminder: 12 weeks to 3.7 feature code cutoff)

[Jelle Zijlstra]

2017-11-03 10:46 GMT-07:00 Brett Cannon :

>
>
> On Fri, 3 Nov 2017 at 08:09 Paul Moore  wrote:
>
>> On 3 November 2017 at 14:50, Serhiy Storchaka 
>> wrote:
>> > 03.11.17 16:36, Guido van Rossum пише:
>> >> Maybe we should remove typing from the stdlib?
>> >> https://github.com/python/typing/issues/495
>> >
>> > I didn't use typing, but AFAIK the most used feature from typing is
>> > NamedTuple. If move NamedTuple and maybe other convenient classes not
>> > directly related to typing into collections or types modules, I think
>> > removing typing from the stdlib will less stress people.
>>
>> (Checks docs) Hmm, I'd missed that this was even in there.
>>
>> Regardless of what happens with the typing module, I think this should
>> be moved. I expect there are many people who never looked at the docs
>> of the typing module because they don't use type annotations. I know
>> the class uses type annotations to define its attributes, but I don't
>> see that as an issue. Data classes do the same, and they won't be in
>> the typing module...
>>
>
> There is another option and that's splitting up the typing module into
> core, abstract things, and then the stuff that is about concrete types. For
> instance, ClassVar, Union, and cast() are all abstract concepts that are
> basic to type hints. But things like Mapping are more concrete and not a
> fundamental concept to type hints. You could argue that the fundamentals
> that won't change could stay in the stdlib while the concrete type classes
> could get pulled out so they can be managed more easily/quickly.
>

I don't think the fundamentals are less likely to change—if anything, it
may be the opposite. Many of the potential innovations we'd want to add to
typing (Protocols, literal types, intersection types) are fundamental to
the type system, and things like Mapping actually don't change very often.


>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> jelle.zijlstra%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Paul Moore]

On 3 November 2017 at 17:47, Antoine Pitrou  wrote:
> On Fri, 3 Nov 2017 12:46:33 -0400
> "Eric V. Smith"  wrote:
>> On 11/3/2017 12:15 PM, Victor Stinner wrote:
>> > Hi,
>> >
>> > 2017-11-03 15:36 GMT+01:00 Guido van Rossum :
>> >> Maybe we should remove typing from the stdlib?
>> >> https://github.com/python/typing/issues/495
>>
>> > The typing module is not used yet in the stdlib, so there is no
>> > technically reason to keep typing part of the stdlib. IMHO it's
>> > perfectly fine to keep typing and annotations out of the stdlib, since
>> > the venv & pip tooling is now rock solid ;-)
>>
>> I'm planning on using it for PEP 557:
>> https://www.python.org/dev/peps/pep-0557/#class-variables
>>
>> The way the code currently checks for this should still work if typing
>> is not in the stdlib, although of course it's assuming that the name
>> "typing" really is the "official" typing library.
>
> I don't think other modules should start relying on the typing module at
> runtime.
> The dataclasses module can define its own "ClassVar" thing and then I
> suspect it's easy to map it to typing._ClassVar.  It seems we should be
> careful not to blur the distinction between declarations that have an
> effect on actual code, and typing declarations which only affect
> type-checking tools.

I'm looking forward to the dataclasses module, and I'm perfectly OK
with the way that it uses type annotations to declare attributes. I
also don't have a problem with it relying on the typing module - but
*only* if the typing module is in the stdlib. I don't think it's good
if a standard feature needs an external library for some of its
functionality.

So I guess the point is, if we're considering moving typing out of the
stdlib, then what's the impact on PEP 557?

Personally, I don't use type annotations myself yet, but I've used
code that does and I'm considering looking into them - for a variety
of reasons, documentation, IDE support, and the ability to type check
my code via mypy. If typing moves out of the stdlib, I'd be much less
inclined to do so - adding a runtime dependency is a non-trivial cost
in terms of admin for deployment, handling within my (peculiar, if you
want to debate workflow) development workflow, etc. Working out how to
add type annotations *without* them being a runtime dependency (just
at test-time) is too much work. So I am concerned that if we move
typing out of the stdlib, it'll reduce adoption rates.

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Antoine Pitrou]

On Fri, 3 Nov 2017 12:46:33 -0400
"Eric V. Smith"  wrote:
> On 11/3/2017 12:15 PM, Victor Stinner wrote:
> > Hi,
> > 
> > 2017-11-03 15:36 GMT+01:00 Guido van Rossum :  
> >> Maybe we should remove typing from the stdlib?
> >> https://github.com/python/typing/issues/495  
> 
> > The typing module is not used yet in the stdlib, so there is no
> > technically reason to keep typing part of the stdlib. IMHO it's
> > perfectly fine to keep typing and annotations out of the stdlib, since
> > the venv & pip tooling is now rock solid ;-)  
> 
> I'm planning on using it for PEP 557:
> https://www.python.org/dev/peps/pep-0557/#class-variables
> 
> The way the code currently checks for this should still work if typing 
> is not in the stdlib, although of course it's assuming that the name 
> "typing" really is the "official" typing library.

I don't think other modules should start relying on the typing module at
runtime.
The dataclasses module can define its own "ClassVar" thing and then I
suspect it's easy to map it to typing._ClassVar.  It seems we should be
careful not to blur the distinction between declarations that have an
effect on actual code, and typing declarations which only affect
type-checking tools.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib (was: Reminder: 12 weeks to 3.7 feature code cutoff)

[Brett Cannon]

On Fri, 3 Nov 2017 at 08:09 Paul Moore  wrote:

> On 3 November 2017 at 14:50, Serhiy Storchaka  wrote:
> > 03.11.17 16:36, Guido van Rossum пише:
> >> Maybe we should remove typing from the stdlib?
> >> https://github.com/python/typing/issues/495
> >
> > I didn't use typing, but AFAIK the most used feature from typing is
> > NamedTuple. If move NamedTuple and maybe other convenient classes not
> > directly related to typing into collections or types modules, I think
> > removing typing from the stdlib will less stress people.
>
> (Checks docs) Hmm, I'd missed that this was even in there.
>
> Regardless of what happens with the typing module, I think this should
> be moved. I expect there are many people who never looked at the docs
> of the typing module because they don't use type annotations. I know
> the class uses type annotations to define its attributes, but I don't
> see that as an issue. Data classes do the same, and they won't be in
> the typing module...
>

There is another option and that's splitting up the typing module into
core, abstract things, and then the stuff that is about concrete types. For
instance, ClassVar, Union, and cast() are all abstract concepts that are
basic to type hints. But things like Mapping are more concrete and not a
fundamental concept to type hints. You could argue that the fundamentals
that won't change could stay in the stdlib while the concrete type classes
could get pulled out so they can be managed more easily/quickly.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Eric V. Smith]

On 11/3/2017 12:15 PM, Victor Stinner wrote:

Hi,

2017-11-03 15:36 GMT+01:00 Guido van Rossum :

Maybe we should remove typing from the stdlib?
https://github.com/python/typing/issues/495



The typing module is not used yet in the stdlib, so there is no
technically reason to keep typing part of the stdlib. IMHO it's
perfectly fine to keep typing and annotations out of the stdlib, since
the venv & pip tooling is now rock solid ;-)


I'm planning on using it for PEP 557:
https://www.python.org/dev/peps/pep-0557/#class-variables

The way the code currently checks for this should still work if typing 
is not in the stdlib, although of course it's assuming that the name 
"typing" really is the "official" typing library.


# If typing has not been imported, then it's impossible for
#  any annotation to be a ClassVar. So, only look for ClassVar
#  if typing has been imported.
typing = sys.modules.get('typing')
if typing is not None:
# This test uses a typing internal class, but it's the best
#  way to test if this is a ClassVar.
if type(a_type) is typing._ClassVar:
# This field is a ClassVar. Ignore it.
continue

See also https://github.com/ericvsmith/dataclasses/issues/14

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib (was: Reminder: 12 weeks to 3.7 feature code cutoff)

[Paul Moore]

On 3 November 2017 at 14:50, Serhiy Storchaka  wrote:
> 03.11.17 16:36, Guido van Rossum пише:
>> Maybe we should remove typing from the stdlib?
>> https://github.com/python/typing/issues/495
>
> I didn't use typing, but AFAIK the most used feature from typing is
> NamedTuple. If move NamedTuple and maybe other convenient classes not
> directly related to typing into collections or types modules, I think
> removing typing from the stdlib will less stress people.

(Checks docs) Hmm, I'd missed that this was even in there.

Regardless of what happens with the typing module, I think this should
be moved. I expect there are many people who never looked at the docs
of the typing module because they don't use type annotations. I know
the class uses type annotations to define its attributes, but I don't
see that as an issue. Data classes do the same, and they won't be in
the typing module...

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com