[Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Guido van Rossum]

[A copy from https://github.com/python/typing/issues/495 to get more
people's attention to this issue.]

I'm wondering if we should remove typing from the stdlib. Now's the time to
think about this, as the feature freeze for 3.7 is about 12 weeks away.

Cons:

   - People have to depend on a PyPI package to use typing (but they do
   anyway for typing_extensions)
   - It's a backward incompatibility for users of Python 3.5 and 3.6 (but
   the typing module was always provisional)

Pros:

   - The typing module can evolve much faster outside the stdlib
   - We could get rid of typing_extensions (and maybe even mypy_extensions)

If we don't do this I worry that we're entering a period where many new
typesystem features end up in typing_extensions and users will be confused
about which items are in typing and which in typing_extensions (not to
mention mypy_extensions). Anything new to be added to typing (e.g. Const,
Final, Literal, or changing ABCs to Protocols) would have to be added to
typing_extensions instead, and users would be confused about which features
exist in which module. Moving typing out of the stdlib can make things
potentially simpler, at the cost of an extra pip install (but they'll need
one anyway for mypy).

Thoughts?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Chris Angelico]

On Sat, Nov 4, 2017 at 1:35 AM, Guido van Rossum  wrote:
> [A copy from https://github.com/python/typing/issues/495 to get more
> people's attention to this issue.]
>
> I'm wondering if we should remove typing from the stdlib. Now's the time to
> think about this, as the feature freeze for 3.7 is about 12 weeks away.
>
> Cons:
>
> People have to depend on a PyPI package to use typing (but they do anyway
> for typing_extensions)

If the lazy evaluation of annotations (PEP 563) also lands in 3.7,
then this would be a very minor downside. You'd need to pip-install
typing as well as mypy *for the actual type checking*, but at run
time, you could ignore both (all those List[...] annotations would be
stored unevaluated). Otherwise, it'd mean that any project that makes
use of type hints would require typing as a run-time dependency.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Guido van Rossum]

On Fri, Nov 3, 2017 at 7:45 AM, Chris Angelico  wrote:

> On Sat, Nov 4, 2017 at 1:35 AM, Guido van Rossum  wrote:
> > [A copy from https://github.com/python/typing/issues/495 to get more
> > people's attention to this issue.]
> >
> > I'm wondering if we should remove typing from the stdlib. Now's the time
> to
> > think about this, as the feature freeze for 3.7 is about 12 weeks away.
> >
> > Cons:
> >
> > People have to depend on a PyPI package to use typing (but they do anyway
> > for typing_extensions)
>
> If the lazy evaluation of annotations (PEP 563) also lands in 3.7,
> then this would be a very minor downside. You'd need to pip-install
> typing as well as mypy *for the actual type checking*, but at run
> time, you could ignore both (all those List[...] annotations would be
> stored unevaluated). Otherwise, it'd mean that any project that makes
> use of type hints would require typing as a run-time dependency.
>

This would not work if you use TypeVar, NewType, or any kind of type alias
involving things imported from typing (e.g. Union or TypedDict). Also
cast().

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Chris Angelico]

On Sat, Nov 4, 2017 at 1:49 AM, Guido van Rossum  wrote:
> On Fri, Nov 3, 2017 at 7:45 AM, Chris Angelico  wrote:
>>
>> On Sat, Nov 4, 2017 at 1:35 AM, Guido van Rossum  wrote:
>> > [A copy from https://github.com/python/typing/issues/495 to get more
>> > people's attention to this issue.]
>> >
>> > I'm wondering if we should remove typing from the stdlib. Now's the time
>> > to
>> > think about this, as the feature freeze for 3.7 is about 12 weeks away.
>> >
>> > Cons:
>> >
>> > People have to depend on a PyPI package to use typing (but they do
>> > anyway
>> > for typing_extensions)
>>
>> If the lazy evaluation of annotations (PEP 563) also lands in 3.7,
>> then this would be a very minor downside. You'd need to pip-install
>> typing as well as mypy *for the actual type checking*, but at run
>> time, you could ignore both (all those List[...] annotations would be
>> stored unevaluated). Otherwise, it'd mean that any project that makes
>> use of type hints would require typing as a run-time dependency.
>
>
> This would not work if you use TypeVar, NewType, or any kind of type alias
> involving things imported from typing (e.g. Union or TypedDict). Also
> cast().

Ah, I forgot about those - they're not just used in the annotations.
Oh well, was a nice idea.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Stéfane Fermigier]

I use typing quite systematically nowadays, even for projects that don't
use mypy (for historical reasons: bringing an older code base to zero mypy
issues can be quite time-consuming).

For instance, adding typing annotation can help autocompletion under
PyCharm (and hopefully other IDEs).

With these annotations, PyCharm is also able to signal typing issues either
directly in the editor, or when running a code check.

I'm quite OK with removing the typing module from the stdlib as it can
easily be added to my projects dependencies, and I can definitively
understand the benefits of a faster release cycle, but I'm worried that
this could hinder adoption of these practices by certain people.

  S.

On Fri, Nov 3, 2017 at 3:35 PM, Guido van Rossum  wrote:

> [A copy from https://github.com/python/typing/issues/495 to get more
> people's attention to this issue.]
>
> I'm wondering if we should remove typing from the stdlib. Now's the time
> to think about this, as the feature freeze for 3.7 is about 12 weeks away.
>
> Cons:
>
>- People have to depend on a PyPI package to use typing (but they do
>anyway for typing_extensions)
>- It's a backward incompatibility for users of Python 3.5 and 3.6 (but
>the typing module was always provisional)
>
> Pros:
>
>- The typing module can evolve much faster outside the stdlib
>- We could get rid of typing_extensions (and maybe even mypy_extensions
>)
>
> If we don't do this I worry that we're entering a period where many new
> typesystem features end up in typing_extensions and users will be
> confused about which items are in typing and which in typing_extensions
> (not to mention mypy_extensions). Anything new to be added to typing
> (e.g. Const, Final, Literal, or changing ABCs to Protocols) would have to
> be added to typing_extensions instead, and users would be confused about
> which features exist in which module. Moving typing out of the stdlib can
> make things potentially simpler, at the cost of an extra pip install (but
> they'll need one anyway for mypy).
>
> Thoughts?
>
> --
> --Guido van Rossum (python.org/~guido)
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>


-- 
Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier -
http://linkedin.com/in/sfermigier
Founder & CEO, Abilian - Enterprise Social Software -
http://www.abilian.com/
Chairman, Free&OSS Group / Systematic Cluster -
http://www.gt-logiciel-libre.org/
Co-Chairman, National Council for Free & Open Source Software (CNLL) -
http://cnll.fr/
Founder & Organiser, PyData Paris - http://pydata.fr/
---
“You never change things by fighting the existing reality. To change
something, build a new model that makes the existing model obsolete.” — R.
Buckminster Fuller
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Antoine Pitrou]

On Fri, 3 Nov 2017 16:04:43 +0100
Stéfane Fermigier  wrote:
> I use typing quite systematically nowadays, even for projects that don't
> use mypy (for historical reasons: bringing an older code base to zero mypy
> issues can be quite time-consuming).
> 
> For instance, adding typing annotation can help autocompletion under
> PyCharm (and hopefully other IDEs).
> 
> With these annotations, PyCharm is also able to signal typing issues either
> directly in the editor, or when running a code check.
> 
> I'm quite OK with removing the typing module from the stdlib as it can
> easily be added to my projects dependencies, and I can definitively
> understand the benefits of a faster release cycle, but I'm worried that
> this could hinder adoption of these practices by certain people.

I don't think casual or beginner users of Python really have to worry
about typing annotations.  As I understand it, they become really
useful on middle- to large-scale projects (disclaimer: I've never used
them myself; the kind of typing the Numba project does -- which I
don't participate in anymore -- is quite different).

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Nick Coghlan]

On 4 November 2017 at 00:35, Guido van Rossum  wrote:
> [A copy from https://github.com/python/typing/issues/495 to get more
> people's attention to this issue.]
>
> I'm wondering if we should remove typing from the stdlib. Now's the time to
> think about this, as the feature freeze for 3.7 is about 12 weeks away.
>
> Cons:
>
> People have to depend on a PyPI package to use typing (but they do anyway
> for typing_extensions)
> It's a backward incompatibility for users of Python 3.5 and 3.6 (but the
> typing module was always provisional)
>
> Pros:
>
> The typing module can evolve much faster outside the stdlib
> We could get rid of typing_extensions (and maybe even mypy_extensions)
>
> If we don't do this I worry that we're entering a period where many new
> typesystem features end up in typing_extensions and users will be confused
> about which items are in typing and which in typing_extensions (not to
> mention mypy_extensions). Anything new to be added to typing (e.g. Const,
> Final, Literal, or changing ABCs to Protocols) would have to be added to
> typing_extensions instead, and users would be confused about which features
> exist in which module. Moving typing out of the stdlib can make things
> potentially simpler, at the cost of an extra pip install (but they'll need
> one anyway for mypy).
>
> Thoughts?

Perhaps typing could switch to being a bundled module, such that it
had its own version, independent of the Python standard library
version, but was still present by default in new installations?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Michel Desmoulin]

I could see the typing module staying but:

- typing_extension (or the new official external module) append types in
the typing module;
- if typing_extension is not installed or too old and missing some
types, any missing type being accessed returns some kind of mock object
that avoid crashing;
- any running type checker encountering mocked types has to show a warning
- you can pip install typing (and it's mocking abilities) in python 2.

This way somebody without the 3rd party module can still use the code
without crashing it. Having types out of date or missing a brand new one
won't crash the code, only have a warning in mypy or pycharm.

This solves the compatibility problem, but also another major problem
currently: several projects I worked on had a fake typing module I could
use it in python 2 and 3 without having to manually install typing.
Sometime I also create scripts and leave type annotations, and my users
really don't want to install the exact typing extension to make it work.
And I don't want to remove the annotations manually after ward.

Le 03/11/2017 à 15:51, Chris Angelico a écrit :
> On Sat, Nov 4, 2017 at 1:49 AM, Guido van Rossum  wrote:
>> On Fri, Nov 3, 2017 at 7:45 AM, Chris Angelico  wrote:
>>>
>>> On Sat, Nov 4, 2017 at 1:35 AM, Guido van Rossum  wrote:
 [A copy from https://github.com/python/typing/issues/495 to get more
 people's attention to this issue.]

 I'm wondering if we should remove typing from the stdlib. Now's the time
 to
 think about this, as the feature freeze for 3.7 is about 12 weeks away.

 Cons:

 People have to depend on a PyPI package to use typing (but they do
 anyway
 for typing_extensions)
>>>
>>> If the lazy evaluation of annotations (PEP 563) also lands in 3.7,
>>> then this would be a very minor downside. You'd need to pip-install
>>> typing as well as mypy *for the actual type checking*, but at run
>>> time, you could ignore both (all those List[...] annotations would be
>>> stored unevaluated). Otherwise, it'd mean that any project that makes
>>> use of type hints would require typing as a run-time dependency.
>>
>>
>> This would not work if you use TypeVar, NewType, or any kind of type alias
>> involving things imported from typing (e.g. Union or TypedDict). Also
>> cast().
> 
> Ah, I forgot about those - they're not just used in the annotations.
> Oh well, was a nice idea.
> 
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Guido van Rossum]

On Sat, Nov 4, 2017 at 7:05 AM, Nick Coghlan  wrote:

> Perhaps typing could switch to being a bundled module, such that it
> had its own version, independent of the Python standard library
> version, but was still present by default in new installations?
>

This is beginning to sound like the most attractive solution. We could
possibly do away with typing_extensions. Are there precedents of how to
bundle a module in this way? Or is it going to be another special case like
pip?

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Nick Coghlan]

On 5 November 2017 at 06:22, Guido van Rossum  wrote:
> On Sat, Nov 4, 2017 at 7:05 AM, Nick Coghlan  wrote:
>>
>> Perhaps typing could switch to being a bundled module, such that it
>> had its own version, independent of the Python standard library
>> version, but was still present by default in new installations?
>
>
> This is beginning to sound like the most attractive solution. We could
> possibly do away with typing_extensions. Are there precedents of how to
> bundle a module in this way? Or is it going to be another special case like
> pip?

You'd be blazing new trails, but you'd be blazing them with a
single-file pure Python module without any binary dependencies outside
CPython itself, which is the simplest possible precedent setting
scenario we could hope for.

I can think of a couple of possible approaches you could take,
depending on how you want to manage development on the typing module
itself.

= External dev =

* Typing moves entirely out of the CPython source tree, with a
separate issue tracker, repository, branch structure, etc (which you
already have)
* We add a "_bundled" directory to the CPython source tree in 3.7+ and
put a typing sdist archive there
* the release process PEP gains a "Confirm the bundled modules are up
to date" step
* the CPython build process is updated to use a venv to generate wheel
files from bundled sdists
* regrtest gains a "bundled" resource (and/or dedicated command line option)
* when asked to test bundled modules, regrtest spins up a test venv,
installs pip and the bundled modules, then runs the tests for those
modules
* ensurepip gains the ability to also install bundled wheel files
* the Windows and Mac OS X installers start including the typing wheel file

= In tree dev =

* Similar to external dev for testing and distribution
* Source code would stay in the main CPython tree and track CPython
branch structure
* The sdist file would be built by CPython's build process, rather
than being checked in

The external dev model is probably a better fit for your use case,
though, since you'd like something that you can readily keep
consistent across all supported 3.x versions (or at least 3.7+), and
that's harder to do when you have multiple copies of the source code
to keep in sync (vs having a single development branch where you run
your CI testing across all supported Python versions).

The external dev option also establishes a more useful precedent,
since we could use it to officially migrate ongoing distutils
maintenance into the setuptools project, and then bring it back via
bundling of setuptools. Similarly, PEP 434 (which already exempts IDLE
from the usual "No new features in maintenance releases" rules) could
potentially be switched over to handling IDLE as an independently
updated bundled application.

The above idea is somewhat similar to what I suggested for the
recommended modules list in
https://mail.python.org/pipermail/python-ideas/2017-October/047508.html,
but the difference is that to simplify the testing and installer
building process, we'd bundle a suitable source archive in the git
repo, rather than requiring PyPI access.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Antoine Pitrou]

On Sun, 5 Nov 2017 13:46:59 +1000
Nick Coghlan  wrote:
> * ensurepip gains the ability to also install bundled wheel files

Why? Why wouldn't you put the wheel directly in site-packages on
install?

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Paul Moore]

On 5 November 2017 at 10:48, Antoine Pitrou  wrote:
> On Sun, 5 Nov 2017 13:46:59 +1000
> Nick Coghlan  wrote:
>> * ensurepip gains the ability to also install bundled wheel files
>
> Why? Why wouldn't you put the wheel directly in site-packages on
> install?

I'm not quite sure what you mean? It needs to be "installed", in the
sense of being unpacked into site-packages, and the ensurepip
mechanism is already able to do that for pip and setuptools, so adding
an extra wheel to install wouldn't be too hard. If we don't install
like that, people won't be able to easily upgrade typing by just using
"pip install --upgrade typing".

Wheels don't support simply being added to sys.path the way that eggs
did, if that's what you mean.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Nick Coghlan]

On 5 November 2017 at 23:30, Paul Moore  wrote:
> On 5 November 2017 at 10:48, Antoine Pitrou  wrote:
>> On Sun, 5 Nov 2017 13:46:59 +1000
>> Nick Coghlan  wrote:
>>> * ensurepip gains the ability to also install bundled wheel files
>>
>> Why? Why wouldn't you put the wheel directly in site-packages on
>> install?
>
> I'm not quite sure what you mean? It needs to be "installed", in the
> sense of being unpacked into site-packages, and the ensurepip
> mechanism is already able to do that for pip and setuptools, so adding
> an extra wheel to install wouldn't be too hard. If we don't install
> like that, people won't be able to easily upgrade typing by just using
> "pip install --upgrade typing".
>
> Wheels don't support simply being added to sys.path the way that eggs
> did, if that's what you mean.

What Paul said here. While wheels *can* be designed to support
use-without-extraction (and pip's own wheel file is constructed that
way so you can use a pip wheel to install itself), the general design
principle is that we expect them to be installed prior to use, such
that they get proper PEP 376 metadata entries, and so that their
subcomponents end up in the right sysconfig directories for the
deployment environment.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Antoine Pitrou]

Le 05/11/2017 à 14:30, Paul Moore a écrit :
> On 5 November 2017 at 10:48, Antoine Pitrou  wrote:
>> On Sun, 5 Nov 2017 13:46:59 +1000
>> Nick Coghlan  wrote:
>>> * ensurepip gains the ability to also install bundled wheel files
>>
>> Why? Why wouldn't you put the wheel directly in site-packages on
>> install?
> 
> I'm not quite sure what you mean? It needs to be "installed", in the
> sense of being unpacked into site-packages, and the ensurepip
> mechanism is already able to do that for pip and setuptools, so adding
> an extra wheel to install wouldn't be too hard.

Ok, perhaps my question wasn't quite clear.  Are you suggesting that
people have to run "python -m ensurepip typing" after they installed
Python?  Or would the typing module be importable as soon as you have an
installed Python, like stdlib modules are?

Regards

Antoine.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Paul Moore]

On 5 November 2017 at 14:47, Antoine Pitrou  wrote:
>
> Le 05/11/2017 à 14:30, Paul Moore a écrit :
>> On 5 November 2017 at 10:48, Antoine Pitrou  wrote:
>>> On Sun, 5 Nov 2017 13:46:59 +1000
>>> Nick Coghlan  wrote:
 * ensurepip gains the ability to also install bundled wheel files
>>>
>>> Why? Why wouldn't you put the wheel directly in site-packages on
>>> install?
>>
>> I'm not quite sure what you mean? It needs to be "installed", in the
>> sense of being unpacked into site-packages, and the ensurepip
>> mechanism is already able to do that for pip and setuptools, so adding
>> an extra wheel to install wouldn't be too hard.
>
> Ok, perhaps my question wasn't quite clear.  Are you suggesting that
> people have to run "python -m ensurepip typing" after they installed
> Python?  Or would the typing module be importable as soon as you have an
> installed Python, like stdlib modules are?

Ah, I get you now. I'd expect typing to be available exactly the same
way as pip is. For me (on Windows) that means that it's available in a
standard install. On Unix, I don't know (I think there's certain
situations where you need to take extra steps in a custom build to
ensure pip is available? I'd expect those steps to also install
typing.

So basically, yes, typing can be expected to be available in all
installations, exactly the same as pip is.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Antoine Pitrou]

On Sun, 5 Nov 2017 18:22:11 +
Paul Moore  wrote:

> On 5 November 2017 at 14:47, Antoine Pitrou  wrote:
> >
> > Le 05/11/2017 à 14:30, Paul Moore a écrit :  
> >> On 5 November 2017 at 10:48, Antoine Pitrou  wrote:  
> >>> On Sun, 5 Nov 2017 13:46:59 +1000
> >>> Nick Coghlan  wrote:  
>  * ensurepip gains the ability to also install bundled wheel files  
> >>>
> >>> Why? Why wouldn't you put the wheel directly in site-packages on
> >>> install?  
> >>
> >> I'm not quite sure what you mean? It needs to be "installed", in the
> >> sense of being unpacked into site-packages, and the ensurepip
> >> mechanism is already able to do that for pip and setuptools, so adding
> >> an extra wheel to install wouldn't be too hard.  
> >
> > Ok, perhaps my question wasn't quite clear.  Are you suggesting that
> > people have to run "python -m ensurepip typing" after they installed
> > Python?  Or would the typing module be importable as soon as you have an
> > installed Python, like stdlib modules are?  
> 
> Ah, I get you now. I'd expect typing to be available exactly the same
> way as pip is. For me (on Windows) that means that it's available in a
> standard install. On Unix, I don't know (I think there's certain
> situations where you need to take extra steps in a custom build to
> ensure pip is available? I'd expect those steps to also install
> typing.

I think typing shouldn't require any extra typing (ha) on Unix either.
I don't remember what the rationale was for having to type
"python -m ensurepip" to get pip installed, but typing is just a
library, not an executable tool that may be able to mess with the
system state, so I don't think it's worthwhile introducing an extra
step for it.

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Paul Moore]

On 5 November 2017 at 18:40, Antoine Pitrou  wrote:
> I think typing shouldn't require any extra typing (ha) on Unix either.
> I don't remember what the rationale was for having to type
> "python -m ensurepip" to get pip installed, but typing is just a
> library, not an executable tool that may be able to mess with the
> system state, so I don't think it's worthwhile introducing an extra
> step for it.

Yes, I agree. I didn't realise that "give me pip" was an extra step on
Unix. I don't know what the arguments for doing it like that on Unix
were, but they certainly don't seem to apply to typing.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Barry Scott]

On Saturday, 4 November 2017 20:22:25 GMT Guido van Rossum wrote:
> On Sat, Nov 4, 2017 at 7:05 AM, Nick Coghlan  wrote:
> > Perhaps typing could switch to being a bundled module, such that it
> > had its own version, independent of the Python standard library
> > version, but was still present by default in new installations?
> 
> This is beginning to sound like the most attractive solution. We could
> possibly do away with typing_extensions. Are there precedents of how to
> bundle a module in this way? Or is it going to be another special case like
> pip?

Is the outcome you want that you ship a version of typing with the python kit,
but if you install from pip it overrides the one shipped in the python kit?

That would be a matter of being having a suitable sys.path/site config I 
guess. pip folder before the bundled packages folder.

If this is a mechanism that python kitting has then you would be able to
bundle other packages like requests or six as well as typing, but because
you can use pip to override the one shipped a user can optionally keep
up with the latest versions.

Barry

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Chris Angelico]

On Mon, Nov 6, 2017 at 9:29 AM, Barry Scott  wrote:
> On Saturday, 4 November 2017 20:22:25 GMT Guido van Rossum wrote:
>> On Sat, Nov 4, 2017 at 7:05 AM, Nick Coghlan  wrote:
>> > Perhaps typing could switch to being a bundled module, such that it
>> > had its own version, independent of the Python standard library
>> > version, but was still present by default in new installations?
>>
>> This is beginning to sound like the most attractive solution. We could
>> possibly do away with typing_extensions. Are there precedents of how to
>> bundle a module in this way? Or is it going to be another special case like
>> pip?
>
> Is the outcome you want that you ship a version of typing with the python kit,
> but if you install from pip it overrides the one shipped in the python kit?
>
> That would be a matter of being having a suitable sys.path/site config I
> guess. pip folder before the bundled packages folder.
>
> If this is a mechanism that python kitting has then you would be able to
> bundle other packages like requests or six as well as typing, but because
> you can use pip to override the one shipped a user can optionally keep
> up with the latest versions.

If this were to happen, I would be inclined to put these "bootstrap"
modules into their own directory in sys.path, after the rest of the
stdlib. Then someone who's paranoid about stdlib shadowing could put
pip-installed modules after the bulk of the stdlib (thus preventing
any third-party package from overriding "import random", for instance)
but still update modules that are specifically intended for updating;
plus it'd mean you can get a directory listing of that, and go grab
all the "blessed by python.org as an extension of the stdlib"
packages.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Nick Coghlan]

On 6 November 2017 at 09:08, Chris Angelico  wrote:
> On Mon, Nov 6, 2017 at 9:29 AM, Barry Scott  wrote:
>> If this is a mechanism that python kitting has then you would be able to
>> bundle other packages like requests or six as well as typing, but because
>> you can use pip to override the one shipped a user can optionally keep
>> up with the latest versions.
>
> If this were to happen, I would be inclined to put these "bootstrap"
> modules into their own directory in sys.path, after the rest of the
> stdlib. Then someone who's paranoid about stdlib shadowing could put
> pip-installed modules after the bulk of the stdlib (thus preventing
> any third-party package from overriding "import random", for instance)
> but still update modules that are specifically intended for updating;
> plus it'd mean you can get a directory listing of that, and go grab
> all the "blessed by python.org as an extension of the stdlib"
> packages.

When we say bundled, we mean bundled: the exact same bits you would
get from PyPI, installed the same way, and if you upgrade it system
wide, there's no trace of the one we bundled.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Nick Coghlan]

On 6 November 2017 at 05:00, Paul Moore  wrote:
> On 5 November 2017 at 18:40, Antoine Pitrou  wrote:
>> I think typing shouldn't require any extra typing (ha) on Unix either.
>> I don't remember what the rationale was for having to type
>> "python -m ensurepip" to get pip installed, but typing is just a
>> library, not an executable tool that may be able to mess with the
>> system state, so I don't think it's worthwhile introducing an extra
>> step for it.
>
> Yes, I agree. I didn't realise that "give me pip" was an extra step on
> Unix. I don't know what the arguments for doing it like that on Unix
> were, but they certainly don't seem to apply to typing.

It's the default on Unix as well - you have to do "make install
ENSUREPIP=no" to avoid getting it. (And some distros also modify their
Python installations so that pip is missing by default)

We provide "python -m ensurepip" because we offer ways to skip doing
it at install time (and venv creation time), so the explicit command
gives folks a straightforward way to change their minds later.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Chris Angelico]

On Mon, Nov 6, 2017 at 5:06 PM, Nick Coghlan  wrote:
> On 6 November 2017 at 09:08, Chris Angelico  wrote:
>> On Mon, Nov 6, 2017 at 9:29 AM, Barry Scott  wrote:
>>> If this is a mechanism that python kitting has then you would be able to
>>> bundle other packages like requests or six as well as typing, but because
>>> you can use pip to override the one shipped a user can optionally keep
>>> up with the latest versions.
>>
>> If this were to happen, I would be inclined to put these "bootstrap"
>> modules into their own directory in sys.path, after the rest of the
>> stdlib. Then someone who's paranoid about stdlib shadowing could put
>> pip-installed modules after the bulk of the stdlib (thus preventing
>> any third-party package from overriding "import random", for instance)
>> but still update modules that are specifically intended for updating;
>> plus it'd mean you can get a directory listing of that, and go grab
>> all the "blessed by python.org as an extension of the stdlib"
>> packages.
>
> When we say bundled, we mean bundled: the exact same bits you would
> get from PyPI, installed the same way, and if you upgrade it system
> wide, there's no trace of the one we bundled.

Oh, that's even better! :+1:

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Michel Desmoulin]

Le 06/11/2017 à 07:07, Nick Coghlan a écrit :

> It's the default on Unix as well - you have to do "make install
> ENSUREPIP=no" to avoid getting it. (And some distros also modify their
> Python installations so that pip is missing by default)

On debian and derivatives (so Ubuntu) you need to install python-pip to
be able to use pip.

Now it's annoying already. Because you have to write every tutorial to
include a special case for them. But at least it's not a required step
to run your program.

However, if you do code using type hints and typing is not installed,
you can't even run the program without installing something. So we
finally have Python 3 by default on most Linux system, but still would
not be able to assume we can run a modern script on it.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Lukasz Langa]

> On 5 Nov, 2017, at 10:30 PM, Michel Desmoulin  
> wrote:
> 
> Le 06/11/2017 à 07:07, Nick Coghlan a écrit :
> 
>> It's the default on Unix as well - you have to do "make install
>> ENSUREPIP=no" to avoid getting it. (And some distros also modify their
>> Python installations so that pip is missing by default)
> 
> On debian and derivatives (so Ubuntu) you need to install python-pip to
> be able to use pip.
> 
> Now it's annoying already. Because you have to write every tutorial to
> include a special case for them. But at least it's not a required step
> to run your program.
> 
> However, if you do code using type hints and typing is not installed,
> you can't even run the program without installing something. So we
> finally have Python 3 by default on most Linux system, but still would
> not be able to assume we can run a modern script on it.

This is a valid concern. Although this particular problem is self-inflicted by 
Debian, I can understand their rationale behind explicit packaging. They need 
to have control over the entire package graph. I wonder if there's a way in 
.deb to specify a required installed package. I'm not calling it a "dependency" 
since obviously it would rather be "python3-typing" that depends on "python3". 
Barry, do you know?

But even if Debian installs python3-typing by default, will "pip install -U 
typing" be possible in this scenario? I guess it wouldn't be terrible if that 
only worked in virtualenvs, although ideally it would work also for the raw 
host installation.

- Ł


signature.asc
Description: Message signed with OpenPGP
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Nick Coghlan]

On 6 November 2017 at 16:42, Lukasz Langa  wrote:
>
>> On 5 Nov, 2017, at 10:30 PM, Michel Desmoulin  
>> wrote:
>>
>> Le 06/11/2017 à 07:07, Nick Coghlan a écrit :
>>
>>> It's the default on Unix as well - you have to do "make install
>>> ENSUREPIP=no" to avoid getting it. (And some distros also modify their
>>> Python installations so that pip is missing by default)
>>
>> On debian and derivatives (so Ubuntu) you need to install python-pip to
>> be able to use pip.
>>
>> Now it's annoying already. Because you have to write every tutorial to
>> include a special case for them. But at least it's not a required step
>> to run your program.
>>
>> However, if you do code using type hints and typing is not installed,
>> you can't even run the program without installing something. So we
>> finally have Python 3 by default on most Linux system, but still would
>> not be able to assume we can run a modern script on it.
>
> This is a valid concern. Although this particular problem is self-inflicted 
> by Debian, I can understand their rationale behind explicit packaging. They 
> need to have control over the entire package graph. I wonder if there's a way 
> in .deb to specify a required installed package. I'm not calling it a 
> "dependency" since obviously it would rather be "python3-typing" that depends 
> on "python3".

Fedora just lives with the circular dependency between python3 and
python3-pip, which ensures both are installed by default (this
arrangement allows "python3 -m venv" to still install pip, without
actually needing a second copy of pip inside the python3 package)

A bundled typing module that "python -m venv" installed by default
would probably require similar treatment.

> But even if Debian installs python3-typing by default, will "pip install -U 
> typing" be possible in this scenario? I guess it wouldn't be terrible if that 
> only worked in virtualenvs, although ideally it would work also for the raw 
> host installation.

"sudo pip install " remains a terrible idea on any distro,
because it leads to pip and the system package manager getting into a
fight over which tool is responsible for managing the affected files.

"pip install --user --upgrade typing" should work OK, though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Paul Moore]

On 6 November 2017 at 06:42, Lukasz Langa  wrote:
>> Now it's annoying already. Because you have to write every tutorial to
>> include a special case for them. But at least it's not a required step
>> to run your program.
>>
>> However, if you do code using type hints and typing is not installed,
>> you can't even run the program without installing something. So we
>> finally have Python 3 by default on most Linux system, but still would
>> not be able to assume we can run a modern script on it.
>
> This is a valid concern. Although this particular problem is self-inflicted 
> by Debian, I can understand their rationale behind explicit packaging. They 
> need to have control over the entire package graph.

It's also a problem for virtual environments, which won't include
typing by default, and don't see the system installed packages. As
Nick says, we could modify venv to always include typing, but most
users, and all tools (tox, pew, pipenv, etc) use virtualenv rather
than venv still, so this would be of very limited benefit.

> I wonder if there's a way in .deb to specify a required installed package. 
> I'm not calling it a "dependency" since obviously it would rather be 
> "python3-typing" that depends on "python3". Barry, do you know?

There's always "include typing in the standard library" :-)

> But even if Debian installs python3-typing by default, will "pip install -U 
> typing" be possible in this scenario? I guess it wouldn't be terrible if that 
> only worked in virtualenvs, although ideally it would work also for the raw 
> host installation.

It would push yet more people into doing "sudo pip install -U typing",
which is something we've been trying really hard to encourage them to
stop doing over on the pip issues list. We strongly recommend using
system supplied packages, but I'd be surprised if Debian is any more
able to keep up with the latest versions of typing than Python core
is.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Antoine Pitrou]

On Mon, 6 Nov 2017 07:30:35 +0100
Michel Desmoulin
 wrote:
> Le 06/11/2017 à 07:07, Nick Coghlan a écrit :
> 
> > It's the default on Unix as well - you have to do "make install
> > ENSUREPIP=no" to avoid getting it. (And some distros also modify their
> > Python installations so that pip is missing by default)  
> 
> On debian and derivatives (so Ubuntu) you need to install python-pip to
> be able to use pip.

I suspect they do this because they want to steer people into trying to
`apt install` their Python dependencies first.

They have no reason to do any such thing for typing, IMO.

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Antoine Pitrou]

On Mon, 6 Nov 2017 16:07:56 +1000
Nick Coghlan  wrote:
> On 6 November 2017 at 05:00, Paul Moore  wrote:
> > On 5 November 2017 at 18:40, Antoine Pitrou  wrote:  
> >> I think typing shouldn't require any extra typing (ha) on Unix either.
> >> I don't remember what the rationale was for having to type
> >> "python -m ensurepip" to get pip installed, but typing is just a
> >> library, not an executable tool that may be able to mess with the
> >> system state, so I don't think it's worthwhile introducing an extra
> >> step for it.  
> >
> > Yes, I agree. I didn't realise that "give me pip" was an extra step on
> > Unix. I don't know what the arguments for doing it like that on Unix
> > were, but they certainly don't seem to apply to typing.  
> 
> It's the default on Unix as well - you have to do "make install
> ENSUREPIP=no" to avoid getting it. (And some distros also modify their
> Python installations so that pip is missing by default)

Oh, thank you.  I stand corrected then :-)

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Gyro Funch]

On 11/4/2017 2:22 PM, Guido van Rossum wrote:
> On Sat, Nov 4, 2017 at 7:05 AM, Nick Coghlan
>  > wrote:
> 
> Perhaps typing could switch to being a bundled module, such that it
> had its own version, independent of the Python standard library
> version, but was still present by default in new installations?
> 
> 
> This is beginning to sound like the most attractive solution. We
> could possibly do away with typing_extensions. Are there precedents
> of how to bundle a module in this way? Or is it going to be another
> special case like pip?
> 
> -- 
> --Guido van Rossum (python.org/~guido )
> 


Would this mean that other packages in the stdlib with development
cycles faster than those of python could use the same bundling
mechanism?

-gyro

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[brent bejot]

On Mon, Nov 6, 2017 at 9:21 AM, Gyro Funch  wrote:

> On 11/4/2017 2:22 PM, Guido van Rossum wrote:
> > On Sat, Nov 4, 2017 at 7:05 AM, Nick Coghlan
> >  > > wrote:
> >
> > Perhaps typing could switch to being a bundled module, such that it
> > had its own version, independent of the Python standard library
> > version, but was still present by default in new installations?
> >
> >
> > This is beginning to sound like the most attractive solution. We
> > could possibly do away with typing_extensions. Are there precedents
> > of how to bundle a module in this way? Or is it going to be another
> > special case like pip?
> >
> > --
> > --Guido van Rossum (python.org/~guido )
> >
>
>
> Would this mean that other packages in the stdlib with development
> cycles faster than those of python could use the same bundling
> mechanism?
>

This is pretty much how I thought things worked for all built-in packages
until this thread came up.  Is there any reason to not do this for all of
stdlib?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Ethan Furman]

On 11/06/2017 06:48 AM, brent bejot wrote:

On Mon, Nov 6, 2017 at 9:21 AM, Gyro Funch wrote:



Would this mean that other packages in the stdlib with development
cycles faster than those of python could use the same bundling
mechanism?


This is pretty much how I thought things worked for all built-in packages until 
this thread came up.  Is there any
reason to not do this for all of stdlib?


Yes.  Stability is probably the first (knowing what is support in a particular version).  Second would be that nearly 
all modules in the stdlib are stable and don't need frequent updates -- in fact, I suspect typing is the only exception.


--
~Ethan~

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Michel Desmoulin]

> 
> 
> This is pretty much how I thought things worked for all built-in
> packages until this thread came up.  Is there any reason to not do this
> for all of stdlib?
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

Yes: https://www.python.org/dev/peps/pep-0206/#batteries-included-philosophy

I can't find it but there is a great story about a guy needing to do a
source code analysis in total isolation (not even a usb key or a cd) and
the computer has nothing installed but Python.

He gets by because Python is very useful "as is".

I can say the same.

When I log in on bare debian / centos or if I'm on a locked mac os. When
i'm on a place or in a remote place in Africa. When I'm with customers
locking down all the network with NDA, security badge and paranoia for
desert...

I can count on Python to be there for me and hashlib the stuff, etree
thingy, zipfile the foo and uuid all the bars.

It's a major feature.

I can live without typing, but I would hate having to pip install a json
parser.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Barry Warsaw]

Guido van Rossum wrote:

> This is beginning to sound like the most attractive solution. We could
> possibly do away with typing_extensions. Are there precedents of how to
> bundle a module in this way? Or is it going to be another special case like
> pip?

With my downstream consumer hat on, *if* we go with a bundling solution,
please make sure it's easy to unbundle.  Downstream consumers may need
to do this for policy, security, etc. reasons, so making this too
difficult will mean delays in releases, convoluted hacks, policy
violations, or all of the above.

See as reference some of the hoops Debian has to go through to handle
ensurepip.

Cheers,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving typing out of the stdlib in Python 3.7?

[Guido van Rossum]

On Mon, Nov 6, 2017 at 5:27 PM, Barry Warsaw  wrote:

>
> Guido van Rossum wrote:
>
> > This is beginning to sound like the most attractive solution. We could
> > possibly do away with typing_extensions. Are there precedents of how to
> > bundle a module in this way? Or is it going to be another special case
> like
> > pip?
>
> With my downstream consumer hat on, *if* we go with a bundling solution,
> please make sure it's easy to unbundle.  Downstream consumers may need
> to do this for policy, security, etc. reasons, so making this too
> difficult will mean delays in releases, convoluted hacks, policy
> violations, or all of the above.
>
> See as reference some of the hoops Debian has to go through to handle
> ensurepip.
>

Having skimmed this thread and some others, I am going in the opposite
direction. Moving typing.py out of the stdlib and then bundling it will
cause too many problems, both psychological and technical.

Therefore I am withdrawing my idea of moving typing.py out of the stdlib,
and replacing it with an extension of its (and PEP 484's) provisional
status by at least one release cycle. This means that at least for the
duration of 3.7 we'll be able to update typing.py pretty aggressively in
bugfix releases.

See https://github.com/python/peps/pull/451 -- basically we only need a
restricted interpretation of provisional, where we can add new features,
but won't break backwards compatibility *for documented aspects of the
API*. With PEP 560, we can then attempt to speed up the import of typing.py
(hopefully 10x by the time 3.8 rolls around).

I propose that the schedule for typing.py's provisional status be added to
PEP 563 (the __future__ import for annotations); I've got a PR with
tentative update to PEP 484 ready (https://github.com/python/peps/pull/451).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/