Re: [Distutils] Platform tags for OS X binary wheels

[Robert McGibbon]

Hi,

> Trust me on that :-)

That's not really the point -- I use both conda and pip, maintain
https://github.com/omnia-md/conda-recipes, and have made multiple upstream
contributions to conda-build.

The point of this thread, from my perspective, was to confirm that there's
a small bug in pip in the way it determines the supported pep425 tags. I
think I've confirmed that, and I'll file a PR shortly.

-Robert

On Fri, Nov 6, 2015 at 4:04 PM, Chris Barker - NOAA Federal <
chris.bar...@noaa.gov> wrote:

> On Nov 6, 2015, at 3:57 PM, Robert McGibbon  wrote:
>
> I'm using the Python from the Miniconda installer with py35 released last
> week.
>
>
> Then you should not expect it to be able to find compatible binary wheels
> on PyPi.
>
> Pretty much the entire point of conda is to support Numpy and friends.
> It's actually really good that it DIDN'T go and install a binary wheel.
>
> You want:
>
> conda install numpy
>
> Trust me on that :-)
>
> There are some cases where pip installing a source package into a conda
> Python is fine -- but mostly only pure-Python packages.
>
> -CHB
>
>
>
> What does the python.org installer build for 10.6+ return for
> `distutils.util.get_platform()`?
>
> -Robert
>
> On Fri, Nov 6, 2015 at 2:50 PM, Ned Deily  wrote:
>
>> In article
>> ,
>>  Robert McGibbon  wrote:
>> > I just tried to run `pip install numpy` on my OS X 10.10.3 box, and it
>> > proceeds to download and compile the tarball from PyPI from source (very
>> > slow). I see, however, that pre-compiled OS X wheel files are available
>> on
>> > PyPI for OS X 10.6 and later.
>> >
>> > Checking the code, it looks like pip is picking up the platform tag
>> through
>> > `distutils.util.get_platform()`, which returns 'macosx-10.5-x86_64' on
>> this
>> > machine. At root, I think this comes from the
>> MACOSX_DEPLOYMENT_TARGET=10.5
>> > entry in the Makefile at `python3.5/config-3.5m/Makefile`. I know that
>> this
>> > value is used by distutils compiling python extension modules --
>> presumably
>> > so that they can be distributed to any target machine with OS X >=10.5
>> --
>> > so that's good. But is this the right thing for pip to be using when
>> > checking whether a binary wheel is compatible? I see it mentioned
>> >  in PEP 425, so perhaps
>> > this was already hashed out on the list.
>>
>> Are you using an OS X Python installed from a python.org installer?  If
>> so, be aware that there are two different OS X installers on Python.org
>> 
>> for each current release.  One is intended for 10.5 systems, although it
>> will work on later OS X systems.  The other is for 10.6 and later
>> systems.  Unless you have a need to run on 10.5 or build something that
>> works on 10.5, download and use the 10.6+ installers instead.  Then the
>> existing whls for products like Numpy should work just fine.
>>
>> --
>>  Ned Deily,
>>  n...@acm.org
>>
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform tags for OS X binary wheels

[Robert McGibbon]

I'm using the Python from the Miniconda installer with py35 released last
week.

What does the python.org installer build for 10.6+ return for
`distutils.util.get_platform()`?

-Robert

On Fri, Nov 6, 2015 at 2:50 PM, Ned Deily  wrote:

> In article
> ,
>  Robert McGibbon  wrote:
> > I just tried to run `pip install numpy` on my OS X 10.10.3 box, and it
> > proceeds to download and compile the tarball from PyPI from source (very
> > slow). I see, however, that pre-compiled OS X wheel files are available
> on
> > PyPI for OS X 10.6 and later.
> >
> > Checking the code, it looks like pip is picking up the platform tag
> through
> > `distutils.util.get_platform()`, which returns 'macosx-10.5-x86_64' on
> this
> > machine. At root, I think this comes from the
> MACOSX_DEPLOYMENT_TARGET=10.5
> > entry in the Makefile at `python3.5/config-3.5m/Makefile`. I know that
> this
> > value is used by distutils compiling python extension modules --
> presumably
> > so that they can be distributed to any target machine with OS X >=10.5 --
> > so that's good. But is this the right thing for pip to be using when
> > checking whether a binary wheel is compatible? I see it mentioned
> >  in PEP 425, so perhaps
> > this was already hashed out on the list.
>
> Are you using an OS X Python installed from a python.org installer?  If
> so, be aware that there are two different OS X installers on Python.org
> for each current release.  One is intended for 10.5 systems, although it
> will work on later OS X systems.  The other is for 10.6 and later
> systems.  Unless you have a need to run on 10.5 or build something that
> works on 10.5, download and use the 10.6+ installers instead.  Then the
> existing whls for products like Numpy should work just fine.
>
> --
>  Ned Deily,
>  n...@acm.org
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

[Chris Barker - NOAA Federal]

> While I understand what you're trying to achieve (and I'm in favour,
> in general) it should be remembered that pip's core goal is installing
> packages - not being a component of a development workflow.

Yes -- clear separation of concerns here!

So what IS supposed to be used in the development workflow? The new
mythical build system?

This brings.  me back to my setuptools-lite concept -- while we are
waiting for a new build system, you can use setuptools-lite, and get a
setup.py install or setup.py develop that does what it's supposed to
do and nothing else

OK, I'll go away now :-)

-Chris


> We absolutely need to make pip useful in the development workflow type
> of situation (that's why pip install -e exists, after all). But I
> don't think it's so much pip "trying to be too clever" as incremental
> rebuilds wasn't the original use case that "pip install ." was
> designed for. What we'll probably have to do is be *more* clever to
> special case out the situations where a development-style support for
> incremental rebuilds is more appropriate than the current behaviour.
>
> Paul
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform tags for OS X binary wheels

[Ned Deily]

In article 
,
 Robert McGibbon  wrote:
> I'm using the Python from the Miniconda installer with py35 released last
> week.
> 
> What does the python.org installer build for 10.6+ return for
> `distutils.util.get_platform()`?

$ /usr/local/bin/python3.5
Python 3.5.0 (v3.5.0:374f501f4567, Sep 12 2015, 11:00:19)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import distutils.util
>>> distutils.util.get_platform()
'macosx-10.6-intel'

I can only assume they built their Python with a deployment target of 
10.5 (MACOSX_DEPLOYMENT_TARGET=10.5).

-- 
 Ned Deily,
 n...@acm.org

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Xavier Fernandez]

On Fri, Nov 6, 2015 at 4:52 AM, Glyph Lefkowitz 
wrote:
>
> Rather than trying to figure out what the "right" way for users to invoke
> `pip´ to begin with is, why not just have Pip start providing more
> *information* about potential problems when you invoke it?
>
> If you invoke 'pip[X.Y]' and it matches 'python -m pip' in your current
> virtualenv, don't say anything; similarly if you invoke 'python -m pip' and
> 'which pip' matches.  But if there's a mismatch, pip can print information
> in both cases.  This would go a long way to alleviating the confusion that
> occurs when users back themselves into one of these corners, and would
> alert users to potential issues before they become a problem; right now you
> have to be a dogged investigative journalist to figure out why pip is doing
> the wrong thing in some cases.
>

I like this solution which is easy to implement and should directly help
the user without any deprecation process.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform tags for OS X binary wheels

[Ned Deily]

In article 
,
 Robert McGibbon  wrote:
> I just tried to run `pip install numpy` on my OS X 10.10.3 box, and it
> proceeds to download and compile the tarball from PyPI from source (very
> slow). I see, however, that pre-compiled OS X wheel files are available on
> PyPI for OS X 10.6 and later.
> 
> Checking the code, it looks like pip is picking up the platform tag through
> `distutils.util.get_platform()`, which returns 'macosx-10.5-x86_64' on this
> machine. At root, I think this comes from the MACOSX_DEPLOYMENT_TARGET=10.5
> entry in the Makefile at `python3.5/config-3.5m/Makefile`. I know that this
> value is used by distutils compiling python extension modules -- presumably
> so that they can be distributed to any target machine with OS X >=10.5 --
> so that's good. But is this the right thing for pip to be using when
> checking whether a binary wheel is compatible? I see it mentioned
>  in PEP 425, so perhaps
> this was already hashed out on the list.

Are you using an OS X Python installed from a python.org installer?  If 
so, be aware that there are two different OS X installers on Python.org 
for each current release.  One is intended for 10.5 systems, although it 
will work on later OS X systems.  The other is for 10.6 and later 
systems.  Unless you have a need to run on 10.5 or build something that 
works on 10.5, download and use the 10.6+ installers instead.  Then the 
existing whls for products like Numpy should work just fine.

-- 
 Ned Deily,
 n...@acm.org

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform tags for OS X binary wheels

[Ralf Gommers]

On Sat, Nov 7, 2015 at 1:04 AM, Chris Barker - NOAA Federal <
chris.bar...@noaa.gov> wrote:

> On Nov 6, 2015, at 3:57 PM, Robert McGibbon  wrote:
>
> I'm using the Python from the Miniconda installer with py35 released last
> week.
>
>
> Then you should not expect it to be able to find compatible binary wheels
> on PyPi.
>
> Pretty much the entire point of conda is to support Numpy and friends.
> It's actually really good that it DIDN'T go and install a binary wheel.
>
> You want:
>
> conda install numpy
>
> Trust me on that :-)
>
> There are some cases where pip installing a source package into a conda
> Python is fine -- but mostly only pure-Python packages.
>

Actually, the situation with pip on OS X is quite good. This should work
with at least python.org Python, MacPython and Homebrew (using wheels):

 pip install numpy scipy matplotlib pandas scikit-image scikit-learn

Ralf
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform tags for OS X binary wheels

[Chris Barker - NOAA Federal]

On Nov 6, 2015, at 3:57 PM, Robert McGibbon  wrote:

I'm using the Python from the Miniconda installer with py35 released last
week.


Then you should not expect it to be able to find compatible binary wheels
on PyPi.

Pretty much the entire point of conda is to support Numpy and friends. It's
actually really good that it DIDN'T go and install a binary wheel.

You want:

conda install numpy

Trust me on that :-)

There are some cases where pip installing a source package into a conda
Python is fine -- but mostly only pure-Python packages.

-CHB



What does the python.org installer build for 10.6+ return for
`distutils.util.get_platform()`?

-Robert

On Fri, Nov 6, 2015 at 2:50 PM, Ned Deily  wrote:

> In article
> ,
>  Robert McGibbon  wrote:
> > I just tried to run `pip install numpy` on my OS X 10.10.3 box, and it
> > proceeds to download and compile the tarball from PyPI from source (very
> > slow). I see, however, that pre-compiled OS X wheel files are available
> on
> > PyPI for OS X 10.6 and later.
> >
> > Checking the code, it looks like pip is picking up the platform tag
> through
> > `distutils.util.get_platform()`, which returns 'macosx-10.5-x86_64' on
> this
> > machine. At root, I think this comes from the
> MACOSX_DEPLOYMENT_TARGET=10.5
> > entry in the Makefile at `python3.5/config-3.5m/Makefile`. I know that
> this
> > value is used by distutils compiling python extension modules --
> presumably
> > so that they can be distributed to any target machine with OS X >=10.5 --
> > so that's good. But is this the right thing for pip to be using when
> > checking whether a binary wheel is compatible? I see it mentioned
> >  in PEP 425, so perhaps
> > this was already hashed out on the list.
>
> Are you using an OS X Python installed from a python.org installer?  If
> so, be aware that there are two different OS X installers on Python.org
> 
> for each current release.  One is intended for 10.5 systems, although it
> will work on later OS X systems.  The other is for 10.6 and later
> systems.  Unless you have a need to run on 10.5 or build something that
> works on 10.5, download and use the 10.6+ installers instead.  Then the
> existing whls for products like Numpy should work just fine.
>
> --
>  Ned Deily,
>  n...@acm.org
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] New PEP : dependency specification

[Nathaniel Smith]

On Thu, Nov 5, 2015 at 7:32 PM, Robert Collins
 wrote:
> Since we ended up with a hard dependency on this for the bootstrap
> thing (regardless of 'smaller step' or not) - I've broken this out of
> PEP 426, made it an encoding of the current status quo rather than an
> aspirational change. Since it has a dependency on markers, I had to
> choose whether to block on James' marker PEP, contribute to that, or
> include it. I think on balance it makes sense to have it in one
> document since the markers bit is actually quite shallow, so I've done
> that (after discussing with James). This could perhaps replace PEP 496
> then, or be given a new number.
>
> Donald has graciously agreed to be a BDFL-delegate for it.
>
> The PR for it is https://github.com/pypa/interoperability-peps/pull/56

Thanks, this is really great!

I made a bunch of fiddly comments inline on the PR, but some more
general comments:

1) Also mentioned this in the PR, but it's probably worth putting up
for more general discussion here: do we want to define some graceful
degradation for how to handle unrecognized variable names in the
environment marker syntax, so as to allow more easily for future
extensions? In the current PEP draft, an unrecognized variable name is
simply a syntax error (all the variable names are effectively
keywords).

One option would be to declare that any expression that contains an
unrecognized variable simply evaluates to False. The downside of this
is that it's somewhat accident prone: "os_name == posix" would
silently be always false (because "posix" should be quoted as a
string, not left unquoted and treated as a variable name), and
"os_nmae == 'posix'" would also be silently accepted.

2) The PEP seems a little uncertain about whether it wants to talk
about the "framing protocol" or not -- like whether there's a
higher-level structure defining the edges of each individual
requirement or not. In setup.py and in existing METADATA files, you
have some higher level list-of-strings syntax and then each string is
parsed as a single individual requirement, and comments don't make
much sense; in requirements.txt then newlines are meaningful and
comments are important. The PEP worries about newlines and newlines,
but doesn't quite want to come out and say that it's defining
requirements.txt -- it wants to be more general.

Maybe it would be clearer to drop the comment and newline handling
stuff from the core requirement specifier syntax (declaring that
newlines are simply a syntax error), and assume that there's some
higher-level framing protocol taking care of that stuff? So METADATA
files would use whatever rules they use to pick out a single
requirement specifier string (probably not allowing e.g. comments) and
then parse it using this PEP's rules, and separately we'd have a
definition for requirements.txt which is basically "a text file where
you strip out comments, delete newlines that are preceded by
backslash, split into lines, discard empty lines, and each resulting
string is parsed as a requirement specifier".

3) The way extras are specified in METADATA files currently (ab)uses
the environment marker syntax. E.g. here's the METADATA files from two
popular packages:

  https://gist.github.com/njsmith/ed74851c0311e858f0f7

(Nice to see Metadata-Version: 2.0 getting some real-world use! I
guess??? I thought I was starting to understand what is going on with
python packaging standards but now I am baffled again. Anyway!)

So apparently this is how you say that there is an extra called "doc"
and that this extra adds a dependency on Sphinx:

Provides-Extra: doc
Requires-Dist: Sphinx (>=1.1); extra == 'doc'

(https://gist.github.com/njsmith/ed74851c0311e858f0f7#file-ipython-4-0-0-wheel-metadata-L39)

And here's how you say that the "terminal" extra adds a dependency on
pyreadline (but only on windows):

Requires-Dist: pyreadline (>=2); sys_platform == "win32" and extra == 'terminal'

(https://gist.github.com/njsmith/ed74851c0311e858f0f7#file-ipython-4-0-0-wheel-metadata-L64)

I'm not sure this is the syntax that I would have come up with, but I
guess it's too late to put the genie back in the bottle, so this PEP
should have some way to cope with these things? Currently they are
simply syntax errors according to the PEP's grammar.

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Paul Moore]

On 6 November 2015 at 03:52, Glyph Lefkowitz  wrote:
> If you invoke 'pip[X.Y]' and it matches 'python -m pip' in your current
> virtualenv, don't say anything; similarly if you invoke 'python -m pip' and
> 'which pip' matches.  But if there's a mismatch, pip can print information
> in both cases.  This would go a long way to alleviating the confusion that
> occurs when users back themselves into one of these corners, and would alert
> users to potential issues before they become a problem; right now you have
> to be a dogged investigative journalist to figure out why pip is doing the
> wrong thing in some cases.

I don't see how such checks would work on Windows. The "simple"
approach would involve invoking a subprocess to check what "python"
resolved to, which is a non-trivial overhead on Windows.

Could you explain how you'd do a check like this on Windows?
Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Paul Moore]

On 5 November 2015 at 22:59, Donald Stufft  wrote:
> I think we could integrate with py on Windows somehow so that we use the same 
> lookup semantics as py does. I don't know enough about Windows and py.exe to 
> know what exactly those are.

Hmm, I'm reluctant to get into complex interactions with C code like
the launcher on Windows. The number of people we'd have able to
maintain such code is approaching zero, I suspect :-( We may be able
to run py.exe as a subprocess, but that could easily get out of hand
(running pip could then easily start up a chain of 6 processes on
Windows, which while not disastrous, certainly isn't cheap).

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] New PEP : dependency specification

[Robert Collins]

On 7 November 2015 at 07:42, Marcus Smith  wrote:
>>
>>
>> No - it specifies the serialisation format for
>> names/specifiers/extras/markers that is in common use, but doesn't
>> specify a programming API. It is intended as an interop building
>> block,
>
>
> I'm not not talking about programming API.
> this PEP would set the format used in interop formats, as you say, but also
> doesn't it effectively standardize the format used in the pip UI?
> or maybe it's just that pip could easily say..  "our syntax for dependencies
> mirrors PEPXX exactly"

Yes, I think that would be a sane thing for pip to say. E.g. the full
description would be something like "Requirements files are a
super-set of PEPXX. You can also include command line options from the
pip CLI. Options on a line on their own are globally applies. Options
on a specification line apply to that specification alone. Empty and
comment-only lines are supported."

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] New PEP : dependency specification

[Marcus Smith]

>
>
> So both the abstract build system PEP and donalds setup.py interface
> depend on having a bootstrap dependency list written into a file in
> the source tree.


your build PEP said stuff like this  "Additional data *may* be included,
but the ``build_requires`` and ``metadata_version`` keys must be present"

that leads me to think you need more than just this specification of a
single dependency.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform tags for OS X binary wheels

[Daniel Holth]

If you would like to fix the problem, figure out how to get the real OSX
version into pip.pep425tags.

On Fri, Nov 6, 2015 at 2:20 PM Robert McGibbon  wrote:

> For OS X, the pip get_platform function eventually calls into here:
> https://github.com/python/cpython/blob/master/Lib/_osx_support.py#L429-L439,
> and I think the comment kind of explains the bug.
>
> -Robert
>
>
>
> On Fri, Nov 6, 2015 at 11:10 AM, Daniel Holth  wrote:
>
>> I see what you mean. Sounds like a bug to me.
>>
>> On Fri, Nov 6, 2015 at 2:07 PM Robert McGibbon 
>> wrote:
>>
>>> I don't think it's the sorting, per se. All of the get_supported() tags
>>> are 10.5 or earlier. Here's the output:
>>> https://gist.github.com/rmcgibbo/1d0f5d166ca48253b5a9
>>>
>>>
>>> On Fri, Nov 6, 2015 at 11:00 AM, Daniel Holth  wrote:
>>>
 It should already be sorted. Try python -c "import pprint,
 pip.pep425tags; pprint.pprint(pip.pep425tags.get_supported())"

 Do none of the tags for the available numpy wheels appear in that list?

 On Fri, Nov 6, 2015 at 1:48 PM Robert McGibbon 
 wrote:

> Hi,
>
> I just tried to run `pip install numpy` on my OS X 10.10.3 box, and it
> proceeds to download and compile the tarball from PyPI from source (very
> slow). I see, however, that pre-compiled OS X wheel files are available on
> PyPI for OS X 10.6 and later.
>
> Checking the code, it looks like pip is picking up the platform tag
> through `distutils.util.get_platform()`, which returns 
> 'macosx-10.5-x86_64'
> on this machine. At root, I think this comes from
> the MACOSX_DEPLOYMENT_TARGET=10.5 entry in the Makefile at
> `python3.5/config-3.5m/Makefile`. I know that this value is used by
> distutils compiling python extension modules -- presumably so that they 
> can
> be distributed to any target machine with OS X >=10.5 -- so that's good.
> But is this the right thing for pip to be using when checking whether a
> binary wheel is compatible? I see it mentioned
>  in PEP 425, so
> perhaps this was already hashed out on the list.
>
> Best,
> Robert
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>

>>>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform tags for OS X binary wheels

[Robert McGibbon]

Sounds good. I'll take a look.

-Robert

On Fri, Nov 6, 2015 at 11:23 AM, Daniel Holth  wrote:

> If you would like to fix the problem, figure out how to get the real OSX
> version into pip.pep425tags.
>
> On Fri, Nov 6, 2015 at 2:20 PM Robert McGibbon  wrote:
>
>> For OS X, the pip get_platform function eventually calls into here:
>> https://github.com/python/cpython/blob/master/Lib/_osx_support.py#L429-L439,
>> and I think the comment kind of explains the bug.
>>
>> -Robert
>>
>>
>>
>> On Fri, Nov 6, 2015 at 11:10 AM, Daniel Holth  wrote:
>>
>>> I see what you mean. Sounds like a bug to me.
>>>
>>> On Fri, Nov 6, 2015 at 2:07 PM Robert McGibbon 
>>> wrote:
>>>
 I don't think it's the sorting, per se. All of the get_supported()
 tags are 10.5 or earlier. Here's the output:
 https://gist.github.com/rmcgibbo/1d0f5d166ca48253b5a9


 On Fri, Nov 6, 2015 at 11:00 AM, Daniel Holth  wrote:

> It should already be sorted. Try python -c "import pprint,
> pip.pep425tags; pprint.pprint(pip.pep425tags.get_supported())"
>
> Do none of the tags for the available numpy wheels appear in that list?
>
> On Fri, Nov 6, 2015 at 1:48 PM Robert McGibbon 
> wrote:
>
>> Hi,
>>
>> I just tried to run `pip install numpy` on my OS X 10.10.3 box, and
>> it proceeds to download and compile the tarball from PyPI from source 
>> (very
>> slow). I see, however, that pre-compiled OS X wheel files are available 
>> on
>> PyPI for OS X 10.6 and later.
>>
>> Checking the code, it looks like pip is picking up the platform tag
>> through `distutils.util.get_platform()`, which returns 
>> 'macosx-10.5-x86_64'
>> on this machine. At root, I think this comes from
>> the MACOSX_DEPLOYMENT_TARGET=10.5 entry in the Makefile at
>> `python3.5/config-3.5m/Makefile`. I know that this value is used by
>> distutils compiling python extension modules -- presumably so that they 
>> can
>> be distributed to any target machine with OS X >=10.5 -- so that's good.
>> But is this the right thing for pip to be using when checking whether a
>> binary wheel is compatible? I see it mentioned
>>  in PEP 425, so
>> perhaps this was already hashed out on the list.
>>
>> Best,
>> Robert
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>

>>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] New PEP : dependency specification

[Robert Collins]

On 7 November 2015 at 07:22, Daniel Holth  wrote:
> LGTM
>
> To clarify in this spec to specify a couple of requirements for the [foo]
> extra would you have to say
>
> [foo] requests
> [foo] sqlalchemy

Oh, so this spec describes how to specify a dependency, not how to
provide the metadata to e.g. setuptools. I may be misinterpreting your
question, but assuming you are referring to the setup.py interface...
it should be insertable in a compatible way with the existing
setuptools idioms.

e.g. in setuptools today you'd say

extras_require={'foo': ["requests", "sqlalchemy"]}

And I think you'd keep doing that.

What you might want to do though is say that for the foo extra that
one of the dependencies only applies sometimes. Today you do this
manual separation thing:

extras_require={
 'foo': ["requests", "sqlalchemy"],
 'foo:python_version<"2.7"': ["requests[security]"]}

A patch to setuptools to allow dependencies per this PEP would allow:

extras_require={
 'foo': ["requests", "sqlalchemy", "requests[security];python_version<'2.7'"]}

> Compare to requires.txt from setuptools which IIRC is a plain text file like
> so, with normal requirements not in a section, and extra or conditional
> requirements in sections named [extra_name;marker]:
>...

OTOH you might be talking about how we'd using this in an PKG_INFO /
egg-info / dist-info metadata serialisation format.

We haven't specified that yet - since there is strictly no new
capabilities over those existing on-disk formats, they are just
different, I think we have an ok status quo. It's not as great as
having a fully specified consistent thing end to end format though :(.

So taking a higher level view, we currently have three formats to describe this:
 - requires.txt, which can't specify markers at the granularity of a
dependency, only by having larger structure which also defines extras
and uses the empty extra to define marker based rules
 - PEP-345 PKG-INFO, which nothing outputs as PKG-INFO, but wheel
outputs as dist-info/METADATA - and is very nearly compatible with
this PEP (it just adds brackets around version specificiers). I think
adding those in as optional but not recommended elements, would permit
a single unified parser for METADATA specifiers and the ones in this
PEP.
 - requirements.txt files which I decided to use as the basis because
its been hammered on most IMO :).

I don't have a PEP yet to specify the *export* of distribution
metadata using this format. If we adopt 'what is in egg-info' as the
interop basis for now, then such a PEP can come later, but at the cost
of having a non-PEP defined format included by reference. If we adopt
what is in dist-info as the format, we get a define thing, but out of
date (because markers have evolved since PEP-345) - but we should be
able to do a very minimal update to 345 that solely references this
new PEP to update the dependency specification (if we do the brackets
thing above), ... which gives us sufficient coverage I think.

-Rob
-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Platform tags for OS X binary wheels

[Robert McGibbon]

Hi,

I just tried to run `pip install numpy` on my OS X 10.10.3 box, and it
proceeds to download and compile the tarball from PyPI from source (very
slow). I see, however, that pre-compiled OS X wheel files are available on
PyPI for OS X 10.6 and later.

Checking the code, it looks like pip is picking up the platform tag through
`distutils.util.get_platform()`, which returns 'macosx-10.5-x86_64' on this
machine. At root, I think this comes from the MACOSX_DEPLOYMENT_TARGET=10.5
entry in the Makefile at `python3.5/config-3.5m/Makefile`. I know that this
value is used by distutils compiling python extension modules -- presumably
so that they can be distributed to any target machine with OS X >=10.5 --
so that's good. But is this the right thing for pip to be using when
checking whether a binary wheel is compatible? I see it mentioned
 in PEP 425, so perhaps
this was already hashed out on the list.

Best,
Robert
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform tags for OS X binary wheels

[Daniel Holth]

It should already be sorted. Try python -c "import pprint, pip.pep425tags;
pprint.pprint(pip.pep425tags.get_supported())"

Do none of the tags for the available numpy wheels appear in that list?

On Fri, Nov 6, 2015 at 1:48 PM Robert McGibbon  wrote:

> Hi,
>
> I just tried to run `pip install numpy` on my OS X 10.10.3 box, and it
> proceeds to download and compile the tarball from PyPI from source (very
> slow). I see, however, that pre-compiled OS X wheel files are available on
> PyPI for OS X 10.6 and later.
>
> Checking the code, it looks like pip is picking up the platform tag
> through `distutils.util.get_platform()`, which returns 'macosx-10.5-x86_64'
> on this machine. At root, I think this comes from
> the MACOSX_DEPLOYMENT_TARGET=10.5 entry in the Makefile at
> `python3.5/config-3.5m/Makefile`. I know that this value is used by
> distutils compiling python extension modules -- presumably so that they can
> be distributed to any target machine with OS X >=10.5 -- so that's good.
> But is this the right thing for pip to be using when checking whether a
> binary wheel is compatible? I see it mentioned
>  in PEP 425, so perhaps
> this was already hashed out on the list.
>
> Best,
> Robert
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform tags for OS X binary wheels

[Robert McGibbon]

I don't think it's the sorting, per se. All of the get_supported() tags are
10.5 or earlier. Here's the output:
https://gist.github.com/rmcgibbo/1d0f5d166ca48253b5a9


On Fri, Nov 6, 2015 at 11:00 AM, Daniel Holth  wrote:

> It should already be sorted. Try python -c "import pprint,
> pip.pep425tags; pprint.pprint(pip.pep425tags.get_supported())"
>
> Do none of the tags for the available numpy wheels appear in that list?
>
> On Fri, Nov 6, 2015 at 1:48 PM Robert McGibbon  wrote:
>
>> Hi,
>>
>> I just tried to run `pip install numpy` on my OS X 10.10.3 box, and it
>> proceeds to download and compile the tarball from PyPI from source (very
>> slow). I see, however, that pre-compiled OS X wheel files are available on
>> PyPI for OS X 10.6 and later.
>>
>> Checking the code, it looks like pip is picking up the platform tag
>> through `distutils.util.get_platform()`, which returns 'macosx-10.5-x86_64'
>> on this machine. At root, I think this comes from
>> the MACOSX_DEPLOYMENT_TARGET=10.5 entry in the Makefile at
>> `python3.5/config-3.5m/Makefile`. I know that this value is used by
>> distutils compiling python extension modules -- presumably so that they can
>> be distributed to any target machine with OS X >=10.5 -- so that's good.
>> But is this the right thing for pip to be using when checking whether a
>> binary wheel is compatible? I see it mentioned
>>  in PEP 425, so perhaps
>> this was already hashed out on the list.
>>
>> Best,
>> Robert
>> ___
>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Michael Merickel]

On Fri, Nov 6, 2015 at 12:33 PM, Ionel Cristian Mărieș 
wrote:

> ​Why not consider having a "pip" launcher?​ Seems the obvious thing to me
> - python has the "py" launcher on windows and it works great!
>
> Eg: "pip -3" to launch pip using python3, "pip -3.5" to launch pip using
> python3.5 - just like the "py" launcher.
>

This sounds similar to node's approach to bundling npm where you have a
version of npm that runs for all projects using that node runtime. In
effect you use the same npm binary independent of which virtualenv you are
using, it's tied to the interpreter installation. So you run the pip that
came with that python, such as pip2.7 installed with python2.7 and then
just tell it which virtualenv to install packages into. You could go
further and install a pip alias into the virtualenv that links to the
appropriate pip, but there's still only one copy running for all
virtualenvs off of that interpreter.

This model is really nice for updating because you don't need to update pip
inside every single virtualenv. Another cool feature of npm is that it's
also effectively auto-using the virtualenv since it defaults to the local
node_modules folder so you don't have to specify it as something like "pip
--virtualenv=env install".

Anyway I'm not suggesting such a drastic change but it's nice to look at
what else is out there and the benefits. It doesn't mesh particularly well
verbatim with how PYTHONPATH+virtualenv works of course.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform tags for OS X binary wheels

[Daniel Holth]

I see what you mean. Sounds like a bug to me.

On Fri, Nov 6, 2015 at 2:07 PM Robert McGibbon  wrote:

> I don't think it's the sorting, per se. All of the get_supported() tags
> are 10.5 or earlier. Here's the output:
> https://gist.github.com/rmcgibbo/1d0f5d166ca48253b5a9
>
>
> On Fri, Nov 6, 2015 at 11:00 AM, Daniel Holth  wrote:
>
>> It should already be sorted. Try python -c "import pprint,
>> pip.pep425tags; pprint.pprint(pip.pep425tags.get_supported())"
>>
>> Do none of the tags for the available numpy wheels appear in that list?
>>
>> On Fri, Nov 6, 2015 at 1:48 PM Robert McGibbon 
>> wrote:
>>
>>> Hi,
>>>
>>> I just tried to run `pip install numpy` on my OS X 10.10.3 box, and it
>>> proceeds to download and compile the tarball from PyPI from source (very
>>> slow). I see, however, that pre-compiled OS X wheel files are available on
>>> PyPI for OS X 10.6 and later.
>>>
>>> Checking the code, it looks like pip is picking up the platform tag
>>> through `distutils.util.get_platform()`, which returns 'macosx-10.5-x86_64'
>>> on this machine. At root, I think this comes from
>>> the MACOSX_DEPLOYMENT_TARGET=10.5 entry in the Makefile at
>>> `python3.5/config-3.5m/Makefile`. I know that this value is used by
>>> distutils compiling python extension modules -- presumably so that they can
>>> be distributed to any target machine with OS X >=10.5 -- so that's good.
>>> But is this the right thing for pip to be using when checking whether a
>>> binary wheel is compatible? I see it mentioned
>>>  in PEP 425, so perhaps
>>> this was already hashed out on the list.
>>>
>>> Best,
>>> Robert
>>> ___
>>> Distutils-SIG maillist  -  Distutils-SIG@python.org
>>> https://mail.python.org/mailman/listinfo/distutils-sig
>>>
>>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] New PEP : dependency specification

[Robert Collins]

On 7 November 2015 at 07:57, Marcus Smith  wrote:
>>
>> So both the abstract build system PEP and donalds setup.py interface
>> depend on having a bootstrap dependency list written into a file in
>> the source tree.
>
>
> your build PEP said stuff like this  "Additional data *may* be included, but
> the ``build_requires`` and ``metadata_version`` keys must be present"
>
> that leads me to think you need more than just this specification of a
> single dependency.

Thats the next layer up - and if we adopt the egg-info-as-carrier
format we don't need to issue a new PEP. I've yet to look closely. But
any which way, it doesn't affect this PEP, and this PEP acts as a
robust building block.

-Rob



-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Platform tags for OS X binary wheels

[Robert McGibbon]

For OS X, the pip get_platform function eventually calls into here:
https://github.com/python/cpython/blob/master/Lib/_osx_support.py#L429-L439,
and I think the comment kind of explains the bug.

-Robert



On Fri, Nov 6, 2015 at 11:10 AM, Daniel Holth  wrote:

> I see what you mean. Sounds like a bug to me.
>
> On Fri, Nov 6, 2015 at 2:07 PM Robert McGibbon  wrote:
>
>> I don't think it's the sorting, per se. All of the get_supported() tags
>> are 10.5 or earlier. Here's the output:
>> https://gist.github.com/rmcgibbo/1d0f5d166ca48253b5a9
>>
>>
>> On Fri, Nov 6, 2015 at 11:00 AM, Daniel Holth  wrote:
>>
>>> It should already be sorted. Try python -c "import pprint,
>>> pip.pep425tags; pprint.pprint(pip.pep425tags.get_supported())"
>>>
>>> Do none of the tags for the available numpy wheels appear in that list?
>>>
>>> On Fri, Nov 6, 2015 at 1:48 PM Robert McGibbon 
>>> wrote:
>>>
 Hi,

 I just tried to run `pip install numpy` on my OS X 10.10.3 box, and it
 proceeds to download and compile the tarball from PyPI from source (very
 slow). I see, however, that pre-compiled OS X wheel files are available on
 PyPI for OS X 10.6 and later.

 Checking the code, it looks like pip is picking up the platform tag
 through `distutils.util.get_platform()`, which returns 'macosx-10.5-x86_64'
 on this machine. At root, I think this comes from
 the MACOSX_DEPLOYMENT_TARGET=10.5 entry in the Makefile at
 `python3.5/config-3.5m/Makefile`. I know that this value is used by
 distutils compiling python extension modules -- presumably so that they can
 be distributed to any target machine with OS X >=10.5 -- so that's good.
 But is this the right thing for pip to be using when checking whether a
 binary wheel is compatible? I see it mentioned
  in PEP 425, so
 perhaps this was already hashed out on the list.

 Best,
 Robert
 ___
 Distutils-SIG maillist  -  Distutils-SIG@python.org
 https://mail.python.org/mailman/listinfo/distutils-sig

>>>
>>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Nathaniel Smith]

On Nov 6, 2015 5:39 AM, "Paul Moore"  wrote:
>
> On 6 November 2015 at 12:10, Donald Stufft  wrote:
> > Doesn’t py.exe just look at some ini files and environment variables to
decide what to invoke? I’m not sure why we couldn’t just replicate that
behavior. Is there something that py.exe does that we can’t also do in
Python?
>
> What's there might be possible in Python, but there are some nasty
> special cases. The launcher code looks in the registry, and in order
> to support running both 32 and 64 bit Pythons from the same launcher
> exe, it needs to play tricks to see both the 32 and 64 bit registry
> values, and I'm not sure the APIs to do that are exposed to Python so
> it may need ctypes.
>
> Doable certainly, easy not so sure. We could of course do a simplified
> version and not try to be precisely equivalent to the launcher
> behaviour - but that may also cause user confusion. The simplest cases
> are easy, it's the corner cases that get nasty.

One option would be to add a "py -which" mode that just does the
configuration lookup and then prints the path to the real python
executable. This would add the cost of one subprocess startup per pip, but
not 6?

-n
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Tim Golden]

On 06/11/2015 14:46, Nathaniel Smith wrote:
> On Nov 6, 2015 5:39 AM, "Paul Moore"  > wrote:
>>
>> On 6 November 2015 at 12:10, Donald Stufft  > wrote:
>> > Doesn’t py.exe just look at some ini files and environment variables
> to decide what to invoke? I’m not sure why we couldn’t just replicate
> that behavior. Is there something that py.exe does that we can’t also do
> in Python?
>>
>> What's there might be possible in Python, but there are some nasty
>> special cases. The launcher code looks in the registry, and in order
>> to support running both 32 and 64 bit Pythons from the same launcher
>> exe, it needs to play tricks to see both the 32 and 64 bit registry
>> values, and I'm not sure the APIs to do that are exposed to Python so
>> it may need ctypes.
>>
>> Doable certainly, easy not so sure. We could of course do a simplified
>> version and not try to be precisely equivalent to the launcher
>> behaviour - but that may also cause user confusion. The simplest cases
>> are easy, it's the corner cases that get nasty.
> 
> One option would be to add a "py -which" mode that just does the
> configuration lookup and then prints the path to the real python
> executable. This would add the cost of one subprocess startup per pip,
> but not 6?

FWIW this will do that:

  py -c "import sys; print(sys.executable)"

TJG
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Marius Gedminas]

On Fri, Nov 06, 2015 at 02:04:38PM +1300, Robert Collins wrote:
> On 6 November 2015 at 10:08, Donald Stufft  wrote:
> > * It's more to type, 10 more characters on *nix and 6 more characters on
> >   Windows which makes it more akward and annoying to use. This is 
> > particularly
> >   annoying inside of a virtual environment where there isn't really any
> >   ambiguity when one is activated.
> 
> cat > /usr/bin/pip << EOF
> python -m pip $@
> EOF
> 
> Seriously - isn't the above entirely sufficient?

It doesn't help with my usual pattern, which used to be

  $ virtualenv .
  $ bin/pip install foo

but then changed[*] into

  $ virtualenv .venv && ln -sfn .venv/bin bin
  $ bin/pip install foo

and is likely to change[+] into

  $ virtualenv .venv && mkdir -p bin && ln -sfn .venv/bin/pip bin/pip
  $ bin/pip install foo

I am not running these by hand -- I have Makefiles to set up my app
environment by creating a local virtualenv and pip installing all the
tools, plus '-e .', into it.  But once the basic environment is done,
I'm often installing ad-hoc one-time-use extra tools with commands like

  $ bin/pip install runsnakerun

---

  [*] because virtualenv's root is becoming too cluttered with files like
  pip-selftest.json, and because some evil packages on PyPI install
  files named README.txt into the virtualenv root.

  [+] because if you symlink just the bin/ directory, bin/python fails to
  set up sys.path correctly

Marius Gedminas
-- 
I'm sure it would be possible to speed apport up a lot, after we're done
making boot and login instantaneous.
-- Lars Wirzenius


signature.asc
Description: Digital signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Installing packages using pip

[Ines Barata]

Hello,

I want to install the following version of pygame in my windows10.pro:install
pygame-1.9.2a0-cp33-none-win_amd64.whl, but the file is in *.whl* format
which I dont have any program to open it
I checked the information in https://docs.python.org/3/installing/ to
understand how to use pip. But didn't make it...

My result is the following, using *python3.5.0 IDLE*:
>>> pip *install* pygame-1.9.2a0-cp34-none-win32.whl
SyntaxError: invalid syntax
>>> pip *pygame*-1.9.2a0-cp34-none-win32.whl
SyntaxError: invalid syntax
>>> python -m *pip* install pygame-1.9.2a0-cp33-none-win_amd64.whl
SyntaxError: invalid syntax
>>> python3.*5*.0 -m pip install pygame-1.9.2a0-cp33-none-win_amd64.whl
SyntaxError: invalid syntax
>>>

Can you help me solve my problem?
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Paul Moore]

On 6 November 2015 at 14:56, Tim Golden  wrote:
>> One option would be to add a "py -which" mode that just does the
>> configuration lookup and then prints the path to the real python
>> executable. This would add the cost of one subprocess startup per pip,
>> but not 6?
>
> FWIW this will do that:
>
>   py -c "import sys; print(sys.executable)"

Indeed. We don't want to require modifications to py.exe, as that
would imply some sort of "Python 3.6 only" requirement (there are
backport options, as the launcher is largely independent from Python
itself, but that's yet more obstacles for end users).

Don't take the 6 processes too literally, but just to explain:

pip launcher
- runs Python. Either directly (= 2 processes so far) or via
py (= 3 processes so far)
- checks what Python it's running (just code to read
sys.executable in pip)
- processes the -p flag
- runs py -c "import sys; print(sys.executable)" to work
out what version the -p flag is requesting
  (unless -p requires the full pathname of an executable,
which is lousy UI)
  That's 2 more processes run, albeit only for a short time
- Finds out it needs a different exe, launches py for the
correct Python (add 2 more processes to the stack, we're at 5 now).
- The subordinate pip runs sys.executable to execute
setup.py (1 more process) which in turn launches the C compiler
  (that's a stack of 7 processes)

It's quite likely that this can be trimmed, and normal cases won't be
nearly as bad. And while creating new processes on Windows is
distinctly worse than on Linux, it's not *that* bad.

But when wheels were introduced, installing via wheels rather than via
setup.py caused *substantial* time savings when installing pure Python
wheels, simply from removing process creation overheads (creating a
virtualenv, which installed setuptools and pip at the time, i.e., 2
wheels instead of 2 sdists, went from something like 30 seconds down
to a couple of seconds, if I recall).

I'm not wanting to make a big issue out of this, but it needs to be
considered...
Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Installing packages using pip

[Paul Moore]

On 6 November 2015 at 14:33, Ines Barata  wrote:
> Hello,
>
> I want to install the following version of pygame in my
> windows10.pro:install pygame-1.9.2a0-cp33-none-win_amd64.whl, but the file
> is in .whl format which I dont have any program to open it
> I checked the information in https://docs.python.org/3/installing/ to
> understand how to use pip. But didn't make it...
>
> My result is the following, using python3.5.0 IDLE:
 pip install pygame-1.9.2a0-cp34-none-win32.whl
> SyntaxError: invalid syntax

That's the correct command, but you need to run it from the Windows
command prompt, not from within IDLE.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Wolfgang Maier]

On 11/05/2015 10:08 PM, Donald Stufft wrote:


* There is a lot of documentation out there in many projects that tell people
   to use ``pip install ...``, the long tail on getting people moved to this
   will be very long.



The deprecation period will probably have to be long, but the current 
situation is not so bad that you could not live with it for a bit longer.



* It's more to type, 10 more characters on *nix and 6 more characters on
   Windows which makes it more akward and annoying to use. This is particularly
   annoying inside of a virtual environment where there isn't really any
   ambiguity when one is activated.



I have no problem with the extra characters, just as I don't have a 
problem with typing: java -jar xy.jar


The extra typing might be annoying for the pypa devs, but remember that 
many regular users have to type this only once in a while when they 
install some new package. For them, its far more important that things 
work reliably than with the shortest possible command.




* It still has the annoyance around having multiple pip installs all over the
   place and needing to manage those.



Another experts problem. People who are just using "Python" and a few 
third-party packages are not suffering from this. Once they get to a 
level where they use multiple versions of python, they will be able to 
cope with the multiple pip installations.




* We still support Python 2.6 which doesn't support executing a package only
   modules via ``-m``. So we'll break Python 2.6 unless people do
   ``python -m pip.__main__`` or we move pip/* to _pip/* and make a pip.py which
   will break things for people using pip as a library (which isn't currently
   supported).



How much longer are you planning to support Python 2.6? Why not just 
deprecate pip and pipX.Y (emit a warning to users) for newer versions of 
Python. Then once you drop Python 2.6 support remove pip and pipX.Y.



python -m pip ... may not read that beautifully, but if lack of beauty 
is the last remaining problem to Python packaging, then that's a reason 
to celebrate, isn't it.


Best,
Wolfgang

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Paul Moore]

On 6 November 2015 at 12:10, Donald Stufft  wrote:
> Doesn’t py.exe just look at some ini files and environment variables to 
> decide what to invoke? I’m not sure why we couldn’t just replicate that 
> behavior. Is there something that py.exe does that we can’t also do in Python?

What's there might be possible in Python, but there are some nasty
special cases. The launcher code looks in the registry, and in order
to support running both 32 and 64 bit Pythons from the same launcher
exe, it needs to play tricks to see both the 32 and 64 bit registry
values, and I'm not sure the APIs to do that are exposed to Python so
it may need ctypes.

Doable certainly, easy not so sure. We could of course do a simplified
version and not try to be precisely equivalent to the launcher
behaviour - but that may also cause user confusion. The simplest cases
are easy, it's the corner cases that get nasty.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] New PEP : dependency specification

[Marcus Smith]

>
>
>
> No - it specifies the serialisation format for
> names/specifiers/extras/markers that is in common use, but doesn't
> specify a programming API. It is intended as an interop building
> block,


I'm not not talking about programming API.
this PEP would set the format used in interop formats, as you say, but also
doesn't it effectively standardize the format used in the pip UI?
or maybe it's just that pip could easily say..  "our syntax for
dependencies mirrors PEPXX exactly"
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] New PEP : dependency specification

[Marcus Smith]

>
>
> The language defined is a compact line based format which is already in
> widespread use


this is the most critical thing for me, and the reason this approach seems
more attractive than the path of PEP426, although I'd certainly like to see
Nick's reaction.

PEP426 tries to cover how names/specifiers/extras/markers would be put
together in abstract "in-memory representation" (that can be serialized to
json), but it's left open to pip (and other tools) to lay down a standard
(via implementation) for how these pieces are put together and used by
users.

this PEP would dictate both, right?  the user way, and the internal
metadata way
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] New PEP : dependency specification

[Daniel Holth]

LGTM

To clarify in this spec to specify a couple of requirements for the [foo]
extra would you have to say

[foo] requests
[foo] sqlalchemy

Compare to requires.txt from setuptools which IIRC is a plain text file
like so, with normal requirements not in a section, and extra or
conditional requirements in sections named [extra_name;marker]:

unconditional
requirements==4.7

[;marker]
non-extra requirement if marker evaluates to true

[extra]
unconditional
requirements
for
extra

[extra;marker]
requirement for extra with if marker evaluates to true

On Fri, Nov 6, 2015 at 12:45 PM Marcus Smith  wrote:

>
>> The language defined is a compact line based format which is already in
>> widespread use
>
>
> this is the most critical thing for me, and the reason this approach seems
> more attractive than the path of PEP426, although I'd certainly like to see
> Nick's reaction.
>
> PEP426 tries to cover how names/specifiers/extras/markers would be put
> together in abstract "in-memory representation" (that can be serialized to
> json), but it's left open to pip (and other tools) to lay down a standard
> (via implementation) for how these pieces are put together and used by
> users.
>
> this PEP would dictate both, right?  the user way, and the internal
> metadata way
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] New PEP : dependency specification

[Robert Collins]

On 7 November 2015 at 06:45, Marcus Smith  wrote:
>>
>> The language defined is a compact line based format which is already in
>> widespread use
>
>
> this is the most critical thing for me, and the reason this approach seems
> more attractive than the path of PEP426, although I'd certainly like to see
> Nick's reaction.
>
> PEP426 tries to cover how names/specifiers/extras/markers would be put
> together in abstract "in-memory representation" (that can be serialized to
> json), but it's left open to pip (and other tools) to lay down a standard
> (via implementation) for how these pieces are put together and used by
> users.
>
> this PEP would dictate both, right?  the user way, and the internal metadata
> way

No - it specifies the serialisation format for
names/specifiers/extras/markers that is in common use, but doesn't
specify a programming API. It is intended as an interop building
block, so we don't have to say 'that thing that pkg_resources is the
defacto std for'.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Ionel Cristian Mărieș]

On Thu, Nov 5, 2015 at 11:08 PM, Donald Stufft  wrote:

> Currently pip installs a number of commands like ``pip``, ``pipX`` and
> ``pipX.Y`` where the X and X.Y corresponds to the version of Python that
> pip
> is installed into. Pip installs into whatever Python is currently
> executing it
> so this gives some ability to control which version of Python you're
> installing
> into (``pip2.7`` for Python 2.7 etc).
>

​Why not consider having a "pip" launcher?​ Seems the obvious thing to me -
python has the "py" launcher on windows and it works great!

Eg: "pip -3" to launch pip using python3, "pip -3.5" to launch pip using
python3.5 - just like the "py" launcher.



Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] New PEP : dependency specification

[Robert Collins]

Here's an update I just pushed into git that addresses the defects found so far.

diff --git a/dependency-specification.rst b/dependency-specification.rst
index f8171d8..486636d 100644
--- a/dependency-specification.rst
+++ b/dependency-specification.rst
@@ -73,7 +73,7 @@ the dependency can be made conditional using
environment markers.
 Grammar
 ---

-We first cover the grammar briefly and the drill into the semantics of each
+We first cover the grammar briefly and then drill into the semantics of each
 section later.

 A distribution specification is written in ASCII text. We use ABNF [#abnf]_ to
@@ -92,7 +92,7 @@ URI is defined in std-66 [#std66]_::

 version-cmp   = "<" / "<=" / "!=" / "==" / ">=" / ">" / "~=" / "==="
 version   = 1*( DIGIT / ALPHA / "-" / "_" / "." / "*" )
-versionspec   = version-cmp version *(',' version-cmp version)
+versionspec   = ["("] version-cmp version *(',' version-cmp version) [")"]
 urlspec   = "@" URI

 Environment markers allow making a specification only take effect in some
@@ -179,7 +179,9 @@ Versions
 See PEP-440 [#pep440]_ for more detail on both version numbers and version
 comparisons. Version specifications limit the versions of a distribution that
 can be used. They only apply to distributions looked up by name, rather than
-via a URL. Version comparison are also used in the markers feature.
+via a URL. Version comparison are also used in the markers feature. The
+optional brackets around a version are present for compatibility with PEP-345
+[#pep345]_ but should not be generated, only accepted.

 Environment Markers
 ---
@@ -302,7 +304,7 @@ Backwards Compatibility
 Most of this PEP is already widely deployed and thus offers no compatibiltiy
 concerns.

-There are however two key points where the PEP differs from the deployed base.
+There are however a few points where the PEP differs from the deployed base.

 Firstly, PEP-440 direct references haven't actually been deployed in the wild,
 but they were designed to be compatibly added, and there are no known
@@ -320,15 +322,21 @@ permitting reasonably graceful upgrade. The new
version comparisons will cause
 errors, so adoption may require waiting some time for deployment to be
 widespread.

+Thirdly, PEP-345 required brackets around version specifiers. In order to
+accept PEP-345 dependency specifications, brackets are accepted, but they
+should not be generated.
+
 Rationale
 =

 In order to move forward with any new PEPs that depend on environment markers,
-we needed a specification that included them.
+we needed a specification that included them in their modern form. This PEP
+brings together all the currently unspecified components into a specified
+form.

 The requirement specifier EBNF is lifted from setuptools pkg_resources
-documentation, since we can't sensible depend on a defacto standard.
-
+documentation, since we wish to avoid depending on a defacto, vs PEP
+specified, standard.

 References
 ==


-Rob

--
Robert Collins 
Distinguished Technologist
HP Converged Cloud
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

[Paul Moore]

On 6 November 2015 at 07:39, Ralf Gommers  wrote:
> On Fri, Nov 6, 2015 at 12:37 AM, Donald Stufft  wrote:
>>
>> If ``pip install —build … —no-clean …`` worked to do incremental builds,
>> would that satisfy this use case? (without the —upgrade and —no-deps,
>> —no-deps is only needed because —upgrade and —upgrade is needed because of
>> another ticket that I think will get fixed at some point).
>
>
> Then there's at least a way to do it, but it's all very unsatisfying. Users
> are again going to have a hard time finding this. And I'd hate to have to
> type that every time.
>
> Robert and Nathaniel have argued the main points already so I'm not going to
> try to go in more detail, but I think the main point is:
>
>   - we want to replace `python setup.py install` with `pip install .` in
> order to get proper uninstalls and dependency handling.
>   - except for those two things, `python setup.py install` does the expected
> thing while pip is trying to be way too clever which is unhelpful.

While I understand what you're trying to achieve (and I'm in favour,
in general) it should be remembered that pip's core goal is installing
packages - not being a component of a development workflow.

We absolutely need to make pip useful in the development workflow type
of situation (that's why pip install -e exists, after all). But I
don't think it's so much pip "trying to be too clever" as incremental
rebuilds wasn't the original use case that "pip install ." was
designed for. What we'll probably have to do is be *more* clever to
special case out the situations where a development-style support for
incremental rebuilds is more appropriate than the current behaviour.

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Donald Stufft]

> On Nov 6, 2015, at 1:33 PM, Ionel Cristian Mărieș  wrote:
> 
> 
>> On Thu, Nov 5, 2015 at 11:08 PM, Donald Stufft  wrote:
>> Currently pip installs a number of commands like ``pip``, ``pipX`` and
>> ``pipX.Y`` where the X and X.Y corresponds to the version of Python that pip
>> is installed into. Pip installs into whatever Python is currently executing 
>> it
>> so this gives some ability to control which version of Python you're 
>> installing
>> into (``pip2.7`` for Python 2.7 etc).
> 
> ​Why not consider having a "pip" launcher?​ Seems the obvious thing to me - 
> python has the "py" launcher on windows and it works great!
> 
> Eg: "pip -3" to launch pip using python3, "pip -3.5" to launch pip using 
> python3.5 - just like the "py" 

Isn't this basically what the third option is?  Just the launcher is also the 
entire program. ___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] [Numpy-discussion] Proposal: stop supporting 'setup.py install'; start requiring 'pip install .' instead

[Nathaniel Smith]

On Mon, Nov 2, 2015 at 5:57 PM, Nathaniel Smith  wrote:
> On Sun, Nov 1, 2015 at 3:16 PM, Ralf Gommers  wrote:
>> 2. ``pip install .`` silences build output, which may make sense for some
>> usecases, but for numpy it just sits there for minutes with no output after
>> printing "Running setup.py install for numpy". Users will think it hangs and
>> Ctrl-C it. https://github.com/pypa/pip/issues/2732
>
> I tend to agree with the commentary there that for end users this is
> different but no worse than the current situation where we spit out
> pages of "errors" that don't mean anything :-). I posted a suggestion
> on that bug that might help with the apparent hanging problem.

For the record, this is now fixed in pip's "develop" branch and should
be in the next release. For commands like 'setup.py install', pip now
displays a spinner that ticks over whenever the underlying process
prints to stdout/stderr. So if the underlying process hangs, then the
spinner will stop (it's not just lying to you), but normally it works
nicely.

https://github.com/pypa/pip/pull/3224

-n

-- 
Nathaniel J. Smith -- http://vorpus.org
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Ionel Cristian Mărieș]

On Fri, Nov 6, 2015 at 11:13 PM, Donald Stufft  wrote:

> Eg: "pip -3" to launch pip using python3, "pip -3.5" to launch pip using
> python3.5 - just like the "py"
>
>
> Isn't this basically what the third option is?  Just the launcher is also
> the entire program.
>

​If you mean the initial mail you have send, I only saw two proposals:
`python -mpip` and something with zipfiles. Not sure if having a zipfile
around counts as a launcher, you wouldn't call something a launcher if it
contains the target completely no?



Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Why github and bitbucket?

[Paul Eipper]

On Thu, Nov 5, 2015 at 6:40 PM, Marcus Smith  wrote:
>>
>>
>> Basically: Historical reasons. The name “PyPA” was a joke by the
>> pip/virtualenv developers and it was only pip and virtualenv so it was on
>> Github.
>
>
> here's an anecdote per the pypa.io history page,  'Other proposed names
> were “ianb-ng”, “cabal”, “pack” and “Ministry of Installation”'
>
> https://www.pypa.io/en/latest/history/
>
> maybe even funnier that we have a history page, but it's easy to forget all
> that's happened, so I made one awhile back...
>

Oh man, too bad “Ministry of Installation” was not chosen, my favorite!

--
Paul Eipper
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] The future of invoking pip

[Donald Stufft]

On November 6, 2015 at 4:59:39 PM, Ionel Cristian Mărieș (cont...@ionelmc.ro) 
wrote:
> On Fri, Nov 6, 2015 at 11:13 PM, Donald Stufft wrote:
>  
> > Eg: "pip -3" to launch pip using python3, "pip -3.5" to launch pip using
> > python3.5 - just like the "py"
> >
> >
> > Isn't this basically what the third option is? Just the launcher is also
> > the entire program.
> >
>  
> If you mean the initial mail you have send, I only saw two proposals:
> `python -mpip` and something with zipfiles. Not sure if having a zipfile
> around counts as a launcher, you wouldn't call something a launcher if it
> contains the target completely no?
>  

Well, it’s not really a launcher no, but you’d do ``pip -p python2 install 
foo`` or something like that. It’s the same UI. Having just a “launcher” I 
think is actually more confusing (and we already had that in the past with -E 
and removed it because it was confusing). Since you’ll have different versions 
of pip in different environments (Python or virtual) things break or act 
confusingly.

-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig