Re: [Python-ideas] Looking for input to help with the pip situation

[Tzu-ping Chung]

Hi everyone,

I just discovered this thread and thought I this is a good chance I do some 
advertisement. I was helping a friend doing Python tutorial and face a 
somewhat similar situation with beginners using Windows. It is not 
particularly difficult to teach them to use py, but with the vast majority 
of resources not mentioning it, it’s a bit of a stretch to expect a 
beginner to know when to substitute what command to what after they finish 
the workshop. And a workshop is next to useless (in my opinion) if 
attendees can’t somehow go on by themselves after the event.

Anyway, I eventually decide it is best if Windows users can mimic how 
people run Python things on POSIX systems. The end product is this:

https://github.com/uranusjr/snafu

This works somehow like Homebrew and APT, but just for CPython. It allows 
you to use almost all the familiar commands, like *python3*, *pip3*, 
anything else you install with pip, on Windows. I made a conscious decision 
to not include python, pip, etc. Although I agree “python” is better in the 
long run, we are always stuck with the Python 2-3 situation for now, and 
need a solution to *somehow* make the distinction. But from what I’ve 
tested, everything else works as expected without much hassle.

The project is still in a very early stage (I only told a handful of 
friends about it), and I’ve put up working on it recently because my day 
job doesn’t allow me to think too much about it at the moment. But reading 
this thread I hope this can provide some ideas to others, and maybe find 
some interested people working with me on this issue.

Thanks!

TP

--
Tzu-ping Chung (@uranusjr)
uranu...@gmail.com
https://uranusjr.com
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

[Excuse any attribution errors, the quoting appears to have got badly
messed up somehow]

On 21 November 2017 at 06:21, Steve Barnes  wrote:
>
> On 21/11/2017 00:32, Chris Barker wrote:
>> I
>> don't know what would be the best approach for adding copies/links of
>> the launcher under other names, though.
>>
>>
>> this is the thing -- I wonder if py.exe has been a success at all :-)
>
> A lot of people, on windows, are using py.exe and pyw.exe without
> realising it - the default behaviour of the recent installer is to
> associate py.exe with *.py and pyw.exe with *.pyw so any user that is
> running python code by just typing the name of the file, or double
> clicking it, will be using it.

I use the py launcher a lot. Often just as "py" or "py -3.6". It
certainly isn't just limited to use via shebang lines.

I will say that I don't see beginners using it as much. That's
probably because there's so much documentation that says "use the
python command to start Python". That is of course an argument that we
should make the canonical command "python" rather than "py", but it
doesn't mean that if people find the "py" command they won't use it.
There's not a lot of evidence that isn't anecdotal either way on that
latter point.

>> So ti comes down to: does anyone USE py.exe???
>>
> As above.

And me.

>> we need th=e basic advice given out to *nix users to work:
>>
>> if you type "python" at the command line, and get python2, then you can
>> type "python3" to get python 3.

Um, we need the basic advice given to *everyone* to work. Insisting
that advice has to be "what Unix people do right now" is not the only
option. The advice "use python to run your default Python interpreter"
is too entrenched to change. The python2 and python3 aliases are a
platform-specific hack to get around the fact that some Linux
distributions couldn't switch the default Python to Python 3. Most of
the beginners I work with have no experience on Unix and no awareness
of python2/python3 commands.

"If you run python at the command line and get the wrong version of
Python, then..." - In that case you already have multiple versions of
Python on your system, and by the definitions we've been working to
here, you're an advanced user so you can fix it yourself.

Obviously, that's too harsh a stance for us to take. But let's not
ignore the fact that Unix users are typically *already* having to deal
with multiple versions of Python, and are typically more familiar with
command lines than Windows users. So there's no guarantee that a
solution that works for them is usable for Windows users who've never
used Python or the command line before, who downloaded Python 2.7 and
installed it, were then told "no, you should be using Python 3", and
installed 3.7, and now don't understand why "upgrading" didn't
uninstall the old version, what they need to do to work with Python 3,
etc...

> One of the nice things about embedding the behaviour into the launcher
> is that if the user/admin has installed a python that includes it it
> provides the behaviour for python installations that were already
> present and discoverable even if they didn't have it so if you have
> python 2.5 then after you install something with py.exe you can,
> currently, use py -2.5 to run it, (or py -2 if it is the only python 2).

+1. This is the biggest benefit for me. Also the fact that all the
older Python versions *don't need to be on PATH*. Having multiple
versions of Python on PATH is a bad thing (because of the Scripts
directory - if you want, I can explain in detail, but it's a side
issue for now).

>> I'm not willing to contradict Steve on this one.
>>
>  > I didn't know he'd made a clear statement about it.
>
> I am not sure that I spelt it out well enough but my personal feeling is
> that the installer should, (and I am reasonably sure that it should
> capable of this), default to add to path IF there is not a currently
> available python, (discoverable), and either default to don't add or
> prompt the installer if there is.

My apologies - I was unclear. I meant Steve Dower, who maintains the
installer, and who has strongly stated that the way the Windows PATH
works (separate system and user parts) makes sensible "add to PATH"
behaviour impossible. The reasons have all been described multiple
times, and I won't go into them again. Search the archives if you want
to see the details. Suffice it to say that wishing it was easy to
"just add Python to PATH" doesn't make it so...

> Of course the whole add to path or not becomes moot if the py.exe
> launcher is present as it is installed on the path, (C:\Windows), and
> then tries very hard just to "do the right thing".

That's why I prefer this option. There are still issues to address
(such as the Scripts directory), but it's much easier to keep the
complexity manageable with this approach.

>> But if people that understand these things say it's a bad idea, then
>> it's a bad idea. 

Re: [Python-ideas] Looking for input to help with the pip situation

[Steve Barnes]

On 21/11/2017 00:32, Chris Barker wrote:
> On Mon, Nov 20, 2017 at 3:24 PM, Paul Moore  > wrote:
> 
> On 20 November 2017 at 21:59, Chris Barker  > wrote:
>  > I don't understand any of this enough to have an opinion, so
> while I'd like
>  > to see py.exe be renamed python.exe, let's not let "the perfect
> be the enemy
>  > of good enough". So, if someone can make the case that they can
> restructure
>  > the whole py.exe / python.exe thing nicely in a way that will
> work, AND
>  > write the code to do it fairly quickly, then great!
> 
> I'm happy enough to modify the py.exe code to base its behaviour on
> the name it's called with, as Steve suggested. I'm not sure if that
> would require a PEP, but I'm willing to do one if it's felt that there
> is a need. It should be possible to get that change in for 3.7.
> 
> 
> very cool -- thanks!

I agree that it would be.

> 
> I
> don't know what would be the best approach for adding copies/links of
> the launcher under other names, though.
> 
> 
> this is the thing -- I wonder if py.exe has been a success at all :-)

A lot of people, on windows, are using py.exe and pyw.exe without 
realising it - the default behaviour of the recent installer is to 
associate py.exe with *.py and pyw.exe with *.pyw so any user that is 
running python code by just typing the name of the file, or double 
clicking it, will be using it.

This is the equivalent of the Unix chmod + shebang - I don't think that 
Windows users are ready for chown/chmod.
> 
> I honestly don't know - it's a cool idea, and I liked the idea of 
> replacing the #! line and all that, but the truth is that people simple 
> don't write scripts on Windows and expect them to "just work" the same 
> way they do on *nix systems. And if you do distutils/setuptools 
> "scripts" or "entry points" or whatever we are calling them, right, then 
> you can write utility scripts and install them, and use them
> 
> So ti comes down to: does anyone USE py.exe???
> 
As above.

> The fact that it's unique to Windows  makes it simply less discoverable, 
> and the fact that Windows users aren't used to the whole #! line thing, 
> means that as useful as it might be, it's not getting actual use.
> 
> In fact, I happened to be paying a lot of attention to this conversion 
> because at the very same moment, a group of us are developing a python 
> curriculum, and working on the "how to run a python file" part. We are 
> producing platform specific instructions, but decided NOT to include the 
> py.exe option for Windows users .. just more confusing things among a 
> lot of confusing things!
> 
> So who does teach people about it???

I quite regularly give people python courses and I always, regardless of 
target platform, tell them about the shebang and encoding lines, explain 
why they need them, including many editors basing syntax highlighting 
and character handling on them. I also point out that some nasty person, 
(me), has embedded the requirement to use them in the company coding 
standard for python and the review criteria for all code. (So carrot and 
stick).

> 
> Anyway, this whole thread has been about making the experience more 
> platform-independent -- ideally users should see docs without 
> Windows-specifc advice, and have it "just work" -- having a python3.exe  
> would make that more likely, so let's do that.
> 
> If we could py.exe enhanced and then use it instead of python.ex,m that 
> would be great. Though I at least still have no idea of that's possible.
> 
> But I don't see why we couldn't make a pyton3.exe NOW (in py3.7) that is 
> a copy (Or link, still don't get linking on Windows...) of python.exe, 
> that would help the situation, and if at some point the in the future, 
> python.exe, python3.exe and py.exe all become the same thing, great!
> 
> I think the launcher change is worthwhile in its own right. Even if we
> don't install any aliases by default, users will be able to manually
> add them.
> 
> 
> sure -- but no one would -- or not really no one, but it would become 
> maybe even a worse hassle of the same thing not working everywhere.
> 
> we need th=e basic advice given out to *nix users to work:
> 
> if you type "python" at the command line, and get python2, then you can 
> type "python3" to get python 3.

> 
> simple, straightforward, and it CAN help.
> 
>  > Otherwise, let's at least get a python3.exe into 3.7 -- and
> ideally into
>  > updates to 3.5, 3.6, and (python2.exe in this case, 2.7)
> 
> It's not going to be acceptable for 3.5, as that is now in security
> fix only mode. I'm not certain it's even acceptable for bugfix-only
> releases. It's a backward incompatible change (3.6.3 has no
> python3.exe but python 3.6.4 does) but maybe an acceptable one - we'd
>

Re: [Python-ideas] Looking for input to help with the pip situation

[Chris Barker]

On Mon, Nov 20, 2017 at 3:24 PM, Paul Moore  wrote:

> On 20 November 2017 at 21:59, Chris Barker  wrote:
> > I don't understand any of this enough to have an opinion, so while I'd
> like
> > to see py.exe be renamed python.exe, let's not let "the perfect be the
> enemy
> > of good enough". So, if someone can make the case that they can
> restructure
> > the whole py.exe / python.exe thing nicely in a way that will work, AND
> > write the code to do it fairly quickly, then great!
>
> I'm happy enough to modify the py.exe code to base its behaviour on
> the name it's called with, as Steve suggested. I'm not sure if that
> would require a PEP, but I'm willing to do one if it's felt that there
> is a need. It should be possible to get that change in for 3.7.


very cool -- thanks!


> I
> don't know what would be the best approach for adding copies/links of
> the launcher under other names, though.


this is the thing -- I wonder if py.exe has been a success at all :-)

I honestly don't know - it's a cool idea, and I liked the idea of replacing
the #! line and all that, but the truth is that people simple don't write
scripts on Windows and expect them to "just work" the same way they do on
*nix systems. And if you do distutils/setuptools "scripts" or "entry
points" or whatever we are calling them, right, then you can write utility
scripts and install them, and use them

So ti comes down to: does anyone USE py.exe???

The fact that it's unique to Windows  makes it simply less discoverable,
and the fact that Windows users aren't used to the whole #! line thing,
means that as useful as it might be, it's not getting actual use.

In fact, I happened to be paying a lot of attention to this conversion
because at the very same moment, a group of us are developing a python
curriculum, and working on the "how to run a python file" part. We are
producing platform specific instructions, but decided NOT to include the
py.exe option for Windows users .. just more confusing things among a lot
of confusing things!

So who does teach people about it???

Anyway, this whole thread has been about making the experience more
platform-independent -- ideally users should see docs without
Windows-specifc advice, and have it "just work" -- having a python3.exe
would make that more likely, so let's do that.

If we could py.exe enhanced and then use it instead of python.ex,m that
would be great. Though I at least still have no idea of that's possible.

But I don't see why we couldn't make a pyton3.exe NOW (in py3.7) that is a
copy (Or link, still don't get linking on Windows...) of python.exe, that
would help the situation, and if at some point the in the future,
python.exe, python3.exe and py.exe all become the same thing, great!


> I think the launcher change is worthwhile in its own right. Even if we
> don't install any aliases by default, users will be able to manually
> add them.


sure -- but no one would -- or not really no one, but it would become maybe
even a worse hassle of the same thing not working everywhere.

we need th=e basic advice given out to *nix users to work:

if you type "python" at the command line, and get python2, then you can
type "python3" to get python 3.

simple, straightforward, and it CAN help.


> > Otherwise, let's at least get a python3.exe into 3.7 -- and ideally into
> > updates to 3.5, 3.6, and (python2.exe in this case, 2.7)
>
> It's not going to be acceptable for 3.5, as that is now in security
> fix only mode. I'm not certain it's even acceptable for bugfix-only
> releases. It's a backward incompatible change (3.6.3 has no
> python3.exe but python 3.6.4 does) but maybe an acceptable one - we'd
> need feedback from the 3.6 and 2.7 release managers on that.
>

Fine -- we can leave that for future discussion


> Also, I'm assuming here you mean "create a copy of python.exe called
> python3.exe". If we do that, we risk making it harder to later switch
> to making "python3.exe" a link to the launcher - for the same reason
> that making "python.exe" be an alternative name for the launcher is
> problematic right now.
>

I'm lost -- how does it make it any harder, rather than exactly the same ???
(except two changes, rather than one...but they are the same change)

> And maybe make "add to PATH" be the default in the installer.
>
> I'm not willing to contradict Steve on this one.


I didn't know he'd made a clear statement about it.

But if people that understand these things say it's a bad idea, then it's a
bad idea. But we SHOULD make sure we are considering not just what could go
wrong, or how often it could go wrong, but who it is going to go wrong
for...

i.e. making it "do the right thing" for newbies is much more important that
making sure it doesn't "do the wrong thing" for more experienced folks that
can uncheck a box, or clean up their PATH after the fact

There are too many
> not-uncommon cases that we could mess up badly here. It's the 

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

>> BTW -- does pip install a "pip3" on Windows?
>
> No. Just "pip.exe".

Looks like I should have checked. As others pointed out, it does
(pip3.exe and pip3.6.exe). Apologies.
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 20 November 2017 at 21:59, Chris Barker  wrote:
> I don't understand any of this enough to have an opinion, so while I'd like
> to see py.exe be renamed python.exe, let's not let "the perfect be the enemy
> of good enough". So, if someone can make the case that they can restructure
> the whole py.exe / python.exe thing nicely in a way that will work, AND
> write the code to do it fairly quickly, then great!

I'm happy enough to modify the py.exe code to base its behaviour on
the name it's called with, as Steve suggested. I'm not sure if that
would require a PEP, but I'm willing to do one if it's felt that there
is a need. It should be possible to get that change in for 3.7. I
don't know what would be the best approach for adding copies/links of
the launcher under other names, though. And I don't have any
experience with the tools we use to build the installer, so that would
be for someone else. That part probably *would* need a PEP, and it may
well be controversial enough that getting agreement in time for 3.7
(i.e. in the next 2 months or so) will be challenging.

I think the launcher change is worthwhile in its own right. Even if we
don't install any aliases by default, users will be able to manually
add them. (And someone could write a 3rd party tool to manage creation
of such aliases, making that capability available to users who don't
have the necessary skills themselves).

> Otherwise, let's at least get a python3.exe into 3.7 -- and ideally into
> updates to 3.5, 3.6, and (python2.exe in this case, 2.7)

It's not going to be acceptable for 3.5, as that is now in security
fix only mode. I'm not certain it's even acceptable for bugfix-only
releases. It's a backward incompatible change (3.6.3 has no
python3.exe but python 3.6.4 does) but maybe an acceptable one - we'd
need feedback from the 3.6 and 2.7 release managers on that.

Also, I'm assuming here you mean "create a copy of python.exe called
python3.exe". If we do that, we risk making it harder to later switch
to making "python3.exe" a link to the launcher - for the same reason
that making "python.exe" be an alternative name for the launcher is
problematic right now.

> And maybe make "add to PATH" be the default in the installer.

I'm not willing to contradict Steve on this one. There are too many
not-uncommon cases that we could mess up badly here. It's the right
choice for "make new users' experience as easy as possible", but if we
do this at the cost of making the experience of people upgrading to
3.7 (possibly by installing 3.7 to gradually switch over, or starting
their migration from 2.7 with 3.7) unpleasant, then we risk getting
bad publicity for the 3.7 release.

The history of how we added Python to PATH across various versions is
messy and inconsistent. Add to that other distributions (Anaconda,
ActiveState) making different choices and adding yet more
inconsistency, means that anyone who *isn't* a brand new user probably
already has a tweaked, and likely fragile, setup. We're not doing them
any favours by adding another new behaviour. (And enabling "Add to
PATH" in 3.7 *would* be a new behaviour - I don't think we've enabled
add to path by default in any version since we switched to per-user
installs as the default).

We need *another* solution here. Not just variations on the existing
mess. That's why I like Steve's suggestion of making the launcher into
the canonical entry point. It's not easy, but at least it stands a
chance of breaking out of the cycle we're currently in, of switching
back and forth between "add to PATH" and "don't add to PATH".

> Those are quick and simple to do, result in little disruption, and make the
> whole cross-platform thing more manageable.

They aren't that simple. I've already discussed "add to PATH". And if
we add python3.exe, we have to consider questions like will venv be
modified to include it in virtual environments? Will virtualenv? If
not, will "python3" run the system Python rather than the current
virtualenv? That's just as much a problem for the option of having
python3 be an alias for the launcher, of course - but my point here is
you have to think about questions like this even for your "simple"
approach.

I just don't think there *is* a quick and simple solution here. So
better not to rush into a solution that isn't fully thought through,
IMO. "Although never is often better than *right* now" is the relevant
Zen here, not "Now is better than never" (nobody's suggesting we
*never* fix this).

> BTW -- does pip install a "pip3" on Windows?

No. Just "pip.exe".

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Mark Lawrence]

On 20/11/17 21:59, Chris Barker wrote:


BTW -- does pip install a "pip3" on Windows?

-CHB

--

Christopher Barker, Ph.D.
Oceanographer



pip.exe, pip3.exe and pip3.6.exe are all in C:\Program 
Files\Python36\Scripts on my Windows 10 setup installed for all users.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Eric Fahlgren]

​
On Mon, Nov 20, 2017 at 1:59 PM, Chris Barker  wrote:

> BTW -- does pip install a "pip3" on Windows?
>
​
In 3.6 it does:

> find t:/Python36/ -iname 'pip*exe'
t:/Python36/Scripts/pip.exe
t:/Python36/Scripts/pip3.6.exe
t:/Python36/Scripts/pip3.exe


​
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Chris Barker]

On Fri, Nov 17, 2017 at 2:14 AM, Steve Barnes 
wrote:

>
> >>> On Wed, Nov 15, 2017 at 11:07 AM, Steve Dower 
>  My preferred solution for this is to rename "py.exe" to "python.exe"
>

...


>
> Note that renaming py.exe to python.exe would have several problems:
>
> ...

I don't understand any of this enough to have an opinion, so while I'd like
to see py.exe be renamed python.exe, let's not let "the perfect be the
enemy of good enough". So, if someone can make the case that they can
restructure the whole py.exe / python.exe thing nicely in a way that will
work, AND write the code to do it fairly quickly, then great!

Otherwise, let's at least get a python3.exe into 3.7 -- and ideally into
updates to 3.5, 3.6, and (python2.exe in this case, 2.7)

And maybe make "add to PATH" be the default in the installer.

Those are quick and simple to do, result in little disruption, and make the
whole cross-platform thing more manageable.

BTW -- does pip install a "pip3" on Windows?

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Steve Barnes]

On 17/11/2017 09:13, Paul Moore wrote:
> On 17 November 2017 at 01:57, Nick Coghlan  wrote:
>> On 17 November 2017 at 05:15, Chris Barker  wrote:
>>>
>>> On Wed, Nov 15, 2017 at 11:07 AM, Steve Dower 
>>> wrote:

 If you write such a PEP, please also research and write up the issues
 with modifying PATH on Windows (they're largely scattered throughout
 bugs.p.o and earlier discussions on python-dev).
>>>
>>>
>>> Is anyone proposing doing anything new with that? (other than changing the
>>> default)
>>>
 My preferred solution for this is to rename "py.exe" to "python.exe"
>>>
>>>
>>> I was going to propose that in this thread, but then thought: "there has
>>> GOT to be a reason why that reall obvious solution wan't done in the first
>>> place", and didn't have time to go back and research it.
>>
>>
>> As far as I recall, the arguments against it are:
>>
>> - wanting the regular executable and the launcher to be able to coexist in
>> the same build target directory
>> - not wanting the regular python executable to prevent access to the
>> launcher at a shell prompt
>> - not wanting the launcher at a shell prompt to prevent access to the
>> regular executable at a shell prompt
>>
>> However, 
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.python.org%2Fdev%2Fpeps%2Fpep-0397%2F=02%7C01%7Cgadgetsteve%40live.co.uk%7C18fce28e3baa4e4e70c708d52d9b80c5%7C84df9e7fe9f640afb435%7C1%7C0%7C636465068254194278=4%2BAusacb7TwM5NY9xEUXx%2B%2Ff7s%2BTvV05xIhUWtn9xjg%3D=0
>>  doesn't spell those out,
>> it just states what the launcher's name will be, and emphasises that the
>> main purpose is to provide a sensible target for file associations after the
>> "always use the most recently installed version" assumption broke down:
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.python.org%2Fdev%2Fpeps%2Fpep-0397%2F%23rationale=02%7C01%7Cgadgetsteve%40live.co.uk%7C18fce28e3baa4e4e70c708d52d9b80c5%7C84df9e7fe9f640afb435%7C1%7C0%7C636465068254194278=OVs0HEVnlob8Rj5LQewOE0YoyIvIA9ydar7i0b1%2BZfY%3D=0
>>
>> Addressing them now:
>>
>> * as long as the extra hard links are only generated at install time, there
>> also won't be any problems with build directory name conflicts.
>> * the launcher will always be available a `py`, regardless of the current
>> PATH
>> * PATH in a venv will still put the regular python executable ahead of the
>> launcher
> 
> Note that if I *do* get round to working on this, my primary intention
> will be to propose altering the launcher to allow it to be used under
> different names, as described by Steve. However, I won't initially be
> proposing that we add any additional links to the launcher by default,
> leaving that for users to do manually, and/or for later revisions of
> the PEP (or further PEPs) to propose this.
> 
> Reasons:
> 
> 1. The launcher is typically in C:\Windows, which has a very high
> priority on PATH, so getting the environment right will be tricky.
> 2. I don't have any real experience with the installer.
> 3. Backporting this change, or dealing with older versions of Python
> that don't include the new launcher, is a complicated question.
> 
> I'd rather keep the initial PEP restricted to the simple behaviour
> change. It's possible that the behaviour change may not even need a
> PEP, but I think it'd be better to have one. The "how do we use the
> new behaviour in the installers" question is likely to be much more
> controversial...
> 
> Paul
> 
> PS This (particularly the replacement of python.exe) is almost
> certainly too late for 3.7, so it's not as if there's a rush to answer
> all the questions at once :-)
> ___

Note that renaming py.exe to python.exe would have several problems:

1. It would only address things on Windows or if py were created for 
other platforms as python there would be issues with trying to ensure 
that it was the one that was called first in every case.
1. We would have to also rename python.exe to avoid py.exe executing itself.
1. The current behaviour of py.exe doesn't quite match what I was 
suggesting in that it always executes a python, (the default or 
specified), or in 3.7 with the -0 lists the ones that it can find, so 
would not really address the users installing to the wrong context. I 
was suggesting that pip, and possibly other tools, would simply execute 
the command if there was only one python in the current context or 
prompt the user if there were more than one. (It occurs to me that it 
might also be valuable to have an all option, e.g. for admins installing 
security updates).

I do like the idea to making this sort of behaviour available, possibly 
as soft-links or alternate entry points, to tools other than pip as that 
would be very handy in some cases, so having this behaviour in a python 
library that pip and other tools could 

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 17 November 2017 at 01:57, Nick Coghlan  wrote:
> On 17 November 2017 at 05:15, Chris Barker  wrote:
>>
>> On Wed, Nov 15, 2017 at 11:07 AM, Steve Dower 
>> wrote:
>>>
>>> If you write such a PEP, please also research and write up the issues
>>> with modifying PATH on Windows (they're largely scattered throughout
>>> bugs.p.o and earlier discussions on python-dev).
>>
>>
>> Is anyone proposing doing anything new with that? (other than changing the
>> default)
>>
>>> My preferred solution for this is to rename "py.exe" to "python.exe"
>>
>>
>> I was going to propose that in this thread, but then thought: "there has
>> GOT to be a reason why that reall obvious solution wan't done in the first
>> place", and didn't have time to go back and research it.
>
>
> As far as I recall, the arguments against it are:
>
> - wanting the regular executable and the launcher to be able to coexist in
> the same build target directory
> - not wanting the regular python executable to prevent access to the
> launcher at a shell prompt
> - not wanting the launcher at a shell prompt to prevent access to the
> regular executable at a shell prompt
>
> However, https://www.python.org/dev/peps/pep-0397/ doesn't spell those out,
> it just states what the launcher's name will be, and emphasises that the
> main purpose is to provide a sensible target for file associations after the
> "always use the most recently installed version" assumption broke down:
> https://www.python.org/dev/peps/pep-0397/#rationale
>
> Addressing them now:
>
> * as long as the extra hard links are only generated at install time, there
> also won't be any problems with build directory name conflicts.
> * the launcher will always be available a `py`, regardless of the current
> PATH
> * PATH in a venv will still put the regular python executable ahead of the
> launcher

Note that if I *do* get round to working on this, my primary intention
will be to propose altering the launcher to allow it to be used under
different names, as described by Steve. However, I won't initially be
proposing that we add any additional links to the launcher by default,
leaving that for users to do manually, and/or for later revisions of
the PEP (or further PEPs) to propose this.

Reasons:

1. The launcher is typically in C:\Windows, which has a very high
priority on PATH, so getting the environment right will be tricky.
2. I don't have any real experience with the installer.
3. Backporting this change, or dealing with older versions of Python
that don't include the new launcher, is a complicated question.

I'd rather keep the initial PEP restricted to the simple behaviour
change. It's possible that the behaviour change may not even need a
PEP, but I think it'd be better to have one. The "how do we use the
new behaviour in the installers" question is likely to be much more
controversial...

Paul

PS This (particularly the replacement of python.exe) is almost
certainly too late for 3.7, so it's not as if there's a rush to answer
all the questions at once :-)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 17 November 2017 at 05:15, Chris Barker  wrote:

> On Wed, Nov 15, 2017 at 11:07 AM, Steve Dower 
> wrote:
>
>> If you write such a PEP, please also research and write up the issues
>> with modifying PATH on Windows (they're largely scattered throughout
>> bugs.p.o and earlier discussions on python-dev).
>>
>
> Is anyone proposing doing anything new with that? (other than changing the
> default)
>
> My preferred solution for this is to rename "py.exe" to "python.exe"
>
>
> I was going to propose that in this thread, but then thought: "there has
> GOT to be a reason why that reall obvious solution wan't done in the first
> place", and didn't have time to go back and research it.
>

As far as I recall, the arguments against it are:

- wanting the regular executable and the launcher to be able to coexist in
the same build target directory
- not wanting the regular python executable to prevent access to the
launcher at a shell prompt
- not wanting the launcher at a shell prompt to prevent access to the
regular executable at a shell prompt

However, https://www.python.org/dev/peps/pep-0397/ doesn't spell those out,
it just states what the launcher's name will be, and emphasises that the
main purpose is to provide a sensible target for file associations after
the "always use the most recently installed version" assumption broke down:
https://www.python.org/dev/peps/pep-0397/#rationale

Addressing them now:

* as long as the extra hard links are only generated at install time, there
also won't be any problems with build directory name conflicts.
* the launcher will always be available a `py`, regardless of the current
PATH
* PATH in a venv will still put the regular python executable ahead of the
launcher

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Chris Barker]

On Wed, Nov 15, 2017 at 11:07 AM, Steve Dower 
wrote:

> If you write such a PEP, please also research and write up the issues with
> modifying PATH on Windows (they're largely scattered throughout bugs.p.o
> and earlier discussions on python-dev).
>

Is anyone proposing doing anything new with that? (other than changing the
default)

My preferred solution for this is to rename "py.exe" to "python.exe"


I was going to propose that in this thread, but then thought: "there has
GOT to be a reason why that reall obvious solution wan't done in the first
place", and didn't have time to go back and research it.

But if it's feasible from a technical and backward-compatible perspective,
great!

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Wes Turner]

Would something like this be helpful?
(`python -m pip` seems to work fine after `conda install pip` after a
miniconda install)

https://gist.github.com/westurner/80e81cd4bf9b79acb5989622f2621655

```python
#!/usr/bin/env python
"""
Something like this as e.g site.discover
(for use as ``python -m site.discover``)
could be helpful for explaining and diagnosing multiple pythons and pips
"""

import os
import re
from distutils.spawn import find_executable
from subprocess import check_output, STDOUT
import collections


def discover(path=None, printsite=False, uniques=True):
"""
Find all ``python*`` and ``pip*`` executables and their version strings
within ``os.environ['PATH']``

Kwargs:
path (str): os.pathsep-delimited path str
(defaults to ``os.environ['PATH']``)
printsite (bool): call `` -m site``
with each found python binary
uniques (bool): print uniques according to ``os.realpath`` at the
end

Returns:
None
"""
if path is None:
path = os.environ['PATH']
rgx = re.compile(r'(python|pip)\-?(\d?\.?\d?)$')
uniques = collections.OrderedDict()
for directory in path.split(os.pathsep):
paths = sorted(
((match.group(0), match.group(2)) for match in
(rgx.match(name) for name in os.listdir(directory))
 if match),
key=lambda x: x[::-1],
reverse=True)
print(u'# %s' % directory)
for (name, ver) in paths:
pathjoined = os.path.join(directory, name)
binpath = find_executable(pathjoined, directory)
realpath = os.path.realpath(binpath)
if binpath is None:
continue
if os.path.exists(pathjoined):
if os.path.islink(pathjoined):
print((
u"%r is a symlink to %r, which doesn't exist"
% (pathjoined, binpath)))
# TODO: %r exists but isnt executable
verstring = check_output((binpath, '-V'),
 stderr=STDOUT).rstrip()
uniques.setdefault(realpath, collections.OrderedDict()) \
.setdefault(binpath, verstring)
print(u'%-11r %-5r %r' % (name, ver, verstring))
if printsite and name.startswith('python'):
sitestring = check_output((binpath, '-m', 'site'))
lines = (u'> %s' % l for l in sitestring.splitlines())
for line in lines:
print(line)
# TODO: check_output((binpath, '-m', 'pip'))

# TODO: AND/OR (?) only print uniques at the end
if uniques:
print('### UNIQUES')
for path in uniques:
print('## %s' % path)
paths = uniques.get(path)
for otherpath in sorted(paths):
print(otherpath)
print('')


if __name__ == '__main__':
discover()

```


On Wednesday, November 15, 2017, Steve Dower  wrote:

> On 15Nov2017 0617, Nick Coghlan wrote:
>
>> On 15 November 2017 at 22:46, Michel Desmoulin > > wrote:
>> Should I do a PEP with a summary of all the stuff we discussed ?
>> I think a Windows-specific PEP covering adding PATH updates back to the
>> default installer behaviour, and adding pythonX and pythonX.Y commands
>> would be useful (and Guido would presumably delegate resolving that to
>> Steve Dower as the Windows installer maintainer).
>>
>
> If you write such a PEP, please also research and write up the issues with
> modifying PATH on Windows (they're largely scattered throughout bugs.p.o
> and earlier discussions on python-dev).
>
> Once you realise the tradeoff involved in modifying these global settings,
> you'll either come around to my point of view or be volunteering to take
> *all* the support questions when they come in :)
>
> The one thing I'd ask is that any such PEP *not* advocate for promoting
>> ther variants as the preferred way of invoking Python on Windows - rather,
>> they should be positioned as a way of making online instructions written
>> for Linux more likely to "just work" for folks on Windows (similar to the
>> utf-8 encoding changes in https://www.python.org/dev/peps/pep-0529/)
>>
>> Instead, the focus should be on ensuring the "python -m pip install" and
>> "pip install" both work after clicking through the installer without
>> changing any settings, and devising a troubleshooting guide to help folks
>> that are familiar with computers and Python, but perhaps not with Windows,
>> guide folks to a properly working environment.
>>
>
> My preferred solution for this is to rename "py.exe" to "python.exe" (or
> rather, make a copy of it with the new name), and extend (or more likely,
> rewrite) the launcher such that:
>
> * if argv[0] == "py.exe", use PEP 514 company/tag resolution to 

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 16 November 2017 at 06:49, Nick Coghlan  wrote:
> On 16 November 2017 at 05:29, Zachary Ware 
> wrote:
>>
>> On Wed, Nov 15, 2017 at 1:07 PM, Steve Dower 
>> wrote:
>> > My preferred solution for this is to rename "py.exe" to "python.exe" (or
>> > rather, make a copy of it with the new name), and extend (or more
>> > likely,
>> > rewrite) the launcher such that:
>> >
>> > * if argv[0] == "py.exe", use PEP 514 company/tag resolution to find and
>> > launch Python based on first command line argument
>> > * if argv[0] == "python.exe", find the matching
>> > PythonCore/ install (where tag may be a partial match - e.g.
>> > "python3.exe" finds the latest PythonCore/3.x)
>> > * else, if argv[0] == ".exe, find the matching
>> > PythonCore/ install and launch "-m "
>> >
>> > With the launcher behaving like this, we can make as many hard links as
>> > we
>> > want in its install directory (it only gets installed once, so only
>> > needs
>> > one PATH entry, and this is C:\Windows for admin installs):
>> > * python.exe
>> > * python2.exe
>> > * python3.exe
>> > * python3.6.exe
>> > * pip.exe
>> > * pip2.exe
>> > * pip3.exe
>>
>> I haven't been following this thread closely, but this sounds lovely.
>> I'm not terribly keen on cluttering up C:\Windows with this, but
>> that's a minor issue.
>
>
> I'd missed Steve's post before writing my last one. This sounds like a
> really nice technical solution to me, too, especially as it will handle
> Python 2 as well (even for Python 2 only systems, the launcher is available
> as an independently installable executable).
>
> Regardless of the underlying implementation details though, a PEP would be a
> helpful way of writing it up so we can make sure packaging.python.org and
> other resources properly account for it.

OK, I'll add that to the list of things to look at.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 16 November 2017 at 05:29, Zachary Ware 
wrote:

> On Wed, Nov 15, 2017 at 1:07 PM, Steve Dower 
> wrote:
> > My preferred solution for this is to rename "py.exe" to "python.exe" (or
> > rather, make a copy of it with the new name), and extend (or more likely,
> > rewrite) the launcher such that:
> >
> > * if argv[0] == "py.exe", use PEP 514 company/tag resolution to find and
> > launch Python based on first command line argument
> > * if argv[0] == "python.exe", find the matching
> > PythonCore/ install (where tag may be a partial match - e.g.
> > "python3.exe" finds the latest PythonCore/3.x)
> > * else, if argv[0] == ".exe, find the matching
> > PythonCore/ install and launch "-m "
> >
> > With the launcher behaving like this, we can make as many hard links as
> we
> > want in its install directory (it only gets installed once, so only needs
> > one PATH entry, and this is C:\Windows for admin installs):
> > * python.exe
> > * python2.exe
> > * python3.exe
> > * python3.6.exe
> > * pip.exe
> > * pip2.exe
> > * pip3.exe
>
> I haven't been following this thread closely, but this sounds lovely.
> I'm not terribly keen on cluttering up C:\Windows with this, but
> that's a minor issue.
>

I'd missed Steve's post before writing my last one. This sounds like a
really nice technical solution to me, too, especially as it will handle
Python 2 as well (even for Python 2 only systems, the launcher is available
as an independently installable executable).

Regardless of the underlying implementation details though, a PEP would be
a helpful way of writing it up so we can make sure packaging.python.org and
other resources properly account for it.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Zachary Ware]

On Wed, Nov 15, 2017 at 1:07 PM, Steve Dower  wrote:
> My preferred solution for this is to rename "py.exe" to "python.exe" (or
> rather, make a copy of it with the new name), and extend (or more likely,
> rewrite) the launcher such that:
>
> * if argv[0] == "py.exe", use PEP 514 company/tag resolution to find and
> launch Python based on first command line argument
> * if argv[0] == "python.exe", find the matching
> PythonCore/ install (where tag may be a partial match - e.g.
> "python3.exe" finds the latest PythonCore/3.x)
> * else, if argv[0] == ".exe, find the matching
> PythonCore/ install and launch "-m "
>
> With the launcher behaving like this, we can make as many hard links as we
> want in its install directory (it only gets installed once, so only needs
> one PATH entry, and this is C:\Windows for admin installs):
> * python.exe
> * python2.exe
> * python3.exe
> * python3.6.exe
> * pip.exe
> * pip2.exe
> * pip3.exe

I haven't been following this thread closely, but this sounds lovely.
I'm not terribly keen on cluttering up C:\Windows with this, but
that's a minor issue.

-- 
Zach
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Steve Dower]

On 15Nov2017 0617, Nick Coghlan wrote:
On 15 November 2017 at 22:46, Michel Desmoulin 
> wrote:

Should I do a PEP with a summary of all the stuff we discussed ?
I think a Windows-specific PEP covering adding PATH updates back to the 
default installer behaviour, and adding pythonX and pythonX.Y commands 
would be useful (and Guido would presumably delegate resolving that to 
Steve Dower as the Windows installer maintainer).


If you write such a PEP, please also research and write up the issues 
with modifying PATH on Windows (they're largely scattered throughout 
bugs.p.o and earlier discussions on python-dev).


Once you realise the tradeoff involved in modifying these global 
settings, you'll either come around to my point of view or be 
volunteering to take *all* the support questions when they come in :)


The one thing I'd ask is that any such PEP *not* advocate for promoting 
ther variants as the preferred way of invoking Python on Windows - 
rather, they should be positioned as a way of making online instructions 
written for Linux more likely to "just work" for folks on Windows 
(similar to the utf-8 encoding changes in 
https://www.python.org/dev/peps/pep-0529/)


Instead, the focus should be on ensuring the "python -m pip install" and 
"pip install" both work after clicking through the installer without 
changing any settings, and devising a troubleshooting guide to help 
folks that are familiar with computers and Python, but perhaps not with 
Windows, guide folks to a properly working environment.


My preferred solution for this is to rename "py.exe" to "python.exe" (or 
rather, make a copy of it with the new name), and extend (or more 
likely, rewrite) the launcher such that:


* if argv[0] == "py.exe", use PEP 514 company/tag resolution to find and 
launch Python based on first command line argument
* if argv[0] == "python.exe", find the matching 
PythonCore/ install (where tag may be a partial match - e.g. 
"python3.exe" finds the latest PythonCore/3.x)
* else, if argv[0] == ".exe, find the matching 
PythonCore/ install and launch "-m "


With the launcher behaving like this, we can make as many hard links as 
we want in its install directory (it only gets installed once, so only 
needs one PATH entry, and this is C:\Windows for admin installs):

* python.exe
* python2.exe
* python3.exe
* python3.6.exe
* pip.exe
* pip2.exe
* pip3.exe

As well as allowing e.g. "py.exe -anaconda36-64 ..." to reliably locate 
and run non-Python.org installs.


It needs to be fully specced out, obviously, and we may want to move the 
all-users install to its own directory to reduce clutter, but part of 
the reason behind PEP 514 was to enable this sort of launcher. It could 
even extend to "you don't have this version right now, want to download 
and install it?"


And finally it should be fairly obvious that this doesn't have to be a 
core Python tool. It has no reliance on anything in core (that isn't 
already specified in a PEP) and could be written totally independently. 
I've tried (weakly) to get work time allocated to this in the past, and 
if it's genuinely not going to get done unless I do it then I'll try again.


Cheers,
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 15 November 2017 at 22:46, Michel Desmoulin 
wrote:

> Le 13/11/2017 à 19:57, Chris Barker a écrit :
> > 3) Make --user be be automatic for pip install. Not actually the
> > default, but pip could do a user install if you don't have the
> > permissions for a non-user install.
>
> Breaking compat ? Not sure people will accept.
>

This isn't actually a Python level change - it's a pip UX level one, and
already on their todo list: https://github.com/pypa/pip/issues/1668

I believe the next concrete step that can be taken there is to actually add
an explicit `--global` flag, so I've belatedly broken that out as its own
issue: https://github.com/pypa/pip/issues/4865

=
>
> Should I do a PEP with a summary of all the stuff we discussed ?
>

I think a Windows-specific PEP covering adding PATH updates back to the
default installer behaviour, and adding pythonX and pythonX.Y commands
would be useful (and Guido would presumably delegate resolving that to
Steve Dower as the Windows installer maintainer).

The one thing I'd ask is that any such PEP *not* advocate for promoting
ther variants as the preferred way of invoking Python on Windows - rather,
they should be positioned as a way of making online instructions written
for Linux more likely to "just work" for folks on Windows (similar to the
utf-8 encoding changes in https://www.python.org/dev/peps/pep-0529/)

Instead, the focus should be on ensuring the "python -m pip install" and
"pip install" both work after clicking through the installer without
changing any settings, and devising a troubleshooting guide to help folks
that are familiar with computers and Python, but perhaps not with Windows,
guide folks to a properly working environment.

The update to the Windows installer would then start offering benefits as
soon as Python 3.7 becomes the default download, while the troubleshooting
guide would be beneficial as soon as folks start learning about it's
existence.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 13/11/2017 à 19:57, Chris Barker a écrit :
> This has gotten to be a big thread, and it's a pretty intractable
> problem, but I think there are a few fairly small things that could be
> done to at least make it a bit easier:
> 
> 1) Add python2.exe and python3.exe files to the Windows installers -- am
> I insane or did Windows used to have that? I really think it did --
> maybe got removed when py.exe was added.

More than that. Add pythonX.X. You may want to run Python3.5 and not
Python3.6.


>   1a) alternatively, we could add a "py" executable to the standard
> linux builds, so there would be THAT one way to do it. But I think
> that's a "BAD IDEA" -- the whole "py" thing is not widely know or used,
> it's not going to show up in package install instructions for a LONG
> time, (actualy we could do both anyway)

It would work better and be cleaner, but is unlikely to happen. 1) is
simpler and have a chance to happen.



> Then "python2 -m pip install" would work everywhere (only with new
> installations, but at least with newbies, that's a bit more likely ...)
> 
>

Yeah, if 1) happens, we should promote -m for pip.

> 2) Make adding to the PATH in Windows the default. I think there are
> really three user groups:
> 
>    - newbies starting from scratch -- they want it on the PATH
> 
>    - newbies with whatever left over cruft from previous installations
> on their systems -- they want it at the FRONT of their PATH. They
> SHOULD  uninstall all the cruft, but if they don't this will still work
> with as few surprises a possible.
> 
>    - not-newbies with a previous version of python they need to continue
> using. They can uncheck the box, or use py.exe

Agreed. But if an existing python exist in the path, there should be a
warning.

> 
> 
> 3) Make --user be be automatic for pip install. Not actually the
> default, but pip could do a user install if you don't have the
> permissions for a non-user install.

Breaking compat ? Not sure people will accept.

=

Should I do a PEP with a summary of all the stuff we discussed ?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 15 November 2017 at 19:51, Paul Moore  wrote:

> On 15 November 2017 at 08:22, Nick Coghlan  wrote:
> > On 15 November 2017 at 16:13, Steve Barnes 
> wrote:
> >>
> >>   - "pip -X[.Y][-32|-64] operation ..." tries to find a python matching
> >> -X[.Y][-32|-64] and if it succeeds executes "python -m pip operation
> >> ..." with that python, (if it doesn't find a matching python is should
> >> fail with a sensible error message and possibly list the valid python
> >> specifiers).
> >
> >
> > This is a genuinely interesting option, especially as `pipenv` has
> already
> > implemented a feature somewhat akin to this:
> > https://docs.pipenv.org/basics.html#specifying-versions-of-python
> >
> > `pipenv` also allows `pipenv --two` and `pipenv --three` when setting up
> > your initial virtual environment.
>
> This is an interesting idea for *any* tool that's about "working with
> Python environments" as opposed to "writing Python code". So pip,
> virtualenv, tox, pipenv, ... Many of these tools are variously
> reinventing "tell me which Python environment to work on" options.
> Having a common way to do this would be really useful. I'm not sure
> how likely it would be for pip to be able to use it (pip introspects
> sys.executable to get site-packages, etc), but it's certainly a
> possibility.
>
> Having a standardised library/wrapper that handles the "select a
> Python environment" process would be a huge plus. There's
> https://github.com/zooba/pep514tools which is a start on handling the
> Windows registry scanning logic, and building on that to include Unix
> and anything else we needed would be great.
>

While I suspect pep514tools itself may not end up being the right place, I
filed https://github.com/zooba/pep514tools/issues/2 to start discussing
that idea further. For other platforms, just naively scanning `PATH` is
often good enough, so it's mainly Windows where discovery needs to be a bit
more aware of platform specific details (i.e. PEP 514's registry entries).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 15 November 2017 at 08:22, Nick Coghlan  wrote:
> On 15 November 2017 at 16:13, Steve Barnes  wrote:
>>
>>   - "pip -X[.Y][-32|-64] operation ..." tries to find a python matching
>> -X[.Y][-32|-64] and if it succeeds executes "python -m pip operation
>> ..." with that python, (if it doesn't find a matching python is should
>> fail with a sensible error message and possibly list the valid python
>> specifiers).
>
>
> This is a genuinely interesting option, especially as `pipenv` has already
> implemented a feature somewhat akin to this:
> https://docs.pipenv.org/basics.html#specifying-versions-of-python
>
> `pipenv` also allows `pipenv --two` and `pipenv --three` when setting up
> your initial virtual environment.

This is an interesting idea for *any* tool that's about "working with
Python environments" as opposed to "writing Python code". So pip,
virtualenv, tox, pipenv, ... Many of these tools are variously
reinventing "tell me which Python environment to work on" options.
Having a common way to do this would be really useful. I'm not sure
how likely it would be for pip to be able to use it (pip introspects
sys.executable to get site-packages, etc), but it's certainly a
possibility.

Having a standardised library/wrapper that handles the "select a
Python environment" process would be a huge plus. There's
https://github.com/zooba/pep514tools which is a start on handling the
Windows registry scanning logic, and building on that to include Unix
and anything else we needed would be great.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 15 November 2017 at 16:13, Steve Barnes  wrote:

>   - "pip -X[.Y][-32|-64] operation ..." tries to find a python matching
> -X[.Y][-32|-64] and if it succeeds executes "python -m pip operation
> ..." with that python, (if it doesn't find a matching python is should
> fail with a sensible error message and possibly list the valid python
> specifiers).
>

This is a genuinely interesting option, especially as `pipenv` has already
implemented a feature somewhat akin to this:
https://docs.pipenv.org/basics.html#specifying-versions-of-python

`pipenv` also allows `pipenv --two` and `pipenv --three` when setting up
your initial virtual environment.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Steve Barnes]

>>> We still have to deal with the fact that basically every Unix
>>> environment is "advanced" in the above sense (the python2/python3
>>> split). I don't have a solution for that (other than "upgrade to
>>> Windows" ;-)).
>>
>> Provide the "py" command on linux and mac. And make it the default
>> recommended way in the documentation.
> 
> +1 from me.
> 


How about update pip so that, from the command line it can accept pretty 
much the same syntax as the py launcher:

  - "python -m pip operation ..." all use the same python as is running 
the pip module.

  - "pip -X[.Y][-32|-64] operation ..." tries to find a python matching 
-X[.Y][-32|-64] and if it succeeds executes "python -m pip operation 
..." with that python, (if it doesn't find a matching python is should 
fail with a sensible error message and possibly list the valid python 
specifiers).

"pip operation ..." searches the available python contexts and, if there 
is only one, (or in a venv), calls "python -m pip operation ..." using 
it but, if there is more than one candidate, (or in not in a venv), 
tells the user that they have more than one python and need to specify 
which, listing the options.

"pip -0" searches the available pythons and lists the available contexts.

Ideally the list contexts operations would differentiate between user 
space and system contexts and display them suitably based on the current 
user privileges.

This would:
  - maintain the current behaviour as much as possible,
  - not break all of the existing examples in the wild & literature
  - avoid trying to introduce a new launcher command on Mac/Linux,
  - propagate reasonably quickly, as people are already fairly used to 
being prompted to update pip,
  - Automatically propagate to the default pip as long as it was back 
ported to at least 2.7, possibly further.
  - Allow users who do need to support multiple python installations to 
do so, (I am one of those).
  - Does not require updates to python itself.

-- 
Steve (Gadget) Barnes
Any opinions in this message are my personal opinions and do not reflect 
those of my employer.

---
This email has been checked for viruses by AVG.
http://www.avg.com

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 15 November 2017 at 01:23, Stephan Houben  wrote:

> Hi Nick,
>
> 2017-11-14 11:07 GMT+01:00 Nick Coghlan :
>
>> On 14 November 2017 at 16:47, Michel Desmoulin > > wrote:
>>
>>> Proposal A:
>>> ---
>>>
>>> Suffix Python executable on Windows like on Unix, so that people will
>>> type pythonX.X if they want a specify version.
>>>
>>> Pros: easy and discoverable.
>>>
>>> Cons: you need a lot of stuff in the system path.
>>>
>>
>> Con: we hope to have the problem resolved on the Linux distro side such
>> that "python" typically means "python" by the time community support for
>> Python 2 ends in 2020. Since Windows has gone the better part of two
>> decades without version Python commands, adding them because we're
>> impatient with the pace of change at the Linux distro level doesn't really
>> make sense (especially when Linux holds such a small fraction of the
>> non-phone client device market).
>>
>
> Perhaps I could sell you on the idea of a Windows "python3" executable,
> not as the New Official Way to do things,
> but rather as a pragmatic measure to make code from those Linux weirdos
> ;-)
>

Aye, I'm not opposed to adding pythonX and pythonX.Y binaries or symlinks
to the Windows installers as a way to make Linux-assuming tutorials
slightly more likely to work for Windows users.

I'm only opposed to promoting that as the new preferred way of launching
Python from the Windows command line, if for no other reason than if the
release after Python 3.9 actually does end up being Python 4.0, then we
only have around 5 more years of "the latest Python" and "the latest Python
3.x" being the same thing :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Chris Barker - NOAA Federal]

On Nov 13, 2017, at 1:53 PM, Paul Moore  wrote:

In principle, I agree with the ideas here, but there are some
practical issues that make them somewhat less straightforward than we
might like.


And practically beats principally ;-)

But yeah.


1) Add python2.exe and python3.exe files to the Windows installers


The only possible way we'd get a python2.exe is by adding it to future
releases of Python 2.7, and we shouldn't be recommending Python 2 for
new users anyway.


No, but it’s going to be in use for a while yet (2020, yes?).

And no, new users should not install py3, but new users may work in an org
that is using py2.

And I'm strongly -1 on promoting "python3.exe" as
the canonical way of getting Python 3.


So what is canonical? Not the py executable, for sure.

*maybe* we are at a point where we expect py3 to be the default on most
systems, but we DO need a way to tell people how to get py3 if:

Typing python gets them py2.

And it’s not an option to remove py2 or re-configure their systems to
change the default ( see your point below about Windows PATH priorities)

The python3 command is a result
of the Unix issues around switching to Python 3 as the default, and we
shouldn't perpetuate that approach on Windows, where it's unneeded.


But the exact same issues are there in windows (except the shebang line)
bit it’s pretty weird that py.exe respects “python3”, when there is no
python3.exe on the system...

And having multiple systems work the same way IS a laudable goal.

Having said that, I don't object to including versioned executables as
a convenience (if it is one). I just dislike promoting the use of them
via standard documentation.


Totally agree.

 1a) alternatively, we could add a "py" executable to the standard linux

builds, so there would be THAT one way to do it. But I think that's a "BAD

IDEA" --


I think that getting a "py" launcher on Unix is a lost cause at this
stage.


Fine — I never liked the idea :-)

Then "python2 -m pip install" would work everywhere (only with new

installations, but at least with newbies, that's a bit more likely ...)


No newcomer should *ever* be getting told to use "python2 -m pip
install". Maybe "python3 -m pip install",


Exactly — that was supposed to be an example. Both should work, but
probably:

Python -m pip install would be recommended, with “you can add 2 or 3 if it
doesn’t do the right thing.

 And of course not all Unix distributions come
with Python 3 installed


Exactly — which is why even newbies that install just python 3 end up with
both on their system.

(Mac OS being an obvious example, and I think
Nick mentioned CentOS and RHEL) so python3 won't work everywhere
either...


But it will give you an error, rather that python 2. That’s exactly what we
want!


2) Make adding to the PATH in Windows the default.

...

Unfortunately, adding Python to the *front* of PATH is not possible.


Well, darn. But we can still make it the default and it will sometimes
work. And all the more reason we need a “python3” executable.

There's also the problem that file associations (i.e., what does
double clicking on a .py/.pyw file do) don't follow the same rules, as
they go through the launcher.


I don’t see anything we can do with that. But I don’t recommend users use
that anyway.

The only really sensible "do what I expect" situation is a single
Python 3 installation on the user's machine.


Sure — but we can’t control that. All we can do is mitigate the damage.

For that case, adding
Python to PATH (it doesn't matter where as there's only one) is a
sensible option. The downside is that optimising for that case makes
the behaviour for other (more experienced) users worse. But it's not
unreasonable to make that trade-off.


Agreed. And not much worse.

3) Make --user be be automatic for pip install. Not actually the default,

but pip could do a user install if you don't have the permissions for a

non-user install.


The problem here is that the user scripts directory isn't on PATH.


How does that have anything to do with it? Either they can launch pip or
they can’t (the above are about that) this is about what should happen when
they do launch pip.


The various options here are under discussion on the pip tracker.


I’ll try to go there to comment.

 but there are
significant backward compatibility and behaviour change issues to
address first.


Always the challenge!

   python myscript.py
   pip install something_or_other

"just work" if the user previously had no Python on their PC.


And as many other configurations as possible :-)

I think the changes we should make should emphasize two things that perhaps
haven’t been given enough priority in the past:

1) things should work as much the same as possible across platforms.

2) making it easier for newbies is more important that for more experienced
folks.


It has a whole section at
https://docs.python.org/3/using/windows.html#python-launcher-for-windows.


What I can tell 

Re: [Python-ideas] Looking for input to help with the pip situation

[Stephan Houben]

Hi Nick,

2017-11-14 11:07 GMT+01:00 Nick Coghlan :

> On 14 November 2017 at 16:47, Michel Desmoulin 
> wrote:
>
>> Proposal A:
>> ---
>>
>> Suffix Python executable on Windows like on Unix, so that people will
>> type pythonX.X if they want a specify version.
>>
>> Pros: easy and discoverable.
>>
>> Cons: you need a lot of stuff in the system path.
>>
>
> Con: we hope to have the problem resolved on the Linux distro side such
> that "python" typically means "python" by the time community support for
> Python 2 ends in 2020. Since Windows has gone the better part of two
> decades without version Python commands, adding them because we're
> impatient with the pace of change at the Linux distro level doesn't really
> make sense (especially when Linux holds such a small fraction of the
> non-phone client device market).
>

Perhaps I could sell you on the idea of a Windows "python3" executable, not
as the New Official Way to do things,
but rather as a pragmatic measure to make code from those Linux weirdos
;-)  more likely to work on Windows.

I would like to compare it with the strings/bytes-for-pathnames issue: the
official recommendation is to use strings
everywhere, problems with round-tripping arbitrary not-valid-UTF8 filenames
on POSIX have been solved now.
Still, POSIX people continued to use bytes, so a pragmatic change was to
make so that bytes now also work
reliably as pathnames under Windows.

Similarly, even when all Linux distributions have switched to
python==python3, people will probably still read and write
tutorials with python3 in it, and perhaps we should accommodate that.

Otherwise I am +1 on your proposal C: if anything this thread has made it
clear that there is so much variety in
third-party Python installers, not just Linux distributions, but also
things like Anaconda, that it seems unreasonable
to require that the official Python documentation covers all those
situations.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Todd]

On Nov 14, 2017 02:39, "Chris Angelico"  wrote:

On Tue, Nov 14, 2017 at 5:47 PM, Michel Desmoulin
 wrote:
> Proposal B:
> 
>
> Make pip and venv part of the standard and request debian that they
> provide it.
>
> Pros: straight forward.
>
> Cons: holy war in the making.

Cons: What does "full" really mean? Does it require a GUI subsystem,
for instance (as a dep of tkinter)? Might just shift the problem.


The solution a lot of other Linux distributions use is to have a
"python(2|3)-base" package that pulls in just the minimal runtime
environment and have the regular "python(2|3)" package gives the full
upstream python install.  Things like tk bindings and pip tend to be
separate packages the normal install depends on.

"Minimal", if I understand it correctly, is the python(2|3) executable and
any module not needing external dependencies, and "full" is everything
upstream provides.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 14 November 2017 at 10:02, Nathaniel Smith  wrote:
> On Tue, Nov 14, 2017 at 12:56 AM, Paul Moore  wrote:
>> On 14 November 2017 at 03:08, Nathaniel Smith  wrote:
>>> On Nov 13, 2017 6:47 PM, "Nick Coghlan"  wrote:
>>
 and a pip.bat with the equivalent contents on Windows?
 (Bonus: maybe this would fix the problem with upgrading pip on
 Windows?)
>>>
>>> Depending on how the batch file was written, I think the answer to
>>> that is "maybe":
>>> https://stackoverflow.com/questions/2888976/how-to-make-bat-file-delete-it-self-after-completion/20333152#20333152
>>>
>>>
>>> Sigh.
>>
>> Batch files are not suitable for this task. The wrappers have to be
>> executables. See
>> http://paul-moores-notes.readthedocs.io/en/latest/wrappers.html for a
>> detailed analysis I did some time ago.
>
> Ah, interesting. My reason for suggesting it in the first place
> because I was hoping to avoid paying the process spawn overhead twice,
> but it sounds like this specific trick is misguided all around :-).

It doesn't even save the process overhead if you're running Powershell
as your main shell, or running pip from a terminal window in your
editor/IDE, ...

I really wish Windows *did* provide a usable "shell script"
implementation (of any form) but sadly it simply doesn't.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Antoine Pitrou]

On Tue, 14 Nov 2017 13:28:20 +0100
Antoine Pitrou  wrote:

> On Tue, 14 Nov 2017 12:14:50 +1000
> Nick Coghlan  wrote:
> > >
> > > I don't think Windows ever had python2.exe/python3.exe, but I could be 
> > > wrong.
> > 
> > Not that I'm aware of in the python.org installers, and I don't think
> > ActivePython does either. I'm less sure about Enthought or Anaconda
> > (since I've never used either of them on Windows).  
> 
> Looking at my Windows VM...
> 
> * from a Miniconda install (equivalent, I think, to a bare-bones
>   Anaconda with the minimal package set required for a function
>   "python" and "conda"):
> 
> c:\>where python  
> C:\Miniconda3\python.exe
> 
> c:\>where python3  
> INFO: Could not find files for the given pattern(s).
> 
> * from a user-created conda environment:
> 
> c:\>activate da36  
> 
> (da36) c:\>where python
> C:\Miniconda3\envs\da36\python.exe
> C:\Miniconda3\python.exe
> 
> (da36) c:\>where python3
> INFO: Could not find files for the given pattern(s).

I forgot to mention that those "python.exe" are all Python 3.6
executables (or hardlinks thereof).

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Antoine Pitrou]

On Tue, 14 Nov 2017 07:45:09 +0100
Michel Desmoulin
 wrote:
> 
> I think it's wrong to have a different setup on different plateforms.
> Python is a portable language. It's also a language you use for quick
> scripts and from a lot of non professional devs.
> 
> Unifying the python experience and making sure it's easy for beginners
> to get started is a very important goal IMO.
> 
> Python is actually quite good at it. But we can make it even better.

If the problem has to do with how Debian decides to repackage upstream
software, I don't think we (python-dev) really can make it better,
though.

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Antoine Pitrou]

On Tue, 14 Nov 2017 12:14:50 +1000
Nick Coghlan  wrote:
> >
> > I don't think Windows ever had python2.exe/python3.exe, but I could be 
> > wrong.  
> 
> Not that I'm aware of in the python.org installers, and I don't think
> ActivePython does either. I'm less sure about Enthought or Anaconda
> (since I've never used either of them on Windows).

Looking at my Windows VM...

* from a Miniconda install (equivalent, I think, to a bare-bones
  Anaconda with the minimal package set required for a function
  "python" and "conda"):

c:\>where python
C:\Miniconda3\python.exe

c:\>where python3
INFO: Could not find files for the given pattern(s).

* from a user-created conda environment:

c:\>activate da36

(da36) c:\>where python
C:\Miniconda3\envs\da36\python.exe
C:\Miniconda3\python.exe

(da36) c:\>where python3
INFO: Could not find files for the given pattern(s).


(no clue about Enthought, sorry)

> Yeah, myself, Barry Warsaw, Matthias Klose, and a number of other
> folks on linux-sig have poked at this idea multiple times since Geoff
> Thomas first posted about https://github.com/geofft/pythonmux, but our
> conclusion each time has been that it wouldn't help enough to justify
> the effort involved in implementing and promoting it. After 20+ years
> of usage in the Linux ecosystem, `/usr/bin/python` and `/usr/bin/env
> python` are simply too entrenched in both people's habits and existing
> code.

+1.  As a matter of fact, Anaconda also doesn't seem to expose "py" on
Windows:

(da36) c:\>py
'py' is not recognized as an internal or external command,
operable program or batch file.

> >> 3) Make --user be be automatic for pip install. Not actually the default,
> >> but pip could do a user install if you don't have the permissions for a
> >> non-user install.  
> >
> > The problem here is that the user scripts directory isn't on PATH. We
> > could put it on, but again we hit the "goes after the system PATH"
> > problem (on Windows).  
> 
> We should still optimise the defaults for the desired system
> configuration (i.e. user-mode installs as the cross-platform default
> outside a venv), and then work on platform-specific troubleshooting
> guides for the common ways that things can go wrong.

Hmm... I liked --user at some point, but if you start using it a lot it
becomes problematic, for example if you want to install two
different scripts/applications with incompatible dependencies.

That said, it *is* much better than system-wide installs of user
packages :-)

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 14 November 2017 at 16:47, Michel Desmoulin 
wrote:

> General summary so far
> ###
>
>
> This debate has been very civil so far and quite productive, so thanks
> to everyone involved.
>
> Several issues and solution proposals have been mentioned, so I will
> make a summary here.
>
> Issue 1: running Python is a different command different OS when several
> versions are installed
> 
> ==
>
> Proposal A:
> ---
>
> Suffix Python executable on Windows like on Unix, so that people will
> type pythonX.X if they want a specify version.
>
> Pros: easy and discoverable.
>
> Cons: you need a lot of stuff in the system path.
>

Con: we hope to have the problem resolved on the Linux distro side such
that "python" typically means "python" by the time community support for
Python 2 ends in 2020. Since Windows has gone the better part of two
decades without version Python commands, adding them because we're
impatient with the pace of change at the Linux distro level doesn't really
make sense (especially when Linux holds such a small fraction of the
non-phone client device market).

Proposal B:
> 
>
> Provide the "py" command on Unix, so that people will type "py -x.x"
>
> Pros: no more issues with system path. Just add this command, and
> register python installations to be listed by "py".
>
> Cons: very few people know about "py", even on windows. This would need
> lots of communication. Harder to change Mac and majors Linux distros
> setup than just the next windows installer.
>

Proposal C:

Make sure 'python' just works on a default Window user install, and treat
any system where that doesn't work by default as a special case that needs
troubleshooting.

Pro: a solid troubleshooting guide will be useful regardless

Con: mainstream Linux distros will still be a consistent exception for at
least the next couple of years (and probably longer for the LTS variants)


> Issue 2: there is no one and only one obvious way to do venv / pip
> ===
>
> Because pip and venv are tied to a Python version, there are numerous
> ways to call them.
>
> Just for pip, you have "pip", "py -x.x -m pip", "pythonx.x -m pip" and
> "pipx".
>
> Proposal A:
> ---
>
> Decide on one form, and promote it everywhere.
>
>
> Pros: simple.
>
> Cons: long and tedious process, with a huge inertia before all 3rd party
> to it. Would need a "PEP 8 for Python command". Requires "issue 1" to be
> solved before.
>

This downside doesn't apply as long as the form we standardise on is "pip
install module".

That already works in a lot of contexts, and the cases where it doesn't
work are typically ones where "python -m pip" may also be a problem.

Given a solid troubleshooting guide for issue 1, we may also be able to add
some implicit checks to pip that provide alerts for problematic
configurations, like checking:

>>> user_scheme = "posix_user" # or "nt_user" or "osx_framework_user"
>>> sysconfig.get_path("scripts", scheme="posix_user") in
os.getenv("PATH").split(os.pathsep)
True

The latter check returning false is a sign that the `PATH` configuration
might be a problem.



> Proposal B:
> 
>
> Provide a pipenv-like behavior
>
> pipenv is a tool created by the author of "requests" that automatically
> "do the right thing" by default. The first time you install a package,
> it creates a virtualenv for the current directory (but don't store it in
> the current directory). It will then install all packages in there.
> "pipenv shell" starts a shell in the virtualenv. Dependencies are
> automatically dumped in a requirement files separating dev and prod.
>
> Pros: bypass the multiple installers issue by prompting you what version
> you wish the first time it creates a virtualenv. Solve the problem of
> admin rights and system path.
>
> Cons: lots of work. Either a new tool or breaking compat with pip. If
> debian didn't install pip by default, this may not help.
>

This is essentially the path that conda takes, and it works well for folks
that are willing to adopt conda's specific approach to managing your Python
runtimes.



> Issue 3: pip and virtualenv are not available everywhere
> ===
>
> Proposal B:
> 
>
> Make pip and venv part of the standard and request debian that they
> provide it.
>
> Pros: straight forward.
>
> Cons: holy war in the making.
>

I don't think it's *that* bad. Debian does have support for weak
dependencies, so the resolution here may be as simple as asking Matthias to
add a Recommends from python3 to python3-venv so that ensurepip is present
by default on typical workstation configurations, but can still be easily
omitted from server configurations.



> Favorite personnal combination
> 

Re: [Python-ideas] Looking for input to help with the pip situation

[Nathaniel Smith]

On Tue, Nov 14, 2017 at 12:56 AM, Paul Moore  wrote:
> On 14 November 2017 at 03:08, Nathaniel Smith  wrote:
>> On Nov 13, 2017 6:47 PM, "Nick Coghlan"  wrote:
>
>>> and a pip.bat with the equivalent contents on Windows?
>>> (Bonus: maybe this would fix the problem with upgrading pip on
>>> Windows?)
>>
>> Depending on how the batch file was written, I think the answer to
>> that is "maybe":
>> https://stackoverflow.com/questions/2888976/how-to-make-bat-file-delete-it-self-after-completion/20333152#20333152
>>
>>
>> Sigh.
>
> Batch files are not suitable for this task. The wrappers have to be
> executables. See
> http://paul-moores-notes.readthedocs.io/en/latest/wrappers.html for a
> detailed analysis I did some time ago.

Ah, interesting. My reason for suggesting it in the first place
because I was hoping to avoid paying the process spawn overhead twice,
but it sounds like this specific trick is misguided all around :-).

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 14 November 2017 at 03:08, Nathaniel Smith  wrote:
> On Nov 13, 2017 6:47 PM, "Nick Coghlan"  wrote:

>> and a pip.bat with the equivalent contents on Windows?
>> (Bonus: maybe this would fix the problem with upgrading pip on
>> Windows?)
>
> Depending on how the batch file was written, I think the answer to
> that is "maybe":
> https://stackoverflow.com/questions/2888976/how-to-make-bat-file-delete-it-self-after-completion/20333152#20333152
>
>
> Sigh.

Batch files are not suitable for this task. The wrappers have to be
executables. See
http://paul-moores-notes.readthedocs.io/en/latest/wrappers.html for a
detailed analysis I did some time ago.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Chris Angelico]

On Tue, Nov 14, 2017 at 6:38 PM, Chris Angelico  wrote:
> On Tue, Nov 14, 2017 at 5:47 PM, Michel Desmoulin
>  wrote:
>> Proposal B:
>> 
>>
>> Make pip and venv part of the standard and request debian that they
>> provide it.
>>
>> Pros: straight forward.
>>
>> Cons: holy war in the making.
>
> If this would really cause such back-lash, how about: Make pip and
> venv part of the standard, but permit them to be omitted from
> "python3" as long as there is a "python3-full" package that will drag
> them in. Several other packages are split up like that.
>
> Pros: Only slightly less straight-forward, and hopefully incites only
> a small war.
>
> Cons: What does "full" really mean? Does it require a GUI subsystem,
> for instance (as a dep of tkinter)? Might just shift the problem.
> OTOH, the sorts of people who run non-graphical Linuxes are usually
> going to be comfortable installing python3-pip explicitly, so maybe
> it's doable.

Oh, forgot to give a clear opinion: Even if the compromise isn't
viable, I am still +1 on auto-installing pip. There's already a
python3-minimal package for people who want "just the interpreter,
please" installs.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Chris Angelico]

On Tue, Nov 14, 2017 at 5:47 PM, Michel Desmoulin
 wrote:
> Proposal B:
> 
>
> Make pip and venv part of the standard and request debian that they
> provide it.
>
> Pros: straight forward.
>
> Cons: holy war in the making.

If this would really cause such back-lash, how about: Make pip and
venv part of the standard, but permit them to be omitted from
"python3" as long as there is a "python3-full" package that will drag
them in. Several other packages are split up like that.

Pros: Only slightly less straight-forward, and hopefully incites only
a small war.

Cons: What does "full" really mean? Does it require a GUI subsystem,
for instance (as a dep of tkinter)? Might just shift the problem.
OTOH, the sorts of people who run non-graphical Linuxes are usually
going to be comfortable installing python3-pip explicitly, so maybe
it's doable.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

General summary so far
###


This debate has been very civil so far and quite productive, so thanks
to everyone involved.

Several issues and solution proposals have been mentioned, so I will
make a summary here.

Issue 1: running Python is a different command different OS when several
versions are installed
==

Proposal A:
---

Suffix Python executable on Windows like on Unix, so that people will
type pythonX.X if they want a specify version.

Pros: easy and discoverable.

Cons: you need a lot of stuff in the system path.

Proposal B:


Provide the "py" command on Unix, so that people will type "py -x.x"

Pros: no more issues with system path. Just add this command, and
register python installations to be listed by "py".

Cons: very few people know about "py", even on windows. This would need
lots of communication. Harder to change Mac and majors Linux distros
setup than just the next windows installer.


Issue 2: there is no one and only one obvious way to do venv / pip
===

Because pip and venv are tied to a Python version, there are numerous
ways to call them.

Just for pip, you have "pip", "py -x.x -m pip", "pythonx.x -m pip" and
"pipx".

Proposal A:
---

Decide on one form, and promote it everywhere.


Pros: simple.

Cons: long and tedious process, with a huge inertia before all 3rd party
to it. Would need a "PEP 8 for Python command". Requires "issue 1" to be
solved before.


Proposal B:


Provide a pipenv-like behavior

pipenv is a tool created by the author of "requests" that automatically
"do the right thing" by default. The first time you install a package,
it creates a virtualenv for the current directory (but don't store it in
the current directory). It will then install all packages in there.
"pipenv shell" starts a shell in the virtualenv. Dependencies are
automatically dumped in a requirement files separating dev and prod.

Pros: bypass the multiple installers issue by prompting you what version
you wish the first time it creates a virtualenv. Solve the problem of
admin rights and system path.

Cons: lots of work. Either a new tool or breaking compat with pip. If
debian didn't install pip by default, this may not help.

Issue 3: pip and virtualenv are not available everywhere
===

Proposal A:


Make a mock "pip" commands for when it's not installed. venv already
does that in some cases (see attachements), alhought it does give you
the wrong command so it should be fixed.

The gist of it: if the users try to run pip install while it's missing,
it prints a warning explaining it's not installed and the command you
can run to solve the problem plus a link to more information in the
documentation.

Pros: low friction to implement.

Cons: requires to put up a hook somewhere so that each linux distro can
input "the proper commands to use" for their system when they package
python. More a workaround than a solution.

Proposal B:


Make pip and venv part of the standard and request debian that they
provide it.

Pros: straight forward.

Cons: holy war in the making.


.


Can I have +1 and -1 from each of you on each point ?

Should I start a new thread for each of them ?


Favorite personnal combination


1 - So far, I like "providing py on unix" better, but I think it's too
hard to do. So provide suffix on windows seems more achievable.

So +1 for having pythonX.X on windows.

2 - Decide on one form, and promote it everywhere is again the simplest
IMO. I love pipenv but the strings to pull to get something like this
means it probably would not happen.

If we stick to suffix for issue 1, this mean we can promote:

pythonX.X -m pip
pythonX.X -m venv

Everywhere. In the docs. In the tutorials. In teaching classes.

This will help a lot to remove all confusions and make the Python start
up experience way easier.

3 - getting angry debian devs is probably not wise. The "mock" pip/venv
command also plays very well with the 2 other solutions.

It means we can promote everywhere:

pythonX.X -m cmd

It will work most of the time. And when it doesn't, give clear
contextual steps on





___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 12/11/2017 à 19:02, Stephan Houben a écrit :
> 
> 2017-11-12 13:20 GMT+01:00 Paul Moore  >:
> 
> On 12 November 2017 at 06:19, Michel Desmoulin
> > wrote:
> 
> > Well, not exactly. Do you do python -m venv, or py -x.x -m venv or
> > pythonx -m venv ? Wait, it's not installed by default on debian.
> 
> Seriously? Debian don't provide venv in the standard Python install?
> That's just broken.
> 
> 
> I think that it is wrong to think of Debian's "python3" package
> as purporting to provide the "standard Python install".
> 
> This is really more properly though of as the Python *runtime*
> environment, i.e. the minimum stuff you need to run programs
> written in Python.
> 

I think it's wrong to have a different setup on different plateforms.
Python is a portable language. It's also a language you use for quick
scripts and from a lot of non professional devs.

Unifying the python experience and making sure it's easy for beginners
to get started is a very important goal IMO.

Python is actually quite good at it. But we can make it even better.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 14 November 2017 at 13:08, Nathaniel Smith  wrote:

> On Nov 13, 2017 6:47 PM, "Nick Coghlan"  wrote:
>
> On 14 November 2017 at 11:51, Nathaniel Smith  wrote:
> > What if instead of installing a standard entry point, the pip
> > executable was installed as
> >
> > #!/bin/sh
> > exec python -m pip "$@"
> >
> > on Unix-likes
>
> It would technically be enough to make the shebang line
> `#!/usr/bin/env python` so the interpreter used was picked up from the
> environment, rather than being preconfigured at install time. However,
> the problem is that you don't know for certain that that python will
> actually have `pip` installed, so it might just fail with a cryptic
> error instead.
>
>
> This would still be a massive improvement over the status quo, which in
> this situation would present a perfect simulacrum of downloading and
> installing the package you asked for, except then when you start python the
> import still fails.
>
> I did think of another issue: when installing into a virtualenv, we
> probably want to keep the current system, so explicit/path/bin/pip
> continues to work as expected.
>

My essential concern is that I don't think we should be opening ourselves
up to *even more* platform dependent behaviours in this area.

Instead, if we decide that we would like to request that pip to start doing
something different when:

* `sys.executable` and `shutil.which('python')` are inconsistent with each
other
* `sys.argv[0]` doesn't end with `site-packages/pip/__main__.py` and isn't
`shutil.which('pip')`

then we'd be better off putting that logic in pip's Python code, rather
than in the platform dependent script wrappers where we can't control the
error messages presented when our expectations aren't met.

I do like the idea of making that distinction in principle though, as it's
a better one than "inside a virtual environment or not?".

While Python level venvs are *a* way to obtain the desired pip/python
consistency, they're not the only one:

- Arch Linux does it by default
- Docker's Python 3 images do it by default (Alpine Linux may even do it by
default in general)
- Windows installs do it by default (if PATH is configured correctly)
- conda env does it
- Software Collections do it (including in the Python 3 Docker images for
RHEL & CentOS)
- *nix environment modules do it
- system admins may set up per-user install profiles to do it
- etc...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nathaniel Smith]

On Nov 13, 2017 6:47 PM, "Nick Coghlan"  wrote:

On 14 November 2017 at 11:51, Nathaniel Smith  wrote:
> What if instead of installing a standard entry point, the pip
> executable was installed as
>
> #!/bin/sh
> exec python -m pip "$@"
>
> on Unix-likes

It would technically be enough to make the shebang line
`#!/usr/bin/env python` so the interpreter used was picked up from the
environment, rather than being preconfigured at install time. However,
the problem is that you don't know for certain that that python will
actually have `pip` installed, so it might just fail with a cryptic
error instead.


This would still be a massive improvement over the status quo, which in
this situation would present a perfect simulacrum of downloading and
installing the package you asked for, except then when you start python the
import still fails.

I did think of another issue: when installing into a virtualenv, we
probably want to keep the current system, so explicit/path/bin/pip
continues to work as expected.


However, `pip` could potentially be updated with a `checkenv`
subcommand that complains if `sys.executable` and
`shutil.which('python')` don't match (and could presumably include
other checks as well).


Unfortunately by the time you know to run this command to have already
understood the problem and how to fix it :-). That said, this is probably
cheap enough we could do it automatically at startup. Does pip know whether
it's pip, pip3, etc. that invoked it? I guess sys.argv[0] should tell us?


> and a pip.bat with the equivalent contents on Windows?
> (Bonus: maybe this would fix the problem with upgrading pip on
> Windows?)

Depending on how the batch file was written, I think the answer to
that is "maybe":
https://stackoverflow.com/questions/2888976/how-to-make-bat-
file-delete-it-self-after-completion/20333152#20333152


Sigh.

-n
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 14 November 2017 at 11:51, Nathaniel Smith  wrote:
> What if instead of installing a standard entry point, the pip
> executable was installed as
>
> #!/bin/sh
> exec python -m pip "$@"
>
> on Unix-likes

It would technically be enough to make the shebang line
`#!/usr/bin/env python` so the interpreter used was picked up from the
environment, rather than being preconfigured at install time. However,
the problem is that you don't know for certain that that python will
actually have `pip` installed, so it might just fail with a cryptic
error instead.

However, `pip` could potentially be updated with a `checkenv`
subcommand that complains if `sys.executable` and
`shutil.which('python')` don't match (and could presumably include
other checks as well).

> and a pip.bat with the equivalent contents on Windows?
> (Bonus: maybe this would fix the problem with upgrading pip on
> Windows?)

Depending on how the batch file was written, I think the answer to
that is "maybe":
https://stackoverflow.com/questions/2888976/how-to-make-bat-file-delete-it-self-after-completion/20333152#20333152

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Stephan Houben]

Hi Chris,

+1 to all three of these proposals!

Stephan

2017-11-13 19:57 GMT+01:00 Chris Barker :

> This has gotten to be a big thread, and it's a pretty intractable problem,
> but I think there are a few fairly small things that could be done to at
> least make it a bit easier:
>
> 1) Add python2.exe and python3.exe files to the Windows installers -- am I
> insane or did Windows used to have that? I really think it did -- maybe got
> removed when py.exe was added.
>   1a) alternatively, we could add a "py" executable to the standard linux
> builds, so there would be THAT one way to do it. But I think that's a "BAD
> IDEA" -- the whole "py" thing is not widely know or used, it's not going to
> show up in package install instructions for a LONG time, (actualy we could
> do both anyway)
>
> Then "python2 -m pip install" would work everywhere (only with new
> installations, but at least with newbies, that's a bit more likely ...)
>
>
> 2) Make adding to the PATH in Windows the default. I think there are
> really three user groups:
>
>- newbies starting from scratch -- they want it on the PATH
>
>- newbies with whatever left over cruft from previous installations on
> their systems -- they want it at the FRONT of their PATH. They SHOULD
> uninstall all the cruft, but if they don't this will still work with as few
> surprises a possible.
>
>- not-newbies with a previous version of python they need to continue
> using. They can uncheck the box, or use py.exe
>
>
> 3) Make --user be be automatic for pip install. Not actually the default,
> but pip could do a user install if you don't have the permissions for a
> non-user install.
>
> This means folks might accidentally install in user mode because they
> forgot to type sudo -- but that would be a mostly-sysadmin/sophisticated
> user problem. And maybe have an environment variable of configuration key
> for  "prefer admin install". If tha was set, pip would only install in user
> mode if specifically asked to. I'm can't imagine a case where a user would
> have admin permissions, but want a user install (except people following
> bad practices!)
>
> Except for the pip change, these would be easy to implement and backward
> compatible. So why not?
>
>
> *NOTE:* even if nothing changes with any of this we need to get py.exe
> better documented and advertised -- it doesn't show up in:
>
> https://docs.python.org/3/faq/windows.html#id2
>
> for instance.
>
> In fact, I knew about py.exe (from this discussion), and was writing up
> notes about how to run a Python file (without access to a Windows box) ,
> and it took a LONG time to find ANY documentation of it (adding "py" to a
> google search about something python does not get far...).
>
> We can do that better, but frankly this may be a lesson on why we can't
> rely on anything "new" to help solve this problem, when maybe we could make
> the "old way" work better and more cross platform.
>
> -Chris
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Chris Barker]

This has gotten to be a big thread, and it's a pretty intractable problem,
but I think there are a few fairly small things that could be done to at
least make it a bit easier:

1) Add python2.exe and python3.exe files to the Windows installers -- am I
insane or did Windows used to have that? I really think it did -- maybe got
removed when py.exe was added.
  1a) alternatively, we could add a "py" executable to the standard linux
builds, so there would be THAT one way to do it. But I think that's a "BAD
IDEA" -- the whole "py" thing is not widely know or used, it's not going to
show up in package install instructions for a LONG time, (actualy we could
do both anyway)

Then "python2 -m pip install" would work everywhere (only with new
installations, but at least with newbies, that's a bit more likely ...)


2) Make adding to the PATH in Windows the default. I think there are really
three user groups:

   - newbies starting from scratch -- they want it on the PATH

   - newbies with whatever left over cruft from previous installations on
their systems -- they want it at the FRONT of their PATH. They SHOULD
uninstall all the cruft, but if they don't this will still work with as few
surprises a possible.

   - not-newbies with a previous version of python they need to continue
using. They can uncheck the box, or use py.exe


3) Make --user be be automatic for pip install. Not actually the default,
but pip could do a user install if you don't have the permissions for a
non-user install.

This means folks might accidentally install in user mode because they
forgot to type sudo -- but that would be a mostly-sysadmin/sophisticated
user problem. And maybe have an environment variable of configuration key
for  "prefer admin install". If tha was set, pip would only install in user
mode if specifically asked to. I'm can't imagine a case where a user would
have admin permissions, but want a user install (except people following
bad practices!)

Except for the pip change, these would be easy to implement and backward
compatible. So why not?


*NOTE:* even if nothing changes with any of this we need to get py.exe
better documented and advertised -- it doesn't show up in:

https://docs.python.org/3/faq/windows.html#id2

for instance.

In fact, I knew about py.exe (from this discussion), and was writing up
notes about how to run a Python file (without access to a Windows box) ,
and it took a LONG time to find ANY documentation of it (adding "py" to a
google search about something python does not get far...).

We can do that better, but frankly this may be a lesson on why we can't
rely on anything "new" to help solve this problem, when maybe we could make
the "old way" work better and more cross platform.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Chris Barker]

On Sun, Nov 12, 2017 at 3:18 PM, Nick Coghlan  wrote:

> P.S. As a user, it's also genuinely irritating to have to always type
> the "python -m " prefix when inside an active virtual environment, as
> in that case, there isn't any ambiguity about which environment pip
> should be manipulating or which version of Python it should be using.
>

and not just there :-)

I decided a year or two ago to recommend "python -m pip install" to my
beginning students, and put it all my course documentation. I've done this
for a while now.

but last week, when I introduced pytest -- I said, if you haven't'
installed pytest yet, do it now -- and then proceeded to type:

pip install pytest

In my terminal, projecting in a huge font  for all the class to see.

It's just too ingrained and easy.

And the truth is, it actually works MOST of the time.

And if it doesn't work right, you are either a semi-sophisticated user that
needs multiple pythons accessible from your shell, or your system is
mis-configured.

Pain though it is, it's probably better to help newbies configure their
systems correctly than to try to get everyone to invoke pip differently.

In fact, I have a little excercise that my students do as teh very first
thing in class:

https://uwpce-pythoncert.github.io/PythonCertDevel/supplemental/HowToRunAPythonFile.html#making-sure-you-are-set-up-correctly

If you don't want to go read that:

I give them this code:

(not as a downloadable python file)

#!/usr/bin/env python
import sysprint("This is my first python program")
version = sys.version_info
if version.major == 3:
if version.minor != 6:
print("You should be running version 3.6")
else:
print("You are running python3.6 -- all good!")else:
print("You need to run Python 3!")
print("This is version: {}.{}".format(version.major, version.minor))


And I have them put it in a file, save it and run it however they want to
run it.

I then know that they:

Know how to create a new python file, put some code in it and run that code.

and

They are running the version of Python I want them to run.

(though to be fair, I had one student struggling for weeks with "strange"
errors, because his PyCharm was setup to run python2. He probably started
using PyCharm after he ran that script some other way...)

I suppose I should do something similar to check that they have pip
properly configured also, but to some extent, that comes up the first time
I have them pip install something -- which is in the first class anyway :-)

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Alex Walters]

> -Original Message-
> From: Python-ideas [mailto:python-ideas-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Nick Coghlan
> Sent: Sunday, November 12, 2017 6:19 PM
> To: Brendan Barnwell 
> Cc: python-ideas@python.org
> Subject: Re: [Python-ideas] Looking for input to help with the pip
situation
> 
> On 13 November 2017 at 02:59, Brendan Barnwell
>  wrote:
> > On 2017-11-12 05:18, Nick Coghlan wrote:
> >>
> >> * the `pip install` option really is nicer looking than `python -m pip
> >> install`, and it only has actual problems in the presence of multiple
> >> Python versions and when upgrading pip itself on Windows (plus: lots
> >> of third party guides recommend it, as do pypi.org project pages)
> >
> >
> > Is there any *advantage* to using `pip install` instead of
`python
> > -m install`?  If not, could we at least change everything under
Python/pip
> > control (e.g., pip documentation) to never recommend `pip` and always
> > recommend `python -m pip` instead, and encourage all third-party
> > documentation to always use `python -m pip` and never use `pip`?
> 
> We've already changed most of them (pypi.org itself is the main one
> that we haven't changed it yet).
> 
> However, there are still per-project READMEs out there that suggest
> "easy_install project" and direct invocation of "python setup.py
> install", so it really isn't appealing to layer yet another mandatory
> change in the recommended spelling of the installation command and
> create yet another point of confusion - it will be much nicer overall
> if we can retroactively make the existing "pip install" instructions
> correct for most users, and leave "python -m pip install" to the
> "Multiple versions of Python" and "Self-upgrading pip on Windows".
> 
> Cheers,
> Nick.
> 
> P.S. As a user, it's also genuinely irritating to have to always type
> the "python -m " prefix when inside an active virtual environment, as
> in that case, there isn't any ambiguity about which environment pip
> should be manipulating or which version of Python it should be using.
> 

You kinda should be typing `env/bin/python -m ...` (or
env\Scripts\python.exe -m ...) but saying that proves your previous point.

> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 13 November 2017 at 02:59, Brendan Barnwell  wrote:
> On 2017-11-12 05:18, Nick Coghlan wrote:
>>
>> * the `pip install` option really is nicer looking than `python -m pip
>> install`, and it only has actual problems in the presence of multiple
>> Python versions and when upgrading pip itself on Windows (plus: lots
>> of third party guides recommend it, as do pypi.org project pages)
>
>
> Is there any *advantage* to using `pip install` instead of `python
> -m install`?  If not, could we at least change everything under Python/pip
> control (e.g., pip documentation) to never recommend `pip` and always
> recommend `python -m pip` instead, and encourage all third-party
> documentation to always use `python -m pip` and never use `pip`?

We've already changed most of them (pypi.org itself is the main one
that we haven't changed it yet).

However, there are still per-project READMEs out there that suggest
"easy_install project" and direct invocation of "python setup.py
install", so it really isn't appealing to layer yet another mandatory
change in the recommended spelling of the installation command and
create yet another point of confusion - it will be much nicer overall
if we can retroactively make the existing "pip install" instructions
correct for most users, and leave "python -m pip install" to the
"Multiple versions of Python" and "Self-upgrading pip on Windows".

Cheers,
Nick.

P.S. As a user, it's also genuinely irritating to have to always type
the "python -m " prefix when inside an active virtual environment, as
in that case, there isn't any ambiguity about which environment pip
should be manipulating or which version of Python it should be using.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Antoine Pitrou]

On Sun, 12 Nov 2017 23:18:26 +1000
Nick Coghlan  wrote:
> 
> We haven't forgotten our early years - we've just spent years (both
> individually and collectively) working on the problem of helping
> people get started with software development, and thus have a very
> good idea as to what *doesn't* work, as well as what *does* work.
> 
> The latter list is currently fairly short:
> 
> * having a friend or colleague walk them through it
> * "Install Parties", like those Django Girls runs (i.e. running a
> pre-tutorial event, specifically focused on getting a working
> environment set up)
> * highly prescriptive learning environments, whether online ones (like
> Grok Learning, trinket.io, PythonAnywhere, etc), or locally installed
> ones (like PyCharm Educational Edition, the Anaconda distribution,
> etc)

Not wanting to nitpick, but I don't think the Anaconda distribution is
"highly prescriptive".  It's a software distribution with scientific
computing as its main focus, but perfectly usable for ordinary Python
programming.  It's not more prescriptive than Debian, which by its
philosophy is directed primarily towards sysadmin crowds but also used
by some people on their personal desktops.

Also, you can create custom Anaconda-like distributions to provide the
desired environment to your students in an executable installer (*).
I'm not sure how usable that option is, but it definitely exists, is
open source and cross-platform.

(*) https://github.com/conda/constructor

Regards

Antoine.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Stephan Houben]

2017-11-12 13:20 GMT+01:00 Paul Moore :

> On 12 November 2017 at 06:19, Michel Desmoulin
>  wrote:
>
> > Well, not exactly. Do you do python -m venv, or py -x.x -m venv or
> > pythonx -m venv ? Wait, it's not installed by default on debian.
>
> Seriously? Debian don't provide venv in the standard Python install?
> That's just broken.
>

I think that it is wrong to think of Debian's "python3" package
as purporting to provide the "standard Python install".

This is really more properly though of as the Python *runtime*
environment, i.e. the minimum stuff you need to run programs
written in Python.

Developers are supposed to install additional packages.

Debian does the same for other languages,, i.e. their "node" package
doesn't contain npm.

I suppose it makes sense; a modern Linux desktop contains
applications written in Python/Perl/Node/Ruby/Tcl/OCaml/..., if
you start including all development tools in the base packages you
are probably ending doubling a typical install.

It's still annoying, I have personally decided that it is simpler
to just install the tarball from python.org than to chase all
the individual packages over which they split the Python install.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Brendan Barnwell]

On 2017-11-12 05:18, Nick Coghlan wrote:

* the `pip install` option really is nicer looking than `python -m pip
install`, and it only has actual problems in the presence of multiple
Python versions and when upgrading pip itself on Windows (plus: lots
of third party guides recommend it, as do pypi.org project pages)


	Is there any *advantage* to using `pip install` instead of `python -m 
install`?  If not, could we at least change everything under Python/pip 
control (e.g., pip documentation) to never recommend `pip` and always 
recommend `python -m pip` instead, and encourage all third-party 
documentation to always use `python -m pip` and never use `pip`? 
Obviously this isn't a full solution, but in the end there's no way we 
change external third-party documentation, which will always eventually 
become outdated.  Absent that, it seems worthwhile to regularize 
existing official documentation.


--
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."

   --author unknown
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 12 November 2017 at 13:18, Nick Coghlan  wrote:
>> Seriously? Debian don't provide venv in the standard Python install?
>> That's just broken.
>
> Yup. And RHEL/CentOS don't provide Python 3.x by default at all - you
> need to grab it via other means.

Wow. I have no problem with not providing Python 3 by default, but
shipping a version that omits features that are non-optional parts of
the standard library is ridiculous. I'm getting more and more inclined
to make my default response to bug reports from people on
Debian/Ubuntu be "report it to your vendor or demonstrate it on a
vanilla build of Python" :-(

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 12 November 2017 at 22:20, Paul Moore  wrote:
> On 12 November 2017 at 06:19, Michel Desmoulin
>  wrote:
>>> We still have to deal with the fact that basically every Unix
>>> environment is "advanced" in the above sense (the python2/python3
>>> split). I don't have a solution for that (other than "upgrade to
>>> Windows" ;-)).
>>
>> Provide the "py" command on linux and mac. And make it the default
>> recommended way in the documentation.
>
> +1 from me.

The current Python launcher is Window specific, and doesn't work on
POSIX systems. So the first barrier here is "Write a py command that
presents the same CLI on POSIX that the Python launcher does on
Windows".

The second problem is that even though we can add a "py" command to
the python.org Mac OS X installers and have that take effect for new
python.org downloads, there are a *lot* of people on Mac OS X and
Linux that don't actually obtain their pre-built Python from
python.org. While some of those would probably adapt and ship a new
command fairly quickly (the cross-platform commercial redistributors,
rolling release and 6-month cadence Linux distros, community
redistributors like pythonz, pyenv, and homebrew), other important
platforms wouldn't (most notably, LTS Linux distros).

The fact it wouldn't actually solve the bootstrapping problem due to
the update policies of various popular platforms then greatly reduces
the incentive to work on the first part of the problem (i.e. actually
creating a POSIX-compatible version of that CLI).

>> Well, not exactly. Do you do python -m venv, or py -x.x -m venv or
>> pythonx -m venv ? Wait, it's not installed by default on debian.
>
> Seriously? Debian don't provide venv in the standard Python install?
> That's just broken.

Yup. And RHEL/CentOS don't provide Python 3.x by default at all - you
need to grab it via other means.

>> Virtualenvs are a hard tool to use for beginners.
>
> Agreed. Genuine beginners just install Python, then use it. If they
> install extra packages, they want them to be available to all of their
> scripts.

Also agreed, but without them (or a functional equivalent, like `conda
env` or `pyenv`), it's incredibly hard to get an arbitrary user on an
arbitrary system to the point where all the following assumptions are
satisfied:

* the user is working at a command prompt (whether inside their IDE,
or in an actual system shell)
* the command `python` will run the desired version of Python
* the commands `pip install` and `python -m pip install` will be
essentially equivalent (aside from the differences when upgrading pip
itself on Windows) and install packages into the desired version of
Python
* any executables installed that way will be executable from that command prompt
* any Python import packages installed that way will be importable
from a Python shell started in that environment

(List of assumptions taken from
https://github.com/pypa/python-packaging-user-guide/issues/396)

Trying to drop any of those assumptions is problematic:

* non-command-shell learning environments vary even more than command shells do
* the `python` command has a much long history than any of the
alternatives (including both `py` and `python3`), and third party
guides reflect that
* the `pip install` option really is nicer looking than `python -m pip
install`, and it only has actual problems in the presence of multiple
Python versions and when upgrading pip itself on Windows (plus: lots
of third party guides recommend it, as do pypi.org project pages)
* if `pip install devtool` doesn't let you run `command-from-devtool`
in your command shell, that's thoroughly unhelpful
* if `pip install dist-package` doesn't let you run `import
library_from_dist_package` in your Python code, that's even less
helpful

It *would* be much nicer if a single user install of the Windows
python.org installer met these criteria by default, though - while the
`py` launcher is a good way of handling multiple versions (and nicely
fills the role of handling file associations for Python file
extensions), it isn't a particularly nice alternative to a properly
configured user PATH on a machine with only one Python installation.

>> A lot of people on this list have forgotten their early years it seems.
>
> Maybe. But the default beginner approach *does* have its problems, and
> guiding beginners to a better approach is a good idea.

We haven't forgotten our early years - we've just spent years (both
individually and collectively) working on the problem of helping
people get started with software development, and thus have a very
good idea as to what *doesn't* work, as well as what *does* work.

The latter list is currently fairly short:

* having a friend or colleague walk them through it
* "Install Parties", like those Django Girls runs (i.e. running a
pre-tutorial event, specifically focused on getting a working
environment set up)
* highly prescriptive learning environments, whether 

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 12 November 2017 at 06:19, Michel Desmoulin
 wrote:
>> 1. Go back to adding Python to PATH. Because our installers don't say
>> "do you want to uninstall the old version", we should probably do a
>> check for a "python" command on PATH in the installer, and if there is
>> one, warn the user "You already have Python installed - if you are
>> upgrading you should manually uninstall the old version first,
>> otherwise your old installation will remain the default". We could get
>> more complex at this point, but that depends on what capabilities we
>> can include in the installer - I don't know how powerful the toolset
>> is.
>
> You don't even have to do that. We can detect it and prompt : "which
> python version do you want to be available by default ?", and then list
> command to run the alternative versions.

I deliberately avoided suggesting automatically changing the default
version, because that's fraught with problems. You need to remove the
previous default from PATH, and if you're doing a user install but the
previous version is in the system PATH you don't have the privileges
to do that. If the previous version was another distribution (e.g.
Anaconda) you've no way to know that and conversely you don't know how
to remove that distribution's path entries (is there a Scripts
directory to remove, did they use bin instead, etc). The list of
potential problems is endless.

>> We still have to deal with the fact that basically every Unix
>> environment is "advanced" in the above sense (the python2/python3
>> split). I don't have a solution for that (other than "upgrade to
>> Windows" ;-)).
>
> Provide the "py" command on linux and mac. And make it the default
> recommended way in the documentation.

+1 from me.

> Well, not exactly. Do you do python -m venv, or py -x.x -m venv or
> pythonx -m venv ? Wait, it's not installed by default on debian.

Seriously? Debian don't provide venv in the standard Python install?
That's just broken.

> Virtualenvs are a hard tool to use for beginners.

Agreed. Genuine beginners just install Python, then use it. If they
install extra packages, they want them to be available to all of their
scripts.

> A lot of people on this list have forgotten their early years it seems.

Maybe. But the default beginner approach *does* have its problems, and
guiding beginners to a better approach is a good idea. It's just that
the starting point needs to be showing them why the problems solved by
tools like virtualenv matter to them.

At this point, though, we've moved into a much bigger issue than "it's
hard to get started with pip". We should keep the discussion focused
on the immediate problem, and not try to solve everything at once.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 12 November 2017 at 16:20, Michel Desmoulin
 wrote:
> Le 10/11/2017 à 09:01, Nick Coghlan a écrit :
>> On 10 November 2017 at 17:05, Michel Desmoulin
>>  wrote:
>> When we choose not to use venv, then it becomes necessary to ensure
>> each of those things individually for each potential system starting
>> state
>>
> The way we do things is not the only way. Take JS: they don't have this
> problem, you npm install the same way everywhere. You don't have
> virtualenv but local node_modules. And you have transpilers to help with
> the langage version differences.

And they have browser manufacturers pouring millions of dollars a year
into their tooling ecosystem into order to influence whose ad networks
get clicked on most often.

Python doesn't have that level of investment, but we do have fine
folks volunteering to work on projects like `pip`, `virtualenv`, PyPI,
and the packaging.python.org documentation project, as well as
backports of standard library modules to earlier Python versions.

When someone attempts to explain to you the practical challenges that
limit both python-dev's and PyPA's ability to reliably control the
starting experience of new Python users, the appropriate response is
to *start listening*, not harangue them for failing to immediately
follow your peremptory orders to make things simpler for you.

"Do it because I said so" is bad management style even in an actual
company - it's even worse in a peer production environment like an
open source community.

> Now I'm not advocating we do it the JS way. I'm just saying that you are
> very keen to defend a statu quo instead of offering ideas to solve the
> problem.

I already opened
https://github.com/pypa/python-packaging-user-guide/issues/396 to
track possible areas of concrete near term improvement (I would have
tagged you on the issue, but I couldn't find a GitHub account under
your name).

> Besides, using venv have the same issues. It's not installed on linux by
> defaut.

That depends greatly on which Linux you install - the choice on
whether to deliberately cripple "python3 -m venv" or not is made by
the maintainers for that distribution, and Python-friendly distros
make sure Python's native tooling works properly by default.

> And on windows you'll have to do py -x.x -m but on mac pythonx -m.

CPython updates take years to reliably roll out to end user systems,
so if you're looking to have an impact in a shorter time frame than
that, the differences in cross-platform invocation are a constraint
you're going to have to learn to live with.

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 10/11/2017 à 12:41, Oleg Broytman a écrit :
> On Fri, Nov 10, 2017 at 09:50:22AM +, Paul Moore  
> wrote:
>> The biggest reason we don't add Python to PATH, as I understand it, is
>> because we need to consider the implications of people having multiple
>> versions of Python installed.
> 
>Why not fix that the same way as on Unix -- by having versioned
> executables: python27.exe, python35.exe? Then python.exe in PATH will be
> from the most recent installed Python.
> 
>> Paul
> 
> Oleg.
> 

That or provider the "py" command on Unix. Either would work.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 10/11/2017 à 09:01, Nick Coghlan a écrit :
> On 10 November 2017 at 17:05, Michel Desmoulin
>  wrote:
>>
>>> Which is why we advise getting into a virtual environment ASAP, such
>>> that the only platform specific thing folks necessarily need to learn
>>> to get started is how to get to that first working virtual
>>> environment.
>>>
>>
>> You can't start by teaching virtualenv. I tried. It doesn't work. And
>> it's a terrible prerequisit if you write docs, tutorial, teaching
>> materials, etc.
> 
> You can't have it both ways - the only way we can systematically mask
> the environmental differences between Windows, Linux and Mac OS X is
> by providing tooling that actually masks those differences, which
> means introducing that tooling becomes a prerequisite for teaching.
> 
> It doesn't completely solve the problem (as getting into and out of
> the environment is still platform specific), but it does mean that the
> ubiquitous online instructions to run "pip install package-name" and
> "python -m command" will actually work once people are inside their
> working environment.
> 
> That tooling is venv:
> 
> * it ensures you have "pip" on your PATH
> * it ensures you have "python" on your PATH
> * it ensures that you have the required permissions to install new packages
> * it ensures that any commands you install from PyPI will be also on your PATH
> 
> When we choose not to use venv, then it becomes necessary to ensure
> each of those things individually for each potential system starting
> state
> 
The way we do things is not the only way. Take JS: they don't have this
problem, you npm install the same way everywhere. You don't have
virtualenv but local node_modules. And you have transpilers to help with
the langage version differences.

Now I'm not advocating we do it the JS way. I'm just saying that you are
very keen to defend a statu quo instead of offering ideas to solve the
problem.

Besides, using venv have the same issues. It's not installed on linux by
defaut. And on windows you'll have to do py -x.x -m but on mac pythonx -m.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> On Windows, which is the only platform I can reasonably comment on,
> the killer issue is that the installer doesn't make the commands
> "python" and "pip" available by default. Just checking my PC, both go
> and rust (which I installed *ages* ago) do appear to, so they "just
> work". Java sort-of also works like that (the runtime is on PATH, but
> the compilers need to be added manually).
> 
> The biggest reason we don't add Python to PATH, as I understand it, is
> because we need to consider the implications of people having multiple
> versions of Python installed. As far as I know, no other language
> allows that. But equally, most beginners wouldn't actually *have*
> multiple versions installed. So maybe we should optimise for that case
> (only one version of Python installed). That would mean:
> 

That seems reasonable. Especially since somebody with several versions
install is more likely to know how to deal with system path issues than
a total beginner installing python for the first time.

> 1. Go back to adding Python to PATH. Because our installers don't say
> "do you want to uninstall the old version", we should probably do a
> check for a "python" command on PATH in the installer, and if there is
> one, warn the user "You already have Python installed - if you are
> upgrading you should manually uninstall the old version first,
> otherwise your old installation will remain the default". We could get
> more complex at this point, but that depends on what capabilities we
> can include in the installer - I don't know how powerful the toolset
> is.

You don't even have to do that. We can detect it and prompt : "which
python version do you want to be available by default ?", and then list
command to run the alternative versions.

> 2. Explicitly document that multiple Python interpreters on one
> machine is considered "advanced", and users with that sort of setup
> should be prepared to manage PATH themselves. I'd put that as
> something like "It is possible to install multiple versions of Python
> at once, but if you do that, you should understand the implications -
> the average user should not need to do this"
> 
> We still have to deal with the fact that basically every Unix
> environment is "advanced" in the above sense (the python2/python3
> split). I don't have a solution for that (other than "upgrade to
> Windows" ;-)).

Provide the "py" command on linux and mac. And make it the default
recommended way in the documentation.

>> It doesn't completely solve the problem (as getting into and out of
>> the environment is still platform specific), but it does mean that the
>> ubiquitous online instructions to run "pip install package-name" and
>> "python -m command" will actually work once people are inside their
>> working environment.
>>
>> That tooling is venv:
>>
>> * it ensures you have "pip" on your PATH
>> * it ensures you have "python" on your PATH
>> * it ensures that you have the required permissions to install new packages
>> * it ensures that any commands you install from PyPI will be also on your 
>> PATH
>>
>> When we choose not to use venv, then it becomes necessary to ensure
>> each of those things individually for each potential system starting
>> state
> 
> Currently, the reality is that people use virtualenv, not venv. All
> higher-level tools I'm aware of wrap virtualenv (to allow Python 2.7
> support). Enhancing the capabilities of venv is fine, but promoting
> venv over virtualenv involves technical challenges across the whole
> toolset, not just documentation/education.
> 
> But agreed, once we get people into a virtual environment (of any
> form) the portability issues become significantly reduced. The main
> outstanding issue is managing multiple environments, which could be
> handled by having a special "default" environment that is the only one
> we'd expect beginners to use/need.

Well, not exactly. Do you do python -m venv, or py -x.x -m venv or
pythonx -m venv ? Wait, it's not installed by default on debian.

And then virtualenv have their own issues:

- it's a contextual mode that you need to activate AND be aware of at
all time
- you need to config tooling to use it (IDE, builders, etc)
- the location of virtualenv is very important
- you should not commit it to git
- you can't relocate it easily
- install jupyter / mymy / flake8 outside a virtualenv and the command
will still seems to work inside a virtualenv it's not installed in. With
terrible consequences.

Virtualenvs are a hard tool to use for beginners.

A lot of people on this list have forgotten their early years it seems.

> Also on Windows, the per-user bin directory isn't added to PATH even
> if you add the system Python to PATH in the installer.
> 
>> That said, I think there is one improvement we could feasibly make,
>> which would be to introduce the notion of a "default user environment"
>> into `venv`, such that there was a single "python -m venv shell"
>> command that:
>>
>> * created a default 

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 10 November 2017 at 11:37, Oleg Broytman  wrote:
> On Fri, Nov 10, 2017 at 07:48:35AM +0100, Michel Desmoulin 
>  wrote:
>> On linux you
>> can't pip install, you need --users, admin rights or a virtualenv.
>
>Isn't it the same on Windows? For an admin-installed Python you need
> --users, admin rights or a virtualenv. And a user-installed Python on
> Windows is equivalen to a user-compiled Python on Linux -- pip installs
> packages to the user-owned site-packages directory.

It is - but the default install on Windows (using the python.org
installer) is a per-user install. So beginners don't encounter
admin-installed Python (unless they ask for it, in which case they
made the choice so they should understand the implications ;-))

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Oleg Broytman]

On Fri, Nov 10, 2017 at 09:50:22AM +, Paul Moore  
wrote:
> The biggest reason we don't add Python to PATH, as I understand it, is
> because we need to consider the implications of people having multiple
> versions of Python installed.

   Why not fix that the same way as on Unix -- by having versioned
executables: python27.exe, python35.exe? Then python.exe in PATH will be
from the most recent installed Python.

> Paul

Oleg.
-- 
 Oleg Broytmanhttp://phdru.name/p...@phdru.name
   Programmers don't die, they just GOSUB without RETURN.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Oleg Broytman]

On Fri, Nov 10, 2017 at 07:48:35AM +0100, Michel Desmoulin 
 wrote:
> On linux you
> can't pip install, you need --users, admin rights or a virtualenv.

   Isn't it the same on Windows? For an admin-installed Python you need
--users, admin rights or a virtualenv. And a user-installed Python on
Windows is equivalen to a user-compiled Python on Linux -- pip installs
packages to the user-owned site-packages directory.

Oleg.
-- 
 Oleg Broytmanhttp://phdru.name/p...@phdru.name
   Programmers don't die, they just GOSUB without RETURN.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 10 November 2017 at 10:01, Nick Coghlan  wrote:
> On 10 November 2017 at 19:50, Paul Moore  wrote:
>> On 10 November 2017 at 08:01, Nick Coghlan  wrote:
>>> That tooling is venv:
>>>
>>> * it ensures you have "pip" on your PATH
>>> * it ensures you have "python" on your PATH
>>> * it ensures that you have the required permissions to install new packages
>>> * it ensures that any commands you install from PyPI will be also on your 
>>> PATH
>>>
>>> When we choose not to use venv, then it becomes necessary to ensure
>>> each of those things individually for each potential system starting
>>> state
>>
>> Currently, the reality is that people use virtualenv, not venv. All
>> higher-level tools I'm aware of wrap virtualenv (to allow Python 2.7
>> support). Enhancing the capabilities of venv is fine, but promoting
>> venv over virtualenv involves technical challenges across the whole
>> toolset, not just documentation/education.
>
> We already assume there will be a step in understanding from "working
> with the latest Python 3.x locally" to "dealing with multiple Python
> versions". Switching from venv to virtualenv just becomes part of that
> process (and will often be hidden behind a higher level tool like
> pipenv, pew, or vex anyway).

OK, that's fair.
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 10 November 2017 at 19:50, Paul Moore  wrote:
> On 10 November 2017 at 08:01, Nick Coghlan  wrote:
>> That tooling is venv:
>>
>> * it ensures you have "pip" on your PATH
>> * it ensures you have "python" on your PATH
>> * it ensures that you have the required permissions to install new packages
>> * it ensures that any commands you install from PyPI will be also on your 
>> PATH
>>
>> When we choose not to use venv, then it becomes necessary to ensure
>> each of those things individually for each potential system starting
>> state
>
> Currently, the reality is that people use virtualenv, not venv. All
> higher-level tools I'm aware of wrap virtualenv (to allow Python 2.7
> support). Enhancing the capabilities of venv is fine, but promoting
> venv over virtualenv involves technical challenges across the whole
> toolset, not just documentation/education.

We already assume there will be a step in understanding from "working
with the latest Python 3.x locally" to "dealing with multiple Python
versions". Switching from venv to virtualenv just becomes part of that
process (and will often be hidden behind a higher level tool like
pipenv, pew, or vex anyway).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 10 November 2017 at 08:01, Nick Coghlan  wrote:
> You can't have it both ways - the only way we can systematically mask
> the environmental differences between Windows, Linux and Mac OS X is
> by providing tooling that actually masks those differences, which
> means introducing that tooling becomes a prerequisite for teaching.

On Windows, which is the only platform I can reasonably comment on,
the killer issue is that the installer doesn't make the commands
"python" and "pip" available by default. Just checking my PC, both go
and rust (which I installed *ages* ago) do appear to, so they "just
work". Java sort-of also works like that (the runtime is on PATH, but
the compilers need to be added manually).

The biggest reason we don't add Python to PATH, as I understand it, is
because we need to consider the implications of people having multiple
versions of Python installed. As far as I know, no other language
allows that. But equally, most beginners wouldn't actually *have*
multiple versions installed. So maybe we should optimise for that case
(only one version of Python installed). That would mean:

1. Go back to adding Python to PATH. Because our installers don't say
"do you want to uninstall the old version", we should probably do a
check for a "python" command on PATH in the installer, and if there is
one, warn the user "You already have Python installed - if you are
upgrading you should manually uninstall the old version first,
otherwise your old installation will remain the default". We could get
more complex at this point, but that depends on what capabilities we
can include in the installer - I don't know how powerful the toolset
is.
2. Explicitly document that multiple Python interpreters on one
machine is considered "advanced", and users with that sort of setup
should be prepared to manage PATH themselves. I'd put that as
something like "It is possible to install multiple versions of Python
at once, but if you do that, you should understand the implications -
the average user should not need to do this".

We still have to deal with the fact that basically every Unix
environment is "advanced" in the above sense (the python2/python3
split). I don't have a solution for that (other than "upgrade to
Windows" ;-)).

> It doesn't completely solve the problem (as getting into and out of
> the environment is still platform specific), but it does mean that the
> ubiquitous online instructions to run "pip install package-name" and
> "python -m command" will actually work once people are inside their
> working environment.
>
> That tooling is venv:
>
> * it ensures you have "pip" on your PATH
> * it ensures you have "python" on your PATH
> * it ensures that you have the required permissions to install new packages
> * it ensures that any commands you install from PyPI will be also on your PATH
>
> When we choose not to use venv, then it becomes necessary to ensure
> each of those things individually for each potential system starting
> state

Currently, the reality is that people use virtualenv, not venv. All
higher-level tools I'm aware of wrap virtualenv (to allow Python 2.7
support). Enhancing the capabilities of venv is fine, but promoting
venv over virtualenv involves technical challenges across the whole
toolset, not just documentation/education.

But agreed, once we get people into a virtual environment (of any
form) the portability issues become significantly reduced. The main
outstanding issue is managing multiple environments, which could be
handled by having a special "default" environment that is the only one
we'd expect beginners to use/need.

> That said, we'd *like* the default to be is per-user package
> installations into the user home directory, but that creates
> additional problems:
>
> * "python" may be missing, and you'll have to use "python3" or "py" instead
> * "pip" may be missing (or mean "install for Python 2.7")
> * you have to specify "--user" on *every* call to pip, and most online
> guides won't say that
> * there's a major backwards compatibility problem with switching pip
> over to per-user package installs as the default (we still want to do
> it eventually, though)
> * on Windows, system-wide Python installs can't adjust per-user PATH
> settings, and per-user installs are subject to being broken by
> system-wide installs
> * on Windows, the distinction between a per-user install of Python,
> and per-user installs of a package is hard to teach
> * on Debian, I believe ~/.local/bin still isn't on PATH by default

Also on Windows, the per-user bin directory isn't added to PATH even
if you add the system Python to PATH in the installer.

> That said, I think there is one improvement we could feasibly make,
> which would be to introduce the notion of a "default user environment"
> into `venv`, such that there was a single "python -m venv shell"
> command that:
>
> * created a default user environment if it didn't already exist
> * launched a subshell with 

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 10 November 2017 at 17:05, Michel Desmoulin
 wrote:
>
>> Which is why we advise getting into a virtual environment ASAP, such
>> that the only platform specific thing folks necessarily need to learn
>> to get started is how to get to that first working virtual
>> environment.
>>
>
> You can't start by teaching virtualenv. I tried. It doesn't work. And
> it's a terrible prerequisit if you write docs, tutorial, teaching
> materials, etc.

You can't have it both ways - the only way we can systematically mask
the environmental differences between Windows, Linux and Mac OS X is
by providing tooling that actually masks those differences, which
means introducing that tooling becomes a prerequisite for teaching.

It doesn't completely solve the problem (as getting into and out of
the environment is still platform specific), but it does mean that the
ubiquitous online instructions to run "pip install package-name" and
"python -m command" will actually work once people are inside their
working environment.

That tooling is venv:

* it ensures you have "pip" on your PATH
* it ensures you have "python" on your PATH
* it ensures that you have the required permissions to install new packages
* it ensures that any commands you install from PyPI will be also on your PATH

When we choose not to use venv, then it becomes necessary to ensure
each of those things individually for each potential system starting
state

That said, we'd *like* the default to be is per-user package
installations into the user home directory, but that creates
additional problems:

* "python" may be missing, and you'll have to use "python3" or "py" instead
* "pip" may be missing (or mean "install for Python 2.7")
* you have to specify "--user" on *every* call to pip, and most online
guides won't say that
* there's a major backwards compatibility problem with switching pip
over to per-user package installs as the default (we still want to do
it eventually, though)
* on Windows, system-wide Python installs can't adjust per-user PATH
settings, and per-user installs are subject to being broken by
system-wide installs
* on Windows, the distinction between a per-user install of Python,
and per-user installs of a package is hard to teach
* on Debian, I believe ~/.local/bin still isn't on PATH by default

That said, I think there is one improvement we could feasibly make,
which would be to introduce the notion of a "default user environment"
into `venv`, such that there was a single "python -m venv shell"
command that:

* created a default user environment if it didn't already exist
* launched a subshell with that environment already activated

This wouldn't be a full environment manager like vex or pew - it would
just be a way to bootstrap a single usable package management
environment in a cross-platform way.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 10 November 2017 at 16:55, Michel Desmoulin
 wrote:
> ensurepip may depend of youself having an internet connection when you
> install it. And without a proxy. And it's not used on debian flavours.

No, ensurepip doesn't depend on internet access (by design - it's the
main reason why CPython bundles the wheel files).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 07/11/2017 à 14:06, אלעזר a écrit :
> On Tue, Nov 7, 2017 at 2:45 PM Nick Coghlan  > wrote:
> 
> On 7 November 2017 at 03:52, Michel Desmoulin
> > wrote:
> 
> > And assume that stuff in any tutorial you make they know this stuff.
> >
> > This is a strong barrier or entry IMO.
> 
> Sure, but it's not one we can readily fix - the user hostility of
> command line environments and the compromises we have to make to abide
> by platform conventions are in the hands of operating system vendors,
> and there's only so much we can do to paper over those distinctions
> when user lock-in and putting barriers in the way of cross-device
> portability is a core part of commercial OS vendors' business models.
> 
> 
> I don't know if you are referring to Microsoft Windows here, but I want
> to note that from my personal experience the Windows subsystem for Linux
> ("Bash on Ubuntu on Windows") is easy to work with, so making Windows
> feel (CLI-wise) like Ubuntu is not so difficult. I'm not sure how easy
> it is for students to set up, but it is an option at least.
> 

It doesn't solve anything:

- requires windows 10
- requires having the several Go install first
- mix windows and linux, making the whole thing confusing to beginers
- we still have the heterogenous setup between the 2 os being a problem.

I love WSL, but it's not a solution to that particular problem.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> Which is why we advise getting into a virtual environment ASAP, such
> that the only platform specific thing folks necessarily need to learn
> to get started is how to get to that first working virtual
> environment.
> 

You can't start by teaching virtualenv. I tried. It doesn't work. And
it's a terrible prerequisit if you write docs, tutorial, teaching
materials, etc.

>> - install pip manually on linux
> 
> s/Linux/Ubuntu/
> 
> Other distros (like Fedora) provide pip by default.

s/Ubuntu/Debian derivates

Debian, Ubuntu, Linux Mint, Linux Tail, etc.

> 
>> - make sure the system path is correctly set
> 
> Recent python.org Windows installers do this automatically, but there
> are unfortunately still lots of ways for things to go wrong.

Just gave a training for python 3.6 this week. It doesn't. You need to
explicitly check a box to do so. Box that no beginer would ever check
without being told to. Actually a box that some of my students even
don't check despite being emphatically told to.

> 
>> Stuff that they will forget on the next install, or miss when changing
>> plateform
> 
> Yes, because computers are awful, and incredibly user hostile. We
> don't have a magic wand to wave to fix that.

No, but we can make some adjustment to make lifes easier. For this
particular issue, checking the box by default would help beginners.
Sysadmin not wanting it would understand the problem and be able to take
the proper decision.

> 
>> And assume that stuff in any tutorial you make they know this stuff.
>>
>> This is a strong barrier or entry IMO.
> 
> Sure, but it's not one we can readily fix - the user hostility of
> command line environments and the compromises we have to make to abide
> by platform conventions are in the hands of operating system vendors,
> and there's only so much we can do to paper over those distinctions
> when user lock-in and putting barriers in the way of cross-device
> portability is a core part of commercial OS vendors' business models.

That is  avery pessimistic that would prevent use from making any
progress in any thing. All the other participants offered at least 3
solutions to problems. They are workable. They may help. We should try them.

> 
> This is a big part of why mobile client devices with cloud backends
> are so popular, even for development purposes: they allow for a much
> simpler developer experience that avoids decades of accumulated cruft
> in the desktop operating system command line experience. Even there
> though, you're faced with the fact that once you choose a provider,
> whatever you do there will probably be locked into that provider and
> not transferable elsewhere.

I agree.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 07/11/2017 à 22:26, Paul Moore a écrit :
> On 7 November 2017 at 20:38, Chris Barker  wrote:
>> On Tue, Nov 7, 2017 at 5:04 AM, Thomas Jollans  wrote:
>>>
>>> As Ivan said earlier, perhaps the Windows installers should provide a
>>> "python3" executable, so "python3 -m pip" works everywhere.
>>
>> absolutely! I really, really thought it did (I'm amazed I never heard
>> from a single student getting bit by that...)
> 
> On Windows, use py -X.Y to select the exact version of Python you
> want. Maybe Unix should have a launcher like this, too? It doesn't
> really need to be any more complex than
> 
> 
> exec pythonX.Y $@

That's my whole point. We should find a way to unify each of those issues.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> 
> there can be multiple python2 or 3s too...
> 
> at least with:
> 
> python? -m pip install
> 
> you will get the pip that matches the python you use...

Not on windows. You have the py command. My all point is that each of
the specific issues I mentioned are A and B but not C propositions. It's
not unified.

> 
> with ensurepip, having pip no installed in a python is getting less
> common, so maybe this isn't needed anymore, but

ensurepip may depend of youself having an internet connection when you
install it. And without a proxy. And it's not used on debian flavours.

> 
> -CHB
>  
> 
> -- 
> 
> Christopher Barker, Ph.D.
> Oceanographer
> 
> Emergency Response Division
> NOAA/NOS/OR            (206) 526-6959    voice
> 7600 Sand Point Way NE   (206) 526-6329    fax
> Seattle, WA  98115       (206) 526-6317    main
> reception
> 
> chris.bar...@noaa.gov 
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> Isn't pip installed by default on Windows and OSX? On Linux some
> distribution will probably do something unexpected and mess up your
> grand plan.

Some installers download pip, which won't work if you don't have an
internet connection or a proxy setup.

But my point is, all this should be unified. All the points I mentioned
worked in a A and B but not C, or a combination of those, different for
each point. This is very unfortunate to write docs, tutorials, or any
teaching material.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> 
> I.e., Install under the "Python 3.6" start menu an additional
> "Python command prompt", which will
> start cmd.exe with an appropriate PATH so that python and pip
> run without further prefix.
> 
> That way, the installer still doesn't need to mess with global PATH and
> you can
> easily have multiple versions of Python, each with their own
> "Python command prompt" submenu.
> 
> At least for Windows users this would simplify the situation a bit.


They already do that on windows. But adding "yet another way" to do
things differently on different OS doesn't feel like a solution ot me.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 06/11/2017 à 23:48, Chris Barker a écrit :
> On Mon, Nov 6, 2017 at 9:52 AM, Michel Desmoulin
> > wrote:
> 
> I know and you still:
> 
> - have to use py -m on windows, python3 linux, python in virtualenv...
> 
> 
> can't you use python3 -m pip install .
> 
> everywhere?

No: on windows you don't have python3, you have py -x.x. On linux you
can't pip install, you need --users, admin rights or a virtualenv.

> 
> That's what I tell my beginner students to do, and I've never had a
> problem. (nce they got Python installed right in the first place) For that:
> 
> The python.og installers for Windows and Mac pretty much 
> "just work"

Nope. E.G: in windows, if you don't check "add python executable to
system path" people won't be able to type "python" in the shell. It's
unchecked by default.

> 
> Linux is a different story, but Linux users are more comfortable with eh
> whole idea of command lines and packages, etc -- so my Linux users have
> never been the hangup.

Not anymore. There is a wave of beginners arriving in an industry using
linux but being unfamiliar with the system. The problem is not that it's
hard. The problem is that it's work to discover it. The first Python
experience should be simple and straight forward.

Another problem is that it's not portable knowledge from an OS to another.

> 
> If/when I'm teaching data analysis for scientific computing, I go
> straight to conda, but for basic python, and most web Development,
> pyton.org  python and pip work great.

You can't teach conda first. It's not portable knowledge and would not
let students benefit from the mass of online tutorials that use pip and
virtualenv.

> 
> A also DO NOT introduce virtualenv right off the bat -- it is another
> complication that is critical to real development, but not important for
> learning python.
> 
> I have done it it the past -- it did not go well
> 
> What's too bad now is that so many docs say "pip install the_package",
> and users with multiple python installs can get burnt. (though most don't)
> 

My point exactly.

> -CHB
> 
> 
> -- 
> 
> Christopher Barker, Ph.D.
> Oceanographer
> 
> Emergency Response Division
> NOAA/NOS/OR            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
> 
> chris.bar...@noaa.gov 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Mark Lawrence]

On 06/11/17 22:53, Ivan Pozdeev via Python-ideas wrote:
You can't. Windows versions don't create versioned executables. Got 
bitten with this myself.


You either use py -x.y -mpip or you can directly use pip.exe, pepx.exe 
or pipx.y.exe.




...Maybe they should?
(This is python-ideas, after all ;-) )

--
Regards,
Ivan




--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Ivan Pozdeev via Python-ideas]

On 07.11.2017 23:38, Chris Barker wrote:
with ensurepip, having pip no installed in a python is getting less 
common, so maybe this isn't needed anymore, but
pip is problematic in environments that have their own package manager 
(i.e. anything but bare Windows) because it doesn't honor its 
conventions (e.g. if it's a non-default Python installation, all its 
packages must use versioned or otherwise custom names for executable 
modules - see e.g. 
https://stackoverflow.com/questions/40718770/pytest-running-with-another-version-of-python 
for an illustration). That's why it's not installed by default in 
environments where that's a significant problem.


--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 7 November 2017 at 20:38, Chris Barker  wrote:
> On Tue, Nov 7, 2017 at 5:04 AM, Thomas Jollans  wrote:
>>
>> As Ivan said earlier, perhaps the Windows installers should provide a
>> "python3" executable, so "python3 -m pip" works everywhere.
>
> absolutely! I really, really thought it did (I'm amazed I never heard
> from a single student getting bit by that...)

On Windows, use py -X.Y to select the exact version of Python you
want. Maybe Unix should have a launcher like this, too? It doesn't
really need to be any more complex than


exec pythonX.Y $@

relying on the existence of versioned executables on Unix. (Excuse my
garbled don't-really-know-how-to-do-it shell scripting...)

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Chris Barker]

On Tue, Nov 7, 2017 at 5:04 AM, Thomas Jollans  wrote:
>
> As Ivan said earlier, perhaps the Windows installers should provide a
> "python3" executable, so "python3 -m pip" works everywhere.
>

absolutely! I really, really thought it did (I'm amazed I never heard
from a single student getting bit by that...)


> I normally use "pip2" and "pip3", but those aren't available everywhere,
> and may be hard to support properly.
>

there can be multiple python2 or 3s too...

at least with:

python? -m pip install

you will get the pip that matches the python you use...

with ensurepip, having pip no installed in a python is getting less common,
so maybe this isn't needed anymore, but

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Ivan Pozdeev via Python-ideas]

On 07.11.2017 18:34, Stephan Houben wrote:

One (smaller) suggestion on the PATH situation on Windows:

I noticed that Visual C++ Build Tools installs a number of  "Command 
prompts"
under its Start menu item, each of which starts a cmd.exe with 
appropriate PATH

set to the appropriate compiler (32/64 bits or ARM cross-compiler), and
assorted environment variables set to the appropriate include/library 
directories.


Could we do something similar for Python?

I.e., Install under the "Python 3.6" start menu an additional
"Python command prompt", which will
start cmd.exe with an appropriate PATH so that python and pip
run without further prefix.

That way, the installer still doesn't need to mess with global PATH 
and you can

easily have multiple versions of Python, each with their own
"Python command prompt" submenu.

At least for Windows users this would simplify the situation a bit.

Stephan

This suggestion is no different from environemnt activation scripts of 
anaconda/pyenv/virtualenv -- without the other benefits of virtualenv.


IPython solves this by creating version-specific executables (even in 
Windows). I see no reason why executables of Python proper cannot do the 
same. In fact, the reason why Windows version went Py instead of this 
seems to rather be to solve a completely different problem -- the 
filetype association (when running a script via ShellExecute, it 
autodetects which Python version to invoke).




2017-11-06 23:53 GMT+01:00 Ivan Pozdeev via Python-ideas 
>:


On 07.11.2017 1:48, Chris Barker wrote:

On Mon, Nov 6, 2017 at 9:52 AM, Michel Desmoulin
> wrote:

I know and you still:

- have to use py -m on windows, python3 linux, python in
virtualenv...


can't you use python3 -m pip install .

everywhere?

You can't. Windows versions don't create versioned executables.
Got bitten with this myself.


...Maybe they should?
(This is python-ideas, after all ;-) )

-- 
Regards,

Ivan


___
Python-ideas mailing list
Python-ideas@python.org 
https://mail.python.org/mailman/listinfo/python-ideas

Code of Conduct: http://python.org/psf/codeofconduct/





--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Stephan Houben]

One (smaller) suggestion on the PATH situation on Windows:

I noticed that Visual C++ Build Tools installs a number of  "Command
prompts"
under its Start menu item, each of which starts a cmd.exe with appropriate
PATH
set to the appropriate compiler (32/64 bits or ARM cross-compiler), and
assorted environment variables set to the appropriate include/library
directories.

Could we do something similar for Python?

I.e., Install under the "Python 3.6" start menu an additional
"Python command prompt", which will
start cmd.exe with an appropriate PATH so that python and pip
run without further prefix.

That way, the installer still doesn't need to mess with global PATH and you
can
easily have multiple versions of Python, each with their own
"Python command prompt" submenu.

At least for Windows users this would simplify the situation a bit.

Stephan



2017-11-06 23:53 GMT+01:00 Ivan Pozdeev via Python-ideas <
python-ideas@python.org>:

> On 07.11.2017 1:48, Chris Barker wrote:
>
> On Mon, Nov 6, 2017 at 9:52 AM, Michel Desmoulin <
> desmoulinmic...@gmail.com> wrote:
>
>> I know and you still:
>>
>> - have to use py -m on windows, python3 linux, python in virtualenv...
>>
>
> can't you use python3 -m pip install .
>
> everywhere?
>
> You can't. Windows versions don't create versioned executables. Got bitten
> with this myself.
>
>
> ...Maybe they should?
> (This is python-ideas, after all ;-) )
>
> --
> Regards,
> Ivan
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Ivan Pozdeev via Python-ideas]

On 07.11.2017 16:11, Paul Moore wrote:

It is, but like any such approach (Cygwin is similar, in principle if
not in execution) that makes one OS "look like" another, whether it's
appropriate is very dependent on circumstances. Training potential
Windows developers in a bash/Ubuntu style environment leaves them at a
disadvantage when they need to develop for actual Windows
environments.
Moreover, one of the key selling points of Python is that it can 
interface with anything and everything found in any environment!
Rather than e.g. conjuring up a closed circle with own replacements for 
everything than forces everyone into the lowest common denominator like 
Java does.


*That's another key reason why I use & suggest using an installation 
"native" to the system whenever possible.*
It's as close to "being an integral part of the environment rather than 
rejecting it and creating an inferior copy" philosophy as one can get.


--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 7 November 2017 at 13:06, אלעזר  wrote:
> On Tue, Nov 7, 2017 at 2:45 PM Nick Coghlan  wrote:
>>
>> On 7 November 2017 at 03:52, Michel Desmoulin 
>> wrote:
>>
>> > And assume that stuff in any tutorial you make they know this stuff.
>> >
>> > This is a strong barrier or entry IMO.
>>
>> Sure, but it's not one we can readily fix - the user hostility of
>> command line environments and the compromises we have to make to abide
>> by platform conventions are in the hands of operating system vendors,
>> and there's only so much we can do to paper over those distinctions
>> when user lock-in and putting barriers in the way of cross-device
>> portability is a core part of commercial OS vendors' business models.
>>
>
> I don't know if you are referring to Microsoft Windows here, but I want to
> note that from my personal experience the Windows subsystem for Linux ("Bash
> on Ubuntu on Windows") is easy to work with, so making Windows feel
> (CLI-wise) like Ubuntu is not so difficult. I'm not sure how easy it is for
> students to set up, but it is an option at least.

It is, but like any such approach (Cygwin is similar, in principle if
not in execution) that makes one OS "look like" another, whether it's
appropriate is very dependent on circumstances. Training potential
Windows developers in a bash/Ubuntu style environment leaves them at a
disadvantage when they need to develop for actual Windows
environments.

It's certainly an option, but so is "train them on Linux".
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Thomas Jollans]

On 2017-11-06 19:04, Michel Desmoulin wrote:
> 
>> I don't see anything particularly bogging here.
>> It's always like this when you have multiple versions of the same
>> software on the system. There's only one PATH, after all.
>>
>> Heck, *the mere fact that Python allows to work like this is already a
>> huge leap forward.* **Doing this cross-platform with exactly the same
>> steps is something few could believe was even possible a couple of years
>> ago!**
> 
> I agree.
> 
>>
>> To simplify things with Python, i do the following:
>> * use the system's Python whenever possible
> 
> So python 2.7 on mac and some linux or none for windows...
> 
>> * if using something else, only install the one version/environment that
>> I'm using day-to-day and add it to PATH (system's if safe & convenient,
>> personal otherwise)
>> * prefer the system's/environment's package manager to pip to install
>> 3rd-party modules, too, if there is one.
> 
> We can't solve the situation perfectly, but we can unify a bit. E.G:
> 
> - provide the "py" command on all OSes to avoid the various naming and
> aliases of python
> - promote it in documentation
> - promote the use of py -x.x -m pip instead of the myriads of alternatives

I'm sure there's a reason why these acrobatics are required on Windows,
but on other OS's separate "python3", "python3.6", "python2",
"python2.7" and "python" executables/symlinks all in the PATH work
perfectly fine.

As Ivan said earlier, perhaps the Windows installers should provide a
"python3" executable, so "python3 -m pip" works everywhere.

I normally use "pip2" and "pip3", but those aren't available everywhere,
and may be hard to support properly.


> - provide an empty pip and venv module. If they are not here, py -m pip
> says "your plateform doesn't provide pip by default, please do x" to
> install it. With "" being plateform specific.

Isn't pip installed by default on Windows and OSX? On Linux some
distribution will probably do something unexpected and mess up your
grand plan.

> - check "add python executable to system path" on the windows installer
> once by defaut

I like this, but I have a suspicion that this is rather unorthodox.


-- 
Thomas Jollans
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Paul Moore]

On 7 November 2017 at 12:44, Nick Coghlan  wrote:
>> - make sure the system path is correctly set
>
> Recent python.org Windows installers do this automatically, but there
> are unfortunately still lots of ways for things to go wrong.

I believe the latest installers switch it off again, because one of
the ways things can go wrong is that stuff put at the start of the
user path is still lower priority than stuff in the system path, and
we now default to user installs. This is an actual problem with mixed
python.org and Anaconda installations, for example - Anaconda adds
itself to the system PATH, and overrides a default user install of
python.org Python. So you can't prioritise python.org over Anaconda
without manual path hacking. (This hit me when I installed Visual
Studio and selected "include Python (Anaconda)" - I can't recall the
exact option, but it broke my python.org install).

As you say command-line environments are just inherently user-hostile,
and we can't do a lot about that. People who buy into the command line
experience learn how to deal with the complexity. People who use IDEs
like Visual Studio or PyCharm can rely on the IDE vendor to provide a
clean experience. But people who want to use core Python but who
aren't comfortable fighting with the command line have a bit of a
problem. We can't solve that problem for them, the best we can do is
offer suggestions on best practices, or tools that help alleviate the
issues.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Thomas Jollans]

On 2017-11-06 23:53, Ivan Pozdeev via Python-ideas wrote:
> On 07.11.2017 1:48, Chris Barker wrote:
>> On Mon, Nov 6, 2017 at 9:52 AM, Michel Desmoulin
>> > wrote:
>>
>> I know and you still:
>>
>> - have to use py -m on windows, python3 linux, python in virtualenv...
>>
>>
>> can't you use python3 -m pip install .
>>
>> everywhere?
> You can't. Windows versions don't create versioned executables. Got
> bitten with this myself.

Once you're worked around that in the first place (lesson: "this is how
you start Python"), then figuring out whether you need "python -m pip"
or "python3 -m pip" shouldn't be a big deal.

Still the fact that the way you call Python is different on different
platforms is rather unfortunate.

-- Thomas

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[אלעזר]

On Tue, Nov 7, 2017 at 2:45 PM Nick Coghlan  wrote:

> On 7 November 2017 at 03:52, Michel Desmoulin 
> wrote:

> And assume that stuff in any tutorial you make they know this stuff.
> >
> > This is a strong barrier or entry IMO.
>
> Sure, but it's not one we can readily fix - the user hostility of
> command line environments and the compromises we have to make to abide
> by platform conventions are in the hands of operating system vendors,
> and there's only so much we can do to paper over those distinctions
> when user lock-in and putting barriers in the way of cross-device
> portability is a core part of commercial OS vendors' business models.
>
>
I don't know if you are referring to Microsoft Windows here, but I want to
note that from my personal experience the Windows subsystem for Linux
("Bash on Ubuntu on Windows") is easy to work with, so making Windows feel
(CLI-wise) like Ubuntu is not so difficult. I'm not sure how easy it is for
students to set up, but it is an option at least.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 7 November 2017 at 03:52, Michel Desmoulin  wrote:
>
>
> Le 06/11/2017 à 09:47, Nick Coghlan a écrit :
>> On 6 November 2017 at 16:47, Michel Desmoulin  
>> wrote:
>>> I really want some people from this list to discuss here so we can find
>>> a way to either unify a bit the way we install and use pip, or find a
>>> way to express a tutorial that always works for people on the most
>>> popular platforms and spread the word so that any doc uses it.
>>
>> https://docs.python.org/3/installing/#basic-usage is as close as we've
>> been able to get to that for the time being.
>
> I know and you still:
>
> - have to use py -m on windows, python3 linux, python in virtualenv...

Which is why we advise getting into a virtual environment ASAP, such
that the only platform specific thing folks necessarily need to learn
to get started is how to get to that first working virtual
environment.

> - install pip manually on linux

s/Linux/Ubuntu/

Other distros (like Fedora) provide pip by default.

> - make sure the system path is correctly set

Recent python.org Windows installers do this automatically, but there
are unfortunately still lots of ways for things to go wrong.

> Stuff that they will forget on the next install, or miss when changing
> plateform.

Yes, because computers are awful, and incredibly user hostile. We
don't have a magic wand to wave to fix that.

> And assume that stuff in any tutorial you make they know this stuff.
>
> This is a strong barrier or entry IMO.

Sure, but it's not one we can readily fix - the user hostility of
command line environments and the compromises we have to make to abide
by platform conventions are in the hands of operating system vendors,
and there's only so much we can do to paper over those distinctions
when user lock-in and putting barriers in the way of cross-device
portability is a core part of commercial OS vendors' business models.

This is a big part of why mobile client devices with cloud backends
are so popular, even for development purposes: they allow for a much
simpler developer experience that avoids decades of accumulated cruft
in the desktop operating system command line experience. Even there
though, you're faced with the fact that once you choose a provider,
whatever you do there will probably be locked into that provider and
not transferable elsewhere.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Ivan Pozdeev via Python-ideas]

On 07.11.2017 1:48, Chris Barker wrote:
On Mon, Nov 6, 2017 at 9:52 AM, Michel Desmoulin 
> wrote:


I know and you still:

- have to use py -m on windows, python3 linux, python in virtualenv...


can't you use python3 -m pip install .

everywhere?
You can't. Windows versions don't create versioned executables. Got 
bitten with this myself.



...Maybe they should?
(This is python-ideas, after all ;-) )

--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Chris Barker]

On Mon, Nov 6, 2017 at 9:52 AM, Michel Desmoulin 
wrote:

> I know and you still:
>
> - have to use py -m on windows, python3 linux, python in virtualenv...
>

can't you use python3 -m pip install .

everywhere?

That's what I tell my beginner students to do, and I've never had a
problem. (nce they got Python installed right in the first place) For that:

The python.og installers for Windows and Mac pretty much
"just work"

Linux is a different story, but Linux users are more comfortable with eh
whole idea of command lines and packages, etc -- so my Linux users have
never been the hangup.

If/when I'm teaching data analysis for scientific computing, I go straight
to conda, but for basic python, and most web Development, pyton.org python
and pip work great.

A also DO NOT introduce virtualenv right off the bat -- it is another
complication that is critical to real development, but not important for
learning python.

I have done it it the past -- it did not go well

What's too bad now is that so many docs say "pip install the_package", and
users with multiple python installs can get burnt. (though most don't)

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> I don't see anything particularly bogging here.
> It's always like this when you have multiple versions of the same
> software on the system. There's only one PATH, after all.
> 
> Heck, *the mere fact that Python allows to work like this is already a
> huge leap forward.* **Doing this cross-platform with exactly the same
> steps is something few could believe was even possible a couple of years
> ago!**

I agree.

> 
> To simplify things with Python, i do the following:
> * use the system's Python whenever possible

So python 2.7 on mac and some linux or none for windows...

> * if using something else, only install the one version/environment that
> I'm using day-to-day and add it to PATH (system's if safe & convenient,
> personal otherwise)
> * prefer the system's/environment's package manager to pip to install
> 3rd-party modules, too, if there is one.

We can't solve the situation perfectly, but we can unify a bit. E.G:

- provide the "py" command on all OSes to avoid the various naming and
aliases of python
- promote it in documentation
- promote the use of py -x.x -m pip instead of the myriads of alternatives
- provide an empty pip and venv module. If they are not here, py -m pip
says "your plateform doesn't provide pip by default, please do x" to
install it. With "" being plateform specific.
- check "add python executable to system path" on the windows installer
once by defaut




___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 06/11/2017 à 09:50, Stephan Houben a écrit :
> Hi Michel,
> 
> That's exactly why I proposed a `pip` function available from the Python
> prompt.
> I suppose you could still tell your students to copy/paste the following
> into their
> Python interpreter.
> 
> def pip(args):
>     import sys
>     import subprocess
>     subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
>     print("Please restart Python now to use installed or upgraded
> packages.")


You still need to install pip on platforms it's not available. And do
that in a virtualenv or do --user. And ask for a restart.

> I suppose an alternative is to set up jupyterhub
> 
> https://jupyterhub.readthedocs.io/en/latest/
> 

I love jupyter, but you can't limit yourself to it. That won't give them
autonomy and will limit very much what they can do. E.G: You can't use
tkinter in jupyter easily.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> 
> Software Carpentry starts out with the Anaconda distribution, as it
> not only improves the cross-platform UX consistent situation, it also
> deals with the external binary dependency problem (at least for the
> core set of packages provided either natively or via conda-forge).
> 

Yeah but the knowledge is not transposable. You can't understand a lot
of tutorials that uses pip. Plus it has licences issues. And if somebody
starts mixing conda and pip, it can results in strange things.

I do use anaconda, but you can't rely solely on it.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Le 06/11/2017 à 09:47, Nick Coghlan a écrit :
> On 6 November 2017 at 16:47, Michel Desmoulin  
> wrote:
>> I really want some people from this list to discuss here so we can find
>> a way to either unify a bit the way we install and use pip, or find a
>> way to express a tutorial that always works for people on the most
>> popular platforms and spread the word so that any doc uses it.
> 
> https://docs.python.org/3/installing/#basic-usage is as close as we've
> been able to get to that for the time being.

I know and you still:

- have to use py -m on windows, python3 linux, python in virtualenv...
- install pip manually on linux
- make sure the system path is correctly set

Stuff that they will forget on the next install, or miss when changing
plateform.

And assume that stuff in any tutorial you make they know this stuff.

This is a strong barrier or entry IMO.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

> 
> Do you know about "fades"?
> 
> https://fades.readthedocs.io/en/release_6_0/

So when they read any tutorial online they don't have reusable knowledge ?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Ivan Pozdeev via Python-ideas]

On 06.11.2017 9:47, Michel Desmoulin wrote:

Hello,

Today I'm going to give a training in Python again.

And again it will go the same way.

On Mac I will have to make people install python, then tell them to use
pip3.

On Windows, I will have to warn them about checking the "add python
executable to system path" (that one of them will ALWAYS miss anyway).
Then tell them to use py -3.x -m pip because some of them will have
several versions of Python installed.

Then on linux, I will tell them to install python-pip and python-venv
and use python3 -m pip.

I'll talk about --user, but commands won't be usable on some machine
where the Scripts or bin dir is not in the system path.

Then I will make them create a virtualenv so that they can avoid messing
with their system python and finally can just use "pip install" like in
most tutorials on the Web.

And eventually I'll talk about pipenv and conda. The first one so they
don't have to think about activating the virtualenv everytime, or pip
freeze, or create the venv, or add it to gitignore, etc. The second
because anaconda is very popular on windows.

There is no way a beginner is going to get any that by themselves
without a lot of time and pain. They will read some tutorial on the web
and struggle to make sens of what pip is and why "pip install" doesn't
work and why "python sucks".

I don't see anything particularly bogging here.
It's always like this when you have multiple versions of the same 
software on the system. There's only one PATH, after all.


Heck, *the mere fact that Python allows to work like this is already a 
huge leap forward.* **Doing this cross-platform with exactly the same 
steps is something few could believe was even possible a couple of years 
ago!**


With most other software packages, it's either not possible at all (and 
an installer for a different version of a package requires uninstalling 
the existing version first), or you need to compile everything from 
source with custom --prefix or something -- and deal with dependency 
hell looming over your head when they unexpectedly override some system 
components, or have to specify paths all the time.


This is why system package managers in Linux distributions are such a 
huge help -- as long as you're using the same repository (or a 
closely-knitted group thereof), you're supposed to have a sane 
environment whatever packages you install, with nothing overriding 
anything else.



To simplify things with Python, i do the following:
* use the system's Python whenever possible
* if using something else, only install the one version/environment that 
I'm using day-to-day and add it to PATH (system's if safe & convenient, 
personal otherwise)
* prefer the system's/environment's package manager to pip to install 
3rd-party modules, too, if there is one.


For personal day-to-day use, it's very rarely critical to use precisely 
specific versions of Python and packages.
For commerical use, I'll either mainly use one environment still (the 
one my production or my client has), or have to deal with multiple 
environments anyway, or set a up a dedicated CI server/environment and 
leave its scripts to deal with them.



I think Python is offering an incredible experience for first timer.
However, the whole "where is walpip" shenanigans is not one of them.

I really want some people from this list to discuss here so we can find
a way to either unify a bit the way we install and use pip, or find a
way to express a tutorial that always works for people on the most
popular platforms and spread the word so that any doc uses it.

Michel

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/


--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 6 November 2017 at 18:50, Stephan Houben  wrote:
> Hi Michel,
>
> That's exactly why I proposed a `pip` function available from the Python
> prompt.
> I suppose you could still tell your students to copy/paste the following
> into their
> Python interpreter.
>
> def pip(args):
> import sys
> import subprocess
> subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
> print("Please restart Python now to use installed or upgraded
> packages.")

Depending on where and how Python is installed, this may still not
work (while I'll grant that Linux users are more likely to already be
familiar with the command line than Windows and Mac OS X users, it's
still a non-trivial step from there to realising why "sudo pip
install" is almost always a bad idea)

> I suppose an alternative is to set up jupyterhub
>
> https://jupyterhub.readthedocs.io/en/latest/
>
> and let all your students just access that from a webbrowser.

Yep, and lots of teachers use services like PythonAnywhere,
trinket.io, and similar, precisely because the only thing the learners
need locally is a web browser. The main downside is that learning that
way isn't quite as transferable a skill, since it completely hides the
code packaging and distribution step. However, it's a good option when
the primary aim is to teach computational skills, and Python is just
the vehicle used for that purpose.

Software Carpentry starts out with the Anaconda distribution, as it
not only improves the cross-platform UX consistent situation, it also
deals with the external binary dependency problem (at least for the
core set of packages provided either natively or via conda-forge).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Stephan Houben]

Hi Michel,

That's exactly why I proposed a `pip` function available from the Python
prompt.
I suppose you could still tell your students to copy/paste the following
into their
Python interpreter.

def pip(args):
import sys
import subprocess
subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
print("Please restart Python now to use installed or upgraded
packages.")

I suppose an alternative is to set up jupyterhub

https://jupyterhub.readthedocs.io/en/latest/

and let all your students just access that from a webbrowser.

Stephan


2017-11-06 7:47 GMT+01:00 Michel Desmoulin :

> Hello,
>
> Today I'm going to give a training in Python again.
>
> And again it will go the same way.
>
> On Mac I will have to make people install python, then tell them to use
> pip3.
>
> On Windows, I will have to warn them about checking the "add python
> executable to system path" (that one of them will ALWAYS miss anyway).
> Then tell them to use py -3.x -m pip because some of them will have
> several versions of Python installed.
>
> Then on linux, I will tell them to install python-pip and python-venv
> and use python3 -m pip.
>
> I'll talk about --user, but commands won't be usable on some machine
> where the Scripts or bin dir is not in the system path.
>
> Then I will make them create a virtualenv so that they can avoid messing
> with their system python and finally can just use "pip install" like in
> most tutorials on the Web.
>
> And eventually I'll talk about pipenv and conda. The first one so they
> don't have to think about activating the virtualenv everytime, or pip
> freeze, or create the venv, or add it to gitignore, etc. The second
> because anaconda is very popular on windows.
>
> There is no way a beginner is going to get any that by themselves
> without a lot of time and pain. They will read some tutorial on the web
> and struggle to make sens of what pip is and why "pip install" doesn't
> work and why "python sucks".
>
> I think Python is offering an incredible experience for first timer.
> However, the whole "where is walpip" shenanigans is not one of them.
>
> I really want some people from this list to discuss here so we can find
> a way to either unify a bit the way we install and use pip, or find a
> way to express a tutorial that always works for people on the most
> popular platforms and spread the word so that any doc uses it.
>
> Michel
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Nick Coghlan]

On 6 November 2017 at 16:47, Michel Desmoulin  wrote:
> I really want some people from this list to discuss here so we can find
> a way to either unify a bit the way we install and use pip, or find a
> way to express a tutorial that always works for people on the most
> popular platforms and spread the word so that any doc uses it.

https://docs.python.org/3/installing/#basic-usage is as close as we've
been able to get to that for the time being.

For Linux, you'll still need to do the initial "python3 -m venv" to
get your students out of the system Python.

I expect you'll also need an initial "py -m venv" for Windows users,
to get their PATH configured appropriately with both a "python"
command and any scripts they install from PyPI.

But the key point: do *NOT* try to teach without creating a virtual
environment as the first step, because it doesn't actually make
anything simpler, and in fact makes a lot of things harder and more
platform dependent.

The tutorial in the Python Packaging User Guide similarly starts with
venv creation: 
https://packaging.python.org/tutorials/installing-packages/#creating-virtual-environments

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Looking for input to help with the pip situation

[Michel Desmoulin]

Hello,

Today I'm going to give a training in Python again.

And again it will go the same way.

On Mac I will have to make people install python, then tell them to use
pip3.

On Windows, I will have to warn them about checking the "add python
executable to system path" (that one of them will ALWAYS miss anyway).
Then tell them to use py -3.x -m pip because some of them will have
several versions of Python installed.

Then on linux, I will tell them to install python-pip and python-venv
and use python3 -m pip.

I'll talk about --user, but commands won't be usable on some machine
where the Scripts or bin dir is not in the system path.

Then I will make them create a virtualenv so that they can avoid messing
with their system python and finally can just use "pip install" like in
most tutorials on the Web.

And eventually I'll talk about pipenv and conda. The first one so they
don't have to think about activating the virtualenv everytime, or pip
freeze, or create the venv, or add it to gitignore, etc. The second
because anaconda is very popular on windows.

There is no way a beginner is going to get any that by themselves
without a lot of time and pain. They will read some tutorial on the web
and struggle to make sens of what pip is and why "pip install" doesn't
work and why "python sucks".

I think Python is offering an incredible experience for first timer.
However, the whole "where is walpip" shenanigans is not one of them.

I really want some people from this list to discuss here so we can find
a way to either unify a bit the way we install and use pip, or find a
way to express a tutorial that always works for people on the most
popular platforms and spread the word so that any doc uses it.

Michel

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/