Re: [Distutils] Module Installation Issues

[Nick Coghlan]

On 16 September 2016 at 05:38, Wes Turner  wrote:
> You can install bash on windows. (GoW, Cygwin, Ming), but Docker (and, soon,
> runC OCP 1.0) is probably your best bet for maximum compatibility with most
> {python, open source} tutorials.

Wes, none of these are beginner friendly recommendations - our target
audience here is folks that *don't yet know what the phrase "command
prompt" means*.

So while there are a myriad of ways to get access to a *nix prompt on
Windows, they're all targeted at folks that are already experienced
developers, and they're also the wrong answer when the goal is to help
people to more effectively use the computer they already have, not the
computer we happen to think they *should* have.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Wes Turner]

You can install bash on windows. (GoW, Cygwin, Ming), but Docker (and,
soon, runC OCP 1.0) is probably your best bet for maximum compatibility
with most {python, open source} tutorials. Installing the code as non-root
is a good idea (e.g. with a .zip or a .tar.gz or a Python .whl wheel)


I think the bash $ and # PS1 should be fairly easy to differentiate from
unprefixed expressions. IDK if those are the POSIX defaults.

These list {Linux,Mac} when Windows is not supported:
https://docs.continuum.io/anaconda/pkg-docs

.rst/ReStructuredText: code directive

.. code:: bash

# Check out the switches for ifconfig/ipconfig /h.


.. code:: python

   """PowerShell is open source now"""


.md/Markdown: fenced code block

```bash
bash --version
help declare -r
```

```python
python -v
```

On Thursday, September 15, 2016, Thomas Kluyver 
wrote:

> On Thu, Sep 15, 2016, at 01:53 PM, Wes Turner wrote:
>
> Shell commands can/could/should be clearly indicated with a '$' prefix:
>
> $ python -m pip install numpy
>
>
> That's a common convention, but:
>
> - It comes from POSIX platforms where the default prompt ends in $. The
> default prompt in Windows ends with >, so $ is not clear.
> - On all platforms, new users may well not recognise that convention.
> - Users may think they have to type the $ as part of the command, leading
> to even more confusion.
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Thomas Kluyver]

On Thu, Sep 15, 2016, at 01:53 PM, Wes Turner wrote:
> Shell commands can/could/should be clearly indicated with a '$'
> prefix:
>
> $ python -m pip install numpy

That's a common convention, but:

- It comes from POSIX platforms where the default prompt ends in $. The
  default prompt in Windows ends with >, so $ is not clear.
- On all platforms, new users may well not recognise that convention.
- Users may think they have to type the $ as part of the command,
  leading to even more confusion.
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Wes Turner]

Useful banking/finance Python resources:

- https://pypi.python.org/pypi/ofxparse
- https://wrdrd.com/docs/consulting/investing#quantopian
- Places that have some Python in their stack:
  - JP Morgan
  - Bank of America Merrill Lynch
  - Quantopian
   - qgrid, zipline, pyfolio, ...


On Thursday, September 15, 2016, Wes Turner  wrote:

>
>
> On Tuesday, September 13, 2016, Mills, Ryan  com >
> wrote:
>
>> I just recently downloaded Python 3.5 and cannot seem to install any
>> packages like Numpy, etc.  I have tried all the instructions on the website
>> and keep getting errors:
>>
>>
>>
>> For example, when I type “python –m pip install Numpy” it returns  a
>> Syntax Error. I am completely new to Python so I must be missing something
>> here – I haven’t altered any files since installing it the other day. Do I
>> use the Python IDLE Shell? Are there other packages I need to install
>> first?  Any help would be greatly appreciated.
>>
>
> Shell commands can/could/should be clearly indicated with a '$' prefix:
>
> $ python -m pip install numpy
>
> Python expressions are sometimes prefixed with '>>>':
>
> >>> import numpy as np
>
> TBH, as a beginner, it's probably way easier to start with Anaconda (conda
> packages) and pip (python packages) (because there are a number of
> libraries required to build numpy and then whatever else):
>
> - https://docs.continuum.io/anaconda/
> - $ conda install pip; python -m pip install
> - $ which python; python -m site
>
> The software carpentry lessons are a good place to start from:
>
> - http://software-carpentry.org/lessons/
>   - http://swcarpentry.github.io/python-novice-inflammation/
>
> Also great resources:
>
> - http://www.scipy-lectures.org
> - https://github.com/jrjohansson/scientific-python-lectures
> (IPython/Jupyter nb)
>
> - https://westurner.org/wiki/awesome-python-testing#python
>   - https://westurner.org/tools/#python
>   - http://learnxinyminutes.com/docs/python/
>   - http://docs.python.org/2/library/unittest.html
>
>   - https://westurner.org/tools/#numpy
>
> - IDLE is pretty cool.
> - IPython is great.
>
> - Jupyter Notebook is a reverse shell.
> - Spyder ($ conda install spyder)
> - [commercial IDE preference]
>
>
> - If you write tests from the start, there's less run/check/run/check
> manual testing and more test coverage.
>
>
>
>>
>> -Ryan
>>
>>
>>
>> Ryan Mills
>>
>> *Quantitative Risk Analyst, Banking Officer*
>>
>> [image: cid:image001.jpg@01CA3228.EF8AEBA0]
>>
>> Capital Analytics & Stress Testing
>> 2000 McKinney Avenue, Suite 700
>>
>> Dallas, TX 75201
>>
>> 214.932.6653 direct
>>
>> 20.6653 internal
>> ryan.mi...@texascapitalbank.com
>>
>> [image: cid:image001.jpg@01CA3228.EF8AEBA0]
>>
>> [image: TCB_horiz_Log_rgb]
>>
>>
>>
>> If you are not the addressee and have received this email in error,
>> please notify me immediately. This email is confidential and may contain
>> privileged or proprietary information that is unlawful for you to read,
>> copy, distribute, disclose or otherwise use in any way.
>>
>>
>>
>>
>>
>
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Nick Coghlan]

On 15 September 2016 at 01:31, Paul Moore  wrote:
> On 13 September 2016 at 23:55, Donald Stufft  wrote:
>> Perhaps a better idea would be to add some smarts to the REPL (but not to 
>> Python itself) that would detect something like:
>>
> pip install
>>
>> And print a better error message that gives a better indication about what’s 
>> gone wrong besides a SyntaxError?
>
> This is pretty easy to do:
>
> def excepthook(typ, value, traceback):
> # Tidy this up to suit
> if typ is SyntaxError and value.text.startswith("pip"):
> print("You may want to run this from a shell prompt")
> return
> sys.__excepthook__(typ, value, traceback)
>
> sys.excepthook = excepthook

Good point, now noted at http://bugs.python.org/issue28140#msg276506

> Of course, it doesn't solve the problem of a user who doesn't know
> what a "shell prompt" is...

As Thomas suggested, even though *we* use "shell" specifically for the
system shell, and "REPL" for the interactive prompt, something like
"system command prompt" is likely the better phrase, since the REPL
does get referred to as a "Python shell" fairly often, including by
IDLE.

Plugging "system command prompt" into Google doesn't give helpful
results, though.

An unqualified "command prompt" is better, as searching for that gives
generic Windows command line access instructions, while qualifying it
with "command prompt mac os x" gives Mac instructions and "command
prompt linux" gives decent Linux instructions.

As far as the error message goes, what I'd suggest is:

1. detect "pip install" specifically (anywhere in the line), not just "pip"
2. report what triggered the custom recommendation in addition to the
recommendation itself

For example:

>>> python –m pip install Numpy
SyntaxError: 'pip install' is a command line operation
Try running this at a command prompt rather than from Python

Once the pip bundled via ensurepip gains "pip.install()" support
(assuming we end up doing that), then the custom message can change
to:

>>> python –m pip install Numpy
SyntaxError: 'pip install' is a command line operation
Try running this at a command prompt rather than from Python, or else try:

import pip
pip.install("NumPy")

Regards,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Thomas Kluyver]

On Wed, Sep 14, 2016, at 04:31 PM, Paul Moore wrote:
> Of course, it doesn't solve the problem of a user who doesn't know
> what a "shell prompt" is...

Especially since the interactive Python interpreter is also a shell
prompt. The title of an IDLE window on my system is "Python 3.5.2 Shell"
;-). I'd suggest the terminology "system command prompt" to refer to...
that thing that you should type 'pip' into.

Even if users don't know exactly what it means, indicating that they're
running something in the wrong place gives them a better chance of
working out what to do.

Thomas
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Paul Moore]

On 13 September 2016 at 23:55, Donald Stufft  wrote:
> Perhaps a better idea would be to add some smarts to the REPL (but not to 
> Python itself) that would detect something like:
>
 pip install
>
> And print a better error message that gives a better indication about what’s 
> gone wrong besides a SyntaxError?

This is pretty easy to do:

def excepthook(typ, value, traceback):
# Tidy this up to suit
if typ is SyntaxError and value.text.startswith("pip"):
print("You may want to run this from a shell prompt")
return
sys.__excepthook__(typ, value, traceback)

sys.excepthook = excepthook

Of course, it doesn't solve the problem of a user who doesn't know
what a "shell prompt" is...

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Nick Coghlan]

On 14 September 2016 at 22:27, Marius Gedminas  wrote:
> On Tue, Sep 13, 2016 at 06:55:49PM -0400, Donald Stufft wrote:
>> >>> pip install
>>
>> And print a better error message that gives a better indication about what’s 
>> gone wrong besides a SyntaxError?
>
> This way you could also make
>
>   >>> exit
>
> work like people expect it to work, without the ().

That's never going to happen, as aside from some additional builtins,
the default REPL will always work the same way as non-interactive
code.

Cheers,
Nick.

P.S. It's also entirely unrelated to the current discussion, as it's
not a SyntaxError, it's a valid reference to the exit builtin injected
by the site module.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Marius Gedminas]

On Tue, Sep 13, 2016 at 06:55:49PM -0400, Donald Stufft wrote:
> >>> pip install
> 
> And print a better error message that gives a better indication about what’s 
> gone wrong besides a SyntaxError?

This way you could also make

  >>> exit

work like people expect it to work, without the ().

Marius Gedminas
-- 
If your company is not involved in something called "ISO 9000" you probably
have no idea what it is.  If your company _is_ involved in ISO 9000 then you
definitely have no idea what it is.
(Scott Adams - The Dilbert principle)


signature.asc
Description: PGP signature
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Nick Coghlan]

On 14 September 2016 at 21:43, Donald Stufft  wrote:
>
>> On Sep 13, 2016, at 7:06 PM, Steve Dower  wrote:
>>
>> This is exactly the argument I was expecting when I said "those who want it 
>> to be perfect in every scenario"
>
> FWIW I don’t have a strong feeling on whether we should add it or not. I 
> haven’t been a beginner in a long time and I am not very good at putting 
> myself in their shoes. I legitimately don’t know what would be more or less 
> confusing here, possible weird state (with or without a warning that they 
> might be getting into weird state territory) or an error message telling them 
> to use a real shell and not the Python REPL.

I think they're both confusing, but one of them we can be more certain
the user will have the ability to resolve, regardless of their
computing background.

Specifically, someone already familiar with the difference between a
system shell and the Python REPL isn't likely to make the mistake in
question in the first place - "$ pip install name" is clearly a shell
command *if* you know what the "$" indicates, while the unqualified
"pip install name" used in Warehouse for cross-platform consistency
still looks more like a system command than it does a Python
expression (it would be a potentially valid function call in some
other languages though, most notably MATLAB in the context of this
discussion).

For someone that *doesn't* know the difference though, the only thing
we can be sure of is that they know how to run a Python REPL, and
hence presumably how to stop and restart that REPL. It's also the case
that for through-the-browser development, the user may not have system
shell access at all, even if they do already know the difference
between the two environments.

So if we're able to emit a warning that says "NOTE: Updated package
installed for already loaded module ''. Restart Python to load
the new version.", then novice users are likely to take us at our word
and restart Python. Intermediate users may say "Hah, I know
imp.reload(), so I can ignore that warning!" and try to selective
reload things. Experienced users more familiar with what a debugging
nightmare selective reloading can be with a deep module dependency
tree and usage of the "from X import Y" early binding syntax may then
end up going back to just restarting their REPL session after
installing new things :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Donald Stufft]

> On Sep 13, 2016, at 7:06 PM, Steve Dower  wrote:
> 
> This is exactly the argument I was expecting when I said "those who want it 
> to be perfect in every scenario" 

FWIW I don’t have a strong feeling on whether we should add it or not. I 
haven’t been a beginner in a long time and I am not very good at putting myself 
in their shoes. I legitimately don’t know what would be more or less confusing 
here, possible weird state (with or without a warning that they might be 
getting into weird state territory) or an error message telling them to use a 
real shell and not the Python REPL.

—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Nick Coghlan]

On 14 September 2016 at 18:02, Paul Moore  wrote:
> On 13 September 2016 at 23:55, Donald Stufft  wrote:
>> The 90% case works when it’s *only* pure python and there’s no 
>> upgrading/downgrading involved, however you can’t control whether there are 
>> going to be upgrades/downgrades or not when dependencies are in play because 
>> of ==, >=, >, <, <=, etc.
>
> I would be OK with an invocation that only ever installed and never
> did an upgrade, as that would avoid the worst situations. Of course,
> (a) AFAIK pip doesn't have such a method (other than --no-deps) at the
> moment (dependencies are always upgraded if needed) and (b) there is
> an ongoing proposal to change "pip install" to always upgrade, and
> this is the exactly the opposite of what we'd need from the Python
> prompt.
>
> I understood Steve's proposal to be that we detect cases which do an
> upgrade rather than a pure install, and in that case report "you
> should restart your Python session". That's a common[1] pattern on
> Windows, so should be familiar to Windows users, who I suspect are
> more likely to hit this issue.
>
> Paul
>
> [1] Common, but not particularly loved - it's the source of the jokes
> about "your mouse has moved, do you want to restart Windows for this
> change to take effect?"

Agreed, but if we're able to limit the warning to cases where:

1. The requested package wasn't already present; and
2. Installing it upgraded one or more existing packages

that should eliminate a lot of the warning noise.

We could even enhance the filtering a little further by having pip
look at the top level module names in the built wheel(s) and only emit
the warning if at least one of those is already present in
sys.modules. That way, doing:

>>> pip.install("i_need_this")
>>> import i_need_this

in a clean REPL wouldn't emit a warning even if something was upgraded
as part of the install step.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Paul Moore]

On 13 September 2016 at 23:55, Donald Stufft  wrote:
> The 90% case works when it’s *only* pure python and there’s no 
> upgrading/downgrading involved, however you can’t control whether there are 
> going to be upgrades/downgrades or not when dependencies are in play because 
> of ==, >=, >, <, <=, etc.

I would be OK with an invocation that only ever installed and never
did an upgrade, as that would avoid the worst situations. Of course,
(a) AFAIK pip doesn't have such a method (other than --no-deps) at the
moment (dependencies are always upgraded if needed) and (b) there is
an ongoing proposal to change "pip install" to always upgrade, and
this is the exactly the opposite of what we'd need from the Python
prompt.

I understood Steve's proposal to be that we detect cases which do an
upgrade rather than a pure install, and in that case report "you
should restart your Python session". That's a common[1] pattern on
Windows, so should be familiar to Windows users, who I suspect are
more likely to hit this issue.

Paul

[1] Common, but not particularly loved - it's the source of the jokes
about "your mouse has moved, do you want to restart Windows for this
change to take effect?"
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Matthias Bussonnier]

On Tue, Sep 13, 2016 at 3:55 PM, Donald Stufft  wrote:

> Perhaps a better idea would be to add some smarts to the REPL (but not to 
> Python itself) that would detect something like:
>
 pip install
>
> And print a better error message that gives a better indication about what’s 
> gone wrong besides a SyntaxError?

I've that for IPython:

https://pypi.python.org/pypi/pip_magic

It need to be loaded explicitly, but I'm[1] considering making it
enabled by default.

We're more and more encountering users that are confused with
installing form within the shell; More and more users are in a Python
REPL[2] long before even knowing what a shell command line is and how
to get one. Worse they can choose environment in a dropdown, so having
to tell them they need to activate the env to install things make no
sens for them.

I think a 'pip install', or whatever else that could hook into the
thing to install package (like something that conda could set to
theirown thing) would be great. A message like "You might need to
restart your Python interpreter for changes to be taken into account"
seem enough to C.Y.A and would get us 90% there.

-- 
M

[1] I think other dev agree with me.
[2] in a notebook most of the time
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Steve Dower]

On 13Sep2016 1559, Matthew Brett wrote:

Perhaps a better idea would be to add some smarts to the REPL (but not to 
Python itself) that would detect something like:


pip install


And print a better error message that gives a better indication about what’s 
gone wrong besides a SyntaxError?


I was thinking the same thing.  If it could also catch accidental


python3 -m pip install


commands, that would be even better.


I feel like these would deserve text adventure style errors:

>>> pip install spam
I don't see a pip here

>>> python
You're already running a Python!

>>> pip uninstall eggs
You ask nicely, but pip refuses to uninstall from inside this shell.

Cheers,
Steve
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Steve Dower]

On 13Sep2016 1555, Donald Stufft wrote:



On Sep 13, 2016, at 6:41 PM, Steve Dower  wrote:

I think it's one of these things where we should suck it up and let the 90% 
case work fine, then display a big fat warning if anything weird may have 
happened and let users sort it out themselves.



I am unsure. One of the really egregious and hard to debug weirdness is going 
to be something like:



import foo.bar  # foo, and foo.bar are in sys.modules
pip.install(“thing”)  # This implicitly upgrades foo
import foo.widget  # the old foo is in sys.modules, but the new foo.widget.




Except my version of this goes more like:

>>> import foo.bar  # foo, and foo.bar are in sys.modules
>>> pip.install("thing")  # This implicitly upgrades foo
WARNING: Packages were updated. You need to restart Python now.
>>> import foo.widget  # because I happily ignore messages starting 
with WARNING


Doesn't matter to me whether it's "safe" to keep going or not - if any 
files are overwritten at all then show the warning. (This is exactly the 
argument I was expecting when I said "those who want it to be perfect in 
every scenario" ;) )


Cheers,
Steve

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Matthew Brett]

Hi,

On Tue, Sep 13, 2016 at 3:55 PM, Donald Stufft  wrote:
>
>> On Sep 13, 2016, at 6:41 PM, Steve Dower  wrote:
>>
>> I think it's one of these things where we should suck it up and let the 90% 
>> case work fine, then display a big fat warning if anything weird may have 
>> happened and let users sort it out themselves.
>
>
> I am unsure. One of the really egregious and hard to debug weirdness is going 
> to be something like:
>
>
 import foo.bar  # foo, and foo.bar are in sys.modules
 pip.install(“thing”)  # This implicitly upgrades foo
 import foo.widget  # the old foo is in sys.modules, but the new foo.widget.
>
> The 90% case works when it’s *only* pure python and there’s no 
> upgrading/downgrading involved, however you can’t control whether there are 
> going to be upgrades/downgrades or not when dependencies are in play because 
> of ==, >=, >, <, <=, etc.
>
> Another problem, particularly on Windows, is going to be stuff like:
>
 import someclibrary
 pip.install(“thing”)  # Implicitly upgrades someclibrary
>
> In this case, this would explode because Python will have the 
> someclibrary.dll locked and the upgrade will attempt to remove it.
>
> My fears here are that people are going to get really confused when they make 
> state modifications to their Python environment and they don’t see those 
> modifications reflected (because of sys.modules caching or what have you). 
> You can possibly get around that with reload(), but we already know the 
> problems that reload() has. Of course, that doesn’t stop Python from offering 
> reload(), but I think the difference is that reload() is targeted towards 
> people with weird edge cases who can understand the caveats while the 
> hypothetical ``pip.install()`` is targeted towards people who are beginners 
> who are still having problems figuring out the difference between the REPL 
> and their shell, much less the vagaries of Python’s global state and 
> attempting to modify that in a now running Python program.
>
> Perhaps a better idea would be to add some smarts to the REPL (but not to 
> Python itself) that would detect something like:
>
 pip install
>
> And print a better error message that gives a better indication about what’s 
> gone wrong besides a SyntaxError?

I was thinking the same thing.  If it could also catch accidental

>>> python3 -m pip install

commands, that would be even better.

Cheers,

Matthew
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Donald Stufft]

> On Sep 13, 2016, at 6:41 PM, Steve Dower  wrote:
> 
> I think it's one of these things where we should suck it up and let the 90% 
> case work fine, then display a big fat warning if anything weird may have 
> happened and let users sort it out themselves.


I am unsure. One of the really egregious and hard to debug weirdness is going 
to be something like:


>>> import foo.bar  # foo, and foo.bar are in sys.modules
>>> pip.install(“thing”)  # This implicitly upgrades foo
>>> import foo.widget  # the old foo is in sys.modules, but the new foo.widget.

The 90% case works when it’s *only* pure python and there’s no 
upgrading/downgrading involved, however you can’t control whether there are 
going to be upgrades/downgrades or not when dependencies are in play because of 
==, >=, >, <, <=, etc.

Another problem, particularly on Windows, is going to be stuff like:

>>> import someclibrary
>>> pip.install(“thing”)  # Implicitly upgrades someclibrary

In this case, this would explode because Python will have the someclibrary.dll 
locked and the upgrade will attempt to remove it.

My fears here are that people are going to get really confused when they make 
state modifications to their Python environment and they don’t see those 
modifications reflected (because of sys.modules caching or what have you). You 
can possibly get around that with reload(), but we already know the problems 
that reload() has. Of course, that doesn’t stop Python from offering reload(), 
but I think the difference is that reload() is targeted towards people with 
weird edge cases who can understand the caveats while the hypothetical 
``pip.install()`` is targeted towards people who are beginners who are still 
having problems figuring out the difference between the REPL and their shell, 
much less the vagaries of Python’s global state and attempting to modify that 
in a now running Python program.

Perhaps a better idea would be to add some smarts to the REPL (but not to 
Python itself) that would detect something like:

>>> pip install

And print a better error message that gives a better indication about what’s 
gone wrong besides a SyntaxError?


—
Donald Stufft



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Jeremy Stanley]

On 2016-09-13 23:00:25 +0100 (+0100), Paul Moore wrote:
[...]
> And things get significantly worse if we allow upgrades from the
> Python prompt rather than just installs (e.g., if you have already
> imported something from the old version and then upgrade).
[...]

If you need it, and of course only if your software is actually
written to handle it (contextually), there is a function to handle
that:

https://docs.python.org/2/library/functions.html#reload
https://docs.python.org/3/library/importlib.html#importlib.reload

I've used it in some very specific situations where you need to
change out parts of a running daemon without restarting the process,
but you really need a lot of extra care to get out of contexts
utilizing the modules being reloaded if you want the replacements to
actually be used.
-- 
Jeremy Stanley
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Steve Dower]

On 13Sep2016 1500, Paul Moore wrote:

On 13 September 2016 at 21:12, Thomas Kluyver  wrote:

One thing I'd quite like to see Python grow is a standard function to
install packages from inside Python.


That's not too hard in principle - pip.main(['install', package]) is
basically all you'd need (modulo various design details, and wrapping
it in a nice user friendly function). The bigger problems are:

1. pip is an independent module (albeit shipped with Python) and it's
not clear to what extent Python would want a builtin function that
depends on a non-stdlib module. But that's not a technical issue.
2. Python's module system isn't really designed around installing new
modules while a process is running. There's caching involved, so the
effects can be unpredictable. It's difficult to know whether the
benefits (avoiding confused users who tried to install from a Python
prompt) would outweigh the costs (confused users having installed a
module but not being able to see it because the module system didn't
notice things had changed).

I'm not honestly sure how big the "installing while a process is
running" issue would be - I did a few simple experiments and couldn't
immediately trigger weirdness, but I believe it can happen. And things
get significantly worse if we allow upgrades from the Python prompt
rather than just installs (e.g., if you have already imported
something from the old version and then upgrade).

Overall, it'd probably be a nice thing to have, but it's nowhere near
as simple as it seems at first glance.
Paul


I think it's one of these things where we should suck it up and let the 
90% case work fine, then display a big fat warning if anything weird may 
have happened and let users sort it out themselves.


A simple builtin that tries to run "sys.executable -m pip {args}" in a 
subprocess and displays "success", "success but you should probably 
restart Python now", "failed because you don't have pip", "failed and 
have all the output" is going to make lots of people happy, even if it 
upsets those who want it to be perfect in every scenario.


Cheers,
Steve

___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Leonardo Rochael Almeida]

On 13 September 2016 at 19:00, Paul Moore  wrote:

> [...]
>
> I'm not honestly sure how big the "installing while a process is
> running" issue would be - I did a few simple experiments and couldn't
> immediately trigger weirdness, but I believe it can happen. And things
> get significantly worse if we allow upgrades from the Python prompt
> rather than just installs (e.g., if you have already imported
> something from the old version and then upgrade).
>

The only weirdness on install I can imagine would happen is if the
installation creates `.pth` files, as those wouldn't be re-read by the
currently running Python.

The only package I remember having these was PIL, which can't be installed
by pip anyway. Pillow, the PIL fork, doesn't use .pth files.

(easy_install creates and manages .pth files, but if you're using
setuptools APIs to install packages from inside a running process then 1.
you already have a system to automatically make the packages available in
your namespace and 2. you know, or should know, what you're getting into).

Overall, it'd probably be a nice thing to have, but it's nowhere near
> as simple as it seems at first glance.
> Paul


Considering that installing a package and importing a package are two very
different things, I don't see why we couldn't handle the simple case of
installing the package for the first time with a nice API. Just document
the usual caveat emptors.

Cheers,

Leo
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Paul Moore]

On 13 September 2016 at 21:12, Thomas Kluyver  wrote:
> One thing I'd quite like to see Python grow is a standard function to
> install packages from inside Python.

That's not too hard in principle - pip.main(['install', package]) is
basically all you'd need (modulo various design details, and wrapping
it in a nice user friendly function). The bigger problems are:

1. pip is an independent module (albeit shipped with Python) and it's
not clear to what extent Python would want a builtin function that
depends on a non-stdlib module. But that's not a technical issue.
2. Python's module system isn't really designed around installing new
modules while a process is running. There's caching involved, so the
effects can be unpredictable. It's difficult to know whether the
benefits (avoiding confused users who tried to install from a Python
prompt) would outweigh the costs (confused users having installed a
module but not being able to see it because the module system didn't
notice things had changed).

I'm not honestly sure how big the "installing while a process is
running" issue would be - I did a few simple experiments and couldn't
immediately trigger weirdness, but I believe it can happen. And things
get significantly worse if we allow upgrades from the Python prompt
rather than just installs (e.g., if you have already imported
something from the old version and then upgrade).

Overall, it'd probably be a nice thing to have, but it's nowhere near
as simple as it seems at first glance.
Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Chris Barker]

>
> I think Ryan may have typed that command at a Python prompt rather than
>> a system command prompt. Unfortunately the distinction often isn't clear
>> in examples, because the experienced developers writing the instructions
>> are used to guessing which commands are Python and which are system
>> commands.
>>
>
indeed -- not a rare error for newbies.


> One thing I'd quite like to see Python grow is a standard function to
>> install packages from inside Python.
>
>
indeed:

import pip

pip.install('package_name')

would make lots of sense, except:

1) It is going to have to be a different command than the shell command, so
the above confusion would still be an issue.

2) are there any technicals reasons this would fail -- i.e. sys.path would
have to be re-calculated at least -- any other re-initializaton that would
have to be done?

3) permissions could be an issue, you don't necessarily want python to be
running with the permissions to write to it's install dir

In short -- I like the idea, but I'm not sure it'll really help the newbie
confusion.

-CHB








> In R, the install.packages()
>> function is the default way to get third party packages, and I think
>> staying in one interactive prompt does make things easier for new users
>> to understand.
>>
>
> Ah, I suspect you're right.
>
> I have considered adding a "Python x.y Shell" start menu item that would
> configure PATH properly for commands like "python" and "pip".
> Instinctively, it seems this would be more useful than a direct shortcut to
> the executable, but at the same time I don't want to start competing with
> all the other app-specific shells out there.
>
> Cheers,
> Steve
>
>
> ___
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Steve Dower]

On 13Sep2016 1312, Thomas Kluyver wrote:

On Tue, Sep 13, 2016, at 08:39 PM, Steve Dower wrote:

It would help if you could post the full error output (sanitizing
paths if needed). But you may just need to upgrade pip (python -m
install -U pip).


I think Ryan may have typed that command at a Python prompt rather than
a system command prompt. Unfortunately the distinction often isn't clear
in examples, because the experienced developers writing the instructions
are used to guessing which commands are Python and which are system
commands.

One thing I'd quite like to see Python grow is a standard function to
install packages from inside Python. In R, the install.packages()
function is the default way to get third party packages, and I think
staying in one interactive prompt does make things easier for new users
to understand.


Ah, I suspect you're right.

I have considered adding a "Python x.y Shell" start menu item that would 
configure PATH properly for commands like "python" and "pip". 
Instinctively, it seems this would be more useful than a direct shortcut 
to the executable, but at the same time I don't want to start competing 
with all the other app-specific shells out there.


Cheers,
Steve


___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Thomas Kluyver]

On Tue, Sep 13, 2016, at 08:39 PM, Steve Dower wrote:
> It would help if you could post the full error output (sanitizing
> paths if needed). But you may just need to upgrade pip (python -m
> install -U pip).

I think Ryan may have typed that command at a Python prompt rather than
a system command prompt. Unfortunately the distinction often isn't
clear in examples, because the experienced developers writing the
instructions are used to guessing which commands are Python and which
are system commands.

One thing I'd quite like to see Python grow is a standard function to
install packages from inside Python. In R, the install.packages()
function is the default way to get third party packages, and I think
staying in one interactive prompt does make things easier for new users
to understand.

Thomas
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Steve Dower]

It would help if you could post the full error output (sanitizing paths if 
needed). But you may just need to upgrade pip (python -m install -U pip).

Knowing exactly where the syntax error is coming from will help us figure out 
which package has the problem. There are at least three involved here.

Cheers,
Steve

Top-posted from my Windows Phone

-Original Message-
From: "Mills, Ryan" <ryan.mi...@texascapitalbank.com>
Sent: ‎9/‎13/‎2016 9:56
To: "distutils-sig@python.org" <distutils-sig@python.org>
Subject: [Distutils] Module Installation Issues

I just recently downloaded Python 3.5 and cannot seem to install any packages 
like Numpy, etc.  I have tried all the instructions on the website and keep 
getting errors:
 
For example, when I type “python –m pip install Numpy” it returns  a Syntax 
Error. I am completely new to Python so I must be missing something here – I 
haven’t altered any files since installing it the other day. Do I use the 
Python IDLE Shell? Are there other packages I need to install first?  Any help 
would be greatly appreciated.  
 
-Ryan
 
Ryan Mills
Quantitative Risk Analyst, Banking Officer

Capital Analytics & Stress Testing
2000 McKinney Avenue, Suite 700
Dallas, TX 75201
214.932.6653 direct
20.6653 internal
ryan.mi...@texascapitalbank.com



 
If you are not the addressee and have received this email in error, please 
notify me immediately. This email is confidential and may contain privileged or 
proprietary information that is unlawful for you to read, copy, distribute, 
disclose or otherwise use in any way.

 
 ___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

Re: [Distutils] Module Installation Issues

[Paul Moore]

On 13 September 2016 at 15:48, Mills, Ryan 
wrote:

> Do I use the Python IDLE Shell?


No, pip is a command line utility so you should go to the command prompt
and run "py -m pip install numpy" (I assume you're on Windows - on Unix
you'd need to run the appropriate python command).

Paul
___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig

[Distutils] Module Installation Issues

[Mills, Ryan]

I just recently downloaded Python 3.5 and cannot seem to install any packages 
like Numpy, etc.  I have tried all the instructions on the website and keep 
getting errors:

For example, when I type "python -m pip install Numpy" it returns  a Syntax 
Error. I am completely new to Python so I must be missing something here - I 
haven't altered any files since installing it the other day. Do I use the 
Python IDLE Shell? Are there other packages I need to install first?  Any help 
would be greatly appreciated.

-Ryan

Ryan Mills
Quantitative Risk Analyst, Banking Officer

[cid:image001.jpg@01CA3228.EF8AEBA0]

Capital Analytics & Stress Testing
2000 McKinney Avenue, Suite 700
Dallas, TX 75201
214.932.6653 direct
20.6653 internal
ryan.mi...@texascapitalbank.com

[cid:image001.jpg@01CA3228.EF8AEBA0]

[TCB_horiz_Log_rgb]
[cid:image004.jpg@01D20DA2.E1988D60]


If you are not the addressee and have received this email in error, please 
notify me immediately. This email is confidential and may contain privileged or 
proprietary information that is unlawful for you to read, copy, distribute, 
disclose or otherwise use in any way.



___
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig