Re: [Python-ideas] Syntax to import modules before running command from the command line

[Nick Coghlan]

On 10 January 2018 at 18:30, Paul Moore  wrote:
> On 10 January 2018 at 02:39, Nick Coghlan  wrote:
>> For the coverage.py use case, an environment-based solution is also
>> genuinely helpful, since you typically can't modify subprocess
>> invocations just because the software is being tested. At the moment,
>> there are approaches that rely on using either `sitecustomize` or
>> `*.pth` files, but being able to write `PYTHONRUNFIRST="import
>> coverage; coverage.process_startup()"` would be a fair bit clearer
>> about what was actually going on.
>
> It's worth remembering that Windows doesn't have the equivalent of the
> Unix "VAR=xxx prog arg arg" syntax for one-time setting of an
> environment variable, so environment variable based solutions are
> strictly less useful than command line arguments. That's one reason I
> prefer -C over PYTHONRUNFIRST.

The proposal is to offer both, not an either/or, but the idea isn't
that folks would need to set the environment variable directly - it's
that coverage.py itself (or a test runner) would set it so that
subprocesses were captured in addition to the directly executed
module.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax to import modules before running command from the command line

[Paul Moore]

On 10 January 2018 at 02:39, Nick Coghlan  wrote:
> For the coverage.py use case, an environment-based solution is also
> genuinely helpful, since you typically can't modify subprocess
> invocations just because the software is being tested. At the moment,
> there are approaches that rely on using either `sitecustomize` or
> `*.pth` files, but being able to write `PYTHONRUNFIRST="import
> coverage; coverage.process_startup()"` would be a fair bit clearer
> about what was actually going on.

It's worth remembering that Windows doesn't have the equivalent of the
Unix "VAR=xxx prog arg arg" syntax for one-time setting of an
environment variable, so environment variable based solutions are
strictly less useful than command line arguments. That's one reason I
prefer -C over PYTHONRUNFIRST.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax to import modules before running command from the command line

[Nick Coghlan]

On 10 January 2018 at 07:54, Barry Warsaw  wrote:
> Steve Barnes wrote:
>> Currently invoking `python -c "some;separated;set of commands;"` will,
>> if you need to use any library functions, require one or more import
>> somelib; sections in the execution string. This results in rather
>> complex "one liners".
>>
>> On the other hand `python -m somelib` will load somelib and attempt to
>> execute its `__main__()` or give an error if there isn't one.
>>
>> What I would like to suggest is a mechanism to pre-load libraries before
>> evaluating the -c option as this would allow the use of code from
>> libraries that don't have a `__main__` function, or those that do but it
>> doesn't do what you want.
>
> It would be really cool if you could somehow write a file with a bunch
> of commands in it, and then get Python to execute those commands.  Then
> it could still be a one line invocation, but you could do much more
> complex things, including import a bunch of modules before executing
> some code.

You jest, but doing that and then going on to process the rest of the
command line the same way the interpreter normally would is genuinely
tricky. Even `python -m runpy [other args]` doesn't emulate it
perfectly, and its responsible for implementing large chunks of the
regular behaviour :)

For the coverage.py use case, an environment-based solution is also
genuinely helpful, since you typically can't modify subprocess
invocations just because the software is being tested. At the moment,
there are approaches that rely on using either `sitecustomize` or
`*.pth` files, but being able to write `PYTHONRUNFIRST="import
coverage; coverage.process_startup()"` would be a fair bit clearer
about what was actually going on.

That example also shows why I'm wary of offering an import-only
version of this: I believe it would encourage folks to write modules
that have side effects on import, which is something we try to avoid
doing.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax to import modules before running command from the command line

[Ivan Pozdeev via Python-ideas]

On 10.01.2018 0:54, Barry Warsaw wrote:

Steve Barnes wrote:

Currently invoking `python -c "some;separated;set of commands;"` will,
if you need to use any library functions, require one or more import
somelib; sections in the execution string. This results in rather
complex "one liners".

On the other hand `python -m somelib` will load somelib and attempt to
execute its `__main__()` or give an error if there isn't one.

What I would like to suggest is a mechanism to pre-load libraries before
evaluating the -c option as this would allow the use of code from
libraries that don't have a `__main__` function, or those that do but it
doesn't do what you want.

It would be really cool if you could somehow write a file with a bunch
of commands in it, and then get Python to execute those commands.  Then
it could still be a one line invocation, but you could do much more
complex things, including import a bunch of modules before executing
some code.  I'm not sure what such a file would look like but here's a
strawman:

IPython's `run -i' does that.


```
import os
import sys
import somelib

path = somelib.get_path()
parent = os.path.dirname(path)
print(parent)

sys.exit(0 if os.path.isdir(parent) else 1)
```

Then you could run it like so:

$ python3 myscript.py

That seems like a nice, compact, one line invocation, but cI don't know,
it probably needs some fleshing out.  It's just a crazy idea, and
there's probably not enough time to implement this for Python 3.7.
Maybe for Python 3.8.

time-machine-winking-ly y'rs,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/


--
Regards,
Ivan

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax to import modules before running command from the command line

[Barry Warsaw]

Steve Barnes wrote:
> Currently invoking `python -c "some;separated;set of commands;"` will, 
> if you need to use any library functions, require one or more import 
> somelib; sections in the execution string. This results in rather 
> complex "one liners".
> 
> On the other hand `python -m somelib` will load somelib and attempt to 
> execute its `__main__()` or give an error if there isn't one.
> 
> What I would like to suggest is a mechanism to pre-load libraries before 
> evaluating the -c option as this would allow the use of code from 
> libraries that don't have a `__main__` function, or those that do but it 
> doesn't do what you want.

It would be really cool if you could somehow write a file with a bunch
of commands in it, and then get Python to execute those commands.  Then
it could still be a one line invocation, but you could do much more
complex things, including import a bunch of modules before executing
some code.  I'm not sure what such a file would look like but here's a
strawman:

```
import os
import sys
import somelib

path = somelib.get_path()
parent = os.path.dirname(path)
print(parent)

sys.exit(0 if os.path.isdir(parent) else 1)
```

Then you could run it like so:

$ python3 myscript.py

That seems like a nice, compact, one line invocation, but cI don't know,
it probably needs some fleshing out.  It's just a crazy idea, and
there's probably not enough time to implement this for Python 3.7.
Maybe for Python 3.8.

time-machine-winking-ly y'rs,
-Barry


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax to import modules before running command from the command line

[Paul Moore]

On 5 January 2018 at 08:12, Nick Coghlan  wrote:
> However, the issue then is that "python -M numpy" would just be a less
> flexible alternative to a command like "python -C 'import numpy as
> np'".

For quick one-liners don't underestimate the value of avoiding punctuation:

# No punctuation at all
python -M numpy
# Needs quotes - if quoted itself, needs nested quotes
python -C "import numpy"
# Needs quotes and semicolon
python -c "import numpy; ..."

This may not be a big deal on Unix shells (IMO, it's still an issue
there, just less critical) but on less-capable shells like Windows'
CMD, avoiding unnecessary punctuation can be a big improvement.

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax to import modules before running command from the command line

[Nick Coghlan]

On 5 January 2018 at 16:28, Steve Barnes  wrote:
> Currently invoking `python -c "some;separated;set of commands;"` will,
> if you need to use any library functions, require one or more import
> somelib; sections in the execution string. This results in rather
> complex "one liners".
>
> On the other hand `python -m somelib` will load somelib and attempt to
> execute its `__main__()` or give an error if there isn't one.

That's not quite how the -m switch works, but that doesn't affect your
core point: "-m somemodule" terminates the option list and defines
what will be run as `__main__`, so you can't compose it with other
options (like `-c`).

> What I would like to suggest is a mechanism to pre-load libraries before
> evaluating the -c option as this would allow the use of code from
> libraries that don't have a `__main__` function, or those that do but it
> doesn't do what you want.

This could be an interesting alternative to the old "python -C" idea
in https://bugs.python.org/issue14803 (which suggests adding an
alternative to "-c" that runs the given code *before* running
`__main__`, rather than running it *as* `__main__`).

To summarise some of the use cases from that discussion:

* setting `__main__.__requires__` to change how `import pkg_resources`
selects default package versions
* reducing the proliferation of `-X` options to enable modules like
tracemalloc and faulthandler
* providing a way to enable code coverage as early as possible on
startup (and have the setting be inherited by subprocesses)

The main downside I'd see compared to that `python -C` and
PYTHONRUNFIRST idea is that it might encourage the definition of
modules with side-effects in order to use it to reconfigure the
interpreter at startup.

That concern could potentially be mitigated if we did both:

* -M/PYTHONIMPORTFIRST: implicit imports in __main__ on startup
* -C/PYTHONRUNFIRST: code to run in __main__ on startup (after the
implicit imports)

However, the issue then is that "python -M numpy" would just be a less
flexible alternative to a command like "python -C 'import numpy as
np'".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax to import modules before running command from the command line

[Wes Turner]

An implicit print() would be convenient, too.

On Friday, January 5, 2018, Wes Turner  wrote:

> Could it just check if -c and -m are both set?
> That way there'd be no need for -p or -M.
>
> (I have an -m  switch in pyline which does exactly this. It makes
> copying and pasting less convenient; but does save having to type 'import
> module;’ for one liners)
>
> On Friday, January 5, 2018, Steve Barnes  wrote:
>
>> Currently invoking `python -c "some;separated;set of commands;"` will,
>> if you need to use any library functions, require one or more import
>> somelib; sections in the execution string. This results in rather
>> complex "one liners".
>>
>> On the other hand `python -m somelib` will load somelib and attempt to
>> execute its `__main__()` or give an error if there isn't one.
>>
>> What I would like to suggest is a mechanism to pre-load libraries before
>> evaluating the -c option as this would allow the use of code from
>> libraries that don't have a `__main__` function, or those that do but it
>> doesn't do what you want.
>>
>> Since -m for module is already taken I would suggest one of:
>> -p for pre-load module
>> -M for load module without attempting to execute `module.__main__()`
>> and without defining "__main__" in the load context or
>> -l for library
>> with the last two having the advantage of appearing next to -m in the
>> --help output.
>>
>> This would change, (for a trivial example):
>> `python -c"import numpy;print(numpy.pi);"`
>> to:
>> `python -M numpy -c"print(numpy.pi);"`
>>
>>
>> --
>> Steve (Gadget) Barnes
>> Any opinions in this message are my personal opinions and do not reflect
>> those of my employer.
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax to import modules before running command from the command line

[Wes Turner]

Could it just check if -c and -m are both set?
That way there'd be no need for -p or -M.

(I have an -m  switch in pyline which does exactly this. It makes
copying and pasting less convenient; but does save having to type 'import
module;’ for one liners)

On Friday, January 5, 2018, Steve Barnes  wrote:

> Currently invoking `python -c "some;separated;set of commands;"` will,
> if you need to use any library functions, require one or more import
> somelib; sections in the execution string. This results in rather
> complex "one liners".
>
> On the other hand `python -m somelib` will load somelib and attempt to
> execute its `__main__()` or give an error if there isn't one.
>
> What I would like to suggest is a mechanism to pre-load libraries before
> evaluating the -c option as this would allow the use of code from
> libraries that don't have a `__main__` function, or those that do but it
> doesn't do what you want.
>
> Since -m for module is already taken I would suggest one of:
> -p for pre-load module
> -M for load module without attempting to execute `module.__main__()`
> and without defining "__main__" in the load context or
> -l for library
> with the last two having the advantage of appearing next to -m in the
> --help output.
>
> This would change, (for a trivial example):
> `python -c"import numpy;print(numpy.pi);"`
> to:
> `python -M numpy -c"print(numpy.pi);"`
>
>
> --
> Steve (Gadget) Barnes
> Any opinions in this message are my personal opinions and do not reflect
> those of my employer.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Syntax to import modules before running command from the command line

[Steve Barnes]

Currently invoking `python -c "some;separated;set of commands;"` will, 
if you need to use any library functions, require one or more import 
somelib; sections in the execution string. This results in rather 
complex "one liners".

On the other hand `python -m somelib` will load somelib and attempt to 
execute its `__main__()` or give an error if there isn't one.

What I would like to suggest is a mechanism to pre-load libraries before 
evaluating the -c option as this would allow the use of code from 
libraries that don't have a `__main__` function, or those that do but it 
doesn't do what you want.

Since -m for module is already taken I would suggest one of:
-p for pre-load module
-M for load module without attempting to execute `module.__main__()` 
and without defining "__main__" in the load context or
-l for library
with the last two having the advantage of appearing next to -m in the 
--help output.

This would change, (for a trivial example):
`python -c"import numpy;print(numpy.pi);"`
to:
`python -M numpy -c"print(numpy.pi);"`


-- 
Steve (Gadget) Barnes
Any opinions in this message are my personal opinions and do not reflect 
those of my employer.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/