[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-14 Thread Jérôme Pouiller via Discussions on Python . org


Thank you Alyssa for this detailed answer.





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/12)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/fc66b3a45a581e65966e6c5028afe19c69f6e9460fd28d63d23954405207238b).


[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-12 Thread Alyssa Coghlan via Discussions on Python.org


[quote="Jérôme Pouiller, post:8, topic:67580, full:true, 
username:jerome-pouiller"]
hmm… so the idea is “a python package should never call a binary provided by 
another package”, that’s it? Yet, I believe it is a common pattern.
[/quote]

Not quite, as the behaviour you're describing is technically a bug (or at least 
a missing feature) in `west` rather than anything behaving unexpectedly in any 
of the packaging tools. Virtual environments only guarantee that `sys.path` 
will be adjusted - it is specifically venv activation that manipulates `PATH`.

For subcommand invocation like that to work reliably (without requiring an 
activated virtual environment), there are a few potential ways to do it:

1. If a dependency offers a Python API, invoke it that way rather than via its 
CLI
2. Invoke the dependency provided subcommands via `[sys.executable, -m, 
relevant_module_name]` rather than via their installer-generated CLI wrapper 
scripts. How viable this is depends on whether or not the dependencies involved 
have defined a `-m` compatible CLI invocation (there may not be a suitable 
module to execute that's equivalent to running the wrapper script)
3. Invoke the dependency subcommands via `[sys.executable, -c, "from 
entry_point_module import entry_point_target; entry_point_target()]` (these can 
be looked up in the project's metadata files or its package build 
configuration). This is the workaround for cases where there's no convenient 
`-m` compatible invocation published by the dependency.
4. Attempt to locate the wrapper scripts relative to `sys.path` entries, invoke 
it via `[sys.executable, path_to_wrapper_script]` once found. Not commonly 
used, since one of the above options will usually be easier.
5. Use `sysconfig` to manipulate `PATH` in the current process to ensure the 
environment's wrapper script folder is present. This is less reliable than the 
above options, since it still involves making potentially invalid assumptions 
about the expected relationship between `sys.path` and `PATH` configuration. 
It's fairly straightforward to make it work for the typical "everything in one 
venv" case, though.

The first three options are the most reliable, but the last two do turn up 
sometimes (especially in code old enough to predate the existence of the `-m` 
switch).





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/11)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/c75bf90784afd80f90d880ed107c933ba622f4ccd34d444e00da8e976dce).


[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-11 Thread Clint Hepner via Discussions on Python.org


No, the idea is that you write the Python code with enough information to 
*identify* the external binary, given a suitable execution environment. 
Ideally, you are going to execute the command by name alone, not an absolute or 
relative path, and configure the (virtual) environment so that the command is 
found in a directory on your `PATH`. You seem to have done that in your first 
two examples, but not in the third.





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/10)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/35f328781814ffaad6c602910fd8ddc11a7a2ce58248f214254363f0b5801bc5).


[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-11 Thread James Parrott via Discussions on Python.org


"Should" is a strong word.  Why not simply import the package and call the 
entry point directly?





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/9)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/feda3e8602b5d5b33cc61f01773c831dd8800e21902146feac5bb673ef42f3f3).


[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-11 Thread Jérôme Pouiller via Discussions on Python . org


hmm... so the idea is "a python package should never call a binary provided by 
another package", that's it? Yet, I believe it is a common pattern.





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/8)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/a8d6fb621757238213deaff5b8b1ba672a696db17dd7c3231c605eefb73ff389).


[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-11 Thread Clint Hepner via Discussions on Python.org


[quote="Jérôme Pouiller, post:5, topic:67580, full:true, 
username:jerome-pouiller"]
But, the [documentation 
says](https://docs.python.org/3/library/venv.html#how-venvs-work):

> Furthermore, all scripts installed in the environment should be runnable 
> without activating it.

My understanding is `/home/jerome/.local/pipx/venvs/west/bin/west` or `sh -c 
'source /home/jerome/.local/pipx/venvs/west/bin/activate; west'` should give 
the same result.
[/quote]

That only applies to commands run directly from the shell.





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/7)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/9df0854018af9fa5887b6725b161ccc4861eab382867cf230610bf151061934e).


[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-11 Thread James Parrott via Discussions on Python.org


It's expected behaviour.

To be able to run a command line tool globally, it could be installed with 
pipx.  Pipx'll put it in its own venv, and add the tool's executable to the 
global path.





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/6)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/5e92f290d7564fe143ba665a7116c9d8aeb9d2f537c691d3cd2d1bd7c0509c19).


[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-11 Thread Jérôme Pouiller via Discussions on Python . org


But, the [documentation says][1]:

> Furthermore, all scripts installed in the environment should be runnable 
> without activating it.

My understanding is `/home/jerome/.local/pipx/venvs/west/bin/west` or `sh -c 
'source /home/jerome/.local/pipx/venvs/west/bin/activate; west'` should give 
the same result.

[1]: https://docs.python.org/3/library/venv.html#how-venvs-work





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/5)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/ecae2372a4edb2e80459b2913c4bedbdd1c875eebf8600fc228fbcfb5c21e776).


[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-11 Thread Clint Hepner via Discussions on Python.org


Still, it boils down to: whatever command `west` executes is either not found 
outside your virtual environment, or a different version exists outside it. 
Path lookup is not exclusive to the shell; `subprocess.run` *also* uses path 
lookup to find `command-provided-by-a-required-package`.





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/4)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/8e095f1f5bf8a5f824a6b98e1799c6a2aa990ffe48c48e44dba1a42603027c21).


[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-11 Thread Jérôme Pouiller via Discussions on Python . org


I forgot to mention:

$ which west
/home/jerome/.local/bin/west
$ ls -l /home/jerome/.local/bin/west
lrwxrwxrwx 1 jerome jerome 44 Feb 22  2024 /home/jerome/.local/bin/west -> 
/home/jerome/.local/pipx/venvs/west/bin/west*

The issue is not about finding the `west` command by itself, but finding the 
commands run by `west`.





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/3)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/8f63e8331331d80adb1f8eefd9538e1324339ac2b44c233a3905afadec3a4419).


[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-11 Thread Clint Hepner via Discussions on Python.org


Your first two examples are equivalent; activating the virtual environment is 
done primarily to modify `PATH` in exactly the way you do so manually in your 
second example.

What happens when you (try to) execute `west` without modifying your `PATH` 
variable depends on whether a command named `west` is even found, or how the 
"other" `west` command compares to the one in your virtual environment.





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/2)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/d20384154a69698ce82133d879096a35e6112fbaec438235ff1a3b116a6cde75).


[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't

2024-10-11 Thread Jérôme Pouiller via Discussions on Python . org


I spend a bunch of time to understand why one of my python command (`west`) 
didn't work as expected. Finally, I found it was because I installed my package 
in a `venv` and `west` it executed commands provided by required packages. eg:

import subprocess
subprocess.run(["command-provided-by-a-required-package"])

So, the snippet below worked:

$ source /home/jerome/.local/pipx/venvs/west/bin/activate
(west) $ west
...  ...

This one also worked:

$ export PATH=$PATH:/home/jerome/.local/pipx/venvs/west/bin
$ west
...  ...

But running `west` without changing the `$PATH` didn't work:

$ west
...  ...

(note since I have no luck, the error message was unrelated to `$PATH`)

I thought that running a binary from a `venv` had exactly the same behavior 
than activating the `venv` and then running the binary. The documentation seems 
to say the same. Do you think this behavior is a bug or is expected? If it is 
expected, what change should I suggest to `west` to properly run with `venv`?





---
[Visit 
Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/1)
 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click 
here](https://discuss.python.org/email/unsubscribe/4ab79d6dc03448ad9d8a4eb449589328c2341222c2431963200d0321645cf726).