[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
Thank you Alyssa for this detailed answer. --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/12) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/fc66b3a45a581e65966e6c5028afe19c69f6e9460fd28d63d23954405207238b).
[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
[quote="Jérôme Pouiller, post:8, topic:67580, full:true, username:jerome-pouiller"] hmm… so the idea is “a python package should never call a binary provided by another package”, that’s it? Yet, I believe it is a common pattern. [/quote] Not quite, as the behaviour you're describing is technically a bug (or at least a missing feature) in `west` rather than anything behaving unexpectedly in any of the packaging tools. Virtual environments only guarantee that `sys.path` will be adjusted - it is specifically venv activation that manipulates `PATH`. For subcommand invocation like that to work reliably (without requiring an activated virtual environment), there are a few potential ways to do it: 1. If a dependency offers a Python API, invoke it that way rather than via its CLI 2. Invoke the dependency provided subcommands via `[sys.executable, -m, relevant_module_name]` rather than via their installer-generated CLI wrapper scripts. How viable this is depends on whether or not the dependencies involved have defined a `-m` compatible CLI invocation (there may not be a suitable module to execute that's equivalent to running the wrapper script) 3. Invoke the dependency subcommands via `[sys.executable, -c, "from entry_point_module import entry_point_target; entry_point_target()]` (these can be looked up in the project's metadata files or its package build configuration). This is the workaround for cases where there's no convenient `-m` compatible invocation published by the dependency. 4. Attempt to locate the wrapper scripts relative to `sys.path` entries, invoke it via `[sys.executable, path_to_wrapper_script]` once found. Not commonly used, since one of the above options will usually be easier. 5. Use `sysconfig` to manipulate `PATH` in the current process to ensure the environment's wrapper script folder is present. This is less reliable than the above options, since it still involves making potentially invalid assumptions about the expected relationship between `sys.path` and `PATH` configuration. It's fairly straightforward to make it work for the typical "everything in one venv" case, though. The first three options are the most reliable, but the last two do turn up sometimes (especially in code old enough to predate the existence of the `-m` switch). --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/11) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/c75bf90784afd80f90d880ed107c933ba622f4ccd34d444e00da8e976dce).
[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
No, the idea is that you write the Python code with enough information to *identify* the external binary, given a suitable execution environment. Ideally, you are going to execute the command by name alone, not an absolute or relative path, and configure the (virtual) environment so that the command is found in a directory on your `PATH`. You seem to have done that in your first two examples, but not in the third. --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/10) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/35f328781814ffaad6c602910fd8ddc11a7a2ce58248f214254363f0b5801bc5).
[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
"Should" is a strong word. Why not simply import the package and call the entry point directly? --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/9) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/feda3e8602b5d5b33cc61f01773c831dd8800e21902146feac5bb673ef42f3f3).
[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
hmm... so the idea is "a python package should never call a binary provided by another package", that's it? Yet, I believe it is a common pattern. --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/8) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/a8d6fb621757238213deaff5b8b1ba672a696db17dd7c3231c605eefb73ff389).
[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
[quote="Jérôme Pouiller, post:5, topic:67580, full:true, username:jerome-pouiller"] But, the [documentation says](https://docs.python.org/3/library/venv.html#how-venvs-work): > Furthermore, all scripts installed in the environment should be runnable > without activating it. My understanding is `/home/jerome/.local/pipx/venvs/west/bin/west` or `sh -c 'source /home/jerome/.local/pipx/venvs/west/bin/activate; west'` should give the same result. [/quote] That only applies to commands run directly from the shell. --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/7) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/9df0854018af9fa5887b6725b161ccc4861eab382867cf230610bf151061934e).
[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
It's expected behaviour. To be able to run a command line tool globally, it could be installed with pipx. Pipx'll put it in its own venv, and add the tool's executable to the global path. --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/6) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/5e92f290d7564fe143ba665a7116c9d8aeb9d2f537c691d3cd2d1bd7c0509c19).
[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
But, the [documentation says][1]: > Furthermore, all scripts installed in the environment should be runnable > without activating it. My understanding is `/home/jerome/.local/pipx/venvs/west/bin/west` or `sh -c 'source /home/jerome/.local/pipx/venvs/west/bin/activate; west'` should give the same result. [1]: https://docs.python.org/3/library/venv.html#how-venvs-work --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/5) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/ecae2372a4edb2e80459b2913c4bedbdd1c875eebf8600fc228fbcfb5c21e776).
[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
Still, it boils down to: whatever command `west` executes is either not found outside your virtual environment, or a different version exists outside it. Path lookup is not exclusive to the shell; `subprocess.run` *also* uses path lookup to find `command-provided-by-a-required-package`. --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/4) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/8e095f1f5bf8a5f824a6b98e1799c6a2aa990ffe48c48e44dba1a42603027c21).
[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
I forgot to mention: $ which west /home/jerome/.local/bin/west $ ls -l /home/jerome/.local/bin/west lrwxrwxrwx 1 jerome jerome 44 Feb 22 2024 /home/jerome/.local/bin/west -> /home/jerome/.local/pipx/venvs/west/bin/west* The issue is not about finding the `west` command by itself, but finding the commands run by `west`. --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/3) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/8f63e8331331d80adb1f8eefd9538e1324339ac2b44c233a3905afadec3a4419).
[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
Your first two examples are equivalent; activating the virtual environment is done primarily to modify `PATH` in exactly the way you do so manually in your second example. What happens when you (try to) execute `west` without modifying your `PATH` variable depends on whether a command named `west` is even found, or how the "other" `west` command compares to the one in your virtual environment. --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/2) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/d20384154a69698ce82133d879096a35e6112fbaec438235ff1a3b116a6cde75).
[Py] [Python Help] Venv: `activate` script changes `$PATH` while executing the binary doesn't
I spend a bunch of time to understand why one of my python command (`west`) didn't work as expected. Finally, I found it was because I installed my package in a `venv` and `west` it executed commands provided by required packages. eg: import subprocess subprocess.run(["command-provided-by-a-required-package"]) So, the snippet below worked: $ source /home/jerome/.local/pipx/venvs/west/bin/activate (west) $ west ... ... This one also worked: $ export PATH=$PATH:/home/jerome/.local/pipx/venvs/west/bin $ west ... ... But running `west` without changing the `$PATH` didn't work: $ west ... ... (note since I have no luck, the error message was unrelated to `$PATH`) I thought that running a binary from a `venv` had exactly the same behavior than activating the `venv` and then running the binary. The documentation seems to say the same. Do you think this behavior is a bug or is expected? If it is expected, what change should I suggest to `west` to properly run with `venv`? --- [Visit Topic](https://discuss.python.org/t/venv-activate-script-changes-path-while-executing-the-binary-doesnt/67580/1) or reply to this email to respond. You are receiving this because you enabled mailing list mode. To unsubscribe from these emails, [click here](https://discuss.python.org/email/unsubscribe/4ab79d6dc03448ad9d8a4eb449589328c2341222c2431963200d0321645cf726).