Re: venv and executing other python programs
Op 15/02/2022 om 8:21 schreef Reto: On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote: > How to people here deal with that? Don't activate the venv for those programs then? The point of a venv is that you only enter it when you actually want that specific python stack. Get yourself a terminal that can either multiplex, or add something like tmux or screen to the mix if you frequently need other python tools during development. Or just install those in you venv, after all if you do use them for dev they are part of your dependencies, so declare them as such. Still feels like a problem to me though. Suppose you're working on a program which, for example, prints json to stdout. And suppose you want to use a tool like jq (let's call it pjq as an example) to process that output, by piping the output of your program to it: python your-program.py | pjq That seems to me a very sensible thing to do. If pjq is written in C, C++, Rust, Ruby, PHP, Go, or whatever, itis not a problem at all. To the user of pjq, it shouldn't matter at all in what language it's written, and mostly it doesn't matter at all indeed. But if pjq happens to be written in Python, it suddenly matters very much: it won't work in your venv (unless your venv happens to have the right modules). There are ways around it of course: - install the tool in your venv, like you say - from outside the venv: venv/bin/python your_program.py | pjq - from inside the venv: python your_program.py | interpreter> Or something like that. At least I think those should work, I haven't tested it. But it feels to wrong to have to use such workarounds simply because pjq happens to be written in the same language as you're writing your program in. -- "A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools." -- Douglas Adams -- https://mail.python.org/mailman/listinfo/python-list
Re: venv and executing other python programs
On Tue, 15 Feb 2022 at 21:19, Roel Schroeven wrote: > > Op 15/02/2022 om 8:21 schreef Reto: > > On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote: > > > How to people here deal with that? > > > > Don't activate the venv for those programs then? > > The point of a venv is that you only enter it when you actually want > > that specific python stack. > > > > Get yourself a terminal that can either multiplex, or add something like > > tmux or screen to the mix if you frequently need other python tools > > during development. > > Or just install those in you venv, after all if you > > do use them for dev they are part of your dependencies, so declare them > > as such. > Still feels like a problem to me though. > > Suppose you're working on a program which, for example, prints json to > stdout. And suppose you want to use a tool like jq (let's call it pjq as > an example) to process that output, by piping the output of your program > to it: > > python your-program.py | pjq > > That seems to me a very sensible thing to do. If pjq is written in C, > C++, Rust, Ruby, PHP, Go, or whatever, itis not a problem at all. To the > user of pjq, it shouldn't matter at all in what language it's written, > and mostly it doesn't matter at all indeed. > > But if pjq happens to be written in Python, it suddenly matters very > much: it won't work in your venv (unless your venv happens to have the > right modules). > > There are ways around it of course: > - install the tool in your venv, like you say > - from outside the venv: venv/bin/python your_program.py | pjq > - from inside the venv: python your_program.py | interpreter> > Or something like that. At least I think those should work, I haven't > tested it. > > But it feels to wrong to have to use such workarounds simply because pjq > happens to be written in the same language as you're writing your > program in. > The pjq shebang shouldn't specify /usr/bin/env unless you want it to run in whatever Python is the current default. In other words, if it matters, be more specific. That usually means either using /usr/bin/python3 for your system Python, or the interpreter out of the venv itself - /path/to/venv/bin/python3 - to specify that. You're absolutely right, workarounds like that are wrong. But the wrongness comes from using env when you don't want env :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: venv and executing other python programs
I did a few experiments in my machine. I created the following foo.py
import pandas
print("foo")
Now "pandas" is installed under Python 3 outside the venv. I can run it
successfully calling "python3 foo.py".
If I add the shebang "#!/usr/bin/env python3" (notice the 3), I can also
run it as "./foo.py".
Calling it as "python foo.py" or using the shebang "#!/usr/bin/env
python" does not work and it makes sense since "pandas" is installed
only for python 3.
New I create a virtual env with "python3 -m venv xxx" and activate it.
Once inside I can run foo.py in 4 different ways:
- python foo.py
- python3 foo.py
- ./foo.py (using the shebang "#!/usr/bin/env python")
- ./foo.py (using the shebang "#!/usr/bin/env python3")
Now all of that was with "pandas" installed outside of venv but not
inside.
I did the same experiments with another library, "cryptonita" which it
is not installed outside but inside and I was able to executed in
4 different ways too (inside the venv of course):
- python foo.py
- python3 foo.py
- ./foo.py (using the shebang "#!/usr/bin/env python")
- ./foo.py (using the shebang "#!/usr/bin/env python3")
Do you have a particular context where you are having troubles? May be
there is something else going on...
Thanks,
Martin.
On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote:
Hi,
I have recently started using venv for my hobby-programming. There
is an annoying problem. Since venv modifies $PATH, python programs
that use the "#!/usr/bin/env python" variant of the hashbang often
fail since their additional modules aren't install inside in venv.
How to people here deal with that?
Please note: I'm not interested in discussing whether the
env-variant is good or bad. ;-) It's not that *I* use it, but
several progs in /usr/bin/.
Thanks for your time.
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Re: Saving/exporting plots from Jupyter-labs?
Den 2022-02-15 skrev Reto : > On Mon, Feb 14, 2022 at 08:54:01PM +, Martin Schöön wrote: >> 1) In notebooks I can save a plot by right-clicking on it and do >> save image as. In Jupyter-lab that does not work and so far I >> have not been able to figure out how to do it. Yes, I have looked >> in the documentation. > > Shift + right click brings up the usual browser menu Thanks, /Martin -- https://mail.python.org/mailman/listinfo/python-list
ERROR IN DOWNLOADING PYTHON
Dear Sir/Mam I have downloaded the latest 3.10.2 version of python for my 64×64 bit laptop but I'm unable to work on it as my system is showing that api-ms-win-crt-runtime-l1-1-0-dll is missing but it's not the case as i have also downloaded this file. Please let me know what should be my immediate step. Looking forward to an early response from you Yours Sincerely Anindita Das -- https://mail.python.org/mailman/listinfo/python-list
Re: ERROR IN DOWNLOADING PYTHON
On 2/15/22, 11_Anindita Das_BCA wrote: > > I have downloaded the latest 3.10.2 version of python for my 64×64 bit > laptop but I'm unable to work on it as my system is showing that > api-ms-win-crt-runtime-l1-1-0-dll is missing but it's not the case as i > have also downloaded this file. I guess that you're using Windows 7 since Windows 10+ includes the Universal C Runtime (ucrt) as a required system component, for which "api-ms-win-crt-runtime-l1-1-0-dll" is a virtual API set (mapped to ucrtbase.dll), which can never be missing, and since Windows 8.1 is relatively uncommon (2-3% of installations). Python 3.10 requires Windows 8.1 and above. If you're using Windows 7, you should be able to install and use Python 3.8.10: https://www.python.org/ftp/python/3.8.10/python-3.8.10-amd64.exe If you need to install ucrt, here's the link for the update that was released in 2016: https://support.microsoft.com/kb/3118401 -- https://mail.python.org/mailman/listinfo/python-list
Re: venv and executing other python programs
Am 15.02.2022 um 08:53 schrieb Barry Scott:
> Or are you running the program from the command line after activating the
> venv?
This ...
Am 15.02.2022 um 11:18 schrieb Roel Schroeven:
> Suppose you're working on a program which, for example, prints json
> to stdout. And suppose you want to use a tool like jq (let's call it
> pjq as an example) to process that output, by piping the output of
> your program to it:
... and this.
Am 15.02.2022 um 08:21 schrieb Reto:
> Don't activate the venv for those programs then?
> The point of a venv is that you only enter it when you actually want
> that specific python stack.
Well, it's not that I activate the venv *for* those troubling
programs. I activate it to work on the particular project. ;-)
> Get yourself a terminal that can either multiplex, or add something like
> tmux or screen to the mix if you frequently need other python tools
> during development.
I already do that (terminator, since I don't get tmux to behave as I
want it).
Am 15.02.2022 um 13:36 schrieb Martin Di Paola:
> I did a few experiments in my machine. I created the following foo.py
[SNIP]
> Do you have a particular context where you are having troubles? May
> be there is something else going on...
It seems to me that on your system, venv includes the site-packages
by default. For that I've found a solution: Create the venv with
--system-site-packages or in the venv's pyvenv.cfg just set/add
"include-system-site-packages = true"
Without this, I get the following (Python 3.6.9, Linux Mint 19,
Pandas installed in /usr/lib/python3/dist-packages/pandas):
[2, 0] mirko@wizbox:~$ vim /home/mirko/bin/foo.py
[2, 0] mirko@wizbox:~$ chmod +x /home/mirko/bin/foo.py
[2, 0] mirko@wizbox:~$ cat /home/mirko/bin/foo.py
#!/usr/bin/env python3
import pandas
print("Works")
[2, 0] mirko@wizbox:~$ foo.py
Works
[2, 0] mirko@wizbox:~$ python3 -m venv xxx
[2, 0] mirko@wizbox:~$ source xxx/bin/activate
(xxx) [2, 0] mirko@wizbox:~$ foo.py
Traceback (most recent call last):
File "/home/mirko/bin/foo.py", line 3, in
import pandas
ModuleNotFoundError: No module named 'pandas'
(xxx) [2, 1] mirko@wizbox:~$ deactivate
[2, 0] mirko@wizbox:~$ foo.py
Works
So, the problem seems to be solved. Thanks all! :-)
--
https://mail.python.org/mailman/listinfo/python-list
Re: venv and executing other python programs
> On 15 Feb 2022, at 14:30, Mirko via Python-list > wrote: > > Well, it's not that I activate the venv *for* those troubling > programs. I activate it to work on the particular project. ;-) It is not necessary to activate the venv to use the python in it. So your solution is to not do the activate. You can run venv/bin/python without the activation. Barry -- https://mail.python.org/mailman/listinfo/python-list
Re: venv and executing other python programs
> On 15 Feb 2022, at 12:36, Martin Di Paola wrote:
>
> I did a few experiments in my machine. I created the following foo.py
>
> import pandas
> print("foo")
>
> Now "pandas" is installed under Python 3 outside the venv. I can run it
> successfully calling "python3 foo.py".
>
> If I add the shebang "#!/usr/bin/env python3" (notice the 3), I can also run
> it as "./foo.py".
>
> Calling it as "python foo.py" or using the shebang "#!/usr/bin/env python"
> does not work and it makes sense since "pandas" is installed
> only for python 3.
>
> New I create a virtual env with "python3 -m venv xxx" and activate it.
>
> Once inside I can run foo.py in 4 different ways:
>
> - python foo.py
> - python3 foo.py
> - ./foo.py (using the shebang "#!/usr/bin/env python")
> - ./foo.py (using the shebang "#!/usr/bin/env python3")
>
> Now all of that was with "pandas" installed outside of venv but not inside.
>
> I did the same experiments with another library, "cryptonita" which it is not
> installed outside but inside and I was able to executed in 4 different ways
> too (inside the venv of course):
>
> - python foo.py
> - python3 foo.py
> - ./foo.py (using the shebang "#!/usr/bin/env python")
> - ./foo.py (using the shebang "#!/usr/bin/env python3")
If you have activated the venv then any script that uses /usr/bin/env will use
executables from the venv
bin folder.
I'm seeing the following in the venv I made on my mac:
% ls venv.tmp/bin
Activate.ps1dmgbuild* modulegraph*pyi-archive_viewer*
pyinstaller*python3.10@
activateds_store* pip*pyi-bindepend*
pylupdate5* pyuic5*
activate.cshmacho_dump* pip3* pyi-grab_version*
pyrcc5* wheel*
activate.fish macho_find* pip3.10*pyi-makespec*
python@
colour-print* macho_standalone* py2applet* pyi-set_version*
python3@
You can see that "python", "python3" and "python3.10" are present.
So it does not help to change the /usr/bin/env to use python3.
Indeed its a feature that if you use /usr/bin/env you obviously want to use the
executable from the activated venv.
I avoid all these issues by not activating the venv. Python has code to know
how to use the venv libraries that are installed in it when invoked. It does
not
depend on the activate script being run.
Barry
>
> Do you have a particular context where you are having troubles? May be there
> is something else going on...
>
> Thanks,
> Martin.
>
> On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote:
>> Hi,
>>
>> I have recently started using venv for my hobby-programming. There
>> is an annoying problem. Since venv modifies $PATH, python programs
>> that use the "#!/usr/bin/env python" variant of the hashbang often
>> fail since their additional modules aren't install inside in venv.
>>
>> How to people here deal with that?
>>
>> Please note: I'm not interested in discussing whether the
>> env-variant is good or bad. ;-) It's not that *I* use it, but
>> several progs in /usr/bin/.
>>
>> Thanks for your time.
>> --
>> https://mail.python.org/mailman/listinfo/python-list
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: venv and executing other python programs
On 2022-02-15 06:35:18 +0100, Mirko via Python-list wrote: > I have recently started using venv for my hobby-programming. There > is an annoying problem. Since venv modifies $PATH, python programs > that use the "#!/usr/bin/env python" variant of the hashbang often > fail since their additional modules aren't install inside in venv. > > How to people here deal with that? Use the hashbang line to couple your scripts to their venvs. For example, here's the start of one of my scripts: #! /usr/local/usradm/venv/bin/python3 import json import os ... import psycopg2 import psycopg2.extras import cx_Oracle ... When this script is run, the python interpreter will inspect it's own pathname and pull in the venv from /usr/local/usradm/venv automatically, regardless of the environment it is started in. I could run this from cron (which has only /usr/bin and /bin in its PATH) or from my normal command line or from a different venv. You could use a different venv for every script, but that's pretty wasteful, so you probably want to organize your venvs so that several scripts can share their environment. For example by project or product (as in my example - "usradm" is a our in-house user administration system) or by capabilities (for example you might want one venv with pandas and all your favourite graph libraries for all your analytics scripts and another with Qt5 for some gui tools). > Please note: I'm not interested in discussing whether the > env-variant is good or bad. ;-) It's not that *I* use it, but > several progs in /usr/bin/. Progs in /usr/bin should IMHO never use /usr/bin/env. They should always use #!/usr/bin/python3. You might want to report that as a bug to your distribution. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: venv and executing other python programs
If you have activated the venv then any script that uses /usr/bin/env will use executables from the venv bin folder. That's correct. I tried to be systematic in the analysis so I tested all the possibilities. I avoid all these issues by not activating the venv. Python has code to know how to use the venv libraries that are installed in it when invoked. It does not depend on the activate script being run. I had to test this myself because I didn't believe it but you are right. Without having the venv activated, if the shebang explicitly points to the python executable of the venv, the program will have access to the libs installed in the environment. The same if I do: /home/user/venv/bin/python foo.py Thanks for the info! Barry Do you have a particular context where you are having troubles? May be there is something else going on... Thanks, Martin. On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote: Hi, I have recently started using venv for my hobby-programming. There is an annoying problem. Since venv modifies $PATH, python programs that use the "#!/usr/bin/env python" variant of the hashbang often fail since their additional modules aren't install inside in venv. How to people here deal with that? Please note: I'm not interested in discussing whether the env-variant is good or bad. ;-) It's not that *I* use it, but several progs in /usr/bin/. Thanks for your time. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: venv and executing other python programs
On 2/15/22, Martin Di Paola wrote: > > That's correct. I tried to be systematic in the analysis so I tested all > the possibilities. Your test results were unexpected for `python3 -m venv xxx`. By default, virtual environments exclude the system and user site packages. Including them should require the command-line argument `--system-site-packages`. I'd check sys.path in the environment. Maybe you have PYTHONPATH set. > I had to test this myself because I didn't believe it but you are right. > Without having the venv activated, if the shebang explicitly points to > the python executable of the venv, the program will have access to the > libs installed in the environment. A virtual environment is configured by a "pyvenv.cfg" file that's either beside the executable or one directory up. Activating an environment is a convenience, not a requirement. -- https://mail.python.org/mailman/listinfo/python-list
