Re: Running a subprocess in a venv
On 10/21/23 07:01, Larry Martell via Python-list wrote: I have a python script, and from that I want to run another script in a subprocess in a venv. What is the best way to do that? I could write a file that activates the venv then runs the script, then run that file, but that seems messy. Is there a better way? You don't need to "activate" a virtualenv. The activation script does some helpful things along the way (setup and cleanup) but none of them are required. The most important thing it does is basically: VIRTUAL_ENV='path-where-you-put-the-virtualenv' export VIRTUAL_ENV _OLD_VIRTUAL_PATH="$PATH" PATH="$VIRTUAL_ENV/bin:$PATH" export PATH and that's really only so that commands that belong to that virtualenv (python, pip, and things where you installed a package in the venv wich creates an "executable" in bin/) are in a directory first in your search path. As long as you deal with necessary paths yourself, you're fine without activating. So as mentioned elsewhere, just use the path to the virtualenv's Python and you're good to go. -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple webserver
On Sun, 22 Oct 2023 at 04:13, Janis Papanagnou via Python-list wrote: > I have a couple decades experience with about a dozen programming > languages (not counting assemblers). Asynchronous processing, IPC, > multi-processing, client/server architectures, multi-threading, > semaphores, etc. etc. are concepts that are not new to me. Oh, sweet, sweet, then you should be fine with the library I suggested. It's certainly served me well (and I have similar experience, having learned networking mainly on OS/2 in the 1990s). > My expectation would be that any sophistically designed socket/ > web-socket library would not impose any risk. And the intended > server by itself has only very limited requirements; listening to > incoming request, storing some client information, broadcasting > to the attached clients. Basically just (informally written): > > init server > forever: > wait for request(s) -> queue > handle requests from queue (sequentially): > store specific information from new registered clients > broadcast some information to all registered clients > > It seems to me that multi-threading or async I/O aren't necessary. Technically that's true, but "wait for request(s)" has to handle (a) new incoming sockets, (b) messages from currently-connected sockets, and possibly (c) sockets now being writable when previously they blocked. So you have most of the work of async I/O. Since the library's been built specifically for asyncio, that's the easiest. > I'd like to ask; where do you see the specific risks with Python > (as language per se) and it's (web-socket-)libraries here? > > If the web-socket IPC is well supported the algorithmic parts in > Python seem trivial to learn and implement. - Or am I missing > something? Pretty trivial, yeah. You shouldn't have too much trouble here I expect. > (A brief search gave me the impression that for JS communication > web-sockets would be the method to use. Otherwise I'd just use > basic Unix domain sockets for the purpose and write it, say, in > C or C++ that I already know. But I don't know whether (or how) > plain sockets are used from JS running in a browser. Here I'm > lacking experience. And that lead me to have a look at Python, > since the web-sockets/server examples that I found looked simple.) Yes, that's correct. You can't use plain sockets from inside a web browser, mainly because they offer way way too much flexibility (JS code is untrusted and is now running on your computer, do you really want that to be able to telnet to anything on your LAN?). So websockets are the way to go. There are other similar technologies, but for this sort of "broadcast to connected clients" messaging system, websockets rule. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Simple webserver
On 20.10.2023 23:05, Paul Rubin wrote: > Janis Papanagnou writes: >> I found a Python sample[*] but I am neither familiar with >> Python nor with the 'simple_websocket_server' package that >> is used in that sample code. But the code looks so simple >> that I'm considering to learn and use Python for the task. > > I've generally used ThreadingServer(SocketServer) for this purpose > and I think threads are less confusing than async, and performance is > fine if the concurrency level is not too high. But, trying to write a > web server in Python if you don't know Python doesn't seem like a great > idea, except as a learning project. I have a couple decades experience with about a dozen programming languages (not counting assemblers). Asynchronous processing, IPC, multi-processing, client/server architectures, multi-threading, semaphores, etc. etc. are concepts that are not new to me. I'm not, literally, intending to write a web-server. It's a JS application that is running in (browser based) clients, and the server is just centrally coordinating the client applications. My expectation would be that any sophistically designed socket/ web-socket library would not impose any risk. And the intended server by itself has only very limited requirements; listening to incoming request, storing some client information, broadcasting to the attached clients. Basically just (informally written): init server forever: wait for request(s) -> queue handle requests from queue (sequentially): store specific information from new registered clients broadcast some information to all registered clients It seems to me that multi-threading or async I/O aren't necessary. I'd like to ask; where do you see the specific risks with Python (as language per se) and it's (web-socket-)libraries here? If the web-socket IPC is well supported the algorithmic parts in Python seem trivial to learn and implement. - Or am I missing something? (A brief search gave me the impression that for JS communication web-sockets would be the method to use. Otherwise I'd just use basic Unix domain sockets for the purpose and write it, say, in C or C++ that I already know. But I don't know whether (or how) plain sockets are used from JS running in a browser. Here I'm lacking experience. And that lead me to have a look at Python, since the web-sockets/server examples that I found looked simple.) Janis -- https://mail.python.org/mailman/listinfo/python-list
Re: Running a subprocess in a venv
On 10/21/2023 11:32 AM, Larry Martell via Python-list wrote: On Sat, Oct 21, 2023 at 9:49 AM Johannes Findeisen wrote: On Sat, 21 Oct 2023 09:01:18 -0400 Larry Martell via Python-list wrote: I have a python script, and from that I want to run another script in a subprocess in a venv. What is the best way to do that? I could write a file that activates the venv then runs the script, then run that file, but that seems messy. Is there a better way? How do you do that? How? Open a file and write the commands I need then invoke that. It sounds messy but not wrong... I would activate the venv and then run my Python script. In the Python script you can call another python script in a subprocess like this: import sys import subprocess # https://docs.python.org/3/library/subprocess.html#popen-constructor proc = subprocess.Popen([sys.executable, "/path/to/an/otherscript.py"]) # https://docs.python.org/3/library/subprocess.html#popen-objects # Do your process communication/handling... proc.communicate(), # proc.wait(), proc.terminate(), proc.kill() etc. Is this the answer you are looking for? Detailed docs: https://docs.python.org/3/library/subprocess.html I know how to use Popen. What I was missing was running the script using sys.executable. Thanks. A nice feature of using sys.executable is that you automatically use the same Python installation as your invoking program is running with. On a system that has several different Python installations, that's a very good thing. -- https://mail.python.org/mailman/listinfo/python-list
Re: Running a subprocess in a venv
On Sat, Oct 21, 2023 at 12:10 PM Johannes Findeisen wrote: > > On Sat, 21 Oct 2023 11:32:03 -0400 > Larry Martell wrote: > > > On Sat, Oct 21, 2023 at 9:49 AM Johannes Findeisen > > wrote: > > > > > > On Sat, 21 Oct 2023 09:01:18 -0400 > > > Larry Martell via Python-list wrote: > > > > > > > I have a python script, and from that I want to run another > > > > script in a subprocess in a venv. What is the best way to do > > > > that? I could write a file that activates the venv then runs the > > > > script, then run that file, but that seems messy. Is there a > > > > better way? > > > > > > How do you do that? > > > > How? Open a file and write the commands I need then invoke that. > > > > > It sounds messy but not wrong... > > > > > > I would activate the venv and then run my Python script. In the > > > Python script you can call another python script in a subprocess > > > like this: > > > > > > import sys > > > import subprocess > > > > > > # > > > https://docs.python.org/3/library/subprocess.html#popen-constructor > > > proc = subprocess.Popen([sys.executable, > > > "/path/to/an/otherscript.py"]) > > > > > > # https://docs.python.org/3/library/subprocess.html#popen-objects > > > # Do your process communication/handling... proc.communicate(), > > > # proc.wait(), proc.terminate(), proc.kill() etc. > > > > > > Is this the answer you are looking for? > > > > > > Detailed docs: https://docs.python.org/3/library/subprocess.html > > > > I know how to use Popen. What I was missing was running the script > > using sys.executable. Thanks. > > sys.executable is the path to the actual Python binary, e.g. > "/usr/bin/python". You could add "/usr/bin/python" there manually but > this is not portable to Windows for example. > > When you add a shebang line to your other script and the file is > executable, you may not need to add sys.executable as first argument to > Popen but using sys.executable is the most reliable way to do this... ;) I need the path to whichever venv is being used so sys.executable works for me. -- https://mail.python.org/mailman/listinfo/python-list
Re: Running a subprocess in a venv
On Sat, 21 Oct 2023 11:32:03 -0400 Larry Martell wrote: > On Sat, Oct 21, 2023 at 9:49 AM Johannes Findeisen > wrote: > > > > On Sat, 21 Oct 2023 09:01:18 -0400 > > Larry Martell via Python-list wrote: > > > > > I have a python script, and from that I want to run another > > > script in a subprocess in a venv. What is the best way to do > > > that? I could write a file that activates the venv then runs the > > > script, then run that file, but that seems messy. Is there a > > > better way? > > > > How do you do that? > > How? Open a file and write the commands I need then invoke that. > > > It sounds messy but not wrong... > > > > I would activate the venv and then run my Python script. In the > > Python script you can call another python script in a subprocess > > like this: > > > > import sys > > import subprocess > > > > # > > https://docs.python.org/3/library/subprocess.html#popen-constructor > > proc = subprocess.Popen([sys.executable, > > "/path/to/an/otherscript.py"]) > > > > # https://docs.python.org/3/library/subprocess.html#popen-objects > > # Do your process communication/handling... proc.communicate(), > > # proc.wait(), proc.terminate(), proc.kill() etc. > > > > Is this the answer you are looking for? > > > > Detailed docs: https://docs.python.org/3/library/subprocess.html > > I know how to use Popen. What I was missing was running the script > using sys.executable. Thanks. sys.executable is the path to the actual Python binary, e.g. "/usr/bin/python". You could add "/usr/bin/python" there manually but this is not portable to Windows for example. When you add a shebang line to your other script and the file is executable, you may not need to add sys.executable as first argument to Popen but using sys.executable is the most reliable way to do this... ;) Regards, Johannes -- https://mail.python.org/mailman/listinfo/python-list
Re: Running a subprocess in a venv
On Sat, Oct 21, 2023 at 9:49 AM Johannes Findeisen wrote: > > On Sat, 21 Oct 2023 09:01:18 -0400 > Larry Martell via Python-list wrote: > > > I have a python script, and from that I want to run another script in > > a subprocess in a venv. What is the best way to do that? I could write > > a file that activates the venv then runs the script, then run that > > file, but that seems messy. Is there a better way? > > How do you do that? How? Open a file and write the commands I need then invoke that. > It sounds messy but not wrong... > > I would activate the venv and then run my Python script. In the Python > script you can call another python script in a subprocess like this: > > import sys > import subprocess > > # https://docs.python.org/3/library/subprocess.html#popen-constructor > proc = subprocess.Popen([sys.executable, "/path/to/an/otherscript.py"]) > > # https://docs.python.org/3/library/subprocess.html#popen-objects > # Do your process communication/handling... proc.communicate(), > # proc.wait(), proc.terminate(), proc.kill() etc. > > Is this the answer you are looking for? > > Detailed docs: https://docs.python.org/3/library/subprocess.html I know how to use Popen. What I was missing was running the script using sys.executable. Thanks. -- https://mail.python.org/mailman/listinfo/python-list
Re: Running a subprocess in a venv
Larry Martell via Python-list schreef op 21/10/2023 om 15:01: I have a python script, and from that I want to run another script in a subprocess in a venv. What is the best way to do that? I could write a file that activates the venv then runs the script, then run that file, but that seems messy. Is there a better way? Activating a venv it is practical when you're working in a shell, but not actually needed. You can execute the python in the venv with the script as parameter. Have a look in the venv directory: there will be a Script subdirectory (on Windows) or bin subdirectory (on Unix-like systems). Within that directory are several executables, one of which will be python or python3. That's the one you need. So use something like subprocess.run(['/path/to/venv/bin/python3', 'yourscript.py', possible other arguments]) -- "Binnen een begrensde ruimte ligt een kritiek punt, waar voorbij de vrijheid afneemt naarmate het aantal individuen stijgt. Dit gaat evenzeer op voor mensen in de begrensde ruimte van een planetair ecosysteem, als voor de gasmoleculen in een hermetisch gesloten vat. Bij mensen is het niet de vraag hoeveel er maximaal in leven kunnen blijven in het systeem, maar wat voor soort bestaan mogelijk is voor diegenen die in leven blijven. -- Pardot Kynes, eerste planetoloog van Arrakis" -- Frank Herbert, Duin -- https://mail.python.org/mailman/listinfo/python-list
Re: Running a subprocess in a venv
On Sat, 21 Oct 2023 09:01:18 -0400 Larry Martell via Python-list wrote: > I have a python script, and from that I want to run another script in > a subprocess in a venv. What is the best way to do that? I could write > a file that activates the venv then runs the script, then run that > file, but that seems messy. Is there a better way? How do you do that? It sounds messy but not wrong... I would activate the venv and then run my Python script. In the Python script you can call another python script in a subprocess like this: import sys import subprocess # https://docs.python.org/3/library/subprocess.html#popen-constructor proc = subprocess.Popen([sys.executable, "/path/to/an/otherscript.py"]) # https://docs.python.org/3/library/subprocess.html#popen-objects # Do your process communication/handling... proc.communicate(), # proc.wait(), proc.terminate(), proc.kill() etc. Is this the answer you are looking for? Detailed docs: https://docs.python.org/3/library/subprocess.html Regards, Johannes -- https://mail.python.org/mailman/listinfo/python-list
Running a subprocess in a venv
I have a python script, and from that I want to run another script in a subprocess in a venv. What is the best way to do that? I could write a file that activates the venv then runs the script, then run that file, but that seems messy. Is there a better way? -- https://mail.python.org/mailman/listinfo/python-list
