Popen Question
Hi,
I usually use csh for my simulation control scripts, but these scripts
are becoming more complex, so I plan to use python for the next
project.
To this end, I am looking at subprocess.Popen() to actually call the
simulations, and have a very basic question which is demonstrated
below.
[sde:st...@lbux03 ~]$ python
Python 2.6 (r26:66714, Feb 21 2009, 02:16:04)
[GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, subprocess
>>> os.environ['MYVAR'] = "myval"
>>> p = subprocess.Popen(['echo', '$MYVAR'],shell=True)
>>>
>>> p = subprocess.Popen(['echo', '$MYVAR'])
>>> $MYVAR
>>> p = subprocess.Popen('echo $MYVAR',shell=True)
>>> myval
>>> p = subprocess.Popen('echo $MYVAR')
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python2.6/subprocess.py", line 595, in __init__
errread, errwrite)
File "/usr/lib64/python2.6/subprocess.py", line 1106, in
_execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
I am not really sure I understand these results.
1) No idea what is going on
2) As (1). What isn't myval printed out (rather than $MYVAR)
3) Works as I wanted it to
4) Why do I need shell=True ?
The documentation isn't very clear to me (it seems you need to
understand the underlying system calls).
Can anyone explain (or provide link) for this behaviour in simple
English?
Thanks,
Steven
--
http://mail.python.org/mailman/listinfo/python-list
popen question
Hello,
look at this function:
--
def test():
child = os.popen('./slow')
for line in child:
print line
-
The program "slow" just writes the numbers 0 through 9 on stdout, one line a
second, and then quits.
I would have expected the python program to spit out a numbers one by one,
instead I see nothing for 10 seconds and then the whole output all at once.
How can I get and process the pipe's output at the pace it is generated?
Thanks,
robert
--
http://mail.python.org/mailman/listinfo/python-list
newbie: popen question
hello everyone :-), I am a newbie to python. I am trying to run a bash script from within a python program. I would greatly appreciate any pointers/comments about how to get around the problem I am facing. I want to run bash script: code.sh from within a python program. code.sh needs to be run like so from the command line [code] $ sudo code.sh arg1 arg2 [/code] I read up on some documentation but am not very clear about how to use popen. I want to relegate the shell to a background process, but it needs to accept the sudo passwd too! I have tried [code] p = subprocess.Popen(['/bin/bash', 'sudo '+mypath+'code.sh '+arg1+' '+arg2], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) [/code] I tried some code from stackoverflow.com/questions/694000/why-doesnt- subprocess-popen-always-return nothing really happens when this executes, the PIPE option pshes it to the background and I can't push in the sudo passwd. Can someone please give me an idea of how to go about this. To recap, I want to run a shell script, which needs to be started with sudo, and then push it into the background. Thanks, -A -- http://mail.python.org/mailman/listinfo/python-list
Re: Popen Question
On Nov 4, 7:06 pm, moogyd wrote:
> Hi,
> I usually use csh for my simulation control scripts, but these scripts
> are becoming more complex, so I plan to use python for the next
> project.
> To this end, I am looking at subprocess.Popen() to actually call the
> simulations, and have a very basic question which is demonstrated
> below.
>
> [sde:st...@lbux03 ~]$ python
> Python 2.6 (r26:66714, Feb 21 2009, 02:16:04)
> [GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>
> >>> import os, subprocess
> >>> os.environ['MYVAR'] = "myval"
> >>> p = subprocess.Popen(['echo', '$MYVAR'],shell=True)
>
> >>> p = subprocess.Popen(['echo', '$MYVAR'])
> >>> $MYVAR
> >>> p = subprocess.Popen('echo $MYVAR',shell=True)
> >>> myval
> >>> p = subprocess.Popen('echo $MYVAR')
>
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/lib64/python2.6/subprocess.py", line 595, in __init__
> errread, errwrite)
> File "/usr/lib64/python2.6/subprocess.py", line 1106, in
> _execute_child
> raise child_exception
> OSError: [Errno 2] No such file or directory
>
> I am not really sure I understand these results.
> 1) No idea what is going on
> 2) As (1). What isn't myval printed out (rather than $MYVAR)
> 3) Works as I wanted it to
> 4) Why do I need shell=True ?
> The documentation isn't very clear to me (it seems you need to
> understand the underlying system calls).
>
> Can anyone explain (or provide link) for this behaviour in simple
> English?
> Thanks,
> Steven
try giving /bin/echo
--
http://mail.python.org/mailman/listinfo/python-list
Re: Popen Question
moogyd writes:
import os, subprocess
os.environ['MYVAR'] = "myval"
p = subprocess.Popen(['echo', '$MYVAR'],shell=True)
p = subprocess.Popen(['echo', '$MYVAR'])
$MYVAR
>
p = subprocess.Popen('echo $MYVAR',shell=True)
myval
>
p = subprocess.Popen('echo $MYVAR')
> Traceback (most recent call last):
> File "", line 1, in
> File "/usr/lib64/python2.6/subprocess.py", line 595, in __init__
> errread, errwrite)
> File "/usr/lib64/python2.6/subprocess.py", line 1106, in
> _execute_child
> raise child_exception
> OSError: [Errno 2] No such file or directory
>
> I am not really sure I understand these results.
> 1) No idea what is going on
> 2) As (1). What isn't myval printed out (rather than $MYVAR)
> 3) Works as I wanted it to
> 4) Why do I need shell=True ?
Expanding $MYVAR into its value is a feature of the shell (afaik all
shells use the same syntax). Popen without shell=True uses the execvp()
system call directly, without going through the shell variable expansion
process (cases 2 and 4 above). For example, case 4 above asks execvp to
(find and) execute a program named "echo $MYVAR" (an 11-letter name,
where the fifth letter is space and the sixth is $ -- a perfectly valid
file/program name).
Then, if you use shell=True with a list, only the first word is used as
a command, and the others are kept in positional parameters. That's why
your first try fails (try 'sh -c echo $HOME' in a shell, without the
single quotes, and you'll get empty output).
> The documentation isn't very clear to me (it seems you need to
> understand the underlying system calls).
You're probably right. The base fact here is: the use of variables is a
feature of the shell. No shell, no variable.
> Can anyone explain (or provide link) for this behaviour in simple
> English?
Shell variables are explained in detail in any shell man page. The
execvp() system call has its own man page.
-- Alain.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Popen Question
In article <891a9a80-c30d-4415-ac81-bddd0b564...@g13g2000yqj.googlegroups.com>
moogyd wrote:
>[sde:st...@lbux03 ~]$ python
>Python 2.6 (r26:66714, Feb 21 2009, 02:16:04)
>[GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2
>Type "help", "copyright", "credits" or "license" for more information.
import os, subprocess
os.environ['MYVAR'] = "myval"
p = subprocess.Popen(['echo', '$MYVAR'],shell=True)
Alain Ketterlin has already explained these to some extent.
Here is a bit more.
This runs, underneath:
['/bin/sh', '-c', 'echo', '$MYVAR']
(with arguments expressed as a Python list). /bin/sh takes the
string after '-c' as a command, and the remaining argument(s) if
any are assigned to positional parameters ($0, $1, etc).
If you replace the command with something a little more explicit,
you can see this:
>>> p = subprocess.Popen(
...[r'echo \$0=$0 \$1=$1', 'arg0', '$MYVAR'], shell=True)
>>> $0=arg0 $1=$MYVAR
p.wait()
0
>>>
(I like to call p.communicate() or p.wait(), although p.communicate()
is pretty much a no-op if you have not done any redirecting. Note that
p.communicate() does a p.wait() for you.)
p = subprocess.Popen(['echo', '$MYVAR'])
$MYVAR
This time, as Alain noted, the shell does not get involved so no
variable expansion occurs. However, you could do it yourself:
>>> p = subprocess.Popen(['echo', os.environ['MYVAR']])
>>> myval
p.wait()
0
>>>
p = subprocess.Popen('echo $MYVAR',shell=True)
myval
(here /bin/sh does the expansion, because you invoked it)
p = subprocess.Popen('echo $MYVAR')
>Traceback (most recent call last):
> File "", line 1, in
> File "/usr/lib64/python2.6/subprocess.py", line 595, in __init__
>errread, errwrite)
> File "/usr/lib64/python2.6/subprocess.py", line 1106, in
>_execute_child
>raise child_exception
>OSError: [Errno 2] No such file or directory
This attempted to run the executable named 'echo $MYVAR'. It did
not exist so the underlying exec (after the fork) failed. The
exception was passed back to the subprocess module, which raised
it in the parent for you to see.
If you were to create an executable named 'echo $MYVAR' (including
the blank and dollar sign) somewhere in your path (or use an explicit
path to it), it would run. I will also capture the actual output
this time:
$ cat '/tmp/echo $MYVAR'
#! /usr/bin/awk NR>1{print}
this is a self-printing file
anything after the first line has NR > 1, so gets printed
$ chmod +x '/tmp/echo $MYVAR'
$ python
Python 2.5.1 (r251:54863, Feb 6 2009, 19:02:12)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> p = subprocess.Popen('/tmp/echo $MYVAR', stdout=subprocess.PIPE)
>>> print p.communicate()[0]
this is a self-printing file
anything after the first line has NR > 1, so gets printed
>>> p.returncode
0
>>>
Incidentally, fun with #!: you can make self-renaming scripts:
sh-3.2$ echo '#! /bin/mv' > /tmp/selfmove; chmod +x /tmp/selfmove
sh-3.2$ ls /tmp/*move*
/tmp/selfmove
sh-3.2$ /tmp/selfmove /tmp/I_moved
sh-3.2$ ls /tmp/*move*
/tmp/I_moved
sh-3.2$
or even self-removing scripts:
sh-3.2$ echo '#! /bin/rm' > /tmp/rmme; chmod +x /tmp/rmme
sh-3.2$ /tmp/rmme
sh-3.2$ /tmp/rmme
sh: /tmp/rmme: No such file or directory
(nothing to do with python, just the way #! interpreter lines work).
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
email: gmail (figure it out) http://web.torek.net/torek/index.html
--
http://mail.python.org/mailman/listinfo/python-list
Re: Popen Question
Hi, Thanks everyone for the replies - it is now clearer. Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Popen Question
In message , Chris Torek wrote: > ['/bin/sh', '-c', 'echo', '$MYVAR'] > > (with arguments expressed as a Python list). /bin/sh takes the > string after '-c' as a command, and the remaining argument(s) if > any are assigned to positional parameters ($0, $1, etc). Doesn’t work. I don’t know what happens to the extra arguments, but they just seem to be ignored if -c is specified. sh -c 'echo hi' echoes “hi”, while sh -c echo hi just outputs a blank line. -- http://mail.python.org/mailman/listinfo/python-list
Re: Popen Question
Lawrence D'Oliveiro writes: > In message , Chris Torek wrote: > > > ['/bin/sh', '-c', 'echo', '$MYVAR'] > > > > (with arguments expressed as a Python list). /bin/sh takes the > > string after '-c' as a command, and the remaining argument(s) if > > any are assigned to positional parameters ($0, $1, etc). > > Doesn’t work. What doesn't work? You were being given an explanation, not a solution. > I don’t know what happens to the extra arguments, but they just seem > to be ignored if -c is specified. The argument to -c is taken as a shell script; the remaining arguments are made available as positional parameters to the script as usual (only starting with $0 rather than $1, for some unknown reason). -- [mdw] -- http://mail.python.org/mailman/listinfo/python-list
Re: Popen Question
On Nov 8, 2:43 am, m...@distorted.org.uk (Mark Wooding) wrote: > > I don’t know what happens to the extra arguments, but they just seem > > to be ignored if -c is specified. > > The argument to -c is taken as a shell script; the remaining arguments > are made available as positional parameters to the script as usual (only > starting with $0 rather than $1, for some unknown reason). Perhaps this example better demonstrates what is going on: >>> p = subprocess.Popen(['echo one $0 three $1 five', 'two', 'four'], ... shell=True) one two three four five Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Popen Question
Ian wrote: On Nov 8, 2:43 am, m...@distorted.org.uk (Mark Wooding) wrote: I don’t know what happens to the extra arguments, but they just seem to be ignored if -c is specified. The argument to -c is taken as a shell script; the remaining arguments are made available as positional parameters to the script as usual (only starting with $0 rather than $1, for some unknown reason). Perhaps this example better demonstrates what is going on: p = subprocess.Popen(['echo one $0 three $1 five', 'two', 'four'], ... shell=True) one two three four five Maybe I'm thick, but I still don't understand. If I run a shell script, then the name of the script is in $0 and the first positional arguments is in $1, similar to how Python sets up sys.argv. But in this case the first positional argument is in $0. Why is that? Puzzled, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: Popen Question
In message <4cd87b24$0$81481$e4fe5...@news.xs4all.nl>, Hans Mulder wrote: > But in this case the first positional argument is in $0. That’s what confused me. -- http://mail.python.org/mailman/listinfo/python-list
Re: Popen Question
On Nov 8, 3:35 pm, Hans Mulder wrote: > > Perhaps this example better demonstrates what is going on: > > p = subprocess.Popen(['echo one $0 three $1 five', 'two', 'four'], > > ... shell=True) > > one two three four five > > Maybe I'm thick, but I still don't understand. If I run a shell script, > then the name of the script is in $0 and the first positional arguments > is in $1, similar to how Python sets up sys.argv. > > But in this case the first positional argument is in $0. Why is that? It's just a quirk in the way the shell handles the -c option. The syntax for the shell invocation boils down to something like this: sh [-c command_string] command_name arg1 arg2 arg3 ... Without the -c option, sh runs the file indicated by command_name, setting $0 to command_name, $1 to arg1, $2 to arg2, etc. With the -c option, it does the same thing; it just runs the command_string instead of a file pointed to by command_name. The latter still conceptually exists as an argument, however, which is why it still gets stored in $0 instead of $1. We could argue about whether this approach is correct or not, but it's what the shell does, and that's not likely to change. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: popen question
On Tue, 08 Jan 2008 09:20:16 +, Robert Latest wrote: > The program "slow" just writes the numbers 0 through 9 on stdout, one line a > second, and then quits. > > I would have expected the python program to spit out a numbers one by one, > instead I see nothing for 10 seconds and then the whole output all at once. > > How can I get and process the pipe's output at the pace it is generated? Both processes have to make their communication ends unbuffered or line buffered. See the documentation of `os.popen()` for the `bufsize` argument. And do whatever is needed to output the numbers from ``slow`` unbuffered or line buffered. Ciao, Marc 'BlackJack' Rintsch -- http://mail.python.org/mailman/listinfo/python-list
Re: popen question
Marc 'BlackJack' Rintsch wrote: > Both processes have to make their communication ends unbuffered or line > buffered. Yeah, I figured something like that. > And do whatever is needed to output the numbers from ``slow`` > unbuffered or line buffered. Hm, "slow" of course is just a little test program I wrote for this purpose. In reality I want to call another program whose behavior I can't influence (well, technically I could because it's open-source, but let's assume it to be a black box for now). If 'slow' or some other program does buffered output, how come I can see its output line-by-line in the shell? robert -- http://mail.python.org/mailman/listinfo/python-list
Re: popen question
Hrvoje Niksic wrote: > stdio uses different buffering strategies depending on the output > type. When the output is a TTY, line buffering is used; when the > output goes to a pipe or file, it is fully buffered. Makes sense. > If you see lines one by one, you are in luck, and you can fix things > on the Python level simply by avoiding buffering in popen. If not, > you will need to resort to more advanced hackery (e.g. fixing stdio > using LD_PRELOAD). Do I really? After all, the shell itself doesn't hack stdio, does it? Anyway, I'm taking this over to comp.unix.programmer since it really isn't a python problem. Thanks, robert -- http://mail.python.org/mailman/listinfo/python-list
Re: popen question
Robert Latest <[EMAIL PROTECTED]> writes: > If 'slow' or some other program does buffered output, how come I can > see its output line-by-line in the shell? stdio uses different buffering strategies depending on the output type. When the output is a TTY, line buffering is used; when the output goes to a pipe or file, it is fully buffered. > In reality I want to call another program whose behavior I can't > influence (well, technically I could because it's open-source, but > let's assume it to be a black box for now). To test whether your black box buffers output to pipe, simply start it like this: $ ./slow | cat If you see lines one by one, you are in luck, and you can fix things on the Python level simply by avoiding buffering in popen. If not, you will need to resort to more advanced hackery (e.g. fixing stdio using LD_PRELOAD). -- http://mail.python.org/mailman/listinfo/python-list
Re: popen question
Robert Latest <[EMAIL PROTECTED]> writes: >> If you see lines one by one, you are in luck, and you can fix things >> on the Python level simply by avoiding buffering in popen. If not, >> you will need to resort to more advanced hackery (e.g. fixing stdio >> using LD_PRELOAD). > > Do I really? After all, the shell itself doesn't hack stdio, does > it? Have you tried the "./slow | cat" test? It's not about the shell doing anything special, it's about slow's stdio choosing buffering "appropriate" for the output device. The hackery I referred to would be necessary to force slow's stdio to use line buffering over file/pipe output simply because there is no documented way to do that (that I know of). > Anyway, I'm taking this over to comp.unix.programmer since it really > isn't a python problem. It's still an often-encountered problem, so I believe a summary of the solution(s) would be appreciated. -- http://mail.python.org/mailman/listinfo/python-list
Re: popen question
On Jan 8, 1:20 am, Robert Latest <[EMAIL PROTECTED]> wrote:
> Hello,
>
> look at this function:
>
> --
> def test():
> child = os.popen('./slow')
> for line in child:
> print line
> -
>
> The program "slow" just writes the numbers 0 through 9 on stdout, one line a
> second, and then quits.
>
> I would have expected the python program to spit out a numbers one by one,
> instead I see nothing for 10 seconds and then the whole output all at once.
>
> How can I get and process the pipe's output at the pace it is generated?
I've seen this problem and it took me a while to figure out what is
happening.
As other posts, I too first suspected it's a problem related to line/
full buffering on the sender side (./slow here).
It turned out that the "for line in child:" line in the iterator is
the culprit. The iterator does something like a child.readlines()
underneath (in it's __iter__ call) instead of a more logical single
line read.
Change your reading to force line-by-line read
e.g.
While True:
line = child.readline()
if not line: break
print line
Karthik
>
> Thanks,
>
> robert
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie: popen question
On May 27, 6:10 pm, thebiggestbangthe...@gmail.com wrote:
> hello everyone :-),
> I am a newbie to python. I am trying to run a
> bash script from within a python program. I would greatly appreciate
> any pointers/comments about how to get around the problem I am facing.
>
> I want to run bash script: code.sh from within a python program.
> code.sh needs to be run like so from the command line
> [code]
> $ sudo code.sh arg1 arg2
> [/code]
>
> I read up on some documentation but am not very clear about how to use
> popen. I want to relegate the shell to a background process, but it
> needs to accept the sudo passwd too!
>
> I have tried
> [code]
> p = subprocess.Popen(['/bin/bash', 'sudo '+mypath+'code.sh '+arg1+'
> '+arg2],
> stdout=subprocess.PIPE,
> stderr=subprocess.STDOUT)
> [/code]
> I tried some code from stackoverflow.com/questions/694000/why-doesnt-
> subprocess-popen-always-return
>
> nothing really happens when this executes, the PIPE option pshes it to
> the background and I can't push in the sudo passwd. Can someone please
> give me an idea of how to go about this.
>
> To recap, I want to run a shell script, which needs to be started with
> sudo, and then push it into the background.
>
> Thanks,
> -A
Your best bet is to make sudo not ask for a password. :) If you
don't have the rights, then you can use pexpect to do what you want to
do. http://pexpect.sourceforge.net/pexpect.html
See the second example on that page.
child = pexpect.spawn('scp foo myn...@host.example.com:.')
child.expect ('Password:')
child.sendline (mypassword)
~Sean
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie: popen question
On Thu, May 28, 2009 at 9:11 AM, Sean DiZazzo wrote: > On May 27, 6:10 pm, thebiggestbangthe...@gmail.com wrote: > > hello everyone :-), > > I am a newbie to python. I am trying to run a > > bash script from within a python program. I would greatly appreciate > > any pointers/comments about how to get around the problem I am facing. > > > > I want to run bash script: code.sh from within a python program. > > code.sh needs to be run like so from the command line > > [code] > > $ sudo code.sh arg1 arg2 > > [/code] > > > > I read up on some documentation but am not very clear about how to use > > popen. I want to relegate the shell to a background process, but it > > needs to accept the sudo passwd too! > > > > I have tried > > [code] > > p = subprocess.Popen(['/bin/bash', 'sudo '+mypath+'code.sh '+arg1+' > > '+arg2], > > stdout=subprocess.PIPE, > > stderr=subprocess.STDOUT) > > [/code] > > I tried some code from stackoverflow.com/questions/694000/why-doesnt- > > subprocess-popen-always-return > > > > nothing really happens when this executes, the PIPE option pshes it to > > the background and I can't push in the sudo passwd. Can someone please > > give me an idea of how to go about this. > > > > To recap, I want to run a shell script, which needs to be started with > > sudo, and then push it into the background. > > > > Thanks, > > -A > > Your best bet is to make sudo not ask for a password. :) If you > don't have the rights, then you can use pexpect to do what you want to > do. http://pexpect.sourceforge.net/pexpect.html > Whoah there. Pexpect, yes. Making sudo not ask for a password? Only if you tell sudo to only not ask for a password for _this_ file. Telling sudo to not require a password at all is asking for trouble. Also, modifying sudo to allow this script to run without a password would require that to be done on every machine that the OP wants to run on. OP: use pexpect. -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie: popen question
> Your best bet is to make sudo not ask for a password. :) If you
> don't have the rights, then you can use pexpect to do what you want to
> do. http://pexpect.sourceforge.net/pexpect.html
>
> See the second example on that page.
>
> child = pexpect.spawn('scp foo myn...@host.example.com:.')
> child.expect ('Password:')
> child.sendline (mypassword)
The sudo password prompt is very configurable, so changing the configuration
to allow execution without password input is really the best option.
--
Freedom is always the freedom of dissenters.
(Rosa Luxemburg)
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie: popen question
On May 28, 5:31 am, Sebastian Wiesner wrote:
>
>
> > Your best bet is to make sudo not ask for a password. :) If you
> > don't have the rights, then you can use pexpect to do what you want to
> > do. http://pexpect.sourceforge.net/pexpect.html
>
> > See the second example on that page.
>
> > child = pexpect.spawn('scp foo myn...@host.example.com:.')
> > child.expect ('Password:')
> > child.sendline (mypassword)
>
> The sudo password prompt is very configurable, so changing the configuration
> to allow execution without password input is really the best option.
>
> --
> Freedom is always the freedom of dissenters.
> (Rosa Luxemburg)
Thanks guys for helping out! very good answers :-)
Before I saw your answers, I tried the following,
output = subprocess.Popen(["sudo","-b", "code.sh", "arg1"],
stdout=subprocess.PIPE).communicate()[0]
This seemed to push the shell execution process to the background and
because my python program was invoked initially with sudo, it seems I
did not need to enter a passwd again.
Any comments about this..any issues that you see will crop up?
Thanks a ton again.
--
http://mail.python.org/mailman/listinfo/python-list
Re: newbie: popen question
thebiggestbangthe...@gmail.com wrote:
> On May 28, 5:31 am, Sebastian Wiesner wrote:
>>
>>
>>> Your best bet is to make sudo not ask for a password. :) If you
>>> don't have the rights, then you can use pexpect to do what you want to
>>> do. http://pexpect.sourceforge.net/pexpect.html
>>> See the second example on that page.
>>> child = pexpect.spawn('scp foo myn...@host.example.com:.')
>>> child.expect ('Password:')
>>> child.sendline (mypassword)
>> The sudo password prompt is very configurable, so changing the configuration
>> to allow execution without password input is really the best option.
>>
>> --
>> Freedom is always the freedom of dissenters.
>> (Rosa Luxemburg)
>
> Thanks guys for helping out! very good answers :-)
>
> Before I saw your answers, I tried the following,
>
> output = subprocess.Popen(["sudo","-b", "code.sh", "arg1"],
> stdout=subprocess.PIPE).communicate()[0]
>
> This seemed to push the shell execution process to the background and
> because my python program was invoked initially with sudo, it seems I
> did not need to enter a passwd again.
>
> Any comments about this..any issues that you see will crop up?
>
> Thanks a ton again.
>
>
>
Is using gksu or kdesu feasible? Or maybe you could run "sudo -v" which
activates sudo then immediately run your "sudo command". This relies on
sudo not configured to not use timestamp though.
--
http://mail.python.org/mailman/listinfo/python-list
Popen question (redundant processes)
Hello World! This is my first post on the list and I'm hoping it is the right forum and not OT, I've searched a bit on this, but, none-the-wiser! My question is on the Popen method, here is my snippet: p1 = Popen(["cat", "georgi_ddr7_allmag_kcor_in_test.dat"], stdout=PIPE ) > p2 = Popen(["fit_coeffs"], stdin=p1.stdout, stdout=PIPE) > p3 = Popen(["reconstruct_maggies"], stdin=p2.stdout,stdout=PIPE) > output_maggies_z=p3.communicate()[0] > > p1 = Popen(["cat", "georgi_ddr7_allmag_kcor_in_test.dat"], stdout=PIPE ) > p2 = Popen(["fit_coeffs"], stdin=p1.stdout, stdout=PIPE) > p4 = Popen(["reconstruct_maggies", "--band-shift", "0.1", "--redshift", > "0."], stdin=p2.stdout,stdout=PIPE) > output_maggies_z0=p4.communicate()[0] > > That is, p1 and p2 are the same, but p3 and p4 which they are passed to, are different. Is there a way to pass p1 and p2 to p3 AND p4 simultaneously, so as to not need to run p1 and p2 twice, as above? What arguments would I need to achieve this? NOTE: "georgi_ddr7_allmag_kcor_in_test.dat" is a very large file (~1E6 records) regards, - Sebastian -- http://mail.python.org/mailman/listinfo/python-list
(SOLVED) Re: popen question
pexpect is the solution. Seems to wrap quite a bit of dirty pseudo-tty hacking. robert -- http://mail.python.org/mailman/listinfo/python-list
a popen question. Please help
Hi all,
I write a small script
texts = os.popen('top').readlines()
print texts
It calls the command line "top" and will print out some texts.
But first I have to press the keyboard "q" to quit the subprocess "top", then
the texts will be printed, otherwise it just stands by with blank.
Question
is. Do you know how to give "q" into my python script so that "top" is
automatically quit immediately or maybe after 1s and print out the texts.
Thank you
--
http://mail.python.org/mailman/listinfo/python-list
Re:a popen question. Please help
First, I think you should use subprocess.Popen (it's recommended by
PEP-324) instead of os.popen. For example:
p = subprocess.Popen(["top"], stdout = PIPE)
p.stdout.readlines()
And to write to stdin (in your case "q") you can use p.stdin.write("q"), or
terminate the process with p.terminate(), or just specify the -n option
(the number of iterations) to the value you desire. It's done in that way:
subprocess.Popen(["top","-n 1"], stdout=PIPE)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Popen question (redundant processes)
On Sun, Aug 30, 2009 at 12:33 PM, Sebastian wrote: > Hello World! > This is my first post on the list and I'm hoping it is the right forum and > not OT, I've searched > a bit on this, but, none-the-wiser! > > My question is on the Popen method, here is my snippet: > >> p1 = Popen(["cat", "georgi_ddr7_allmag_kcor_in_test.dat"], stdout=PIPE ) >> p2 = Popen(["fit_coeffs"], stdin=p1.stdout, stdout=PIPE) >> p3 = Popen(["reconstruct_maggies"], stdin=p2.stdout,stdout=PIPE) >> output_maggies_z=p3.communicate()[0] >> >> p1 = Popen(["cat", "georgi_ddr7_allmag_kcor_in_test.dat"], stdout=PIPE ) >> p2 = Popen(["fit_coeffs"], stdin=p1.stdout, stdout=PIPE) >> p4 = Popen(["reconstruct_maggies", "--band-shift", "0.1", "--redshift", >> "0."], stdin=p2.stdout,stdout=PIPE) >> output_maggies_z0=p4.communicate()[0] >> > > That is, p1 and p2 are the same, but p3 and p4 which they are passed to, are > different. > Is there a way to pass p1 and p2 to p3 AND p4 simultaneously, so as to not > need to > run p1 and p2 twice, as above? > What arguments would I need to achieve this? > > NOTE: "georgi_ddr7_allmag_kcor_in_test.dat" is a very large file (~1E6 > records) Send the output of p2 through the unix command "tee" (http://unixhelp.ed.ac.uk/CGI/man-cgi?tee). Then put the output of tee into p3 and set p4's input to the file you specified to tee. Cheers, Chris -- http://blog.rebertia.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Popen question (redundant processes)
En Sun, 30 Aug 2009 17:25:40 -0300, Chris Rebert
escribió:
On Sun, Aug 30, 2009 at 12:33 PM, Sebastian wrote:
Hello World!
This is my first post on the list and I'm hoping it is the right forum
and
not OT, I've searched
a bit on this, but, none-the-wiser!
My question is on the Popen method, here is my snippet:
p1 = Popen(["cat", "georgi_ddr7_allmag_kcor_in_test.dat"], stdout=PIPE
)
p2 = Popen(["fit_coeffs"], stdin=p1.stdout, stdout=PIPE)
p3 = Popen(["reconstruct_maggies"], stdin=p2.stdout,stdout=PIPE)
output_maggies_z=p3.communicate()[0]
p1 = Popen(["cat", "georgi_ddr7_allmag_kcor_in_test.dat"], stdout=PIPE
)
p2 = Popen(["fit_coeffs"], stdin=p1.stdout, stdout=PIPE)
p4 = Popen(["reconstruct_maggies", "--band-shift", "0.1", "--redshift",
"0."], stdin=p2.stdout,stdout=PIPE)
output_maggies_z0=p4.communicate()[0]
That is, p1 and p2 are the same, but p3 and p4 which they are passed
to, are
different.
Is there a way to pass p1 and p2 to p3 AND p4 simultaneously, so as to
not
need to
run p1 and p2 twice, as above?
What arguments would I need to achieve this?
NOTE: "georgi_ddr7_allmag_kcor_in_test.dat" is a very large file (~1E6
records)
Send the output of p2 through the unix command "tee"
(http://unixhelp.ed.ac.uk/CGI/man-cgi?tee). Then put the output of tee
into p3 and set p4's input to the file you specified to tee.
In addition to that, you can avoid the cat command (and omit p1
completely), just pass the georgi file as p2 stdin:
p2 = Popen(["fit_coeffs"], stdin=open("georgi..."), ...
In other words, the original shell commands are:
cat georgi.dat | fit | reconstruct
cat georgi.dat | fit | reconstruct --otherargs
and we suggest doing:
fit < georgi.dat | tee /tmp/foo | reconstruct
reconstruct --otherargs < /tmp/foo
--
Gabriel Genellina
--
http://mail.python.org/mailman/listinfo/python-list
Re: a popen question. Please help
texts = os.popen('top').readlines()
print texts
It calls the command line "top" and will print out some texts.
But first I have to press the keyboard "q" to quit the subprocess "top", then
the texts will be printed, otherwise it just stands by with blank.
Question
is. Do you know how to give "q" into my python script so that "top" is
automatically quit immediately or maybe after 1s and print out the texts.
Well as a workaround, my version of top (on Debian) supports a
"-n" parameter so you can tell it how many iterations you want it
to perform before quitting. So you should be able to just use
os.popen('top -n1').readlines()
-tkc
--
http://mail.python.org/mailman/listinfo/python-list
Re: a popen question. Please help
On Sun, Aug 30, 2009 at 4:43 AM, Tim Chase wrote:
>> texts = os.popen('top').readlines()
>> print texts
>>
>> It calls the command line "top" and will print out some texts.
>> But first I have to press the keyboard "q" to quit the subprocess "top",
>> then the texts will be printed, otherwise it just stands by with blank.
>>
>> Question
>> is. Do you know how to give "q" into my python script so that "top" is
>> automatically quit immediately or maybe after 1s and print out the texts.
>
> Well as a workaround, my version of top (on Debian) supports a "-n"
> parameter so you can tell it how many iterations you want it to perform
> before quitting. So you should be able to just use
>
> os.popen('top -n1').readlines()
Hm, interesting. On Mac OS X's (and BSD's?) top, -n instead specifies
the number of processes to list at a time (i.e. list only the top N
processes), which is entirely different.
Cheers,
Chris
--
http://mail.python.org/mailman/listinfo/python-list
Re: a popen question. Please help
os.popen('top -n1').readlines()
Hm, interesting. On Mac OS X's (and BSD's?) top, -n instead specifies
the number of processes to list at a time (i.e. list only the top N
processes), which is entirely different.
[reaching over to my Mac] Looks like "top" there supports a -l
parameter which does something similar.
Darn "standards" :-/
-tkc
--
http://mail.python.org/mailman/listinfo/python-list
Re: a popen question. Please help
In article , Tim Chase wrote: > >Darn "standards" :-/ The wonderful thing about standards is that there are so many to choose from. -- Aahz (a...@pythoncraft.com) <*> http://www.pythoncraft.com/ "I support family values -- Addams family values" --www.nancybuttons.com -- http://mail.python.org/mailman/listinfo/python-list
maybe a popen question... or something else?
Working on Windows XP
Say I have a Windows executable, foo.exe.
foo.exe is a command line tool that can take a number of different
arguments and perform corresponding actions.
I want to invoke foo.exe from a Python script (using whatever will
work best). I want to continuously pass arguments to foo.exe in
between doing other stuff from within my Python script.
Some pseudo code:
processHandle = invoke("foo.exe") # what python module should "invoke"
be here?
doUnrelatedStuff()
processHandle.passArgs("arg1 arg2") # The same foo.exe I invoked above
gets these args for processing
doMoreUnrelatedStuff()
processHandle.passArgs("arg3 arg4") # The same foo.exe I invoked above
gets these args for processing
processHandle.close() # "foo.exe is destroyed"
Thanks for any help.
--
http://mail.python.org/mailman/listinfo/python-list
Re: maybe a popen question... or something else?
dude wrote:
> Working on Windows XP
> Say I have a Windows executable, foo.exe.
> foo.exe is a command line tool that can take a number of different
> arguments and perform corresponding actions.
>
> I want to invoke foo.exe from a Python script (using whatever will
> work best). I want to continuously pass arguments to foo.exe in
> between doing other stuff from within my Python script.
>
> Some pseudo code:
>
> processHandle = invoke("foo.exe") # what python module should "invoke"
> be here?
>
> doUnrelatedStuff()
>
> processHandle.passArgs("arg1 arg2") # The same foo.exe I invoked above
> gets these args for processing
>
> doMoreUnrelatedStuff()
>
> processHandle.passArgs("arg3 arg4") # The same foo.exe I invoked above
> gets these args for processing
>
> processHandle.close() # "foo.exe is destroyed"
>
> Thanks for any help.
>
When you say "pass arguments", the methodology you outline definitely
isn't going to fly with argument passing. When a command is invoked the
arguments are taken from the command line, so it isn't possible to pass
further arguments at a later time.
You probably need to write to you sub-process's standard input stream.
which you can do whit the subprocess module (the modern way to do it) or
one of the various popen() functions.
regards
Steve
--
Steve Holden+1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
--- Asciimercial --
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
--- Thank You for Reading -
--
http://mail.python.org/mailman/listinfo/python-list
