Re: subprocess problem
On 2017-02-11 02:23:12 +, Cameron Simpson said:
def your_function(...):
with open('/path/to/your/logfile.txt', 'a') as logfp:
print("PATH=".os.environ['PATH'], file=logfp)
p=Popen(...)
p.communicate(...)
print("p.returncode=%r" % (p.returncode))
and any other interesting values that occur to you. This is really simple, and
doesn't require a terminal or using the logging module.
You can 'tail -f /path/to/your/logfile.txt' in another terminal to watch stuff
happen as you exercise the program.
Thanks for the great advice.
I found out, that the frozen app does not pass the normal path
variable. All the tricks with passing the correct environment did not
work. I just hardcoded the existing standard path. After that it worked.
I think it is a problem with frozen apps in general. Cx_Freeze has
other problems than Pyinstaller. But all need some workarounds for the
problems.
--
https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On 10Feb2017 11:54, Andreas Paeffgen wrote:
Thanks for all the great advice.
I tested all the possibilities. So far no luck. If i start the app
from a terminal, the solutions work. But not, if i start the
app-bundle (There is no connection to a terminal)
1. Copied the path to p.subprocess
2. Used the def wich: python function
3. Used shutil
When there is no Terminal, there is no error message / status code message.
Please show us you code containing the status code check.
I would do some debugging inside the function. Consider writing to a special
purpose log file, for example like this:
def your_function(...):
with open('/path/to/your/logfile.txt', 'a') as logfp:
print("PATH=".os.environ['PATH'], file=logfp)
p=Popen(...)
p.communicate(...)
print("p.returncode=%r" % (p.returncode))
and any other interesting values that occur to you. This is really simple, and
doesn't require a terminal or using the logging module.
You can 'tail -f /path/to/your/logfile.txt' in another terminal to watch stuff
happen as you exercise the program.
Cheers,
Cameron Simpson
--
https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
Thanks for all the great advice. I tested all the possibilities. So far no luck. If i start the app from a terminal, the solutions work. But not, if i start the app-bundle (There is no connection to a terminal) 1. Copied the path to p.subprocess 2. Used the def wich: python function 3. Used shutil When there is no Terminal, there is no error message / status code message. -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On 10Feb2017 00:03, eryk sun wrote: On Thu, Feb 9, 2017 at 10:50 PM, Cameron Simpson wrote: This is why I suggested the check_returncode() method, which examines the error code. You must be thinking of the returncode attribute, which isn't a method. check_returncode() is a method of the CompletedProcess object that's returned by subprocess.run(), which was added in 3.5. The OP is using both 3.4 and 3.5, so run() isn't a practical option. The communicate method mentioned it; I hadn't realised it was new with 3.5, good point. The older check_output() function also checks the return code and raises a subprocess.CalledProcessError if the command fails. For example: [...] Cool, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On Fri, Feb 10, 2017 at 12:05 AM, Wildman via Python-list wrote: > > Corrected code: > > def which(target) > for p in pathlist: > fullpath = p + "/" + target > if os.path.isfile(fullpath) and os.access(fullpath, os.X_OK): > return fullpath, True > return None, False Use shutil.which: https://docs.python.org/3/library/shutil.html#shutil.which -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On 2017-02-10 00:05, Wildman via Python-list wrote:
On Fri, 10 Feb 2017 09:53:32 +1100, Cameron Simpson wrote:
On 09Feb2017 11:59, Wildman wrote:
Here is a method I frequently use to replace the which
command. (air code)
import os
pathlist = os.environ["PATH"].split(":")
def which(target)
for p in pathlist:
fullpath = p + "/" + target
if os.path.isfile(fullpath):
return fullpath, True
return "", False
Cleaner is to return the path if found, or None if not.
I don't see this as an important issue but you are
right.
target, found = which("pandoc")
if found:
target will contain the full path to pandoc
else:
pandoc was not found
You want to check for execute permssion too.
For my uses of the above code, it is unlikely that
the 'target' would not have the exec bit set. But,
your suggestion is valid due to the fact that I
should not take anything for granted.
Cheers,
Cameron Simpson
Corrected code:
def which(target)
for p in pathlist:
fullpath = p + "/" + target
if os.path.isfile(fullpath) and os.access(fullpath, os.X_OK):
return fullpath, True
return None, False
Thanks.
The implication was that you don't need the True/False, just the
string/None.
--
https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On Fri, 10 Feb 2017 09:53:32 +1100, Cameron Simpson wrote:
> On 09Feb2017 11:59, Wildman wrote:
>>Here is a method I frequently use to replace the which
>>command. (air code)
>>
>>import os
>>pathlist = os.environ["PATH"].split(":")
>>
>>def which(target)
>>for p in pathlist:
>>fullpath = p + "/" + target
>>if os.path.isfile(fullpath):
>>return fullpath, True
>>return "", False
>
> Cleaner is to return the path if found, or None if not.
I don't see this as an important issue but you are
right.
>>target, found = which("pandoc")
>>if found:
>>target will contain the full path to pandoc
>>else:
>>pandoc was not found
>
> You want to check for execute permssion too.
For my uses of the above code, it is unlikely that
the 'target' would not have the exec bit set. But,
your suggestion is valid due to the fact that I
should not take anything for granted.
> Cheers,
> Cameron Simpson
Corrected code:
def which(target)
for p in pathlist:
fullpath = p + "/" + target
if os.path.isfile(fullpath) and os.access(fullpath, os.X_OK):
return fullpath, True
return None, False
Thanks.
--
GNU/Linux user #557453
Keyboard not detected! Press any key to continue...
--
https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On Thu, Feb 9, 2017 at 10:50 PM, Cameron Simpson wrote: > This is why I suggested the check_returncode() method, which examines the > error code. You must be thinking of the returncode attribute, which isn't a method. check_returncode() is a method of the CompletedProcess object that's returned by subprocess.run(), which was added in 3.5. The OP is using both 3.4 and 3.5, so run() isn't a practical option. The older check_output() function also checks the return code and raises a subprocess.CalledProcessError if the command fails. For example: >>> subprocess.check_output(['which', 'ls']) b'/bin/ls\n' >>> subprocess.check_output(['which', 'pandoc']) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.5/subprocess.py", line 626, in check_output **kwargs).stdout File "/usr/lib/python3.5/subprocess.py", line 708, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['which', 'pandoc']' returned non-zero exit status 1 -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On 09Feb2017 11:59, Wildman wrote:
Here is a method I frequently use to replace the which
command. (air code)
import os
pathlist = os.environ["PATH"].split(":")
def which(target)
for p in pathlist:
fullpath = p + "/" + target
if os.path.isfile(fullpath):
return fullpath, True
return "", False
Cleaner is to return the path if found, or None if not.
target, found = which("pandoc")
if found:
target will contain the full path to pandoc
else:
pandoc was not found
You want to check for execute permssion too.
Cheers,
Cameron Simpson
--
https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On 09Feb2017 11:16, Andreas Paeffgen wrote: I guess which does not return an error code. If it does not find anything, the return is just blank. If it finds something, the path is returned. So the change of code did not help, because there is just no error message. Could there be a $path problem in the subprocess started inside the binary? You're confusing the error code (an integer returned from _every_ exiting process) and the process' stdout and stderr, which are data streams. This is why I suggested the check_returncode() method, which examines the error code. Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
Am 09.02.17 um 18:16 schrieb Andreas Paeffgen: > I guess which does not return an error code. In fact, id does return a return code. Here an example: | honk:~ gdie$ which bash; echo $? | /bin/bash | 0 | honk:~ gdie$ which wzlbrmpf; echo $? | 1 It is 0 on success, 1 for a failure. Exactly the result I would expect from an unixoid program. On Ubuntu 16.04, 'man which' tells of three possible exit states: 0, 1 and 2. On Mac OS, the return codes are not documented. It seems, there are only 0 (success) and 1 (failure). Normally, on unixoid operating systems, return code 0 indicates success, anything else non-success, failure or error. > If it does not find > anything, the return is just blank. If it finds something, the path is > returned. That is the result on STDOUT. It is _not_ the return code! > So the change of code did not help, because there is just no error message. Do you expect an error message, or a return code? An error message would -- normally -- appear on STDERR. > Could there be a $path problem in the subprocess started inside the binary? That's for sure. I already had this problem with py2exe. I solved it by passing a directory containing a suitable PATH entry to the env argument of subprocess.Popen(). Problem gone. If you call 'python myscript.py' on the shell command line, the python script will inherit a copy the shell's environment. If you start a frozen python script, there is no shell's environment, it could inherit. It is up to the tool you used to freeze the script (cx_freeze, py2exe, etc.), which environment contents it will see. Best regards, Günther -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On Thu, 09 Feb 2017 11:16:18 -0600, Andreas Paeffgen wrote:
> I guess which does not return an error code. If it does not find
> anything, the return is just blank. If it finds something, the path is
> returned.
>
> So the change of code did not help, because there is just no error message.
> Could there be a $path problem in the subprocess started inside the binary?
Here is a method I frequently use to replace the which
command. (air code)
import os
pathlist = os.environ["PATH"].split(":")
def which(target)
for p in pathlist:
fullpath = p + "/" + target
if os.path.isfile(fullpath):
return fullpath, True
return "", False
target, found = which("pandoc")
if found:
target will contain the full path to pandoc
else:
pandoc was not found
--
GNU/Linux user #557453
The cow died so I don't need your bull!
--
https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
I guess which does not return an error code. If it does not find anything, the return is just blank. If it finds something, the path is returned. So the change of code did not help, because there is just no error message. Could there be a $path problem in the subprocess started inside the binary? -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
Maybe i could use another trick to circumvent the problems in the frozen app? The frozen apps can be downloaded here: https://sourceforge.net/projects/panconvert/files/Newest/ @Cameron: 1. I use PyQT5 for a creating a gui app. To run the app on other systems, where no QT5 and PyQT5 is installed, a bundle is created with all relevant libraries, so it will start independently. This app bundle, is called a frozen app. Not only the source code, but all it dependencies are distributed including python3 itself. 2. In the test-environment pandoc is installed in /usr/local/bin/pandoc (Mac) or /bin/pandoc (Linux). Which returns the path correctly, at the terminal (shell), in Python and in the Gui-App Panconvert. 3. I am aware of the limitations of which. There is a fallback to manually insert the path. So the app will function anyway. And i want to avoid to search the whole filesystem. This takes minutes on my test machines, so this is no option. 4. I suppose that something is different from executing the code in the frozen state. The app e.g. on Mac is not started from a python shell or a comand shell. But even on linux, when started from a shell it does not work. 5. I will try the hint with dev.null and the p.check_returncode() @Wolfgang 1. The function should be executed only, if the path is unknown. If the user enters the path manually in the settings and pandoc exists, the function should not be executed to save computation time. 2. Only if the manually entered path is not correct or empty the function should be executed, hence 'if not os.path.isfile(path_pandoc)' 3. The function fills in the path automatically if which returns a value. This works in the source code -- https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On 09.02.2017 01:56, Andreas Paeffgen wrote:
The Problem with the subprocess code is: Using the sourcecode
functioning as normal.
The frozen app with cx_freeze on every platform just returns an empty
result
Here is the code in short:
def get_path_pandoc():
settings = QSettings('Pandoc', 'PanConvert')
path_pandoc = settings.value('path_pandoc','')
if not os.path.isfile(path_pandoc):
if platform.system() == 'Darwin' or os.name == 'posix':
args = ['which', 'pandoc']
p = subprocess.Popen(
args,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
path_pandoc =
str.rstrip(p.communicate(path_pandoc.encode('utf-8'))[0].decode('utf-8'))
The whole problematic code can be checked on
http://github.com/apaeffgen/panconvert
in source/converters/interface_pandoc.py
Checking your repo I found that get_path_pandoc, the function from which
you took the code snippet above, will always return None if
os.path.isfile(path_pandoc).
This is probably not what you are intending. Do you know if path_pandoc
is maybe set to an existing file in your frozen code already so the
whole 'which' or 'where' branch is never executed?
Best,
Wolfgang
--
https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On 09.02.2017 01:56, Andreas Paeffgen wrote:
The Problem with the subprocess code is: Using the sourcecode
functioning as normal.
The frozen app with cx_freeze on every platform just returns an empty
result
Here is the code in short:
def get_path_pandoc():
settings = QSettings('Pandoc', 'PanConvert')
path_pandoc = settings.value('path_pandoc','')
if not os.path.isfile(path_pandoc):
if platform.system() == 'Darwin' or os.name == 'posix':
args = ['which', 'pandoc']
p = subprocess.Popen(
args,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
path_pandoc =
str.rstrip(p.communicate(path_pandoc.encode('utf-8'))[0].decode('utf-8'))
The whole problematic code can be checked on
http://github.com/apaeffgen/panconvert
in source/converters/interface_pandoc.py
Checking your repo I found that get_path_pandoc, the function from which
you took the code snippet above, will always return None if
os.path.isfile(path_pandoc).
This is probably not what you are intending. Do you know if path_pandoc
is maybe set to an existing file in your frozen code already so the
whole 'which' or 'where' branch is never executed?
Best,
Wolfgang
--
https://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
On 08Feb2017 18:56, Andreas Paeffgen wrote:
The Problem with the subprocess code is: Using the sourcecode
functioning as normal.
The frozen app with cx_freeze on every platform just returns an empty result
I don't know what the sentence above above cx_freeze means.
Here is the code in short:
def get_path_pandoc():
settings = QSettings('Pandoc', 'PanConvert')
path_pandoc = settings.value('path_pandoc','')
if not os.path.isfile(path_pandoc):
if platform.system() == 'Darwin' or os.name == 'posix':
args = ['which', 'pandoc']
p = subprocess.Popen(
args,
stdin=subprocess.PIPE,
"which" doesn't read any input. You probably want "subprocess.DEVNULL" instead
of "subprocess.PIPE" for stdin.
stdout=subprocess.PIPE)
path_pandoc =
str.rstrip(p.communicate(path_pandoc.encode('utf-8'))[0].decode('utf-8'))
The whole problematic code can be checked on
http://github.com/apaeffgen/panconvert
in source/converters/interface_pandoc.py
I debugged it with some QMessage-Boxes. I just know, that the returnd
result is empty.
I tried some stderr on this code also. But also the error message is empty.
What does "which pandoc" say to you at a terminal prompt? The code above relies
on that returning you the path to the pandoc executable. If it is not in your
path then you will get an empty result.
Also, the "which" programme should have a zero exit status if it finds and
prints the path to pandoc, and a nonzero exit status otherwise? Find out what
exit status it returns. This is available from the "p" object after
communicate() returns. See the subprocess doco for details. The easy thing to
do is probably to call "p.check_returncode()" after communicate() returns.
Any hints what could be changed in the code, so it also works in the
frozen app?
What's a frozen app?
Maybe that is your problem: when you run the programme by hand it works, but in
this "forzen" state it doesn't.
The "which" command isn't magic. It searchs for commands by looking for
executables in the directories listed in the $PATH environment variable. Get
your program to print or display the value of "os.environ['PATH']" before
invoking "which". The difference may tell you something useful.
At the moment i use python3.4 to 3.5 depending on the platform (Mac,
Win, Linux)
Thank you for this information; many people forget to supply it.
P.S. Tried also some code with invoking the bash and afterwords the which
pandoc code. To know avail also
From a shell prompt (such as bash's prompt) issuing the command:
which pandoc
should print the executable path, _if_ your $PATH includes the right directory.
You can look at $PATH from the shell with the command:
echo $PATH
Do you know where pandoc normally resides on your system?
Cheers,
Cameron Simpson
--
https://mail.python.org/mailman/listinfo/python-list
subprocess problem
The Problem with the subprocess code is: Using the sourcecode
functioning as normal.
The frozen app with cx_freeze on every platform just returns an empty result
Here is the code in short:
def get_path_pandoc():
settings = QSettings('Pandoc', 'PanConvert')
path_pandoc = settings.value('path_pandoc','')
if not os.path.isfile(path_pandoc):
if platform.system() == 'Darwin' or os.name == 'posix':
args = ['which', 'pandoc']
p = subprocess.Popen(
args,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE)
path_pandoc =
str.rstrip(p.communicate(path_pandoc.encode('utf-8'))[0].decode('utf-8'))
The whole problematic code can be checked on
http://github.com/apaeffgen/panconvert
in source/converters/interface_pandoc.py
I debugged it with some QMessage-Boxes. I just know, that the returnd
result is empty.
I tried some stderr on this code also. But also the error message is empty.
Any hints what could be changed in the code, so it also works in the
frozen app?
At the moment i use python3.4 to 3.5 depending on the platform (Mac,
Win, Linux)
P.S. Tried also some code with invoking the bash and afterwords the
which pandoc code. To know avail also
--
https://mail.python.org/mailman/listinfo/python-list
Re: Subprocess Problem (Wait and while)
On 09/08/2010 17:08, Alban Nona wrote:
Hi,
I have some problem with my actual code.
In fact, the script is done to work within nuke from the foundry which is a
compositing software.
Homever, I have some code difficulties as I quite new in the area.
Here the deal:
Im using subprocess command to copy some files into local directory, like
this:
cmd = ['xcopy', '/E', '/I', '/Q', '/Y', '%s' % _folder.replace('/',
'\\'), '%s' % _temp.replace('/', '\\')]
subprocess.Popen(cmd, shell=True)
Its copying quite well. I have no problem with that. But, after the copy I
would like to continue my code, but ONLY if the copy is done.
If Im using a WAIT, its freezing the main program (nuke) until the copy in
done. I want to avoid that. I also try "call" but same thing. Its
freezing...
I though about something like:
while subprocess not finished:
Do nothing
else:
Continue Program
Have a look at the .poll method of subprocess.Popen
Like this its checking if the files is copying and do nothing until its
done. I also tough:
while os.listdir(src) != os.listdir(dst):
Do nothing
Else:
Continue program
That's unlikely to work because Windows will create the directory
entry first (and at the complete size) and will then fill it in
as it copies.
TJG
--
http://mail.python.org/mailman/listinfo/python-list
Subprocess Problem (Wait and while)
Hi,
I have some problem with my actual code.
In fact, the script is done to work within nuke from the foundry which is a
compositing software.
Homever, I have some code difficulties as I quite new in the area.
Here the deal:
Im using subprocess command to copy some files into local directory, like
this:
cmd = ['xcopy', '/E', '/I', '/Q', '/Y', '%s' % _folder.replace('/',
'\\'), '%s' % _temp.replace('/', '\\')]
subprocess.Popen(cmd, shell=True)
Its copying quite well. I have no problem with that. But, after the copy I
would like to continue my code, but ONLY if the copy is done.
If Im using a WAIT, its freezing the main program (nuke) until the copy in
done. I want to avoid that. I also try "call" but same thing. Its
freezing...
I though about something like:
while subprocess not finished:
Do nothing
else:
Continue Program
Like this its checking if the files is copying and do nothing until its
done. I also tough:
while os.listdir(src) != os.listdir(dst):
Do nothing
Else:
Continue program
But Im not sure its quite good solution.
Someone can help me with this one please ?
Thank you ! :)
--
http://mail.python.org/mailman/listinfo/python-list
Re: socket and subprocess problem
On Thu, Dec 18, 2008 at 8:00 PM, Bryan Olson wrote: > I'd swear James copied my response, except his came first. Even the > formatting came out similar. I hadn't seen his response when I wrote mine, > and wouldn't have bothered posing the same thing again. Great minds think alike huh :) You should check out my circuits (1) library! cheers James 1: http://trac.softcircuit.net.au/circuits/ -- http://mail.python.org/mailman/listinfo/python-list
Re: socket and subprocess problem
James Mills wrote: subprocess process: #1. When my subprocess process has successfully started notify the parent. #2. When my subprocess process has successfully created a listening socket, notify the parent. parent process: #1. When our subprocess process has successfully started a listening socket initiate a connection. I'd swear James copied my response, except his came first. Even the formatting came out similar. I hadn't seen his response when I wrote mine, and wouldn't have bothered posing the same thing again. -- --Bryan -- http://mail.python.org/mailman/listinfo/python-list
Re: socket and subprocess problem
goat...@gmail.com wrote: Guys thanks to point it out. Yes, it's a race problem. I tried sleep long enough, then I can connect to the socket. I should add code to try to connect to the socket for a given time out. As Roy noted, that's "the cheesy way". Are the kind of programmers who accept cheesy solutions? Of course not. The right solution is for the child process to tell the parent when the port is ready for connections. There are a variety of ways to transfer that message; the most straightforward is for the child process to write something to its standard output. Parent process: Launch child. read() child's output, waiting for the ready message. connect() to the port the child established. Child process: Create the socket; bind(); listen(). Write the ready message to stdout. accept(). There's a subtle yet important point here, that frequently gets lost: Roy Smith wrote: Sounds like a timing problem. I assume that the process started by a.run() creates a socket and does a bind/listen/accept sequence on it. The problem is, there's nothing in your code which guarantees that this happens before b.run() executes the connect() call. Actually, it's just bind() and listen() that have to happen before the client can connect(); accept() will not succeed until after a client connects. In our example the child process is the server, and the server's readiness alert should go after his call to listen() returns, but before he calls accept(). -- --Bryan -- http://mail.python.org/mailman/listinfo/python-list
Re: socket and subprocess problem
On Tue, Dec 16, 2008 at 3:30 PM, wrote: > Guys thanks to point it out. > Yes, it's a race problem. I tried sleep long enough, then I can > connect to the socket. I should add code to try to connect to the > socket for a given time out. This is where event-driven approaches become really useful :) subprocess process: #1. When my subprocess process has successfully started notify the parent. #2. When my subprocess process has successfully created a listening socket, notify the parent. parent process: #1. When our subprocess process has successfully started a listening socket initiate a connection. I could implement a prototype of this if the OP is interested. --JamesMills -- http://mail.python.org/mailman/listinfo/python-list
Re: socket and subprocess problem
Guys thanks to point it out. Yes, it's a race problem. I tried sleep long enough, then I can connect to the socket. I should add code to try to connect to the socket for a given time out. Roy Smith wrote: > In article > <6d3291c3-4e12-4bdd-884a-21f15f38d...@a12g2000pro.googlegroups.com>, > goat...@gmail.com wrote: > > > In my python code I use subprocess.Popen to run and external program > > who will listen to a TCP port. And I also create a socket to connect > > to the TCP port that the external program is listening. > > I will get 'Connection refused, errno=111' when I try to > > socket.connect(). > > > Class a: > > def run() > > subprocess.Popen(..) > > Class b: > > def run(): > > sock = socket.socket() > > sock.connect(..) > > # > > test.py > > # socket connect will fail here > > a.run() > > b.run() > > ### > > test1.py > > if __name__ = '__main__': > > a.run() > > > > test2.py > > # socket will connect fine > > if __name__ = '__main__': > > b.run > > Sounds like a timing problem. I assume that the process started by a.run() > creates a socket and does a bind/listen/accept sequence on it. The problem > is, there's nothing in your code which guarantees that this happens before > b.run() executes the connect() call. > > The cheesy way to test this is to sleep for a second somewhere between > a.run() and b.run(). See if that helps. > > If it doesn't, then it's possible the process started by a.run() isn't > doing what it's supposed to do. Try running test1.py, and while it's > running, run netstat to see if you've got something listening on the port > you expect. -- http://mail.python.org/mailman/listinfo/python-list
Re: socket and subprocess problem
In article <6d3291c3-4e12-4bdd-884a-21f15f38d...@a12g2000pro.googlegroups.com>, goat...@gmail.com wrote: > In my python code I use subprocess.Popen to run and external program > who will listen to a TCP port. And I also create a socket to connect > to the TCP port that the external program is listening. > I will get 'Connection refused, errno=111' when I try to > socket.connect(). > Class a: > def run() > subprocess.Popen(..) > Class b: > def run(): > sock = socket.socket() > sock.connect(..) > # > test.py > # socket connect will fail here > a.run() > b.run() > ### > test1.py > if __name__ = '__main__': > a.run() > > test2.py > # socket will connect fine > if __name__ = '__main__': > b.run Sounds like a timing problem. I assume that the process started by a.run() creates a socket and does a bind/listen/accept sequence on it. The problem is, there's nothing in your code which guarantees that this happens before b.run() executes the connect() call. The cheesy way to test this is to sleep for a second somewhere between a.run() and b.run(). See if that helps. If it doesn't, then it's possible the process started by a.run() isn't doing what it's supposed to do. Try running test1.py, and while it's running, run netstat to see if you've got something listening on the port you expect. -- http://mail.python.org/mailman/listinfo/python-list
Re: socket and subprocess problem
goat...@gmail.com wrote: In my python code I use subprocess.Popen to run and external program who will listen to a TCP port. And I also create a socket to connect to the TCP port that the external program is listening. I will get 'Connection refused, errno=111' when I try to socket.connect (). Looks like a race. The first process tries to connect before the external program gets through socket.listen(). If you think about it, it's not that surprising. But if I run my subprocess.Popen code and socket code in two separate python process. Socket will connect just fine. It's still a race condition even if the side you want to win almost always does. -- --Bryan -- http://mail.python.org/mailman/listinfo/python-list
socket and subprocess problem
Hi all, Here is my problem, see if any one else met this before In my python code I use subprocess.Popen to run and external program who will listen to a TCP port. And I also create a socket to connect to the TCP port that the external program is listening. I will get 'Connection refused, errno=111' when I try to socket.connect (). But if I run my subprocess.Popen code and socket code in two separate python process. Socket will connect just fine. OS is RHEL5 x86_64, python version is 2.6.1 Pseudo code as below Class a: def run() subprocess.Popen(..) Class b: def run(): sock = socket.socket() sock.connect(..) # test.py # socket connect will fail here a.run() b.run() ### test1.py if __name__ = '__main__': a.run() test2.py # socket will connect fine if __name__ = '__main__': b.run -- http://mail.python.org/mailman/listinfo/python-list
Subprocess problem on multiple OS's
Subprocess issues with platform independence
Postby ajamin on Wed Oct 08, 2008 10:46 am
I am writing a python script that will act as a wrapper for another
program. The python script will provide the inputs for this program
and will verify that the output is correct. The application runs on
multiple OS's including windows and *nix.
This is the code:
Code: Select all
Help with Code Tags
python Syntax (Toggle Plain Text)
1.
import os, sys, string, time, subprocess
2.
command = "OK\r\n"
3.
p = subprocess.Popen( ("C:\setup-winnt.exe", "-console"),
4.
stdin = subprocess.PIPE,
5.
stdout = subprocess.PIPE,
6.
stderr = subprocess.PIPE)
7.
stdout_text, stderr_text = p.communicate(command)
8.
sys.stdout.writelines(stdout_text.rstrip())
import os, sys, string, time, subprocess command = "OK\r\n" p =
subprocess.Popen( ("C:\setup-winnt.exe", "-console"), stdin =
subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
stdout_text, stderr_text = p.communicate(command)
sys.stdout.writelines(stdout_text.rstrip())
On the *nix version, the application to open ("C:\setup-winnt.exe")
and the command ("OK\r\n") are different. The plan is to refactor this
as the code develops.
The above code works fine on *nix. On these OS's the application
launches in the same console that the command is issued from. On
windows the behavior is different. When the command is executed an
initialization window opens. When this window closes, a command window
is opened. It is this command window that I wish to send commands to.
I believe this command window may be started as a child process of the
application. However it is opened, it does not accept input from the
stdin of the process.
Any suggestions?
--
http://mail.python.org/mailman/listinfo/python-list
Re: python 2.5 scripting in vim on windows: subprocess problem
On 22/10/2007, Andy Kittner <[EMAIL PROTECTED]> wrote: > >> Are you running this on vim or gvim? If you are running on gvim, my > >> guess is that the handles that you are passing are not valid. In > >> either case, try creating explicit handles that are valid (such as for > >> /dev/null) and create the process with these handles. > Just as a side note: my vim was a gvim-7.1.140 with dynamic python > support, so it doesn't look like a general problem. I've also tried *-1-140 from cream's sourceforge website and it works just like my custom-built one. > > I'm passing hadles that I get from subprocess.Popen. Just passing > > command to Popen constructor and using his handles to read data. No > > other handle-manipulations. > When exactly does it throw the exception? Directly on creation of the > Popen object, or when you try to read stdout? It throws exception on subprocess.Popen object instantiation. os.system() works fine but I want Popen functionality. -- http://mail.python.org/mailman/listinfo/python-list
Re: python 2.5 scripting in vim on windows: subprocess problem
Hi!
VIM can, also, to be OLE-COM-server.
Try with:
from win32com.client.dynamic import Dispatch
vim = Dispatch('Vim.Application')
(+google)
@-salutations
Michel Claveau
--
http://mail.python.org/mailman/listinfo/python-list
python 2.5 scripting in vim on windows: subprocess problem
Hello! I'm using subprocess.Popen in python script in vim. It called this way: def some_func(): p = subprocess.Popen( command , stdout = subprocess.PIPE, stderr = subprocess.STDOUT) while True: s = p.stdout.readline() if not s: break self.__output( '... %s' % s ) return p.wait() It filters command's output and re-ouputs it in stdout. Being called from console, it works fine. Being called from vim with :python some_func() it says: file "...subprocess.py", line 586 in __init__ ... file "...subprocess.py", line 699, in _get_handles ... file "...subprocess.py", line 744 in _make_inheritable DUPLICATE_SAME_ACCESS WindowsError: [Error 6] -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem on WinXP
Simon Forman schrieb: > Wolfgang wrote: >> Hi Simon, >> >> I did not know that library! I'm still new to python and I still have >> problems to find the right commands. > > Welcome. : ) Python comes with "batteries included". I'm always > finding cool new modules myself, and I've been using it for years. In > fact, I didn't notice the bz2 module until about a week ago. > the main problem is to find out how to use all these nice tools ;-) > > Read the docs. There seems to be api for (de)compressing both > "streams" of data and whole files. Understanding the docs is the next issue! for example the subprocess module: there is startupinfo=None, creationflags=0 but absolutely no details what flags are valid! I suppose these values are passed through without any checking and for a stupid user (like me) it is impossible to find out how to use this. > > I don't know about performance, as I've never tried to use the module > before, but I would bet that it's good. It almost certainly uses the > same bzip2 library as the bzip2 program itself and it avoids the > overhead of creating a new process for each file. > > But if you're in doubt (and performance really matters for this > application) test and measure it. I've read about measuring the performance somewhere in the mailing list but I doubt that I can implement an algorithm without knowing the file structure of the (to be compressed) files. I will test your script tomorrow (but my one is running for more than an hour now (on 16GB of data) Cheers Wolfgang -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem on WinXP
Wolfgang wrote: > Hi Simon, > > I did not know that library! I'm still new to python and I still have > problems to find the right commands. Welcome. : ) Python comes with "batteries included". I'm always finding cool new modules myself, and I've been using it for years. In fact, I didn't notice the bz2 module until about a week ago. Browse the standard library docs for fun: http://docs.python.org/lib/lib.html there's all kinds of cool stuff in there. Whenever you say to yourself, "Hmm, somebody must have had this problem before," reach for the standard library. The solution's likely already in there. > > But I suppose this library is mainly for partially > compressing/decompressing of files. How can I use that library to > compress/decompress full files without reading them into memory? And > what about performance? Read the docs. There seems to be api for (de)compressing both "streams" of data and whole files. I don't know about performance, as I've never tried to use the module before, but I would bet that it's good. It almost certainly uses the same bzip2 library as the bzip2 program itself and it avoids the overhead of creating a new process for each file. But if you're in doubt (and performance really matters for this application) test and measure it. I think your script could be rewritten as follows with good speed and memory performance, but I haven't tested it (and the output filepaths may not be what you want...): import os import bz2 dir_ = r"g:\messtech" for root, dirs, files in os.walk(dir_): for file_ in files: f = os.path.join(root, file_) bzf = os.path.join(f, '.bz2') F = open(f) BZF = BZ2File(bzf, 'w') try: for line in F: BZF.write(line) finally: F.close() BZF.close() Also, note that I changed 'dir' and 'file' to 'dir_' and 'file_'. Both dir and file are python built-ins, so you shouldn't reuse those names for your variables. Peace, ~Simon -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem on WinXP
Hi Simon, I did not know that library! I'm still new to python and I still have problems to find the right commands. But I suppose this library is mainly for partially compressing/decompressing of files. How can I use that library to compress/decompress full files without reading them into memory? And what about performance? Thanks Wolfgang Simon Forman schrieb: > Wolfgang wrote: >> Hi, >> >> I want to compress all files (also in subfolder). The code is working >> more or less, but I get a black popup window (command line window) for >> every file to compress. How do I cave to change my system call that >> nothing pops up? >> >> Wolfgang >> >> import os >> import subprocess >> >> dir="g:\\messtech" >> >> for root, dirs, files in os.walk(dir): >> for file in files: >> f=os.path.join(root,file) >> subprocess.Popen([r"bzip2",'', f],shell=False).wait() > > How about forgetting the system call and just using the bz2 standard > library module? > > http://docs.python.org/lib/module-bz2.html > > Peace, > ~Simon > -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem on WinXP
Wolfgang wrote: > Hi, > > I want to compress all files (also in subfolder). The code is working > more or less, but I get a black popup window (command line window) for > every file to compress. How do I cave to change my system call that > nothing pops up? > > Wolfgang > > import os > import subprocess > > dir="g:\\messtech" > > for root, dirs, files in os.walk(dir): > for file in files: > f=os.path.join(root,file) > subprocess.Popen([r"bzip2",'', f],shell=False).wait() How about forgetting the system call and just using the bz2 standard library module? http://docs.python.org/lib/module-bz2.html Peace, ~Simon -- http://mail.python.org/mailman/listinfo/python-list
subprocess problem on WinXP
Hi, I want to compress all files (also in subfolder). The code is working more or less, but I get a black popup window (command line window) for every file to compress. How do I cave to change my system call that nothing pops up? Wolfgang import os import subprocess dir="g:\\messtech" for root, dirs, files in os.walk(dir): for file in files: f=os.path.join(root,file) subprocess.Popen([r"bzip2",'', f],shell=False).wait() -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem in cygwin with Tkinter
that's it! Thanks, that sorted me out. The readme at the following location was very helpful: http://www.tishler.net/jason/software/rebase/rebase-2.2.README I couldn't get rebaseall to work until I installed all of the packages mentioned in the readme. Now I have a different problem, regarding Pmw menus, but I've posted a separate item on that. thanks again. -- http://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem in cygwin with Tkinter
Stewart Midwinter wrote: > [...] > I'm using this version of Cygwin: > $ uname -a > CYGWIN_NT-5.1 Mulata 1.5.18(0.132/4/2) 2005-07-02 20:30 i686 unknown unknown > Cyg > win > [...] > When I run the same command in a Tkinter app, I get an exception: > > [EMAIL PROTECTED] /cygdrive/c/programs/pipeworksb/lib/python/popt > $ python test_subprocess.py > d:\cygwin\bin\python2.4.exe (7752): *** unable to remap > d:\cygwin\bin\tk84.dll t > o same address as parent(0x18C4) != 0x18C5 > 9 [main] python 4696 fork_parent: child 7752 died waiting for dll > loading > [...] > any ideas? This is a dll problem that sometimes happens in Cygwin. Run 'rebaseall' to fix this. HTH, AdSR -- http://mail.python.org/mailman/listinfo/python-list
subprocess problem in cygwin with Tkinter
this has me puzzled; I've created a small test app to show the problem
I'm having.
I want to use subprocess to execute system commands from inside a
Tkinter app running under Cygwin.
When I open a python interpreter and run my subprocess command, all is
well. But when I run the same command from inside a Tkinter app, I'm
getting errors.
I'm using this version of Cygwin:
$ uname -a
CYGWIN_NT-5.1 Mulata 1.5.18(0.132/4/2) 2005-07-02 20:30 i686 unknown unknown Cyg
win
When I run my command in a python shell, all is well:
[EMAIL PROTECTED] /cygdrive/c/programs/pipeworksb/lib/python/popt
$ python
Python 2.4.1 (#1, May 27 2005, 18:02:40)
[GCC 3.3.3 (cygwin special)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> dirlist = subprocess.Popen('/usr/bin/ls', shell=True, bufsize=2048, stdout=
subprocess.PIPE).stdout
>>> dirlist
', mode 'rb' at 0x49d848>
>>> for dir in dirlist:
... print dir
...
__init__.pyc
browser.bat
test_subprocess.py
test_subprocess.py.bak
>>>
When I run the same command in a Tkinter app, I get an exception:
[EMAIL PROTECTED] /cygdrive/c/programs/pipeworksb/lib/python/popt
$ python test_subprocess.py
d:\cygwin\bin\python2.4.exe (7752): *** unable to remap d:\cygwin\bin\tk84.dll t
o same address as parent(0x18C4) != 0x18C5
9 [main] python 4696 fork_parent: child 7752 died waiting for dll loading
Traceback (most recent call last):
File "test_subprocess.py", line 19, in ?
l = Tkinter.Label(text=dir_list())
File "test_subprocess.py", line 8, in dir_list
dirlist = subprocess.Popen('/usr/bin/ls', shell=True, bufsize=2048, stdout=s
ubprocess.PIPE).stdout
File "/usr/lib/python2.4/subprocess.py", line 558, in __init__
errread, errwrite)
File "/usr/lib/python2.4/subprocess.py", line 918, in _execute_child
self.pid = os.fork()
OSError: [Errno 2] No such file or directory
Here's a sample Tkinter app that reproduces the problem:
#test_subprocess.py
import subprocess, os
import Tkinter
def dir_list():
'''provide a listing of files in a directory'''
#need to know which system to draw from
curdir = os.getcwd()
dirlist = subprocess.Popen('/usr/bin/ls', shell=True,
bufsize=2048, stdout=subprocess.PIPE).stdout
#dirlist = subprocess.Popen(['ls',' -l'],
stdout=subprocess.PIPE).stdout.communicate()[0]
print dirlist
msgList = []
for dir in dirlist:
print dir
msgList.append(dir)
msg = '\n'.join(msgList)
return msg
root = Tkinter.Tk()
l = Tkinter.Label(text=dir_list())
l.pack()
b = Tkinter.Button(text='ok',command=root.destroy)
b.pack()
root.mainloop()
BTW, the test_subprocess.py test script that ships with Cygwin python runs fine.
any ideas?
thanks,
--
Stewart Midwinter
[EMAIL PROTECTED]
[EMAIL PROTECTED]
Skype, GoogleTalk, iChatAV, MSN, Yahoo: midtoad
AIM:midtoad1
--
http://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
Hi !
Thank you very much.
With your tip, this script :
import os
p = os.popen4(r'cmd /k')
p[0].write('dir *.bat /B\r\n')
p[0].flush()
p[0].write('dir *.cfg \r\n')
p[0].flush()
p[0].write('exit\r\n')
p[0].flush()
print ''.join(p[1].readlines())
run perfectly
@-salutations
Michel Claveau
--
http://mail.python.org/mailman/listinfo/python-list
Re: subprocess problem
Hi, I had a similar problem recently, and found that using pipes with os.popen* helped in my case. You can check the thread at: http://mail.python.org/pipermail/python-list/2005-September/300744.html. Cheers, Uri Do Re Mi chel La Si Do wrote: > Hi! > > This script (under Win-XP + P-2.4.1) : > import subprocess > p1=subprocess.Popen(r'cmd /cdir *.* /S /W /B', stdout=subprocess.PIPE) > chaineretour=p1.stdout.read() > > run OK if called from DOS-console. But, from another Python's script (by COM > + exec) give me an error. > > Here, the traceback : > > Traceback (most recent call last): > File "D:\dev\Python\ponx.py", line 4446, in PRun > exec(ccod,globals(),globals()) > File "C:\Python24\lib\subprocess.py", line 546, in __init__ >(p2cread, p2cwrite, > File "C:\Python24\lib\subprocess.py", line 606, in __get_handles__ >p2cread = self._make_inheritable(p2cread) > File "C:\Python24\lib\subprocess.py", line 647, in _make_inheritable >DUPLICATE_SAME_ACCESS) > error: (6, 'DuplicateHandle','Descripteur non valide') > > > I know the bug-signaled : http://python.org/sf/1124861 > > > But... a idea for a solution ? > > Thanks, and sorry for my bad english. > > > Michel Claveau -- http://mail.python.org/mailman/listinfo/python-list
subprocess problem
Hi! This script (under Win-XP + P-2.4.1) : import subprocess p1=subprocess.Popen(r'cmd /cdir *.* /S /W /B', stdout=subprocess.PIPE) chaineretour=p1.stdout.read() run OK if called from DOS-console. But, from another Python's script (by COM + exec) give me an error. Here, the traceback : Traceback (most recent call last): File "D:\dev\Python\ponx.py", line 4446, in PRun exec(ccod,globals(),globals()) File "C:\Python24\lib\subprocess.py", line 546, in __init__ (p2cread, p2cwrite, File "C:\Python24\lib\subprocess.py", line 606, in __get_handles__ p2cread = self._make_inheritable(p2cread) File "C:\Python24\lib\subprocess.py", line 647, in _make_inheritable DUPLICATE_SAME_ACCESS) error: (6, 'DuplicateHandle','Descripteur non valide') I know the bug-signaled : http://python.org/sf/1124861 But... a idea for a solution ? Thanks, and sorry for my bad english. Michel Claveau -- http://mail.python.org/mailman/listinfo/python-list
subprocess problem on Windows in IDLE and PythonWin
Python 2.4 on Windows XP
In the python command-line the following works fine:
>>> from subprocess import *
>>> p = Popen('dir', stdout=PIPE)
>From within IDLE or PythonWin I get the following exception:
Traceback (most recent call last):
File "", line 1, in -toplevel-
p = Popen('dir', stdout=PIPE)
File "c:\python24\lib\subprocess.py", line 545, in __init__
(p2cread, p2cwrite,
File "c:\python24\lib\subprocess.py", line 605, in _get_handles
p2cread = self._make_inheritable(p2cread)
File "c:\python24\lib\subprocess.py", line 646, in _make_inheritable
DUPLICATE_SAME_ACCESS)
TypeError: an integer is required
Note it works fine on Linux also. I tested it with
>>> p = Popen('ls', stdout=PIPE)
... and had no trouble.
--
http://mail.python.org/mailman/listinfo/python-list
