Re: Popen3 and capturestderr
On Wed, Mar 09, 2005 at 06:17:50AM -, Donn Cave wrote: > Quoth Kenneth Pronovici <[EMAIL PROTECTED]>: > ... > | If ignoreStderr=False, I use popen2.Popen4 so that stderr and stdout are > | intermingled. If ignoreStderr=True, I use popen2.Popen3 with > | capturestderr=True so stderr doesn't appear in the output. This > | functionality exists so I have an equivalent of command-line redirection > | of stderr, i.e. command 2>/dev/null. > ... > | After some digging, I've decided that this behavior probably occurs > | because I am ignoring the pipe.childerr file object. Indeed, if I call > | pipe.childerr.close() right after opening the pipe, my "ls" command that > | had been hanging completes normally. However, other commands which > | actually attempt to write to stderr don't seem to like this very much. > | > | What is the right way to discard stderr when working with a pipe? I > | want to consistently throw it away, and I don't see a good way to do > | this with the popen2 implementation. > > Right, popen2 gives you about 3 options, out of probably dozens that > you could get with shell redirections. On the other hand, the source > is available, and Python is an OOP language, so I assume there is no > reason you can't make a derived class that does just what you want. > In the present case I guess that would mean something like >null = os.open('/dev/null', os.O_RDWR) >os.dup2(null, 0) >os.dup2(null, 2) (depending) >os.close(null) > along with other stuff you can just copy from Popen4. Ah... ok, subclassing is an option I hadn't considered. I'll give that a whirl and see whether I can make it work. KEN -- Kenneth J. Pronovici <[EMAIL PROTECTED]> Personal Homepage: http://www.skyjammer.com/~pronovic/ "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." - Benjamin Franklin, Historical Review of Pennsylvania, 1759 -- http://mail.python.org/mailman/listinfo/python-list
Re: Popen3 and capturestderr
Quoth Kenneth Pronovici <[EMAIL PROTECTED]>: ... | If ignoreStderr=False, I use popen2.Popen4 so that stderr and stdout are | intermingled. If ignoreStderr=True, I use popen2.Popen3 with | capturestderr=True so stderr doesn't appear in the output. This | functionality exists so I have an equivalent of command-line redirection | of stderr, i.e. command 2>/dev/null. ... | After some digging, I've decided that this behavior probably occurs | because I am ignoring the pipe.childerr file object. Indeed, if I call | pipe.childerr.close() right after opening the pipe, my "ls" command that | had been hanging completes normally. However, other commands which | actually attempt to write to stderr don't seem to like this very much. | | What is the right way to discard stderr when working with a pipe? I | want to consistently throw it away, and I don't see a good way to do | this with the popen2 implementation. Right, popen2 gives you about 3 options, out of probably dozens that you could get with shell redirections. On the other hand, the source is available, and Python is an OOP language, so I assume there is no reason you can't make a derived class that does just what you want. In the present case I guess that would mean something like null = os.open('/dev/null', os.O_RDWR) os.dup2(null, 0) os.dup2(null, 2) (depending) os.close(null) along with other stuff you can just copy from Popen4. Donn -- http://mail.python.org/mailman/listinfo/python-list
Popen3 and capturestderr
Hi, I have a problem with a popen2 pipe hanging partway through execution of a command. This happens in my function executeCommand() that is used for every "shell" command execution in my program. Source code for this function is below. My development environment is Debian unstable with Python 2.3.5. Within executeCommand(), output of every executed command is always logged using a Python logger. If outputFile is passed in, then output will also be written into the outputFile, which might have been created using open(), gzip.GzipFile() or bz2.BZ2File(), etc. This functionality exists so I have an equivalent of command-line redirection of stdout, i.e. command > file. If ignoreStderr=False, I use popen2.Popen4 so that stderr and stdout are intermingled. If ignoreStderr=True, I use popen2.Popen3 with capturestderr=True so stderr doesn't appear in the output. This functionality exists so I have an equivalent of command-line redirection of stderr, i.e. command 2>/dev/null. I am using popen2 rather than popen because I want to avoid all of the issues around shell interpolation of arguments, etc. This whole thing has been working pretty well for a while. Then lately, I started using it on commands that produce a large amount of output, and it's begun to hang partway through execution. A good way to reproduce this behavior on my system is to use command "ls -laR /". The function hangs whever the output file reaches a certain size (around 20 MB). When I interrupt the program, the stack trace shows that it's stuck at pipe.fromchild.readline(). After some digging, I've decided that this behavior probably occurs because I am ignoring the pipe.childerr file object. Indeed, if I call pipe.childerr.close() right after opening the pipe, my "ls" command that had been hanging completes normally. However, other commands which actually attempt to write to stderr don't seem to like this very much. What is the right way to discard stderr when working with a pipe? I want to consistently throw it away, and I don't see a good way to do this with the popen2 implementation. Thanks for the help, KEN --- snip --- def executeCommand(command, args, returnOutput=False, ignoreStderr=False, outputFile=None): """ @param command: Shell command to execute in a list like [ "ls", ] @param args: List of arguments to the command, like [ "-laR", "/", ] @param returnOutput: Indicates whether to return the output of the command @param outputFile: File object that all output should be written to. @return: Tuple of C{(result, output)}. """ output = [] fields = command[:] fields.extend(args) if ignoreStderr: pipe = popen2.Popen3(fields, capturestderr=True) else: pipe = popen2.Popen4(fields) pipe.tochild.close() while True: line = pipe.fromchild.readline() if not line: break if returnOutput: output.append(line) if outputFile is not None: outputFile.write(line) outputLogger.info(line[:-1]) if returnOutput: return (pipe.wait(), output) else: return (pipe.wait(), None) --- snip --- -- Kenneth J. Pronovici <[EMAIL PROTECTED]> -- http://mail.python.org/mailman/listinfo/python-list