Eric Brunson wrote: > I'm coming in late to the discussion and thought that someone would > explain it succinctly, but there have been so many correct statements > which I feel fail to nail down the problem that I thought I'd chime in. >
Hi Eric, Thank you for your considerate response. Your analysis is spot on -- speaking for myself, of course -- I'm mostly just flailing here; not having a very strong grasp of how all the components work together (kernel, shell, python, and the subprocess module). But I'm really enjoying this thread and learning a lot in the 'process' ... ahem, sorry for the pun ... painful :) > Here's the important concepts to understand. The pipe is a construct of > the shell, you are spawning a shell to run your entire command line. > When the "make" finishes and stops accepting the input from "yes", the > kernel sends a SIGPIPE to the parent process, i.e. the shell. The > shell's default behavior when receiving the SIGPIPE is to print an error > message. You're right. Considering the shell is definitely important and I missed it. Yet given the following, slightly adapted from the subprocess module docs (your reference link below) ... from subprocess import Popen, PIPE p1 = Popen(["yes", "Hello"], stdout=PIPE) p2 = Popen(["head", "-n", "10"], stdin=p1.stdout, stdout=PIPE) output = p2.communicate()[0] ... still produces ... yes: standard output: Broken pipe yes: write error If run as a script, the message above shows up in the terminal after the python script terminates (or apparently so; shell prompt first, then error). And when using the interactive interpreter... nothing until exiting the interpreter, when the message again appears in the terminal. Perhaps this is some peculiarity of my environment, or more likely I am still missing something. > Your problem, as I interpret it, is that you don't like seeing the error > message. So you have two choices: 1) find some way to filter it, > because all it is is a message, or 2) get rid of the shell and handle > the management of the subprocesses yourself. It's been discussed how to > suppress the output and it's been discussed how to intercept the signal, > but I don't thing that anyone has pointed to the definitive python > construct to handle it properly. > > You've had several responses that correctly tell you how to handle a > signal, but without any testing I'm pretty sure that none of that will > do you any good with the subprocess.call() construct you're using. The > shell is your subprocess and it is already handling the signal, so you > should never see it. The trick is to spawn both processes via python > *without* an intervening shell to interfere. Here's the canonical > example: http://docs.python.org/lib/node536.html. You can spawn each > subprocess, 'yes' and 'make' without the shell being required to pipe > the output of one to the other, you can do it all completely in python. Based on my own tests, I don't think it makes a difference. I suspect the python override for sigpipe is inherited by all child processes, shell or otherwise, and the SIG_IGN is allowing 'yes' to see the broken pipe, report it and terminate -- where the default action would normally be to terminate only. This is, of course, not that far from wild speculation. Though, I am curious if others observe the same behavior. Then again, the error is *not* displayed for any of tests where signal.signal(signal.SIGPIPE, signal.SIG_DFL) is added, in either the global scope or in the form of a preexec_fn argument passed to subprocess, with shell=True or without. IMHO, this is certainly a tripping point if true. Granted, probably a fringe case and relatively rare since it requires spawning a subprocess, which writes to a pipe that is broken before the subprocess ends, and is not implemented to handle the event gracefully. But I still find it weirdly intriguing. > > Again, sorry to come in late, but while reading many absolutely factual <off-topic-blather> As far as I'm concerned, no apologies are necessary -- not that you are apologizing to me necessarily :) But nonetheless, thank you for helping. In fact, a big thank you to all the tutors. I've been a member of the list for quite a few years now, reaping the benefits of your collective experience (like many others I would imagine), mostly by lurking. You provide a tremendous service to the python community, especially those new to python or programming in general, with near super-human patience and skill. Quite a feat! FWIW, thanks. </off-topic-blather> Marty > an correct responses, they all seemed to be either handling the output > or discussing signal handling without, in my mind, pointing out that > it's the shell that's giving you the problems. I think it's Python's > default behavior to ignore SIGPIPE, as Martin comments on in his latest > reply. It's just that the shell has already accepted and handled the > signal. > > Interestingly, when you eliminate the shell and use something similar to > the example code from above, you can pretty easily see how to get rid of > the 'yes' and just feed the make subprocess input yourself, further > simplifying the code. > > I'll admit, I've stated all this without actually running any test, so > please, if anyone can show differently, I'm happy to have my > understanding corrected. > > I hope that helps, not only with your specific problem, but from a "big > picture" standpoint, also. Nothing I hate more than writing code that I > don't really understand. :-) > > e. > > James wrote: >> Hi, >> >> I have a snippet of code in a Python script I'm whipping up that's >> causing a not-so-pretty output. Here's the code: >> >> subprocess.call( "yes '' | make oldconfig" , shell=True ) >> >> When I run this code, Python loyally executes the command, and then I >> see the following error on my console: >> >> ----- >> >> yes: standard output: Broken pipe >> yes: write error >> >> ----- >> >> I did some Googling and I believe found the reason for this error >> ("yes" executes forever, and complains when Python kills the process >> off). However, I'd like to figure out a way to get rid of the error >> (or hide it) so that it's not visible to the person running the >> script (it's not the prettiest thing to see scroll by your screen :)). >> >> Thoughts / ideas? >> >> Thanks! >> .james >> _______________________________________________ >> Tutor maillist - Tutor@python.org >> http://mail.python.org/mailman/listinfo/tutor >> > > _______________________________________________ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
