Thanks everyone for your replies. Chris, Unfortunately, I can't redirect the output to a file because there are other processes which use this processes output as stdin.
Rob, I will give this a try. On Tue, Feb 22, 2011 at 7:48 PM, Chris Kaynor <ckay...@zindagigames.com>wrote: > > On Tue, Feb 22, 2011 at 3:44 PM, Rita <rmorgan...@gmail.com> wrote: > >> I have a process like this, >> >> def run(cmd): >> #cmd=a process which writes a lot of data. Binary/ASCII data >> p=subprocess.Popen(cmd,stdout=subprocess.PIPE) >> >> I would like to get cmd's return code so I am doing this, >> >> def run(cmd): >> p=subprocess.Popen(cmd,stdout=subprocess.PIPE) >> rc=p.poll() >> print rc # and I get 'None' regardless of what cmd gives me (0 thru 255 >> are valid return codes) >> return p.stdout >> > > Here, you are accessing the return code, however the process has not exited > yet, and thus you get None. > > >> >> >> When using wait() it works a bit better but not consistent >> def run(cmd): >> p=subprocess.Popen(cmd,stdout=subprocess.PIPE) >> rc=p.wait() >> print rc >> return p.stdout >> >> When the output of cmd is a small ascii file it works perfectly fine, but >> when the file is large (more than 2MB) the process just waits for ever (I am >> guessing its blocking?). When I use the communicate call it works perfectly >> but my process is consuming way too much memory. > > > Here, you wait for the process to exit, with wait returning the exit code. > As you have waited for the process, it has exited and thus you get the > return code. > > The issue here is that the stdout pipe has a limited size, and the process > will block once the pipe becomes full. You are entering a dead-lock as the > pipe is full, and thus the sub process is blocking waiting for room to write > more output. As nothing is reading that output, that will never happen. It > works fine on smaller output as you don't hit the limit. See the warning on > http://docs.python.org/library/subprocess.html#popen-objects under wait. > > .communicate works as it reads the output, however it eats up memory as all > of the data has to be buffered in memory. > > A possible work around would be to redirect the output into a file rather > than to a pipe (see the comments about stdin, stdout and stderr at > http://docs.python.org/library/subprocess.html#using-the-subprocess-module). > In this way, the data will be buffered to a file rather than memory. > > >> >> Is there a better way to get my return code consistently efficiently and >> not take up so much memory? >> >> >> >> >> >> >> >> -- >> --- Get your facts first, then you can distort them as you please.-- >> >> -- >> http://mail.python.org/mailman/listinfo/python-list >> >> > > -- > http://mail.python.org/mailman/listinfo/python-list > > -- --- Get your facts first, then you can distort them as you please.--
-- http://mail.python.org/mailman/listinfo/python-list