Hi David and Matt. I appreciate your help which has got me moving forward again so many thanks for your reply. I have been using subprocess.Popen a fair bit but this was the first time I had to use subprocess to capture large file output. The trouble I was having was with the process would just hang. Chunking was the solution. I guess I assumed this would be taken care of in the internals.
Overall, I wish subprocess had some better documentation since it is definitely not a drop in replacement for os.system. In other circumstances I am using subprocess.call() for simple calls which works fine. The speed of this solution is slower than os.system. Would a queue of some kind be needed to speed this up? Has anyone implemented something like this? Many thanks. Regards, David Matt Nordhoff wrote: > David Pratt wrote: >> Hi. I am trying to replace a system call with a subprocess call. I have >> tried subprocess.Popen and subprocess.call with but have not been >> successful. The command line would be: >> >> svnadmin dump /my/repository > svndump.db >> >> This is what I am using currently: >> >> os.system('svnadmin dump %s > %s' % (svn_dir, >> os.path.join(backup_dir, 'svndump.db'))) >> >> Many thanks. > > Try this: > > import os.path > import subprocess > > p = subprocess.Popen( > ['svnadmin', 'dump', svndir], > stdout=subprocess.PIPE, > ) > > fh = open(os.path.join(backup_dir, 'svndump.db'), 'wb') > > while True: > chunk = p.stdout.read(2**20) # 1 MB > if not chunk: > break > fh.write(chunk) > > fh.close() > > It reads svnadmin's stdout in 1 MB chunks, in case it's large enough > that reading the whole thing into RAM at once would be a bad idea. > > No error handling. For one, you might want to add a try...finally to > ensure that fh will get closed. (Or if you have Python 2.5, use a with > statement! :-) ) Also, Popen will raise an OSError if svnadmin can't be > found or something. And this isn't even considering svnadmin erroring out... > > svnadmin's stderr will go to your stderr. > > I didn't test it, but I'm pretty sure it will work. (I spotted a syntax > error while writing that though.) I don't have much experience with > Popen's stdio objects, so it's possible you'd need to do something like > call p.wait() to wait for it to exit before being able to read its stdout. > > It could be slower than the os.system version, since now Python is doing > all of the I/O, instead of your shell, but I doubt that'll be a big problem. > > (Also, insert suggestion about using a good VCS. ;-) ) -- http://mail.python.org/mailman/listinfo/python-list