Nadeem Vawda <nadeem.va...@gmail.com> added the comment: Yes, that's a good idea. I've been testing a similar change, and it seems to drop the peak memory usage for test_tarfile from around 810MB down to under 200MB. It looks like 2GB genuinely isn't enough to reliably use LZMA compression with preset=9.
You might want to use preset=None instead of explicitly saying preset=6, though. This tells LZMAFile to use the default preset, and will allow you to get rid of the if-statement on lines 1821-1823. Something unrelated that I noticed in the surrounding code: gzopen and bz2open validate the mode by testing 'len(mode) > 1 or mode not in "rw"'. This would be simpler as 'mode not in ("r", "w")' (like you've done in xzopen), and it would accept only "r" and "w" (but not "" or "rw"). ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue5689> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com