On Sat, Jan 13, 2007 at 10:36:05PM +0100, Jim Meyering wrote: > Craig Macdonald <[EMAIL PROTECTED]> wrote: > > On some occasions, I have the need to sort extremely large files, but > > which compress well using programs such as gzip or bzip. > ... > > This task has been on the TODO list for some time: > > sort: Compress temporary files when doing large external sort/merges. > This improves performance when you can compress/uncompress faster than > you can read/write, which is common in these days of fast CPUs. > suggestion from Charles Randall on 2001-08-10 > > Just waiting for someone to work on it.
I already wrote a patch to sort quite a while ago that added compression of the temporary files. Unfortunately, with my code, I saw no performance increase, so I discarded the patch. However, if it can fix the problem of running out of space, there may still be some use for it after all. The patch is old and needs some conflicts to be worked out, but I'll get on it and submit ASAP. In the meantime, I've posted the original C code <http://linux.ucla.edu/~dsh/patches/compressed-sort.c> in case you want to check it out and send comments. It uses LZO compression because it was supposed to be fast. I also had patches to the autoconf related stuff, but they seem to be hiding at the moment. That part shouldn't be too hard to rewrite. _______________________________________________ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
