On Wed, Dec 28, 2016 at 3:12 AM, Magnus Hagander <mag...@hagander.net> wrote: > On Tue, Dec 27, 2016 at 1:16 PM, Michael Paquier <michael.paqu...@gmail.com> > wrote: >> On Tue, Dec 27, 2016 at 6:34 PM, Magnus Hagander <mag...@hagander.net> >> wrote: >> > On Tue, Dec 27, 2016 at 2:23 AM, Michael Paquier >> > <michael.paqu...@gmail.com> >> > wrote: >> >> Magnus, you have mentioned me as well that you had a couple of ideas >> >> on the matter, feel free to jump in and let's mix our thoughts! >> > >> > >> > Yeah, I've been wondering what the actual usecase is here :) >> >> There is value to compress segments finishing with trailing zeros, >> even if they are not the species with the highest representation in >> the WAL archive. > > Agreed on that part -- that's the value in compression though, and not > necessarily the TAR format itself. > > Is there any value of the TAR format *without* compression in your scenario?
Hm. I cannot think of one. >> > I can see the point of being able to compress the individual segments >> > directly in pg_receivexlog in smaller systems though, without the need >> > to >> > rely on an external compression program as well. But in that case, is >> > there >> > any reason we need to wrap it in a tarfile, and can't just write it to >> > <segment>.gz natively? >> >> You mean having a --compress=0|9 option that creates individual gz >> files for each segment? Definitely we could just do that. It would be > > Yes, that's what I meant. OK, I have bitten the bullet and attached is a patch to add just --compress=0|9. So there is one .gz file generated per segment, and things are rather straight-forward. Adding tests is unfortunately not in scope as not all builds are compiled with libz. A couple of things to be aware of though: - gzopen, gzwrite and gzclose are used to handle the gz files. That's unconsistent with the tar method that is based on mainly deflate() and more low level routines. - I have switched the directory method to use a file pointer instead of a file descriptor as gzwrite returns int as the number of uncompressed bytes written. - history and timeline files are gzip'd as well. Perhaps we don't want to do that. What do you think about this approach? I'll add that to the next CF. -- Michael
receivexlog-gzip-v1.patch
Description: invalid/octet-stream
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers