On Wed, Jan 5, 2011 at 22:58, Dimitri Fontaine <dimi...@2ndquadrant.fr> wrote: > Magnus Hagander <mag...@hagander.net> writes: >> Attached is an updated streaming base backup patch, based off the work > > Thanks! :) > >> * Compression: Do we want to be able to compress the backups server-side? Or >> defer that to whenever we get compression in libpq? (you can still tunnel >> it >> through for example SSH to get compression if you want to) My thinking is >> defer it. > > Compression in libpq would be a nice way to solve it, later.
Yeah, I'm pretty much set on postponing that one. >> * Compression: We could still implement compression of the tar files in >> pg_streamrecv (probably easier, possibly more useful?) > > What about pg_streamrecv | gzip > …, which has the big advantage of > being friendly to *any* compression command line tool, whatever patents > and licenses? That's part of what I meant with "easier and more useful". Right now though, pg_streamrecv will output one tar file for each tablespace, so you can't get it on stdout. But that can be changed of course. The easiest step 1 is to just use gzopen() from zlib on the files and use the same code as now :-) >> * Stefan mentiond it might be useful to put some >> posix_fadvise(POSIX_FADV_DONTNEED) >> in the process that streams all the files out. Seems useful, as long as >> that >> doesn't kick them out of the cache *completely*, for other backends as >> well. >> Do we know if that is the case? > > Maybe have a look at pgfincore to only tag DONTNEED for blocks that are > not already in SHM? I think that's way more complex than we want to go here. >> * include all the necessary WAL files in the backup. This way we could >> generate >> a tar file that would work on it's own - right now, you still need to set >> up >> log archiving (or use streaming repl) to get the remaining logfiles from >> the >> master. This is fine for replication setups, but not for backups. >> This would also require us to block recycling of WAL files during the >> backup, >> of course. > > Well, I would guess that if you're streaming the WAL files in parallel > while the base backup is taken, then you're able to have it all without > archiving setup, and the server could still recycling them. Yes, this was mostly for the use-case of "getting a single tarfile that you can actually use to restore from without needing the log archive at all". -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
