For reference what I'm picturing is this:
When a table is compressed it's marked read-only which bars any new
tuples from being inserted or existing tuples being deleted. Then it's
frozen and any pages which contain tuples wich can't be frozen are
waited on until they can be. When it's finished every tuple has to be
guaranteed to be fully frozen.
Then the relation is rewritten in compressed form. Each block is
compressed one by one and written one after the other to disk.
At the same time a new fork is written which contains a pointer to
each block. It could just be a directly addressed array of offsets and
lengths. All block lookups have to first load the page of the
indirection map, then read the appropriate section of the original
file and decompress it into shared buffers.
I had pondered the idea of a fork storing the compressed status of each
page, because it has advantages :
- no need to change the page layout to insert a "is compressed" flag
- possible to compress any data, not just standard pages
- if you know the compressed size of a page in advance, it is much easier
to prefetch it entirely and not just the first chunk, or read too much...
From a programming point of view this is nice and simple. From a
user's point of view it's a bit of a pain since it means you have to
rewrite your whole table when you want to compress it. And it means
you have to rewrite it all again if you decide you want to set it back
to read-write. My experience with people who have very large tables is
that they design their whole process around the goal of avoiding
having to move the data once it's written.
Note that if a table is huge, it is always cut in (currently) 1GB slices,
so you could operate on one slice at a time, then release a lock, let the
backlog of queries flow, and resume.
Realtime compression would be much less of a hassle to use, though...
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
