Re: XLog size reductions: smaller XLRec block header for PG17

2024-02-02 Thread Robert Haas
On Fri, Feb 2, 2024 at 8:52 AM Heikki Linnakangas wrote: > To shrink OIDs fields, you could refer to earlier WAL records. A special > value for "same relation as in previous record", or something like that. > Now we're just re-inventing LZ-style compression though. Might as well > use LZ4 or

Re: XLog size reductions: smaller XLRec block header for PG17

2024-02-02 Thread Heikki Linnakangas
On 22/01/2024 19:23, Robert Haas wrote: In the case of this particular patch, I think the problem is that there's no consensus on the design. There's not a ton of debate on this thread, but thread [1] linked in the original post contains a lot of vigorous debate about what the right thing to do

Re: XLog size reductions: smaller XLRec block header for PG17

2024-01-22 Thread Robert Haas
On Mon, Jan 22, 2024 at 5:38 AM Aleksander Alekseev wrote: > I don't think that closing CF entries purely due to inactivity is a > good practice (neither something we did before) as long as there is > code, it applies, etc. There are a lot of patches and few people > working on them. Inactivity

Re: XLog size reductions: smaller XLRec block header for PG17

2024-01-22 Thread vignesh C
On Mon, 22 Jan 2024 at 16:08, Aleksander Alekseev wrote: > > Hi, > > > I'm seeing that there has been no activity in this thread for nearly 4 > > months, I'm planning to close this in the current commitfest unless > > someone is planning to take it forward. > > I don't think that closing CF

Re: XLog size reductions: smaller XLRec block header for PG17

2024-01-22 Thread Aleksander Alekseev
Hi, > I'm seeing that there has been no activity in this thread for nearly 4 > months, I'm planning to close this in the current commitfest unless > someone is planning to take it forward. I don't think that closing CF entries purely due to inactivity is a good practice (neither something we did

Re: XLog size reductions: smaller XLRec block header for PG17

2024-01-20 Thread vignesh C
On Tue, 26 Sept 2023 at 02:09, Matthias van de Meent wrote: > > On Tue, 19 Sept 2023 at 01:03, Andres Freund wrote: > > > > Hi, > > > > On 2023-05-18 19:22:26 +0300, Heikki Linnakangas wrote: > > > On 18/05/2023 17:59, Matthias van de Meent wrote: > > > > It changes the block IDs used to fit in

Re: XLog size reductions: smaller XLRec block header for PG17

2023-09-26 Thread Aleksander Alekseev
Hi, > This scheme is reused later for the XLogRecord xl_tot_len field over > at [0], and FWIW is thus being reused. Sure, it's tailored to this WAL > use case, but IMO we're getting good value from it. We don't use > protobuf or JSON for WAL, we use our own serialization format. Having > some

Re: XLog size reductions: smaller XLRec block header for PG17

2023-09-25 Thread Matthias van de Meent
On Tue, 19 Sept 2023 at 01:03, Andres Freund wrote: > > Hi, > > On 2023-05-18 19:22:26 +0300, Heikki Linnakangas wrote: > > On 18/05/2023 17:59, Matthias van de Meent wrote: > > > It changes the block IDs used to fit in 6 bits, using the upper 2 bits > > > of the block_id field to store how much

Re: XLog size reductions: smaller XLRec block header for PG17

2023-09-18 Thread Michael Paquier
On Mon, Sep 18, 2023 at 04:03:38PM -0700, Andres Freund wrote: > I think it's a bad tradeoff to write lots of custom varint encodings, just to > eek out a bit more space savings. The increase in code complexity IMO makes it > a bad tradeoff. +1. -- Michael signature.asc Description: PGP

Re: XLog size reductions: smaller XLRec block header for PG17

2023-09-18 Thread Andres Freund
Hi, On 2023-05-18 19:22:26 +0300, Heikki Linnakangas wrote: > On 18/05/2023 17:59, Matthias van de Meent wrote: > > Attached is a patch that reduces this overhead by up to 2 bytes by > > encoding how large the block data length field is into the block ID, > > and thus optionally reducing the

Re: XLog size reductions: smaller XLRec block header for PG17

2023-09-18 Thread Matthias van de Meent
On Tue, 5 Sept 2023 at 15:04, Aleksander Alekseev wrote: > > Hi, > > I noticed that the patch needs review and decided to take a look. Thanks for reviewing! > All in all the patch looks good to me, but I have a couple of nitpicks: > > * The comment for XLogSizeClass seems to be somewhat

Re: XLog size reductions: smaller XLRec block header for PG17

2023-09-05 Thread Aleksander Alekseev
Hi, I noticed that the patch needs review and decided to take a look. > No meaningful savings in the pgbench workload, mostly due to xlog > record length MAXALIGNs currently not being favorable in the pgbench > workload. But, record sizes have dropped by 1 or 2 bytes in several > cases, as can

Re: XLog size reductions: smaller XLRec block header for PG17

2023-05-18 Thread Matthias van de Meent
On Thu, 18 May 2023 at 18:22, Heikki Linnakangas wrote: > > On 18/05/2023 17:59, Matthias van de Meent wrote: > Perhaps we should introduce a few generic inline functions to do varint > encoding. That could be useful in many places, while this scheme is very > tailored for XLogRecordBlockHeader.

Re: XLog size reductions: smaller XLRec block header for PG17

2023-05-18 Thread Heikki Linnakangas
On 18/05/2023 17:59, Matthias van de Meent wrote: Attached is a patch that reduces this overhead by up to 2 bytes by encoding how large the block data length field is into the block ID, and thus optionally reducing the block data's length field to 0 bytes. Examples: cross-page update records

XLog size reductions: smaller XLRec block header for PG17

2023-05-18 Thread Matthias van de Meent
Hi all, As was previously discussed at the thread surrounding [1]: Currently any block registration in WAL takes up at least 8 bytes in xlog overhead, regardless of how much data is included in that block: 1 byte for block ID, 1 byte for fork and flags, 2 bytes for the block data length, and 4