Re: [HACKERS] [RFC] LSN Map

2015-07-06 Thread Heikki Linnakangas
On 02/24/2015 04:55 AM, Robert Haas wrote: On Mon, Feb 23, 2015 at 12:52 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Dunno, but Jim's got a point. This is a maintenance burden to all indexams, if they all have to remember to update the LSN map separately. It needs to be done in some

Re: [HACKERS] [RFC] LSN Map

2015-02-23 Thread Heikki Linnakangas
On 01/13/2015 01:22 PM, Marco Nenciarini wrote: Il 08/01/15 20:18, Jim Nasby ha scritto: On 1/7/15, 3:50 AM, Marco Nenciarini wrote: The current implementation tracks only heap LSN. It currently does not track any kind of indexes, but this can be easily added later. Would it make sense to do

Re: [HACKERS] [RFC] LSN Map

2015-02-23 Thread Robert Haas
On Mon, Feb 23, 2015 at 12:52 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Dunno, but Jim's got a point. This is a maintenance burden to all indexams, if they all have to remember to update the LSN map separately. It needs to be done in some common code, like in PageSetLSN or

Re: [HACKERS] [RFC] LSN Map

2015-01-13 Thread Marco Nenciarini
Il 08/01/15 20:18, Jim Nasby ha scritto: On 1/7/15, 3:50 AM, Marco Nenciarini wrote: The current implementation tracks only heap LSN. It currently does not track any kind of indexes, but this can be easily added later. Would it make sense to do this at a buffer level, instead of at the heap

Re: [HACKERS] [RFC] LSN Map

2015-01-08 Thread Jim Nasby
On 1/7/15, 3:50 AM, Marco Nenciarini wrote: The current implementation tracks only heap LSN. It currently does not track any kind of indexes, but this can be easily added later. Would it make sense to do this at a buffer level, instead of at the heap level? That means it would handle both

[HACKERS] [RFC] LSN Map

2015-01-07 Thread Marco Nenciarini
Hi Hackers, In order to make incremental backup (https://wiki.postgresql.org/wiki/Incremental_backup) efficient we need a way to track the LSN of a page in a way that we can retrieve it without reading the actual block. Below there is my proposal on how to achieve it. LSN Map --- The

Re: [HACKERS] [RFC] LSN Map

2015-01-07 Thread Bruce Momjian
On Wed, Jan 7, 2015 at 10:50:38AM +0100, Marco Nenciarini wrote: Implementation -- We create an additional fork which contains a raw stream of LSNs. To limit the space used, every entry represent the maximum LSN of a group of blocks of a fixed size. I chose arbitrarily the size

Re: [HACKERS] [RFC] LSN Map

2015-01-07 Thread Alvaro Herrera
Bruce Momjian wrote: Have you done any measurements to determine how much backup can be skipped using this method for a typical workload, i.e. how many 16MB page ranges are not modified in a typical span between incremental backups? That seems entirely dependent on the specific workload. --

Re: [HACKERS] [RFC] LSN Map

2015-01-07 Thread Bruce Momjian
On Wed, Jan 7, 2015 at 12:33:20PM -0300, Alvaro Herrera wrote: Bruce Momjian wrote: Have you done any measurements to determine how much backup can be skipped using this method for a typical workload, i.e. how many 16MB page ranges are not modified in a typical span between incremental

Re: [HACKERS] [RFC] LSN Map

2015-01-07 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Bruce Momjian wrote: Have you done any measurements to determine how much backup can be skipped using this method for a typical workload, i.e. how many 16MB page ranges are not modified in a typical span between incremental backups? That seems