Re: [HACKERS] B-tree parent pointer and checkpoints

2012-08-23 Thread Bruce Momjian
On Tue, Aug 21, 2012 at 11:55:20AM -0400, Robert Haas wrote: > On Wed, Aug 15, 2012 at 6:23 PM, Bruce Momjian wrote: > > Has this been addressed? A TODO? > > I don't think anything's been done about it. According to your email > of October 11, 2011, you already did add a TODO for this. Ah, I s

Re: [HACKERS] B-tree parent pointer and checkpoints

2012-08-21 Thread Robert Haas
On Wed, Aug 15, 2012 at 6:23 PM, Bruce Momjian wrote: > Has this been addressed? A TODO? I don't think anything's been done about it. According to your email of October 11, 2011, you already did add a TODO for this. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postg

Re: [HACKERS] B-tree parent pointer and checkpoints

2012-08-15 Thread Bruce Momjian
Has this been addressed? A TODO? --- On Tue, Sep 6, 2011 at 09:49:39AM -0400, Robert Haas wrote: > On Tue, Sep 6, 2011 at 9:45 AM, Heikki Linnakangas > wrote: > >> Do you really need to hold the page locks for all that ti

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-10-11 Thread Bruce Momjian
Heikki Linnakangas wrote: > On 11.03.2011 19:41, Tom Lane wrote: > > Heikki Linnakangas writes: > >> On 11.03.2011 17:59, Tom Lane wrote: > >>> But that will be fixed during WAL replay. > > > >> Not under the circumstances that started the original thread: > > > >> 1. Backend splits a page > >> 2.

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-09-06 Thread Robert Haas
On Tue, Sep 6, 2011 at 10:03 AM, Bruce Momjian wrote: > Robert Haas wrote: >> On Tue, Sep 6, 2011 at 9:45 AM, Heikki Linnakangas >> wrote: >> >> Do you really need to hold the page locks for all that time, or could >> >> you cheat? ?Like... release the locks on the split pages but then go >> >> b

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-09-06 Thread Bruce Momjian
Robert Haas wrote: > On Tue, Sep 6, 2011 at 9:45 AM, Heikki Linnakangas > wrote: > >> Do you really need to hold the page locks for all that time, or could > >> you cheat? ?Like... release the locks on the split pages but then go > >> back and reacquire them to clear the flag... > > > > Hmm, there

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-09-06 Thread Robert Haas
On Tue, Sep 6, 2011 at 9:45 AM, Heikki Linnakangas wrote: >> Do you really need to hold the page locks for all that time, or could >> you cheat?  Like... release the locks on the split pages but then go >> back and reacquire them to clear the flag... > > Hmm, there's two issues with that: > > 1. W

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-09-06 Thread Heikki Linnakangas
On 06.09.2011 16:40, Robert Haas wrote: On Tue, Sep 6, 2011 at 6:21 AM, Heikki Linnakangas wrote: The way it would work is that on page split the right page is flagged with MISSING_DOWNLINK flag. When the downlink is inserted into the parent, the flag is cleared in the same critical section as

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-09-06 Thread Robert Haas
On Tue, Sep 6, 2011 at 6:21 AM, Heikki Linnakangas wrote: > Nope. > > On a closer look, this isn't only a problem for page deletion. Page > splitting also barfs if it can't find the parent of a page. As the code > stands, a missing downlink is not harmless, but causes all sorts of trouble. > > The

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-09-06 Thread Heikki Linnakangas
On 05.09.2011 21:55, Bruce Momjian wrote: Heikki Linnakangas wrote: On 11.03.2011 19:41, Tom Lane wrote: Heikki Linnakangas writes: On 11.03.2011 17:59, Tom Lane wrote: But that will be fixed during WAL replay. Not under the circumstances that started the original thread: 1. Backend s

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-09-05 Thread Bruce Momjian
Heikki Linnakangas wrote: > On 11.03.2011 19:41, Tom Lane wrote: > > Heikki Linnakangas writes: > >> On 11.03.2011 17:59, Tom Lane wrote: > >>> But that will be fixed during WAL replay. > > > >> Not under the circumstances that started the original thread: > > > >> 1. Backend splits a page > >> 2.

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-03-11 Thread Heikki Linnakangas
On 11.03.2011 19:41, Tom Lane wrote: Heikki Linnakangas writes: On 11.03.2011 17:59, Tom Lane wrote: But that will be fixed during WAL replay. Not under the circumstances that started the original thread: 1. Backend splits a page 2. Checkpoint starts 3. Checkpoint runs to completion 4. C

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-03-11 Thread Tom Lane
Heikki Linnakangas writes: > On 11.03.2011 17:59, Tom Lane wrote: >> But that will be fixed during WAL replay. > Not under the circumstances that started the original thread: > 1. Backend splits a page > 2. Checkpoint starts > 3. Checkpoint runs to completion > 4. Crash > (5. Backend never got t

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-03-11 Thread Heikki Linnakangas
On 11.03.2011 17:59, Tom Lane wrote: Heikki Linnakangas writes: On 10.03.2011 22:50, Bruce Momjian wrote: Bruce Momjian wrote: Has this been addressed? I see we have with this commit: 9de3aa65f01fb51cbc725e8508ea233e4e92c46c We fixed GiST. B-tree still has the issue that if you have a

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-03-11 Thread Tom Lane
Heikki Linnakangas writes: > On 10.03.2011 22:50, Bruce Momjian wrote: >> Bruce Momjian wrote: > > Has this been addressed? >> >> I see we have with this commit: >> >> 9de3aa65f01fb51cbc725e8508ea233e4e92c46c > We fixed GiST. B-tree still has the issue that if you have a checkpoint > in the m

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-03-11 Thread Heikki Linnakangas
On 10.03.2011 22:50, Bruce Momjian wrote: Bruce Momjian wrote: Has this been addressed? I see we have with this commit: 9de3aa65f01fb51cbc725e8508ea233e4e92c46c We fixed GiST. B-tree still has the issue that if you have a checkpoint in the middle of an insert, and crash, you might

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-03-10 Thread Bruce Momjian
Bruce Momjian wrote: > > Has this been addressed? I see we have with this commit: 9de3aa65f01fb51cbc725e8508ea233e4e92c46c --- > > --- > >

Re: [HACKERS] B-tree parent pointer and checkpoints

2011-03-10 Thread Bruce Momjian
Has this been addressed? --- Heikki Linnakangas wrote: > On 13.11.2010 00:34, Greg Stark wrote: > > On Fri, Nov 12, 2010 at 7:20 PM, Heikki Linnakangas > > wrote: > >> I think we can work around that with a small modificat

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-16 Thread Heikki Linnakangas
On 13.11.2010 00:34, Greg Stark wrote: On Fri, Nov 12, 2010 at 7:20 PM, Heikki Linnakangas wrote: I think we can work around that with a small modification to the page split algorithm. In a nutshell, when the child page is split, put a flag on the left half indicating that the rightlink must a

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-12 Thread Greg Stark
On Fri, Nov 12, 2010 at 7:20 PM, Heikki Linnakangas wrote: > I think we can work around that with a small modification to the page split > algorithm. In a nutshell, when the child page is split, put a flag on the > left half indicating that the rightlink must always be followed, regardless > of th

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-12 Thread Heikki Linnakangas
On 11.11.2010 20:34, Tom Lane wrote: Heikki Linnakangas writes: Hmm, we don't currently keep track of that when we descend the tree to choose the target page, but perhaps an extra Consistent call to check that would be acceptable. We already call Penalty for every tuple on each internal node on

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-11 Thread Tom Lane
Heikki Linnakangas writes: > Hmm, we don't currently keep track of that when we descend the tree to > choose the target page, but perhaps an extra Consistent call to check > that would be acceptable. We already call Penalty for every tuple on > each internal node on the way, so compared to that

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-11 Thread Heikki Linnakangas
On 11.11.2010 17:16, Tom Lane wrote: Heikki Linnakangas writes: GiST is different. When you insert a key to a leaf page, you (sometimes) need to adjust the parent pointer to reflect the new key as well. B-tree tolerates incomplete splits with the 'next page' pointer, but that is not applicable

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-11 Thread Tom Lane
Heikki Linnakangas writes: > GiST is different. When you insert a key to a leaf page, you (sometimes) > need to adjust the parent pointer to reflect the new key as well. B-tree > tolerates incomplete splits with the 'next page' pointer, but that is > not applicable to gist. Teodor described the

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-11 Thread Heikki Linnakangas
On 11.11.2010 00:49, Tom Lane wrote: I wrote: What happens if you error out in between? Or is it assumed that the *entire* sequence is a critical section? If it has to be that way, one might wonder what's the point of trying to split it into multiple WAL records. Or, to be more concrete: I'm

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-11 Thread Heikki Linnakangas
On 11.11.2010 00:49, Tom Lane wrote: I wrote: What happens if you error out in between? Or is it assumed that the *entire* sequence is a critical section? If it has to be that way, one might wonder what's the point of trying to split it into multiple WAL records. Or, to be more concrete: I'm

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-10 Thread Tom Lane
I wrote: > What happens if you error out in between? Or is it assumed that the > *entire* sequence is a critical section? If it has to be that way, > one might wonder what's the point of trying to split it into multiple > WAL records. Or, to be more concrete: I'm wondering if this *entire* mecha

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-10 Thread Tom Lane
Heikki Linnakangas writes: > The new rule is that before you start a multi-WAL-record operation that > needs to be completed at end of recovery if you crash in the middle, you > call HoldCheckpoint(), and once you're finished, ResumeCheckpoint(). What happens if you error out in between? Or i

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-10 Thread Heikki Linnakangas
On 08.11.2010 15:40, Heikki Linnakangas wrote: Here's a first draft of this, using the inCommit flag as is. It works, but suffers from starvation if you have a lot of concurrent multi-WAL-record actions. I tested that by running INSERTs to a table with tsvector field with a GiST index on it from

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-08 Thread Heikki Linnakangas
On 02.11.2010 16:40, Heikki Linnakangas wrote: On 02.11.2010 16:30, Tom Lane wrote: Heikki Linnakangas writes: I think we can fix this by requiring that any multi-WAL-record actions that are in-progress when a checkpoint starts (at the REDO-pointer) must finish before the checkpoint record is w

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-02 Thread Heikki Linnakangas
On 02.11.2010 16:30, Tom Lane wrote: Heikki Linnakangas writes: I think we can fix this by requiring that any multi-WAL-record actions that are in-progress when a checkpoint starts (at the REDO-pointer) must finish before the checkpoint record is written. What happens if someone wants to star

Re: [HACKERS] B-tree parent pointer and checkpoints

2010-11-02 Thread Tom Lane
Heikki Linnakangas writes: > I think we can fix this by requiring that any multi-WAL-record actions > that are in-progress when a checkpoint starts (at the REDO-pointer) must > finish before the checkpoint record is written. What happens if someone wants to start a new split while the checkpoin

[HACKERS] B-tree parent pointer and checkpoints

2010-11-02 Thread Heikki Linnakangas
We have the rm_safe_restartpoint mechanism to ensure that we don't use a checkpoint that splits a multi-level B-tree insertion as a restart point. But to my surprise, we don't have anything to protect against the analogous case during normal operation. This is possible: 1. Split child page. Wr