[HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-28 Thread Simon Riggs
On Sat, 2010-03-27 at 22:39 +, Greg Stark wrote: > On Sat, Mar 27, 2010 at 7:36 PM, Simon Riggs wrote: > > On Sat, 2010-03-27 at 19:15 +, Greg Stark wrote: > > > If we're pruning an index entry to a heap tuple that has been HOT > > > pruned wouldn't the HOT pruning record have already conf

[HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-27 Thread Greg Stark
On Sat, Mar 27, 2010 at 7:36 PM, Simon Riggs wrote: > On Sat, 2010-03-27 at 19:15 +, Greg Stark wrote: > > If we're pruning an index entry to a heap tuple that has been HOT > > pruned wouldn't the HOT pruning record have already conflicted with > > any queries that could see it? > > Quite pro

[HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-27 Thread Simon Riggs
On Sat, 2010-03-27 at 19:15 +, Greg Stark wrote: > On Sat, Mar 27, 2010 at 10:10 AM, Simon Riggs wrote: > > It appears that in practice many of the index items point to heap items > > that are LP_DEAD. So for the purposes of accessing a heap tuple's xmin, > > then we're both right. To the curr

[HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-27 Thread Greg Stark
On Sat, Mar 27, 2010 at 10:10 AM, Simon Riggs wrote: > It appears that in practice many of the index items point to heap items > that are LP_DEAD. So for the purposes of accessing a heap tuple's xmin, > then we're both right. To the current purpose the tuple has been > removed, though you are also

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-27 Thread Simon Riggs
On Fri, 2010-03-26 at 16:16 -0400, Tom Lane wrote: > Simon Riggs writes: > > On Sun, 2010-01-31 at 23:43 +0200, Heikki Linnakangas wrote: > >> When replaying the deletion record, the standby could look at all the > >> heap tuples whose index pointers are being removed, to see which one > >> was ne

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-26 Thread Tom Lane
Simon Riggs writes: > On Sun, 2010-01-31 at 23:43 +0200, Heikki Linnakangas wrote: >> When replaying the deletion record, the standby could look at all the >> heap tuples whose index pointers are being removed, to see which one >> was newest. > Long after coding this, I now realise this really is

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-26 Thread Simon Riggs
On Sun, 2010-01-31 at 23:43 +0200, Heikki Linnakangas wrote: > When replaying the deletion record, the standby could look at all the > heap tuples whose index pointers are being removed, to see which one > was newest. Long after coding this, I now realise this really is a dumb-ass idea. There is

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-09 Thread Simon Riggs
On Tue, 2010-03-09 at 11:20 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > XLOG_BTREE_DELETE records would not carry latestRemovedXid, this would > > be calculated by inspection of heap tuples. > > You might still want to keep the conservative latestRemovedXid value in > the WAL record

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-09 Thread Heikki Linnakangas
Simon Riggs wrote: > XLOG_BTREE_DELETE records would not carry latestRemovedXid, this would > be calculated by inspection of heap tuples. You might still want to keep the conservative latestRemovedXid value in the WAL record to avoid the extra work when latestRemovedXid alone allows. -- Heikki

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-03-09 Thread Simon Riggs
On Sun, 2010-01-31 at 16:53 -0500, Tom Lane wrote: > Heikki Linnakangas writes: > > IIRC it was Greg Stark who suggested last time this was discussed that > > we could calculate the exact value for latestRemovedXid in the standby. > > When replaying the deletion record, the standby could look at a

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 17:10 -0500, Tom Lane wrote: > Simon Riggs writes: > > Should I revoke this change? > > Please. Will do, in morning. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscriptio

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Tom Lane
Simon Riggs writes: > Should I revoke this change? Please. We can always put it back later if nothing better gets implemented. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.post

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 23:43 +0200, Heikki Linnakangas wrote: > IIRC it was Greg Stark who suggested last time this was discussed that > we could calculate the exact value for latestRemovedXid in the > standby. When replaying the deletion record, the standby could look at > all the heap tuples whose

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Tom Lane
Heikki Linnakangas writes: > IIRC it was Greg Stark who suggested last time this was discussed that > we could calculate the exact value for latestRemovedXid in the standby. > When replaying the deletion record, the standby could look at all the > heap tuples whose index pointers are being removed

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Heikki Linnakangas
Simon Riggs wrote: > On Sun, 2010-01-31 at 22:05 +0100, Stefan Kaltenbrunner wrote: > >> hmm makes sense from the HS side - but I think to make a case for a GUC >> it would help a lot to see actual numbers(as in benchmarks) showing how >> much of a hit it is to the master. > > Agreed, though my

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 22:05 +0100, Stefan Kaltenbrunner wrote: > hmm makes sense from the HS side - but I think to make a case for a GUC > it would help a lot to see actual numbers(as in benchmarks) showing how > much of a hit it is to the master. Agreed, though my approach to benchmarking was

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Stefan Kaltenbrunner
Simon Riggs wrote: On Sun, 2010-01-31 at 20:48 +0100, Stefan Kaltenbrunner wrote: Simon Riggs wrote: On Sun, 2010-01-31 at 14:07 -0500, Tom Lane wrote: The commit is a one line change, with parameter to control it, discussed by Heikki and myself in December 2008. I stand by the accuracy of th

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 15:41 -0500, Tom Lane wrote: > Simon Riggs writes: > > At the moment a btree delete record will cancel every request > > 1. no matter how long they have been running > > 2. no matter if they haven't accessed the index being cleaned (they > > might later, is the thinking...) >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Tom Lane
Simon Riggs writes: > At the moment a btree delete record will cancel every request > 1. no matter how long they have been running > 2. no matter if they haven't accessed the index being cleaned (they > might later, is the thinking...) That seems seriously horrid. What is the rationale for #2 in

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 20:48 +0100, Stefan Kaltenbrunner wrote: > Simon Riggs wrote: > > On Sun, 2010-01-31 at 14:07 -0500, Tom Lane wrote: > > > >>> The commit is a one line change, with parameter to control it, discussed > >>> by Heikki and myself in December 2008. I stand by the accuracy of the

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Stefan Kaltenbrunner
Simon Riggs wrote: On Sun, 2010-01-31 at 14:07 -0500, Tom Lane wrote: The commit is a one line change, with parameter to control it, discussed by Heikki and myself in December 2008. I stand by the accuracy of the change; the parameter is really to ensure we can test during beta. Well, I was w

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Simon Riggs
On Sun, 2010-01-31 at 14:07 -0500, Tom Lane wrote: > > The commit is a one line change, with parameter to control it, discussed > > by Heikki and myself in December 2008. I stand by the accuracy of the > > change; the parameter is really to ensure we can test during beta. > > Well, I was waiting

Re: [HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-31 Thread Tom Lane
Simon Riggs writes: > On Fri, 2010-01-29 at 14:04 -0500, Tom Lane wrote: >> WTF? Simon, this seems to be getting way way beyond anything the >> community has agreed to. Particularly, inventing GUCs is not something >> to be doing without consensus. > If you or anybody else would like me to revo

[HACKERS] Re: [COMMITTERS] pgsql: Augment WAL records for btree delete with GetOldestXmin() to

2010-01-29 Thread Simon Riggs
On Fri, 2010-01-29 at 14:04 -0500, Tom Lane wrote: > sri...@postgresql.org (Simon Riggs) writes: > > Log Message: > > --- > > Augment WAL records for btree delete with GetOldestXmin() to reduce > > false positives during Hot Standby conflict processing. Simple > > patch to enhance conflict