Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 13:38 +, Simon Riggs wrote: > > I see now that in the presence of prepared transactions, we would fail > > to clean up failed transations with XID > the oldest prepared > > transaction > > Good! I just spotted that also, just prior to posting my fix, so > rewriting it ag

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 15:18 +0200, Heikki Linnakangas wrote: > That removed piece of code was executed in the standby whenever we saw a > shutdown checkpoint. It calls ProcArrayApplyRecoveryInfo(), which calls > ExpireOldKnownAssignedTransactionIds() and StandbyReleaseOldLocks() to > clean up know

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Heikki Linnakangas
Simon Riggs wrote: > On Thu, 2009-12-17 at 13:24 +0200, Heikki Linnakangas wrote: > >> Hmm, looking at the code, I think Simon threw that baby with the >> bathwater when he removed support for starting standby from a shutdown >> checkpoint. > > Hmm, I think that code was just for starting points

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 13:24 +0200, Heikki Linnakangas wrote: > Hmm, looking at the code, I think Simon threw that baby with the > bathwater when he removed support for starting standby from a shutdown > checkpoint. Hmm, I think that code was just for starting points only. It would not have been e

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Simon Riggs
On Thu, 2009-12-17 at 19:55 +0900, Hiroyuki Yamada wrote: > Well, I've read some more and have a question. > > The implementation assumes that transactions write COMMIT/ABORT WAL at the end > of them, while it does not seem to write ABORT WAL in immediate shutdown. So, > > 1. acquire ACCESS EXCL

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Heikki Linnakangas
Hiroyuki Yamada wrote: > The implementation assumes that transactions write COMMIT/ABORT WAL at the end > of them, while it does not seem to write ABORT WAL in immediate shutdown. So, > > 1. acquire ACCESS EXCLUSIVE lock in table A in xact 1 > 2. execute immediate shutdown of the active node > 3.

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-17 Thread Hiroyuki Yamada
>On Wed, 2009-12-16 at 19:35 +0900, Hiroyuki Yamada wrote: > >> * There is a window beween gathering lock information in >> GetRunningTransactionLocks() >>and writing WAL in LogAccessExclusiveLocks(). >> * In current lock redo algorithm, locks are released when the transaction >> holding t

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Simon Riggs
On Wed, 2009-12-16 at 19:35 +0900, Hiroyuki Yamada wrote: > * There is a window beween gathering lock information in > GetRunningTransactionLocks() >and writing WAL in LogAccessExclusiveLocks(). > * In current lock redo algorithm, locks are released when the transaction > holding the lock

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Simon Riggs
On Wed, 2009-12-16 at 19:35 +0900, Hiroyuki Yamada wrote: > Sorry for annoying you. Not at all! Good to get fresh eyes on this. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Heikki Linnakangas
Hiroyuki Yamada wrote: > By the way, reading LogStandbySnapshot() and GetRunningTransactionLocks() > raised following questions. > > * There is a window beween gathering lock information in > GetRunningTransactionLocks() >and writing WAL in LogAccessExclusiveLocks(). > * In current lock red

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Hiroyuki Yamada
>On Wed, 2009-12-16 at 18:08 +0900, Hiroyuki Yamada wrote: >> >That fixes or explains all known issues, from me. Are there any other >> >things you know about that I haven't responded to? Do you think we have >> >addressed every issue, except deferred items? >> > >> >I will be looking to commit to

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Simon Riggs
On Wed, 2009-12-16 at 18:08 +0900, Hiroyuki Yamada wrote: > >That fixes or explains all known issues, from me. Are there any other > >things you know about that I haven't responded to? Do you think we have > >addressed every issue, except deferred items? > > > >I will be looking to commit to CVS la

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Hiroyuki Yamada
>That fixes or explains all known issues, from me. Are there any other >things you know about that I haven't responded to? Do you think we have >addressed every issue, except deferred items? > >I will be looking to commit to CVS later today; waiting on any >objections. > Is following problem repo

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-16 Thread Simon Riggs
On Wed, 2009-12-16 at 09:10 +0200, Heikki Linnakangas wrote: > Simon Riggs wrote: > > Investigating how that could come about, it looks like there is some > > fairly strange stuff going on here. StandbyRecoverPreparedTransactions() > > is never called at all. > > I told you so: > http://archives.p

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-15 Thread Heikki Linnakangas
Simon Riggs wrote: > Investigating how that could come about, it looks like there is some > fairly strange stuff going on here. StandbyRecoverPreparedTransactions() > is never called at all. I told you so: http://archives.postgresql.org/message-id/4b260b5e.3010...@enterprisedb.com StandbyRecoverP

Re: [HACKERS] Hot Standby and prepared transactions

2009-12-15 Thread Simon Riggs
On Tue, 2009-12-15 at 18:49 +, Simon Riggs wrote: > This looks like the code is setting the parent to be zero. [Sorry, that comment is completely off, misread the line number.] The assertion is failing because the parent entry for that subxid had already been set. Investigating how that coul

[HACKERS] Hot Standby and prepared transactions

2009-12-15 Thread Simon Riggs
I get this when testing prepared transactions, which looks like it is unrelated to Hot Standby. 2009-12-15 17:28:08 GMT[10385]LOG: archive recovery complete 2009-12-15 17:28:08 GMT[10428]LOG: checkpoint starting: end-of-recovery immediate wait 2009-12-15 17:28:08 GMT[10428]DEBUG: creating and