Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-02-03 Thread Andres Freund
On 2015-02-03 14:18:02 +0900, Michael Paquier wrote: > - "RecoveryLockList contains entry for lock no longer recorded by > lock manager: xid %u database %u relation %u", > - lock->xid, lock->dbOid, lock->relOid); > +"RecoveryLockList contains entry for lock no longer recorded by > loc

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-02-02 Thread Michael Paquier
On Sat, Jan 31, 2015 at 5:34 AM, Andres Freund wrote: > On 2015-01-29 11:01:51 -0500, Robert Haas wrote: >> On Wed, Jan 28, 2015 at 2:41 AM, Michael Paquier >> wrote: >> > Andres Freund wrote: >> >> I think this isn't particularly pretty, but it seems to be working well >> >> enough, and changing

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-01-30 Thread Andres Freund
On 2015-01-29 11:01:51 -0500, Robert Haas wrote: > On Wed, Jan 28, 2015 at 2:41 AM, Michael Paquier > wrote: > > Andres Freund wrote: > >> I think this isn't particularly pretty, but it seems to be working well > >> enough, and changing it would be pretty invasive. So keeping in line > >> with all

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-01-29 Thread Robert Haas
On Wed, Jan 28, 2015 at 2:41 AM, Michael Paquier wrote: > Andres Freund wrote: >> I think this isn't particularly pretty, but it seems to be working well >> enough, and changing it would be pretty invasive. So keeping in line >> with all that code seems to be easier. > OK, I'm convinced with this

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-01-27 Thread Michael Paquier
Andres Freund wrote: > I think this isn't particularly pretty, but it seems to be working well > enough, and changing it would be pretty invasive. So keeping in line > with all that code seems to be easier. OK, I'm convinced with this part to remove the call of LockSharedObjectForSession that uses

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-01-27 Thread Andres Freund
On 2015-01-27 16:23:53 +0900, Michael Paquier wrote: > On Tue, Jan 27, 2015 at 6:24 AM, Andres Freund wrote: > > Unfortunately that Assert()s when there's a lock conflict because > > e.g. another backend is currently connecting. That's because ProcSleep() > > does a enable_timeout_after(DEADLOCK_T

Re: [HACKERS] Hot Standby WAL reply uses heavyweight session locks, but doesn't have enough infrastructure set up

2015-01-26 Thread Michael Paquier
On Tue, Jan 27, 2015 at 6:24 AM, Andres Freund wrote: > Unfortunately that Assert()s when there's a lock conflict because > e.g. another backend is currently connecting. That's because ProcSleep() > does a enable_timeout_after(DEADLOCK_TIMEOUT, DeadlockTimeout) - and > there's no deadlock timeout

Re: [HACKERS] Hot standby doesn't come up on some situation.

2014-03-07 Thread Kyotaro HORIGUCHI
Hello, > > After all, I have confirmed that this fixes the problem on crash > > recovery of hot-standby botfor 9.3 and HEAD and no problem was > > found except unreadability :( > > Ok, committed. Thanks! Thank you. > Any concrete suggestions about the readability? Is there some > particular spo

Re: [HACKERS] Hot standby doesn't come up on some situation.

2014-03-05 Thread Heikki Linnakangas
On 03/05/2014 10:51 AM, Kyotaro HORIGUCHI wrote: Hello, After all, I have confirmed that this fixes the problem on crash recovery of hot-standby botfor 9.3 and HEAD and no problem was found except unreadability :( Ok, committed. Thanks! Any concrete suggestions about the readability? Is there

Re: [HACKERS] Hot standby doesn't come up on some situation.

2014-03-05 Thread Kyotaro HORIGUCHI
Hello, After all, I have confirmed that this fixes the problem on crash recovery of hot-standby botfor 9.3 and HEAD and no problem was found except unreadability :( By the way, I moderately want to fix an assertion message to a ordinary one. Details are below. The server stops with followi

Re: [HACKERS] Hot standby doesn't come up on some situation.

2014-03-02 Thread Kyotaro HORIGUCHI
Hello, > | * as if we had just replayed the record before the REDO location > | * (or the checkpoint record itself, if it's a shutdown checkpoint). > > The test script following raises assertion failure. It's added > with 'non-shutdown' checkpoint' just before shutting down > immediately. Startin

Re: [HACKERS] Hot standby doesn't come up on some situation.

2014-03-02 Thread Kyotaro HORIGUCHI
Correcting one point of my last mail. > Ouch! It brought another bug. My patch also did. regards, > > I completely understood the behavior thanks to your detailed > > explanation. (And how to use log messages effectively :-) > > Sorry, I just found that it's wrong, and found another probl

Re: [HACKERS] Hot standby doesn't come up on some situation.

2014-03-02 Thread Kyotaro HORIGUCHI
Ouch! It brought another bug. > I completely understood the behavior thanks to your detailed > explanation. (And how to use log messages effectively :-) Sorry, I just found that it's wrong, and found another problem brought by your patch. > I agree that the fix is appropriate. > > > I believe t

Re: [HACKERS] Hot standby doesn't come up on some situation.

2014-03-02 Thread Kyotaro HORIGUCHI
Hello, At Fri, 28 Feb 2014 14:45:58 +0200, Heikki Linnakangas wrote in <53108506.2010...@vmware.com> > > Yes, but the same stuation could be made by restarting crashed > > secondary. > > Yeah. > > > I have no idea about the scenario on whitch this behavior was regarded > > as > > undesirable b

Re: [HACKERS] Hot standby doesn't come up on some situation.

2014-02-28 Thread Heikki Linnakangas
On 02/28/2014 11:51 AM, Kyotaro HORIGUCHI wrote: Hello, 2014/02/28 18:07 "Andres Freund" : On 2014-02-28 17:55:21 +0900, Kyotaro HORIGUCHI wrote: The recovery process stays on 'incosistent' state forever when the server has crashed before any wal record is inserted after the last checkpoint.

Re: [HACKERS] Hot standby doesn't come up on some situation.

2014-02-28 Thread Kyotaro HORIGUCHI
Hello, 2014/02/28 18:07 "Andres Freund" : > > On 2014-02-28 17:55:21 +0900, Kyotaro HORIGUCHI wrote: > > The recovery process stays on 'incosistent' state forever when > > the server has crashed before any wal record is inserted after > > the last checkpoint. > > > # killall postgres > > # rm -rf

Re: [HACKERS] Hot standby doesn't come up on some situation.

2014-02-28 Thread Andres Freund
On 2014-02-28 17:55:21 +0900, Kyotaro HORIGUCHI wrote: > The recovery process stays on 'incosistent' state forever when > the server has crashed before any wal record is inserted after > the last checkpoint. > # killall postgres > # rm -rf $PGDATA/* > initdb > pg_ctl start -w > sleep 1 > pg_ctl st

Re: [HACKERS] Hot standby doesn't come up on some situation.

2014-02-28 Thread Kyotaro HORIGUCHI
Ouch. this is the same issue to the mail below, http://www.postgresql.org/message-id/53104595.6060...@lab.ntt.co.jp regards, -- Kyotaro Horiguchi NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http:/

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-17 Thread Andres Freund
On 2013-01-17 10:19:23 -0500, Tom Lane wrote: > Pavan Deolasee writes: > > On Thu, Jan 17, 2013 at 12:08 PM, Tom Lane wrote: > >> ISTM that if we dare not interrupt for fear of confusing OpenSSL, we > >> cannot safely attempt to send an error message to the client either; > >> but ereport(FATAL)

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-17 Thread Tom Lane
Pavan Deolasee writes: > On Thu, Jan 17, 2013 at 12:08 PM, Tom Lane wrote: >> ISTM that if we dare not interrupt for fear of confusing OpenSSL, we >> cannot safely attempt to send an error message to the client either; >> but ereport(FATAL) will try exactly that. > I thought since FATAL will for

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-17 Thread Andres Freund
On 2013-01-17 01:38:31 -0500, Tom Lane wrote: > But having said that ... are we sure this code is not actually broken? > ISTM that if we dare not interrupt for fear of confusing OpenSSL, we > cannot safely attempt to send an error message to the client either; > but ereport(FATAL) will try exactly

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-16 Thread Pavan Deolasee
On Thu, Jan 17, 2013 at 12:08 PM, Tom Lane wrote: > > But having said that ... are we sure this code is not actually broken? > I'm not. > ISTM that if we dare not interrupt for fear of confusing OpenSSL, we > cannot safely attempt to send an error message to the client either; > but ereport(FA

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-16 Thread Tom Lane
Abhijit Menon-Sen writes: > Sorry for nitpicking, but "we can't long jumps" made me cringe. Agreed :-( > Here's a slightly more condensed version: I find this still not quite right, because where it's placed, it applies to both the DoingCommandRead and !DoingCommandRead branches of the if/else

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-16 Thread Pavan Deolasee
On Thu, Jan 17, 2013 at 9:26 AM, Abhijit Menon-Sen wrote: > At 2012-12-29 14:23:45 -0500, sfr...@snowman.net wrote: > > > > Regarding the actual comment, here's the wording that I'd use: > > Sorry for nitpicking, but "we can't long jumps" made me cringe. > Here's a slightly more condensed version:

Re: [HACKERS] Hot Standby conflict resolution handling

2013-01-16 Thread Abhijit Menon-Sen
At 2012-12-29 14:23:45 -0500, sfr...@snowman.net wrote: > > Regarding the actual comment, here's the wording that I'd use: Sorry for nitpicking, but "we can't long jumps" made me cringe. Here's a slightly more condensed version: /* * We can't use ereport(ERROR) here, because any longjmps

Re: [HACKERS] Hot Standby conflict resolution handling

2012-12-29 Thread Stephen Frost
Pavan, * Pavan Deolasee (pavan.deola...@gmail.com) wrote: > On Tue, Dec 4, 2012 at 6:01 PM, Pavan Deolasee > wrote: > > Thanks Andres. I also read the original thread and I now understand why we > > are using FATAL here, at least until we have a better solution. Obviously > > the connection reset

Re: [HACKERS] Hot Standby conflict resolution handling

2012-12-04 Thread Pavan Deolasee
On Tue, Dec 4, 2012 at 6:01 PM, Pavan Deolasee wrote: > > > Thanks Andres. I also read the original thread and I now understand why we > are using FATAL here, at least until we have a better solution. Obviously > the connection reset is no good either because as someone commented in the > original

Re: [HACKERS] Hot Standby conflict resolution handling

2012-12-04 Thread Pavan Deolasee
On Tue, Dec 4, 2012 at 1:44 PM, Andres Freund wrote: > > > > > After max_standby_streaming_delay, the standby starts cancelling the > > queries. I get an error like this on the standby: > > postgres=# explain verbose select count(b) from test WHERE a > 10; > > FATAL: terminating connection du

Re: [HACKERS] Hot Standby conflict resolution handling

2012-12-04 Thread Andres Freund
Hi, On 2012-12-04 12:30:43 +0530, Pavan Deolasee wrote: > I was trying some simple queries on a Hot Standby with streaming > replication. > > On standby, I do this: > postgres=# begin transaction isolation level repeatable read; > BEGIN > postgres=# explain verbose select count(b) from test WHERE

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-12-03 Thread Robert Haas
On Fri, Nov 30, 2012 at 6:41 PM, Andres Freund wrote: >> While we're talking about changing defaults, how about changing the >> default value of the recovery.conf parameter 'standby_mode' to on? >> Not sure about anybody else, but I never want it any other way. > > Hm. But only if there is a recov

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-12-01 Thread Albe Laurenz
Magnus Hagander wrote: >> On 30.11.2012 21:02, Andres Freund wrote: >>> There are workloads where its detrimental, but in general having it >>> default to on improver experience tremendously because getting conflicts >>> because of vacuum is rather confusing. >>> >>> In the workloads where it might

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Andres Freund
On 2012-11-30 16:09:15 -0500, Tom Lane wrote: > Claudio Freire writes: > > Without hot standby feedback, reporting queries are impossible. I've > > experienced it. Cancellations make it impossible to finish any > > decently complex reporting query. > > The original expectation was that slave-side

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Josh Berkus
All: Well, the problem is that we have three configurations which only work for one very common scenario: - reporting slave: feedback off, very long replication_delay - load-balancing slave: feedback on, short replication_delay - backup/failover slave: feedback off, short replication_delay I don

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Andres Freund
On 2012-11-30 22:46:06 +0200, Heikki Linnakangas wrote: > On 30.11.2012 21:02, Andres Freund wrote: > >Hi, > > > >The subject says it all. > > > >There are workloads where its detrimental, but in general having it > >default to on improver experience tremendously because getting conflicts > >becaus

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Magnus Hagander
On Fri, Nov 30, 2012 at 10:09 PM, Tom Lane wrote: > Claudio Freire writes: >> Without hot standby feedback, reporting queries are impossible. I've >> experienced it. Cancellations make it impossible to finish any >> decently complex reporting query. > > The original expectation was that slave-sid

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Magnus Hagander
On Fri, Nov 30, 2012 at 9:46 PM, Heikki Linnakangas wrote: > On 30.11.2012 21:02, Andres Freund wrote: >> >> Hi, >> >> The subject says it all. >> >> There are workloads where its detrimental, but in general having it >> default to on improver experience tremendously because getting conflicts >> b

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Daniel Farina
On Fri, Nov 30, 2012 at 11:35 AM, Robert Haas wrote: > On Fri, Nov 30, 2012 at 2:02 PM, Andres Freund wrote: >> Does anybody have an argument against changing the default value? > > Well, the disadvantage of it is that the standby can bloat the master, > which might be surprising to some people,

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Andres Freund
On 2012-11-30 14:35:37 -0500, Robert Haas wrote: > On Fri, Nov 30, 2012 at 2:02 PM, Andres Freund wrote: > > Does anybody have an argument against changing the default value? > > Well, the disadvantage of it is that the standby can bloat the master, > which might be surprising to some people, too.

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 6:49 PM, Heikki Linnakangas wrote: >>> I have most certainly managed databases where holding up vacuuming >>> on the source would cripple performance to the point that users >>> would have demanded that any other process causing it must be >>> immediately canceled. And canc

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Heikki Linnakangas
On 30.11.2012 23:40, Claudio Freire wrote: On Fri, Nov 30, 2012 at 6:20 PM, Kevin Grittner wrote: Claudio Freire wrote: With what setting of max_standby_streaming_delay? I would rather default that to -1 than default hot_standby_feedback on. That way what you do on the standby only affects th

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 6:20 PM, Kevin Grittner wrote: > Claudio Freire wrote: > >>> With what setting of max_standby_streaming_delay? I would rather >>> default that to -1 than default hot_standby_feedback on. That >>> way what you do on the standby only affects the standby. >> >> 1d > > Was ther

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Kevin Grittner
Claudio Freire wrote: >> With what setting of max_standby_streaming_delay? I would rather >> default that to -1 than default hot_standby_feedback on. That >> way what you do on the standby only affects the standby. > > 1d Was there actually a transaction hanging open for an entire day on the sta

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Tom Lane
Claudio Freire writes: > Without hot standby feedback, reporting queries are impossible. I've > experienced it. Cancellations make it impossible to finish any > decently complex reporting query. The original expectation was that slave-side cancels would be infrequent. Maybe there's some fixing/t

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 6:06 PM, Kevin Grittner wrote: > >> Without hot standby feedback, reporting queries are impossible. >> I've experienced it. Cancellations make it impossible to finish >> any decently complex reporting query. > > With what setting of max_standby_streaming_delay? I would rath

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Kevin Grittner
Claudio Freire wrote: > Without hot standby feedback, reporting queries are impossible. > I've experienced it. Cancellations make it impossible to finish > any decently complex reporting query. With what setting of max_standby_streaming_delay? I would rather default that to -1 than default hot_st

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Heikki Linnakangas
On 30.11.2012 22:49, Claudio Freire wrote: On Fri, Nov 30, 2012 at 5:46 PM, Heikki Linnakangas wrote: Think of someone setting up a test server, by setting it up as a standby from the master. Now, when someone holds a transaction open in the test server, you get bloat in the master. Or if you

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Tom Lane
Robert Haas writes: > While we're talking about changing defaults, how about changing the > default value of the recovery.conf parameter 'standby_mode' to on? > Not sure about anybody else, but I never want it any other way. Dunno, it's been only a couple of days since there was a thread about so

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 5:46 PM, Heikki Linnakangas wrote: > > Think of someone setting up a test server, by setting it up as a standby > from the master. Now, when someone holds a transaction open in the test > server, you get bloat in the master. Or if you set up a standby for > reporting purpos

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Heikki Linnakangas
On 30.11.2012 21:02, Andres Freund wrote: Hi, The subject says it all. There are workloads where its detrimental, but in general having it default to on improver experience tremendously because getting conflicts because of vacuum is rather confusing. In the workloads where it might not be a go

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Josh Berkus
> In the workloads where it might not be a good idea (very long queries on > the standby, many dead tuples on the primary) you need to think very > carefuly about the strategy of avoiding conflicts anyway, and explicit > configuration is required as well. > > Does anybody have an argument against

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Robert Haas
On Fri, Nov 30, 2012 at 2:02 PM, Andres Freund wrote: > Does anybody have an argument against changing the default value? Well, the disadvantage of it is that the standby can bloat the master, which might be surprising to some people, too. But I don't really have a lot of skin in this game. Whi

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Simon Riggs
On 30 November 2012 19:02, Andres Freund wrote: > The subject says it all. > > There are workloads where its detrimental, but in general having it > default to on improver experience tremendously because getting conflicts > because of vacuum is rather confusing. > > In the workloads where it migh

Re: [HACKERS] hot standby PSQL 9.1 Windows 2008 Servers

2012-06-26 Thread Robert Haas
On Tue, May 22, 2012 at 12:15 PM, chinnaobi wrote: > You mean when the primary which is going to switch its role to standby might > not have sent all the WAL records to the standby and If it is switched to > standby it has more WAL records than the standby which is now serves as > primary. Is it ?

Re: [HACKERS] hot standby PSQL 9.1 Windows 2008 Servers

2012-05-31 Thread chinnaobi
Sorry to mention, In my setup the primary and standby servers receive same traffic, so no issue with the "network fault between the primary and the standby, but not between the primary and some of the clients updating it" -- View this message in context: http://postgresql.1045698.n5.nabble.com/R

Re: [HACKERS] hot standby PSQL 9.1 Windows 2008 Servers

2012-05-31 Thread chinnaobi
Dear Kevin, Thank you for your reply. Yeah I am writing an application using powershell, it's true it is not trivial and especially a guy like me who has no idea on database. You raised all the cases which I am muddling with, But currently I am testing this setup: Always standby server is confi

Re: [HACKERS] hot standby PSQL 9.1 Windows 2008 Servers

2012-05-31 Thread Kevin Grittner
chinnaobi wrote: > You mean when the primary which is going to switch its role to > standby might not have sent all the WAL records to the standby and > If it is switched to standby it has more WAL records than the > standby which is now serves as primary. Is it ?? What happens when there is a

Re: [HACKERS] hot standby PSQL 9.1 Windows 2008 Servers

2012-05-22 Thread chinnaobi
Dear Robert, Thank you very much for the reply. You mean when the primary which is going to switch its role to standby might not have sent all the WAL records to the standby and If it is switched to standby it has more WAL records than the standby which is now serves as primary. Is it ?? It is

Re: [HACKERS] hot standby PSQL 9.1 Windows 2008 Servers

2012-05-22 Thread Robert Haas
On Mon, May 14, 2012 at 8:18 AM, chinnaobi wrote: > I do base backup only first time on standby when it is going to be > replicated. when ever primary goes down,  standby becomes primary and > primary becomes standby when primary comes up. When primary becomes standby > I am restoring data from WA

Re: [HACKERS] Hot Standby Failover Scenario

2012-02-28 Thread Greg Smith
On 02/27/2012 10:05 PM, Lucky Haryadi wrote: 3. When Master A fails to service, the database will failovered to Slave B by triggering with trigger file. As soon as you trigger a standby, it changes it to a new timeline. At that point, the series of WAL files diverges. It's no longer possible

Re: [HACKERS] Hot standby fails if any backend crashes

2012-02-04 Thread Simon Riggs
On Fri, Feb 3, 2012 at 4:48 AM, Tom Lane wrote: > I think saner behavior might only require this change: > >            /* >             * Any unexpected exit (including FATAL exit) of the startup >             * process is treated as a crash, except that we don't want to >             * reinitia

Re: [HACKERS] Hot standby fails if any backend crashes

2012-02-03 Thread Daniel Farina
On Thu, Feb 2, 2012 at 8:48 PM, Tom Lane wrote: > It's a bit disturbing that nobody has reported this from the field yet. > Seems to imply that hot standby isn't being used much. I have seen this, but didn't get to dig in, as I thought it could be a problem from other things done outside Postgres

Re: [HACKERS] Hot standby fails if any backend crashes

2012-02-02 Thread Fujii Masao
On Fri, Feb 3, 2012 at 1:48 PM, Tom Lane wrote: > I wrote: >> I'm currently working with Duncan Rance's test case for bug #6425, and >> I am observing a very nasty behavior in HEAD: once one of the >> hot-standby query backends crashes, the standby postmaster SIGQUIT's >> all its children and then

Re: [HACKERS] Hot standby fails if any backend crashes

2012-02-02 Thread Tom Lane
I wrote: > I'm currently working with Duncan Rance's test case for bug #6425, and > I am observing a very nasty behavior in HEAD: once one of the > hot-standby query backends crashes, the standby postmaster SIGQUIT's > all its children and then just quits itself, with no log message and > apparentl

Re: [HACKERS] Hot standby off of hot standby?

2012-01-30 Thread Josh Berkus
On 1/29/12 8:36 PM, Igor Schtein wrote: > Hi, > > Is it possible to use a standby instance as a master/primary for another > standby in Postgres 9.0? In other words, does PG 9.0 supports cascading > standby configuration? No, that's a 9.2 feature in development. If you can build PostgreSQL from

Re: [HACKERS] Hot standby off of hot standby?

2012-01-30 Thread Simon Riggs
On Mon, Jan 30, 2012 at 4:36 AM, Igor Schtein wrote: > Is it possible to use a standby instance as a master/primary for another > standby in Postgres 9.0?  In other words, does PG 9.0 supports cascading > standby configuration? No, but 9.2 will support that feature, known as cascading replicatio

Re: [HACKERS] Hot Standby startup with overflowed snapshots

2011-11-02 Thread Chris Redekop
oopsreply-to-all -- Forwarded message -- From: Chris Redekop Date: Wed, Nov 2, 2011 at 8:41 AM Subject: Re: [HACKERS] Hot Standby startup with overflowed snapshots To: Simon Riggs Sure, I've got quite a few logs lying around - I've attached 3 of 'em...let m

Re: [HACKERS] Hot Standby startup with overflowed snapshots

2011-11-02 Thread Simon Riggs
On Fri, Oct 28, 2011 at 3:42 AM, Chris Redekop wrote: > On a side note I am sporadically seeing another error on hotstandby startup. >  I'm not terribly concerned about it as it is pretty rare and it will work > on a retry so it's not a big deal.  The error is "FATAL:  out-of-order XID > insertio

Re: [HACKERS] Hot Standby startup with overflowed snapshots

2011-10-27 Thread Robert Haas
On Thu, Oct 27, 2011 at 6:55 PM, Simon Riggs wrote: > It seems cheap to add in a call to LogStandbySnapshot() after each > call to pg_stop_backup(). > > Does anyone think this case is worth adding code for? Seems like one > more thing to break. Why at that particular time? It would maybe nice if

Re: [HACKERS] Hot Standby startup with overflowed snapshots

2011-10-27 Thread Chris Redekop
Sorry..."designed" was poor choice of words, I meant "not unexpected". Doing the checkpoint right after pg_stop_backup() looks like it will work perfectly for me, so thanks for all your help! On a side note I am sporadically seeing another error on hotstandby startup. I'm not terribly concerned

Re: [HACKERS] Hot Standby startup with overflowed snapshots

2011-10-27 Thread Simon Riggs
On Thu, Oct 27, 2011 at 10:09 PM, Chris Redekop wrote: > hrmz, still basically the same behaviour.  I think it might be a *little* > better with this patch.  Before when under load it would start up quickly > maybe 2 or 3 times out of 10 attemptswith this patch it might be up to 4 > or 5 time

Re: [HACKERS] Hot Standby startup with overflowed snapshots

2011-10-27 Thread Chris Redekop
hrmz, still basically the same behaviour. I think it might be a *little* better with this patch. Before when under load it would start up quickly maybe 2 or 3 times out of 10 attemptswith this patch it might be up to 4 or 5 times out of 10...ish...or maybe it was just fluke *shrug*. I'm stil

Re: [HACKERS] Hot Standby startup with overflowed snapshots

2011-10-27 Thread Simon Riggs
On Thu, Oct 27, 2011 at 5:26 PM, Chris Redekop wrote: > Thanks for the patch Simon, but unfortunately it does not resolve the issue > I am seeing.  The standby still refuses to finish starting up until long > after all clients have disconnected from the primary (>10 minutes).  I do > see your new

Re: [HACKERS] Hot Standby startup with overflowed snapshots

2011-10-27 Thread Chris Redekop
Thanks for the patch Simon, but unfortunately it does not resolve the issue I am seeing. The standby still refuses to finish starting up until long after all clients have disconnected from the primary (>10 minutes). I do see your new log statement on startup, but only once - it does not repeat.

Re: [HACKERS] hot standby startup, visibility map, clog

2011-06-11 Thread Robert Haas
On Thu, Jun 9, 2011 at 5:14 AM, Daniel Farina wrote: > The first fact is that turning off hot standby will let the cluster > start up, but only after seeing a spate of messages like these (dozen > or dozens, not thousands): > > 2011-06-09 08:02:32 UTC  LOG:  restored log file > "0002002C00

Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age

2011-03-11 Thread Fujii Masao
On Fri, Mar 11, 2011 at 4:46 PM, Heikki Linnakangas wrote: >> Is this an open item for 9.1? > > Simon fixed it, commit b9075a6d2f9b07a00262a670dd60272904c79dce. Oh, thanks! Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-ha

Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age

2011-03-10 Thread Heikki Linnakangas
On 11.03.2011 06:21, Fujii Masao wrote: On Thu, Dec 9, 2010 at 4:55 PM, Simon Riggs wrote: On Thu, 2010-12-09 at 00:16 +0100, Heikki Linnakangas wrote: On 09.12.2010 00:10, Heikki Linnakangas wrote: On 08.12.2010 16:00, Simon Riggs wrote: Heikki pointed out to me that the btree delete record

Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age

2011-03-10 Thread Fujii Masao
On Thu, Dec 9, 2010 at 4:55 PM, Simon Riggs wrote: > On Thu, 2010-12-09 at 00:16 +0100, Heikki Linnakangas wrote: >> On 09.12.2010 00:10, Heikki Linnakangas wrote: >> > On 08.12.2010 16:00, Simon Riggs wrote: >> >> Heikki pointed out to me that the btree delete record processing does >> >> not res

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-12-15 Thread Joachim Wieland
On Tue, Dec 7, 2010 at 3:42 AM, Heikki Linnakangas wrote: > Ok, I've committed this patch now. I can confirm that I could continue replaying the logfiles on the standby host with this patch. Thanks a lot, Joachim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

Re: [HACKERS] Hot Standby tuning for btree_xlog_vacuum()

2010-12-09 Thread Simon Riggs
Just wanted to say thanks for the review, since I haven't yet managed to fix and commit this. I expect to later this month. On Mon, 2010-09-27 at 23:06 -0400, Robert Haas wrote: > On Thu, Apr 29, 2010 at 4:12 PM, Simon Riggs wrote: > > Simple tuning of btree_xlog_vacuum() using an idea I had a wh

Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age

2010-12-08 Thread Simon Riggs
On Thu, 2010-12-09 at 00:16 +0100, Heikki Linnakangas wrote: > On 09.12.2010 00:10, Heikki Linnakangas wrote: > > On 08.12.2010 16:00, Simon Riggs wrote: > >> Heikki pointed out to me that the btree delete record processing does > >> not respect vacuum_defer_cleanup_age. It should. > >> > >> Attach

Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age

2010-12-08 Thread Simon Riggs
On Thu, 2010-12-09 at 00:39 +0100, Heikki Linnakangas wrote: > vacuum_defer_cleanup_age should take effect in > the master, not during recovery. Hmmm, more to the point, it does take effect on the master and so there is no need for this at all. What were you thinking? What was I? Doh. -- Sim

Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age

2010-12-08 Thread Heikki Linnakangas
On 08.12.2010 16:00, Simon Riggs wrote: Heikki pointed out to me that the btree delete record processing does not respect vacuum_defer_cleanup_age. It should. Attached patch to implement that. This doesn't look right to me. btree_xlog_delete_get_latestRemovedXid() function calculates the lat

Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age

2010-12-08 Thread Heikki Linnakangas
On 09.12.2010 00:10, Heikki Linnakangas wrote: On 08.12.2010 16:00, Simon Riggs wrote: Heikki pointed out to me that the btree delete record processing does not respect vacuum_defer_cleanup_age. It should. Attached patch to implement that. Looking to commit in next few hours barring objections

Re: [HACKERS] Hot Standby btree delete records and vacuum_defer_cleanup_age

2010-12-08 Thread Heikki Linnakangas
On 08.12.2010 16:00, Simon Riggs wrote: Heikki pointed out to me that the btree delete record processing does not respect vacuum_defer_cleanup_age. It should. Attached patch to implement that. Looking to commit in next few hours barring objections/suggestions, to both HEAD and 9_0_STABLE, in ti

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-12-07 Thread Heikki Linnakangas
On 02.12.2010 12:31, Heikki Linnakangas wrote: On 02.12.2010 13:25, Simon Riggs wrote: On Thu, 2010-12-02 at 12:41 +0200, Heikki Linnakangas wrote: On 02.12.2010 11:02, Simon Riggs wrote: The cause of the issue is that replay starts at one LSN and there is a delay until the RunningXacts WAL re

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-12-02 Thread Heikki Linnakangas
On 02.12.2010 13:25, Simon Riggs wrote: On Thu, 2010-12-02 at 12:41 +0200, Heikki Linnakangas wrote: On 02.12.2010 11:02, Simon Riggs wrote: The cause of the issue is that replay starts at one LSN and there is a delay until the RunningXacts WAL record occurs. If there was no delay, there would

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-12-02 Thread Simon Riggs
On Thu, 2010-12-02 at 12:41 +0200, Heikki Linnakangas wrote: > On 02.12.2010 11:02, Simon Riggs wrote: > > The cause of the issue is that replay starts at one LSN and there is a > > delay until the RunningXacts WAL record occurs. If there was no delay, > > there would be no issue at all. In CreateC

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-12-02 Thread Heikki Linnakangas
On 02.12.2010 11:02, Simon Riggs wrote: The cause of the issue is that replay starts at one LSN and there is a delay until the RunningXacts WAL record occurs. If there was no delay, there would be no issue at all. In CreateCheckpoint() we start by grabbing the WAInsertLock and later recording tha

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-12-02 Thread Simon Riggs
On Thu, 2010-12-02 at 10:39 +0200, Heikki Linnakangas wrote: > On 01.12.2010 20:51, Heikki Linnakangas wrote: > > Another approach would be to revisit the way the running-xacts snapshot > > is taken. Currently, we first take a snapshot, and then WAL-log it. > > There is a small window between the s

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-12-02 Thread Heikki Linnakangas
On 01.12.2010 20:51, Heikki Linnakangas wrote: Another approach would be to revisit the way the running-xacts snapshot is taken. Currently, we first take a snapshot, and then WAL-log it. There is a small window between the steps where backends can begin/end transactions, and recovery has to deal

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-12-01 Thread Heikki Linnakangas
On 24.11.2010 12:48, Heikki Linnakangas wrote: When recovery starts, we fetch the oldestActiveXid from the checkpoint record. Let's say that it's 100. We then start replaying WAL records from the Redo pointer, and the first record (heap insert in your case) contains an Xid that's much larger than

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-24 Thread Simon Riggs
On Wed, 2010-11-24 at 12:48 +0200, Heikki Linnakangas wrote: > When recovery starts, we fetch the oldestActiveXid from the checkpoint > record. Let's say that it's 100. We then start replaying WAL records > from the Redo pointer, and the first record (heap insert in your case) > contains an Xid th

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-24 Thread Heikki Linnakangas
On 24.11.2010 13:38, Heikki Linnakangas wrote: It's dangerous to initialize latestObservedXid to anything to an older value. older value than the nextXid-1 from the checkpoint record, I meant to say. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hacker

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-24 Thread Heikki Linnakangas
On 24.11.2010 12:48, Heikki Linnakangas wrote: On 24.11.2010 06:56, Joachim Wieland wrote: On Tue, Nov 23, 2010 at 8:45 AM, Heikki Linnakangas wrote: On 19.11.2010 23:46, Joachim Wieland wrote: FATAL: too many KnownAssignedXids. head: 0, tail: 0, nxids: 9978, pArray->maxKnownAssignedXids: 68

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-24 Thread Heikki Linnakangas
On 24.11.2010 06:56, Joachim Wieland wrote: On Tue, Nov 23, 2010 at 8:45 AM, Heikki Linnakangas wrote: On 19.11.2010 23:46, Joachim Wieland wrote: FATAL: too many KnownAssignedXids. head: 0, tail: 0, nxids: 9978, pArray->maxKnownAssignedXids: 6890 Hmm, that's a lot of entries in KnownAssi

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-23 Thread Joachim Wieland
On Tue, Nov 23, 2010 at 8:45 AM, Heikki Linnakangas wrote: > On 19.11.2010 23:46, Joachim Wieland wrote: >> >> FATAL:  too many KnownAssignedXids. head: 0, tail: 0, nxids: 9978, >> pArray->maxKnownAssignedXids: 6890 > > Hmm, that's a lot of entries in KnownAssignedXids. > > Can you recompile with

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-22 Thread Heikki Linnakangas
On 19.11.2010 23:46, Joachim Wieland wrote: FATAL: too many KnownAssignedXids. head: 0, tail: 0, nxids: 9978, pArray->maxKnownAssignedXids: 6890 Hmm, that's a lot of entries in KnownAssignedXids. Can you recompile with WAL_DEBUG, and run the recovery again with wal_debug=on ? That will print

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-22 Thread Joachim Wieland
On Sun, Nov 21, 2010 at 11:48 PM, Fujii Masao wrote: > -- > If you suspect a bug in Hot Standby, please set >        trace_recovery_messages = DEBUG2 > in postgresql.conf and repeat the action > > Always useful to know > * max_connections > * current number of sessions > * whether we h

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-21 Thread Fujii Masao
On Sat, Nov 20, 2010 at 6:46 AM, Joachim Wieland wrote: > I still have the server, if you want me to debug anything or send a > patch against 9.0.1 that gives more output, just let me know. Per previous Simon's comment, the following information would be useful. http://archives.postgresql.org/pgs

Re: [HACKERS] Hot Standby b-tree delete records review

2010-11-09 Thread Heikki Linnakangas
(cleaning up my inbox, and bumped into this..) On 22.04.2010 12:31, Simon Riggs wrote: On Thu, 2010-04-22 at 12:18 +0300, Heikki Linnakangas wrote: Simon Riggs wrote: On Thu, 2010-04-22 at 11:56 +0300, Heikki Linnakangas wrote: If none of the removed heap tuples were present anymore, we curr

  1   2   3   4   5   6   7   8   9   10   >