Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-13 Thread Fujii Masao
On Thu, Apr 1, 2010 at 8:24 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Apr 1, 2010 at 7:18 AM, Simon Riggs si...@2ndquadrant.com wrote: I'm not willing to investigate this further myself at this stage. This looks like risk for little benefit. That's kind of what I figured.  I'll see

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-13 Thread Robert Haas
On Tue, Apr 13, 2010 at 9:18 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Apr 1, 2010 at 8:24 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Apr 1, 2010 at 7:18 AM, Simon Riggs si...@2ndquadrant.com wrote: I'm not willing to investigate this further myself at this stage. This

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-13 Thread Fujii Masao
On Tue, Apr 13, 2010 at 10:27 PM, Robert Haas robertmh...@gmail.com wrote: Can you explain how to recreate the problem that this patch fixes? 1. Configure and start the primary server. 2. Configure the standby server. 3. Remove all of the WAL files in pg_xlog of the standby. 4. Start the

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-01 Thread Fujii Masao
On Thu, Apr 1, 2010 at 12:16 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 31, 2010 at 5:02 AM, Simon Riggs si...@2ndquadrant.com wrote: From what I have seen, the comment about PM_WAIT_BACKENDS is incorrect. backends might be waiting for the WAL record that conflicts with their

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-01 Thread Robert Haas
On Thu, Apr 1, 2010 at 4:42 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Apr 1, 2010 at 12:16 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 31, 2010 at 5:02 AM, Simon Riggs si...@2ndquadrant.com wrote: From what I have seen, the comment about PM_WAIT_BACKENDS is incorrect.

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-01 Thread Simon Riggs
On Thu, 2010-04-01 at 06:48 -0400, Robert Haas wrote: On Thu, Apr 1, 2010 at 4:42 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Apr 1, 2010 at 12:16 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 31, 2010 at 5:02 AM, Simon Riggs si...@2ndquadrant.com wrote: From what I

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-04-01 Thread Robert Haas
On Thu, Apr 1, 2010 at 7:18 AM, Simon Riggs si...@2ndquadrant.com wrote: I'm not willing to investigate this further myself at this stage. This looks like risk for little benefit. That's kind of what I figured. I'll see about fixing up Fujii-san's patch and documenting the behavior; but it

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Simon Riggs
On Wed, 2010-03-31 at 10:48 +0900, Fujii Masao wrote: On Wed, Mar 31, 2010 at 9:47 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Mar 30, 2010 at 5:09 AM, Fujii Masao masao.fu...@gmail.com wrote: I rebased the patch to HEAD. Is the patch still required for 9.0? If not, I'd remove the

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Fujii Masao
On Wed, Mar 31, 2010 at 5:00 PM, Simon Riggs si...@2ndquadrant.com wrote: Please add some docs that a) explains what the patch does and b) notes any changes from behaviour in previous releases. ISTM this is a major change in behaviour. How about adding the following description into 17.5.

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Simon Riggs
On Wed, 2010-03-31 at 17:48 +0900, Fujii Masao wrote: On Wed, Mar 31, 2010 at 5:00 PM, Simon Riggs si...@2ndquadrant.com wrote: Please add some docs that a) explains what the patch does and b) notes any changes from behaviour in previous releases. ISTM this is a major change in behaviour.

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Fujii Masao
On Wed, Mar 31, 2010 at 6:02 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-03-31 at 17:48 +0900, Fujii Masao wrote: On Wed, Mar 31, 2010 at 5:00 PM, Simon Riggs si...@2ndquadrant.com wrote: Please add some docs that a) explains what the patch does and b) notes any changes from

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Robert Haas
On Wed, Mar 31, 2010 at 4:00 AM, Simon Riggs si...@2ndquadrant.com wrote: Please add some docs that a) explains what the patch does and b) notes any changes from behaviour in previous releases. ISTM this is a major change in behaviour. I guess I see this a little bit differently. If you do a

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-31 Thread Robert Haas
On Wed, Mar 31, 2010 at 5:02 AM, Simon Riggs si...@2ndquadrant.com wrote: From what I have seen, the comment about PM_WAIT_BACKENDS is incorrect. backends might be waiting for the WAL record that conflicts with their queries to be replayed. Recovery sometimes waits for backends, but

pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-30 Thread Fujii Masao
On Mon, Feb 1, 2010 at 11:49 AM, Fujii Masao masao.fu...@gmail.com wrote: On Sat, Jan 30, 2010 at 12:54 PM, Fujii Masao masao.fu...@gmail.com wrote: HOWEVER, I do believe this is an issue we could live with for 9.0 if it's going to lead to a whole lot of additional debugging of SR.  But if

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-30 Thread Robert Haas
On Tue, Mar 30, 2010 at 5:09 AM, Fujii Masao masao.fu...@gmail.com wrote: I rebased the patch to HEAD. Is the patch still required for 9.0? If not, I'd remove the open item of the smart shutdown during recovery. I am by no means an expert on this area of the code, but in the interest of moving

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-30 Thread Fujii Masao
On Wed, Mar 31, 2010 at 9:47 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Mar 30, 2010 at 5:09 AM, Fujii Masao masao.fu...@gmail.com wrote: I rebased the patch to HEAD. Is the patch still required for 9.0? If not, I'd remove the open item of the smart shutdown during recovery. I am

Re: pending patch: Re: [HACKERS] HS/SR and smart shutdown

2010-03-30 Thread Robert Haas
On Tue, Mar 30, 2010 at 9:48 PM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Mar 31, 2010 at 9:47 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Mar 30, 2010 at 5:09 AM, Fujii Masao masao.fu...@gmail.com wrote: I rebased the patch to HEAD. Is the patch still required for 9.0? If

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Fujii Masao
On Mon, Feb 1, 2010 at 11:49 AM, Fujii Masao masao.fu...@gmail.com wrote: On Sat, Jan 30, 2010 at 12:54 PM, Fujii Masao masao.fu...@gmail.com wrote: HOWEVER, I do believe this is an issue we could live with for 9.0 if it's going to lead to a whole lot of additional debugging of SR.  But if

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Greg Stark
On Thu, Mar 4, 2010 at 12:11 PM, Fujii Masao masao.fu...@gmail.com wrote: There is no post about this for over a month. Can I remove this from TODO item of SR for 9.0? Thought? Objection? Does smart shutdown still fail to shut down a slave? -- greg -- Sent via pgsql-hackers mailing list

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Fujii Masao
On Thu, Mar 4, 2010 at 11:55 PM, Greg Stark gsst...@mit.edu wrote: On Thu, Mar 4, 2010 at 12:11 PM, Fujii Masao masao.fu...@gmail.com wrote: There is no post about this for over a month. Can I remove this from TODO item of SR for 9.0? Thought? Objection? Does smart shutdown still fail to

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Robert Haas
On Thu, Mar 4, 2010 at 10:17 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Mar 4, 2010 at 11:55 PM, Greg Stark gsst...@mit.edu wrote: On Thu, Mar 4, 2010 at 12:11 PM, Fujii Masao masao.fu...@gmail.com wrote: There is no post about this for over a month. Can I remove this from TODO item

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Greg Stark
On Thu, Mar 4, 2010 at 3:56 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Mar 4, 2010 at 10:17 AM, Fujii Masao masao.fu...@gmail.com wrote: Yes. More precisely, smart shutdown during recovery does not complete until recovery ends. Well, I don't think we should let smart shutdown just

Re: [HACKERS] HS/SR and smart shutdown

2010-03-04 Thread Robert Haas
On Thu, Mar 4, 2010 at 12:39 PM, Greg Stark gsst...@mit.edu wrote: On Thu, Mar 4, 2010 at 3:56 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Mar 4, 2010 at 10:17 AM, Fujii Masao masao.fu...@gmail.com wrote: Yes. More precisely, smart shutdown during recovery does not complete until

Re: [HACKERS] HS/SR and smart shutdown

2010-01-31 Thread Magnus Hagander
On Sat, Jan 30, 2010 at 01:05, Robert Haas robertmh...@gmail.com wrote: On Fri, Jan 29, 2010 at 7:01 PM, Josh Berkus j...@agliodbs.com wrote: It's a good question if that still makes sense with Hot Standby. Perhaps we should redefine smart shutdown in standby mode to shut down as soon as all

Re: [HACKERS] HS/SR and smart shutdown

2010-01-31 Thread Fujii Masao
On Sat, Jan 30, 2010 at 12:54 PM, Fujii Masao masao.fu...@gmail.com wrote: HOWEVER, I do believe this is an issue we could live with for 9.0 if it's going to lead to a whole lot of additional debugging of SR. But if it's an easy fix, it'll avoid a lot of complaints on pgsql-general. I think

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Fujii Masao
On Thu, Jan 21, 2010 at 4:27 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: It's a good question if that still makes sense with Hot Standby. Perhaps we should redefine smart shutdown in standby mode to shut down as soon as all read-only connections have died. Okay. Let's

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Josh Berkus
Fujii, I guess that the startup process and the walreceiver should wait for all read only backends to exit in smart shutdown case. It's because those backends might be waiting for the record that conflicts with their queries to be replayed. Is this OK? Or we should kill the startup process

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Heikki Linnakangas
Josh Berkus wrote: I guess that the startup process and the walreceiver should wait for all read only backends to exit in smart shutdown case. It's because those backends might be waiting for the record that conflicts with their queries to be replayed. Is this OK? Or we should kill the

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Simon Riggs
On Thu, 2010-01-21 at 09:27 +0200, Heikki Linnakangas wrote: Right, that's the way a standby server (= one still in recovery) has always behaved. It has made sense in the past: it's not in the spirit of smart shutdown to kill the WAL replay immediately. smart means wait for recovery to

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Josh Berkus
It's a good question if that still makes sense with Hot Standby. Perhaps we should redefine smart shutdown in standby mode to shut down as soon as all read-only connections have died. It's clear that smart shutdown doesn't work while something is active. Recovery is active and so we

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Robert Haas
On Fri, Jan 29, 2010 at 7:01 PM, Josh Berkus j...@agliodbs.com wrote: It's a good question if that still makes sense with Hot Standby. Perhaps we should redefine smart shutdown in standby mode to shut down as soon as all read-only connections have died. It's clear that smart shutdown doesn't

Re: [HACKERS] HS/SR and smart shutdown

2010-01-29 Thread Fujii Masao
On Sat, Jan 30, 2010 at 9:01 AM, Josh Berkus j...@agliodbs.com wrote: I don't think it's clear, or intuitive for users.  In SR, recovery is *never* done, so smart shutdown never completes (even if the master is shut down, when I tested it). If you specify the trigger_file parameter in the

Re: [HACKERS] HS/SR and smart shutdown

2010-01-21 Thread Greg Smith
Heikki Linnakangas wrote: It's a good question if that still makes sense with Hot Standby. Perhaps we should redefine smart shutdown in standby mode to shut down as soon as all read-only connections have died. I've advocated in the past that an escalating shutdown procedure would be

[HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Josh Berkus
I've been working on my demo, and I'm discovering that due to the connection from the walsender and walreceiver, smart shutdown from pg_ctl doesn't work if replication is active. This seems worth fixing; if we don't fix it, we should at least document it. Comments? --Josh -- Sent via

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Fujii Masao
On Thu, Jan 21, 2010 at 8:04 AM, Josh Berkus j...@agliodbs.com wrote: I've been working on my demo, and I'm discovering that due to the connection from the walsender and walreceiver, smart shutdown from pg_ctl doesn't work if replication is active. This seems worth fixing; if we don't fix it,

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Josh Berkus
If it's standby, it's a previously-existing behavior that a smart shutdown doesn't work immediately during recovery. After a recovery has been completed, it would work. Of course, I agree that such a behavior should be documented. Well, as long as streaming rep is running, you can't do a

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Robert Haas
On Wed, Jan 20, 2010 at 8:44 PM, Josh Berkus j...@agliodbs.com wrote: If it's standby, it's a previously-existing behavior that a smart shutdown doesn't work immediately during recovery. After a recovery has been completed, it would work. Of course, I agree that such a behavior should be

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jan 20, 2010 at 8:44 PM, Josh Berkus j...@agliodbs.com wrote: Well, as long as streaming rep is running, you can't do a smart shutdown ... smart shutdown seems to treat the walreciever as a client connection.  At the very least, this should be

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Robert Haas
On Wed, Jan 20, 2010 at 8:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jan 20, 2010 at 8:44 PM, Josh Berkus j...@agliodbs.com wrote: Well, as long as streaming rep is running, you can't do a smart shutdown ... smart shutdown seems to treat the

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Mark Kirkwood
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jan 20, 2010 at 8:44 PM, Josh Berkus j...@agliodbs.com wrote: Well, as long as streaming rep is running, you can't do a smart shutdown ... smart shutdown seems to treat the walreciever as a client connection. At the

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Fujii Masao
On Thu, Jan 21, 2010 at 10:44 AM, Josh Berkus j...@agliodbs.com wrote: If it's standby, it's a previously-existing behavior that a smart shutdown doesn't work immediately during recovery. After a recovery has been completed, it would work. Of course, I agree that such a behavior should be

Re: [HACKERS] HS/SR and smart shutdown

2010-01-20 Thread Heikki Linnakangas
Fujii Masao wrote: On Thu, Jan 21, 2010 at 10:44 AM, Josh Berkus j...@agliodbs.com wrote: If it's standby, it's a previously-existing behavior that a smart shutdown doesn't work immediately during recovery. After a recovery has been completed, it would work. Of course, I agree that such a