Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-05-16 Thread Kyotaro HORIGUCHI
Hello, On Sun, May 13, 2012 at 10:38 PM, Kyotaro HORIGUCHI wrote: > I've overlooked that startup process of the standby reads > archives first, and then WAL. But the current patch enables > progress governing based on checkpoint_segments during archive > recovery on the standby. I forcused on Wa

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-05-13 Thread Kyotaro HORIGUCHI
Hello, I've returned from my overseas trip for just over one week. # and I'll leave Japan again after this... > >     restorePtr <= replayPtr <= receivePtr > > > > But XLByteLT(recievePtr, replayPtr) this should not return true > > under the condition above.. Something wrong in my assumption? >

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-27 Thread Kyotaro HORIGUCHI
Sorry for broken message. > >From I said that the former (spinlock) could be dropped, but the > latter (read as volatile) should be needed. I said that the former (spinlock) could be dropped from the view of functionarity, but the latter (read as volatile) should be needed. > >From the view of

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-27 Thread Kyotaro HORIGUCHI
Thank you for sugestion. > This still makes catching up in standby mode slower, as you get > many more restartpoints. The reason for ignoring > checkpoint_segments during recovery was to avoid that. I may have a misunderstanding around there, or your intention. I understand that standby creates

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-26 Thread Kyotaro HORIGUCHI
Hello, At Wed, 25 Apr 2012 02:31:24 +0900, Fujii Masao wrote in > > If we are allowed to be tolerant of the temporary lack of > > coherence in shared memory there, the spinlock could be removed. > > But the possibility to read garbage by using XLogCtl itself to > > access standbyMode does not s

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-24 Thread Fujii Masao
On Tue, Apr 24, 2012 at 3:47 PM, Kyotaro HORIGUCHI wrote: > Hello, > >> >  - xlog.c: Make StandbyMode shared. >> > >> >  - checkpointer.c: Use IsStandbyMode() to check if postmaster is >> >   under standby mode. >> >> IsStandbyMode() looks overkill to me. The standby mode flag is forcibly >> turne

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-24 Thread Fujii Masao
On Tue, Apr 24, 2012 at 3:53 PM, Heikki Linnakangas wrote: > On 23.04.2012 02:59, Fujii Masao wrote: >> >> On Thu, Apr 19, 2012 at 2:20 PM, Kyotaro HORIGUCHI >>  wrote: >>> >>> Hello, this is new version of standby checkpoint_segments patch. >> >> >> Thanks for the patch! > > > This still makes c

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-23 Thread Heikki Linnakangas
On 23.04.2012 02:59, Fujii Masao wrote: On Thu, Apr 19, 2012 at 2:20 PM, Kyotaro HORIGUCHI wrote: Hello, this is new version of standby checkpoint_segments patch. Thanks for the patch! This still makes catching up in standby mode slower, as you get many more restartpoints. The reason for

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-23 Thread Kyotaro HORIGUCHI
Hello, > >  - xlog.c: Make StandbyMode shared. > > > >  - checkpointer.c: Use IsStandbyMode() to check if postmaster is > >   under standby mode. > > IsStandbyMode() looks overkill to me. The standby mode flag is forcibly > turned off at the end of recovery, but its change doesn't need to be shar

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-22 Thread Fujii Masao
On Thu, Apr 19, 2012 at 2:20 PM, Kyotaro HORIGUCHI wrote: > Hello, this is new version of standby checkpoint_segments patch. Thanks for the patch! >  - xlog.c: Make StandbyMode shared. > >  - checkpointer.c: Use IsStandbyMode() to check if postmaster is >   under standby mode. IsStandbyMode() l

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-18 Thread Kyotaro HORIGUCHI
Hello, this is new version of standby checkpoint_segments patch. - xlog.c: Make StandbyMode shared. - checkpointer.c: Use IsStandbyMode() to check if postmaster is under standby mode. regards, -- Kyotaro Horiguchi NTT Open Source Software Center == My e-mail address has been changed sin

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-18 Thread Kyotaro HORIGUCHI
I convinced that current patch has a problem, and will come up with the new patch later. > > I tried that at first. But I suppose the requirement here is 'if > > reading segments comes via replication stream, enable throttling > > by checkpoint_segments.' and WalRcvInProgress() seems fit to >

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-18 Thread Fujii Masao
On Wed, Apr 18, 2012 at 10:22 AM, Kyotaro HORIGUCHI wrote: > I tried that at first. But I suppose the requirement here is 'if > reading segments comes via replication stream, enable throttling > by checkpoint_segments.' and WalRcvInProgress() seems fit to > check that. If so, what if replication

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
> > Hmm. StandbyMode is a local variable which cannot be accessed in > > checkpointer. But WalRcvInProgress() which shows if wal receiver > > is running seems to be usable to ENABLE governing progress by > > checkpoint_segments. > > Even when walreceiver is not running and WAL files are read from

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Fujii Masao
On Tue, Apr 17, 2012 at 11:50 PM, Kyotaro HORIGUCHI wrote: > Hmm. StandbyMode is a local variable which cannot be accessed in > checkpointer. But WalRcvInProgress() which shows if wal receiver > is running seems to be usable to ENABLE governing progress by > checkpoint_segments. Even when walrece

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Fujii Masao
On Tue, Apr 17, 2012 at 3:50 PM, Kyotaro HORIGUCHI wrote: > These seems quite reasonable. These conditions make following > conditional expression. > >     restorePtr <= replayPtr <= receivePtr > > But XLByteLT(recievePtr, replayPtr) this should not return true > under the condition above.. Someth

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
Hello, this message is attached with the patch which did not tested. That is for show the way. On Tue, Apr 17, 2012 at 9:38 PM, Kyotaro HORIGUCHI wrote: > But I think referring checkpoint_segment on such case should be > inhibited, and I suppose it is possible using StandbyMode in > IsCheckpointO

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
Sorry, I've wrote something wrong. >> The reason we haven't historically obeyed checkpoint_segments >> during recovery is that it slows down the recovery >> unnecessarily if you're restoring from a backup and you replay, > > The variable StandbyMode is false on archive recovery, so no > checkpoint

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Kyotaro HORIGUCHI
Hello, > The reason we haven't historically obeyed checkpoint_segments > during recovery is that it slows down the recovery > unnecessarily if you're restoring from a backup and you replay, The variable StandbyMode is false on archive recovery, so no checkpoint triggerred during then. xlog.c:100

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-17 Thread Heikki Linnakangas
On 17.04.2012 09:50, Kyotaro HORIGUCHI wrote: This is new version of the patch. I replaced GetStandbyFlushRecPtr with GetXLogReplayRecPtr to check progress of checkpoint following Fujii's sugestion. The reason we haven't historically obeyed checkpoint_segments during recovery is that it slows

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-16 Thread Kyotaro HORIGUCHI
This is new version of the patch. I replaced GetStandbyFlushRecPtr with GetXLogReplayRecPtr to check progress of checkpoint following Fujii's sugestion. The first one is for 9.2dev, and the second is 9.1.3 backported version. === By the way, I took a close look around there, > I agree with it ba

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-16 Thread Kyotaro HORIGUCHI
Hello, thank you for comment. > > In the backported version to 9.1.3, bgwriter.c is modified > > instead of checkpointer.c in 9.2. And GetWalRcvWriteRecPtr() is > > used as the equivalent of GetStandbyFlushRecPtr() in 9.2. > > In 9,2, GetXLogReplayRecPtr() should be used instead of > GetStandbyFl

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-16 Thread Fujii Masao
On Mon, Apr 16, 2012 at 9:05 PM, Kyotaro HORIGUCHI wrote: > In the backported version to 9.1.3, bgwriter.c is modified > instead of checkpointer.c in 9.2. And GetWalRcvWriteRecPtr() is > used as the equivalent of GetStandbyFlushRecPtr() in 9.2. In 9,2, GetXLogReplayRecPtr() should be used instead

Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-16 Thread Simon Riggs
On Mon, Apr 16, 2012 at 1:05 PM, Kyotaro HORIGUCHI wrote: > Hello, this is bug report and a patch for it. > > The first patch in the attachments is for 9.2dev and next one is > for 9.1.3. > > On the current 9.2dev, IsCheckpointOnSchedule@checkpointer.c does > not check against WAL segments writte

[HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments

2012-04-16 Thread Kyotaro HORIGUCHI
Hello, this is bug report and a patch for it. The first patch in the attachments is for 9.2dev and next one is for 9.1.3. On the current 9.2dev, IsCheckpointOnSchedule@checkpointer.c does not check against WAL segments written. This makes checkpointer always run at the speed according to checkpoi