Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-25 Thread Heikki Linnakangas
On 24.01.2013 19:44, Simon Riggs wrote: On 24 January 2013 16:52, Heikki Linnakangashlinnakan...@vmware.com wrote: I may be missing something, but it looks like after a fast promotion, you don't request a new checkpoint. So it can take quite a while for the next checkpoint to be triggered by

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-25 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: There's no hard correctness reason here for any particular behavior, I just feel that that would make most sense. It seems prudent to initiate a checkpoint right after timeline switch, so that you get a new checkpoint on the new timeline

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-25 Thread Simon Riggs
On 25 January 2013 12:15, Heikki Linnakangas hlinnakan...@vmware.com wrote: 1) an immediate checkpoint can cause a disk/resource usage spike, which is definitely not what you need just when a spike of connections and new SQL hits the system. It doesn't need to be an immediate checkpoint,

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-24 Thread Simon Riggs
On 6 January 2013 21:58, Simon Riggs si...@2ndquadrant.com wrote: On 9 August 2012 10:45, Simon Riggs si...@2ndquadrant.com wrote: On 22 June 2012 05:03, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: I hope this is promising. I've reviewed this and thought about it over some

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-24 Thread Heikki Linnakangas
On 24.01.2013 18:24, Simon Riggs wrote: On 6 January 2013 21:58, Simon Riggssi...@2ndquadrant.com wrote: I've been torn between the need to remove the checkpoint for speed and being worried about the implications of doing so. We promote in multiple use cases. When we end a PITR, or are

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-24 Thread Simon Riggs
On 24 January 2013 16:52, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 24.01.2013 18:24, Simon Riggs wrote: On 6 January 2013 21:58, Simon Riggssi...@2ndquadrant.com wrote: I've been torn between the need to remove the checkpoint for speed and being worried about the implications

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-24 Thread Simon Riggs
On 24 January 2013 17:44, Simon Riggs si...@2ndquadrant.com wrote: At replay, an end-of-recovery record should be a signal to the hot standby mechanism that there are no transactions running in the master at that point, same as a shutdown checkpoint. I had a reason why I didn't do that, but

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2013-01-06 Thread Simon Riggs
On 9 August 2012 10:45, Simon Riggs si...@2ndquadrant.com wrote: On 22 June 2012 05:03, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: I hope this is promising. I've reviewed this and thought about it over some time. I've been torn between the need to remove the checkpoint for

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-10-18 Thread Alvaro Herrera
This patch seems to have been neglected by both its submitter and the reviewer. Also, Simon said he was going to set it returned-with-feedback on his last reply, but I see it as needs-review still in the CF app. Is this something that is going to be reconsidered and resubmitted for the next

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-10-18 Thread Simon Riggs
On 18 October 2012 21:22, Alvaro Herrera alvhe...@2ndquadrant.com wrote: This patch seems to have been neglected by both its submitter and the reviewer. Also, Simon said he was going to set it returned-with-feedback on his last reply, but I see it as needs-review still in the CF app. Is

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-08-30 Thread Kyotaro HORIGUCHI
Hello, sorry for long absense. At first I was unhappy that you'd removed the restriction that timelines only change on a shutdown checkpoint. But the reality is timelines change at any point in the WAL stream - the only way to tell between end of WAL and a timeline change is by looking for

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-08-09 Thread Simon Riggs
On 22 June 2012 05:03, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: I hope this is promising. I've reviewed this and thought about it over some time. At first I was unhappy that you'd removed the restriction that timelines only change on a shutdown checkpoint. But the reality is

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-21 Thread Kyotaro HORIGUCHI
Hello, Is it guaranteed that all the files (e.g., the latest timeline history file) required for such crash recovery exist in pg_xlog? If yes, your approach might work well. Particularly regarding the promotion, the files reuiqred are the history file of the latest timeline, the WAL file

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-19 Thread Kyotaro HORIGUCHI
Thank you. What happens if the server skips an end-of-recovery checkpoint, is promoted to the master, runs some write transactions, crashes and restarts automatically before it completes checkpoint? In this case, the server needs to do crash recovery from the last checkpoint record with old

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-19 Thread Fujii Masao
On Tue, Jun 19, 2012 at 5:30 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Thank you. What happens if the server skips an end-of-recovery checkpoint, is promoted to the master, runs some write transactions, crashes and restarts automatically before it completes checkpoint? In

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-18 Thread Kyotaro HORIGUCHI
Hello, This is the new version of the patch. Your patch introduced new WAL record type XLOG_END_OF_RECOVERY to mark the chenge point of TLI. But I think the information is already stored in history files and already ready to use in current code. I looked into your first patch and looked over the

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-18 Thread Fujii Masao
On Mon, Jun 18, 2012 at 5:42 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: What do you think about this? What happens if the server skips an end-of-recovery checkpoint, is promoted to the master, runs some write transactions, crashes and restarts automatically before it completes

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-12 Thread Simon Riggs
On 12 June 2012 03:38, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Hello, sorry for vague understanding. I depend on this and suppose we can omit it if latest checkpoint has been taken so as to be able to do crash recovery thereafter. I don't see any reason to special case

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-12 Thread Kyotaro HORIGUCHI
Hello, Thank you to head me the previous discussion. I'll consider them from now. I want the standby to start to serve as soon as possible even by a few seconds on failover in a HA cluster. Please implement a prototype and measure how many seconds we are discussing. I'm sorry to have

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-11 Thread Kyotaro HORIGUCHI
Hello, sorry for vague understanding. I depend on this and suppose we can omit it if latest checkpoint has been taken so as to be able to do crash recovery thereafter. I don't see any reason to special case this. If a checkpoint has no work to do, then it will go very quickly. Why seek to

[HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-08 Thread Kyotaro HORIGUCHI
Hello, I have a problem with promoting from hot-standby that exclusive checkpointing retards completion of promotion. This checkpoint is shutdown checkpoint as a convention in realtion to TLI increment according to the comment shown below. I suppose shutdown checkpoint means exclusive checkpoint

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-08 Thread Simon Riggs
On 8 June 2012 09:22, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: I have a problem with promoting from hot-standby that exclusive checkpointing retards completion of promotion. Agreed, we have that problem. I depend on this and suppose we can omit it if latest checkpoint has