Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2015-01-02 Thread Dennis Kögel
Am 31.12.2014 um 11:40 schrieb Michael Paquier : >> As long as master is fixed, I don't actually care. But I agree with Dennis >> that it's hard to see what's been commited with all the different issues >> found, and if any commits were done, in which branch. I'd like to be able to >> tell my custo

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-31 Thread Michael Paquier
On Wed, Dec 31, 2014 at 4:44 PM, Guillaume Lelarge wrote: > 2014-12-12 14:58 GMT+01:00 Heikki Linnakangas : >> Now, what do we do with the back-branches? I'm not sure. Changing the >> behaviour in back-branches could cause nasty surprises. Perhaps it's best to >> just leave it as it is, even thoug

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-30 Thread Guillaume Lelarge
2014-12-12 14:58 GMT+01:00 Heikki Linnakangas : > On 12/10/2014 04:32 PM, Dennis Kögel wrote: > >> Hi, >> >> Am 04.09.2014 um 17:50 schrieb Jehan-Guillaume de Rorthais < >> j...@dalibo.com>: >> >>> Since few months, we occasionally see .ready files appearing on some >>> slave >>> instances from va

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-12 Thread Heikki Linnakangas
On 12/10/2014 04:32 PM, Dennis Kögel wrote: Hi, Am 04.09.2014 um 17:50 schrieb Jehan-Guillaume de Rorthais : Since few months, we occasionally see .ready files appearing on some slave instances from various context. The two I have in mind are under 9.2.x. […] So it seems for some reasons, these

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-10 Thread Dennis Kögel
Hi, Am 04.09.2014 um 17:50 schrieb Jehan-Guillaume de Rorthais : > Since few months, we occasionally see .ready files appearing on some slave > instances from various context. The two I have in mind are under 9.2.x. […] > So it seems for some reasons, these old WALs were "forgotten" by the > resta

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-28 Thread Heikki Linnakangas
On 10/27/2014 06:12 PM, Heikki Linnakangas wrote: On 10/27/2014 02:12 PM, Fujii Masao wrote: >On Fri, Oct 24, 2014 at 10:05 PM, Heikki Linnakangas > wrote: >>On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: >>> >>>At least for master, we should consider changing the way the archiving >>>work

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-28 Thread Fujii Masao
On Tue, Oct 28, 2014 at 1:12 AM, Heikki Linnakangas wrote: > On 10/27/2014 02:12 PM, Fujii Masao wrote: >> >> On Fri, Oct 24, 2014 at 10:05 PM, Heikki Linnakangas >> wrote: >>> >>> On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: At least for master, we should consider changing th

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-27 Thread Heikki Linnakangas
On 10/27/2014 02:12 PM, Fujii Masao wrote: On Fri, Oct 24, 2014 at 10:05 PM, Heikki Linnakangas wrote: On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: At least for master, we should consider changing the way the archiving works so that we only archive WAL that was generated in the same serv

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-27 Thread Fujii Masao
On Fri, Oct 24, 2014 at 10:05 PM, Heikki Linnakangas wrote: > On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: >> >> At least for master, we should consider changing the way the archiving >> works so that we only archive WAL that was generated in the same server. >> I.e. we should never try to ar

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-24 Thread Robert Haas
On Fri, Oct 24, 2014 at 9:05 AM, Heikki Linnakangas wrote: > So, this is what I came up with for master. Does anyone see a problem with > it? In the proposed commit message, you mis-spelled "significant" as "signicant". -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Pos

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-24 Thread Michael Paquier
On Fri, Oct 24, 2014 at 3:05 PM, Heikki Linnakangas wrote: > On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: >> >> At least for master, we should consider changing the way the archiving >> works so that we only archive WAL that was generated in the same server. >> I.e. we should never try to arc

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-24 Thread Heikki Linnakangas
On 10/23/2014 11:09 AM, Heikki Linnakangas wrote: At least for master, we should consider changing the way the archiving works so that we only archive WAL that was generated in the same server. I.e. we should never try to archive WAL files belonging to another timeline. I just remembered that we

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Michael Paquier
On Thu, Oct 23, 2014 at 10:16 PM, Michael Paquier wrote: > On Thu, Oct 23, 2014 at 1:45 PM, Heikki Linnakangas < > hlinnakan...@vmware.com> wrote: > That's not right. Should check *after* the write if the segment was > completed, and close it if so. Like the attached. > > Just tested this patch w

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Michael Paquier
On Thu, Oct 23, 2014 at 1:45 PM, Heikki Linnakangas wrote: > On 10/23/2014 01:25 PM, Michael Paquier wrote: > >> On Thu, Oct 23, 2014 at 10:09 AM, Heikki Linnakangas < >> hlinnakan...@vmware.com> wrote: >> >> On 10/23/2014 08:59 AM, Fujii Masao wrote: >>> Sounds reasonable, for back-branches. Al

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Michael Paquier
On Thu, Oct 23, 2014 at 2:34 PM, Fujii Masao wrote: > On Thu, Oct 23, 2014 at 9:23 PM, Fujii Masao > wrote: > > On Thu, Oct 23, 2014 at 8:45 PM, Heikki Linnakangas > > wrote: > >> On 10/23/2014 01:25 PM, Michael Paquier wrote: > >>> > >>> On Thu, Oct 23, 2014 at 10:09 AM, Heikki Linnakangas < >

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Fujii Masao
On Thu, Oct 23, 2014 at 9:23 PM, Fujii Masao wrote: > On Thu, Oct 23, 2014 at 8:45 PM, Heikki Linnakangas > wrote: >> On 10/23/2014 01:25 PM, Michael Paquier wrote: >>> >>> On Thu, Oct 23, 2014 at 10:09 AM, Heikki Linnakangas < >>> hlinnakan...@vmware.com> wrote: >>> On 10/23/2014 08:59 AM,

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Fujii Masao
On Thu, Oct 23, 2014 at 8:45 PM, Heikki Linnakangas wrote: > On 10/23/2014 01:25 PM, Michael Paquier wrote: >> >> On Thu, Oct 23, 2014 at 10:09 AM, Heikki Linnakangas < >> hlinnakan...@vmware.com> wrote: >> >>> On 10/23/2014 08:59 AM, Fujii Masao wrote: >>> Sounds reasonable, for back-branches. Al

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Fujii Masao
On Thu, Oct 23, 2014 at 5:09 PM, Heikki Linnakangas wrote: > On 10/23/2014 08:59 AM, Fujii Masao wrote: >> >> On Mon, Oct 20, 2014 at 3:26 PM, Michael Paquier >> wrote: >>> >>> On Fri, Oct 17, 2014 at 10:37 PM, Michael Paquier >>> wrote: On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masa

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Heikki Linnakangas
On 10/23/2014 01:25 PM, Michael Paquier wrote: On Thu, Oct 23, 2014 at 10:09 AM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: On 10/23/2014 08:59 AM, Fujii Masao wrote: Sounds reasonable, for back-branches. Although I'm still worried we might miss some corner-case unless we go with a mo

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Michael Paquier
On Thu, Oct 23, 2014 at 10:09 AM, Heikki Linnakangas < hlinnakan...@vmware.com> wrote: > On 10/23/2014 08:59 AM, Fujii Masao wrote: > Sounds reasonable, for back-branches. Although I'm still worried we might > miss some corner-case unless we go with a more wholesale solution. > Don't really want

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Heikki Linnakangas
On 10/23/2014 08:59 AM, Fujii Masao wrote: On Mon, Oct 20, 2014 at 3:26 PM, Michael Paquier wrote: On Fri, Oct 17, 2014 at 10:37 PM, Michael Paquier wrote: On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao wrote: In this case, the patch seems to make the restartpoint recycle even WAL files whi

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-23 Thread Fujii Masao
On Mon, Oct 20, 2014 at 5:15 PM, Michael Paquier wrote: > > > On Fri, Oct 17, 2014 at 10:12 PM, Fujii Masao wrote: >> >> On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao >> wrote: >> > On Thu, Oct 9, 2014 at 3:26 PM, Michael Paquier >> > wrote: >> >> >> >> >> >> On Wed, Oct 8, 2014 at 10:00 PM, Mic

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-22 Thread Fujii Masao
On Mon, Oct 20, 2014 at 3:26 PM, Michael Paquier wrote: > On Fri, Oct 17, 2014 at 10:37 PM, Michael Paquier > wrote: >> >> On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao wrote: >>> >>> In this case, the patch seems to make the restartpoint recycle even WAL >>> files >>> which have .ready files an

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-22 Thread Heikki Linnakangas
On 10/22/2014 04:24 PM, Michael Paquier wrote: On Wed, Oct 22, 2014 at 1:53 PM, Heikki Linnakangas wrote: I think we should take a more wholesale approach to this. We should enforce the rule that the server only ever archives WAL files belonging to the same timeline that the server generates.

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-22 Thread Michael Paquier
On Fri, Oct 17, 2014 at 2:23 PM, Fujii Masao wrote: > I found one problem in the 0002 patch. The patch changes the recovery so > that > it creates .done files for every WAL files which exist in pg_xlog > directory at > the end of recovery. But even WAL files which will have to be archived > later

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-22 Thread Heikki Linnakangas
On 10/20/2014 09:26 AM, Michael Paquier wrote: On Fri, Oct 17, 2014 at 10:37 PM, Michael Paquier wrote: On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao wrote: In this case, the patch seems to make the restartpoint recycle even WAL files which have .ready files and will have to be archived late

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-20 Thread Michael Paquier
On Fri, Oct 17, 2014 at 10:12 PM, Fujii Masao wrote: > On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao > wrote: > > On Thu, Oct 9, 2014 at 3:26 PM, Michael Paquier > > wrote: > >> > >> > >> On Wed, Oct 8, 2014 at 10:00 PM, Michael Paquier < > michael.paqu...@gmail.com> > >> wrote: > >>> > >>> The

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-19 Thread Michael Paquier
On Fri, Oct 17, 2014 at 10:37 PM, Michael Paquier wrote: > > On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao wrote: >> >> In this case, the patch seems to make the restartpoint recycle even WAL files >> which have .ready files and will have to be archived later. Thought? > > The real problem current

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-17 Thread Michael Paquier
On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao wrote: > In this case, the patch seems to make the restartpoint recycle even WAL > files > which have .ready files and will have to be archived later. Thought? > The real problem currently is that it is possible to have a segment file not marked as .do

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-17 Thread Fujii Masao
On Fri, Oct 17, 2014 at 9:23 PM, Fujii Masao wrote: > On Thu, Oct 9, 2014 at 3:26 PM, Michael Paquier > wrote: >> >> >> On Wed, Oct 8, 2014 at 10:00 PM, Michael Paquier >> wrote: >>> >>> The additional process at promotion sounds like a good idea, I'll try to >>> get a patch done tomorrow. This

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-17 Thread Fujii Masao
On Thu, Oct 9, 2014 at 3:26 PM, Michael Paquier wrote: > > > On Wed, Oct 8, 2014 at 10:00 PM, Michael Paquier > wrote: >> >> The additional process at promotion sounds like a good idea, I'll try to >> get a patch done tomorrow. This would result as well in removing the >> XLogArchiveForceDone stu

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-09 Thread Michael Paquier
On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas wrote: > 1. Where do the FF files come from? In 9.2, FF-segments are not supposed to > created, ever. > > I think we should add a check in walreceiver, to throw an error if the > master sends an invalid WAL pointer, pointing to an FF segment. Atta

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Michael Paquier
On Wed, Oct 8, 2014 at 10:00 PM, Michael Paquier wrote: > The additional process at promotion sounds like a good idea, I'll try to > get a patch done tomorrow. This would result as well in removing the > XLogArchiveForceDone stuff. Either way, not that I have been able to > reproduce the problem

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Michael Paquier
On Wed, Oct 8, 2014 at 11:38 PM, Heikki Linnakangas wrote: > On 10/08/2014 04:59 PM, Fujii Masao wrote: > >> On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas >> wrote: >> >>> Instead of creating any .done files during recovery, we could scan >>> pg_xlog >>> at promotion, and create a .done fil

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Heikki Linnakangas
On 10/08/2014 04:59 PM, Fujii Masao wrote: On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas wrote: Instead of creating any .done files during recovery, we could scan pg_xlog at promotion, and create a .done file for every WAL segment that's present at that point. That would be more robust. An

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Fujii Masao
On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas wrote: > On 10/08/2014 10:44 AM, Michael Paquier wrote: >> >> On Fri, Sep 19, 2014 at 1:07 AM, Jehan-Guillaume de Rorthais < >> j...@dalibo.com> wrote: >> >>> We kept the WAL files and log files for further analysis. How can we help >>> regarding

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Michael Paquier
On Wed, Oct 8, 2014 at 6:54 PM, Heikki Linnakangas wrote: > 1. Where do the FF files come from? In 9.2, FF-segments are not supposed > to created, ever. > Since this only happens with streaming replication, the FF segments are > probably being created by walreceiver. XLogWalRcvWrite is the functi

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Heikki Linnakangas
On 10/08/2014 10:44 AM, Michael Paquier wrote: On Fri, Sep 19, 2014 at 1:07 AM, Jehan-Guillaume de Rorthais < j...@dalibo.com> wrote: We kept the WAL files and log files for further analysis. How can we help regarding this issue? Commit c2f79ba has added as assumption that the WAL receiver s

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-10-08 Thread Michael Paquier
On Fri, Sep 19, 2014 at 1:07 AM, Jehan-Guillaume de Rorthais < j...@dalibo.com> wrote: > We kept the WAL files and log files for further analysis. How can we help > regarding this issue? > Commit c2f79ba has added as assumption that the WAL receiver should always enforce the create of .done files

[HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-09-18 Thread Jehan-Guillaume de Rorthais
Hi hackers, We spend some time with Guillaume Lelarge studying this issue. CreateRestartPoint() calls RemoveOldXlogFiles() to drop/recycle old WALs. This one is calling XLogArchiveCheckDone() to check if the given WAL can be dropped. As our slave has "archive_mode" & "archive_command" set, XLogAr