Re: Why standby restores some WALs many times from archive?

2018-01-10 Thread Jeff Janes
On Sat, Dec 30, 2017 at 4:20 AM, Michael Paquier wrote: > On Sat, Dec 30, 2017 at 04:30:07AM +0300, Sergey Burladyan wrote: > > We use this scripts: > > https://github.com/avito-tech/dba-utils/tree/master/pg_archive > > > > But I can reproduce problem with simple cp & mv: > > archive_command: > >

Re: Why standby restores some WALs many times from archive?

2018-01-10 Thread Sergey Burladyan
I think I found what happened here. One WAL record can be split between WAL files. In XLogReadRecord, if last WAL record is incomplete, it try to get next WAL: /* Copy the first fragment of the record from the first page. */ memcpy(state->readRecordBuf,

Re: Why standby restores some WALs many times from archive?

2017-12-31 Thread Michael Paquier
On Sun, Dec 31, 2017 at 10:22:37AM +0300, Sergey Burladyan wrote: > Michael Paquier writes: >> Don't be surprised if you get corrupted instances or that you >> are not able to recover up to a consistent point if you need to roll in >> a backup. > > But only if archive was reboot unexpectedly, am

Re: Why standby restores some WALs many times from archive?

2017-12-30 Thread Sergey Burladyan
Michael Paquier writes: > On Sat, Dec 30, 2017 at 04:30:07AM +0300, Sergey Burladyan wrote: > > We use this scripts: > > https://github.com/avito-tech/dba-utils/tree/master/pg_archive > > > > But I can reproduce problem with simple cp & mv: > > archive_command: > > test ! -f /var/lib/postgresq

Re: Why standby restores some WALs many times from archive?

2017-12-30 Thread Michael Paquier
On Sat, Dec 30, 2017 at 04:30:07AM +0300, Sergey Burladyan wrote: > We use this scripts: > https://github.com/avito-tech/dba-utils/tree/master/pg_archive > > But I can reproduce problem with simple cp & mv: > archive_command: > test ! -f /var/lib/postgresql/wals/%f && \ > test ! -f /var/lib/po

Re: Why standby restores some WALs many times from archive?

2017-12-29 Thread Sergey Burladyan
Jeff Janes writes: > On Thu, Dec 28, 2017 at 7:02 AM, Victor Yagofarov wrote: > > I have postgres 9.4 standby with archive-based replication (via > > restore_command). > Can you show us both your archive_command and your restore_command? We use this scripts: https://github.com/avito-tech/dba-u

Re: Why standby restores some WALs many times from archive?

2017-12-28 Thread Jeff Janes
On Thu, Dec 28, 2017 at 7:02 AM, Victor Yagofarov wrote: > Hi, > > I tried to ask a question in pgsql-general@ but got no response. > > I have postgres 9.4 standby with archive-based replication (via > restore_command). > Can you show us both your archive_command and your restore_command? Cheer

Why standby restores some WALs many times from archive?

2017-12-28 Thread Victor Yagofarov
Hi, I tried to ask a question in pgsql-general@ but got no response. I have postgres 9.4 standby with archive-based replication (via restore_command). PostgreSQL 9.4.12 on x86_64-unknown-linux-gnu, compiled by gcc (Debian 4.9.2-10) 4.9.2, 64-bit One day I noticed strange behavior in postgres l