Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Satyanarayana Narlapuram
Thank you! Got it. 

-Original Message-
From: Stephen Frost [mailto:sfr...@snowman.net] 
Sent: Monday, September 25, 2017 10:57 AM
To: Magnus Hagander <mag...@hagander.net>
Cc: Satyanarayana Narlapuram <satyanarayana.narlapu...@microsoft.com>; 
PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: [HACKERS] Reading backup label file for checkpoint and redo 
location during crash recovery

* Magnus Hagander (mag...@hagander.net) wrote:
> On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost <sfr...@snowman.net> wrote:
> > * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote:
> > > During crash recovery, last checkpoint record information is 
> > > obtained
> > from the backup label if present, instead of getting it from the 
> > control file. This behavior is causing PostgreSQL database cluster 
> > not to come up until the backup label file is deleted (as the error message 
> > says).
> > >
> > > if (checkPoint.redo < checkPointLoc)
> > >   {
> > >  if (!ReadRecord(xlogreader,
> > checkPoint.redo, LOG, false))
> > > ereport(FATAL,
> > >   (errmsg("could 
> > > not
> > find redo location referenced by checkpoint record"),
> > >   errhint("If you 
> > > are
> > not restoring from a backup, try removing the file 
> > \"%s/backup_label\".", DataDir)));
> > >   }
> > >
> > > If we are recovering from a dump file, reading from the backup 
> > > label
> > files makes sense as the control file could be archived after a few 
> > checkpoints. But this is not the case for crash recovery, and is 
> > always safe to read the checkpoint record information from the control file.
> > > Is this behavior kept this way as there is no clear way to 
> > > distinguish
> > between the recovery from the dump and the regular crash recovery?
> >
> > This is why the exclusive backup method has been deprecated in PG10 
> > in favor of the non-exclusive backup method, which avoids this by 
> > not creating a backup label file (it's up to the backup software to 
> > store the necessary information and create the file for use during 
> > recovery).
> 
> Actally, it was deprecated already in 9.6, not just 10.

Whoops, right.  Thanks for the clarification. :)

Stephen


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote:
> On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost  wrote:
> > * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote:
> > > During crash recovery, last checkpoint record information is obtained
> > from the backup label if present, instead of getting it from the control
> > file. This behavior is causing PostgreSQL database cluster not to come up
> > until the backup label file is deleted (as the error message says).
> > >
> > > if (checkPoint.redo < checkPointLoc)
> > >   {
> > >  if (!ReadRecord(xlogreader,
> > checkPoint.redo, LOG, false))
> > > ereport(FATAL,
> > >   (errmsg("could not
> > find redo location referenced by checkpoint record"),
> > >   errhint("If you are
> > not restoring from a backup, try removing the file \"%s/backup_label\".",
> > DataDir)));
> > >   }
> > >
> > > If we are recovering from a dump file, reading from the backup label
> > files makes sense as the control file could be archived after a few
> > checkpoints. But this is not the case for crash recovery, and is always
> > safe to read the checkpoint record information from the control file.
> > > Is this behavior kept this way as there is no clear way to distinguish
> > between the recovery from the dump and the regular crash recovery?
> >
> > This is why the exclusive backup method has been deprecated in PG10 in
> > favor of the non-exclusive backup method, which avoids this by not
> > creating a backup label file (it's up to the backup software to store
> > the necessary information and create the file for use during recovery).
> 
> Actally, it was deprecated already in 9.6, not just 10.

Whoops, right.  Thanks for the clarification. :)

Stephen


signature.asc
Description: Digital signature


Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Magnus Hagander
On Mon, Sep 25, 2017 at 7:43 PM, Stephen Frost  wrote:

> Greetings Satya,
>
> * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote:
> > During crash recovery, last checkpoint record information is obtained
> from the backup label if present, instead of getting it from the control
> file. This behavior is causing PostgreSQL database cluster not to come up
> until the backup label file is deleted (as the error message says).
> >
> > if (checkPoint.redo < checkPointLoc)
> >   {
> >  if (!ReadRecord(xlogreader,
> checkPoint.redo, LOG, false))
> > ereport(FATAL,
> >   (errmsg("could not
> find redo location referenced by checkpoint record"),
> >   errhint("If you are
> not restoring from a backup, try removing the file \"%s/backup_label\".",
> DataDir)));
> >   }
> >
> > If we are recovering from a dump file, reading from the backup label
> files makes sense as the control file could be archived after a few
> checkpoints. But this is not the case for crash recovery, and is always
> safe to read the checkpoint record information from the control file.
> > Is this behavior kept this way as there is no clear way to distinguish
> between the recovery from the dump and the regular crash recovery?
>
> This is why the exclusive backup method has been deprecated in PG10 in
> favor of the non-exclusive backup method, which avoids this by not
> creating a backup label file (it's up to the backup software to store
> the necessary information and create the file for use during recovery).
>


Actally, it was deprecated already in 9.6, not just 10.


-- 
 Magnus Hagander
 Me: https://www.hagander.net/ 
 Work: https://www.redpill-linpro.com/ 


Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Andres Freund
On 2017-09-25 13:43:32 -0400, Stephen Frost wrote:
> Greetings Satya,
> 
> * Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote:
> > During crash recovery, last checkpoint record information is obtained from 
> > the backup label if present, instead of getting it from the control file. 
> > This behavior is causing PostgreSQL database cluster not to come up until 
> > the backup label file is deleted (as the error message says).
> > 
> > if (checkPoint.redo < checkPointLoc)
> >   {
> >  if (!ReadRecord(xlogreader, checkPoint.redo, 
> > LOG, false))
> > ereport(FATAL,
> >   (errmsg("could not find 
> > redo location referenced by checkpoint record"),
> >   errhint("If you are not 
> > restoring from a backup, try removing the file \"%s/backup_label\".", 
> > DataDir)));
> >   }
> > 
> > If we are recovering from a dump file, reading from the backup label files 
> > makes sense as the control file could be archived after a few checkpoints. 
> > But this is not the case for crash recovery, and is always safe to read the 
> > checkpoint record information from the control file.
> > Is this behavior kept this way as there is no clear way to distinguish 
> > between the recovery from the dump and the regular crash recovery?
> 
> This is why the exclusive backup method has been deprecated in PG10 in
> favor of the non-exclusive backup method, which avoids this by not
> creating a backup label file (it's up to the backup software to store
> the necessary information and create the file for use during recovery).
> 
> Please see:
> 
> https://www.postgresql.org/docs/10/static/continuous-archiving.html
> 
> In particular, section 25.3.3.

Might not be obvious for the more casual contributor:

And even before then, using pg_basebackup / the streaming replication
protocol version of creating a base backup, it was avoided.

Greetings,

Andres Freund


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Stephen Frost
Greetings Satya,

* Satyanarayana Narlapuram (satyanarayana.narlapu...@microsoft.com) wrote:
> During crash recovery, last checkpoint record information is obtained from 
> the backup label if present, instead of getting it from the control file. 
> This behavior is causing PostgreSQL database cluster not to come up until the 
> backup label file is deleted (as the error message says).
> 
> if (checkPoint.redo < checkPointLoc)
>   {
>  if (!ReadRecord(xlogreader, checkPoint.redo, 
> LOG, false))
> ereport(FATAL,
>   (errmsg("could not find 
> redo location referenced by checkpoint record"),
>   errhint("If you are not 
> restoring from a backup, try removing the file \"%s/backup_label\".", 
> DataDir)));
>   }
> 
> If we are recovering from a dump file, reading from the backup label files 
> makes sense as the control file could be archived after a few checkpoints. 
> But this is not the case for crash recovery, and is always safe to read the 
> checkpoint record information from the control file.
> Is this behavior kept this way as there is no clear way to distinguish 
> between the recovery from the dump and the regular crash recovery?

This is why the exclusive backup method has been deprecated in PG10 in
favor of the non-exclusive backup method, which avoids this by not
creating a backup label file (it's up to the backup software to store
the necessary information and create the file for use during recovery).

Please see:

https://www.postgresql.org/docs/10/static/continuous-archiving.html

In particular, section 25.3.3.

Thanks!

Stephen


signature.asc
Description: Digital signature


[HACKERS] Reading backup label file for checkpoint and redo location during crash recovery

2017-09-25 Thread Satyanarayana Narlapuram
Hi there,

During crash recovery, last checkpoint record information is obtained from the 
backup label if present, instead of getting it from the control file. This 
behavior is causing PostgreSQL database cluster not to come up until the backup 
label file is deleted (as the error message says).

if (checkPoint.redo < checkPointLoc)
  {
 if (!ReadRecord(xlogreader, checkPoint.redo, LOG, 
false))
ereport(FATAL,
  (errmsg("could not find redo 
location referenced by checkpoint record"),
  errhint("If you are not 
restoring from a backup, try removing the file \"%s/backup_label\".", 
DataDir)));
  }

If we are recovering from a dump file, reading from the backup label files 
makes sense as the control file could be archived after a few checkpoints. But 
this is not the case for crash recovery, and is always safe to read the 
checkpoint record information from the control file.
Is this behavior kept this way as there is no clear way to distinguish between 
the recovery from the dump and the regular crash recovery?


Thanks,
Satya