On Tue, Jan 27, 2015 at 10:16:48PM -0500, David Steele wrote: > This is definitely an edge case. Not only does the file have to be > modified in the same second *after* rsync has done the copy, but the > file also has to not be modified in *any other subsequent second* before > the next incremental backup. If the file is busy enough to have a > collision with rsync in that second, then it is very likely to be > modified before the next incremental backup which is generally a day or > so later. And, of course, the backup where the issue occurs is fine - > it's the next backup that is invalid. > > However, the hot/cold backup scheme as documented does make the race > condition more likely since the two backups are done in close proximity > temporally. Ultimately, the most reliable method is to use checksums. > > For me the biggest issue is that there is no way to discover if a db in > consistent no matter how much time/resources you are willing to spend. > I could live with the idea of the occasional bad backup (since I keep as > many as possible), but having no way to know whether it is good or not > is very frustrating. I know data checksums are a step in that > direction, but they are a long way from providing the optimal solution. > I've implemented rigorous checksums in PgBackRest but something closer > to the source would be even better.
Agreed. I have update the two mentions of rsync in our docs to clarify this. Thank you. The patch also has pg_upgrade doc improvements suggested by comments from Josh Berkus. -- Bruce Momjian <br...@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. +
diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml new file mode 100644 index 07ca0dc..e25e0d0 *** a/doc/src/sgml/backup.sgml --- b/doc/src/sgml/backup.sgml *************** tar -cf backup.tar /usr/local/pgsql/data *** 438,445 **** Another option is to use <application>rsync</> to perform a file system backup. This is done by first running <application>rsync</> while the database server is running, then shutting down the database ! server just long enough to do a second <application>rsync</>. The ! second <application>rsync</> will be much quicker than the first, because it has relatively little data to transfer, and the end result will be consistent because the server was down. This method allows a file system backup to be performed with minimal downtime. --- 438,447 ---- Another option is to use <application>rsync</> to perform a file system backup. This is done by first running <application>rsync</> while the database server is running, then shutting down the database ! server long enough to do an <command>rsync --checksum</>. ! (<option>--checksum</> is necessary because <command>rsync</> only ! has file modification-time granularity of one second.) The ! second <application>rsync</> will be quicker than the first, because it has relatively little data to transfer, and the end result will be consistent because the server was down. This method allows a file system backup to be performed with minimal downtime. diff --git a/doc/src/sgml/pgupgrade.sgml b/doc/src/sgml/pgupgrade.sgml new file mode 100644 index e1cd260..ed65def *** a/doc/src/sgml/pgupgrade.sgml --- b/doc/src/sgml/pgupgrade.sgml *************** pg_upgrade.exe *** 409,414 **** --- 409,504 ---- </step> <step> + <title>Upgrade any Log-Shipping Standby Servers</title> + + <para> + If you have Log-Shipping Standby Servers (<xref + linkend="warm-standby">), follow these steps to upgrade them (before + starting any servers): + </para> + + <procedure> + + <step> + <title>Install the new PostgreSQL binaries on standby servers</title> + + <para> + Make sure the new binaries and support files are installed + on all the standby servers. Do <emphasis>not</> run + <application>initdb</>. If <application>initdb</> was run, delete + the standby server data directories. Also, install any custom + shared object files on the new standbys that you installed in the + new master cluster. + </para> + </step> + + <step> + <title>Shutdown the Standby Servers</title> + + <para> + If the standby servers are still running, shut them down. Save any + configuration files from the standbys you need to keep, e.g. + <filename>postgresql.conf</>, <literal>recovery.conf</>, as these + will be overwritten or removed in the next step. + </para> + </step> + + <step> + <title>Run <application>rsync</></title> + + <para> + From a directory that is above the old and new database cluster + directories, run this for each slave: + + <programlisting> + rsync --archive --hard-links --size-only old_pgdata new_pgdata remote_dir + </programlisting> + + where <option>old_pgdata</> and <option>new_pgdata</> are relative + to the current directory, and <option>remote_dir</> is + <emphasis>above</> the old and new cluster directories on + the standby server. The old and new relative cluster paths + must match on the master and standby server. Consult the + <application>rsync</> manual page for details on specifying the + remote directory, e.g. <literal>standbyhost:/var/lib/postgresql/</>. + <application>rsync</> will be fast when <option>--link</> mode is + used because it will create hard links on the remote server rather + than transferring user data. + </para> + + <para> + If you have tablespaces, you will need to run a similar + <application>rsync</> command for each tablespace directory. If you + have relocated <filename>pg_xlog</> outside the data directories, + <application>rsync</> must be run on those directories too. + </para> + </step> + + <step> + <title>Configure log-shipping to standby servers</title> + + <para> + Configure the servers for log shipping. (You do not need to run + <function>pg_start_backup()</> and <function>pg_stop_backup()</> + or take a file system backup as the slaves are still synchronized + with the master.) + </para> + </step> + + </procedure> + + </step> + + <step> + <title>Start the new server</title> + + <para> + The new server and any <application>rsync</>'ed standby servers can + now be safely started. + </para> + </step> + + <step> <title>Post-Upgrade processing</title> <para> *************** psql --username postgres --file script.s *** 548,569 **** </para> <para> - A Log-Shipping Standby Server (<xref linkend="warm-standby">) cannot - be upgraded because the server must allow writes. The simplest way - is to upgrade the primary and use <command>rsync</> to rebuild the - standbys. You can run <command>rsync</> while the primary is down, - or as part of a base backup (<xref linkend="backup-base-backup">) - which overwrites the old standby cluster. - </para> - - <para> If you want to use link mode and you do not want your old cluster to be modified when the new cluster is started, make a copy of the old cluster and upgrade that in link mode. To make a valid copy of the old cluster, use <command>rsync</> to create a dirty copy of the old cluster while the server is running, then shut down ! the old server and run <command>rsync</> again to update the copy with any ! changes to make it consistent. You might want to exclude some files, e.g. <filename>postmaster.pid</>, as documented in <xref linkend="backup-lowlevel-base-backup">. If your file system supports file system snapshots or copy-on-write file copies, you can use that --- 638,652 ---- </para> <para> If you want to use link mode and you do not want your old cluster to be modified when the new cluster is started, make a copy of the old cluster and upgrade that in link mode. To make a valid copy of the old cluster, use <command>rsync</> to create a dirty copy of the old cluster while the server is running, then shut down ! the old server and run <command>rsync --checksum</> again to update the ! copy with any changes to make it consistent. (<option>--checksum</> ! is necessary because <command>rsync</> only has file modification-time ! granularity of one second.) You might want to exclude some files, e.g. <filename>postmaster.pid</>, as documented in <xref linkend="backup-lowlevel-base-backup">. If your file system supports file system snapshots or copy-on-write file copies, you can use that
-- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers