On Tue, Jan 27, 2015 at 10:16:48PM -0500, David Steele wrote:
> This is definitely an edge case. Not only does the file have to be
> modified in the same second *after* rsync has done the copy, but the
> file also has to not be modified in *any other subsequent second* before
> the next incremental backup. If the file is busy enough to have a
> collision with rsync in that second, then it is very likely to be
> modified before the next incremental backup which is generally a day or
> so later. And, of course, the backup where the issue occurs is fine -
> it's the next backup that is invalid.
>
> However, the hot/cold backup scheme as documented does make the race
> condition more likely since the two backups are done in close proximity
> temporally. Ultimately, the most reliable method is to use checksums.
>
> For me the biggest issue is that there is no way to discover if a db in
> consistent no matter how much time/resources you are willing to spend.
> I could live with the idea of the occasional bad backup (since I keep as
> many as possible), but having no way to know whether it is good or not
> is very frustrating. I know data checksums are a step in that
> direction, but they are a long way from providing the optimal solution.
> I've implemented rigorous checksums in PgBackRest but something closer
> to the source would be even better.
Agreed. I have update the two mentions of rsync in our docs to clarify
this. Thank you.
The patch also has pg_upgrade doc improvements suggested by comments
from Josh Berkus.
--
Bruce Momjian <[email protected]> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ Everyone has their own god. +
diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml
new file mode 100644
index 07ca0dc..e25e0d0
*** a/doc/src/sgml/backup.sgml
--- b/doc/src/sgml/backup.sgml
*************** tar -cf backup.tar /usr/local/pgsql/data
*** 438,445 ****
Another option is to use <application>rsync</> to perform a file
system backup. This is done by first running <application>rsync</>
while the database server is running, then shutting down the database
! server just long enough to do a second <application>rsync</>. The
! second <application>rsync</> will be much quicker than the first,
because it has relatively little data to transfer, and the end result
will be consistent because the server was down. This method
allows a file system backup to be performed with minimal downtime.
--- 438,447 ----
Another option is to use <application>rsync</> to perform a file
system backup. This is done by first running <application>rsync</>
while the database server is running, then shutting down the database
! server long enough to do an <command>rsync --checksum</>.
! (<option>--checksum</> is necessary because <command>rsync</> only
! has file modification-time granularity of one second.) The
! second <application>rsync</> will be quicker than the first,
because it has relatively little data to transfer, and the end result
will be consistent because the server was down. This method
allows a file system backup to be performed with minimal downtime.
diff --git a/doc/src/sgml/pgupgrade.sgml b/doc/src/sgml/pgupgrade.sgml
new file mode 100644
index e1cd260..ed65def
*** a/doc/src/sgml/pgupgrade.sgml
--- b/doc/src/sgml/pgupgrade.sgml
*************** pg_upgrade.exe
*** 409,414 ****
--- 409,504 ----
</step>
<step>
+ <title>Upgrade any Log-Shipping Standby Servers</title>
+
+ <para>
+ If you have Log-Shipping Standby Servers (<xref
+ linkend="warm-standby">), follow these steps to upgrade them (before
+ starting any servers):
+ </para>
+
+ <procedure>
+
+ <step>
+ <title>Install the new PostgreSQL binaries on standby servers</title>
+
+ <para>
+ Make sure the new binaries and support files are installed
+ on all the standby servers. Do <emphasis>not</> run
+ <application>initdb</>. If <application>initdb</> was run, delete
+ the standby server data directories. Also, install any custom
+ shared object files on the new standbys that you installed in the
+ new master cluster.
+ </para>
+ </step>
+
+ <step>
+ <title>Shutdown the Standby Servers</title>
+
+ <para>
+ If the standby servers are still running, shut them down. Save any
+ configuration files from the standbys you need to keep, e.g.
+ <filename>postgresql.conf</>, <literal>recovery.conf</>, as these
+ will be overwritten or removed in the next step.
+ </para>
+ </step>
+
+ <step>
+ <title>Run <application>rsync</></title>
+
+ <para>
+ From a directory that is above the old and new database cluster
+ directories, run this for each slave:
+
+ <programlisting>
+ rsync --archive --hard-links --size-only old_pgdata new_pgdata remote_dir
+ </programlisting>
+
+ where <option>old_pgdata</> and <option>new_pgdata</> are relative
+ to the current directory, and <option>remote_dir</> is
+ <emphasis>above</> the old and new cluster directories on
+ the standby server. The old and new relative cluster paths
+ must match on the master and standby server. Consult the
+ <application>rsync</> manual page for details on specifying the
+ remote directory, e.g. <literal>standbyhost:/var/lib/postgresql/</>.
+ <application>rsync</> will be fast when <option>--link</> mode is
+ used because it will create hard links on the remote server rather
+ than transferring user data.
+ </para>
+
+ <para>
+ If you have tablespaces, you will need to run a similar
+ <application>rsync</> command for each tablespace directory. If you
+ have relocated <filename>pg_xlog</> outside the data directories,
+ <application>rsync</> must be run on those directories too.
+ </para>
+ </step>
+
+ <step>
+ <title>Configure log-shipping to standby servers</title>
+
+ <para>
+ Configure the servers for log shipping. (You do not need to run
+ <function>pg_start_backup()</> and <function>pg_stop_backup()</>
+ or take a file system backup as the slaves are still synchronized
+ with the master.)
+ </para>
+ </step>
+
+ </procedure>
+
+ </step>
+
+ <step>
+ <title>Start the new server</title>
+
+ <para>
+ The new server and any <application>rsync</>'ed standby servers can
+ now be safely started.
+ </para>
+ </step>
+
+ <step>
<title>Post-Upgrade processing</title>
<para>
*************** psql --username postgres --file script.s
*** 548,569 ****
</para>
<para>
- A Log-Shipping Standby Server (<xref linkend="warm-standby">) cannot
- be upgraded because the server must allow writes. The simplest way
- is to upgrade the primary and use <command>rsync</> to rebuild the
- standbys. You can run <command>rsync</> while the primary is down,
- or as part of a base backup (<xref linkend="backup-base-backup">)
- which overwrites the old standby cluster.
- </para>
-
- <para>
If you want to use link mode and you do not want your old cluster
to be modified when the new cluster is started, make a copy of the
old cluster and upgrade that in link mode. To make a valid copy
of the old cluster, use <command>rsync</> to create a dirty
copy of the old cluster while the server is running, then shut down
! the old server and run <command>rsync</> again to update the copy with any
! changes to make it consistent. You might want to exclude some
files, e.g. <filename>postmaster.pid</>, as documented in <xref
linkend="backup-lowlevel-base-backup">. If your file system supports
file system snapshots or copy-on-write file copies, you can use that
--- 638,652 ----
</para>
<para>
If you want to use link mode and you do not want your old cluster
to be modified when the new cluster is started, make a copy of the
old cluster and upgrade that in link mode. To make a valid copy
of the old cluster, use <command>rsync</> to create a dirty
copy of the old cluster while the server is running, then shut down
! the old server and run <command>rsync --checksum</> again to update the
! copy with any changes to make it consistent. (<option>--checksum</>
! is necessary because <command>rsync</> only has file modification-time
! granularity of one second.) You might want to exclude some
files, e.g. <filename>postmaster.pid</>, as documented in <xref
linkend="backup-lowlevel-base-backup">. If your file system supports
file system snapshots or copy-on-write file copies, you can use that
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
