On 06/29/2015 09:44 AM, Michael Paquier wrote:
On Mon, Jun 29, 2015 at 4:55 AM, Heikki Linnakangas wrote:
But we'll still need to handle the pg_xlog symlink case somehow. Perhaps it
would be enough to special-case pg_xlog for now.
Well, sure, pg_rewind does not copy the soft links either way. Now it
would be nice to have an option to be able to recreate the soft link
of at least pg_xlog even if it can be scripted as well after a run.
Hmm. I'm starting to think that pg_rewind should ignore pg_xlog
entirely. In any non-trivial scenarios, just copying all the files from
pg_xlog isn't enough anyway, and you need to set up a recovery.conf
after running pg_rewind that contains a restore_command or
primary_conninfo, to fetch the WAL. So you can argue that by not copying
pg_xlog automatically, we're actually doing a favour to the DBA, by
forcing him to set up the recovery.conf file correctly. Because if you
just test simple scenarios where not much time has passed between the
failover and running pg_rewind, it might be enough to just copy all the
WAL currently in pg_xlog, but it would not be enough if more time had
passed and not all the required WAL is present in pg_xlog anymore. And
by not copying the WAL, we can avoid some copying, as restore_command or
streaming replication will only copy what's needed, while pg_rewind
would copy all WAL it can find the target's data directory.
pg_basebackup also doesn't include any WAL, unless you pass the --xlog
option. It would be nice to also add an optional --xlog option to
pg_rewind, but with pg_rewind it's possible that all the required WAL
isn't present in the pg_xlog directory anymore, so you wouldn't always
achieve the same effect of making the backup self-contained.
So, I propose the attached. It makes pg_rewind ignore the pg_xlog
directory in both the source and the target.
- Heikki
diff --git a/doc/src/sgml/ref/pg_rewind.sgml b/doc/src/sgml/ref/pg_rewind.sgml
index 32dc83f..bb1d640 100644
--- a/doc/src/sgml/ref/pg_rewind.sgml
+++ b/doc/src/sgml/ref/pg_rewind.sgml
@@ -49,12 +49,13 @@ PostgreSQL documentation
<para>
The result is equivalent to replacing the target data directory with the
- source one. All files are copied, including configuration files. The
- advantage of <application>pg_rewind</> over taking a new base backup, or
- tools like <application>rsync</>, is that <application>pg_rewind</> does
- not require reading through all unchanged files in the cluster. That makes
- it a lot faster when the database is large and only a small portion of it
- differs between the clusters.
+ source one. All files in the data directory are copied, including
+ configuration files, but excluding the WAL log directory,
+ <filename>pg_xlog</>. The advantage of <application>pg_rewind</> over
+ taking a new base backup, or tools like <application>rsync</>, is that
+ <application>pg_rewind</> does not require reading through all unchanged
+ files in the cluster. That makes it a lot faster when the database is
+ large and only a small portion of it differs between the clusters.
</para>
<para>
@@ -74,12 +75,12 @@ PostgreSQL documentation
When the target server is started up for the first time after running
<application>pg_rewind</>, it will go into recovery mode and replay all
WAL generated in the source server after the point of divergence.
- If some of the WAL was no longer available in the source server when
- <application>pg_rewind</> was run, and therefore could not be copied by
- <application>pg_rewind</> session, it needs to be made available when the
- target server is started up. That can be done by creating a
- <filename>recovery.conf</> file in the target data directory with a
- suitable <varname>restore_command</>.
+ Like after restoring from a base backup using continous archiving,
+ the source server's WAL must be made available to the rewound server.
+ This can be done by creating a <filename>recovery.conf</> file in the
+ target data directory with a suitable <varname>restore_command</> or
+ <varname>primary_conninfo</> line, or by copying the required WAL files
+ manually into <filename>pg_xlog</> in the target data directory.
</para>
</refsect1>
diff --git a/src/bin/pg_rewind/RewindTest.pm b/src/bin/pg_rewind/RewindTest.pm
index 5219ec9..82e3f4c 100644
--- a/src/bin/pg_rewind/RewindTest.pm
+++ b/src/bin/pg_rewind/RewindTest.pm
@@ -284,14 +284,12 @@ sub run_pg_rewind
# Keep a temporary postgresql.conf for master node or it would be
# overwritten during the rewind.
- copy(
- "$test_master_datadir/postgresql.conf",
- "$testroot/master-postgresql.conf.tmp");
+ copy("$test_master_datadir/postgresql.conf",
+ "$testroot/master-postgresql.conf.tmp");
# Now run pg_rewind
if ($test_mode eq "local")
{
-
# Do rewind using a local pgdata as source
# Stop the master and be ready to perform the rewind
system_or_bail(
@@ -306,10 +304,22 @@ sub run_pg_rewind
$log_path,
'2>&1');
ok($result, 'pg_rewind local');
+
+ # Copy WAL files from the source server so that recovery will find
+ # them. (In remote mode, the source server is still running, and the
+ # rewound server will use streaming replication to connect to the
+ # source server and fetch the WAL from there.)
+ opendir(DIR,"$test_standby_datadir/pg_xlog") or die "Cannot open $test_standby_datadir/pg_xlo\n";
+ my @xlogfiles = readdir(DIR);
+ closedir(DIR);
+
+ foreach my $xlogfile (@xlogfiles) {
+ copy("$test_standby_datadir/pg_xlog/$xlogfile",
+ "$test_master_datadir/pg_xlog/");
+ }
}
elsif ($test_mode eq "remote")
{
-
# Do rewind using a remote connection as source
my $result = run(
[ 'pg_rewind',
diff --git a/src/bin/pg_rewind/copy_fetch.c b/src/bin/pg_rewind/copy_fetch.c
index 224fad1..f437391 100644
--- a/src/bin/pg_rewind/copy_fetch.c
+++ b/src/bin/pg_rewind/copy_fetch.c
@@ -74,6 +74,9 @@ recurse_dir(const char *datadir, const char *parentpath,
strcmp(xlde->d_name, "..") == 0)
continue;
+ if (parentpath == NULL && strcmp(xlde->d_name, "pg_xlog") == 0)
+ continue;
+
snprintf(fullpath, MAXPGPATH, "%s/%s", fullparentpath, xlde->d_name);
if (lstat(fullpath, &fst) < 0)
diff --git a/src/bin/pg_rewind/libpq_fetch.c b/src/bin/pg_rewind/libpq_fetch.c
index 05aa133..2f411e3 100644
--- a/src/bin/pg_rewind/libpq_fetch.c
+++ b/src/bin/pg_rewind/libpq_fetch.c
@@ -151,6 +151,7 @@ libpqProcessFileList(void)
" SELECT '' AS path, filename, size, isdir FROM\n"
" (SELECT pg_ls_dir('.', true, false) AS filename) AS fn,\n"
" pg_stat_file(fn.filename, true) AS this\n"
+ " WHERE filename <> 'pg_xlog'"
" UNION ALL\n"
" SELECT parent.path || parent.filename || '/' AS path,\n"
" fn, this.size, this.isdir\n"
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
