On Sat, Jan 15, 2011 at 23:10, Dimitri Fontaine <dimi...@2ndquadrant.fr> wrote:
>>> That should be -D --pgdata, for consistency with pg_dump.
>>
>> pg_dump doesn't have a -D. I assume you mean pg_ctl / initdb?
>
> Yes, sorry, been too fast.
Ok. Updated patch that includes this change attached. I also changed
the tar directory from -t to -T, for consistency.
It also includes the change to take -h host, -U user, -w/-W for
password -p port instead of a conninfo string.
>> Another option, I think Heikki mentioned this on IM at some point, is
>> to do something like name it <oid>-<name>.tar. That would give us best
>> of both worlds?
>
> Well I'd think we know the pg_tablespace columns encoding, so the
> problem might be the filesystem encodings, right? Well there's also the
Do we really? That's one of the global catalogs that don't really have
an encoding, isn't it?
> option of creating <oid>.tar and have a symlink to it called <name>.tar
> but that's pushing it. I don't think naming after OIDs is a good
> service for users, but if that's all we can reasonably do…
Yeah, symlink seems to be making things way too complex. <oid>-<name>
seems is perhaps a reasonable compromise?
> Will continue reviewing and post something more polished and
> comprehensive next week — mainly wanted to see if you wanted to include
> pg_ctl <command> in the patch already.
Ok, thanks.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
diff --git a/doc/src/sgml/backup.sgml b/doc/src/sgml/backup.sgml
index db7c834..c14ae43 100644
--- a/doc/src/sgml/backup.sgml
+++ b/doc/src/sgml/backup.sgml
@@ -813,6 +813,16 @@ SELECT pg_stop_backup();
</para>
<para>
+ You can also use the <xref linkend="app-pgbasebackup"> tool to take
+ the backup, instead of manually copying the files. This tool will take
+ care of the <function>pg_start_backup()</>, copy and
+ <function>pg_stop_backup()</> steps automatically, and transfers the
+ backup over a regular <productname>PostgreSQL</productname> connection
+ using the replication protocol, instead of requiring filesystem level
+ access.
+ </para>
+
+ <para>
Some file system backup tools emit warnings or errors
if the files they are trying to copy change while the copy proceeds.
When taking a base backup of an active database, this situation is normal
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index f40fa9d..c44d11e 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -160,6 +160,7 @@ Complete list of usable sgml source files in this directory.
<!entity dropuser system "dropuser.sgml">
<!entity ecpgRef system "ecpg-ref.sgml">
<!entity initdb system "initdb.sgml">
+<!entity pgBasebackup system "pg_basebackup.sgml">
<!entity pgConfig system "pg_config-ref.sgml">
<!entity pgControldata system "pg_controldata.sgml">
<!entity pgCtl system "pg_ctl-ref.sgml">
diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
new file mode 100644
index 0000000..dc8e2f4
--- /dev/null
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -0,0 +1,313 @@
+<!--
+doc/src/sgml/ref/pg_basebackup.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="app-pgbasebackup">
+ <refmeta>
+ <refentrytitle>pg_basebackup</refentrytitle>
+ <manvolnum>1</manvolnum>
+ <refmiscinfo>Application</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>pg_basebackup</refname>
+ <refpurpose>take a base backup of a <productname>PostgreSQL</productname> cluster</refpurpose>
+ </refnamediv>
+
+ <indexterm zone="app-pgbasebackup">
+ <primary>pg_basebackup</primary>
+ </indexterm>
+
+ <refsynopsisdiv>
+ <cmdsynopsis>
+ <command>pg_basebackup</command>
+ <arg rep="repeat"><replaceable>option</></arg>
+ </cmdsynopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>
+ Description
+ </title>
+ <para>
+ <application>pg_basebackup</application> is used to take base backups of
+ a running <productname>PostgreSQL</productname> database cluster. These
+ are taken without affecting other clients to the database, and can be used
+ both for point-in-time recovery (see <xref linkend="continuous-archiving">)
+ and as the starting point for a log shipping or streaming replication standby
+ server (see <xref linkend="warm-standby">).
+ </para>
+
+ <para>
+ <application>pg_basebackup</application> makes a binary copy of the database
+ cluster files, while making sure the system is automatically put in and
+ out of backup mode automatically. Backups are always taken of the entire
+ database cluster, it is not possible to back up individual databases or
+ database objects. For individual database backups, a tool such as
+ <xref linkend="APP-PGDUMP"> must be used.
+ </para>
+
+ <para>
+ The backup is made over a regular <productname>PostgreSQL</productname>
+ connection, and uses the replication protocol. The connection must be
+ made with a user having <literal>REPLICATION</literal> permissions (see
+ <xref linkend="role-attributes">).
+ </para>
+
+ <para>
+ Only one backup can be concurrently active in
+ <productname>PostgreSQL</productname>, meaning that only one instance of
+ <application>pg_basebackup</application> can run at the same time
+ against a single database cluster.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Options</title>
+
+ <para>
+ <variablelist>
+ <varlistentry>
+ <term><option>-D <replaceable class="parameter">directory</replaceable></option></term>
+ <term><option>--pgdata=<replaceable class="parameter">directory</replaceable></option></term>
+ <listitem>
+ <para>
+ Directory to restore the base data directory to. When the cluster has
+ no additional tablespaces, the whole database will be placed in this
+ directory. If the cluster contains additional tablespaces, the main
+ data directory will be placed in this directory, but all other
+ tablespaces will be placed in the same absolute path as they have
+ on the server.
+ </para>
+ <para>
+ Only one of <literal>-D</> and <literal>-T</> can be specified.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-l <replaceable class="parameter">label</replaceable></option></term>
+ <term><option>--label=<replaceable class="parameter">label</replaceable></option></term>
+ <listitem>
+ <para>
+ Sets the label for the backup. If none is specified, a default value of
+ <literal>pg_basebackup base backup</literal> will be used.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-p</option></term>
+ <term><option>--progress</option></term>
+ <listitem>
+ <para>
+ Enables progress reporting. Turning this on will deliver an approximate
+ progress report during the backup. Since the database may change during
+ the backup, this is only an approximation and may not end at exactly
+ <literal>100%</literal>.
+ </para>
+ <para>
+ When this is enabled, the backup will start by enumerating the size of
+ the entire database, and then go back and send the actual contents.
+ This may make the backup take slightly longer, and in particular it
+ will take longer before the first data is sent.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-T <replaceable class="parameter">directory</replaceable></option></term>
+ <term><option>--tardir=<replaceable class="parameter">directory</replaceable></option></term>
+ <listitem>
+ <para>
+ Directory to place tar format files in. When this is specified, the
+ backup will consist of a number of tar files, one for each tablespace
+ in the database, stored in this directory. The tar file for the main
+ data directory will be named <filename>base.tar</>, and all other
+ tablespaces will be named after the tablespace oid.
+ </para>
+ <para>
+ If the value <literal>-</> (dash) is specified as tar directory,
+ the tar contents will be written to standard output, suitable for
+ piping to for example <productname>gzip</>. This is only possible if
+ the cluster has no additional tablespaces.
+ </para>
+ <para>
+ Only one of <literal>-D</> and <literal>-T</> can be specified.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-v</option></term>
+ <term><option>--verbose</option></term>
+ <listitem>
+ <para>
+ Enables verbose mode. Will output some extra steps during startup and
+ shutdown, as well as show the exact filename that is currently being
+ processed if progress reporting is also enabled.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-Z <replaceable class="parameter">level</replaceable></option></term>
+ <term><option>--compress=<replaceable class="parameter">level</replaceable></option></term>
+ <listitem>
+ <para>
+ Enables gzip compression of tar file output. Compression is only
+ available when generating tar files, and is not available when sending
+ output to standard output.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ The following command-line options control the database connection parameters.
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-h <replaceable class="parameter">host</replaceable></option></term>
+ <term><option>--host=<replaceable class="parameter">host</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the host name of the machine on which the server is
+ running. If the value begins with a slash, it is used as the
+ directory for the Unix domain socket. The default is taken
+ from the <envar>PGHOST</envar> environment variable, if set,
+ else a Unix domain socket connection is attempted.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-p <replaceable class="parameter">port</replaceable></option></term>
+ <term><option>--port=<replaceable class="parameter">port</replaceable></option></term>
+ <listitem>
+ <para>
+ Specifies the TCP port or local Unix domain socket file
+ extension on which the server is listening for connections.
+ Defaults to the <envar>PGPORT</envar> environment variable, if
+ set, or a compiled-in default.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-U <replaceable>username</replaceable></option></term>
+ <term><option>--username=<replaceable class="parameter">username</replaceable></option></term>
+ <listitem>
+ <para>
+ User name to connect as.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-w</></term>
+ <term><option>--no-password</></term>
+ <listitem>
+ <para>
+ Never issue a password prompt. If the server requires
+ password authentication and a password is not available by
+ other means such as a <filename>.pgpass</filename> file, the
+ connection attempt will fail. This option can be useful in
+ batch jobs and scripts where no user is present to enter a
+ password.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-W</option></term>
+ <term><option>--password</option></term>
+ <listitem>
+ <para>
+ Force <application>pg_basebackup</application> to prompt for a
+ password before connecting to a database.
+ </para>
+
+ <para>
+ This option is never essential, since
+ <application>pg_bsaebackup</application> will automatically prompt
+ for a password if the server demands password authentication.
+ However, <application>pg_basebackup</application> will waste a
+ connection attempt finding out that the server wants a password.
+ In some cases it is worth typing <option>-W</> to avoid the extra
+ connection attempt.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Other, less commonly used, parameters are also available:
+
+ <variablelist>
+ <varlistentry>
+ <term><option>-V</></term>
+ <term><option>--version</></term>
+ <listitem>
+ <para>
+ Print the <application>pg_basebackup</application> version and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><option>-?</></term>
+ <term><option>--help</></term>
+ <listitem>
+ <para>
+ Show help about <application>pg_basebackup</application> command line
+ arguments, and exit.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ </variablelist>
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Environment</title>
+
+ <para>
+ This utility, like most other <productname>PostgreSQL</> utilities,
+ uses the environment variables supported by <application>libpq</>
+ (see <xref linkend="libpq-envars">).
+ </para>
+
+ </refsect1>
+
+ <refsect1>
+ <title>Notes</title>
+
+ <para>
+ The backup will include all files in the data directory and tablespaces,
+ including the configuration files and any additional files placed in the
+ directory by third parties. Only regular files and directories are allowed
+ in the data directory, no symbolic links or special device files.
+ </para>
+
+ <para>
+ The way <productname>PostgreSQL</productname> manages tablespaces, the path
+ for all additional tablespaces must be identical whenever a backup is
+ restored. The main data directory, however, is relocatable to any location.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="APP-PGDUMP"></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index 84babf6..6ee8e5b 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -202,6 +202,7 @@
&droplang;
&dropuser;
&ecpgRef;
+ &pgBasebackup;
&pgConfig;
&pgDump;
&pgDumpall;
diff --git a/src/bin/Makefile b/src/bin/Makefile
index c18c05c..3809412 100644
--- a/src/bin/Makefile
+++ b/src/bin/Makefile
@@ -14,7 +14,7 @@ top_builddir = ../..
include $(top_builddir)/src/Makefile.global
SUBDIRS = initdb pg_ctl pg_dump \
- psql scripts pg_config pg_controldata pg_resetxlog
+ psql scripts pg_config pg_controldata pg_resetxlog pg_basebackup
ifeq ($(PORTNAME), win32)
SUBDIRS+=pgevent
endif
diff --git a/src/bin/pg_basebackup/Makefile b/src/bin/pg_basebackup/Makefile
new file mode 100644
index 0000000..ccb1502
--- /dev/null
+++ b/src/bin/pg_basebackup/Makefile
@@ -0,0 +1,38 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for src/bin/pg_basebackup
+#
+# Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/bin/pg_basebackup/Makefile
+#
+#-------------------------------------------------------------------------
+
+PGFILEDESC = "pg_basebackup - takes a streaming base backup of a PostgreSQL instance"
+PGAPPICON=win32
+
+subdir = src/bin/pg_basebackup
+top_builddir = ../../..
+include $(top_builddir)/src/Makefile.global
+
+override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
+
+OBJS= pg_basebackup.o $(WIN32RES)
+
+all: pg_basebackup
+
+pg_basebackup: $(OBJS) | submake-libpq submake-libpgport
+ $(CC) $(CFLAGS) $(OBJS) $(libpq_pgport) $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+install: all installdirs
+ $(INSTALL_PROGRAM) pg_basebackup$(X) '$(DESTDIR)$(bindir)/pg_basebackup$(X)'
+
+installdirs:
+ $(MKDIR_P) '$(DESTDIR)$(bindir)'
+
+uninstall:
+ rm -f '$(DESTDIR)$(bindir)/pg_basebackup$(X)'
+
+clean distclean maintainer-clean:
+ rm -f pg_basebackup$(X) $(OBJS)
diff --git a/src/bin/pg_basebackup/nls.mk b/src/bin/pg_basebackup/nls.mk
new file mode 100644
index 0000000..760ee1d
--- /dev/null
+++ b/src/bin/pg_basebackup/nls.mk
@@ -0,0 +1,5 @@
+# src/bin/pg_basebackup/nls.mk
+CATALOG_NAME := pg_basebackup
+AVAIL_LANGUAGES :=
+GETTEXT_FILES := pg_basebackup.c
+GETTEXT_TRIGGERS:= _
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
new file mode 100644
index 0000000..bb9dea0
--- /dev/null
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -0,0 +1,999 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_basebackup.c - receive a base backup using streaming replication protocol
+ *
+ * Author: Magnus Hagander <mag...@hagander.net>
+ *
+ * Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/bin/pg_basebackup.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+#include "libpq-fe.h"
+
+#include <unistd.h>
+#include <dirent.h>
+#include <sys/stat.h>
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+#endif
+
+#include "getopt_long.h"
+
+
+/* Global options */
+static const char *progname;
+char *basedir = NULL;
+char *tardir = NULL;
+char *label = "pg_basebackup base backup";
+bool showprogress = false;
+int verbose = 0;
+int compresslevel = 0;
+char *dbhost = NULL;
+char *dbuser = NULL;
+char *dbport = NULL;
+int dbgetpassword = 0; /* 0=auto, -1=never, 1=always */
+
+/* Progress counters */
+static uint64 totalsize;
+static uint64 totaldone;
+static int tablespacecount;
+
+/* Function headers */
+static char *xstrdup(const char *s);
+static void *xmalloc0(int size);
+static void usage(void);
+static void verify_dir_is_empty_or_create(char *dirname);
+static void progress_report(int tablespacenum, char *fn);
+static PGconn *GetConnection(void);
+
+static void ReceiveTarFile(PGconn *conn, PGresult *res, int rownum);
+static void ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum);
+static void BaseBackup();
+
+#ifdef HAVE_LIBZ
+static const char *
+get_gz_error(gzFile *gzf)
+{
+ int errnum;
+ const char *errmsg;
+
+ errmsg = gzerror(gzf, &errnum);
+ if (errnum == Z_ERRNO)
+ return strerror(errno);
+ else
+ return errmsg;
+}
+#endif
+
+/*
+ * strdup() and malloc() replacements that prints an error and exits
+ * if something goes wrong. Can never return NULL.
+ */
+static char *
+xstrdup(const char *s)
+{
+ char *result;
+
+ result = strdup(s);
+ if (!result)
+ {
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+ exit(1);
+ }
+ return result;
+}
+
+static void *
+xmalloc0(int size)
+{
+ void *result;
+
+ result = malloc(size);
+ if (!result)
+ {
+ fprintf(stderr, _("%s: out of memory\n"), progname);
+ exit(1);
+ }
+ MemSet(result, 0, size);
+ return result;
+}
+
+static void
+usage(void)
+{
+ printf(_("%s takes base backups of running PostgreSQL servers\n\n"),
+ progname);
+ printf(_("Usage:\n"));
+ printf(_(" %s [OPTION]...\n"), progname);
+ printf(_("\nOptions:\n"));
+ printf(_(" -D, --pgdata=directory receive base backup into directory\n"));
+ printf(_(" -T, --tardir=directory receive base backup into tar files\n"
+ " stored in specified directory\n"));
+ printf(_(" -Z, --compress=0-9 compress tar output\n"));
+ printf(_(" -l, --label=label set backup label\n"));
+ printf(_(" -p, --progress show progress information\n"));
+ printf(_(" -v, --verbose output verbose messages\n"));
+ printf(_("\nConnection options:\n"));
+ printf(_(" -h, --host=HOSTNAME database server host or socket directory\n"));
+ printf(_(" -p, --port=PORT database server port number\n"));
+ printf(_(" -U, --username=NAME connect as specified database user\n"));
+ printf(_(" -w, --no-password never prompt for password\n"));
+ printf(_(" -W, --password force password prompt (should happen automatically)\n"));
+ printf(_("\nOther options:\n"));
+ printf(_(" -?, --help show this help, then exit\n"));
+ printf(_(" -V, --version output version information, then exit\n"));
+ printf(_("\nReport bugs to <pgsql-b...@postgresql.org>.\n"));
+}
+
+
+/*
+ * Verify that the given directory exists and is empty. If it does not
+ * exist, it is created. If it exists but is not empty, an error will
+ * be give and the process ended.
+ */
+static void
+verify_dir_is_empty_or_create(char *dirname)
+{
+ switch (pg_check_dir(dirname))
+ {
+ case 0:
+
+ /*
+ * Does not exist, so create
+ */
+ if (pg_mkdir_p(dirname, S_IRWXU) == -1)
+ {
+ fprintf(stderr,
+ _("%s: could not create directory \"%s\": %s\n"),
+ progname, dirname, strerror(errno));
+ exit(1);
+ }
+ return;
+ case 1:
+
+ /*
+ * Exists, empty
+ */
+ return;
+ case 2:
+
+ /*
+ * Exists, not empty
+ */
+ fprintf(stderr,
+ _("%s: directory \"%s\" exists but is not empty\n"),
+ progname, dirname);
+ exit(1);
+ case -1:
+
+ /*
+ * Access problem
+ */
+ fprintf(stderr, _("%s: could not access directory \"%s\": %s\n"),
+ progname, dirname, strerror(errno));
+ exit(1);
+ }
+}
+
+
+/*
+ * Print a progress report based on the global variables. If verbose output
+ * is disabled, also print the current file name.
+ */
+static void
+progress_report(int tablespacenum, char *fn)
+{
+ if (verbose)
+ fprintf(stderr,
+ INT64_FORMAT "/" INT64_FORMAT " kB (%i%%) %i/%i tablespaces (%-30s)\r",
+ totaldone / 1024, totalsize,
+ (int) ((totaldone / 1024) * 100 / totalsize),
+ tablespacenum, tablespacecount, fn);
+ else
+ fprintf(stderr, INT64_FORMAT "/" INT64_FORMAT " kB (%i%%) %i/%i tablespaces\r",
+ totaldone / 1024, totalsize,
+ (int) ((totaldone / 1024) * 100 / totalsize),
+ tablespacenum, tablespacecount);
+}
+
+
+/*
+ * Receive a tar format file from the connection to the server, and write
+ * the data from this file directly into a tar file. If compression is
+ * enabled, the data will be compressed while written to the file.
+ *
+ * The file will be named base.tar[.gz] if it's for the main data directory
+ * or <tablespaceoid>.tar[.gz] if it's for another tablespace.
+ *
+ * No attempt to inspect or validate the contents of the file is done.
+ */
+static void
+ReceiveTarFile(PGconn *conn, PGresult *res, int rownum)
+{
+ char fn[MAXPGPATH];
+ char *copybuf = NULL;
+ FILE *tarfile = NULL;
+
+#ifdef HAVE_LIBZ
+ gzFile *ztarfile = NULL;
+#endif
+
+ if (PQgetisnull(res, rownum, 0))
+
+ /*
+ * Base tablespaces
+ */
+ if (strcmp(tardir, "-") == 0)
+ tarfile = stdout;
+ else
+ {
+#ifdef HAVE_LIBZ
+ if (compresslevel > 0)
+ {
+ snprintf(fn, sizeof(fn), "%s/base.tar.gz", tardir);
+ ztarfile = gzopen(fn, "wb");
+ if (gzsetparams(ztarfile, compresslevel, Z_DEFAULT_STRATEGY) != Z_OK)
+ {
+ fprintf(stderr, _("%s: could not set compression level %i\n"),
+ progname, compresslevel);
+ exit(1);
+ }
+ }
+ else
+#endif
+ {
+ snprintf(fn, sizeof(fn), "%s/base.tar", tardir);
+ tarfile = fopen(fn, "wb");
+ }
+ }
+ else
+ {
+ /*
+ * Specific tablespace
+ */
+#ifdef HAVE_LIBZ
+ if (compresslevel > 0)
+ {
+ snprintf(fn, sizeof(fn), "%s/%s.tar.gz", tardir, PQgetvalue(res, rownum, 0));
+ ztarfile = gzopen(fn, "wb");
+ }
+ else
+#endif
+ {
+ snprintf(fn, sizeof(fn), "%s/%s.tar", tardir, PQgetvalue(res, rownum, 0));
+ tarfile = fopen(fn, "wb");
+ }
+ }
+
+#ifdef HAVE_LIBZ
+ if (!tarfile && !ztarfile)
+#else
+ if (!tarfile)
+#endif
+ {
+ fprintf(stderr, _("%s: could not create file \"%s\": %s\n"),
+ progname, fn, strerror(errno));
+ exit(1);
+ }
+
+ /*
+ * Get the COPY data stream
+ */
+ res = PQgetResult(conn);
+ if (!res || PQresultStatus(res) != PGRES_COPY_OUT)
+ {
+ fprintf(stderr, _("%s: could not get COPY data stream: %s\n"),
+ progname, PQerrorMessage(conn));
+ exit(1);
+ }
+
+ while (1)
+ {
+ int r;
+
+ if (copybuf != NULL)
+ {
+ PQfreemem(copybuf);
+ copybuf = NULL;
+ }
+
+ r = PQgetCopyData(conn, ©buf, 0);
+ if (r == -1)
+ {
+ /*
+ * End of chunk. Close file (but not stdout).
+ *
+ * Also, write two completely empty blocks at the end of the tar
+ * file, as required by some tar programs.
+ */
+ char zerobuf[1024];
+
+ MemSet(zerobuf, 0, sizeof(zerobuf));
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, zerobuf, sizeof(zerobuf)) != sizeof(zerobuf))
+ {
+ fprintf(stderr, _("%s: could not write to compressed file '%s': %s\n"),
+ progname, fn, get_gz_error(ztarfile));
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(zerobuf, sizeof(zerobuf), 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file '%s': %m\n"),
+ progname, fn);
+ exit(1);
+ }
+ }
+
+ if (strcmp(tardir, "-") != 0)
+ {
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ gzclose(ztarfile);
+#endif
+ if (tarfile != NULL)
+ fclose(tarfile);
+ }
+
+ break;
+ }
+ else if (r == -2)
+ {
+ fprintf(stderr, _("%s: could not read COPY data: %s\n"),
+ progname, PQerrorMessage(conn));
+ exit(1);
+ }
+
+#ifdef HAVE_LIBZ
+ if (ztarfile != NULL)
+ {
+ if (gzwrite(ztarfile, copybuf, r) != r)
+ {
+ fprintf(stderr, _("%s: could not write to compressed file '%s': %s\n"),
+ progname, fn, get_gz_error(ztarfile));
+ }
+ }
+ else
+#endif
+ {
+ if (fwrite(copybuf, r, 1, tarfile) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file '%s': %m\n"),
+ progname, fn);
+ exit(1);
+ }
+ }
+ totaldone += r;
+ if (showprogress)
+ progress_report(rownum, fn);
+ } /* while (1) */
+
+ if (copybuf != NULL)
+ PQfreemem(copybuf);
+}
+
+/*
+ * Receive a tar format stream from the connection to the server, and unpack
+ * the contents of it into a directory. Only files, directories and
+ * symlinks are supported, no other kinds of special files.
+ *
+ * If the data is for the main data directory, it will be restored in the
+ * specified directory. If it's for another tablespace, it will be restored
+ * in the original directory, since relocation of tablespaces is not
+ * supported.
+ */
+static void
+ReceiveAndUnpackTarFile(PGconn *conn, PGresult *res, int rownum)
+{
+ char current_path[MAXPGPATH];
+ char fn[MAXPGPATH];
+ int current_len_left;
+ int current_padding;
+ char *copybuf = NULL;
+ FILE *file = NULL;
+
+ if (PQgetisnull(res, rownum, 0))
+ strcpy(current_path, basedir);
+ else
+ strcpy(current_path, PQgetvalue(res, rownum, 1));
+
+ /*
+ * Make sure we're unpacking into an empty directory
+ */
+ verify_dir_is_empty_or_create(current_path);
+
+ /*
+ * Get the COPY data
+ */
+ res = PQgetResult(conn);
+ if (!res || PQresultStatus(res) != PGRES_COPY_OUT)
+ {
+ fprintf(stderr, _("%s: could not get COPY data stream: %s\n"),
+ progname, PQerrorMessage(conn));
+ exit(1);
+ }
+
+ while (1)
+ {
+ int r;
+
+ if (copybuf != NULL)
+ {
+ PQfreemem(copybuf);
+ copybuf = NULL;
+ }
+
+ r = PQgetCopyData(conn, ©buf, 0);
+
+ if (r == -1)
+ {
+ /*
+ * End of chunk
+ */
+ if (file)
+ fclose(file);
+
+ break;
+ }
+ else if (r == -2)
+ {
+ fprintf(stderr, _("%s: could not read COPY data: %s\n"),
+ progname, PQerrorMessage(conn));
+ exit(1);
+ }
+
+ if (file == NULL)
+ {
+#ifndef WIN32
+ mode_t filemode;
+#endif
+
+ /*
+ * No current file, so this must be the header for a new file
+ */
+ if (r != 512)
+ {
+ fprintf(stderr, _("%s: Invalid tar block header size: %i\n"),
+ progname, r);
+ exit(1);
+ }
+ totaldone += 512;
+
+ if (sscanf(copybuf + 124, "%11o", ¤t_len_left) != 1)
+ {
+ fprintf(stderr, _("%s: could not parse file size!\n"),
+ progname);
+ exit(1);
+ }
+
+ /* Set permissions on the file */
+ if (sscanf(©buf[100], "%07o ", &filemode) != 1)
+ {
+ fprintf(stderr, _("%s: could not parse file mode!\n"),
+ progname);
+ exit(1);
+ }
+
+ /*
+ * All files are padded up to 512 bytes
+ */
+ current_padding =
+ ((current_len_left + 511) & ~511) - current_len_left;
+
+ /*
+ * First part of header is zero terminated filename
+ */
+ snprintf(fn, sizeof(fn), "%s/%s", current_path, copybuf);
+ if (fn[strlen(fn) - 1] == '/')
+ {
+ /*
+ * Ends in a slash means directory or symlink to directory
+ */
+ if (copybuf[156] == '5')
+ {
+ /*
+ * Directory
+ */
+ fn[strlen(fn) - 1] = '\0'; /* Remove trailing slash */
+ if (mkdir(fn, S_IRWXU) != 0)
+ {
+ fprintf(stderr,
+ _("%s: could not create directory \"%s\": %m\n"),
+ progname, fn);
+ exit(1);
+ }
+#ifndef WIN32
+ if (chmod(fn, filemode))
+ fprintf(stderr, _("%s: could not set permissions on directory '%s': %m\n"),
+ progname, fn);
+#endif
+ }
+ else if (copybuf[156] == '2')
+ {
+ /*
+ * Symbolic link
+ */
+ fn[strlen(fn) - 1] = '\0'; /* Remove trailing slash */
+ if (symlink(©buf[157], fn) != 0)
+ {
+ fprintf(stderr,
+ _("%s: could not create symbolic link from %s to %s: %m\n"),
+ progname, fn, ©buf[157]);
+ exit(1);
+ }
+ }
+ else
+ {
+ fprintf(stderr, _("%s: unknown link indicator '%c'\n"),
+ progname, copybuf[156]);
+ exit(1);
+ }
+ continue; /* directory or link handled */
+ }
+
+ /*
+ * regular file
+ */
+ file = fopen(fn, "wb");
+ if (!file)
+ {
+ fprintf(stderr, _("%s: could not create file '%s': %m\n"),
+ progname, fn);
+ exit(1);
+ }
+
+#ifndef WIN32
+ if (chmod(fn, filemode))
+ fprintf(stderr, _("%s: could not set permissions on file '%s': %m\n"),
+ progname, fn);
+#endif
+
+ if (current_len_left == 0)
+ {
+ /*
+ * Done with this file, next one will be a new tar header
+ */
+ fclose(file);
+ file = NULL;
+ continue;
+ }
+ } /* new file */
+ else
+ {
+ /*
+ * Continuing blocks in existing file
+ */
+ if (current_len_left == 0 && r == current_padding)
+ {
+ /*
+ * Received the padding block for this file, ignore it and
+ * close the file, then move on to the next tar header.
+ */
+ fclose(file);
+ file = NULL;
+ totaldone += r;
+ continue;
+ }
+
+ if (fwrite(copybuf, r, 1, file) != 1)
+ {
+ fprintf(stderr, _("%s: could not write to file '%s': %m\n"),
+ progname, fn);
+ exit(1);
+ }
+ totaldone += r;
+ if (showprogress)
+ progress_report(rownum, fn);
+
+ current_len_left -= r;
+ if (current_len_left == 0 && current_padding == 0)
+ {
+ /*
+ * Received the last block, and there is no padding to be
+ * expected. Close the file and move on to the next tar
+ * header.
+ */
+ fclose(file);
+ file = NULL;
+ continue;
+ }
+ } /* continuing data in existing file */
+ } /* loop over all data blocks */
+
+ if (file != NULL)
+ {
+ fprintf(stderr, _("%s: last file was never finsihed!\n"), progname);
+ exit(1);
+ }
+
+ if (copybuf != NULL)
+ PQfreemem(copybuf);
+}
+
+
+static PGconn *
+GetConnection(void)
+{
+ PGconn *conn;
+ int argcount = 3; /* dbname, replication, password */
+ int i;
+ const char **keywords;
+ const char **values;
+ char *password = NULL;
+
+ if (dbhost)
+ argcount++;
+ if (dbuser)
+ argcount++;
+ if (dbport)
+ argcount++;
+
+ keywords = xmalloc0((argcount + 1) * sizeof(*keywords));
+ values = xmalloc0((argcount + 1) * sizeof(*values));
+
+ keywords[0] = "dbname";
+ values[0] = "replication";
+ keywords[1] = "replication";
+ values[1] = "true";
+ i = 2;
+ if (dbhost)
+ {
+ keywords[i] = "host";
+ values[i] = dbhost;
+ i++;
+ }
+ if (dbuser)
+ {
+ keywords[i] = "user";
+ values[i] = dbuser;
+ i++;
+ }
+ if (dbport)
+ {
+ keywords[i] = "port";
+ values[i] = dbport;
+ i++;
+ }
+
+ while (true)
+ {
+ if (dbgetpassword == 1)
+ {
+ /* Prompt for a password */
+ password = simple_prompt(_("Password: "), 100, false);
+ keywords[argcount - 1] = "password";
+ values[argcount - 1] = password;
+ }
+
+ conn = PQconnectdbParams(keywords, values, true);
+ if (password)
+ free(password);
+
+ if (PQstatus(conn) == CONNECTION_BAD &&
+ PQconnectionNeedsPassword(conn) &&
+ dbgetpassword != -1)
+ {
+ dbgetpassword = 1; /* ask for password next time */
+ continue;
+ }
+
+ if (PQstatus(conn) != CONNECTION_OK)
+ {
+ fprintf(stderr, _("%s: could not connect to server: %s\n"),
+ progname, PQerrorMessage(conn));
+ exit(1);
+ }
+
+ /* Connection ok! */
+ free(values);
+ free(keywords);
+ return conn;
+ }
+}
+
+static void
+BaseBackup()
+{
+ PGconn *conn;
+ PGresult *res;
+ char current_path[MAXPGPATH];
+ char escaped_label[MAXPGPATH];
+ int i;
+
+ /*
+ * Connect in replication mode to the server
+ */
+ conn = GetConnection();
+
+ PQescapeStringConn(conn, escaped_label, label, sizeof(escaped_label), &i);
+ snprintf(current_path, sizeof(current_path), "BASE_BACKUP LABEL '%s' %s",
+ escaped_label,
+ showprogress ? "PROGRESS" : "");
+
+ if (PQsendQuery(conn, current_path) == 0)
+ {
+ fprintf(stderr, _("%s: coult not start base backup: %s\n"),
+ progname, PQerrorMessage(conn));
+ PQfinish(conn);
+ exit(1);
+ }
+
+ /*
+ * Get the header
+ */
+ res = PQgetResult(conn);
+ if (!res || PQresultStatus(res) != PGRES_TUPLES_OK)
+ {
+ fprintf(stderr, _("%s: could not initiate base backup: %s\n"),
+ progname, PQerrorMessage(conn));
+ PQfinish(conn);
+ exit(1);
+ }
+ if (PQntuples(res) < 1)
+ {
+ fprintf(stderr, _("%s: no data returned from server.\n"), progname);
+ PQfinish(conn);
+ exit(1);
+ }
+
+ /*
+ * Sum up the total size, for progress reporting
+ */
+ totalsize = totaldone = 0;
+ tablespacecount = PQntuples(res);
+ for (i = 0; i < PQntuples(res); i++)
+ {
+ if (showprogress)
+ totalsize += atol(PQgetvalue(res, i, 2));
+
+ /*
+ * Verify tablespace directories are empty Don't bother with the first
+ * once since it can be relocated, and it will be checked before we do
+ * anything anyway.
+ */
+ if (basedir != NULL && i > 0)
+ verify_dir_is_empty_or_create(PQgetvalue(res, i, 1));
+ }
+
+ /*
+ * When writing to stdout, require a single tablespace
+ */
+ if (tardir != NULL && strcmp(tardir, "-") == 0 && PQntuples(res) > 1)
+ {
+ fprintf(stderr, _("%s: can only write single tablespace to stdout, database has %i.\n"),
+ progname, PQntuples(res));
+ PQfinish(conn);
+ exit(1);
+ }
+
+ /*
+ * Start receiving chunks
+ */
+ for (i = 0; i < PQntuples(res); i++)
+ {
+ if (tardir != NULL)
+ ReceiveTarFile(conn, res, i);
+ else
+ ReceiveAndUnpackTarFile(conn, res, i);
+ } /* Loop over all tablespaces */
+
+ if (showprogress)
+ {
+ progress_report(PQntuples(res), "");
+ fprintf(stderr, "\n"); /* Need to move to next line */
+ }
+ PQclear(res);
+
+ res = PQgetResult(conn);
+ if (!res || PQresultStatus(res) != PGRES_COMMAND_OK)
+ {
+ fprintf(stderr, _("%s: final receive failed: %s\n"),
+ progname, PQerrorMessage(conn));
+ exit(1);
+ }
+
+ /*
+ * End of copy data. Final result is already checked inside the loop.
+ */
+ PQfinish(conn);
+
+ if (verbose)
+ fprintf(stderr, "%s: base backup completed.\n", progname);
+}
+
+
+int
+main(int argc, char **argv)
+{
+ static struct option long_options[] = {
+ {"help", no_argument, NULL, '?'},
+ {"version", no_argument, NULL, 'V'},
+ {"pgdata", required_argument, NULL, 'D'},
+ {"tardir", required_argument, NULL, 'T'},
+ {"compress", required_argument, NULL, 'Z'},
+ {"label", required_argument, NULL, 'l'},
+ {"host", required_argument, NULL, 'h'},
+ {"port", required_argument, NULL, 'p'},
+ {"username", required_argument, NULL, 'U'},
+ {"no-password", no_argument, NULL, 'w'},
+ {"password", no_argument, NULL, 'W'},
+ {"verbose", no_argument, NULL, 'v'},
+ {"progress", no_argument, NULL, 'P'},
+ {NULL, 0, NULL, 0}
+ };
+ int c;
+
+ int option_index;
+
+ progname = get_progname(argv[0]);
+ set_pglocale_pgservice(argv[0], PG_TEXTDOMAIN("pg_basebackup"));
+
+ if (argc > 1)
+ {
+ if (strcmp(argv[1], "-h") == 0 || strcmp(argv[1], "--help") == 0 ||
+ strcmp(argv[1], "-?") == 0)
+ {
+ usage();
+ exit(0);
+ }
+ else if (strcmp(argv[1], "-V") == 0
+ || strcmp(argv[1], "--version") == 0)
+ {
+ puts("pg_basebackup (PostgreSQL) " PG_VERSION);
+ exit(0);
+ }
+ }
+
+ while ((c = getopt_long(argc, argv, "D:T:l:Z:h:p:U:wWvP",
+ long_options, &option_index)) != -1)
+ {
+ switch (c)
+ {
+ case 'D':
+ basedir = xstrdup(optarg);
+ break;
+ case 'T':
+ tardir = xstrdup(optarg);
+ break;
+ case 'l':
+ label = xstrdup(optarg);
+ break;
+ case 'Z':
+ compresslevel = atoi(optarg);
+ break;
+ case 'h':
+ dbhost = xstrdup(optarg);
+ break;
+ case 'p':
+ if (atoi(optarg) == 0)
+ {
+ fprintf(stderr, _("%s: invalid port number \"%s\""),
+ progname, optarg);
+ exit(1);
+ }
+ dbport = xstrdup(optarg);
+ break;
+ case 'U':
+ dbuser = xstrdup(optarg);
+ break;
+ case 'w':
+ dbgetpassword = -1;
+ break;
+ case 'W':
+ dbgetpassword = 1;
+ break;
+ case 'v':
+ verbose++;
+ break;
+ case 'P':
+ showprogress = true;
+ break;
+ default:
+
+ /*
+ * getopt_long already emitted a complaint
+ */
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+ }
+
+ /*
+ * Any non-option arguments?
+ */
+ if (optind < argc)
+ {
+ fprintf(stderr,
+ _("%s: too many command-line arguments (first is \"%s\")\n"),
+ progname, argv[optind + 1]);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /*
+ * Required arguments
+ */
+ if (basedir == NULL && tardir == NULL)
+ {
+ fprintf(stderr, _("%s: no target directory specified\n"), progname);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ /*
+ * Mutually exclusive arguments
+ */
+ if (basedir != NULL && tardir != NULL)
+ {
+ fprintf(stderr,
+ _("%s: both directory mode and tar mode cannot be specified\n"),
+ progname);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+ if (basedir != NULL && compresslevel > 0)
+ {
+ fprintf(stderr,
+ _("%s: only tar mode backups can be compressed\n"),
+ progname);
+ fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
+ progname);
+ exit(1);
+ }
+
+#ifndef HAVE_LIBZ
+ if (compresslevel > 0)
+ {
+ fprintf(stderr,
+ _("%s: this build does not support compression\n"),
+ progname);
+ exit(1);
+ }
+#else
+ if (compresslevel > 0 && strcmp(tardir, "-") == 0)
+ {
+ fprintf(stderr,
+ _("%s: compression is not supported on standard output\n"),
+ progname);
+ exit(1);
+ }
+#endif
+
+ /*
+ * Verify directories
+ */
+ if (basedir)
+ verify_dir_is_empty_or_create(basedir);
+ else if (strcmp(tardir, "-") != 0)
+ verify_dir_is_empty_or_create(tardir);
+
+
+
+ BaseBackup();
+
+ return 0;
+}
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index 29c3c77..40fb130 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -273,6 +273,8 @@ sub mkvcbuild
$initdb->AddLibrary('wsock32.lib');
$initdb->AddLibrary('ws2_32.lib');
+ my $pgbasebackup = AddSimpleFrontend('pg_basebackup', 1);
+
my $pgconfig = AddSimpleFrontend('pg_config');
my $pgcontrol = AddSimpleFrontend('pg_controldata');
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
