On Thu, Mar 01, 2012 at 09:06:10PM -0500, Bruce Momjian wrote:
> OK, combining your and Robert's ideas, how about I have pg_upgrade write
> the server log to a file, and the pg_dump output to a file (with its
> stderr), and if pg_upgrade fails, I report the failure and mention those
> files. If pg_upgrade succeeds, I remove the files? pg_upgrade already
> creates temporary files that it removes on completion.
OK, I have completed a rework of pg_upgrade logging. pg_upgrade had 4
logging options, -g, -G, -l, and -v, and still it wasn't possible to get
useful logging. :-(
What I have done with this patch is to remove -g, -G, and -l, and
unconditionally write to 4 log files in the current directory (in
addition to the 3 SQL files I already create).
If pg_upgrade succeeds, the files are removed, but if it fails (or if
the new -r/retain option is used), the files remain. Here is a sample
failure when I create a plpgsql function in the old server, but truncate
plpgsql.so in the new server:
Performing Upgrade
------------------
Analyzing all rows in the new cluster ok
Freezing all rows on the new cluster ok
Deleting new commit clogs ok
Copying old commit clogs to new server ok
Setting next transaction ID for new cluster ok
Resetting WAL archives ok
Setting frozenxid counters in new cluster ok
Creating databases in the new cluster ok
Adding support functions to new cluster ok
Restoring database schema to new cluster
Consult the last few lines of "pg_upgrade_restore.log" for
the probable cause of the failure.
Failure, exiting
$ tail pg_upgrade_restore.log
COMMENT ON LANGUAGE plpythonu IS 'PL/PythonU untrusted procedural
language';
COMMENT
SET search_path = public, pg_catalog;
SET
CREATE FUNCTION x() RETURNS integer
LANGUAGE plpgsql
AS $$begin
select pg_sleep(1);
end$$;
psql:/usr/local/pgdev/pg_upgrade/pg_upgrade_dump_db.sql:233:
ERROR: could not load library
"/usr/local/pgsql/lib/plpgsql.so":
/usr/local/pgsql/lib/plpgsql.so: file too short
That seems quite clear; I enabled --echo-queries in psql so you can see
the query that generated the error.
These changes should make pg_upgrade errors much easier to diagnose. I
hope to apply this for PG 9.2.
--
Bruce Momjian <br...@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
diff --git a/contrib/pg_upgrade/check.c b/contrib/pg_upgrade/check.c
new file mode 100644
index a5f63eb..63f6645
*** a/contrib/pg_upgrade/check.c
--- b/contrib/pg_upgrade/check.c
*************** issue_warnings(char *sequence_script_fil
*** 165,176 ****
if (sequence_script_file_name)
{
prep_status("Adjusting sequences");
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
"-f \"%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user,
! sequence_script_file_name, log_opts.filename2);
unlink(sequence_script_file_name);
check_ok();
}
--- 165,177 ----
if (sequence_script_file_name)
{
prep_status("Adjusting sequences");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
"-f \"%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user,
! sequence_script_file_name, UTILITY_LOG_FILE);
unlink(sequence_script_file_name);
check_ok();
}
diff --git a/contrib/pg_upgrade/controldata.c b/contrib/pg_upgrade/controldata.c
new file mode 100644
index 5239601..debb444
*** a/contrib/pg_upgrade/controldata.c
--- b/contrib/pg_upgrade/controldata.c
*************** get_control_data(ClusterInfo *cluster, b
*** 126,136 ****
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
! if (log_opts.debug)
! fputs(bufin, log_opts.debug_fd);
#ifdef WIN32
-
/*
* Due to an installer bug, LANG=C doesn't work for PG 8.3.3, but does
* work 8.2.6 and 8.3.7, so check for non-ASCII output and suggest a
--- 126,135 ----
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
! if (log_opts.verbose)
! pg_log(PG_INFO, "%s", bufin);
#ifdef WIN32
/*
* Due to an installer bug, LANG=C doesn't work for PG 8.3.3, but does
* work 8.2.6 and 8.3.7, so check for non-ASCII output and suggest a
diff --git a/contrib/pg_upgrade/dump.c b/contrib/pg_upgrade/dump.c
new file mode 100644
index 772ca37..36f9a0f
*** a/contrib/pg_upgrade/dump.c
--- b/contrib/pg_upgrade/dump.c
*************** generate_old_dump(void)
*** 22,31 ****
* --binary-upgrade records the width of dropped columns in pg_class, and
* restores the frozenid's for databases and relations.
*/
! exec_prog(true,
SYSTEMQUOTE "\"%s/pg_dumpall\" --port %d --username \"%s\" "
! "--schema-only --binary-upgrade > \"%s/" ALL_DUMP_FILE "\""
! SYSTEMQUOTE, new_cluster.bindir, old_cluster.port, os_info.user, os_info.cwd);
check_ok();
}
--- 22,32 ----
* --binary-upgrade records the width of dropped columns in pg_class, and
* restores the frozenid's for databases and relations.
*/
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/pg_dumpall\" --port %d --username \"%s\" "
! "--schema-only --binary-upgrade >> \"%s/%s\" 2> \"%s\""
! SYSTEMQUOTE, new_cluster.bindir, old_cluster.port, os_info.user,
! os_info.cwd, ALL_DUMP_FILE, UTILITY_LOG_FILE);
check_ok();
}
diff --git a/contrib/pg_upgrade/exec.c b/contrib/pg_upgrade/exec.c
new file mode 100644
index b870ded..2aa3cc8
*** a/contrib/pg_upgrade/exec.c
--- b/contrib/pg_upgrade/exec.c
*************** static int win32_check_directory_write_p
*** 34,40 ****
* instead of returning should an error occur.
*/
int
! exec_prog(bool throw_error, const char *fmt,...)
{
va_list args;
int result;
--- 34,40 ----
* instead of returning should an error occur.
*/
int
! exec_prog(bool throw_error, const char *log_file, const char *fmt,...)
{
va_list args;
int result;
*************** exec_prog(bool throw_error, const char *
*** 50,57 ****
if (result != 0)
{
pg_log(throw_error ? PG_FATAL : PG_INFO,
! "There were problems executing \"%s\"\n", cmd);
return 1;
}
--- 50,60 ----
if (result != 0)
{
+ pg_log(PG_INFO, "There were problems executing \"%s\"\n", cmd);
pg_log(throw_error ? PG_FATAL : PG_INFO,
! "Consult the last few lines of \"%s\" for\n"
! "the probable cause of the failure.\n",
! log_file);
return 1;
}
diff --git a/contrib/pg_upgrade/info.c b/contrib/pg_upgrade/info.c
new file mode 100644
index 692cdc2..a1f6210
*** a/contrib/pg_upgrade/info.c
--- b/contrib/pg_upgrade/info.c
*************** create_rel_filename_map(const char *old_
*** 132,150 ****
void
print_maps(FileNameMap *maps, int n_maps, const char *db_name)
{
! if (log_opts.debug)
{
int mapnum;
! pg_log(PG_DEBUG, "mappings for database \"%s\":\n", db_name);
for (mapnum = 0; mapnum < n_maps; mapnum++)
! pg_log(PG_DEBUG, "%s.%s: %u to %u\n",
maps[mapnum].nspname, maps[mapnum].relname,
maps[mapnum].old_relfilenode,
maps[mapnum].new_relfilenode);
! pg_log(PG_DEBUG, "\n\n");
}
}
--- 132,150 ----
void
print_maps(FileNameMap *maps, int n_maps, const char *db_name)
{
! if (log_opts.verbose)
{
int mapnum;
! pg_log(PG_INFO, "mappings for database \"%s\":\n", db_name);
for (mapnum = 0; mapnum < n_maps; mapnum++)
! pg_log(PG_INFO, "%s.%s: %u to %u\n",
maps[mapnum].nspname, maps[mapnum].relname,
maps[mapnum].old_relfilenode,
maps[mapnum].new_relfilenode);
! pg_log(PG_INFO, "\n\n");
}
}
*************** get_db_and_rel_infos(ClusterInfo *cluste
*** 168,176 ****
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
get_rel_infos(cluster, &cluster->dbarr.dbs[dbnum]);
! if (log_opts.debug)
{
! pg_log(PG_DEBUG, "\n%s databases:\n", CLUSTER_NAME(cluster));
print_db_infos(&cluster->dbarr);
}
}
--- 168,176 ----
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
get_rel_infos(cluster, &cluster->dbarr.dbs[dbnum]);
! if (log_opts.verbose)
{
! pg_log(PG_INFO, "\n%s databases:\n", CLUSTER_NAME(cluster));
print_db_infos(&cluster->dbarr);
}
}
*************** print_db_infos(DbInfoArr *db_arr)
*** 368,376 ****
for (dbnum = 0; dbnum < db_arr->ndbs; dbnum++)
{
! pg_log(PG_DEBUG, "Database: %s\n", db_arr->dbs[dbnum].db_name);
print_rel_infos(&db_arr->dbs[dbnum].rel_arr);
! pg_log(PG_DEBUG, "\n\n");
}
}
--- 368,378 ----
for (dbnum = 0; dbnum < db_arr->ndbs; dbnum++)
{
! if (log_opts.verbose)
! pg_log(PG_INFO, "Database: %s\n", db_arr->dbs[dbnum].db_name);
print_rel_infos(&db_arr->dbs[dbnum].rel_arr);
! if (log_opts.verbose)
! pg_log(PG_INFO, "\n\n");
}
}
*************** print_rel_infos(RelInfoArr *arr)
*** 380,387 ****
{
int relnum;
! for (relnum = 0; relnum < arr->nrels; relnum++)
! pg_log(PG_DEBUG, "relname: %s.%s: reloid: %u reltblspace: %s\n",
! arr->rels[relnum].nspname, arr->rels[relnum].relname,
! arr->rels[relnum].reloid, arr->rels[relnum].tablespace);
}
--- 382,390 ----
{
int relnum;
! if (log_opts.verbose)
! for (relnum = 0; relnum < arr->nrels; relnum++)
! pg_log(PG_INFO, "relname: %s.%s: reloid: %u reltblspace: %s\n",
! arr->rels[relnum].nspname, arr->rels[relnum].relname,
! arr->rels[relnum].reloid, arr->rels[relnum].tablespace);
}
diff --git a/contrib/pg_upgrade/option.c b/contrib/pg_upgrade/option.c
new file mode 100644
index 0a105ef..2f30625
*** a/contrib/pg_upgrade/option.c
--- b/contrib/pg_upgrade/option.c
*************** parseCommandLine(int argc, char *argv[])
*** 46,63 ****
{"user", required_argument, NULL, 'u'},
{"check", no_argument, NULL, 'c'},
- {"debug", no_argument, NULL, 'g'},
- {"debugfile", required_argument, NULL, 'G'},
{"link", no_argument, NULL, 'k'},
! {"logfile", required_argument, NULL, 'l'},
{"verbose", no_argument, NULL, 'v'},
{NULL, 0, NULL, 0}
};
int option; /* Command line option */
int optindex = 0; /* used by getopt_long */
int os_user_effective_id;
char *return_buf;
!
user_opts.transfer_mode = TRANSFER_MODE_COPY;
os_info.progname = get_progname(argv[0]);
--- 46,63 ----
{"user", required_argument, NULL, 'u'},
{"check", no_argument, NULL, 'c'},
{"link", no_argument, NULL, 'k'},
! {"retain", no_argument, NULL, 'r'},
{"verbose", no_argument, NULL, 'v'},
{NULL, 0, NULL, 0}
};
int option; /* Command line option */
int optindex = 0; /* used by getopt_long */
int os_user_effective_id;
+ FILE *fp;
char *return_buf;
! int i;
!
user_opts.transfer_mode = TRANSFER_MODE_COPY;
os_info.progname = get_progname(argv[0]);
*************** parseCommandLine(int argc, char *argv[])
*** 98,104 ****
if (return_buf == NULL)
pg_log(PG_FATAL, "Could not access current working directory: %s\n", getErrorText(errno));
! while ((option = getopt_long(argc, argv, "d:D:b:B:cgG:kl:o:O:p:P:u:v",
long_options, &optindex)) != -1)
{
switch (option)
--- 98,104 ----
if (return_buf == NULL)
pg_log(PG_FATAL, "Could not access current working directory: %s\n", getErrorText(errno));
! while ((option = getopt_long(argc, argv, "d:D:b:B:cko:O:p:P:ru:v",
long_options, &optindex)) != -1)
{
switch (option)
*************** parseCommandLine(int argc, char *argv[])
*** 125,151 ****
new_cluster.pgconfig = pg_strdup(optarg);
break;
- case 'g':
- pg_log(PG_REPORT, "Running in debug mode\n");
- log_opts.debug = true;
- break;
-
- case 'G':
- if ((log_opts.debug_fd = fopen(optarg, "w")) == NULL)
- {
- pg_log(PG_FATAL, "cannot open debug file\n");
- exit(1);
- }
- break;
-
case 'k':
user_opts.transfer_mode = TRANSFER_MODE_LINK;
break;
- case 'l':
- log_opts.filename = pg_strdup(optarg);
- break;
-
case 'o':
old_cluster.pgopts = pg_strdup(optarg);
break;
--- 125,134 ----
*************** parseCommandLine(int argc, char *argv[])
*** 175,180 ****
--- 158,167 ----
}
break;
+ case 'r':
+ log_opts.retain = true;
+ break;
+
case 'u':
pg_free(os_info.user);
os_info.user = pg_strdup(optarg);
*************** parseCommandLine(int argc, char *argv[])
*** 189,194 ****
--- 176,183 ----
case 'v':
pg_log(PG_REPORT, "Running in verbose mode\n");
log_opts.verbose = true;
+ /* renable retain too */
+ log_opts.retain = true;
break;
default:
*************** parseCommandLine(int argc, char *argv[])
*** 199,235 ****
}
}
! if (log_opts.filename != NULL)
{
! /*
! * We must use append mode so output generated by child processes via
! * ">>" will not be overwritten, and we want the file truncated on
! * start.
! */
! /* truncate */
! if ((log_opts.fd = fopen(log_opts.filename, "w")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", log_opts.filename);
! fclose(log_opts.fd);
! if ((log_opts.fd = fopen(log_opts.filename, "a")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", log_opts.filename);
}
- else
- log_opts.filename = pg_strdup(DEVNULL);
! /* WIN32 files do not accept writes from multiple processes */
! #ifndef WIN32
! log_opts.filename2 = pg_strdup(log_opts.filename);
! #else
! log_opts.filename2 = pg_strdup(DEVNULL);
! #endif
!
! /* if no debug file name, output to the terminal */
! if (log_opts.debug && !log_opts.debug_fd)
! {
! log_opts.debug_fd = fopen(DEVTTY, "w");
! if (!log_opts.debug_fd)
! pg_log(PG_FATAL, "cannot write to terminal\n");
! }
/* Get values from env if not already set */
check_required_directory(&old_cluster.bindir, "PGBINOLD", "-b",
--- 188,204 ----
}
}
! /* truncate log files because we append to them */
! for (i = 0; i < NUM_OUTPUT_FILES; i++)
{
! if ((fp = fopen(output_files[i], "w")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n",
! output_files[i]);
! fclose(fp);
}
! if ((log_opts.fp = fopen(INTERNAL_LOG_FILE, "w")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", INTERNAL_LOG_FILE);
/* Get values from env if not already set */
check_required_directory(&old_cluster.bindir, "PGBINOLD", "-b",
*************** Options:\n\
*** 256,271 ****
-c, --check check clusters only, don't change any data\n\
-d, --old-datadir=OLDDATADIR old cluster data directory\n\
-D, --new-datadir=NEWDATADIR new cluster data directory\n\
- -g, --debug enable debugging\n\
- -G, --debugfile=FILENAME output debugging activity to file\n\
-k, --link link instead of copying files to new cluster\n\
- -l, --logfile=FILENAME log internal activity to file\n\
-o, --old-options=OPTIONS old cluster options to pass to the server\n\
-O, --new-options=OPTIONS new cluster options to pass to the server\n\
-p, --old-port=OLDPORT old cluster port number (default %d)\n\
-P, --new-port=NEWPORT new cluster port number (default %d)\n\
-u, --user=NAME cluster superuser (default \"%s\")\n\
! -v, --verbose enable verbose output\n\
-V, --version display version information, then exit\n\
-h, --help show this help, then exit\n\
\n\
--- 225,238 ----
-c, --check check clusters only, don't change any data\n\
-d, --old-datadir=OLDDATADIR old cluster data directory\n\
-D, --new-datadir=NEWDATADIR new cluster data directory\n\
-k, --link link instead of copying files to new cluster\n\
-o, --old-options=OPTIONS old cluster options to pass to the server\n\
-O, --new-options=OPTIONS new cluster options to pass to the server\n\
-p, --old-port=OLDPORT old cluster port number (default %d)\n\
-P, --new-port=NEWPORT new cluster port number (default %d)\n\
+ -r, --retain retain SQL and log files after success\n\
-u, --user=NAME cluster superuser (default \"%s\")\n\
! -v, --verbose enable verbose logging, enables -r/--retain\n\
-V, --version display version information, then exit\n\
-h, --help show this help, then exit\n\
\n\
*************** adjust_data_dir(ClusterInfo *cluster)
*** 354,372 ****
{
char filename[MAXPGPATH];
char cmd[MAXPGPATH], cmd_output[MAX_STRING];
! FILE *fd, *output;
/* If there is no postgresql.conf, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/postgresql.conf", cluster->pgconfig);
! if ((fd = fopen(filename, "r")) == NULL)
return;
! fclose(fd);
/* If PG_VERSION exists, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/PG_VERSION", cluster->pgconfig);
! if ((fd = fopen(filename, "r")) != NULL)
{
! fclose(fd);
return;
}
--- 321,339 ----
{
char filename[MAXPGPATH];
char cmd[MAXPGPATH], cmd_output[MAX_STRING];
! FILE *fp, *output;
/* If there is no postgresql.conf, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/postgresql.conf", cluster->pgconfig);
! if ((fp = fopen(filename, "r")) == NULL)
return;
! fclose(fp);
/* If PG_VERSION exists, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/PG_VERSION", cluster->pgconfig);
! if ((fp = fopen(filename, "r")) != NULL)
{
! fclose(fp);
return;
}
diff --git a/contrib/pg_upgrade/pg_upgrade.c b/contrib/pg_upgrade/pg_upgrade.c
new file mode 100644
index 3078bcd..3579c9d
*** a/contrib/pg_upgrade/pg_upgrade.c
--- b/contrib/pg_upgrade/pg_upgrade.c
*************** ClusterInfo old_cluster,
*** 55,60 ****
--- 55,71 ----
new_cluster;
OSInfo os_info;
+ char *output_files[NUM_OUTPUT_FILES] = {
+ ALL_DUMP_FILE,
+ GLOBALS_DUMP_FILE,
+ DB_DUMP_FILE,
+ SERVER_LOG_FILE,
+ RESTORE_LOG_FILE,
+ UTILITY_LOG_FILE,
+ INTERNAL_LOG_FILE
+ };
+
+
int
main(int argc, char **argv)
{
*************** main(int argc, char **argv)
*** 127,135 ****
* because there is no need to have the schema load use new oids.
*/
prep_status("Setting next OID for new cluster");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -o %u \"%s\" > "
! DEVNULL SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid, new_cluster.pgdata);
check_ok();
create_script_for_old_cluster_deletion(&deletion_script_file_name);
--- 138,148 ----
* because there is no need to have the schema load use new oids.
*/
prep_status("Setting next OID for new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE "\"%s/pg_resetxlog\" -o %u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
create_script_for_old_cluster_deletion(&deletion_script_file_name);
*************** prepare_new_cluster(void)
*** 193,202 ****
* --analyze so autovacuum doesn't update statistics later
*/
prep_status("Analyzing all rows in the new cluster");
! exec_prog(true,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --analyze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, log_opts.filename2);
check_ok();
/*
--- 206,215 ----
* --analyze so autovacuum doesn't update statistics later
*/
prep_status("Analyzing all rows in the new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --analyze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, UTILITY_LOG_FILE);
check_ok();
/*
*************** prepare_new_cluster(void)
*** 206,215 ****
* later.
*/
prep_status("Freezing all rows on the new cluster");
! exec_prog(true,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --freeze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, log_opts.filename2);
check_ok();
get_pg_database_relfilenode(&new_cluster);
--- 219,228 ----
* later.
*/
prep_status("Freezing all rows on the new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --freeze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, UTILITY_LOG_FILE);
check_ok();
get_pg_database_relfilenode(&new_cluster);
*************** prepare_new_databases(void)
*** 243,255 ****
* support functions in template1 but pg_dumpall creates database using
* the template0 template.
*/
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
! /* --no-psqlrc prevents AUTOCOMMIT=off */
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! GLOBALS_DUMP_FILE, log_opts.filename2);
check_ok();
/* we load this to get a current list of databases */
--- 256,269 ----
* support functions in template1 but pg_dumpall creates database using
* the template0 template.
*/
! exec_prog(true, RESTORE_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
! /* --no-psqlrc prevents AUTOCOMMIT=off */
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! GLOBALS_DUMP_FILE, RESTORE_LOG_FILE);
check_ok();
/* we load this to get a current list of databases */
*************** create_new_objects(void)
*** 275,286 ****
check_ok();
prep_status("Restoring database schema to new cluster");
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! DB_DUMP_FILE, log_opts.filename2);
check_ok();
/* regenerate now that we have objects in the databases */
--- 289,301 ----
check_ok();
prep_status("Restoring database schema to new cluster");
! exec_prog(true, RESTORE_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! DB_DUMP_FILE, RESTORE_LOG_FILE);
check_ok();
/* regenerate now that we have objects in the databases */
*************** copy_clog_xlog_xid(void)
*** 306,334 ****
check_ok();
prep_status("Copying old commit clogs to new server");
#ifndef WIN32
! exec_prog(true, SYSTEMQUOTE "%s \"%s\" \"%s\"" SYSTEMQUOTE,
"cp -Rf",
#else
/* flags: everything, no confirm, quiet, overwrite read-only */
! exec_prog(true, SYSTEMQUOTE "%s \"%s\" \"%s\\\"" SYSTEMQUOTE,
"xcopy /e /y /q /r",
#endif
! old_clog_path, new_clog_path);
check_ok();
/* set the next transaction id of the new cluster */
prep_status("Setting next transaction ID for new cluster");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -f -x %u \"%s\" > " DEVNULL SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtxid, new_cluster.pgdata);
check_ok();
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -l %u,%u,%u \"%s\" >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_tli,
! old_cluster.controldata.logid, old_cluster.controldata.nxtlogseg,
! new_cluster.pgdata, log_opts.filename2);
check_ok();
}
--- 321,358 ----
check_ok();
prep_status("Copying old commit clogs to new server");
+ exec_prog(true, UTILITY_LOG_FILE,
#ifndef WIN32
! SYSTEMQUOTE "%s \"%s\" \"%s\" >> \"%s\" 2>&1" SYSTEMQUOTE,
"cp -Rf",
#else
/* flags: everything, no confirm, quiet, overwrite read-only */
! SYSTEMQUOTE "%s \"%s\" \"%s\\\" >> \"%s\" 2>&1" SYSTEMQUOTE,
"xcopy /e /y /q /r",
#endif
! old_clog_path, new_clog_path, UTILITY_LOG_FILE);
check_ok();
/* set the next transaction id of the new cluster */
prep_status("Setting next transaction ID for new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE
! "\"%s/pg_resetxlog\" -f -x %u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE, new_cluster.bindir,
! old_cluster.controldata.chkpnt_nxtxid,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE
! "\"%s/pg_resetxlog\" -l %u,%u,%u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE, new_cluster.bindir,
! old_cluster.controldata.chkpnt_tli,
! old_cluster.controldata.logid,
! old_cluster.controldata.nxtlogseg,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
}
*************** set_frozenxids(void)
*** 421,438 ****
static void
cleanup(void)
{
! char filename[MAXPGPATH];
!
! if (log_opts.fd)
! fclose(log_opts.fd);
! if (log_opts.debug_fd)
! fclose(log_opts.debug_fd);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, ALL_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, GLOBALS_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, DB_DUMP_FILE);
! unlink(filename);
}
--- 445,463 ----
static void
cleanup(void)
{
! fclose(log_opts.fp);
! /* Remove dump and log files? */
! if (!log_opts.retain)
! {
! int i;
! char filename[MAXPGPATH];
! for (i = 0; i < NUM_OUTPUT_FILES; i++)
! {
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd,
! output_files[i]);
! unlink(filename);
! }
! }
}
diff --git a/contrib/pg_upgrade/pg_upgrade.h b/contrib/pg_upgrade/pg_upgrade.h
new file mode 100644
index a954815..2a7f7e1
*** a/contrib/pg_upgrade/pg_upgrade.h
--- b/contrib/pg_upgrade/pg_upgrade.h
***************
*** 35,40 ****
--- 35,68 ----
#define GLOBALS_DUMP_FILE "pg_upgrade_dump_globals.sql"
#define DB_DUMP_FILE "pg_upgrade_dump_db.sql"
+ #define SERVER_LOG_FILE "pg_upgrade_server.log"
+ #define RESTORE_LOG_FILE "pg_upgrade_restore.log"
+ #define UTILITY_LOG_FILE "pg_upgrade_utility.log"
+ #define INTERNAL_LOG_FILE "pg_upgrade_internal.log"
+
+ #define NUM_OUTPUT_FILES 7
+ extern char *output_files[];
+
+ /*
+ * WIN32 files do not accept writes from multiple processes
+ *
+ * On Win32, we can't send both pg_upgrade output and command output to the
+ * same file because we get the error: "The process cannot access the file
+ * because it is being used by another process." so send the pg_ctl
+ * command-line output to the utility log file on Windows, rather than
+ * into the server log file.
+ *
+ * We could use the Windows pgwin32_open() flags to allow shared file
+ * writes but is unclear how all other tools would use those flags, so
+ * we just avoid it and log a little differently on Windows; we adjust
+ * the error message appropriately.
+ */
+ #ifndef WIN32
+ #define SERVER_LOG_FILE2 SERVER_LOG_FILE
+ #else
+ #define SERVER_LOG_FILE2 UTILITY_LOG_FILE
+ #endif
+
#ifndef WIN32
#define pg_copy_file copy_file
#define pg_mv_file rename
*************** typedef enum
*** 169,176 ****
PG_INFO,
PG_REPORT,
PG_WARNING,
! PG_FATAL,
! PG_DEBUG
} eLogType;
--- 197,203 ----
PG_INFO,
PG_REPORT,
PG_WARNING,
! PG_FATAL
} eLogType;
*************** typedef struct
*** 204,227 ****
*/
typedef struct
{
! char *filename; /* name of log file (may be /dev/null) */
! /*
! * WIN32 files do not accept writes from multiple processes
! *
! * On Win32, we can't send both pg_upgrade output and command output to the
! * same file because we get the error: "The process cannot access the file
! * because it is being used by another process." so we have to send all
! * other output to 'nul'. Therefore, we set this to DEVNULL on Win32, and
! * it equals 'filename' on all other platforms.
! *
! * We could use the Windows pgwin32_open() flags to allow shared file
! * writes but is unclear how all other tools would use those flags, so
! * we just avoid it and log a little less on Windows.
! */
! char *filename2;
! FILE *fd; /* log FILE */
! bool debug; /* TRUE -> log more information */
! FILE *debug_fd; /* debug-level log FILE */
bool verbose; /* TRUE -> be verbose in messages */
} LogOpts;
--- 231,238 ----
*/
typedef struct
{
! bool retain; /* retain log files on success */
! FILE *fp; /* log FILE */
bool verbose; /* TRUE -> be verbose in messages */
} LogOpts;
*************** void split_old_dump(void);
*** 294,301 ****
/* exec.c */
! int exec_prog(bool throw_error, const char *cmd, ...)
! __attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
void verify_directories(void);
bool is_server_running(const char *datadir);
--- 305,312 ----
/* exec.c */
! int exec_prog(bool throw_error, const char *log_file, const char *cmd, ...)
! __attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
void verify_directories(void);
bool is_server_running(const char *datadir);
diff --git a/contrib/pg_upgrade/server.c b/contrib/pg_upgrade/server.c
new file mode 100644
index 989af63..22c4f65
*** a/contrib/pg_upgrade/server.c
--- b/contrib/pg_upgrade/server.c
*************** executeQueryOrDie(PGconn *conn, const ch
*** 80,86 ****
vsnprintf(command, sizeof(command), fmt, args);
va_end(args);
! pg_log(PG_DEBUG, "executing: %s\n", command);
result = PQexec(conn, command);
status = PQresultStatus(result);
--- 80,87 ----
vsnprintf(command, sizeof(command), fmt, args);
va_end(args);
! if (log_opts.verbose)
! pg_log(PG_INFO, "executing: %s\n", command);
result = PQexec(conn, command);
status = PQresultStatus(result);
*************** start_postmaster(ClusterInfo *cluster)
*** 161,177 ****
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" "
"-o \"-p %d %s %s\" start >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, log_opts.filename2, cluster->pgconfig, cluster->port,
(cluster->controldata.cat_ver >=
BINARY_UPGRADE_SERVER_FLAG_CAT_VER) ? "-b" :
"-c autovacuum=off -c autovacuum_freeze_max_age=2000000000",
! cluster->pgopts ? cluster->pgopts : "", log_opts.filename2);
/*
* Don't throw an error right away, let connecting throw the error because
* it might supply a reason for the failure.
*/
! pg_ctl_return = exec_prog(false, "%s", cmd);
/* Check to see if we can connect to the server; if not, report it. */
if ((conn = get_db_conn(cluster, "template1")) == NULL ||
--- 162,183 ----
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" "
"-o \"-p %d %s %s\" start >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, SERVER_LOG_FILE, cluster->pgconfig, cluster->port,
(cluster->controldata.cat_ver >=
BINARY_UPGRADE_SERVER_FLAG_CAT_VER) ? "-b" :
"-c autovacuum=off -c autovacuum_freeze_max_age=2000000000",
! cluster->pgopts ? cluster->pgopts : "", SERVER_LOG_FILE2);
/*
* Don't throw an error right away, let connecting throw the error because
* it might supply a reason for the failure.
*/
! pg_ctl_return = exec_prog(false,
! /* pass both file names if the differ */
! (strcmp(SERVER_LOG_FILE, SERVER_LOG_FILE2) == 0) ?
! SERVER_LOG_FILE :
! SERVER_LOG_FILE " or " SERVER_LOG_FILE2,
! "%s", cmd);
/* Check to see if we can connect to the server; if not, report it. */
if ((conn = get_db_conn(cluster, "template1")) == NULL ||
*************** stop_postmaster(bool fast)
*** 211,221 ****
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" -o \"%s\" "
"%s stop >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, log_opts.filename2, cluster->pgconfig,
cluster->pgopts ? cluster->pgopts : "",
! fast ? "-m fast" : "", log_opts.filename2);
! exec_prog(fast ? false : true, "%s", cmd);
os_info.running_cluster = NULL;
}
--- 217,227 ----
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" -o \"%s\" "
"%s stop >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, SERVER_LOG_FILE2, cluster->pgconfig,
cluster->pgopts ? cluster->pgopts : "",
! fast ? "-m fast" : "", SERVER_LOG_FILE2);
! exec_prog(fast ? false : true, SERVER_LOG_FILE2, "%s", cmd);
os_info.running_cluster = NULL;
}
diff --git a/contrib/pg_upgrade/util.c b/contrib/pg_upgrade/util.c
new file mode 100644
index 94eaa18..ab78e3e
*** a/contrib/pg_upgrade/util.c
--- b/contrib/pg_upgrade/util.c
*************** pg_log(eLogType type, char *fmt,...)
*** 77,89 ****
vsnprintf(message, sizeof(message), fmt, args);
va_end(args);
! if (log_opts.fd != NULL)
{
! fwrite(message, strlen(message), 1, log_opts.fd);
/* if we are using OVERWRITE_MESSAGE, add newline */
if (strchr(message, '\r') != NULL)
! fwrite("\n", 1, 1, log_opts.fd);
! fflush(log_opts.fd);
}
switch (type)
--- 77,89 ----
vsnprintf(message, sizeof(message), fmt, args);
va_end(args);
! if (type != PG_INFO || log_opts.verbose)
{
! fwrite(message, strlen(message), 1, log_opts.fp);
/* if we are using OVERWRITE_MESSAGE, add newline */
if (strchr(message, '\r') != NULL)
! fwrite("\n", 1, 1, log_opts.fp);
! fflush(log_opts.fp);
}
switch (type)
*************** pg_log(eLogType type, char *fmt,...)
*** 104,114 ****
exit(1);
break;
- case PG_DEBUG:
- if (log_opts.debug)
- fprintf(log_opts.debug_fd, "%s\n", _(message));
- break;
-
default:
break;
}
--- 104,109 ----
diff --git a/doc/src/sgml/pgupgrade.sgml b/doc/src/sgml/pgupgrade.sgml
new file mode 100644
index 4f263fe..84b01a8
*** a/doc/src/sgml/pgupgrade.sgml
--- b/doc/src/sgml/pgupgrade.sgml
***************
*** 91,120 ****
</varlistentry>
<varlistentry>
- <term><option>-g</option></term>
- <term><option>--debug</option></term>
- <listitem><para>enable debugging</para></listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-G</option> <replaceable>debug_filename</></term>
- <term><option>--debugfile=</option><replaceable>debug_filename</></term>
- <listitem><para>output debugging activity to file</para></listitem>
- </varlistentry>
-
- <varlistentry>
<term><option>-k</option></term>
<term><option>--link</option></term>
<listitem><para>use hard links instead of copying files to the new cluster</para></listitem>
</varlistentry>
<varlistentry>
- <term><option>-l</option> <replaceable>log_filename</></term>
- <term><option>--logfile=</option><replaceable>log_filename</></term>
- <listitem><para>log internal activity to file</para></listitem>
- </varlistentry>
-
- <varlistentry>
<term><option>-o</option> <replaceable class="parameter">options</replaceable></term>
<term><option>--old-options</option> <replaceable class="parameter">options</replaceable></term>
<listitem><para>options to be passed directly to the
--- 91,102 ----
***************
*** 143,148 ****
--- 125,137 ----
</varlistentry>
<varlistentry>
+ <term><option>-r</option></term>
+ <term><option>--retain</option></term>
+ <listitem><para>retain SQL and log files even after successful completion
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-u</option> <replaceable>user_name</></term>
<term><option>--user=</option><replaceable>user_name</></term>
<listitem><para>cluster's super user name; environment
***************
*** 152,158 ****
<varlistentry>
<term><option>-v</option></term>
<term><option>--verbose</option></term>
! <listitem><para>enable verbose output</para></listitem>
</varlistentry>
<varlistentry>
--- 141,148 ----
<varlistentry>
<term><option>-v</option></term>
<term><option>--verbose</option></term>
! <listitem><para>enable verbose internal logging, also enables
! <option>-r</option>/<option>--retain</option></para></listitem>
</varlistentry>
<varlistentry>
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
