On Thu, Mar 08, 2012 at 10:19:05AM -0500, Tom Lane wrote:
> Bruce Momjian <[email protected]> writes:
> > On Thu, Mar 08, 2012 at 08:34:53AM -0500, A.M. wrote:
> >> It looks like the patch will overwrite the logs in the current working
> >> directory, for example, if pg_upgrade is run twice in the same place. Is
> >> that intentional? I had imagined that the logs would have been dumped in
>
> > Yes. I was afraid that continually appending to a log file on every run
> > would be too confusing. I could do only appends, or number the log
> > files, that those seemed confusing.
>
> Use one (set of) files, and always append, but at the beginning of each
> run print "\npg_upgrade starting at [timestamp]\n\n". Should make it
> reasonably clear, while not losing information.
OK, it seems people do care about keeping log files from multiple runs
so I went with Tom's idea and have:
-----------------------------------------------------------------
pg_upgrade run on Thu Mar 8 19:30:12 2012
-----------------------------------------------------------------
Performing Consistency Checks
-----------------------------
Updated patch attached.
FYI, in retain mode, these are the files left in the current directory:
delete_old_cluster.sh
pg_upgrade_dump_all.sql
pg_upgrade_dump_db.sql
pg_upgrade_dump_globals.sql
pg_upgrade_internal.log
pg_upgrade_restore.log
pg_upgrade_server.log
pg_upgrade_utility.log
I will address the idea of using /tmp in another email.
--
Bruce Momjian <[email protected]> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
diff --git a/contrib/pg_upgrade/check.c b/contrib/pg_upgrade/check.c
new file mode 100644
index a5f63eb..7905534
*** a/contrib/pg_upgrade/check.c
--- b/contrib/pg_upgrade/check.c
*************** issue_warnings(char *sequence_script_fil
*** 165,176 ****
if (sequence_script_file_name)
{
prep_status("Adjusting sequences");
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user,
! sequence_script_file_name, log_opts.filename2);
unlink(sequence_script_file_name);
check_ok();
}
--- 165,177 ----
if (sequence_script_file_name)
{
prep_status("Adjusting sequences");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user,
! sequence_script_file_name, UTILITY_LOG_FILE);
unlink(sequence_script_file_name);
check_ok();
}
diff --git a/contrib/pg_upgrade/controldata.c b/contrib/pg_upgrade/controldata.c
new file mode 100644
index 5239601..e01280d
*** a/contrib/pg_upgrade/controldata.c
--- b/contrib/pg_upgrade/controldata.c
*************** get_control_data(ClusterInfo *cluster, b
*** 126,136 ****
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
! if (log_opts.debug)
! fputs(bufin, log_opts.debug_fd);
#ifdef WIN32
-
/*
* Due to an installer bug, LANG=C doesn't work for PG 8.3.3, but does
* work 8.2.6 and 8.3.7, so check for non-ASCII output and suggest a
--- 126,134 ----
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
! pg_log(PG_VERBOSE, "%s", bufin);
#ifdef WIN32
/*
* Due to an installer bug, LANG=C doesn't work for PG 8.3.3, but does
* work 8.2.6 and 8.3.7, so check for non-ASCII output and suggest a
diff --git a/contrib/pg_upgrade/dump.c b/contrib/pg_upgrade/dump.c
new file mode 100644
index 772ca37..7ec94ad
*** a/contrib/pg_upgrade/dump.c
--- b/contrib/pg_upgrade/dump.c
*************** generate_old_dump(void)
*** 22,31 ****
* --binary-upgrade records the width of dropped columns in pg_class, and
* restores the frozenid's for databases and relations.
*/
! exec_prog(true,
SYSTEMQUOTE "\"%s/pg_dumpall\" --port %d --username \"%s\" "
! "--schema-only --binary-upgrade > \"%s/" ALL_DUMP_FILE "\""
! SYSTEMQUOTE, new_cluster.bindir, old_cluster.port, os_info.user, os_info.cwd);
check_ok();
}
--- 22,32 ----
* --binary-upgrade records the width of dropped columns in pg_class, and
* restores the frozenid's for databases and relations.
*/
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/pg_dumpall\" --port %d --username \"%s\" "
! "--schema-only --binary-upgrade > \"%s/%s\" 2>> \"%s\""
! SYSTEMQUOTE, new_cluster.bindir, old_cluster.port, os_info.user,
! os_info.cwd, ALL_DUMP_FILE, UTILITY_LOG_FILE);
check_ok();
}
diff --git a/contrib/pg_upgrade/exec.c b/contrib/pg_upgrade/exec.c
new file mode 100644
index b870ded..571ad23
*** a/contrib/pg_upgrade/exec.c
--- b/contrib/pg_upgrade/exec.c
*************** static int win32_check_directory_write_p
*** 34,40 ****
* instead of returning should an error occur.
*/
int
! exec_prog(bool throw_error, const char *fmt,...)
{
va_list args;
int result;
--- 34,40 ----
* instead of returning should an error occur.
*/
int
! exec_prog(bool throw_error, const char *log_file, const char *fmt,...)
{
va_list args;
int result;
*************** exec_prog(bool throw_error, const char *
*** 44,57 ****
vsnprintf(cmd, MAXPGPATH, fmt, args);
va_end(args);
! pg_log(PG_INFO, "%s\n", cmd);
result = system(cmd);
if (result != 0)
{
! pg_log(throw_error ? PG_FATAL : PG_INFO,
! "There were problems executing \"%s\"\n", cmd);
return 1;
}
--- 44,60 ----
vsnprintf(cmd, MAXPGPATH, fmt, args);
va_end(args);
! pg_log(PG_VERBOSE, "%s\n", cmd);
result = system(cmd);
if (result != 0)
{
! pg_log(PG_REPORT, "There were problems executing \"%s\"\n", cmd);
! pg_log(throw_error ? PG_FATAL : PG_REPORT,
! "Consult the last few lines of \"%s\" for\n"
! "the probable cause of the failure.\n",
! log_file);
return 1;
}
diff --git a/contrib/pg_upgrade/info.c b/contrib/pg_upgrade/info.c
new file mode 100644
index 692cdc2..36683fa
*** a/contrib/pg_upgrade/info.c
--- b/contrib/pg_upgrade/info.c
*************** create_rel_filename_map(const char *old_
*** 132,150 ****
void
print_maps(FileNameMap *maps, int n_maps, const char *db_name)
{
! if (log_opts.debug)
{
int mapnum;
! pg_log(PG_DEBUG, "mappings for database \"%s\":\n", db_name);
for (mapnum = 0; mapnum < n_maps; mapnum++)
! pg_log(PG_DEBUG, "%s.%s: %u to %u\n",
maps[mapnum].nspname, maps[mapnum].relname,
maps[mapnum].old_relfilenode,
maps[mapnum].new_relfilenode);
! pg_log(PG_DEBUG, "\n\n");
}
}
--- 132,150 ----
void
print_maps(FileNameMap *maps, int n_maps, const char *db_name)
{
! if (log_opts.verbose)
{
int mapnum;
! pg_log(PG_VERBOSE, "mappings for database \"%s\":\n", db_name);
for (mapnum = 0; mapnum < n_maps; mapnum++)
! pg_log(PG_VERBOSE, "%s.%s: %u to %u\n",
maps[mapnum].nspname, maps[mapnum].relname,
maps[mapnum].old_relfilenode,
maps[mapnum].new_relfilenode);
! pg_log(PG_VERBOSE, "\n\n");
}
}
*************** get_db_and_rel_infos(ClusterInfo *cluste
*** 168,178 ****
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
get_rel_infos(cluster, &cluster->dbarr.dbs[dbnum]);
! if (log_opts.debug)
! {
! pg_log(PG_DEBUG, "\n%s databases:\n", CLUSTER_NAME(cluster));
print_db_infos(&cluster->dbarr);
- }
}
--- 168,176 ----
for (dbnum = 0; dbnum < cluster->dbarr.ndbs; dbnum++)
get_rel_infos(cluster, &cluster->dbarr.dbs[dbnum]);
! pg_log(PG_VERBOSE, "\n%s databases:\n", CLUSTER_NAME(cluster));
! if (log_opts.verbose)
print_db_infos(&cluster->dbarr);
}
*************** print_db_infos(DbInfoArr *db_arr)
*** 368,376 ****
for (dbnum = 0; dbnum < db_arr->ndbs; dbnum++)
{
! pg_log(PG_DEBUG, "Database: %s\n", db_arr->dbs[dbnum].db_name);
print_rel_infos(&db_arr->dbs[dbnum].rel_arr);
! pg_log(PG_DEBUG, "\n\n");
}
}
--- 366,374 ----
for (dbnum = 0; dbnum < db_arr->ndbs; dbnum++)
{
! pg_log(PG_VERBOSE, "Database: %s\n", db_arr->dbs[dbnum].db_name);
print_rel_infos(&db_arr->dbs[dbnum].rel_arr);
! pg_log(PG_VERBOSE, "\n\n");
}
}
*************** print_rel_infos(RelInfoArr *arr)
*** 381,387 ****
int relnum;
for (relnum = 0; relnum < arr->nrels; relnum++)
! pg_log(PG_DEBUG, "relname: %s.%s: reloid: %u reltblspace: %s\n",
arr->rels[relnum].nspname, arr->rels[relnum].relname,
arr->rels[relnum].reloid, arr->rels[relnum].tablespace);
}
--- 379,385 ----
int relnum;
for (relnum = 0; relnum < arr->nrels; relnum++)
! pg_log(PG_VERBOSE, "relname: %s.%s: reloid: %u reltblspace: %s\n",
arr->rels[relnum].nspname, arr->rels[relnum].relname,
arr->rels[relnum].reloid, arr->rels[relnum].tablespace);
}
diff --git a/contrib/pg_upgrade/option.c b/contrib/pg_upgrade/option.c
new file mode 100644
index 0a105ef..e365ee1
*** a/contrib/pg_upgrade/option.c
--- b/contrib/pg_upgrade/option.c
***************
*** 11,18 ****
#include "pg_upgrade.h"
! #include "getopt_long.h"
!
#ifdef WIN32
#include <io.h>
#endif
--- 11,20 ----
#include "pg_upgrade.h"
! #include <getopt_long.h>
! #include <time.h>
! #include <sys/types.h>
! #include <sys/stat.h>
#ifdef WIN32
#include <io.h>
#endif
*************** parseCommandLine(int argc, char *argv[])
*** 46,63 ****
{"user", required_argument, NULL, 'u'},
{"check", no_argument, NULL, 'c'},
- {"debug", no_argument, NULL, 'g'},
- {"debugfile", required_argument, NULL, 'G'},
{"link", no_argument, NULL, 'k'},
! {"logfile", required_argument, NULL, 'l'},
{"verbose", no_argument, NULL, 'v'},
{NULL, 0, NULL, 0}
};
int option; /* Command line option */
int optindex = 0; /* used by getopt_long */
int os_user_effective_id;
char *return_buf;
!
user_opts.transfer_mode = TRANSFER_MODE_COPY;
os_info.progname = get_progname(argv[0]);
--- 48,66 ----
{"user", required_argument, NULL, 'u'},
{"check", no_argument, NULL, 'c'},
{"link", no_argument, NULL, 'k'},
! {"retain", no_argument, NULL, 'r'},
{"verbose", no_argument, NULL, 'v'},
{NULL, 0, NULL, 0}
};
int option; /* Command line option */
int optindex = 0; /* used by getopt_long */
int os_user_effective_id;
+ FILE *fp;
char *return_buf;
! int i;
! time_t run_time = time(NULL);
!
user_opts.transfer_mode = TRANSFER_MODE_COPY;
os_info.progname = get_progname(argv[0]);
*************** parseCommandLine(int argc, char *argv[])
*** 90,95 ****
--- 93,100 ----
}
}
+ umask(0077);
+
/* Allow help and version to be run as root, so do the test here. */
if (os_user_effective_id == 0)
pg_log(PG_FATAL, "%s: cannot be run as root\n", os_info.progname);
*************** parseCommandLine(int argc, char *argv[])
*** 98,104 ****
if (return_buf == NULL)
pg_log(PG_FATAL, "Could not access current working directory: %s\n", getErrorText(errno));
! while ((option = getopt_long(argc, argv, "d:D:b:B:cgG:kl:o:O:p:P:u:v",
long_options, &optindex)) != -1)
{
switch (option)
--- 103,112 ----
if (return_buf == NULL)
pg_log(PG_FATAL, "Could not access current working directory: %s\n", getErrorText(errno));
! if ((log_opts.fp = fopen(INTERNAL_LOG_FILE, "a")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", INTERNAL_LOG_FILE);
!
! while ((option = getopt_long(argc, argv, "d:D:b:B:cko:O:p:P:ru:v",
long_options, &optindex)) != -1)
{
switch (option)
*************** parseCommandLine(int argc, char *argv[])
*** 125,151 ****
new_cluster.pgconfig = pg_strdup(optarg);
break;
- case 'g':
- pg_log(PG_REPORT, "Running in debug mode\n");
- log_opts.debug = true;
- break;
-
- case 'G':
- if ((log_opts.debug_fd = fopen(optarg, "w")) == NULL)
- {
- pg_log(PG_FATAL, "cannot open debug file\n");
- exit(1);
- }
- break;
-
case 'k':
user_opts.transfer_mode = TRANSFER_MODE_LINK;
break;
- case 'l':
- log_opts.filename = pg_strdup(optarg);
- break;
-
case 'o':
old_cluster.pgopts = pg_strdup(optarg);
break;
--- 133,142 ----
*************** parseCommandLine(int argc, char *argv[])
*** 175,180 ****
--- 166,175 ----
}
break;
+ case 'r':
+ log_opts.retain = true;
+ break;
+
case 'u':
pg_free(os_info.user);
os_info.user = pg_strdup(optarg);
*************** parseCommandLine(int argc, char *argv[])
*** 199,234 ****
}
}
! if (log_opts.filename != NULL)
! {
! /*
! * We must use append mode so output generated by child processes via
! * ">>" will not be overwritten, and we want the file truncated on
! * start.
! */
! /* truncate */
! if ((log_opts.fd = fopen(log_opts.filename, "w")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", log_opts.filename);
! fclose(log_opts.fd);
! if ((log_opts.fd = fopen(log_opts.filename, "a")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n", log_opts.filename);
! }
! else
! log_opts.filename = pg_strdup(DEVNULL);
!
! /* WIN32 files do not accept writes from multiple processes */
! #ifndef WIN32
! log_opts.filename2 = pg_strdup(log_opts.filename);
! #else
! log_opts.filename2 = pg_strdup(DEVNULL);
! #endif
!
! /* if no debug file name, output to the terminal */
! if (log_opts.debug && !log_opts.debug_fd)
{
! log_opts.debug_fd = fopen(DEVTTY, "w");
! if (!log_opts.debug_fd)
! pg_log(PG_FATAL, "cannot write to terminal\n");
}
/* Get values from env if not already set */
--- 194,211 ----
}
}
! /* label start of upgrade in logfiles */
! for (i = 0; i < NUM_LOG_FILES; i++)
{
! if ((fp = fopen(output_files[i], "a")) == NULL)
! pg_log(PG_FATAL, "cannot write to log file %s\n",
! output_files[i]);
! fprintf(fp, "\n"
! "-----------------------------------------------------------------\n"
! " pg_upgrade run on %s"
! "-----------------------------------------------------------------\n\n",
! ctime(&run_time));
! fclose(fp);
}
/* Get values from env if not already set */
*************** Options:\n\
*** 256,271 ****
-c, --check check clusters only, don't change any data\n\
-d, --old-datadir=OLDDATADIR old cluster data directory\n\
-D, --new-datadir=NEWDATADIR new cluster data directory\n\
- -g, --debug enable debugging\n\
- -G, --debugfile=FILENAME output debugging activity to file\n\
-k, --link link instead of copying files to new cluster\n\
- -l, --logfile=FILENAME log internal activity to file\n\
-o, --old-options=OPTIONS old cluster options to pass to the server\n\
-O, --new-options=OPTIONS new cluster options to pass to the server\n\
-p, --old-port=OLDPORT old cluster port number (default %d)\n\
-P, --new-port=NEWPORT new cluster port number (default %d)\n\
-u, --user=NAME cluster superuser (default \"%s\")\n\
! -v, --verbose enable verbose output\n\
-V, --version display version information, then exit\n\
-h, --help show this help, then exit\n\
\n\
--- 233,246 ----
-c, --check check clusters only, don't change any data\n\
-d, --old-datadir=OLDDATADIR old cluster data directory\n\
-D, --new-datadir=NEWDATADIR new cluster data directory\n\
-k, --link link instead of copying files to new cluster\n\
-o, --old-options=OPTIONS old cluster options to pass to the server\n\
-O, --new-options=OPTIONS new cluster options to pass to the server\n\
-p, --old-port=OLDPORT old cluster port number (default %d)\n\
-P, --new-port=NEWPORT new cluster port number (default %d)\n\
+ -r, --retain retain SQL and log files after success\n\
-u, --user=NAME cluster superuser (default \"%s\")\n\
! -v, --verbose enable verbose internal logging\n\
-V, --version display version information, then exit\n\
-h, --help show this help, then exit\n\
\n\
*************** adjust_data_dir(ClusterInfo *cluster)
*** 354,372 ****
{
char filename[MAXPGPATH];
char cmd[MAXPGPATH], cmd_output[MAX_STRING];
! FILE *fd, *output;
/* If there is no postgresql.conf, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/postgresql.conf", cluster->pgconfig);
! if ((fd = fopen(filename, "r")) == NULL)
return;
! fclose(fd);
/* If PG_VERSION exists, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/PG_VERSION", cluster->pgconfig);
! if ((fd = fopen(filename, "r")) != NULL)
{
! fclose(fd);
return;
}
--- 329,347 ----
{
char filename[MAXPGPATH];
char cmd[MAXPGPATH], cmd_output[MAX_STRING];
! FILE *fp, *output;
/* If there is no postgresql.conf, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/postgresql.conf", cluster->pgconfig);
! if ((fp = fopen(filename, "r")) == NULL)
return;
! fclose(fp);
/* If PG_VERSION exists, it can't be a config-only dir */
snprintf(filename, sizeof(filename), "%s/PG_VERSION", cluster->pgconfig);
! if ((fp = fopen(filename, "r")) != NULL)
{
! fclose(fp);
return;
}
diff --git a/contrib/pg_upgrade/pg_upgrade.c b/contrib/pg_upgrade/pg_upgrade.c
new file mode 100644
index 3078bcd..a4511bb
*** a/contrib/pg_upgrade/pg_upgrade.c
--- b/contrib/pg_upgrade/pg_upgrade.c
*************** ClusterInfo old_cluster,
*** 55,60 ****
--- 55,68 ----
new_cluster;
OSInfo os_info;
+ char *output_files[NUM_LOG_FILES] = {
+ SERVER_LOG_FILE,
+ RESTORE_LOG_FILE,
+ UTILITY_LOG_FILE,
+ INTERNAL_LOG_FILE
+ };
+
+
int
main(int argc, char **argv)
{
*************** main(int argc, char **argv)
*** 127,135 ****
* because there is no need to have the schema load use new oids.
*/
prep_status("Setting next OID for new cluster");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -o %u \"%s\" > "
! DEVNULL SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid, new_cluster.pgdata);
check_ok();
create_script_for_old_cluster_deletion(&deletion_script_file_name);
--- 135,145 ----
* because there is no need to have the schema load use new oids.
*/
prep_status("Setting next OID for new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE "\"%s/pg_resetxlog\" -o %u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtoid,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
create_script_for_old_cluster_deletion(&deletion_script_file_name);
*************** prepare_new_cluster(void)
*** 193,202 ****
* --analyze so autovacuum doesn't update statistics later
*/
prep_status("Analyzing all rows in the new cluster");
! exec_prog(true,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --analyze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, log_opts.filename2);
check_ok();
/*
--- 203,212 ----
* --analyze so autovacuum doesn't update statistics later
*/
prep_status("Analyzing all rows in the new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --analyze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, UTILITY_LOG_FILE);
check_ok();
/*
*************** prepare_new_cluster(void)
*** 206,215 ****
* later.
*/
prep_status("Freezing all rows on the new cluster");
! exec_prog(true,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --freeze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, log_opts.filename2);
check_ok();
get_pg_database_relfilenode(&new_cluster);
--- 216,225 ----
* later.
*/
prep_status("Freezing all rows on the new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
SYSTEMQUOTE "\"%s/vacuumdb\" --port %d --username \"%s\" "
"--all --freeze >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, new_cluster.port, os_info.user, UTILITY_LOG_FILE);
check_ok();
get_pg_database_relfilenode(&new_cluster);
*************** prepare_new_databases(void)
*** 243,255 ****
* support functions in template1 but pg_dumpall creates database using
* the template0 template.
*/
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
! /* --no-psqlrc prevents AUTOCOMMIT=off */
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! GLOBALS_DUMP_FILE, log_opts.filename2);
check_ok();
/* we load this to get a current list of databases */
--- 253,266 ----
* support functions in template1 but pg_dumpall creates database using
* the template0 template.
*/
! exec_prog(true, RESTORE_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
! /* --no-psqlrc prevents AUTOCOMMIT=off */
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! GLOBALS_DUMP_FILE, RESTORE_LOG_FILE);
check_ok();
/* we load this to get a current list of databases */
*************** create_new_objects(void)
*** 275,286 ****
check_ok();
prep_status("Restoring database schema to new cluster");
! exec_prog(true,
! SYSTEMQUOTE "\"%s/psql\" --set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\"" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! DB_DUMP_FILE, log_opts.filename2);
check_ok();
/* regenerate now that we have objects in the databases */
--- 286,298 ----
check_ok();
prep_status("Restoring database schema to new cluster");
! exec_prog(true, RESTORE_LOG_FILE,
! SYSTEMQUOTE "\"%s/psql\" --echo-queries "
! "--set ON_ERROR_STOP=on "
"--no-psqlrc --port %d --username \"%s\" "
! "-f \"%s/%s\" --dbname template1 >> \"%s\" 2>&1" SYSTEMQUOTE,
new_cluster.bindir, new_cluster.port, os_info.user, os_info.cwd,
! DB_DUMP_FILE, RESTORE_LOG_FILE);
check_ok();
/* regenerate now that we have objects in the databases */
*************** copy_clog_xlog_xid(void)
*** 306,334 ****
check_ok();
prep_status("Copying old commit clogs to new server");
#ifndef WIN32
! exec_prog(true, SYSTEMQUOTE "%s \"%s\" \"%s\"" SYSTEMQUOTE,
"cp -Rf",
#else
/* flags: everything, no confirm, quiet, overwrite read-only */
! exec_prog(true, SYSTEMQUOTE "%s \"%s\" \"%s\\\"" SYSTEMQUOTE,
"xcopy /e /y /q /r",
#endif
! old_clog_path, new_clog_path);
check_ok();
/* set the next transaction id of the new cluster */
prep_status("Setting next transaction ID for new cluster");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -f -x %u \"%s\" > " DEVNULL SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_nxtxid, new_cluster.pgdata);
check_ok();
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
! exec_prog(true, SYSTEMQUOTE "\"%s/pg_resetxlog\" -l %u,%u,%u \"%s\" >> \"%s\" 2>&1" SYSTEMQUOTE,
! new_cluster.bindir, old_cluster.controldata.chkpnt_tli,
! old_cluster.controldata.logid, old_cluster.controldata.nxtlogseg,
! new_cluster.pgdata, log_opts.filename2);
check_ok();
}
--- 318,355 ----
check_ok();
prep_status("Copying old commit clogs to new server");
+ exec_prog(true, UTILITY_LOG_FILE,
#ifndef WIN32
! SYSTEMQUOTE "%s \"%s\" \"%s\" >> \"%s\" 2>&1" SYSTEMQUOTE,
"cp -Rf",
#else
/* flags: everything, no confirm, quiet, overwrite read-only */
! SYSTEMQUOTE "%s \"%s\" \"%s\\\" >> \"%s\" 2>&1" SYSTEMQUOTE,
"xcopy /e /y /q /r",
#endif
! old_clog_path, new_clog_path, UTILITY_LOG_FILE);
check_ok();
/* set the next transaction id of the new cluster */
prep_status("Setting next transaction ID for new cluster");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE
! "\"%s/pg_resetxlog\" -f -x %u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE, new_cluster.bindir,
! old_cluster.controldata.chkpnt_nxtxid,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
/* now reset the wal archives in the new cluster */
prep_status("Resetting WAL archives");
! exec_prog(true, UTILITY_LOG_FILE,
! SYSTEMQUOTE
! "\"%s/pg_resetxlog\" -l %u,%u,%u \"%s\" >> \"%s\" 2>&1"
! SYSTEMQUOTE, new_cluster.bindir,
! old_cluster.controldata.chkpnt_tli,
! old_cluster.controldata.logid,
! old_cluster.controldata.nxtlogseg,
! new_cluster.pgdata, UTILITY_LOG_FILE);
check_ok();
}
*************** set_frozenxids(void)
*** 421,438 ****
static void
cleanup(void)
{
! char filename[MAXPGPATH];
! if (log_opts.fd)
! fclose(log_opts.fd);
! if (log_opts.debug_fd)
! fclose(log_opts.debug_fd);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, ALL_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, GLOBALS_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd, DB_DUMP_FILE);
! unlink(filename);
}
--- 442,472 ----
static void
cleanup(void)
{
!
! fclose(log_opts.fp);
! /* Remove dump and log files? */
! if (!log_opts.retain)
! {
! char filename[MAXPGPATH];
! int i;
! for (i = 0; i < NUM_LOG_FILES; i++)
! {
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd,
! output_files[i]);
! unlink(filename);
! }
! /* remove SQL files */
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd,
! ALL_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd,
! GLOBALS_DUMP_FILE);
! unlink(filename);
! snprintf(filename, sizeof(filename), "%s/%s", os_info.cwd,
! DB_DUMP_FILE);
! unlink(filename);
! }
}
diff --git a/contrib/pg_upgrade/pg_upgrade.h b/contrib/pg_upgrade/pg_upgrade.h
new file mode 100644
index a954815..cf7d78f
*** a/contrib/pg_upgrade/pg_upgrade.h
--- b/contrib/pg_upgrade/pg_upgrade.h
***************
*** 35,40 ****
--- 35,68 ----
#define GLOBALS_DUMP_FILE "pg_upgrade_dump_globals.sql"
#define DB_DUMP_FILE "pg_upgrade_dump_db.sql"
+ #define SERVER_LOG_FILE "pg_upgrade_server.log"
+ #define RESTORE_LOG_FILE "pg_upgrade_restore.log"
+ #define UTILITY_LOG_FILE "pg_upgrade_utility.log"
+ #define INTERNAL_LOG_FILE "pg_upgrade_internal.log"
+
+ #define NUM_LOG_FILES 4
+ extern char *output_files[];
+
+ /*
+ * WIN32 files do not accept writes from multiple processes
+ *
+ * On Win32, we can't send both pg_upgrade output and command output to the
+ * same file because we get the error: "The process cannot access the file
+ * because it is being used by another process." so send the pg_ctl
+ * command-line output to the utility log file on Windows, rather than
+ * into the server log file.
+ *
+ * We could use the Windows pgwin32_open() flags to allow shared file
+ * writes but is unclear how all other tools would use those flags, so
+ * we just avoid it and log a little differently on Windows; we adjust
+ * the error message appropriately.
+ */
+ #ifndef WIN32
+ #define SERVER_LOG_FILE2 SERVER_LOG_FILE
+ #else
+ #define SERVER_LOG_FILE2 UTILITY_LOG_FILE
+ #endif
+
#ifndef WIN32
#define pg_copy_file copy_file
#define pg_mv_file rename
*************** typedef enum
*** 166,176 ****
*/
typedef enum
{
! PG_INFO,
PG_REPORT,
PG_WARNING,
! PG_FATAL,
! PG_DEBUG
} eLogType;
--- 194,203 ----
*/
typedef enum
{
! PG_VERBOSE,
PG_REPORT,
PG_WARNING,
! PG_FATAL
} eLogType;
*************** typedef struct
*** 204,228 ****
*/
typedef struct
{
! char *filename; /* name of log file (may be /dev/null) */
! /*
! * WIN32 files do not accept writes from multiple processes
! *
! * On Win32, we can't send both pg_upgrade output and command output to the
! * same file because we get the error: "The process cannot access the file
! * because it is being used by another process." so we have to send all
! * other output to 'nul'. Therefore, we set this to DEVNULL on Win32, and
! * it equals 'filename' on all other platforms.
! *
! * We could use the Windows pgwin32_open() flags to allow shared file
! * writes but is unclear how all other tools would use those flags, so
! * we just avoid it and log a little less on Windows.
! */
! char *filename2;
! FILE *fd; /* log FILE */
! bool debug; /* TRUE -> log more information */
! FILE *debug_fd; /* debug-level log FILE */
bool verbose; /* TRUE -> be verbose in messages */
} LogOpts;
--- 231,239 ----
*/
typedef struct
{
! FILE *fp; /* log FILE */
bool verbose; /* TRUE -> be verbose in messages */
+ bool retain; /* retain log files on success */
} LogOpts;
*************** void split_old_dump(void);
*** 294,301 ****
/* exec.c */
! int exec_prog(bool throw_error, const char *cmd, ...)
! __attribute__((format(PG_PRINTF_ATTRIBUTE, 2, 3)));
void verify_directories(void);
bool is_server_running(const char *datadir);
--- 305,312 ----
/* exec.c */
! int exec_prog(bool throw_error, const char *log_file, const char *cmd, ...)
! __attribute__((format(PG_PRINTF_ATTRIBUTE, 3, 4)));
void verify_directories(void);
bool is_server_running(const char *datadir);
diff --git a/contrib/pg_upgrade/relfilenode.c b/contrib/pg_upgrade/relfilenode.c
new file mode 100644
index a1e30b1..45d6c54
*** a/contrib/pg_upgrade/relfilenode.c
--- b/contrib/pg_upgrade/relfilenode.c
*************** transfer_relfile(pageCnvCtx *pageConvert
*** 267,273 ****
if (user_opts.transfer_mode == TRANSFER_MODE_COPY)
{
! pg_log(PG_INFO, "copying \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = copyAndUpdateFile(pageConverter, old_file, new_file, true)) != NULL)
pg_log(PG_FATAL, "error while copying relation \"%s.%s\" (\"%s\" to \"%s\"): %s\n",
--- 267,273 ----
if (user_opts.transfer_mode == TRANSFER_MODE_COPY)
{
! pg_log(PG_VERBOSE, "copying \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = copyAndUpdateFile(pageConverter, old_file, new_file, true)) != NULL)
pg_log(PG_FATAL, "error while copying relation \"%s.%s\" (\"%s\" to \"%s\"): %s\n",
*************** transfer_relfile(pageCnvCtx *pageConvert
*** 275,281 ****
}
else
{
! pg_log(PG_INFO, "linking \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = linkAndUpdateFile(pageConverter, old_file, new_file)) != NULL)
pg_log(PG_FATAL,
--- 275,281 ----
}
else
{
! pg_log(PG_VERBOSE, "linking \"%s\" to \"%s\"\n", old_file, new_file);
if ((msg = linkAndUpdateFile(pageConverter, old_file, new_file)) != NULL)
pg_log(PG_FATAL,
diff --git a/contrib/pg_upgrade/server.c b/contrib/pg_upgrade/server.c
new file mode 100644
index 989af63..f8d7326
*** a/contrib/pg_upgrade/server.c
--- b/contrib/pg_upgrade/server.c
*************** executeQueryOrDie(PGconn *conn, const ch
*** 80,86 ****
vsnprintf(command, sizeof(command), fmt, args);
va_end(args);
! pg_log(PG_DEBUG, "executing: %s\n", command);
result = PQexec(conn, command);
status = PQresultStatus(result);
--- 80,86 ----
vsnprintf(command, sizeof(command), fmt, args);
va_end(args);
! pg_log(PG_VERBOSE, "executing: %s\n", command);
result = PQexec(conn, command);
status = PQresultStatus(result);
*************** start_postmaster(ClusterInfo *cluster)
*** 161,177 ****
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" "
"-o \"-p %d %s %s\" start >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, log_opts.filename2, cluster->pgconfig, cluster->port,
(cluster->controldata.cat_ver >=
BINARY_UPGRADE_SERVER_FLAG_CAT_VER) ? "-b" :
"-c autovacuum=off -c autovacuum_freeze_max_age=2000000000",
! cluster->pgopts ? cluster->pgopts : "", log_opts.filename2);
/*
* Don't throw an error right away, let connecting throw the error because
* it might supply a reason for the failure.
*/
! pg_ctl_return = exec_prog(false, "%s", cmd);
/* Check to see if we can connect to the server; if not, report it. */
if ((conn = get_db_conn(cluster, "template1")) == NULL ||
--- 161,182 ----
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" "
"-o \"-p %d %s %s\" start >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, SERVER_LOG_FILE, cluster->pgconfig, cluster->port,
(cluster->controldata.cat_ver >=
BINARY_UPGRADE_SERVER_FLAG_CAT_VER) ? "-b" :
"-c autovacuum=off -c autovacuum_freeze_max_age=2000000000",
! cluster->pgopts ? cluster->pgopts : "", SERVER_LOG_FILE2);
/*
* Don't throw an error right away, let connecting throw the error because
* it might supply a reason for the failure.
*/
! pg_ctl_return = exec_prog(false,
! /* pass both file names if the differ */
! (strcmp(SERVER_LOG_FILE, SERVER_LOG_FILE2) == 0) ?
! SERVER_LOG_FILE :
! SERVER_LOG_FILE " or " SERVER_LOG_FILE2,
! "%s", cmd);
/* Check to see if we can connect to the server; if not, report it. */
if ((conn = get_db_conn(cluster, "template1")) == NULL ||
*************** stop_postmaster(bool fast)
*** 211,221 ****
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" -o \"%s\" "
"%s stop >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, log_opts.filename2, cluster->pgconfig,
cluster->pgopts ? cluster->pgopts : "",
! fast ? "-m fast" : "", log_opts.filename2);
! exec_prog(fast ? false : true, "%s", cmd);
os_info.running_cluster = NULL;
}
--- 216,226 ----
snprintf(cmd, sizeof(cmd),
SYSTEMQUOTE "\"%s/pg_ctl\" -w -l \"%s\" -D \"%s\" -o \"%s\" "
"%s stop >> \"%s\" 2>&1" SYSTEMQUOTE,
! cluster->bindir, SERVER_LOG_FILE2, cluster->pgconfig,
cluster->pgopts ? cluster->pgopts : "",
! fast ? "-m fast" : "", SERVER_LOG_FILE2);
! exec_prog(fast ? false : true, SERVER_LOG_FILE2, "%s", cmd);
os_info.running_cluster = NULL;
}
diff --git a/contrib/pg_upgrade/util.c b/contrib/pg_upgrade/util.c
new file mode 100644
index 94eaa18..b039269
*** a/contrib/pg_upgrade/util.c
--- b/contrib/pg_upgrade/util.c
*************** pg_log(eLogType type, char *fmt,...)
*** 77,94 ****
vsnprintf(message, sizeof(message), fmt, args);
va_end(args);
! if (log_opts.fd != NULL)
{
! fwrite(message, strlen(message), 1, log_opts.fd);
/* if we are using OVERWRITE_MESSAGE, add newline */
if (strchr(message, '\r') != NULL)
! fwrite("\n", 1, 1, log_opts.fd);
! fflush(log_opts.fd);
}
switch (type)
{
! case PG_INFO:
if (log_opts.verbose)
printf("%s", _(message));
break;
--- 77,95 ----
vsnprintf(message, sizeof(message), fmt, args);
va_end(args);
! /* PG_VERBOSE is only output in verbose mode */
! if (type != PG_VERBOSE || log_opts.verbose)
{
! fwrite(message, strlen(message), 1, log_opts.fp);
/* if we are using OVERWRITE_MESSAGE, add newline */
if (strchr(message, '\r') != NULL)
! fwrite("\n", 1, 1, log_opts.fp);
! fflush(log_opts.fp);
}
switch (type)
{
! case PG_VERBOSE:
if (log_opts.verbose)
printf("%s", _(message));
break;
*************** pg_log(eLogType type, char *fmt,...)
*** 104,114 ****
exit(1);
break;
- case PG_DEBUG:
- if (log_opts.debug)
- fprintf(log_opts.debug_fd, "%s\n", _(message));
- break;
-
default:
break;
}
--- 105,110 ----
diff --git a/doc/src/sgml/pgupgrade.sgml b/doc/src/sgml/pgupgrade.sgml
new file mode 100644
index 4f263fe..6ecdfda
*** a/doc/src/sgml/pgupgrade.sgml
--- b/doc/src/sgml/pgupgrade.sgml
***************
*** 91,120 ****
</varlistentry>
<varlistentry>
- <term><option>-g</option></term>
- <term><option>--debug</option></term>
- <listitem><para>enable debugging</para></listitem>
- </varlistentry>
-
- <varlistentry>
- <term><option>-G</option> <replaceable>debug_filename</></term>
- <term><option>--debugfile=</option><replaceable>debug_filename</></term>
- <listitem><para>output debugging activity to file</para></listitem>
- </varlistentry>
-
- <varlistentry>
<term><option>-k</option></term>
<term><option>--link</option></term>
<listitem><para>use hard links instead of copying files to the new cluster</para></listitem>
</varlistentry>
<varlistentry>
- <term><option>-l</option> <replaceable>log_filename</></term>
- <term><option>--logfile=</option><replaceable>log_filename</></term>
- <listitem><para>log internal activity to file</para></listitem>
- </varlistentry>
-
- <varlistentry>
<term><option>-o</option> <replaceable class="parameter">options</replaceable></term>
<term><option>--old-options</option> <replaceable class="parameter">options</replaceable></term>
<listitem><para>options to be passed directly to the
--- 91,102 ----
***************
*** 143,148 ****
--- 125,137 ----
</varlistentry>
<varlistentry>
+ <term><option>-r</option></term>
+ <term><option>--retain</option></term>
+ <listitem><para>retain SQL and log files even after successful completion
+ </para></listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><option>-u</option> <replaceable>user_name</></term>
<term><option>--user=</option><replaceable>user_name</></term>
<listitem><para>cluster's super user name; environment
***************
*** 152,158 ****
<varlistentry>
<term><option>-v</option></term>
<term><option>--verbose</option></term>
! <listitem><para>enable verbose output</para></listitem>
</varlistentry>
<varlistentry>
--- 141,147 ----
<varlistentry>
<term><option>-v</option></term>
<term><option>--verbose</option></term>
! <listitem><para>enable verbose internal logging</para></listitem>
</varlistentry>
<varlistentry>
--
Sent via pgsql-hackers mailing list ([email protected])
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
