On Wed, Dec 15, 2021 at 10:47:24AM +0900, Michael Paquier wrote: > Missed that, thanks! I'll think about all that a bit more before > sending a long-overdue rebased version.
Okay, here is finally a rebase of this patch, where I have fixed a couple of existing issues, and I have extended the patch to the point where the support range is what I expect should be. In short: - Added support for MSVC for the TAP test. I have considered making upgradecheck silent, but after thinking about it I have just filtered pg_upgrade from bincheck, and simplified upgradecheck to launch the new test. It is simple to switch from one approach to another. This shaves some code in vcregress.pl. - Fixed a set of issues with various chdir commands in the previous patches. The command of pg_regress has been tweaked so as all results are part of src/bin/pg_upgrade/. Any logs generated by pg_upgrade stay in this location, same way as HEAD. - Adapted to the new modules of src/test/perl/. - Support for cross-upgrades now uses upgrade_adapt.sql (I have sent a patch for the buildfarm client about that yesterday actually), same way as test.sh on HEAD. Like HEAD, attempting to use the cross-version HEAD causes diffs between the old and the new dumps. But there is nothing new here. This could be improved more but the attached does already a lot. - Like the previous versions, this supports two modes when setting up the to-be-upgraded cluster: setup things from an old dump or use pg_regress. The buildfarm does the former for upgrades down to 9.2. The core code does the latter. I may have missed one thing or two, but I think that's pretty much what we should be looking for to do the switch to TAP in terms of coverage. -- Michael
From 47fcfac534f947595a880eacdbcfe4f8bd6af516 Mon Sep 17 00:00:00 2001 From: Michael Paquier <mich...@paquier.xyz> Date: Thu, 16 Dec 2021 11:50:54 +0900 Subject: [PATCH v4] Switch tests of pg_upgrade to use TAP --- src/bin/pg_upgrade/.gitignore | 5 + src/bin/pg_upgrade/Makefile | 23 +- src/bin/pg_upgrade/TESTING | 33 ++- src/bin/pg_upgrade/t/001_basic.pl | 9 + src/bin/pg_upgrade/t/002_pg_upgrade.pl | 275 ++++++++++++++++++++++ src/bin/pg_upgrade/test.sh | 279 ----------------------- src/test/perl/PostgreSQL/Test/Cluster.pm | 25 ++ src/tools/msvc/vcregress.pl | 91 +------- 8 files changed, 355 insertions(+), 385 deletions(-) create mode 100644 src/bin/pg_upgrade/t/001_basic.pl create mode 100644 src/bin/pg_upgrade/t/002_pg_upgrade.pl delete mode 100644 src/bin/pg_upgrade/test.sh diff --git a/src/bin/pg_upgrade/.gitignore b/src/bin/pg_upgrade/.gitignore index 2d3bfeaa50..3b64522ab6 100644 --- a/src/bin/pg_upgrade/.gitignore +++ b/src/bin/pg_upgrade/.gitignore @@ -7,3 +7,8 @@ /loadable_libraries.txt /log/ /tmp_check/ + +# Generated by pg_regress +/sql/ +/expected/ +/results/ diff --git a/src/bin/pg_upgrade/Makefile b/src/bin/pg_upgrade/Makefile index 44d06be5a6..35b6c123a5 100644 --- a/src/bin/pg_upgrade/Makefile +++ b/src/bin/pg_upgrade/Makefile @@ -28,6 +28,12 @@ OBJS = \ override CPPFLAGS := -DDLSUFFIX=\"$(DLSUFFIX)\" -I$(srcdir) -I$(libpq_srcdir) $(CPPFLAGS) LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport) +# required for 002_pg_upgrade.pl +REGRESS_SHLIB=$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX) +export REGRESS_SHLIB +REGRESS_OUTPUTDIR=$(abs_top_builddir)/src/bin/pg_upgrade +export REGRESS_OUTPUTDIR + all: pg_upgrade pg_upgrade: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils @@ -49,17 +55,8 @@ clean distclean maintainer-clean: pg_upgrade_dump_globals.sql \ pg_upgrade_dump_*.custom pg_upgrade_*.log -# When $(MAKE) is present, make automatically infers that this is a -# recursive make. which is not actually what we want here, as that -# e.g. prevents output synchronization from working (as make thinks -# that the subsidiary make knows how to deal with that itself, but -# we're invoking a shell script that doesn't know). Referencing -# $(MAKE) indirectly avoids that behaviour. -# See https://www.gnu.org/software/make/manual/html_node/MAKE-Variable.html#MAKE-Variable -NOTSUBMAKEMAKE=$(MAKE) +check: + $(prove_check) -check: test.sh all temp-install - MAKE=$(NOTSUBMAKEMAKE) $(with_temp_install) bindir=$(abs_top_builddir)/tmp_install/$(bindir) EXTRA_REGRESS_OPTS="$(EXTRA_REGRESS_OPTS)" $(SHELL) $< - -# installcheck is not supported because there's no meaningful way to test -# pg_upgrade against a single already-running server +installcheck: + $(prove_installcheck) diff --git a/src/bin/pg_upgrade/TESTING b/src/bin/pg_upgrade/TESTING index e69874b42d..acf769386a 100644 --- a/src/bin/pg_upgrade/TESTING +++ b/src/bin/pg_upgrade/TESTING @@ -2,21 +2,30 @@ THE SHORT VERSION ----------------- On non-Windows machines, you can execute the testing process -described below by running +described below by running the following command in this directory: make check -in this directory. This will run the shell script test.sh, performing -an upgrade from the version in this source tree to a new instance of -the same version. -To test an upgrade from a different version, you must have a built -source tree for the old version as well as this version, and you -must have done "make install" for both versions. Then do: +This will run the TAP tests to run pg_upgrade, performing an upgrade +from the version in this source tree to a new instance of the same +version. + +To test an upgrade from a different version, there are two options +available: + +1) You have a built source tree for the old version as well as this +version's binaries. Then set up the following variables before +launching the test: export oldsrc=...somewhere/postgresql (old version's source tree) -export oldbindir=...otherversion/bin (old version's installed bin dir) -export bindir=...thisversion/bin (this version's installed bin dir) -export libdir=...thisversion/lib (this version's installed lib dir) -sh test.sh +export oldinstall=...otherversion/ (old version's install base path) + +2) You have a dump that can be used to set up the old version, as well +as this version's binaries. Then set up the following variables: +export olddump=...somewhere/dump.sql (old version's dump) +export oldinstall=...otherversion/ (old version's install base path) + +Finally, the tests can be done by running + make check In this case, you will have to manually eyeball the resulting dump diff for version-specific differences, as explained below. @@ -87,3 +96,5 @@ steps: 7) Diff the regression database dump file with the regression dump file loaded into the old server. + +The generated dump may be reusable with "olddump", as defined above. diff --git a/src/bin/pg_upgrade/t/001_basic.pl b/src/bin/pg_upgrade/t/001_basic.pl new file mode 100644 index 0000000000..75b0f98b08 --- /dev/null +++ b/src/bin/pg_upgrade/t/001_basic.pl @@ -0,0 +1,9 @@ +use strict; +use warnings; + +use PostgreSQL::Test::Utils; +use Test::More tests => 8; + +program_help_ok('pg_upgrade'); +program_version_ok('pg_upgrade'); +program_options_handling_ok('pg_upgrade'); diff --git a/src/bin/pg_upgrade/t/002_pg_upgrade.pl b/src/bin/pg_upgrade/t/002_pg_upgrade.pl new file mode 100644 index 0000000000..721741d0f8 --- /dev/null +++ b/src/bin/pg_upgrade/t/002_pg_upgrade.pl @@ -0,0 +1,275 @@ +# Set of tests for pg_upgrade, including cross-version checks. +use strict; +use warnings; + +use Cwd qw(abs_path getcwd); +use File::Basename qw(dirname); + +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More tests => 4; + +# Generate a database with a name made of a range of ASCII characters. +sub generate_db +{ + my ($node, $from_char, $to_char) = @_; + + my $dbname = ''; + for my $i ($from_char .. $to_char) + { + next if $i == 7 || $i == 10 || $i == 13; # skip BEL, LF, and CR + $dbname = $dbname . sprintf('%c', $i); + } + $node->run_log([ 'createdb', '--port', $node->port, $dbname ]); +} + +# From now on, the test of pg_upgrade consists in setting up an instance. +# This is the source instance used for the upgrade. Then a new and fresh +# instance is created, and is used as the target instance for the +# upgrade. Before running an upgrade a logical dump of the old instance +# is taken, and a second logical dump of the new instance is taken after +# the upgrade. The upgrade test passes if there are no differences after +# running pg_upgrade. + +# Testing upgrades with an older instance of PostgreSQL requires +# setting up two environment variables, among the following: +# - "oldsrc", to point to the code source of the older version. +# This is required to set up the old instance with pg_upgrade. +# - "olddump", to point to a dump file that will be used to set +# up the old instance to upgrade from. +# - "oldinstall", to point to the installation path of the older +# version. + +# "oldsrc" and "olddump" cannot be used together. Setting up +# "olddump" and "oldinstall" will use the dump pointed to to +# set up the old instance. If "oldsrc" is used instead of "olddump", +# the full set of regression tests of the old instance is run +# instead. + +if (defined($ENV{oldsrc}) && defined($ENV{olddump})) +{ + die "oldsrc and olddump are both defined"; +} +elsif (defined($ENV{oldsrc})) +{ + if ( (defined($ENV{oldsrc}) && !defined($ENV{oldinstall})) + || (!defined($ENV{oldsrc}) && defined($ENV{oldinstall}))) + { + # Not all variables are defined, so leave and die if test is + # done with an older installation. + die "oldsrc or oldinstall is undefined"; + } +} +elsif (defined($ENV{olddump})) +{ + if ( (defined($ENV{olddump}) && !defined($ENV{oldinstall})) + || (!defined($ENV{olddump}) && defined($ENV{oldinstall}))) + { + # Not all variables are defined, so leave and die if test is + # done with an older installation. + die "olddump or oldinstall is undefined"; + } +} + +if ((defined($ENV{oldsrc}) || defined($ENV{olddump})) && $windows_os) +{ + # This configuration is not supported on Windows, as regress.so + # location diverges across the compilation methods used on this + # platform. + die "No support for older version tests on Windows"; +} + +# Default is the location of this source code for both nodes used with +# the upgrade. +my $newsrc = abs_path("../../.."); +my $oldsrc = $ENV{oldsrc} || $newsrc; +$oldsrc = abs_path($oldsrc); + +# Temporary location for the dumps taken +my $tempdir = PostgreSQL::Test::Utils::tempdir; + +# Initialize node to upgrade +my $oldnode = PostgreSQL::Test::Cluster->new('old_node', + install_path => $ENV{oldinstall}); + +$oldnode->init(extra => [ '--locale', 'C', '--encoding', 'LATIN1' ]); +$oldnode->start; + +# Set up the data of the old instance with pg_regress or an old dump. +if (defined($ENV{olddump})) +{ + # Use the dump specified. + my $olddumpfile = $ENV{olddump}; + die "no dump file found!" unless -e $olddumpfile; + + # Load the dump, and we are done here. + $oldnode->command_ok( + [ + 'psql', '-X', '-f', $olddumpfile, + '--port', $oldnode->port, 'regression' + ]); +} +else +{ + # Default is to just use pg_regress to setup the old instance + # Creating databases with names covering most ASCII bytes + generate_db($oldnode, 1, 45); + generate_db($oldnode, 46, 90); + generate_db($oldnode, 91, 127); + + # Run core regression tests on the old instance. + $oldnode->run_log([ "createdb", '--port', $oldnode->port, 'regression' ]); + + # Grab any regression options that may be passed down by caller. + my $extra_opts_val = $ENV{EXTRA_REGRESS_OPT} || ""; + my @extra_opts = split(/\s+/, $extra_opts_val); + + # --dlpath is needed to be able to find the location of regress.so and + # any libraries the regression tests required. This needs to point to + # in the old instance when using it. In the default case, fallback to + # what the caller provided for REGRESS_SHLIB. + my $dlpath; + if (defined($ENV{oldinstall})) + { + $dlpath = "$oldsrc/src/test/regress"; + } + else + { + $dlpath = dirname($ENV{REGRESS_SHLIB}); + } + $dlpath = PostgreSQL::Test::Utils::perl2host($dlpath); + + # --outputdir points to the path where to place the output files, which + # had better be this directory. + my $outputdir = + PostgreSQL::Test::Utils::perl2host($ENV{REGRESS_OUTPUTDIR}); + + # --inputdir needs to point to the location of the input files, from the + # cluster to-be-upgraded. + my $inputdir = + PostgreSQL::Test::Utils::perl2host("$oldsrc/src/test/regress"); + + my @regress_command = [ + $ENV{PG_REGRESS}, '--make-testtablespace-dir', + '--schedule', "$oldsrc/src/test/regress/parallel_schedule", + '--bindir', $oldnode->config_data('--bindir'), + '--dlpath', $dlpath, + '--port', $oldnode->port, + '--outputdir', $outputdir, + '--inputdir', $inputdir, + '--use-existing' + ]; + @regress_command = (@regress_command, @extra_opts); + + $oldnode->command_ok(@regress_command, + 'regression test run on old instance'); +} + +# Before dumping, get rid of objects not existing or not supported in later +# versions. This depends on the version of the old server used, and matters +# only if different versions are used for the dump. +if (defined($ENV{oldinstall})) +{ + # Note that upgrade_adapt.sql from the new version is used, to + # cope with an upgrade to this version. + $oldnode->run_log( + [ + 'psql', '-X', '-f', + PostgreSQL::Test::Utils::perl2host( + "$newsrc/src/bin/pg_upgrade/upgrade_adapt.sql"), + '--port', + $oldnode->port, + 'regression' + ]); +} + +# Initialize a new node for the upgrade. This is done early so as it is +# possible to know with which node's PATH the initial dump needs to be +# taken. +my $newnode = PostgreSQL::Test::Cluster->new('new_node'); +$newnode->init(extra => [ '--locale=C', '--encoding=LATIN1' ]); +my $newbindir = $newnode->config_data('--bindir'); +my $oldbindir = $oldnode->config_data('--bindir'); + +# Take a dump before performing the upgrade as a base comparison. Note +# that we need to use pg_dumpall from the new node here. +$newnode->command_ok( + [ + 'pg_dumpall', '--no-sync', + '-d', $oldnode->connstr('postgres'), + '-f', "$tempdir/dump1.sql" + ], + 'dump before running pg_upgrade'); + +# After dumping, update references to the old source tree's regress.so +# to point to the new tree. +if (defined($ENV{oldinstall})) +{ + # First, fetch all the references to libraries that are not part + # of the default path $libdir. + my $output = $oldnode->safe_psql('regression', + "SELECT DISTINCT probin::text FROM pg_proc WHERE probin NOT LIKE '\$libdir%';" + ); + chomp($output); + my @libpaths = split("\n", $output); + + my $dump_data = slurp_file("$tempdir/dump1.sql"); + + my $newregresssrc = "$newsrc/src/test/regress"; + foreach (@libpaths) + { + my $libpath = $_; + $libpath = dirname($libpath); + $dump_data =~ s/$libpath/$newregresssrc/g; + } + + open my $fh, ">", "$tempdir/dump1.sql" or die "could not open dump file"; + print $fh $dump_data; + close $fh; + + # This replaces any references to the old tree's regress.so + # the new tree's regress.so. Any references that do *not* + # match $libdir are switched so as this request does not + # depend on the path of the old source tree. This is useful + # when using an old dump. Do the operation on all the database + # that allow connections so as this includes the regression + # database and anything the user has set up. + $output = $oldnode->safe_psql('postgres', + "SELECT datname FROM pg_database WHERE datallowconn;"); + chomp($output); + my @datnames = split("\n", $output); + foreach (@datnames) + { + my $datname = $_; + $oldnode->safe_psql( + $datname, "UPDATE pg_proc SET probin = + regexp_replace(probin, '.*/', '$newregresssrc/') + WHERE probin NOT LIKE '\$libdir/%'"); + } +} + +# Update the instance. +$oldnode->stop; + +# Time for the real run. +$newnode->command_ok( + [ + 'pg_upgrade', '-d', $oldnode->data_dir, '-D', + $newnode->data_dir, '-b', $oldbindir, '-B', + $newbindir, '-p', $oldnode->port, '-P', + $newnode->port + ], + 'run of pg_upgrade for new instance'); +$newnode->start; + +# Take a second dump on the upgraded instance. +$newnode->run_log( + [ + 'pg_dumpall', '--no-sync', + '-d', $newnode->connstr('postgres'), + '-f', "$tempdir/dump2.sql" + ]); + +# Compare the two dumps, there should be no differences. +command_ok([ 'diff', '-q', "$tempdir/dump1.sql", "$tempdir/dump2.sql" ], + 'old and new dump match after pg_upgrade'); diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh deleted file mode 100644 index 32d186d897..0000000000 --- a/src/bin/pg_upgrade/test.sh +++ /dev/null @@ -1,279 +0,0 @@ -#!/bin/sh - -# src/bin/pg_upgrade/test.sh -# -# Test driver for pg_upgrade. Initializes a new database cluster, -# runs the regression tests (to put in some data), runs pg_dumpall, -# runs pg_upgrade, runs pg_dumpall again, compares the dumps. -# -# Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group -# Portions Copyright (c) 1994, Regents of the University of California - -set -e - -: ${MAKE=make} - -# Guard against parallel make issues (see comments in pg_regress.c) -unset MAKEFLAGS -unset MAKELEVEL - -# Run a given "initdb" binary and overlay the regression testing -# authentication configuration. -standard_initdb() { - # To increase coverage of non-standard segment size and group access - # without increasing test runtime, run these tests with a custom setting. - # Also, specify "-A trust" explicitly to suppress initdb's warning. - # --allow-group-access and --wal-segsize have been added in v11. - "$1" -N --wal-segsize 1 --allow-group-access -A trust - if [ -n "$TEMP_CONFIG" -a -r "$TEMP_CONFIG" ] - then - cat "$TEMP_CONFIG" >> "$PGDATA/postgresql.conf" - fi - ../../test/regress/pg_regress --config-auth "$PGDATA" -} - -# What flavor of host are we on? -# Treat MINGW* (msys1) and MSYS* (msys2) the same. -testhost=`uname -s | sed 's/^MSYS/MINGW/'` - -# Establish how the server will listen for connections -case $testhost in - MINGW*) - LISTEN_ADDRESSES="localhost" - PG_REGRESS_SOCKET_DIR="" - PGHOST=localhost - ;; - *) - LISTEN_ADDRESSES="" - # Select a socket directory. The algorithm is from the "configure" - # script; the outcome mimics pg_regress.c:make_temp_sockdir(). - if [ x"$PG_REGRESS_SOCKET_DIR" = x ]; then - set +e - dir=`(umask 077 && - mktemp -d /tmp/pg_upgrade_check-XXXXXX) 2>/dev/null` - if [ ! -d "$dir" ]; then - dir=/tmp/pg_upgrade_check-$$-$RANDOM - (umask 077 && mkdir "$dir") - if [ ! -d "$dir" ]; then - echo "could not create socket temporary directory in \"/tmp\"" - exit 1 - fi - fi - set -e - PG_REGRESS_SOCKET_DIR=$dir - trap 'rm -rf "$PG_REGRESS_SOCKET_DIR"' 0 - trap 'exit 3' 1 2 13 15 - fi - PGHOST=$PG_REGRESS_SOCKET_DIR - ;; -esac - -POSTMASTER_OPTS="-F -c listen_addresses=\"$LISTEN_ADDRESSES\" -k \"$PG_REGRESS_SOCKET_DIR\"" -export PGHOST - -# don't rely on $PWD here, as old shells don't set it -temp_root=`pwd`/tmp_check -rm -rf "$temp_root" -mkdir "$temp_root" - -: ${oldbindir=$bindir} - -: ${oldsrc=../../..} -oldsrc=`cd "$oldsrc" && pwd` -newsrc=`cd ../../.. && pwd` - -# We need to make pg_regress use psql from the desired installation -# (likely a temporary one), because otherwise the installcheck run -# below would try to use psql from the proper installation directory -# of the target version, which might be outdated or not exist. But -# don't override anything else that's already in EXTRA_REGRESS_OPTS. -EXTRA_REGRESS_OPTS="$EXTRA_REGRESS_OPTS --bindir='$oldbindir'" -export EXTRA_REGRESS_OPTS - -# While in normal cases this will already be set up, adding bindir to -# path allows test.sh to be invoked with different versions as -# described in ./TESTING -PATH=$bindir:$PATH -export PATH - -BASE_PGDATA="$temp_root/data" -PGDATA="${BASE_PGDATA}.old" -export PGDATA - -# Send installcheck outputs to a private directory. This avoids conflict when -# check-world runs pg_upgrade check concurrently with src/test/regress check. -# To retrieve interesting files after a run, use pattern tmp_check/*/*.diffs. -outputdir="$temp_root/regress" -EXTRA_REGRESS_OPTS="$EXTRA_REGRESS_OPTS --outputdir=$outputdir" -export EXTRA_REGRESS_OPTS -mkdir "$outputdir" - -# pg_regress --make-tablespacedir would take care of that in 14~, but this is -# still required for older versions where this option is not supported. -if [ "$newsrc" != "$oldsrc" ]; then - mkdir "$outputdir"/testtablespace - mkdir "$outputdir"/sql - mkdir "$outputdir"/expected -fi - -logdir=`pwd`/log -rm -rf "$logdir" -mkdir "$logdir" - -# Clear out any environment vars that might cause libpq to connect to -# the wrong postmaster (cf pg_regress.c) -# -# Some shells, such as NetBSD's, return non-zero from unset if the variable -# is already unset. Since we are operating under 'set -e', this causes the -# script to fail. To guard against this, set them all to an empty string first. -PGDATABASE=""; unset PGDATABASE -PGUSER=""; unset PGUSER -PGSERVICE=""; unset PGSERVICE -PGSSLMODE=""; unset PGSSLMODE -PGREQUIRESSL=""; unset PGREQUIRESSL -PGCONNECT_TIMEOUT=""; unset PGCONNECT_TIMEOUT -PGHOSTADDR=""; unset PGHOSTADDR - -# Select a non-conflicting port number, similarly to pg_regress.c -PG_VERSION_NUM=`grep '#define PG_VERSION_NUM' "$newsrc"/src/include/pg_config.h | awk '{print $3}'` -PGPORT=`expr $PG_VERSION_NUM % 16384 + 49152` -export PGPORT - -i=0 -while psql -X postgres </dev/null 2>/dev/null -do - i=`expr $i + 1` - if [ $i -eq 16 ] - then - echo port $PGPORT apparently in use - exit 1 - fi - PGPORT=`expr $PGPORT + 1` - export PGPORT -done - -# buildfarm may try to override port via EXTRA_REGRESS_OPTS ... -EXTRA_REGRESS_OPTS="$EXTRA_REGRESS_OPTS --port=$PGPORT" -export EXTRA_REGRESS_OPTS - -standard_initdb "$oldbindir"/initdb -"$oldbindir"/pg_ctl start -l "$logdir/postmaster1.log" -o "$POSTMASTER_OPTS" -w - -# Create databases with names covering the ASCII bytes other than NUL, BEL, -# LF, or CR. BEL would ring the terminal bell in the course of this test, and -# it is not otherwise a special case. PostgreSQL doesn't support the rest. -dbname1=`awk 'BEGIN { for (i= 1; i < 46; i++) - if (i != 7 && i != 10 && i != 13) printf "%c", i }' </dev/null` -# Exercise backslashes adjacent to double quotes, a Windows special case. -dbname1='\"\'$dbname1'\\"\\\' -dbname2=`awk 'BEGIN { for (i = 46; i < 91; i++) printf "%c", i }' </dev/null` -dbname3=`awk 'BEGIN { for (i = 91; i < 128; i++) printf "%c", i }' </dev/null` -createdb "regression$dbname1" || createdb_status=$? -createdb "regression$dbname2" || createdb_status=$? -createdb "regression$dbname3" || createdb_status=$? - -# Extra options to apply to the dump. This may be changed later. -extra_dump_options="" - -if "$MAKE" -C "$oldsrc" installcheck-parallel; then - oldpgversion=`psql -X -A -t -d regression -c "SHOW server_version_num"` - - # Before dumping, tweak the database of the old instance depending - # on its version. - if [ "$newsrc" != "$oldsrc" ]; then - # This SQL script has its own idea of the cleanup that needs to be - # done on the cluster to-be-upgraded, and includes version checks. - # Note that this uses the script stored on the new branch. - psql -X -d regression -f "$newsrc/src/bin/pg_upgrade/upgrade_adapt.sql" \ - || psql_fix_sql_status=$? - - # Handling of --extra-float-digits gets messy after v12. - # Note that this changes the dumps from the old and new - # instances if involving an old cluster of v11 or older. - if [ $oldpgversion -lt 120000 ]; then - extra_dump_options="--extra-float-digits=0" - fi - fi - - pg_dumpall $extra_dump_options --no-sync \ - -f "$temp_root"/dump1.sql || pg_dumpall1_status=$? - - if [ "$newsrc" != "$oldsrc" ]; then - # update references to old source tree's regress.so etc - fix_sql="" - case $oldpgversion in - *) - fix_sql="UPDATE pg_proc SET probin = replace(probin, '$oldsrc', '$newsrc') WHERE probin LIKE '$oldsrc%';" - ;; - esac - psql -X -d regression -c "$fix_sql;" || psql_fix_sql_status=$? - - mv "$temp_root"/dump1.sql "$temp_root"/dump1.sql.orig - sed "s;$oldsrc;$newsrc;g" "$temp_root"/dump1.sql.orig >"$temp_root"/dump1.sql - fi -else - make_installcheck_status=$? -fi -"$oldbindir"/pg_ctl -m fast stop -if [ -n "$createdb_status" ]; then - exit 1 -fi -if [ -n "$make_installcheck_status" ]; then - exit 1 -fi -if [ -n "$psql_fix_sql_status" ]; then - exit 1 -fi -if [ -n "$pg_dumpall1_status" ]; then - echo "pg_dumpall of pre-upgrade database cluster failed" - exit 1 -fi - -PGDATA="$BASE_PGDATA" - -standard_initdb 'initdb' - -pg_upgrade $PG_UPGRADE_OPTS -d "${PGDATA}.old" -D "$PGDATA" -b "$oldbindir" -p "$PGPORT" -P "$PGPORT" - -# make sure all directories and files have group permissions, on Unix hosts -# Windows hosts don't support Unix-y permissions. -case $testhost in - MINGW*|CYGWIN*) ;; - *) if [ `find "$PGDATA" -type f ! -perm 640 | wc -l` -ne 0 ]; then - echo "files in PGDATA with permission != 640"; - exit 1; - fi ;; -esac - -case $testhost in - MINGW*|CYGWIN*) ;; - *) if [ `find "$PGDATA" -type d ! -perm 750 | wc -l` -ne 0 ]; then - echo "directories in PGDATA with permission != 750"; - exit 1; - fi ;; -esac - -pg_ctl start -l "$logdir/postmaster2.log" -o "$POSTMASTER_OPTS" -w - -pg_dumpall $extra_dump_options --no-sync \ - -f "$temp_root"/dump2.sql || pg_dumpall2_status=$? -pg_ctl -m fast stop - -if [ -n "$pg_dumpall2_status" ]; then - echo "pg_dumpall of post-upgrade database cluster failed" - exit 1 -fi - -case $testhost in - MINGW*) MSYS2_ARG_CONV_EXCL=/c cmd /c delete_old_cluster.bat ;; - *) sh ./delete_old_cluster.sh ;; -esac - -if diff "$temp_root"/dump1.sql "$temp_root"/dump2.sql >/dev/null; then - echo PASSED - exit 0 -else - echo "Files $temp_root/dump1.sql and $temp_root/dump2.sql differ" - echo "dumps were not identical" - exit 1 -fi diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index c061e850fb..2bf2fcb69d 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -327,6 +327,31 @@ sub install_path =pod +=item $node->config_data($option) + +Grab some data from pg_config, with $option being the switched +used. + +=cut + +sub config_data +{ + my ($self, $option) = @_; + local %ENV = $self->_get_env(); + + my ($stdout, $stderr); + my $result = + IPC::Run::run [ $self->installed_command('pg_config'), $option ], + '>', \$stdout, '2>', \$stderr + or die "could not execute pg_config"; + chomp($stdout); + $stdout =~ s/\r$//; + + return $stdout; +} + +=pod + =item $node->info() Return a string containing human-readable diagnostic information (paths, etc) diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl index dc5b2df7d7..d62a15883e 100644 --- a/src/tools/msvc/vcregress.pl +++ b/src/tools/msvc/vcregress.pl @@ -287,6 +287,10 @@ sub bincheck foreach my $dir (@bin_dirs) { next unless -d "$dir/t"; + # Do not consider pg_upgrade, as it is handled by + # upgradecheck. + next if ($dir =~ "/pg_upgrade/"); + my $status = tap_check($dir); $mstat ||= $status; } @@ -591,89 +595,12 @@ sub generate_db sub upgradecheck { - my $status; - my $cwd = getcwd(); + InstallTemp(); + # Tweak environment for the upgrade tests + $ENV{REGRESS_OUTPUTDIR} = "$topdir/src/bin/pg_upgrade"; - # Much of this comes from the pg_upgrade test.sh script, - # but it only covers the --install case, and not the case - # where the old and new source or bin dirs are different. - # i.e. only this version to this version check. That's - # what pg_upgrade's "make check" does. - - $ENV{PGHOST} = 'localhost'; - $ENV{PGPORT} ||= 50432; - my $tmp_root = "$topdir/src/bin/pg_upgrade/tmp_check"; - rmtree($tmp_root); - mkdir $tmp_root || die $!; - my $upg_tmp_install = "$tmp_root/install"; # unshared temp install - print "Setting up temp install\n\n"; - Install($upg_tmp_install, "all", $config); - - # Install does a chdir, so change back after that - chdir $cwd; - my ($bindir, $libdir, $oldsrc, $newsrc) = - ("$upg_tmp_install/bin", "$upg_tmp_install/lib", $topdir, $topdir); - $ENV{PATH} = "$bindir;$ENV{PATH}"; - my $data = "$tmp_root/data"; - $ENV{PGDATA} = "$data.old"; - my $outputdir = "$tmp_root/regress"; - my @EXTRA_REGRESS_OPTS = ("--outputdir=$outputdir"); - mkdir "$outputdir" || die $!; - - my $logdir = "$topdir/src/bin/pg_upgrade/log"; - rmtree($logdir); - mkdir $logdir || die $!; - print "\nRunning initdb on old cluster\n\n"; - standard_initdb() or exit 1; - print "\nStarting old cluster\n\n"; - my @args = ('pg_ctl', 'start', '-l', "$logdir/postmaster1.log"); - system(@args) == 0 or exit 1; - - print "\nCreating databases with names covering most ASCII bytes\n\n"; - generate_db("\\\"\\", 1, 45, "\\\\\"\\\\\\"); - generate_db('', 46, 90, ''); - generate_db('', 91, 127, ''); - - print "\nSetting up data for upgrading\n\n"; - installcheck_internal('parallel', @EXTRA_REGRESS_OPTS); - - # now we can chdir into the source dir - chdir "$topdir/src/bin/pg_upgrade"; - print "\nDumping old cluster\n\n"; - @args = ('pg_dumpall', '-f', "$tmp_root/dump1.sql"); - system(@args) == 0 or exit 1; - print "\nStopping old cluster\n\n"; - system("pg_ctl stop") == 0 or exit 1; - $ENV{PGDATA} = "$data"; - print "\nSetting up new cluster\n\n"; - standard_initdb() or exit 1; - print "\nRunning pg_upgrade\n\n"; - @args = ('pg_upgrade', '-d', "$data.old", '-D', $data, '-b', $bindir); - system(@args) == 0 or exit 1; - print "\nStarting new cluster\n\n"; - @args = ('pg_ctl', '-l', "$logdir/postmaster2.log", 'start'); - system(@args) == 0 or exit 1; - print "\nDumping new cluster\n\n"; - @args = ('pg_dumpall', '-f', "$tmp_root/dump2.sql"); - system(@args) == 0 or exit 1; - print "\nStopping new cluster\n\n"; - system("pg_ctl stop") == 0 or exit 1; - print "\nDeleting old cluster\n\n"; - system(".\\delete_old_cluster.bat") == 0 or exit 1; - print "\nComparing old and new cluster dumps\n\n"; - - @args = ('diff', '-q', "$tmp_root/dump1.sql", "$tmp_root/dump2.sql"); - system(@args); - $status = $?; - if (!$status) - { - print "PASSED\n"; - } - else - { - print "dumps not identical!\n"; - exit(1); - } + my $mstat = tap_check("$topdir/src/bin/pg_upgrade"); + exit $mstat if $mstat; return; } -- 2.34.1
signature.asc
Description: PGP signature
