On Tue, Jul 26, 2022 at 04:24:25PM +1200, Thomas Munro wrote: > 3. You can't really run PostgreSQL on Cygwin for real, because its > implementation of signals does not have reliable signal masking, so > unsubtle and probably also subtle breakage occurs. That was reported > upstream by Noah years ago, but they aren't working on a fix. > lorikeet shows random failures, and presumably any CI system will do > the same...
Reference: https://www.postgresql.org/message-id/20170321034703.GB2097809%40tornado.leadboat.com On my 2nd try: https://cirrus-ci.com/task/5311911574110208 TRAP: FailedAssertion("mq->mq_sender == NULL", File: "shm_mq.c", Line: 230, PID: 16370) 2022-07-26 06:32:35.525 PDT [15538][postmaster] LOG: background worker "parallel worker" (PID 16370) was terminated by signal 6: Aborted > XXX Doesn't get all the way through yet... Mainly because getopt was causing all tap tests to fail. I tried to fix that in configure, but ended up changing the callers. This is getting close, but I don't think has actually managed to pass all tests yet.. https://cirrus-ci.com/task/5274721116749824 > 4. When building with Cygwin GCC 11.3 you get a bunch of warnings > that don't show up on other platforms, seemingly indicating that it > interprets -Wimplicit-fallthrough=3 differently. Huh? Evidently due to the same getopt issues. > XXX This should use a canned Docker image with all the right packages > installed Has anyone tried using non-canned images ? It sounds like this could reduce the 4min startup time for windows. https://cirrus-ci.org/guide/docker-builder-vm/#dockerfile-as-a-ci-environment > XXX configure is soooo slooow, can we cache it?! Compiling is also > insanely slow, but ccache gets it down to a couple of minutes if you're > lucky One reason compiling was slow is because you ended up with -O2. You can cache configure as long as you're willing to re-run it whenever options were changed. That also applies to the existing headerscheck. > XXX I don't know how to put variables like BUILD_JOBS into the scripts WDYM ? If it's outside of bash and in windows shell it's like %var%, right ? https://cirrus-ci.org/guide/writing-tasks/#environment-variables I just noticed that cirrus is misbehaving: if there's a variable called CI (which there is), then it expands $CI_FOO like ${CI}_FOO rather than ${CI_FOO}. I've also seen weirdness when variable names or operators appear in the commit message... > XXX Needs some --with-X options Done > XXX We would never want this to run by default in CI, but it'd be nice > to be able to ask for it with ci-os-only! (See commented out line) > only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*cygwin.*' Doesn't this already do what's needed? As long as it doesn't also check: CHANGE_MESSAGE !~ 'ci-os-only', the task will runs only on request. > XXX I have no idea if crash dump works, and if this should share > elements with the msys work in commitfest #3575 Based on the crash above, it wasn't working. And after some changes ... it still doesn't work. windows_os is probably skipping too many things. -- Justin
>From 174cc603ff951b86794f57fcc8c7d326d948e006 Mon Sep 17 00:00:00 2001 From: Thomas Munro <thomas.mu...@gmail.com> Date: Mon, 25 Jul 2022 23:05:10 +1200 Subject: [PATCH] WIP CI support for Cygwin. ci-os-only: cygwin, xindows, xinux https://cirrus-ci.com/task/5145086722834432 XXX This should use a canned Docker image with all the right packages installed XXX I have no idea if crash dump works, and if this should share elements with the msys work in commitfest #3575 --- .cirrus.yml | 59 +++++++++++++++++++ configure | 18 +++--- configure.ac | 6 +- src/interfaces/libpq/t/001_uri.pl | 11 +++- src/interfaces/libpq/t/002_api.pl | 13 +++- .../libpq_pipeline/t/001_libpq_pipeline.pl | 14 ++++- src/test/perl/PostgreSQL/Test/Cluster.pm | 2 +- src/test/perl/PostgreSQL/Test/Utils.pm | 12 +++- src/tools/ci/cores_backtrace.sh | 6 +- 9 files changed, 120 insertions(+), 21 deletions(-) diff --git a/.cirrus.yml b/.cirrus.yml index f23d6cae552..0389b7758d1 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -456,6 +456,65 @@ task: path: "crashlog-*.txt" type: text/plain +task: + name: Windows - Cygwin + + env: + CPUS: 4 + BUILD_JOBS: 4 + TEST_JOBS: 3 + CCACHE_DIR: /tmp/ccache + CCACHE_LOGFILE: ccache.log + CONFIGURE_FLAGS: --enable-cassert --enable-debug --enable-tap-tests --with-ldap --with-ssl=openssl --with-gssapi + CONFIGURE_CACHE: /tmp/ccache/configure.cache + PG_TEST_USE_UNIX_SOCKETS: 1 + + only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*cygwin.*' + + windows_container: + image: cirrusci/windowsservercore:2019 + os_version: 2019 + cpu: $CPUS + memory: 4G + + ccache_cache: + folder: C:\tools\cygwin\tmp\ccache + + setup_additional_packages_script: | + choco install --cache C:\tools\cygwin\choco -y --no-progress cygwin + C:\tools\cygwin\cygwinsetup.exe -q -P cygrunsrv,make,gcc-core,ccache,binutils,libtool,pkg-config,flex,bison,zlib-devel,libssl-devel,libkrb5-devel,openldap-devel,libreadline-devel,perl,perl-IPC-Run + C:\tools\cygwin\bin\bash.exe --login -c "cygserver-config -y" || EXIT /b 1 + C:\tools\cygwin\bin\bash.exe --login -c "echo 'kern.ipc.semmni 1024' >> /etc/cygserver.conf" || EXIT /b 1 + C:\tools\cygwin\bin\bash.exe --login -c "echo 'kern.ipc.semmns 1024' >> /etc/cygserver.conf" || EXIT /b 1 + C:\tools\cygwin\bin\bash.exe --login -c "net start cygserver" || EXIT /b 1 + + sysinfo_script: | + chcp + systeminfo + powershell -Command get-psdrive -psprovider filesystem + set + C:\tools\cygwin\bin\bash.exe --login -c "id; uname -a; ulimit -a -H; ulimit -a -S; export" || EXIT /b 1 + + configure_script: + # Try to configure with the cache file, and retry without if it fails, in case the flags changed. + - C:\tools\cygwin\bin\bash.exe --login -xc "cd '%cd%' && for i in 1 2; do ./configure --cache-file=${CONFIGURE_CACHE} ${CONFIGURE_FLAGS} CC='ccache gcc' CFLAGS='-Og -ggdb' && break; rm -v ${CONFIGURE_CACHE}; done" || EXIT /b 1 + + build_script: + - C:\tools\cygwin\bin\bash.exe --login -xc "ccache --zero-stats" || EXIT /b 1 + - C:\tools\cygwin\bin\bash.exe --login -xc "cd '%cd%' && make -s -j '%BUILD_JOBS%' world-bin" || EXIT /b 1 + - C:\tools\cygwin\bin\bash.exe --login -xc "ccache --show-stats" || EXIT /b 1 + + always: + upload_caches: ccache + + test_world_script: + - C:\tools\cygwin\bin\bash.exe --login -xc "cd '%cd%' && timeout 33m make -s -j '%TEST_JOBS%' check-world -Otarget" || EXIT /b 1 + + on_failure: + <<: *on_failure + cores_script: + - C:\tools\cygwin\bin\bash.exe --login -xc "cd '%cd%' && src/tools/ci/cores_backtrace.sh linux ." || EXIT /b 1 + task: name: CompilerWarnings diff --git a/configure b/configure index c5bc3823958..8c6ca4381fb 100755 --- a/configure +++ b/configure @@ -5675,15 +5675,15 @@ fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CC} supports -Wimplicit-fallthrough=3, for CFLAGS" >&5 -$as_echo_n "checking whether ${CC} supports -Wimplicit-fallthrough=3, for CFLAGS... " >&6; } +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CC} supports -Wimplicit-fallthrough 3, for CFLAGS" >&5 +$as_echo_n "checking whether ${CC} supports -Wimplicit-fallthrough 3, for CFLAGS... " >&6; } if ${pgac_cv_prog_CC_cflags__Wimplicit_fallthrough_3+:} false; then : $as_echo_n "(cached) " >&6 else pgac_save_CFLAGS=$CFLAGS pgac_save_CC=$CC CC=${CC} -CFLAGS="${CFLAGS} -Wimplicit-fallthrough=3" +CFLAGS="${CFLAGS} -Wimplicit-fallthrough 3" ac_save_c_werror_flag=$ac_c_werror_flag ac_c_werror_flag=yes cat confdefs.h - <<_ACEOF >conftest.$ac_ext @@ -5710,19 +5710,19 @@ fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_prog_CC_cflags__Wimplicit_fallthrough_3" >&5 $as_echo "$pgac_cv_prog_CC_cflags__Wimplicit_fallthrough_3" >&6; } if test x"$pgac_cv_prog_CC_cflags__Wimplicit_fallthrough_3" = x"yes"; then - CFLAGS="${CFLAGS} -Wimplicit-fallthrough=3" + CFLAGS="${CFLAGS} -Wimplicit-fallthrough 3" fi - { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CXX} supports -Wimplicit-fallthrough=3, for CXXFLAGS" >&5 -$as_echo_n "checking whether ${CXX} supports -Wimplicit-fallthrough=3, for CXXFLAGS... " >&6; } + { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CXX} supports -Wimplicit-fallthrough 3, for CXXFLAGS" >&5 +$as_echo_n "checking whether ${CXX} supports -Wimplicit-fallthrough 3, for CXXFLAGS... " >&6; } if ${pgac_cv_prog_CXX_cxxflags__Wimplicit_fallthrough_3+:} false; then : $as_echo_n "(cached) " >&6 else pgac_save_CXXFLAGS=$CXXFLAGS pgac_save_CXX=$CXX CXX=${CXX} -CXXFLAGS="${CXXFLAGS} -Wimplicit-fallthrough=3" +CXXFLAGS="${CXXFLAGS} -Wimplicit-fallthrough 3" ac_save_cxx_werror_flag=$ac_cxx_werror_flag ac_cxx_werror_flag=yes ac_ext=cpp @@ -5761,7 +5761,7 @@ fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_prog_CXX_cxxflags__Wimplicit_fallthrough_3" >&5 $as_echo "$pgac_cv_prog_CXX_cxxflags__Wimplicit_fallthrough_3" >&6; } if test x"$pgac_cv_prog_CXX_cxxflags__Wimplicit_fallthrough_3" = x"yes"; then - CXXFLAGS="${CXXFLAGS} -Wimplicit-fallthrough=3" + CXXFLAGS="${CXXFLAGS} -Wimplicit-fallthrough 3" fi @@ -17000,7 +17000,7 @@ fi # mingw has adopted a GNU-centric interpretation of optind/optreset, # so always use our version on Windows. -if test "$PORTNAME" = "win32"; then +if test "$PORTNAME" = "win32" -o test "$PORTNAME" = "cygwin"; then case " $LIBOBJS " in *" getopt.$ac_objext "* ) ;; *) LIBOBJS="$LIBOBJS getopt.$ac_objext" diff --git a/configure.ac b/configure.ac index 61d0dd5d586..2be0cae8784 100644 --- a/configure.ac +++ b/configure.ac @@ -504,8 +504,8 @@ if test "$GCC" = yes -a "$ICC" = no; then PGAC_PROG_CXX_CFLAGS_OPT([-Wendif-labels]) PGAC_PROG_CC_CFLAGS_OPT([-Wmissing-format-attribute]) PGAC_PROG_CXX_CFLAGS_OPT([-Wmissing-format-attribute]) - PGAC_PROG_CC_CFLAGS_OPT([-Wimplicit-fallthrough=3]) - PGAC_PROG_CXX_CFLAGS_OPT([-Wimplicit-fallthrough=3]) + PGAC_PROG_CC_CFLAGS_OPT([-Wimplicit-fallthrough 3]) + PGAC_PROG_CXX_CFLAGS_OPT([-Wimplicit-fallthrough 3]) PGAC_PROG_CC_CFLAGS_OPT([-Wcast-function-type]) PGAC_PROG_CXX_CFLAGS_OPT([-Wcast-function-type]) # This was included in -Wall/-Wformat in older GCC versions @@ -1947,7 +1947,7 @@ fi # mingw has adopted a GNU-centric interpretation of optind/optreset, # so always use our version on Windows. -if test "$PORTNAME" = "win32"; then +if test "$PORTNAME" = "win32" -o test "$PORTNAME" = "cygwin"; then AC_LIBOBJ(getopt) AC_LIBOBJ(getopt_long) fi diff --git a/src/interfaces/libpq/t/001_uri.pl b/src/interfaces/libpq/t/001_uri.pl index beaf13b49ca..c46aeb71853 100644 --- a/src/interfaces/libpq/t/001_uri.pl +++ b/src/interfaces/libpq/t/001_uri.pl @@ -224,7 +224,16 @@ sub test_uri $expect{'exit'} = $expect{stderr} eq ''; - my $cmd = [ 'libpq_uri_regress', $uri ]; + my $cmd; + if (-e "$ENV{'TESTDIR'}/test/libpq_uri_regress.exe") + { + $cmd = [ "$ENV{'TESTDIR'}/test/libpq_uri_regress.exe", $uri ]; + } + else + { + $cmd = [ 'libpq_uri_regress', $uri ]; + } + $result{exit} = IPC::Run::run $cmd, '>', \$result{stdout}, '2>', \$result{stderr}; diff --git a/src/interfaces/libpq/t/002_api.pl b/src/interfaces/libpq/t/002_api.pl index fa00221ae29..3bf89e85823 100644 --- a/src/interfaces/libpq/t/002_api.pl +++ b/src/interfaces/libpq/t/002_api.pl @@ -6,7 +6,18 @@ use PostgreSQL::Test::Utils; use Test::More; # Test PQsslAttribute(NULL, "library") -my ($out, $err) = run_command([ 'libpq_testclient', '--ssl' ]); + +my $cmd; +if (-e "$ENV{'TESTDIR'}/test/libpq_uri_regress.exe") +{ + $cmd = "$ENV{'TESTDIR'}/test/libpq_testclient.exe"; +} +else +{ + $cmd = 'libpq_testclient' +} + +my ($out, $err) = run_command([ $cmd, '--ssl' ]); if ($ENV{with_ssl} eq 'openssl') { diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl index 0821329c8d3..126cd7a0085 100644 --- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl +++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl @@ -14,7 +14,17 @@ $node->start; my $numrows = 700; -my ($out, $err) = run_command([ 'libpq_pipeline', 'tests' ]); +my $libpq_pipeline; +if (-e "libpq_pipeline.exe") +{ + $libpq_pipeline = "libpq_pipeline.exe"; +} +else +{ + $libpq_pipeline = "libpq_pipeline"; +} + +my ($out, $err) = run_command([ $libpq_pipeline, 'tests' ]); die "oops: $err" unless $err eq ''; my @tests = split(/\s+/, $out); @@ -39,7 +49,7 @@ for my $testname (@tests) # Execute the test $node->command_ok( [ - 'libpq_pipeline', @extraargs, + $libpq_pipeline, @extraargs, $testname, $node->connstr('postgres') ], "libpq_pipeline $testname"); diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index c8c7bc5045a..febf5988434 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -827,7 +827,7 @@ sub start # compatibility with older versions. $ret = PostgreSQL::Test::Utils::system_log( 'pg_ctl', '-w', '-D', $self->data_dir, - '-l', $self->logfile, '-o', "--cluster-name=$name", + '-l', $self->logfile, '-o', "-c cluster-name=$name", 'start'); if ($ret != 0) diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm index 1ca2cc59170..f10ef55de9a 100644 --- a/src/test/perl/PostgreSQL/Test/Utils.pm +++ b/src/test/perl/PostgreSQL/Test/Utils.pm @@ -88,10 +88,11 @@ our @EXPORT = qw( $windows_os $is_msys2 + $is_cygwin $use_unix_sockets ); -our ($windows_os, $is_msys2, $use_unix_sockets, $timeout_default, +our ($windows_os, $is_msys2, $is_cygwin, $use_unix_sockets, $timeout_default, $tmp_check, $log_path, $test_logfile); BEGIN @@ -140,13 +141,18 @@ BEGIN $ENV{PGAPPNAME} = basename($0); # Must be set early - $windows_os = $Config{osname} eq 'MSWin32' || $Config{osname} eq 'msys'; + $windows_os = $Config{osname} eq 'MSWin32' || $Config{osname} eq 'msys' + || $Config{osname} eq 'cygwin'; + # Check if this environment is MSYS2. $is_msys2 = $windows_os && -x '/usr/bin/uname' && `uname -or` =~ /^[2-9].*Msys/; + # Check if this environment is Cygwin + $is_cygwin = $Config{osname} eq 'cygwin'; + if ($windows_os) { require Win32API::File; @@ -707,7 +713,7 @@ sub dir_symlink { my $oldname = shift; my $newname = shift; - if ($windows_os) + if ($windows_os && !$is_cygwin) { $oldname =~ s,/,\\,g; $newname =~ s,/,\\,g; diff --git a/src/tools/ci/cores_backtrace.sh b/src/tools/ci/cores_backtrace.sh index 28d3cecfc67..475dd609a22 100755 --- a/src/tools/ci/cores_backtrace.sh +++ b/src/tools/ci/cores_backtrace.sh @@ -1,5 +1,8 @@ #! /bin/sh +#set -e +set -x + if [ $# -ne 2 ]; then echo "cores_backtrace.sh <os> <directory>" exit 1 @@ -18,7 +21,7 @@ case $os in esac first=1 -for corefile in $(find "$directory" -type f) ; do +for corefile in $(find "$directory" -type f \( -name 'core.*' -o -name core \) ) ; do if [ "$first" -eq 1 ]; then first=0 else @@ -30,6 +33,7 @@ for corefile in $(find "$directory" -type f) ; do lldb -c $corefile --batch -o 'thread backtrace all' -o 'quit' else auxv=$(gdb --quiet --core ${corefile} --batch -ex 'info auxv' 2>/dev/null) + echo "auxv $auxv" if [ $? -ne 0 ]; then echo "could not process ${corefile}" continue -- 2.17.1
