Inline notes about changes since the last version.
On Thu, Jul 28, 2022 at 05:44:28PM -0500, Justin Pryzby wrote:
> I think the "only_if" should allow separately running one but not both of the
> windows instances, like:
>
> + only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' ||
> $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*mingw64'
>
> I'm not sure, but maybe this task should only run "by request", and omit the
> first condition:
>
> + only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*mingw64'
The patch shouldn't say this during development, or else cfbot doesn't run it..
Oops.
> I think it should include something like
>
> + setup_additional_packages_script: |
> + REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ...
>
> Let's see what others think about those.
>
> Do you know if this handles logging of crash dumps ?
It does now, although I hardcoded "postgres.exe" ...
> + setup_additional_packages_script: |
> + REM C:\msys64\usr\bin\pacman.exe -S --noconfirm busybox
This should include choco, too.
> - CXXFLAGS='-Og -ggdb'"
> + CXXFLAGS='-Og -ggdb' && break;
> + rm -v ${CCACHE_DIR}/configure.cache;
> + done
I noticed that this doesn't seem to do the right thing with the exit status -
configure can fail without cirrusci noticing, and then the build fails at the
next step.
> for item in `find "$sourcetree" -name Makefile -print -o -name GNUmakefile
> -print | grep -v "$sourcetree/doc/src/sgml/images/"`; do
> - filename=`expr "$item" : "$sourcetree\(.*\)"`
> - if test ! -f "${item}.in"; then
> - if cmp "$item" "$buildtree/$filename" >/dev/null 2>&1; then : ; else
> - ln -fs "$item" "$buildtree/$filename" || exit 1
> - fi
> - fi
> + filename=${item#$sourcetree}
> + [ -e "$buildtree/$filename" ] && continue
I fixed this to check for ".in" files as intended.
It'd be a lot better if the image didn't take so long to start. :(
--
Justin
>From 32646786299672d333cbf1f49bafac9e90e0d3be Mon Sep 17 00:00:00 2001
From: Melih Mutlu <memu...@microsoft.com>
Date: Mon, 21 Feb 2022 14:46:05 +0300
Subject: [PATCH 1/2] Added Windows with MinGW environment in Cirrus CI
---
.cirrus.yml | 78 ++++++++++++++++++++++++++++++++++++++++++++---------
1 file changed, 65 insertions(+), 13 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index a9193c2c34f..472661c0936 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -339,13 +339,29 @@ task:
cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores"
+WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE
+ env:
+ # Half the allowed per-user CPU cores
+ CPUS: 4
+ # The default working dir is in a directory msbuild complains about
+ CIRRUS_WORKING_DIR: "c:/cirrus"
+
+ # Avoids port conflicts between concurrent tap test runs
+ PG_TEST_USE_UNIX_SOCKETS: 1
+
+ only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*windows.*'
+
+ sysinfo_script: |
+ chcp
+ systeminfo
+ powershell -Command get-psdrive -psprovider filesystem
+ set
+
task:
+ << : *WINDOWS_ENVIRONMENT_BASE
name: Windows - Server 2019, VS 2019
env:
- # Half the allowed per-user CPU cores
- CPUS: 4
-
# Our windows infrastructure doesn't have test concurrency above the level
# of a single vcregress test target. Due to that, it's useful to run prove
# with multiple jobs. For the other tasks it isn't, because two sources
@@ -355,15 +371,11 @@ task:
# likely can be improved upon further.
PROVE_FLAGS: -j10 --timer
- # The default cirrus working dir is in a directory msbuild complains about
- CIRRUS_WORKING_DIR: "c:/cirrus"
# Avoid re-installing over and over
NO_TEMP_INSTALL: 1
# git's tar doesn't deal with drive letters, see
# https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net
TAR: "c:/windows/system32/tar.exe"
- # Avoids port conflicts between concurrent tap test runs
- PG_TEST_USE_UNIX_SOCKETS: 1
PG_REGRESS_SOCK_DIR: "c:/cirrus/"
# -m enables parallelism
# verbosity:minimal + Summary reduce verbosity, while keeping a summary of
@@ -398,12 +410,6 @@ task:
cpu: $CPUS
memory: 4G
- sysinfo_script: |
- chcp
- systeminfo
- powershell -Command get-psdrive -psprovider filesystem
- set
-
setup_additional_packages_script: |
REM 3min
REM choco install -y --no-progress --version=1.0.0 visualstudio2022-workload-vctools --install-args="--add Microsoft.VisualStudio.Component.VC.CLI.Support"
@@ -475,6 +481,52 @@ task:
path: "crashlog-*.txt"
type: text/plain
+task:
+ << : *WINDOWS_ENVIRONMENT_BASE
+ name: Windows - Server 2019, MinGW64
+ windows_container:
+ image: $CONTAINER_REPO/windows_ci_mingw64:latest
+ cpu: $CPUS
+ memory: 4G
+ env:
+ CCACHE_DIR: C:/msys64/ccache
+ BUILD_DIR: "%CIRRUS_WORKING_DIR%/build"
+
+ ccache_cache:
+ folder: ${CCACHE_DIR}
+
+ mingw_info_script:
+ - C:\msys64\usr\bin\dash.exe -lc "where gcc"
+ - C:\msys64\usr\bin\dash.exe -lc "gcc --version"
+ - C:\msys64\usr\bin\dash.exe -lc "where perl"
+ - C:\msys64\usr\bin\dash.exe -lc "perl --version"
+
+ configure_script:
+ - C:\msys64\usr\bin\dash.exe -lc "mkdir %BUILD_DIR% &&
+ cd %BUILD_DIR% &&
+ %CIRRUS_WORKING_DIR%/configure
+ --enable-cassert
+ --enable-tap-tests
+ --with-icu
+ --with-libxml
+ --with-libxslt
+ --with-lz4
+ --enable-debug
+ CC='ccache gcc'
+ CXX='ccache g++'
+ CFLAGS='-Og -ggdb -pipe'
+ CXXFLAGS='-Og -ggdb'"
+
+ build_script:
+ C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s world-bin -j${CPUS}"
+
+ upload_caches: ccache
+
+ tests_script:
+ - set "NoDefaultCurrentDirectoryInExePath=0"
+ - C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s ${CHECK} ${CHECKFLAGS} -j${CPUS} TMPDIR=%BUILD_DIR%/tmp_install"
+
+ on_failure: *on_failure
task:
name: CompilerWarnings
--
2.17.1
>From 49a49939827dde731f513d334e53e1f56be44b5f Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Thu, 28 Jul 2022 10:57:26 -0500
Subject: [PATCH 2/2] f!
ci-os-only: mingw
---
.cirrus.yml | 31 +++++++++++++++++++++++--------
config/prep_buildtree | 20 ++++++++------------
src/tools/ci/cores_backtrace.sh | 17 +++++++++++++++--
3 files changed, 46 insertions(+), 22 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 472661c0936..b51a625d210 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -34,6 +34,7 @@ on_failure: &on_failure
- "**/*.log"
- "**/*.diffs"
- "**/regress_log_*"
+ - "**/*.stackdump"
type: text/plain
task:
@@ -349,8 +350,6 @@ WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE
# Avoids port conflicts between concurrent tap test runs
PG_TEST_USE_UNIX_SOCKETS: 1
- only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*windows.*'
-
sysinfo_script: |
chcp
systeminfo
@@ -361,6 +360,8 @@ task:
<< : *WINDOWS_ENVIRONMENT_BASE
name: Windows - Server 2019, VS 2019
+ only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*windows.*'
+
env:
# Our windows infrastructure doesn't have test concurrency above the level
# of a single vcregress test target. Due to that, it's useful to run prove
@@ -484,6 +485,8 @@ task:
task:
<< : *WINDOWS_ENVIRONMENT_BASE
name: Windows - Server 2019, MinGW64
+ #XXX only_if: $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*mingw.*'
+
windows_container:
image: $CONTAINER_REPO/windows_ci_mingw64:latest
cpu: $CPUS
@@ -495,6 +498,10 @@ task:
ccache_cache:
folder: ${CCACHE_DIR}
+ setup_additional_packages_script: |
+ REM choco install -y --no-progress ...
+ REM C:\msys64\usr\bin\pacman.exe -S --noconfirm busybox
+
mingw_info_script:
- C:\msys64\usr\bin\dash.exe -lc "where gcc"
- C:\msys64\usr\bin\dash.exe -lc "gcc --version"
@@ -502,20 +509,25 @@ task:
- C:\msys64\usr\bin\dash.exe -lc "perl --version"
configure_script:
+ # Try to configure with the cache file, and retry without if it fails, in case the flags changed.
- C:\msys64\usr\bin\dash.exe -lc "mkdir %BUILD_DIR% &&
- cd %BUILD_DIR% &&
+ cd %BUILD_DIR% && for i in 1 2; do
%CIRRUS_WORKING_DIR%/configure
+ --cache-file=${CCACHE_DIR}/configure.cache
--enable-cassert
+ --enable-debug
--enable-tap-tests
--with-icu
--with-libxml
--with-libxslt
--with-lz4
- --enable-debug
CC='ccache gcc'
CXX='ccache g++'
CFLAGS='-Og -ggdb -pipe'
- CXXFLAGS='-Og -ggdb'"
+ CXXFLAGS='-Og -ggdb' && break;
+ rm -v ${CCACHE_DIR}/configure.cache;
+ done
+ "
build_script:
C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s world-bin -j${CPUS}"
@@ -523,10 +535,13 @@ task:
upload_caches: ccache
tests_script:
- - set "NoDefaultCurrentDirectoryInExePath=0"
- - C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s ${CHECK} ${CHECKFLAGS} -j${CPUS} TMPDIR=%BUILD_DIR%/tmp_install"
+ - set "NoDefaultCurrentDirectoryInExePath=0"
+ - C:\msys64\usr\bin\dash.exe -lc "cd %BUILD_DIR% && make -s ${CHECK} ${CHECKFLAGS} -j${CPUS} TMPDIR=%BUILD_DIR%/tmp_install"
- on_failure: *on_failure
+ on_failure:
+ <<: *on_failure
+ cores_script:
+ - C:\tools\cygwin\bin\dash.exe --login -c "cd '%cd%' && src/tools/ci/cores_backtrace.sh msys ."
task:
name: CompilerWarnings
diff --git a/config/prep_buildtree b/config/prep_buildtree
index a0eabd3dee2..97bf65967f1 100644
--- a/config/prep_buildtree
+++ b/config/prep_buildtree
@@ -26,20 +26,16 @@ buildtree=`cd ${2:-'.'} && pwd`
# If we did, it would interfere with installation of prebuilt docs from
# the source tree, if a VPATH build is done from a distribution tarball.
# See bug #5595.
-for item in `find "$sourcetree" -type d \( \( -name CVS -prune \) -o \( -name .git -prune \) -o -print \) | grep -v "$sourcetree/doc/src/sgml/\+"`; do
- subdir=`expr "$item" : "$sourcetree\(.*\)"`
- if test ! -d "$buildtree/$subdir"; then
- mkdir -p "$buildtree/$subdir" || exit 1
- fi
-done
+( cd "$sourcetree" && find . -type d \( \( -name CVS -prune \) -o \( -name .git -prune \) -o -print \) |grep -v "doc/src/sgml/\+" ) |
+ ( cd "$buildtree" && xargs mkdir -p )
for item in `find "$sourcetree" -name Makefile -print -o -name GNUmakefile -print | grep -v "$sourcetree/doc/src/sgml/images/"`; do
- filename=`expr "$item" : "$sourcetree\(.*\)"`
- if test ! -f "${item}.in"; then
- if cmp "$item" "$buildtree/$filename" >/dev/null 2>&1; then : ; else
- ln -fs "$item" "$buildtree/$filename" || exit 1
- fi
- fi
+ [ -e "$item.in" ] && echo "skipping: $item" &&
+ continue
+
+ filename=${item#$sourcetree}
+ cmp "$item" "$buildtree/$filename" >/dev/null 2>&1 ||
+ ln -fs "$item" "$buildtree/$filename"
done
exit 0
diff --git a/src/tools/ci/cores_backtrace.sh b/src/tools/ci/cores_backtrace.sh
index 28d3cecfc67..93e90f284e6 100755
--- a/src/tools/ci/cores_backtrace.sh
+++ b/src/tools/ci/cores_backtrace.sh
@@ -10,11 +10,24 @@ directory=$2
case $os in
freebsd|linux|macos)
- ;;
+ ;;
+
+ msys)
+ # XXX Evidently I don't know how to write two arguments here without pathname expansion later, other than eval.
+ #findargs='-name "*.stackdump"'
+ for corefile in $(find "$directory" -type f -name "*.stackdump") ; do
+ binary=`basename "$corefile" .stackdump`
+ echo;echo;
+ echo "dumping ${corefile} for ${binary}"
+ awk '/^0/{print $2}' $corefile |addr2line -f -i -e ./src/backend/postgres.exe
+ done
+ exit 0
+ ;;
+
*)
echo "unsupported operating system ${os}"
exit 1
- ;;
+ ;;
esac
first=1
--
2.17.1
