Hi,
Attached is an large incremental patch onto Bilal's version:
- I hacked enough on the pg-vm-images repo to make it store containers in
github (on the gha_main branch, for now). That makes the container faster
and cheaper to retrieve.
We might want to split the containers further (e.g. cross building and 32bit
support), but for now I just split the docs stuff into a separate container.
- The reason that sometimes cancelling took a long time was, afaict, the use
of always() in the compiler-warning job. That apparently prevents GHA from
cancelling the job in a timely fashion. I turned those into !cancelled().
Also addded a !cancelled() to SanityCheck.
- In the end I didn't like the matrix all that much, makes it considerably
harder to understand everything and the restrictions GHA puts on it are just
too annoying.
I replaced it much more heavy use of yaml anchors/aliases and by updating
the environment programattically. I'm not sure how much better it is now.
- The macports cache keys included the run_id, I think that's a bad idea,
because it leads to the cache being newly uploaded even if there's been no
change. That'll lead to even more quickly churning through the cache space.
- ccache:
- There was too much duplication around the ccache handling for my taste. I
moved that into a yaml anchor and reused it everywhere. By using
${{github.job_id}} the cache names don't need to be manually disambiguated.
- The ccache names weren't unique enough. The run_id doesn't change during
reruns which would lead to warnings.
- Made it so that the cache is saved immediately after the build, so that
cancelled builds still save the cache.
- I wanted to share commands like meson test between the tasks, made that work
with a bit of hackery.
- I didn't see why
find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \;
was needed.
- I found it hard to find actual warnings in the output of CompilerWarnings,
due to to all the configure output. I added some magic output stuff to make
the configure output collapse once it ran.
- I converted most things to use ${{env.varname}} instead of $VARNAME or
%VARNAME%, as that allows sharing code between windows and other OSs if one
is a bit careful.
- Deduplicated a few other things like logging.
- Added a commit removing cirrus, mainly because I was tired of it also
running while hacking on this.
- Also added a commit to reduce the segment size in tests to 1MB, that makes
them a decent bit less IO intensive.
- Added a commit to just run regress/regress, makes it a lot faster to test
"complete-ish" cycles.
- Changed sanitizer using builds to use -O2, to reduce the CPU cost a bit.
This makes uncached builds noticeably slower, but does appear to be a
win. But I could see counter-arguments to that too.
- Removed :detect_stack_use_after_return=0, as that's been made unnecessary
since you "forked off" from .cirrus.tasks.yml.
A few other comments:
- All the caches in GHA apparently are branch specific, except that branches
can access the caches of the main branch.
I think that'll make particularly macports very expensive for cfbot. I
wonder if we ought to build the "base" macports cache in pg-vm-images.
Similarly, I think we probably should do that for mingw64.
- src/tools/ci/README needs updating
- I experimented with making the slowest jobs faster by using
meson test --slice 1/2
that actually does nicely work. But it didn't quite seem critical for now.
Here's a run running with just regress/regress:
https://github.com/anarazel/postgres/actions/runs/26782558353
(the CompilerWarnings ccache hit had a miss, that's why it's not faster)
Here's a (not yet completed) run with the full tests:
https://github.com/anarazel/postgres/actions/runs/26784248887
Thoughts?
Greetings,
Andres Freund
>From 844833b49963726af8832bcfeadd93e6564d3f75 Mon Sep 17 00:00:00 2001
From: Nazir Bilal Yavuz <[email protected]>
Date: Thu, 28 May 2026 19:31:34 +0300
Subject: [PATCH v6a 1/5] Add GitHub Actions workflow for CI
Cirrus CI, which the project has used to run CI, is shutting down on
June 1, 2026. Replace it with a GitHub Actions. Github Actions is
selected because it has unlimited runner time for public repositories.
Github Action currently covers:
- SanityCheck
- Linux (Autoconf)
- Linux (Meson, 32- and 64-bit)
- macOS (Meson)
- Windows (Visual Studio + Meson and MinGW + Meson)
- CompilerWarnings
BSD coverage is left for later, as it requires more work.
Back-branches will be updated later, after being sure that workflow runs
correctly on master.
Author: Jelte Fennema-Nio <[email protected]>
Author: Nazir Bilal Yavuz <[email protected]>
Reviewed-by: Jacob Champion <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Andres Freund <[email protected]>
Reviewed-by: Zsolt Parragi <[email protected]>
Discussion: https://postgr.es/m/3ydjipcr7kbss57nvi67noplncqhesl5eyb6wgol4ccjxynspv%40yatlykpribmm
---
.github/workflows/postgresql-ci.yml | 1026 ++++++++++++++++++++++++++
src/tools/ci/ci_macports_packages.sh | 19 +-
2 files changed, 1042 insertions(+), 3 deletions(-)
create mode 100644 .github/workflows/postgresql-ci.yml
diff --git a/.github/workflows/postgresql-ci.yml b/.github/workflows/postgresql-ci.yml
new file mode 100644
index 00000000000..a7ef0bee94d
--- /dev/null
+++ b/.github/workflows/postgresql-ci.yml
@@ -0,0 +1,1026 @@
+# GitHub Actions CI configuration for PostgreSQL
+
+name: GitHub Actions CI
+
+on:
+ push:
+
+# Restrict GITHUB_TOKEN to the minimum the jobs need: reading repo
+# contents during checkout.
+permissions:
+ contents: read
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ # Never cancel in-progress runs on master to ensure all commits are tested.
+ cancel-in-progress: ${{ github.ref != 'refs/heads/master' }}
+
+env:
+ # The lower depth accelerates git clone. Use a bit of depth so that
+ # concurrent jobs and retrying older runs have a chance of working.
+ CLONE_DEPTH: 500
+
+ CCACHE_MAXSIZE: "250M"
+
+ # check target for the autoconf builds
+ CHECK: check-world PROVE_FLAGS=--timer
+ CHECKFLAGS: -Otarget
+
+ # Build test dependencies as part of the build step, to see compiler
+ # errors/warnings in one place.
+ MBUILD_TARGET: all testprep
+ MTEST_ARGS: --print-errorlogs --no-rebuild -C build
+ PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
+ TEMP_CONFIG: ${{ github.workspace }}/src/tools/ci/pg_ci_base.conf
+ PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth
+
+ # Postgres config args for the meson builds, shared between all meson tasks
+ # except the 'SanityCheck' task
+ MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true
+
+ # Meson feature flags shared by all meson tasks, except:
+ # SanityCheck: uses almost no dependencies.
+ # Windows - VS: has fewer dependencies than listed here, so defines its own.
+ # Linux: uses the 'auto' feature option to test meson feature autodetection.
+ MESON_COMMON_FEATURES: >-
+ -Dauto_features=disabled
+ -Dldap=enabled
+ -Dssl=openssl
+ -Dtap_tests=enabled
+ -Dplperl=enabled
+ -Dplpython=enabled
+ -Ddocs=enabled
+ -Dicu=enabled
+ -Dlibxml=enabled
+ -Dlibxslt=enabled
+ -Dlz4=enabled
+ -Dpltcl=enabled
+ -Dreadline=enabled
+ -Dzlib=enabled
+ -Dzstd=enabled
+
+ # Shared between the Linux autoconf job and the CompilerWarnings jobs
+ LINUX_CONFIGURE_FEATURES: >-
+ --with-gssapi
+ --with-icu
+ --with-ldap
+ --with-libcurl
+ --with-libxml
+ --with-libxslt
+ --with-llvm
+ --with-lz4
+ --with-pam
+ --with-perl
+ --with-python
+ --with-selinux
+ --with-ssl=openssl
+ --with-systemd
+ --with-tcl --with-tclconfig=/usr/lib/tcl8.6/
+ --with-uuid=ossp
+ --with-zstd
+
+ # Debian Trixie container image used by all Linux jobs. Built by
+ # 'https://github.com/anarazel/pg-vm-images/'.
+ LINUX_CI_IMAGE: us-docker.pkg.dev/pg-ci-images/ci/linux_debian_trixie_ci:latest
+
+ # The full set of OS / job selectors recognized by the `ci-os-only:`
+ # commit-message directive parsed in the `setup` job below.
+ CI_OS_ONLY_JOBS: "linux macos windows mingw compilerwarnings sanitycheck"
+
+ _LOG_PATHS: &log_paths |
+ build*/testrun/**/*.log
+ build*/testrun/**/*.diffs
+ build*/testrun/**/regress_log_*
+ build*/meson-logs/*.txt
+
+
+jobs:
+ # Parse "ci-os-only: ..." from the commit message and expose flags
+ # consumed by the jobs' `if:` conditions.
+ setup:
+ name: Determine enabled jobs
+ runs-on: ubuntu-latest
+ timeout-minutes: 1
+ outputs:
+ linux: ${{ steps.os.outputs.linux }}
+ macos: ${{ steps.os.outputs.macos }}
+ windows: ${{ steps.os.outputs.windows }}
+ mingw: ${{ steps.os.outputs.mingw }}
+ compilerwarnings: ${{ steps.os.outputs.compilerwarnings }}
+ sanitycheck: ${{ steps.os.outputs.sanitycheck }}
+ # Re-export workflow-level env vars that other jobs need to reference
+ # from contexts (e.g. `jobs.<id>.container.image`) where the `env`
+ # context is not available.
+ linux_ci_image: ${{ env.LINUX_CI_IMAGE }}
+ steps:
+ - id: os
+ env:
+ MSG: ${{ github.event.head_commit.message }}
+ shell: bash
+ run: |
+ set -e
+ all_os=${CI_OS_ONLY_JOBS}
+ if printf '%s\n' "$MSG" | grep -qE '^ci-os-only: '; then
+ sel=$(printf '%s\n' "$MSG" | sed -n 's/^ci-os-only: //p' | head -n 1)
+ echo "ci-os-only selection: $sel"
+ else
+ sel="$all_os"
+ fi
+ for o in $all_os; do
+ if echo " $sel " | grep -qE "[ ,]$o[ ,]"; then
+ echo "$o=true" >> "$GITHUB_OUTPUT"
+ else
+ echo "$o=false" >> "$GITHUB_OUTPUT"
+ fi
+ done
+ cat "$GITHUB_OUTPUT"
+
+
+ # To avoid unnecessarily spinning up a lot of VMs / containers for entirely
+ # broken commits, have a minimal task that all others depend on.
+ #
+ # SPECIAL:
+ # - Builds with --auto-features=disabled and thus almost no enabled
+ # dependencies
+ sanity-check:
+ name: SanityCheck
+ needs: setup
+ if: needs.setup.outputs.sanitycheck == 'true'
+ runs-on: ubuntu-latest
+ timeout-minutes: 15
+ container:
+ image: ${{ needs.setup.outputs.linux_ci_image }}
+ # --privileged is needed so the prepare step can write to sysctls
+ # under /proc/sys (it's mounted read-only without it). We use it to
+ # set kernel.core_pattern.
+ options: --privileged
+ env:
+ BUILD_JOBS: 8
+ TEST_JOBS: 8
+ CCACHE_DIR: ${{ github.workspace }}/ccache_dir
+ # no options enabled, should be small
+ CCACHE_MAXSIZE: "150M"
+ steps:
+ # Anchor reused by other jobs further down. GitHub Actions supports
+ # YAML anchors/aliases but not merge keys, so the alias copies the
+ # whole step verbatim. The anchor is resolved at YAML parse time, so the
+ # alias keeps working even if this job is skipped at runtime.
+ - &checkout_step
+ uses: actions/checkout@v6
+ with:
+ fetch-depth: ${{ env.CLONE_DEPTH }}
+
+ - name: Restore ccache
+ uses: actions/cache@v5
+ with:
+ path: ${{ env.CCACHE_DIR }}
+ key: ccache-sanitycheck-${{ github.ref_name }}-${{ github.run_id }}
+ restore-keys: |
+ ccache-sanitycheck-${{ github.ref_name }}-
+ ccache-sanitycheck-
+
+ - name: Prepare workspace
+ run: |
+ whoami
+ useradd -m postgres
+ chown -R postgres:postgres .
+ mkdir -p "$CCACHE_DIR"
+ chown -R postgres:postgres "$CCACHE_DIR"
+
+ - name: Configure
+ run: |
+ su postgres <<-'EOF'
+ set -e
+ meson setup \
+ --buildtype=debug \
+ --auto-features=disabled \
+ -Ddefault_library=shared \
+ -Dtap_tests=enabled \
+ build
+ EOF
+
+ - name: Build
+ run: |
+ su postgres <<EOF
+ set -e
+ ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}
+ EOF
+
+ # Run a minimal set of tests. The main regression tests take too long
+ # for this purpose. For now this is a random quick pg_regress style
+ # test, and a tap test that exercises both a frontend binary and the
+ # backend.
+ - name: Test
+ run: |
+ su postgres <<EOF
+ set -e
+ ulimit -c unlimited
+ meson test ${MTEST_ARGS} --suite setup
+ meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS} \
+ cube/regress pg_ctl/001_start_stop
+ EOF
+
+ - name: Core backtraces
+ if: failure()
+ run: |
+ mkdir -m 770 /tmp/cores
+ find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \;
+ src/tools/ci/cores_backtrace.sh linux /tmp/cores
+
+ - name: Upload logs
+ if: failure()
+ uses: actions/upload-artifact@v7
+ with:
+ name: sanitycheck-logs-${{ github.run_id }}
+ path: *log_paths
+ if-no-files-found: ignore
+
+
+ # Build & test postgres on Linux in three configurations.
+ #
+ # Autoconf:
+ # - Uses address sanitizer (sanitizer failures are typically printed in
+ # the server log)
+ # - Configures postgres with a small segment size
+ # - Uses PG_TEST_PG_COMBINEBACKUP_MODE=--copy-file-range
+ #
+ # Meson:
+ # - Test both 64- and 32-bit builds
+ # - Uses undefined behaviour and alignment sanitizers, (sanitizer failures
+ # are typically printed in the server log)
+ # - Uses io_method=io_uring
+ # - Uses meson feature autodetection
+ # - 32-bit build tests with LANG=C to give ICU some buildfarm-uncovered
+ # coverage. Also, newer Python insists on changing LC_CTYPE away from C,
+ # prevent that with PYTHONCOERCECLOCALE.
+ #
+ # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes
+ # print_stacktraces=1,verbosity=2, duh
+ # detect_leaks=0: too many uninteresting leak errors in short-lived binaries
+ linux:
+ name: Linux - ${{ matrix.name }}
+ needs: [setup, sanity-check]
+ if: |
+ !cancelled() &&
+ needs.setup.outputs.linux == 'true' &&
+ needs.sanity-check.result != 'failure'
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+ strategy:
+ fail-fast: false
+ matrix:
+ include:
+ - name: Autoconf
+ slug: autoconf
+ cc: ccache gcc
+ cxx: ccache g++
+ sanitizer_flags: -fsanitize=address
+ pg_test_pg_combinebackup_mode: '--copy-file-range'
+ configure: |
+ ./configure \
+ --enable-cassert --enable-injection-points --enable-debug \
+ --enable-tap-tests --enable-nls \
+ --with-segsize-blocks=6 \
+ --with-libnuma \
+ --with-liburing \
+ ${LINUX_CONFIGURE_FEATURES} \
+ CLANG="ccache clang"
+ build: |
+ make -s -j${BUILD_JOBS} world-bin
+ test: |
+ make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS}
+ logs_paths: |
+ **/*.log
+ **/*.diffs
+ **/regress_log_*
+
+ - name: Meson (64-bit)
+ slug: meson-64
+ cc: ccache gcc
+ cxx: ccache g++
+ sanitizer_flags: -fsanitize=alignment,undefined
+ pg_test_initdb_extra_opts: '-c io_method=io_uring'
+ configure: |
+ meson setup \
+ ${MESON_COMMON_PG_CONFIG_ARGS} \
+ -Duuid=e2fs \
+ --buildtype=debug \
+ -Dllvm=enabled \
+ build
+ build: |
+ ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}
+ ninja -C build -t missingdeps
+ test: |
+ meson test ${MTEST_ARGS} -C build --num-processes ${TEST_JOBS}
+ logs_paths: *log_paths
+
+ - name: Meson (32-bit)
+ slug: meson-32
+ cc: ccache gcc -m32
+ cxx: ccache g++ -m32
+ sanitizer_flags: -fsanitize=alignment,undefined
+ pg_test_initdb_extra_opts: '-c io_method=io_uring'
+ configure: |
+ meson setup \
+ ${MESON_COMMON_PG_CONFIG_ARGS} \
+ -Duuid=e2fs \
+ --buildtype=debug \
+ --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
+ -DPERL=perl5.40-i386-linux-gnu \
+ -Dlibnuma=disabled \
+ build
+ build: |
+ ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}
+ ninja -C build -t missingdeps
+ test: |
+ PYTHONCOERCECLOCALE=0 LANG=C \
+ meson test ${MTEST_ARGS} -C build --num-processes ${TEST_JOBS}
+ logs_paths: *log_paths
+ container:
+ image: ${{ needs.setup.outputs.linux_ci_image }}
+ # Share the host PID + IPC namespaces. 017_shm.pl rapidly creates,
+ # kill9's, and restarts postgres; with the container's small PID
+ # space a new postgres can recycle the dead postmaster's PID before
+ # pg_ctl's postmaster.pid check notices, producing spurious "node X
+ # is already running" failures. SysV shm in the test also relies on
+ # host-like IPC behavior.
+ #
+ # --ulimit raises memlock and core dump size. Memlock is needed for
+ # running the AIO tests.
+ #
+ # --privileged is needed so the prepare step can write to sysctls
+ # under /proc/sys (it's mounted read-only without it). We use it to
+ # set kernel.core_pattern and (for the meson entries) to flip
+ # kernel.io_uring_disabled (default 2 on recent GH runner kernels).
+ options: --pid=host --ipc=host --ulimit memlock=-1:-1 --privileged
+ env:
+ BUILD_JOBS: 4
+ TEST_JOBS: 8
+ CCACHE_DIR: /tmp/ccache_dir
+ DEBUGINFOD_URLS: "https://debuginfod.debian.net";
+
+ UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2
+ ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0:detect_stack_use_after_return=0
+ CFLAGS: -Og -ggdb -fno-sanitize-recover=all ${{ matrix.sanitizer_flags }}
+ CXXFLAGS: -Og -ggdb -fno-sanitize-recover=all ${{ matrix.sanitizer_flags }}
+ LDFLAGS: ${{ matrix.sanitizer_flags }}
+ CC: ${{ matrix.cc }}
+ CXX: ${{ matrix.cxx }}
+
+ PG_TEST_INITDB_EXTRA_OPTS: ${{ matrix.pg_test_initdb_extra_opts }}
+ PG_TEST_PG_COMBINEBACKUP_MODE: ${{ matrix.pg_test_pg_combinebackup_mode }}
+ steps:
+ - *checkout_step
+
+ - name: Restore ccache
+ uses: actions/cache@v5
+ with:
+ path: ${{ env.CCACHE_DIR }}
+ key: ccache-linux-${{ matrix.slug }}-${{ github.ref_name }}-${{ github.run_id }}
+ restore-keys: |
+ ccache-linux-${{ matrix.slug }}-${{ github.ref_name }}-
+ ccache-linux-${{ matrix.slug }}-
+
+ - name: Prepare workspace
+ run: |
+ useradd -m postgres
+ chown -R postgres:postgres .
+ mkdir -p "$CCACHE_DIR"
+ chown -R postgres:postgres "$CCACHE_DIR"
+ mkdir -m 770 /tmp/cores
+ chown root:postgres /tmp/cores
+ sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core'
+ # This is only needed on Linux Meson but it doesn't harm to have
+ # this enabled.
+ sysctl -w kernel.io_uring_disabled=0
+
+ cat >> /etc/hosts <<-EOF
+ 127.0.0.1 pg-loadbalancetest
+ 127.0.0.2 pg-loadbalancetest
+ 127.0.0.3 pg-loadbalancetest
+ EOF
+
+ - name: Configure
+ run: |
+ su postgres <<EOF
+ set -e
+ ${{ matrix.configure }}
+ EOF
+
+ - name: Build
+ run: |
+ su postgres <<EOF
+ set -e
+ ${{ matrix.build }}
+ EOF
+
+ - name: Test world
+ run: |
+ su postgres <<EOF
+ set -e
+ ulimit -c unlimited
+ ${{ matrix.test }}
+ EOF
+
+ - name: Core backtraces
+ if: failure()
+ run: src/tools/ci/cores_backtrace.sh linux /tmp/cores
+
+ - name: Upload logs
+ if: failure()
+ uses: actions/upload-artifact@v7
+ with:
+ name: linux-${{ matrix.slug }}-logs-${{ github.run_id }}
+ path: ${{ matrix.logs_paths }}
+ if-no-files-found: ignore
+
+
+ # SPECIAL:
+ # - Enables --clone for pg_upgrade and pg_combinebackup
+ # - Specifies configuration options that test reading/writing/copying of node trees
+ # - Specifies debug_parallel_query=regress, to catch related issues during CI
+ macos:
+ name: macOS - Meson
+ needs: [setup, sanity-check]
+ if: |
+ !cancelled() &&
+ needs.setup.outputs.macos == 'true' &&
+ needs.sanity-check.result != 'failure'
+ runs-on: macos-15
+ timeout-minutes: 60
+ env:
+ BUILD_JOBS: 4
+ # Test performance regresses noticeably when using all cores. 8 works OK.
+ # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de
+ # Fix: Needs to be re-tested for GitHub Actions.
+ TEST_JOBS: 8
+
+ CCACHE_DIR: ${{ github.workspace }}/ccache_dir
+ MACPORTS_CACHE: ${{ github.workspace }}/macports-cache
+
+ MESON_FEATURES: >-
+ -Dbonjour=enabled
+ -Ddtrace=enabled
+ -Dgssapi=enabled
+ -Dlibcurl=enabled
+ -Dnls=enabled
+ -Duuid=e2fs
+
+ MACOS_PACKAGE_LIST: >-
+ ccache
+ icu
+ kerberos5
+ lz4
+ meson
+ openldap
+ openssl
+ p5.34-io-tty
+ p5.34-ipc-run
+ python312
+ tcl
+ zstd
+
+ CC: ccache cc
+ CXX: ccache c++
+ CFLAGS: -Og -ggdb
+ CXXFLAGS: -Og -ggdb
+ PG_TEST_PG_UPGRADE_MODE: --clone
+ PG_TEST_PG_COMBINEBACKUP_MODE: --clone
+
+ # Several buildfarm animals enable these options. Without testing them
+ # during CI, it would be easy to cause breakage on the buildfarm with CI
+ # passing.
+ PG_TEST_INITDB_EXTRA_OPTS: >-
+ -c debug_copy_parse_plan_trees=on
+ -c debug_write_read_parse_plan_trees=on
+ -c debug_raw_expression_coverage_test=on
+ -c debug_parallel_query=regress
+
+ steps:
+ - *checkout_step
+
+ - name: Sysinfo
+ run: |
+ id
+ uname -a
+ ulimit -a -H && ulimit -a -S
+ env
+
+ - name: Setup core files
+ run: |
+ mkdir -p $HOME/cores
+ sudo sysctl kern.corefile="$HOME/cores/core.%P"
+
+ - name: Restore ccache
+ uses: actions/cache@v5
+ with:
+ path: ${{ env.CCACHE_DIR }}
+ key: ccache-macos-${{ github.ref_name }}-${{ github.run_id }}
+ restore-keys: |
+ ccache-macos-${{ github.ref_name }}-
+ ccache-macos-
+
+ - name: Compute MacPorts cache key
+ id: mpkey
+ run: |
+ macos_major=$(sw_vers -productVersion | sed 's/\..*//')
+ pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q)
+ script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh)
+ echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}-${GITHUB_RUN_ID}" >> "$GITHUB_OUTPUT"
+ echo "restore-key=macports-${macos_major}-${pkglist_hash}-${script_hash}-" >> "$GITHUB_OUTPUT"
+
+ - name: Restore MacPorts cache
+ uses: actions/cache@v5
+ with:
+ path: ${{ env.MACPORTS_CACHE }}
+ key: ${{ steps.mpkey.outputs.key }}
+ restore-keys: ${{ steps.mpkey.outputs.restore-key }}
+
+ # Use MacPorts, even though Homebrew is installed. The installation
+ # of the additional packages we need would take quite a while with
+ # Homebrew, even if we cache the downloads. We can't cache all of
+ # Homebrew, because it's already large. So we use MacPorts. To cache
+ # the installation we create a .dmg file that we mount if it already
+ # exists.
+ # XXX: The reason for the direct p5.34* references is that we'd need
+ # the large MacPort tree around to figure out that p5-io-tty is
+ # actually p5.34-io-tty. Using the unversioned name works, but
+ # updates MacPorts every time.
+ - name: Install dependencies (MacPorts)
+ env:
+ # Pass token so the script's GitHub API call to list MacPorts
+ # releases isn't subject to the 60/h/IP unauthenticated rate
+ # limit (shared across all jobs on the runner's IP).
+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+ run: |
+ sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST
+ # system python doesn't provide headers
+ sudo /opt/local/bin/port select python3 python312
+ # Make macports install visible to subsequent steps
+ echo /opt/local/sbin >> "$GITHUB_PATH"
+ echo /opt/local/bin >> "$GITHUB_PATH"
+
+ - name: Configure
+ run: |
+ export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/"
+ meson setup \
+ ${MESON_COMMON_PG_CONFIG_ARGS} \
+ --buildtype=debug \
+ -Dextra_include_dirs=/opt/local/include \
+ -Dextra_lib_dirs=/opt/local/lib \
+ -Ddarwin_sysroot=none \
+ ${MESON_COMMON_FEATURES} \
+ ${MESON_FEATURES} \
+ build
+
+ - name: Build
+ run: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}
+
+ - name: Test world
+ run: |
+ ulimit -c unlimited # default is 0
+ ulimit -n 1024 # default is 256, pretty low
+ meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS}
+
+ - name: Core backtraces
+ if: failure()
+ run: src/tools/ci/cores_backtrace.sh macos "$HOME/cores"
+
+ - name: Upload logs
+ if: failure()
+ uses: actions/upload-artifact@v7
+ with:
+ name: macos-logs-${{ github.run_id }}
+ path: *log_paths
+ if-no-files-found: ignore
+
+
+ windows-vs:
+ name: Windows - VS - Meson & ninja
+ needs: [setup, sanity-check]
+ if: |
+ !cancelled() &&
+ needs.setup.outputs.windows == 'true' &&
+ needs.sanity-check.result != 'failure'
+ runs-on: windows-2022
+ timeout-minutes: 60
+ env:
+ TEST_JOBS: 8
+ # Avoid port conflicts between concurrent tap tests
+ PG_TEST_USE_UNIX_SOCKETS: 1
+ PG_REGRESS_SOCK_DIR: 'd:\pgsock'
+
+ MESON_FEATURES: >-
+ -Dcpp_args=/std:c++20
+ -Dauto_features=disabled
+ -Dtap_tests=enabled
+ -Dldap=enabled
+ -Dssl=openssl
+ -Dplperl=enabled
+ -Dplpython=enabled
+ TAR: "c:/windows/system32/tar.exe"
+
+ defaults:
+ run:
+ shell: cmd
+ steps:
+ - name: Disable Windows Defender
+ shell: powershell
+ run: |
+ Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable
+ # Verify Defender status
+ $status = Get-MpComputerStatus -ErrorAction SilentlyContinue
+ if ($status) {
+ Write-Host "RealTimeProtectionEnabled: $($status.RealTimeProtectionEnabled)"
+ Write-Host "AntivirusEnabled: $($status.AntivirusEnabled)"
+ }
+
+ - *checkout_step
+
+ - name: Sysinfo
+ run: |
+ chcp
+ systeminfo
+ set
+
+ # The TAP tests build an initdb template under build/tmp_install and
+ # then `robocopy` it into per-test data directories. Robocopy with the
+ # default /COPY:DAT flag doesn't copy ACLs — destinations inherit from
+ # their parent dir. On GitHub-hosted Windows runners the workspace's
+ # inherited ACL grants Administrators:(F) and Users:(RX) but does NOT
+ # grant the runner user (runneradmin) directly. That matters because
+ # pg_ctl on Windows uses CreateRestrictedProcess to drop admin
+ # privileges from postmaster, so the postmaster process has the user
+ # SID in its token but no longer the Administrators group — leaving it
+ # with only "Users:(RX)" on pg_control and friends, which causes
+ # "PANIC: could not open file global/pg_control: Permission denied".
+ #
+ # Fix it once on the workspace dir with (OI)(CI) inheritance flags so
+ # every file/dir created underneath gets an explicit grant for the
+ # current user.
+ - name: Grant workspace ACL to runner user
+ shell: pwsh
+ run: |
+ icacls "${{ github.workspace }}" /grant "${env:USERNAME}:(OI)(CI)F" /Q | Out-Null
+ Write-Host "Granted Full Control to $env:USERNAME on ${{ github.workspace }}"
+
+ # postgres' plpython3u loads python3.dll (the stable-ABI forwarder)
+ # which in turn loads whichever python3NN.dll the Windows loader finds
+ # first on PATH. On windows-2022 `C:\Program Files\Mercurial\` ships
+ # its own python3.dll + python39.dll and appears on PATH *before* the
+ # hostedtoolcache Python 3.12 — so without intervention the backend
+ # ends up running Python 3.9 while postgres' stdlib search uses 3.12,
+ # producing `ImportError: cannot import name 'text_encoding' from
+ # 'io'` (the 3.12 `io.py` calling into 3.9's `_io`).
+ #
+ # Drop Mercurial's directory from PATH so the hostedtoolcache
+ # python3.dll wins the DLL search.
+ - name: Remove Mercurial from PATH
+ shell: pwsh
+ run: |
+ $filtered = ($env:PATH -split ';' |
+ Where-Object { $_ -and ($_ -notmatch '\\Mercurial\\?$') }) -join ';'
+ Add-Content $env:GITHUB_ENV "PATH=$filtered"
+ Write-Host "Removed Mercurial entries from PATH"
+
+ - name: Install dependencies
+ shell: pwsh
+ run: |
+ choco install -y --no-progress --limitoutput diffutils winflexbison3
+ # meson + ninja aren't preinstalled on windows-2022. Install via pip
+ python -m pip install --upgrade meson ninja
+
+ # OpenSSL 1.1 via the slproweb installer (pinned to match the
+ # version used elsewhere in postgres CI).
+ curl.exe -fsSL -o openssl-setup.exe https://slproweb.com/download/Win64OpenSSL-1_1_1w.exe
+ Start-Process -Wait -FilePath ./openssl-setup.exe `
+ -ArgumentList '/DIR=d:\openssl\1.1\ /VERYSILENT /SP- /SUPPRESSMSGBOXES'
+ # The slproweb installer puts libcrypto-1_1-x64.dll / libssl-1_1-x64.dll
+ # in d:\openssl\1.1\bin\ and updates the system PATH. GH Actions
+ # snapshots PATH at job start though, so the running job won't
+ # see those DLLs and initdb.exe would crash silently at runtime.
+ # Push the bin dir onto GITHUB_PATH so it persists for later steps.
+ Add-Content $env:GITHUB_PATH "d:\openssl\1.1\bin"
+
+ # Install IPC::Run.
+ # - recommends_policy=0 keeps cpan from pulling in IO::Tty / IO::Pty,
+ # which don't build on Windows ("This module requires a POSIX
+ # compliant system to work").
+ # - Pin to NJM/IPC-Run-20250809.0 because TODDR/IPC-Run-20260322.0
+ # broke postgres tap tests on Windows (changed pipe stdio
+ # handling). See upstream pg-vm-images commit ff5238afa3 and
+ # the thread at
+ # https://postgr.es/m/CAN55FZ06xanSbJdHe-CurjX_qNuBWZDEvS1kAk36L38YCtZXnw%40mail.gmail.com
+ "o conf recommends_policy 0`no conf commit`nnotest install NJM/IPC-Run-20250809.0.tar.gz" | cpan
+ perl -mIPC::Run -e 1
+
+ - name: Setup hosts file
+ shell: pwsh
+ run: |
+ Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.1 pg-loadbalancetest"
+ Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.2 pg-loadbalancetest"
+ Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.3 pg-loadbalancetest"
+
+ - name: Setup socket directory
+ shell: cmd
+ run: mkdir %PG_REGRESS_SOCK_DIR%
+
+ - name: Configure
+ run: |
+ call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
+ meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% %MESON_FEATURES% --buildtype debug -Db_pch=true -Dextra_lib_dirs=d:\openssl\1.1\lib -Dextra_include_dirs=d:\openssl\1.1\include -DTAR=%TAR% build
+
+ - name: Build
+ run: |
+ call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
+ ninja -C build %MBUILD_TARGET%
+ ninja -C build -t missingdeps
+
+ - name: Test world
+ run: |
+ call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
+ meson test %MTEST_ARGS% --num-processes %TEST_JOBS%
+
+ # FIX: We need to collect crashlogs but they are not collected. cdb.exe
+ # is installed on the runner so it needs to be configured.
+ - name: Upload logs
+ if: failure()
+ uses: actions/upload-artifact@v7
+ with:
+ name: windows-vs-logs-${{ github.run_id }}
+ path: *log_paths
+ if-no-files-found: ignore
+
+
+ windows-mingw:
+ name: Windows - MinGW - Meson
+ needs: [setup, sanity-check]
+ if: |
+ !cancelled() &&
+ needs.setup.outputs.mingw == 'true' &&
+ needs.sanity-check.result != 'failure'
+ runs-on: windows-2022
+ timeout-minutes: 60
+ env:
+ TEST_JOBS: 4 # higher concurrency causes occasional failures
+ PG_TEST_USE_UNIX_SOCKETS: 1
+ PG_REGRESS_SOCK_DIR: 'd:\pgsock'
+ TAR: "c:/windows/system32/tar.exe"
+
+ MSYS: winjitdebug
+ CHERE_INVOKING: 1
+ MSYSTEM: UCRT64
+
+ # Keep -Dnls explicitly disabled, as the number of files it creates
+ # causes a noticeable slowdown.
+ MESON_FEATURES: >-
+ -Dnls=disabled
+
+ CCACHE_DIR: D:/a/ccache
+ CCACHE_MAXSIZE: "500M"
+ CCACHE_SLOPPINESS: pch_defines,time_macros
+ CCACHE_DEPEND: 1
+
+ defaults:
+ run:
+ shell: 'D:\msys64\usr\bin\bash.exe --login -eo pipefail "{0}"'
+
+ steps:
+ - name: Disable Windows Defender
+ shell: powershell
+ run: |
+ Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable
+ # Verify Defender status
+ $status = Get-MpComputerStatus -ErrorAction SilentlyContinue
+ if ($status) {
+ Write-Host "RealTimeProtectionEnabled: $($status.RealTimeProtectionEnabled)"
+ Write-Host "AntivirusEnabled: $($status.AntivirusEnabled)"
+ }
+
+ - *checkout_step
+
+ # Relocate the preinstalled MSYS2 tree from C:\ (slow system disk) to
+ # D:\ (faster ephemeral data disk). Every subsequent MSYS2 step uses
+ # D:\msys64\usr\bin\bash.exe via the job's `defaults.run.shell`.
+ #
+ # This reduces the total runtime of this task by ~15 minutes.
+ #
+ # robocopy returns 0-7 on success (with various "files copied" bits
+ # set) and 8+ on real failure, so we have to translate its exit code.
+ - name: Relocate MSYS2 to D
+ shell: powershell
+ run: |
+ robocopy C:\msys64 D:\msys64 /E /MT:16 /NJS /NJH /NFL /NDL /NP
+ if ($LASTEXITCODE -ge 8) { exit $LASTEXITCODE }
+ exit 0
+
+ - name: Setup MSYS2
+ run: |
+ # ${MINGW_PACKAGE_PREFIX} is an environment variable used in the
+ # MSYS2. It dynamically expands to the correct prefix for the active
+ # shell environment.
+ pacman -S --noconfirm --needed \
+ git bison flex make diffutils \
+ ${MINGW_PACKAGE_PREFIX}-ccache \
+ ${MINGW_PACKAGE_PREFIX}-gcc \
+ ${MINGW_PACKAGE_PREFIX}-icu \
+ ${MINGW_PACKAGE_PREFIX}-libbacktrace \
+ ${MINGW_PACKAGE_PREFIX}-libxml2 \
+ ${MINGW_PACKAGE_PREFIX}-libxslt \
+ ${MINGW_PACKAGE_PREFIX}-lz4 \
+ ${MINGW_PACKAGE_PREFIX}-make \
+ ${MINGW_PACKAGE_PREFIX}-meson \
+ ${MINGW_PACKAGE_PREFIX}-perl \
+ ${MINGW_PACKAGE_PREFIX}-pkg-config \
+ ${MINGW_PACKAGE_PREFIX}-readline \
+ ${MINGW_PACKAGE_PREFIX}-zlib
+
+ - name: Install additional dependencies
+ run: |
+ # Pin IPC::Run to NJM/IPC-Run-20250809.0; TODDR/IPC-Run-20260322.0
+ # broke postgres tap tests on Windows (pipe stdio handling).
+ # See pg-vm-images commit ff5238afa3.
+ (echo; echo o conf recommends_policy 0; echo notest install NJM/IPC-Run-20250809.0.tar.gz) | cpan
+ perl -mIPC::Run -e 1
+
+ - name: Setup socket directory
+ shell: cmd
+ run: mkdir %PG_REGRESS_SOCK_DIR%
+
+ - name: Restore ccache
+ uses: actions/cache@v5
+ with:
+ path: ${{ env.CCACHE_DIR }}
+ key: ccache-mingw-${{ github.ref_name }}-${{ github.run_id }}
+ restore-keys: |
+ ccache-mingw-${{ github.ref_name }}-
+ ccache-mingw-
+
+ - name: Configure
+ run: |
+ meson setup \
+ ${MESON_COMMON_PG_CONFIG_ARGS} \
+ -Ddebug=true -Doptimization=g -Db_pch=true \
+ ${MESON_COMMON_FEATURES} \
+ ${MESON_FEATURES} \
+ -DTAR=${TAR} \
+ build
+
+ - name: Build
+ run: ninja -C build ${MBUILD_TARGET}
+
+ - name: Test world
+ run: meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS}
+
+ # FIX: We need to collect crashlogs but they are not collected. cdb.exe
+ # is installed on the runner so it needs to be configured.
+ - name: Upload logs
+ if: failure()
+ uses: actions/upload-artifact@v7
+ with:
+ name: windows-mingw-logs-${{ github.run_id }}
+ path: *log_paths
+ if-no-files-found: ignore
+
+ # Test that code can be built with both gcc and clang without warnings,
+ # with various combinations of cassert/dtrace flags. Trace probes have
+ # a history of getting accidentally broken; the matrix is there to
+ # catch that.
+ #
+ # The autoconf cache files (gcc.cache / clang.cache) are intentionally
+ # reused across the matrix entries that share a compiler, so we don't
+ # pay for full feature detection on every entry.
+ compiler-warnings:
+ name: CompilerWarnings
+ needs: [setup, sanity-check]
+ if: |
+ !cancelled() &&
+ needs.setup.outputs.compilerwarnings == 'true' &&
+ needs.sanity-check.result != 'failure'
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+ container:
+ image: ${{ needs.setup.outputs.linux_ci_image }}
+ env:
+ BUILD_JOBS: 4
+ CCACHE_DIR: /tmp/ccache_dir
+ # Use larger ccache cache as this job compiles with multiple
+ # compilers / flag combinations.
+ CCACHE_MAXSIZE: "1G"
+ steps:
+ - *checkout_step
+
+ - name: Restore ccache
+ uses: actions/cache@v5
+ with:
+ path: ${{ env.CCACHE_DIR }}
+ key: ccache-compiler-warnings-${{ github.ref_name }}-${{ github.run_id }}
+ restore-keys: |
+ ccache-compiler-warnings-${{ github.ref_name }}-
+ ccache-compiler-warnings-
+
+ - name: Sysinfo
+ run: |
+ id
+ uname -a
+ cat /proc/cmdline
+ ulimit -a -H && ulimit -a -S
+ gcc -v
+ clang -v
+ env
+
+ - name: Setup workspace
+ run: |
+ echo "COPT=-Werror" > src/Makefile.custom
+ mkdir -p "$CCACHE_DIR"
+
+ # gcc, cassert off, dtrace on
+ - name: gcc warnings + (dtrace)
+ if: always()
+ run: |
+ ./configure \
+ --cache gcc.cache \
+ --enable-dtrace \
+ ${LINUX_CONFIGURE_FEATURES} \
+ CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
+ make -s -j${BUILD_JOBS} clean
+ make -s -j${BUILD_JOBS} world-bin
+
+ # gcc, cassert on, dtrace off
+ - name: gcc warnings + (cassert)
+ if: always()
+ run: |
+ ./configure \
+ --cache gcc.cache \
+ --enable-cassert \
+ ${LINUX_CONFIGURE_FEATURES} \
+ CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
+ make -s -j${BUILD_JOBS} clean
+ make -s -j${BUILD_JOBS} world-bin
+
+ # clang, cassert off, dtrace off
+ - name: clang warnings
+ if: always()
+ run: |
+ ./configure \
+ --cache clang.cache \
+ ${LINUX_CONFIGURE_FEATURES} \
+ CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang"
+ make -s -j${BUILD_JOBS} clean
+ make -s -j${BUILD_JOBS} world-bin
+
+ # clang, cassert on, dtrace on
+ - name: clang warnings + (cassert + dtrace)
+ if: always()
+ run: |
+ ./configure \
+ --cache clang.cache \
+ --enable-cassert \
+ --enable-dtrace \
+ ${LINUX_CONFIGURE_FEATURES} \
+ CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang"
+ make -s -j${BUILD_JOBS} clean
+ make -s -j${BUILD_JOBS} world-bin
+
+ - name: mingw warnings (cross compilation)
+ if: always()
+ run: |
+ ./configure \
+ --host=x86_64-w64-mingw32ucrt \
+ --enable-cassert \
+ --without-icu \
+ CC="ccache x86_64-w64-mingw32ucrt-gcc" \
+ CXX="ccache x86_64-w64-mingw32ucrt-g++"
+ make -s -j${BUILD_JOBS} clean
+ make -s -j${BUILD_JOBS} world-bin
+
+ ###
+ # Verify docs can be built
+ ###
+ # XXX: Only do this if there have been changes in doc/ since last build
+ - name: Build documentation
+ if: always()
+ run: |
+ ./configure \
+ --cache gcc.cache \
+ CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
+ make -s -j${BUILD_JOBS} clean
+ make -s -j${BUILD_JOBS} -C doc
+
+ ###
+ # Verify headerscheck / cpluspluscheck succeed
+ #
+ # - Run both in same script to increase parallelism, use -k to get
+ # result of both
+ # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose
+ ###
+ - name: headerscheck + cpluspluscheck
+ if: always()
+ run: |
+ ./configure \
+ ${LINUX_CONFIGURE_FEATURES} \
+ --cache gcc.cache \
+ --quiet \
+ CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
+ make -s -j${BUILD_JOBS} clean
+ make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10'
diff --git a/src/tools/ci/ci_macports_packages.sh b/src/tools/ci/ci_macports_packages.sh
index 63e97b37c78..18a06f96119 100755
--- a/src/tools/ci/ci_macports_packages.sh
+++ b/src/tools/ci/ci_macports_packages.sh
@@ -20,13 +20,26 @@ echo "macOS major version: $macos_major_version"
# macOS release.
macports_release_list_url="https://api.github.com/repos/macports/macports-base/releases";
macports_version_pattern="2\.10\.1"
-macports_url="$( curl -s $macports_release_list_url | grep "\"https://github.com/macports/macports-base/releases/download/v$macports_version_pattern/MacPorts-$macports_version_pattern-$macos_major_version-[A-Za-z]*\.pkg\""; | sed 's/.*: "//;s/".*//' | head -1 )"
+# Authenticate the GitHub API request when a token is available (e.g. on
+# GitHub Actions). Unauthenticated requests share a 60/h/IP rate limit
+# with every other job on the runner's IP and frequently return an error
+# JSON, leaving $macports_url empty and breaking the subsequent curl.
+auth_header=""
+if [ -n "$GITHUB_TOKEN" ]; then
+ auth_header="Authorization: Bearer $GITHUB_TOKEN"
+fi
+macports_url="$( curl -fsSL ${auth_header:+-H "$auth_header"} "$macports_release_list_url" | grep "\"https://github.com/macports/macports-base/releases/download/v$macports_version_pattern/MacPorts-$macports_version_pattern-$macos_major_version-[A-Za-z]*\.pkg\""; | sed 's/.*: "//;s/".*//' | head -1 )"
echo "MacPorts package URL: $macports_url"
+if [ -z "$macports_url" ]; then
+ echo "error: could not determine MacPorts package URL for macOS $macos_major_version (version pattern: $macports_version_pattern)" 1>&2
+ exit 1
+fi
+
cache_dmg="macports.hfs.dmg"
-if [ "$CIRRUS_CI" != "true" ]; then
- echo "expect to be called within cirrus-ci" 1>2
+if [ "$CIRRUS_CI" != "true" ] && [ "$GITHUB_ACTIONS" != "true" ]; then
+ echo "expect to be called within cirrus-ci or github actions" 1>2
exit 1
fi
--
2.54.0.380.gc69baaf57b
>From 828954a35ae25ba9331834eeb19ce42a3ed17a3f Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Mon, 1 Jun 2026 15:09:49 -0400
Subject: [PATCH v6a 2/5] gha: Andres' revisions
---
.github/workflows/postgresql-ci.yml | 834 +++++++++++++++-------------
1 file changed, 447 insertions(+), 387 deletions(-)
diff --git a/.github/workflows/postgresql-ci.yml b/.github/workflows/postgresql-ci.yml
index a7ef0bee94d..e2795ca0ffb 100644
--- a/.github/workflows/postgresql-ci.yml
+++ b/.github/workflows/postgresql-ci.yml
@@ -4,6 +4,7 @@ name: GitHub Actions CI
on:
push:
+ # FIXME: Should we also run on PRs?
# Restrict GITHUB_TOKEN to the minimum the jobs need: reading repo
# contents during checkout.
@@ -13,6 +14,7 @@ permissions:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
# Never cancel in-progress runs on master to ensure all commits are tested.
+ # FIXME: Should also not cancel REL_XY_STABLE
cancel-in-progress: ${{ github.ref != 'refs/heads/master' }}
env:
@@ -20,9 +22,19 @@ env:
# concurrent jobs and retrying older runs have a chance of working.
CLONE_DEPTH: 500
+ # At the moment all jobs use 4vcore runners, and none seems to benefit from
+ # increasing concurrency further.
+ BUILD_JOBS: 4
+
+ # It's possible that some jobs benefit from an increased test concurrency,
+ # but a default of 4 is a safe bet. Individual jobs can override.
+ TEST_JOBS: 4
+
CCACHE_MAXSIZE: "250M"
+ CCACHE_DIR: ${{ github.workspace }}/ccache_dir
- # check target for the autoconf builds
+ # Check target for the autoconf builds. Can be set to e.g. check to only
+ # only test the main regression tests.
CHECK: check-world PROVE_FLAGS=--timer
CHECKFLAGS: -Otarget
@@ -30,6 +42,11 @@ env:
# errors/warnings in one place.
MBUILD_TARGET: all testprep
MTEST_ARGS: --print-errorlogs --no-rebuild -C build
+
+ # Can be set to a non-empty value to run a limited set of tests
+ # (e.g. --suite regress to only run the main regression tests).
+ MTEST_TARGET:
+
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${{ github.workspace }}/src/tools/ci/pg_ci_base.conf
PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth
@@ -79,20 +96,16 @@ env:
--with-uuid=ossp
--with-zstd
- # Debian Trixie container image used by all Linux jobs. Built by
+ # Debian Trixie containers used by all Linux jobs. Built by
# 'https://github.com/anarazel/pg-vm-images/'.
- LINUX_CI_IMAGE: us-docker.pkg.dev/pg-ci-images/ci/linux_debian_trixie_ci:latest
+ CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/gha_main
+ CONTAINER_LINUX_CI: linux_debian_trixie_ci:latest
+ CONTAINER_LINUX_CI_DOCS: linux_debian_trixie_ci_docs:latest
# The full set of OS / job selectors recognized by the `ci-os-only:`
# commit-message directive parsed in the `setup` job below.
CI_OS_ONLY_JOBS: "linux macos windows mingw compilerwarnings sanitycheck"
- _LOG_PATHS: &log_paths |
- build*/testrun/**/*.log
- build*/testrun/**/*.diffs
- build*/testrun/**/regress_log_*
- build*/meson-logs/*.txt
-
jobs:
# Parse "ci-os-only: ..." from the commit message and expose flags
@@ -111,14 +124,26 @@ jobs:
# Re-export workflow-level env vars that other jobs need to reference
# from contexts (e.g. `jobs.<id>.container.image`) where the `env`
# context is not available.
- linux_ci_image: ${{ env.LINUX_CI_IMAGE }}
+ container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }}
+ container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }}
steps:
+ # Anchor reused by other jobs further down. GitHub Actions supports YAML
+ # anchors/aliases but not merge keys, so the alias copies the whole step
+ # verbatim. The anchor is resolved at YAML parse time, so the alias
+ # keeps working even if this job were to be skipped at runtime.
+ - &nix_sysinfo_step
+ name: sysinfo
+ run: |
+ id
+ uname -a
+ ulimit -a -H && ulimit -a -S
+ env
+
- id: os
env:
MSG: ${{ github.event.head_commit.message }}
shell: bash
run: |
- set -e
all_os=${CI_OS_ONLY_JOBS}
if printf '%s\n' "$MSG" | grep -qE '^ci-os-only: '; then
sel=$(printf '%s\n' "$MSG" | sed -n 's/^ci-os-only: //p' | head -n 1)
@@ -145,199 +170,22 @@ jobs:
sanity-check:
name: SanityCheck
needs: setup
- if: needs.setup.outputs.sanitycheck == 'true'
+ if: |
+ !cancelled() &&
+ needs.setup.outputs.sanitycheck == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
- container:
- image: ${{ needs.setup.outputs.linux_ci_image }}
+ container: &linux_ci_container
+ image: ${{ needs.setup.outputs.container_linux_ci }}
+
+ # Options passed to all linux containers. Not all of the jobs need
+ # all of them, but it's easier to just define them centrally.
+ #
# --privileged is needed so the prepare step can write to sysctls
# under /proc/sys (it's mounted read-only without it). We use it to
- # set kernel.core_pattern.
- options: --privileged
- env:
- BUILD_JOBS: 8
- TEST_JOBS: 8
- CCACHE_DIR: ${{ github.workspace }}/ccache_dir
- # no options enabled, should be small
- CCACHE_MAXSIZE: "150M"
- steps:
- # Anchor reused by other jobs further down. GitHub Actions supports
- # YAML anchors/aliases but not merge keys, so the alias copies the
- # whole step verbatim. The anchor is resolved at YAML parse time, so the
- # alias keeps working even if this job is skipped at runtime.
- - &checkout_step
- uses: actions/checkout@v6
- with:
- fetch-depth: ${{ env.CLONE_DEPTH }}
-
- - name: Restore ccache
- uses: actions/cache@v5
- with:
- path: ${{ env.CCACHE_DIR }}
- key: ccache-sanitycheck-${{ github.ref_name }}-${{ github.run_id }}
- restore-keys: |
- ccache-sanitycheck-${{ github.ref_name }}-
- ccache-sanitycheck-
-
- - name: Prepare workspace
- run: |
- whoami
- useradd -m postgres
- chown -R postgres:postgres .
- mkdir -p "$CCACHE_DIR"
- chown -R postgres:postgres "$CCACHE_DIR"
-
- - name: Configure
- run: |
- su postgres <<-'EOF'
- set -e
- meson setup \
- --buildtype=debug \
- --auto-features=disabled \
- -Ddefault_library=shared \
- -Dtap_tests=enabled \
- build
- EOF
-
- - name: Build
- run: |
- su postgres <<EOF
- set -e
- ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}
- EOF
-
- # Run a minimal set of tests. The main regression tests take too long
- # for this purpose. For now this is a random quick pg_regress style
- # test, and a tap test that exercises both a frontend binary and the
- # backend.
- - name: Test
- run: |
- su postgres <<EOF
- set -e
- ulimit -c unlimited
- meson test ${MTEST_ARGS} --suite setup
- meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS} \
- cube/regress pg_ctl/001_start_stop
- EOF
-
- - name: Core backtraces
- if: failure()
- run: |
- mkdir -m 770 /tmp/cores
- find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \;
- src/tools/ci/cores_backtrace.sh linux /tmp/cores
-
- - name: Upload logs
- if: failure()
- uses: actions/upload-artifact@v7
- with:
- name: sanitycheck-logs-${{ github.run_id }}
- path: *log_paths
- if-no-files-found: ignore
-
-
- # Build & test postgres on Linux in three configurations.
- #
- # Autoconf:
- # - Uses address sanitizer (sanitizer failures are typically printed in
- # the server log)
- # - Configures postgres with a small segment size
- # - Uses PG_TEST_PG_COMBINEBACKUP_MODE=--copy-file-range
- #
- # Meson:
- # - Test both 64- and 32-bit builds
- # - Uses undefined behaviour and alignment sanitizers, (sanitizer failures
- # are typically printed in the server log)
- # - Uses io_method=io_uring
- # - Uses meson feature autodetection
- # - 32-bit build tests with LANG=C to give ICU some buildfarm-uncovered
- # coverage. Also, newer Python insists on changing LC_CTYPE away from C,
- # prevent that with PYTHONCOERCECLOCALE.
- #
- # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes
- # print_stacktraces=1,verbosity=2, duh
- # detect_leaks=0: too many uninteresting leak errors in short-lived binaries
- linux:
- name: Linux - ${{ matrix.name }}
- needs: [setup, sanity-check]
- if: |
- !cancelled() &&
- needs.setup.outputs.linux == 'true' &&
- needs.sanity-check.result != 'failure'
- runs-on: ubuntu-latest
- timeout-minutes: 60
- strategy:
- fail-fast: false
- matrix:
- include:
- - name: Autoconf
- slug: autoconf
- cc: ccache gcc
- cxx: ccache g++
- sanitizer_flags: -fsanitize=address
- pg_test_pg_combinebackup_mode: '--copy-file-range'
- configure: |
- ./configure \
- --enable-cassert --enable-injection-points --enable-debug \
- --enable-tap-tests --enable-nls \
- --with-segsize-blocks=6 \
- --with-libnuma \
- --with-liburing \
- ${LINUX_CONFIGURE_FEATURES} \
- CLANG="ccache clang"
- build: |
- make -s -j${BUILD_JOBS} world-bin
- test: |
- make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS}
- logs_paths: |
- **/*.log
- **/*.diffs
- **/regress_log_*
-
- - name: Meson (64-bit)
- slug: meson-64
- cc: ccache gcc
- cxx: ccache g++
- sanitizer_flags: -fsanitize=alignment,undefined
- pg_test_initdb_extra_opts: '-c io_method=io_uring'
- configure: |
- meson setup \
- ${MESON_COMMON_PG_CONFIG_ARGS} \
- -Duuid=e2fs \
- --buildtype=debug \
- -Dllvm=enabled \
- build
- build: |
- ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}
- ninja -C build -t missingdeps
- test: |
- meson test ${MTEST_ARGS} -C build --num-processes ${TEST_JOBS}
- logs_paths: *log_paths
-
- - name: Meson (32-bit)
- slug: meson-32
- cc: ccache gcc -m32
- cxx: ccache g++ -m32
- sanitizer_flags: -fsanitize=alignment,undefined
- pg_test_initdb_extra_opts: '-c io_method=io_uring'
- configure: |
- meson setup \
- ${MESON_COMMON_PG_CONFIG_ARGS} \
- -Duuid=e2fs \
- --buildtype=debug \
- --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
- -DPERL=perl5.40-i386-linux-gnu \
- -Dlibnuma=disabled \
- build
- build: |
- ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}
- ninja -C build -t missingdeps
- test: |
- PYTHONCOERCECLOCALE=0 LANG=C \
- meson test ${MTEST_ARGS} -C build --num-processes ${TEST_JOBS}
- logs_paths: *log_paths
- container:
- image: ${{ needs.setup.outputs.linux_ci_image }}
+ # set kernel.core_pattern and (for the meson entries) to flip
+ # kernel.io_uring_disabled (default 2 on recent GH runner kernels).
+ #
# Share the host PID + IPC namespaces. 017_shm.pl rapidly creates,
# kill9's, and restarts postgres; with the container's small PID
# space a new postgres can recycle the dead postmaster's PID before
@@ -347,46 +195,36 @@ jobs:
#
# --ulimit raises memlock and core dump size. Memlock is needed for
# running the AIO tests.
- #
- # --privileged is needed so the prepare step can write to sysctls
- # under /proc/sys (it's mounted read-only without it). We use it to
- # set kernel.core_pattern and (for the meson entries) to flip
- # kernel.io_uring_disabled (default 2 on recent GH runner kernels).
- options: --pid=host --ipc=host --ulimit memlock=-1:-1 --privileged
+ options: &linux_container_options |
+ --privileged --pid=host --ipc=host --ulimit memlock=-1:-1
env:
- BUILD_JOBS: 4
- TEST_JOBS: 8
- CCACHE_DIR: /tmp/ccache_dir
- DEBUGINFOD_URLS: "https://debuginfod.debian.net";
-
- UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2
- ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0:detect_stack_use_after_return=0
- CFLAGS: -Og -ggdb -fno-sanitize-recover=all ${{ matrix.sanitizer_flags }}
- CXXFLAGS: -Og -ggdb -fno-sanitize-recover=all ${{ matrix.sanitizer_flags }}
- LDFLAGS: ${{ matrix.sanitizer_flags }}
- CC: ${{ matrix.cc }}
- CXX: ${{ matrix.cxx }}
-
- PG_TEST_INITDB_EXTRA_OPTS: ${{ matrix.pg_test_initdb_extra_opts }}
- PG_TEST_PG_COMBINEBACKUP_MODE: ${{ matrix.pg_test_pg_combinebackup_mode }}
+ # no options enabled, should be small
+ CCACHE_MAXSIZE: "150M"
steps:
- - *checkout_step
+ - *nix_sysinfo_step
- - name: Restore ccache
- uses: actions/cache@v5
+ - &checkout_step
+ uses: actions/checkout@v6
+ with:
+ fetch-depth: ${{ env.CLONE_DEPTH }}
+
+ - &ccache_restore_step
+ name: Restore ccache
+ id: ccache_restore
+ uses: actions/cache/restore@v5
with:
path: ${{ env.CCACHE_DIR }}
- key: ccache-linux-${{ matrix.slug }}-${{ github.ref_name }}-${{ github.run_id }}
+ key: &ccache_key |
+ ccache-${{ github.job }}-${{ github.ref_name }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
- ccache-linux-${{ matrix.slug }}-${{ github.ref_name }}-
- ccache-linux-${{ matrix.slug }}-
+ ccache-${{ github.job }}-${{ github.ref_name }}-
+ ccache-${{ github.job }}-
- - name: Prepare workspace
+ - &linux_prepare_workspace
+ name: Prepare workspace
run: |
useradd -m postgres
chown -R postgres:postgres .
- mkdir -p "$CCACHE_DIR"
- chown -R postgres:postgres "$CCACHE_DIR"
mkdir -m 770 /tmp/cores
chown root:postgres /tmp/cores
sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core'
@@ -400,41 +238,283 @@ jobs:
127.0.0.3 pg-loadbalancetest
EOF
+ # By using a shell that includes su, the run commands themselves get
+ # simpler. As there are quite a few commands that need to use su...
- name: Configure
+ shell: &su_postgres_shell |
+ su postgres -c "bash --noprofile --norc -eo pipefail {0}"
run: |
- su postgres <<EOF
- set -e
- ${{ matrix.configure }}
- EOF
+ meson setup \
+ --buildtype=debug \
+ --auto-features=disabled \
+ -Ddefault_library=shared \
+ -Dtap_tests=enabled \
+ build
- name: Build
- run: |
- su postgres <<EOF
- set -e
- ${{ matrix.build }}
- EOF
+ shell: *su_postgres_shell
+ run: &ninja_build_command |
+ ninja -C build -j${{env.BUILD_JOBS}} ${{env.MBUILD_TARGET}}
+ ninja -C build -t missingdeps
- - name: Test world
- run: |
- su postgres <<EOF
- set -e
- ulimit -c unlimited
- ${{ matrix.test }}
- EOF
+ # FIXME: As long as we use per-run ccache caches, we should probably add
+ # a step that checks if there is sufficient new content to warrant
+ # saving the new cache.
+ - &ccache_save_step
+ name: Save ccache
+ uses: actions/cache/save@v5
+ with:
+ path: ${{ env.CCACHE_DIR }}
+ key: ${{ steps.ccache_restore.outputs.cache-primary-key }}
- - name: Core backtraces
- if: failure()
+ # Run a minimal set of tests. The main regression tests take too long
+ # for this purpose. For now this is a random quick pg_regress style
+ # test, and a tap test that exercises both a frontend binary and the
+ # backend.
+ #
+ # To allow the command below to be reused by later tasks, we allow
+ # adding "setup" commands to be specified via the ADDITIONAL_SETUP
+ # environment variable.
+ #
+ # Note that this command is used on all platforms, therefore one needs
+ # to be careful about using only ${{env.}} variable references,
+ # linebreaks etc.
+ - name: Test
+ shell: *su_postgres_shell
+ env:
+ MTEST_TARGET: cube/regress pg_ctl/001_start_stop
+ run: &meson_test_world_cmd |
+ ${{case(runner.os == 'Windows', '', 'ulimit -c unlimited')}}
+
+ ${{env.ADDITIONAL_SETUP}}
+
+ echo ::group::test_setup
+ meson test ${{env.MTEST_ARGS}} --suite setup --logbase setup
+ echo ::endgroup::
+
+ meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} ${{env.MTEST_TARGET}}
+
+ - &linux_collect_cores
+ name: Core backtraces
+ if: failure() && !cancelled()
run: src/tools/ci/cores_backtrace.sh linux /tmp/cores
- - name: Upload logs
- if: failure()
+ # Note that this is used for both meson and autoconf builds
+ - &upload_logs_step
+ name: Upload logs
+ if: failure() && !cancelled()
uses: actions/upload-artifact@v7
with:
- name: linux-${{ matrix.slug }}-logs-${{ github.run_id }}
- path: ${{ matrix.logs_paths }}
+ name: logs-${{ github.job }}-${{ github.run_id }}-${{ github.run_attempt }}
+ path: |
+ **/*.log
+ **/*.diffs
+ **/regress_log_*
+ **/crashlog-*.txt
if-no-files-found: ignore
+ # Linux, Autoconf
+ #
+ # SPECIAL:
+ # - Uses address sanitizer (sanitizer failures are typically printed in
+ # the server log)
+ # - Configures postgres with a small segment size
+ # - Uses PG_TEST_PG_COMBINEBACKUP_MODE=--copy-file-range
+ linux-autoconf:
+ name: Linux - Autoconf
+ needs: [setup, sanity-check]
+ if: &linux_job_if |
+ !cancelled() &&
+ needs.setup.outputs.linux == 'true' &&
+ needs.sanity-check.result != 'failure'
+ runs-on: ubuntu-latest
+ container: *linux_ci_container
+ timeout-minutes: 60
+
+ env: &linux_env
+ # Add both debian and linux, as symbols from the host can be visible during profiling
+ DEBUGINFOD_URLS: "https://debuginfod.debian.net https://debuginfod.ubuntu.com";
+ # Use -O2 to reduce the test times, use -fno-sanitize-recover=all to make sanitizer test
+ # failures visible.
+ CFLAGS: -O2 -ggdb -fno-sanitize-recover=all
+ CXXFLAGS: -O2 -ggdb -fno-sanitize-recover=all
+ LDFLAGS:
+ CC: ccache gcc
+ CXX: ccache g++
+ CLANG: ccache clang
+
+ # Configure sanitizer runtime behavior to be suitable for running tests:
+ # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes
+ # print_stacktraces=1,verbosity=2, duh
+ # detect_leaks=0: too many uninteresting leak errors in short-lived binaries
+ UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2
+ ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0
+
+ steps:
+ # GitHub Actions does not make it easy to share some, but not all,
+ # environment variables between related tasks. We solve that for the
+ # linux- tasks by updating the environment variables programmatically.
+ - name: Update Environment
+ env:
+ SANITIZER_FLAGS: -fsanitize=address
+ PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range
+ run: &linux_update_config_cmd |
+ echo "CFLAGS=$CFLAGS ${SANITIZER_FLAGS}" >> "$GITHUB_ENV"
+ echo "CXXFLAGS=$CXXFLAGS ${SANITIZER_FLAGS}" >> "$GITHUB_ENV"
+ echo "LDFLAGS=$LDFLAGS ${SANITIZER_FLAGS}" >> "$GITHUB_ENV"
+
+ echo "CC=${CC}" >> "$GITHUB_ENV"
+ echo "CXX=${CXX}" >> "$GITHUB_ENV"
+
+ echo "PG_TEST_INITDB_EXTRA_OPTS=${PG_TEST_INITDB_EXTRA_OPTS}" >> "$GITHUB_ENV"
+ echo "PG_TEST_PG_COMBINEBACKUP_MODE=${PG_TEST_PG_COMBINEBACKUP_MODE}" >> "$GITHUB_ENV"
+
+ - *nix_sysinfo_step
+ - *checkout_step
+ - *ccache_restore_step
+ - *linux_prepare_workspace
+
+ - name: Configure
+ shell: *su_postgres_shell
+ run: |
+ ./configure \
+ --enable-cassert --enable-injection-points --enable-debug \
+ --enable-tap-tests --enable-nls \
+ --with-segsize-blocks=6 \
+ --with-libnuma \
+ --with-liburing \
+ ${LINUX_CONFIGURE_FEATURES}
+
+ - name: Build
+ shell: *su_postgres_shell
+ run: |
+ make -s -j${BUILD_JOBS} world-bin
+
+ - *ccache_save_step
+
+ - name: Test world
+ shell: *su_postgres_shell
+ run: |
+ make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS}
+
+ - *linux_collect_cores
+ - *upload_logs_step
+
+
+ # Linux Meson, 32 bit
+ #
+ # SPECIAL:
+ # - Uses undefined behaviour and alignment sanitizers, (sanitizer failures
+ # are typically printed in the server log)
+ # - Uses io_method=io_uring
+ # - Uses meson feature autodetection
+ # - tests with LANG=C to give ICU some buildfarm-uncovered coverage. Also,
+ # newer Python insists on changing LC_CTYPE away from C, prevent that with
+ # PYTHONCOERCECLOCALE.
+ linux-meson-32:
+ name: Linux - Meson (32-bit)
+ needs: [setup, sanity-check]
+ if: *linux_job_if
+ runs-on: ubuntu-latest
+ container: *linux_ci_container
+ timeout-minutes: 60
+ env: *linux_env
+
+ steps:
+ - name: Update Environment
+ env:
+ SANITIZER_FLAGS: -fsanitize=alignment,undefined
+ PG_TEST_INITDB_EXTRA_OPTS: -c io_method=io_uring
+ CC: ccache gcc -m32
+ CXX: ccache g++ -m32
+ run: *linux_update_config_cmd
+
+ - *nix_sysinfo_step
+ - *checkout_step
+ - *ccache_restore_step
+ - *linux_prepare_workspace
+
+ - name: Configure
+ shell: *su_postgres_shell
+ run: |
+ meson setup \
+ ${MESON_COMMON_PG_CONFIG_ARGS} \
+ -Duuid=e2fs \
+ --buildtype=debug \
+ --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
+ -DPERL=perl5.40-i386-linux-gnu \
+ -Dlibnuma=disabled \
+ build
+
+ - name: Build
+ shell: *su_postgres_shell
+ run: *ninja_build_command
+
+ - *ccache_save_step
+
+ - name: Test world
+ shell: *su_postgres_shell
+ env:
+ PYTHONCOERCECLOCALE: 0
+ LANG: C
+ run: *meson_test_world_cmd
+
+ - *linux_collect_cores
+ - *upload_logs_step
+
+ # Linux Meson, 64 bit
+ #
+ # SPECIAL:
+ # - Uses undefined behaviour and alignment sanitizers, (sanitizer failures
+ # are typically printed in the server log)
+ # - Uses io_method=io_uring
+ # - Uses meson feature autodetection
+ linux-meson-64:
+ name: Linux - Meson (64-bit)
+ needs: [setup, sanity-check]
+ if: *linux_job_if
+ runs-on: ubuntu-latest
+ container: *linux_ci_container
+ timeout-minutes: 60
+ env: *linux_env
+
+ steps:
+ - name: Update Environment
+ env:
+ SANITIZER_FLAGS: -fsanitize=alignment,undefined
+ PG_TEST_INITDB_EXTRA_OPTS: -c io_method=io_uring
+ run: *linux_update_config_cmd
+
+ - *nix_sysinfo_step
+ - *checkout_step
+ - *ccache_restore_step
+ - *linux_prepare_workspace
+
+ - name: Configure
+ shell: *su_postgres_shell
+ run: |
+ meson setup \
+ ${MESON_COMMON_PG_CONFIG_ARGS} \
+ -Duuid=e2fs \
+ --buildtype=debug \
+ -Dllvm=enabled \
+ build
+
+ - name: Build
+ shell: *su_postgres_shell
+ run: *ninja_build_command
+
+ - *ccache_save_step
+
+ - name: Test world
+ shell: *su_postgres_shell
+ run: *meson_test_world_cmd
+
+ - *linux_collect_cores
+ - *upload_logs_step
+
+
# SPECIAL:
# - Enables --clone for pg_upgrade and pg_combinebackup
# - Specifies configuration options that test reading/writing/copying of node trees
@@ -449,13 +529,6 @@ jobs:
runs-on: macos-15
timeout-minutes: 60
env:
- BUILD_JOBS: 4
- # Test performance regresses noticeably when using all cores. 8 works OK.
- # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de
- # Fix: Needs to be re-tested for GitHub Actions.
- TEST_JOBS: 8
-
- CCACHE_DIR: ${{ github.workspace }}/ccache_dir
MACPORTS_CACHE: ${{ github.workspace }}/macports-cache
MESON_FEATURES: >-
@@ -497,44 +570,28 @@ jobs:
-c debug_parallel_query=regress
steps:
- - *checkout_step
+ - *nix_sysinfo_step
- - name: Sysinfo
- run: |
- id
- uname -a
- ulimit -a -H && ulimit -a -S
- env
+ - *checkout_step
- name: Setup core files
run: |
mkdir -p $HOME/cores
sudo sysctl kern.corefile="$HOME/cores/core.%P"
- - name: Restore ccache
- uses: actions/cache@v5
- with:
- path: ${{ env.CCACHE_DIR }}
- key: ccache-macos-${{ github.ref_name }}-${{ github.run_id }}
- restore-keys: |
- ccache-macos-${{ github.ref_name }}-
- ccache-macos-
-
- - name: Compute MacPorts cache key
+ - name: "Macports: Compute cache key"
id: mpkey
run: |
macos_major=$(sw_vers -productVersion | sed 's/\..*//')
pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q)
script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh)
- echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}-${GITHUB_RUN_ID}" >> "$GITHUB_OUTPUT"
- echo "restore-key=macports-${macos_major}-${pkglist_hash}-${script_hash}-" >> "$GITHUB_OUTPUT"
+ echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT"
- - name: Restore MacPorts cache
+ - name: "MacPorts: Restore cache"
uses: actions/cache@v5
with:
path: ${{ env.MACPORTS_CACHE }}
key: ${{ steps.mpkey.outputs.key }}
- restore-keys: ${{ steps.mpkey.outputs.restore-key }}
# Use MacPorts, even though Homebrew is installed. The installation
# of the additional packages we need would take quite a while with
@@ -546,7 +603,7 @@ jobs:
# the large MacPort tree around to figure out that p5-io-tty is
# actually p5.34-io-tty. Using the unversioned name works, but
# updates MacPorts every time.
- - name: Install dependencies (MacPorts)
+ - name: "MacPorts: Install dependencies"
env:
# Pass token so the script's GitHub API call to list MacPorts
# releases isn't subject to the 60/h/IP unauthenticated rate
@@ -560,11 +617,14 @@ jobs:
echo /opt/local/sbin >> "$GITHUB_PATH"
echo /opt/local/bin >> "$GITHUB_PATH"
+ - *ccache_restore_step
+
- name: Configure
+ env:
+ PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/
run: |
- export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/"
meson setup \
- ${MESON_COMMON_PG_CONFIG_ARGS} \
+ ${{env.MESON_COMMON_PG_CONFIG_ARGS}} \
--buildtype=debug \
-Dextra_include_dirs=/opt/local/include \
-Dextra_lib_dirs=/opt/local/lib \
@@ -574,25 +634,21 @@ jobs:
build
- name: Build
- run: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}
+ run: *ninja_build_command
+
+ - *ccache_save_step
- name: Test world
- run: |
- ulimit -c unlimited # default is 0
- ulimit -n 1024 # default is 256, pretty low
- meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS}
+ env:
+ # default is 256, pretty low
+ ADDITIONAL_SETUP: ulimit -n 1024
+ run: *meson_test_world_cmd
- name: Core backtraces
- if: failure()
+ if: failure() && !cancelled()
run: src/tools/ci/cores_backtrace.sh macos "$HOME/cores"
- - name: Upload logs
- if: failure()
- uses: actions/upload-artifact@v7
- with:
- name: macos-logs-${{ github.run_id }}
- path: *log_paths
- if-no-files-found: ignore
+ - *upload_logs_step
windows-vs:
@@ -605,10 +661,10 @@ jobs:
runs-on: windows-2022
timeout-minutes: 60
env:
- TEST_JOBS: 8
# Avoid port conflicts between concurrent tap tests
PG_TEST_USE_UNIX_SOCKETS: 1
PG_REGRESS_SOCK_DIR: 'd:\pgsock'
+ TAR: "c:/windows/system32/tar.exe"
MESON_FEATURES: >-
-Dcpp_args=/std:c++20
@@ -618,13 +674,13 @@ jobs:
-Dssl=openssl
-Dplperl=enabled
-Dplpython=enabled
- TAR: "c:/windows/system32/tar.exe"
defaults:
run:
shell: cmd
steps:
- - name: Disable Windows Defender
+ - &windows_disable_defender
+ name: Disable Windows Defender
shell: powershell
run: |
Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable
@@ -723,33 +779,36 @@ jobs:
- name: Setup socket directory
shell: cmd
- run: mkdir %PG_REGRESS_SOCK_DIR%
+ run: mkdir ${{env.PG_REGRESS_SOCK_DIR}}
- name: Configure
run: |
call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
- meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% %MESON_FEATURES% --buildtype debug -Db_pch=true -Dextra_lib_dirs=d:\openssl\1.1\lib -Dextra_include_dirs=d:\openssl\1.1\include -DTAR=%TAR% build
+ meson setup ^
+ --backend ninja ^
+ ${{env.MESON_COMMON_PG_CONFIG_ARGS}} ^
+ ${{env.MESON_FEATURES}} ^
+ --buildtype debug ^
+ -Db_pch=true ^
+ -Dextra_lib_dirs=d:\openssl\1.1\lib -Dextra_include_dirs=d:\openssl\1.1\include ^
+ -DTAR=${{env.TAR}} ^
+ build
- name: Build
run: |
call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
- ninja -C build %MBUILD_TARGET%
+ ninja -C build ${{env.MBUILD_TARGET}}
ninja -C build -t missingdeps
- name: Test world
- run: |
- call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
- meson test %MTEST_ARGS% --num-processes %TEST_JOBS%
+ env:
+ ADDITIONAL_SETUP: |
+ call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
+ run: *meson_test_world_cmd
# FIX: We need to collect crashlogs but they are not collected. cdb.exe
# is installed on the runner so it needs to be configured.
- - name: Upload logs
- if: failure()
- uses: actions/upload-artifact@v7
- with:
- name: windows-vs-logs-${{ github.run_id }}
- path: *log_paths
- if-no-files-found: ignore
+ - *upload_logs_step
windows-mingw:
@@ -762,7 +821,7 @@ jobs:
runs-on: windows-2022
timeout-minutes: 60
env:
- TEST_JOBS: 4 # higher concurrency causes occasional failures
+ # Avoid port conflicts between concurrent tap tests
PG_TEST_USE_UNIX_SOCKETS: 1
PG_REGRESS_SOCK_DIR: 'd:\pgsock'
TAR: "c:/windows/system32/tar.exe"
@@ -776,7 +835,6 @@ jobs:
MESON_FEATURES: >-
-Dnls=disabled
- CCACHE_DIR: D:/a/ccache
CCACHE_MAXSIZE: "500M"
CCACHE_SLOPPINESS: pch_defines,time_macros
CCACHE_DEPEND: 1
@@ -786,17 +844,7 @@ jobs:
shell: 'D:\msys64\usr\bin\bash.exe --login -eo pipefail "{0}"'
steps:
- - name: Disable Windows Defender
- shell: powershell
- run: |
- Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable
- # Verify Defender status
- $status = Get-MpComputerStatus -ErrorAction SilentlyContinue
- if ($status) {
- Write-Host "RealTimeProtectionEnabled: $($status.RealTimeProtectionEnabled)"
- Write-Host "AntivirusEnabled: $($status.AntivirusEnabled)"
- }
-
+ - *windows_disable_defender
- *checkout_step
# Relocate the preinstalled MSYS2 tree from C:\ (slow system disk) to
@@ -835,6 +883,8 @@ jobs:
${MINGW_PACKAGE_PREFIX}-readline \
${MINGW_PACKAGE_PREFIX}-zlib
+ - *nix_sysinfo_step
+
- name: Install additional dependencies
run: |
# Pin IPC::Run to NJM/IPC-Run-20250809.0; TODDR/IPC-Run-20260322.0
@@ -845,42 +895,32 @@ jobs:
- name: Setup socket directory
shell: cmd
- run: mkdir %PG_REGRESS_SOCK_DIR%
+ run: mkdir ${{env.PG_REGRESS_SOCK_DIR}}
- - name: Restore ccache
- uses: actions/cache@v5
- with:
- path: ${{ env.CCACHE_DIR }}
- key: ccache-mingw-${{ github.ref_name }}-${{ github.run_id }}
- restore-keys: |
- ccache-mingw-${{ github.ref_name }}-
- ccache-mingw-
+ - *ccache_restore_step
- name: Configure
run: |
meson setup \
- ${MESON_COMMON_PG_CONFIG_ARGS} \
+ ${{env.MESON_COMMON_PG_CONFIG_ARGS}} \
-Ddebug=true -Doptimization=g -Db_pch=true \
- ${MESON_COMMON_FEATURES} \
- ${MESON_FEATURES} \
- -DTAR=${TAR} \
+ ${{env.MESON_COMMON_FEATURES}} \
+ ${{env.MESON_FEATURES}} \
+ -DTAR=${{env.TAR}} \
build
- name: Build
- run: ninja -C build ${MBUILD_TARGET}
+ run: *ninja_build_command
+
+ - *ccache_save_step
- name: Test world
- run: meson test ${MTEST_ARGS} --num-processes ${TEST_JOBS}
+ run: *meson_test_world_cmd
# FIX: We need to collect crashlogs but they are not collected. cdb.exe
# is installed on the runner so it needs to be configured.
- - name: Upload logs
- if: failure()
- uses: actions/upload-artifact@v7
- with:
- name: windows-mingw-logs-${{ github.run_id }}
- path: *log_paths
- if-no-files-found: ignore
+ - *upload_logs_step
+
# Test that code can be built with both gcc and clang without warnings,
# with various combinations of cassert/dtrace flags. Trace probes have
@@ -900,24 +940,12 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 60
container:
- image: ${{ needs.setup.outputs.linux_ci_image }}
+ image: ${{ needs.setup.outputs.container_linux_ci_docs }}
env:
- BUILD_JOBS: 4
- CCACHE_DIR: /tmp/ccache_dir
# Use larger ccache cache as this job compiles with multiple
# compilers / flag combinations.
CCACHE_MAXSIZE: "1G"
steps:
- - *checkout_step
-
- - name: Restore ccache
- uses: actions/cache@v5
- with:
- path: ${{ env.CCACHE_DIR }}
- key: ccache-compiler-warnings-${{ github.ref_name }}-${{ github.run_id }}
- restore-keys: |
- ccache-compiler-warnings-${{ github.ref_name }}-
- ccache-compiler-warnings-
- name: Sysinfo
run: |
@@ -929,83 +957,108 @@ jobs:
clang -v
env
+ - *checkout_step
+
+ - *ccache_restore_step
+
- name: Setup workspace
run: |
echo "COPT=-Werror" > src/Makefile.custom
- mkdir -p "$CCACHE_DIR"
# gcc, cassert off, dtrace on
- name: gcc warnings + (dtrace)
- if: always()
+ if: ${{ !cancelled() }}
run: |
+ echo "::group::configure"
./configure \
--cache gcc.cache \
--enable-dtrace \
- ${LINUX_CONFIGURE_FEATURES} \
+ ${{env.LINUX_CONFIGURE_FEATURES}} \
CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- make -s -j${BUILD_JOBS} world-bin
+ echo "::endgroup::"
+
+ make -s -j${{env.BUILD_JOBS}} clean
+ make -s -j${{env.BUILD_JOBS}} world-bin
+
# gcc, cassert on, dtrace off
- name: gcc warnings + (cassert)
- if: always()
+ if: ${{ !cancelled() }}
run: |
+ echo "::group::configure"
./configure \
--cache gcc.cache \
--enable-cassert \
- ${LINUX_CONFIGURE_FEATURES} \
+ ${{env.LINUX_CONFIGURE_FEATURES}} \
CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- make -s -j${BUILD_JOBS} world-bin
+ echo "::endgroup::"
+
+ make -s -j${{env.BUILD_JOBS}} clean
+ make -s -j${{env.BUILD_JOBS}} world-bin
# clang, cassert off, dtrace off
- name: clang warnings
- if: always()
+ if: ${{ !cancelled() }}
run: |
+ echo "::group::configure"
./configure \
--cache clang.cache \
- ${LINUX_CONFIGURE_FEATURES} \
+ ${{env.LINUX_CONFIGURE_FEATURES}} \
CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- make -s -j${BUILD_JOBS} world-bin
+ echo "::endgroup::"
+
+ make -s -j${{env.BUILD_JOBS}} clean
+ make -s -j${{env.BUILD_JOBS}} world-bin
+
# clang, cassert on, dtrace on
- name: clang warnings + (cassert + dtrace)
- if: always()
+ if: ${{ !cancelled() }}
run: |
+ echo "::group::configure"
./configure \
--cache clang.cache \
--enable-cassert \
--enable-dtrace \
- ${LINUX_CONFIGURE_FEATURES} \
+ ${{env.LINUX_CONFIGURE_FEATURES}} \
CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- make -s -j${BUILD_JOBS} world-bin
+ echo "::endgroup::"
+
+ make -s -j${{env.BUILD_JOBS}} clean
+ make -s -j${{env.BUILD_JOBS}} world-bin
+
- name: mingw warnings (cross compilation)
- if: always()
+ if: ${{ !cancelled() }}
run: |
+ echo "::group::configure"
./configure \
--host=x86_64-w64-mingw32ucrt \
--enable-cassert \
--without-icu \
CC="ccache x86_64-w64-mingw32ucrt-gcc" \
CXX="ccache x86_64-w64-mingw32ucrt-g++"
- make -s -j${BUILD_JOBS} clean
- make -s -j${BUILD_JOBS} world-bin
+ echo "::endgroup::"
+
+ make -s -j${{env.BUILD_JOBS}} clean
+ make -s -j${{env.BUILD_JOBS}} world-bin
+
###
# Verify docs can be built
###
# XXX: Only do this if there have been changes in doc/ since last build
- name: Build documentation
- if: always()
+ if: ${{ !cancelled() }}
run: |
+ echo "::group::configure"
./configure \
--cache gcc.cache \
CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- make -s -j${BUILD_JOBS} -C doc
+ echo "::endgroup::"
+
+ make -s -j${{env.BUILD_JOBS}} clean
+ make -s -j${{env.BUILD_JOBS}} -C doc
###
# Verify headerscheck / cpluspluscheck succeed
@@ -1015,12 +1068,19 @@ jobs:
# - Use -fmax-errors, as particularly cpluspluscheck can be very verbose
###
- name: headerscheck + cpluspluscheck
- if: always()
+ if: ${{ !cancelled() }}
run: |
+ echo "::group::configure"
./configure \
- ${LINUX_CONFIGURE_FEATURES} \
+ ${{env.LINUX_CONFIGURE_FEATURES}} \
--cache gcc.cache \
--quiet \
CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10'
+ echo "::endgroup::"
+
+ make -s -j${{env.BUILD_JOBS}} clean
+ make -s -j${{env.BUILD_JOBS}} -k ${{env.CHECKFLAGS}} \
+ headerscheck cpluspluscheck \
+ EXTRAFLAGS='-fmax-errors=10'
+
+ - *ccache_save_step
--
2.54.0.380.gc69baaf57b
>From d97c1280b85bcfe0ae4f46be1d4e91a1bc44335a Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Thu, 28 May 2026 13:31:41 -0400
Subject: [PATCH v6a 3/5] disable cirrus
Author:
Reviewed-by:
Discussion: https://postgr.es/m/
Backpatch-through:
---
.cirrus.yml | 91 ----
.cirrus.star | 143 -------
.cirrus.tasks.yml | 1022 ---------------------------------------------
3 files changed, 1256 deletions(-)
delete mode 100644 .cirrus.yml
delete mode 100644 .cirrus.star
delete mode 100644 .cirrus.tasks.yml
diff --git a/.cirrus.yml b/.cirrus.yml
deleted file mode 100644
index 3f75852e84e..00000000000
--- a/.cirrus.yml
+++ /dev/null
@@ -1,91 +0,0 @@
-# CI configuration file for CI utilizing cirrus-ci.org
-#
-# For instructions on how to enable the CI integration in a repository and
-# further details, see src/tools/ci/README
-#
-#
-# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute
-# resources for CI configurable on a repository level, the "final" CI
-# configuration is the combination of:
-#
-# 1) the contents of this file
-#
-# 2) computed environment variables
-#
-# Used to enable/disable tasks based on the execution environment. See
-# .cirrus.star: compute_environment_vars()
-#
-# 3) if defined, the contents of the file referenced by the, repository
-# level, REPO_CI_CONFIG_GIT_URL variable (see
-# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted
-# format)
-#
-# This allows running tasks in a different execution environment than the
-# default, e.g. to have sufficient resources for cfbot.
-#
-# 4) .cirrus.tasks.yml
-#
-# This composition is done by .cirrus.star
-
-
-env:
- # Source of images / containers
- GCP_PROJECT: pg-ci-images
- IMAGE_PROJECT: $GCP_PROJECT
- CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci
- DISK_SIZE: 25
-
-
-# Define how to run various types of tasks.
-
-# VMs provided by cirrus-ci. Each user has a limited number of "free" credits
-# for testing.
-cirrus_community_vm_template: &cirrus_community_vm_template
- compute_engine_instance:
- image_project: $IMAGE_PROJECT
- image: family/$IMAGE_FAMILY
- platform: $PLATFORM
- cpu: $CPUS
- disk: $DISK_SIZE
-
-
-default_linux_task_template: &linux_task_template
- env:
- PLATFORM: linux
- <<: *cirrus_community_vm_template
-
-
-default_freebsd_task_template: &freebsd_task_template
- env:
- PLATFORM: freebsd
- <<: *cirrus_community_vm_template
-
-default_netbsd_task_template: &netbsd_task_template
- env:
- PLATFORM: netbsd
- <<: *cirrus_community_vm_template
-
-default_openbsd_task_template: &openbsd_task_template
- env:
- PLATFORM: openbsd
- <<: *cirrus_community_vm_template
-
-
-default_windows_task_template: &windows_task_template
- env:
- PLATFORM: windows
- <<: *cirrus_community_vm_template
-
-
-# macos workers provided by cirrus-ci
-default_macos_task_template: &macos_task_template
- env:
- PLATFORM: macos
- macos_instance:
- image: $IMAGE
-
-
-# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here,
-# followed by the contents .cirrus.tasks.yml. This allows
-# REPO_CI_CONFIG_GIT_URL to override how the task types above will be
-# executed, e.g. using a custom compute account or permanent workers.
diff --git a/.cirrus.star b/.cirrus.star
deleted file mode 100644
index e9bb672b959..00000000000
--- a/.cirrus.star
+++ /dev/null
@@ -1,143 +0,0 @@
-"""Additional CI configuration, using the starlark language. See
-https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark
-
-See also the starlark specification at
-https://github.com/bazelbuild/starlark/blob/master/spec.md
-
-See also .cirrus.yml and src/tools/ci/README
-"""
-
-load("cirrus", "env", "fs", "re", "yaml")
-
-
-def main():
- """The main function is executed by cirrus-ci after loading .cirrus.yml and can
- extend the CI definition further.
-
- As documented in .cirrus.yml, the final CI configuration is composed of
-
- 1) the contents of .cirrus.yml
-
- 2) computed environment variables
-
- 3) if defined, the contents of the file referenced by the, repository
- level, REPO_CI_CONFIG_GIT_URL variable (see
- https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted
- format)
-
- 4) .cirrus.tasks.yml
- """
-
- output = ""
-
- # 1) is evaluated implicitly
-
-
- # Add 2)
- additional_env = compute_environment_vars()
- env_fmt = """
-###
-# Computed environment variables start here
-###
-{0}
-###
-# Computed environment variables end here
-###
-"""
- output += env_fmt.format(yaml.dumps({'env': additional_env}))
-
-
- # Add 3)
- repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL")
- if repo_config_url != None:
- print("loading additional configuration from \"{}\"".format(repo_config_url))
- output += config_from(repo_config_url)
- else:
- output += "\n# REPO_CI_CONFIG_URL was not set\n"
-
-
- # Add 4)
- output += config_from(".cirrus.tasks.yml")
-
-
- return output
-
-
-def compute_environment_vars():
- cenv = {}
-
- ###
- # Some tasks are manually triggered by default because they might use too
- # many resources for users of free Cirrus credits, but they can be
- # triggered automatically by naming them in an environment variable e.g.
- # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository
- # Settings" on Cirrus CI's website.
-
- default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd']
-
- repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '')
- for task in default_manual_trigger_tasks:
- name = 'CI_TRIGGER_TYPE_' + task.upper()
- if repo_ci_automatic_trigger_tasks.find(task) != -1:
- value = 'automatic'
- else:
- value = 'manual'
- cenv[name] = value
- ###
-
- ###
- # Parse "ci-os-only:" tag in commit message and set
- # CI_{$OS}_ENABLED variable for each OS
-
- # We want to disable SanityCheck if testing just a specific OS. This
- # shortens push-wait-for-ci cycle time a bit when debugging operating
- # system specific failures. Just treating it as an OS in that case
- # suffices.
-
- operating_systems = [
- 'compilerwarnings',
- 'freebsd',
- 'linux',
- 'macos',
- 'mingw',
- 'netbsd',
- 'openbsd',
- 'sanitycheck',
- 'windows',
- ]
- commit_message = env.get('CIRRUS_CHANGE_MESSAGE')
- match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)"
-
- # re.match() returns an array with a tuple of (matched-string, match_1, ...)
- m = re.match(match_re, commit_message)
- if m and len(m) > 0:
- os_only = m[0][2]
- os_only_list = re.split(r'[, ]+', os_only)
- else:
- os_only_list = operating_systems
-
- for os in operating_systems:
- os_enabled = os in os_only_list
- cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled
- ###
-
- return cenv
-
-
-def config_from(config_src):
- """return contents of config file `config_src`, surrounded by markers
- indicating start / end of the included file
- """
-
- config_contents = fs.read(config_src)
- config_fmt = """
-
-###
-# contents of config file `{0}` start here
-###
-{1}
-###
-# contents of config file `{0}` end here
-###
-"""
- return config_fmt.format(config_src, config_contents)
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
deleted file mode 100644
index 8683d1ae9c7..00000000000
--- a/.cirrus.tasks.yml
+++ /dev/null
@@ -1,1022 +0,0 @@
-# CI configuration file for CI utilizing cirrus-ci.org
-#
-# For instructions on how to enable the CI integration in a repository and
-# further details, see src/tools/ci/README
-#
-#
-# NB: Different tasks intentionally test with different, non-default,
-# configurations, to increase the chance of catching problems. Each task with
-# non-obvious non-default documents their oddity at the top of the task,
-# prefixed by "SPECIAL:".
-
-
-env:
- # The lower depth accelerates git clone. Use a bit of depth so that
- # concurrent tasks and retrying older jobs have a chance of working.
- CIRRUS_CLONE_DEPTH: 500
- # Useful to be able to analyse what in a script takes long
- CIRRUS_LOG_TIMESTAMP: true
-
- CCACHE_MAXSIZE: "250M"
-
- # target to test, for all but windows
- CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS
- CHECKFLAGS: -Otarget
- PROVE_FLAGS: --timer
- # Build test dependencies as part of the build step, to see compiler
- # errors/warnings in one place.
- MBUILD_TARGET: all testprep
- MTEST_ARGS: --print-errorlogs --no-rebuild -C build
- PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
- TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
- PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth
-
- # Postgres config args for the meson builds, shared between all meson tasks
- # except the 'SanityCheck' task
- MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true
-
- # Meson feature flags shared by all meson tasks, except:
- # SanityCheck: uses almost no dependencies.
- # Windows - VS: has fewer dependencies than listed here, so defines its own.
- # Linux: uses the 'auto' feature option to test meson feature autodetection.
- MESON_COMMON_FEATURES: >-
- -Dauto_features=disabled
- -Dldap=enabled
- -Dssl=openssl
- -Dtap_tests=enabled
- -Dplperl=enabled
- -Dplpython=enabled
- -Ddocs=enabled
- -Dicu=enabled
- -Dlibxml=enabled
- -Dlibxslt=enabled
- -Dlz4=enabled
- -Dpltcl=enabled
- -Dreadline=enabled
- -Dzlib=enabled
- -Dzstd=enabled
-
-
-# What files to preserve in case tests fail
-on_failure_ac: &on_failure_ac
- log_artifacts:
- paths:
- - "**/*.log"
- - "**/*.diffs"
- - "**/regress_log_*"
- type: text/plain
-
-on_failure_meson: &on_failure_meson
- testrun_artifacts:
- paths:
- - "build*/testrun/**/*.log"
- - "build*/testrun/**/*.diffs"
- - "build*/testrun/**/regress_log_*"
- type: text/plain
-
- # In theory it'd be nice to upload the junit files meson generates, so that
- # cirrus will nicely annotate the commit. Unfortunately the files don't
- # contain identifiable file + line numbers right now, so the annotations
- # don't end up useful. We could probably improve on that with a some custom
- # conversion script, but ...
- meson_log_artifacts:
- path: "build*/meson-logs/*.txt"
- type: text/plain
-
-
-# To avoid unnecessarily spinning up a lot of VMs / containers for entirely
-# broken commits, have a minimal task that all others depend on.
-#
-# SPECIAL:
-# - Builds with --auto-features=disabled and thus almost no enabled
-# dependencies
-task:
- name: SanityCheck
-
- # If a specific OS is requested, don't run the sanity check. This shortens
- # push-wait-for-ci cycle time a bit when debugging operating system specific
- # failures. Uses skip instead of only_if, as cirrus otherwise warns about
- # only_if conditions not matching.
- skip: $CI_SANITYCHECK_ENABLED == false
-
- env:
- CPUS: 4
- BUILD_JOBS: 8
- TEST_JOBS: 8
- IMAGE_FAMILY: pg-ci-trixie
- CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir
- # no options enabled, should be small
- CCACHE_MAXSIZE: "150M"
-
- # While containers would start up a bit quicker, building is a bit
- # slower. This way we don't have to maintain a container image.
- <<: *linux_task_template
-
- ccache_cache:
- folder: $CCACHE_DIR
-
- create_user_script: |
- useradd -m postgres
- chown -R postgres:postgres .
- mkdir -p ${CCACHE_DIR}
- chown -R postgres:postgres ${CCACHE_DIR}
- echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf
- su postgres -c "ulimit -l -H && ulimit -l -S"
- # Can't change container's kernel.core_pattern. Postgres user can't write
- # to / normally. Change that.
- chown root:postgres /
- chmod g+rwx /
-
- configure_script: |
- su postgres <<-EOF
- set -e
- meson setup \
- --buildtype=debug \
- --auto-features=disabled \
- -Ddefault_library=shared \
- -Dtap_tests=enabled \
- build
- EOF
- build_script: |
- su postgres <<-EOF
- set -e
- ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}
- EOF
- upload_caches: ccache
-
- # Run a minimal set of tests. The main regression tests take too long for
- # this purpose. For now this is a random quick pg_regress style test, and a
- # tap test that exercises both a frontend binary and the backend.
- test_minimal_script: |
- su postgres <<-EOF
- set -e
- ulimit -c unlimited
- meson test $MTEST_ARGS --suite setup
- meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \
- cube/regress pg_ctl/001_start_stop
- EOF
-
- on_failure:
- <<: *on_failure_meson
- cores_script: |
- mkdir -m 770 /tmp/cores
- find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \;
- src/tools/ci/cores_backtrace.sh linux /tmp/cores
-
-
-# SPECIAL:
-# - Uses postgres specific CPPFLAGS that increase test coverage
-# - Specifies configuration options that test reading/writing/copying of node trees
-# - Specifies debug_parallel_query=regress, to catch related issues during CI
-# - Also runs tests against a running postgres instance, see test_running_script
-task:
- name: FreeBSD - Meson
-
- env:
- CPUS: 4
- BUILD_JOBS: 4
- TEST_JOBS: 8
- IMAGE_FAMILY: pg-ci-freebsd
- DISK_SIZE: 50
-
- CCACHE_DIR: /tmp/ccache_dir
- CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
- CFLAGS: -Og -ggdb
-
- # Several buildfarm animals enable these options. Without testing them
- # during CI, it would be easy to cause breakage on the buildfarm with CI
- # passing.
- PG_TEST_INITDB_EXTRA_OPTS: >-
- -c debug_copy_parse_plan_trees=on
- -c debug_write_read_parse_plan_trees=on
- -c debug_raw_expression_coverage_test=on
- -c debug_parallel_query=regress
- PG_TEST_PG_UPGRADE_MODE: --link
-
- MESON_FEATURES: >-
- -Ddtrace=enabled
- -Dgssapi=enabled
- -Dlibcurl=enabled
- -Dnls=enabled
- -Dpam=enabled
- -Dtcl_version=tcl86
- -Duuid=bsd
-
- <<: *freebsd_task_template
-
- depends_on: SanityCheck
- only_if: $CI_FREEBSD_ENABLED
-
- sysinfo_script: |
- id
- uname -a
- ulimit -a -H && ulimit -a -S
- export
-
- ccache_cache:
- folder: $CCACHE_DIR
- setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh
- create_user_script: |
- pw useradd postgres
- chown -R postgres:postgres .
- mkdir -p ${CCACHE_DIR}
- chown -R postgres:postgres ${CCACHE_DIR}
- setup_core_files_script: |
- mkdir -m 770 /tmp/cores
- chown root:postgres /tmp/cores
- sysctl kern.corefile='/tmp/cores/%N.%P.core'
- setup_additional_packages_script: |
- #pkg install -y ...
-
- # NB: Intentionally build without -Dllvm. The freebsd image size is already
- # large enough to make VM startup slow, and even without llvm freebsd
- # already takes longer than other platforms except for windows.
- configure_script: |
- su postgres <<-EOF
- set -e
- meson setup \
- ${MESON_COMMON_PG_CONFIG_ARGS} \
- --buildtype=debug \
- -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
- ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \
- build
- EOF
- build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}'
- upload_caches: ccache
-
- test_world_script: |
- su postgres <<-EOF
- set -e
- ulimit -c unlimited
- meson test $MTEST_ARGS --num-processes ${TEST_JOBS}
- EOF
-
- # test runningcheck, freebsd chosen because it's currently fast enough
- test_running_script: |
- su postgres <<-EOF
- set -e
- ulimit -c unlimited
- meson test $MTEST_ARGS --quiet --suite setup
- export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH"
- mkdir -p build/testrun
- build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust
- echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf
- build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start
- meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running
- build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop
- EOF
-
- on_failure:
- # if the server continues running, it often causes cirrus-ci to fail
- # during upload, as it doesn't expect artifacts to change size
- stop_running_script: |
- su postgres <<-EOF
- set -e
- build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true
- EOF
- <<: *on_failure_meson
- cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores
-
-
-task:
- depends_on: SanityCheck
-
- env:
- # Below are experimentally derived to be a decent choice.
- CPUS: 4
- BUILD_JOBS: 8
- TEST_JOBS: 8
-
- # Default working directory is /tmp, but its total size (1.2 GB) is not
- # enough, so different working and cache directory are set.
- CIRRUS_WORKING_DIR: /home/postgres/postgres
- CCACHE_DIR: /home/postgres/cache
-
- PATH: /usr/sbin:$PATH
- CORE_DUMP_DIR: /var/crash
-
- matrix:
- - name: NetBSD - Meson
- # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star
- trigger_type: $CI_TRIGGER_TYPE_NETBSD
- only_if: $CI_NETBSD_ENABLED
- env:
- OS_NAME: netbsd
- IMAGE_FAMILY: pg-ci-netbsd-postgres
- PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig'
- # initdb fails with: 'invalid locale settings' error on NetBSD.
- # Force 'LANG' and 'LC_*' variables to be 'C'.
- # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us
- LANG: "C"
- LC_ALL: "C"
- # -Duuid is not set for the NetBSD, see the comment below, above
- # configure_script, for more information.
- MESON_FEATURES: >-
- -Dgssapi=enabled
- -Dlibcurl=enabled
- -Dnls=enabled
- -Dpam=enabled
-
- setup_additional_packages_script: |
- #pkgin -y install ...
- <<: *netbsd_task_template
-
- - name: OpenBSD - Meson
- # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star
- trigger_type: $CI_TRIGGER_TYPE_OPENBSD
- only_if: $CI_OPENBSD_ENABLED
- env:
- OS_NAME: openbsd
- IMAGE_FAMILY: pg-ci-openbsd-postgres
- PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig'
- CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin
-
- MESON_FEATURES: >-
- -Dbsd_auth=enabled
- -Dlibcurl=enabled
- -Dtcl_version=tcl86
- -Duuid=e2fs
-
- setup_additional_packages_script: |
- #pkg_add -I ...
- # Always core dump to ${CORE_DUMP_DIR}
- set_core_dump_script: sysctl -w kern.nosuidcoredump=2
- <<: *openbsd_task_template
-
- sysinfo_script: |
- locale
- id
- uname -a
- ulimit -a -H && ulimit -a -S
- env
-
- ccache_cache:
- folder: $CCACHE_DIR
- setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh
- create_user_script: |
- useradd postgres
- chown -R postgres:users /home/postgres
- mkdir -p ${CCACHE_DIR}
- chown -R postgres:users ${CCACHE_DIR}
- setup_core_files_script: |
- mkdir -p ${CORE_DUMP_DIR}
- chmod -R 770 ${CORE_DUMP_DIR}
- chown -R postgres:users ${CORE_DUMP_DIR}
-
- # -Duuid=bsd is not set since 'bsd' uuid option
- # is not working on NetBSD & OpenBSD. See
- # https://www.postgresql.org/message-id/[email protected]
- # And other uuid options are not available on NetBSD.
- configure_script: |
- su postgres <<-EOF
- set -e
- meson setup \
- ${MESON_COMMON_PG_CONFIG_ARGS} \
- --buildtype=debugoptimized \
- --pkg-config-path ${PKGCONFIG_PATH} \
- ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \
- build
- EOF
-
- build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}'
- upload_caches: ccache
-
- test_world_script: |
- su postgres <<-EOF
- set -e
- ulimit -c unlimited
- meson test $MTEST_ARGS --num-processes ${TEST_JOBS}
- EOF
-
- on_failure:
- <<: *on_failure_meson
- cores_script: |
- # Although we try to configure the OS to core dump inside
- # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the
- # ${CORE_DUMP_DIR} directory.
- find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \;
- src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR}
-
-
-# configure feature flags, shared between the task running the linux tests and
-# the CompilerWarnings task
-LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
- --with-gssapi
- --with-icu
- --with-ldap
- --with-libcurl
- --with-libxml
- --with-libxslt
- --with-llvm
- --with-lz4
- --with-pam
- --with-perl
- --with-python
- --with-selinux
- --with-ssl=openssl
- --with-systemd
- --with-tcl --with-tclconfig=/usr/lib/tcl8.6/
- --with-uuid=ossp
- --with-zstd
-
-
-# Check SPECIAL in the matrix: below
-task:
- env:
- CPUS: 4
- BUILD_JOBS: 4
- TEST_JOBS: 8 # experimentally derived to be a decent choice
- IMAGE_FAMILY: pg-ci-trixie
-
- CCACHE_DIR: /tmp/ccache_dir
- DEBUGINFOD_URLS: "https://debuginfod.debian.net";
-
- # Enable a reasonable set of sanitizers. Use the linux task for that, as
- # it's one of the fastest tasks (without sanitizers). Also several of the
- # sanitizers work best on linux.
- #
- # The overhead of alignment sanitizer is low, undefined behaviour has
- # moderate overhead. Test alignment sanitizer in the meson task, as it
- # does both 32 and 64 bit builds and is thus more likely to expose
- # alignment bugs.
- #
- # Address sanitizer in contrast is somewhat expensive. Enable it in the
- # autoconf task, as the meson task tests both 32 and 64bit.
- #
- # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes
- # print_stacktraces=1,verbosity=2, duh
- # detect_leaks=0: too many uninteresting leak errors in short-lived binaries
- UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2
- ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0
-
- # SANITIZER_FLAGS is set in the tasks below
- CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS
- CXXFLAGS: $CFLAGS
- LDFLAGS: $SANITIZER_FLAGS
- CC: ccache gcc
- CXX: ccache g++
-
- LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES
- LINUX_MESON_FEATURES: >-
- -Duuid=e2fs
-
- <<: *linux_task_template
-
- depends_on: SanityCheck
- only_if: $CI_LINUX_ENABLED
-
- ccache_cache:
- folder: ${CCACHE_DIR}
-
- sysinfo_script: |
- id
- uname -a
- cat /proc/cmdline
- ulimit -a -H && ulimit -a -S
- export
- create_user_script: |
- useradd -m postgres
- chown -R postgres:postgres .
- mkdir -p ${CCACHE_DIR}
- chown -R postgres:postgres ${CCACHE_DIR}
- echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf
- su postgres -c "ulimit -l -H && ulimit -l -S"
- setup_core_files_script: |
- mkdir -m 770 /tmp/cores
- chown root:postgres /tmp/cores
- sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core'
-
- setup_hosts_file_script: |
- cat >> /etc/hosts <<-EOF
- 127.0.0.1 pg-loadbalancetest
- 127.0.0.2 pg-loadbalancetest
- 127.0.0.3 pg-loadbalancetest
- EOF
-
- setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
-
- matrix:
- # SPECIAL:
- # - Uses address sanitizer, sanitizer failures are typically printed in
- # the server log
- # - Configures postgres with a small segment size
- - name: Linux - Debian Trixie - Autoconf
-
- env:
- SANITIZER_FLAGS: -fsanitize=address
- PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range
-
- # Normally, the "relation segment" code basically has no coverage in our
- # tests, because we (quite reasonably) don't generate tables large
- # enough in tests. We've had plenty bugs that we didn't notice due the
- # code not being exercised much. Thus specify a very small segment size
- # here. Use a non-power-of-two segment size, given we currently allow
- # that.
- configure_script: |
- su postgres <<-EOF
- set -e
- ./configure \
- --enable-cassert --enable-injection-points --enable-debug \
- --enable-tap-tests --enable-nls \
- --with-segsize-blocks=6 \
- --with-libnuma \
- --with-liburing \
- \
- ${LINUX_CONFIGURE_FEATURES} \
- \
- CLANG="ccache clang"
- EOF
- build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin"
- upload_caches: ccache
-
- test_world_script: |
- su postgres <<-EOF
- set -e
- ulimit -c unlimited # default is 0
- make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS}
- EOF
-
- on_failure:
- <<: *on_failure_ac
-
- # SPECIAL:
- # - Uses undefined behaviour and alignment sanitizers, sanitizer failures
- # are typically printed in the server log
- # - Test both 64bit and 32 bit builds
- # - uses io_method=io_uring
- # - Uses meson feature autodetection
- - name: Linux - Debian Trixie - Meson
-
- env:
- CCACHE_MAXSIZE: "400M" # tests two different builds
- SANITIZER_FLAGS: -fsanitize=alignment,undefined
- PG_TEST_INITDB_EXTRA_OPTS: >-
- -c io_method=io_uring
-
- configure_script: |
- su postgres <<-EOF
- set -e
- meson setup \
- ${MESON_COMMON_PG_CONFIG_ARGS} \
- --buildtype=debug \
- ${LINUX_MESON_FEATURES} -Dllvm=enabled \
- build
- EOF
-
- # Also build & test in a 32bit build - it's gotten rare to test that
- # locally.
- configure_32_script: |
- su postgres <<-EOF
- set -e
- export CC='ccache gcc -m32'
- export CXX='ccache g++ -m32'
- meson setup \
- ${MESON_COMMON_PG_CONFIG_ARGS} \
- --buildtype=debug \
- --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
- -DPERL=perl5.40-i386-linux-gnu \
- ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \
- build-32
- EOF
-
- build_script: |
- su postgres <<-EOF
- set -e
- ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}
- ninja -C build -t missingdeps
- EOF
-
- build_32_script: |
- su postgres <<-EOF
- set -e
- ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET}
- ninja -C build -t missingdeps
- EOF
-
- upload_caches: ccache
-
- test_world_script: |
- su postgres <<-EOF
- set -e
- ulimit -c unlimited
- meson test $MTEST_ARGS --num-processes ${TEST_JOBS}
- EOF
- # so that we don't upload 64bit logs if 32bit fails
- rm -rf build/
-
- # There's currently no coverage of icu with LANG=C in the buildfarm. We
- # can easily provide some here by running one of the sets of tests that
- # way. Newer versions of python insist on changing the LC_CTYPE away
- # from C, prevent that with PYTHONCOERCECLOCALE.
- test_world_32_script: |
- su postgres <<-EOF
- set -e
- ulimit -c unlimited
- PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS}
- EOF
-
- on_failure:
- <<: *on_failure_meson
-
- on_failure:
- cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores
-
-
-# NB: macOS is by far the most expensive OS to run CI for, therefore no
-# expensive additional checks should be added.
-#
-# SPECIAL:
-# - Enables --clone for pg_upgrade and pg_combinebackup
-task:
- name: macOS - Sequoia - Meson
-
- env:
- CPUS: 4 # always get that much for cirrusci macOS instances
- BUILD_JOBS: $CPUS
- # Test performance regresses noticeably when using all cores. 8 seems to
- # work OK. See
- # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de
- TEST_JOBS: 8
- IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia
-
- CIRRUS_WORKING_DIR: ${HOME}/pgsql/
- CCACHE_DIR: ${HOME}/ccache
- MACPORTS_CACHE: ${HOME}/macports-cache
-
- MESON_FEATURES: >-
- -Dbonjour=enabled
- -Ddtrace=enabled
- -Dgssapi=enabled
- -Dlibcurl=enabled
- -Dnls=enabled
- -Duuid=e2fs
-
- MACOS_PACKAGE_LIST: >-
- ccache
- icu
- kerberos5
- lz4
- meson
- openldap
- openssl
- p5.34-io-tty
- p5.34-ipc-run
- python312
- tcl
- zstd
-
- CC: ccache cc
- CXX: ccache c++
- CFLAGS: -Og -ggdb
- CXXFLAGS: -Og -ggdb
-
- PG_TEST_PG_UPGRADE_MODE: --clone
- PG_TEST_PG_COMBINEBACKUP_MODE: --clone
-
- <<: *macos_task_template
-
- depends_on: SanityCheck
- only_if: $CI_MACOS_ENABLED
-
- sysinfo_script: |
- id
- uname -a
- ulimit -a -H && ulimit -a -S
- export
-
- setup_core_files_script:
- - mkdir ${HOME}/cores
- - sudo sysctl kern.corefile="${HOME}/cores/core.%P"
-
- # Use macports, even though homebrew is installed. The installation
- # of the additional packages we need would take quite a while with
- # homebrew, even if we cache the downloads. We can't cache all of
- # homebrew, because it's already large. So we use macports. To cache
- # the installation we create a .dmg file that we mount if it already
- # exists.
- # XXX: The reason for the direct p5.34* references is that we'd need
- # the large macport tree around to figure out that p5-io-tty is
- # actually p5.34-io-tty. Using the unversioned name works, but
- # updates macports every time.
- macports_cache:
- folder: ${MACPORTS_CACHE}
- fingerprint_script: |
- # Reinstall packages if the OS major version, the list of the packages
- # to install or the MacPorts install script changes.
- sw_vers -productVersion | sed 's/\..*//'
- echo $MACOS_PACKAGE_LIST
- md5 src/tools/ci/ci_macports_packages.sh
- reupload_on_changes: true
- setup_additional_packages_script: |
- sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST
- # system python doesn't provide headers
- sudo /opt/local/bin/port select python3 python312
- # Make macports install visible for subsequent steps
- echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV
- upload_caches: macports
-
- ccache_cache:
- folder: $CCACHE_DIR
- configure_script: |
- export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/"
- meson setup \
- ${MESON_COMMON_PG_CONFIG_ARGS} \
- --buildtype=debug \
- -Dextra_include_dirs=/opt/local/include \
- -Dextra_lib_dirs=/opt/local/lib \
- ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \
- build
-
- build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}
- upload_caches: ccache
-
- test_world_script: |
- ulimit -c unlimited # default is 0
- ulimit -n 1024 # default is 256, pretty low
- meson test $MTEST_ARGS --num-processes ${TEST_JOBS}
-
- on_failure:
- <<: *on_failure_meson
- cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores"
-
-
-WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE
- env:
- # Half the allowed per-user CPU cores
- CPUS: 4
-
- # The default cirrus working dir is in a directory msbuild complains about
- CIRRUS_WORKING_DIR: "c:/cirrus"
- # git's tar doesn't deal with drive letters, see
- # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net
- TAR: "c:/windows/system32/tar.exe"
- # Avoids port conflicts between concurrent tap test runs
- PG_TEST_USE_UNIX_SOCKETS: 1
- PG_REGRESS_SOCK_DIR: "c:/cirrus/"
- DISK_SIZE: 50
- IMAGE_FAMILY: pg-ci-windows-ci
-
- sysinfo_script: |
- chcp
- systeminfo
- powershell -Command get-psdrive -psprovider filesystem
- set
-
-
-task:
- name: Windows - Server 2022, VS 2019 - Meson & ninja
- << : *WINDOWS_ENVIRONMENT_BASE
-
- env:
- TEST_JOBS: 8 # wild guess, data based value welcome
-
- # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That
- # prevents crash reporting from working unless binaries do SetErrorMode()
- # themselves. Furthermore, it appears that either python or, more likely,
- # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very
- # occasionally *trigger* a crash on process exit - which is hard to debug,
- # given that it explicitly prevents crash dumps from working...
- # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX
- CIRRUS_WINDOWS_ERROR_MODE: 0x8001
-
- MESON_FEATURES:
- -Dcpp_args=/std:c++20
- -Dauto_features=disabled
- -Dldap=enabled
- -Dssl=openssl
- -Dtap_tests=enabled
- -Dplperl=enabled
- -Dplpython=enabled
-
- <<: *windows_task_template
-
- depends_on: SanityCheck
- only_if: $CI_WINDOWS_ENABLED
-
- setup_additional_packages_script: |
- REM choco install -y --no-progress ...
-
- setup_hosts_file_script: |
- echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts
- echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts
- echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts
- type c:\Windows\System32\Drivers\etc\hosts
-
- configure_script: |
- vcvarsall x64
- meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build
-
- build_script: |
- vcvarsall x64
- ninja -C build %MBUILD_TARGET%
- ninja -C build -t missingdeps
-
- check_world_script: |
- vcvarsall x64
- meson test %MTEST_ARGS% --num-processes %TEST_JOBS%
-
- on_failure:
- <<: *on_failure_meson
- crashlog_artifacts:
- path: "crashlog-*.txt"
- type: text/plain
-
-
-task:
- << : *WINDOWS_ENVIRONMENT_BASE
- name: Windows - Server 2022, MinGW64 - Meson
-
- # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star.
- trigger_type: $CI_TRIGGER_TYPE_MINGW
-
- depends_on: SanityCheck
- only_if: $CI_MINGW_ENABLED
-
- env:
- TEST_JOBS: 4 # higher concurrency causes occasional failures
- CCACHE_DIR: C:/msys64/ccache
- CCACHE_MAXSIZE: "500M"
- CCACHE_SLOPPINESS: pch_defines,time_macros
- CCACHE_DEPEND: 1
- # for some reason mingw plpython cannot find its installation without this
- PYTHONHOME: C:/msys64/ucrt64
- # prevents MSYS bash from resetting error mode
- MSYS: winjitdebug
- # Start bash in current working directory
- CHERE_INVOKING: 1
- BASH: C:\msys64\usr\bin\bash.exe -l
-
- # Keep -Dnls explicitly disabled, as the number of files it creates causes a
- # noticeable slowdown.
- MESON_FEATURES: >-
- -Dnls=disabled
-
- <<: *windows_task_template
-
- ccache_cache:
- folder: ${CCACHE_DIR}
-
- setup_additional_packages_script: |
- REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ...
-
- mingw_info_script: |
- %BASH% -c "where gcc"
- %BASH% -c "gcc --version"
- %BASH% -c "where perl"
- %BASH% -c "perl --version"
-
- configure_script: |
- %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build"
-
- build_script: |
- %BASH% -c "ninja -C build ${MBUILD_TARGET}"
-
- upload_caches: ccache
-
- test_world_script: |
- %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%"
-
- on_failure:
- <<: *on_failure_meson
- crashlog_artifacts:
- path: "crashlog-*.txt"
- type: text/plain
-
-
-task:
- name: CompilerWarnings
-
- # To limit unnecessary work only run this once the SanityCheck
- # succeeds. This is particularly important for this task as we intentionally
- # use always: to continue after failures.
- depends_on: SanityCheck
- only_if: $CI_COMPILERWARNINGS_ENABLED
-
- env:
- CPUS: 4
- BUILD_JOBS: 4
- IMAGE_FAMILY: pg-ci-trixie
-
- # Use larger ccache cache, as this task compiles with multiple compilers /
- # flag combinations
- CCACHE_MAXSIZE: "1G"
- CCACHE_DIR: "/tmp/ccache_dir"
-
- LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES
-
- <<: *linux_task_template
-
- sysinfo_script: |
- id
- uname -a
- cat /proc/cmdline
- ulimit -a -H && ulimit -a -S
- gcc -v
- clang -v
- export
-
- ccache_cache:
- folder: $CCACHE_DIR
-
- setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
-
- ###
- # Test that code can be built with gcc/clang without warnings
- ###
-
- setup_script: echo "COPT=-Werror" > src/Makefile.custom
-
- # Trace probes have a history of getting accidentally broken. Use the
- # different compilers to build with different combinations of dtrace on/off
- # and cassert on/off.
-
- # gcc, cassert off, dtrace on
- always:
- gcc_warning_script: |
- time ./configure \
- --cache gcc.cache \
- --enable-dtrace \
- ${LINUX_CONFIGURE_FEATURES} \
- CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- time make -s -j${BUILD_JOBS} world-bin
-
- # gcc, cassert on, dtrace off
- always:
- gcc_a_warning_script: |
- time ./configure \
- --cache gcc.cache \
- --enable-cassert \
- ${LINUX_CONFIGURE_FEATURES} \
- CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- time make -s -j${BUILD_JOBS} world-bin
-
- # clang, cassert off, dtrace off
- always:
- clang_warning_script: |
- time ./configure \
- --cache clang.cache \
- ${LINUX_CONFIGURE_FEATURES} \
- CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- time make -s -j${BUILD_JOBS} world-bin
-
- # clang, cassert on, dtrace on
- always:
- clang_a_warning_script: |
- time ./configure \
- --cache clang.cache \
- --enable-cassert \
- --enable-dtrace \
- ${LINUX_CONFIGURE_FEATURES} \
- CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- time make -s -j${BUILD_JOBS} world-bin
-
- # cross-compile to windows
- always:
- mingw_cross_warning_script: |
- time ./configure \
- --host=x86_64-w64-mingw32ucrt \
- --enable-cassert \
- --without-icu \
- CC="ccache x86_64-w64-mingw32ucrt-gcc" \
- CXX="ccache x86_64-w64-mingw32ucrt-g++"
- make -s -j${BUILD_JOBS} clean
- time make -s -j${BUILD_JOBS} world-bin
-
- ###
- # Verify docs can be built
- ###
- # XXX: Only do this if there have been changes in doc/ since last build
- always:
- docs_build_script: |
- time ./configure \
- --cache gcc.cache \
- CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- time make -s -j${BUILD_JOBS} -C doc
-
- ###
- # Verify headerscheck / cpluspluscheck succeed
- #
- # - Run both in same script to increase parallelism, use -k to get result of both
- # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose
- ###
- always:
- headers_headerscheck_script: |
- time ./configure \
- ${LINUX_CONFIGURE_FEATURES} \
- --cache gcc.cache \
- --quiet \
- CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10'
-
- always:
- upload_caches: ccache
--
2.54.0.380.gc69baaf57b
>From c877607c5a0f2878fb9e28e93a40e5be22d7eb6c Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Thu, 28 May 2026 16:17:06 -0400
Subject: [PATCH v6a 4/5] smaller segsize for tests
---
meson.build | 3 ++-
contrib/test_decoding/expected/ddl.out | 2 +-
contrib/test_decoding/sql/ddl.sql | 3 +--
src/test/perl/PostgreSQL/Test/Cluster.pm | 1 +
src/test/recovery/t/039_end_of_wal.pl | 3 ++-
src/test/recovery/t/051_effective_wal_level.pl | 2 +-
src/test/regress/pg_regress.c | 2 +-
src/Makefile.global.in | 2 +-
8 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/meson.build b/meson.build
index 20b887f1a1b..b7a42e4373a 100644
--- a/meson.build
+++ b/meson.build
@@ -3927,7 +3927,8 @@ sys.exit(sp.returncode)
test_initdb_template,
temp_install_bindir / 'initdb',
'--auth', 'trust', '--no-sync', '--no-instructions', '--lc-messages=C',
- '--no-clean'
+ '--no-clean',
+ '--wal-segsize=1',
],
priority: setup_tests_priority - 1,
timeout: 300,
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 6819812e806..30a1877a76f 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -58,7 +58,7 @@ SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_d
SELECT slot_name, plugin, slot_type, active,
NOT catalog_xmin IS NULL AS catalog_xmin_set,
xmin IS NULl AS data_xmin_not_set,
- pg_wal_lsn_diff(restart_lsn, '0/01000000') > 0 AS some_wal
+ pg_wal_lsn_diff(restart_lsn, '0/00000001') > 0 AS some_wal
FROM pg_replication_slots;
slot_name | plugin | slot_type | active | catalog_xmin_set | data_xmin_not_set | some_wal
-----------------+---------------+-----------+--------+------------------+-------------------+----------
diff --git a/contrib/test_decoding/sql/ddl.sql b/contrib/test_decoding/sql/ddl.sql
index 6d0b7d77778..ba250e5d529 100644
--- a/contrib/test_decoding/sql/ddl.sql
+++ b/contrib/test_decoding/sql/ddl.sql
@@ -22,14 +22,13 @@ SELECT 'init' FROM pg_create_physical_replication_slot('repl');
SELECT data FROM pg_logical_slot_get_changes('repl', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
SELECT pg_drop_replication_slot('repl');
-
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
/* check whether status function reports us, only reproduceable columns */
SELECT slot_name, plugin, slot_type, active,
NOT catalog_xmin IS NULL AS catalog_xmin_set,
xmin IS NULl AS data_xmin_not_set,
- pg_wal_lsn_diff(restart_lsn, '0/01000000') > 0 AS some_wal
+ pg_wal_lsn_diff(restart_lsn, '0/00000001') > 0 AS some_wal
FROM pg_replication_slots;
/*
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index 4fcb1f6be56..5c74a8b690a 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -671,6 +671,7 @@ sub init
'initdb', '--no-sync',
'--pgdata' => $pgdata,
'--auth' => 'trust',
+ '--wal-segsize' => '1',
@{ $params{extra} });
}
else
diff --git a/src/test/recovery/t/039_end_of_wal.pl b/src/test/recovery/t/039_end_of_wal.pl
index f46d089a0fb..af6a6ced73d 100644
--- a/src/test/recovery/t/039_end_of_wal.pl
+++ b/src/test/recovery/t/039_end_of_wal.pl
@@ -112,7 +112,8 @@ sub build_page_header
# set to "minimal" avoids random standby snapshot records. Autovacuum
# could also trigger randomly, generating random WAL activity of its own.
my $node = PostgreSQL::Test::Cluster->new("node");
-$node->init;
+$node->init(extra=>['--wal-segsize=16']);
+
$node->append_conf(
'postgresql.conf',
q[wal_level = minimal
diff --git a/src/test/recovery/t/051_effective_wal_level.pl b/src/test/recovery/t/051_effective_wal_level.pl
index c862073c34e..6329947a33f 100644
--- a/src/test/recovery/t/051_effective_wal_level.pl
+++ b/src/test/recovery/t/051_effective_wal_level.pl
@@ -34,7 +34,7 @@ sub wait_for_logical_decoding_disabled
# Initialize the primary server with wal_level = 'replica'.
my $primary = PostgreSQL::Test::Cluster->new('primary');
-$primary->init(allows_streaming => 1);
+$primary->init(allows_streaming => 1, extra=>['--wal-segsize=16']);
$primary->append_conf('postgresql.conf', "log_min_messages = debug1");
$primary->start();
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 1c052cc0fbf..23f9dfe781f 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -2410,7 +2410,7 @@ regression_main(int argc, char *argv[],
note("initializing database system by running initdb");
appendStringInfo(&cmd,
- "\"%s%sinitdb\" -D \"%s/data\" --no-clean --no-sync",
+ "\"%s%sinitdb\" -D \"%s/data\" --no-clean --no-sync --wal-segsize=1",
bindir ? bindir : "",
bindir ? "/" : "",
temp_instance);
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index cef1ad7f87d..f529bacc4b2 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -442,7 +442,7 @@ ifeq ($(MAKELEVEL),0)
$(MAKE) -C '$(top_builddir)' DESTDIR='$(abs_top_builddir)'/tmp_install install >'$(abs_top_builddir)'/tmp_install/log/install.log 2>&1
$(MAKE) -j1 $(if $(CHECKPREP_TOP),-C $(CHECKPREP_TOP),) checkprep >>'$(abs_top_builddir)'/tmp_install/log/install.log 2>&1
- $(with_temp_install) initdb --auth trust --no-sync --no-instructions --lc-messages=C --no-clean '$(abs_top_builddir)'/tmp_install/initdb-template >>'$(abs_top_builddir)'/tmp_install/log/initdb-template.log 2>&1
+ $(with_temp_install) initdb --auth trust --no-sync --no-instructions --lc-messages=C --no-clean --wal-segsize=1 '$(abs_top_builddir)'/tmp_install/initdb-template >>'$(abs_top_builddir)'/tmp_install/log/initdb-template.log 2>&1
endif
endif
endif
--
2.54.0.380.gc69baaf57b
>From fb5c68339f49aba97ab8cfd9dd358a3b8e59228d Mon Sep 17 00:00:00 2001
From: Andres Freund <[email protected]>
Date: Mon, 1 Jun 2026 15:31:32 -0400
Subject: [PATCH v6a 5/5] gha: Only run main regression tests
---
.github/workflows/postgresql-ci.yml | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/.github/workflows/postgresql-ci.yml b/.github/workflows/postgresql-ci.yml
index e2795ca0ffb..c6d4c960a55 100644
--- a/.github/workflows/postgresql-ci.yml
+++ b/.github/workflows/postgresql-ci.yml
@@ -35,7 +35,8 @@ env:
# Check target for the autoconf builds. Can be set to e.g. check to only
# only test the main regression tests.
- CHECK: check-world PROVE_FLAGS=--timer
+ # FIXME: Reset
+ CHECK: check PROVE_FLAGS=--timer
CHECKFLAGS: -Otarget
# Build test dependencies as part of the build step, to see compiler
@@ -45,7 +46,8 @@ env:
# Can be set to a non-empty value to run a limited set of tests
# (e.g. --suite regress to only run the main regression tests).
- MTEST_TARGET:
+ # FIXME: Reset
+ MTEST_TARGET: --suite regress
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${{ github.workspace }}/src/tools/ci/pg_ci_base.conf
--
2.54.0.380.gc69baaf57b
