Hi, On 2026-06-03 20:34:41 +0200, Peter Eisentraut wrote: > On 03.06.26 17:35, Andres Freund wrote: > > > From a067f35829f5db7b7b03d2ac1eed934820597a5c Mon Sep 17 00:00:00 2001 > > > From: Peter Eisentraut <[email protected]> > > > Date: Wed, 3 Jun 2026 12:47:35 +0200 > > > Subject: [PATCH v7.2.pe] Move more coverage from previously FreeBSD > > > elsewhere > > > > > > The following testing aspects were previously (Cirrus CI) covered by > > > the FreeBSD job. Since we currently (GitHub Actions) don't have > > > FreeBSD support, we move these elsewhere for now: > > > > > > - RELCACHE_FORCE_RELEASE (moved to macOS) > > > - ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS (moved to macOS) > > > - PG_TEST_INITDB_EXTRA_OPTS test reading/writing/copying of node trees > > > as well as debug_parallel_query=regress (moved to macOS) > > > - PG_TEST_PG_UPGRADE_MODE --link (moved to Linux 64-bit (macOS already > > > tests --clone)) > > > > I'd move all of these to linux-autoconf, if you're ok with that? > > > > > > > - meson test --setup running (moved to macOS) > > > > And this to linux-meson-32? > > That seems fine as well.
Done in the attached (0015) Other changes: - added a commit to split the tests in windows across two jobs using meson test's --slice. At the expense of the uglier display and use of one more runner "slot", it brings down the test times from about 33min to 23min. I think that's worth it? - use ubuntu-24.04 instead of ubuntu-latest, worried that the upgrade will break stuff - formatting and naming polish - some more missing stored logs and similar stuff Greetings, Andres Freund
>From 1830f1f38b17749c5cee6de9c31d3d0e6b503e57 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Tue, 2 Jun 2026 18:57:01 -0400 Subject: [PATCH v9a 01/22] ci: Add GitHub Actions based CI Cirrus CI, which the project used for CI until now, has shut down on June 1, 2026. Replace it with GitHub Actions. GitHub Actions was selected because it has unlimited runner time for public repositories. The GitHub Actions based CI currently covers: - SanityCheck - Linux - Autoconf - Linux - Meson, (32-bit and 64-bit) - macOS - Meson - Windows (Visual Studio + Meson and MinGW + Meson) - CompilerWarnings BSD coverage is left for later, as it requires more work. Note that, for performance reasons, use of address sanitizer was moved to the Linux - Meson (64-bit) task. Back-branches will be updated later, after being sure that workflow runs correctly on master. Author: Nazir Bilal Yavuz <[email protected]> Author: Andres Freund <[email protected]> Author: Jelte Fennema-Nio <[email protected]> Reviewed-by: Jacob Champion <[email protected]> Reviewed-by: Peter Eisentraut <[email protected]> Reviewed-by: Andres Freund <[email protected]> Reviewed-by: Zsolt Parragi <[email protected]> Discussion: https://postgr.es/m/3ydjipcr7kbss57nvi67noplncqhesl5eyb6wgol4ccjxynspv%40yatlykpribmm --- .github/workflows/pg-ci.yml | 1083 ++++++++++++++++++++++++++ src/tools/ci/ci_macports_packages.sh | 19 +- 2 files changed, 1099 insertions(+), 3 deletions(-) create mode 100644 .github/workflows/pg-ci.yml diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml new file mode 100644 index 00000000000..8594d540f6a --- /dev/null +++ b/.github/workflows/pg-ci.yml @@ -0,0 +1,1083 @@ +# GitHub Actions CI configuration for PostgreSQL +# +# For instructions on how to enable / disable CI integration in a repository +# and further details, see src/tools/ci/README +# +# https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax +# is a good starting point for documentation about GitHub Actions. + +name: CI for PostgreSQL + +on: + push: + # TODO: It might make sense to also add PR based triggers, to make it easier + # to use PRs on one's own repo, but it's a tad more complicated than just + # adding the 'pull_request' event, as naively doing so would often lead to + # running CI twice. + +# Restrict GITHUB_TOKEN to the minimum the jobs need: reading repo +# contents during checkout. +permissions: + contents: read + +concurrency: + # For anything other than stable branches, we want there to only be one + # workflow active for that branch. But on stable branches & master, we + # neither want to wait for prior runs, nor to cancel them, so that each + # separately pushed commit is tested. We achieve that by setting a unique + # concurrency group when on such a branch. + group: | + ${{github.workflow }}-${{ + case(github.ref == 'refs/heads/master' || + (startsWith(github.ref, 'refs/heads/REL_') && endsWith(github.ref, '_STABLE')), + github.run_id, + github.ref) + }} + cancel-in-progress: true + +env: + # The lower depth accelerates git clone. Use a bit of depth so that + # concurrent jobs and retrying older runs have a chance of working. + CLONE_DEPTH: 500 + + # At the moment all jobs use 4vcore runners, and none seems to benefit from + # increasing concurrency further. + BUILD_JOBS: 4 + + # It's possible that some jobs benefit from an increased test concurrency, + # but a default of 4 is a safe bet. Individual jobs can override. + TEST_JOBS: 4 + + CCACHE_MAXSIZE: "250M" + CCACHE_DIR: ${{ github.workspace }}/ccache_dir + + # Check target for the autoconf builds. Can be set to e.g. check to only + # test the main regression tests. + CHECK: check-world PROVE_FLAGS=--timer + CHECKFLAGS: -Otarget + + # Build test dependencies as part of the build step, to see compiler + # errors/warnings in one place. + MBUILD_TARGET: all testprep + MTEST_ARGS: --print-errorlogs --no-rebuild -C build + + # Can be set to a non-empty value to run a limited set of tests + # (e.g. --suite regress to only run the main regression tests). + MTEST_TARGET: + + PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests + TEMP_CONFIG: ${{ github.workspace }}/src/tools/ci/pg_ci_base.conf + PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth + + # Postgres config args for the meson builds, shared between all meson tasks + # except the 'SanityCheck' task + MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true + + # Meson feature flags shared by all meson tasks, except: + # SanityCheck: uses almost no dependencies. + # Windows - VS: has fewer dependencies than listed here, so defines its own. + # Linux: uses the 'auto' feature option to test meson feature autodetection. + MESON_COMMON_FEATURES: >- + -Dauto_features=disabled + -Dldap=enabled + -Dssl=openssl + -Dtap_tests=enabled + -Dplperl=enabled + -Dplpython=enabled + -Ddocs=enabled + -Dicu=enabled + -Dlibxml=enabled + -Dlibxslt=enabled + -Dlz4=enabled + -Dpltcl=enabled + -Dreadline=enabled + -Dzlib=enabled + -Dzstd=enabled + + # Shared between the Linux autoconf job and the CompilerWarnings jobs + LINUX_CONFIGURE_FEATURES: >- + --with-gssapi + --with-icu + --with-ldap + --with-libcurl + --with-libxml + --with-libxslt + --with-llvm + --with-lz4 + --with-pam + --with-perl + --with-python + --with-selinux + --with-ssl=openssl + --with-systemd + --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ + --with-uuid=ossp + --with-zstd + + # Debian Trixie containers used by all Linux jobs. Built by + # 'https://github.com/anarazel/pg-vm-images/'. + CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main + CONTAINER_LINUX_CI: linux_debian_trixie_ci:latest + CONTAINER_LINUX_CI_DOCS: linux_debian_trixie_ci_docs:latest + + # The full set of OS / job selectors recognized by the `ci-os-only:` + # commit-message directive parsed in the `setup` job below. + CI_OS_ONLY_JOBS: "linux macos windows mingw compilerwarnings sanitycheck" + + +jobs: + # Parse "ci-os-only: ..." from the commit message and expose flags + # consumed by the jobs' `if:` conditions. + setup: + name: Determine enabled jobs + runs-on: ubuntu-latest + timeout-minutes: 1 + outputs: + linux: ${{ steps.os.outputs.linux }} + macos: ${{ steps.os.outputs.macos }} + windows: ${{ steps.os.outputs.windows }} + mingw: ${{ steps.os.outputs.mingw }} + compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} + sanitycheck: ${{ steps.os.outputs.sanitycheck }} + # Re-export workflow-level env vars that other jobs need to reference + # from contexts (e.g. `jobs.<id>.container.image`) where the `env` + # context is not available. + container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} + container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + steps: + # Anchor reused by other jobs further down. GitHub Actions supports YAML + # anchors/aliases but not merge keys, so the alias copies the whole step + # verbatim. The anchor is resolved at YAML parse time, so the alias + # keeps working even if this job were to be skipped at runtime. + - &nix_sysinfo_step + name: sysinfo + run: | + id + uname -a + ulimit -a -H && ulimit -a -S + env + + - id: os + env: + MSG: ${{ github.event.head_commit.message }} + shell: bash + run: | + all_os=${CI_OS_ONLY_JOBS} + if printf '%s\n' "$MSG" | grep -qE '^ci-os-only: '; then + sel=$(printf '%s\n' "$MSG" | sed -n 's/^ci-os-only: //p' | head -n 1) + echo "ci-os-only selection: $sel" + else + sel="$all_os" + fi + for o in $all_os; do + if echo " $sel " | grep -qE "[ ,]$o[ ,]"; then + echo "$o=true" >> "$GITHUB_OUTPUT" + else + echo "$o=false" >> "$GITHUB_OUTPUT" + fi + done + cat "$GITHUB_OUTPUT" + + + # To avoid unnecessarily spinning up a lot of VMs / containers for entirely + # broken commits, have a minimal task that all others depend on. + # + # SPECIAL: + # - Builds with --auto-features=disabled and thus almost no enabled + # dependencies + sanity-check: + name: SanityCheck + needs: setup + if: | + !cancelled() && + needs.setup.outputs.sanitycheck == 'true' + runs-on: ubuntu-latest + timeout-minutes: 15 + container: &linux_ci_container + image: ${{ needs.setup.outputs.container_linux_ci }} + + # Options passed to all linux containers. Not all of the jobs need + # all of them, but it's easier to just define them centrally. + # + # --privileged is needed so the prepare step can write to sysctls + # under /proc/sys (it's mounted read-only without it). We use it to + # set kernel.core_pattern and (for the meson entries) to flip + # kernel.io_uring_disabled (default 2 on recent GH runner kernels). + # + # Share the host PID + IPC namespaces. 017_shm.pl rapidly creates, + # kill9's, and restarts postgres; with the container's small PID + # space a new postgres can recycle the dead postmaster's PID before + # pg_ctl's postmaster.pid check notices, producing spurious "node X + # is already running" failures. SysV shm in the test also relies on + # host-like IPC behavior. + # + # --ulimit raises memlock and core dump size. Memlock is needed for + # running the AIO tests. + options: &linux_container_options | + --privileged --pid=host --ipc=host --ulimit memlock=-1:-1 + env: + # no options enabled, should be small + CCACHE_MAXSIZE: "150M" + steps: + - *nix_sysinfo_step + + - &checkout_step + uses: actions/checkout@v6 + with: + fetch-depth: ${{ env.CLONE_DEPTH }} + + - &ccache_restore_step + name: Restore ccache + id: ccache_restore + uses: actions/cache/restore@v5 + with: + path: ${{ env.CCACHE_DIR }} + key: ccache-${{ github.job }}-${{ github.ref_name }}-${{ github.run_id }}-${{ github.run_attempt }} + restore-keys: | + ccache-${{ github.job }}-${{ github.ref_name }}- + ccache-${{ github.job }}- + + - &linux_prepare_workspace + name: Prepare workspace + run: | + useradd -m postgres + chown -R postgres:postgres . + mkdir -m 770 /tmp/cores + chown root:postgres /tmp/cores + sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' + # This is only needed for some of the tasks using this, but it + # doesn't harm to have this enabled. + sysctl -w kernel.io_uring_disabled=0 + + cat >> /etc/hosts <<-EOF + 127.0.0.1 pg-loadbalancetest + 127.0.0.2 pg-loadbalancetest + 127.0.0.3 pg-loadbalancetest + EOF + + # By using a shell that includes su, the run commands themselves get + # simpler. As there are quite a few commands that need to use su... + - name: Configure + shell: &su_postgres_shell | + su postgres -c "bash --noprofile --norc -eo pipefail {0}" + run: | + meson setup \ + --buildtype=debug \ + --auto-features=disabled \ + -Ddefault_library=shared \ + -Dtap_tests=enabled \ + build + + - name: Build + shell: *su_postgres_shell + run: &ninja_build_command | + ninja -C build -j${{env.BUILD_JOBS}} ${{env.MBUILD_TARGET}} + ninja -C build -t missingdeps + + # TODO: As long as we use per-run ccache caches, we should probably add + # a step that checks if there is sufficient new content to warrant + # saving the new cache. + - &ccache_save_step + name: Save ccache + uses: actions/cache/save@v5 + with: + path: ${{ env.CCACHE_DIR }} + key: ${{ steps.ccache_restore.outputs.cache-primary-key }} + + # Run a minimal set of tests. The main regression tests take too long + # for this purpose. For now this is a random quick pg_regress style + # test, and a tap test that exercises both a frontend binary and the + # backend. + # + # To allow the command below to be reused by later tasks, we allow + # adding "setup" commands to be specified via the ADDITIONAL_SETUP + # environment variable. + # + # Note that this command is used on all platforms, therefore one needs + # to be careful about using only ${{env.}} variable references, + # linebreaks etc. + - name: Test + shell: *su_postgres_shell + env: + MTEST_TARGET: cube/regress pg_ctl/001_start_stop + run: &meson_test_world_cmd | + ${{case(runner.os == 'Windows', '', 'ulimit -c unlimited')}} + + ${{env.ADDITIONAL_SETUP}} + + echo ::group::test_setup + meson test ${{env.MTEST_ARGS}} --suite setup --logbase setup + echo ::endgroup:: + + meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} ${{env.MTEST_TARGET}} + + - &linux_collect_cores + name: Core backtraces + if: failure() && !cancelled() + run: src/tools/ci/cores_backtrace.sh linux /tmp/cores + + # Note that this is used for both meson and autoconf builds + - &upload_logs_step + name: Upload logs + if: failure() && !cancelled() + uses: actions/upload-artifact@v7 + with: + name: logs-${{ github.job }}-${{ github.run_id }}-${{ github.run_attempt }} + path: | + **/*.log + **/*.diffs + **/regress_log_* + **/crashlog-*.txt + if-no-files-found: ignore + + + # Linux, Autoconf + # + # SPECIAL: + # - Uses undefined & alignment sanitizers (sanitizer failures are typically + # printed in the server log) + # - Configures postgres with a small segment size + # - Uses PG_TEST_PG_COMBINEBACKUP_MODE=--copy-file-range + linux-autoconf: + name: Linux - Autoconf + needs: [setup, sanity-check] + if: &linux_job_if | + !cancelled() && + needs.setup.outputs.linux == 'true' && + needs.sanity-check.result != 'failure' + runs-on: ubuntu-latest + container: *linux_ci_container + timeout-minutes: 60 + + env: &linux_env + # Add both debian and ubuntu, as symbols from the host can be visible during profiling + DEBUGINFOD_URLS: "https://debuginfod.debian.net https://debuginfod.ubuntu.com"; + # Use -O2 to reduce the test times, use -fno-sanitize-recover=all to make sanitizer test + # failures visible. + CFLAGS: -O2 -ggdb -fno-sanitize-recover=all + CXXFLAGS: -O2 -ggdb -fno-sanitize-recover=all + LDFLAGS: + CC: ccache gcc + CXX: ccache g++ + CLANG: ccache clang + + # Configure sanitizer runtime behavior to be suitable for running tests: + # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes + # print_stacktraces=1,verbosity=2, duh + # detect_leaks=0: too many uninteresting leak errors in short-lived binaries + UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 + ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 + + steps: + # GitHub Actions does not make it easy to share some, but not all, + # environment variables between related tasks. We solve that for the + # linux- tasks by updating the environment variables programmatically. + - name: Update Environment + env: + SANITIZER_FLAGS: -fsanitize=alignment,undefined + PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range + run: &linux_update_config_cmd | + echo "CFLAGS=$CFLAGS ${SANITIZER_FLAGS}" >> "$GITHUB_ENV" + echo "CXXFLAGS=$CXXFLAGS ${SANITIZER_FLAGS}" >> "$GITHUB_ENV" + echo "LDFLAGS=$LDFLAGS ${SANITIZER_FLAGS}" >> "$GITHUB_ENV" + + echo "CC=${CC}" >> "$GITHUB_ENV" + echo "CXX=${CXX}" >> "$GITHUB_ENV" + + echo "PG_TEST_INITDB_EXTRA_OPTS=${PG_TEST_INITDB_EXTRA_OPTS}" >> "$GITHUB_ENV" + echo "PG_TEST_PG_COMBINEBACKUP_MODE=${PG_TEST_PG_COMBINEBACKUP_MODE}" >> "$GITHUB_ENV" + + - *nix_sysinfo_step + - *checkout_step + - *ccache_restore_step + - *linux_prepare_workspace + + - name: Configure + shell: *su_postgres_shell + run: | + ./configure \ + --enable-cassert --enable-injection-points --enable-debug \ + --enable-tap-tests --enable-nls \ + --with-segsize-blocks=6 \ + --with-libnuma \ + --with-liburing \ + ${LINUX_CONFIGURE_FEATURES} + + - name: Build + shell: *su_postgres_shell + run: | + make -s -j${BUILD_JOBS} world-bin + + - *ccache_save_step + + - name: Test world + shell: *su_postgres_shell + run: | + make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} + + - *linux_collect_cores + - *upload_logs_step + + + # Linux Meson, 32 bit + # + # SPECIAL: + # - Uses undefined behaviour and alignment sanitizers, (sanitizer failures + # are typically printed in the server log) + # - Uses io_method=io_uring + # - Uses meson feature autodetection + # - tests with LANG=C to give ICU some buildfarm-uncovered coverage. Also, + # newer Python insists on changing LC_CTYPE away from C, prevent that with + # PYTHONCOERCECLOCALE. + linux-meson-32: + name: Linux - Meson (32-bit) + needs: [setup, sanity-check] + if: *linux_job_if + runs-on: ubuntu-latest + container: *linux_ci_container + timeout-minutes: 60 + env: *linux_env + + steps: + - name: Update Environment + env: + SANITIZER_FLAGS: -fsanitize=alignment,undefined + PG_TEST_INITDB_EXTRA_OPTS: -c io_method=io_uring + CC: ccache gcc -m32 + CXX: ccache g++ -m32 + run: *linux_update_config_cmd + + - *nix_sysinfo_step + - *checkout_step + - *ccache_restore_step + - *linux_prepare_workspace + + - name: Configure + shell: *su_postgres_shell + run: | + meson setup \ + ${MESON_COMMON_PG_CONFIG_ARGS} \ + -Duuid=e2fs \ + --buildtype=debug \ + --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ + -DPERL=perl5.40-i386-linux-gnu \ + -Dlibnuma=disabled \ + build + + - name: Build + shell: *su_postgres_shell + run: *ninja_build_command + + - *ccache_save_step + + - name: Test world + shell: *su_postgres_shell + env: + PYTHONCOERCECLOCALE: 0 + LANG: C + run: *meson_test_world_cmd + + - *linux_collect_cores + - *upload_logs_step + + + # Linux Meson, 64 bit + # + # SPECIAL: + # - Uses address sanitizer, (sanitizer failures are typically printed in the + # server log). We test asan with meson rather than autoconf, as it's a bit + # faster at running the tests. + # - Uses io_method=io_uring + # - Uses meson feature autodetection + linux-meson-64: + name: Linux - Meson (64-bit) + needs: [setup, sanity-check] + if: *linux_job_if + runs-on: ubuntu-latest + container: *linux_ci_container + timeout-minutes: 60 + env: *linux_env + + steps: + - name: Update Environment + env: + SANITIZER_FLAGS: -fsanitize=address + PG_TEST_INITDB_EXTRA_OPTS: -c io_method=io_uring + run: *linux_update_config_cmd + + - *nix_sysinfo_step + - *checkout_step + - *ccache_restore_step + - *linux_prepare_workspace + + - name: Configure + shell: *su_postgres_shell + run: | + meson setup \ + ${MESON_COMMON_PG_CONFIG_ARGS} \ + -Duuid=e2fs \ + --buildtype=debug \ + -Dllvm=enabled \ + build + + - name: Build + shell: *su_postgres_shell + run: *ninja_build_command + + - *ccache_save_step + + - name: Test world + shell: *su_postgres_shell + run: *meson_test_world_cmd + + - *linux_collect_cores + - *upload_logs_step + + + # SPECIAL: + # - Enables --clone for pg_upgrade and pg_combinebackup + # - Specifies configuration options that test reading/writing/copying of node trees + # - Specifies debug_parallel_query=regress, to catch related issues during CI + macos: + name: macOS - Meson + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.macos == 'true' && + needs.sanity-check.result != 'failure' + runs-on: macos-15 + timeout-minutes: 60 + env: + MACPORTS_CACHE: ${{ github.workspace }}/macports-cache + + MESON_FEATURES: >- + -Dbonjour=enabled + -Ddtrace=enabled + -Dgssapi=enabled + -Dlibcurl=enabled + -Dnls=enabled + -Duuid=e2fs + + MACOS_PACKAGE_LIST: >- + ccache + icu + kerberos5 + lz4 + meson + openldap + openssl + p5.34-io-tty + p5.34-ipc-run + python312 + tcl + zstd + + CC: ccache cc + CXX: ccache c++ + CFLAGS: -Og -ggdb + CXXFLAGS: -Og -ggdb + PG_TEST_PG_UPGRADE_MODE: --clone + PG_TEST_PG_COMBINEBACKUP_MODE: --clone + + # Several buildfarm animals enable these options. Without testing them + # during CI, it would be easy to cause breakage on the buildfarm with CI + # passing. + PG_TEST_INITDB_EXTRA_OPTS: >- + -c debug_copy_parse_plan_trees=on + -c debug_write_read_parse_plan_trees=on + -c debug_raw_expression_coverage_test=on + -c debug_parallel_query=regress + + steps: + - *nix_sysinfo_step + + - *checkout_step + + - name: Setup core files + run: | + mkdir -p $HOME/cores + sudo sysctl kern.corefile="$HOME/cores/core.%P" + + - name: "Macports: Compute cache key" + id: mpkey + run: | + macos_major=$(sw_vers -productVersion | sed 's/\..*//') + pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) + script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) + echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + + - name: "MacPorts: Restore cache" + uses: actions/cache@v5 + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mpkey.outputs.key }} + + # Use MacPorts, even though Homebrew is installed. The installation + # of the additional packages we need would take quite a while with + # Homebrew, even if we cache the downloads. We can't cache all of + # Homebrew, because it's already large. So we use MacPorts. To cache + # the installation we create a .dmg file that we mount if it already + # exists. + # XXX: The reason for the direct p5.34* references is that we'd need + # the large MacPort tree around to figure out that p5-io-tty is + # actually p5.34-io-tty. Using the unversioned name works, but + # updates MacPorts every time. + - name: "MacPorts: Install dependencies" + env: + # Pass token so the script's GitHub API call to list MacPorts + # releases isn't subject to the 60/h/IP unauthenticated rate + # limit (shared across all jobs on the runner's IP). + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST + # system python doesn't provide headers + sudo /opt/local/bin/port select python3 python312 + # Make macports install visible to subsequent steps + echo /opt/local/sbin >> "$GITHUB_PATH" + echo /opt/local/bin >> "$GITHUB_PATH" + + - *ccache_restore_step + + - name: Configure + env: + PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ + run: | + meson setup \ + ${{env.MESON_COMMON_PG_CONFIG_ARGS}} \ + --buildtype=debug \ + -Dextra_include_dirs=/opt/local/include \ + -Dextra_lib_dirs=/opt/local/lib \ + -Ddarwin_sysroot=none \ + ${MESON_COMMON_FEATURES} \ + ${MESON_FEATURES} \ + build + + - name: Build + run: *ninja_build_command + + - *ccache_save_step + + - name: Test world + env: + # default is 256, pretty low + ADDITIONAL_SETUP: ulimit -n 1024 + run: *meson_test_world_cmd + + - name: Core backtraces + if: failure() && !cancelled() + run: src/tools/ci/cores_backtrace.sh macos "$HOME/cores" + + - *upload_logs_step + + + windows-vs: + name: Windows - Visual Studio + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.windows == 'true' && + needs.sanity-check.result != 'failure' + runs-on: windows-2022 + timeout-minutes: 60 + env: + # Avoid port conflicts between concurrent tap tests + PG_TEST_USE_UNIX_SOCKETS: 1 + PG_REGRESS_SOCK_DIR: 'd:\pgsock' + TAR: "c:/windows/system32/tar.exe" + + MESON_FEATURES: >- + -Dcpp_args=/std:c++20 + -Dauto_features=disabled + -Dtap_tests=enabled + -Dldap=enabled + -Dssl=openssl + -Dplperl=enabled + -Dplpython=enabled + + defaults: + run: + shell: cmd + steps: + - &windows_disable_defender + name: Disable Windows Defender + shell: powershell + run: | + Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable + # Verify Defender status + $status = Get-MpComputerStatus -ErrorAction SilentlyContinue + if ($status) { + Write-Host "RealTimeProtectionEnabled: $($status.RealTimeProtectionEnabled)" + Write-Host "AntivirusEnabled: $($status.AntivirusEnabled)" + } + + - *checkout_step + + - name: Sysinfo + run: | + chcp + systeminfo + set + + # The TAP tests build an initdb template under build/tmp_install and + # then `robocopy` it into per-test data directories. Robocopy with the + # default /COPY:DAT flag doesn't copy ACLs — destinations inherit from + # their parent dir. On GitHub-hosted Windows runners the workspace's + # inherited ACL grants Administrators:(F) and Users:(RX) but does NOT + # grant the runner user (runneradmin) directly. That matters because + # pg_ctl on Windows uses CreateRestrictedProcess to drop admin + # privileges from postmaster, so the postmaster process has the user + # SID in its token but no longer the Administrators group — leaving it + # with only "Users:(RX)" on pg_control and friends, which causes + # "PANIC: could not open file global/pg_control: Permission denied". + # + # Fix it once on the workspace dir with (OI)(CI) inheritance flags so + # every file/dir created underneath gets an explicit grant for the + # current user. + - name: Grant workspace ACL to runner user + shell: pwsh + run: | + icacls "${{ github.workspace }}" /grant "${env:USERNAME}:(OI)(CI)F" /Q | Out-Null + Write-Host "Granted Full Control to $env:USERNAME on ${{ github.workspace }}" + + # postgres' plpython3u loads python3.dll (the stable-ABI forwarder) + # which in turn loads whichever python3NN.dll the Windows loader finds + # first on PATH. On windows-2022 `C:\Program Files\Mercurial\` ships + # its own python3.dll + python39.dll and appears on PATH *before* the + # hostedtoolcache Python 3.12 — so without intervention the backend + # ends up running Python 3.9 while postgres' stdlib search uses 3.12, + # producing `ImportError: cannot import name 'text_encoding' from + # 'io'` (the 3.12 `io.py` calling into 3.9's `_io`). + # + # Drop Mercurial's directory from PATH so the hostedtoolcache + # python3.dll wins the DLL search. + - name: Remove Mercurial from PATH + shell: pwsh + run: | + $filtered = ($env:PATH -split ';' | + Where-Object { $_ -and ($_ -notmatch '\\Mercurial\\?$') }) -join ';' + Add-Content $env:GITHUB_ENV "PATH=$filtered" + Write-Host "Removed Mercurial entries from PATH" + + - name: Install dependencies + shell: pwsh + run: | + choco install -y --no-progress --limitoutput diffutils winflexbison3 + # meson + ninja aren't preinstalled on windows-2022. Install via pip + python -m pip install --upgrade meson ninja + + # OpenSSL 1.1 via the slproweb installer (pinned to match the + # version used elsewhere in postgres CI). + curl.exe -fsSL -o openssl-setup.exe https://slproweb.com/download/Win64OpenSSL-1_1_1w.exe + Start-Process -Wait -FilePath ./openssl-setup.exe ` + -ArgumentList '/DIR=d:\openssl\1.1\ /VERYSILENT /SP- /SUPPRESSMSGBOXES' + # The slproweb installer puts libcrypto-1_1-x64.dll / libssl-1_1-x64.dll + # in d:\openssl\1.1\bin\ and updates the system PATH. GH Actions + # snapshots PATH at job start though, so the running job won't + # see those DLLs and initdb.exe would crash silently at runtime. + # Push the bin dir onto GITHUB_PATH so it persists for later steps. + Add-Content $env:GITHUB_PATH "d:\openssl\1.1\bin" + + # Install IPC::Run. + # - recommends_policy=0 keeps cpan from pulling in IO::Tty / IO::Pty, + # which don't build on Windows ("This module requires a POSIX + # compliant system to work"). + # - Pin to NJM/IPC-Run-20250809.0 because TODDR/IPC-Run-20260322.0 + # broke postgres tap tests on Windows (changed pipe stdio + # handling). See upstream pg-vm-images commit ff5238afa3 and + # the thread at + # https://postgr.es/m/CAN55FZ06xanSbJdHe-CurjX_qNuBWZDEvS1kAk36L38YCtZXnw%40mail.gmail.com + "o conf recommends_policy 0`no conf commit`nnotest install NJM/IPC-Run-20250809.0.tar.gz" | cpan + perl -mIPC::Run -e 1 + + - name: Setup hosts file + shell: pwsh + run: | + Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.1 pg-loadbalancetest" + Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.2 pg-loadbalancetest" + Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.3 pg-loadbalancetest" + + - name: Setup socket directory + shell: cmd + run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + + - name: Configure + run: | + call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 + meson setup ^ + --backend ninja ^ + ${{env.MESON_COMMON_PG_CONFIG_ARGS}} ^ + ${{env.MESON_FEATURES}} ^ + --buildtype debug ^ + -Db_pch=true ^ + -Dextra_lib_dirs=d:\openssl\1.1\lib -Dextra_include_dirs=d:\openssl\1.1\include ^ + -DTAR=${{env.TAR}} ^ + build + + - name: Build + run: | + call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 + ninja -C build ${{env.MBUILD_TARGET}} + ninja -C build -t missingdeps + + - name: Test world + env: + ADDITIONAL_SETUP: | + call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 + run: *meson_test_world_cmd + + # TODO: We need to collect crashlogs but for them to be generated, we'd + # have to configure the JIT Debugger to do so. cdb.exe is installed on + # the runner so that is possible. + - *upload_logs_step + + + windows-mingw: + name: Windows - MinGW - Meson + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.mingw == 'true' && + needs.sanity-check.result != 'failure' + runs-on: windows-2022 + timeout-minutes: 60 + env: + # Avoid port conflicts between concurrent tap tests + PG_TEST_USE_UNIX_SOCKETS: 1 + PG_REGRESS_SOCK_DIR: 'd:\pgsock' + TAR: "c:/windows/system32/tar.exe" + + MSYS: winjitdebug + CHERE_INVOKING: 1 + MSYSTEM: UCRT64 + + # Keep -Dnls explicitly disabled, as the number of files it creates + # causes a noticeable slowdown. + MESON_FEATURES: >- + -Dnls=disabled + + CCACHE_MAXSIZE: "500M" + CCACHE_SLOPPINESS: pch_defines,time_macros + CCACHE_DEPEND: 1 + + defaults: + run: + shell: 'D:\msys64\usr\bin\bash.exe --login -eo pipefail "{0}"' + + steps: + - *windows_disable_defender + - *checkout_step + + # Relocate the preinstalled MSYS2 tree from C:\ (slow system disk) to + # D:\ (faster ephemeral data disk). Every subsequent MSYS2 step uses + # D:\msys64\usr\bin\bash.exe via the job's `defaults.run.shell`. + # + # This reduces the total runtime of this task by ~15 minutes. + # + # robocopy returns 0-7 on success (with various "files copied" bits + # set) and 8+ on real failure, so we have to translate its exit code. + - name: Relocate MSYS2 to D + shell: powershell + run: | + robocopy C:\msys64 D:\msys64 /E /MT:16 /NJS /NJH /NFL /NDL /NP + if ($LASTEXITCODE -ge 8) { exit $LASTEXITCODE } + exit 0 + + - name: Setup MSYS2 + run: | + # ${MINGW_PACKAGE_PREFIX} is an environment variable used in the + # MSYS2. It dynamically expands to the correct prefix for the active + # shell environment. + pacman -S --noconfirm --needed \ + git bison flex make diffutils \ + ${MINGW_PACKAGE_PREFIX}-ccache \ + ${MINGW_PACKAGE_PREFIX}-gcc \ + ${MINGW_PACKAGE_PREFIX}-icu \ + ${MINGW_PACKAGE_PREFIX}-libbacktrace \ + ${MINGW_PACKAGE_PREFIX}-libxml2 \ + ${MINGW_PACKAGE_PREFIX}-libxslt \ + ${MINGW_PACKAGE_PREFIX}-lz4 \ + ${MINGW_PACKAGE_PREFIX}-make \ + ${MINGW_PACKAGE_PREFIX}-meson \ + ${MINGW_PACKAGE_PREFIX}-perl \ + ${MINGW_PACKAGE_PREFIX}-pkg-config \ + ${MINGW_PACKAGE_PREFIX}-readline \ + ${MINGW_PACKAGE_PREFIX}-zlib + + - *nix_sysinfo_step + + - name: Install additional dependencies + run: | + # Pin IPC::Run to NJM/IPC-Run-20250809.0; TODDR/IPC-Run-20260322.0 + # broke postgres tap tests on Windows (pipe stdio handling). + # See pg-vm-images commit ff5238afa3. + (echo; echo o conf recommends_policy 0; echo notest install NJM/IPC-Run-20250809.0.tar.gz) | cpan + perl -mIPC::Run -e 1 + + - name: Setup socket directory + shell: cmd + run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + + - *ccache_restore_step + + - name: Configure + run: | + meson setup \ + ${{env.MESON_COMMON_PG_CONFIG_ARGS}} \ + -Ddebug=true -Doptimization=g -Db_pch=true \ + ${{env.MESON_COMMON_FEATURES}} \ + ${{env.MESON_FEATURES}} \ + -DTAR=${{env.TAR}} \ + build + + - name: Build + run: *ninja_build_command + + - *ccache_save_step + + - name: Test world + run: *meson_test_world_cmd + + # FIX: We need to collect crashlogs but they are not collected. cdb.exe + # is installed on the runner so it needs to be configured. + - *upload_logs_step + + + # Test that code can be built with both gcc and clang without warnings, + # with various combinations of cassert/dtrace flags. Trace probes have + # a history of getting accidentally broken; the matrix is there to + # catch that. + # + # The autoconf cache files (gcc.cache / clang.cache) are intentionally + # reused across the matrix entries that share a compiler, so we don't + # pay for full feature detection on every entry. + compiler-warnings: + name: CompilerWarnings + needs: [setup, sanity-check] + if: | + !cancelled() && + needs.setup.outputs.compilerwarnings == 'true' && + needs.sanity-check.result != 'failure' + runs-on: ubuntu-latest + timeout-minutes: 60 + container: + image: ${{ needs.setup.outputs.container_linux_ci_docs }} + env: + # Use larger ccache cache as this job compiles with multiple + # compilers / flag combinations. + CCACHE_MAXSIZE: "1G" + DEFAULT_BUILD: world-bin + steps: + + - name: Sysinfo + run: | + id + uname -a + cat /proc/cmdline + ulimit -a -H && ulimit -a -S + gcc -v + clang -v + env + + - *checkout_step + + - *ccache_restore_step + + - name: Setup workspace + run: | + echo "COPT=-Werror" > src/Makefile.custom + + # gcc, cassert off, dtrace on + - name: gcc warnings + (dtrace) + if: ${{ !cancelled() }} + env: + CONF: ${{env.LINUX_CONFIGURE_FEATURES}} --cache gcc.cache --enable-dtrace + CC: ccache gcc + CXX: ccache g++ + CLANG: ccache clang + run: &compiler_warnings_cmd | + echo "::group::configure" + ./configure \ + ${{env.CONF}} \ + CLANG="ccache clang" + echo "::endgroup::" + + make -s -j${{env.BUILD_JOBS}} clean + make -s -j${{env.BUILD_JOBS}} ${{env.DEFAULT_BUILD}} + + + # gcc, cassert on, dtrace off + - name: gcc warnings + (cassert) + if: ${{ !cancelled() }} + env: + CONF: ${{env.LINUX_CONFIGURE_FEATURES}} --cache gcc.cache --enable-cassert + CC: ccache gcc + CXX: ccache g++ + run: *compiler_warnings_cmd + + + # clang, cassert off, dtrace off + - name: clang warnings + if: ${{ !cancelled() }} + env: + CONF: ${{env.LINUX_CONFIGURE_FEATURES}} --cache clang.cache + CC: ccache clang + CXX: ccache clang++ + run: *compiler_warnings_cmd + + + # clang, cassert on, dtrace on + - name: clang warnings + (cassert + dtrace) + if: ${{ !cancelled() }} + env: + CONF: ${{env.LINUX_CONFIGURE_FEATURES}} --cache clang.cache --enable-cassert --enable-dtrace + CC: ccache clang + CXX: ccache clang++ + run: *compiler_warnings_cmd + + + - name: mingw warnings (cross compilation) + if: ${{ !cancelled() }} + env: + CONF: --host=x86_64-w64-mingw32ucrt --enable-cassert --without-icu + CC: ccache x86_64-w64-mingw32ucrt-gcc + CXX: ccache x86_64-w64-mingw32ucrt-g++ + run: *compiler_warnings_cmd + + + ### + # Verify docs can be built + ### + # XXX: Only do this if there have been changes in doc/ since last build + - name: Build documentation + if: ${{ !cancelled() }} + env: + CONF: --cache gcc.cache + CC: ccache gcc + CXX: ccache g++ + DEFAULT_BUILD: -C doc + run: *compiler_warnings_cmd + + ### + # Verify headerscheck / cpluspluscheck succeed + # + # - Run both in same script to increase parallelism, use -k to get + # result of both + # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose + ### + - name: headerscheck + cpluspluscheck + if: ${{ !cancelled() }} + run: | + echo "::group::configure" + ./configure \ + ${{env.LINUX_CONFIGURE_FEATURES}} \ + --cache gcc.cache \ + --quiet \ + CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" + echo "::endgroup::" + + make -s -j${{env.BUILD_JOBS}} clean + make -s -j${{env.BUILD_JOBS}} -k ${{env.CHECKFLAGS}} \ + headerscheck cpluspluscheck \ + EXTRAFLAGS='-fmax-errors=10' + + - *ccache_save_step diff --git a/src/tools/ci/ci_macports_packages.sh b/src/tools/ci/ci_macports_packages.sh index 63e97b37c78..c7c4a1c0c60 100755 --- a/src/tools/ci/ci_macports_packages.sh +++ b/src/tools/ci/ci_macports_packages.sh @@ -20,13 +20,26 @@ echo "macOS major version: $macos_major_version" # macOS release. macports_release_list_url="https://api.github.com/repos/macports/macports-base/releases"; macports_version_pattern="2\.10\.1" -macports_url="$( curl -s $macports_release_list_url | grep "\"https://github.com/macports/macports-base/releases/download/v$macports_version_pattern/MacPorts-$macports_version_pattern-$macos_major_version-[A-Za-z]*\.pkg\""; | sed 's/.*: "//;s/".*//' | head -1 )" +# Authenticate the GitHub API request when a token is available (e.g. on +# GitHub Actions). Unauthenticated requests share a 60/h/IP rate limit +# with every other job on the runner's IP and frequently return an error +# JSON, leaving $macports_url empty and breaking the subsequent curl. +auth_header="" +if [ -n "$GITHUB_TOKEN" ]; then + auth_header="Authorization: Bearer $GITHUB_TOKEN" +fi +macports_url="$( curl -fsSL ${auth_header:+-H "$auth_header"} "$macports_release_list_url" | grep "\"https://github.com/macports/macports-base/releases/download/v$macports_version_pattern/MacPorts-$macports_version_pattern-$macos_major_version-[A-Za-z]*\.pkg\""; | sed 's/.*: "//;s/".*//' | head -1 )" echo "MacPorts package URL: $macports_url" +if [ -z "$macports_url" ]; then + echo "error: could not determine MacPorts package URL for macOS $macos_major_version (version pattern: $macports_version_pattern)" 1>&2 + exit 1 +fi + cache_dmg="macports.hfs.dmg" -if [ "$CIRRUS_CI" != "true" ]; then - echo "expect to be called within cirrus-ci" 1>2 +if [ "$CIRRUS_CI" != "true" ] && [ "$GITHUB_ACTIONS" != "true" ]; then + echo "expect to be called within cirrus-ci or github actions" 1>&2 exit 1 fi -- 2.54.0.380.gc69baaf57b
>From ad8ecd09261a5b370b0a53c3ca1e5f7dd415b195 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Tue, 2 Jun 2026 19:07:55 -0400 Subject: [PATCH v9a 02/22] ci: Rewrite src/tools/ci/README To be merged with the prior commit. --- src/tools/ci/README | 77 +++++++++++++-------------------------------- 1 file changed, 21 insertions(+), 56 deletions(-) diff --git a/src/tools/ci/README b/src/tools/ci/README index d183648a8d0..8776d82ffcc 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -17,42 +17,35 @@ Postgres has two forms of CI: Configuring CI on personal repositories ======================================= -Currently postgres contains CI support utilizing cirrus-ci. cirrus-ci -currently is only available for github. +Currently postgres contains CI support utilizing GitHub actions. -Enabling cirrus-ci in a github repository -========================================= +Enabling and using CI in a GitHub repository +============================================ -To enable cirrus-ci on a repository, go to -https://github.com/marketplace/cirrus-ci and select "Public -Repositories". Then "Install it for free" and "Complete order". The next page -allows to configure which repositories cirrus-ci has access to. Choose the -relevant repository and "Install". +The GitHub Actions based CI workflow is active by default, therefore no +configuration is necessary. -See also https://cirrus-ci.org/guide/quick-start/ +CI runs are visible at https://github.com/<username>/<reponame>/actions -Once enabled on a repository, future commits and pull-requests in that -repository will automatically trigger CI builds. These are visible from the -commit history / PRs, and can also be viewed in the cirrus-ci UI at -https://cirrus-ci.com/github/<username>/<reponame>/ +The high-level status of workflow runs on public repositories are visible +without being logged into GitHub, however details including logs require being +logged in. -Hint: all build log files are uploaded to cirrus-ci and can be downloaded -from the "Artifacts" section from the cirrus-ci UI after clicking into a -specific task on a build's summary page. +To disable CI on a repository, navigate to +https://github.com/<username>/<reponame>/actions/workflows/pg-ci.yml +and click on the '...' on the right and choose 'Disable workflow'. +Containers / Images used for CI +=============================== -Images used for CI -================== +To keep CI times tolerable, several platforms use pre-generated containers / +images. The containers and images are generated separately from CI runs, +otherwise each git repository that is being tested would need to build its own +set of containers, which would be wasteful (both in space and time). -To keep CI times tolerable, most platforms use pre-generated images. Some -platforms use containers, others use full VMs. Images for both are generated -separately from CI runs, otherwise each git repository that is being tested -would need to build its own set of containers, which would be wasteful (both -in space and time. - -These images are built, on a daily basis, from the specifications in -github.com/anarazel/pg-vm-images/ +These containers / images are built, on a daily basis, from the specifications +in github.com/anarazel/pg-vm-images/ Controlling CI via commit messages @@ -61,35 +54,7 @@ Controlling CI via commit messages The behavior of CI can be controlled by special content in commit messages. Currently the following controls are available: -- ci-os-only: {(freebsd|linux|macos|mingw|netbsd|openbsd|windows)} +- ci-os-only: {(compilerwarnings|linux|macos|mingw|sanitycheck|windows)} Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. - - -Using custom compute resources for CI -===================================== - -When running a lot of tests in a repository, cirrus-ci's free credits do not -suffice. In those cases a repository can be configured to use other -infrastructure for running tests. To do so, the REPO_CI_CONFIG_GIT_URL -variable can be configured for the repository in the cirrus-ci web interface, -at https://cirrus-ci.com/github/<user or organization>. The file referenced -(see https://cirrus-ci.org/guide/programming-tasks/#fs) by the variable can -overwrite the default execution method for different operating systems, -defined in .cirrus.yml, by redefining the relevant yaml anchors. - -Custom compute resources can be provided using -- https://cirrus-ci.org/guide/supported-computing-services/ -- https://cirrus-ci.org/guide/persistent-workers/ - - -Enabling manual tasks by default -================================ - -Some tasks are not triggered automatically by default, to avoid using up CI -credits too quickly. This can be changed on the repository level, e.g. when -custom compute resources are configured. - -The following repository level environment variables are recognized: -- REPO_CI_AUTOMATIC_TRIGGER_TASKS - space-separated list of (mingw|netbsd|openbsd) -- 2.54.0.380.gc69baaf57b
>From e8fadbaaddddc1d1a66ec2f094f4a6c98f7dacf7 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Thu, 28 May 2026 13:31:41 -0400 Subject: [PATCH v9a 03/22] ci: Remove support for cirrus-ci based CI As mentioned in the earlier commit, cirrus-ci has shut down. Therefore remove all files related to running CI via cirrus. Also update comments / code that were referencing cirrus-ci. Discussion: https://postgr.es/m/3ydjipcr7kbss57nvi67noplncqhesl5eyb6wgol4ccjxynspv%40yatlykpribmm --- src/bin/pg_combinebackup/t/010_hardlink.pl | 12 +- .cirrus.yml | 91 -- src/test/perl/PostgreSQL/Test/Cluster.pm | 5 +- .cirrus.star | 143 --- .cirrus.tasks.yml | 1022 -------------------- src/tools/ci/ci_macports_packages.sh | 6 +- src/tools/ci/gcp_ram_disk.sh | 27 - 7 files changed, 11 insertions(+), 1295 deletions(-) delete mode 100644 .cirrus.yml delete mode 100644 .cirrus.star delete mode 100644 .cirrus.tasks.yml delete mode 100755 src/tools/ci/gcp_ram_disk.sh diff --git a/src/bin/pg_combinebackup/t/010_hardlink.pl b/src/bin/pg_combinebackup/t/010_hardlink.pl index b6e8a9128af..5fbbe6a923b 100644 --- a/src/bin/pg_combinebackup/t/010_hardlink.pl +++ b/src/bin/pg_combinebackup/t/010_hardlink.pl @@ -18,13 +18,13 @@ $primary->append_conf('postgresql.conf', 'autovacuum = off'); $primary->start; # Create a couple of tables (~264KB each). -# Note: Cirrus CI runs some tests with a very small segment size, so, in that +# Note: CI runs some tests with a very small segment size, so, in that # environment, a single table of 264KB would have both a segment with a link -# count of 1 and also one with a link count of 2. But in a normal installation, -# segment size is 1GB. Therefore, we use 2 different tables here: for test_1, -# all segments (or the only one) will have two hard links; for test_2, the -# last segment (or the only one) will have 1 hard link, and any others will -# have 2. +# count of 1 and also one with a link count of 2. But in a normal +# installation, segment size is 1GB. Therefore, we use 2 different tables +# here: for test_1, all segments (or the only one) will have two hard links; +# for test_2, the last segment (or the only one) will have 1 hard link, and +# any others will have 2. my $query = <<'EOM'; CREATE TABLE test_%s AS SELECT x.id::bigint, diff --git a/.cirrus.yml b/.cirrus.yml deleted file mode 100644 index 3f75852e84e..00000000000 --- a/.cirrus.yml +++ /dev/null @@ -1,91 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# The actual CI tasks are defined in .cirrus.tasks.yml. To make the compute -# resources for CI configurable on a repository level, the "final" CI -# configuration is the combination of: -# -# 1) the contents of this file -# -# 2) computed environment variables -# -# Used to enable/disable tasks based on the execution environment. See -# .cirrus.star: compute_environment_vars() -# -# 3) if defined, the contents of the file referenced by the, repository -# level, REPO_CI_CONFIG_GIT_URL variable (see -# https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted -# format) -# -# This allows running tasks in a different execution environment than the -# default, e.g. to have sufficient resources for cfbot. -# -# 4) .cirrus.tasks.yml -# -# This composition is done by .cirrus.star - - -env: - # Source of images / containers - GCP_PROJECT: pg-ci-images - IMAGE_PROJECT: $GCP_PROJECT - CONTAINER_REPO: us-docker.pkg.dev/${GCP_PROJECT}/ci - DISK_SIZE: 25 - - -# Define how to run various types of tasks. - -# VMs provided by cirrus-ci. Each user has a limited number of "free" credits -# for testing. -cirrus_community_vm_template: &cirrus_community_vm_template - compute_engine_instance: - image_project: $IMAGE_PROJECT - image: family/$IMAGE_FAMILY - platform: $PLATFORM - cpu: $CPUS - disk: $DISK_SIZE - - -default_linux_task_template: &linux_task_template - env: - PLATFORM: linux - <<: *cirrus_community_vm_template - - -default_freebsd_task_template: &freebsd_task_template - env: - PLATFORM: freebsd - <<: *cirrus_community_vm_template - -default_netbsd_task_template: &netbsd_task_template - env: - PLATFORM: netbsd - <<: *cirrus_community_vm_template - -default_openbsd_task_template: &openbsd_task_template - env: - PLATFORM: openbsd - <<: *cirrus_community_vm_template - - -default_windows_task_template: &windows_task_template - env: - PLATFORM: windows - <<: *cirrus_community_vm_template - - -# macos workers provided by cirrus-ci -default_macos_task_template: &macos_task_template - env: - PLATFORM: macos - macos_instance: - image: $IMAGE - - -# Contents of REPO_CI_CONFIG_GIT_URL, if defined, will be inserted here, -# followed by the contents .cirrus.tasks.yml. This allows -# REPO_CI_CONFIG_GIT_URL to override how the task types above will be -# executed, e.g. using a custom compute account or permanent workers. diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm index 4fcb1f6be56..529f49efee1 100644 --- a/src/test/perl/PostgreSQL/Test/Cluster.pm +++ b/src/test/perl/PostgreSQL/Test/Cluster.pm @@ -363,9 +363,8 @@ This tries to connect to the server, to test whether it works or not,, so the server is up and running. Otherwise this can return 0 even if there's nothing wrong with raw_connect() itself. -Notably, raw_connect() does not work on Unix domain sockets on -Strawberry perl 5.26.3.1 on Windows, which we use in Cirrus CI images -as of this writing. It dies with "not implemented on this +Notably, raw_connect() does not work on Unix domain sockets on at least +Strawberry perl 5.26.3.1 on Windows. It dies with "not implemented on this architecture". =cut diff --git a/.cirrus.star b/.cirrus.star deleted file mode 100644 index e9bb672b959..00000000000 --- a/.cirrus.star +++ /dev/null @@ -1,143 +0,0 @@ -"""Additional CI configuration, using the starlark language. See -https://cirrus-ci.org/guide/programming-tasks/#introduction-into-starlark - -See also the starlark specification at -https://github.com/bazelbuild/starlark/blob/master/spec.md - -See also .cirrus.yml and src/tools/ci/README -""" - -load("cirrus", "env", "fs", "re", "yaml") - - -def main(): - """The main function is executed by cirrus-ci after loading .cirrus.yml and can - extend the CI definition further. - - As documented in .cirrus.yml, the final CI configuration is composed of - - 1) the contents of .cirrus.yml - - 2) computed environment variables - - 3) if defined, the contents of the file referenced by the, repository - level, REPO_CI_CONFIG_GIT_URL variable (see - https://cirrus-ci.org/guide/programming-tasks/#fs for the accepted - format) - - 4) .cirrus.tasks.yml - """ - - output = "" - - # 1) is evaluated implicitly - - - # Add 2) - additional_env = compute_environment_vars() - env_fmt = """ -### -# Computed environment variables start here -### -{0} -### -# Computed environment variables end here -### -""" - output += env_fmt.format(yaml.dumps({'env': additional_env})) - - - # Add 3) - repo_config_url = env.get("REPO_CI_CONFIG_GIT_URL") - if repo_config_url != None: - print("loading additional configuration from \"{}\"".format(repo_config_url)) - output += config_from(repo_config_url) - else: - output += "\n# REPO_CI_CONFIG_URL was not set\n" - - - # Add 4) - output += config_from(".cirrus.tasks.yml") - - - return output - - -def compute_environment_vars(): - cenv = {} - - ### - # Some tasks are manually triggered by default because they might use too - # many resources for users of free Cirrus credits, but they can be - # triggered automatically by naming them in an environment variable e.g. - # REPO_CI_AUTOMATIC_TRIGGER_TASKS="task_name other_task" under "Repository - # Settings" on Cirrus CI's website. - - default_manual_trigger_tasks = ['mingw', 'netbsd', 'openbsd'] - - repo_ci_automatic_trigger_tasks = env.get('REPO_CI_AUTOMATIC_TRIGGER_TASKS', '') - for task in default_manual_trigger_tasks: - name = 'CI_TRIGGER_TYPE_' + task.upper() - if repo_ci_automatic_trigger_tasks.find(task) != -1: - value = 'automatic' - else: - value = 'manual' - cenv[name] = value - ### - - ### - # Parse "ci-os-only:" tag in commit message and set - # CI_{$OS}_ENABLED variable for each OS - - # We want to disable SanityCheck if testing just a specific OS. This - # shortens push-wait-for-ci cycle time a bit when debugging operating - # system specific failures. Just treating it as an OS in that case - # suffices. - - operating_systems = [ - 'compilerwarnings', - 'freebsd', - 'linux', - 'macos', - 'mingw', - 'netbsd', - 'openbsd', - 'sanitycheck', - 'windows', - ] - commit_message = env.get('CIRRUS_CHANGE_MESSAGE') - match_re = r"(^|.*\n)ci-os-only: ([^\n]+)($|\n.*)" - - # re.match() returns an array with a tuple of (matched-string, match_1, ...) - m = re.match(match_re, commit_message) - if m and len(m) > 0: - os_only = m[0][2] - os_only_list = re.split(r'[, ]+', os_only) - else: - os_only_list = operating_systems - - for os in operating_systems: - os_enabled = os in os_only_list - cenv['CI_{0}_ENABLED'.format(os.upper())] = os_enabled - ### - - return cenv - - -def config_from(config_src): - """return contents of config file `config_src`, surrounded by markers - indicating start / end of the included file - """ - - config_contents = fs.read(config_src) - config_fmt = """ - -### -# contents of config file `{0}` start here -### -{1} -### -# contents of config file `{0}` end here -### -""" - return config_fmt.format(config_src, config_contents) diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml deleted file mode 100644 index 8683d1ae9c7..00000000000 --- a/.cirrus.tasks.yml +++ /dev/null @@ -1,1022 +0,0 @@ -# CI configuration file for CI utilizing cirrus-ci.org -# -# For instructions on how to enable the CI integration in a repository and -# further details, see src/tools/ci/README -# -# -# NB: Different tasks intentionally test with different, non-default, -# configurations, to increase the chance of catching problems. Each task with -# non-obvious non-default documents their oddity at the top of the task, -# prefixed by "SPECIAL:". - - -env: - # The lower depth accelerates git clone. Use a bit of depth so that - # concurrent tasks and retrying older jobs have a chance of working. - CIRRUS_CLONE_DEPTH: 500 - # Useful to be able to analyse what in a script takes long - CIRRUS_LOG_TIMESTAMP: true - - CCACHE_MAXSIZE: "250M" - - # target to test, for all but windows - CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS - CHECKFLAGS: -Otarget - PROVE_FLAGS: --timer - # Build test dependencies as part of the build step, to see compiler - # errors/warnings in one place. - MBUILD_TARGET: all testprep - MTEST_ARGS: --print-errorlogs --no-rebuild -C build - PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests - TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf - PG_TEST_EXTRA: kerberos ldap ssl libpq_encryption load_balance oauth - - # Postgres config args for the meson builds, shared between all meson tasks - # except the 'SanityCheck' task - MESON_COMMON_PG_CONFIG_ARGS: -Dcassert=true -Dinjection_points=true - - # Meson feature flags shared by all meson tasks, except: - # SanityCheck: uses almost no dependencies. - # Windows - VS: has fewer dependencies than listed here, so defines its own. - # Linux: uses the 'auto' feature option to test meson feature autodetection. - MESON_COMMON_FEATURES: >- - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - -Ddocs=enabled - -Dicu=enabled - -Dlibxml=enabled - -Dlibxslt=enabled - -Dlz4=enabled - -Dpltcl=enabled - -Dreadline=enabled - -Dzlib=enabled - -Dzstd=enabled - - -# What files to preserve in case tests fail -on_failure_ac: &on_failure_ac - log_artifacts: - paths: - - "**/*.log" - - "**/*.diffs" - - "**/regress_log_*" - type: text/plain - -on_failure_meson: &on_failure_meson - testrun_artifacts: - paths: - - "build*/testrun/**/*.log" - - "build*/testrun/**/*.diffs" - - "build*/testrun/**/regress_log_*" - type: text/plain - - # In theory it'd be nice to upload the junit files meson generates, so that - # cirrus will nicely annotate the commit. Unfortunately the files don't - # contain identifiable file + line numbers right now, so the annotations - # don't end up useful. We could probably improve on that with a some custom - # conversion script, but ... - meson_log_artifacts: - path: "build*/meson-logs/*.txt" - type: text/plain - - -# To avoid unnecessarily spinning up a lot of VMs / containers for entirely -# broken commits, have a minimal task that all others depend on. -# -# SPECIAL: -# - Builds with --auto-features=disabled and thus almost no enabled -# dependencies -task: - name: SanityCheck - - # If a specific OS is requested, don't run the sanity check. This shortens - # push-wait-for-ci cycle time a bit when debugging operating system specific - # failures. Uses skip instead of only_if, as cirrus otherwise warns about - # only_if conditions not matching. - skip: $CI_SANITYCHECK_ENABLED == false - - env: - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-trixie - CCACHE_DIR: ${CIRRUS_WORKING_DIR}/ccache_dir - # no options enabled, should be small - CCACHE_MAXSIZE: "150M" - - # While containers would start up a bit quicker, building is a bit - # slower. This way we don't have to maintain a container image. - <<: *linux_task_template - - ccache_cache: - folder: $CCACHE_DIR - - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - # Can't change container's kernel.core_pattern. Postgres user can't write - # to / normally. Change that. - chown root:postgres / - chmod g+rwx / - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - --buildtype=debug \ - --auto-features=disabled \ - -Ddefault_library=shared \ - -Dtap_tests=enabled \ - build - EOF - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - EOF - upload_caches: ccache - - # Run a minimal set of tests. The main regression tests take too long for - # this purpose. For now this is a random quick pg_regress style test, and a - # tap test that exercises both a frontend binary and the backend. - test_minimal_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --suite setup - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} \ - cube/regress pg_ctl/001_start_stop - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - mkdir -m 770 /tmp/cores - find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \; - src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# SPECIAL: -# - Uses postgres specific CPPFLAGS that increase test coverage -# - Specifies configuration options that test reading/writing/copying of node trees -# - Specifies debug_parallel_query=regress, to catch related issues during CI -# - Also runs tests against a running postgres instance, see test_running_script -task: - name: FreeBSD - Meson - - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 - IMAGE_FAMILY: pg-ci-freebsd - DISK_SIZE: 50 - - CCACHE_DIR: /tmp/ccache_dir - CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS - CFLAGS: -Og -ggdb - - # Several buildfarm animals enable these options. Without testing them - # during CI, it would be easy to cause breakage on the buildfarm with CI - # passing. - PG_TEST_INITDB_EXTRA_OPTS: >- - -c debug_copy_parse_plan_trees=on - -c debug_write_read_parse_plan_trees=on - -c debug_raw_expression_coverage_test=on - -c debug_parallel_query=regress - PG_TEST_PG_UPGRADE_MODE: --link - - MESON_FEATURES: >- - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - -Dtcl_version=tcl86 - -Duuid=bsd - - <<: *freebsd_task_template - - depends_on: SanityCheck - only_if: $CI_FREEBSD_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - pw useradd postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kern.corefile='/tmp/cores/%N.%P.core' - setup_additional_packages_script: | - #pkg install -y ... - - # NB: Intentionally build without -Dllvm. The freebsd image size is already - # large enough to make VM startup slow, and even without llvm freebsd - # already takes longer than other platforms except for windows. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - # test runningcheck, freebsd chosen because it's currently fast enough - test_running_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --quiet --suite setup - export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH" - mkdir -p build/testrun - build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust - echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf - build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - EOF - - on_failure: - # if the server continues running, it often causes cirrus-ci to fail - # during upload, as it doesn't expect artifacts to change size - stop_running_script: | - su postgres <<-EOF - set -e - build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true - EOF - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh freebsd /tmp/cores - - -task: - depends_on: SanityCheck - - env: - # Below are experimentally derived to be a decent choice. - CPUS: 4 - BUILD_JOBS: 8 - TEST_JOBS: 8 - - # Default working directory is /tmp, but its total size (1.2 GB) is not - # enough, so different working and cache directory are set. - CIRRUS_WORKING_DIR: /home/postgres/postgres - CCACHE_DIR: /home/postgres/cache - - PATH: /usr/sbin:$PATH - CORE_DUMP_DIR: /var/crash - - matrix: - - name: NetBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_NETBSD - only_if: $CI_NETBSD_ENABLED - env: - OS_NAME: netbsd - IMAGE_FAMILY: pg-ci-netbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/pkg/lib/pkgconfig' - # initdb fails with: 'invalid locale settings' error on NetBSD. - # Force 'LANG' and 'LC_*' variables to be 'C'. - # See https://postgr.es/m/2490325.1734471752%40sss.pgh.pa.us - LANG: "C" - LC_ALL: "C" - # -Duuid is not set for the NetBSD, see the comment below, above - # configure_script, for more information. - MESON_FEATURES: >- - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Dpam=enabled - - setup_additional_packages_script: | - #pkgin -y install ... - <<: *netbsd_task_template - - - name: OpenBSD - Meson - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star - trigger_type: $CI_TRIGGER_TYPE_OPENBSD - only_if: $CI_OPENBSD_ENABLED - env: - OS_NAME: openbsd - IMAGE_FAMILY: pg-ci-openbsd-postgres - PKGCONFIG_PATH: '/usr/lib/pkgconfig:/usr/local/lib/pkgconfig' - CORE_DUMP_EXECUTABLE_DIR: $CIRRUS_WORKING_DIR/build/tmp_install/usr/local/pgsql/bin - - MESON_FEATURES: >- - -Dbsd_auth=enabled - -Dlibcurl=enabled - -Dtcl_version=tcl86 - -Duuid=e2fs - - setup_additional_packages_script: | - #pkg_add -I ... - # Always core dump to ${CORE_DUMP_DIR} - set_core_dump_script: sysctl -w kern.nosuidcoredump=2 - <<: *openbsd_task_template - - sysinfo_script: | - locale - id - uname -a - ulimit -a -H && ulimit -a -S - env - - ccache_cache: - folder: $CCACHE_DIR - setup_ram_disk_script: src/tools/ci/gcp_ram_disk.sh - create_user_script: | - useradd postgres - chown -R postgres:users /home/postgres - mkdir -p ${CCACHE_DIR} - chown -R postgres:users ${CCACHE_DIR} - setup_core_files_script: | - mkdir -p ${CORE_DUMP_DIR} - chmod -R 770 ${CORE_DUMP_DIR} - chown -R postgres:users ${CORE_DUMP_DIR} - - # -Duuid=bsd is not set since 'bsd' uuid option - # is not working on NetBSD & OpenBSD. See - # https://www.postgresql.org/message-id/[email protected] - # And other uuid options are not available on NetBSD. - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debugoptimized \ - --pkg-config-path ${PKGCONFIG_PATH} \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - EOF - - build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET}' - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - cores_script: | - # Although we try to configure the OS to core dump inside - # ${CORE_DUMP_DIR}, they may not obey this. So, move core files to the - # ${CORE_DUMP_DIR} directory. - find build/ -type f -name '*.core' -exec mv '{}' ${CORE_DUMP_DIR} \; - src/tools/ci/cores_backtrace.sh ${OS_NAME} ${CORE_DUMP_DIR} ${CORE_DUMP_EXECUTABLE_DIR} - - -# configure feature flags, shared between the task running the linux tests and -# the CompilerWarnings task -LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >- - --with-gssapi - --with-icu - --with-ldap - --with-libcurl - --with-libxml - --with-libxslt - --with-llvm - --with-lz4 - --with-pam - --with-perl - --with-python - --with-selinux - --with-ssl=openssl - --with-systemd - --with-tcl --with-tclconfig=/usr/lib/tcl8.6/ - --with-uuid=ossp - --with-zstd - - -# Check SPECIAL in the matrix: below -task: - env: - CPUS: 4 - BUILD_JOBS: 4 - TEST_JOBS: 8 # experimentally derived to be a decent choice - IMAGE_FAMILY: pg-ci-trixie - - CCACHE_DIR: /tmp/ccache_dir - DEBUGINFOD_URLS: "https://debuginfod.debian.net"; - - # Enable a reasonable set of sanitizers. Use the linux task for that, as - # it's one of the fastest tasks (without sanitizers). Also several of the - # sanitizers work best on linux. - # - # The overhead of alignment sanitizer is low, undefined behaviour has - # moderate overhead. Test alignment sanitizer in the meson task, as it - # does both 32 and 64 bit builds and is thus more likely to expose - # alignment bugs. - # - # Address sanitizer in contrast is somewhat expensive. Enable it in the - # autoconf task, as the meson task tests both 32 and 64bit. - # - # disable_coredump=0, abort_on_error=1: for useful backtraces in case of crashes - # print_stacktraces=1,verbosity=2, duh - # detect_leaks=0: too many uninteresting leak errors in short-lived binaries - UBSAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:verbosity=2 - ASAN_OPTIONS: print_stacktrace=1:disable_coredump=0:abort_on_error=1:detect_leaks=0 - - # SANITIZER_FLAGS is set in the tasks below - CFLAGS: -Og -ggdb -fno-sanitize-recover=all $SANITIZER_FLAGS - CXXFLAGS: $CFLAGS - LDFLAGS: $SANITIZER_FLAGS - CC: ccache gcc - CXX: ccache g++ - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - LINUX_MESON_FEATURES: >- - -Duuid=e2fs - - <<: *linux_task_template - - depends_on: SanityCheck - only_if: $CI_LINUX_ENABLED - - ccache_cache: - folder: ${CCACHE_DIR} - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - export - create_user_script: | - useradd -m postgres - chown -R postgres:postgres . - mkdir -p ${CCACHE_DIR} - chown -R postgres:postgres ${CCACHE_DIR} - echo '* - memlock 134217728' > /etc/security/limits.d/postgres.conf - su postgres -c "ulimit -l -H && ulimit -l -S" - setup_core_files_script: | - mkdir -m 770 /tmp/cores - chown root:postgres /tmp/cores - sysctl kernel.core_pattern='/tmp/cores/%e-%s-%p.core' - - setup_hosts_file_script: | - cat >> /etc/hosts <<-EOF - 127.0.0.1 pg-loadbalancetest - 127.0.0.2 pg-loadbalancetest - 127.0.0.3 pg-loadbalancetest - EOF - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - matrix: - # SPECIAL: - # - Uses address sanitizer, sanitizer failures are typically printed in - # the server log - # - Configures postgres with a small segment size - - name: Linux - Debian Trixie - Autoconf - - env: - SANITIZER_FLAGS: -fsanitize=address - PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range - - # Normally, the "relation segment" code basically has no coverage in our - # tests, because we (quite reasonably) don't generate tables large - # enough in tests. We've had plenty bugs that we didn't notice due the - # code not being exercised much. Thus specify a very small segment size - # here. Use a non-power-of-two segment size, given we currently allow - # that. - configure_script: | - su postgres <<-EOF - set -e - ./configure \ - --enable-cassert --enable-injection-points --enable-debug \ - --enable-tap-tests --enable-nls \ - --with-segsize-blocks=6 \ - --with-libnuma \ - --with-liburing \ - \ - ${LINUX_CONFIGURE_FEATURES} \ - \ - CLANG="ccache clang" - EOF - build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin" - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited # default is 0 - make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_ac - - # SPECIAL: - # - Uses undefined behaviour and alignment sanitizers, sanitizer failures - # are typically printed in the server log - # - Test both 64bit and 32 bit builds - # - uses io_method=io_uring - # - Uses meson feature autodetection - - name: Linux - Debian Trixie - Meson - - env: - CCACHE_MAXSIZE: "400M" # tests two different builds - SANITIZER_FLAGS: -fsanitize=alignment,undefined - PG_TEST_INITDB_EXTRA_OPTS: >- - -c io_method=io_uring - - configure_script: | - su postgres <<-EOF - set -e - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - ${LINUX_MESON_FEATURES} -Dllvm=enabled \ - build - EOF - - # Also build & test in a 32bit build - it's gotten rare to test that - # locally. - configure_32_script: | - su postgres <<-EOF - set -e - export CC='ccache gcc -m32' - export CXX='ccache g++ -m32' - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \ - -DPERL=perl5.40-i386-linux-gnu \ - ${LINUX_MESON_FEATURES} -Dlibnuma=disabled \ - build-32 - EOF - - build_script: | - su postgres <<-EOF - set -e - ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - build_32_script: | - su postgres <<-EOF - set -e - ninja -C build-32 -j${BUILD_JOBS} ${MBUILD_TARGET} - ninja -C build -t missingdeps - EOF - - upload_caches: ccache - - test_world_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - EOF - # so that we don't upload 64bit logs if 32bit fails - rm -rf build/ - - # There's currently no coverage of icu with LANG=C in the buildfarm. We - # can easily provide some here by running one of the sets of tests that - # way. Newer versions of python insist on changing the LC_CTYPE away - # from C, prevent that with PYTHONCOERCECLOCALE. - test_world_32_script: | - su postgres <<-EOF - set -e - ulimit -c unlimited - PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS} - EOF - - on_failure: - <<: *on_failure_meson - - on_failure: - cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores - - -# NB: macOS is by far the most expensive OS to run CI for, therefore no -# expensive additional checks should be added. -# -# SPECIAL: -# - Enables --clone for pg_upgrade and pg_combinebackup -task: - name: macOS - Sequoia - Meson - - env: - CPUS: 4 # always get that much for cirrusci macOS instances - BUILD_JOBS: $CPUS - # Test performance regresses noticeably when using all cores. 8 seems to - # work OK. See - # https://postgr.es/m/20220927040208.l3shfcidovpzqxfh%40awork3.anarazel.de - TEST_JOBS: 8 - IMAGE: ghcr.io/cirruslabs/macos-runner:sequoia - - CIRRUS_WORKING_DIR: ${HOME}/pgsql/ - CCACHE_DIR: ${HOME}/ccache - MACPORTS_CACHE: ${HOME}/macports-cache - - MESON_FEATURES: >- - -Dbonjour=enabled - -Ddtrace=enabled - -Dgssapi=enabled - -Dlibcurl=enabled - -Dnls=enabled - -Duuid=e2fs - - MACOS_PACKAGE_LIST: >- - ccache - icu - kerberos5 - lz4 - meson - openldap - openssl - p5.34-io-tty - p5.34-ipc-run - python312 - tcl - zstd - - CC: ccache cc - CXX: ccache c++ - CFLAGS: -Og -ggdb - CXXFLAGS: -Og -ggdb - - PG_TEST_PG_UPGRADE_MODE: --clone - PG_TEST_PG_COMBINEBACKUP_MODE: --clone - - <<: *macos_task_template - - depends_on: SanityCheck - only_if: $CI_MACOS_ENABLED - - sysinfo_script: | - id - uname -a - ulimit -a -H && ulimit -a -S - export - - setup_core_files_script: - - mkdir ${HOME}/cores - - sudo sysctl kern.corefile="${HOME}/cores/core.%P" - - # Use macports, even though homebrew is installed. The installation - # of the additional packages we need would take quite a while with - # homebrew, even if we cache the downloads. We can't cache all of - # homebrew, because it's already large. So we use macports. To cache - # the installation we create a .dmg file that we mount if it already - # exists. - # XXX: The reason for the direct p5.34* references is that we'd need - # the large macport tree around to figure out that p5-io-tty is - # actually p5.34-io-tty. Using the unversioned name works, but - # updates macports every time. - macports_cache: - folder: ${MACPORTS_CACHE} - fingerprint_script: | - # Reinstall packages if the OS major version, the list of the packages - # to install or the MacPorts install script changes. - sw_vers -productVersion | sed 's/\..*//' - echo $MACOS_PACKAGE_LIST - md5 src/tools/ci/ci_macports_packages.sh - reupload_on_changes: true - setup_additional_packages_script: | - sh src/tools/ci/ci_macports_packages.sh $MACOS_PACKAGE_LIST - # system python doesn't provide headers - sudo /opt/local/bin/port select python3 python312 - # Make macports install visible for subsequent steps - echo PATH=/opt/local/sbin/:/opt/local/bin/:$PATH >> $CIRRUS_ENV - upload_caches: macports - - ccache_cache: - folder: $CCACHE_DIR - configure_script: | - export PKG_CONFIG_PATH="/opt/local/lib/pkgconfig/" - meson setup \ - ${MESON_COMMON_PG_CONFIG_ARGS} \ - --buildtype=debug \ - -Dextra_include_dirs=/opt/local/include \ - -Dextra_lib_dirs=/opt/local/lib \ - ${MESON_COMMON_FEATURES} ${MESON_FEATURES} \ - build - - build_script: ninja -C build -j${BUILD_JOBS} ${MBUILD_TARGET} - upload_caches: ccache - - test_world_script: | - ulimit -c unlimited # default is 0 - ulimit -n 1024 # default is 256, pretty low - meson test $MTEST_ARGS --num-processes ${TEST_JOBS} - - on_failure: - <<: *on_failure_meson - cores_script: src/tools/ci/cores_backtrace.sh macos "${HOME}/cores" - - -WINDOWS_ENVIRONMENT_BASE: &WINDOWS_ENVIRONMENT_BASE - env: - # Half the allowed per-user CPU cores - CPUS: 4 - - # The default cirrus working dir is in a directory msbuild complains about - CIRRUS_WORKING_DIR: "c:/cirrus" - # git's tar doesn't deal with drive letters, see - # https://postgr.es/m/b6782dc3-a7b0-ed56-175f-f8f54cb08d67%40dunslane.net - TAR: "c:/windows/system32/tar.exe" - # Avoids port conflicts between concurrent tap test runs - PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: "c:/cirrus/" - DISK_SIZE: 50 - IMAGE_FAMILY: pg-ci-windows-ci - - sysinfo_script: | - chcp - systeminfo - powershell -Command get-psdrive -psprovider filesystem - set - - -task: - name: Windows - Server 2022, VS 2019 - Meson & ninja - << : *WINDOWS_ENVIRONMENT_BASE - - env: - TEST_JOBS: 8 # wild guess, data based value welcome - - # Cirrus defaults to SetErrorMode(SEM_NOGPFAULTERRORBOX | ...). That - # prevents crash reporting from working unless binaries do SetErrorMode() - # themselves. Furthermore, it appears that either python or, more likely, - # the C runtime has a bug where SEM_NOGPFAULTERRORBOX can very - # occasionally *trigger* a crash on process exit - which is hard to debug, - # given that it explicitly prevents crash dumps from working... - # 0x8001 is SEM_FAILCRITICALERRORS | SEM_NOOPENFILEERRORBOX - CIRRUS_WINDOWS_ERROR_MODE: 0x8001 - - MESON_FEATURES: - -Dcpp_args=/std:c++20 - -Dauto_features=disabled - -Dldap=enabled - -Dssl=openssl - -Dtap_tests=enabled - -Dplperl=enabled - -Dplpython=enabled - - <<: *windows_task_template - - depends_on: SanityCheck - only_if: $CI_WINDOWS_ENABLED - - setup_additional_packages_script: | - REM choco install -y --no-progress ... - - setup_hosts_file_script: | - echo 127.0.0.1 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts - type c:\Windows\System32\Drivers\etc\hosts - - configure_script: | - vcvarsall x64 - meson setup --backend ninja %MESON_COMMON_PG_CONFIG_ARGS% --buildtype debug -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% %MESON_FEATURES% build - - build_script: | - vcvarsall x64 - ninja -C build %MBUILD_TARGET% - ninja -C build -t missingdeps - - check_world_script: | - vcvarsall x64 - meson test %MTEST_ARGS% --num-processes %TEST_JOBS% - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - << : *WINDOWS_ENVIRONMENT_BASE - name: Windows - Server 2022, MinGW64 - Meson - - # See REPO_CI_AUTOMATIC_TRIGGER_TASKS in .cirrus.star. - trigger_type: $CI_TRIGGER_TYPE_MINGW - - depends_on: SanityCheck - only_if: $CI_MINGW_ENABLED - - env: - TEST_JOBS: 4 # higher concurrency causes occasional failures - CCACHE_DIR: C:/msys64/ccache - CCACHE_MAXSIZE: "500M" - CCACHE_SLOPPINESS: pch_defines,time_macros - CCACHE_DEPEND: 1 - # for some reason mingw plpython cannot find its installation without this - PYTHONHOME: C:/msys64/ucrt64 - # prevents MSYS bash from resetting error mode - MSYS: winjitdebug - # Start bash in current working directory - CHERE_INVOKING: 1 - BASH: C:\msys64\usr\bin\bash.exe -l - - # Keep -Dnls explicitly disabled, as the number of files it creates causes a - # noticeable slowdown. - MESON_FEATURES: >- - -Dnls=disabled - - <<: *windows_task_template - - ccache_cache: - folder: ${CCACHE_DIR} - - setup_additional_packages_script: | - REM C:\msys64\usr\bin\pacman.exe -S --noconfirm ... - - mingw_info_script: | - %BASH% -c "where gcc" - %BASH% -c "gcc --version" - %BASH% -c "where perl" - %BASH% -c "perl --version" - - configure_script: | - %BASH% -c "meson setup %MESON_COMMON_PG_CONFIG_ARGS% -Ddebug=true -Doptimization=g -Db_pch=true %MESON_COMMON_FEATURES% %MESON_FEATURES% -DTAR=%TAR% build" - - build_script: | - %BASH% -c "ninja -C build ${MBUILD_TARGET}" - - upload_caches: ccache - - test_world_script: | - %BASH% -c "meson test %MTEST_ARGS% --num-processes %TEST_JOBS%" - - on_failure: - <<: *on_failure_meson - crashlog_artifacts: - path: "crashlog-*.txt" - type: text/plain - - -task: - name: CompilerWarnings - - # To limit unnecessary work only run this once the SanityCheck - # succeeds. This is particularly important for this task as we intentionally - # use always: to continue after failures. - depends_on: SanityCheck - only_if: $CI_COMPILERWARNINGS_ENABLED - - env: - CPUS: 4 - BUILD_JOBS: 4 - IMAGE_FAMILY: pg-ci-trixie - - # Use larger ccache cache, as this task compiles with multiple compilers / - # flag combinations - CCACHE_MAXSIZE: "1G" - CCACHE_DIR: "/tmp/ccache_dir" - - LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES - - <<: *linux_task_template - - sysinfo_script: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - export - - ccache_cache: - folder: $CCACHE_DIR - - setup_additional_packages_script: | - #apt-get update - #DEBIAN_FRONTEND=noninteractive apt-get -y install ... - - ### - # Test that code can be built with gcc/clang without warnings - ### - - setup_script: echo "COPT=-Werror" > src/Makefile.custom - - # Trace probes have a history of getting accidentally broken. Use the - # different compilers to build with different combinations of dtrace on/off - # and cassert on/off. - - # gcc, cassert off, dtrace on - always: - gcc_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # gcc, cassert on, dtrace off - always: - gcc_a_warning_script: | - time ./configure \ - --cache gcc.cache \ - --enable-cassert \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert off, dtrace off - always: - clang_warning_script: | - time ./configure \ - --cache clang.cache \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # clang, cassert on, dtrace on - always: - clang_a_warning_script: | - time ./configure \ - --cache clang.cache \ - --enable-cassert \ - --enable-dtrace \ - ${LINUX_CONFIGURE_FEATURES} \ - CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - # cross-compile to windows - always: - mingw_cross_warning_script: | - time ./configure \ - --host=x86_64-w64-mingw32ucrt \ - --enable-cassert \ - --without-icu \ - CC="ccache x86_64-w64-mingw32ucrt-gcc" \ - CXX="ccache x86_64-w64-mingw32ucrt-g++" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} world-bin - - ### - # Verify docs can be built - ### - # XXX: Only do this if there have been changes in doc/ since last build - always: - docs_build_script: | - time ./configure \ - --cache gcc.cache \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -C doc - - ### - # Verify headerscheck / cpluspluscheck succeed - # - # - Run both in same script to increase parallelism, use -k to get result of both - # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose - ### - always: - headers_headerscheck_script: | - time ./configure \ - ${LINUX_CONFIGURE_FEATURES} \ - --cache gcc.cache \ - --quiet \ - CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang" - make -s -j${BUILD_JOBS} clean - time make -s -j${BUILD_JOBS} -k ${CHECKFLAGS} headerscheck cpluspluscheck EXTRAFLAGS='-fmax-errors=10' - - always: - upload_caches: ccache diff --git a/src/tools/ci/ci_macports_packages.sh b/src/tools/ci/ci_macports_packages.sh index c7c4a1c0c60..304b9b43fd4 100755 --- a/src/tools/ci/ci_macports_packages.sh +++ b/src/tools/ci/ci_macports_packages.sh @@ -6,7 +6,7 @@ # when packages are installed or removed. Any package this script is # not instructed to install, will be removed again. # -# This currently expects to be run in a macos cirrus-ci environment. +# This currently expects to be run in a macos github actions environment. set -e # set -x @@ -38,8 +38,8 @@ fi cache_dmg="macports.hfs.dmg" -if [ "$CIRRUS_CI" != "true" ] && [ "$GITHUB_ACTIONS" != "true" ]; then - echo "expect to be called within cirrus-ci or github actions" 1>&2 +if [ "$GITHUB_ACTIONS" != "true" ]; then + echo "expect to be called within github actions" 1>&2 exit 1 fi diff --git a/src/tools/ci/gcp_ram_disk.sh b/src/tools/ci/gcp_ram_disk.sh deleted file mode 100755 index 18dbb2037f5..00000000000 --- a/src/tools/ci/gcp_ram_disk.sh +++ /dev/null @@ -1,27 +0,0 @@ -#!/bin/sh -# Move working directory into a RAM disk for better performance. - -set -e -set -x - -mv $CIRRUS_WORKING_DIR $CIRRUS_WORKING_DIR.orig -mkdir $CIRRUS_WORKING_DIR - -case "`uname`" in - FreeBSD|NetBSD) - mount -t tmpfs tmpfs $CIRRUS_WORKING_DIR - ;; - OpenBSD) - umount /dev/sd0j # unused /usr/obj partition - printf "m j\n\n\nswap\nw\nq\n" | disklabel -E sd0 - swapon /dev/sd0j - # Remove the per-process data segment limit so that mount_mfs can allocate - # large memory filesystems. Without this, mount_mfs mmap() may fail with - # "Cannot allocate memory" if the requested size exceeds the current - # datasize limit. - ulimit -d unlimited - mount -t mfs -o rw,noatime,nodev,-s=10000000 swap $CIRRUS_WORKING_DIR - ;; -esac - -cp -a $CIRRUS_WORKING_DIR.orig/. $CIRRUS_WORKING_DIR/ -- 2.54.0.380.gc69baaf57b
>From 445f03bfa26fd7cb98ade4c8f8bd41b2faf22348 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 01:31:36 -0400 Subject: [PATCH v9a 04/22] ci: Store build/meson-logs and name "id: os" task --- .github/workflows/pg-ci.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 8594d540f6a..a4fd4f42a50 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -157,7 +157,8 @@ jobs: ulimit -a -H && ulimit -a -S env - - id: os + - name: Parse ci-os-only + id: os env: MSG: ${{ github.event.head_commit.message }} shell: bash @@ -328,6 +329,7 @@ jobs: **/*.diffs **/regress_log_* **/crashlog-*.txt + build/meson-logs/** if-no-files-found: ignore -- 2.54.0.380.gc69baaf57b
>From 0c5c07bb37f0159aa80fc7c5de2ff89335f207ef Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 01:40:22 -0400 Subject: [PATCH v9a 05/22] ci: Use mingw -pkgconf instead of -pkg-config package Before there's this complaint: warning: removing 'mingw-w64-ucrt-x86_64-pkg-config-0.29.2-6' from target list because it conflicts with 'mingw-w64-ucrt-x86_64-pkgconf-1~2.5.1-1' --- .github/workflows/pg-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index a4fd4f42a50..5ae8162d635 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -901,7 +901,7 @@ jobs: ${MINGW_PACKAGE_PREFIX}-make \ ${MINGW_PACKAGE_PREFIX}-meson \ ${MINGW_PACKAGE_PREFIX}-perl \ - ${MINGW_PACKAGE_PREFIX}-pkg-config \ + ${MINGW_PACKAGE_PREFIX}-pkgconf \ ${MINGW_PACKAGE_PREFIX}-readline \ ${MINGW_PACKAGE_PREFIX}-zlib -- 2.54.0.380.gc69baaf57b
>From 72edc7be510d87436aca0ba44acccfb16c06eea9 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 02:19:42 -0400 Subject: [PATCH v9a 06/22] ci: windows-2022 already openssl installed The other installer is outdated and takes a while. --- .github/workflows/pg-ci.yml | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 5ae8162d635..b05c3539ab6 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -767,17 +767,6 @@ jobs: # meson + ninja aren't preinstalled on windows-2022. Install via pip python -m pip install --upgrade meson ninja - # OpenSSL 1.1 via the slproweb installer (pinned to match the - # version used elsewhere in postgres CI). - curl.exe -fsSL -o openssl-setup.exe https://slproweb.com/download/Win64OpenSSL-1_1_1w.exe - Start-Process -Wait -FilePath ./openssl-setup.exe ` - -ArgumentList '/DIR=d:\openssl\1.1\ /VERYSILENT /SP- /SUPPRESSMSGBOXES' - # The slproweb installer puts libcrypto-1_1-x64.dll / libssl-1_1-x64.dll - # in d:\openssl\1.1\bin\ and updates the system PATH. GH Actions - # snapshots PATH at job start though, so the running job won't - # see those DLLs and initdb.exe would crash silently at runtime. - # Push the bin dir onto GITHUB_PATH so it persists for later steps. - Add-Content $env:GITHUB_PATH "d:\openssl\1.1\bin" # Install IPC::Run. # - recommends_policy=0 keeps cpan from pulling in IO::Tty / IO::Pty, @@ -811,7 +800,6 @@ jobs: ${{env.MESON_FEATURES}} ^ --buildtype debug ^ -Db_pch=true ^ - -Dextra_lib_dirs=d:\openssl\1.1\lib -Dextra_include_dirs=d:\openssl\1.1\include ^ -DTAR=${{env.TAR}} ^ build -- 2.54.0.380.gc69baaf57b
>From 402eadee220098158efd11d94df824ffdd551a44 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 02:21:00 -0400 Subject: [PATCH v9a 07/22] ci: windows: Install bison flex via msys That's a fair bit faster and fails less often. --- .github/workflows/pg-ci.yml | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index b05c3539ab6..e35170956e8 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -760,10 +760,25 @@ jobs: Add-Content $env:GITHUB_ENV "PATH=$filtered" Write-Host "Removed Mercurial entries from PATH" + # Install some dependencies via msys64, that seems to be the fastest and + # most reliable + - name: Install dependencies, Mingw + shell: 'C:\msys64\usr\bin\bash.exe --login -eo pipefail "{0}"' + run: | + # Install some dependencies via msys64, that seems to be the fastest + # and most reliable + pacman -S --noconfirm --needed --asdeps \ + bison flex + + # Make bison and flex visible + echo C:/msys64/usr/bin >> "$GITHUB_PATH" + + # Don't prefer mingw's perl + echo C:/Strawberry/perl/bin >> "$GITHUB_PATH" + - name: Install dependencies shell: pwsh run: | - choco install -y --no-progress --limitoutput diffutils winflexbison3 # meson + ninja aren't preinstalled on windows-2022. Install via pip python -m pip install --upgrade meson ninja -- 2.54.0.380.gc69baaf57b
>From fe084a4de90e98480629957ebd2dd09133bbe55b Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 02:21:51 -0400 Subject: [PATCH v9a 08/22] ci: mingw: Don't install make, use --asdeps --asdeps can sometimes lead to fewer packages being installed. --- .github/workflows/pg-ci.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index e35170956e8..9611686af90 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -892,8 +892,8 @@ jobs: # ${MINGW_PACKAGE_PREFIX} is an environment variable used in the # MSYS2. It dynamically expands to the correct prefix for the active # shell environment. - pacman -S --noconfirm --needed \ - git bison flex make diffutils \ + pacman -S --noconfirm --needed --asdeps \ + git bison flex diffutils \ ${MINGW_PACKAGE_PREFIX}-ccache \ ${MINGW_PACKAGE_PREFIX}-gcc \ ${MINGW_PACKAGE_PREFIX}-icu \ -- 2.54.0.380.gc69baaf57b
>From 79a69eb410d99636e5c6bee338b75f4b7da2a5ce Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 02:22:37 -0400 Subject: [PATCH v9a 09/22] ci: mingw: Don't rely on zstd implicitly being installed --- .github/workflows/pg-ci.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9611686af90..f2543689fe5 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -906,6 +906,7 @@ jobs: ${MINGW_PACKAGE_PREFIX}-perl \ ${MINGW_PACKAGE_PREFIX}-pkgconf \ ${MINGW_PACKAGE_PREFIX}-readline \ + ${MINGW_PACKAGE_PREFIX}-zstd \ ${MINGW_PACKAGE_PREFIX}-zlib - *nix_sysinfo_step -- 2.54.0.380.gc69baaf57b
>From c308e3050ced7f3a239bfc1d5020b95b82a7ae05 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 02:24:28 -0400 Subject: [PATCH v9a 10/22] ci: windows: Check for errors, cmd/powershell don't have set -e behavior --- .github/workflows/pg-ci.yml | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index f2543689fe5..ad9bbf745fb 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -307,7 +307,7 @@ jobs: ${{env.ADDITIONAL_SETUP}} echo ::group::test_setup - meson test ${{env.MTEST_ARGS}} --suite setup --logbase setup + meson test ${{env.MTEST_ARGS}} --suite setup --logbase setup || exit 1 echo ::endgroup:: meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} ${{env.MTEST_TARGET}} @@ -781,6 +781,7 @@ jobs: run: | # meson + ninja aren't preinstalled on windows-2022. Install via pip python -m pip install --upgrade meson ninja + if (!$?) { throw 'cmdfail' } # Install IPC::Run. @@ -793,7 +794,9 @@ jobs: # the thread at # https://postgr.es/m/CAN55FZ06xanSbJdHe-CurjX_qNuBWZDEvS1kAk36L38YCtZXnw%40mail.gmail.com "o conf recommends_policy 0`no conf commit`nnotest install NJM/IPC-Run-20250809.0.tar.gz" | cpan + if (!$?) { throw 'cmdfail' } perl -mIPC::Run -e 1 + if (!$?) { throw 'cmdfail' } - name: Setup hosts file shell: pwsh @@ -821,7 +824,7 @@ jobs: - name: Build run: | call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 - ninja -C build ${{env.MBUILD_TARGET}} + ninja -C build ${{env.MBUILD_TARGET}} || exit 1 ninja -C build -t missingdeps - name: Test world -- 2.54.0.380.gc69baaf57b
>From 7cfd86e15b2455e118f90012df03ef99dffa2799 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 02:25:41 -0400 Subject: [PATCH v9a 11/22] ci: windows: ninja is already installed --- .github/workflows/pg-ci.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index ad9bbf745fb..1600a5a8095 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -779,8 +779,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson + ninja aren't preinstalled on windows-2022. Install via pip - python -m pip install --upgrade meson ninja + # meson is not preinstalled on windows-2022. Install via pip + python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } -- 2.54.0.380.gc69baaf57b
>From 011a8c193bdca5491d22ae04c24fc26bf19c6fd1 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 02:42:33 -0400 Subject: [PATCH v9a 12/22] ci: windows: make argument order alphabetical --- .github/workflows/pg-ci.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 1600a5a8095..3bb559ab9f4 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -688,13 +688,13 @@ jobs: TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- - -Dcpp_args=/std:c++20 -Dauto_features=disabled - -Dtap_tests=enabled + -Dcpp_args=/std:c++20 -Dldap=enabled - -Dssl=openssl -Dplperl=enabled -Dplpython=enabled + -Dssl=openssl + -Dtap_tests=enabled defaults: run: -- 2.54.0.380.gc69baaf57b
>From 4c4316c020449e7852dba901a07dcf114a4779b2 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Tue, 2 Jun 2026 20:59:53 -0400 Subject: [PATCH v9a 13/22] ci: Add groups to windows install, it's pretty hard to read as-is --- .github/workflows/pg-ci.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 3bb559ab9f4..af986b351bf 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -780,8 +780,10 @@ jobs: shell: pwsh run: | # meson is not preinstalled on windows-2022. Install via pip + echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } + echo ::endgroup:: # Install IPC::Run. @@ -793,10 +795,12 @@ jobs: # handling). See upstream pg-vm-images commit ff5238afa3 and # the thread at # https://postgr.es/m/CAN55FZ06xanSbJdHe-CurjX_qNuBWZDEvS1kAk36L38YCtZXnw%40mail.gmail.com + echo ::group::cpan_ipc_run "o conf recommends_policy 0`no conf commit`nnotest install NJM/IPC-Run-20250809.0.tar.gz" | cpan if (!$?) { throw 'cmdfail' } perl -mIPC::Run -e 1 if (!$?) { throw 'cmdfail' } + echo ::endgroup:: - name: Setup hosts file shell: pwsh @@ -919,8 +923,10 @@ jobs: # Pin IPC::Run to NJM/IPC-Run-20250809.0; TODDR/IPC-Run-20260322.0 # broke postgres tap tests on Windows (pipe stdio handling). # See pg-vm-images commit ff5238afa3. + echo ::group::cpan_ipc_run (echo; echo o conf recommends_policy 0; echo notest install NJM/IPC-Run-20250809.0.tar.gz) | cpan perl -mIPC::Run -e 1 + echo ::endgroup:: - name: Setup socket directory shell: cmd -- 2.54.0.380.gc69baaf57b
>From 05d0c033d764818f3d3540478850638eacc52712 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Tue, 2 Jun 2026 20:30:06 -0400 Subject: [PATCH v9a 14/22] ci: Add back running check support --- .github/workflows/pg-ci.yml | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index af986b351bf..1be402b44ce 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -479,6 +479,36 @@ jobs: LANG: C run: *meson_test_world_cmd + # Test running against existing PG instance. + # + # linux-meson-32 chosen because it's currently comparatively fast + - name: Test running + shell: *su_postgres_shell + run: | + ulimit -c unlimited + + # Ensure install exists, in case somebody is debugging a failing + # test within this an reorders this before "Test world" + echo ::group::test_setup + meson test ${{env.MTEST_ARGS}} --suite setup --logbase setup + echo ::endgroup:: + + # Make libraries discoverable (the x86_64 reference is a meson + # oddity) + export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/x86_64-linux-gnu/:$LD_LIBRARY_PATH" + + build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust + echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf + + # Log into a place that will be archived in case of failure + mkdir -p build/testrun + build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start + + # Run the tests supporting running against an already running + meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} --setup running + + build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop + - *linux_collect_cores - *upload_logs_step -- 2.54.0.380.gc69baaf57b
>From b97c3a6ac0465ea7c4e025ef72a214822a32edb7 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 11:37:29 -0400 Subject: [PATCH v9a 15/22] ci: Move more coverage from previously FreeBSD elsewhere Discussion: https://postgr.es/m/[email protected] --- .github/workflows/pg-ci.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 1be402b44ce..72d306feb14 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -340,6 +340,8 @@ jobs: # printed in the server log) # - Configures postgres with a small segment size # - Uses PG_TEST_PG_COMBINEBACKUP_MODE=--copy-file-range + # - Uses postgres specific CPPFLAGS that increase test coverage + # - Enables --link for pg_upgrade linux-autoconf: name: Linux - Autoconf needs: [setup, sanity-check] @@ -378,7 +380,10 @@ jobs: env: SANITIZER_FLAGS: -fsanitize=alignment,undefined PG_TEST_PG_COMBINEBACKUP_MODE: --copy-file-range + CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS + PG_TEST_PG_UPGRADE_MODE: --link run: &linux_update_config_cmd | + echo "CPPFLAGS=$CPPFLAGS" >> "$GITHUB_ENV" echo "CFLAGS=$CFLAGS ${SANITIZER_FLAGS}" >> "$GITHUB_ENV" echo "CXXFLAGS=$CXXFLAGS ${SANITIZER_FLAGS}" >> "$GITHUB_ENV" echo "LDFLAGS=$LDFLAGS ${SANITIZER_FLAGS}" >> "$GITHUB_ENV" @@ -386,6 +391,7 @@ jobs: echo "CC=${CC}" >> "$GITHUB_ENV" echo "CXX=${CXX}" >> "$GITHUB_ENV" + echo "PG_TEST_PG_UPGRADE_MODE=${PG_TEST_PG_UPGRADE_MODE}" >> "$GITHUB_ENV" echo "PG_TEST_INITDB_EXTRA_OPTS=${PG_TEST_INITDB_EXTRA_OPTS}" >> "$GITHUB_ENV" echo "PG_TEST_PG_COMBINEBACKUP_MODE=${PG_TEST_PG_COMBINEBACKUP_MODE}" >> "$GITHUB_ENV" -- 2.54.0.380.gc69baaf57b
>From 5f291ee3e48c273339455a3563ad896ecb186899 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 13:15:38 -0400 Subject: [PATCH v9a 16/22] ci: Formatting, Naming and consistency improvements --- .github/workflows/pg-ci.yml | 76 ++++++++++++++++++------------------- 1 file changed, 36 insertions(+), 40 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 72d306feb14..a570b9e2c8d 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -126,7 +126,10 @@ env: jobs: - # Parse "ci-os-only: ..." from the commit message and expose flags + + # Job: Determine enabled jobs + # + # Parses "ci-os-only: ..." from the commit message and exposes flags # consumed by the jobs' `if:` conditions. setup: name: Determine enabled jobs @@ -144,6 +147,7 @@ jobs: # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -180,6 +184,8 @@ jobs: cat "$GITHUB_OUTPUT" + # Job: SanityCheck + # # To avoid unnecessarily spinning up a lot of VMs / containers for entirely # broken commits, have a minimal task that all others depend on. # @@ -219,6 +225,7 @@ jobs: env: # no options enabled, should be small CCACHE_MAXSIZE: "150M" + steps: - *nix_sysinfo_step @@ -238,7 +245,7 @@ jobs: ccache-${{ github.job }}-${{ github.ref_name }}- ccache-${{ github.job }}- - - &linux_prepare_workspace + - &linux_prepare_workspace_step name: Prepare workspace run: | useradd -m postgres @@ -271,7 +278,7 @@ jobs: - name: Build shell: *su_postgres_shell - run: &ninja_build_command | + run: &ninja_build_cmd | ninja -C build -j${{env.BUILD_JOBS}} ${{env.MBUILD_TARGET}} ninja -C build -t missingdeps @@ -312,7 +319,7 @@ jobs: meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} ${{env.MTEST_TARGET}} - - &linux_collect_cores + - &linux_collect_cores_step name: Core backtraces if: failure() && !cancelled() run: src/tools/ci/cores_backtrace.sh linux /tmp/cores @@ -333,7 +340,7 @@ jobs: if-no-files-found: ignore - # Linux, Autoconf + # Job: Linux - Autoconf # # SPECIAL: # - Uses undefined & alignment sanitizers (sanitizer failures are typically @@ -398,7 +405,7 @@ jobs: - *nix_sysinfo_step - *checkout_step - *ccache_restore_step - - *linux_prepare_workspace + - *linux_prepare_workspace_step - name: Configure shell: *su_postgres_shell @@ -423,11 +430,11 @@ jobs: run: | make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS} - - *linux_collect_cores + - *linux_collect_cores_step - *upload_logs_step - # Linux Meson, 32 bit + # Job: Linux - Meson (32-bit) # # SPECIAL: # - Uses undefined behaviour and alignment sanitizers, (sanitizer failures @@ -458,7 +465,7 @@ jobs: - *nix_sysinfo_step - *checkout_step - *ccache_restore_step - - *linux_prepare_workspace + - *linux_prepare_workspace_step - name: Configure shell: *su_postgres_shell @@ -474,7 +481,7 @@ jobs: - name: Build shell: *su_postgres_shell - run: *ninja_build_command + run: *ninja_build_cmd - *ccache_save_step @@ -515,11 +522,11 @@ jobs: build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop - - *linux_collect_cores + - *linux_collect_cores_step - *upload_logs_step - # Linux Meson, 64 bit + # Linux - Meson (64-bit) # # SPECIAL: # - Uses address sanitizer, (sanitizer failures are typically printed in the @@ -546,7 +553,7 @@ jobs: - *nix_sysinfo_step - *checkout_step - *ccache_restore_step - - *linux_prepare_workspace + - *linux_prepare_workspace_step - name: Configure shell: *su_postgres_shell @@ -560,7 +567,7 @@ jobs: - name: Build shell: *su_postgres_shell - run: *ninja_build_command + run: *ninja_build_cmd - *ccache_save_step @@ -568,10 +575,12 @@ jobs: shell: *su_postgres_shell run: *meson_test_world_cmd - - *linux_collect_cores + - *linux_collect_cores_step - *upload_logs_step + # Job: macOS - Meson + # # SPECIAL: # - Enables --clone for pg_upgrade and pg_combinebackup # - Specifies configuration options that test reading/writing/copying of node trees @@ -628,8 +637,8 @@ jobs: steps: - *nix_sysinfo_step - - *checkout_step + - *ccache_restore_step - name: Setup core files run: | @@ -674,8 +683,6 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" - - *ccache_restore_step - - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ @@ -691,7 +698,7 @@ jobs: build - name: Build - run: *ninja_build_command + run: *ninja_build_cmd - *ccache_save_step @@ -708,6 +715,7 @@ jobs: - *upload_logs_step + # Job: Windows - Visual Studio windows-vs: name: Windows - Visual Studio needs: [setup, sanity-check] @@ -735,8 +743,9 @@ jobs: defaults: run: shell: cmd + steps: - - &windows_disable_defender + - &windows_disable_defender_step name: Disable Windows Defender shell: powershell run: | @@ -821,7 +830,6 @@ jobs: if (!$?) { throw 'cmdfail' } echo ::endgroup:: - # Install IPC::Run. # - recommends_policy=0 keeps cpan from pulling in IO::Tty / IO::Pty, # which don't build on Windows ("This module requires a POSIX @@ -879,6 +887,7 @@ jobs: - *upload_logs_step + # Job: Windows - MinGW - Meson windows-mingw: name: Windows - MinGW - Meson needs: [setup, sanity-check] @@ -912,7 +921,7 @@ jobs: shell: 'D:\msys64\usr\bin\bash.exe --login -eo pipefail "{0}"' steps: - - *windows_disable_defender + - *windows_disable_defender_step - *checkout_step # Relocate the preinstalled MSYS2 tree from C:\ (slow system disk) to @@ -981,7 +990,7 @@ jobs: build - name: Build - run: *ninja_build_command + run: *ninja_build_cmd - *ccache_save_step @@ -993,6 +1002,8 @@ jobs: - *upload_logs_step + # Job: CompilerWarnings + # # Test that code can be built with both gcc and clang without warnings, # with various combinations of cassert/dtrace flags. Trace probes have # a history of getting accidentally broken; the matrix is there to @@ -1017,20 +1028,10 @@ jobs: # compilers / flag combinations. CCACHE_MAXSIZE: "1G" DEFAULT_BUILD: world-bin + steps: - - - name: Sysinfo - run: | - id - uname -a - cat /proc/cmdline - ulimit -a -H && ulimit -a -S - gcc -v - clang -v - env - + - *nix_sysinfo_step - *checkout_step - - *ccache_restore_step - name: Setup workspace @@ -1055,7 +1056,6 @@ jobs: make -s -j${{env.BUILD_JOBS}} clean make -s -j${{env.BUILD_JOBS}} ${{env.DEFAULT_BUILD}} - # gcc, cassert on, dtrace off - name: gcc warnings + (cassert) if: ${{ !cancelled() }} @@ -1065,7 +1065,6 @@ jobs: CXX: ccache g++ run: *compiler_warnings_cmd - # clang, cassert off, dtrace off - name: clang warnings if: ${{ !cancelled() }} @@ -1075,7 +1074,6 @@ jobs: CXX: ccache clang++ run: *compiler_warnings_cmd - # clang, cassert on, dtrace on - name: clang warnings + (cassert + dtrace) if: ${{ !cancelled() }} @@ -1085,7 +1083,6 @@ jobs: CXX: ccache clang++ run: *compiler_warnings_cmd - - name: mingw warnings (cross compilation) if: ${{ !cancelled() }} env: @@ -1094,7 +1091,6 @@ jobs: CXX: ccache x86_64-w64-mingw32ucrt-g++ run: *compiler_warnings_cmd - ### # Verify docs can be built ### -- 2.54.0.380.gc69baaf57b
>From da626d9c12f4fdb23714aac1cc85b4eab0f3e80a Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 13:38:47 -0400 Subject: [PATCH v9a 17/22] ci: Don't run setup twice --- .github/workflows/pg-ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index a570b9e2c8d..fa347847e02 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -317,7 +317,7 @@ jobs: meson test ${{env.MTEST_ARGS}} --suite setup --logbase setup || exit 1 echo ::endgroup:: - meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} ${{env.MTEST_TARGET}} + meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} --no-suite setup ${{env.MTEST_TARGET}} - &linux_collect_cores_step name: Core backtraces -- 2.54.0.380.gc69baaf57b
>From c7baf96653181f0a2ff360edc30fe49b0703c752 Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 13:49:09 -0400 Subject: [PATCH v9a 18/22] ci: Only use pwsh, not pwsh and powershell --- .github/workflows/pg-ci.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index fa347847e02..3d8845bbda6 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -747,7 +747,7 @@ jobs: steps: - &windows_disable_defender_step name: Disable Windows Defender - shell: powershell + shell: pwsh run: | Set-MpPreference -DisableRealtimeMonitoring $true -SubmitSamplesConsent NeverSend -MAPSReporting Disable # Verify Defender status @@ -933,7 +933,7 @@ jobs: # robocopy returns 0-7 on success (with various "files copied" bits # set) and 8+ on real failure, so we have to translate its exit code. - name: Relocate MSYS2 to D - shell: powershell + shell: pwsh run: | robocopy C:\msys64 D:\msys64 /E /MT:16 /NJS /NJH /NFL /NDL /NP if ($LASTEXITCODE -ge 8) { exit $LASTEXITCODE } -- 2.54.0.380.gc69baaf57b
>From 4c1553cb8eed704f2ecd37ccf7023fb336ebf88d Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 13:49:29 -0400 Subject: [PATCH v9a 19/22] ci: windows-mingw, also setup hosts file --- .github/workflows/pg-ci.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 3d8845bbda6..adb5ef5780e 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -846,7 +846,8 @@ jobs: if (!$?) { throw 'cmdfail' } echo ::endgroup:: - - name: Setup hosts file + - &window_setup_hosts_step + name: Setup hosts file shell: pwsh run: | Add-Content c:\Windows\System32\Drivers\etc\hosts "127.0.0.1 pg-loadbalancetest" @@ -922,6 +923,7 @@ jobs: steps: - *windows_disable_defender_step + - *window_setup_hosts_step - *checkout_step # Relocate the preinstalled MSYS2 tree from C:\ (slow system disk) to -- 2.54.0.380.gc69baaf57b
>From b22ae7bff1c9b882a269013a72b933db1ca81fda Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 13:58:08 -0400 Subject: [PATCH v9a 20/22] ci: Collect config.log from autoconf builds after failures --- .github/workflows/pg-ci.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index adb5ef5780e..b3997ab42ad 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -337,6 +337,7 @@ jobs: **/regress_log_* **/crashlog-*.txt build/meson-logs/** + **/config.log if-no-files-found: ignore @@ -1130,3 +1131,4 @@ jobs: EXTRAFLAGS='-fmax-errors=10' - *ccache_save_step + - *upload_logs_step -- 2.54.0.380.gc69baaf57b
>From d92ed4fbbe48da68ec4d4a167191a022d34db13f Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 14:04:07 -0400 Subject: [PATCH v9a 21/22] ci: linux: Run on ubuntu-24.04 and define what we run on centrally Centrally define the version of linux runners, to make it easier to update. We don't just want to use ubuntu-latest, as it's not implausible --- .github/workflows/pg-ci.yml | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index b3997ab42ad..93b17af46df 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -114,6 +114,12 @@ env: --with-uuid=ossp --with-zstd + # Centrally define the version of linux runners, to make it easier to + # update. We don't just want to use ubuntu-latest, as it's not implausible + # there will be breakage when that switches to the next ubuntu version. + _LINUX_RUNS_ON: &linux_runs_on | + ubuntu-24.04 + # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/'. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -133,7 +139,7 @@ jobs: # consumed by the jobs' `if:` conditions. setup: name: Determine enabled jobs - runs-on: ubuntu-latest + runs-on: *linux_runs_on timeout-minutes: 1 outputs: linux: ${{ steps.os.outputs.linux }} @@ -198,7 +204,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: ubuntu-latest + runs-on: *linux_runs_on timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -357,7 +363,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: ubuntu-latest + runs-on: *linux_runs_on container: *linux_ci_container timeout-minutes: 60 @@ -449,7 +455,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: ubuntu-latest + runs-on: *linux_runs_on container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -539,7 +545,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: ubuntu-latest + runs-on: *linux_runs_on container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -1022,7 +1028,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: ubuntu-latest + runs-on: *linux_runs_on timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} -- 2.54.0.380.gc69baaf57b
>From b13f1fc1e089c8b52437d2895fd08f14c766debb Mon Sep 17 00:00:00 2001 From: Andres Freund <[email protected]> Date: Wed, 3 Jun 2026 13:41:53 -0400 Subject: [PATCH v9a 22/22] ci: Slice tests on windows-vs across two runners Without windows-vs is about 10min slower than the others tasks (windows-vs is currently 31m40s, windows-mingw is 20m45s), which makes for a frustrating experience. It seems worth "sacrificing" one of the 20 available concurrent jobs to avoid that. This reduces the time down to about 18m. ci-os-only: windows --- .github/workflows/pg-ci.yml | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 93b17af46df..83ee93ce5d6 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -723,8 +723,13 @@ jobs: # Job: Windows - Visual Studio + # + # If we were to execute tests in this job serially, this would be the + # slowest job by a good margin. To avoid that, use a matrix in combination + # with meson test's --slice SLICE/NUM_SLICES mechanism to split the tests + # across two runners. windows-vs: - name: Windows - Visual Studio + name: Windows - Visual Studio - Slice ${{ matrix.slice}}/${{ matrix.num_slices}} needs: [setup, sanity-check] if: | !cancelled() && @@ -732,6 +737,17 @@ jobs: needs.sanity-check.result != 'failure' runs-on: windows-2022 timeout-minutes: 60 + + # As described at the top of the task, split the tests across two runners + # for performance. The gains from additional concurrency diminish + # relatively quickly, due to each instance having to install dependencies + # and build postgres. + strategy: + fail-fast: false + matrix: + num_slices: [2] + slice: [1, 2] + env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 @@ -885,6 +901,11 @@ jobs: - name: Test world env: + # As described at the top of the task, split the tests across two + # runners for performance. It's not the prettiest to implement this + # by prepending to MTEST_TARGET, but a more complicated solution + # doesn't seem worth it. + MTEST_TARGET: --slice ${{ matrix.slice}}/${{ matrix.num_slices}} ${{env.MTEST_TARGET}} ADDITIONAL_SETUP: | call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 run: *meson_test_world_cmd -- 2.54.0.380.gc69baaf57b
