This is an automated email from the ASF dual-hosted git repository.
zeroshade pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-go.git
The following commit(s) were added to refs/heads/main by this push:
new 9e0b802a ci: use clang ASan runtime for -asan tests (#1299)
9e0b802a is described below
commit 9e0b802a81ef7796af80bde49a631f54b284e872
Author: Matt Topol <[email protected]>
AuthorDate: Fri Sep 18 10:14:35 2026 -0700
ci: use clang ASan runtime for -asan tests (#1299)
### Rationale for this change
The Ubuntu RC verification job intermittently fails with
AddressSanitizer *internal* CHECK failures under `go test -asan`,
striking arbitrary pure-Go packages with no failing Go assertion. Most
recent occurrence on `main`:
https://github.com/apache/arrow-go/actions/runs/34250658983/job/102145334765
—
```text
AddressSanitizer: CHECK failed: sanitizer_common.h:522 "((i)) < ((size_))"
#2 InternalMmapVectorNoCtor<ThreadContextBase*>::operator[]
#3 ThreadRegistry::StartThread sanitizer_thread_registry.cpp:314
#4 AsanThread::ThreadStart
FAIL github.com/apache/arrow-go/v18/arrow/util 0.287s
```
Sibling signatures over the last month include
`sanitizer_thread_registry.cpp:161/316`,
`sanitizer_allocator_secondary.h:297`, and plain segfaults, hitting
`arrow/float16`, `arrow/internal/dictutils`, `arrow/util`, and
`arrow/flight/flightsql/example`. Every crash site is GCC libasan's
thread-registry/allocator bookkeeping at thread start or exit: the Go
runtime creates and retires threads in ways GCC's libasan thread
registry does not tolerate. This is the dominant recurring RC
verification flake (~5 of the last ~11 Ubuntu failures).
### What changes are included in this PR?
When `ci/scripts/test.sh` selects `-asan`, export
`CC=clang`/`CXX=clang++` if clang is available and `CC` is not already
set, so the test binaries link LLVM's compiler-rt ASan runtime instead
of GCC's libasan. Environments without clang (e.g. the Debian golang
images) keep the current behavior unchanged.
`ubuntu-latest` runners ship clang, so the RC `Verify (ubuntu-latest)`
job — the only recurring victim — picks up the LLVM runtime
automatically.
### Are these changes tested?
- `bash -n` and `shellcheck` pass on the edited script
- This PR's own RC `Verify (ubuntu-latest)` check executes exactly this
code path with clang
### Are there any user-facing changes?
No, CI-only.
---------
Signed-off-by: Matt Topol <[email protected]>
---
.github/workflows/rc.yml | 9 +++++++++
ci/docker/debian-12.dockerfile | 9 +++++++++
ci/scripts/test.sh | 45 +++++++++++++++++++++++++++++++++++++++++-
3 files changed, 62 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/rc.yml b/.github/workflows/rc.yml
index e0ee7944..a20c24ab 100644
--- a/.github/workflows/rc.yml
+++ b/.github/workflows/rc.yml
@@ -96,6 +96,15 @@ jobs:
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e #
v6.2.0
with:
go-version-file: go.mod
+ - name: Install ASan runtime
+ # ci/scripts/test.sh only runs -asan against an LLVM >= 19 runtime;
+ # ubuntu-latest still defaults to clang 18.
+ if: runner.os == 'Linux'
+ run: |
+ sudo apt-get update
+ sudo apt-get install -y --no-install-recommends \
+ clang-19 \
+ libclang-rt-19-dev
- name: Verify
run: |
tar_gz=$(echo apache-arrow-go-*.tar.gz)
diff --git a/ci/docker/debian-12.dockerfile b/ci/docker/debian-12.dockerfile
index 95187272..e2885812 100644
--- a/ci/docker/debian-12.dockerfile
+++ b/ci/docker/debian-12.dockerfile
@@ -19,6 +19,15 @@ ARG arch=amd64
ARG go=1.24
FROM ${arch}/golang:${go}-bookworm
+# ci/scripts/test.sh only runs -asan against an LLVM >= 19 runtime; the
+# libsanitizer shipped with the image's GCC predates the thread-registry fix.
+RUN apt-get update -y -q && \
+ apt-get install -y -q --no-install-recommends \
+ clang-19 \
+ libclang-rt-19-dev && \
+ apt-get clean && \
+ rm -rf /var/lib/apt/lists/*
+
# Copy the go.mod and go.sum over and pre-download all the dependencies
COPY . /arrow-go
RUN cd /arrow-go && \
diff --git a/ci/scripts/test.sh b/ci/scripts/test.sh
index 411fd35a..45719d45 100755
--- a/ci/scripts/test.sh
+++ b/ci/scripts/test.sh
@@ -25,6 +25,22 @@ export PARQUET_TEST_DATA=${1}/parquet-testing/data
export PARQUET_TEST_BAD_DATA=${1}/parquet-testing/bad_data
export ARROW_TEST_DATA=${1}/arrow-testing/data
+# Go's -asan links whatever AddressSanitizer runtime the C toolchain ships.
+# Runtimes older than LLVM 19 allocate thread contexts from the global
+# low-level allocator, which is not thread safe, so they corrupt their own
+# thread registry when the Go runtime creates and retires threads
+# concurrently and abort test binaries at random in arbitrary packages
+# (llvm/llvm-project#87324, fixed by llvm/llvm-project#88177). GCC's
+# libsanitizer snapshot predates that fix as well, so -asan is only reliable
+# when a clang >= 19 runtime is available.
+asan_runtime_major() {
+ local version
+ version=$("${1}" --version 2>/dev/null | head -1) || return 1
+ [[ "${version}" = *"clang version "* ]] || return 1
+ version=${version##*clang version }
+ echo "${version%%.*}"
+}
+
case "$(uname)" in
MINGW*)
# -race and -asan don't work on Windows currently
@@ -39,7 +55,34 @@ MINGW*)
# -asan not supported on darwin/amd64
test_args=("-race")
else
- test_args=("-asan")
+ asan_cc=${CC:-}
+ if [[ -z "${asan_cc}" ]]; then
+ # Prefer the default clang when it is new enough, otherwise the
+ # newest explicitly installed one.
+ for candidate in clang clang-21 clang-20 clang-19; do
+ command -v "${candidate}" >/dev/null 2>&1 || continue
+ major=$(asan_runtime_major "${candidate}") || continue
+ if [[ "${major}" -ge 19 ]]; then
+ asan_cc=${candidate}
+ break
+ fi
+ done
+ fi
+
+ major=$(asan_runtime_major "${asan_cc:-false}") || major=0
+ if [[ "${major}" -ge 19 ]]; then
+ test_args=("-asan")
+ if [[ -z "${CC:-}" ]]; then
+ export CC=${asan_cc}
+ if [[ -z "${CXX:-}" ]] && command -v "${asan_cc/clang/clang++}"
>/dev/null 2>&1; then
+ export CXX=${asan_cc/clang/clang++}
+ fi
+ fi
+ else
+ # Every available ASan runtime predates the fix; -asan would abort at
+ # random, so run the race detector instead.
+ test_args=("-race")
+ fi
fi
fi
;;
