This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new e797e99fcbe [fix](thirdparty) Upgrade libunwind to 1.8.3 to fix
aarch64 BE startup crash on 64 KiB-page kernels (#67877)
e797e99fcbe is described below
commit e797e99fcbe14193871655af8cc91886c820751e
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Sat Sep 12 21:22:53 2026 +0800
[fix](thirdparty) Upgrade libunwind to 1.8.3 to fix aarch64 BE startup
crash on 64 KiB-page kernels (#67877)
### What problem does this PR solve?
Related PR: #64093
Problem Summary:
**Every aarch64 BE built from master or branch-4.1 (since 4.1.4-rc01)
crashes before `main()` on a 64 KiB-page kernel** (Kunpeng 920 + Kylin
V10 / openEuler / CentOS aarch64, the common ARM server stack). The core
from a 4.1.4 arm64 deployment shows:
```
access_mem (addr=0) src/aarch64/Ginit.c
is_plt_entry src/aarch64/Gstep.c
_ULaarch64_step
_ULaarch64_tdep_trace
unw_backtrace
<bthread/mutex.cpp static initializer: backtrace(dummy_buf, 4)> ← bRPC
"Warm up backtrace before main()"
__libc_csu_init
```
SIGSEGV with `si_addr=0`, AUXV `PAGESZ=65536`, and inside the libunwind
cursor `ip=0`, `validate=0`, `mem_validate_func=msync_validate`,
`last_good_addr[]={0,0,0,0}`.
How it happens:
1. #64093 links GNU libunwind statically into the aarch64 BE (it forces
`USE_UNWIND=ON` on Linux, references `unw_*` from `phdr_cache.cpp`
without the `__x86_64__` guard, and builds jemalloc with
`--enable-prof-libunwind`). libunwind's weak `backtrace` alias then
replaces glibc's `backtrace()` for every caller in the process: bRPC's
pre-main warm-up, `StackTrace::tryCapture()` on aarch64, jemalloc heap
profiling. The 4.1.3 arm64 package was immune only because it was built
with `USE_UNWIND=OFF` (which the ARM compilation doc still prescribes
and #64093 removed): it has `backtrace@GLIBC_2.17` and zero
`_ULaarch64_*` symbols.
2. libunwind 1.6.2 hard-codes `PAGE_SIZE 4096` in `src/aarch64/Ginit.c`.
On a 64 KiB-page kernel `mincore()`/`msync()` reject its 4 KiB-aligned
probes with `EINVAL`, so `validate_mem()` fails for 15/16 of all
addresses (the core's `msync_validate` + empty `last_good_addr` are
exactly this).
3. `unw_step()` therefore gets `unw_is_signal_frame() < 0` for a valid
IP, takes the "IP points to non-mapped memory, use LR" recovery path and
loads `uc.regs[30]` — a slot `_Uaarch64_getcontext_trace` never writes
(it saves only FP/SP/PC). On the reporting host that stack garbage is 0.
4. `dwarf_step(ip=0)` finds nothing, and the fallback calls
`is_plt_entry()` with `c->validate` restored to 0, so `access_mem`
executes `ldr x8, [x1]` with `x1=0`.
The PHDR cache being empty before `main()` is not what breaks this (step
3 has already poisoned `ip` before any FDE lookup, and the same crash
reproduces without the Doris hook); changing the cache initialisation
order would not fix it, and every later `backtrace()` on such a host
rolls the same dice.
Fix: **upgrade libunwind 1.6.2 → 1.8.3**, which was never bumped since
it was introduced in 2023 (#21938):
- 1.7.0+: `unw_page_size = sysconf(_SC_PAGESIZE)` instead of the
hard-coded 4096.
- 1.8.0+: `src/mi/Gaddress_validator.c` validates through the pipe-write
probe only, so `mincore`/`msync` alignment no longer matters at all;
`unw_step()` sets `c->validate = 1` before the DWARF-failure fallback,
so a garbage IP is rejected instead of dereferenced.
Changes:
- `thirdparty/vars.sh`: libunwind 1.6.2 → 1.8.3 (md5
`13bc7b41462ac6ea157d350eaf6c1503`).
- `thirdparty/patches/libunwind-1.8.3-doris-phdr-cache.patch`: the
#64093 hook rebased onto 1.8.3's `as->iterate_phdr_function` call site;
semantics unchanged (`doris_unwind_iterate_phdr` still takes precedence
whenever it is linked in). 1.8 also exposes
`unw_set_iterate_phdr_function()`, which could replace the source patch
in a follow-up.
- `thirdparty/build-thirdparty.sh`: `--disable-tests
--disable-documentation`; only `libunwind.a` is consumed.
- `thirdparty/test/libunwind-page-size-test.sh` (+ run at the end of the
Linux thirdparty CI build): reproduces the failure on any Linux host. An
`LD_PRELOAD` shim gives libunwind a 64 KiB-page kernel's behaviour
(`EINVAL` for unaligned `mincore`/`msync`,
`sysconf(_SC_PAGESIZE)=65536`), and a program shaped like the bRPC
warm-up calls `backtrace()` from a constructor with a zeroed stack.
Against 1.6.2 it segfaults with the customer's exact stack and cursor
state; against 1.8.3 it passes.
No BE source change: every `unw_*` API the BE, jemalloc and the x86_64
stack-trace code use is unchanged in 1.8.3, `cxx_exceptions` stays off
on x86_64/aarch64, and the weak `backtrace` alias is still provided.
Follow-ups outside this PR: the doris-website ARM compilation page still
says `export USE_UNWIND=OFF` is required, which has been a silent no-op
since #64093.
### Release note
Fix the aarch64 BE crashing before `main()` (SIGSEGV in
`access_mem`/`is_plt_entry` under `unw_backtrace`) on 64 KiB-page
kernels such as Kunpeng 920 with Kylin, openEuler or CentOS, by
upgrading the bundled libunwind from 1.6.2 to 1.8.3.
### Check List (For Author)
- Test
- [x] Manual test (add detailed scripts or steps below)
- Symbolised the customer core against the matching 4.1.4 arm64
`doris_be` (gdb): crash stack, `si_addr=0`, `PAGESZ=65536`, cursor
`ip=0`/`validate=0`, `mem_validate_func=msync_validate`,
`last_good_addr={0}`, `uc.regs[30]=0`; `backtrace` == `unw_backtrace` in
the 4.1.4 binary vs `backtrace@GLIBC_2.17` in 4.1.3.
- Reproduced the identical crash on a 4 KiB arm64 Linux container by
running the real 4.1.4 `doris_be --version` under the 64 KiB shim and
zeroing `uc.regs[30]` in gdb.
- Built libunwind 1.6.2 (+Doris patch) and 1.8.3 (+rebased patch) for
aarch64 with the exact recipe from `build_libunwind()`; ran
`thirdparty/test/libunwind-page-size-test.sh` against both: 1.6.2 FAIL
(SIGSEGV, same stack), 1.8.3 PASS with and without the Doris hook, and
with `ip` forced to 0 in gdb.
- Same build + test on x86_64 (emulated ubuntu-22.04): PASS.
- The rebased patch applies with both GNU `patch` and Apple `patch`;
`thirdparty/test/download-thirdparty-*-test.sh`,
`juicefs-default-mirror-test.sh`, `azure-vcpkg-retry-test.sh` pass;
shellcheck reports no new findings.
- Not done here: a Linux `doris_be` rebuilt with 1.8.3 on a real 64
KiB-page host. That is the one check that still needs an aarch64
machine.
- Behavior changed:
- [x] Yes. Bundled libunwind 1.6.2 → 1.8.3; libunwind's own test
programs and man pages are no longer built. Unwinding semantics for the
BE otherwise unchanged.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_016MdauUJh8AtE8SozZ29Dmv
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
---
.github/workflows/build-thirdparty.yml | 5 +
thirdparty/CHANGELOG.md | 11 ++
thirdparty/build-thirdparty.sh | 4 +-
thirdparty/download-thirdparty.sh | 4 +-
...atch => libunwind-1.8.3-doris-phdr-cache.patch} | 8 +-
thirdparty/test/libunwind-page-size-test.sh | 172 +++++++++++++++++++++
thirdparty/vars.sh | 8 +-
7 files changed, 201 insertions(+), 11 deletions(-)
diff --git a/.github/workflows/build-thirdparty.yml
b/.github/workflows/build-thirdparty.yml
index 6baff66f3b7..5e25d5fdc87 100644
--- a/.github/workflows/build-thirdparty.yml
+++ b/.github/workflows/build-thirdparty.yml
@@ -196,6 +196,11 @@ jobs:
#./build-thirdparty.sh -j "$(nproc)"
./build-thirdparty.sh -j 2
+ # CI has no 64 KiB-page aarch64 host; this drives the libunwind just
built
+ # through an emulated one, so it also runs on this x86_64 4 KiB
runner.
+ cd "${GITHUB_WORKSPACE}"
+ thirdparty/test/libunwind-page-size-test.sh
+
build_macos:
name: Build Third Party Libraries (macOS)
needs: changes
diff --git a/thirdparty/CHANGELOG.md b/thirdparty/CHANGELOG.md
index f56ce806730..f0df535e5f6 100644
--- a/thirdparty/CHANGELOG.md
+++ b/thirdparty/CHANGELOG.md
@@ -2,6 +2,17 @@
This file contains version of the third-party dependency libraries in the
build-env image. The docker build-env image is apache/doris, and the tag is
`build-env-${version}`
+## 20260911
+
+- Modified: libunwind 1.6.2 -> 1.8.3. 1.6.2 hard-codes a 4 KiB page size in the
+ AArch64 memory validator (`mincore`/`msync` on 4 KiB-aligned addresses), so
on a
+ 64 KiB-page kernel every validation fails, `unw_step` falls back to a link
register
+ that `getcontext_trace` never saved and dereferences it unvalidated. This
crashed
+ every aarch64 BE on such kernels before `main()`, in bRPC's `backtrace()`
warm-up.
+ 1.8.x reads the page size at runtime, validates through the pipe-write probe
only,
+ and validates addresses before the DWARF-failure fallback. The Doris
PHDR-cache hook
+ patch is rebased onto 1.8.3; the test programs and man pages are no longer
built.
+
## 20260909
- Modified: snappy 1.1.10 -> 1.2.1. Enable x86 SSE4.2 paths and AVX2 by
default;
diff --git a/thirdparty/build-thirdparty.sh b/thirdparty/build-thirdparty.sh
index 6329b1276fc..d0b19c01622 100755
--- a/thirdparty/build-thirdparty.sh
+++ b/thirdparty/build-thirdparty.sh
@@ -1834,7 +1834,9 @@ build_libunwind() {
# LIBUNWIND_IS_NATIVE_ONLY:
https://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20160523/159802.html
# -nostdinc++ only required for gcc compilation
cflags="-I${TP_INCLUDE_DIR} -std=c99 -D_LIBUNWIND_NO_HEAP=1 -D_DEBUG
-D_LIBUNWIND_IS_NATIVE_ONLY -O3 -fno-exceptions -funwind-tables
-fno-sanitize=all -nostdinc++ -fno-rtti -Wno-error=incompatible-pointer-types"
- CFLAGS="${cflags}" LDFLAGS="-L${TP_LIB_DIR} -llzma" ../configure
--prefix="${TP_INSTALL_DIR}" --disable-shared --enable-static
+ # Only the library is consumed; the test programs and man pages are
not.
+ CFLAGS="${cflags}" LDFLAGS="-L${TP_LIB_DIR} -llzma" ../configure
--prefix="${TP_INSTALL_DIR}" --disable-shared --enable-static \
+ --disable-tests --disable-documentation
make -j "${PARALLEL}"
make install
diff --git a/thirdparty/download-thirdparty.sh
b/thirdparty/download-thirdparty.sh
index 7beb47291f8..8527f6c423d 100755
--- a/thirdparty/download-thirdparty.sh
+++ b/thirdparty/download-thirdparty.sh
@@ -526,10 +526,10 @@ fi
# patch libunwind so Doris can force GNU libunwind to use the BE PHDR cache
# without changing ordinary dl_iterate_phdr callers.
if [[ " ${TP_ARCHIVES[*]} " =~ " LIBUNWIND " ]]; then
- if [[ "${LIBUNWIND_SOURCE}" = "libunwind-1.6.2" ]]; then
+ if [[ "${LIBUNWIND_SOURCE}" = "libunwind-1.8.3" ]]; then
cd "${TP_SOURCE_DIR}/${LIBUNWIND_SOURCE}"
if [[ ! -f "${PATCHED_MARK}" ]]; then
- patch -p1 <"${TP_PATCH_DIR}/libunwind-1.6.2-doris-phdr-cache.patch"
+ patch -p1 <"${TP_PATCH_DIR}/libunwind-1.8.3-doris-phdr-cache.patch"
touch "${PATCHED_MARK}"
fi
cd -
diff --git a/thirdparty/patches/libunwind-1.6.2-doris-phdr-cache.patch
b/thirdparty/patches/libunwind-1.8.3-doris-phdr-cache.patch
similarity index 83%
rename from thirdparty/patches/libunwind-1.6.2-doris-phdr-cache.patch
rename to thirdparty/patches/libunwind-1.8.3-doris-phdr-cache.patch
index e534780b725..ea561b39a05 100644
--- a/thirdparty/patches/libunwind-1.6.2-doris-phdr-cache.patch
+++ b/thirdparty/patches/libunwind-1.8.3-doris-phdr-cache.patch
@@ -1,8 +1,8 @@
diff --git a/src/dwarf/Gfind_proc_info-lsb.c b/src/dwarf/Gfind_proc_info-lsb.c
-index 1d0d6a4..8f47463 100644
+index c11345e..837fbfe 100644
--- a/src/dwarf/Gfind_proc_info-lsb.c
+++ b/src/dwarf/Gfind_proc_info-lsb.c
-@@ -47,6 +47,14 @@ struct table_entry
+@@ -46,6 +46,14 @@ struct table_entry
#ifndef UNW_REMOTE_ONLY
@@ -21,11 +21,11 @@ index 1d0d6a4..8f47463 100644
cb_data.di_debug.format = -1;
SIGPROCMASK (SIG_SETMASK, &unwi_full_mask, &saved_mask);
-- ret = dl_iterate_phdr (dwarf_callback, &cb_data);
+- ret = as->iterate_phdr_function (dwarf_callback, &cb_data);
+ if (doris_unwind_iterate_phdr)
+ ret = doris_unwind_iterate_phdr (dwarf_callback, &cb_data, ip);
+ else
-+ ret = dl_iterate_phdr (dwarf_callback, &cb_data);
++ ret = as->iterate_phdr_function (dwarf_callback, &cb_data);
SIGPROCMASK (SIG_SETMASK, &saved_mask, NULL);
if (ret > 0)
diff --git a/thirdparty/test/libunwind-page-size-test.sh
b/thirdparty/test/libunwind-page-size-test.sh
new file mode 100755
index 00000000000..451316769cd
--- /dev/null
+++ b/thirdparty/test/libunwind-page-size-test.sh
@@ -0,0 +1,172 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# libunwind 1.6.2 hard-coded a 4 KiB page size in its AArch64 memory validator.
+# On a 64 KiB-page kernel (Kunpeng/Kylin/openEuler/CentOS aarch64) mincore()
and
+# msync() reject its 4 KiB-aligned probes with EINVAL, every validation fails,
+# unw_step() falls back to a link register that getcontext_trace never saved,
+# and is_plt_entry() dereferences that garbage unvalidated. Because the static
+# libunwind's weak `backtrace` alias replaces glibc's, bRPC's pre-main
+# `backtrace()` warm-up crashed every aarch64 BE on such hosts before main().
+#
+# CI has no 64 KiB-page host, so this drives the installed libunwind.a through
a
+# program shaped like that warm-up while an LD_PRELOAD shim makes the kernel
+# look like a 64 KiB-page one to libunwind: the same EINVAL for unaligned
+# probes and a 64 KiB sysconf(_SC_PAGESIZE). Run it after
+# `build-thirdparty.sh libunwind` on Linux; it only needs a C compiler.
+
+set -eo pipefail
+
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." &>/dev/null && pwd)"
+TP_INSTALL_DIR="${TP_INSTALL_DIR:-${ROOT}/installed}"
+
+fail() {
+ echo "FAIL: $*" >&2
+ exit 1
+}
+
+if [[ "$(uname -s)" != 'Linux' ]]; then
+ echo "SKIP: GNU libunwind is only built on Linux"
+ exit 0
+fi
+
+if [[ -n "${CC:-}" ]]; then
+ cc="${CC}"
+else
+ for candidate in gcc clang cc; do
+ if command -v "${candidate}" >/dev/null 2>&1; then
+ cc="${candidate}"
+ break
+ fi
+ done
+fi
+[[ -n "${cc:-}" ]] || fail "no C compiler found; set CC"
+[[ -f "${TP_INSTALL_DIR}/lib/libunwind.a" ]] || fail
"${TP_INSTALL_DIR}/lib/libunwind.a not found; build libunwind first"
+
+tmpdir="$(mktemp -d)"
+trap 'rm -rf "${tmpdir}"' EXIT
+
+cat >"${tmpdir}/shim64k.c" <<'EOF'
+#define _GNU_SOURCE
+#include <dlfcn.h>
+#include <errno.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <sys/mman.h>
+#include <unistd.h>
+
+/* What a 64 KiB-page kernel does with libunwind's validation probes. */
+#define EMU_PAGE 65536UL
+
+int mincore(void *addr, size_t len, unsigned char *vec) {
+ static int (*real)(void *, size_t, unsigned char *);
+ if (!real) real = dlsym(RTLD_NEXT, "mincore");
+ if ((uintptr_t)addr & (EMU_PAGE - 1)) { errno = EINVAL; return -1; }
+ size_t n = (len + EMU_PAGE - 1) / EMU_PAGE;
+ for (size_t i = 0; i < n; i++) {
+ unsigned char tmp[EMU_PAGE / 4096];
+ if (real((char *)addr + i * EMU_PAGE, EMU_PAGE, tmp) != 0) return -1;
+ unsigned char any = 0;
+ for (size_t j = 0; j < sizeof tmp; j++) any |= tmp[j] & 1;
+ vec[i] = any;
+ }
+ return 0;
+}
+int msync(void *addr, size_t len, int flags) {
+ static int (*real)(void *, size_t, int);
+ if (!real) real = dlsym(RTLD_NEXT, "msync");
+ if ((uintptr_t)addr & (EMU_PAGE - 1)) { errno = EINVAL; return -1; }
+ return real(addr, len, flags);
+}
+long sysconf(int name) {
+ static long (*real)(int);
+ if (!real) real = dlsym(RTLD_NEXT, "sysconf");
+ if (name == _SC_PAGESIZE) return EMU_PAGE;
+ return real(name);
+}
+int getpagesize(void) { return (int)EMU_PAGE; }
+EOF
+
+cat >"${tmpdir}/premain.c" <<'EOF'
+#define _GNU_SOURCE
+#include <execinfo.h>
+#include <link.h>
+#include <stdio.h>
+#include <string.h>
+#include <unistd.h>
+
+/* be/src/common/phdr_cache.cpp before main(): the PHDR cache is still NULL,
so the
+ * Doris hook that the libunwind patch routes dwarf_find_proc_info() through
finds
+ * nothing. */
+int doris_unwind_iterate_phdr(int (*callback)(struct dl_phdr_info *, size_t,
void *), void *data,
+ unsigned long ip) {
+ (void)callback; (void)data; (void)ip;
+ return 0;
+}
+
+static void *dummy_buf[4];
+static int dummy_bt = -1;
+
+/* Zero the stack below us, like the never-touched region the warm-up ran on,
so the
+ * uc.regs[30] slot that _Uaarch64_getcontext_trace never writes reads as 0. */
+static void __attribute__((noinline)) scrub_stack(void) {
+ volatile char buf[256 * 1024];
+ memset((char *)buf, 0, sizeof(buf));
+ __asm__ volatile("" ::: "memory");
+}
+
+/* bRPC src/bthread/mutex.cpp: "Warm up backtrace before main()." */
+__attribute__((constructor)) static void warm_up(void) {
+ scrub_stack();
+ dummy_bt = backtrace(dummy_buf, 4);
+}
+
+int main(void) {
+ void *buf[32];
+ int n = backtrace(buf, 32);
+ printf("premain=%d runtime=%d pagesz=%ld\n", dummy_bt, n,
sysconf(_SC_PAGESIZE));
+ return (dummy_bt < 0 || n < 0) ? 1 : 0;
+}
+EOF
+
+extra_libs=()
+[[ -f "${TP_INSTALL_DIR}/lib/libz.a" ]] && extra_libs+=(-lz)
+
+"${cc}" -O2 -g -shared -fPIC -o "${tmpdir}/shim64k.so" "${tmpdir}/shim64k.c"
-ldl
+# -u unw_backtrace pulls libunwind's backtrace.o in, whose weak `backtrace`
alias then
+# takes over the call, exactly as it does in doris_be.
+"${cc}" -O2 -g -fno-omit-frame-pointer -o "${tmpdir}/premain"
"${tmpdir}/premain.c" \
+ -I"${TP_INSTALL_DIR}/include" -L"${TP_INSTALL_DIR}/lib" \
+ -Wl,-u,unw_backtrace -lunwind -llzma "${extra_libs[@]}" -lpthread
+
+if command -v nm >/dev/null 2>&1; then
+ nm "${tmpdir}/premain" | grep -Eq ' [WT] backtrace$' ||
+ fail "backtrace() is not bound to libunwind's unw_backtrace in the
test program"
+fi
+
+echo "== native page size =="
+out="$("${tmpdir}/premain")" || fail "pre-main backtrace() crashed on the
native page size: ${out}"
+echo "${out}"
+
+echo "== emulated 64 KiB page size =="
+out="$(LD_PRELOAD="${tmpdir}/shim64k.so" "${tmpdir}/premain")" ||
+ fail "pre-main backtrace() crashed on an emulated 64 KiB-page kernel:
${out}"
+echo "${out}"
+[[ "${out}" == *"pagesz=65536"* ]] || fail "the 64 KiB page emulation did not
take effect: ${out}"
+
+echo "PASS"
diff --git a/thirdparty/vars.sh b/thirdparty/vars.sh
index 3bf11ff80cf..c0d4e0079e9 100644
--- a/thirdparty/vars.sh
+++ b/thirdparty/vars.sh
@@ -353,10 +353,10 @@ JEMALLOC_DORIS_SOURCE="jemalloc-5.3.0"
JEMALLOC_DORIS_MD5SUM="09a8328574dab22a7df848eae6dbbf53"
# libunwind
-LIBUNWIND_DOWNLOAD="https://github.com/libunwind/libunwind/releases/download/v1.6.2/libunwind-1.6.2.tar.gz";
-LIBUNWIND_NAME="libunwind-1.6.2.tar.gz"
-LIBUNWIND_SOURCE="libunwind-1.6.2"
-LIBUNWIND_MD5SUM="f625b6a98ac1976116c71708a73dc44a"
+LIBUNWIND_DOWNLOAD="https://github.com/libunwind/libunwind/releases/download/v1.8.3/libunwind-1.8.3.tar.gz";
+LIBUNWIND_NAME="libunwind-1.8.3.tar.gz"
+LIBUNWIND_SOURCE="libunwind-1.8.3"
+LIBUNWIND_MD5SUM="13bc7b41462ac6ea157d350eaf6c1503"
# cctz
CCTZ_DOWNLOAD="https://github.com/google/cctz/archive/refs/tags/v2.5.tar.gz";
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
