Doris-Breakwater commented on issue #67448:
URL: https://github.com/apache/doris/issues/67448#issuecomment-5509507696
Breakwater-GitHub-Analysis-Slot: slot_f68ed80a2dc5
### Initial assessment
**Triage: confirmed master build regression / macOS arm64 build blocker.**
The issue is open with no labels or assignee. Suggested existing labels are
`kind/fix` and `area/aarch64`, with routing to the BE build owner.
### Verified facts
- The reported commit wires `kuromoji_build_dict` to the complete
`${DORIS_LINK_LIBS}` closure, and `kuromoji_dict` is an `ALL` target in normal
(`MAKE_TEST=OFF`) builds ([CMake target
definition](https://github.com/apache/doris/blob/ddbaaab13882dab3dfa51ffb8038218df869e360/be/CMakeLists.txt#L1101-L1141)).
The install failure is therefore downstream: the link failure prevents
generation of the four required dictionary files, and the install guard then
correctly rejects an incomplete package.
- On Darwin, `build.sh` defaults `USE_JEMALLOC=OFF` ([selection
logic](https://github.com/apache/doris/blob/ddbaaab13882dab3dfa51ffb8038218df869e360/build.sh#L618-L627)),
so a Release build selects and links tcmalloc. Doris already has a macOS-arm64
workaround for the same `arm64_b26` failure, but it is limited to `MAKE_TEST`
or `BUILD_BENCHMARK` ([existing allocator
exclusion](https://github.com/apache/doris/blob/ddbaaab13882dab3dfa51ffb8038218df869e360/be/CMakeLists.txt#L860-L873));
this new tool is created only when `NOT MAKE_TEST`, so that protection cannot
apply.
- The exact `ddbaaab1388` workflow run was canceled by concurrency, but the
immediately following master run
[#153446](https://github.com/apache/doris/actions/runs/33609174370/job/100180175805)
failed in `Build BE` at `bin/kuromoji_build_dict` with the same tcmalloc
fixup: displacement `-135769416`, `__TEXT` size `0x08324000`. The later master
run
[#153448](https://github.com/apache/doris/actions/runs/33620759231/job/100217120489)
repeated the identical failure and layout. This independently reproduces the
report on the repository's `macos-15` arm64 CI image.
- The CI verbose link command confirms that this small offline converter is
presented with all BE component archives, the full third-party closure, and
`libtcmalloc.a`. Its source directly needs the Kuromoji builder/parser plus
Doris `Status`; it does not conceptually need the BE service/runtime or a
production allocator. The target wiring was introduced by
[#64667](https://github.com/apache/doris/commit/11171329d6d0b56bf5d45b4f212aa311b92da946).
### Root-cause judgment
The evidence supports the issue's mechanism: the new offline tool reuses the
whole BE link closure in a normal macOS build, while that build selects
gperftools. The resulting Mach-O `__TEXT` places tcmalloc's custom
`google_malloc`/`malloc_hook` sections beyond the arm64 direct-branch range
from `___clang_call_terminate`; the linker reports that exact out-of-range
branch. This is not a missing-dictionary root cause, and it is no longer only a
reporter-specific/toolchain hypothesis.
The same failure family and allocator workaround already exist in
[#66615](https://github.com/apache/doris/commit/cc86cb3a3cb1061ff463128fb8758f49393dca22),
which is a useful project-local reference.
### Recommended next step
1. **Preferred durable fix:** give the dictionary converter a dedicated,
small dependency target containing the builder/parser implementation and only
the support libraries it actually needs; do not link it with
`${DORIS_LINK_LIBS}` or a custom allocator. Audit the actual closure rather
than assuming that `Storage`/`Common` alone is small, because those targets use
unity objects and can pull additional unresolved dependencies at archive
granularity.
2. **Contained unblocker if the dependency split is not immediately
practical:** omit tcmalloc only for `kuromoji_build_dict` on macOS arm64 and
provide the required no-op gperftools symbols using the established #66615
pattern. Keep `doris_be` allocator behavior unchanged. This removes the failing
custom sections but leaves an unnecessarily large build tool, so it is less
maintainable than dependency isolation.
3. Keep `kuromoji_dict` in `ALL` and keep the install-time file validation.
Dropping either only hides the failure and can produce a package without data
required by the runtime feature.
### Evidence required on the fixing PR
- A clean macOS arm64 `ninja -v kuromoji_build_dict` showing the reduced
link line, no `libtcmalloc.a`, and a comfortably sub-128-MB `__TEXT` (record
`size`/`otool` output before and after).
- A clean `ninja kuromoji_dict` followed by validation that `system.bin`,
`matrix.bin`, `chardef.bin`, and `unkdict.bin` are generated and non-empty.
- A full `DISABLE_BUILD_UI=ON sh build.sh --be` on macOS arm64, including
installation into `output/`, plus the relevant Kuromoji BE unit tests on a
supported test host.
- A Linux BE build to catch any dependency-list portability regression.
No additional reporter logs or reproduction details are required for initial
triage. The remaining unknown is patch-specific: the exact minimal
support-library closure must be demonstrated by the candidate link command and
clean build, not guessed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
