zclllyybb commented on issue #64187: URL: https://github.com/apache/doris/issues/64187#issuecomment-4643405783
Breakwater-GitHub-Analysis-Slot: slot_0e8b225c034a Thanks for the detailed report. I checked the current master code path, and the diagnosis looks consistent. Assessment: - This is best treated as an ARM64 + LSAN test-environment incompatibility, not as evidence that the set-operator overflow conversion logic is broken. - In `be/test/exec/operator/set_operator_test.cpp`, both affected tests build 4,200 rows of 1 MiB strings. The build side is accumulated before the final overflow conversion step. - `SetSinkOperatorX::sink()` first merges incoming build blocks through `MutableBlock::merge_ignore_overflow()`. That reaches `ColumnStr::insert_range_from_ignore_overflow()`, which resizes the destination `chars` buffer. - `PODArrayBase::reserve()` rounds the requested allocation up to the next power of two before `realloc`. Once the logical `chars` size crosses slightly over 4 GiB, the allocation request can become 8 GiB. - LLVM LeakSanitizer limits a single allocation to 4 GiB on `__aarch64__`, so the process aborts before the test reaches `_process_build_block()` / `convert_column_if_overflow()`. Recommended next step: - Add a targeted `GTEST_SKIP()` at the start of both `test_sink_large_string_data_over_4g` tests when `defined(__aarch64__) && defined(LEAK_SANITIZER)`. - Keep the tests enabled for x86_64 LSAN and for non-LSAN ARM64 builds, because they still cover the intended over-4GiB `ColumnString` conversion path where the allocator permits it. - I would not change `PODArray` growth behavior for this issue. Its power-of-two reserve policy is shared by many vector columns, and changing it would have broad performance and memory-behavior impact for a sanitizer-specific synthetic UT limit. Useful verification details to include in the PR: - The exact compiler / LLVM version used by the failing ARM64 LSAN build. - The full BE UT command and the first sanitizer error line showing the 8 GiB request versus the 4 GiB limit. - Confirmation that the tests still run on x86_64 LSAN and on ARM64 non-LSAN builds after adding the targeted skip. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
