Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
amoeba commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3049481847 Thanks @kou that seems like a good idea. I changed the milestone. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
kou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3047745382 @amoeba We may want to add this to 21.0.0 for stable CI results. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
kou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3047743782 Issue resolved by pull request 47028 https://github.com/apache/arrow/pull/47028 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
kou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3047516497 Let's use it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
pitrou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3047506937 I can confirm that, if I install `mold` (`mamba install mold`) and then pass `ARROW_USE_MOLD=ON`, the crashes disappear. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
pitrou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3047494212 > 1. Use `mold` instead of GNU `ld` I like this too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
kou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3047493162 Oh, interesting... Here are available workarounds, right? 1. Use `mold` instead of GNU `ld` 2. Call `std::this_thread::sleep_for(std::chrono::nanoseconds(1))` before `std::call_once()` I like 1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
pitrou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3045787198 cc @bkietz @kou @zanmato1984 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
pitrou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3045786106 Upgrading to gcc 14.3.0 also does not fix the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
pitrou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3045750318 I tried the static singleton pattern and it doesn't work any better, unless I also add the `sleep_for` hack. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
pitrou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3045673113 Also libpthread is linked by that particular test, which crashes: ``` # ldd /build/cpp/debug/arrow-acero-fetch-node-test linux-vdso.so.1 (0x7dfb001cd000) libarrow_testing.so.2100 => /opt/conda/envs/arrow/lib/libarrow_testing.so.2100 (0x7dfaffeae000) libarrow_acero.so.2100 => /opt/conda/envs/arrow/lib/libarrow_acero.so.2100 (0x7dfaffb15000) libarrow_compute.so.2100 => /opt/conda/envs/arrow/lib/libarrow_compute.so.2100 (0x7dfafe0f4000) libarrow.so.2100 => /opt/conda/envs/arrow/lib/libarrow.so.2100 (0x7dfafc347000) libarrow_gtest.so.1.16.0 => /opt/conda/envs/arrow/lib/libarrow_gtest.so.1.16.0 (0x7dfafc299000) libstdc++.so.6 => /opt/conda/envs/arrow/lib/libstdc++.so.6 (0x7dfafc0aa000) libgcc_s.so.1 => /opt/conda/envs/arrow/lib/libgcc_s.so.1 (0x7dfafc08b000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7dfafbe52000) /lib64/ld-linux-x86-64.so.2 (0x7dfb001cf000) libboost_filesystem.so.1.85.0 => /opt/conda/envs/arrow/lib/libboost_filesystem.so.1.85.0 (0x7dfafbe27000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x7dfafbd4) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x7dfafbd3b000) libre2.so.9 => /opt/conda/envs/arrow/lib/libre2.so.9 (0x7dfafbcd5000) libutf8proc.so.3 => /opt/conda/envs/arrow/lib/libutf8proc.so.3 (0x7dfafbc7c000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x7dfafbc77000) libbrotlienc.so.1 => /opt/conda/envs/arrow/lib/libbrotlienc.so.1 (0x7dfafbbc9000) libbrotlidec.so.1 => /opt/conda/envs/arrow/lib/libbrotlidec.so.1 (0x7dfaf000) libbz2.so.1.0 => /opt/conda/envs/arrow/lib/libbz2.so.1.0 (0x7dfafbba7000) liblz4.so.1 => /opt/conda/envs/arrow/lib/liblz4.so.1 (0x7dfafbb7a000) libsnappy.so.1 => /opt/conda/envs/arrow/lib/libsnappy.so.1 (0x7dfafbb6d000) libz.so.1 => /opt/conda/envs/arrow/lib/libz.so.1 (0x7dfafbb52000) libzstd.so.1 => /opt/conda/envs/arrow/lib/libzstd.so.1 (0x7dfafba39000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x7dfafba34000) libbrotlicommon.so.1 => /opt/conda/envs/arrow/lib/./libbrotlicommon.so.1 (0x7dfafba0f000) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
pitrou commented on issue #47015:
URL: https://github.com/apache/arrow/issues/47015#issuecomment-3045686189
This comment gives us a clue:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60662#c5
And indeed, if I apply this hack, then `arrow-acero-fetch-node-test `
doesn't crash anymore:
```diff
diff --git a/cpp/src/arrow/acero/test_nodes.cc
b/cpp/src/arrow/acero/test_nodes.cc
index d95c22ca45..b1b2f4d831 100644
--- a/cpp/src/arrow/acero/test_nodes.cc
+++ b/cpp/src/arrow/acero/test_nodes.cc
@@ -17,10 +17,12 @@
#include "arrow/acero/test_nodes.h"
+#include
#include
#include
#include
#include
+#include
#include
#include "arrow/acero/exec_plan.h"
@@ -357,6 +359,7 @@ struct GatedNode : public ExecNode, public TracedNode {
} // namespace
void RegisterTestNodes() {
+ std::this_thread::sleep_for(std::chrono::nanoseconds(1));
static std::once_flag registered;
std::call_once(registered, [] {
ExecFactoryRegistry* registry = default_exec_factory_registry();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
pitrou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3045644386 It looks like an instance of https://stackoverflow.com/questions/51584960/stdcall-once-throws-stdsystem-error-unknown-error-1 , but `-pthread` appears in the compiler arguments, which makes it rather strange. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
pitrou commented on issue #47015:
URL: https://github.com/apache/arrow/issues/47015#issuecomment-3045581601
The backtrace for one of the crashes looks like this. It happens during test
initialization:
```
#0 __pthread_kill_implementation (no_tid=0, signo=6,
threadid=140652913054144) at ./nptl/pthread_kill.c:44
#1 __pthread_kill_internal (signo=6, threadid=140652913054144) at
./nptl/pthread_kill.c:78
#2 __GI___pthread_kill (threadid=140652913054144, signo=signo@entry=6) at
./nptl/pthread_kill.c:89
#3 0x7fec4f33e476 in __GI_raise (sig=sig@entry=6) at
../sysdeps/posix/raise.c:26
#4 0x7fec4f3247f3 in __GI_abort () at ./stdlib/abort.c:79
#5 0x7fec4f6110d9 in __gnu_cxx::__verbose_terminate_handler () at
../../../../libstdc++-v3/libsupc++/vterminate.cc:95
#6 0x7fec4f60f6bb in __cxxabiv1::__terminate (handler=)
at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:48
#7 0x7fec4f6090e3 in std::terminate () at
../../../../libstdc++-v3/libsupc++/eh_terminate.cc:58
#8 0x7fec4f60f8be in __cxxabiv1::__cxa_throw (obj=,
tinfo=0x7fec4f7328f8 , dest=0x7fec4f62bee6
)
at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:98
#9 0x7fec4f609d9f in std::__throw_system_error (__i=-1) at
../../../../../libstdc++-v3/src/c++11/system_error.cc:599
#10 0x566a4b0442a0 in
std::call_once >(std::once_flag
&, struct {...} &&) (__once=..., __f=...)
at
/opt/conda/envs/arrow/lib/gcc/x86_64-conda-linux-gnu/13.3.0/include/c++/mutex:908
#11 0x566a4b040cc9 in arrow::acero::RegisterTestNodes () at
/arrow/cpp/src/arrow/acero/test_nodes.cc:361
#12 0x566a4b0739e0 in arrow::acero::CheckFetch (options=...) at
/arrow/cpp/src/arrow/acero/fetch_node_test.cc:43
#13 0x566a4b074df2 in arrow::acero::FetchNode_Basic_Test::TestBody
(this=0x566a4daf23d0) at /arrow/cpp/src/arrow/acero/fetch_node_test.cc:75
#14 0x7fec4f7a185c in
testing::internal::HandleExceptionsInMethodIfSupported
(object=0x566a4daf23d0, method=&virtual testing::Test::TestBody(),
location=0x7fec4f7c5f53 "the test body") at
/build/cpp/_deps/googletest-src/googletest/src/gtest.cc:2711
#15 0x7fec4f781b47 in testing::Test::Run (this=0x566a4daf23d0) at
/build/cpp/_deps/googletest-src/googletest/src/gtest.cc:2728
#16 0x7fec4f7826a6 in testing::TestInfo::Run (this=0x566a4d9a3cb0) at
/build/cpp/_deps/googletest-src/googletest/src/gtest.cc:2874
#17 0x7fec4f78316f in testing::TestSuite::Run (this=0x566a4d9a4150) at
/build/cpp/_deps/googletest-src/googletest/src/gtest.cc:3052
#18 0x7fec4f7950b9 in testing::internal::UnitTestImpl::RunAllTests
(this=0x566a4d9a3dd0) at
/build/cpp/_deps/googletest-src/googletest/src/gtest.cc:6004
#19 0x7fec4f7a2ca8 in
testing::internal::HandleExceptionsInMethodIfSupported (object=0x566a4d9a3dd0,
method=(bool
(testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const))
0x7fec4f794c7a ,
location=0x7fec4f7c69f8 "auxiliary test code (environments or event
listeners)") at /build/cpp/_deps/googletest-src/googletest/src/gtest.cc:2711
#20 0x7fec4f792f18 in testing::UnitTest::Run (this=0x7fec4f7f0260
) at
/build/cpp/_deps/googletest-src/googletest/src/gtest.cc:5583
#21 0x566a4b072335 in RUN_ALL_TESTS () at
/build/cpp/_deps/googletest-src/googletest/include/gtest/gtest.h:2334
#22 0x566a4b0725c9 in main (argc=1, argv=0x7ffdf0112e08) at
/arrow/cpp/src/arrow/compute/test_env.cc:50
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
xhochy commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3045540297 I sadly have no idea, I have never seen a linker make a run-time impact. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
pitrou commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3045517236 @xhochy Do you have any idea what might be causing this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] [CI][C++] Crashes on AMD64 Conda C++ AVX2 [arrow]
xhochy commented on issue #47015: URL: https://github.com/apache/arrow/issues/47015#issuecomment-3045516716 > Then I noticed that with binutils 2.44, conda-forge also removed the faster ld.gold linker. This is not conda-forge specific, this was a subtle (i.e. not mentioned) change in `binutils` themselves. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
