Okay, I confirmed that by running the following two commands, the test failures that Jacob found on Manjarolinux were solved (at least in the container I was using)
$ pacman -S libxcrypt-compat $ ln -s /usr/share/zoneinfo/America/New_York /etc/localtime For the second command it looks like the /etc/localtime symbolic link wasn't being set in the container and is leveraged by the Orc adapter tests. So setting the localtime (to any valid zone info) was sufficient to let the tests run and pass. Hope this helps! On Fri, Apr 28, 2023 at 11:49 AM Matthew Topol <m...@voltrondata.com> wrote: > Looks like this might be related: > https://unix.stackexchange.com/questions/691479/how-to-deal-with-missing-libcrypt-so-1-on-arch-linux > as manjaro also uses pacman and Arch Linux's packages. > > I'm re-running the verification right now after installing the recommended > package in that thread. I'll report back if it solves the issue. > > --Matt > > On Fri, Apr 28, 2023 at 11:29 AM Matthew Topol <m...@voltrondata.com> > wrote: > >> @Kou: I was able to reproduce the libcrypto failure that Jacob saw using >> https://hub.docker.com/r/manjarolinux/base though i did need to manually >> install git first since it doesn't come with it. >> >> $ pacman -Syu git >> $ git clone https://github.com/apache/arrow.git >> $ cd arrow >> $ TEST_DEFAULT=0 TEST_SOURCE=0 TEST_CPP=1 USE_CONDA=1 >> dev/release/verify-release-candidate.sh 12.0.0 0 >> >> That set of commands was sufficient to reproduce the error I believe (I >> did this on monday when I was poking around the failures but I definitely >> managed to see the same error pop up in a run). I'm running it again right >> now to confirm. >> >> --Matt >> >> On Thu, Apr 27, 2023 at 8:28 PM Sutou Kouhei <k...@clear-code.com> wrote: >> >>> Hi, >>> >>> Thanks for sharing the log. >>> >>> libcrypto.so isn't related on the segmentation fault. It's >>> just for relating to showing backtrace. >>> >>> > perl: error while loading shared libraries: libcrypt.so.1: >>> > cannot open shared object file: No such file or directory >>> >>> This is happen at >>> >>> https://github.com/apache/arrow/blob/main/cpp/build-support/run-test.sh#L42 >>> : >>> >>> TEST_NAME=$(echo $TEST_FILENAME | perl -pe 's/\..+?$//') # Remove path >>> and extension (if any). >>> >>> BTW, it seems that we should remove a Perl dependency from >>> https://github.com/apache/arrow/blob/main/cpp/build-support/run-test.sh >>> ... >>> >>> >>> I want to reproduce this problem on my environment. Could >>> you share your environment information? Did you use Manjaro >>> Linux this too? >>> >>> >>> Thanks, >>> -- >>> kou >>> >>> >>> In <canva0dgkodpfde7_b8xuvmtkh5kdmzvmtpbofo82hqj17gu...@mail.gmail.com> >>> "Re: [VOTE] Release Apache Arrow 12.0.0 - RC0" on Thu, 27 Apr 2023 >>> 23:54:58 +0200, >>> Jacob Wujciak <ja...@voltrondata.com.INVALID> wrote: >>> >>> > I have uploaded the log [1] for the run using conda with gandiva >>> active. It >>> > looks like there is an issue with libcrypt.so causing these tests to >>> > segfault. >>> > >>> > 1: https://gist.github.com/assignUser/cba0a13875de9d6a4f31000f585244f0 >>> > >>> > On Thu, Apr 27, 2023 at 11:32 PM Will Jones <will.jones...@gmail.com> >>> wrote: >>> > >>> >> Hi Raul, >>> >> >>> >> It might be worth creating a new RC that fixes more of the test >>> issues, >>> >> even if they shouldn't be blockers. I've run the release script a few >>> >> different times, and after 1.5 hours (is that a normal runtime for >>> >> verification?) I get various test failures. So far the errors are in >>> the >>> >> TEST_PYTHON, TEST_WHEELS, and TEST_JAVA. I'll have more time tomorrow >>> to >>> >> look at the failures I am seeing. >>> >> >>> >> On Thu, Apr 27, 2023 at 8:09 AM Raúl Cumplido <raulcumpl...@gmail.com >>> > >>> >> wrote: >>> >> >>> >> > Hi, >>> >> > >>> >> > The vote for the RC has been open for 5 days. >>> >> > >>> >> > I will wait until tomorrow, if no more +1 votes are casted I >>> >> > understand that the issue related to the pandas failures >>> >> > (https://github.com/apache/arrow/issues/35321) is causing >>> verification >>> >> > to fail and we require a new RC with the above fix. >>> >> > >>> >> > Let me know if there are other blockers that should be included in >>> that >>> >> > case. >>> >> > >>> >> > Kind regards, >>> >> > Raúl >>> >> > >>> >> > El jue, 27 abr 2023 a las 16:08, Sutou Kouhei (<k...@clear-code.com >>> >) >>> >> > escribió: >>> >> > > >>> >> > > Hi, >>> >> > > >>> >> > > I tried this on a manjarolinux/base Docker image. >>> >> > > >>> >> > > I think that this is a problem of the Arch Linux's llvm >>> >> > > package. LLVMExports.cmake in the package doesn't provide >>> >> > > the LLVMX86CodeGen target: >>> >> > > >>> >> > > # grep add_library /usr/lib/cmake/llvm/LLVMExports.cmake >>> >> > > add_library(LLVMDemangle STATIC IMPORTED) >>> >> > > add_library(LLVMSupport STATIC IMPORTED) >>> >> > > add_library(LLVMTableGen STATIC IMPORTED) >>> >> > > add_library(LTO SHARED IMPORTED) >>> >> > > add_library(LLVMgold MODULE IMPORTED) >>> >> > > add_library(LLVM SHARED IMPORTED) >>> >> > > add_library(Remarks SHARED IMPORTED) >>> >> > > >>> >> > > FYI: LLVMExports.cmake on Debian GNU/Linux provides many >>> >> > > targets: >>> >> > > >>> >> > > $ grep add_library >>> /usr/lib/llvm-15/lib/cmake/llvm/LLVMExports.cmake | >>> >> > head >>> >> > > add_library(LLVMDemangle STATIC IMPORTED) >>> >> > > add_library(LLVMSupport STATIC IMPORTED) >>> >> > > add_library(LLVMTableGen STATIC IMPORTED) >>> >> > > add_library(LLVMTableGenGlobalISel STATIC IMPORTED) >>> >> > > add_library(LLVMCore STATIC IMPORTED) >>> >> > > add_library(LLVMFuzzerCLI STATIC IMPORTED) >>> >> > > add_library(LLVMFuzzMutate STATIC IMPORTED) >>> >> > > add_library(LLVMFileCheck STATIC IMPORTED) >>> >> > > add_library(LLVMInterfaceStub STATIC IMPORTED) >>> >> > > add_library(LLVMIRReader STATIC IMPORTED) >>> >> > > >>> >> > > $ grep add_library >>> /usr/lib/llvm-15/lib/cmake/llvm/LLVMExports.cmake | >>> >> > wc -l >>> >> > > 195 >>> >> > > >>> >> > > >>> >> > > Thanks, >>> >> > > -- >>> >> > > kou >>> >> > > >>> >> > > In <CANva0dgrkkaAb_dbnSLXuVxi= >>> h22i0yjhbtde-y3-rflox8...@mail.gmail.com >>> >> > >>> >> > > "Re: [VOTE] Release Apache Arrow 12.0.0 - RC0" on Tue, 25 Apr >>> 2023 >>> >> > 23:50:21 +0200, >>> >> > > Jacob Wujciak <ja...@voltrondata.com.INVALID> wrote: >>> >> > > >>> >> > > > I checked out a trace for the cmake issue and LLVM 15.07 is >>> found >>> >> > > > correctly. The issue come from `llvm_map_components_to_libnames` >>> >> which >>> >> > > > complains about X86 not being in the lsit of libraries. But we >>> don't >>> >> > add >>> >> > > > that but rather it gets appended in the function? >>> >> > > > >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(228): >>> >> > > > get_property(LLVM_TARGETS_CONFIGURED GLOBAL PROPERTY >>> >> > > > LLVM_TARGETS_CONFIGURED ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(234): if(NOT >>> >> > LLVM_TARGETS_CONFIGURED >>> >> > > > ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(244): list(FIND >>> >> > > > LLVM_TARGETS_TO_BUILD X86 have_native_backend ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(245): list(FIND >>> >> link_components >>> >> > > > engine engine_required ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(246): if(NOT >>> engine_required >>> >> > EQUAL >>> >> > > > -1 ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(255): list(FIND >>> >> link_components >>> >> > > > native native_required ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(256): if(NOT >>> native_required >>> >> > EQUAL >>> >> > > > -1 ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(257): if(NOT >>> >> have_native_backend >>> >> > > > EQUAL -1 ) >>> >> > > > */usr/lib/cmake/llvm/LLVM-Config.cmake(258): list(APPEND >>> >> > link_components >>> >> > > > X86 )* >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(263): >>> >> > > > llvm_expand_pseudo_components(link_components >>> >> > > > >>> >> > >>> >> >>> core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(127): set(link_components >>> >> > > > >>> >> > >>> >> >>> core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(128): foreach(c >>> >> > > > >>> >> > >>> >> >>> core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(130): list(FIND >>> >> > > > LLVM_TARGETS_TO_BUILD core idx ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(131): if(NOT idx LESS 0 ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(159): elseif(c STREQUAL >>> >> > > > nativecodegen ) >>> >> > > > [snip] >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(212): list(APPEND >>> >> > > > expanded_components debuginfodwarf ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(130): list(FIND >>> >> > > > LLVM_TARGETS_TO_BUILD X86 idx ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(131): if(NOT idx LESS 0 ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(132): if(TARGET >>> >> LLVMX86CodeGen ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(134): else() >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(135): if(TARGET LLVMX86 ) >>> >> > > > /usr/lib/cmake/llvm/LLVM-Config.cmake(137): else() >>> >> > > > */usr/lib/cmake/llvm/LLVM-Config.cmake(138): >>> message(FATAL_ERROR >>> >> > Target >>> >> > > > X86 is not in the set of libraries. )* >>> >> > > > >>> >> > > > On Tue, Apr 25, 2023 at 10:57 AM Raúl Cumplido < >>> >> raulcumpl...@gmail.com >>> >> > > >>> >> > > > wrote: >>> >> > > > >>> >> > > >> I have created the following issue for the new wheels test >>> failure >>> >> > > >> around pandas 2.0.1 : >>> https://github.com/apache/arrow/issues/35321 >>> >> > > >> >>> >> > > >> I don't think we should create a new RC for that issue but I'm >>> happy >>> >> > > >> to know other people's thoughts around that. >>> >> > > >> >>> >> > > >> El lun, 24 abr 2023 a las 21:12, Raúl Cumplido >>> >> > > >> (<raulcumpl...@gmail.com>) escribió: >>> >> > > >> > >>> >> > > >> > El lun, 24 abr 2023 a las 18:53, Will Jones >>> >> > > >> > (<will.jones...@gmail.com>) escribió: >>> >> > > >> > > >>> >> > > >> > > I'm seeing failing Pandas tests in PyArrow when verifying >>> with >>> >> > > >> > > >>> >> > > >> > > USE_CONDA=1 dev/release/verify-release-candidate.sh 12.0.0 >>> 0 >>> >> > > >> > > >>> >> > > >> > > >>> >> > > >> >>> >> > >>> >> >>> pyarrow/tests/test_extension_type.py::test_extension_to_pandas_storage_type[registered_period_type0] >>> >> > > >> > > - NotImplementedError: extension<test.period<PeriodType>> >>> >> > > >> > >>> >> > > >> > This is also happening on our nightlies from today: >>> >> > > >> > >>> >> > > >> >>> >> > >>> >> >>> https://github.com/ursacomputing/crossbow/actions/runs/4786502455/jobs/8510514881 >>> >> > > >> > >>> >> > > >> > There has been a new pandas release: 2.0.1 around 9 hours ago >>> >> which >>> >> > > >> > seems to be the causing issue: >>> >> > > >> > https://pypi.org/project/pandas/#history >>> >> > > >> > >>> >> > > >> > > >>> >> > > >> > > No one else is getting that? >>> >> > > >> > > >>> >> > > >> > > >>> >> > > >> > > On Sun, Apr 23, 2023 at 9:21 AM Raúl Cumplido < >>> >> > raulcumpl...@gmail.com> >>> >> > > >> > > wrote: >>> >> > > >> > > >>> >> > > >> > > > +1 (non binding) >>> >> > > >> > > > >>> >> > > >> > > > I have tested both SOURCES and BINARIES successfully >>> with: >>> >> > > >> > > > TEST_DEFAULT=0 TEST_SOURCE=1 >>> >> > dev/release/verify-release-candidate.sh >>> >> > > >> > > > 12.0.0 0 >>> >> > > >> > > > TEST_DEFAULT=0 TEST_WHEELS=1 >>> >> > dev/release/verify-release-candidate.sh >>> >> > > >> > > > 12.0.0 0 >>> >> > > >> > > > TEST_DEFAULT=0 TEST_BINARIES=1 >>> >> > > >> dev/release/verify-release-candidate.sh >>> >> > > >> > > > 12.0.0 0 >>> >> > > >> > > > with: >>> >> > > >> > > > * Python 3.10.6 >>> >> > > >> > > > * gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0 >>> >> > > >> > > > * NVIDIA CUDA cuda_11.5.r11.5/compiler.30672275_0 >>> >> > > >> > > > * openjdk version "17.0.6" 2023-01-17 >>> >> > > >> > > > * ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) >>> >> > > >> [x86_64-linux-gnu] >>> >> > > >> > > > * dotnet 7.0.203 >>> >> > > >> > > > * Ubuntu 22.04 LTS >>> >> > > >> > > > >>> >> > > >> > > > El dom, 23 abr 2023 a las 12:59, Yibo Cai (< >>> yibo....@arm.com >>> >> >) >>> >> > > >> escribió: >>> >> > > >> > > > > >>> >> > > >> > > > > +1 >>> >> > > >> > > > > >>> >> > > >> > > > > I ran the followings on Ubuntu-22.04, aarch64. >>> >> > > >> > > > > >>> >> > > >> > > > > TEST_DEFAULT=0 \ >>> >> > > >> > > > > TEST_CPP=1 \ >>> >> > > >> > > > > TEST_PYTHON=1 \ >>> >> > > >> > > > > TEST_GO=1 \ >>> >> > > >> > > > > dev/release/verify-release-candidate.sh 12.0.0 0 >>> >> > > >> > > > > >>> >> > > >> > > > > TEST_DEFAULT=0 \ >>> >> > > >> > > > > TEST_WHEELS=1 \ >>> >> > > >> > > > > dev/release/verify-release-candidate.sh 12.0.0 0 >>> >> > > >> > > > > >>> >> > > >> > > > > >>> >> > > >> > > > > On 4/23/23 14:40, Sutou Kouhei wrote: >>> >> > > >> > > > > > +1 >>> >> > > >> > > > > > >>> >> > > >> > > > > > I ran the followings on Debian GNU/Linux sid: >>> >> > > >> > > > > > >>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>> >> > > >> > > > > > TEST_SOURCE=1 \ >>> >> > > >> > > > > > LANG=C \ >>> >> > > >> > > > > > TZ=UTC \ >>> >> > > >> > > > > > CUDAToolkit_ROOT=/usr \ >>> >> > > >> > > > > > ARROW_CMAKE_OPTIONS="-DBoost_NO_BOOST_CMAKE=ON >>> >> > > >> > > > -Dxsimd_SOURCE=BUNDLED" \ >>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>> 12.0.0 0 >>> >> > > >> > > > > > >>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>> >> > > >> > > > > > TEST_APT=1 \ >>> >> > > >> > > > > > LANG=C \ >>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>> 12.0.0 0 >>> >> > > >> > > > > > >>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>> >> > > >> > > > > > TEST_BINARY=1 \ >>> >> > > >> > > > > > LANG=C \ >>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>> 12.0.0 0 >>> >> > > >> > > > > > >>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>> >> > > >> > > > > > TEST_JARS=1 \ >>> >> > > >> > > > > > LANG=C \ >>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>> 12.0.0 0 >>> >> > > >> > > > > > >>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>> >> > > >> > > > > > TEST_PYTHON_VERSIONS=3.11 \ >>> >> > > >> > > > > > TEST_WHEELS=1 \ >>> >> > > >> > > > > > LANG=C \ >>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>> 12.0.0 0 >>> >> > > >> > > > > > >>> >> > > >> > > > > > * TEST_DEFAULT=0 \ >>> >> > > >> > > > > > TEST_YUM=1 \ >>> >> > > >> > > > > > LANG=C \ >>> >> > > >> > > > > > dev/release/verify-release-candidate.sh >>> 12.0.0 0 >>> >> > > >> > > > > > >>> >> > > >> > > > > > with: >>> >> > > >> > > > > > >>> >> > > >> > > > > > * .NET SDK (6.0.406) >>> >> > > >> > > > > > * Python 3.11.2 >>> >> > > >> > > > > > * gcc (Debian 12.2.0-14) 12.2.0 >>> >> > > >> > > > > > * nvidia-cuda-dev 11.7.99~11.7.1-4 >>> >> > > >> > > > > > * openjdk version "17.0.6" 2023-01-17 >>> >> > > >> > > > > > * ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) >>> >> > > >> [x86_64-linux-gnu] >>> >> > > >> > > > > > >>> >> > > >> > > > > > >>> >> > > >> > > > > > Thanks, >>> >> > > >> > > > >>> >> > > >> >>> >> > >>> >> >>> >>
