Hi,
I don't think https://github.com/apache/arrow/issues/35321
is a blocker for 12.0.0 RC0 because our fix
https://github.com/apache/arrow/pull/35324 just skips these
tests only for pandas 2.0.1.
If it's a blocker of verification, can we skip these tests
by adding some pytest arguments in our verification script
something like the following?
----
diff --git a/ci/scripts/python_wheel_unix_test.sh
b/ci/scripts/python_wheel_unix_test.sh
index a6cc3bb7b..31c1a027a 100755
--- a/ci/scripts/python_wheel_unix_test.sh
+++ b/ci/scripts/python_wheel_unix_test.sh
@@ -90,5 +90,6 @@ if [ "${CHECK_UNITTESTS}" == "ON" ]; then
# Execute unittest, test dependencies must be installed
python -c 'import pyarrow; pyarrow.create_library_symlinks()'
- python -m pytest -r s --pyargs pyarrow
+ # REMOVE ME: Workaround for 12.0.0 with pandas 2.0.1.
+ python -m pytest -r s --pyargs pyarrow -k 'not
test_extension_to_pandas_storage_type'
fi
----
Thanks,
--
kou
In <cad1rbrqxx8549-raqlm7qcvmgq08e10pwm9sxosupuyhmay...@mail.gmail.com>
"Re: [VOTE] Release Apache Arrow 12.0.0 - RC0" on Thu, 27 Apr 2023 17:09:23
+0200,
Raúl Cumplido <[email protected]> wrote:
> Hi,
>
> The vote for the RC has been open for 5 days.
>
> I will wait until tomorrow, if no more +1 votes are casted I
> understand that the issue related to the pandas failures
> (https://github.com/apache/arrow/issues/35321) is causing verification
> to fail and we require a new RC with the above fix.
>
> Let me know if there are other blockers that should be included in that case.
>
> Kind regards,
> Raúl
>
> El jue, 27 abr 2023 a las 16:08, Sutou Kouhei (<[email protected]>)
> escribió:
>>
>> Hi,
>>
>> I tried this on a manjarolinux/base Docker image.
>>
>> I think that this is a problem of the Arch Linux's llvm
>> package. LLVMExports.cmake in the package doesn't provide
>> the LLVMX86CodeGen target:
>>
>> # grep add_library /usr/lib/cmake/llvm/LLVMExports.cmake
>> add_library(LLVMDemangle STATIC IMPORTED)
>> add_library(LLVMSupport STATIC IMPORTED)
>> add_library(LLVMTableGen STATIC IMPORTED)
>> add_library(LTO SHARED IMPORTED)
>> add_library(LLVMgold MODULE IMPORTED)
>> add_library(LLVM SHARED IMPORTED)
>> add_library(Remarks SHARED IMPORTED)
>>
>> FYI: LLVMExports.cmake on Debian GNU/Linux provides many
>> targets:
>>
>> $ grep add_library /usr/lib/llvm-15/lib/cmake/llvm/LLVMExports.cmake | head
>> add_library(LLVMDemangle STATIC IMPORTED)
>> add_library(LLVMSupport STATIC IMPORTED)
>> add_library(LLVMTableGen STATIC IMPORTED)
>> add_library(LLVMTableGenGlobalISel STATIC IMPORTED)
>> add_library(LLVMCore STATIC IMPORTED)
>> add_library(LLVMFuzzerCLI STATIC IMPORTED)
>> add_library(LLVMFuzzMutate STATIC IMPORTED)
>> add_library(LLVMFileCheck STATIC IMPORTED)
>> add_library(LLVMInterfaceStub STATIC IMPORTED)
>> add_library(LLVMIRReader STATIC IMPORTED)
>>
>> $ grep add_library /usr/lib/llvm-15/lib/cmake/llvm/LLVMExports.cmake | wc -l
>> 195
>>
>>
>> Thanks,
>> --
>> kou
>>
>> In <CANva0dgrkkaAb_dbnSLXuVxi=h22i0yjhbtde-y3-rflox8...@mail.gmail.com>
>> "Re: [VOTE] Release Apache Arrow 12.0.0 - RC0" on Tue, 25 Apr 2023
>> 23:50:21 +0200,
>> Jacob Wujciak <[email protected]> wrote:
>>
>> > I checked out a trace for the cmake issue and LLVM 15.07 is found
>> > correctly. The issue come from `llvm_map_components_to_libnames` which
>> > complains about X86 not being in the lsit of libraries. But we don't add
>> > that but rather it gets appended in the function?
>> >
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(228):
>> > get_property(LLVM_TARGETS_CONFIGURED GLOBAL PROPERTY
>> > LLVM_TARGETS_CONFIGURED )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(234): if(NOT LLVM_TARGETS_CONFIGURED
>> > )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(244): list(FIND
>> > LLVM_TARGETS_TO_BUILD X86 have_native_backend )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(245): list(FIND link_components
>> > engine engine_required )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(246): if(NOT engine_required EQUAL
>> > -1 )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(255): list(FIND link_components
>> > native native_required )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(256): if(NOT native_required EQUAL
>> > -1 )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(257): if(NOT have_native_backend
>> > EQUAL -1 )
>> > */usr/lib/cmake/llvm/LLVM-Config.cmake(258): list(APPEND link_components
>> > X86 )*
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(263):
>> > llvm_expand_pseudo_components(link_components
>> > core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(127): set(link_components
>> > core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(128): foreach(c
>> > core;mcjit;native;ipo;bitreader;target;linker;analysis;debuginfodwarf;X86 )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(130): list(FIND
>> > LLVM_TARGETS_TO_BUILD core idx )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(131): if(NOT idx LESS 0 )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(159): elseif(c STREQUAL
>> > nativecodegen )
>> > [snip]
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(212): list(APPEND
>> > expanded_components debuginfodwarf )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(130): list(FIND
>> > LLVM_TARGETS_TO_BUILD X86 idx )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(131): if(NOT idx LESS 0 )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(132): if(TARGET LLVMX86CodeGen )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(134): else()
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(135): if(TARGET LLVMX86 )
>> > /usr/lib/cmake/llvm/LLVM-Config.cmake(137): else()
>> > */usr/lib/cmake/llvm/LLVM-Config.cmake(138): message(FATAL_ERROR Target
>> > X86 is not in the set of libraries. )*
>> >
>> > On Tue, Apr 25, 2023 at 10:57 AM Raúl Cumplido <[email protected]>
>> > wrote:
>> >
>> >> I have created the following issue for the new wheels test failure
>> >> around pandas 2.0.1 : https://github.com/apache/arrow/issues/35321
>> >>
>> >> I don't think we should create a new RC for that issue but I'm happy
>> >> to know other people's thoughts around that.
>> >>
>> >> El lun, 24 abr 2023 a las 21:12, Raúl Cumplido
>> >> (<[email protected]>) escribió:
>> >> >
>> >> > El lun, 24 abr 2023 a las 18:53, Will Jones
>> >> > (<[email protected]>) escribió:
>> >> > >
>> >> > > I'm seeing failing Pandas tests in PyArrow when verifying with
>> >> > >
>> >> > > USE_CONDA=1 dev/release/verify-release-candidate.sh 12.0.0 0
>> >> > >
>> >> > >
>> >> pyarrow/tests/test_extension_type.py::test_extension_to_pandas_storage_type[registered_period_type0]
>> >> > > - NotImplementedError: extension<test.period<PeriodType>>
>> >> >
>> >> > This is also happening on our nightlies from today:
>> >> >
>> >> https://github.com/ursacomputing/crossbow/actions/runs/4786502455/jobs/8510514881
>> >> >
>> >> > There has been a new pandas release: 2.0.1 around 9 hours ago which
>> >> > seems to be the causing issue:
>> >> > https://pypi.org/project/pandas/#history
>> >> >
>> >> > >
>> >> > > No one else is getting that?
>> >> > >
>> >> > >
>> >> > > On Sun, Apr 23, 2023 at 9:21 AM Raúl Cumplido <[email protected]>
>> >> > > wrote:
>> >> > >
>> >> > > > +1 (non binding)
>> >> > > >
>> >> > > > I have tested both SOURCES and BINARIES successfully with:
>> >> > > > TEST_DEFAULT=0 TEST_SOURCE=1 dev/release/verify-release-candidate.sh
>> >> > > > 12.0.0 0
>> >> > > > TEST_DEFAULT=0 TEST_WHEELS=1 dev/release/verify-release-candidate.sh
>> >> > > > 12.0.0 0
>> >> > > > TEST_DEFAULT=0 TEST_BINARIES=1
>> >> dev/release/verify-release-candidate.sh
>> >> > > > 12.0.0 0
>> >> > > > with:
>> >> > > > * Python 3.10.6
>> >> > > > * gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
>> >> > > > * NVIDIA CUDA cuda_11.5.r11.5/compiler.30672275_0
>> >> > > > * openjdk version "17.0.6" 2023-01-17
>> >> > > > * ruby 3.0.2p107 (2021-07-07 revision 0db68f0233)
>> >> [x86_64-linux-gnu]
>> >> > > > * dotnet 7.0.203
>> >> > > > * Ubuntu 22.04 LTS
>> >> > > >
>> >> > > > El dom, 23 abr 2023 a las 12:59, Yibo Cai (<[email protected]>)
>> >> escribió:
>> >> > > > >
>> >> > > > > +1
>> >> > > > >
>> >> > > > > I ran the followings on Ubuntu-22.04, aarch64.
>> >> > > > >
>> >> > > > > TEST_DEFAULT=0 \
>> >> > > > > TEST_CPP=1 \
>> >> > > > > TEST_PYTHON=1 \
>> >> > > > > TEST_GO=1 \
>> >> > > > > dev/release/verify-release-candidate.sh 12.0.0 0
>> >> > > > >
>> >> > > > > TEST_DEFAULT=0 \
>> >> > > > > TEST_WHEELS=1 \
>> >> > > > > dev/release/verify-release-candidate.sh 12.0.0 0
>> >> > > > >
>> >> > > > >
>> >> > > > > On 4/23/23 14:40, Sutou Kouhei wrote:
>> >> > > > > > +1
>> >> > > > > >
>> >> > > > > > I ran the followings on Debian GNU/Linux sid:
>> >> > > > > >
>> >> > > > > > * TEST_DEFAULT=0 \
>> >> > > > > > TEST_SOURCE=1 \
>> >> > > > > > LANG=C \
>> >> > > > > > TZ=UTC \
>> >> > > > > > CUDAToolkit_ROOT=/usr \
>> >> > > > > > ARROW_CMAKE_OPTIONS="-DBoost_NO_BOOST_CMAKE=ON
>> >> > > > -Dxsimd_SOURCE=BUNDLED" \
>> >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 0
>> >> > > > > >
>> >> > > > > > * TEST_DEFAULT=0 \
>> >> > > > > > TEST_APT=1 \
>> >> > > > > > LANG=C \
>> >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 0
>> >> > > > > >
>> >> > > > > > * TEST_DEFAULT=0 \
>> >> > > > > > TEST_BINARY=1 \
>> >> > > > > > LANG=C \
>> >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 0
>> >> > > > > >
>> >> > > > > > * TEST_DEFAULT=0 \
>> >> > > > > > TEST_JARS=1 \
>> >> > > > > > LANG=C \
>> >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 0
>> >> > > > > >
>> >> > > > > > * TEST_DEFAULT=0 \
>> >> > > > > > TEST_PYTHON_VERSIONS=3.11 \
>> >> > > > > > TEST_WHEELS=1 \
>> >> > > > > > LANG=C \
>> >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 0
>> >> > > > > >
>> >> > > > > > * TEST_DEFAULT=0 \
>> >> > > > > > TEST_YUM=1 \
>> >> > > > > > LANG=C \
>> >> > > > > > dev/release/verify-release-candidate.sh 12.0.0 0
>> >> > > > > >
>> >> > > > > > with:
>> >> > > > > >
>> >> > > > > > * .NET SDK (6.0.406)
>> >> > > > > > * Python 3.11.2
>> >> > > > > > * gcc (Debian 12.2.0-14) 12.2.0
>> >> > > > > > * nvidia-cuda-dev 11.7.99~11.7.1-4
>> >> > > > > > * openjdk version "17.0.6" 2023-01-17
>> >> > > > > > * ruby 3.1.2p20 (2022-04-12 revision 4491bb740a)
>> >> [x86_64-linux-gnu]
>> >> > > > > >
>> >> > > > > >
>> >> > > > > > Thanks,
>> >> > > >
>> >>
