[jira] [Resolved] (ARROW-18086) [Ruby] Importing table containing float16 array throws error
[
https://issues.apache.org/jira/browse/ARROW-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18086.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 15204
https://github.com/apache/arrow/pull/15204
> [Ruby] Importing table containing float16 array throws error
>
>
> Key: ARROW-18086
> URL: https://issues.apache.org/jira/browse/ARROW-18086
> Project: Apache Arrow
> Issue Type: Bug
> Components: Ruby
>Affects Versions: 9.0.0
>Reporter: Atte Keinänen
>Assignee: Kouhei Sutou
>Priority: Minor
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In Red Arrow, loading table containing float16 array leads to this error when
> using IPC streaming format:
> {code:java}
> > Arrow::Table.load(Arrow::Buffer.new(resp.body), format: :arrow_streaming)
> cannot create instance of abstract (non-instantiatable) type 'GArrowDataType'
> from
> /usr/local/bundle/gems/gobject-introspection-4.0.3/lib/gobject-introspection/loader.rb:688:in
> `invoke' from
> /usr/local/bundle/gems/gobject-introspection-4.0.3/lib/gobject-introspection/loader.rb:559:in
> `get_field'{code}
> At least using float64 list array this does not happen.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-17144) [C++][Gandiva] Add sqrt function
[ https://issues.apache.org/jira/browse/ARROW-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou updated ARROW-17144: - Summary: [C++][Gandiva] Add sqrt function (was: Adding sqrt Function) > [C++][Gandiva] Add sqrt function > > > Key: ARROW-17144 > URL: https://issues.apache.org/jira/browse/ARROW-17144 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ - Gandiva >Reporter: Sahaj Gupta >Assignee: Sahaj Gupta >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Adding Sqrt Function. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-17144) [C++][Gandiva] Add sqrt function
[ https://issues.apache.org/jira/browse/ARROW-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-17144. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 13656 https://github.com/apache/arrow/pull/13656 > [C++][Gandiva] Add sqrt function > > > Key: ARROW-17144 > URL: https://issues.apache.org/jira/browse/ARROW-17144 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ - Gandiva >Reporter: Sahaj Gupta >Assignee: Sahaj Gupta >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Adding Sqrt Function. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ARROW-17144) Adding sqrt Function
[ https://issues.apache.org/jira/browse/ARROW-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou updated ARROW-17144: - Component/s: C++ - Gandiva > Adding sqrt Function > > > Key: ARROW-17144 > URL: https://issues.apache.org/jira/browse/ARROW-17144 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ - Gandiva >Reporter: Sahaj Gupta >Assignee: Sahaj Gupta >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Adding Sqrt Function. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (ARROW-17144) Adding sqrt Function
[ https://issues.apache.org/jira/browse/ARROW-17144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou reassigned ARROW-17144: Assignee: Sahaj Gupta > Adding sqrt Function > > > Key: ARROW-17144 > URL: https://issues.apache.org/jira/browse/ARROW-17144 > Project: Apache Arrow > Issue Type: New Feature >Reporter: Sahaj Gupta >Assignee: Sahaj Gupta >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Adding Sqrt Function. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-15206) [Ruby] Allow to pass schema when loading table from file
[
https://issues.apache.org/jira/browse/ARROW-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-15206.
--
Fix Version/s: 11.0.0
Resolution: Fixed
https://github.com/apache/arrow/pull/15148
> [Ruby] Allow to pass schema when loading table from file
>
>
> Key: ARROW-15206
> URL: https://issues.apache.org/jira/browse/ARROW-15206
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Ruby
>Reporter: Kanstantsin Ilchanka
>Assignee: Kouhei Sutou
>Priority: Minor
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> There is ability to do this in C++, but not in Ruby
> {code:java}
> schema = Arrow::Schema.new(a: :int64, b: :double)
> Arrow::Table.load(URI('file:///tmp/example.csv'), format: :csv, schema:
> schema){code}
> This should also work when loading multiple files from folder
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-17538) [C++] Importing an ArrowArrayStream can't handle errors from get_schema
[ https://issues.apache.org/jira/browse/ARROW-17538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-17538. -- Resolution: Fixed Issue resolved by pull request 15037 https://github.com/apache/arrow/pull/15037 > [C++] Importing an ArrowArrayStream can't handle errors from get_schema > --- > > Key: ARROW-17538 > URL: https://issues.apache.org/jira/browse/ARROW-17538 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Affects Versions: 9.0.0 >Reporter: David Li >Assignee: David Li >Priority: Major > Labels: good-first-issue, pull-request-available > Fix For: 11.0.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > As indicated in the code: > https://github.com/apache/arrow/blob/cd3c6ead97d584366aafd2f14d99a1cb8ace9ca2/cpp/src/arrow/c/bridge.cc#L1823 > > This probably needs a static initializer so we can catch things. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18231) [C++] Cannot override optimization level using CXXFLAGS
[
https://issues.apache.org/jira/browse/ARROW-18231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18231.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 15022
https://github.com/apache/arrow/pull/15022
> [C++] Cannot override optimization level using CXXFLAGS
> ---
>
> Key: ARROW-18231
> URL: https://issues.apache.org/jira/browse/ARROW-18231
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Antoine Pitrou
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> In release mode, Arrow C++ unconditionally adds {{-O2}} _at the end_ of the
> compiler flags.
> So, if you do something like:
> {code:bash}
> export CXXFLAGS=-O0
> cmake ...
> {code}
> the final compilation flags will look like {{-O0 -O2}}, meaning the
> user-provided optimization level is ignored.
> One can instead use the {{ARROW_CXXFLAGS}} CMake variable, but it only
> overrides the flags for Arrow itself, not the bundled dependencies.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-17838) [Python] Unify CMakeLists.txt at python/CMakeLists.txt and python/src/CMakeLists.txt
[ https://issues.apache.org/jira/browse/ARROW-17838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-17838. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14925 https://github.com/apache/arrow/pull/14925 > [Python] Unify CMakeLists.txt at python/CMakeLists.txt and > python/src/CMakeLists.txt > > > Key: ARROW-17838 > URL: https://issues.apache.org/jira/browse/ARROW-17838 > Project: Apache Arrow > Issue Type: Improvement > Components: Python >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 5h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (ARROW-15997) [CI] Nightly turbodbc build is failing (C++ compilation error)
[
https://issues.apache.org/jira/browse/ARROW-15997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou closed ARROW-15997.
Resolution: Won't Fix
We've removed the job: https://github.com/apache/arrow/pull/15036
> [CI] Nightly turbodbc build is failing (C++ compilation error)
> --
>
> Key: ARROW-15997
> URL: https://issues.apache.org/jira/browse/ARROW-15997
> Project: Apache Arrow
> Issue Type: Bug
> Components: Continuous Integration
>Reporter: Joris Van den Bossche
>Priority: Major
>
> See eg
> https://github.com/ursacomputing/crossbow/runs/5637809188?check_suite_focus=true
> The error seems related to boost (and not Arrow), and happens in the C++ code
> of turbodbc. But it is strange that it happens in both the latest and master
> turbodbc build (so it's not caused by a change on turbodbc's side). And I
> also didn't see a change in the boost version compared to the last successful
> build.
> cc [~uwe]
> {code}
> [102/156] Building CXX object
> cpp/turbodbc/Test/CMakeFiles/turbodbc_test.dir/tests/field_translator_test.cpp.o
> FAILED:
> cpp/turbodbc/Test/CMakeFiles/turbodbc_test.dir/tests/field_translator_test.cpp.o
>
> /opt/conda/envs/arrow/bin/x86_64-conda-linux-gnu-c++
> -I/turbodbc/cpp/turbodbc/Library -I/turbodbc/cpp/turbodbc/../cpp_odbc/Library
> -I/turbodbc/cpp/turbodbc/Test -fvisibility-inlines-hidden -std=c++17
> -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC
> -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem
> /opt/conda/envs/arrow/include -Wall -Wextra -g -O0 -pedantic -std=c++11 -MD
> -MT
> cpp/turbodbc/Test/CMakeFiles/turbodbc_test.dir/tests/field_translator_test.cpp.o
> -MF
> cpp/turbodbc/Test/CMakeFiles/turbodbc_test.dir/tests/field_translator_test.cpp.o.d
> -o
> cpp/turbodbc/Test/CMakeFiles/turbodbc_test.dir/tests/field_translator_test.cpp.o
> -c /turbodbc/cpp/turbodbc/Test/tests/field_translator_test.cpp
> In file included from
> /opt/conda/envs/arrow/include/boost/type_index/stl_type_index.hpp:32,
> from /opt/conda/envs/arrow/include/boost/type_index.hpp:29,
> from
> /opt/conda/envs/arrow/include/boost/variant/variant.hpp:21,
> from /turbodbc/cpp/turbodbc/Library/turbodbc/field.h:3,
> from
> /turbodbc/cpp/turbodbc/Library/turbodbc/field_translator.h:3,
> from
> /turbodbc/cpp/turbodbc/Test/tests/field_translator_test.cpp:1:
> /opt/conda/envs/arrow/include/boost/optional/optional.hpp: In instantiation
> of 'std::basic_ostream<_CharT, _Traits>&
> boost::operator<<(std::basic_ostream<_CharT, _Traits>&, const
> boost::optional_detail::optional_tag&) [with CharType = char; CharTrait =
> std::char_traits]':
> /opt/conda/envs/arrow/include/gtest/gtest-printers.h:215:9: required from
> 'static void
> testing::internal::internal_stream_operator_without_lexical_name_lookup::StreamPrinter::PrintValue(const
> T&, std::ostream*) [with T = boost::optional std::__cxx11::basic_string, std::allocator
> >, bool, double, boost::gregorian::date, boost::posix_time::ptime> >;
> = void; =
> std::basic_ostream&; std::ostream = std::basic_ostream]'
> /opt/conda/envs/arrow/include/gtest/gtest-printers.h:312:22: required from
> 'void testing::internal::PrintWithFallback(const T&, std::ostream*) [with T =
> boost::optional std::char_traits, std::allocator >, bool, double,
> boost::gregorian::date, boost::posix_time::ptime> >; std::ostream =
> std::basic_ostream]'
> /opt/conda/envs/arrow/include/gtest/gtest-printers.h:441:30: required from
> 'void testing::internal::PrintTo(const T&, std::ostream*) [with T =
> boost::optional std::char_traits, std::allocator >, bool, double,
> boost::gregorian::date, boost::posix_time::ptime> >; std::ostream =
> std::basic_ostream]'
> /opt/conda/envs/arrow/include/gtest/gtest-printers.h:691:12: required from
> 'static void testing::internal::UniversalPrinter::Print(const T&,
> std::ostream*) [with T = boost::optional std::__cxx11::basic_string, std::allocator
> >, bool, double, boost::gregorian::date, boost::posix_time::ptime> >;
> std::ostream = std::basic_ostream]'
> /opt/conda/envs/arrow/include/gtest/gtest-printers.h:980:30: required from
> 'void testing::internal::UniversalPrint(const T&, std::ostream*) [with T =
> boost::optional std::char_traits, std::allocator >, bool, double,
> boost::gregorian::date, boost::posix_time::ptime> >; std::ostream =
> std::basic_ostream]'
> /opt/conda/envs/arrow/include/gtest/gtest-printers.h:865:19: [ skipping 2
> instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
> /opt/conda/envs/arrow/include/gtest/gtest-printers.h:334:36: required from
> 'static std::string testing::internal::FormatForComparison
[jira] [Resolved] (ARROW-17692) [R] Arrow Package Installation: undefined symbol error
[
https://issues.apache.org/jira/browse/ARROW-17692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-17692.
--
Resolution: Fixed
Issue resolved by pull request 14235
https://github.com/apache/arrow/pull/14235
> [R] Arrow Package Installation: undefined symbol error
> ---
>
> Key: ARROW-17692
> URL: https://issues.apache.org/jira/browse/ARROW-17692
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Reporter: Wayne Tu
>Assignee: Nicola Crane
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 10h 50m
> Remaining Estimate: 0h
>
> Hi,
> I encountered "undefined symbol: _ZTIN3Aws4Auth22AWSCredentialsProviderE
> {noformat}
> Error: loading failed
> Execution halted
> ERROR: loading failed" errors when trying to install arrow under R 4.1.3 with
> devtoolset-8 (gcc version 8.3.1).
> > Sys.getenv("LD_LIBRARY_PATH")
> [1]
> "/usr/local/lib64:/usr/local/lib64/cmake:/lib64:/opt/rh/devtoolset-8/root/usr/lib64:/opt/rh/devtoolset-8/root/usr/lib/gcc/x86_64-redhat-linux/8:/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8:/opt/R/4.1.3/lib/R/lib:/usr/local/lib:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.322.b06-1.el7_9.x86_64/jre/lib/amd64/server"
> > Sys.getenv("PATH")
> [1]
> "/apps/Python/3.9.12/bin:/usr/local/cmake-3.21.4-linux-x86_64/bin:/opt/rh/devtoolset-8/root/usr/bin:/apps/bin:/usr/local/bin:/bin:/usr/bin"
> > Sys.setenv("NOT_CRAN"=TRUE)
> > Sys.setenv("LIBARROW_BINARY" = FALSE)
> > Sys.setenv("ARROW_R_DEV" = TRUE)
> > Sys.setenv("ARROW_USE_PKG_CONFIG" = FALSE)
> > Sys.setenv(ARROW_S3 = "ON")
> > Sys.setenv(CMAKE = "/apps/cmake-3.21.4-linux-x86_64/bin/cmake")
> > sessionInfo()
> R version 4.1.3 (2022-03-10)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Red Hat Enterprise Linux Server 7.9 (Maipo)
> Matrix products: default
> BLAS/LAPACK: /usr/lib64/libopenblasp-r0.3.3.so
> locale:
> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
> [9] LC_ADDRESS=C LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
> loaded via a namespace (and not attached):
> [1] compiler_4.1.3
> > arrow::arrow_available()
> Error in loadNamespace(x) : there is no package called ‘arrow’
> > system("gcc -v")
> Using built-in specs.
> COLLECT_GCC=gcc
> COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-8/root/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper
> Target: x86_64-redhat-linux
> Configured with: ../configure --enable-bootstrap
> --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-8/root/usr
> --mandir=/opt/rh/devtoolset-8/root/usr/share/man
> --infodir=/opt/rh/devtoolset-8/root/usr/share/info
> --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared
> --enable-threads=posix --enable-checking=release --enable-multilib
> --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions
> --enable-gnu-unique-object --enable-linker-build-id
> --with-gcc-major-version-only --with-linker-hash-style=gnu
> --with-default-libstdcxx-abi=gcc4-compatible --enable-plugin
> --enable-initfini-array
> --with-isl=/builddir/build/BUILD/gcc-8.3.1-20190311/obj-x86_64-redhat-linux/isl-install
> --disable-libmpx --enable-gnu-indirect-function --with-tune=generic
> --with-arch_32=x86-64 --build=x86_64-redhat-linux
> Thread model: posix
> gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC)
>
> > install.packages(mpkg, repos=NULL, type="source")
> ..
> ..
> ** building package indices
> ** installing vignettes
> ** testing if installed package can be loaded from temporary location
> Error: package or namespace load failed for ‘arrow’ in dyn.load(file, DLLpath
> = DLLpath, ...):
> unable to load shared object
> '/home/user1/R/x86_64-pc-linux-gnu/4.1.3/00LOCK-arrow/00new/arrow/libs/arrow.so':
>
> /home/user1/R/x86_64-pc-linux-gnu/4.1.3/00LOCK-arrow/00new/arrow/libs/arrow.so:
> undefined symbol: _ZTIN3Aws4Auth22AWSCredentialsProviderE
> Error: loading failed
> Execution halted
> ERROR: loading failed
> * removing ‘/home/user1/R/x86_64-pc-linux-gnu/4.1.3/arrow’
> Warning message:
> In install.packages(mpkg, repos = NULL, type = "source") :
> installation of package
> ‘/apps/tmp/RtmpEqJN3J/downloaded_packages/arrow_8.0.0.tar.gz’ had non-zero
> exit status
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18339) [CI][Python] Add DYLD_LIBRARY_PATH to avoid requiring PYARROW_BUNDLE_ARROW_CPP on macOS job
[ https://issues.apache.org/jira/browse/ARROW-18339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18339. -- Resolution: Fixed Issue resolved by pull request 14643 https://github.com/apache/arrow/pull/14643 > [CI][Python] Add DYLD_LIBRARY_PATH to avoid requiring > PYARROW_BUNDLE_ARROW_CPP on macOS job > --- > > Key: ARROW-18339 > URL: https://issues.apache.org/jira/browse/ARROW-18339 > Project: Apache Arrow > Issue Type: Bug > Components: Continuous Integration, Python >Reporter: Raúl Cumplido >Assignee: Raúl Cumplido >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > As discussed on this PR [https://github.com/apache/arrow/pull/14643] opening > Jira ticket -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18426) Update committers and PMC members on website
[ https://issues.apache.org/jira/browse/ARROW-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18426. -- Fix Version/s: 11.0.0 Resolution: Fixed https://github.com/apache/arrow-site/pull/282 > Update committers and PMC members on website > > > Key: ARROW-18426 > URL: https://issues.apache.org/jira/browse/ARROW-18426 > Project: Apache Arrow > Issue Type: Improvement > Components: Website >Reporter: Benson Muite >Assignee: Benson Muite >Priority: Trivial > Fix For: 11.0.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Update committers and PMC members -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (ARROW-15206) [Ruby] Allow to pass schema when loading table from file
[
https://issues.apache.org/jira/browse/ARROW-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou reassigned ARROW-15206:
Assignee: Kouhei Sutou
> [Ruby] Allow to pass schema when loading table from file
>
>
> Key: ARROW-15206
> URL: https://issues.apache.org/jira/browse/ARROW-15206
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Ruby
>Reporter: Kanstantsin Ilchanka
>Assignee: Kouhei Sutou
>Priority: Minor
>
> There is ability to do this in C++, but not in Ruby
> {code:java}
> schema = Arrow::Schema.new(a: :int64, b: :double)
> Arrow::Table.load(URI('file:///tmp/example.csv'), format: :csv, schema:
> schema){code}
> This should also work when loading multiple files from folder
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (ARROW-14604) [Ruby] Add API to read/write by partitions
[
https://issues.apache.org/jira/browse/ARROW-14604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou reassigned ARROW-14604:
Assignee: Kouhei Sutou
> [Ruby] Add API to read/write by partitions
> --
>
> Key: ARROW-14604
> URL: https://issues.apache.org/jira/browse/ARROW-14604
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Ruby
>Reporter: Kanstantsin Ilchanka
>Assignee: Kouhei Sutou
>Priority: Minor
>
> We should add convenient API for {{Arrow::Table#save}}/{{Arrow::Table.load to
> allow read or write multiple files at once}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17404) [Java] Consolidate JNI compilation #2
[ https://issues.apache.org/jira/browse/ARROW-17404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17644519#comment-17644519 ] Kouhei Sutou commented on ARROW-17404: -- [~dsusanibara] Can we close this? Is there any more task to be resolved? > [Java] Consolidate JNI compilation #2 > - > > Key: ARROW-17404 > URL: https://issues.apache.org/jira/browse/ARROW-17404 > Project: Apache Arrow > Issue Type: Bug > Components: Documentation, Java >Reporter: David Dali Susanibar Arce >Priority: Major > > *Umbrella ticket for consolidating Java JNI compilation initiative #2* > Initial part of consolidate JNI Java initiative was: Consolidate ORC/Dataset > code and Separate JNI CMakeLists.txt compilation. > This 2nd part consist on: > 1.- Make the Java library able to compile with a single mvn command > 2.- Make Java library able to compile from an installed libarrow > 3.- Migrate remaining C++ CMakeLists.txt specific to Java into the Java > project: ORC / Dataset / Gandiva > 4.- Add windows build script that produces DLLs > 5.- Incorporate Windows DLLs into the maven packages > 6.- Migrate ORC JNI to use C-Data-Interface -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-17374.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 14818
https://github.com/apache/arrow/pull/14818
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Assignee: Kouhei Sutou
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Attachments: build-images.out, environment.yml
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou reassigned ARROW-17374:
Assignee: Kouhei Sutou
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Assignee: Kouhei Sutou
>Priority: Blocker
> Labels: pull-request-available
> Attachments: build-images.out, environment.yml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642281#comment-17642281
]
Kouhei Sutou commented on ARROW-17374:
--
Thanks!
I could reproduce this problem.
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out, environment.yml
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642209#comment-17642209
]
Kouhei Sutou commented on ARROW-17374:
--
I tried with {{centos:7}} image but I got the following error:
{noformat}
(testme) [root@f186ef90db02 /]# R -e 'install.packages("decor",
repos="http://cran.us.r-project.org;)'
R version 4.1.3 (2022-03-10) -- "One Push-Up"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-conda-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
During startup - Warning messages:
1: package 'utils' in options("defaultPackages") was not found
2: package 'stats' in options("defaultPackages") was not found
> install.packages("decor", repos="http://cran.us.r-project.org;)
Error in install.packages("decor", repos = "http://cran.us.r-project.org;) :
could not find function "install.packages"
Execution halted
{noformat}
What is missing?
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out, environment.yml
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-17374) [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
[
https://issues.apache.org/jira/browse/ARROW-17374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642170#comment-17642170
]
Kouhei Sutou commented on ARROW-17374:
--
Sorry. I forgot to try this.
Could you show your CentOS version? 7?
> [R] R Arrow install fails with SNAPPY_LIB-NOTFOUND
> --
>
> Key: ARROW-17374
> URL: https://issues.apache.org/jira/browse/ARROW-17374
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Affects Versions: 8.0.0, 8.0.1, 9.0.0
> Environment: Amazon Linux 2 (RHEL) - 5.10.102-99.473.amzn2.x86_64
>Reporter: Shane Brennan
>Priority: Blocker
> Attachments: build-images.out, environment.yml
>
>
> I've been trying to install Arrow on an R notebook within AWS SageMaker.
> SageMaker provides Jupyter-like notebooks, with each instance running Amazon
> Linux 2 as its OS, itself based on RHEL.
> Trying to install a few ways, e.g., using the standard binaries, using the
> nightly builds, setting ARROW_WITH_SNAPPY to ON and LIBARROW_MINIMAL all
> still result in the following error.
> {noformat}
> x86_64-conda-linux-gnu-c++ -std=gnu++11 -shared
> -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -Wl,-O2 -Wl,--sort-common
> -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags
> -Wl,--gc-sections -Wl,--allow-shlib-undefined
> -Wl,-rpath,/home/ec2-user/anaconda3/envs/R/lib
> -Wl,-rpath-link,/home/ec2-user/anaconda3/envs/R/lib
> -L/home/ec2-user/anaconda3/envs/R/lib -o arrow.so RTasks.o altrep.o array.o
> array_to_vector.o arraydata.o arrowExports.o bridge.o buffer.o chunkedarray.o
> compression.o compute-exec.o compute.o config.o csv.o dataset.o datatype.o
> expression.o extension-impl.o feather.o field.o filesystem.o imports.o io.o
> json.o memorypool.o message.o parquet.o r_to_arrow.o recordbatch.o
> recordbatchreader.o recordbatchwriter.o safe-call-into-r-impl.o scalar.o
> schema.o symbols.o table.o threadpool.o type_infer.o
> -L/tmp/Rtmpuh87oc/R.INSTALL67114493a3de/arrow/libarrow/arrow-9.0.0.20220809/lib
> -larrow_dataset -lparquet -larrow -larrow_bundled_dependencies -lz
> SNAPPY_LIB-NOTFOUND /home/ec2-user/anaconda3/envs/R/lib/libbz2.so -pthread
> -larrow -larrow_bundled_dependencies -larrow_dataset -lparquet -lssl -lcrypto
> -lcurl -lssl -lcrypto -lcurl -L/home/ec2-user/anaconda3/envs/R/lib/R/lib -lR
> x86_64-conda-linux-gnu-c++: error: SNAPPY_LIB-NOTFOUND: No such file or
> directory
> make: *** [/home/ec2-user/anaconda3/envs/R/lib/R/share/make/shlib.mk:10:
> arrow.so] Error 1{noformat}
> Snappy is installed on the systems, and both shared object (.so) and cmake
> files are there, where I've tried setting the system env variables Snappy_DIR
> and Snappy_LIB to point at them, but to no avail.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (ARROW-14681) [C++] vcpkg arrow port ArrowDataset component usage
[
https://issues.apache.org/jira/browse/ARROW-14681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou closed ARROW-14681.
Resolution: Duplicate
We can use {{find_package(ArrowDataset)}} with Apache Arrow 10.0.0.
If there is a problem with it, please open a new issue at
https://github.com/apache/arrow/issues/new/choose .
> [C++] vcpkg arrow port ArrowDataset component usage
> ---
>
> Key: ARROW-14681
> URL: https://issues.apache.org/jira/browse/ARROW-14681
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Yue Ni
>Priority: Major
>
> vcpkg arrow port
> ([https://github.com/microsoft/vcpkg/tree/master/ports/arrow)] installs not
> only arrow but also other arrow components such as ArrowDataset/ArrowFlight.
> After installation, I can use `find_package(Arrow)` in CMake/vcpkg to find
> the arrow package and use it. But I wonder how I can use other components
> such as ArrowDataset. I tried using it in CMake like below:
> 1) `find_package(ArrowDataset REQUIRED)`
> CMake will report error like below:
> {quote}– _find_package pkg_name=ArrowDataset args=REQUIRED
> CMake Error at /Users/ss/dev/tools/vcpkg/scripts/buildsystems/vcpkg.cmake:789
> (_find_package):
> By not providing "FindArrowDataset.cmake" in CMAKE_MODULE_PATH this project
> has asked CMake to find a package configuration file provided by
> "ArrowDataset", but CMake did not find one.
> Could not find a package configuration file provided by "ArrowDataset" with
> any of the following names:
> ArrowDatasetConfig.cmake
> arrowdataset-config.cmake
> Add the installation prefix of "ArrowDataset" to CMAKE_PREFIX_PATH or set
> "ArrowDataset_DIR" to a directory containing one of the above files. If
> "ArrowDataset" provides a separate development package or SDK, be sure it
> has been installed.
> {quote}
>
> 2) `find_package(Arrow CONFIGS ArrowDatasetConfig.cmake REQUIRED)`
> The ArrowDatasetConfig.cmake can be found, however, it tries to find Parquet
> internally and still failed with error like below:
> {quote}– _find_package pkg_name=Arrow
> args=CONFIGS;ArrowDatasetConfig.cmake;REQUIRED
> – _find_package pkg_name=Arrow args=REQUIRED
> – _find_package pkg_name=Threads args=REQUIRED
> – _find_package pkg_name=Snappy args=REQUIRED
> – _find_package pkg_name=PkgConfig args=QUIET
> – _find_package pkg_name=Threads args=REQUIRED
> – _find_package pkg_name=Thrift args=REQUIRED
> – Found thrift: /Users/ss/dev/tools/vcpkg/installed/x64-osx
> – _find_package pkg_name=Threads args=QUIET
> – _find_package pkg_name=lz4 args=REQUIRED
> – _find_package pkg_name=zstd args=REQUIRED
> – _find_package pkg_name=re2Alt args=REQUIRED
> – _find_package pkg_name=re2 args=
> – _find_package pkg_name=BZip2 args=REQUIRED
> – _find_package pkg_name=unofficial-utf8proc args=REQUIRED
> – _find_package pkg_name=c-ares args=
> – _find_package pkg_name=c-ares args=CONFIG
> – _find_package pkg_name=absl args=CONFIG
> – _find_package pkg_name=Threads args=
> – _find_package pkg_name=upb args=CONFIG;REQUIRED
> – _find_package pkg_name=Parquet args=REQUIRED
> CMake Error at /Users/ss/dev/tools/vcpkg/scripts/buildsystems/vcpkg.cmake:789
> (_find_package):
> By not providing "FindParquet.cmake" in CMAKE_MODULE_PATH this project has
> asked CMake to find a package configuration file provided by "Parquet", but
> CMake did not find one.
> Could not find a package configuration file provided by "Parquet" with any
> of the following names:
> ParquetConfig.cmake
> parquet-config.cmake
> Add the installation prefix of "Parquet" to CMAKE_PREFIX_PATH or set
> "Parquet_DIR" to a directory containing one of the above files. If
> "Parquet" provides a separate development package or SDK, be sure it has
> been installed.
> Call Stack (most recent call first):
>
> /usr/local/Cellar/cmake/3.21.3_1/share/cmake/Modules/CMakeFindDependencyMacro.cmake:47
> (find_package)
>
> /Users/ss/dev/tools/vcpkg/installed/x64-osx/share/arrow/ArrowDatasetConfig.cmake:55
> (find_dependency)
> /Users/ss/dev/tools/vcpkg/scripts/buildsystems/vcpkg.cmake:789
> (_find_package)
> {quote}
> Is there any instruction describing how this could be done? Thanks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-18419) [C++] Update vendored fast_float
Kouhei Sutou created ARROW-18419: Summary: [C++] Update vendored fast_float Key: ARROW-18419 URL: https://issues.apache.org/jira/browse/ARROW-18419 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Kouhei Sutou Assignee: Kouhei Sutou For https://github.com/fastfloat/fast_float/pull/147 . -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ARROW-18039) [C++][CI] Reduce MinGW build times
[
https://issues.apache.org/jira/browse/ARROW-18039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640998#comment-17640998
]
Kouhei Sutou commented on ARROW-18039:
--
{noformat}
1: 124.79s:
google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep-stamp/google_cloud_cpp_ep-build,
google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_storage.a,
google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_rest_internal.a,
google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_common.a,
C:/Users/kou/work/arrow/build-msys/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep-stamp/google_cloud_cpp_ep-build,
C:/Users/kou/work/arrow/build-msys/google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_storage.a,
C:/Users/kou/work/arrow/build-msys/google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_rest_internal.a,
C:/Users/kou/work/arrow/build-msys/google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_common.a
{noformat}
We can reduce this by creating a MSYS2 package for google-cloud-cpp.
> [C++][CI] Reduce MinGW build times
> --
>
> Key: ARROW-18039
> URL: https://issues.apache.org/jira/browse/ARROW-18039
> Project: Apache Arrow
> Issue Type: Wish
> Components: C++, Continuous Integration
>Reporter: Antoine Pitrou
>Assignee: Kouhei Sutou
>Priority: Major
>
> The MinGW C++ builds on CI currently build in release mode. This is probably
> because debug builds on Windows are complicated (you must get all the
> dependencies also compiled in debug mode, AFAIU).
> However, we could probably disable optimizations, so as to reduce compilation
> times.
> The compilation flags are currently as follows:
> {code}
> -- CMAKE_C_FLAGS: -O2 -DNDEBUG -ftree-vectorize -Wa,-mbig-obj -Wall
> -Wno-conversion -Wno-sign-conversion -Wunused-result
> -fno-semantic-interposition -mxsave -msse4.2
> -- CMAKE_CXX_FLAGS: -Wno-noexcept-type -fdiagnostics-color=always -O2
> -DNDEBUG -ftree-vectorize -Wa,-mbig-obj -Wall -Wno-conversion
> -Wno-sign-conversion -Wunused-result -fno-semantic-interposition -mxsave
> -msse4.2
> {code}
> Perhaps we can pass {{-O0}}?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-18039) [C++][CI] Reduce MinGW build times
[
https://issues.apache.org/jira/browse/ARROW-18039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640997#comment-17640997
]
Kouhei Sutou commented on ARROW-18039:
--
{noformat}
2: 103.922000s:
src/parquet/CMakeFiles/parquet-arrow-test.dir/arrow/arrow_reader_writer_test.cc.obj
{noformat}
Building without optimization (removing {{-O2}}, {{-DNDEBUG}} and so on from
command line, no {{-OX}} flag) is faster: 103s -> 81s
Building with {{-O0}} is faster: 103s -> 84s
Building with {{-O0 -ggdb}} is slower: 103s -> 123s
> [C++][CI] Reduce MinGW build times
> --
>
> Key: ARROW-18039
> URL: https://issues.apache.org/jira/browse/ARROW-18039
> Project: Apache Arrow
> Issue Type: Wish
> Components: C++, Continuous Integration
>Reporter: Antoine Pitrou
>Assignee: Kouhei Sutou
>Priority: Major
>
> The MinGW C++ builds on CI currently build in release mode. This is probably
> because debug builds on Windows are complicated (you must get all the
> dependencies also compiled in debug mode, AFAIU).
> However, we could probably disable optimizations, so as to reduce compilation
> times.
> The compilation flags are currently as follows:
> {code}
> -- CMAKE_C_FLAGS: -O2 -DNDEBUG -ftree-vectorize -Wa,-mbig-obj -Wall
> -Wno-conversion -Wno-sign-conversion -Wunused-result
> -fno-semantic-interposition -mxsave -msse4.2
> -- CMAKE_CXX_FLAGS: -Wno-noexcept-type -fdiagnostics-color=always -O2
> -DNDEBUG -ftree-vectorize -Wa,-mbig-obj -Wall -Wno-conversion
> -Wno-sign-conversion -Wunused-result -fno-semantic-interposition -mxsave
> -msse4.2
> {code}
> Perhaps we can pass {{-O0}}?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-13221) [C++] arrow_reader_writer_test.cc slow to compile
[
https://issues.apache.org/jira/browse/ARROW-13221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640993#comment-17640993
]
Kouhei Sutou commented on ARROW-13221:
--
It seems that 1) may not reduce total build time. Because it just splits whole
tests and doesn't change the total number of tests and the current build
approach (templated tests).
It seems that 2) will reduce total build time.
bq. step 2 above will probably be slightly cumbersome...
I agree with it...
> [C++] arrow_reader_writer_test.cc slow to compile
> -
>
> Key: ARROW-13221
> URL: https://issues.apache.org/jira/browse/ARROW-13221
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Antoine Pitrou
>Priority: Minor
>
> As soon as some optimizations are enabled,
> {{src/parquet/arrow/arrow_reader_writer_test.cc}} becomes extremely slow to
> compile (more than one minute just for itself). This is perceivable on e.g.
> the {{conda-cpp-valgrind}} build where we add {{-Og}} to the gcc flags in
> order to make the tests less slow under emulation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18418) [Website] do not delete /datafusion-python
[ https://issues.apache.org/jira/browse/ARROW-18418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou updated ARROW-18418: - Summary: [Website] do not delete /datafusion-python (was: [WEBSITE] do not delete /datafusion-python) > [Website] do not delete /datafusion-python > -- > > Key: ARROW-18418 > URL: https://issues.apache.org/jira/browse/ARROW-18418 > Project: Apache Arrow > Issue Type: Improvement > Components: Website >Reporter: Andy Grove >Assignee: Andy Grove >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > do not delete /datafusion-python when publishing -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18418) [Website] do not delete /datafusion-python
[ https://issues.apache.org/jira/browse/ARROW-18418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18418. -- Fix Version/s: 11.0.0 Resolution: Fixed https://github.com/apache/arrow-site/pull/278 > [Website] do not delete /datafusion-python > -- > > Key: ARROW-18418 > URL: https://issues.apache.org/jira/browse/ARROW-18418 > Project: Apache Arrow > Issue Type: Improvement > Components: Website >Reporter: Andy Grove >Assignee: Andy Grove >Priority: Major > Fix For: 11.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > do not delete /datafusion-python when publishing -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18412) [C++][R] Windows build fails because of missing ChunkResolver symbols
[
https://issues.apache.org/jira/browse/ARROW-18412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18412.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 14774
[https://github.com/apache/arrow/pull/14774]
> [C++][R] Windows build fails because of missing ChunkResolver symbols
> -
>
> Key: ARROW-18412
> URL: https://issues.apache.org/jira/browse/ARROW-18412
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, R
>Reporter: Dewey Dunnington
>Assignee: Dewey Dunnington
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In recent nightly builds of the Windows package we have a build failure
> because some symbols related to the {{ChunkResolver}} are not found in the
> linking stage.
> https://github.com/ursacomputing/crossbow/actions/runs/3559717769/jobs/5979255297#step:9:2818
> [~kou] suggested the following patch might fix the build:
> https://github.com/apache/arrow/pull/14530#issuecomment-1328341447
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18412) [R] Windows build fails because of missing ChunkResolver symbols
[
https://issues.apache.org/jira/browse/ARROW-18412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-18412:
-
Component/s: C++
> [R] Windows build fails because of missing ChunkResolver symbols
>
>
> Key: ARROW-18412
> URL: https://issues.apache.org/jira/browse/ARROW-18412
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, R
>Reporter: Dewey Dunnington
>Assignee: Dewey Dunnington
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In recent nightly builds of the Windows package we have a build failure
> because some symbols related to the {{ChunkResolver}} are not found in the
> linking stage.
> https://github.com/ursacomputing/crossbow/actions/runs/3559717769/jobs/5979255297#step:9:2818
> [~kou] suggested the following patch might fix the build:
> https://github.com/apache/arrow/pull/14530#issuecomment-1328341447
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18412) [C++][R] Windows build fails because of missing ChunkResolver symbols
[
https://issues.apache.org/jira/browse/ARROW-18412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-18412:
-
Summary: [C++][R] Windows build fails because of missing ChunkResolver
symbols (was: [R] Windows build fails because of missing ChunkResolver symbols)
> [C++][R] Windows build fails because of missing ChunkResolver symbols
> -
>
> Key: ARROW-18412
> URL: https://issues.apache.org/jira/browse/ARROW-18412
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, R
>Reporter: Dewey Dunnington
>Assignee: Dewey Dunnington
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In recent nightly builds of the Windows package we have a build failure
> because some symbols related to the {{ChunkResolver}} are not found in the
> linking stage.
> https://github.com/ursacomputing/crossbow/actions/runs/3559717769/jobs/5979255297#step:9:2818
> [~kou] suggested the following patch might fix the build:
> https://github.com/apache/arrow/pull/14530#issuecomment-1328341447
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-18039) [C++][CI] Reduce MinGW build times
[
https://issues.apache.org/jira/browse/ARROW-18039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640567#comment-17640567
]
Kouhei Sutou commented on ARROW-18039:
--
Wow! Thanks for the info!
> [C++][CI] Reduce MinGW build times
> --
>
> Key: ARROW-18039
> URL: https://issues.apache.org/jira/browse/ARROW-18039
> Project: Apache Arrow
> Issue Type: Wish
> Components: C++, Continuous Integration
>Reporter: Antoine Pitrou
>Assignee: Kouhei Sutou
>Priority: Major
>
> The MinGW C++ builds on CI currently build in release mode. This is probably
> because debug builds on Windows are complicated (you must get all the
> dependencies also compiled in debug mode, AFAIU).
> However, we could probably disable optimizations, so as to reduce compilation
> times.
> The compilation flags are currently as follows:
> {code}
> -- CMAKE_C_FLAGS: -O2 -DNDEBUG -ftree-vectorize -Wa,-mbig-obj -Wall
> -Wno-conversion -Wno-sign-conversion -Wunused-result
> -fno-semantic-interposition -mxsave -msse4.2
> -- CMAKE_CXX_FLAGS: -Wno-noexcept-type -fdiagnostics-color=always -O2
> -DNDEBUG -ftree-vectorize -Wa,-mbig-obj -Wall -Wno-conversion
> -Wno-sign-conversion -Wunused-result -fno-semantic-interposition -mxsave
> -msse4.2
> {code}
> Perhaps we can pass {{-O0}}?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-18039) [C++][CI] Reduce MinGW build times
[
https://issues.apache.org/jira/browse/ARROW-18039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640478#comment-17640478
]
Kouhei Sutou commented on ARROW-18039:
--
{noformat}
2: 103.922000s:
src/parquet/CMakeFiles/parquet-arrow-test.dir/arrow/arrow_reader_writer_test.cc.obj
{noformat}
{{TestParquetIO}} is heavy:
https://github.com/apache/arrow/blob/apache-arrow-10.0.0/cpp/src/parquet/arrow/arrow_reader_writer_test.cc#L598-L1611
Without {{TestParquetIO}}: 103s -> 25s
> [C++][CI] Reduce MinGW build times
> --
>
> Key: ARROW-18039
> URL: https://issues.apache.org/jira/browse/ARROW-18039
> Project: Apache Arrow
> Issue Type: Wish
> Components: C++, Continuous Integration
>Reporter: Antoine Pitrou
>Assignee: Kouhei Sutou
>Priority: Major
>
> The MinGW C++ builds on CI currently build in release mode. This is probably
> because debug builds on Windows are complicated (you must get all the
> dependencies also compiled in debug mode, AFAIU).
> However, we could probably disable optimizations, so as to reduce compilation
> times.
> The compilation flags are currently as follows:
> {code}
> -- CMAKE_C_FLAGS: -O2 -DNDEBUG -ftree-vectorize -Wa,-mbig-obj -Wall
> -Wno-conversion -Wno-sign-conversion -Wunused-result
> -fno-semantic-interposition -mxsave -msse4.2
> -- CMAKE_CXX_FLAGS: -Wno-noexcept-type -fdiagnostics-color=always -O2
> -DNDEBUG -ftree-vectorize -Wa,-mbig-obj -Wall -Wno-conversion
> -Wno-sign-conversion -Wunused-result -fno-semantic-interposition -mxsave
> -msse4.2
> {code}
> Perhaps we can pass {{-O0}}?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-18039) [C++][CI] Reduce MinGW build times
[
https://issues.apache.org/jira/browse/ARROW-18039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640470#comment-17640470
]
Kouhei Sutou commented on ARROW-18039:
--
Worst 100 build time on my local Windows:
{noformat}
1: 124.79s:
google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep-stamp/google_cloud_cpp_ep-build,
google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_storage.a,
google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_rest_internal.a,
google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_common.a,
C:/Users/kou/work/arrow/build-msys/google_cloud_cpp_ep-prefix/src/google_cloud_cpp_ep-stamp/google_cloud_cpp_ep-build,
C:/Users/kou/work/arrow/build-msys/google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_storage.a,
C:/Users/kou/work/arrow/build-msys/google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_rest_internal.a,
C:/Users/kou/work/arrow/build-msys/google_cloud_cpp_ep-install/lib/libgoogle_cloud_cpp_common.a
2: 103.922000s:
src/parquet/CMakeFiles/parquet-arrow-test.dir/arrow/arrow_reader_writer_test.cc.obj
3: 94.791000s:
src/arrow/compute/kernels/CMakeFiles/arrow-compute-vector-test.dir/vector_replace_test.cc.obj
4: 81.723000s:
src/arrow/compute/kernels/CMakeFiles/arrow-compute-scalar-test.dir/scalar_if_else_test.cc.obj
5: 72.075000s:
src/arrow/compute/kernels/CMakeFiles/arrow-compute-aggregate-test.dir/aggregate_test.cc.obj
6: 64.858000s:
src/arrow/CMakeFiles/arrow-array-test.dir/array/array_test.cc.obj
7: 57.531000s:
src/arrow/compute/kernels/CMakeFiles/arrow-compute-scalar-test.dir/scalar_arithmetic_test.cc.obj
8: 42.816000s:
src/arrow/CMakeFiles/arrow_shared.dir/compute/kernels/vector_sort.cc.obj
9: 41.387000s:
src/arrow/CMakeFiles/arrow_shared.dir/compute/kernels/scalar_arithmetic.cc.obj
10: 37.235000s:
src/arrow/compute/kernels/CMakeFiles/arrow-compute-vector-test.dir/vector_selection_test.cc.obj
11: 37.111000s:
src/arrow/compute/kernels/CMakeFiles/arrow-compute-scalar-test.dir/scalar_compare_test.cc.obj
12: 36.954000s:
src/arrow/CMakeFiles/arrow-array-test.dir/array/array_dict_test.cc.obj
13: 36.64s:
src/arrow/CMakeFiles/arrow_shared.dir/compute/kernels/hash_aggregate.cc.obj
14: 33.652000s:
src/arrow/CMakeFiles/arrow_shared.dir/compute/kernels/scalar_temporal_binary.cc.obj
15: 33.567000s:
src/arrow/compute/kernels/CMakeFiles/arrow-compute-scalar-test.dir/scalar_string_test.cc.obj
16: 30.662000s:
src/arrow/CMakeFiles/arrow_shared.dir/compute/kernels/scalar_temporal_unary.cc.obj
17: 30.403000s:
src/arrow/dataset/CMakeFiles/arrow-dataset-scanner-test.dir/scanner_test.cc.obj
18: 29.843000s: src/arrow/CMakeFiles/arrow_shared.dir/builder.cc.obj
19: 29.193000s:
src/arrow/CMakeFiles/arrow-sparse-tensor-test.dir/sparse_tensor_test.cc.obj
20: 29.089000s:
src/arrow/util/CMakeFiles/arrow-utility-test.dir/small_vector_test.cc.obj
21: 28.395000s:
src/arrow/filesystem/CMakeFiles/arrow-gcsfs-test.dir/gcsfs_test.cc.obj
22: 28.305000s:
src/arrow/ipc/CMakeFiles/arrow-ipc-read-write-test.dir/read_write_test.cc.obj
23: 26.873000s: src/arrow/CMakeFiles/arrow-scalar-test.dir/scalar_test.cc.obj
24: 26.222000s:
src/gandiva/CMakeFiles/gandiva_shared.dir/function_registry_arithmetic.cc.obj
25: 25.608000s:
src/arrow/util/CMakeFiles/arrow-async-utility-test.dir/async_generator_test.cc.obj
26: 25.414000s:
src/arrow/compute/kernels/CMakeFiles/arrow-compute-aggregate-test.dir/hash_aggregate_test.cc.obj
27: 23.884000s:
src/arrow/compute/kernels/CMakeFiles/arrow-compute-vector-test.dir/vector_sort_test.cc.obj
28: 22.756000s:
src/gandiva/tests/CMakeFiles/gandiva-projector-test.dir/projector_test.cc.obj
29: 20.789000s:
src/arrow/ipc/CMakeFiles/arrow-ipc-json-simple-test.dir/json_simple_test.cc.obj
30: 20.29s:
src/parquet/CMakeFiles/parquet-writer-test.dir/column_writer_test.cc.obj
31: 19.539000s:
src/arrow/CMakeFiles/arrow_shared.dir/compute/kernels/aggregate_basic.cc.obj
32: 19.115000s:
src/arrow/CMakeFiles/arrow_shared.dir/compute/kernels/scalar_string_ascii.cc.obj
33: 19.045000s:
src/arrow/util/CMakeFiles/arrow-utility-test.dir/decimal_test.cc.obj
34: 18.993000s:
src/arrow/dataset/CMakeFiles/arrow-dataset-file-parquet-test.dir/file_parquet_test.cc.obj
35: 18.743000s:
src/arrow/compute/exec/CMakeFiles/arrow-compute-plan-test.dir/plan_test.cc.obj
36: 18.203000s:
src/arrow/compute/kernels/CMakeFiles/arrow-compute-vector-test.dir/vector_hash_test.cc.obj
37: 17.673000s:
src/arrow/dataset/CMakeFiles/arrow-dataset-partition-test.dir/partition_test.cc.obj
38: 16.936000s:
src/arrow/CMakeFiles/arrow-array-test.dir/array/array_binary_test.cc.obj
39: 16.518000s:
src/arrow/c/CMakeFiles/arrow-c-bridge-test.dir/bridge_test.cc.obj
40: 16.342000s:
src/parquet/CMakeFiles/parquet-internals-test.dir/statistics_test.cc.obj
41: 16.215000s:
[jira] [Created] (ARROW-18414) [Release] Add a post script to generate announce email
Kouhei Sutou created ARROW-18414:
Summary: [Release] Add a post script to generate announce email
Key: ARROW-18414
URL: https://issues.apache.org/jira/browse/ARROW-18414
Project: Apache Arrow
Issue Type: Improvement
Components: Developer Tools
Reporter: Kouhei Sutou
Fix For: 11.0.0
We want to generate an announce email like a vote email.
e.g.: [ANNOUNCE] Apache Arrow 10.0.0 released
https://lists.apache.org/thread/zdsogdwj3r7wjv93o84go4ykgrcwtr0p .
FYI: We can generate a vote email by {{SOURCE_DEFAULT=0 SOURCE_VOTE=1
dev/release/02-source.sh ...}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18407) [Release][Website] Use UTC for release date
[ https://issues.apache.org/jira/browse/ARROW-18407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18407. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14737 [https://github.com/apache/arrow/pull/14737] > [Release][Website] Use UTC for release date > --- > > Key: ARROW-18407 > URL: https://issues.apache.org/jira/browse/ARROW-18407 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools, Website >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18405) [Ruby] Raw table converter rebuilds chunked arrays
[
https://issues.apache.org/jira/browse/ARROW-18405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18405.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 14738
[https://github.com/apache/arrow/pull/14738]
> [Ruby] Raw table converter rebuilds chunked arrays
> --
>
> Key: ARROW-18405
> URL: https://issues.apache.org/jira/browse/ARROW-18405
> Project: Apache Arrow
> Issue Type: Bug
> Components: Ruby
>Affects Versions: 10.0.0
>Reporter: Sten Larsson
>Assignee: Kouhei Sutou
>Priority: Minor
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Consider the following Ruby script:
> {code:ruby}
> require 'arrow'
> data = Arrow::ChunkedArray.new([Arrow::Int64Array.new([1])])
> table = Arrow::Table.new('column' => data)
> puts table['column'].data_type
> {code}
> This prints "int64" with red-arrow 9.0.0 and "uint8" in 10.0.0.
> From my understanding it is due to this commit:
> [https://github.com/apache/arrow/commit/913d9c0a9a1a4398ed5f56d713d586770b4f702c#diff-f7f19bbc3945ea30ba06d851705f2d58f7666507bb101c4e151014ca398bd635R42]
> The old version would not call ArrayBuilder.build on a ChunkedArray, but the
> new version does. This is a problem for us, because we need the column to
> stay int64.
> A workaround is to specify a schema and list of arrays instead to bypass the
> raw table converter:
> {code:ruby}
> table = Arrow::Table.new([{name: 'column', type: 'int64'}], [data])
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18409) [GLib][Plasma] Suppress deprecated warning in building plasma-glib
[
https://issues.apache.org/jira/browse/ARROW-18409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18409.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 14739
[https://github.com/apache/arrow/pull/14739]
> [GLib][Plasma] Suppress deprecated warning in building plasma-glib
> --
>
> Key: ARROW-18409
> URL: https://issues.apache.org/jira/browse/ARROW-18409
> Project: Apache Arrow
> Issue Type: Improvement
> Components: GLib
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> If we always get "Plasma is deprecated since Arrow 10.0.0. ..." warning from
> {{plasma/common.h}}, we can't use {{-Dwerror=true}} Meson option with
> plama-glib.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18410) [Packaging][Ubuntu] Add support for Ubuntu 22.10
[ https://issues.apache.org/jira/browse/ARROW-18410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18410. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14740 [https://github.com/apache/arrow/pull/14740] > [Packaging][Ubuntu] Add support for Ubuntu 22.10 > > > Key: ARROW-18410 > URL: https://issues.apache.org/jira/browse/ARROW-18410 > Project: Apache Arrow > Issue Type: Improvement > Components: Packaging >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (ARROW-18334) [C++] add function for timestamp/duration is not commutative
[
https://issues.apache.org/jira/browse/ARROW-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou reassigned ARROW-18334:
Assignee: Ben Kietzman
> [C++] add function for timestamp/duration is not commutative
>
>
> Key: ARROW-18334
> URL: https://issues.apache.org/jira/browse/ARROW-18334
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Weston Pace
>Assignee: Ben Kietzman
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> The expression simplification currently has a small set of functions which it
> knows are commutative (IsBinaryAssociativeCommutative). "add" (and
> "add_checked" are in this list. This should be ok for
> add(timestamp,duration) since this boils down to add(int64,int64) which is
> commutative. However, the way the kernels are currently implemented, we are
> getting the incorrect output type.
> Concretely, we have kernels:
> {noformat}
> add_checked() -> types[0]
> add_checked() -> types[1]
> {noformat}
> A call is made with expression {{field_ref("x") + duration_literal}}. This
> call is bound to {{add_checked}}. However, the
> expression is then simplified to {{duration_literal + field_ref("x")}}.
> Oddly enough, the math in this case is correct, since it is just addition,
> but the output type is not. It assigns an output type of duration instead of
> timestamp.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18334) [C++] add function for timestamp/duration is not commutative
[
https://issues.apache.org/jira/browse/ARROW-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18334.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 14659
[https://github.com/apache/arrow/pull/14659]
> [C++] add function for timestamp/duration is not commutative
>
>
> Key: ARROW-18334
> URL: https://issues.apache.org/jira/browse/ARROW-18334
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Weston Pace
>Assignee: Ben Kietzman
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> The expression simplification currently has a small set of functions which it
> knows are commutative (IsBinaryAssociativeCommutative). "add" (and
> "add_checked" are in this list. This should be ok for
> add(timestamp,duration) since this boils down to add(int64,int64) which is
> commutative. However, the way the kernels are currently implemented, we are
> getting the incorrect output type.
> Concretely, we have kernels:
> {noformat}
> add_checked() -> types[0]
> add_checked() -> types[1]
> {noformat}
> A call is made with expression {{field_ref("x") + duration_literal}}. This
> call is bound to {{add_checked}}. However, the
> expression is then simplified to {{duration_literal + field_ref("x")}}.
> Oddly enough, the math in this case is correct, since it is just addition,
> but the output type is not. It assigns an output type of duration instead of
> timestamp.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18334) [C++] Add function for timestamp/duration is not commutative
[
https://issues.apache.org/jira/browse/ARROW-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-18334:
-
Summary: [C++] Add function for timestamp/duration is not commutative
(was: add function for timestamp/duration is not commutative)
> [C++] Add function for timestamp/duration is not commutative
>
>
> Key: ARROW-18334
> URL: https://issues.apache.org/jira/browse/ARROW-18334
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Weston Pace
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> The expression simplification currently has a small set of functions which it
> knows are commutative (IsBinaryAssociativeCommutative). "add" (and
> "add_checked" are in this list. This should be ok for
> add(timestamp,duration) since this boils down to add(int64,int64) which is
> commutative. However, the way the kernels are currently implemented, we are
> getting the incorrect output type.
> Concretely, we have kernels:
> {noformat}
> add_checked() -> types[0]
> add_checked() -> types[1]
> {noformat}
> A call is made with expression {{field_ref("x") + duration_literal}}. This
> call is bound to {{add_checked}}. However, the
> expression is then simplified to {{duration_literal + field_ref("x")}}.
> Oddly enough, the math in this case is correct, since it is just addition,
> but the output type is not. It assigns an output type of duration instead of
> timestamp.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18334) [C++] add function for timestamp/duration is not commutative
[
https://issues.apache.org/jira/browse/ARROW-18334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-18334:
-
Summary: [C++] add function for timestamp/duration is not commutative
(was: [C++] Add function for timestamp/duration is not commutative)
> [C++] add function for timestamp/duration is not commutative
>
>
> Key: ARROW-18334
> URL: https://issues.apache.org/jira/browse/ARROW-18334
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Weston Pace
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> The expression simplification currently has a small set of functions which it
> knows are commutative (IsBinaryAssociativeCommutative). "add" (and
> "add_checked" are in this list. This should be ok for
> add(timestamp,duration) since this boils down to add(int64,int64) which is
> commutative. However, the way the kernels are currently implemented, we are
> getting the incorrect output type.
> Concretely, we have kernels:
> {noformat}
> add_checked() -> types[0]
> add_checked() -> types[1]
> {noformat}
> A call is made with expression {{field_ref("x") + duration_literal}}. This
> call is bound to {{add_checked}}. However, the
> expression is then simplified to {{duration_literal + field_ref("x")}}.
> Oddly enough, the math in this case is correct, since it is just addition,
> but the output type is not. It assigns an output type of duration instead of
> timestamp.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-18410) [Packaging][Ubuntu] Add support for Ubuntu 22.10
Kouhei Sutou created ARROW-18410: Summary: [Packaging][Ubuntu] Add support for Ubuntu 22.10 Key: ARROW-18410 URL: https://issues.apache.org/jira/browse/ARROW-18410 Project: Apache Arrow Issue Type: Improvement Components: Packaging Reporter: Kouhei Sutou Assignee: Kouhei Sutou -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18361) [CI][Conan] Merge upstream changes
[ https://issues.apache.org/jira/browse/ARROW-18361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18361. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14671 [https://github.com/apache/arrow/pull/14671] > [CI][Conan] Merge upstream changes > -- > > Key: ARROW-18361 > URL: https://issues.apache.org/jira/browse/ARROW-18361 > Project: Apache Arrow > Issue Type: Improvement > Components: Continuous Integration >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Updated: https://github.com/conan-io/conan-center-index/pull/14111 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18390) [CI][Python] Nightly python test for spark master missing test module
[
https://issues.apache.org/jira/browse/ARROW-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18390.
--
Resolution: Fixed
Issue resolved by pull request 14715
[https://github.com/apache/arrow/pull/14715]
> [CI][Python] Nightly python test for spark master missing test module
> -
>
> Key: ARROW-18390
> URL: https://issues.apache.org/jira/browse/ARROW-18390
> Project: Apache Arrow
> Issue Type: Bug
> Components: Continuous Integration, Python
>Reporter: Raúl Cumplido
>Assignee: Raúl Cumplido
>Priority: Major
> Labels: Nightly, pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> Currently the nightly test with spark master
> [test-conda-python-3.9-spark-master|[https://github.com/ursacomputing/crossbow/actions/runs/3528196313/jobs/5918037939]]
> fail with:
> {code:java}
> Starting test(python): pyspark.sql.tests.test_pandas_map (temp output:
> /spark/python/target/cbca1b18-4af7-4205-aa41-8c945bf1cf58/python__pyspark.sql.tests.test_pandas_map__9ptzo8sa.log)
> /opt/conda/envs/arrow/bin/python: No module named
> pyspark.sql.tests.test_pandas_grouped_map {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18390) [CI][Python] Nightly python test for spark master missing test module
[
https://issues.apache.org/jira/browse/ARROW-18390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-18390:
-
Fix Version/s: 10.0.2
> [CI][Python] Nightly python test for spark master missing test module
> -
>
> Key: ARROW-18390
> URL: https://issues.apache.org/jira/browse/ARROW-18390
> Project: Apache Arrow
> Issue Type: Bug
> Components: Continuous Integration, Python
>Reporter: Raúl Cumplido
>Assignee: Raúl Cumplido
>Priority: Major
> Labels: Nightly, pull-request-available
> Fix For: 10.0.2, 11.0.0
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> Currently the nightly test with spark master
> [test-conda-python-3.9-spark-master|[https://github.com/ursacomputing/crossbow/actions/runs/3528196313/jobs/5918037939]]
> fail with:
> {code:java}
> Starting test(python): pyspark.sql.tests.test_pandas_map (temp output:
> /spark/python/target/cbca1b18-4af7-4205-aa41-8c945bf1cf58/python__pyspark.sql.tests.test_pandas_map__9ptzo8sa.log)
> /opt/conda/envs/arrow/bin/python: No module named
> pyspark.sql.tests.test_pandas_grouped_map {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-18409) [GLib][Plasma] Suppress deprecated warning in building plasma-glib
Kouhei Sutou created ARROW-18409:
Summary: [GLib][Plasma] Suppress deprecated warning in building
plasma-glib
Key: ARROW-18409
URL: https://issues.apache.org/jira/browse/ARROW-18409
Project: Apache Arrow
Issue Type: Improvement
Components: GLib
Reporter: Kouhei Sutou
Assignee: Kouhei Sutou
If we always get "Plasma is deprecated since Arrow 10.0.0. ..." warning from
{{plasma/common.h}}, we can't use {{-Dwerror=true}} Meson option with
plama-glib.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-18405) [Ruby] Raw table converter rebuilds chunked arrays
[
https://issues.apache.org/jira/browse/ARROW-18405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17638759#comment-17638759
]
Kouhei Sutou commented on ARROW-18405:
--
Good catch!
> [Ruby] Raw table converter rebuilds chunked arrays
> --
>
> Key: ARROW-18405
> URL: https://issues.apache.org/jira/browse/ARROW-18405
> Project: Apache Arrow
> Issue Type: Bug
> Components: Ruby
>Affects Versions: 10.0.0
>Reporter: Sten Larsson
>Assignee: Kouhei Sutou
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Consider the following Ruby script:
> {code:ruby}
> require 'arrow'
> data = Arrow::ChunkedArray.new([Arrow::Int64Array.new([1])])
> table = Arrow::Table.new('column' => data)
> puts table['column'].data_type
> {code}
> This prints "int64" with red-arrow 9.0.0 and "uint8" in 10.0.0.
> From my understanding it is due to this commit:
> [https://github.com/apache/arrow/commit/913d9c0a9a1a4398ed5f56d713d586770b4f702c#diff-f7f19bbc3945ea30ba06d851705f2d58f7666507bb101c4e151014ca398bd635R42]
> The old version would not call ArrayBuilder.build on a ChunkedArray, but the
> new version does. This is a problem for us, because we need the column to
> stay int64.
> A workaround is to specify a schema and list of arrays instead to bypass the
> raw table converter:
> {code:ruby}
> table = Arrow::Table.new([{name: 'column', type: 'int64'}], [data])
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-18407) [Release][Website] Use UTC for release date
Kouhei Sutou created ARROW-18407: Summary: [Release][Website] Use UTC for release date Key: ARROW-18407 URL: https://issues.apache.org/jira/browse/ARROW-18407 Project: Apache Arrow Issue Type: Improvement Components: Developer Tools, Website Reporter: Kouhei Sutou Assignee: Kouhei Sutou -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (ARROW-18405) [Ruby] Raw table converter rebuilds chunked arrays
[
https://issues.apache.org/jira/browse/ARROW-18405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou reassigned ARROW-18405:
Assignee: Kouhei Sutou
> [Ruby] Raw table converter rebuilds chunked arrays
> --
>
> Key: ARROW-18405
> URL: https://issues.apache.org/jira/browse/ARROW-18405
> Project: Apache Arrow
> Issue Type: Bug
> Components: Ruby
>Affects Versions: 10.0.0
>Reporter: Sten Larsson
>Assignee: Kouhei Sutou
>Priority: Minor
>
> Consider the following Ruby script:
> {code:ruby}
> require 'arrow'
> data = Arrow::ChunkedArray.new([Arrow::Int64Array.new([1])])
> table = Arrow::Table.new('column' => data)
> puts table['column'].data_type
> {code}
> This prints "int64" with red-arrow 9.0.0 and "uint8" in 10.0.0.
> From my understanding it is due to this commit:
> [https://github.com/apache/arrow/commit/913d9c0a9a1a4398ed5f56d713d586770b4f702c#diff-f7f19bbc3945ea30ba06d851705f2d58f7666507bb101c4e151014ca398bd635R42]
> The old version would not call ArrayBuilder.build on a ChunkedArray, but the
> new version does. This is a problem for us, because we need the column to
> stay int64.
> A workaround is to specify a schema and list of arrays instead to bypass the
> raw table converter:
> {code:ruby}
> table = Arrow::Table.new([{name: 'column', type: 'int64'}], [data])
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18292) [Release][Python] Upload .wheel/.tar.gz for release not RC
[
https://issues.apache.org/jira/browse/ARROW-18292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18292.
--
Resolution: Fixed
Issue resolved by pull request 14708
[https://github.com/apache/arrow/pull/14708]
> [Release][Python] Upload .wheel/.tar.gz for release not RC
> --
>
> Key: ARROW-18292
> URL: https://issues.apache.org/jira/browse/ARROW-18292
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Developer Tools
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{dev/release/post-09-python.sh}} uploads {{.wheel}}/{{.tar.gz}} for RC (
> https://apache.jfrog.io/ui/native/arrow/python-rc/ ) not release (
> https://apache.jfrog.io/ui/native/arrow/python/ ) . They are the same content
> (because we copy artifacts of passed RC to release) but we should upload
> {{.wheel}}/{{.tar.gz}} for release to clarify that we use vote passed
> artifacts.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18384) [Release][MSYS2] Show pull request title
[ https://issues.apache.org/jira/browse/ARROW-18384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18384. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14709 [https://github.com/apache/arrow/pull/14709] > [Release][MSYS2] Show pull request title > > > Key: ARROW-18384 > URL: https://issues.apache.org/jira/browse/ARROW-18384 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ARROW-18381) MIGRATION: Create milestones for every needed fix version
[ https://issues.apache.org/jira/browse/ARROW-18381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637958#comment-17637958 ] Kouhei Sutou commented on ARROW-18381: -- Oh. It seems that https://github.com/community/community/discussions/4566 is a feature request for the feature. I'm OK with associating with only the lower version. > MIGRATION: Create milestones for every needed fix version > - > > Key: ARROW-18381 > URL: https://issues.apache.org/jira/browse/ARROW-18381 > Project: Apache Arrow > Issue Type: Task >Reporter: Todd Farmer >Priority: Major > Attachments: Screenshot from 2022-11-22 11-53-07.png, Screenshot from > 2022-11-22 11-54-26.png > > > The Apache Arrow projects uses the "Fix version" field in ASF Jira issue to > track the version in which issues were resolved/fixed/implemented. The most > equivalent field in GitHub issues is the "milestone" field. This field is > explicitly managed - the versions need to be added to the repository > configuration before they can be used. This mapping needs to be established > as a prerequisite for completing the import from ASF Jira. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18389) [CI][Python] Update nightly test-conda-python-3.7-pandas-0.24 to pandas >= 1.0
[ https://issues.apache.org/jira/browse/ARROW-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18389. -- Resolution: Fixed Issue resolved by pull request 14714 [https://github.com/apache/arrow/pull/14714] > [CI][Python] Update nightly test-conda-python-3.7-pandas-0.24 to pandas >= 1.0 > -- > > Key: ARROW-18389 > URL: https://issues.apache.org/jira/browse/ARROW-18389 > Project: Apache Arrow > Issue Type: Bug > Components: Continuous Integration, Python >Reporter: Raúl Cumplido >Assignee: Raúl Cumplido >Priority: Major > Labels: Nightly, pull-request-available > Fix For: 11.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > https://issues.apache.org/jira/browse/ARROW-18173 Removed support for pandas > < 1.0. We should upgrade the nightly test. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18397) [C++] Clear S3 region resolver client at S3 shutdown
[ https://issues.apache.org/jira/browse/ARROW-18397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18397. -- Resolution: Fixed Issue resolved by pull request 14718 [https://github.com/apache/arrow/pull/14718] > [C++] Clear S3 region resolver client at S3 shutdown > > > Key: ARROW-18397 > URL: https://issues.apache.org/jira/browse/ARROW-18397 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Antoine Pitrou >Assignee: Antoine Pitrou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0, 10.0.2 > > Time Spent: 50m > Remaining Estimate: 0h > > The S3 region resolver caches a S3 client at module scope. This client can be > destroyed very late and trigger an assertion error in the AWS SDK because it > was already shutdown: > https://github.com/aws/aws-sdk-cpp/issues/2204 > When explicitly finalizing S3, we should ensure we also destroy the cached S3 > client. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-18384) [Release][MSYS2] Show pull request title
Kouhei Sutou created ARROW-18384: Summary: [Release][MSYS2] Show pull request title Key: ARROW-18384 URL: https://issues.apache.org/jira/browse/ARROW-18384 Project: Apache Arrow Issue Type: Improvement Components: Developer Tools Reporter: Kouhei Sutou Assignee: Kouhei Sutou -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (ARROW-18292) [Release][Python] Upload .wheel/.tar.gz for release not RC
[
https://issues.apache.org/jira/browse/ARROW-18292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou reassigned ARROW-18292:
Assignee: Kouhei Sutou
> [Release][Python] Upload .wheel/.tar.gz for release not RC
> --
>
> Key: ARROW-18292
> URL: https://issues.apache.org/jira/browse/ARROW-18292
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Developer Tools
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Blocker
> Fix For: 11.0.0
>
>
> {{dev/release/post-09-python.sh}} uploads {{.wheel}}/{{.tar.gz}} for RC (
> https://apache.jfrog.io/ui/native/arrow/python-rc/ ) not release (
> https://apache.jfrog.io/ui/native/arrow/python/ ) . They are the same content
> (because we copy artifacts of passed RC to release) but we should upload
> {{.wheel}}/{{.tar.gz}} for release to clarify that we use vote passed
> artifacts.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18340) [Python] PyArrow C++ header files no longer always included in installed pyarrow
[
https://issues.apache.org/jira/browse/ARROW-18340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18340.
--
Resolution: Fixed
Issue resolved by pull request 14656
[https://github.com/apache/arrow/pull/14656]
> [Python] PyArrow C++ header files no longer always included in installed
> pyarrow
>
>
> Key: ARROW-18340
> URL: https://issues.apache.org/jira/browse/ARROW-18340
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
>Affects Versions: 10.0.0
>Reporter: Joris Van den Bossche
>Assignee: Alenka Frim
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0, 10.0.2
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> We have a python build env var to control whether the Arrow C++ header files
> are included in the python package or not
> ({{PYARROW_BUNDLE_ARROW_CPP_HEADERS}}). This is set to True by default, and
> only in the conda recipe set to False.
> After the cmake refactor, the Python C++ header files no longer live in the
> Arrow C++ package, and so should _always_ be included in the python package,
> regardless of how arrow-cpp is installed.
> Initially this was done, but it seems that
> https://github.com/apache/arrow/pull/13892 removed this unconditional copy of
> the PyArrow header files to {{pyarrow/include}}. Now it is only copied if
> {{PYARROW_BUNDLE_ARROW_CPP_HEADERS}} is enabled.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18382) [C++] "ADDRESS_SANITIZER" not defined in fuzzing builds
[
https://issues.apache.org/jira/browse/ARROW-18382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18382.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 14702
[https://github.com/apache/arrow/pull/14702]
> [C++] "ADDRESS_SANITIZER" not defined in fuzzing builds
> ---
>
> Key: ARROW-18382
> URL: https://issues.apache.org/jira/browse/ARROW-18382
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Antoine Pitrou
>Assignee: Antoine Pitrou
>Priority: Minor
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Fuzzing builds (as run by OSS-Fuzz) enable Address Sanitizer through their
> own set of options rather than by enabling {{ARROW_USE_ASAN}}. However, we
> need to be informed this situation in the Arrow source code.
> One example of where this matters is that eternal thread pools produce
> spurious leaks at shutdown because of the vector of at-fork handlers; it
> therefore needs to be worked around on those builds.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-16340) [C++][Python] Move all Python related code into PyArrow
[
https://issues.apache.org/jira/browse/ARROW-16340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17637098#comment-17637098
]
Kouhei Sutou commented on ARROW-16340:
--
I don't recommend the approach because the approach mixes 2 Apache Arrow C++
libraries.
bq. 1) Do you think if it helps if I keep pyarrow's version and Arrow C++
library version always consistent (for example, both using 10.0.0)?
It's better but I don't recommend mixing Apache Arrow C++ library in pyarrow's
wheel and Apache Arrow C++ library from vcpkg.
bq. 2) If I use the official pyarrow (in Python) wheel and (pyarrow C++ library
+ Arrow C++ library, both compiled from vcpkg),
I don't recommend mixing them.
bq. is it any better than using the C data stream API?
You can use simpler API. But it's not the biggest problem. You can use either
API you like if you don't mix Apache Arrow C++ libraries.
bq. You said `you mix multiple Apache Arrow C++ libraries` and this could cause
unexpected behavior, but it seems even if I don't use this approach, as long as
I use pyarrow wheel in Python, I may run into such some unexpected problem, is
it correct?
Sorry. I couldn't understand this. Does it mean that you use only pyarrow
wheel? (You don't mix 2 Apache Arrow C++ libraries?)
> [C++][Python] Move all Python related code into PyArrow
> ---
>
> Key: ARROW-16340
> URL: https://issues.apache.org/jira/browse/ARROW-16340
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, Python
>Reporter: Alenka Frim
>Assignee: Alenka Frim
>Priority: Major
> Labels: pull-request-available
> Fix For: 10.0.0
>
> Time Spent: 33h 10m
> Remaining Estimate: 0h
>
> Move {{src/arrow/python}} directory into {{pyarrow}} and arrange PyArrow to
> build it.
> More details can be found on this thread:
> https://lists.apache.org/thread/jbxyldhqff4p9z53whhs95y4jcomdgd2
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18366) [Packaging][RPM][Gandiva] Failed to link on AlmaLinux 9
[
https://issues.apache.org/jira/browse/ARROW-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18366.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 14680
[https://github.com/apache/arrow/pull/14680]
> [Packaging][RPM][Gandiva] Failed to link on AlmaLinux 9
>
>
> Key: ARROW-18366
> URL: https://issues.apache.org/jira/browse/ARROW-18366
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Packaging
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> https://github.com/ursacomputing/crossbow/actions/runs/3502784911/jobs/5867407921#step:6:4748
> {noformat}
> FAILED: gandiva-glib/Gandiva-1.0.gir
> env
> PKG_CONFIG_PATH=/usr/lib64/pkgconfig:/usr/share/pkgconfig:/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/meson-uninstalled
> /usr/bin/g-ir-scanner --quiet --no-libtool --namespace=Gandiva
> --nsversion=1.0 --warn-all --output gandiva-glib/Gandiva-1.0.gir
> --c-include=gandiva-glib/gandiva-glib.h --warn-all
> --include-uninstalled=./arrow-glib/Arrow-1.0.gir
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/gandiva-glib
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/gandiva-glib
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/redhat-linux-build/src
>
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/redhat-linux-build/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/redhat-linux-build/src
>
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/redhat-linux-build/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/src
> --filelist=/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/gandiva-glib/libgandiva-glib.so.1100.0.0.p/Gandiva_1.0_gir_filelist
> --include=Arrow-1.0 --symbol-prefix=ggandiva --identifier-prefix=GGandiva
> --pkg-export=gandiva-glib --cflags-begin
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/redhat-linux-build/src
>
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/redhat-linux-build/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/src
> -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
> -I/usr/include/sysprof-4 -I/usr/include/gobject-introspection-1.0
> --cflags-end
> --add-include-path=/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/arrow-glib
> --add-include-path=/usr/share/gir-1.0
> -L/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/gandiva-glib
> --library gandiva-glib
> -L/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/arrow-glib
> -L/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../../cpp/redhat-linux-build/release
> --extra-library=gobject-2.0 --extra-library=glib-2.0
> --extra-library=girepository-1.0 --sources-top-dirs
> /build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/ --sources-top-dirs
> /build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/ --warn-error
> /usr/bin/ld:
> /build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../../cpp/redhat-linux-build/release/libgandiva.so.1100:
> undefined reference to `std::__glibcxx_assert_fail(char const*, int, char
> const*, char const*)'
> collect2: error: ld returned 1 exit status
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18366) [Packaging][RPM][Gandiva] Failed to link on AlmaLinux 9
[
https://issues.apache.org/jira/browse/ARROW-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-18366:
-
Fix Version/s: 10.0.2
> [Packaging][RPM][Gandiva] Failed to link on AlmaLinux 9
>
>
> Key: ARROW-18366
> URL: https://issues.apache.org/jira/browse/ARROW-18366
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Packaging
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 10.0.2, 11.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> https://github.com/ursacomputing/crossbow/actions/runs/3502784911/jobs/5867407921#step:6:4748
> {noformat}
> FAILED: gandiva-glib/Gandiva-1.0.gir
> env
> PKG_CONFIG_PATH=/usr/lib64/pkgconfig:/usr/share/pkgconfig:/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/meson-uninstalled
> /usr/bin/g-ir-scanner --quiet --no-libtool --namespace=Gandiva
> --nsversion=1.0 --warn-all --output gandiva-glib/Gandiva-1.0.gir
> --c-include=gandiva-glib/gandiva-glib.h --warn-all
> --include-uninstalled=./arrow-glib/Arrow-1.0.gir
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/gandiva-glib
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/gandiva-glib
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/redhat-linux-build/src
>
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/redhat-linux-build/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/redhat-linux-build/src
>
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/redhat-linux-build/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/src
> --filelist=/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/gandiva-glib/libgandiva-glib.so.1100.0.0.p/Gandiva_1.0_gir_filelist
> --include=Arrow-1.0 --symbol-prefix=ggandiva --identifier-prefix=GGandiva
> --pkg-export=gandiva-glib --cflags-begin
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/.
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/redhat-linux-build/src
>
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/redhat-linux-build/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/src
> -I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/src
> -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
> -I/usr/include/sysprof-4 -I/usr/include/gobject-introspection-1.0
> --cflags-end
> --add-include-path=/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/arrow-glib
> --add-include-path=/usr/share/gir-1.0
> -L/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/gandiva-glib
> --library gandiva-glib
> -L/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/arrow-glib
> -L/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../../cpp/redhat-linux-build/release
> --extra-library=gobject-2.0 --extra-library=glib-2.0
> --extra-library=girepository-1.0 --sources-top-dirs
> /build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/ --sources-top-dirs
> /build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/ --warn-error
> /usr/bin/ld:
> /build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../../cpp/redhat-linux-build/release/libgandiva.so.1100:
> undefined reference to `std::__glibcxx_assert_fail(char const*, int, char
> const*, char const*)'
> collect2: error: ld returned 1 exit status
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ARROW-18366) [Packaging][RPM][Gandiva] Failed to link on AlmaLinux 9
Kouhei Sutou created ARROW-18366:
Summary: [Packaging][RPM][Gandiva] Failed to link on AlmaLinux 9
Key: ARROW-18366
URL: https://issues.apache.org/jira/browse/ARROW-18366
Project: Apache Arrow
Issue Type: Improvement
Components: Packaging
Reporter: Kouhei Sutou
Assignee: Kouhei Sutou
https://github.com/ursacomputing/crossbow/actions/runs/3502784911/jobs/5867407921#step:6:4748
{noformat}
FAILED: gandiva-glib/Gandiva-1.0.gir
env
PKG_CONFIG_PATH=/usr/lib64/pkgconfig:/usr/share/pkgconfig:/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/meson-uninstalled
/usr/bin/g-ir-scanner --quiet --no-libtool --namespace=Gandiva --nsversion=1.0
--warn-all --output gandiva-glib/Gandiva-1.0.gir
--c-include=gandiva-glib/gandiva-glib.h --warn-all
--include-uninstalled=./arrow-glib/Arrow-1.0.gir
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/gandiva-glib
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/gandiva-glib
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/.
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/.
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/redhat-linux-build/src
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/redhat-linux-build/src
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/src
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/src
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/.
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/.
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/redhat-linux-build/src
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/redhat-linux-build/src
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/src
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/src
--filelist=/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/gandiva-glib/libgandiva-glib.so.1100.0.0.p/Gandiva_1.0_gir_filelist
--include=Arrow-1.0 --symbol-prefix=ggandiva --identifier-prefix=GGandiva
--pkg-export=gandiva-glib --cflags-begin
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/.
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/.
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/redhat-linux-build/src
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/redhat-linux-build/src
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/../cpp/src
-I/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../cpp/src
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -I/usr/include/sysprof-4
-I/usr/include/gobject-introspection-1.0 --cflags-end
--add-include-path=/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/arrow-glib
--add-include-path=/usr/share/gir-1.0
-L/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/gandiva-glib
--library gandiva-glib
-L/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/arrow-glib
-L/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../../cpp/redhat-linux-build/release
--extra-library=gobject-2.0 --extra-library=glib-2.0
--extra-library=girepository-1.0 --sources-top-dirs
/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/ --sources-top-dirs
/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/ --warn-error
/usr/bin/ld:
/build/rpmbuild/BUILD/apache-arrow-11.0.0.dev130/c_glib/build/../../cpp/redhat-linux-build/release/libgandiva.so.1100:
undefined reference to `std::__glibcxx_assert_fail(char const*, int, char
const*, char const*)'
collect2: error: ld returned 1 exit status
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18362) [Parquet][C++] Accelerate bit-packing decoding with AVX-512
[ https://issues.apache.org/jira/browse/ARROW-18362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou updated ARROW-18362: - Summary: [Parquet][C++] Accelerate bit-packing decoding with AVX-512 (was: [C++] Accelerate Parquet bit-packing decoding with AVX-512) > [Parquet][C++] Accelerate bit-packing decoding with AVX-512 > --- > > Key: ARROW-18362 > URL: https://issues.apache.org/jira/browse/ARROW-18362 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, Parquet >Reporter: zhaoyaqi >Assignee: zhaoyaqi >Priority: Major > > Accelerate Parquet bit-packing decoding with AVX-512 instructions? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (ARROW-18362) [C++] Accelerate Parquet bit-packing decoding with AVX-512
[ https://issues.apache.org/jira/browse/ARROW-18362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou reassigned ARROW-18362: Assignee: zhaoyaqi > [C++] Accelerate Parquet bit-packing decoding with AVX-512 > -- > > Key: ARROW-18362 > URL: https://issues.apache.org/jira/browse/ARROW-18362 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: zhaoyaqi >Assignee: zhaoyaqi >Priority: Major > > Accelerate Parquet bit-packing decoding with AVX-512 instructions? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ARROW-18362) [C++] Accelerate Parquet bit-packing decoding with AVX-512
[ https://issues.apache.org/jira/browse/ARROW-18362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou updated ARROW-18362: - Component/s: Parquet > [C++] Accelerate Parquet bit-packing decoding with AVX-512 > -- > > Key: ARROW-18362 > URL: https://issues.apache.org/jira/browse/ARROW-18362 > Project: Apache Arrow > Issue Type: Improvement > Components: C++, Parquet >Reporter: zhaoyaqi >Assignee: zhaoyaqi >Priority: Major > > Accelerate Parquet bit-packing decoding with AVX-512 instructions? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-18361) [CI][Conan] Merge upstream changes
Kouhei Sutou created ARROW-18361: Summary: [CI][Conan] Merge upstream changes Key: ARROW-18361 URL: https://issues.apache.org/jira/browse/ARROW-18361 Project: Apache Arrow Issue Type: Improvement Components: Continuous Integration Reporter: Kouhei Sutou Assignee: Kouhei Sutou Updated: https://github.com/conan-io/conan-center-index/pull/14111 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18256) [C++][Windows] External/shared libthrift requires IMPORTED_IMPLIB in FindThrift.cmake
[
https://issues.apache.org/jira/browse/ARROW-18256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18256.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 14595
[https://github.com/apache/arrow/pull/14595]
> [C++][Windows] External/shared libthrift requires IMPORTED_IMPLIB in
> FindThrift.cmake
> --
>
> Key: ARROW-18256
> URL: https://issues.apache.org/jira/browse/ARROW-18256
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 10.0.0
>Reporter: Maarten Pronk
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> As mentioned in ARROW-18255, over at
> [https://github.com/JuliaPackaging/Yggdrasil/pull/5425,] we cross-compile
> Arrow to make it easily available within the Julia ecosystem.
> When compiling with `DARROW_THRIFT_USE_SHARED=ON` and using an external
> libthrift, CMAKE warns with:
>
> {quote}[12:00:54] CMake Warning (dev) in CMakeLists.txt:
> [12:00:54] Policy CMP0111 is not set: An imported target missing its
> location property
> [12:00:54] fails during generation. Run "cmake --help-policy CMP0111" for
> policy
> [12:00:54] details. Use the cmake_policy command to set the policy and
> suppress this
> [12:00:54] warning.
> [12:00:54]
> [12:00:54] IMPORTED_IMPLIB not set for imported target "thrift::thrift"
> configuration
> [12:00:54] "RELEASE".
> [12:00:54] This warning is for project developers. Use -Wno-dev to suppress
> it.
> {quote}
> This will lead to linking errors later on. I've fixed the warning with the
> following patch for v10.
>
> {quote}{{diff --git a/cpp/cmake_modules/FindThrift.cmake
> b/cpp/cmake_modules/FindThrift.cmake}}
> {{index 2f20a5cb5..2d1e728aa 100644}}
> {{--- a/cpp/cmake_modules/FindThrift.cmake}}
> {{+++ b/cpp/cmake_modules/FindThrift.cmake}}
> {{@@ -146,6 +146,7 @@ if(Thrift_FOUND)}}
> {{ endif()}}
> {{ set_target_properties(thrift::thrift}}
> {{ PROPERTIES IMPORTED_LOCATION "${THRIFT_LIB}"}}
> {{+ IMPORTED_IMPLIB "${THRIFT_LIB}"}}
> {{ INTERFACE_INCLUDE_DIRECTORIES
> "${THRIFT_INCLUDE_DIR}")}}
> {{ if(WIN32 AND NOT MSVC_TOOLCHAIN)}}
> {{ # We don't need this for Visual C++ because Thrift uses}}
> {quote}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18348) [CI][Release][Yum] redhat-rpm-config is needed on AlmaLinux 9
[
https://issues.apache.org/jira/browse/ARROW-18348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18348.
--
Resolution: Fixed
Issue resolved by pull request 14661
[https://github.com/apache/arrow/pull/14661]
> [CI][Release][Yum] redhat-rpm-config is needed on AlmaLinux 9
> -
>
> Key: ARROW-18348
> URL: https://issues.apache.org/jira/browse/ARROW-18348
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Continuous Integration, Packaging
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0, 10.0.2
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> https://github.com/ursacomputing/crossbow/actions/runs/3485133283/jobs/5830385419#step:7:1909
> {noformat}
> Building native extensions. This could take a while...
> ERROR: Error installing gobject-introspection:
> ERROR: Failed to build gem native extension.
>
> current directory: /usr/local/share/gems/gems/glib2-4.0.3/ext/glib2
> /usr/bin/ruby -I /usr/share/rubygems -r ./siteconf20221117-855-v8bktd.rb
> extconf.rb
> checking for --enable-debug-build option... no
> checking for -Wall option to compiler... *** extconf.rb failed ***
> Could not create Makefile due to some reason, probably lack of necessary
> libraries and/or headers. Check the mkmf.log file for more details. You
> may
> need configuration options.
>
> Provided configuration options:
> --with-opt-dir
> --without-opt-dir
> --with-opt-include
> --without-opt-include=${opt-dir}/include
> --with-opt-lib
> --without-opt-lib=${opt-dir}/lib64
> --with-make-prog
> --without-make-prog
> --srcdir=.
> --curdir
> --ruby=/usr/bin/$(RUBY_BASE_NAME)
> --enable-debug-build
> --disable-debug-build
> /usr/share/ruby/mkmf.rb:471:in `try_do': The compiler failed to generate an
> executable file. (RuntimeError)
> You have to install development tools first.
> from /usr/share/ruby/mkmf.rb:597:in `block in try_compile'
> from /usr/share/ruby/mkmf.rb:546:in `with_werror'
> from /usr/share/ruby/mkmf.rb:597:in `try_compile'
> from /usr/local/share/gems/gems/glib2-4.0.3/lib/mkmf-gnome.rb:65:in
> `block in try_compiler_option'
> from /usr/share/ruby/mkmf.rb:971:in `block in checking_for'
> from /usr/share/ruby/mkmf.rb:361:in `block (2 levels) in postpone'
> from /usr/share/ruby/mkmf.rb:331:in `open'
> from /usr/share/ruby/mkmf.rb:361:in `block in postpone'
> from /usr/share/ruby/mkmf.rb:331:in `open'
> from /usr/share/ruby/mkmf.rb:357:in `postpone'
> from /usr/share/ruby/mkmf.rb:970:in `checking_for'
> from /usr/local/share/gems/gems/glib2-4.0.3/lib/mkmf-gnome.rb:64:in
> `try_compiler_option'
> from /usr/local/share/gems/gems/glib2-4.0.3/lib/mkmf-gnome.rb:74:in
> `'
> from
> :85:in
> `require'
> from
> :85:in
> `require'
> from extconf.rb:27:in `'
>
> To see why this extension failed to compile, please check the mkmf.log
> which can be found here:
>
> /usr/local/lib64/gems/ruby/glib2-4.0.3/mkmf.log
>
> extconf failed, exit code 1
>
> Gem files will remain installed in /usr/local/share/gems/gems/glib2-4.0.3
> for inspection.
> Results logged to /usr/local/lib64/gems/ruby/glib2-4.0.3/gem_make.out
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18120) [Release][Dev] Automate running binaries/wheels verifications
[ https://issues.apache.org/jira/browse/ARROW-18120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18120. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14469 [https://github.com/apache/arrow/pull/14469] > [Release][Dev] Automate running binaries/wheels verifications > - > > Key: ARROW-18120 > URL: https://issues.apache.org/jira/browse/ARROW-18120 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > We have a script (02-source.sh) that runs source verifications. > But we don't have a script that runs binaries/wheels verifications yet. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-18348) [CI][Release][Yum] redhat-rpm-config is needed on AlmaLinux 9
Kouhei Sutou created ARROW-18348:
Summary: [CI][Release][Yum] redhat-rpm-config is needed on
AlmaLinux 9
Key: ARROW-18348
URL: https://issues.apache.org/jira/browse/ARROW-18348
Project: Apache Arrow
Issue Type: Improvement
Components: Continuous Integration, Packaging
Reporter: Kouhei Sutou
Assignee: Kouhei Sutou
Fix For: 10.0.2, 11.0.0
https://github.com/ursacomputing/crossbow/actions/runs/3485133283/jobs/5830385419#step:7:1909
{noformat}
Building native extensions. This could take a while...
ERROR: Error installing gobject-introspection:
ERROR: Failed to build gem native extension.
current directory: /usr/local/share/gems/gems/glib2-4.0.3/ext/glib2
/usr/bin/ruby -I /usr/share/rubygems -r ./siteconf20221117-855-v8bktd.rb
extconf.rb
checking for --enable-debug-build option... no
checking for -Wall option to compiler... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib64
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/bin/$(RUBY_BASE_NAME)
--enable-debug-build
--disable-debug-build
/usr/share/ruby/mkmf.rb:471:in `try_do': The compiler failed to generate an
executable file. (RuntimeError)
You have to install development tools first.
from /usr/share/ruby/mkmf.rb:597:in `block in try_compile'
from /usr/share/ruby/mkmf.rb:546:in `with_werror'
from /usr/share/ruby/mkmf.rb:597:in `try_compile'
from /usr/local/share/gems/gems/glib2-4.0.3/lib/mkmf-gnome.rb:65:in
`block in try_compiler_option'
from /usr/share/ruby/mkmf.rb:971:in `block in checking_for'
from /usr/share/ruby/mkmf.rb:361:in `block (2 levels) in postpone'
from /usr/share/ruby/mkmf.rb:331:in `open'
from /usr/share/ruby/mkmf.rb:361:in `block in postpone'
from /usr/share/ruby/mkmf.rb:331:in `open'
from /usr/share/ruby/mkmf.rb:357:in `postpone'
from /usr/share/ruby/mkmf.rb:970:in `checking_for'
from /usr/local/share/gems/gems/glib2-4.0.3/lib/mkmf-gnome.rb:64:in
`try_compiler_option'
from /usr/local/share/gems/gems/glib2-4.0.3/lib/mkmf-gnome.rb:74:in
`'
from
:85:in
`require'
from
:85:in
`require'
from extconf.rb:27:in `'
To see why this extension failed to compile, please check the mkmf.log which
can be found here:
/usr/local/lib64/gems/ruby/glib2-4.0.3/mkmf.log
extconf failed, exit code 1
Gem files will remain installed in /usr/local/share/gems/gems/glib2-4.0.3 for
inspection.
Results logged to /usr/local/lib64/gems/ruby/glib2-4.0.3/gem_make.out
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18121) [Release][CI] Use Ubuntu 22.04 for verifying binaries
[
https://issues.apache.org/jira/browse/ARROW-18121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-18121:
-
Fix Version/s: 10.0.2
> [Release][CI] Use Ubuntu 22.04 for verifying binaries
> -
>
> Key: ARROW-18121
> URL: https://issues.apache.org/jira/browse/ARROW-18121
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Continuous Integration
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 10.0.2, 11.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> APT/Yum verifications use Docker. If we use old libseccomp on host, some
> operations may cause errors:
> e.g.:
> https://github.com/ursacomputing/crossbow/actions/runs/3294870946/jobs/5432835953#step:7:5437
> {noformat}
> + valac --pkg arrow-glib --pkg posix build.vala
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18121) [Release][CI] Use Ubuntu 22.04 for verifying binaries
[
https://issues.apache.org/jira/browse/ARROW-18121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18121.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 14470
[https://github.com/apache/arrow/pull/14470]
> [Release][CI] Use Ubuntu 22.04 for verifying binaries
> -
>
> Key: ARROW-18121
> URL: https://issues.apache.org/jira/browse/ARROW-18121
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Continuous Integration
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> APT/Yum verifications use Docker. If we use old libseccomp on host, some
> operations may cause errors:
> e.g.:
> https://github.com/ursacomputing/crossbow/actions/runs/3294870946/jobs/5432835953#step:7:5437
> {noformat}
> + valac --pkg arrow-glib --pkg posix build.vala
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-18121) [Release][CI] Use Ubuntu 22.04 for verifying binaries
[
https://issues.apache.org/jira/browse/ARROW-18121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17635108#comment-17635108
]
Kouhei Sutou commented on ARROW-18121:
--
This problem was solved by https://github.com/apache/arrow/pull/14477 .
> [Release][CI] Use Ubuntu 22.04 for verifying binaries
> -
>
> Key: ARROW-18121
> URL: https://issues.apache.org/jira/browse/ARROW-18121
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Continuous Integration
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> APT/Yum verifications use Docker. If we use old libseccomp on host, some
> operations may cause errors:
> e.g.:
> https://github.com/ursacomputing/crossbow/actions/runs/3294870946/jobs/5432835953#step:7:5437
> {noformat}
> + valac --pkg arrow-glib --pkg posix build.vala
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> error: Failed to close file descriptor for child process (Operation not
> permitted)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18222) [Release][MSYS2] Detect reverse dependencies automatically
[ https://issues.apache.org/jira/browse/ARROW-18222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18222. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14565 [https://github.com/apache/arrow/pull/14565] > [Release][MSYS2] Detect reverse dependencies automatically > -- > > Key: ARROW-18222 > URL: https://issues.apache.org/jira/browse/ARROW-18222 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Minor > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18223) [Release][Homebrew] Detect reverse dependencies automatically
[ https://issues.apache.org/jira/browse/ARROW-18223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18223. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14566 [https://github.com/apache/arrow/pull/14566] > [Release][Homebrew] Detect reverse dependencies automatically > - > > Key: ARROW-18223 > URL: https://issues.apache.org/jira/browse/ARROW-18223 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ARROW-18120) [Release][Dev] Automate running binaries/wheels verifications
[ https://issues.apache.org/jira/browse/ARROW-18120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou updated ARROW-18120: - Summary: [Release][Dev] Automate running binaries/wheels verifications (was: [Release][Dev] Run binaries/wheels verifications in 05-binary-upload.sh) > [Release][Dev] Automate running binaries/wheels verifications > - > > Key: ARROW-18120 > URL: https://issues.apache.org/jira/browse/ARROW-18120 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > We have a script (02-source.sh) that runs source verifications. > But we don't have a script that runs binaries/wheels verifications yet. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ARROW-18340) [Python] PyArrow C++ header files no longer always included in installed pyarrow
[
https://issues.apache.org/jira/browse/ARROW-18340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-18340:
-
Fix Version/s: 10.0.2
(was: 10.0.1)
> [Python] PyArrow C++ header files no longer always included in installed
> pyarrow
>
>
> Key: ARROW-18340
> URL: https://issues.apache.org/jira/browse/ARROW-18340
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
>Affects Versions: 10.0.0
>Reporter: Joris Van den Bossche
>Assignee: Alenka Frim
>Priority: Major
> Labels: pull-request-available
> Fix For: 10.0.2, 11.0.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> We have a python build env var to control whether the Arrow C++ header files
> are included in the python package or not
> ({{PYARROW_BUNDLE_ARROW_CPP_HEADERS}}). This is set to True by default, and
> only in the conda recipe set to False.
> After the cmake refactor, the Python C++ header files no longer live in the
> Arrow C++ package, and so should _always_ be included in the python package,
> regardless of how arrow-cpp is installed.
> Initially this was done, but it seems that
> https://github.com/apache/arrow/pull/13892 removed this unconditional copy of
> the PyArrow header files to {{pyarrow/include}}. Now it is only copied if
> {{PYARROW_BUNDLE_ARROW_CPP_HEADERS}} is enabled.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18336) [Release][Docs] Don't update versions not in major release
[ https://issues.apache.org/jira/browse/ARROW-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou updated ARROW-18336: - Fix Version/s: 10.0.2 (was: 10.0.1) > [Release][Docs] Don't update versions not in major release > -- > > Key: ARROW-18336 > URL: https://issues.apache.org/jira/browse/ARROW-18336 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 10.0.2, 11.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18235) [C++][Gandiva] Fix the like function implementation for escape chars
[ https://issues.apache.org/jira/browse/ARROW-18235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18235. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14579 [https://github.com/apache/arrow/pull/14579] > [C++][Gandiva] Fix the like function implementation for escape chars > > > Key: ARROW-18235 > URL: https://issues.apache.org/jira/browse/ARROW-18235 > Project: Apache Arrow > Issue Type: New Feature > Components: C++ - Gandiva >Reporter: Siddhant Rao >Assignee: Siddhant Rao >Priority: Minor > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 2h > Remaining Estimate: 0h > > The current conversion of sql like patterns to pcre doesn't work if the > escape character is one of the special characters to pcre. Fix this by not > escaping those characters if it the escape character itself. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (ARROW-18129) [Python] get_include() gives wrong directory in conda environment
[
https://issues.apache.org/jira/browse/ARROW-18129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634821#comment-17634821
]
Kouhei Sutou edited comment on ARROW-18129 at 11/16/22 8:11 PM:
Yes, in case of using conda the Arrow C++ headers are not found in the python
package ({{{}pyarrow/include){}}} due to conda recipe using
{{{}PYARROW_BUNDLE_ARROW_CPP_HEADERS=0{}}}, see:
[https://github.com/apache/arrow/blob/16fd7f3fabbe663c161899d197deb66a7ec188b5/dev/tasks/conda-recipes/arrow-cpp/build-pyarrow.sh#L11]
With Apache Arrow 10.0.0 the C++ code connected to PyArrow (PyArrow C\+\+) has
been moved to the python package and so the header files for PyArrow C\+\+
should always be included in the package itself. To track that work we created
another issue https://issues.apache.org/jira/browse/ARROW-18340.
Regarding the headers for Arrow C\+\+: we should add info in the
documentation/docstrings to make it clear that the Arrow C\+\+ header files
are not located in the python package if one is using conda.
We could also add {{$CONDA_PREFIX/include/arrow}} path in the {{get_include()}}
function if one is using conda. [~apitrou] what is your opinion on that?
was (Author: alenkaf):
Yes, in case of using conda the Arrow C++ headers are not found in the python
package ({{{}pyarrow/include){}}} due to conda recipe using
{{{}PYARROW_BUNDLE_ARROW_CPP_HEADERS=0{}}}, see:
[https://github.com/apache/arrow/blob/16fd7f3fabbe663c161899d197deb66a7ec188b5/dev/tasks/conda-recipes/arrow-cpp/build-pyarrow.sh#L11]
With Apache Arrow 10.0.0 the C++ code connected to PyArrow (PyArrow C++) has
been moved to the python package and so the header files for PyArrow C++ should
always be included in the package itself. To track that work we created another
issue https://issues.apache.org/jira/browse/ARROW-18340.
Regarding the headers for Arrow C++: we should add info in the
documentation/docstrings to make it clear that the Arrow C+ header files are
not located in the python package if one is using conda.
We could also add {{$CONDA_PREFIX/include/arrow}} path in the {{get_include()}}
function if one is using conda. [~apitrou] what is your opinion on that?
> [Python] get_include() gives wrong directory in conda environment
> -
>
> Key: ARROW-18129
> URL: https://issues.apache.org/jira/browse/ARROW-18129
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 9.0.0
> Environment: conda
>Reporter: Left Screen
>Priority: Critical
> Labels: triaged
>
> {{get_include}} seems to do:
>
> {code:java}
> def get_include():
> """
> Return absolute path to directory containing Arrow C++ include
> headers. Similar to numpy.get_include
> """
> return _os.path.join(_os.path.dirname(__file__), 'include') {code}
> This returns something like:
> {code:java}
> /path/to/myconda/envs/envname/lib/python3.8/site-packages/pyarrow/include{code}
> which does not exist in a conda environment. The path where the headers
> actually get installed is to:
>
> {code:java}
> $ echo $CONDA_PREFIX
> /path/to/myconda/envs/envname
> $ ls $CONDA_PREFIX/include/arrow | head
> adapters
> api.h
> array
> array.h
> buffer_builder.h
> buffer.h
> builder.h
> c
> chunked_array.h
> chunk_resolver.h
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18335) [CI][Release][JS] verify-rc-source-js is failed with Node.js 18
[
https://issues.apache.org/jira/browse/ARROW-18335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18335.
--
Fix Version/s: 11.0.0
10.0.1
Resolution: Fixed
Issue resolved by pull request 14652
[https://github.com/apache/arrow/pull/14652]
> [CI][Release][JS] verify-rc-source-js is failed with Node.js 18
> ---
>
> Key: ARROW-18335
> URL: https://issues.apache.org/jira/browse/ARROW-18335
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Continuous Integration, JavaScript
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0, 10.0.1
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> https://github.com/ursacomputing/crossbow/actions/runs/3470647227/jobs/5799108266
> {noformat}
> $ gulp bundle
> [13:54:58] Using gulpfile /arrow/js/gulpfile.js
> [13:54:58] Starting 'bundle'...
> [13:54:58] Starting 'bundle:clean'...
> [13:54:58] Finished 'bundle:clean' after 25 ms
> [13:54:58] Starting 'bundle:esbuild'...
> [13:54:58] field-bundle.js: 195.51 kB (gzipped: 47.27 kB)
> [13:54:58] makeTable-bundle.js: 197.11 kB (gzipped: 47.57 kB)
> [13:54:58] makeVector-bundle.js: 197.02 kB (gzipped: 47.54 kB)
> [13:54:58] schema-bundle.js: 195.51 kB (gzipped: 47.27 kB)
> [13:54:58] table-bundle.js: 195.51 kB (gzipped: 47.27 kB)
> [13:54:58] tableFromArrays-bundle.js: 198.74 kB (gzipped: 48.2 kB)
> [13:54:58] tableFromIPC-bundle.js: 196.75 kB (gzipped: 47.54 kB)
> [13:54:58] vector-bundle.js: 195.51 kB (gzipped: 47.27 kB)
> [13:54:58] vectorFromArray-bundle.js: 198.66 kB (gzipped: 48.15 kB)
> [13:54:58] Finished 'bundle:esbuild' after 430 ms
> [13:54:58] Starting 'bundle:rollup'...
> [13:55:18] field-bundle.js: 799 B (gzipped: 365 B)
> [13:55:18] makeTable-bundle.js: 85.62 kB (gzipped: 18.67 kB)
> [13:55:18] makeVector-bundle.js: 69.37 kB (gzipped: 15.21 kB)
> [13:55:18] schema-bundle.js: 12.27 kB (gzipped: 3.16 kB)
> [13:55:18] table-bundle.js: 82.19 kB (gzipped: 18.14 kB)
> [13:55:18] tableFromArrays-bundle.js: 105.2 kB (gzipped: 23.27 kB)
> [13:55:18] tableFromIPC-bundle.js: 188.2 kB (gzipped: 39.54 kB)
> [13:55:18] vector-bundle.js: 65.99 kB (gzipped: 14.69 kB)
> [13:55:18] vectorFromArray-bundle.js: 94.33 kB (gzipped: 20.48 kB)
> [13:55:18] Finished 'bundle:rollup' after 20 s
> [13:55:18] Starting 'bundle:webpack'...
> [13:55:19] 'bundle:webpack' errored after 439 ms
> [13:55:19] Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
> at new NodeError (node:internal/errors:393:5)
> at Stream. (node:internal/streams/pipeline:352:14)
> at Stream.emit (node:events:525:35)
> at Stream.emit (node:domain:552:15)
> at stream.destroy (/arrow/js/node_modules/through/index.js:84:12)
> at _end (/arrow/js/node_modules/through/index.js:67:14)
> at stream.end (/arrow/js/node_modules/through/index.js:74:5)
> at DestroyableTransform.onend
> (/arrow/js/node_modules/readable-stream/lib/_stream_readable.js:577:10)
> at Object.onceWrapper (node:events:627:28)
> at DestroyableTransform.emit (node:events:525:35)
> [13:55:19] 'bundle' errored after 21 s
> error Command failed with exit code 1.
> info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this
> command.
> Failed to verify release candidate. See /tmp/arrow-HEAD.ksyeB for details.
> 1
> Error: `docker-compose --file
> /home/runner/work/crossbow/crossbow/arrow/docker-compose.yml run --rm -e
> VERIFY_VERSION= -e VERIFY_RC= -e TEST_DEFAULT=0 -e TEST_JS=1
> ubuntu-verify-rc` exited with a non-zero exit code 1, see the process log
> above.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (ARROW-18232) [Release][macOS][wheel] Can't disable GCS/S3 tests
[ https://issues.apache.org/jira/browse/ARROW-18232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou closed ARROW-18232. Resolution: Not A Problem > [Release][macOS][wheel] Can't disable GCS/S3 tests > -- > > Key: ARROW-18232 > URL: https://issues.apache.org/jira/browse/ARROW-18232 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18233) [Release][JS] don't install yarn to system
[ https://issues.apache.org/jira/browse/ARROW-18233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18233. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14577 [https://github.com/apache/arrow/pull/14577] > [Release][JS] don't install yarn to system > -- > > Key: ARROW-18233 > URL: https://issues.apache.org/jira/browse/ARROW-18233 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18259) [C++][CMake] Add support for system Thrift CMake package
[ https://issues.apache.org/jira/browse/ARROW-18259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18259. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14597 [https://github.com/apache/arrow/pull/14597] > [C++][CMake] Add support for system Thrift CMake package > > > Key: ARROW-18259 > URL: https://issues.apache.org/jira/browse/ARROW-18259 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18287) [C++][CMake] Add support for Brotli/utf8proc provided by vcpkg
[ https://issues.apache.org/jira/browse/ARROW-18287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18287. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14609 [https://github.com/apache/arrow/pull/14609] > [C++][CMake] Add support for Brotli/utf8proc provided by vcpkg > -- > > Key: ARROW-18287 > URL: https://issues.apache.org/jira/browse/ARROW-18287 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > vcpkg provides CMake packages for Brotli and utf8proc with "unofficial-" > prefix. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18289) [Release][vcpkg] Add a script to update vcpkg's arrow port
[ https://issues.apache.org/jira/browse/ARROW-18289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18289. -- Fix Version/s: 11.0.0 Resolution: Fixed Issue resolved by pull request 14610 [https://github.com/apache/arrow/pull/14610] > [Release][vcpkg] Add a script to update vcpkg's arrow port > -- > > Key: ARROW-18289 > URL: https://issues.apache.org/jira/browse/ARROW-18289 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (ARROW-18278) [Java] Maven generate-libs-jni-macos-linux on M1 fails due to cmake error
[
https://issues.apache.org/jira/browse/ARROW-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18278.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 14623
[https://github.com/apache/arrow/pull/14623]
> [Java] Maven generate-libs-jni-macos-linux on M1 fails due to cmake error
> -
>
> Key: ARROW-18278
> URL: https://issues.apache.org/jira/browse/ARROW-18278
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Java
>Reporter: Rok Mihevc
>Assignee: Rok Mihevc
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> When building with maven on M1 [as per
> docs|https://arrow.apache.org/docs/dev/developers/java/building.html#id3]:
> {code:bash}
> mvn clean install
> mvn generate-resources -Pgenerate-libs-jni-macos-linux -N
> {code}
> I get the following error:
> {code:bash}
> [INFO] --- exec-maven-plugin:3.1.0:exec (jni-cmake) @ arrow-java-root ---
> -- Building using CMake version: 3.24.2
> -- The C compiler identification is AppleClang 14.0.0.1429
> -- The CXX compiler identification is AppleClang 14.0.0.1429
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working C compiler:
> /Library/Developer/CommandLineTools/usr/bin/cc - skipped
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Check for working CXX compiler:
> /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- Found Java:
> /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/bin/java (found
> version "11.0.16")
> -- Found JNI:
> /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/include found
> components: AWT JVM
> CMake Error at dataset/CMakeLists.txt:18 (find_package):
> By not providing "FindArrowDataset.cmake" in CMAKE_MODULE_PATH this project
> has asked CMake to find a package configuration file provided by
> "ArrowDataset", but CMake did not find one.
> Could not find a package configuration file provided by "ArrowDataset" with
> any of the following names:
> ArrowDatasetConfig.cmake
> arrowdataset-config.cmake
> Add the installation prefix of "ArrowDataset" to CMAKE_PREFIX_PATH or set
> "ArrowDataset_DIR" to a directory containing one of the above files. If
> "ArrowDataset" provides a separate development package or SDK, be sure it
> has been installed.
> -- Configuring incomplete, errors occurred!
> See also
> "/Users/rok/Documents/repos/arrow/java-jni/CMakeFiles/CMakeOutput.log".
> See also
> "/Users/rok/Documents/repos/arrow/java-jni/CMakeFiles/CMakeError.log".
> [ERROR] Command execution failed.
> org.apache.commons.exec.ExecuteException: Process exited with an error: 1
> (Exit value: 1)
> at org.apache.commons.exec.DefaultExecutor.executeInternal
> (DefaultExecutor.java:404)
> at org.apache.commons.exec.DefaultExecutor.execute
> (DefaultExecutor.java:166)
> at org.codehaus.mojo.exec.ExecMojo.executeCommandLine (ExecMojo.java:1000)
> at org.codehaus.mojo.exec.ExecMojo.executeCommandLine (ExecMojo.java:947)
> at org.codehaus.mojo.exec.ExecMojo.execute (ExecMojo.java:471)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo
> (DefaultBuildPluginManager.java:137)
> at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2
> (MojoExecutor.java:370)
> at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute
> (MojoExecutor.java:351)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:215)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:171)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:163)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:81)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:56)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:294)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:960)
> at
[jira] [Resolved] (ARROW-18336) [Release][Docs] Don't update versions not in major release
[ https://issues.apache.org/jira/browse/ARROW-18336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou resolved ARROW-18336. -- Fix Version/s: 11.0.0 10.0.1 Resolution: Fixed Issue resolved by pull request 14653 [https://github.com/apache/arrow/pull/14653] > [Release][Docs] Don't update versions not in major release > -- > > Key: ARROW-18336 > URL: https://issues.apache.org/jira/browse/ARROW-18336 > Project: Apache Arrow > Issue Type: Improvement > Components: Developer Tools >Reporter: Kouhei Sutou >Assignee: Kouhei Sutou >Priority: Major > Labels: pull-request-available > Fix For: 11.0.0, 10.0.1 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-18336) [Release][Docs] Don't update versions not in major release
Kouhei Sutou created ARROW-18336: Summary: [Release][Docs] Don't update versions not in major release Key: ARROW-18336 URL: https://issues.apache.org/jira/browse/ARROW-18336 Project: Apache Arrow Issue Type: Improvement Components: Developer Tools Reporter: Kouhei Sutou Assignee: Kouhei Sutou -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ARROW-18335) [CI][Release][JS] verify-rc-source-js is failed with Node.js 18
Kouhei Sutou created ARROW-18335:
Summary: [CI][Release][JS] verify-rc-source-js is failed with
Node.js 18
Key: ARROW-18335
URL: https://issues.apache.org/jira/browse/ARROW-18335
Project: Apache Arrow
Issue Type: Improvement
Components: Continuous Integration, JavaScript
Reporter: Kouhei Sutou
Assignee: Kouhei Sutou
https://github.com/ursacomputing/crossbow/actions/runs/3470647227/jobs/5799108266
{noformat}
$ gulp bundle
[13:54:58] Using gulpfile /arrow/js/gulpfile.js
[13:54:58] Starting 'bundle'...
[13:54:58] Starting 'bundle:clean'...
[13:54:58] Finished 'bundle:clean' after 25 ms
[13:54:58] Starting 'bundle:esbuild'...
[13:54:58] field-bundle.js: 195.51 kB (gzipped: 47.27 kB)
[13:54:58] makeTable-bundle.js: 197.11 kB (gzipped: 47.57 kB)
[13:54:58] makeVector-bundle.js: 197.02 kB (gzipped: 47.54 kB)
[13:54:58] schema-bundle.js: 195.51 kB (gzipped: 47.27 kB)
[13:54:58] table-bundle.js: 195.51 kB (gzipped: 47.27 kB)
[13:54:58] tableFromArrays-bundle.js: 198.74 kB (gzipped: 48.2 kB)
[13:54:58] tableFromIPC-bundle.js: 196.75 kB (gzipped: 47.54 kB)
[13:54:58] vector-bundle.js: 195.51 kB (gzipped: 47.27 kB)
[13:54:58] vectorFromArray-bundle.js: 198.66 kB (gzipped: 48.15 kB)
[13:54:58] Finished 'bundle:esbuild' after 430 ms
[13:54:58] Starting 'bundle:rollup'...
[13:55:18] field-bundle.js: 799 B (gzipped: 365 B)
[13:55:18] makeTable-bundle.js: 85.62 kB (gzipped: 18.67 kB)
[13:55:18] makeVector-bundle.js: 69.37 kB (gzipped: 15.21 kB)
[13:55:18] schema-bundle.js: 12.27 kB (gzipped: 3.16 kB)
[13:55:18] table-bundle.js: 82.19 kB (gzipped: 18.14 kB)
[13:55:18] tableFromArrays-bundle.js: 105.2 kB (gzipped: 23.27 kB)
[13:55:18] tableFromIPC-bundle.js: 188.2 kB (gzipped: 39.54 kB)
[13:55:18] vector-bundle.js: 65.99 kB (gzipped: 14.69 kB)
[13:55:18] vectorFromArray-bundle.js: 94.33 kB (gzipped: 20.48 kB)
[13:55:18] Finished 'bundle:rollup' after 20 s
[13:55:18] Starting 'bundle:webpack'...
[13:55:19] 'bundle:webpack' errored after 439 ms
[13:55:19] Error [ERR_STREAM_PREMATURE_CLOSE]: Premature close
at new NodeError (node:internal/errors:393:5)
at Stream. (node:internal/streams/pipeline:352:14)
at Stream.emit (node:events:525:35)
at Stream.emit (node:domain:552:15)
at stream.destroy (/arrow/js/node_modules/through/index.js:84:12)
at _end (/arrow/js/node_modules/through/index.js:67:14)
at stream.end (/arrow/js/node_modules/through/index.js:74:5)
at DestroyableTransform.onend
(/arrow/js/node_modules/readable-stream/lib/_stream_readable.js:577:10)
at Object.onceWrapper (node:events:627:28)
at DestroyableTransform.emit (node:events:525:35)
[13:55:19] 'bundle' errored after 21 s
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this
command.
Failed to verify release candidate. See /tmp/arrow-HEAD.ksyeB for details.
1
Error: `docker-compose --file
/home/runner/work/crossbow/crossbow/arrow/docker-compose.yml run --rm -e
VERIFY_VERSION= -e VERIFY_RC= -e TEST_DEFAULT=0 -e TEST_JS=1 ubuntu-verify-rc`
exited with a non-zero exit code 1, see the process log above.
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ARROW-18331) [Packaging][Release] Add conan to post-release tasks and publish 9.0.0 and 10.0.0
[
https://issues.apache.org/jira/browse/ARROW-18331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17634539#comment-17634539
]
Kouhei Sutou commented on ARROW-18331:
--
I worked on removing patches from Conan as much as possible. And all of them
are included in 10.0.0. (See also: ARROW-16089)
So we can do this now. :-)
We create a post release script for Conan into {{dev/release/}} and run it to
open a pull request to update the arrow Conan recipe like we did for
MSYS2/Homebrew/vcpkg.
> [Packaging][Release] Add conan to post-release tasks and publish 9.0.0 and
> 10.0.0
> -
>
> Key: ARROW-18331
> URL: https://issues.apache.org/jira/browse/ARROW-18331
> Project: Apache Arrow
> Issue Type: Bug
> Components: Packaging
>Affects Versions: 9.0.0, 10.0.0
>Reporter: Raúl Cumplido
>Priority: Major
> Fix For: 11.0.0
>
>
> It does seem we have not updated conan recipes since 8.0.1:
> https://github.com/conan-io/conan-center-index/tree/master/recipes/arrow
> That's why the latest version of arrow appears to be 8.0.1 here:
> https://conan.io/center/arrow
> Is this something that we should do as part of post-release tasks? Should we
> add 9.0.0 and 10.0.0?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (ARROW-18322) [Python] source installation from pip without flight
[
https://issues.apache.org/jira/browse/ARROW-18322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou resolved ARROW-18322.
--
Fix Version/s: 11.0.0
Resolution: Fixed
Issue resolved by pull request 14642
[https://github.com/apache/arrow/pull/14642]
> [Python] source installation from pip without flight
>
>
> Key: ARROW-18322
> URL: https://issues.apache.org/jira/browse/ARROW-18322
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 10.0.0
>Reporter: Alan Snow
>Assignee: Alenka Frim
>Priority: Major
> Labels: pull-request-available
> Fix For: 11.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} ARROW_FLIGHT
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} ARROW_GANDIVA
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} ARROW_PLASMA
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} PYARROW_WITH_FLIGHT
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} PYARROW_WITH_GANDIVA
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} PYARROW_WITH_PLASMA
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#ce9178}python -m pip install pyarrow{color}}}
> {{ }}
> {{{color:#ce9178}...{color}}}
> {{ }}
> {{CMake Error at CMakeLists.txt:262 (find_package):}}
> {{ }}
> {{By not providing "FindArrowFlight.cmake" in CMAKE_MODULE_PATH this project}}
> {{ }}
> {{has asked CMake to find a package configuration file provided by}}
> {{ }}
> {{"ArrowFlight", but CMake did not find one.}}
> {{ }}
> {{ }}
> {{ }}
> {{Could not find a package configuration file provided by "ArrowFlight" with}}
> {{ }}
> {{any of the following names:}}
> {{ }}
> {{ }}
> {{ }}
> {{ArrowFlightConfig.cmake}}
> {{ }}
> {{arrowflight-config.cmake}}
> {{ }}
> {{Add the installation prefix of "ArrowFlight" to CMAKE_PREFIX_PATH or set}}
> {{ }}
> {{"ArrowFlight_DIR" to a directory containing one of the above files. If}}
> {{ }}
> {{"ArrowFlight" provides a separate development package or SDK, be sure it}}
> {{ {color:#ce9178}{\{ has been installed.}}{color}}}
> {{ }}
> OS: ubuntu 22
> Python: 3.9
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18322) [Python] source installation from pip without flight
[
https://issues.apache.org/jira/browse/ARROW-18322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-18322:
-
Fix Version/s: 10.0.1
> [Python] source installation from pip without flight
>
>
> Key: ARROW-18322
> URL: https://issues.apache.org/jira/browse/ARROW-18322
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 10.0.0
>Reporter: Alan Snow
>Assignee: Alenka Frim
>Priority: Major
> Labels: pull-request-available
> Fix For: 10.0.1, 11.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} ARROW_FLIGHT
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} ARROW_GANDIVA
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} ARROW_PLASMA
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} PYARROW_WITH_FLIGHT
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} PYARROW_WITH_GANDIVA
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#569cd6}ENV{color}{color:#d4d4d4} PYARROW_WITH_PLASMA
> {color}{color:#ce9178}'OFF'{color}}}
> {{{color:#ce9178}python -m pip install pyarrow{color}}}
> {{ }}
> {{{color:#ce9178}...{color}}}
> {{ }}
> {{CMake Error at CMakeLists.txt:262 (find_package):}}
> {{ }}
> {{By not providing "FindArrowFlight.cmake" in CMAKE_MODULE_PATH this project}}
> {{ }}
> {{has asked CMake to find a package configuration file provided by}}
> {{ }}
> {{"ArrowFlight", but CMake did not find one.}}
> {{ }}
> {{ }}
> {{ }}
> {{Could not find a package configuration file provided by "ArrowFlight" with}}
> {{ }}
> {{any of the following names:}}
> {{ }}
> {{ }}
> {{ }}
> {{ArrowFlightConfig.cmake}}
> {{ }}
> {{arrowflight-config.cmake}}
> {{ }}
> {{Add the installation prefix of "ArrowFlight" to CMAKE_PREFIX_PATH or set}}
> {{ }}
> {{"ArrowFlight_DIR" to a directory containing one of the above files. If}}
> {{ }}
> {{"ArrowFlight" provides a separate development package or SDK, be sure it}}
> {{ {color:#ce9178}{\{ has been installed.}}{color}}}
> {{ }}
> OS: ubuntu 22
> Python: 3.9
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-17635) [Python][CI] Sync conda recipe with the arrow-cpp feedstock
[
https://issues.apache.org/jira/browse/ARROW-17635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-17635:
-
Fix Version/s: 10.0.1
> [Python][CI] Sync conda recipe with the arrow-cpp feedstock
> ---
>
> Key: ARROW-17635
> URL: https://issues.apache.org/jira/browse/ARROW-17635
> Project: Apache Arrow
> Issue Type: Task
> Components: Continuous Integration, Python
>Reporter: Antoine Pitrou
>Assignee: H. Vetinari
>Priority: Critical
> Labels: pull-request-available
> Fix For: 10.0.1, 11.0.0
>
> Time Spent: 12.5h
> Remaining Estimate: 0h
>
> As explained in the README below, some files in the conda recipe must be
> sync'ed with the "upstream" conda-forge {{arrow-cpp}} feedstock:
> https://github.com/apache/arrow/blob/a5ecb0ff0774805b0f912e231eaedf42e7194c36/dev/tasks/conda-recipes/README.md
> I'll also note that the "upstream" build matrices have become huge and we
> should only enable a couple of these jobs to avoid blowing up our own CI
> resources:
> https://github.com/conda-forge/arrow-cpp-feedstock/tree/main/.azure-pipelines
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18080) [C++] Remove gcc <= 4.9 workarounds
[ https://issues.apache.org/jira/browse/ARROW-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou updated ARROW-18080: - Fix Version/s: 10.0.1 > [C++] Remove gcc <= 4.9 workarounds > --- > > Key: ARROW-18080 > URL: https://issues.apache.org/jira/browse/ARROW-18080 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Neal Richardson >Assignee: Neal Richardson >Priority: Major > Labels: pull-request-available > Fix For: 10.0.1, 11.0.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Since we require gcc 7ish or greater now that we're on C++17 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ARROW-18162) [C++] Add Arm SVE compiler options
[
https://issues.apache.org/jira/browse/ARROW-18162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-18162:
-
Fix Version/s: 10.0.1
> [C++] Add Arm SVE compiler options
> --
>
> Key: ARROW-18162
> URL: https://issues.apache.org/jira/browse/ARROW-18162
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Yibo Cai
>Assignee: Yibo Cai
>Priority: Major
> Labels: pull-request-available
> Fix For: 10.0.1, 11.0.0
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> {{xsimd}} 9.0+ supports Arm SVE (fixed size). Some additional compiler
> options are required to enable SVE.
> Per my test on Amazon Graviton3 (SVE-256). SVE256 performs much better than
> NEON for some cases. E.g., utf8 benchmark {{ValidateLargeAscii}} improves
> from *38.6* (NEON) to *51.5* (SVE256) GB/s.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18260) [C++][CMake] Add support for x64 for CMAKE_SYSTEM_PROCESSOR
[
https://issues.apache.org/jira/browse/ARROW-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kouhei Sutou updated ARROW-18260:
-
Fix Version/s: 10.0.1
> [C++][CMake] Add support for x64 for CMAKE_SYSTEM_PROCESSOR
> ---
>
> Key: ARROW-18260
> URL: https://issues.apache.org/jira/browse/ARROW-18260
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Kouhei Sutou
>Assignee: Kouhei Sutou
>Priority: Major
> Labels: pull-request-available
> Fix For: 10.0.1, 11.0.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> vcpkg uses {{x64}}:
> https://vcpkg.readthedocs.io/en/latest/users/triplets/
> bq. Valid options are x86, x64, arm, arm64 and wasm32.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ARROW-18054) [Python][CI] Enable Cython tests on windows wheels
[ https://issues.apache.org/jira/browse/ARROW-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou updated ARROW-18054: - Fix Version/s: 10.0.1 > [Python][CI] Enable Cython tests on windows wheels > -- > > Key: ARROW-18054 > URL: https://issues.apache.org/jira/browse/ARROW-18054 > Project: Apache Arrow > Issue Type: Task > Components: Continuous Integration, Python >Reporter: Raúl Cumplido >Assignee: Raúl Cumplido >Priority: Major > Labels: pull-request-available > Fix For: 10.0.1, 11.0.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > We currently have `set PYARROW_TEST_CYTHON=OFF` on the windows wheel tests. > We should run the cython tests for windows too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
