[jira] [Created] (ARROW-7049) warnings building on mingw-w64
Jeroen created ARROW-7049:
-
Summary: warnings building on mingw-w64
Key: ARROW-7049
URL: https://issues.apache.org/jira/browse/ARROW-7049
Project: Apache Arrow
Issue Type: Bug
Components: C++
Affects Versions: 0.15.1
Reporter: Jeroen
Two warnings when building libarrow 0.15.1 on mingw-w64:
{code}
[ 2%] Running thrift compiler on parquet.thrift
[WARNING:C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/parquet.thrift:297]
The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
signedness of this type.
{code}
And later:
{code}
81%] Building CXX object
src/parquet/CMakeFiles/parquet_static.dir/column_reader.cc.obj
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:
In member function 'virtual arrow::Status
parquet::arrow::FileWriterImpl::WriteColumnChunk(const
std::shared_ptr&, int64_t, int64_t)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:79:41:
warning: 'schema_field' may be used uninitialized in this function
[-Wmaybe-uninitialized]
schema_manifest_(schema_manifest) {}
^
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:466:24:
note: 'schema_field' was declared here
const SchemaField* schema_field;
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ARROW-7049) [C++] warnings building on mingw-w64
[
https://issues.apache.org/jira/browse/ARROW-7049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen updated ARROW-7049:
--
Description:
Two warnings when building libarrow 0.15.1 on mingw-w64:
{code}
[ 2%] Running thrift compiler on parquet.thrift
[WARNING:C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/parquet.thrift:297]
The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
signedness of this type.
{code}
And later:
{code}
81%] Building CXX object
src/parquet/CMakeFiles/parquet_static.dir/column_reader.cc.obj
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:
In member function 'virtual arrow::Status
parquet::arrow::FileWriterImpl::WriteColumnChunk(const
std::shared_ptr&, int64_t, int64_t)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:79:41:
warning: 'schema_field' may be used uninitialized in this function
[-Wmaybe-uninitialized]
schema_manifest_(schema_manifest) {}
^
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:466:24:
note: 'schema_field' was declared here
const SchemaField* schema_field;
{code}
Maybe CI with `CXXFLAGS += -Werror` ?
was:
Two warnings when building libarrow 0.15.1 on mingw-w64:
{code}
[ 2%] Running thrift compiler on parquet.thrift
[WARNING:C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/parquet.thrift:297]
The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
signedness of this type.
{code}
And later:
{code}
81%] Building CXX object
src/parquet/CMakeFiles/parquet_static.dir/column_reader.cc.obj
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:
In member function 'virtual arrow::Status
parquet::arrow::FileWriterImpl::WriteColumnChunk(const
std::shared_ptr&, int64_t, int64_t)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:79:41:
warning: 'schema_field' may be used uninitialized in this function
[-Wmaybe-uninitialized]
schema_manifest_(schema_manifest) {}
^
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:466:24:
note: 'schema_field' was declared here
const SchemaField* schema_field;
{code}
> [C++] warnings building on mingw-w64
>
>
> Key: ARROW-7049
> URL: https://issues.apache.org/jira/browse/ARROW-7049
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.15.1
>Reporter: Jeroen
>Priority: Minor
>
> Two warnings when building libarrow 0.15.1 on mingw-w64:
> {code}
> [ 2%] Running thrift compiler on parquet.thrift
> [WARNING:C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/parquet.thrift:297]
> The "byte" type is a compatibility alias for "i8". Use "i8" to emphasize the
> signedness of this type.
> {code}
> And later:
> {code}
> 81%] Building CXX object
> src/parquet/CMakeFiles/parquet_static.dir/column_reader.cc.obj
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:
> In member function 'virtual arrow::Status
> parquet::arrow::FileWriterImpl::WriteColumnChunk(const
> std::shared_ptr&, int64_t, int64_t)':
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:79:41:
> warning: 'schema_field' may be used uninitialized in this function
> [-Wmaybe-uninitialized]
> schema_manifest_(schema_manifest) {}
> ^
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/apache-arrow-0.15.1/cpp/src/parquet/arrow/writer.cc:466:24:
> note: 'schema_field' was declared here
> const SchemaField* schema_field;
> {code}
> Maybe CI with `CXXFLAGS += -Werror` ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-7050) [R] Compiler warning in R bindings
Jeroen created ARROW-7050:
-
Summary: [R] Compiler warning in R bindings
Key: ARROW-7050
URL: https://issues.apache.org/jira/browse/ARROW-7050
Project: Apache Arrow
Issue Type: Bug
Components: R
Affects Versions: 0.15.1
Reporter: Jeroen
As reported by gcc 8.3.0 on Windows. Maybe you should CI with -Werror?
{code}
array_from_vector.cpp: In member function 'arrow::Status
arrow::r::TypedVectorConverter::Ingest(SEXP) [with Type =
arrow::DoubleType; Derived =
arrow::r::NumericVectorConverter]':
array_from_vector.cpp:383:16: warning: 'value' may be used uninitialized in
this function [-Wmaybe-uninitialized]
double value;
^
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-5849) Compiler warnings on mingw-w64
Jeroen created ARROW-5849:
-
Summary: Compiler warnings on mingw-w64
Key: ARROW-5849
URL: https://issues.apache.org/jira/browse/ARROW-5849
Project: Apache Arrow
Issue Type: Bug
Components: C++
Affects Versions: 0.14.0
Reporter: Jeroen
In mingw64 we see the following warnings:
{code}
[ 54%] Building CXX object
src/arrow/CMakeFiles/arrow_static.dir/util/io-util.cc.obj
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/decimal.cc:
In static member function 'static arrow::Status
arrow::Decimal128::FromString(const string_view&, arrow::Decimal128*, int32_t*,
int32_t*)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/decimal.cc:313:35:
warning: 'dec.arrow::{anonymous}::DecimalComponents::exponent' may be used
uninitialized in this function [-Wmaybe-uninitialized]
*scale = -adjusted_exponent + len - 1;
~~~^~~~
{code}
{code}
[ 56%] Building CXX object
src/arrow/CMakeFiles/arrow_static.dir/util/string_builder.cc.obj
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:
In static member function 'static arrow::Status
arrow::internal::TemporaryDir::Make(const string&,
std::unique_ptr*)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:897:3:
warning: 'created' may be used uninitialized in this function
[-Wmaybe-uninitialized]
if (!created) {
^~
{code}
And on mingw32 we also see these:
{code}
[ 56%] Building CXX object
src/arrow/CMakeFiles/arrow_static.dir/util/string_builder.cc.obj
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:
In static member function 'static arrow::Status
arrow::internal::TemporaryDir::Make(const string&,
std::unique_ptr*)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:897:3:
warning: 'created' may be used uninitialized in this function
[-Wmaybe-uninitialized]
if (!created) {
^~
{code}
{code}
54%] Building CXX object
src/arrow/CMakeFiles/arrow_static.dir/util/logging.cc.obj
In file included from
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:63:
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:
In function 'void* mmap(void*, size_t, int, int, int, off_t)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:94:62:
warning: right shift count >= width of type [-Wshift-count-overflow]
const DWORD dwMaxSizeHigh = static_cast((maxSize >> 32) &
0xL);
^~
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:
In function 'arrow::Status arrow::internal::MemoryMapRemap(void*, size_t,
size_t, int, void**)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:568:55:
warning: right shift count >= width of type [-Wshift-count-overflow]
LONG new_size_high = static_cast((new_size >> 32) & 0xL);
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-5849) Compiler warnings on mingw-w64
[
https://issues.apache.org/jira/browse/ARROW-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen updated ARROW-5849:
--
Description:
In mingw64 we see the following warnings:
{code}
[ 54%] Building CXX object
src/arrow/CMakeFiles/arrow_static.dir/util/io-util.cc.obj
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/decimal.cc:
In static member function 'static arrow::Status
arrow::Decimal128::FromString(const string_view&, arrow::Decimal128*, int32_t*,
int32_t*)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/decimal.cc:313:35:
warning: 'dec.arrow::{anonymous}::DecimalComponents::exponent' may be used
uninitialized in this function [-Wmaybe-uninitialized]
*scale = -adjusted_exponent + len - 1;
~~~^~~~
{code}
{code}
[ 56%] Building CXX object
src/arrow/CMakeFiles/arrow_static.dir/util/string_builder.cc.obj
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:
In static member function 'static arrow::Status
arrow::internal::TemporaryDir::Make(const string&,
std::unique_ptr*)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:897:3:
warning: 'created' may be used uninitialized in this function
[-Wmaybe-uninitialized]
if (!created) {
^~
{code}
And on mingw32 we also see these:
{code}
In file included from
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/file.cc:25:
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:
In function 'void* mmap(void*, size_t, int, int, int, off_t)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:94:62:
warning: right shift count >= width of type [-Wshift-count-overflow]
const DWORD dwMaxSizeHigh = static_cast((maxSize >> 32) &
0xL);
^~
{code}
{code}
54%] Building CXX object
src/arrow/CMakeFiles/arrow_static.dir/util/logging.cc.obj
In file included from
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:63:
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:
In function 'void* mmap(void*, size_t, int, int, int, off_t)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:94:62:
warning: right shift count >= width of type [-Wshift-count-overflow]
const DWORD dwMaxSizeHigh = static_cast((maxSize >> 32) &
0xL);
^~
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:
In function 'arrow::Status arrow::internal::MemoryMapRemap(void*, size_t,
size_t, int, void**)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:568:55:
warning: right shift count >= width of type [-Wshift-count-overflow]
LONG new_size_high = static_cast((new_size >> 32) & 0xL);
{code}
was:
In mingw64 we see the following warnings:
{code}
[ 54%] Building CXX object
src/arrow/CMakeFiles/arrow_static.dir/util/io-util.cc.obj
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/decimal.cc:
In static member function 'static arrow::Status
arrow::Decimal128::FromString(const string_view&, arrow::Decimal128*, int32_t*,
int32_t*)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/decimal.cc:313:35:
warning: 'dec.arrow::{anonymous}::DecimalComponents::exponent' may be used
uninitialized in this function [-Wmaybe-uninitialized]
*scale = -adjusted_exponent + len - 1;
~~~^~~~
{code}
{code}
[ 56%] Building CXX object
src/arrow/CMakeFiles/arrow_static.dir/util/string_builder.cc.obj
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:
In static member function 'static arrow::Status
arrow::internal::TemporaryDir::Make(const string&,
std::unique_ptr*)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:897:3:
warning: 'created' may be used uninitialized in this function
[-Wmaybe-uninitialized]
if (!created) {
^~
{code}
And on mingw32 we also see these:
{code}
[ 56%] Building CXX object
src/arrow/CMakeFiles/arrow_static.dir/util/string_builder.cc.obj
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:
In static member function 'static arrow::Status
arrow::internal::TemporaryDir::Make(const string&,
std::unique_ptr*)':
C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:897:3:
warning: 'created' may be used uninitialized in this function
[-Wmaybe-uninitialized]
if (!created) {
^~
{code}
{code}
54%] Building CXX object
src/arrow/CMakeFiles/arrow_static.dir/util/logging.cc.obj
In file inc
[jira] [Commented] (ARROW-5849) Compiler warnings on mingw-w64
[
https://issues.apache.org/jira/browse/ARROW-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878633#comment-16878633
]
Jeroen commented on ARROW-5849:
---
The first two -Wmaybe-uninitialized appear on both in 32 and 64 bit. The bottom
two -Wshift-count-overflow are 32 bit only.
> Can you propose a PR for this?
No sorry I'm not an arrow developer. I just need to build binaries.
> Compiler warnings on mingw-w64
> --
>
> Key: ARROW-5849
> URL: https://issues.apache.org/jira/browse/ARROW-5849
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.14.0
>Reporter: Jeroen
>Priority: Minor
>
> In mingw64 we see the following warnings:
> {code}
> [ 54%] Building CXX object
> src/arrow/CMakeFiles/arrow_static.dir/util/io-util.cc.obj
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/decimal.cc:
> In static member function 'static arrow::Status
> arrow::Decimal128::FromString(const string_view&, arrow::Decimal128*,
> int32_t*, int32_t*)':
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/decimal.cc:313:35:
> warning: 'dec.arrow::{anonymous}::DecimalComponents::exponent' may be used
> uninitialized in this function [-Wmaybe-uninitialized]
>*scale = -adjusted_exponent + len - 1;
> ~~~^~~~
> {code}
> {code}
> [ 56%] Building CXX object
> src/arrow/CMakeFiles/arrow_static.dir/util/string_builder.cc.obj
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:
> In static member function 'static arrow::Status
> arrow::internal::TemporaryDir::Make(const string&,
> std::unique_ptr*)':
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:897:3:
> warning: 'created' may be used uninitialized in this function
> [-Wmaybe-uninitialized]
>if (!created) {
>^~
> {code}
> And on mingw32 we also see these:
> {code}
> In file included from
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/file.cc:25:
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:
> In function 'void* mmap(void*, size_t, int, int, int, off_t)':
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:94:62:
> warning: right shift count >= width of type [-Wshift-count-overflow]
>const DWORD dwMaxSizeHigh = static_cast((maxSize >> 32) &
> 0xL);
> ^~
> {code}
> {code}
> 54%] Building CXX object
> src/arrow/CMakeFiles/arrow_static.dir/util/logging.cc.obj
> In file included from
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:63:
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:
> In function 'void* mmap(void*, size_t, int, int, int, off_t)':
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:94:62:
> warning: right shift count >= width of type [-Wshift-count-overflow]
>const DWORD dwMaxSizeHigh = static_cast((maxSize >> 32) &
> 0xL);
> ^~
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:
> In function 'arrow::Status arrow::internal::MemoryMapRemap(void*, size_t,
> size_t, int, void**)':
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:568:55:
> warning: right shift count >= width of type [-Wshift-count-overflow]
>LONG new_size_high = static_cast((new_size >> 32) & 0xL);
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (ARROW-5849) Compiler warnings on mingw-w64
[
https://issues.apache.org/jira/browse/ARROW-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen reassigned ARROW-5849:
-
Assignee: Neal Richardson
> Compiler warnings on mingw-w64
> --
>
> Key: ARROW-5849
> URL: https://issues.apache.org/jira/browse/ARROW-5849
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.14.0
>Reporter: Jeroen
>Assignee: Neal Richardson
>Priority: Minor
>
> In mingw64 we see the following warnings:
> {code}
> [ 54%] Building CXX object
> src/arrow/CMakeFiles/arrow_static.dir/util/io-util.cc.obj
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/decimal.cc:
> In static member function 'static arrow::Status
> arrow::Decimal128::FromString(const string_view&, arrow::Decimal128*,
> int32_t*, int32_t*)':
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/decimal.cc:313:35:
> warning: 'dec.arrow::{anonymous}::DecimalComponents::exponent' may be used
> uninitialized in this function [-Wmaybe-uninitialized]
>*scale = -adjusted_exponent + len - 1;
> ~~~^~~~
> {code}
> {code}
> [ 56%] Building CXX object
> src/arrow/CMakeFiles/arrow_static.dir/util/string_builder.cc.obj
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:
> In static member function 'static arrow::Status
> arrow::internal::TemporaryDir::Make(const string&,
> std::unique_ptr*)':
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:897:3:
> warning: 'created' may be used uninitialized in this function
> [-Wmaybe-uninitialized]
>if (!created) {
>^~
> {code}
> And on mingw32 we also see these:
> {code}
> In file included from
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/file.cc:25:
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:
> In function 'void* mmap(void*, size_t, int, int, int, off_t)':
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:94:62:
> warning: right shift count >= width of type [-Wshift-count-overflow]
>const DWORD dwMaxSizeHigh = static_cast((maxSize >> 32) &
> 0xL);
> ^~
> {code}
> {code}
> 54%] Building CXX object
> src/arrow/CMakeFiles/arrow_static.dir/util/logging.cc.obj
> In file included from
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:63:
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:
> In function 'void* mmap(void*, size_t, int, int, int, off_t)':
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/io/mman.h:94:62:
> warning: right shift count >= width of type [-Wshift-count-overflow]
>const DWORD dwMaxSizeHigh = static_cast((maxSize >> 32) &
> 0xL);
> ^~
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:
> In function 'arrow::Status arrow::internal::MemoryMapRemap(void*, size_t,
> size_t, int, void**)':
> C:/msys64/home/mingw-packages/mingw-w64-arrow/src/arrow/cpp/src/arrow/util/io-util.cc:568:55:
> warning: right shift count >= width of type [-Wshift-count-overflow]
>LONG new_size_high = static_cast((new_size >> 32) & 0xL);
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ARROW-6214) Sanitizer errors triggered via R bindings
Jeroen created ARROW-6214: - Summary: Sanitizer errors triggered via R bindings Key: ARROW-6214 URL: https://issues.apache.org/jira/browse/ARROW-6214 Project: Apache Arrow Issue Type: Bug Components: C++, R Affects Versions: 0.14.1 Environment: Linux Reporter: Jeroen When we run the examples of the R package through the sanitizers, several errors show up. These could be related to the segfaults we saw on the macos builder on CRAN. Steps to reproduce + example outputs at: https://gist.github.com/jeroen/111901c351a4089a9effa90691a1dd81 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (ARROW-6214) Sanitizer errors triggered via R bindings
[ https://issues.apache.org/jira/browse/ARROW-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeroen updated ARROW-6214: -- Description: When we run the examples of the R package through the sanitizers, several errors show up. These could be related to the segfaults we saw on the macos builder on CRAN. We use the docker container provided by Winston Chang to test this: https://github.com/wch/r-debug Steps to reproduce + example outputs at: https://gist.github.com/jeroen/111901c351a4089a9effa90691a1dd81 was: When we run the examples of the R package through the sanitizers, several errors show up. These could be related to the segfaults we saw on the macos builder on CRAN. Steps to reproduce + example outputs at: https://gist.github.com/jeroen/111901c351a4089a9effa90691a1dd81 > Sanitizer errors triggered via R bindings > - > > Key: ARROW-6214 > URL: https://issues.apache.org/jira/browse/ARROW-6214 > Project: Apache Arrow > Issue Type: Bug > Components: C++, R >Affects Versions: 0.14.1 > Environment: Linux >Reporter: Jeroen >Priority: Major > > When we run the examples of the R package through the sanitizers, several > errors show up. These could be related to the segfaults we saw on the macos > builder on CRAN. > We use the docker container provided by Winston Chang to test this: > https://github.com/wch/r-debug > Steps to reproduce + example outputs at: > https://gist.github.com/jeroen/111901c351a4089a9effa90691a1dd81 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
[
https://issues.apache.org/jira/browse/ARROW-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908545#comment-16908545
]
Jeroen commented on ARROW-4844:
---
I'm working around it now by linking an external libdouble-conversion rather
than the vendored one.
> Static libarrow is missing vendored libdouble-conversion
>
>
> Key: ARROW-4844
> URL: https://issues.apache.org/jira/browse/ARROW-4844
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.1
>Reporter: Jeroen
>Priority: Major
>
> When trying to statically link the R bindings to libarrow.a, I get linking
> errors which suggest that libdouble-conversion.a was not properly embedded in
> libarrow.a. This problem happens on both MacOS and Windows.
> Here is the arrow build log:
> https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
> {code}
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (ARROW-6214) [R] Sanitizer errors triggered via R bindings
[
https://issues.apache.org/jira/browse/ARROW-6214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16912292#comment-16912292
]
Jeroen commented on ARROW-6214:
---
I have updated the output at the gist. Some of these look pretty bad, it is
unclear to me if the originate in libarrow or in the R bindings. There are also
warnings from RDthreadcheck:
{code}
Fatal error: Wrong thread calling 'Rf_protect'
Fatal error: Wrong thread calling 'RunFinalizers'
Fatal error: Wrong thread calling 'RunFinalizers'
segfault
{code}
It's very likely that these problems are related to the crashes in the unit
tests on the CRAN mac builder.
> [R] Sanitizer errors triggered via R bindings
> -
>
> Key: ARROW-6214
> URL: https://issues.apache.org/jira/browse/ARROW-6214
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, R
>Affects Versions: 0.14.1
> Environment: Linux
>Reporter: Jeroen
>Priority: Critical
> Fix For: 0.15.0
>
>
> When we run the examples of the R package through the sanitizers, several
> errors show up. These could be related to the segfaults we saw on the macos
> builder on CRAN.
> We use the docker container provided by Winston Chang to test this:
> https://github.com/wch/r-debug
> Steps to reproduce + example outputs at:
> https://gist.github.com/jeroen/111901c351a4089a9effa90691a1dd81
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (ARROW-4848) [C++] Static libparquet not compiled with -DARROW_STATIC on Windows
[ https://issues.apache.org/jira/browse/ARROW-4848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913096#comment-16913096 ] Jeroen commented on ARROW-4848: --- [~npr] Note we hardcode `-DARROW_STATIC` during the build: https://github.com/apache/arrow/blob/master/ci/PKGBUILD#L67 I'm not sure if this is still needed, but previously it was. > [C++] Static libparquet not compiled with -DARROW_STATIC on Windows > --- > > Key: ARROW-4848 > URL: https://issues.apache.org/jira/browse/ARROW-4848 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Affects Versions: 0.12.1 >Reporter: Jeroen >Assignee: Uwe L. Korn >Priority: Major > Fix For: 0.15.0 > > > When trying to link the R bindings against static libparquet.a + libarrow.a > we get a lot of missing arrow symbol warnings from libparquet.a. I think the > problem is that libparquet.a was not compiled -DARROW_STATIC, and therefore > cannot be linked against libarrow.a. > When arrow cmake is configured with -DARROW_BUILD_SHARED=OFF I think it > should automatically use -DARROW_STATIC when compiling libparquet on Windows? -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (ARROW-6679) [RELEASE] autobrew license in LICENSE.txt is not acceptable
[
https://issues.apache.org/jira/browse/ARROW-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937457#comment-16937457
]
Jeroen commented on ARROW-6679:
---
Added a file: https://github.com/jeroen/autobrew/blob/gh-pages/LICENCE.txt
> [RELEASE] autobrew license in LICENSE.txt is not acceptable
> ---
>
> Key: ARROW-6679
> URL: https://issues.apache.org/jira/browse/ARROW-6679
> Project: Apache Arrow
> Issue Type: Bug
> Components: R
>Reporter: Wes McKinney
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.15.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> This project includes code from the autobrew project.
> * r/tools/autobrew and dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb
> are based on code from the autobrew project.
> Copyright: Copyright (c) 2017 - 2019, Jeroen Ooms.
> All rights reserved.
> Homepage: https://github.com/jeroen/autobrew
> {code}
> This code needs to be made available under a Category A license
> https://apache.org/legal/resolved.html#category-a
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-3204) [R] Release to CRAN
[ https://issues.apache.org/jira/browse/ARROW-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16747277#comment-16747277 ] Jeroen commented on ARROW-3204: --- The Homebrew lead maintainer has dropped arrow from homebrew because it isn't building: https://github.com/Homebrew/homebrew-core/commit/6cd7f61d2d07d5fd9f8747ea0f620169cc8a2434 . So this further complicates things for R bindings on MacOS. I am guessing that the build broke because the 0.11 tarball has been removed from the apache website, even though 0.12 hasn't even been announced yet? [https://www.apache.org/dist/arrow/] > [R] Release to CRAN > --- > > Key: ARROW-3204 > URL: https://issues.apache.org/jira/browse/ARROW-3204 > Project: Apache Arrow > Issue Type: New Feature > Components: R >Reporter: James Lamb >Priority: Major > > As of [ARROW-1325|https://issues.apache.org/jira/browse/ARROW-1325], the > project contains a minimal working R package that takes advantage of R's > built-in support for calling C++ code and uses Rcpp for added support. > Though the exact public interface of the package hasn't been developed yet > (so this issue may be running before we walk), one major feature in its > development will be release to [CRAN|https://cran.r-project.org/], the > official package repository for the R language. > Completing this story would mean: > * Getting source code to state where [R CMD > CHECK|https://www.r-bloggers.com/how-i-learned-to-stop-worrying-and-love-r-cmd-check/] > passes with 0 ERRORS, 0 WARNINGS, and 0 NOTES > * Setting up build process for source and precompiled binary packages (see > [data.table|https://github.com/Rdatatable/data.table] and > [XGBoost|https://github.com/dmlc/xgboost] as examples of packages that do > this) > * Submission to CRAN and acceptance of the first release > > Distribution via CRAN would be a much more natural fit for most R users' > workflows than the current build-from-source workflow, and comes with all the > other benefits of package managers (e.g. version pegging, easy distribution > of platform-specific binaries). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ARROW-4351) Fail to build with static parquet
Jeroen created ARROW-4351:
-
Summary: Fail to build with static parquet
Key: ARROW-4351
URL: https://issues.apache.org/jira/browse/ARROW-4351
Project: Apache Arrow
Issue Type: Bug
Reporter: Jeroen
Trying to build a static version of arrow with parquet fails as below. Is this
configuration possible?
{{ -DARROW_BUILD_STATIC=ON}}
{{ -DARROW_BUILD_TESTS=OFF}}
{{ -DARROW_PYTHON=OFF}}
{{ -DARROW_BOOST_USE_SHARED=OFF}}
{{ -DARROW_WITH_SNAPPY=OFF}}
{{ -DARROW_WITH_ZSTD=OFF}}
{{ -DARROW_WITH_LZ4=OFF}}
{{ -DARROW_JEMALLOC=OFF}}
{{ -DARROW_BUILD_SHARED=OFF}}
{{ -DARROW_BOOST_VENDORED=OFF}}
{{ -DARROW_WITH_ZLIB=OFF}}
{{ -DARROW_WITH_BROTLI=OFF}}
{{ -DARROW_USE_GLOG=OFF}}
{{ -DPTHREAD_LIBRARY=OFF}}
{{ -DARROW_BUILD_UTILITIES=ON}}
{{ -DARROW_TEST_LINKAGE="static"}}
{{ -DARROW_HDFS=OFF}}
{{ -DARROW_PARQUET=ON}}
...
{{==> cmake . -DCMAKE_C_FLAGS_RELEASE=-DNDEBUG
-DCMAKE_CXX_FLAGS_RELEASE=-DNDEBUG}}
{{Last 15 lines from
/Users/jeroen/Library/Logs/Homebrew/apache-arrow/01.cmake:}}
{{-- CMAKE_CXX_FLAGS: -Qunused-arguments -O3 -DNDEBUG -Wall
-Wno-unknown-warning-option -msse4.2 -maltivec -march=armv8-a+crc
-stdlib=libc++}}
{{-- Looking for backtrace}}
{{-- Looking for backtrace - found}}
{{-- backtrace facility detected in default set of libraries}}
{{-- Found Backtrace: /usr/include}}
{{-- Configuring done}}
{{CMake Error at cmake_modules/BuildUtils.cmake:143 (add_dependencies):}}
{{ The dependency target "arrow_shared" of target "parquet_objlib" does not}}
{{ exist.}}
{{Call Stack (most recent call first):}}
{{ src/parquet/CMakeLists.txt:214 (ADD_ARROW_LIB)}}
{{-- Generating done}}
{{-- Build files have been written to:
/tmp/apache-arrow-20190123-44858-1as3l4q/apache-arrow-0.12.0/cpp}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
Jeroen created ARROW-4844:
-
Summary: Static libarrow is missing vendored libdouble-conversion
Key: ARROW-4844
URL: https://issues.apache.org/jira/browse/ARROW-4844
Project: Apache Arrow
Issue Type: Bug
Components: C++
Affects Versions: 0.12.1
Reporter: Jeroen
When trying to statically link libarrow.a I get linking errors which suggest
that libdouble-conversion.a was not properly embedded in libarrow.a. This
problem happens on both MacOS and Windows.
For the R bindings we need static libraries.
{code}
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
int*) const'
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ARROW-4845) Compiler warnings on Windows
Jeroen created ARROW-4845:
-
Summary: Compiler warnings on Windows
Key: ARROW-4845
URL: https://issues.apache.org/jira/browse/ARROW-4845
Project: Apache Arrow
Issue Type: Bug
Components: R
Affects Versions: 0.12.1
Reporter: Jeroen
I am seeing the warnings below when compiling the R bindings on Windows. Most
of these seem easy to fix (comparing int with size_t or int32 with int64).
{code}
array.cpp: In function 'Rcpp::LogicalVector Array__Mask(const
std::shared_ptr&)':
array.cpp:102:24: warning: comparison of integer expressions of different
signedness: 'size_t' {aka 'long long unsigned int'} and 'int64_t' {aka 'long
long int'} [-Wsign-compare]
for (size_t i = 0; i < array->length(); i++, bitmap_reader.Next()) {
~~^
/mingw64/bin/g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-testing/include" -DNDEBUG
-DARROW_STATIC -I"C:/R/library/Rcpp/include"-O2 -Wall -mtune=generic
-c array__to_vector.cpp -o array__to_vector.o
array__to_vector.cpp: In member function 'virtual arrow::Status
arrow::r::Converter_Boolean::Ingest_some_nulls(SEXP, const
std::shared_ptr&, R_xlen_t, R_xlen_t) const':
array__to_vector.cpp:254:28: warning: comparison of integer expressions of
different signedness: 'size_t' {aka 'long long unsigned int'} and 'R_xlen_t'
{aka 'long long int'} [-Wsign-compare]
for (size_t i = 0; i < n; i++, data_reader.Next(), null_reader.Next(),
++p_data) {
~~^~~
array__to_vector.cpp:258:28: warning: comparison of integer expressions of
different signedness: 'size_t' {aka 'long long unsigned int'} and 'R_xlen_t'
{aka 'long long int'} [-Wsign-compare]
for (size_t i = 0; i < n; i++, data_reader.Next(), ++p_data) {
~~^~~
array__to_vector.cpp: In member function 'virtual arrow::Status
arrow::r::Converter_Decimal::Ingest_some_nulls(SEXP, const
std::shared_ptr&, R_xlen_t, R_xlen_t) const':
array__to_vector.cpp:473:28: warning: comparison of integer expressions of
different signedness: 'size_t' {aka 'long long unsigned int'} and 'R_xlen_t'
{aka 'long long int'} [-Wsign-compare]
for (size_t i = 0; i < n; i++, bitmap_reader.Next(), ++p_data) {
~~^~~
array__to_vector.cpp:478:28: warning: comparison of integer expressions of
different signedness: 'size_t' {aka 'long long unsigned int'} and 'R_xlen_t'
{aka 'long long int'} [-Wsign-compare]
for (size_t i = 0; i < n; i++, ++p_data) {
~~^~~
array__to_vector.cpp: In member function 'virtual arrow::Status
arrow::r::Converter_Int64::Ingest_some_nulls(SEXP, const
std::shared_ptr&, R_xlen_t, R_xlen_t) const':
array__to_vector.cpp:515:28: warning: comparison of integer expressions of
different signedness: 'size_t' {aka 'long long unsigned int'} and 'R_xlen_t'
{aka 'long long int'} [-Wsign-compare]
for (size_t i = 0; i < n; i++, bitmap_reader.Next(), ++p_data) {
~~^~~
array__to_vector.cpp: In instantiation of 'arrow::Status
arrow::r::SomeNull_Ingest(SEXP, R_xlen_t, R_xlen_t, const array_value_type*,
const std::shared_ptr&, Lambda) [with int RTYPE = 14;
array_value_type = long long int; Lambda =
arrow::r::Converter_Date64::Ingest_some_nulls(SEXP, const
std::shared_ptr&, R_xlen_t, R_xlen_t) const::;
SEXP = SEXPREC*; R_xlen_t = long long int]':
array__to_vector.cpp:366:77: required from here
array__to_vector.cpp:116:26: warning: comparison of integer expressions of
different signedness: 'size_t' {aka 'long long unsigned int'} and 'R_xlen_t'
{aka 'long long int'} [-Wsign-compare]
for (size_t i = 0; i < n; i++, bitmap_reader.Next(), ++p_data, ++p_values)
{
~~^~~
array__to_vector.cpp: In instantiation of 'arrow::Status
arrow::r::SomeNull_Ingest(SEXP, R_xlen_t, R_xlen_t, const array_value_type*,
const std::shared_ptr&, Lambda) [with int RTYPE = 13;
array_value_type = unsigned char; Lambda =
arrow::r::Converter_Dictionary::Ingest_some_nulls_Impl(SEXP, const
std::shared_ptr&, R_xlen_t, R_xlen_t) const [with Type =
arrow::UInt8Type; SEXP = SEXPREC*; R_xlen_t = long long
int]::; SEXP = SEXPREC*; R_xlen_t = long long int]':
array__to_vector.cpp:341:47: required from 'arrow::Status
arrow::r::Converter_Dictionary::Ingest_some_nulls_Impl(SEXP, const
std::shared_ptr&, R_xlen_t, R_xlen_t) const [with Type =
arrow::UInt8Type; SEXP = SEXPREC*; R_xlen_t = long long int]'
array__to_vector.cpp:313:78: required from here
array__to_vector.cpp:116:26: warning: comparison of integer expressions of
different signedness: 'size_t' {aka 'long long unsigned int'} and 'R_xlen_t'
{aka 'long long int'} [-Wsign-compare]
array__to_vector.cpp: In instantiation of 'arrow::Status
arrow::r::SomeNull_Ingest(SEXP, R_xlen_t, R_xlen_t, const array_value_type*,
const std::shared_ptr&, Lambda) [with int RTYPE = 13;
array_value
[jira] [Updated] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
[
https://issues.apache.org/jira/browse/ARROW-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen updated ARROW-4844:
--
Description:
When trying to statically link the R bindings to libarrow.a, I get linking
errors which suggest that libdouble-conversion.a was not properly embedded in
libarrow.a. This problem happens on both MacOS and Windows.
Here is the arrow build log:
https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
{code}
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
int*) const'
{code}
was:
When trying to statically link libarrow.a I get linking errors which suggest
that libdouble-conversion.a was not properly embedded in libarrow.a. This
problem happens on both MacOS and Windows.
For the R bindings we need static libraries.
{code}
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
int*) const'
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
undefined reference to
`double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
int*) const'
{code}
> Static libarrow is missing vendored libdouble-conversion
>
>
> Key: ARROW-4844
> URL: https://issues.apache.org/jira/browse/ARROW-4844
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.1
>Reporter: Jeroen
>Priority: Major
>
> When trying to statically link the R bindings to libarrow.a, I get linking
> errors which suggest that libdouble-conversion.a was not properly embedded in
> libarrow.a. This problem happens on both MacOS and Windows.
> Here is the arrow build log:
> https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
> {co
[jira] [Commented] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
[
https://issues.apache.org/jira/browse/ARROW-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790990#comment-16790990
]
Jeroen commented on ARROW-4844:
---
Hmm I don't understand. How are bindings supposed to use `libarrow.a` if it
depends on internal vendored libraries which are not shipped with the
installation? There is no way to link to this?
It seems to me you either need to include the internal libdouble-conversion
object files into `libarrow.a`, or alternatively ship libdouble-conversion.a
along with libarrow.a in the installation and update arrow.pc to link with
"-larrow -ldouble-conversion"
> Static libarrow is missing vendored libdouble-conversion
>
>
> Key: ARROW-4844
> URL: https://issues.apache.org/jira/browse/ARROW-4844
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.1
>Reporter: Jeroen
>Priority: Major
>
> When trying to statically link the R bindings to libarrow.a, I get linking
> errors which suggest that libdouble-conversion.a was not properly embedded in
> libarrow.a. This problem happens on both MacOS and Windows.
> Here is the arrow build log:
> https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
> {code}
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
[
https://issues.apache.org/jira/browse/ARROW-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790990#comment-16790990
]
Jeroen edited comment on ARROW-4844 at 3/12/19 8:51 PM:
Hmm I don't understand. How are bindings supposed to use `libarrow.a` if it
depends on internal vendored libraries which are not shipped with the
installation? There is no way to link to this?
It seems to me you either need to include the internal libdouble-conversion
object files into `libarrow.a`, or alternatively ship libdouble-conversion.a
along with libarrow.a in the installation (perhaps rename to
libarrow_double-conversion.a) and update arrow.pc to link with "-larrow
-ldouble-conversion"
Shared libs (dll files) are not permitted in R so we need a static libarrow.
was (Author: jeroenooms):
Hmm I don't understand. How are bindings supposed to use `libarrow.a` if it
depends on internal vendored libraries which are not shipped with the
installation? There is no way to link to this?
It seems to me you either need to include the internal libdouble-conversion
object files into `libarrow.a`, or alternatively ship libdouble-conversion.a
along with libarrow.a in the installation and update arrow.pc to link with
"-larrow -ldouble-conversion"
Shared libs (dll files) are not permitted in R so we need a static libarrow.
> Static libarrow is missing vendored libdouble-conversion
>
>
> Key: ARROW-4844
> URL: https://issues.apache.org/jira/browse/ARROW-4844
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.1
>Reporter: Jeroen
>Priority: Major
>
> When trying to statically link the R bindings to libarrow.a, I get linking
> errors which suggest that libdouble-conversion.a was not properly embedded in
> libarrow.a. This problem happens on both MacOS and Windows.
> Here is the arrow build log:
> https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
> {code}
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
[
https://issues.apache.org/jira/browse/ARROW-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790990#comment-16790990
]
Jeroen edited comment on ARROW-4844 at 3/12/19 8:52 PM:
Hmm I don't understand. How are bindings supposed to use `libarrow.a` if it
depends on internal vendored libraries which are not shipped with the
installation? There is no way to link to this?
It seems to me you either need to include the internal libdouble-conversion
object files into `libarrow.a`, or alternatively ship libdouble-conversion.a
along with libarrow.a in the installation (perhaps rename to
libarrow_double-conversion.a) and update arrow.pc to link with "-larrow
-ldouble-conversion"
Shared libs (dll files) are not permitted in R so we need a complete static
build of libarrow.
was (Author: jeroenooms):
Hmm I don't understand. How are bindings supposed to use `libarrow.a` if it
depends on internal vendored libraries which are not shipped with the
installation? There is no way to link to this?
It seems to me you either need to include the internal libdouble-conversion
object files into `libarrow.a`, or alternatively ship libdouble-conversion.a
along with libarrow.a in the installation (perhaps rename to
libarrow_double-conversion.a) and update arrow.pc to link with "-larrow
-ldouble-conversion"
Shared libs (dll files) are not permitted in R so we need a static libarrow.
> Static libarrow is missing vendored libdouble-conversion
>
>
> Key: ARROW-4844
> URL: https://issues.apache.org/jira/browse/ARROW-4844
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.1
>Reporter: Jeroen
>Priority: Major
>
> When trying to statically link the R bindings to libarrow.a, I get linking
> errors which suggest that libdouble-conversion.a was not properly embedded in
> libarrow.a. This problem happens on both MacOS and Windows.
> Here is the arrow build log:
> https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
> {code}
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
[
https://issues.apache.org/jira/browse/ARROW-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16790990#comment-16790990
]
Jeroen edited comment on ARROW-4844 at 3/12/19 8:46 PM:
Hmm I don't understand. How are bindings supposed to use `libarrow.a` if it
depends on internal vendored libraries which are not shipped with the
installation? There is no way to link to this?
It seems to me you either need to include the internal libdouble-conversion
object files into `libarrow.a`, or alternatively ship libdouble-conversion.a
along with libarrow.a in the installation and update arrow.pc to link with
"-larrow -ldouble-conversion"
Shared libs (dll files) are not permitted in R so we need a static libarrow.
was (Author: jeroenooms):
Hmm I don't understand. How are bindings supposed to use `libarrow.a` if it
depends on internal vendored libraries which are not shipped with the
installation? There is no way to link to this?
It seems to me you either need to include the internal libdouble-conversion
object files into `libarrow.a`, or alternatively ship libdouble-conversion.a
along with libarrow.a in the installation and update arrow.pc to link with
"-larrow -ldouble-conversion"
> Static libarrow is missing vendored libdouble-conversion
>
>
> Key: ARROW-4844
> URL: https://issues.apache.org/jira/browse/ARROW-4844
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.1
>Reporter: Jeroen
>Priority: Major
>
> When trying to statically link the R bindings to libarrow.a, I get linking
> errors which suggest that libdouble-conversion.a was not properly embedded in
> libarrow.a. This problem happens on both MacOS and Windows.
> Here is the arrow build log:
> https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
> {code}
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ARROW-4848) Static libparquet not compiled with -DARROW_STATIC on Windows
Jeroen created ARROW-4848: - Summary: Static libparquet not compiled with -DARROW_STATIC on Windows Key: ARROW-4848 URL: https://issues.apache.org/jira/browse/ARROW-4848 Project: Apache Arrow Issue Type: Bug Components: C++ Affects Versions: 0.12.1 Reporter: Jeroen When trying to link the R bindings against static libparquet.a + libarrow.a we get a lot of missing arrow symbol warnings from libparquet.a. I think the problem is that libparquet.a was not compiled -DARROW_STATIC, and therefore cannot be linked against libarrow.a. When arrow cmake is configured with -DARROW_BUILD_SHARED=OFF I think it should automatically use -DARROW_STATIC when compiling libparquet on Windows? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
[
https://issues.apache.org/jira/browse/ARROW-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791069#comment-16791069
]
Jeroen edited comment on ARROW-4844 at 3/12/19 10:42 PM:
-
OK thank you. FWIW: I was able to make the R bindings work by building an
external libdouble-conversion, and then rebuilding libarrow using this external
library by setting -DDOUBLE_CONVERSION_HOME. By avoiding the vendored library,
I can now statically link against libarrow by also including
`-ldouble-conversion`.
Still, I do think that when arrow does get built with vendored libs, those
should be shipped one way or another. Otherwise there is no way to link with
the resulting libarrow library.
was (Author: jeroenooms):
OK thank you. FWIW: I was able to make the R bindings work by building an
external libdouble-conversion, and then rebuilding libarrow using this external
library by setting -DDOUBLE_CONVERSION_HOME. By avoiding the vendored library,
I can now statically link with libarrow by also including `-ldouble-conversion`.
Still, I do think that when arrow does get built with vendored libs, those
should be shipped one way or another. Otherwise there is no way to link with
the resulting libarrow library.
> Static libarrow is missing vendored libdouble-conversion
>
>
> Key: ARROW-4844
> URL: https://issues.apache.org/jira/browse/ARROW-4844
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.1
>Reporter: Jeroen
>Assignee: Uwe L. Korn
>Priority: Major
>
> When trying to statically link the R bindings to libarrow.a, I get linking
> errors which suggest that libdouble-conversion.a was not properly embedded in
> libarrow.a. This problem happens on both MacOS and Windows.
> Here is the arrow build log:
> https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
> {code}
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
[
https://issues.apache.org/jira/browse/ARROW-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791069#comment-16791069
]
Jeroen edited comment on ARROW-4844 at 3/12/19 10:42 PM:
-
OK thank you. FWIW: I was able to make the R bindings work by building an
external libdouble-conversion, and then rebuilding libarrow using this external
library by setting -DDOUBLE_CONVERSION_HOME. By avoiding the vendored library,
I can now statically link against libarrow by also including
`-ldouble-conversion`. So I have a workaround.
Still, I do think that when arrow does get built with vendored libs, those
should be shipped one way or another. Otherwise there is no way to link with
the resulting libarrow library.
was (Author: jeroenooms):
OK thank you. FWIW: I was able to make the R bindings work by building an
external libdouble-conversion, and then rebuilding libarrow using this external
library by setting -DDOUBLE_CONVERSION_HOME. By avoiding the vendored library,
I can now statically link against libarrow by also including
`-ldouble-conversion`.
Still, I do think that when arrow does get built with vendored libs, those
should be shipped one way or another. Otherwise there is no way to link with
the resulting libarrow library.
> Static libarrow is missing vendored libdouble-conversion
>
>
> Key: ARROW-4844
> URL: https://issues.apache.org/jira/browse/ARROW-4844
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.1
>Reporter: Jeroen
>Assignee: Uwe L. Korn
>Priority: Major
>
> When trying to statically link the R bindings to libarrow.a, I get linking
> errors which suggest that libdouble-conversion.a was not properly embedded in
> libarrow.a. This problem happens on both MacOS and Windows.
> Here is the arrow build log:
> https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
> {code}
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
[
https://issues.apache.org/jira/browse/ARROW-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791069#comment-16791069
]
Jeroen commented on ARROW-4844:
---
OK thank you. FWIW: I was able to make the R bindings work by building an
external libdouble-conversion, and then rebuilding libarrow using this external
library by setting -DDOUBLE_CONVERSION_HOME. By avoiding the vendored library,
I can now statically link with libarrow by also including `-ldouble-conversion`.
Still, I do think that when arrow does get built with vendored libs, those
should be shipped one way or another. Otherwise there is no way to link with
the resulting libarrow library.
> Static libarrow is missing vendored libdouble-conversion
>
>
> Key: ARROW-4844
> URL: https://issues.apache.org/jira/browse/ARROW-4844
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.1
>Reporter: Jeroen
>Assignee: Uwe L. Korn
>Priority: Major
>
> When trying to statically link the R bindings to libarrow.a, I get linking
> errors which suggest that libdouble-conversion.a was not properly embedded in
> libarrow.a. This problem happens on both MacOS and Windows.
> Here is the arrow build log:
> https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
> {code}
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ARROW-4900) mingw-w64 < 5 does not have __cpuidex
Jeroen created ARROW-4900: - Summary: mingw-w64 < 5 does not have __cpuidex Key: ARROW-4900 URL: https://issues.apache.org/jira/browse/ARROW-4900 Project: Apache Arrow Issue Type: Bug Components: C++ Reporter: Jeroen The current R toolchain uses mingw-w64 version 3. Building arrow fails because arrow uses __cpuidex which was introduced in mingw-w64 version 5: https://github.com/mingw-w64/mingw-w64/commit/8e921f2a87fb70564cbcf40676daee93ea0cae3a We need a polyfill for this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-4900) mingw-w64 < 5 does not have __cpuidex
[ https://issues.apache.org/jira/browse/ARROW-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeroen updated ARROW-4900: -- Description: The current R toolchain uses mingw-w64 runtime v3. Building libarrow fails because arrow uses __cpuidex which was introduced in mingw-w64 runtime v5: https://github.com/mingw-w64/mingw-w64/commit/8e921f2a87fb70564cbcf40676daee93ea0cae3a We need a polyfill for this. was: The current R toolchain uses mingw-w64 version 3. Building arrow fails because arrow uses __cpuidex which was introduced in mingw-w64 version 5: https://github.com/mingw-w64/mingw-w64/commit/8e921f2a87fb70564cbcf40676daee93ea0cae3a We need a polyfill for this. > mingw-w64 < 5 does not have __cpuidex > - > > Key: ARROW-4900 > URL: https://issues.apache.org/jira/browse/ARROW-4900 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Jeroen >Priority: Major > > The current R toolchain uses mingw-w64 runtime v3. Building libarrow fails > because arrow uses __cpuidex which was introduced in mingw-w64 runtime v5: > https://github.com/mingw-w64/mingw-w64/commit/8e921f2a87fb70564cbcf40676daee93ea0cae3a > We need a polyfill for this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
[
https://issues.apache.org/jira/browse/ARROW-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16805787#comment-16805787
]
Jeroen commented on ARROW-4844:
---
For example opencv ships the vendored static libs in a special dir
lib/opencv4/3rdparty. I think that is more user friendly than restricting
static builds by "refusing to vendor anything at all".
{code}
[MSYS2 CI] mingw-w64-opencv: Checking Binaries
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_annotation.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_interactive-calibration.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_version.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_version_win32.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_visualisation.exe
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_calib3d.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_core.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_features2d.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_flann.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_gapi.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_highgui.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_imgcodecs.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_imgproc.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_ml.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_objdetect.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_photo.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_stitching.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_video.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_videoio.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/opencv4/3rdparty/libade.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/opencv4/3rdparty/libquirc.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/pkgconfig/opencv4.pc
{code}
> Static libarrow is missing vendored libdouble-conversion
>
>
> Key: ARROW-4844
> URL: https://issues.apache.org/jira/browse/ARROW-4844
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.1
>Reporter: Jeroen
>Assignee: Uwe L. Korn
>Priority: Major
>
> When trying to statically link the R bindings to libarrow.a, I get linking
> errors which suggest that libdouble-conversion.a was not properly embedded in
> libarrow.a. This problem happens on both MacOS and Windows.
> Here is the arrow build log:
> https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
> {code}
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x5fda):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6097):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6589):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(converter.cc.obj):(.text+0x6647):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToFloat(char const*, int,
> int*) const'
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (ARROW-4844) Static libarrow is missing vendored libdouble-conversion
[
https://issues.apache.org/jira/browse/ARROW-4844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16805787#comment-16805787
]
Jeroen edited comment on ARROW-4844 at 3/30/19 12:10 PM:
-
For example opencv ships the vendored static libs in a special dir
lib/opencv4/3rdparty. Thereby we can statically link to the library, and the
vendored libs won't conflict with anything else on the system. I think that is
more user friendly than restricting static builds by "refusing to vendor
anything at all".
{code}
[MSYS2 CI] mingw-w64-opencv: Checking Binaries
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_annotation.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_interactive-calibration.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_version.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_version_win32.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_visualisation.exe
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_calib3d.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_core.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_features2d.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_flann.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_gapi.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_highgui.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_imgcodecs.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_imgproc.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_ml.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_objdetect.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_photo.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_stitching.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_video.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_videoio.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/opencv4/3rdparty/libade.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/opencv4/3rdparty/libquirc.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/pkgconfig/opencv4.pc
{code}
was (Author: jeroenooms):
For example opencv ships the vendored static libs in a special dir
lib/opencv4/3rdparty. I think that is more user friendly than restricting
static builds by "refusing to vendor anything at all".
{code}
[MSYS2 CI] mingw-w64-opencv: Checking Binaries
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_annotation.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_interactive-calibration.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_version.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_version_win32.exe
./pkg/mingw-w64-i686-opencv/mingw32/bin/opencv_visualisation.exe
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_calib3d.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_core.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_features2d.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_flann.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_gapi.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_highgui.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_imgcodecs.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_imgproc.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_ml.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_objdetect.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_photo.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_stitching.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_video.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/libopencv_videoio.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/opencv4/3rdparty/libade.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/opencv4/3rdparty/libquirc.a
./pkg/mingw-w64-i686-opencv/mingw32/lib/pkgconfig/opencv4.pc
{code}
> Static libarrow is missing vendored libdouble-conversion
>
>
> Key: ARROW-4844
> URL: https://issues.apache.org/jira/browse/ARROW-4844
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.12.1
>Reporter: Jeroen
>Assignee: Uwe L. Korn
>Priority: Major
>
> When trying to statically link the R bindings to libarrow.a, I get linking
> errors which suggest that libdouble-conversion.a was not properly embedded in
> libarrow.a. This problem happens on both MacOS and Windows.
> Here is the arrow build log:
> https://ci.appveyor.com/project/jeroen/rtools-packages/builds/23015303/job/mtgl6rvfde502iu7
> {code}
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../lib/libarrow.a(cast.cc.obj):(.text+0x1c77c):
> undefined reference to
> `double_conversion::StringToDoubleConverter::StringToDouble(char const*, int,
> int*) const'
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
>
> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../.
[jira] [Created] (ARROW-5090) Linking failure on MacOS due to @rpath in dylib
Jeroen created ARROW-5090:
-
Summary: Linking failure on MacOS due to @rpath in dylib
Key: ARROW-5090
URL: https://issues.apache.org/jira/browse/ARROW-5090
Project: Apache Arrow
Issue Type: Bug
Components: C++
Reporter: Jeroen
Linking an application or bindings against the arrow shared lib (from homebrew)
fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
I don't think an installed lib should contain @rpath. The workaround is to set
the rpath when linking. The R package [apparently
hardcodes](https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21)
this to be /usr/local/lib which is appropriate.
I think the proper solution is to build with: CMAKE_BUILD_WITH_INSTALL_RPATH=ON
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-5090) Linking failure on MacOS due to @rpath in dylib
[
https://issues.apache.org/jira/browse/ARROW-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen updated ARROW-5090:
--
Description:
Linking an application or bindings against the arrow shared lib (from homebrew)
fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
Indeed if we check the dylib dependencies:
{code}
otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
version 13.0.0, current version 13.0.0)
@rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
13.0.0)
/usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
19.0.0, current version 19.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
{code}
I don't think an installed lib should contain @rpath. The workaround is to set
the rpath when linking. The R package [apparently
hardcodes](https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21)
this to be /usr/local/lib which is appropriate.
I think the proper solution is to build with: CMAKE_BUILD_WITH_INSTALL_RPATH=ON
was:
Linking an application or bindings against the arrow shared lib (from homebrew)
fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
I don't think an installed lib should contain @rpath. The workaround is to set
the rpath when linking. The R package [apparently
hardcodes](https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21)
this to be /usr/local/lib which is appropriate.
I think the proper solution is to build with: CMAKE_BUILD_WITH_INSTALL_RPATH=ON
> Linking failure on MacOS due to @rpath in dylib
> ---
>
> Key: ARROW-5090
> URL: https://issues.apache.org/jira/browse/ARROW-5090
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Jeroen
>Priority: Major
>
> Linking an application or bindings against the arrow shared lib (from
> homebrew) fails with:
> {code}
> dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so,
> 6): Library not loaded: @rpath/libarrow.13.dylib
> Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> Reason: image not found
> {code}
> Indeed if we check the dylib dependencies:
> {code}
> otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
> version 13.0.0, current version 13.0.0)
> @rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
> 13.0.0)
> /usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
> 19.0.0, current version 19.1.0)
> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
> 400.9.4)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.250.1)
> {code}
> I don't think an installed lib should contain @rpath. The workaround is to
> set the rpath when linking. The R package [apparently
> hardcodes](https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21)
> this to be /usr/local/lib which is appropriate.
> I think the proper solution is to build with:
> CMAKE_BUILD_WITH_INSTALL_RPATH=ON
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-5090) Linking failure on MacOS due to @rpath in dylib
[
https://issues.apache.org/jira/browse/ARROW-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen updated ARROW-5090:
--
Description:
Linking an application or bindings against the arrow shared lib (from homebrew)
fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
Indeed if we check the dylib dependencies:
{code}
otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
version 13.0.0, current version 13.0.0)
@rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
13.0.0)
/usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
19.0.0, current version 19.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
{code}
I don't think an installed lib should contain @rpath. Does this mean
ARROW_INSTALL_NAME_RPATH is not working?
The workaround is to set the rpath when linking. The R package [apparently
hardcodes](https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21)
this to be /usr/local/lib which is appropriate.
was:
Linking an application or bindings against the arrow shared lib (from homebrew)
fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
Indeed if we check the dylib dependencies:
{code}
otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
version 13.0.0, current version 13.0.0)
@rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
13.0.0)
/usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
19.0.0, current version 19.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
{code}
I don't think an installed lib should contain @rpath. The workaround is to set
the rpath when linking. The R package [apparently
hardcodes](https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21)
this to be /usr/local/lib which is appropriate.
I think the proper solution is to build with: CMAKE_BUILD_WITH_INSTALL_RPATH=ON
> Linking failure on MacOS due to @rpath in dylib
> ---
>
> Key: ARROW-5090
> URL: https://issues.apache.org/jira/browse/ARROW-5090
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Jeroen
>Priority: Major
>
> Linking an application or bindings against the arrow shared lib (from
> homebrew) fails with:
> {code}
> dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so,
> 6): Library not loaded: @rpath/libarrow.13.dylib
> Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> Reason: image not found
> {code}
> Indeed if we check the dylib dependencies:
> {code}
> otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
> version 13.0.0, current version 13.0.0)
> @rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
> 13.0.0)
> /usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
> 19.0.0, current version 19.1.0)
> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
> 400.9.4)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.250.1)
> {code}
> I don't think an installed lib should contain @rpath. Doe
[jira] [Updated] (ARROW-5090) Linking failure on MacOS due to @rpath in dylib
[
https://issues.apache.org/jira/browse/ARROW-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen updated ARROW-5090:
--
Description:
Linking an application or bindings against the arrow shared lib (from homebrew)
fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
Indeed if we check the dylib dependencies:
{code}
otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
version 13.0.0, current version 13.0.0)
@rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
13.0.0)
/usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
19.0.0, current version 19.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
{code}
I don't think an installed lib should contain @rpath. Does this mean
ARROW_INSTALL_NAME_RPATH is not working?
The workaround is to set the rpath when linking. The R package [apparently
hardcodes|https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21]
this to be /usr/local/lib which is appropriate.
was:
Linking an application or bindings against the arrow shared lib (from homebrew)
fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
Indeed if we check the dylib dependencies:
{code}
otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
version 13.0.0, current version 13.0.0)
@rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
13.0.0)
/usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
19.0.0, current version 19.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
{code}
I don't think an installed lib should contain @rpath. Does this mean
ARROW_INSTALL_NAME_RPATH is not working?
The workaround is to set the rpath when linking. The R package [apparently
hardcodes](https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21)
this to be /usr/local/lib which is appropriate.
> Linking failure on MacOS due to @rpath in dylib
> ---
>
> Key: ARROW-5090
> URL: https://issues.apache.org/jira/browse/ARROW-5090
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Jeroen
>Priority: Major
>
> Linking an application or bindings against the arrow shared lib (from
> homebrew) fails with:
> {code}
> dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so,
> 6): Library not loaded: @rpath/libarrow.13.dylib
> Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> Reason: image not found
> {code}
> Indeed if we check the dylib dependencies:
> {code}
> otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
> version 13.0.0, current version 13.0.0)
> @rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
> 13.0.0)
> /usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
> 19.0.0, current version 19.1.0)
> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
> 400.9.4)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.250.1)
> {code}
> I don't think an installed lib should contain @rpath. Does this mean
> ARROW_INST
[jira] [Updated] (ARROW-5090) Linking failure on MacOS due to @rpath in dylib
[
https://issues.apache.org/jira/browse/ARROW-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen updated ARROW-5090:
--
Description:
Linking an application or bindings against the arrow shared lib (from homebrew)
fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
Indeed if we check the dylib dependencies there is an unresolved reference to
_@rpath_.
{code}
otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
version 13.0.0, current version 13.0.0)
@rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
13.0.0)
/usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
19.0.0, current version 19.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
{code}
Does this mean ARROW_INSTALL_NAME_RPATH is not working?
The workaround is to set the rpath when linking. The R package [apparently
hardcodes|https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21]
this to be /usr/local/lib which is appropriate.
was:
Linking an application or bindings against the arrow shared lib (from homebrew)
fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
Indeed if we check the dylib dependencies:
{code}
otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
version 13.0.0, current version 13.0.0)
@rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
13.0.0)
/usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
19.0.0, current version 19.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
{code}
I don't think an installed lib should contain @rpath. Does this mean
ARROW_INSTALL_NAME_RPATH is not working?
The workaround is to set the rpath when linking. The R package [apparently
hardcodes|https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21]
this to be /usr/local/lib which is appropriate.
> Linking failure on MacOS due to @rpath in dylib
> ---
>
> Key: ARROW-5090
> URL: https://issues.apache.org/jira/browse/ARROW-5090
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Jeroen
>Priority: Major
>
> Linking an application or bindings against the arrow shared lib (from
> homebrew) fails with:
> {code}
> dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so,
> 6): Library not loaded: @rpath/libarrow.13.dylib
> Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> Reason: image not found
> {code}
> Indeed if we check the dylib dependencies there is an unresolved reference to
> _@rpath_.
> {code}
> otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
> version 13.0.0, current version 13.0.0)
> @rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
> 13.0.0)
> /usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
> 19.0.0, current version 19.1.0)
> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
> 400.9.4)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.250.1)
> {code}
> Does this mean ARROW_INSTALL_NAME_RPATH is n
[jira] [Updated] (ARROW-5090) Linking failure on MacOS due to @rpath in dylib
[
https://issues.apache.org/jira/browse/ARROW-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen updated ARROW-5090:
--
Description:
Linking an application or bindings against the parquet shared lib (from
homebrew) fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
Indeed if we check the dylib dependencies there is an unresolved reference to
_@rpath_.
{code}
otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
version 13.0.0, current version 13.0.0)
@rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
13.0.0)
/usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
19.0.0, current version 19.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
{code}
Does this mean ARROW_INSTALL_NAME_RPATH is not working?
The workaround is to set the rpath when linking. The R package [apparently
hardcodes|https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21]
this to be /usr/local/lib which is appropriate.
was:
Linking an application or bindings against the arrow shared lib (from homebrew)
fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
Indeed if we check the dylib dependencies there is an unresolved reference to
_@rpath_.
{code}
otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
version 13.0.0, current version 13.0.0)
@rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
13.0.0)
/usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
19.0.0, current version 19.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
{code}
Does this mean ARROW_INSTALL_NAME_RPATH is not working?
The workaround is to set the rpath when linking. The R package [apparently
hardcodes|https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21]
this to be /usr/local/lib which is appropriate.
> Linking failure on MacOS due to @rpath in dylib
> ---
>
> Key: ARROW-5090
> URL: https://issues.apache.org/jira/browse/ARROW-5090
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Jeroen
>Priority: Major
>
> Linking an application or bindings against the parquet shared lib (from
> homebrew) fails with:
> {code}
> dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so,
> 6): Library not loaded: @rpath/libarrow.13.dylib
> Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> Reason: image not found
> {code}
> Indeed if we check the dylib dependencies there is an unresolved reference to
> _@rpath_.
> {code}
> otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
> version 13.0.0, current version 13.0.0)
> @rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
> 13.0.0)
> /usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
> 19.0.0, current version 19.1.0)
> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
> 400.9.4)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.250.1)
> {code}
> Does this mean ARROW_INSTALL_NAME_RPATH is not wo
[jira] [Updated] (ARROW-5090) Parquet linking fails on MacOS due to @rpath in dylib
[
https://issues.apache.org/jira/browse/ARROW-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen updated ARROW-5090:
--
Summary: Parquet linking fails on MacOS due to @rpath in dylib (was:
Linking failure on MacOS due to @rpath in dylib)
> Parquet linking fails on MacOS due to @rpath in dylib
> -
>
> Key: ARROW-5090
> URL: https://issues.apache.org/jira/browse/ARROW-5090
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Jeroen
>Priority: Major
>
> Linking an application or bindings against the parquet shared lib (from
> homebrew) fails with:
> {code}
> dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so,
> 6): Library not loaded: @rpath/libarrow.13.dylib
> Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> Reason: image not found
> {code}
> Indeed if we check the dylib dependencies there is an unresolved reference to
> _@rpath_.
> {code}
> otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
> version 13.0.0, current version 13.0.0)
> @rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
> 13.0.0)
> /usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
> 19.0.0, current version 19.1.0)
> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
> 400.9.4)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.250.1)
> {code}
> Does this mean ARROW_INSTALL_NAME_RPATH is not working?
> The workaround is to set the rpath when linking. The R package [apparently
> hardcodes|https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21]
> this to be /usr/local/lib which is appropriate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-5090) Parquet linking fails on MacOS due to @rpath in dylib
[
https://issues.apache.org/jira/browse/ARROW-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeroen updated ARROW-5090:
--
Description:
Linking an application or bindings against the parquet shared lib (from
homebrew) fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
Indeed if we check the dylib dependencies there is an unresolved reference to
_@rpath_ to libarrow.
{code}
otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
version 13.0.0, current version 13.0.0)
@rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
13.0.0)
/usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
19.0.0, current version 19.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
{code}
Does this mean ARROW_INSTALL_NAME_RPATH is not working?
The workaround is to set the rpath when linking. The R package [apparently
hardcodes|https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21]
this to be /usr/local/lib which is appropriate.
was:
Linking an application or bindings against the parquet shared lib (from
homebrew) fails with:
{code}
dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so, 6):
Library not loaded: @rpath/libarrow.13.dylib
Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
Reason: image not found
{code}
Indeed if we check the dylib dependencies there is an unresolved reference to
_@rpath_.
{code}
otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
/usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
version 13.0.0, current version 13.0.0)
@rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
13.0.0)
/usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
0.0.0, current version 0.0.0)
/usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
19.0.0, current version 19.1.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
400.9.4)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
version 1252.250.1)
{code}
Does this mean ARROW_INSTALL_NAME_RPATH is not working?
The workaround is to set the rpath when linking. The R package [apparently
hardcodes|https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21]
this to be /usr/local/lib which is appropriate.
> Parquet linking fails on MacOS due to @rpath in dylib
> -
>
> Key: ARROW-5090
> URL: https://issues.apache.org/jira/browse/ARROW-5090
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Jeroen
>Priority: Major
>
> Linking an application or bindings against the parquet shared lib (from
> homebrew) fails with:
> {code}
> dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so,
> 6): Library not loaded: @rpath/libarrow.13.dylib
> Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> Reason: image not found
> {code}
> Indeed if we check the dylib dependencies there is an unresolved reference to
> _@rpath_ to libarrow.
> {code}
> otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
> version 13.0.0, current version 13.0.0)
> @rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
> 13.0.0)
> /usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
> 19.0.0, current version 19.1.0)
> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
> 400.9.4)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.250.1)
> {code}
> Does this m
[jira] [Commented] (ARROW-5090) Parquet linking fails on MacOS due to @rpath in dylib
[
https://issues.apache.org/jira/browse/ARROW-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808724#comment-16808724
]
Jeroen commented on ARROW-5090:
---
Fixed by setting -DARROW_INSTALL_NAME_RPATH=OFF in homebrew:
https://github.com/Homebrew/homebrew-core/pull/38636
> Parquet linking fails on MacOS due to @rpath in dylib
> -
>
> Key: ARROW-5090
> URL: https://issues.apache.org/jira/browse/ARROW-5090
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Jeroen
>Priority: Major
>
> Linking an application or bindings against the parquet shared lib (from
> homebrew) fails with:
> {code}
> dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so,
> 6): Library not loaded: @rpath/libarrow.13.dylib
> Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> Reason: image not found
> {code}
> Indeed if we check the dylib dependencies there is an unresolved reference to
> _@rpath_ to libarrow.
> {code}
> otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
> version 13.0.0, current version 13.0.0)
> @rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
> 13.0.0)
> /usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
> 19.0.0, current version 19.1.0)
> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
> 400.9.4)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.250.1)
> {code}
> Does this mean ARROW_INSTALL_NAME_RPATH is not working?
> The workaround is to set the rpath when linking. The R package [apparently
> hardcodes|https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21]
> this to be /usr/local/lib which is appropriate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (ARROW-5090) Parquet linking fails on MacOS due to @rpath in dylib
[
https://issues.apache.org/jira/browse/ARROW-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16808724#comment-16808724
]
Jeroen edited comment on ARROW-5090 at 4/3/19 1:33 PM:
---
Fixed by building the homebrew recipe with:
{code} -DARROW_INSTALL_NAME_RPATH=OFF{code}
See: https://github.com/Homebrew/homebrew-core/pull/38636
was (Author: jeroenooms):
Fixed by setting -DARROW_INSTALL_NAME_RPATH=OFF in homebrew:
https://github.com/Homebrew/homebrew-core/pull/38636
> Parquet linking fails on MacOS due to @rpath in dylib
> -
>
> Key: ARROW-5090
> URL: https://issues.apache.org/jira/browse/ARROW-5090
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Jeroen
>Priority: Major
>
> Linking an application or bindings against the parquet shared lib (from
> homebrew) fails with:
> {code}
> dlopen(/Users/travis/build/r-lib/arrow/arrow.Rcheck/arrow/libs/arrow.so,
> 6): Library not loaded: @rpath/libarrow.13.dylib
> Referenced from: /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> Reason: image not found
> {code}
> Indeed if we check the dylib dependencies there is an unresolved reference to
> _@rpath_ to libarrow.
> {code}
> otool -L /usr/local/opt/apache-arrow/lib/libparquet.13.dylib
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib:
> /usr/local/opt/apache-arrow/lib/libparquet.13.dylib (compatibility
> version 13.0.0, current version 13.0.0)
> @rpath/libarrow.13.dylib (compatibility version 13.0.0, current version
> 13.0.0)
> /usr/local/opt/boost/lib/libboost_regex-mt.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/thrift/lib/libthrift-0.12.0.dylib (compatibility version
> 0.0.0, current version 0.0.0)
> /usr/local/opt/protobuf/lib/libprotobuf.18.dylib (compatibility version
> 19.0.0, current version 19.1.0)
> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version
> 400.9.4)
> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
> version 1252.250.1)
> {code}
> Does this mean ARROW_INSTALL_NAME_RPATH is not working?
> The workaround is to set the rpath when linking. The R package [apparently
> hardcodes|https://github.com/apache/arrow/blob/master/r/src/Makevars.in#L21]
> this to be /usr/local/lib which is appropriate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5154) [R] [CI] Build failure
[
https://issues.apache.org/jira/browse/ARROW-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813857#comment-16813857
]
Jeroen commented on ARROW-5154:
---
This was a temporary bug in the R travis CI, it has been fixed:
https://github.com/r-lib/devtools/issues/2020#issuecomment-481331453
> [R] [CI] Build failure
> --
>
> Key: ARROW-5154
> URL: https://issues.apache.org/jira/browse/ARROW-5154
> Project: Apache Arrow
> Issue Type: Bug
> Components: Continuous Integration, R
>Reporter: Antoine Pitrou
>Priority: Critical
>
> The Travis-CI R build job is failing. Example here:
> https://travis-ci.org/apache/arrow/jobs/517479176
> Snippet:
> {code}
> $ Rscript -e 'deps <- devtools::dev_package_deps(dependencies =
> NA);devtools::install_deps(dependencies = TRUE);if (!all(deps$package %in%
> installed.packages())) { message("missing: ", paste(setdiff(deps$package,
> installed.packages()), collapse=", ")); q(status = 1, save = "no")}'
> Error in match.arg(upgrade, c("ask", "always", "never")) :
> 'arg' must be of length 1
> Calls: ... upgradable_packages -> resolve_upgrade -> match.arg
> Execution halted
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (ARROW-9616) [C++] Support LTO for R
Jeroen created ARROW-9616:
-
Summary: [C++] Support LTO for R
Key: ARROW-9616
URL: https://issues.apache.org/jira/browse/ARROW-9616
Project: Apache Arrow
Issue Type: Bug
Components: C++, R
Affects Versions: 1.0.0
Reporter: Jeroen
The next version of R might enable LTO on Windows, i.e. R packages will be
compiled with {{-flto}} by default. This works out of the box for most
packages, but for arrow, the linker crashes as below.
{code}
C:/rtools40/mingw64/bin/g++ -shared -O2 -Wall -mfpmath=sse -msse2
-mstackrealign -flto -s -static-libgcc -o arrow.dll tmp.def array.o
array_from_vector.o array_to_vector.o arraydata.o arrowExports.o buffer.o
chunkedarray.o compression.o compute.o csv.o dataset.o datatype.o expression.o
feather.o field.o filesystem.o imports.o io.o json.o memorypool.o message.o
parquet.o py-to-r.o recordbatch.o recordbatchreader.o recordbatchwriter.o
scalar.o schema.o symbols.o table.o threadpool.o -L../windows//lib-8.3.0/x64
-L../windows//lib/x64 -lparquet -larrow_dataset -larrow -lthrift -lsnappy -lz
-lzstd -llz4 -lbcrypt -lpsapi -lcrypto -lcrypt32 -lws2_32
-LC:/PROGRA~1/R/R-devel/bin/x64 -lR
lto1.exe: internal compiler error: in add_symbol_to_partition_1, at
lto/lto-partition.c:153
libbacktrace could not find executable to open
Please submit a full bug report,
with preprocessed source if appropriate.
See <[https://github.com/r-windows]> for instructions.
lto-wrapper.exe: fatal error: C:\rtools40\mingw64\bin\g++.exe returned 1 exit
status
compilation terminated.
C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe:
error: lto-wrapper failed
{code}
You can easily produce this for example like this:
{code:r}
dir.create("~/.R")
writeLines("CXXFLAGS=-flto", file = "~/.R/Makevars")
install.packages("arrow", type = 'source')
{code}
I am not sure if this is a bug in the toolchain, or in arrow. I tried with both
gcc-8.3.0 and gcc-9.3.0, and the result is the same. I did find [this
issue|https://github.com/cycfi/elements/pull/56] in another project which
suggests to enable `INTERPROCEDURAL_OPTIMIZATION` in cmake.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-11781) Reading small amount of files from a partitioned dataset is unexpectedly slow
Jeroen created ARROW-11781:
--
Summary: Reading small amount of files from a partitioned dataset
is unexpectedly slow
Key: ARROW-11781
URL: https://issues.apache.org/jira/browse/ARROW-11781
Project: Apache Arrow
Issue Type: Bug
Reporter: Jeroen
I posted this on StackOverflow and was told I should probably create an issue
here.
I managed to create a relative minimal example:
{code:java}
df = spark.createDataFrame(
[
(str(a), b, c, random.randint(0, 1000))
for a in range(100)
for b in range(10)
for c in range(1)
],
['a', 'b', 'c', 'd']
)
print("Writing the spark dataframe to the file system in partitioned folders.")
df.repartition('a').write.partitionBy('a', 'b').parquet(str(data_dir),
compression='snappy', mode='overwrite')
def time_it(func, repetition=10):
start = time.time()
for _ in range(repetition):
func()
return (time.time() - start) / repetition
print("Loading the entire dataset")
print(time_it(lambda: pd.read_parquet(data_dir, engine='pyarrow')))
print("Loading a single file using filters")
print(time_it(lambda: pd.read_parquet(data_dir, engine='pyarrow',
filters=[[('a', '=', '0'), ('b', '=', '0')]])))
print("Loading a single file using filters and a specified partitioning")
partitioning = pa.dataset.HivePartitioning(
pa.schema([
pa.field('a', pa.string()),
pa.field('b', pa.string())
])
)
print(time_it(lambda: pd.read_parquet(data_dir, engine='pyarrow',
filters=[[('a', '=', '0'), ('b', '=', '0')]], partitioning=partitioning)))
print("Loading a single file by specifying the path")
print(time_it(lambda: pd.read_parquet(data_dir / 'a=0' / 'b=0',
engine='pyarrow')))
{code}
Which gives me the following output:
{code:java}
Writing the spark dataframe to the file system in partitioned folders.
21/02/25 14:37:58 WARN TaskSetManager: Stage 0 contains a task of very large
size (18240 KiB). The maximum recommended task size is 1000 KiB.
Loading the entire dataset
0.23926825523376466
Loading a single file using filters
0.04788286685943603
Loading a single file using filters and a specified partitioning
0.0323061466217041
Loading a single file by specifying the path
0.0017130613327026368
{code}
Loading the small amount of files is about 20 times faster if you address the
paths directly, compared to the pyarrow filters.
The question as I posted it on StackOverflow:
I am having some problems with the speed of loading `.parquet` files. However,
I don't know what I am doing wrong.
*Problem*
I am trying to read a single `.parquet` file from from my local filesystem
which is the partitioned output from a spark job. Such that there are
`.parquet` files in hierarchical directories named `a=x` and `b=y`.
To achieve this, I am using `pandas.read_parquet` (which uses
`pyarrow.parquet.read_table`) for which I include the `filters` kwarg. The run
time of using the `filters` is way longer than I would expect.
{code:java}
# The following runs for about 55 seconds
pd.read_parquet(, filters=[[('a', '=', 'x'), ('b', '=',
'y')]])
# The following runs for about 0.04 seconds
pd.read_parquet(/a=x/b=y/)
# The following runs for about 70 seconds
pd.read_parquet(){code}
Reading a single parquet file by specifying filters is only slightly faster
than loading the entire dataset, where I would expect a run time approximately
linear in the amount of files.
*What mistake do I make here?*
I realize that simply putting the filters in the path would work, however this
will quickly become complex as what I want to filter on will / can change.
Besides, I think `read_table` should be able to load this data efficiently.
PS: The entire dataset contains many millions of rows, the data I want to load
is only a few thousand rows.
*Edit 1:*
As suggested by 0x26res I manually defined the partitioning, this lead to a
significant speed up, but still not as much as I would have expected. In this
situation the run time was about 5 seconds.
{code:java}
partitioning = HivePartitioning(
pa.schema([
pa.field('a', pa.string()),
pa.field('b', pa.int32()),
])
)
pd.read_parquet(
,
engine='pyarrow',
filters=[
[
('a', '=', x),
('b', '=', y),
]
],
partitioning=partitioning
)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-16903) Install failure on Ubuntu-22.04 when libboost-filesystem-dev is missing
Jeroen created ARROW-16903:
--
Summary: Install failure on Ubuntu-22.04 when
libboost-filesystem-dev is missing
Key: ARROW-16903
URL: https://issues.apache.org/jira/browse/ARROW-16903
Project: Apache Arrow
Issue Type: Bug
Components: R
Reporter: Jeroen
Trying to install the R package on Ubuntu 22.04 gives the following unhelpful
error for any configuration, even when using LIBARROW_MINIMAL=true or
FORCE_BUNDLED_BUILD=true
{code:java}
* installing *source* package ‘arrow’ ...
** using staged installation
Replacing default rmarkdown theme...
*** No libarrow binary found for version 8.0.0.9000 on ubuntu-22.04
*** Proceeding without libarrow
- NOTE ---
There was an issue preparing the Arrow C++ libraries.
See https://arrow.apache.org/docs/r/articles/install.html
-
{code}
It would be helpful in the message above to mention `ARROW_R_DEV=true` to debug
this further because that is not obvious.
Now the issue turned out to be this:
{code:java}
-- Could NOT find Boost: missing: system filesystem (found
/usr/lib/x86_64-linux-gnu/cmake/Boost-1.74.0/BoostConfig.cmake (found suitable
version "1.74.0", minimum required is "1.58"))
CMake Error at cmake_modules/ThirdpartyToolchain.cmake:892 (add_library):
add_library cannot create imported target "Boost::headers" because another
target with the same name already exists.
Call Stack (most recent call first):
cmake_modules/ThirdpartyToolchain.cmake:154 (build_boost)
cmake_modules/ThirdpartyToolchain.cmake:255 (build_dependency)
cmake_modules/ThirdpartyToolchain.cmake:1005 (resolve_dependency)
CMakeLists.txt:567 (include)
{code}
So my system had boost, but it did not have libboost-filesystem-dev. This leads
to a fatal error, even when we build with bundled dependencies. Is that
expected? It is not documented on
https://arrow.apache.org/docs/r/articles/install.html that this is a hard
requirement.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
