[jira] [Resolved] (ARROW-7702) [C++][Dataset] Provide (optional) deterministic order of batches
[
https://issues.apache.org/jira/browse/ARROW-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joris Van den Bossche resolved ARROW-7702.
--
Fix Version/s: 1.0.0
Assignee: Francois Saint-Jacques
Resolution: Duplicate
> [C++][Dataset] Provide (optional) deterministic order of batches
>
>
> Key: ARROW-7702
> URL: https://issues.apache.org/jira/browse/ARROW-7702
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python
>Reporter: Joris Van den Bossche
>Assignee: Francois Saint-Jacques
>Priority: Major
> Labels: dataset
> Fix For: 1.0.0
>
>
> Example with python:
> {code}
> import pyarrow as pa
> import pyarrow.parquet as pq
> table = pa.table({'a': range(12)})
> pq.write_table(table, "test_chunks.parquet", chunk_size=3)
> # reading with dataset
> import pyarrow.dataset as ds
> ds.dataset("test_chunks.parquet").to_table().to_pandas()
> {code}
> gives non-deterministic result (order of the row groups in the parquet file):
> {code}
> In [25]: ds.dataset("test_chunks.parquet").to_table().to_pandas()
>
>
> Out[25]:
> a
> 00
> 11
> 22
> 33
> 44
> 55
> 66
> 77
> 88
> 99
> 10 10
> 11 11
> In [26]: ds.dataset("test_chunks.parquet").to_table().to_pandas()
>
>
> Out[26]:
> a
> 00
> 11
> 22
> 33
> 48
> 59
> 6 10
> 7 11
> 84
> 95
> 10 6
> 11 7
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-7702) [C++][Dataset] Provide (optional) deterministic order of batches
[
https://issues.apache.org/jira/browse/ARROW-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140294#comment-17140294
]
Joris Van den Bossche commented on ARROW-7702:
--
Indeed, this was fixed. Thanks for noting
> [C++][Dataset] Provide (optional) deterministic order of batches
>
>
> Key: ARROW-7702
> URL: https://issues.apache.org/jira/browse/ARROW-7702
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python
>Reporter: Joris Van den Bossche
>Assignee: Francois Saint-Jacques
>Priority: Major
> Labels: dataset
> Fix For: 1.0.0
>
>
> Example with python:
> {code}
> import pyarrow as pa
> import pyarrow.parquet as pq
> table = pa.table({'a': range(12)})
> pq.write_table(table, "test_chunks.parquet", chunk_size=3)
> # reading with dataset
> import pyarrow.dataset as ds
> ds.dataset("test_chunks.parquet").to_table().to_pandas()
> {code}
> gives non-deterministic result (order of the row groups in the parquet file):
> {code}
> In [25]: ds.dataset("test_chunks.parquet").to_table().to_pandas()
>
>
> Out[25]:
> a
> 00
> 11
> 22
> 33
> 44
> 55
> 66
> 77
> 88
> 99
> 10 10
> 11 11
> In [26]: ds.dataset("test_chunks.parquet").to_table().to_pandas()
>
>
> Out[26]:
> a
> 00
> 11
> 22
> 33
> 48
> 59
> 6 10
> 7 11
> 84
> 95
> 10 6
> 11 7
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-7945) [C++][Dataset] Implement InMemoryDatasetFactory
[
https://issues.apache.org/jira/browse/ARROW-7945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140306#comment-17140306
]
Joris Van den Bossche commented on ARROW-7945:
--
[~bivald] that would indeed enable that, but not that you can already filter
Tables/RecordBatches right now (only a different syntax, with the {{filter()}}
method expecting a boolean mask)
> [C++][Dataset] Implement InMemoryDatasetFactory
> ---
>
> Key: ARROW-7945
> URL: https://issues.apache.org/jira/browse/ARROW-7945
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Affects Versions: 0.16.0
>Reporter: Ben Kietzman
>Assignee: Ben Kietzman
>Priority: Major
> Labels: dataset
>
> This will allow in memory datasets (such as tables) to participate in
> discovery through {{UnionDatasetFactory}}. This class will be trivial since
> Inspect will do nothing but return the table's schema, but is necessary to
> ensure that the resulting {{UnionDataset}}'s unified schema accommodates the
> table's schema (for example including fields present only in the table's
> schema or emitting an error when unification is not possible)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-7945) [C++][Dataset] Implement InMemoryDatasetFactory
[
https://issues.apache.org/jira/browse/ARROW-7945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140365#comment-17140365
]
Niklas B commented on ARROW-7945:
-
[~jorisvandenbossche] That's a good point! I have tried using numpy to create a
boolean mask and while that operations is zero copy and lightning fast, the
actual filter takes a fair bit of time (2 seconds) to filter out about 50% of
700k rows (~200 columns). I'm guessing its iterating each column, where each
column takes about 0.01s. Possibly I can improve this if I write a function in
cython/c++, but it's slightly out of my abilities. I'll probably wait for the
Dataset API :)
> [C++][Dataset] Implement InMemoryDatasetFactory
> ---
>
> Key: ARROW-7945
> URL: https://issues.apache.org/jira/browse/ARROW-7945
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Affects Versions: 0.16.0
>Reporter: Ben Kietzman
>Assignee: Ben Kietzman
>Priority: Major
> Labels: dataset
>
> This will allow in memory datasets (such as tables) to participate in
> discovery through {{UnionDatasetFactory}}. This class will be trivial since
> Inspect will do nothing but return the table's schema, but is necessary to
> ensure that the resulting {{UnionDataset}}'s unified schema accommodates the
> table's schema (for example including fields present only in the table's
> schema or emitting an error when unification is not possible)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ARROW-7084) [C++] ArrayRangeEquals should check for full type equality?
[ https://issues.apache.org/jira/browse/ARROW-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-7084: -- Labels: pull-request-available (was: ) > [C++] ArrayRangeEquals should check for full type equality? > > > Key: ARROW-7084 > URL: https://issues.apache.org/jira/browse/ARROW-7084 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Micah Kornfield >Assignee: Ji Liu >Priority: Major > Labels: pull-request-available > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > It looks like ArrayRangeEquals in compare.cc only checks type IDs before > doing comparison actual values. This is inconsistent with ArrayEquals which > checks for type equality and also seems incorrect for cases like Decimal128. > > I presume this was an oversight when fixing ARROW-2567 but maybe it was > intentional? > [~uwe]? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-8149) [C++/Python] Enable CUDA Support in conda recipes
[ https://issues.apache.org/jira/browse/ARROW-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Korn reassigned ARROW-8149: --- Assignee: Uwe Korn > [C++/Python] Enable CUDA Support in conda recipes > - > > Key: ARROW-8149 > URL: https://issues.apache.org/jira/browse/ARROW-8149 > Project: Apache Arrow > Issue Type: New Feature > Components: C++, Packaging >Reporter: Uwe Korn >Assignee: Uwe Korn >Priority: Major > Fix For: 1.0.0 > > > See the changes in > [https://github.com/conda-forge/arrow-cpp-feedstock/pull/123], we need to > copy this into the Arrow repository and also test CUDA in these recipes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8149) [C++/Python] Enable CUDA Support in conda recipes
[ https://issues.apache.org/jira/browse/ARROW-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-8149: -- Labels: pull-request-available (was: ) > [C++/Python] Enable CUDA Support in conda recipes > - > > Key: ARROW-8149 > URL: https://issues.apache.org/jira/browse/ARROW-8149 > Project: Apache Arrow > Issue Type: New Feature > Components: C++, Packaging >Reporter: Uwe Korn >Assignee: Uwe Korn >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > See the changes in > [https://github.com/conda-forge/arrow-cpp-feedstock/pull/123], we need to > copy this into the Arrow repository and also test CUDA in these recipes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-8895) [C++] Add C++ unit tests for filter and take functions on temporal type inputs, including timestamps
[ https://issues.apache.org/jira/browse/ARROW-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-8895. - Resolution: Fixed Issue resolved by pull request 7485 [https://github.com/apache/arrow/pull/7485] > [C++] Add C++ unit tests for filter and take functions on temporal type > inputs, including timestamps > > > Key: ARROW-8895 > URL: https://issues.apache.org/jira/browse/ARROW-8895 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > These are used in R but not tested in C++, so I only found out that I had > missed adding the kernels to the Filter VectorFunction when running the R > test suite -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9091) [C++] Utilize function's default options when passing no options to CallFunction for a function that requires them
[
https://issues.apache.org/jira/browse/ARROW-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-9091:
--
Labels: pull-request-available (was: )
> [C++] Utilize function's default options when passing no options to
> CallFunction for a function that requires them
> --
>
> Key: ARROW-9091
> URL: https://issues.apache.org/jira/browse/ARROW-9091
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
>Reporter: Wes McKinney
>Assignee: Ben Kietzman
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Otherwise benign usage of {{CallFunction}} can cause an unintuitive segfault
> in some cases
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-9186) [R] Allow specifying CSV file encoding
Neal Richardson created ARROW-9186: -- Summary: [R] Allow specifying CSV file encoding Key: ARROW-9186 URL: https://issues.apache.org/jira/browse/ARROW-9186 Project: Apache Arrow Issue Type: Improvement Components: R Reporter: Neal Richardson Fix For: 2.0.0 ARROW-9106 did this for Python and we should have the same in R -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8999) [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" build
[ https://issues.apache.org/jira/browse/ARROW-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neal Richardson updated ARROW-8999: --- Fix Version/s: (was: 1.0.0) 2.0.0 > [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" > build > > > Key: ARROW-8999 > URL: https://issues.apache.org/jira/browse/ARROW-8999 > Project: Apache Arrow > Issue Type: Bug > Components: Python >Reporter: Wes McKinney >Assignee: Krisztian Szucs >Priority: Major > Fix For: 2.0.0 > > > I've been seeing this segfault periodically the last week, does anyone have > an idea what might be wrong? > https://github.com/apache/arrow/pull/7273/checks?check_run_id=717249862 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-8999) [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" build
[ https://issues.apache.org/jira/browse/ARROW-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140654#comment-17140654 ] Wes McKinney commented on ARROW-8999: - [~kszucs] seems like GHA does not have segfault reporting, any way we could get a backtrace to show up when the crash happens? > [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" > build > > > Key: ARROW-8999 > URL: https://issues.apache.org/jira/browse/ARROW-8999 > Project: Apache Arrow > Issue Type: Bug > Components: Python >Reporter: Wes McKinney >Assignee: Krisztian Szucs >Priority: Major > Fix For: 2.0.0 > > > I've been seeing this segfault periodically the last week, does anyone have > an idea what might be wrong? > https://github.com/apache/arrow/pull/7273/checks?check_run_id=717249862 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9187) [R] Add bindings for arithmetic and is_null kernels
Neal Richardson created ARROW-9187: -- Summary: [R] Add bindings for arithmetic and is_null kernels Key: ARROW-9187 URL: https://issues.apache.org/jira/browse/ARROW-9187 Project: Apache Arrow Issue Type: Improvement Components: R Reporter: Neal Richardson Fix For: 2.0.0 These were done in Python in ARROW-9162 and ARROW-9159 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-9121) [C++] Do not wipe the filesystem when path is empty
[
https://issues.apache.org/jira/browse/ARROW-9121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140663#comment-17140663
]
Krisztian Szucs commented on ARROW-9121:
[~apitrou] What would your suggestion here? Have another method for wiping the
entire fs and raise from DeleteDirContents if empty path was passed?
> [C++] Do not wipe the filesystem when path is empty
> ---
>
> Key: ARROW-9121
> URL: https://issues.apache.org/jira/browse/ARROW-9121
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Mohamed Zenadi
>Assignee: Krisztian Szucs
>Priority: Minor
> Fix For: 1.0.0
>
>
> The `DeleteDirContents` method in the filesystems api has a default behavior
> or *wiping* the whole filesystem if we give it an empty path.
>
> It's documented as:
> > Like DeleteDir, but doesn’t delete the directory itself. Passing an empty
> path (“”) will wipe the entire filesystem tree.
>
> And the corresponding code confirms that:
> {code:java}
> auto parts = SplitAbstractPath(path);
> RETURN_NOT_OK(ValidateAbstractPathParts(parts));
>
> if (parts.empty()) {
> // Wipe filesystem
> impl_->RootDir().entries.clear();
> return Status::OK();
> }
> {code}
>
> This is a weird default that does not make sense. If the user wanted really
> to wipe his filesystem, he'd pass a `/`.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ARROW-9188) [C++] Do not always statically link Brotli libraries
Wes McKinney created ARROW-9188:
---
Summary: [C++] Do not always statically link Brotli libraries
Key: ARROW-9188
URL: https://issues.apache.org/jira/browse/ARROW-9188
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: Wes McKinney
Fix For: 1.0.0
I discovered by symbol sleuthing that even if the Brotli shared libraries are
available on the system, the static libraries will be used for linking. Here's
a dump of the symbols on my machine
https://gist.github.com/wesm/b7e911424ce3ca3a175b652666fe6f89
The largest of these symbols {{kBrotliDictionaryData}} is 120KB
Since we statically link everything in our Python wheels, we might consider
disabling Brotli in manylinux builds at least since it adds 800KB to libarrow.so
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (ARROW-9182) [C++] Use "applicator" namespace for kernel operator-to-kernel functors, streamline argument unboxing
[ https://issues.apache.org/jira/browse/ARROW-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-9182. - Resolution: Fixed Issue resolved by pull request 7491 [https://github.com/apache/arrow/pull/7491] > [C++] Use "applicator" namespace for kernel operator-to-kernel functors, > streamline argument unboxing > - > > Key: ARROW-9182 > URL: https://issues.apache.org/jira/browse/ARROW-9182 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > I have some changes to improve readability -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9189) [Website] Improve contributor guide
Neal Richardson created ARROW-9189: -- Summary: [Website] Improve contributor guide Key: ARROW-9189 URL: https://issues.apache.org/jira/browse/ARROW-9189 Project: Apache Arrow Issue Type: Improvement Components: Documentation, Website Reporter: Neal Richardson Assignee: Neal Richardson Fix For: 1.0.0 https://spark.apache.org/contributing.html is much richer and welcoming than ours, for example. We probably don't need everything on that page but we can do better. Includes ARROW-4429 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9190) [Website][C++] Add blog post on efforts to make building lighter and easier
Neal Richardson created ARROW-9190: -- Summary: [Website][C++] Add blog post on efforts to make building lighter and easier Key: ARROW-9190 URL: https://issues.apache.org/jira/browse/ARROW-9190 Project: Apache Arrow Issue Type: Improvement Components: Website Reporter: Neal Richardson Assignee: Neal Richardson Fix For: 1.0.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-8999) [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" build
[
https://issues.apache.org/jira/browse/ARROW-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140711#comment-17140711
]
Uwe Korn commented on ARROW-8999:
-
We could activate dumping of {{core}} files in the worker and check on failure
wether one exists and then get a traceback out of {{lldb}} for inspection.
> [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7"
> build
>
>
> Key: ARROW-8999
> URL: https://issues.apache.org/jira/browse/ARROW-8999
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Reporter: Wes McKinney
>Assignee: Krisztian Szucs
>Priority: Major
> Fix For: 2.0.0
>
>
> I've been seeing this segfault periodically the last week, does anyone have
> an idea what might be wrong?
> https://github.com/apache/arrow/pull/7273/checks?check_run_id=717249862
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ARROW-9179) [R] Replace usage of iris dataset in tests
[ https://issues.apache.org/jira/browse/ARROW-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-9179: -- Labels: pull-request-available (was: ) > [R] Replace usage of iris dataset in tests > -- > > Key: ARROW-9179 > URL: https://issues.apache.org/jira/browse/ARROW-9179 > Project: Apache Arrow > Issue Type: Improvement > Components: R >Reporter: Neal Richardson >Assignee: Neal Richardson >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > It's not a particularly interesting dataset to test with anyway. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-8800) [C++] Split arrow::ChunkedArray into arrow/chunked_array.h
[ https://issues.apache.org/jira/browse/ARROW-8800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-8800. - Resolution: Fixed Issue resolved by pull request 7487 [https://github.com/apache/arrow/pull/7487] > [C++] Split arrow::ChunkedArray into arrow/chunked_array.h > -- > > Key: ARROW-8800 > URL: https://issues.apache.org/jira/browse/ARROW-8800 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > There are plenty of scenarios where ChunkedArray is used separate from Table, > it would probably make sense to split up the headers, implementation, and > unit tests -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (ARROW-9169) [C++] Undefined reference when building flight code
[
https://issues.apache.org/jira/browse/ARROW-9169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140167#comment-17140167
]
Kouhei Sutou edited comment on ARROW-9169 at 6/19/20, 8:43 PM:
---
[~kou] Thanks a lot for your attention.
This is my cmake command line:
{noformat}
cmake -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_TESTS=ON
-DARROW_BUILD_BENCHMARKS=ON -DARROW_FLIGHT=ON -DARROW_DATASET=ON
-DARROW_PARQUET=ON -DARROW_JNI=ON -DARROW_BUILD_EXAMPLES=ON
-DBOOST_SOURCE=SYSTEM -DBoost_INCLUDE_DIR=~/software/boost_1_71_0
-DCMAKE_CXX_COMPILER=~/software/gcc-7.2.0/bin/c++
-DCMAKE_C_COMPILER=~/software/gcc-7.2.0/bin/gcc
-DTHRIFT_COMPILER=~/software/thrift-0.12.0/bin/thrift
-DTHRIFT_INCLUDE_DIR=~/software/thrift-0.12.0/include
-DTHRIFT_STATIC_LIB=~/software/thrift-0.12.0/lib/libthrift-0.12.0.so
-DGRPC_ADDRESS_SORTING_LIB=~/software/grpc/lib/libaddress_sorting.a
-DGRPC_CPP_PLUGIN=~/software/grpc/bin/grpc_cpp_plugin
-DGRPC_GPR_LIB=~/software/grpc/lib/libgpr.a
-DGRPC_GRPCPP_LIB=~/software/grpc/lib/libgrpc++.a
-DGRPC_GRPC_LIB=~/software/grpc/lib/libgrpc.a
-DGRPC_INCLUDE_DIR=~/software/grpc/include
-DGRPC_UPB_LIB=~/software/grpc/lib/libupb.a
-DProtobuf_INCLUDE_DIR=~/software/protobuf/include
-DProtobuf_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf.a
-DProtobuf_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf.a
-DProtobuf_LITE_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf-lite.a
-DProtobuf_LITE_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf-lite.a
-DProtobuf_PROTOC_EXECUTABLE=~/software/protobuf/bin/protoc
-DProtobuf_PROTOC_LIBRARY_DEBUG=~/software/protobuf/lib/libprotoc.a
-DProtobuf_PROTOC_LIBRARY_RELEASE=~/software/protobuf/lib/libprotoc.a
-Dabsl_DIR=~/src/github/build/arrow-cpp-debug/absl_ep-install ../../arrow/cpp
{noformat}
was (Author: fan_li_ya):
[~kou] Thanks a lot for your attention.
This is my cmake command line:
{{cmake -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_TESTS=ON
-DARROW_BUILD_BENCHMARKS=ON -DARROW_FLIGHT=ON -DARROW_DATASET=ON
-DARROW_PARQUET=ON -DARROW_JNI=ON -DARROW_BUILD_EXAMPLES=ON
-DBOOST_SOURCE=SYSTEM -DBoost_INCLUDE_DIR=~/software/boost_1_71_0
-DCMAKE_CXX_COMPILER=~/software/gcc-7.2.0/bin/c++
-DCMAKE_C_COMPILER=~/software/gcc-7.2.0/bin/gcc
-DTHRIFT_COMPILER=~/software/thrift-0.12.0/bin/thrift
-DTHRIFT_INCLUDE_DIR=~/software/thrift-0.12.0/include
-DTHRIFT_STATIC_LIB=~/software/thrift-0.12.0/lib/libthrift-0.12.0.so
-DGRPC_ADDRESS_SORTING_LIB=~/software/grpc/lib/libaddress_sorting.a
-DGRPC_CPP_PLUGIN=~/software/grpc/bin/grpc_cpp_plugin
-DGRPC_GPR_LIB=~/software/grpc/lib/libgpr.a
-DGRPC_GRPCPP_LIB=~/software/grpc/lib/libgrpc++.a
-DGRPC_GRPC_LIB=~/software/grpc/lib/libgrpc.a
-DGRPC_INCLUDE_DIR=~/software/grpc/include
-DGRPC_UPB_LIB=~/software/grpc/lib/libupb.a
-DProtobuf_INCLUDE_DIR=~/software/protobuf/include
-DProtobuf_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf.a
-DProtobuf_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf.a
-DProtobuf_LITE_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf-lite.a
-DProtobuf_LITE_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf-lite.a
-DProtobuf_PROTOC_EXECUTABLE=~/software/protobuf/bin/protoc
-DProtobuf_PROTOC_LIBRARY_DEBUG=~/software/protobuf/lib/libprotoc.a
-DProtobuf_PROTOC_LIBRARY_RELEASE=~/software/protobuf/lib/libprotoc.a
-Dabsl_DIR=~/src/github/build/arrow-cpp-debug/absl_ep-install ../../arrow/cpp}}
> [C++] Undefined reference when building flight code
> ---
>
> Key: ARROW-9169
> URL: https://issues.apache.org/jira/browse/ARROW-9169
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Liya Fan
>Priority: Major
>
> When linking flight-test-server with
> CMakeFiles/flight-test-server.dir/link.txt, I get the following error:
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::StrCat[abi:cxx11](absl::AlphaNum const&, absl::AlphaNum const&,
> absl::AlphaNum const&)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::strings_internal::CatPieces[abi:cxx11](std::initializer_list)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::numbers_internal::FastIntToBuffer(long, char*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::numbers_internal::FastIntToBuffer(unsigned long, char*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::str_format_internal::FormatPack[abi:cxx11](absl::str_format_internal::UntypedFormatSpecImpl,
> absl::Span)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::numbers_internal::SixDigitsToBuffer(double, char*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::base_internal::ThrowStdOutOfRange(char const*)'
> ../..
[jira] [Comment Edited] (ARROW-9169) [C++] Undefined reference when building flight code
[
https://issues.apache.org/jira/browse/ARROW-9169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140167#comment-17140167
]
Kouhei Sutou edited comment on ARROW-9169 at 6/19/20, 8:49 PM:
---
[~kou] Thanks a lot for your attention.
This is my cmake command line:
{noformat}
cmake \
-DCMAKE_BUILD_TYPE=Debug \
-DARROW_BUILD_TESTS=ON \
-DARROW_BUILD_BENCHMARKS=ON \
-DARROW_FLIGHT=ON \
-DARROW_DATASET=ON \
-DARROW_PARQUET=ON \
-DARROW_JNI=ON \
-DARROW_BUILD_EXAMPLES=ON \
-DBOOST_SOURCE=SYSTEM \
-DBoost_INCLUDE_DIR=~/software/boost_1_71_0 \
-DCMAKE_CXX_COMPILER=~/software/gcc-7.2.0/bin/c++ \
-DCMAKE_C_COMPILER=~/software/gcc-7.2.0/bin/gcc \
-DTHRIFT_COMPILER=~/software/thrift-0.12.0/bin/thrift \
-DTHRIFT_INCLUDE_DIR=~/software/thrift-0.12.0/include \
-DTHRIFT_STATIC_LIB=~/software/thrift-0.12.0/lib/libthrift-0.12.0.so \
-DGRPC_ADDRESS_SORTING_LIB=~/software/grpc/lib/libaddress_sorting.a \
-DGRPC_CPP_PLUGIN=~/software/grpc/bin/grpc_cpp_plugin \
-DGRPC_GPR_LIB=~/software/grpc/lib/libgpr.a \
-DGRPC_GRPCPP_LIB=~/software/grpc/lib/libgrpc++.a \
-DGRPC_GRPC_LIB=~/software/grpc/lib/libgrpc.a \
-DGRPC_INCLUDE_DIR=~/software/grpc/include \
-DGRPC_UPB_LIB=~/software/grpc/lib/libupb.a \
-DProtobuf_INCLUDE_DIR=~/software/protobuf/include \
-DProtobuf_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf.a \
-DProtobuf_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf.a \
-DProtobuf_LITE_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf-lite.a \
-DProtobuf_LITE_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf-lite.a \
-DProtobuf_PROTOC_EXECUTABLE=~/software/protobuf/bin/protoc \
-DProtobuf_PROTOC_LIBRARY_DEBUG=~/software/protobuf/lib/libprotoc.a \
-DProtobuf_PROTOC_LIBRARY_RELEASE=~/software/protobuf/lib/libprotoc.a \
-Dabsl_DIR=~/src/github/build/arrow-cpp-debug/absl_ep-install \
../../arrow/cpp
{noformat}
was (Author: fan_li_ya):
[~kou] Thanks a lot for your attention.
This is my cmake command line:
{noformat}
cmake -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_TESTS=ON
-DARROW_BUILD_BENCHMARKS=ON -DARROW_FLIGHT=ON -DARROW_DATASET=ON
-DARROW_PARQUET=ON -DARROW_JNI=ON -DARROW_BUILD_EXAMPLES=ON
-DBOOST_SOURCE=SYSTEM -DBoost_INCLUDE_DIR=~/software/boost_1_71_0
-DCMAKE_CXX_COMPILER=~/software/gcc-7.2.0/bin/c++
-DCMAKE_C_COMPILER=~/software/gcc-7.2.0/bin/gcc
-DTHRIFT_COMPILER=~/software/thrift-0.12.0/bin/thrift
-DTHRIFT_INCLUDE_DIR=~/software/thrift-0.12.0/include
-DTHRIFT_STATIC_LIB=~/software/thrift-0.12.0/lib/libthrift-0.12.0.so
-DGRPC_ADDRESS_SORTING_LIB=~/software/grpc/lib/libaddress_sorting.a
-DGRPC_CPP_PLUGIN=~/software/grpc/bin/grpc_cpp_plugin
-DGRPC_GPR_LIB=~/software/grpc/lib/libgpr.a
-DGRPC_GRPCPP_LIB=~/software/grpc/lib/libgrpc++.a
-DGRPC_GRPC_LIB=~/software/grpc/lib/libgrpc.a
-DGRPC_INCLUDE_DIR=~/software/grpc/include
-DGRPC_UPB_LIB=~/software/grpc/lib/libupb.a
-DProtobuf_INCLUDE_DIR=~/software/protobuf/include
-DProtobuf_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf.a
-DProtobuf_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf.a
-DProtobuf_LITE_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf-lite.a
-DProtobuf_LITE_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf-lite.a
-DProtobuf_PROTOC_EXECUTABLE=~/software/protobuf/bin/protoc
-DProtobuf_PROTOC_LIBRARY_DEBUG=~/software/protobuf/lib/libprotoc.a
-DProtobuf_PROTOC_LIBRARY_RELEASE=~/software/protobuf/lib/libprotoc.a
-Dabsl_DIR=~/src/github/build/arrow-cpp-debug/absl_ep-install ../../arrow/cpp
{noformat}
> [C++] Undefined reference when building flight code
> ---
>
> Key: ARROW-9169
> URL: https://issues.apache.org/jira/browse/ARROW-9169
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Liya Fan
>Priority: Major
>
> When linking flight-test-server with
> CMakeFiles/flight-test-server.dir/link.txt, I get the following error:
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::StrCat[abi:cxx11](absl::AlphaNum const&, absl::AlphaNum const&,
> absl::AlphaNum const&)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::strings_internal::CatPieces[abi:cxx11](std::initializer_list)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::numbers_internal::FastIntToBuffer(long, char*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::numbers_internal::FastIntToBuffer(unsigned long, char*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::str_format_internal::FormatPack[abi:cxx11](absl::str_format_internal::UntypedFormatSpecImpl,
> absl::Span)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::numbers_internal::SixDigitsToBuffer(double, char*)'
> ../../../d
[jira] [Commented] (ARROW-9169) [C++] Undefined reference when building flight code
[
https://issues.apache.org/jira/browse/ARROW-9169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140831#comment-17140831
]
Kouhei Sutou commented on ARROW-9169:
-
Thanks.
It seems that Abseil isn't found by {{find_package(absl CONFIG)}} in
{{cpp/cmake_modules/FindRPCAlt.cmake}}.
Does
{{~/src/github/build/arrow-cpp-debug/absl_ep-install/lib/cmake/absl/abslConfig.cmake}}
exist?
If it exists, could you try {{absl_DIR=~/src/... cmake ...}} command line?
{{_DIR}} is an environment variable not a CMake variable.
> [C++] Undefined reference when building flight code
> ---
>
> Key: ARROW-9169
> URL: https://issues.apache.org/jira/browse/ARROW-9169
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Reporter: Liya Fan
>Priority: Major
>
> When linking flight-test-server with
> CMakeFiles/flight-test-server.dir/link.txt, I get the following error:
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::StrCat[abi:cxx11](absl::AlphaNum const&, absl::AlphaNum const&,
> absl::AlphaNum const&)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::strings_internal::CatPieces[abi:cxx11](std::initializer_list)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::numbers_internal::FastIntToBuffer(long, char*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::numbers_internal::FastIntToBuffer(unsigned long, char*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::str_format_internal::FormatPack[abi:cxx11](absl::str_format_internal::UntypedFormatSpecImpl,
> absl::Span)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::numbers_internal::SixDigitsToBuffer(double, char*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::base_internal::ThrowStdOutOfRange(char const*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::string_view::find(char, unsigned long) const'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::StrCat[abi:cxx11](absl::AlphaNum const&, absl::AlphaNum const&,
> absl::AlphaNum const&, absl::AlphaNum const&)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::FormatTime[abi:cxx11](absl::Time)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to `bool
> absl::str_format_internal::FormatArgImpl::Dispatch const*>(absl::str_format_internal::FormatArgImpl::Data,
> absl::str_format_internal::FormatConversionSpecImpl, void*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to `bool
> absl::str_format_internal::FormatArgImpl::Dispatch std::char_traits, std::allocator >
> >(absl::str_format_internal::FormatArgImpl::Data,
> absl::str_format_internal::FormatConversionSpecImpl, void*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::numbers_internal::FastIntToBuffer(unsigned int, char*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::EqualsIgnoreCase(absl::string_view, absl::string_view)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::StrCat[abi:cxx11](absl::AlphaNum const&, absl::AlphaNum const&)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::numbers_internal::FastIntToBuffer(int, char*)'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::optional_internal::throw_bad_optional_access()'
> ../../../debug/libarrow_flight.so.100.0.0: undefined reference to
> `absl::ByChar::Find(absl::string_view, unsigned long) const'
> it seems absl has been installed successfully by cmake.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ARROW-9183) [C++] Failed to build arrow-cpp with gcc 4.9.2
[ https://issues.apache.org/jira/browse/ARROW-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou reassigned ARROW-9183: --- Assignee: Shuai Zhang > [C++] Failed to build arrow-cpp with gcc 4.9.2 > -- > > Key: ARROW-9183 > URL: https://issues.apache.org/jira/browse/ARROW-9183 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Affects Versions: 0.17.1 >Reporter: Shuai Zhang >Assignee: Shuai Zhang >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Attachments: fix.patch, image-2020-06-19-11-29-13-641.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > The [Building Arrow > C++|https://github.com/apache/arrow/blob/apache-arrow-0.17.1/docs/source/developers/cpp/building.rst] > document say "building requires a C++11-enabled compiler. On Linux, gcc 4.8 > and higher should be sufficient." > But actually, it failed to build with gcc 4.9.2 (see attached > image-2020-06-19-11-29-13-641.png). This is because a bug in gcc [Bug > 57250|https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57250] > Please note it in the document to lead using a higher version of gcc. Current > document is still misleading. > > NEED TRIAGE, because the > [atomic_shared_ptr.h|https://github.com/apache/arrow/blob/apache-arrow-0.17.1/cpp/src/arrow/util/atomic_shared_ptr.h] > file is aim to eliminate the bug. But the error occurred when compiling it. > OK, got it. I installed clang but still using the gcc 4.9.2's libstdc++. The > bug appeared then. > I think it more reliable to check libstdc++ version instead of the gcc > version, because this is a libstdc++ bug instead of a gcc bug. I found it > possible to [check the libstdc++ version via MACRO > \_\_GLIBCXX\_\_|https://stackoverflow.com/questions/10354636/how-do-you-find-what-version-of-libstdc-library-is-installed-on-your-linux-mac] > with help of [this timeline table|https://gcc.gnu.org/develop.html#timeline]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9191) [Rust] Do not panic when int96 milliseconds are negative
Max Burke created ARROW-9191: Summary: [Rust] Do not panic when int96 milliseconds are negative Key: ARROW-9191 URL: https://issues.apache.org/jira/browse/ARROW-9191 Project: Apache Arrow Issue Type: Improvement Reporter: Max Burke See GitHub PR: -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9191) [Rust] Do not panic when int96 milliseconds are negative
[ https://issues.apache.org/jira/browse/ARROW-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-9191: -- Labels: pull-request-available (was: ) > [Rust] Do not panic when int96 milliseconds are negative > > > Key: ARROW-9191 > URL: https://issues.apache.org/jira/browse/ARROW-9191 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Max Burke >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > See GitHub PR: [https://github.com/apache/arrow/pull/7500] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9191) [Rust] Do not panic when int96 milliseconds are negative
[ https://issues.apache.org/jira/browse/ARROW-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Burke updated ARROW-9191: - Description: See GitHub PR: [https://github.com/apache/arrow/pull/7500] (was: See GitHub PR: ) > [Rust] Do not panic when int96 milliseconds are negative > > > Key: ARROW-9191 > URL: https://issues.apache.org/jira/browse/ARROW-9191 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Max Burke >Priority: Major > > See GitHub PR: [https://github.com/apache/arrow/pull/7500] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-9179) [R] Replace usage of iris dataset in tests
[ https://issues.apache.org/jira/browse/ARROW-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neal Richardson resolved ARROW-9179. Resolution: Fixed Issue resolved by pull request 7499 [https://github.com/apache/arrow/pull/7499] > [R] Replace usage of iris dataset in tests > -- > > Key: ARROW-9179 > URL: https://issues.apache.org/jira/browse/ARROW-9179 > Project: Apache Arrow > Issue Type: Improvement > Components: R >Reporter: Neal Richardson >Assignee: Neal Richardson >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > It's not a particularly interesting dataset to test with anyway. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9192) [Rust] Enable clippy linting for arrow crate in CI pipeline
QP Hou created ARROW-9192: - Summary: [Rust] Enable clippy linting for arrow crate in CI pipeline Key: ARROW-9192 URL: https://issues.apache.org/jira/browse/ARROW-9192 Project: Apache Arrow Issue Type: Improvement Components: Rust Reporter: QP Hou Assignee: QP Hou -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9192) [Rust] Enable clippy linting for arrow crate in CI pipeline
[ https://issues.apache.org/jira/browse/ARROW-9192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-9192: -- Labels: pull-request-available (was: ) > [Rust] Enable clippy linting for arrow crate in CI pipeline > > > Key: ARROW-9192 > URL: https://issues.apache.org/jira/browse/ARROW-9192 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Reporter: QP Hou >Assignee: QP Hou >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
