[jira] [Resolved] (ARROW-7702) [C++][Dataset] Provide (optional) deterministic order of batches
[ https://issues.apache.org/jira/browse/ARROW-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joris Van den Bossche resolved ARROW-7702. -- Fix Version/s: 1.0.0 Assignee: Francois Saint-Jacques Resolution: Duplicate > [C++][Dataset] Provide (optional) deterministic order of batches > > > Key: ARROW-7702 > URL: https://issues.apache.org/jira/browse/ARROW-7702 > Project: Apache Arrow > Issue Type: Bug > Components: C++, Python >Reporter: Joris Van den Bossche >Assignee: Francois Saint-Jacques >Priority: Major > Labels: dataset > Fix For: 1.0.0 > > > Example with python: > {code} > import pyarrow as pa > import pyarrow.parquet as pq > table = pa.table({'a': range(12)}) > pq.write_table(table, "test_chunks.parquet", chunk_size=3) > # reading with dataset > import pyarrow.dataset as ds > ds.dataset("test_chunks.parquet").to_table().to_pandas() > {code} > gives non-deterministic result (order of the row groups in the parquet file): > {code} > In [25]: ds.dataset("test_chunks.parquet").to_table().to_pandas() > > > Out[25]: > a > 00 > 11 > 22 > 33 > 44 > 55 > 66 > 77 > 88 > 99 > 10 10 > 11 11 > In [26]: ds.dataset("test_chunks.parquet").to_table().to_pandas() > > > Out[26]: > a > 00 > 11 > 22 > 33 > 48 > 59 > 6 10 > 7 11 > 84 > 95 > 10 6 > 11 7 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7702) [C++][Dataset] Provide (optional) deterministic order of batches
[ https://issues.apache.org/jira/browse/ARROW-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140294#comment-17140294 ] Joris Van den Bossche commented on ARROW-7702: -- Indeed, this was fixed. Thanks for noting > [C++][Dataset] Provide (optional) deterministic order of batches > > > Key: ARROW-7702 > URL: https://issues.apache.org/jira/browse/ARROW-7702 > Project: Apache Arrow > Issue Type: Bug > Components: C++, Python >Reporter: Joris Van den Bossche >Assignee: Francois Saint-Jacques >Priority: Major > Labels: dataset > Fix For: 1.0.0 > > > Example with python: > {code} > import pyarrow as pa > import pyarrow.parquet as pq > table = pa.table({'a': range(12)}) > pq.write_table(table, "test_chunks.parquet", chunk_size=3) > # reading with dataset > import pyarrow.dataset as ds > ds.dataset("test_chunks.parquet").to_table().to_pandas() > {code} > gives non-deterministic result (order of the row groups in the parquet file): > {code} > In [25]: ds.dataset("test_chunks.parquet").to_table().to_pandas() > > > Out[25]: > a > 00 > 11 > 22 > 33 > 44 > 55 > 66 > 77 > 88 > 99 > 10 10 > 11 11 > In [26]: ds.dataset("test_chunks.parquet").to_table().to_pandas() > > > Out[26]: > a > 00 > 11 > 22 > 33 > 48 > 59 > 6 10 > 7 11 > 84 > 95 > 10 6 > 11 7 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7945) [C++][Dataset] Implement InMemoryDatasetFactory
[ https://issues.apache.org/jira/browse/ARROW-7945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140306#comment-17140306 ] Joris Van den Bossche commented on ARROW-7945: -- [~bivald] that would indeed enable that, but not that you can already filter Tables/RecordBatches right now (only a different syntax, with the {{filter()}} method expecting a boolean mask) > [C++][Dataset] Implement InMemoryDatasetFactory > --- > > Key: ARROW-7945 > URL: https://issues.apache.org/jira/browse/ARROW-7945 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Affects Versions: 0.16.0 >Reporter: Ben Kietzman >Assignee: Ben Kietzman >Priority: Major > Labels: dataset > > This will allow in memory datasets (such as tables) to participate in > discovery through {{UnionDatasetFactory}}. This class will be trivial since > Inspect will do nothing but return the table's schema, but is necessary to > ensure that the resulting {{UnionDataset}}'s unified schema accommodates the > table's schema (for example including fields present only in the table's > schema or emitting an error when unification is not possible) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-7945) [C++][Dataset] Implement InMemoryDatasetFactory
[ https://issues.apache.org/jira/browse/ARROW-7945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140365#comment-17140365 ] Niklas B commented on ARROW-7945: - [~jorisvandenbossche] That's a good point! I have tried using numpy to create a boolean mask and while that operations is zero copy and lightning fast, the actual filter takes a fair bit of time (2 seconds) to filter out about 50% of 700k rows (~200 columns). I'm guessing its iterating each column, where each column takes about 0.01s. Possibly I can improve this if I write a function in cython/c++, but it's slightly out of my abilities. I'll probably wait for the Dataset API :) > [C++][Dataset] Implement InMemoryDatasetFactory > --- > > Key: ARROW-7945 > URL: https://issues.apache.org/jira/browse/ARROW-7945 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Affects Versions: 0.16.0 >Reporter: Ben Kietzman >Assignee: Ben Kietzman >Priority: Major > Labels: dataset > > This will allow in memory datasets (such as tables) to participate in > discovery through {{UnionDatasetFactory}}. This class will be trivial since > Inspect will do nothing but return the table's schema, but is necessary to > ensure that the resulting {{UnionDataset}}'s unified schema accommodates the > table's schema (for example including fields present only in the table's > schema or emitting an error when unification is not possible) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-7084) [C++] ArrayRangeEquals should check for full type equality?
[ https://issues.apache.org/jira/browse/ARROW-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-7084: -- Labels: pull-request-available (was: ) > [C++] ArrayRangeEquals should check for full type equality? > > > Key: ARROW-7084 > URL: https://issues.apache.org/jira/browse/ARROW-7084 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Micah Kornfield >Assignee: Ji Liu >Priority: Major > Labels: pull-request-available > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > It looks like ArrayRangeEquals in compare.cc only checks type IDs before > doing comparison actual values. This is inconsistent with ArrayEquals which > checks for type equality and also seems incorrect for cases like Decimal128. > > I presume this was an oversight when fixing ARROW-2567 but maybe it was > intentional? > [~uwe]? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-8149) [C++/Python] Enable CUDA Support in conda recipes
[ https://issues.apache.org/jira/browse/ARROW-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Korn reassigned ARROW-8149: --- Assignee: Uwe Korn > [C++/Python] Enable CUDA Support in conda recipes > - > > Key: ARROW-8149 > URL: https://issues.apache.org/jira/browse/ARROW-8149 > Project: Apache Arrow > Issue Type: New Feature > Components: C++, Packaging >Reporter: Uwe Korn >Assignee: Uwe Korn >Priority: Major > Fix For: 1.0.0 > > > See the changes in > [https://github.com/conda-forge/arrow-cpp-feedstock/pull/123], we need to > copy this into the Arrow repository and also test CUDA in these recipes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8149) [C++/Python] Enable CUDA Support in conda recipes
[ https://issues.apache.org/jira/browse/ARROW-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-8149: -- Labels: pull-request-available (was: ) > [C++/Python] Enable CUDA Support in conda recipes > - > > Key: ARROW-8149 > URL: https://issues.apache.org/jira/browse/ARROW-8149 > Project: Apache Arrow > Issue Type: New Feature > Components: C++, Packaging >Reporter: Uwe Korn >Assignee: Uwe Korn >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > See the changes in > [https://github.com/conda-forge/arrow-cpp-feedstock/pull/123], we need to > copy this into the Arrow repository and also test CUDA in these recipes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-8895) [C++] Add C++ unit tests for filter and take functions on temporal type inputs, including timestamps
[ https://issues.apache.org/jira/browse/ARROW-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-8895. - Resolution: Fixed Issue resolved by pull request 7485 [https://github.com/apache/arrow/pull/7485] > [C++] Add C++ unit tests for filter and take functions on temporal type > inputs, including timestamps > > > Key: ARROW-8895 > URL: https://issues.apache.org/jira/browse/ARROW-8895 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > These are used in R but not tested in C++, so I only found out that I had > missed adding the kernels to the Filter VectorFunction when running the R > test suite -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9091) [C++] Utilize function's default options when passing no options to CallFunction for a function that requires them
[ https://issues.apache.org/jira/browse/ARROW-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-9091: -- Labels: pull-request-available (was: ) > [C++] Utilize function's default options when passing no options to > CallFunction for a function that requires them > -- > > Key: ARROW-9091 > URL: https://issues.apache.org/jira/browse/ARROW-9091 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Ben Kietzman >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Otherwise benign usage of {{CallFunction}} can cause an unintuitive segfault > in some cases -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9186) [R] Allow specifying CSV file encoding
Neal Richardson created ARROW-9186: -- Summary: [R] Allow specifying CSV file encoding Key: ARROW-9186 URL: https://issues.apache.org/jira/browse/ARROW-9186 Project: Apache Arrow Issue Type: Improvement Components: R Reporter: Neal Richardson Fix For: 2.0.0 ARROW-9106 did this for Python and we should have the same in R -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-8999) [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" build
[ https://issues.apache.org/jira/browse/ARROW-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neal Richardson updated ARROW-8999: --- Fix Version/s: (was: 1.0.0) 2.0.0 > [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" > build > > > Key: ARROW-8999 > URL: https://issues.apache.org/jira/browse/ARROW-8999 > Project: Apache Arrow > Issue Type: Bug > Components: Python >Reporter: Wes McKinney >Assignee: Krisztian Szucs >Priority: Major > Fix For: 2.0.0 > > > I've been seeing this segfault periodically the last week, does anyone have > an idea what might be wrong? > https://github.com/apache/arrow/pull/7273/checks?check_run_id=717249862 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-8999) [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" build
[ https://issues.apache.org/jira/browse/ARROW-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140654#comment-17140654 ] Wes McKinney commented on ARROW-8999: - [~kszucs] seems like GHA does not have segfault reporting, any way we could get a backtrace to show up when the crash happens? > [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" > build > > > Key: ARROW-8999 > URL: https://issues.apache.org/jira/browse/ARROW-8999 > Project: Apache Arrow > Issue Type: Bug > Components: Python >Reporter: Wes McKinney >Assignee: Krisztian Szucs >Priority: Major > Fix For: 2.0.0 > > > I've been seeing this segfault periodically the last week, does anyone have > an idea what might be wrong? > https://github.com/apache/arrow/pull/7273/checks?check_run_id=717249862 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9187) [R] Add bindings for arithmetic and is_null kernels
Neal Richardson created ARROW-9187: -- Summary: [R] Add bindings for arithmetic and is_null kernels Key: ARROW-9187 URL: https://issues.apache.org/jira/browse/ARROW-9187 Project: Apache Arrow Issue Type: Improvement Components: R Reporter: Neal Richardson Fix For: 2.0.0 These were done in Python in ARROW-9162 and ARROW-9159 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-9121) [C++] Do not wipe the filesystem when path is empty
[ https://issues.apache.org/jira/browse/ARROW-9121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140663#comment-17140663 ] Krisztian Szucs commented on ARROW-9121: [~apitrou] What would your suggestion here? Have another method for wiping the entire fs and raise from DeleteDirContents if empty path was passed? > [C++] Do not wipe the filesystem when path is empty > --- > > Key: ARROW-9121 > URL: https://issues.apache.org/jira/browse/ARROW-9121 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Mohamed Zenadi >Assignee: Krisztian Szucs >Priority: Minor > Fix For: 1.0.0 > > > The `DeleteDirContents` method in the filesystems api has a default behavior > or *wiping* the whole filesystem if we give it an empty path. > > It's documented as: > > Like DeleteDir, but doesn’t delete the directory itself. Passing an empty > path (“”) will wipe the entire filesystem tree. > > And the corresponding code confirms that: > {code:java} > auto parts = SplitAbstractPath(path); > RETURN_NOT_OK(ValidateAbstractPathParts(parts)); > > if (parts.empty()) { > // Wipe filesystem > impl_->RootDir().entries.clear(); > return Status::OK(); > } > {code} > > This is a weird default that does not make sense. If the user wanted really > to wipe his filesystem, he'd pass a `/`. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9188) [C++] Do not always statically link Brotli libraries
Wes McKinney created ARROW-9188: --- Summary: [C++] Do not always statically link Brotli libraries Key: ARROW-9188 URL: https://issues.apache.org/jira/browse/ARROW-9188 Project: Apache Arrow Issue Type: Improvement Components: C++ Reporter: Wes McKinney Fix For: 1.0.0 I discovered by symbol sleuthing that even if the Brotli shared libraries are available on the system, the static libraries will be used for linking. Here's a dump of the symbols on my machine https://gist.github.com/wesm/b7e911424ce3ca3a175b652666fe6f89 The largest of these symbols {{kBrotliDictionaryData}} is 120KB Since we statically link everything in our Python wheels, we might consider disabling Brotli in manylinux builds at least since it adds 800KB to libarrow.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-9182) [C++] Use "applicator" namespace for kernel operator-to-kernel functors, streamline argument unboxing
[ https://issues.apache.org/jira/browse/ARROW-9182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-9182. - Resolution: Fixed Issue resolved by pull request 7491 [https://github.com/apache/arrow/pull/7491] > [C++] Use "applicator" namespace for kernel operator-to-kernel functors, > streamline argument unboxing > - > > Key: ARROW-9182 > URL: https://issues.apache.org/jira/browse/ARROW-9182 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > I have some changes to improve readability -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9189) [Website] Improve contributor guide
Neal Richardson created ARROW-9189: -- Summary: [Website] Improve contributor guide Key: ARROW-9189 URL: https://issues.apache.org/jira/browse/ARROW-9189 Project: Apache Arrow Issue Type: Improvement Components: Documentation, Website Reporter: Neal Richardson Assignee: Neal Richardson Fix For: 1.0.0 https://spark.apache.org/contributing.html is much richer and welcoming than ours, for example. We probably don't need everything on that page but we can do better. Includes ARROW-4429 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9190) [Website][C++] Add blog post on efforts to make building lighter and easier
Neal Richardson created ARROW-9190: -- Summary: [Website][C++] Add blog post on efforts to make building lighter and easier Key: ARROW-9190 URL: https://issues.apache.org/jira/browse/ARROW-9190 Project: Apache Arrow Issue Type: Improvement Components: Website Reporter: Neal Richardson Assignee: Neal Richardson Fix For: 1.0.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ARROW-8999) [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" build
[ https://issues.apache.org/jira/browse/ARROW-8999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140711#comment-17140711 ] Uwe Korn commented on ARROW-8999: - We could activate dumping of {{core}} files in the worker and check on failure wether one exists and then get a traceback out of {{lldb}} for inspection. > [Python][C++] Non-deterministic segfault in "AMD64 MacOS 10.15 Python 3.7" > build > > > Key: ARROW-8999 > URL: https://issues.apache.org/jira/browse/ARROW-8999 > Project: Apache Arrow > Issue Type: Bug > Components: Python >Reporter: Wes McKinney >Assignee: Krisztian Szucs >Priority: Major > Fix For: 2.0.0 > > > I've been seeing this segfault periodically the last week, does anyone have > an idea what might be wrong? > https://github.com/apache/arrow/pull/7273/checks?check_run_id=717249862 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9179) [R] Replace usage of iris dataset in tests
[ https://issues.apache.org/jira/browse/ARROW-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-9179: -- Labels: pull-request-available (was: ) > [R] Replace usage of iris dataset in tests > -- > > Key: ARROW-9179 > URL: https://issues.apache.org/jira/browse/ARROW-9179 > Project: Apache Arrow > Issue Type: Improvement > Components: R >Reporter: Neal Richardson >Assignee: Neal Richardson >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > It's not a particularly interesting dataset to test with anyway. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-8800) [C++] Split arrow::ChunkedArray into arrow/chunked_array.h
[ https://issues.apache.org/jira/browse/ARROW-8800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney resolved ARROW-8800. - Resolution: Fixed Issue resolved by pull request 7487 [https://github.com/apache/arrow/pull/7487] > [C++] Split arrow::ChunkedArray into arrow/chunked_array.h > -- > > Key: ARROW-8800 > URL: https://issues.apache.org/jira/browse/ARROW-8800 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ >Reporter: Wes McKinney >Assignee: Wes McKinney >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > There are plenty of scenarios where ChunkedArray is used separate from Table, > it would probably make sense to split up the headers, implementation, and > unit tests -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (ARROW-9169) [C++] Undefined reference when building flight code
[ https://issues.apache.org/jira/browse/ARROW-9169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140167#comment-17140167 ] Kouhei Sutou edited comment on ARROW-9169 at 6/19/20, 8:43 PM: --- [~kou] Thanks a lot for your attention. This is my cmake command line: {noformat} cmake -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_TESTS=ON -DARROW_BUILD_BENCHMARKS=ON -DARROW_FLIGHT=ON -DARROW_DATASET=ON -DARROW_PARQUET=ON -DARROW_JNI=ON -DARROW_BUILD_EXAMPLES=ON -DBOOST_SOURCE=SYSTEM -DBoost_INCLUDE_DIR=~/software/boost_1_71_0 -DCMAKE_CXX_COMPILER=~/software/gcc-7.2.0/bin/c++ -DCMAKE_C_COMPILER=~/software/gcc-7.2.0/bin/gcc -DTHRIFT_COMPILER=~/software/thrift-0.12.0/bin/thrift -DTHRIFT_INCLUDE_DIR=~/software/thrift-0.12.0/include -DTHRIFT_STATIC_LIB=~/software/thrift-0.12.0/lib/libthrift-0.12.0.so -DGRPC_ADDRESS_SORTING_LIB=~/software/grpc/lib/libaddress_sorting.a -DGRPC_CPP_PLUGIN=~/software/grpc/bin/grpc_cpp_plugin -DGRPC_GPR_LIB=~/software/grpc/lib/libgpr.a -DGRPC_GRPCPP_LIB=~/software/grpc/lib/libgrpc++.a -DGRPC_GRPC_LIB=~/software/grpc/lib/libgrpc.a -DGRPC_INCLUDE_DIR=~/software/grpc/include -DGRPC_UPB_LIB=~/software/grpc/lib/libupb.a -DProtobuf_INCLUDE_DIR=~/software/protobuf/include -DProtobuf_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf.a -DProtobuf_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf.a -DProtobuf_LITE_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf-lite.a -DProtobuf_LITE_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf-lite.a -DProtobuf_PROTOC_EXECUTABLE=~/software/protobuf/bin/protoc -DProtobuf_PROTOC_LIBRARY_DEBUG=~/software/protobuf/lib/libprotoc.a -DProtobuf_PROTOC_LIBRARY_RELEASE=~/software/protobuf/lib/libprotoc.a -Dabsl_DIR=~/src/github/build/arrow-cpp-debug/absl_ep-install ../../arrow/cpp {noformat} was (Author: fan_li_ya): [~kou] Thanks a lot for your attention. This is my cmake command line: {{cmake -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_TESTS=ON -DARROW_BUILD_BENCHMARKS=ON -DARROW_FLIGHT=ON -DARROW_DATASET=ON -DARROW_PARQUET=ON -DARROW_JNI=ON -DARROW_BUILD_EXAMPLES=ON -DBOOST_SOURCE=SYSTEM -DBoost_INCLUDE_DIR=~/software/boost_1_71_0 -DCMAKE_CXX_COMPILER=~/software/gcc-7.2.0/bin/c++ -DCMAKE_C_COMPILER=~/software/gcc-7.2.0/bin/gcc -DTHRIFT_COMPILER=~/software/thrift-0.12.0/bin/thrift -DTHRIFT_INCLUDE_DIR=~/software/thrift-0.12.0/include -DTHRIFT_STATIC_LIB=~/software/thrift-0.12.0/lib/libthrift-0.12.0.so -DGRPC_ADDRESS_SORTING_LIB=~/software/grpc/lib/libaddress_sorting.a -DGRPC_CPP_PLUGIN=~/software/grpc/bin/grpc_cpp_plugin -DGRPC_GPR_LIB=~/software/grpc/lib/libgpr.a -DGRPC_GRPCPP_LIB=~/software/grpc/lib/libgrpc++.a -DGRPC_GRPC_LIB=~/software/grpc/lib/libgrpc.a -DGRPC_INCLUDE_DIR=~/software/grpc/include -DGRPC_UPB_LIB=~/software/grpc/lib/libupb.a -DProtobuf_INCLUDE_DIR=~/software/protobuf/include -DProtobuf_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf.a -DProtobuf_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf.a -DProtobuf_LITE_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf-lite.a -DProtobuf_LITE_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf-lite.a -DProtobuf_PROTOC_EXECUTABLE=~/software/protobuf/bin/protoc -DProtobuf_PROTOC_LIBRARY_DEBUG=~/software/protobuf/lib/libprotoc.a -DProtobuf_PROTOC_LIBRARY_RELEASE=~/software/protobuf/lib/libprotoc.a -Dabsl_DIR=~/src/github/build/arrow-cpp-debug/absl_ep-install ../../arrow/cpp}} > [C++] Undefined reference when building flight code > --- > > Key: ARROW-9169 > URL: https://issues.apache.org/jira/browse/ARROW-9169 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Liya Fan >Priority: Major > > When linking flight-test-server with > CMakeFiles/flight-test-server.dir/link.txt, I get the following error: > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::StrCat[abi:cxx11](absl::AlphaNum const&, absl::AlphaNum const&, > absl::AlphaNum const&)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::strings_internal::CatPieces[abi:cxx11](std::initializer_list)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::numbers_internal::FastIntToBuffer(long, char*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::numbers_internal::FastIntToBuffer(unsigned long, char*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::str_format_internal::FormatPack[abi:cxx11](absl::str_format_internal::UntypedFormatSpecImpl, > absl::Span)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::numbers_internal::SixDigitsToBuffer(double, char*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::base_internal::ThrowStdOutOfRange(char const*)' > ../..
[jira] [Comment Edited] (ARROW-9169) [C++] Undefined reference when building flight code
[ https://issues.apache.org/jira/browse/ARROW-9169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140167#comment-17140167 ] Kouhei Sutou edited comment on ARROW-9169 at 6/19/20, 8:49 PM: --- [~kou] Thanks a lot for your attention. This is my cmake command line: {noformat} cmake \ -DCMAKE_BUILD_TYPE=Debug \ -DARROW_BUILD_TESTS=ON \ -DARROW_BUILD_BENCHMARKS=ON \ -DARROW_FLIGHT=ON \ -DARROW_DATASET=ON \ -DARROW_PARQUET=ON \ -DARROW_JNI=ON \ -DARROW_BUILD_EXAMPLES=ON \ -DBOOST_SOURCE=SYSTEM \ -DBoost_INCLUDE_DIR=~/software/boost_1_71_0 \ -DCMAKE_CXX_COMPILER=~/software/gcc-7.2.0/bin/c++ \ -DCMAKE_C_COMPILER=~/software/gcc-7.2.0/bin/gcc \ -DTHRIFT_COMPILER=~/software/thrift-0.12.0/bin/thrift \ -DTHRIFT_INCLUDE_DIR=~/software/thrift-0.12.0/include \ -DTHRIFT_STATIC_LIB=~/software/thrift-0.12.0/lib/libthrift-0.12.0.so \ -DGRPC_ADDRESS_SORTING_LIB=~/software/grpc/lib/libaddress_sorting.a \ -DGRPC_CPP_PLUGIN=~/software/grpc/bin/grpc_cpp_plugin \ -DGRPC_GPR_LIB=~/software/grpc/lib/libgpr.a \ -DGRPC_GRPCPP_LIB=~/software/grpc/lib/libgrpc++.a \ -DGRPC_GRPC_LIB=~/software/grpc/lib/libgrpc.a \ -DGRPC_INCLUDE_DIR=~/software/grpc/include \ -DGRPC_UPB_LIB=~/software/grpc/lib/libupb.a \ -DProtobuf_INCLUDE_DIR=~/software/protobuf/include \ -DProtobuf_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf.a \ -DProtobuf_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf.a \ -DProtobuf_LITE_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf-lite.a \ -DProtobuf_LITE_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf-lite.a \ -DProtobuf_PROTOC_EXECUTABLE=~/software/protobuf/bin/protoc \ -DProtobuf_PROTOC_LIBRARY_DEBUG=~/software/protobuf/lib/libprotoc.a \ -DProtobuf_PROTOC_LIBRARY_RELEASE=~/software/protobuf/lib/libprotoc.a \ -Dabsl_DIR=~/src/github/build/arrow-cpp-debug/absl_ep-install \ ../../arrow/cpp {noformat} was (Author: fan_li_ya): [~kou] Thanks a lot for your attention. This is my cmake command line: {noformat} cmake -DCMAKE_BUILD_TYPE=Debug -DARROW_BUILD_TESTS=ON -DARROW_BUILD_BENCHMARKS=ON -DARROW_FLIGHT=ON -DARROW_DATASET=ON -DARROW_PARQUET=ON -DARROW_JNI=ON -DARROW_BUILD_EXAMPLES=ON -DBOOST_SOURCE=SYSTEM -DBoost_INCLUDE_DIR=~/software/boost_1_71_0 -DCMAKE_CXX_COMPILER=~/software/gcc-7.2.0/bin/c++ -DCMAKE_C_COMPILER=~/software/gcc-7.2.0/bin/gcc -DTHRIFT_COMPILER=~/software/thrift-0.12.0/bin/thrift -DTHRIFT_INCLUDE_DIR=~/software/thrift-0.12.0/include -DTHRIFT_STATIC_LIB=~/software/thrift-0.12.0/lib/libthrift-0.12.0.so -DGRPC_ADDRESS_SORTING_LIB=~/software/grpc/lib/libaddress_sorting.a -DGRPC_CPP_PLUGIN=~/software/grpc/bin/grpc_cpp_plugin -DGRPC_GPR_LIB=~/software/grpc/lib/libgpr.a -DGRPC_GRPCPP_LIB=~/software/grpc/lib/libgrpc++.a -DGRPC_GRPC_LIB=~/software/grpc/lib/libgrpc.a -DGRPC_INCLUDE_DIR=~/software/grpc/include -DGRPC_UPB_LIB=~/software/grpc/lib/libupb.a -DProtobuf_INCLUDE_DIR=~/software/protobuf/include -DProtobuf_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf.a -DProtobuf_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf.a -DProtobuf_LITE_LIBRARY_DEBUG=~/software/protobuf/lib/libprotobuf-lite.a -DProtobuf_LITE_LIBRARY_RELEASE=~/software/protobuf/lib/libprotobuf-lite.a -DProtobuf_PROTOC_EXECUTABLE=~/software/protobuf/bin/protoc -DProtobuf_PROTOC_LIBRARY_DEBUG=~/software/protobuf/lib/libprotoc.a -DProtobuf_PROTOC_LIBRARY_RELEASE=~/software/protobuf/lib/libprotoc.a -Dabsl_DIR=~/src/github/build/arrow-cpp-debug/absl_ep-install ../../arrow/cpp {noformat} > [C++] Undefined reference when building flight code > --- > > Key: ARROW-9169 > URL: https://issues.apache.org/jira/browse/ARROW-9169 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Liya Fan >Priority: Major > > When linking flight-test-server with > CMakeFiles/flight-test-server.dir/link.txt, I get the following error: > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::StrCat[abi:cxx11](absl::AlphaNum const&, absl::AlphaNum const&, > absl::AlphaNum const&)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::strings_internal::CatPieces[abi:cxx11](std::initializer_list)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::numbers_internal::FastIntToBuffer(long, char*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::numbers_internal::FastIntToBuffer(unsigned long, char*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::str_format_internal::FormatPack[abi:cxx11](absl::str_format_internal::UntypedFormatSpecImpl, > absl::Span)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::numbers_internal::SixDigitsToBuffer(double, char*)' > ../../../d
[jira] [Commented] (ARROW-9169) [C++] Undefined reference when building flight code
[ https://issues.apache.org/jira/browse/ARROW-9169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140831#comment-17140831 ] Kouhei Sutou commented on ARROW-9169: - Thanks. It seems that Abseil isn't found by {{find_package(absl CONFIG)}} in {{cpp/cmake_modules/FindRPCAlt.cmake}}. Does {{~/src/github/build/arrow-cpp-debug/absl_ep-install/lib/cmake/absl/abslConfig.cmake}} exist? If it exists, could you try {{absl_DIR=~/src/... cmake ...}} command line? {{_DIR}} is an environment variable not a CMake variable. > [C++] Undefined reference when building flight code > --- > > Key: ARROW-9169 > URL: https://issues.apache.org/jira/browse/ARROW-9169 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Reporter: Liya Fan >Priority: Major > > When linking flight-test-server with > CMakeFiles/flight-test-server.dir/link.txt, I get the following error: > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::StrCat[abi:cxx11](absl::AlphaNum const&, absl::AlphaNum const&, > absl::AlphaNum const&)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::strings_internal::CatPieces[abi:cxx11](std::initializer_list)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::numbers_internal::FastIntToBuffer(long, char*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::numbers_internal::FastIntToBuffer(unsigned long, char*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::str_format_internal::FormatPack[abi:cxx11](absl::str_format_internal::UntypedFormatSpecImpl, > absl::Span)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::numbers_internal::SixDigitsToBuffer(double, char*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::base_internal::ThrowStdOutOfRange(char const*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::string_view::find(char, unsigned long) const' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::StrCat[abi:cxx11](absl::AlphaNum const&, absl::AlphaNum const&, > absl::AlphaNum const&, absl::AlphaNum const&)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::FormatTime[abi:cxx11](absl::Time)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to `bool > absl::str_format_internal::FormatArgImpl::Dispatch const*>(absl::str_format_internal::FormatArgImpl::Data, > absl::str_format_internal::FormatConversionSpecImpl, void*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to `bool > absl::str_format_internal::FormatArgImpl::Dispatch std::char_traits, std::allocator > > >(absl::str_format_internal::FormatArgImpl::Data, > absl::str_format_internal::FormatConversionSpecImpl, void*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::numbers_internal::FastIntToBuffer(unsigned int, char*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::EqualsIgnoreCase(absl::string_view, absl::string_view)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::StrCat[abi:cxx11](absl::AlphaNum const&, absl::AlphaNum const&)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::numbers_internal::FastIntToBuffer(int, char*)' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::optional_internal::throw_bad_optional_access()' > ../../../debug/libarrow_flight.so.100.0.0: undefined reference to > `absl::ByChar::Find(absl::string_view, unsigned long) const' > it seems absl has been installed successfully by cmake. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ARROW-9183) [C++] Failed to build arrow-cpp with gcc 4.9.2
[ https://issues.apache.org/jira/browse/ARROW-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kouhei Sutou reassigned ARROW-9183: --- Assignee: Shuai Zhang > [C++] Failed to build arrow-cpp with gcc 4.9.2 > -- > > Key: ARROW-9183 > URL: https://issues.apache.org/jira/browse/ARROW-9183 > Project: Apache Arrow > Issue Type: Bug > Components: C++ >Affects Versions: 0.17.1 >Reporter: Shuai Zhang >Assignee: Shuai Zhang >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Attachments: fix.patch, image-2020-06-19-11-29-13-641.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > The [Building Arrow > C++|https://github.com/apache/arrow/blob/apache-arrow-0.17.1/docs/source/developers/cpp/building.rst] > document say "building requires a C++11-enabled compiler. On Linux, gcc 4.8 > and higher should be sufficient." > But actually, it failed to build with gcc 4.9.2 (see attached > image-2020-06-19-11-29-13-641.png). This is because a bug in gcc [Bug > 57250|https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57250] > Please note it in the document to lead using a higher version of gcc. Current > document is still misleading. > > NEED TRIAGE, because the > [atomic_shared_ptr.h|https://github.com/apache/arrow/blob/apache-arrow-0.17.1/cpp/src/arrow/util/atomic_shared_ptr.h] > file is aim to eliminate the bug. But the error occurred when compiling it. > OK, got it. I installed clang but still using the gcc 4.9.2's libstdc++. The > bug appeared then. > I think it more reliable to check libstdc++ version instead of the gcc > version, because this is a libstdc++ bug instead of a gcc bug. I found it > possible to [check the libstdc++ version via MACRO > \_\_GLIBCXX\_\_|https://stackoverflow.com/questions/10354636/how-do-you-find-what-version-of-libstdc-library-is-installed-on-your-linux-mac] > with help of [this timeline table|https://gcc.gnu.org/develop.html#timeline]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9191) [Rust] Do not panic when int96 milliseconds are negative
Max Burke created ARROW-9191: Summary: [Rust] Do not panic when int96 milliseconds are negative Key: ARROW-9191 URL: https://issues.apache.org/jira/browse/ARROW-9191 Project: Apache Arrow Issue Type: Improvement Reporter: Max Burke See GitHub PR: -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9191) [Rust] Do not panic when int96 milliseconds are negative
[ https://issues.apache.org/jira/browse/ARROW-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-9191: -- Labels: pull-request-available (was: ) > [Rust] Do not panic when int96 milliseconds are negative > > > Key: ARROW-9191 > URL: https://issues.apache.org/jira/browse/ARROW-9191 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Max Burke >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > See GitHub PR: [https://github.com/apache/arrow/pull/7500] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9191) [Rust] Do not panic when int96 milliseconds are negative
[ https://issues.apache.org/jira/browse/ARROW-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Burke updated ARROW-9191: - Description: See GitHub PR: [https://github.com/apache/arrow/pull/7500] (was: See GitHub PR: ) > [Rust] Do not panic when int96 milliseconds are negative > > > Key: ARROW-9191 > URL: https://issues.apache.org/jira/browse/ARROW-9191 > Project: Apache Arrow > Issue Type: Improvement >Reporter: Max Burke >Priority: Major > > See GitHub PR: [https://github.com/apache/arrow/pull/7500] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ARROW-9179) [R] Replace usage of iris dataset in tests
[ https://issues.apache.org/jira/browse/ARROW-9179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neal Richardson resolved ARROW-9179. Resolution: Fixed Issue resolved by pull request 7499 [https://github.com/apache/arrow/pull/7499] > [R] Replace usage of iris dataset in tests > -- > > Key: ARROW-9179 > URL: https://issues.apache.org/jira/browse/ARROW-9179 > Project: Apache Arrow > Issue Type: Improvement > Components: R >Reporter: Neal Richardson >Assignee: Neal Richardson >Priority: Major > Labels: pull-request-available > Fix For: 1.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > It's not a particularly interesting dataset to test with anyway. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ARROW-9192) [Rust] Enable clippy linting for arrow crate in CI pipeline
QP Hou created ARROW-9192: - Summary: [Rust] Enable clippy linting for arrow crate in CI pipeline Key: ARROW-9192 URL: https://issues.apache.org/jira/browse/ARROW-9192 Project: Apache Arrow Issue Type: Improvement Components: Rust Reporter: QP Hou Assignee: QP Hou -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ARROW-9192) [Rust] Enable clippy linting for arrow crate in CI pipeline
[ https://issues.apache.org/jira/browse/ARROW-9192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated ARROW-9192: -- Labels: pull-request-available (was: ) > [Rust] Enable clippy linting for arrow crate in CI pipeline > > > Key: ARROW-9192 > URL: https://issues.apache.org/jira/browse/ARROW-9192 > Project: Apache Arrow > Issue Type: Improvement > Components: Rust >Reporter: QP Hou >Assignee: QP Hou >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)