[Help Needed] Arrow IPC Reader in Rust

Hi Arrow developers, I'm "done" with the Arrow IPC Reader in Rust (for supported data types), but am having issues with reading some of the test data. Specifically, I've noticed that when reading the integration test data (primitve_generated), where I expect an array with 17 values, the arrow

[jira] [Created] (ARROW-7187) [C++][Doc] doxygen broken on master because of @

Neal Richardson created ARROW-7187: -- Summary: [C++][Doc] doxygen broken on master because of @ Key: ARROW-7187 URL: https://issues.apache.org/jira/browse/ARROW-7187 Project: Apache Arrow

[jira] [Created] (ARROW-7186) [R] Add inline comments to document the dplyr code

Neal Richardson created ARROW-7186: -- Summary: [R] Add inline comments to document the dplyr code Key: ARROW-7186 URL: https://issues.apache.org/jira/browse/ARROW-7186 Project: Apache Arrow

[jira] [Created] (ARROW-7185) [R][Dataset] Add bindings for IN, IS_VALID expressions

Neal Richardson created ARROW-7185: -- Summary: [R][Dataset] Add bindings for IN, IS_VALID expressions Key: ARROW-7185 URL: https://issues.apache.org/jira/browse/ARROW-7185 Project: Apache Arrow

Creating arrays from existing arrays in Cython

Hi, I'm trying to create arrays from an existing array but I'm not sure how exactly to do it. I tried using the ArrayBuilder class, but I keep getting compiler errors when trying to instantiate one... So I have a couple questions then: 1. How would I instantiate and use an ArrayBuilder class?

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-11-15-0

Others: * I created https://issues.apache.org/jira/browse/ARROW-7105 over the weekend for the homebrew-cpp failure. * Here's the spark failure, not sure how to interpret it. It is the first time it has failed so maybe it is transient. https://circleci.com/gh/ursa-labs/crossbow/4951 * Dask failure

[jira] [Created] (ARROW-7184) [C++][Dataset] Nightly ubuntu 14.04 fails because of dataset filter tests

Neal Richardson created ARROW-7184: -- Summary: [C++][Dataset] Nightly ubuntu 14.04 fails because of dataset filter tests Key: ARROW-7184 URL: https://issues.apache.org/jira/browse/ARROW-7184 Project:

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-11-15-0

Thanks Neal! I'll try to allocate time for the wheels (including python 3.8) next week. On Fri, Nov 15, 2019 at 7:05 PM Neal Richardson wrote: > > I ticketed a few of the nightly failures: > > * https://issues.apache.org/jira/browse/ARROW-7181 (wheels) > *

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-11-15-0

I ticketed a few of the nightly failures: * https://issues.apache.org/jira/browse/ARROW-7181 (wheels) * https://issues.apache.org/jira/browse/ARROW-7182 (fuzzit, looks like an easy fix) * https://issues.apache.org/jira/browse/ARROW-7183 (re-skip r-sanitizer, I'll handle) I didn't have time this

[jira] [Created] (ARROW-7183) [CI][Crossbow] Re-skip r-sanitizer nightly tests

Neal Richardson created ARROW-7183: -- Summary: [CI][Crossbow] Re-skip r-sanitizer nightly tests Key: ARROW-7183 URL: https://issues.apache.org/jira/browse/ARROW-7183 Project: Apache Arrow

[jira] [Created] (ARROW-7182) [CI][Crossbow] NIghtly fuzzit build broken in docker-compose refactor

Neal Richardson created ARROW-7182: -- Summary: [CI][Crossbow] NIghtly fuzzit build broken in docker-compose refactor Key: ARROW-7182 URL: https://issues.apache.org/jira/browse/ARROW-7182 Project:

[jira] [Created] (ARROW-7181) [Python][Nightly] Wheel builds could NOT find ArrowPython

Neal Richardson created ARROW-7181: -- Summary: [Python][Nightly] Wheel builds could NOT find ArrowPython Key: ARROW-7181 URL: https://issues.apache.org/jira/browse/ARROW-7181 Project: Apache Arrow

[jira] [Created] (ARROW-7180) [CI] Java builds are not triggered on the master branch

Krisztian Szucs created ARROW-7180: -- Summary: [CI] Java builds are not triggered on the master branch Key: ARROW-7180 URL: https://issues.apache.org/jira/browse/ARROW-7180 Project: Apache Arrow

[jira] [Created] (ARROW-7179) [C++][Compute] Coalesce kernel

Ben Kietzman created ARROW-7179: --- Summary: [C++][Compute] Coalesce kernel Key: ARROW-7179 URL: https://issues.apache.org/jira/browse/ARROW-7179 Project: Apache Arrow Issue Type: Improvement

[NIGHTLY] Arrow Build Report for Job nightly-2019-11-15-0

Arrow Build Report for Job nightly-2019-11-15-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-11-15-0 Failed Tasks: - homebrew-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-11-15-0-travis-homebrew-cpp -

Re: [C++][Parquet]: Stream API handling of optional fields

I'm all for it. Created [1] it would also enable an operator[] for arrays of primitive types [2]. [1] https://issues.apache.org/jira/browse/ARROW-7178 [2] https://issues.apache.org/jira/browse/ARROW-6276 On Fri, Nov 15, 2019 at 12:40 AM Micah Kornfield wrote: > > I think there are potentially

[jira] [Created] (ARROW-7178) [C++] Vendor forward compatible std::optional

Francois Saint-Jacques created ARROW-7178: - Summary: [C++] Vendor forward compatible std::optional Key: ARROW-7178 URL: https://issues.apache.org/jira/browse/ARROW-7178 Project: Apache Arrow

[jira] [Created] (ARROW-7177) [Java] Provide a utility to improve the performance of vector loading/unloading

Liya Fan created ARROW-7177: --- Summary: [Java] Provide a utility to improve the performance of vector loading/unloading Key: ARROW-7177 URL: https://issues.apache.org/jira/browse/ARROW-7177 Project: Apache

Re: Parquet cpp status

The parquet c++ implementation has all the facilities to expose the required information to implement predicate pushdown. The experimental Dataset API does make use of this with parquet. See [1] for an example of the API. Or a real-life usage with the nyc-tlc taxi dataset [2]. The relevant

Re: [Discuss][Java] 64-bit lengths for ValueVectors

I think the 2GB limit is overly restrictive for modern computers. This is a problem we must face anyway. Best, Liya Fan On Fri, Nov 15, 2019 at 5:07 PM Micah Kornfield wrote: > Apologies for the long delay, I chose to do the minimal work of limiting > this change [1] to allowing ArrowBuf to

Re: [Discuss][Java] 64-bit lengths for ValueVectors

Apologies for the long delay, I chose to do the minimal work of limiting this change [1] to allowing ArrowBuf to 64-bit lengths. This would unblock work on LargeString and LargeBinary. If this change looks OK, I think there is some follow-up work to add more thorough unit/integration tests. As

[jira] [Created] (ARROW-7176) [Resolve] Fix arrow::ipc compiler warning

Jiajia Li created ARROW-7176: Summary: [Resolve] Fix arrow::ipc compiler warning Key: ARROW-7176 URL: https://issues.apache.org/jira/browse/ARROW-7176 Project: Apache Arrow Issue Type: