Re: Parquet+Arrow Java

2017-07-21 Thread Jacques Nadeau
We do have relational operators as well in our code. We're trying to figure out what to contribute back and how to factor. For now, the code is under Apache license you are free to use. Our relational operations are under here: https://github.com/dremio/dremio-oss/tree/master/sabot/kernel/src/main

[jira] [Created] (ARROW-1249) Expose the fillEmpties function from NullableVector.mutator

2017-07-21 Thread SIDDHARTH TEOTIA (JIRA)
SIDDHARTH TEOTIA created ARROW-1249: --- Summary: Expose the fillEmpties function from NullableVector.mutator Key: ARROW-1249 URL: https://issues.apache.org/jira/browse/ARROW-1249 Project: Apache Arrow

Re: 64-Byte Buffer Alignment

2017-07-21 Thread Wes McKinney
hi Brian -- have you had any luck reproducing this issue? Thanks Wes On Sun, Jul 16, 2017 at 7:06 PM, Wes McKinney wrote: > I just created https://issues.apache.org/jira/browse/ARROW-1224 > > On Sun, Jul 16, 2017 at 7:03 PM, Wes McKinney wrote: >> hi Brian, >> >> In the record batch IPC formats

Re: [VOTE] Release Apache Arrow 0.5.0 - RC2

2017-07-21 Thread Wes McKinney
+1 (binding) * Verified signature and checksum * Ran Java unit tests, packaged to JARs * Build and run C++ unit tests on Ubuntu 14.04 / gcc 4.8 * Built Parquet C++ against Arrow 0.5.0 and ran its unit tests * Ran Python unit tests including --with-parquet * Ran GLib C Ruby unit tests * Ran integra

Re: Use case for R Arrow Bindings

2017-07-21 Thread Bryan Cutler
Thanks Clark. I know that SparkR would benefit a lot from Arrow bindings and many people would like to see that, but to my knowledge no one has started working on this yet. Please keep us updated with what you find! Bryan On Fri, Jul 21, 2017 at 9:15 AM, Clark Fitzgerald wrote: > Regarding th

[jira] [Created] (ARROW-1248) [Python] C linkage warnings in Clang with public Cython API

2017-07-21 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1248: --- Summary: [Python] C linkage warnings in Clang with public Cython API Key: ARROW-1248 URL: https://issues.apache.org/jira/browse/ARROW-1248 Project: Apache Arrow

Re: Use case for R Arrow Bindings

2017-07-21 Thread Clark Fitzgerald
Regarding the R Consortium, the Distributed Computing Working Group led by Michael Lawrence would be interested in this. It would be nice to go to them with some working examples and use cases. Next week I will start looking into R / Arrow bindings. A couple other people at the UC Davis Data Scien

[jira] [Created] (ARROW-1247) pyarrow causes python to crash errors on parquet.dll

2017-07-21 Thread Aditi Breed (JIRA)
Aditi Breed created ARROW-1247: -- Summary: pyarrow causes python to crash errors on parquet.dll Key: ARROW-1247 URL: https://issues.apache.org/jira/browse/ARROW-1247 Project: Apache Arrow Issue T

Re: Parquet+Arrow Java

2017-07-21 Thread Michael Shtelma
Hi Wes, It is really great, that you have open-sourced all this! As far as I understand, you have also open-sourced the engine that can execute relational operators on arrow ? Is it possible to use it as library ? Are you also planning to donate it arrow project at some point? Sincerely, Michael