[jira] [Created] (ARROW-8239) [Java] fix param checks in splitAndTransfer method

2020-03-26 Thread Prudhvi Porandla (Jira)
Prudhvi Porandla created ARROW-8239: --- Summary: [Java] fix param checks in splitAndTransfer method Key: ARROW-8239 URL: https://issues.apache.org/jira/browse/ARROW-8239 Project: Apache Arrow

[jira] [Created] (ARROW-8238) [C++][Compute] Failed to build compute tests on windows with msvc2015

2020-03-26 Thread Yibo Cai (Jira)
Yibo Cai created ARROW-8238: --- Summary: [C++][Compute] Failed to build compute tests on windows with msvc2015 Key: ARROW-8238 URL: https://issues.apache.org/jira/browse/ARROW-8238 Project: Apache Arrow

Re: Proposal to use Black for automatic formatting of Python code

2020-03-26 Thread Micah Kornfield
+1 from me as well. On Thursday, March 26, 2020, Neal Richardson wrote: > I'm also in favor, very much so. Life is too short to hold strong opinions > about code style; you get used to whatever you're accustomed to seeing. And > I support using automation to remove manual nuisances like this. >

Re: Proposal to use Black for automatic formatting of Python code

2020-03-26 Thread Neal Richardson
I'm also in favor, very much so. Life is too short to hold strong opinions about code style; you get used to whatever you're accustomed to seeing. And I support using automation to remove manual nuisances like this. Neal On Thu, Mar 26, 2020 at 3:49 PM Wes McKinney wrote: > I'm in favor of this

[NIGHTLY] Arrow Build Report for Job nightly-2020-03-26-1

2020-03-26 Thread Crossbow
Arrow Build Report for Job nightly-2020-03-26-1 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-03-26-1 Failed Tasks: - gandiva-jar-trusty: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-03-26-1-travis-gandiva-jar-trusty - test-co

Re: Proposal to use Black for automatic formatting of Python code

2020-03-26 Thread Wes McKinney
I'm in favor of this even though I also probably won't like some of the formatting decisions it makes. Is there a sense of how far away Black is from having Cython support? I saw it was being worked on a while back. On Thu, Mar 26, 2020 at 2:37 PM Joris Van den Bossche wrote: > > Hi all, > > I wo

[jira] [Created] (ARROW-8237) [Python] Developer build instructions for non-conda users appear to have grown out of date

2020-03-26 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8237: --- Summary: [Python] Developer build instructions for non-conda users appear to have grown out of date Key: ARROW-8237 URL: https://issues.apache.org/jira/browse/ARROW-8237

Re: [DISCUSS] Adding "trivial" buffer compression option to IPC protocol (ARROW-300)

2020-03-26 Thread Wes McKinney
Here are the results: File size: https://ibb.co/71sBsg3 Read time: https://ibb.co/4ZncdF8 Write time: https://ibb.co/xhNkRS2 Code: https://github.com/wesm/notebooks/blob/master/20190919file_benchmarks/FeatherCompression.ipynb (based on https://github.com/apache/arrow/pull/6694) High level summa

[jira] [Created] (ARROW-8236) [Rust] Linting GitHub Actions task failing

2020-03-26 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8236: --- Summary: [Rust] Linting GitHub Actions task failing Key: ARROW-8236 URL: https://issues.apache.org/jira/browse/ARROW-8236 Project: Apache Arrow Issue Type: Bug

Re: Flight API for c_glib

2020-03-26 Thread Sutou Kouhei
Hi, In "Flight API for c_glib" on Thu, 26 Mar 2020 16:01:25 +, Tanveer Ahmad - EWI wrote: > I am wondering whether some work is being done on Flight API for c_glib? We'll do in a few months. Thanks, -- kou

Proposal to use Black for automatic formatting of Python code

2020-03-26 Thread Joris Van den Bossche
Hi all, I would like to propose adopting Black as code formatter within the python project. There is an older JIRA issue about this ( https://issues.apache.org/jira/browse/ARROW-5176), but bringing it to the mailing list for wider attention. Black (https://github.com/ambv/black) is a tool for aut

[jira] [Created] (ARROW-8235) [C++][Compute] Filter out nulls by default

2020-03-26 Thread Ben Kietzman (Jira)
Ben Kietzman created ARROW-8235: --- Summary: [C++][Compute] Filter out nulls by default Key: ARROW-8235 URL: https://issues.apache.org/jira/browse/ARROW-8235 Project: Apache Arrow Issue Type: Imp

[jira] [Created] (ARROW-8234) [CI] Build timeouts on "AMD64 Windows RTools 35"

2020-03-26 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-8234: - Summary: [CI] Build timeouts on "AMD64 Windows RTools 35" Key: ARROW-8234 URL: https://issues.apache.org/jira/browse/ARROW-8234 Project: Apache Arrow Issue

[jira] [Created] (ARROW-8233) [CI] Build timeouts on "AMD64 Windows MinGW 64 GLib & Ruby "

2020-03-26 Thread Antoine Pitrou (Jira)
Antoine Pitrou created ARROW-8233: - Summary: [CI] Build timeouts on "AMD64 Windows MinGW 64 GLib & Ruby " Key: ARROW-8233 URL: https://issues.apache.org/jira/browse/ARROW-8233 Project: Apache Arrow

[jira] [Created] (ARROW-8232) [Python] Deprecate pa.open_file and pa.open_stream in favor of pa.ipc.open_file/open_stream

2020-03-26 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-8232: --- Summary: [Python] Deprecate pa.open_file and pa.open_stream in favor of pa.ipc.open_file/open_stream Key: ARROW-8232 URL: https://issues.apache.org/jira/browse/ARROW-8232

Re: [DISCUSS] Adding "trivial" buffer compression option to IPC protocol (ARROW-300)

2020-03-26 Thread Wes McKinney
I'll run a grid of batch sizes (from 1024 to 64K or 128K) and let you know the read/write times and compression ratios. Shouldn't take too long On Wed, Mar 25, 2020 at 10:37 PM Fan Liya wrote: > > Thanks a lot for sharing the good results. > > As investigated by Wes, we have existing zstd library

[jira] [Created] (ARROW-8231) Parse key_value_metadata from parquet FileMetaData into arrow schema metadata

2020-03-26 Thread Jira
Jörn Horstmann created ARROW-8231: - Summary: Parse key_value_metadata from parquet FileMetaData into arrow schema metadata Key: ARROW-8231 URL: https://issues.apache.org/jira/browse/ARROW-8231 Project

Flight API for c_glib

2020-03-26 Thread Tanveer Ahmad - EWI
Hi, I am wondering whether some work is being done on Flight API for c_glib? Regards, Tanveer Ahmad

Re: [NIGHTLY] Arrow Build Report for Job nightly-2020-03-26-0

2020-03-26 Thread Krisztián Szűcs
Hi, The wheel and conda tasks are failing because I've updated the credentials required to upload the produced artifacts. So these are temporary failures which should not affect the upcoming nightly builds. - Krisztian On Thu, Mar 26, 2020 at 2:01 PM Crossbow wrote: > > > Arrow Build Report for

[NIGHTLY] Arrow Build Report for Job nightly-2020-03-26-0

2020-03-26 Thread Crossbow
Arrow Build Report for Job nightly-2020-03-26-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-03-26-0 Failed Tasks: - gandiva-jar-trusty: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-03-26-0-travis-gandiva-jar-trusty - test-co

[jira] [Created] (ARROW-8230) [Java] Move Netty memory manager into a separate module

2020-03-26 Thread Liya Fan (Jira)
Liya Fan created ARROW-8230: --- Summary: [Java] Move Netty memory manager into a separate module Key: ARROW-8230 URL: https://issues.apache.org/jira/browse/ARROW-8230 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-8229) [Java] Move ArrowBuf into the Arrow package

2020-03-26 Thread Liya Fan (Jira)
Liya Fan created ARROW-8229: --- Summary: [Java] Move ArrowBuf into the Arrow package Key: ARROW-8229 URL: https://issues.apache.org/jira/browse/ARROW-8229 Project: Apache Arrow Issue Type: Improvemen

Re: Summary of RLE and other compression efforts?

2020-03-26 Thread Micah Kornfield
Hi Evan, > Hope everyone is staying safe! Thanks you too. A fairly substantial amount of CPU is needed for translating from Parquet; > main memory bandwidth becomes a factor. Thus, it seems speed and > constraining factors varies widely by application I agree performance is going to be applic