Value of Date64 type over Date32

2020-08-10 Thread Eric Erhardt
I don't understand what the value of the Date64 type is over using Date32: >From https://github.com/apache/arrow/blob/master/format/Schema.fbs#L193-L206 enum DateUnit: short { DAY, MILLISECOND } /// Date is either a 32-bit or 64-bit type representing elapsed time since UNIX /// epoch

Re: [DISCUSS] Adding a pull-style iterator API to the C data interface

2020-08-10 Thread Wes McKinney
I'm still in need of it. I'd be interested in developing a solution that can be used in some database APIs, e.g. using it for the result interface for an embedded SQL database like SQLite or DuckDB would be an interesting motivating use case. One approach would be to create something unofficial

Re: [DISCUSS] Adding a pull-style iterator API to the C data interface

2020-08-10 Thread Antoine Pitrou
From the absence of response, it would seem there isn't much interest in this. Please speak up if you think this would be useful to you. Regards Antoine. On Tue, 7 Jul 2020 07:49:17 -0500 Wes McKinney wrote: > Any opinions about this? It seems the next steps would be a concrete > API

Re: [DISCUSS][C++] Group by operation for RecordBatch and Table

2020-08-10 Thread Kenta Murata
Hi Wes, Thank you very much giving us the detail explanation of your thoughts. I need the knowledge of the SOTA of query engine you pointed out if I’ll contribute to C++ Query Engine or just write the binding of it. I’m studying the article and the codes. Regards, Kenta Murata On Thu, Aug 6,

[NIGHTLY] Arrow Build Report for Job nightly-2020-08-10-0

2020-08-10 Thread Crossbow
Arrow Build Report for Job nightly-2020-08-10-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-08-10-0 Failed Tasks: - test-conda-python-3.7-hdfs-2.9.2: URL:

Re: [DISSCUSS][JAVA] Avoid set reader/writer indices in FieldVector#getFieldBuffers

2020-08-10 Thread Ji Liu
Hi Micah, I am afraid it's not a reasonable solution. 1. The status is that getFieldBuffers has right order buffer and was used in IPC, getBuffers was not used in IPC. 2. The purpose of this PR is to use getBuffers in IPC instead, and making changes in getFieldBuffers dose not seem to help this