Any opinions about this? It seems the next steps would be a concrete
API proposal and perhaps a reference implementation thereof.
On Sun, Jun 28, 2020 at 11:26 PM Wes McKinney <wesmck...@gmail.com> wrote:
>
> In ARROW-8301 [1] and elsewhere we've been discussing how to
> communicate what amounts to a sequence of arrays or a sequence of
> RecordBatch objects using the C data interface.
>
> Example use cases:
>
> * Returning a sequence of record / row batches from a database driver
> * Sending a C++ arrow::ChunkedArray or arrow::Table to a consumer
> using only the C interface
>
> Applications could define their own custom iterator interfaces to
> communicate what amounts to a sequence of the ArrowArray C interface
> objects, but it is likely a common enough use case to have an
> off-the-shelf solution so that we can support this solution in our
> reference libraries (e.g. Arrow C++, pyarrow, Arrow R)
>
> I suggested a C structure as follows
>
> struct ArrowArrayStream {
> void (*get_schema)(struct ArrowSchema*);
> // Non-zero return value indicates an error?
> int (*get_next)(struct ArrowArray*);
> void (*get_error)(... ERROR HANDLING TODO );
> void (*release)(struct ArrowArrayStream*);
> void* private_data;
> };
>
> The producer would populate this object with pointers to its
> implementations of these functions.
>
> Thoughts about this?
>
> Thanks,
> Wes
>
> [1]: https://issues.apache.org/jira/browse/ARROW-8301
