Thanks Wes,
I will follow what is happening between Arrow and Kudu.
In the short term, if you would have to define a storage for Arrow which
has good (enough) performance, not too costly to operate... what would
you choose ? I saw there is an example to store Parquet files on Azure
Blob Storage, would it be ok to start ? Or there is a better choice ?
---
sh...@metaverse.fr
GPG public key ID : B517C4C8
Le 21/01/2020 17:54, Wes McKinney a écrit :
I'm interested to see an Arrow adapter for Apache Kudu developed. My
gut feeling is that this work should be undertaken in Kudu itself,
potentially having the tablet servers producing Arrow Record Batches
locally and sending them to the client rather than converting to
Kudu's own on-the-wire record format and then deserializing into Arrow
on the receiver side. It might be worth a conversation with the Kudu
community to see what they think.
Of course one can build an Arrow deserializer for the current Kudu C++
client API and probably get pretty good performance. see also
ARROW-814
https://issues.apache.org/jira/browse/ARROW-814
On Tue, Jan 21, 2020 at 12:32 PM Shazz <sh...@metaverse.fr> wrote:
Hi,
I'm thinking of an architecture to store and access efficiently
tabular
data and I was told to look at Arrow and Kudu.
I saw on the frontpage a diagram where Arrow can be integrated with
Kudu
but nothing in the documentation. Is there an example available
somewhere ?
Thanks !
--
sh...@metaverse.fr
GPG public key ID : B517C4C8
