Re: Arrow Flight + Go, Arrow for Realtime

2020-08-12 Thread fredgan
Hi Sebastien, I am really interested in this feature. I would like to join in the development. But I have to take time to familiarize myself with it. I will try it! Fred -- Original -- From: Sebastien Binet

Re: [EXTERNAL] Re: Value of Date64 type over Date32

2020-08-12 Thread Wes McKinney
I think we should validate optionally in ValidateFull in C++. I think to validate unconditionally would be too computationally expensive https://issues.apache.org/jira/browse/ARROW-9705 On Tue, Aug 11, 2020 at 1:34 PM Eric Erhardt wrote: > > Thanks for the info, Wes. > > Looking through the

Re: Arrow Flight + Go, Arrow for Realtime

2020-08-12 Thread Sebastien Binet
Mark, AFAIK, nobody's actively working on Arrow-Flight for Go (I think somebody started that work at some point but I don't remember anything hitting the main repo) as for Go+WASM:

Re: Arrow Flight + Go, Arrow for Realtime

2020-08-12 Thread Wes McKinney
There's a WIP patch for Flight support in Go https://github.com/apache/arrow/pull/6731 I hope to see someone taking up this work as first-class Flight support in Go would be very useful for building data services. On Wed, Aug 12, 2020 at 5:08 AM Adam Lippai wrote: > > Arrow is mainly about

[NIGHTLY] Arrow Build Report for Job nightly-2020-08-12-0

2020-08-12 Thread Crossbow
Arrow Build Report for Job nightly-2020-08-12-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-08-12-0 Failed Tasks: - test-conda-python-3.7-hdfs-2.9.2: URL:

Re: Arrow Flight + Go, Arrow for Realtime

2020-08-12 Thread Adam Lippai
Arrow is mainly about batching data and leveraging all the opportunities this gives. This means you either have to buffer the data yourself and flush it when a reasonable sized batch is complete or play with preallocating Arrow structures This was discussed recently, you might be interested in the

Arrow Flight + Go, Arrow for Realtime

2020-08-12 Thread mark
I'm looking at using Arrow for a realtime IoT project which includes use cases both on server, and also for transferring /using in a Browser via WASM, and have a few questions. Language in use is Go. Is anyone working on implementing Arrow-Flight in Go ? (According to the