Re: [VOTE][RUST] Release Apache Arrow Rust 7.0.0 RC1

2022-01-11 Thread Jorge Cardoso Leitão
+1 On Tue, Jan 11, 2022 at 9:17 PM QP Hou wrote: > +1 (non-binding) > > On Mon, Jan 10, 2022 at 3:14 PM Andy Grove wrote: > > > > +1 (binding) > > > > Thanks, > > > > Andy. > > > > On Sat, Jan 8, 2022 at 3:43 AM Andrew Lamb wrote: > > > > > Hi, > > > > > > I would like to propose a release of

Re: Arrow streaming computation engine

2022-01-11 Thread QP Hou
For datafusion (the Rust engine that Weston mentioned), the community is about to start building a PoC for streaming engine. The discussion is happening at https://github.com/apache/arrow-datafusion/issues/1544. On Tue, Jan 11, 2022 at 3:29 PM Weston Pace wrote: > > First, note that there are

Re: Help drafting Arrow 2022-01 board report

2022-01-11 Thread Sutou Kouhei
Hi Wes, Could you submit the draft to ASF? Thanks to everyone who help drafting this report. Thanks, -- kou In <20220106.094048.190616206847894889@clear-code.com> "Help drafting Arrow 2022-01 board report" on Thu, 06 Jan 2022 09:40:48 +0900 (JST), Sutou Kouhei wrote: > Hi, > > I

Re: [DISCUSS] Adding new columnar memory layouts to Arrow (in-memory, IPC, C ABI)

2022-01-11 Thread Wes McKinney
hi all, Thank you for all the comments on this mailing list thread and in the Google document. There is definitely a lot of work to take some next steps from here, so I think it would make sense to fork off each of the proposed additions into dedicated discussions. The most contentious issue, it

Re: Arrow streaming computation engine

2022-01-11 Thread Weston Pace
First, note that there are different computation engines in different languages. The Rust implementation has datafusion[1] for example. For the rest of this email, I will speak in more detail specifically about the C++ computation engine (which I am more familiar with) that is in place today.

Arrow streaming computation engine

2022-01-11 Thread Li Jin
Hi, This is a somewhat lengthy email about thoughts around a streaming computation engine for Arrow dataset that I would like to hear feedback from Arrow devs. The main use cases that we are thinking for the streaming engine are time series data, i.e., data arrives in time order (e.g. daily US

Re: [Rust] Async record batch reader?

2022-01-11 Thread Andrew Lamb
A (very belated) follow up to this thread -- there is now a POC PR from Raphael Taylor-Davies with an async Arrow reader for parquet files [1] if anyone is interested. Andrew [1] https://github.com/apache/arrow-rs/pull/1154# On Wed, Aug 26, 2020 at 12:52 PM Andy Grove wrote: > Hi Max, > > I