Re: Arrow sync call June 8 at 12:00 US/Eastern, 16:00 UTC

Hi Gavin, There was no detailed discussion in the meeting about this, just some general comments, but I'll share a few areas of collaboration that I'm aware of: - There is work ongoing to enable the Arrow C++ compute engine (aka "Acero") to consume Substrait plans, change them into ExecPlans, and

Re: [VOTE] Mark C Stream Interface as Stable

+1 In "[VOTE] Mark C Stream Interface as Stable" on Wed, 8 Jun 2022 11:15:29 -0700, Will Jones wrote: > Hi, > > Given all feedback to discussion [1] has been positive, I would like to > propose marking the C Stream Interface as stable. > > I have prepared PRs in apache/arrow [2] and

Re: [VOTE] Mark C Stream Interface as Stable

+1 (non binding) -Jon On Wed, Jun 8, 2022 at 4:52 PM Jorge Cardoso Leitão wrote: > > Sorry, I got a bit confused on what we were voting on. Thank you for the > clarification. > > +1 > > Best, > Jorge > > > On Wed, Jun 8, 2022 at 9:53 PM Antoine Pitrou wrote: > > > > > Le 08/06/2022 à 20:55,

Re: Arrow sync call June 8 at 12:00 US/Eastern, 16:00 UTC

Thanks Ian -- can I ask whether there was any discussion of note that happened around Arrow + Substrait stuff? On Wed, Jun 8, 2022 at 5:31 PM Ian Cook wrote: > Attendees: > > Ian Cook > Raúl Cumplido > Alenka Frim > Ian Joiner > Will Jones > Jorge Leitão > David Li > Rok Mihevc > Ashish

Re: Arrow sync call June 8 at 12:00 US/Eastern, 16:00 UTC

Attendees: Ian Cook Raúl Cumplido Alenka Frim Ian Joiner Will Jones Jorge Leitão David Li Rok Mihevc Ashish Paliwal Matthew Topol Jacob Wujciak Discussion: Recent changes to the merge script for apache/arrow PRs - Now uses a personal access token (PAT) to authenticate to the ASF Jira - Now

Re: [VOTE] Mark C Stream Interface as Stable

Sorry, I got a bit confused on what we were voting on. Thank you for the clarification. +1 Best, Jorge On Wed, Jun 8, 2022 at 9:53 PM Antoine Pitrou wrote: > > Le 08/06/2022 à 20:55, Jorge Cardoso Leitão a écrit : > > 0 (binding) - imo there is some unclarity over what is expected to be > >

Re: [VOTE] Mark C Stream Interface as Stable

Le 08/06/2022 à 20:55, Jorge Cardoso Leitão a écrit : 0 (binding) - imo there is some unclarity over what is expected to be passed over the C streaming interface - an Array or a StructArray. I think the spec claims the former, but the C++ implementation (which I assume is the reference here)

Re: [VOTE] Mark C Stream Interface as Stable

Thanks for raising that Jorge! I should have looked for open issues related to the C Streaming Interface. I confirmed just now that this is the only one. I am still +1 (non-binding) for formalizing as-written. From what is described in the linked issue, the case of non-struct arrays seems to

Re: [VOTE] Mark C Stream Interface as Stable

0 (binding) - imo there is some unclarity over what is expected to be passed over the C streaming interface - an Array or a StructArray. I think the spec claims the former, but the C++ implementation (which I assume is the reference here) expects the latter [1]. Would it be possible to clarify

Re: [VOTE] Mark C Stream Interface as Stable

+1 (binding) On Wed, Jun 8, 2022, at 14:42, Dewey Dunnington wrote: > +1 (non-binding) > > On Wed, Jun 8, 2022 at 2:29 PM Antoine Pitrou wrote: > >> >> +1 (binding) >> >> >> Le 08/06/2022 à 20:15, Will Jones a écrit : >> > Hi, >> > >> > Given all feedback to discussion [1] has been positive, I

Re: [VOTE] Mark C Stream Interface as Stable

+1 (non-binding) On Wed, Jun 8, 2022 at 2:29 PM Antoine Pitrou wrote: > > +1 (binding) > > > Le 08/06/2022 à 20:15, Will Jones a écrit : > > Hi, > > > > Given all feedback to discussion [1] has been positive, I would like to > > propose marking the C Stream Interface as stable. > > > > I have

Re: [VOTE] Mark C Stream Interface as Stable

+1 (binding) Le 08/06/2022 à 20:15, Will Jones a écrit : Hi, Given all feedback to discussion [1] has been positive, I would like to propose marking the C Stream Interface as stable. I have prepared PRs in apache/arrow [2] and apache/arrow-rs [3] to remove all "experimental" markers from

Re: [VOTE] Mark C Stream Interface as Stable

+1 (binding) On Wed, Jun 8, 2022 at 2:16 PM Will Jones wrote: > Hi, > > Given all feedback to discussion [1] has been positive, I would like to > propose marking the C Stream Interface as stable. > > I have prepared PRs in apache/arrow [2] and apache/arrow-rs [3] to remove > all "experimental"

[VOTE] Mark C Stream Interface as Stable

Hi, Given all feedback to discussion [1] has been positive, I would like to propose marking the C Stream Interface as stable. I have prepared PRs in apache/arrow [2] and apache/arrow-rs [3] to remove all "experimental" markers from the interface and update the support grid for the interface.

Re: int64_t vs size_t

> > Is it an oversight or a conscious design decision? If latter, what is the > reason behind it? This comes from the style guide (Google) [1] the project adapted [1] https://google.github.io/styleguide/cppguide.html#Integer_Types On Wed, Jun 8, 2022 at 10:39 AM Arkadiy Vertleyb (BLOOMBERG/ 120

Re: int8_t vs size_t

No, Arrow should definitely compile in 32 bits. Feel free to open a JIRA and/or submit a PR for it. Le 08/06/2022 à 19:48, Arkadiy Vertleyb (BLOOMBERG/ 120 PARK) a écrit : Hi Antoine, I need the 32 bit support because our project needs to support 32 bit. These are my constrains. As

Re: int8_t vs size_t

Hi Antoine, I need the 32 bit support because our project needs to support 32 bit. These are my constrains. As of now, arrow doesn't even compile in 32 bit. I can fix it, but has the decision been made to stop supporting it? Thanks, Arkadiy From: dev@arrow.apache.org At: 06/08/22 13:38:43

Re:int64_t vs size_t

Corrected the message title. From: dev@arrow.apache.org At: 06/08/22 13:35:23 UTC-4:00To: dev@arrow.apache.org Subject: int8_t vs size_t Hi all. Throughout the entire project, int64_t rather than size_t is consistently used to denote size and offset. This causes massive amount of compiler

Re: int8_t vs size_t

Hi, It is a conscious decision of following the Google C++ style guide: https://google.github.io/styleguide/cppguide.html#Integer_Types I agree that size_t (or ssize_t) would have been a better choice for in-memory lengths and sizes. Unfortunately, that ship has sailed now. 32-bit systems

int8_t vs size_t

Hi all. Throughout the entire project, int64_t rather than size_t is consistently used to denote size and offset. This causes massive amount of compiler warnings in the 32 bit system. Is it an oversight or a conscious design decision? If latter, what is the reason behind it? Thanks, Arkadiy

Re: [MATLAB] Integrating a framework for connecting MATLAB and C++ objects using MEX

Hi Kou, --- Note: I am replying to your email as a forward from Fiona (Cc'd) since your original email was accidentally blocked by my email client). --- The way that we expected the object dispatch layer to be used by client code is as follows: 1. A developer would author a custom MEX

Arrow sync call June 8 at 12:00 US/Eastern, 16:00 UTC

Hi all, Our biweekly sync call is today at 12:00 noon Eastern time. The Zoom meeting URL for this and other biweekly Arrow sync calls is: https://zoom.us/j/87649033008?pwd=SitsRHluQStlREM0TjJVYkRibVZsUT09 Alternatively, enter this information into the Zoom website or app to join the call:

[Rust] DataFusion 9.0.0 release plan

I am planning on creating the first 9.0.0 RC this Friday. This is the first release in our new monthly release cadence. Here is the tracking issue for reference. https://github.com/apache/arrow-datafusion/issues/2676 Thanks, Andy.

Re: [C++] Adding Run-Length Encoding to Arrow

RLE would probably have some benefits that it makes sense to evaluate, I would personally go in the direction of having a minimal benchmarking suite for some of the cases where we expect to seem most benefit (IE: filtering) so we can discuss with real numbers. Also, the currently proposed format