Re: [C++] struct/class conventions

2021-03-16 Thread Wes McKinney
I think the struct/class decisions have mostly been restricted to public header files. Inside .cc files I think we generally use what seems most convenient (which can often be a struct for pimpl types). In the public headers we should be trying to use struct only for objects that are principally si

Re: [DISCUSS][C++] Reduce usage of KernelContext in compute::

2021-03-16 Thread Benjamin Kietzman
I've opened https://issues.apache.org/jira/browse/ARROW-11945 to track this improvement. On Tue, Mar 16, 2021 at 3:40 PM Wes McKinney wrote: > It seems fine to me to return Result from KernelInit (as long as a > context of some kind of still passed in to the function to have the > possibility of

Re: [DISCUSS][C++] Reduce usage of KernelContext in compute::

2021-03-16 Thread Wes McKinney
It seems fine to me to return Result from KernelInit (as long as a context of some kind of still passed in to the function to have the possibility of passing other configuration bits), and it would make development more convenient to use a common error handling strategy in more places On Fri, Mar

Re: [DISCUSS] [Rust] Donate Ballista to Apache Arrow

2021-03-16 Thread Jorge Cardoso Leitão
Hi, I agree. I skimmed through the code and it is a solid addition to me, so, again, thank you so much for donating it to Arrow. I look very much forward to learning more on how to build a scheduler that can adapt to both in-node and across-node executions :-) Best, Jorge On Tue, Mar 16, 2021

Re: [ALL] Integration tests for dense and sparse tensor

2021-03-16 Thread Antoine Pitrou
Hi Fernando, Le 16/03/2021 à 17:01, Fernando Herrera a écrit : Thanks for the update. It would be interesting to add a centralized plan for tensors in Arrow. It would allow sharing data between packages like numpy, ndarray, pytorch, tensorflow really easy. Those Python packages already exch

Re: [ALL] Integration tests for dense and sparse tensor

2021-03-16 Thread Fernando Herrera
Hi Wes, Thanks for the update. It would be interesting to add a centralized plan for tensors in Arrow. It would allow sharing data between packages like numpy, ndarray, pytorch, tensorflow really easy. Don't you think so? Let me have a look at how the integration tests are created in Archery so I

Re: [Release] Request for patch release of arrow 3.x

2021-03-16 Thread Prem Sagar Gali
Thanks Neal, like you mentioned maybe explicitly mentioning in the release page of arrow that major releases are done quarterly would be helpful. cuDF can then standby for next release of arrow in April. Regards, Prem From: Neal Richardson Date: Monday, March 15, 2021 at 4:01 PM To: dev Subje

Re: [ALL] Integration tests for dense and sparse tensor

2021-03-16 Thread Wes McKinney
hi Fernando — for clarity, there is no centralized planning in this project. If a volunteer wants to do something and there are no objections from other people, then they are free to go ahead and do it. If there aren't any Jira issues about adding integration tests, it would make sense to go ahead

Re: [DISCUSS] [Rust] Donate Ballista to Apache Arrow

2021-03-16 Thread Andy Grove
Thank you for all the responses so far. Based on this thread and the conversations happening in the Ballista project, I would say that the feedback is mostly positive and supportive of this donation, so I have started work on a PR [1] and will start a VOTE email thread once the PR is ready for revi

[NIGHTLY] Arrow Build Report for Job nightly-2021-03-16-0

2021-03-16 Thread Crossbow
Arrow Build Report for Job nightly-2021-03-16-0 All tasks: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-03-16-0 Failed Tasks: - conda-linux-gcc-py37-aarch64: URL: https://github.com/ursacomputing/crossbow/branches/all?query=nightly-2021-03-16-0-drone-conda-linux

Re: [ALL] Integration tests for dense and sparse tensor

2021-03-16 Thread Antoine Pitrou
Hi Fernando, Currently there are no explicit plans to do it, but that would be certainly useful if other implementation start implementing tensor IPC support. One should start by defining a reference format (probably JSON) such as exists for other IPC types: https://arrow.apache.org/docs/

Re: [ALL] Integration tests for dense and sparse tensor

2021-03-16 Thread Fernando Herrera
Are there any plans to include integration testing for tensors in the pipeline? Thanks, Fernando On Mon, Mar 15, 2021 at 8:16 PM Antoine Pitrou wrote: > On Mon, 15 Mar 2021 19:48:22 + > Fernando Herrera wrote: > > Hi Neal, > > > > Thanks for the update and the link. > > > > I found that th

Re: How to compose nullable return types without Option

2021-03-16 Thread John Coleman
Lol – it’s a Kotlin Arrow question, not sure how I got here… From: Wes McKinney Date: Monday, 15 March BE 2564 20:26 To: dev@arrow.apache.org Subject: Re: How to compose nullable return types without Option Hi John, is this a question about Apache Arrow or Arrow the Kotlin library (a different p