Re: [RESULT][VOTE] Release Apache Arrow 10.0.0 - RC0

2022-11-07 Thread Sutou Kouhei
Current status: - [Done] Make the released version as “RELEASED” on JIRA - [Done] Make the CPP PARQUET related version as “RELEASED” on JIRA - [Done] Start the new version on JIRA on the ARROW project - [Done] Start the new version on JIRA for the related CPP PARQUET version - [Done] Merge

Re: [DISCUSS] Docs site for ADBC

2022-11-07 Thread Sutou Kouhei
Hi, I don't object it. Are you going to use https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-Specifyingasub-directorytopublishto ? If it works, we may want to use it for other sub directories such as https://arrow.apache.org/datafusion/ and

Struct evolution

2022-11-07 Thread Matthew Scanlon
Good afternoon, I wanted to reach out and open a dialog about structs, the evolution of them in schemas, and if support for such a feature is on the road map or a hard pass for the arrow team. Currently, it appears structs support removing a field, but will there be support for adding fields

[RESULT][VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 14.0.0 RC1

2022-11-07 Thread Andy Grove
The vote has passed with three binding +1 votes (and seven +1 votes in total). Thank you to all who helped with the release verification. The crates have been published. On Sat, Nov 5, 2022 at 9:35 PM Ian Joiner wrote: > +1 (Non-binding) > > Verified on my System76/Ubuntu 22.04/AMD64 > > On

[DISCUSS] Docs site for ADBC

2022-11-07 Thread David Li
I'd like to publish (nightly) API docs for ADBC, is there any objection to configuring this under something like arrow.apache.org/adbc/docs/... using asf.yaml? -David

Re: [DISCUSS] Pyarrow wheels for Python 3.11

2022-11-07 Thread Antoine Pitrou
On Mon, 7 Nov 2022 18:21:34 +0100 Antoine Pitrou wrote: > Le 07/11/2022 à 15:37, Raúl Cumplido a écrit : > > El lun, 7 nov 2022 a las 14:14, Neal Richardson (< > > neal.p.richard...@gmail.com>) escribió: > > > >> Two unrelated thoughts: > >> > >> 1. Since it sounds like we need to do a

Re: [DISCUSS] Pyarrow wheels for Python 3.11

2022-11-07 Thread Antoine Pitrou
Le 07/11/2022 à 15:37, Raúl Cumplido a écrit : El lun, 7 nov 2022 a las 14:14, Neal Richardson (< neal.p.richard...@gmail.com>) escribió: Two unrelated thoughts: 1. Since it sounds like we need to do a patch release for the wheels, should we include any other critical bugfixes that have

Re: [DISCUSS] Integrate existing Spark connector for Flight

2022-11-07 Thread David Li
Sorry for the late reply. To be clear, even if you are not looking for maintainer help: - You will (most likely) have to go through an IP clearance process. - You will not be able to push code without a committer or PMC. - You will not be able to release code without a PMC vote. Is that really

Re: [DISCUSS] Pyarrow wheels for Python 3.11

2022-11-07 Thread Raúl Cumplido
El lun, 7 nov 2022 a las 14:14, Neal Richardson (< neal.p.richard...@gmail.com>) escribió: > Two unrelated thoughts: > > 1. Since it sounds like we need to do a patch release for the wheels, > should we include any other critical bugfixes that have been reported > affecting 10.0.0? Do we have a

Re: [Rust][Blog] Fast and Memory Efficient Multi-Column Sorts

2022-11-07 Thread Gavin Ray
This is awesome, thanks for sharing! I was at the All Things Open conference recently, and Influx had a booth there. I went over to try to ask about the IOx/Datafusion stuff but unfortunately nobody at the booth knew anything about the technical details. Maybe next time =) On Mon, Nov 7, 2022

Re: [Rust][Blog] Fast and Memory Efficient Multi-Column Sorts

2022-11-07 Thread Andrew Lamb
The blog has been published: https://arrow.apache.org/blog/2022/11/07/multi-column-sorts-in-arrow-rust-part-1/ https://arrow.apache.org/blog/2022/11/07/multi-column-sorts-in-arrow-rust-part-2/ Thank you to all who contributed content and suggestions Andrew On Tue, Nov 1, 2022 at 4:56 PM Andrew

Re: [DISCUSS] Pyarrow wheels for Python 3.11

2022-11-07 Thread Neal Richardson
Two unrelated thoughts: 1. Since it sounds like we need to do a patch release for the wheels, should we include any other critical bugfixes that have been reported affecting 10.0.0? Do we have a 10.0.1 Fix Version in Jira already, and/or any known issues we would want to include? 2. Since I said

Re: [DISCUSS] Pyarrow wheels for Python 3.11

2022-11-07 Thread Raúl Cumplido
Thanks Jarek, > I am not sure whether the changes included any code that should find > its way into the official source package on the > https://downloads.apache.org/arrow/ - but if yes, then there is not > much choice from the Apache By-laws but the convenience/compiled > packages (and this is

Re: [DISCUSS] Pyarrow wheels for Python 3.11

2022-11-07 Thread Jarek Potiuk
Just a comment from an outsider (but one that peeks at the communication and I am very much interested as PyArrow binary wheel release is one of the important prerequisites for Apache Airflow official 3.11 support - mostly as transitive dependencies for other dependencies we use). I am not sure

[DISCUSS] Pyarrow wheels for Python 3.11

2022-11-07 Thread Raúl Cumplido
Hi, As you might be aware, the release of Python 3.11 happened around the time we were releasing Apache Arrow 10.0.0. There seems to be quite a lot of users that would like to use Pyarrow on the new Python 3.11 version, this can be seen on the amount of comments on the PR that added wheels for