[IPC] Delta Dictionary Flag Clarification for Multi-Batch IPC

2024-01-19 Thread Chris Larsen
Hi folks, I'm working on multi-batch dictionary with delta support in Java [1] and would like some clarifications. Given the "isDelta" flag in the dictionary message [2], when should this be set to "true"? 1) If we have dictionary with an ID of 1 that we want to delta encode and it is used

Re: [DISC] Improve Arrow Release verification process

2024-01-19 Thread Jacob Wujciak-Jens
I concur, a minimally scoped verification script for the actual voting process without any binary verification etc. should be created. The ease in verifying a release will lower the burden to participate in the vote which is good for the community and will even be necessary if we ever want to

Re: [DISCUSS][MATLAB] Proposed "Category B" License for Bundling MATLAB MEX Build Artifacts in Official Arrow Release

2024-01-19 Thread Sarah Gilmore
Hi Roman, > FWIW: while these are all excellent questions for the pre-work, if there > needs to be an ultimate statement on this -- you'll have to file a LEGAL > JIRA. E.g.: https://issues.apache.org/jira/browse/LEGAL-506 > > (plz include all the relevant details when filing it -- whatever comes

Re: [DISC] Improve Arrow Release verification process

2024-01-19 Thread Andrew Lamb
I would second this notion that manually running tests that are already covered as part of CI as part of the release process is of (very) limited value. While we do the same thing (compile and run some tests) as part of the Rust release this has never caught any serious defect I am aware of and

Re: [DISC] Improve Arrow Release verification process

2024-01-19 Thread Antoine Pitrou
Well, if the main objective is to just follow the ASF Release guidelines, then our verification process can be simplified drastically. The ASF indeed just requires: """ Every ASF release MUST contain one or more source packages, which MUST be sufficient for a user to build and test the

Re: [DISC] Improve Arrow Release verification process

2024-01-19 Thread Dane Pitkin
I agree that this is a great time to look at improving the verification process. One solution I've seen work fairly well is to convert large bash scripts into a lightweight ETL pipeline that caches the results/status of each node as it executes. That way, restarting a pipeline at the right

Re: [VOTE] Release Apache Arrow 15.0.0 - RC1

2024-01-19 Thread Raúl Cumplido
El vie, 19 ene 2024 a las 15:56, David Li () escribió: > > +1 (binding) > > Verified sources on Debian 12 'bookworm'. I had issues with binaries but that > was because of AlmaLinux failing to verify its own GPG key for some reason. I had the same issue locally. I fixed it by removing the local

Re: [DISCUSS][MATLAB] Proposed "Category B" License for Bundling MATLAB MEX Build Artifacts in Official Arrow Release

2024-01-19 Thread Roman Shaposhnik
On Thu, Jan 18, 2024 at 12:24 PM Ian Cook wrote: > > Hi Sarah, > > Thanks for pursuing this. > > The ASF 3rd Party License Policy lists a number of standard, > off-the-shelf licenses that are compatible with Category B, but the > policy does not include any provision for custom-written licenses.

Re: [VOTE] Release Apache Arrow 15.0.0 - RC1

2024-01-19 Thread Matt Topol
> As the issue seems to have been around for 1.5 or 2 years I don't think this issue requires a new RC Fair enough. Everything else seems good to me, +1 (binding). Verified on Ubuntu 22.04 On Fri, Jan 19, 2024 at 9:55 AM David Li wrote: > +1 (binding) > > Verified sources on Debian 12

Re: [VOTE] Release Apache Arrow 15.0.0 - RC1

2024-01-19 Thread David Li
+1 (binding) Verified sources on Debian 12 'bookworm'. I had issues with binaries but that was because of AlmaLinux failing to verify its own GPG key for some reason. On Thu, Jan 18, 2024, at 04:40, Raúl Cumplido wrote: > El mié, 17 ene 2024 a las 23:37, Matt Topol () > escribió: >> >> Yea, I

[DISC] Improve Arrow Release verification process

2024-01-19 Thread Raúl Cumplido
Hi, One of the challenges we have when doing a release is verification and voting. Currently the Arrow verification process is quite long, tedious and error prone. I would like to use this email to get feedback and user requests in order to improve the process. Several things already on my