Re: [DISCUSS] [Rust] Move Rust components to new repos and process

2021-04-10 Thread Jorge Cardoso Leitão
Hi Jacob, My understanding is that our integration tests can be roughly summarized as follows: * Every Implementation compiles 4 binaries: * a consumer (validate) * a producer (json_to_file) * a flight server * a flight client These binaries are used via a CLI. * The

Re: 4.0 release preparation

2021-04-10 Thread Neal Richardson
Thanks Weston. Regarding jpype, there was a discussion about doing this on the April 6 nightly build thread: https://lists.apache.org/thread.html/rc945a730da4b8820cba3b121de02fa165e28c475f1de9ef4a848339f%40%3Cdev.arrow.apache.org%3E On Sat, Apr 10, 2021 at 2:14 PM Ian Cook wrote: > I can

Re: 4.0 release preparation

2021-04-10 Thread Ian Cook
I can confirm that the test-r-minimal-build failure was fixed in ARROW-12197. Ian On Sat, Apr 10, 2021 at 17:06 Weston Pace wrote: > Nightly build triage (based on nightly builds from 4/9): > > Failed Tasks: > - conda-linux-gcc-py36-aarch64: > ARROW-12324 (conda builds timing out, conda

Re: Status of Arrow Julia implementation?

2021-04-10 Thread Micah Kornfield
> > Ok, I've had a chance to discuss with a few other Julia developers and > review various options. I think it's best to drop the Julia code from the > physical apache/arrow repo. The extra overhead on development, release > process, and user issue reporting and PR contributing are too much in >

Re: 4.0 release preparation

2021-04-10 Thread Weston Pace
Nightly build triage (based on nightly builds from 4/9): Failed Tasks: - conda-linux-gcc-py36-aarch64: ARROW-12324 (conda builds timing out, conda slow) - conda-linux-gcc-py37-aarch64: ARROW-12324 (conda builds timing out, conda slow) - conda-osx-clang-py37-r40: Appears to have been an

Re: [DISCUSS] [Rust] Move Rust components to new repos and process

2021-04-10 Thread Dominik Moritz
I have been working on an arrow-was library that uses the rust crate. I was hoping to eventually merge this into the arrow repo. Since wasm is often used on the web, I was hoping to maybe use the wasm library as a core for the arrow-js library (which is in the main arrow repo). So there is just

Re: [DISCUSS] [Rust] Move Rust components to new repos and process

2021-04-10 Thread Jacob Quinn
Jorge, * in rust, run integration tests against the latest apache/master on every > PR > I've started to familiarize myself with the archery integration framework over the last few days. Could you clarify for the "archery novices" what exactly ^ this line would mean? Does apache/master refer to

[Rust] [Ballista] PR reviews / merges

2021-04-10 Thread Andy Grove
Now that Ballista is merged into the Arrow repo, there is some follow-up work around CI and updating documentation. Given that Apache Arrow follows a Commit-then-review policy, I am planning on merging some of these early trivial PRs (that do not involve code changes) without requesting or

Re: [DISCUSS] [Rust] Move Rust components to new repos and process

2021-04-10 Thread Jorge Cardoso Leitão
Hi, Wrt to integration tests, I agree that it is important to have a plan prior to this. What we have been doing in the apache/arrow: 1. only release if integration tests pass against each other 2. release the signed tar with the latest of every implementation (i.e. master) My suggestion for