I also think release on demand is a good strategy. The primary reasons to do an arrow-rs release every 2 weeks were: 1. To have predictable cadence into downstream projects (e.g. datafusion and others) 2. Amortize the overhead associated with each release (the process is non trivial and the current 72 hour voting window adds some backpressure as well -- I remember Wes may have said windows shorter than 72 hours might be fine too)
On Wed, Sep 8, 2021 at 12:19 AM QP Hou <q...@scribd.com.invalid> wrote: > A minor note on the Rust side of things. arrow-rs has a 2 weeks > release cycle, but arrow-datafusion mostly does release on demand at > the moment. Our most uptodate release processes are documented at [1] > and [2]. > > [1]: https://github.com/apache/arrow-rs/blob/master/dev/release/README.md > [2]: > https://github.com/apache/arrow-datafusion/blob/master/dev/release/README.md > > On Tue, Sep 7, 2021 at 4:01 PM Jacob Quinn <quinn.jac...@gmail.com> wrote: > > > > Thanks kou. > > > > I think the TODO action list looks good. > > > > The one point I think could use some additional discussion is around the > > release cadence: it IS desirable to be able to release more frequently > than > > the parent repo 3-4 month cadence. But we also haven't had the frequency > of > > commits to necessarily warrant a release every 2 weeks. I can think of > two > > possible options, not sure if one or the other would be more compatible > > with the apache release process: > > > > 1) Allow for release-on-demand; this is idiomatic for most Julia packages > > I'm aware of. When a particular bug is fixed, or feature added, a user > can > > request a release, a little discussion happens, and a new release is > made. > > This approach would work well for the "bursty" kind of contributions > we've > > seen to Arrow.jl where development by certain people will happen > frequently > > for a while, then take a break for other things. This also avoids having > > "scheduled" releases (every 2 weeks, 3 months, etc.) where there hasn't > > been significant updates to necessarily warrant a new release. This > > approach may also facilitate differentiating between bugfix (patch) > > releases vs. new functionality releases (minor), since when a release is > > requested, it could be specified whether it should be patch or minor (or > > major). > > > > 2) Commit to a scheduled release pattern like every 2 weeks, once a > month, > > etc. This has the advantage of consistency and clearer expectations for > > users/devs involved. A release also doesn't need to be requested, because > > we can just wait for the scheduled time to release. In terms of the > > "unnecessary releases" mentioned above, it could be as simple as > > "cancelling" a release if there hasn't been significant updates in the > > elapsed time period. > > > > My preference would be for 1), but that's influenced from what I'm > familiar > > with in the Julia package ecosystem. It seems like it would still fit in > > the apache way since we would formally request a new release, wait the > > elapsed amount of time for voting (24 hours would be preferrable), then > at > > the end of the voting period, a new release could be made. > > > > Thanks again kou for helping support the Julia implementation here. > > > > -Jacob > > > > 2) > > > > On Sun, Sep 5, 2021 at 3:25 PM Sutou Kouhei <k...@clear-code.com> wrote: > > > > > Hi, > > > > > > Sorry for the delay. This is a continuation of the "Status > > > of Arrow Julia implementation?" thread: > > > > > > > > > > https://lists.apache.org/x/thread.html/r6d91286686d92837fbe21dd042801a57e3a7b00b5903ea90a754ac7b%40%3Cdev.arrow.apache.org%3E > > > > > > I summarize the current status, the next actions and items > > > to be discussed. > > > > > > The current status: > > > > > > * The Julia Arrow implementation uses > > > https://github.com/JuliaData/Arrow.jl as a "dev branch" > > > instead of creating a branch in > > > https://github.com/apache/arrow > > > * The Julia Arrow implementation wants to use GitHub > > > for the main issue management platform > > > * The Julia Arrow implementation wants to release > > > more frequency than 1 release per 3-4 months > > > * The current workflow of the Rust Arrow implementation > > > will also fit the Julia Arrow implementation > > > > > > The current workflow of the Rust Arrow implementation: > > > > > > > > > > https://docs.google.com/document/d/1TyrUP8_UWXqk97a8Hvb1d0UYWigch0HAephIjW7soSI/edit#heading=h.kv1hwbhi3cmi > > > > > > * Uses apache/arrow-rs and apache/arrow-datafusion instead > > > of apache/arrow for repository > > > > > > * Uses GitHub instead of JIRA for issue management > > > platform > > > > > > > > > > https://docs.google.com/document/d/1tMQ67iu8XyGGZuj--h9WQYB9inCk6c2sL_4xMTwENGc/edit > > > > > > * Releases a new minor and patch version every 2 weeks > > > in addition to the quarterly release of the other releases > > > > > > The next actions after we get a consensus about this > > > discussion: > > > > > > 1. Start voting the Julia Arrow implementation move like > > > the Rust's one: > > > > > > > > > > https://lists.apache.org/x/thread.html/r44390a18b3fbb08ddb68aa4d12f37245d948984fae11a41494e5fc1d@%3Cdev.arrow.apache.org%3E > > > > > > 2. Create apache/arrow-julia > > > > > > 3. Start IP clearance process to import JuliaData/Arrow.jl > > > to apache/arrow-julia > > > > > > (We don't use julia/Arrow/ in apache/arrow.) > > > > > > 4. Import JuliaData/Arrow.jl to apache/arrow-julia > > > > > > 5. Prepare integration tests CI in apache/arrow-julia and > apache/arrow > > > > > > 6. Prepare releasing tools in apache/arrow-julia and apache/arrow > > > > > > 7. Remove julia/... from apache/arrow and leave > > > julia/README.md pointing to apache/arrow-julia > > > > > > > > > Items to be discussed: > > > > > > * Interval of minor and patch releases > > > > > > * The Rust Arrow implementation uses 2 weeks. > > > > > > * Does the Julia Arrow implementation also wants to use > > > 2 weeks? > > > > > > * Can we accordance with the Apache way with this workflow > > > without pain? > > > > > > The Rust Arrow implementation workflow includes the > > > following for this: > > > > > > > > > > https://docs.google.com/document/d/1TyrUP8_UWXqk97a8Hvb1d0UYWigch0HAephIjW7soSI/edit#heading=h.kv1hwbhi3cmi > > > > > > > Contributors will be required to write issues for > > > > planned features and bug fixes so that we have > > > > visibility and opportunities for collaboration > > > > before a PR shows up. > > > > > > * More items? > > > > > > > > > Thanks, > > > -- > > > kou > > > >
