So the idea behind regularly-scheduled, predictable patch/minor/major releases was a couple of things:
1. setting release schedule expectations with the open source community (as Dave said). i.e. you can expect a minor/major release 1x a quarter and patch releases in between (monthly) 2. removing the ambiguity of what a minor/major release contains. it simply contains whatever was merged in that quarter. nothing to plan. nothing to debate. The truth is feature-based release planning is hard and requires a lot of communication. I don't believe we have the level of communication required (via working group or mailing list) to effectively pull off feature.-based releases, hence, time-based seems like the better option for us IMO and if we go that route, we need to stick to the schedule to be successful. Yeah, I get it. Going from quarterly releases (which we've failed at) to monthly seems like the wrong direction. What if we change NOTHING and recommit to our quarterly major/minor releases with patch releases being on an as-needed basis coordinated by the release manager? So basically a big no-op. :) Jeremy On Tue, Oct 26, 2021 at 7:16 PM Dave Neuman <[email protected]> wrote: > Resending because I don’t think it went to the list last time. > > I agree that releasing whenever we have something worth releasing is ideal, > however I think with an open source project we should set expectations of > some sort of release schedule. > > I also agree that we should be striving to automate as much as possible to > make our releases easier. That is something that takes time and effort, we > have made great progress and I have no doubt that we will continue to do > so. > > On Tue, Oct 26, 2021 at 17:27 Gray, Jonathan > <[email protected]> wrote: > > > Instead of trying to focus on a cadence of an arbitrary calendar > schedule, > > why not instead ask “Is it good enough?” and “Is it better than what we > > presently have?” as often as possible. The first question addresses > > partially implemented features or roadmap items. The latter addresses > code > > stability and the state of confidence in the software. > > > > I have a moderately high bar as it is exercising the latest merged > commits > > every day from scratch on new infrastructure to at minimum successfully > > pass known traffic on a variety of delivery service configurations. If > > that passes, it’s “good enough” to me. That’s not to say it’s bug free > or > > perfect, but it’s an increasing bar on top of what we already enforce on > > the merge of PRs as it is. Also, to say something is “good enough” > doesn’t > > mean it can’t be better. How you know what “good enough” is evolutionary > > and hopefully cumulative. Our project has a difficult challenge that > it’s > > not just one single application. It’s a complex system of interactions > of > > multiple pieces of software in particular ways. With the current state > to > > support anything technically possible via both ATS+ATC, there is no > “done” > > when it comes to improving automated stability validation. > > > > Cutting releases is real work, so it’s something that we’d have to weigh > > out too. Release whiplash asking folks to vote on new RC every few weeks > > isn’t feasible unless it’s a standing “good enough” process that didn’t > > require humans to do anything. Don’t get me wrong, calendars and goals > > aren’t bad, but those basic questions don’t have to wait 1-3 months > before > > they’re asked and really are more important. To me at least it’s also ok > > to abandon a release if it turns out during the RC process it’s got > > critical flaws, so we don’t end up cherry-picking and fixing up patches > > moving the next cadence window further and further out. > > > > Jonathan G > > > > > > From: Dave Neuman <[email protected]> > > Date: Tuesday, October 26, 2021 at 4:02 PM > > To: [email protected] <[email protected]> > > Subject: [EXTERNAL] Re: [PROPOSAL] ATC Release Schedule > > So we are trying to address our inability to meet our current release > goals > > by adding more releases? > > Why don't we just try to get more consistent with the quarterly releases > > before we change things? > > If we need to do a patch release it is probably because some critical > issue > > was found, are we really going to wait for the beginning of the month > > instead of just releasing it and fixing that issue ASAP? > > Maybe I am missing some context here but this seems like we are trying to > > solve a problem with more process instead of trying to get better at what > > we already agreed to? > > > > On Tue, Oct 26, 2021 at 1:04 PM Taylor Frey <[email protected]> > wrote: > > > > > I mentioned this on the WG call this morning, but I thought I'd add my > > > thoughts in response to this mailing list item since there may have > been > > > some confusion with my explanation. > > > > > > First, I think delivering on a time-based schedule is grand. I think > > > cutting a release once a month might be an aggressive goal, but once we > > get > > > in a rhythm I believe it will become easier. > > > > > > Choosing a version number is an arbitrary decision. But since we appear > > to > > > be following Semantic Versioning ( > > > https://urldefense.com/v3/__https://semver.org/__;!!CQl3mcHX2A!Q7m483_Mi3eCFQqXNwoXRG_EA0fdl7zl1R6ecyfz7auYIYCuI5rcDvl2EfkIcooWVEU3$ > > ) for our numbering > > > scheme then I believe it's best to follow those practices for choosing > > the > > > next version number. Thusly, the version number will "choose itself" > > based > > > on the SemVer tenants. Given the schedule, this *might* result in > > something > > > like: > > > > > > 11/1/21: 6.0.1 - PATCH - Only bug fixes, performance changes, tooling > > > 12/1/21: 6.1.0 - MINOR - New feature (Refetch), optionally contains > PATCH > > > items > > > 1/1/22: 6.1.1 - PATCH - Only bug fixes, performance changes, tooling > > > 2/1/22: 6.1.2 - PATCH - Only bug fixes, performance changes, tooling > > > 3/1/22: 6.2.0 - MINOR - New feature (Roles/Permissions), optionally > > > contains PATCH items > > > 4/1/22: 7.0.0 - MAJOR - "Breaking" backwards compatibility promise. > > Removal > > > of an API version or endpoint, forced changes to clients reliant on an > > API > > > (like switching from Cookies to Token/Auth on all API versions) > > > 5/1/22: 7.0.1 - PATCH - Only bug fixes, performance changes, tooling > > > ... > > > > > > There are a plethora of choices in versioning. One alternative we may > all > > > be familiar with is sometimes called CalVer. Ubuntu's scheduled > releases > > > follow this scheme so their versioning represents it as a date 21.04 or > > > 21.10. We could do something similar. The MAJOR version could still be > > > similar to SemVer, but without the restrictions allowing us to > increment > > > once a MAJOR change has happened (Removal of PERL, for past example). > The > > > versioning scheme and schedule could then resemble: > > > > > > 11/1/21: 6.21.11 - November, 2021 > > > 12/1/21: 6.21.12 - December, 2021 > > > 1/1/22: 6.22.01 - January, 2022 > > > 2/1/22: 6.22.02 - February, 2022 > > > 3/1/22: 6.22.03 - March, 2022 > > > 4/1/22: 7.0.0 - MAJOR - Arbitrary > > > 5/1/22: 7.22.05 - May, 2022 > > > ... > > > > > > Windows, MacOS pick MAJOR versions in seemingly arbitrary fashion. I > > recall > > > Linux choosing their MAJOR versions based on numbers of commits to the > > > repository. > > > > > > I'm certainly welcome to whatever scheme we want to implement moving > > > forward, but if we are going to deviate from SemVer's practices then I > > > think it should be explicit. > > > > > > T > > > > > > On Tue, Oct 26, 2021 at 11:58 AM Zach Hoffman <[email protected]> > > > wrote: > > > > > > > Seeing 2 differences: > > > > • Aiming for a minor release every quarter instead of what we aspired > > > > to previously, new major release every quarter > > > > • Aiming for a patch release every month at the start of the month > > > > > > > > Makes sense to me, +1. > > > > > > > > > > > > -Zach > > > > > > > > On Tue, Oct 26, 2021 at 11:44 AM Jeremy Mitchell < > > [email protected]> > > > > wrote: > > > > > > > > > > Resending to make sure it's clear this is simply a proposal. Please > > > > provide > > > > > feedback. > > > > > > > > > > In the past, ATC has targeted quarterly official releases but we've > > > > failed > > > > > on many occasions due to "holding the bus" for certain features or > > slow > > > > > release voting. In the 10/26/2021 TC working group, we discussed > that > > > we > > > > > should stick to this schedule regardless of feature readiness. > > > (remember > > > > if > > > > > you want a certain feature that was merged post-release, you can > > always > > > > > pull from the master branch as we are all in agreement that master > > must > > > > > ALWAYS be releasable). > > > > > > > > > > Here is the schedule that was proposed: > > > > > > > > > > - Quarterly minor releases (unless a Major is warranted and will > > be > > > > > determined on case-by-case basis) > > > > > - Monthly patch releases > > > > > > > > > > Example: > > > > > > > > > > 11/1/21: 6.0.1 > > > > > 12/1/21: 6.0.2 > > > > > 1/1/22: 6.1.0 > > > > > 2/1/22: 6.1.1 > > > > > 3/1/22: 6.1.2 > > > > > 4/1/22: 6.2.0 > > > > > 5/1/22: 6.2.1 > > > > > 6/1/22: 6.2.2 > > > > > 7/1/22: 6.3.0 > > > > > 8/1/22: 6.3.1 > > > > > 9/1/22: 6.3.2 > > > > > 10/1/22: 7.0.0 <-- Apparently something "big" or non backwards > > > compatible > > > > > got merged (i.e. API version removal) > > > > > 11/1/22: 7.0.1 > > > > > 12/1/22: 7.0.2 > > > > > > > > > > Please provide feedback on this approach. > > > > > > > > > > Thanks, > > > > > > > > > > Jeremy > > > > > > > > > >
