More frequent releases require planning, decisions, voting, etc. All things that we have limited bandwidth for, hence, the reason I originally suggested time-boxed quarterly releases. I, personally, am still in favor of that approach and IF a feature gets merged between official quarterly releases and somebody wants that feature NOW, they can simply pull master to get that feature. This however assumes that master is always "releasable" and I think we've all agreed on that.
Jeremy On Wed, Nov 3, 2021 at 9:24 AM Eric Friedrich <[email protected]> wrote: > I like the idea of more frequent releases. Trying to think ahead a month. > > Are there any problems we foresee with cutting a 6.1 RC and voting? Do > we have a plan for how to address these, or do we just handle as > issues arise? > (Partial features in progress, important bugs we want to squeeze in, > etc...) > > --Eric > > On Tue, Nov 2, 2021 at 5:45 PM Dave Neuman <[email protected]> wrote: > > > > sure, seems like what we originally agreed to :) > > > > > > On Tue, Nov 2, 2021 at 3:17 PM Jeremy Mitchell <[email protected]> > > wrote: > > > > > OK, so can we agree on this? > > > > > > - minor/major releases every start of a quarter (1/1, 4/1, 7/1, 10/1) > > > - patch releases in between as determined by release manager > > > > > > by that logic, we may have N patch releases between now and 12/31 and > on > > > 1/1 we will have 6.1.0 (or 7.0 if a major is warranted). > > > > > > Jeremy > > > > > > > > > > > > On Wed, Oct 27, 2021 at 12:52 PM ocket 8888 <[email protected]> > wrote: > > > > > > > Fortunately bugfixes rarely require migrations, those are usually > only > > > for > > > > new features, or backporting would probably be impossible. > > > > > > > > On Wed, Oct 27, 2021 at 10:06 AM Jeremy Mitchell < > [email protected]> > > > > wrote: > > > > > > > > > Just to be clear, cherry-picks (aka backports) (from master to the > > > > release > > > > > branch) should be limited to bug fixes and would only apply to > patch > > > > > releases (i.e. 6.0.1) and I would expect the release manager to > > > determine > > > > > if the cherry-pick is safe or not. Yes, some can be very messy and > > > > > dangerous and IMO those should probably be punted to the next > release > > > > > branch. > > > > > > > > > > jeremy > > > > > > > > > > On Wed, Oct 27, 2021 at 8:59 AM Gray, Jonathan > > > > > <[email protected]> wrote: > > > > > > > > > > > > > why not instead ask “Is it good enough?” and “Is it better > than > > > > what > > > > > we > > > > > > presently have?” as often as possible. > > > > > > > > > > > > > may not help very much, because fixing a bug is always "better > than > > > > > what > > > > > > we > > > > > > presently have". > > > > > > > > > > > > Not always. Sometimes the risks entailed with cherry-picking and > > > > > > repatching bugfixes isn’t small. Sometimes bugfixes rely on > subtle > > > > side > > > > > > effects or behaviors not captured in the PR. It’s not safe to > assume > > > > > that > > > > > > taking code out of the context in which it was written and > tested to > > > be > > > > > > inherently stable or better. > > > > > > > > > > > > TO specifically has a problem with the idea of cherry-picking > with > > > TODB > > > > > > migrations. Since the migrations are linear and there still has > to > > > be > > > > a > > > > > > forward path from one of these cherry-picked releases, you could > end > > > up > > > > > in > > > > > > a situation where you’ve got database schema conflicts > introduced by > > > a > > > > > > cherry-pick. It feels like the desire I’m reading here is > wanting > > > the > > > > > > benefits of adopting GitFlow, while trying to still use > GitHubFlow in > > > > the > > > > > > repo. > > > > > > > > > > > > Jonathan G > > > > > > > > > > > > From: ocket 8888 <[email protected]> > > > > > > Date: Tuesday, October 26, 2021 at 10:23 PM > > > > > > To: [email protected] <[email protected] > > > > > > > > Subject: Re: [EXTERNAL] Re: [PROPOSAL] ATC Release Schedule > > > > > > > Thusly, the version number will "choose itself" based on the > SemVer > > > > > > tenants. > > > > > > > > > > > > only really works for major vs minor. Micro versions are > > > backport-only, > > > > > so > > > > > > they don't necessarily have anything to do with whether or not > > > breaking > > > > > > changes have been made on master. Which is also why > > > > > > > > > > > > > why not instead ask “Is it good enough?” and “Is it better than > > > what > > > > we > > > > > > presently have?” as often as possible. > > > > > > > > > > > > may not help very much, because fixing a bug is always "better > than > > > > what > > > > > we > > > > > > presently have". > > > > > > > > > > > > Essentially it seems like our release cadence wouldn't actually > > > change > > > > > very > > > > > > much, the proposal isn't to get rid of the quarterly releases we > > > strive > > > > > > for, but just adding a plan to backport bug fixes every month. > Which, > > > > we > > > > > > could also decide not to do, if no bugs were fixed that month, > or if > > > > it's > > > > > > too much trouble to backport the fix - "as-needed" as Jeremy > puts it. > > > > The > > > > > > big difference is we typically only do micro releases for > critical > > > bugs > > > > > > that slipped through the validation process, or for security > fixes, > > > > > whereas > > > > > > under the terms of the proposal we'd make an effort to collect > things > > > > > that > > > > > > could be fixed in supported releases every month. > > > > > > > > > > > > > asking folks to vote on new RC every few weeks isn’t feasible > > > unless > > > > > it’s > > > > > > a standing “good enough” process > > > > > > > > > > > > that would definitely be better, but I think we hope that few > enough > > > > > things > > > > > > are going into micro releases, with very little unknown since the > > > last > > > > > > minor, that validation could be focused just on "are these bugs > > > > actually > > > > > > fixed?" Could be harder with non-trivial backports, to be sure. > It > > > > would > > > > > > definitely be better to have that process, and it's definitely > > > > possible a > > > > > > micro version gets too big or complex to validate cleanly. In > that > > > > case, > > > > > > the release fails, which honestly isn't a big deal if there's no > > > > critical > > > > > > bug or security fix. > > > > > > > > > > > > On Tue, Oct 26, 2021 at 9:02 PM Jeremy Mitchell < > > > [email protected] > > > > > > > > > > > wrote: > > > > > > > > > > > > > So the idea behind regularly-scheduled, predictable > > > patch/minor/major > > > > > > > releases was a couple of things: > > > > > > > > > > > > > > 1. setting release schedule expectations with the open source > > > > community > > > > > > (as > > > > > > > Dave said). i.e. you can expect a minor/major release 1x a > quarter > > > > and > > > > > > > patch releases in between (monthly) > > > > > > > 2. removing the ambiguity of what a minor/major release > contains. > > > it > > > > > > simply > > > > > > > contains whatever was merged in that quarter. nothing to plan. > > > > nothing > > > > > to > > > > > > > debate. > > > > > > > > > > > > > > The truth is feature-based release planning is hard and > requires a > > > > lot > > > > > of > > > > > > > communication. I don't believe we have the level of > communication > > > > > > required > > > > > > > (via working group or mailing list) to effectively pull off > > > > > > feature.-based > > > > > > > releases, hence, time-based seems like the better option for > us IMO > > > > and > > > > > > if > > > > > > > we go that route, we need to stick to the schedule to be > > > successful. > > > > > > > > > > > > > > Yeah, I get it. Going from quarterly releases (which we've > failed > > > at) > > > > > to > > > > > > > monthly seems like the wrong direction. What if we change > NOTHING > > > and > > > > > > > recommit to our quarterly major/minor releases with patch > releases > > > > > being > > > > > > on > > > > > > > an as-needed basis coordinated by the release manager? So > > > basically a > > > > > big > > > > > > > no-op. :) > > > > > > > > > > > > > > Jeremy > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Oct 26, 2021 at 7:16 PM Dave Neuman <[email protected] > > > > > > wrote: > > > > > > > > > > > > > > > Resending because I don’t think it went to the list last > time. > > > > > > > > > > > > > > > > I agree that releasing whenever we have something worth > releasing > > > > is > > > > > > > ideal, > > > > > > > > however I think with an open source project we should set > > > > > expectations > > > > > > of > > > > > > > > some sort of release schedule. > > > > > > > > > > > > > > > > I also agree that we should be striving to automate as much > as > > > > > possible > > > > > > > to > > > > > > > > make our releases easier. That is something that takes time > and > > > > > > effort, > > > > > > > we > > > > > > > > have made great progress and I have no doubt that we will > > > continue > > > > to > > > > > > do > > > > > > > > so. > > > > > > > > > > > > > > > > On Tue, Oct 26, 2021 at 17:27 Gray, Jonathan > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > Instead of trying to focus on a cadence of an arbitrary > > > calendar > > > > > > > > schedule, > > > > > > > > > why not instead ask “Is it good enough?” and “Is it better > than > > > > > what > > > > > > we > > > > > > > > > presently have?” as often as possible. The first question > > > > > addresses > > > > > > > > > partially implemented features or roadmap items. The > latter > > > > > > addresses > > > > > > > > code > > > > > > > > > stability and the state of confidence in the software. > > > > > > > > > > > > > > > > > > I have a moderately high bar as it is exercising the latest > > > > merged > > > > > > > > commits > > > > > > > > > every day from scratch on new infrastructure to at minimum > > > > > > successfully > > > > > > > > > pass known traffic on a variety of delivery service > > > > configurations. > > > > > > If > > > > > > > > > that passes, it’s “good enough” to me. That’s not to say > it’s > > > > bug > > > > > > free > > > > > > > > or > > > > > > > > > perfect, but it’s an increasing bar on top of what we > already > > > > > enforce > > > > > > > on > > > > > > > > > the merge of PRs as it is. Also, to say something is “good > > > > enough” > > > > > > > > doesn’t > > > > > > > > > mean it can’t be better. How you know what “good enough” > is > > > > > > > evolutionary > > > > > > > > > and hopefully cumulative. Our project has a difficult > > > challenge > > > > > that > > > > > > > > it’s > > > > > > > > > not just one single application. It’s a complex system of > > > > > > interactions > > > > > > > > of > > > > > > > > > multiple pieces of software in particular ways. With the > > > current > > > > > > state > > > > > > > > to > > > > > > > > > support anything technically possible via both ATS+ATC, > there > > > is > > > > no > > > > > > > > “done” > > > > > > > > > when it comes to improving automated stability validation. > > > > > > > > > > > > > > > > > > Cutting releases is real work, so it’s something that we’d > have > > > > to > > > > > > > weigh > > > > > > > > > out too. Release whiplash asking folks to vote on new RC > every > > > > few > > > > > > > weeks > > > > > > > > > isn’t feasible unless it’s a standing “good enough” process > > > that > > > > > > didn’t > > > > > > > > > require humans to do anything. Don’t get me wrong, > calendars > > > and > > > > > > goals > > > > > > > > > aren’t bad, but those basic questions don’t have to wait > 1-3 > > > > months > > > > > > > > before > > > > > > > > > they’re asked and really are more important. To me at > least > > > it’s > > > > > > also > > > > > > > ok > > > > > > > > > to abandon a release if it turns out during the RC process > it’s > > > > got > > > > > > > > > critical flaws, so we don’t end up cherry-picking and > fixing up > > > > > > patches > > > > > > > > > moving the next cadence window further and further out. > > > > > > > > > > > > > > > > > > Jonathan G > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Dave Neuman <[email protected]> > > > > > > > > > Date: Tuesday, October 26, 2021 at 4:02 PM > > > > > > > > > To: [email protected] < > > > [email protected] > > > > > > > > > > > > > > Subject: [EXTERNAL] Re: [PROPOSAL] ATC Release Schedule > > > > > > > > > So we are trying to address our inability to meet our > current > > > > > release > > > > > > > > goals > > > > > > > > > by adding more releases? > > > > > > > > > Why don't we just try to get more consistent with the > quarterly > > > > > > > releases > > > > > > > > > before we change things? > > > > > > > > > If we need to do a patch release it is probably because > some > > > > > critical > > > > > > > > issue > > > > > > > > > was found, are we really going to wait for the beginning > of the > > > > > month > > > > > > > > > instead of just releasing it and fixing that issue ASAP? > > > > > > > > > Maybe I am missing some context here but this seems like > we are > > > > > > trying > > > > > > > to > > > > > > > > > solve a problem with more process instead of trying to get > > > better > > > > > at > > > > > > > what > > > > > > > > > we already agreed to? > > > > > > > > > > > > > > > > > > On Tue, Oct 26, 2021 at 1:04 PM Taylor Frey < > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > I mentioned this on the WG call this morning, but I > thought > > > I'd > > > > > add > > > > > > > my > > > > > > > > > > thoughts in response to this mailing list item since > there > > > may > > > > > have > > > > > > > > been > > > > > > > > > > some confusion with my explanation. > > > > > > > > > > > > > > > > > > > > First, I think delivering on a time-based schedule is > grand. > > > I > > > > > > think > > > > > > > > > > cutting a release once a month might be an aggressive > goal, > > > but > > > > > > once > > > > > > > we > > > > > > > > > get > > > > > > > > > > in a rhythm I believe it will become easier. > > > > > > > > > > > > > > > > > > > > Choosing a version number is an arbitrary decision. But > since > > > > we > > > > > > > appear > > > > > > > > > to > > > > > > > > > > be following Semantic Versioning ( > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://semver.org/__;!!CQl3mcHX2A!Q7m483_Mi3eCFQqXNwoXRG_EA0fdl7zl1R6ecyfz7auYIYCuI5rcDvl2EfkIcooWVEU3$ > > > > > > < > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https:/semver.org/__;!!CQl3mcHX2A!Q7m483_Mi3eCFQqXNwoXRG_EA0fdl7zl1R6ecyfz7auYIYCuI5rcDvl2EfkIcooWVEU3$ > > > > > > > > > > > > > > > > ) for our numbering > > > > > > > > > > scheme then I believe it's best to follow those > practices for > > > > > > > choosing > > > > > > > > > the > > > > > > > > > > next version number. Thusly, the version number will > "choose > > > > > > itself" > > > > > > > > > based > > > > > > > > > > on the SemVer tenants. Given the schedule, this *might* > > > result > > > > in > > > > > > > > > something > > > > > > > > > > like: > > > > > > > > > > > > > > > > > > > > 11/1/21: 6.0.1 - PATCH - Only bug fixes, performance > changes, > > > > > > tooling > > > > > > > > > > 12/1/21: 6.1.0 - MINOR - New feature (Refetch), > optionally > > > > > contains > > > > > > > > PATCH > > > > > > > > > > items > > > > > > > > > > 1/1/22: 6.1.1 - PATCH - Only bug fixes, performance > changes, > > > > > > tooling > > > > > > > > > > 2/1/22: 6.1.2 - PATCH - Only bug fixes, performance > changes, > > > > > > tooling > > > > > > > > > > 3/1/22: 6.2.0 - MINOR - New feature (Roles/Permissions), > > > > > optionally > > > > > > > > > > contains PATCH items > > > > > > > > > > 4/1/22: 7.0.0 - MAJOR - "Breaking" backwards > compatibility > > > > > promise. > > > > > > > > > Removal > > > > > > > > > > of an API version or endpoint, forced changes to clients > > > > reliant > > > > > on > > > > > > > an > > > > > > > > > API > > > > > > > > > > (like switching from Cookies to Token/Auth on all API > > > versions) > > > > > > > > > > 5/1/22: 7.0.1 - PATCH - Only bug fixes, performance > changes, > > > > > > tooling > > > > > > > > > > ... > > > > > > > > > > > > > > > > > > > > There are a plethora of choices in versioning. One > > > alternative > > > > we > > > > > > may > > > > > > > > all > > > > > > > > > > be familiar with is sometimes called CalVer. Ubuntu's > > > scheduled > > > > > > > > releases > > > > > > > > > > follow this scheme so their versioning represents it as a > > > date > > > > > > 21.04 > > > > > > > or > > > > > > > > > > 21.10. We could do something similar. The MAJOR version > could > > > > > still > > > > > > > be > > > > > > > > > > similar to SemVer, but without the restrictions allowing > us > > > to > > > > > > > > increment > > > > > > > > > > once a MAJOR change has happened (Removal of PERL, for > past > > > > > > example). > > > > > > > > The > > > > > > > > > > versioning scheme and schedule could then resemble: > > > > > > > > > > > > > > > > > > > > 11/1/21: 6.21.11 - November, 2021 > > > > > > > > > > 12/1/21: 6.21.12 - December, 2021 > > > > > > > > > > 1/1/22: 6.22.01 - January, 2022 > > > > > > > > > > 2/1/22: 6.22.02 - February, 2022 > > > > > > > > > > 3/1/22: 6.22.03 - March, 2022 > > > > > > > > > > 4/1/22: 7.0.0 - MAJOR - Arbitrary > > > > > > > > > > 5/1/22: 7.22.05 - May, 2022 > > > > > > > > > > ... > > > > > > > > > > > > > > > > > > > > Windows, MacOS pick MAJOR versions in seemingly arbitrary > > > > > fashion. > > > > > > I > > > > > > > > > recall > > > > > > > > > > Linux choosing their MAJOR versions based on numbers of > > > commits > > > > > to > > > > > > > the > > > > > > > > > > repository. > > > > > > > > > > > > > > > > > > > > I'm certainly welcome to whatever scheme we want to > implement > > > > > > moving > > > > > > > > > > forward, but if we are going to deviate from SemVer's > > > practices > > > > > > then > > > > > > > I > > > > > > > > > > think it should be explicit. > > > > > > > > > > > > > > > > > > > > T > > > > > > > > > > > > > > > > > > > > On Tue, Oct 26, 2021 at 11:58 AM Zach Hoffman < > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Seeing 2 differences: > > > > > > > > > > > • Aiming for a minor release every quarter instead of > what > > > we > > > > > > > aspired > > > > > > > > > > > to previously, new major release every quarter > > > > > > > > > > > • Aiming for a patch release every month at the start > of > > > the > > > > > > month > > > > > > > > > > > > > > > > > > > > > > Makes sense to me, +1. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -Zach > > > > > > > > > > > > > > > > > > > > > > On Tue, Oct 26, 2021 at 11:44 AM Jeremy Mitchell < > > > > > > > > > [email protected]> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > Resending to make sure it's clear this is simply a > > > > proposal. > > > > > > > Please > > > > > > > > > > > provide > > > > > > > > > > > > feedback. > > > > > > > > > > > > > > > > > > > > > > > > In the past, ATC has targeted quarterly official > releases > > > > but > > > > > > > we've > > > > > > > > > > > failed > > > > > > > > > > > > on many occasions due to "holding the bus" for > certain > > > > > features > > > > > > > or > > > > > > > > > slow > > > > > > > > > > > > release voting. In the 10/26/2021 TC working group, > we > > > > > > discussed > > > > > > > > that > > > > > > > > > > we > > > > > > > > > > > > should stick to this schedule regardless of feature > > > > > readiness. > > > > > > > > > > (remember > > > > > > > > > > > if > > > > > > > > > > > > you want a certain feature that was merged > post-release, > > > > you > > > > > > can > > > > > > > > > always > > > > > > > > > > > > pull from the master branch as we are all in > agreement > > > that > > > > > > > master > > > > > > > > > must > > > > > > > > > > > > ALWAYS be releasable). > > > > > > > > > > > > > > > > > > > > > > > > Here is the schedule that was proposed: > > > > > > > > > > > > > > > > > > > > > > > > - Quarterly minor releases (unless a Major is > > > warranted > > > > > and > > > > > > > will > > > > > > > > > be > > > > > > > > > > > > determined on case-by-case basis) > > > > > > > > > > > > - Monthly patch releases > > > > > > > > > > > > > > > > > > > > > > > > Example: > > > > > > > > > > > > > > > > > > > > > > > > 11/1/21: 6.0.1 > > > > > > > > > > > > 12/1/21: 6.0.2 > > > > > > > > > > > > 1/1/22: 6.1.0 > > > > > > > > > > > > 2/1/22: 6.1.1 > > > > > > > > > > > > 3/1/22: 6.1.2 > > > > > > > > > > > > 4/1/22: 6.2.0 > > > > > > > > > > > > 5/1/22: 6.2.1 > > > > > > > > > > > > 6/1/22: 6.2.2 > > > > > > > > > > > > 7/1/22: 6.3.0 > > > > > > > > > > > > 8/1/22: 6.3.1 > > > > > > > > > > > > 9/1/22: 6.3.2 > > > > > > > > > > > > 10/1/22: 7.0.0 <-- Apparently something "big" or non > > > > > backwards > > > > > > > > > > compatible > > > > > > > > > > > > got merged (i.e. API version removal) > > > > > > > > > > > > 11/1/22: 7.0.1 > > > > > > > > > > > > 12/1/22: 7.0.2 > > > > > > > > > > > > > > > > > > > > > > > > Please provide feedback on this approach. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > Jeremy > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
