+1 to " If there are no contributors for an LTS release, there will be no LTS release. If there *are* contributors, then we'll find a way to make some sort of LTS model work within the other constraints we have."
+10000 to let's get some folks who are interested a place to collaborate and talk to each other and get things up and running. That would be the first step. If it ends up in LTS and skip-level upgrades for all projects, we all win! That's a destination, not the first step! Thanks, Dims On Sun, Nov 12, 2017 at 6:04 AM, Doug Hellmann <d...@doughellmann.com> wrote: > Excerpts from Clint Byrum's message of 2017-11-11 08:41:15 -0800: >> Excerpts from Doug Hellmann's message of 2017-11-10 13:11:45 -0500: >> > Excerpts from Clint Byrum's message of 2017-11-08 23:15:15 -0800: >> > > Excerpts from Samuel Cassiba's message of 2017-11-08 08:27:12 -0800: >> > > > On Tue, Nov 7, 2017 at 3:28 PM, Erik McCormick >> > > > <emccorm...@cirrusseven.com> wrote: >> > > > > Hello Ops folks, >> > > > > >> > > > > This morning at the Sydney Summit we had a very well attended and >> > > > > very >> > > > > productive session about how to go about keeping a selection of past >> > > > > releases available and maintained for a longer period of time (LTS). >> > > > > >> > > > > There was agreement in the room that this could be accomplished by >> > > > > moving the responsibility for those releases from the Stable Branch >> > > > > team down to those who are already creating and testing patches for >> > > > > old releases: The distros, deployers, and operators. >> > > > > >> > > > > The concept, in general, is to create a new set of cores from these >> > > > > groups, and use 3rd party CI to validate patches. There are lots of >> > > > > details to be worked out yet, but our amazing UC (User Committee) >> > > > > will >> > > > > be begin working out the details. >> > > > > >> > > > > Please take a look at the Etherpad from the session if you'd like to >> > > > > see the details. More importantly, if you would like to contribute to >> > > > > this effort, please add your name to the list starting on line 133. >> > > > > >> > > > > https://etherpad.openstack.org/p/SYD-forum-upstream-lts-releases >> > > > > >> > > > > Thanks to everyone who participated! >> > > > > >> > > > > Cheers, >> > > > > Erik >> > > > > >> > > > > __________________________________________________________________________ >> > > > > OpenStack Development Mailing List (not for usage questions) >> > > > > Unsubscribe: >> > > > > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> > > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > >> > > > In advance, pardon the defensive tone. I was not in a position to >> > > > attend, or even be in Sydney. However, as this comes across the ML, I >> > > > can't help but get the impression this effort would be forcing more >> > > > work on already stretched teams, ie. deployment-focused development >> > > > teams already under a crunch as contributor count continues to decline >> > > > in favor of other projects inside and out of OpenStack. >> > > > >> > > >> > > I suspect if LTS's become a normal part of OpenStack, most deployment >> > > projects will decline to support the interim releases. We can infer this >> > > from the way Ubuntu is used. This might actually be a good thing for the >> > > chef OpenStack community. 3 out of 3.5 of you can focus on the LTS bits, >> > > and the 0.5 person can do some best effort to cover the weird corner >> > > case of "previous stable release to master". >> > > >> > > The biggest challenge will be ensuring that the skip-level upgrades >> > > work. The current grenade based upgrade jobs are already quite a bear to >> > > keep working IIRC. I've not seen if chef or any of the deployment >> > > projects >> > > test upgrades like that. >> > > >> > > However, if people can stop caring much about the interim releases and >> > > just keep "previous LTS to master" upgrades working, then that might be >> > > good for casual adoption. >> > > >> > > Personally I'd rather we make it easier to run "rolling release" >> > > OpenStack. Maybe we can do that if we stop cutting stable releases every >> > > 6 months. >> > > >> > >> > We should stop calling what we're talking about "LTS". It isn't >> > going to match the expectations of anyone receiving LTS releases >> > for other products, at least at first. Perhaps "Deployer Supported" >> > or "User Supported" are better terms for what we're talking about. >> > >> >> I believe this state we're in is a stop-gap on the way to the full >> LTS. People are already getting stuck. We're going to help them stay stuck >> by upstreaming bug fixes. We should be mindful of that and provide a way >> to get less-stuck. The LTS model from other projects has proven quite >> popular, and it would make sense for us to embrace it if our operators >> are hurting with the current model, which I believe they are. >> >> > In the "LTS" room we did not agree to stop cutting stable releases >> > or to start supporting upgrades directly from N-2 (or older) to N. >> > Both of those changes would require modifying the support the >> > existing contributor base has committed to provide. >> > >> >> Thanks, I am just inferring those things from what was agreed on. However, >> It would make a lot of sense to discuss the plans for the future, even >> if we don't have data from the present proposal. >> >> > Fast-forward upgrades will still need to run the migration steps >> > of each release in order, one at a time. The team working on that >> > is going to produce a document describing what works today so we >> > can analyze it for ways to improve the upgrade experience, for both >> > fast-forward and "regular" upgrades. That was all discussed in a >> > separate session. >> > >> >> We are what we test. If we're going to test fast-forwards, how far into >> the past do we test? It makes a lot of sense to pick a release every few >> cycles and say "we're going to support fast-forwards from this point >> to master for the next 2 years". Then users can safely clamber their >> way onto that release, and spend the next 2 years paying down debt, >> knowing they have a chance for a skip-level upgrade. It's not a model >> I recommend, but I do think it's the way many companies work today. > > That seems like a good approach. When the team working on fast-forward > upgrades finishes the process document it should become more clear how > we could potentially start running upgrade tests, and what changes might > be needed to support doing so. Having some tags for supporting > fast-forward and skip-level upgrades would be good, too, so deployers > can understand easily what sort of upgrade models are supported by > different projects. > >> Anyway, let me be clear. I think we should listen to our operators. I >> wasn't at the forum, so I couldn't do that there. But what I have seen >> at every org I've been at, is one of two things: >> >> 1) Running close to master, merging in and deploying on a regular >> cadence. Confidence in the pipeline and actual CD nirvana. This >> is rare, and requires commitment and trust at multiple levels. But >> it's the way I personally recommend people run OpenStack. >> >> 2) Falling 1 release behind the moment they deploy the stable release, >> and then falling further and further behind from there. >> >> This is my own anecdotal evidence, but we have some data to back it up >> too. As of the April survey[1] we see that _most_ users were on Mitaka or >> earlier. Mitaka went EOL in _April_. So users are getting run over by the >> EOL policy, and no amount of "you should upgrade faster" is going to fix >> that. Perhaps we should slow down the tagging. Some have suggested going >> to once per year. That makes some sense to me. Especially if we continue >> to support CD'ing from master. > > The problem with stable maintenance isn't so much the number of > releases involved. Stable branches become more brittle as they age, > because the underlying OS changes with upgrades in dependencies. > We discussed a bunch of different ways to address that, from > containers to static images to managing a "frozen" repo of packages > to install during tests. Any decisions on what approach to use will > be left to the people actually doing the work, along with the folks > maintaining the tools used (in particular, the infra and QA teams). > > In preparation for the summit I went back through all of the notes > I could find about stable branches from previous summits. The first > mention of stable branches I found was at the Folsom summit, and > that was a discussion of doing stable releases "more often," which > implies that we had the stable branches at least as early as Essex. > I wasn't around before Folsom, so I'm not sure when they actually > started. The first mention of an LTS release I found was the Juno > summit, which was later than I expected. > > No one in the room disputed the assertion that what we're doing for > stable releases is insufficient. We are all trying to listen to > users' needs. Continuing to just say "we should do LTS releases" > however doesn't acknowledge the other long standing fact, which is > that over all of the time we have talked about it we have had *no > contributors willing to actually support an LTS release model > upstream.* > > We act like people have been saying "no, you are not allowed to > maintain branches for longer than the stable team says is OK," or > "no, we'll never provide an LTS release," but that's not how our > community works. The policies are set by the contributors. If there > are no contributors for an LTS release, there will be no LTS release. > If there *are* contributors, then we'll find a way to make some > sort of LTS model work within the other constraints we have. > > It seems now that we have people saying they would do some amount > of maintenance work (probably less than we try to do for stable > branches under our current model), if they could. The first change, > what has been proposed, is to give them a place to do the work. > Then we'll see if anyone actually does it, and if so we can plan > further improvements. > >> Then we can ask the operators if they would prefer that we stop EOL'ing >> things out from under them. We can make it a community goal to have a >> "feature" that is "you can upgrade from the last one" and have "the last >> one" be something older than 6 months, maybe even older than 1 year. >> >> [1] https://www.openstack.org/assets/survey/April2017SurveyReport.pdf >> > > _______________________________________________ > OpenStack-operators mailing list > OpenStack-operators@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators -- Davanum Srinivas :: https://twitter.com/dims _______________________________________________ OpenStack-operators mailing list OpenStack-operators@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators