It is not just about BVT. I'd suggest to monitor situation overall, including failures of system tests [1]. If we see regressions there, or some test cases will start flapping (what is even worse), then we'd have to revert back to CentOS 7.1.
[1] https://github.com/openstack/fuel-qa On Wed, Mar 2, 2016 at 10:16 AM Dmitry Borodaenko <dborodae...@mirantis.com> wrote: > I agree with Mike's concerns, and propose to make these limitations (4 > weeks before FF for OS upgrades, 2 weeks for upgrades of key > dependencies -- RabbitMQ, MCollective, Puppet, MySQL, PostgreSQL, > anything else?) official for 10.0/Newton. > > For 9.0/Mitaka, it is too late to impose them, so we just have to be > very careful and conservative with this upgrade. First of all, we need > to have a green BVT before and after switching to the CentOS 7.2 repo > snapshot, so while I approved the spec, we can't move forward with this > until BVT is green again, and right now it's red: > > https://ci.fuel-infra.org/job/9.0.fuel_community.ubuntu.bvt_2/ > > If we get it back to green but it becomes red after the upgrade, you > must switch back to CentOS 7.1 *immediately*. If you are able to stick > to this plan, there is still time to complete the transition today > without requiring an FFE. > > -- > Dmitry Borodaenko > > > On Wed, Mar 02, 2016 at 05:53:53PM +0000, Mike Scherbakov wrote: > > Formally, we can merge it today. Historically, every update of OS caused > us > > instability for some time: from days to a couple of month. > > Taking this into account and number of other exceptions requested, > overall > > stability of code, my opinion would be to postpone this to 10.0. > > > > Also, I'd suggest to change the process, and have freeze date for all OS > > updates no later than a month before official FF date. This will give us > > time to stabilize, and ensure that base on which all new code is being > > developed is stable when approaching FF. > > > > I'd also propose to have freeze for major upgrades of 3rd party packages > no > > later than 2 weeks before FF, which Fuel depends heavily upon. For > > instance, such will include RabbitMQ, MCollective, Puppet. > > > > On Wed, Mar 2, 2016 at 7:34 AM Igor Marnat <imar...@mirantis.com> wrote: > > > > > Igor, > > > couple of points from my side. > > > > > > CentOS 7.2 will be getting updates for several more months, and we have > > > snapshots and all the mechanics in place to switch to the next version > when > > > needed. > > > > > > Speaking of getting this update into 9.0, we actually don't need FFE, > we > > > can merge remaining staff today. It has enough reviews, so if you add > your > > > +1 today, we don't need FFE. > > > > > > https://review.openstack.org/#/c/280338/ > > > https://review.fuel-infra.org/#/c/17400/ > > > > > > > > > > > > Regards, > > > Igor Marnat > > > > > > On Wed, Mar 2, 2016 at 6:23 PM, Dmitry Teselkin < > dtesel...@mirantis.com> > > > wrote: > > > > > >> Igor, > > >> > > >> Your statement about updates for 7.2 isn't correct - it will receive > > >> updates, because it's the latest release ATM. There is *no* pinning > inside > > >> ISO, and the only place where it was 8.0 were docker containers just > > >> because we had to workaround some issues. But there are no docker > > >> containers in 9.0, so that's not the case. > > >> The proposed solution to switch to CentOS-7.2 in fact is based on > > >> selecting the right snapshot with packages. There is no pinning in > ISO (it > > >> was in earlier versions of the spec but was removed). > > >> > > >> On Wed, Mar 2, 2016 at 6:11 PM, Igor Kalnitsky < > ikalnit...@mirantis.com> > > >> wrote: > > >> > > >>> Dmitry, Igor, > > >>> > > >>> > Very important thing is that CentOS 7.1 which master node is based > now > > >>> > don't get updates any longer. > > >>> > > >>> If you are using "fixed" release you must be ready that you won't get > > >>> any updates. So with CentOS 7.2 the problem still the same. > > >>> > > >>> However, let's wait for Fuel PTL decision. I only shared my POV: > > >>> that's not a critical feature, and taking into account the risks of > > >>> regression - I'd prefer to do not accept it in 9.0. > > >>> > > >>> Regards, > > >>> Igor > > >>> > > >>> On Wed, Mar 2, 2016 at 4:42 PM, Igor Marnat <imar...@mirantis.com> > > >>> wrote: > > >>> > Igor, > > >>> > please note that this is pretty much not like update of master node > > >>> which we > > >>> > had in 8.0. This is minor _update_ of CentOS from 7.1 to 7.2 which > team > > >>> > tested for more than 2 months already. > > >>> > > > >>> > We don't expect it to require any additional efforts from core or > qa > > >>> team. > > >>> > > > >>> > Very important thing is that CentOS 7.1 which master node is based > now > > >>> don't > > >>> > get updates any longer. Updates are only provided for CentOS 7.2. > > >>> > > > >>> > So we'll have to switch CentOS 7.1 to CentOS 7.2 anyways. > > >>> > > > >>> > We can do it now for more or less free, later in release cycle for > > >>> higher > > >>> > risk and QA efforts and after the release for 2x price because of > > >>> additional > > >>> > QA cycle we'll need to pass through. > > >>> > > > >>> > > > >>> > > > >>> > Regards, > > >>> > Igor Marnat > > >>> > > > >>> > On Wed, Mar 2, 2016 at 2:57 PM, Dmitry Teselkin < > > >>> dtesel...@mirantis.com> > > >>> > wrote: > > >>> >> > > >>> >> Hi Igor, > > >>> >> > > >>> >> Postponing this till Fuel 10 means we have to elaborate a plan to > do > > >>> such > > >>> >> upgrade for Fuel 9 after the release - the underlying system will > not > > >>> get > > >>> >> updated on it's own, and the security issues will not close > > >>> themselves. The > > >>> >> problem here is that such upgrade of deployed master node > requires a > > >>> lot of > > >>> >> QA work also. > > >>> >> > > >>> >> Since we are not going to update package we build on our own (they > > >>> still > > >>> >> targeted 7.1) switching master node base to CentOS-72 is not that > > >>> dangerous > > >>> >> as it seems. > > >>> >> > > >>> >> On Wed, Mar 2, 2016 at 1:54 PM, Igor Kalnitsky < > > >>> ikalnit...@mirantis.com> > > >>> >> wrote: > > >>> >>> > > >>> >>> Hey Dmitry, > > >>> >>> > > >>> >>> No offence, but I rather against that exception. We have too many > > >>> >>> things to do in Mitaka, and moving to CentOS 7.2 means > > >>> >>> > > >>> >>> * extra effort from core team > > >>> >>> * extra effort from qa team > > >>> >>> > > >>> >>> Moreover, it might block development by introducing unpredictable > > >>> >>> regressions. Remember 8.0? So I think it'd be better to reduce > risk > > >>> of > > >>> >>> regressions that affects so many developers by postponing CentOS > 7.2 > > >>> >>> till Fuel 10. > > >>> >>> > > >>> >>> Thanks, > > >>> >>> Igor > > >>> >>> > > >>> >>> > > >>> >>> On Mon, Feb 29, 2016 at 7:13 PM, Dmitry Teselkin < > > >>> dtesel...@mirantis.com> > > >>> >>> wrote: > > >>> >>> > I'd like to ask for a feature freeze exception for switching to > > >>> >>> > CentOS-7.2 > > >>> >>> > feature [0]. > > >>> >>> > > > >>> >>> > CentOS-7.2 ISO's have been tested periodically since the > beginning > > >>> of > > >>> >>> > the > > >>> >>> > year, and all major issues were addressed / fixed at the > moment. > > >>> During > > >>> >>> > the > > >>> >>> > last weekend I've made 70 BVT runs to verify that the solution > > >>> [2] for > > >>> >>> > the > > >>> >>> > last issue - e1000 transmit unit hangs works. And it works, 0 > > >>> tests of > > >>> >>> > 35 > > >>> >>> > failed [3]. > > >>> >>> > > > >>> >>> > Benefits of switching to CentOS-7.2 are quite obvious - we will > > >>> return > > >>> >>> > to > > >>> >>> > latest supported CentOS release, will fix a lot of bugs / > security > > >>> >>> > issues > > >>> >>> > [4] and will make further updates easier. > > >>> >>> > > > >>> >>> > Risk of regression still exists, but it's quite low, 35 > successful > > >>> BVTs > > >>> >>> > can't be wrong. > > >>> >>> > > > >>> >>> > To finish that feature the following should be done: > > >>> >>> > * review and merge e1000 workaround [2] > > >>> >>> > * review and merge spec [0] > > >>> >>> > * review and merge request that switches build CI to > CentOS-7.2 [5] > > >>> >>> > > > >>> >>> > I expect the last day it could be done is March, 4. > > >>> >>> > > > >>> >>> > [0] https://review.openstack.org/#/c/280338/ > > >>> >>> > [1] https://bugs.launchpad.net/fuel/+bug/1526544 > > >>> >>> > [2] https://review.openstack.org/#/c/285306/ > > >>> >>> > [3] > > >>> https://etherpad.openstack.org/p/r.1c4cfee8185326d6922d6c9321404350 > > >>> >>> > [4] > > >>> https://etherpad.openstack.org/p/r.a7fe0b575d891ed81206765fa5be6630 > > >>> >>> > [5] https://review.fuel-infra.org/#/c/17400/ > > >>> >>> > > > >>> >>> > > > >>> >>> > -- > > >>> >>> > Thanks, > > >>> >>> > Dmitry Teselkin > > >>> >>> > Mirantis > > >>> >>> > http://www.mirantis.com > > >>> >>> > > > >>> >>> > > > >>> >>> > > > >>> > __________________________________________________________________________ > > >>> >>> > OpenStack Development Mailing List (not for usage questions) > > >>> >>> > Unsubscribe: > > >>> >>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > >>> >>> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > >>> >>> > > > >>> >>> > > >>> >>> > > >>> >>> > > >>> > __________________________________________________________________________ > > >>> >>> OpenStack Development Mailing List (not for usage questions) > > >>> >>> Unsubscribe: > > >>> >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > >>> >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > >>> >> > > >>> >> > > >>> >> > > >>> >> > > >>> >> -- > > >>> >> Thanks, > > >>> >> Dmitry Teselkin > > >>> >> Mirantis > > >>> >> http://www.mirantis.com > > >>> >> > > >>> >> > > >>> > __________________________________________________________________________ > > >>> >> OpenStack Development Mailing List (not for usage questions) > > >>> >> Unsubscribe: > > >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > >>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > >>> >> > > >>> > > > >>> > > > >>> > > > >>> > __________________________________________________________________________ > > >>> > OpenStack Development Mailing List (not for usage questions) > > >>> > Unsubscribe: > > >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > >>> > > > >>> > > >>> > > >>> > __________________________________________________________________________ > > >>> OpenStack Development Mailing List (not for usage questions) > > >>> Unsubscribe: > > >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > >>> > > >> > > >> > > >> > > >> -- > > >> Thanks, > > >> Dmitry Teselkin > > >> Mirantis > > >> http://www.mirantis.com > > >> > > >> > __________________________________________________________________________ > > >> OpenStack Development Mailing List (not for usage questions) > > >> Unsubscribe: > > >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > >> > > >> > > > > __________________________________________________________________________ > > > OpenStack Development Mailing List (not for usage questions) > > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > -- > > Mike Scherbakov > > #mihgen > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Mike Scherbakov #mihgen
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev