On 08/05/2016 12:58 PM, Steven Hardy wrote: > On Thu, Aug 04, 2016 at 09:46:20PM -0400, Emilien Macchi wrote: >> Hi, >> >> I'm currently working by iteration to get a new upstream job that test >> upgrades and update. >> Until now, I'm doing baby steps. I bootstrapped the work to upgrade >> undercloud, see https> ://review.openstack.org/#/c/346995/ for details >> (it's almost working hitting a packaging issue now). >> >> Now I am interested by having 2 overcloud jobs: >> >> - update: Newton -> Newton: basically, we already have it with >> gate-tripleo-ci-centos-7-ovb-upgrades - but my proposal is to use >> multinode work that James started. >> I have a PoC (2 lines of code): >> https://review.openstack.org/#/c/351330/1 that works, it deploys an >> overcloud using packaging, applies the patch in THT and run overcloud >> update. I tested it and it works fine, (I tried to break Keystone). >> Right now the job name is >> gate-tripleo-ci-centos-7-nonha-multinode-upgrades-nv because I took >> example from the existing ovb job that does the exact same thing. >> I propose to rename it to >> gate-tripleo-ci-centos-7-nonha-multinode-updates-nv. What do you >> think? > > This sounds good, and it seems to be a valid replacement for the old > "upgrades" job - it won't catch all kinds of update bugs (in particular it > obviously won't run any packaged based updates at all), but it will catch > the most serious template regressions, which will be useful coverage to > maintain I think. > >> - upgrade: Mitaka -> Newton: I haven't started anything yet but the >> idea is to test the upgrade from stable to master, using multinode job >> now (not ovb). >> I can prototype something but I would like to hear from our community before. > > I think getting this coverage in place is very important, we're > experiencing a lot of post-release pain due to the lack of this coverage, > so +1 on any steps we can take to get some coverage here, I'd say go ahead > and do the prototype if you have time to do it. > > You may want to chat with weshay, as I know there are some RDO upgrade > tests which were planned to be run as third-party jobs to get some upgrade > coverage - I'm not sure if there is any scope for reuse here, or if it will > be easier to just wire in the upgrade via our current scripts (obviously > some form of reuse would be good if possible). > >> Please give some feedback if you are interested by this work and I >> will spend some time during the next weeks on $topic. >> >> Note: please also look my thread about undercloud upgrade job, I need >> your feedback too. > > My only question about undercloud upgrades is whether we might combine the > overcloud upgrade job with this, e.g upgrade undercloud, then updgrade > overcloud. Probably the blocker here will be the gate timeout I guess, > even if we're using pre-cached images etc.
Yeah, we'd probably have to cut a bunch of runtime off somewhere. Just the undercloud upgrade alone starting from a pre-built Mitaka image (which we might be able to do in CI, but it would be a little tricky and I'm not positive it would work) was taking 20-25 minutes when I was running it a while ago. It's probably even slower now. Add that to the already long time the overcloud part takes and I don't see how it would fit in under the timeout. This is why I was running a job locally instead of adding it to tripleo-ci (with the intent that some day I would figure out how to move it into upstream infra like Emilien did :-). > > Thanks for looking into this! > > Steve > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
