Folks This is what we should discuss. Let's think of better testing coverage when we decide to switch to new tag. We MUST NOT skip a single bug which we fixed in downstream. So this means that for each bug fixed in downstream there should be a set of tests merged into our testing framework along with the fix. This should be an obligatory requirement for each change. This testing framework MUST test each change to review.fuel-infra.org or to fuel-library so that we can catch such issues immediately.
I would start with covering each bug with particular unit and/or puppet-noop test and then, when we have working data-driven testing, there should be a template of test for each environment. Again, Sergey and Ivan, their MUST be 0 (zero) commits skipped by downstream if they are not in upstream, unless it is 100% proven that this commit is not needed anymore. And this can be proven only either by this bugfix test passing against upstream or by SME's consensus on it. I strongly urge you not to skip such fixes silently and discuss them with Fuel Library core reviewers. On Tue, Oct 20, 2015 at 1:26 PM, Sergii Golovatiuk <sgolovat...@mirantis.com > wrote: > Hi, > > On Mon, Oct 19, 2015 at 10:46 PM, Sergey Kolekonov < > skoleko...@mirantis.com> wrote: > >> Hi, >> >> I agree with Ivan. Getting rid of forks and moving to puppet-librarian is >> complicated work and such problems are nearly unavoidable. It's hard to >> cover all possible corner cases with regular tests. >> > > This case shows the lack of integration tests. We do not validate if > config files for all services are exactly the same before and after > switching to librarian. We rely on deployment only. This way is not > accurate. > > > >> openstacklib module provides basic functionality for many OpenStack >> modules, so reverting it to Kilo code means breaking the whole Liberty >> deployment. >> Let's don't block development process and merge all lost fixes. >> > > The main problem is not breaking deployment but recurring regressions and > how to address them. > > > >> >> Thanks Matthew for reporting this issue. >> >> On Mon, Oct 19, 2015 at 10:10 PM, Ivan Berezovskiy < >> iberezovs...@mirantis.com> wrote: >> >>> Hi, >>> >>> First of all, I want to mention (I don't blame anyone), that two >>> patchsets in bug description >>> ([0], [1]) were not merged into upstream puppet-openstacklib module (and >>> commit >>> messages don't contain links to upstream review). I see only one >>> proposed patch [2] >>> from Dmitry Ilyin, which was abandoned at Sep 18. Now it's restored and >>> those issues should be fixed using it. >>> >>> Second, our patches (moving to librarian) were tested several times >>> under Fuel CI jobs, >>> on BVTs, smoke_neutron tests with Kilo and Liberty code. Unfortunately, >>> we didn't find >>> problems with deployment. >>> >>> Third, two weeks passed after merging of our patches for librarian, and >>> only now >>> we are speaking about regressions. >>> >>> Patch [2] covers missing two commits [0], [1], that's why I suggest to >>> get it merged >>> and then recheck issues, because it's very late for reverting. >>> >>> >>> [0] - https://review.openstack.org/#/c/219668/ >>> [1] - https://review.openstack.org/#/c/223676/ >>> [2] - https://review.openstack.org/#/c/220224/ >>> >>> 2015-10-19 20:59 GMT+03:00 Sergii Golovatiuk <sgolovat...@mirantis.com>: >>> >>>> Hi, >>>> >>>> The policy should be revert, IMHO. cherry-pick doesn't guarantee the >>>> consistency, so it will take more time... Also this way gives time to write >>>> tests to exclude the regression in future. >>>> >>>> >>>> -- >>>> Best regards, >>>> Sergii Golovatiuk, >>>> Skype #golserge >>>> IRC #holser >>>> >>>> On Mon, Oct 19, 2015 at 6:52 PM, Matthew Mosesohn < >>>> mmoses...@mirantis.com> wrote: >>>> >>>>> Hi Fuelers, >>>>> >>>>> It seems we have a regression on two critical bugs because of >>>>> switching Fuel to puppet-openstacklib: >>>>> https://bugs.launchpad.net/fuel/+bug/1507685 >>>>> >>>>> This regressed to patches that were in Fuel Library that addressed two >>>>> bugs marked as Critical. >>>>> >>>>> We should improve the acceptance criteria for moving to upstream >>>>> modules to ensure no bugs are regressed that relate to the particular >>>>> Puppet module being migrated. >>>>> >>>>> Secondly, what should our policy be? Revert the switch to upstream >>>>> module? Or just work on cherry-picking the appropriate fixes? >>>>> >>>>> Best Regards, >>>>> Matthew Mosesohn >>>>> >>>>> >>>>> __________________________________________________________________________ >>>>> OpenStack Development Mailing List (not for usage questions) >>>>> Unsubscribe: >>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>>> >>>> >>>> >>>> __________________________________________________________________________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> >>> >>> -- >>> Thanks, Ivan Berezovskiy >>> MOS Puppet Team Lead >>> at Mirantis <https://www.mirantis.com/> >>> >>> slack: iberezovskiy >>> skype: bouhforever >>> phone: + 7-960-343-42-46 >>> >>> >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> >> -- >> Regards, >> Sergey Kolekonov >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Yours Faithfully, Vladimir Kuklin, Fuel Library Tech Lead, Mirantis, Inc. +7 (495) 640-49-04 +7 (926) 702-39-68 Skype kuklinvv 35bk3, Vorontsovskaya Str. Moscow, Russia, www.mirantis.com <http://www.mirantis.ru/> www.mirantis.ru vkuk...@mirantis.com
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev