On 23 January 2014 09:39, Sean Dague <s...@dague.net> wrote: > ================================ > Changes coming in gate structure > ================================
> Svelt Gate > ========== > > The gate jobs will be trimmed down immensely. Nothing project > specific, so pep8 / unit tests all ripped out, no functional test > runs. Less overall configs. Exactly how minimal we'll figure out as we > decide what we can live without. The floor for this would be > devstack-tempest-full and grenade. > > This is basically sanity check that the combination of patches in > flight doesn't ruin the world for everyone. So two things occur to me here - - this increases thread-the-needle risks. - what value does the sanity check still offer? To illustrate this, consider that we have one patch for consideration. We'll run it in check, then in gate, then it lands. If a second patch is submitted, it will run in isolation in check, so we have a cross-p*atch* race - e.g. a change(1) to nova won't be tested with nova(2) until the gate run runs. If the change is only applicable to one configured scenario, that scenario has to be in the gate, or its not tested at all and we find out that the thing is broken when *every nova check run from that point on breaks* - instant firedrill. The risk here seems to be roughly P(broken-in-combination) * patches-not-tested-in-combination : e.g. if checks and gate are both 0.5h long [under estimated round number for ease of calculations] then from clicking approve we have one hour until the change is in trunk and we're immune to interactions being undetected. If we're pushing 500 through a day at peak, then in that interval 20 patches could be submitted on average. So for anything we take out of the gate and make check only our risk of cross-patch needle getting through is up to twenty times higher than it is of such a thing breaking the gate today. OTOH as you say, our gate breaks aren't typically in e.g. unit tests. For things that are in the gate, obviously it's business as usual. So - I'm pretty worried that we're going to have more firedrills with this, in things like unittests and functional tests, even while tempest scenarios are kept running. -Rob -- Robert Collins <rbtcoll...@hp.com> Distinguished Technologist HP Converged Cloud _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev