Robert Collins <robe...@robertcollins.net> writes: > On 23 January 2014 09:39, Sean Dague <s...@dague.net> wrote: >> ================================ >> Changes coming in gate structure >> ================================ > >> Svelt Gate >> ========== >> >> The gate jobs will be trimmed down immensely. Nothing project >> specific, so pep8 / unit tests all ripped out, no functional test >> runs. Less overall configs. Exactly how minimal we'll figure out as we >> decide what we can live without. The floor for this would be >> devstack-tempest-full and grenade. >> >> This is basically sanity check that the combination of patches in >> flight doesn't ruin the world for everyone. > > So two things occur to me here - > - this increases thread-the-needle risks. > - what value does the sanity check still offer?
Here's how I see this: we have a process that ensures that the code remains as perfect as its tests. We have seen that the combined system of openstack-and-its-tests is not perfect. The complexity of that system is great, and it seems we will likely never remove all non-deterministic failures from this system. At least, we seem to be able to introduce them as fast as they are removed. So this is an adaptation to two aspects of our current situation: single-project-test failures are causing gate failures, and the non-deterministic failure rate in cross-project-tests are as well. The first is addressed by moving single-project tests out of the gate. So a single project is more likely to break itself but less likely to break others. This is a trade-off, but it localizes the pain in case of error. The second is addressed by removing some of our cross-project-test variants from the gate. The more variants we run, the more likely we are to hit non-deterministic bugs. You have argued before that rate is not high enough for us to prevent these bugs from entering, but perhaps with the reduction it will be low enough that it allows us to get work done. It's possible both of those decisions may let in more bugs than we are comfortable with. However, given the current state where cross-project frustrations are high and development has slowed dramatically, localizing failures when possible and focusing the gate on ensuring that there's at least a basic workable system seems worth trying. If we take too much out of the gate, we can put it back in. Regardless, I think the changes to require recent check votes will help and be a long-lasting improvement. -Jim _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev