Because this has come up a few times during freeze, it's probably worth actively advertising the policy we've developed on dropping / skipping a Tempest test.
Tempest tests encode the behavior of the system (to some degree), which means that once we know the behavior of the system, if code in a core project can only land if we skip or drop a tempest test, that's clearly a behavior change. We want to be really deliberate about this, so in these situations we require: * an extremely clear commit message about why this is required (and why backwards compatible behavior is not an option) * a link in the commit message to the dependent review (you can just put the idempotent change id in there) * a +2 on the dependent review in the project by a core member The 3rd part is important, because incompatible behavior changes are something we want to make the core team for any given project is sure they want to do before we drop or skip a test and let those changes come into the project. This system isn't perfect, but it's trying to strike a balance. My hope is in the future we could implement cross project dependencies in zuul so that you could gather test results for a project change - assuming the tempest change was applied, that would prevent all these changes from being -1ed until the tempest change is landed. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net
signature.asc
Description: OpenPGP digital signature
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev