On 06/18/2013 01:17 PM, Jeremy Stanley wrote: > On 2013-06-18 18:43:58 +0200 (+0200), Martina Kollarova wrote: > [...] >> I propose that it should fail and stop all of the other tests once >> there is a failure in a voting test. For non-voting tests, it should >> stop only itself, not the others. > [...] > > We did this for a while (ran pep8 first in the check pipeline, then > ran other jobs if that passed), but it turned out to have a rather > confusing interaction with the requirements validation jobs. > > The long and short of it is that there are roughly two classes of > jobs we want to consider... quick ones which are not very resource > intensive, and more involved tests which are unlikely to succeed > anyway if the quick ones fail. To that end there has been discussion > of adding a "fast-fail" option for certain jobs causing them to > abort others already running and return the result set right away so > that developers can benefit from a more immediate testing feedback > loop. > > I don't think we have a timeline for implementation there, but it's > my recollection that's the direction we resolved to take it.
Yeah. What Jeremy said. Actually, fast-fail is one of the reasons that we've been pushing so hard to get people on to testr and thus have the option for the tests to emit subunit output streams. Our current thinking is that we will teach zuul to read subunit streams as they happen, so that fast-fail can report back overall failure and also re-order the gate queue as needed because we know a job will fail - but we can still continue running the jobs. Note, that since this involves us reading a streaming protocol, it doesn't depend on a job failing first. We should be able to report failure for the set of jobs on the first instance of a single test case failing. As of right now, we have not begun development on this feature. I believe we'd be more than happy to point someone in the right direction if they wanted to work on it before we can get to it. Monty _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev