Re: Windows and --race tests are now gating
On 2016-07-11 11:57 AM, Mark Shuttleworth wrote: > On 11/07/16 09:26, Aaron Bentley wrote: >> But is this week really any different from any other week? Won't there >> always be something critical that has to land? > > I think it's fine to make it a plan that we will have Windows tests on > for 2.0 GA, but at the moment we're focused on all the breaking changes > to the APIs and those are not Windows-relevant. > >>> With the race tests, we got all of >>> those passing before turning on gating. We need to do the same for the >>> Windows >>> tests. We need to deactivate gating on Windows at this stage. >> But there are some tests that do pass under windows. By disabling all >> tests, we risk further regressions on Windows. Instead, you could >> disable the tests that don't pass (only when running under Windows, of >> course), and then work to fix them. > > Please do that now - leave the passing Windows tests in place and ask > the Windows teams to enable further tests which we then add to the gate. I have talked with Alexis and I'll work with her and the core team to get that done. Aaron signature.asc Description: OpenPGP digital signature -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Windows and --race tests are now gating
On 11/07/16 09:26, Aaron Bentley wrote: > But is this week really any different from any other week? Won't there > always be something critical that has to land? I think it's fine to make it a plan that we will have Windows tests on for 2.0 GA, but at the moment we're focused on all the breaking changes to the APIs and those are not Windows-relevant. >> With the race tests, we got all of >> those passing before turning on gating. We need to do the same for the >> Windows >> tests. We need to deactivate gating on Windows at this stage. > But there are some tests that do pass under windows. By disabling all > tests, we risk further regressions on Windows. Instead, you could > disable the tests that don't pass (only when running under Windows, of > course), and then work to fix them. Please do that now - leave the passing Windows tests in place and ask the Windows teams to enable further tests which we then add to the gate. Blocking all trunk landings with a chunk of known-failing tests is not good practice. Mark signature.asc Description: OpenPGP digital signature -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Windows and --race tests are now gating
On 2016-07-10 08:34 PM, Ian Booth wrote: > Turning on gating for Windows tests before all tests were passing is premature > and is now blocking us from landing critical fixes for beta12 that we need to > release this week for inclusion into xenial. Hey, this is why I pointed out that it would block all landings. And believe me, I double-checked that this was really what Torsten wanted. But is this week really any different from any other week? Won't there always be something critical that has to land? > With the race tests, we got all of > those passing before turning on gating. We need to do the same for the Windows > tests. We need to deactivate gating on Windows at this stage. But there are some tests that do pass under windows. By disabling all tests, we risk further regressions on Windows. Instead, you could disable the tests that don't pass (only when running under Windows, of course), and then work to fix them. That way, we have a passing test suite and a way to work incrementally on fixing the Windows testing instead of requiring a flag-day coordinated change. Aaron signature.asc Description: OpenPGP digital signature -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Re: Windows and --race tests are now gating
Turning on gating for Windows tests before all tests were passing is premature and is now blocking us from landing critical fixes for beta12 that we need to release this week for inclusion into xenial. With the race tests, we got all of those passing before turning on gating. We need to do the same for the Windows tests. We need to deactivate gating on Windows at this stage. Of course we need to fix the tests, but turning on gating before that is done is counterproductive given what we need to get done this week. On 08/07/16 03:26, Aaron Bentley wrote: > Hi Juju developers, > > As requested by juju-core, we have added --race and Windows unit tests > to gated landings. These tests are run concurrently, not sequentially, > but all must complete before code can be landed. > > As a practical matter, this means that landings are now impossible until > the Windows and --race tests can pass. > > The output from this initial version is a bit crude-- it will tell you > which tasks failed (e.g. "windows"), and you then need to look at the > corresponding logs under the Jenkins artifacts. We aim to improve this > soon. > > Aaron > > > -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
Windows and --race tests are now gating
Hi Juju developers, As requested by juju-core, we have added --race and Windows unit tests to gated landings. These tests are run concurrently, not sequentially, but all must complete before code can be landed. As a practical matter, this means that landings are now impossible until the Windows and --race tests can pass. The output from this initial version is a bit crude-- it will tell you which tasks failed (e.g. "windows"), and you then need to look at the corresponding logs under the Jenkins artifacts. We aim to improve this soon. Aaron signature.asc Description: OpenPGP digital signature -- Juju-dev mailing list Juju-dev@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
