John
Thank you for discovering and working on this!
This sounds like an awful can of worms and should be addressed in a
dedicated manner.
I'll add it to the tech debt board and bug squad board with a reference
to your branch.
I am not sure that anyone on core has a capacity to tackle it right this
second but I am hoping that the bug squad can take it head-on soon \o/
Sincerely Yours,
Anastasia
On 17/03/16 19:43, John Meinel wrote:
> tl; dr: I need some help fixing our tests that are incorrectly not
> isolated from the real world.
>
> So...
>
> In investigating the bug with PatchValue and non pointer receivers, we
> realized that PatchValue calls AddCleanup. Which means that If you are
> doing:
> func ... SetUpSuite() {
> s.PatchValue(&foo, safeFoo)
> }
>
> That PatchValue gets cleaned up in the first TearDownTest().Which
> means that the isolation you thought you were adding to the *Suite* is
> actually isolated only to the first test that gets run.
>
> I have a branch of "juju/testing" that fixes it so AddCleanup can be
> called at any time. If it is called before SetUpTest(), then it adds a
> cleanup to the Suite stack, otherwise it adds it to the current Test
> stack.
>
> But that breaks about 100 tests that weren't as isolated as they
> thought they were.
>
> It also breaks a few tests that were Patching values before calling
> IsolationSuite.SetUpTest/Suite(), eg:
> SetUpTest() {
> PatchValue()
> Base.SetUpTest()
> }
>
> My branch that starts working on this is at:
> g...@github.com:jameinel/juju unsafe-cleanups
>
> One of the big examples of bad tests is "provider/ec2". 52 tests fail
> because they either try to read from cloud-images.ubunt.com
> <http://cloud-images.ubunt.com> directly, or because they try to read
> from test:/streams/v1/index.sjson but don't accept the signature on
> those files.
>
>
> Is anyone able to help me tackle cleaning up our test suite?
>
> Thanks,
> John
> =:->
>
>
>
--
Juju-dev mailing list
Juju-dev@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/juju-dev
