Another option is to just run separate test jobs. Like Django tests in one runner, and angular in another. As different panels are deprecated and removed, and we alter the scope of the tests, two jobs might act as a good stopgap.
Rob On 21 July 2016 at 01:58, Richard Jones <r1chardj0...@gmail.com<mailto:r1chardj0...@gmail.com>> wrote: On 21 July 2016 at 10:13, Timur Sufiev <tsuf...@mirantis.com<mailto:tsuf...@mirantis.com>> wrote: I'm not sure if (3) was the thing we agreed upon (and if it will help us to detect issues in Horizon) but the rest of options definitely make sense to me. I think that's reasonable - moving the tests to tempest will make it more difficult to run them locally. Also there is 5. Parallelize integration tests. Ah, yep, good catch. I'm not sure whether this will negatively impact our stability on some of the test nodes though :/ Richard On Wed, Jul 20, 2016 at 4:28 PM Richard Jones <r1chardj0...@gmail.com<mailto:r1chardj0...@gmail.com>> wrote: Hi folks, Sorry I didn't make the meeting this morning :-( We touched on testing at the mid-cycle and again it came up in the meeting this morning. We identified two issues: 1. Our integration test suite cannot grow to too many more tests because they take too long to run (and get auto-killed). We could increase the amount of time allowed for them, but we agreed that we shouldn't do this. 2. We don't have enough higher-level (integration leve) test coverage for our newer angular interfaces. These two are obviously at odds :-) What we agreed on, I think, is that we're going to: 1. Reduce the granularity of the integration tests - use them to broadly test whether an interface works at all when presented to a browser, 2. Replace many single interface tests with a fewer tests that poke at many interfaces (thus removing the significant per-test overhead) - even potentially having a single test that attempts to access every panel (as a guard against gross Javascript incompatibilities or configuration issues breaking panels), 3. Move the integration tests to tempest, and 4. Write some "django unit test suite" functional tests for our angular code that tests more than just the single units we currently test (ie. mock at a more remote level, checking that broader interactions work). Does this sound about right? Richard __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
