On 4/21/2017 8:36 AM, Ferenc Horváth wrote:
Dear OpenStackerz,
I'd like to improve the coverage of the current test suite over some
special code parts in Nova.
My main target is to add a few scenarios [1] that would exercise the
AggregateMultiTenancyIsolation scheduler filter.
I'm also planning on adding new test cases for other filters and for
some libvirt related features [2] as well.
Unfortunately, [1] and [2] could not be merged into Tempest for various
reasons, hence I started working on functional tests in Nova.
However, since functional tests cannot be used to verify that a deployed
system behaves correctly, we still need end to end tests.
Therefore, I'm proposing a new Tempest test plug-in [3,4] that would be
the home of currently out of tree tests.
The idea is that these tests would run separately on a weekly basis to
not use too much resources, but the rest of the questions are still open.
Therefore, I'd appreciate any advice or review on this topic.
Thank You all in advance.
Best regards,
Ferenc Horváth
[1] https://review.openstack.org/#/c/374887/
[2] https://review.openstack.org/#/c/315786/
[3] https://review.openstack.org/#/c/448482/
[4] https://review.openstack.org/#/c/451227/
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
As discussed in the nova meeting today [1] I think some of this could
make sense as a Tempest plugin, or an in-tree functional test using
fixtures, or an in-tree dsvm integration style job that's not Tempest
but runs real tests against a live devstack configuration.
For the scheduler filter testing, I think that's something that is
totally doable with the in-tree functional tests using our fixtures. You
don't need devstack for that since it's just testing the logic of the
scheduler filters. We have running services, database, and a live API in
those tests, and external services like cinder/glance/neutron are
stubbed out. An example of a test like this is here [2]. I understand
you have some internal requirements for how these tests are performed
though, so I can't really help you there. Keep in mind if you do it with
an in-tree functional test, it gets tested on every change and is
voting, whereas a periodic job is not and you only find out it's broken
after we break it.
For testing the libvirt watchdog action we obviously need a real
deployment with running libvirt. I think you could do that as a tempest
plugin or as an in-tree dsvm integration job, much like how the
novaclient functional tests work (those aren't tempest but they run
against a live devstack and execute real APIs via the nova CLI). The
downside is we don't have any dsvm-integration infrastructure setup in
Nova today, so you'd have to blaze that trail. But we've talked about
this as a need for a long time, so it'd be great if someone worked on
it. Alternatively it could be an in-tree Tempest plugin...but then we
can't test things like the libvirt image cache which is something we
could do with a dsvm-integration job I think, or the evacuate API (we'd
have to run that in serial so it doesn't break other concurrently
running tests). Actually testing evacuate would be awesome though.
In general I feel like writing a CI job for a very specific scheduler
filter configuration is overdoing it, unless you also made that job
re-configurable on the fly, like how the live migration job works [3].
[1]
http://eavesdrop.openstack.org/meetings/nova/2017/nova.2017-04-27-21.00.log.html#l-111
[2]
https://github.com/openstack/nova/blob/master/nova/tests/functional/regressions/test_bug_1671648.py
[3]
https://github.com/openstack/nova/blob/master/nova/tests/live_migration/hooks/run_tests.sh
--
Thanks,
Matt
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
