Hi All,
This is a proposal to abstract all the setups, test run,
post test validation, cleanup of the tests from tp-libvirt/tp-qemu
into avocado-vt so that we have one unique testcase source code
and existing test config can grow based on the new scenarios/variants.
>From testconfig we could pass following params,
`test_class` - Actual functionality test
`pre_events` - integrate any other events or test before actual test
`post_events`- integrate any other events or test after actual test
Test case file source can be like below common for all the tests,
def run(test, params, env):
"""
Test KVM scenarios
"""
try:
test_class = eval(params.get("test_class"))
pre_events = params.get("test_pre_events").split()
post_events = params.get("test_post_events").split()
vms = env.get_all_vms()
try:
for each in pre_events:
preevent = eval(each)
preevent_obj = preevent(vms, params)
preevent_obj.run()
test_obj = test_class(vms, params)
test_obj.run()
for each in post_events:
postevent = eval(each)
postevent_obj = postevent(vms, params)
postevent_obj.run()
except Exception as info:
test.fail(info)
finally:
if preevent_obj:
preevent_obj.cleanup()
if test_obj:
test_obj.cleanup()
if postevent:
postevent_obj.cleanup()
except NameError as info:
test.error(info)
Avocado-VT should have class for each functionality test or events
class TestClass(object):
def __init__(self, vms, params):
self.vms = vms
self.params = params
# perform pre test setup
def setup(self):
def run(self):
self.setup()
# code that performs actual test
self.validation
# perform post test validation
def validation(self):
def cleanup():
As example we can take migration test,
One of the test performs CPU hotplug before migration, perform migration
and CPU hotunplug after migration
* preevent will be CPU hotplug Test
* test_class will be Migration Test
* postevent will be CPU hotunplug Test
We can control the preevent or postevent from params,
1. require it or not (migration with/without any events)
2. require same no of multiple events before/after test (hotplug event
before migration and hotunplug event after migration)
3. require variable no of multiple events before/after test (hotplug
event, stress event before and hotunplug event after migration)
By this design,
* Single testcase source for all the functional tests.
* All the functional code abstracted to Avocado-VT into its own class
for each features respectively.
* Avoids all the duplicate named params for every test introduced and
similar logics that reduces the code bloat in Autotest and Avocado-VT.
* Make almost all the attributes configurable from params.
* New testcases can be generated dynamically from avocado commandline by
giving different param values using --vt-extra-params,
Example: we can change no of cpus to be hotplugged/hotunplugged while
triggering the test from avocado command line instead of writing new
testcase for it.
Initially it would take effort to refactor Avocado-VT to adopt it but,
I think it will greatly change the way we can test going forward and
make the framework more simple, powerful and robost.
Please share your thoughts on it, I can make a working model based on
the feedback and comments on the proposal.
-- Bala
