On 12/29/2011 08:38 AM, Dor Laor wrote:
On 12/28/2011 07:21 PM, Avi Kivity wrote:
Would you add live migration testing to qemu-test? If yes, you're
duplicating some more. If not, you're not doing functional or coverage
tests for that functionality.
From the recent threads it looks to me that the 2 advantages of qemu-test over
kvm-autotest are:
1. python is not used within the guest
2. qemu-test is smaller and simpler
Except for the above 2, kvm autotest is superior.
Sorry, but that's non-sense.
qemu-test builds a custom guest, entirely from source. This makes it possible
to efficiently test non-native architectures. The tests are written for this
minimal environment (which is not straight forward to do). This tests will
never run on Windows as they are 100% tied to their environment.
autotest (let's be clear, there is no such thing as kvm autotest anymore) is a
framework for executing third party tests. It's got fancy magic to execute in
client mode vs. server mode. There is a suite of tests that are integrated in
autotest that exercise various types of virtualization functionality. This
suite of tests use a special config format to determine what they do.
Included in this, is the ability to create a guest from an ISO either using step
files or via a guest-specific mechanism (kickstart, etc.). The tests are
written to be mostly guest agnostic and are therefore written in Python in a
cross platform way.
There is essentially no overlap between the two and calling kvm autotest
superior is like calling the Sun superior to the Moon.
My main motivation to merge
qemu-test and kvm autotest is that I fear that qemu-test will be the only
requirement for committing stuff for qemu and developers will be excused from
their 'duty' by writing a 50 LOC shell script and assume they done w/ testing.
There is no requirement to write autotest tests to get things merged into QEMU.
That's is how things are today.
And I don't think there will ever a requirement to write anything that isn't
merged directly into qemu.git. I'm not going to hold up merging a feature until
another project merges something into their tree.
So unless we're going to merge KVM autotest into qemu.git, I don't think there's
every going to be a requirement to write a KVM autotest test in order to get
something merged into qemu.git.
But since autotest is a framework for running third-party tests, it seems to
make sense for qemu.git to have a test framework that autotest can execute.
And since what we call KVM autotest actually tests a heck of a lot more than
just QEMU, it makes sense for KVM autotest to continue to focus on full stack
testing where QEMU is but one of the many components that it tests.
In addition to that, the 50 LOC will be too basic and only provide value for
basic functionality tests and kvm autotest folks will need to rewrite all of the
matching content and beyond.
Those above 2 advantages can be solve:
1. python is not used within the guest
- One option is just to use the same shell code in kvm autotest w/o
no python code and use the same kernel/initramfs.
Yes, you can make a directory in kvm autotest that just imports qemu-test, but
what's the point of doing that? Isn't it better for this to live in qemu.git?
- Another way is to use a plain linux distro w/ python and boot it
into single user test mode and potentially use a S4 guest snapshot
or external snapshot to shorten its boot time.
You cannot easily create a "plain linux distro" for an ARM target. If you don't
believe me, add ARM guest support to KVM autotest and see how well it works out :-)
You'll get faster boot time and friendlier code.
Adding S4 resume for "simple tests" seems to be a bit odd to me...
2. qemu-test is smaller and simpler
kvm autotest will have a fast, out of the box mode especially
designed to answer your requirements.
It's mainly a matter of preparing a fastTestMode.py that will setup
all of the above in a similar way that today's
https://github.com/autotest/autotest/blob/master/client/tests/kvm/get_started.py
I hope you see from the above that this isn't just about having something new
that has fewer features and is therefore simpler for the time being.
The approach is fundamentally different.
Regards,
Anthony Liguori
