On 10/25/2011 01:03 PM, Eduardo Habkost wrote:
On Tue, Oct 25, 2011 at 03:27:35PM +0200, Gerd Hoffmann wrote:
[...]
A while ago I played with some simple IDE tests. It basically was a
small x86 kernel with an empty image that sends IDE commands and prints
some results, and a script that invokes the guest and checks whether the
test has passed or failed.
That reminds me that I've started toying with running tests inside a
guest too. Stopped working on it a while back due to other priorities.
Attached what I have so far.
So at first I started with my own multiboot kernel and copied over some
parts of kvm-unittest's libc. Clearly not the best idea once it's more
than a couple of lines, so at some point I took the code and integrated
with my real kvm-unittests repository.
Now I don't have to duplicate code any more, but at the same time
there's no chance that a 'make check' in an upstream qemu tree could run
this. Tests for other devices will have exactly the same problem.
Any suggestions on how to go forward with this kind of tests? Should
this go into qemu or into kvm-unittests? Or should kvm-unittests be
merged into the qemu tree? Or is the approach completely wrong?
I think we should have some framework to run tests inside the guest in
the qemu source tree. I'm not sure kvm-unittests is the right tool for
the job though. It is quite low-level and mainly targets the kvm bits
inside the linux kernel. Testing -- for example -- usb device emulation
would pretty much require writing a usb stack for kvm-unitests ...
We have a framework to run tests inside a fully-installed guest, that's
KVM-Autotest. But maybe it's too much for the kind of tests you need, I
don't know. There are different "levels" of testing, with different
reequirements, and we need to have good tools for all levels.
Just trying to enumerate the kind of tests somebody may need:
A) Simple unit tests for internal qemu C functions
- 'make check' can run them, using either libcheck or gtest.
B) Functional tests that tests actual virtualization/emulation, but only
of some specific subsystems, not using a fully-featured qemu process.
- We don't have anything like that, today, right? I am not sure we
need it.
C) Functional tests that just need to run a small binary with no OS
installed in the guest, but running a fully-feature qemu process.
- The tests in the 'tests' directory do this, right? kvm-unittests
does this, right?
D) Functional tests that need a minimal OS installed, with, e.g., at
least a Linux kernel and a shell.
- This is what Gerd's patch below does, right? Also, KVM-Autotest can
be used for this.
Yes, it can. There's instrumentation to get ssh sessions, serial
sessions, execute monitor commands... Only need to specify the path to a
pre-installed minimal guest, skip install tests and have the bare
minimal subset of tests needed. If only a small subset of tests are run
with a minimal image, a kvm autotest job is quick to run and suitable
for development use. I've looked at Gerd's initial patch to add an
initramfs for such lightweight testing, and the init ramdisk generation
could be implemented in autotest.
However...
Problem today is that KVM autotest is considered to be hard to approach
and integrate with qemu developer's workflow. I won't argue it's not, so
I'm admitting failure here. We've been juggling to keep it stable for
the regular, full blown QE case, with expanding test and targets
coverage (see libvirt test), and yet trying to make it better documented
and sufficiently easy to approach to do simpler ad hoc testing. I wish
it was easier to accomplish the later, though.
So, an in-tree solution for D) like the one Gerd started to work on
seems better, as it won't generate frustration from folks having to
learn kvm autotest, which will ultimately lead to them ignoring the effort.
E) Functional tests that need a full OS installed and configured.
- Today we use KVM-Autotest for this.
Does the above model look correct/complete, or is there some case I
missed?
cheers,
Gerd
From 096f68ea08c3c4baf1bbdc549b257a67ecc87e25 Mon Sep 17 00:00:00 2001
From: Gerd Hoffmann<kra...@redhat.com>
Date: Tue, 13 Sep 2011 17:38:37 +0200
Subject: [PATCH] initramfs test framework
Signed-off-by: Gerd Hoffmann<kra...@redhat.com>
---
initramfs/.gitignore | 3 +
initramfs/10-qemu-udev.rules | 5 +
initramfs/Makefile | 36 +++++++
initramfs/README | 44 ++++++++
initramfs/init.c | 225 ++++++++++++++++++++++++++++++++++++++++++
initramfs/initramfs-boot | 32 ++++++
initramfs/initramfs-create | 111 +++++++++++++++++++++
initramfs/test-ehci | 3 +
initramfs/test-ehci.good | 8 ++
initramfs/test-hello.c | 7 ++
initramfs/test-hello.good | 1 +
initramfs/test-uhci | 3 +
initramfs/test-uhci.good | 3 +
13 files changed, 481 insertions(+), 0 deletions(-)
create mode 100644 initramfs/.gitignore
create mode 100644 initramfs/10-qemu-udev.rules
create mode 100644 initramfs/Makefile
create mode 100644 initramfs/README
create mode 100644 initramfs/init.c
create mode 100755 initramfs/initramfs-boot
create mode 100755 initramfs/initramfs-create
create mode 100755 initramfs/test-ehci
create mode 100644 initramfs/test-ehci.good
create mode 100644 initramfs/test-hello.c
create mode 100644 initramfs/test-hello.good
create mode 100755 initramfs/test-uhci
create mode 100644 initramfs/test-uhci.good