Hi, In the last weeks I am realizing more and more how hard FAI development without automated testing is.
When testing installs, the process, depending an what part you work on the process is - build fai - remove old fai - install new fai - maybe adjust config - make nfsroot - make base images (mutli-distribution) - activate installation for testclient with fai-chboot - restart testclient - wait until FAI finishes or you see an error, or see it hanging - if FAI finishes, log into the console - check /tmp/fai/fai.log (mainly searching for "ERR" which indicates software installation troubles which cause a lot harm mostly) - check /tmp/target/boot if there's a kernel image - check /tmp/target/boot/grub/menu.lst if there's an entry for the kernel image - reboot - check if machine boots up - check if specified services run and things work (often I only check if X runs for a gnome demo install, but that's enough in most cases, real problems appear on other places, mostly) I already talked about that long before, but now the "pain" is big enough, I plan to put some efforts in making installs automatically testable before getting further on stuff like class directory restructuring and further support for other Distributions. The key to automated testing is using virtual Hardware, so also the start and stop of Hardware can be checked by a script easily. With qemu, I already have parts of the things described above, mostly the build, package-install and run install part, not the test/checks part. But, with an ssh master-key, the mentioned checks should be easily possible. I am also starting to play with xen these days. I am a bit unsure now if I should finish the testing stuff I started with qemu to get stuff done, or forget about that and concentrate on xen and forget about qemu. I already know that xen is generally faster, but qemu doesn't have so many hardware problems/dependencies. Xen doesn't run nice and easy on all hardware, especially with unclean ACPI tables, while I saw none such troubles with qemu. Does anybody else has experiences with testing installs and their results automatically? How do you do it? Are there important checks I forgot? Henning
