Hi Ian, On Sat, Sep 23, 2023 at 11:19:27AM +0100, Ian Jackson wrote: > To summarise that discussion: at that time the best available solution > that worked in ci.d.n seemed to be to write an ad-hoc script to run > the tests in qemu; three packes had done that, each separately, with > complex scripts with many moving parts.
In principle, debvm is supposed to target that particular use case. There are two limitations that currently make this infeasible. > I saw debvm, and wondered if it was suitable for this purpose. > But, then I looked at its debian/test/control and I see that the tests > are marked as flaky.[2] So maybe it isn't reliable enough :-/. The reliability of tests is ok. The reason for marking them flaky is that they currently test the "wrong" packages. ci.d.n sets up chroots in a delicate way to combine particular packages to see which combinations cause breakage. Then debvm just creates an unstable system and tests that. In effect, it currently tests unstable (inside those virtual machines) rather than what it is supposed to be testing. Johannes solved this problem on the mmdebstrap side and mmdebstrap's tests no longer are flaky in this way. Therefore this should be solvable on the debvm side. I just haven't gotten do figuring out the right runes thus far. Roughly speaking, the hosts' apt configuration, pinning and sources.lists should be used inside the created virtual machine. > I have other questions too, particularly to do with the way I would > need autopkgtest to be able to influence package selection in the > nested testbeds. Exactly. That's currently the missing piece to remove the flakiness annotation. There is another practical problem. None of the autopkgtest nodes support kvm. Emulation will always use tcg. For one thing, tcg is slow. It can be so slow on some architectures that RCU becomes unhappy as its grace periods become too long. For another, tcg is buggy. It has emulation bugs even on release architectures that make some expected functionality fail. For instance, gdb reliably segfaults when run in s390x tcg emulation. That kvm aspect kinda seems like an unresolvable blocker. While most autopkgtest machines are physical machines, they use kvm for running the actual autopkgtest nodes and then lxc for individual test isolation. We'd have to use nested kvm here and somehow get it through lxc. > Everyone else: has there been any other progress on the multi-node > autopkgtest problem ? Disregarding these two aspects, debvm should get you quite far. You probably need to take network management into your own hands. I expect that vde2 would be a good way to implement this in an unprivileged way. Your debvm invocations may become a little longer that way, but that should be fine given that you store all of that in scripts. Helmut
