Re: debci / salsa ci: support for qemu runner
On 7/25/23 19:49, Johannes Schauer Marin Rodrigues wrote: having kvm available in debci and salsaci would be a big plus. The issue with nested virt, is that they prevent live-migration of VMs (Qemu doesn't support live-migrating nested MMU pages...). There's ongoing work to allow this in upstream Qemu, but as far as I know, this isn't ready yet. Once we have the feature in, we'll go from a "mostly unsupported" in public clouds to "mostly, everyone has it". Until then, I would recommend against using nested-virt whenever possible. Cheers, Thomas Goirand (zigo)
Re: debci / salsa ci: support for qemu runner
On Sat, Jul 29, 2023 at 06:02:26AM +0200, Helmut Grohne wrote: > Hi, > > On Tue, Jul 25, 2023 at 09:37:35PM +0200, Paul Gevers wrote: > > For ci.d.n, the issue is not money, but the required work to integrate it > > into the infrastructure. We need volunteers (or pay people to do the work), > > but unless they can and want to figure out everything from source [1], the > > bottleneck remains that the current volunteers would need to help those > > people understand the setup and guide them coming up with good solutions. > > I second this on another level. While the lxc backend is exercised very > often, the qemu backend evidently experiences rare use. The default > --ram-size is 1G and that happens to be too little for a number of > packages already. This soon will be configurable (#1037245). I expect > that there are more aspects where qemu and lxc differ in a way that > causes test failures as most of the existing tests only ever ran on lxc. > Some of these aspects will have to be fixed in tests, but others (like > the --ram-size) will need addressing in infrastructure. Please expect > more work in this area. FWIW, the debci codebase has gained support for specifying a backend on a per-package basis, so we will be able to switch packages one by one, and only the ones that really need (and/or benefit from) running on QEMU. signature.asc Description: PGP signature
Re: debci / salsa ci: support for qemu runner
Hi, On Tue, Jul 25, 2023 at 09:37:35PM +0200, Paul Gevers wrote: > For ci.d.n, the issue is not money, but the required work to integrate it > into the infrastructure. We need volunteers (or pay people to do the work), > but unless they can and want to figure out everything from source [1], the > bottleneck remains that the current volunteers would need to help those > people understand the setup and guide them coming up with good solutions. I second this on another level. While the lxc backend is exercised very often, the qemu backend evidently experiences rare use. The default --ram-size is 1G and that happens to be too little for a number of packages already. This soon will be configurable (#1037245). I expect that there are more aspects where qemu and lxc differ in a way that causes test failures as most of the existing tests only ever ran on lxc. Some of these aspects will have to be fixed in tests, but others (like the --ram-size) will need addressing in infrastructure. Please expect more work in this area. Helmut
Re: debci / salsa ci: support for qemu runner
Hi, On 25-07-2023 16:16, Michael Biebl wrote: apparently, we in Debian struggle to find good opportunities where to spend our money. For ci.d.n, the issue is not money, but the required work to integrate it into the infrastructure. We need volunteers (or pay people to do the work), but unless they can and want to figure out everything from source [1], the bottleneck remains that the current volunteers would need to help those people understand the setup and guide them coming up with good solutions. I think support for qemu runners, i.e. supporting isolation-machine in autopkgtest on both debci and salsa ci would be an excellent opportunity. Please note that we wouldn't need this (initially) on all architectures. As long as tests run consistently, some architectures could support it and some not. Even more tuned, if we only had some workers on some architectures supporting this, we could list (manually or better) the tests that would need to be tested there explicitly somehow. All this doesn't exist yet. One step that has recently been made in debci, thanks Antonio for maintaining that, is that workers nodes should now be able to support multiple backends. No publicly available work has been done yet to utilize that. Paul [1] https://salsa.debian.org/ci-team/debian-ci-config/ OpenPGP_signature Description: OpenPGP digital signature
Re: debci / salsa ci: support for qemu runner
Am 25.07.23 um 19:49 schrieb Johannes Schauer Marin Rodrigues: Hi, Quoting Michael Biebl (2023-07-25 16:16:35) apparently, we in Debian struggle to find good opportunities where to spend our money. I think support for qemu runners, i.e. supporting isolation-machine in autopkgtest on both debci and salsa ci would be an excellent opportunity. is this just about isolation-machine or is this about architectures where qemu support exists but no real hardware? It's about having support for isolation-machine OpenPGP_signature Description: OpenPGP digital signature
Re: debci / salsa ci: support for qemu runner
El 25/07/23 a las 19:49, Johannes Schauer Marin Rodrigues escribió: > Hi, > > Quoting Michael Biebl (2023-07-25 16:16:35) > > apparently, we in Debian struggle to find good opportunities where to spend > > our money. > > > > I think support for qemu runners, i.e. supporting isolation-machine in > > autopkgtest on both debci and salsa ci would be an excellent opportunity. > > is this just about isolation-machine or is this about architectures where qemu > support exists but no real hardware? > > If it's about isolation-machine I have "solved" this problem in the past by > creating a qemu virtual machine as part of the autopkgtest and then run my > test > inside that. This is slow because kvm is not available but it works for small > tests. > > As far as I'm concerned, having kvm available in debci and salsaci would be a > big plus. > > No idea if that is easier or harder than what Michael proposed. FTR: https://salsa.debian.org/salsa/support/-/issues/345 For Salsa CI, AFAIU, this means Salsa Admins would need to use for the gitlab-runner another type of google machine supporting nested-virtualisation (such as N1 or N2, instead of E1). I cannot say if that is possible/feasible. Thanks to the Salsa Admins for their work, BTW! I can talk for some of the debci *ARM* runners, and the current setup doesn't support nested virtualisation either. Cheers, -- Santiago signature.asc Description: PGP signature
Re: debci / salsa ci: support for qemu runner
Hi, Quoting Michael Biebl (2023-07-25 16:16:35) > apparently, we in Debian struggle to find good opportunities where to spend > our money. > > I think support for qemu runners, i.e. supporting isolation-machine in > autopkgtest on both debci and salsa ci would be an excellent opportunity. is this just about isolation-machine or is this about architectures where qemu support exists but no real hardware? If it's about isolation-machine I have "solved" this problem in the past by creating a qemu virtual machine as part of the autopkgtest and then run my test inside that. This is slow because kvm is not available but it works for small tests. As far as I'm concerned, having kvm available in debci and salsaci would be a big plus. No idea if that is easier or harder than what Michael proposed. Thanks! cheers, josch signature.asc Description: signature
