Re: [libvirt] UID/GID during kvm/qemu migrate
On Tue, 30 Jul 2019 14:00:48 +0100 Daniel P. Berrangé wrote: > On Tue, Jul 30, 2019 at 02:49:02PM +0200, Stephan von Krawczynski wrote: > > On Tue, 30 Jul 2019 11:35:45 +0100 > > Daniel P. Berrangé wrote: > > > > > On Mon, Jul 29, 2019 at 02:04:14PM +0200, Stephan von Krawczynski wrote: > > > > > > > Hello, > > > > > > > > is there some immanent code in libvirt that forces UID/GID of the > > > > libvirt standard user to be the same on two boxes migrating qemu vms > > > > against each other? > > > > The migration itself uses root obviously (password is requested). But > > > > if a vm xml does not contain any definition regarding UID/GID what > > > > else could prevent this from working? > > > > > > > > I believe I ran into such a problem trying to migrate and ending up in > > > > an error, a vm still working on original host but its fs (netfs pool > > > > (nfs/raw)) being switched to read-only... > > > > > > When migrating a VM whose image is hosted on NFS, you have 2 QEMU > > > processes which both need to be able to open the same image file at the > > > same time. QEMU runs as an unprivileged user normally, and so the disk > > > images get chowned to this unprivileged user by libvirt when QEMU is > > > started. If the QEMU on the target host is given a UID/GID that's > > > different from the QEMU on the source host, then the target QEMU will > > > likely have problems opening the image. > > > > > > Basically when using shared FS storage, the rule is to have all your > > > hosts configured in the same way from libvirt's POV. > > > > > > Regards, > > > Daniel > > > > Hello Daniel, > > > > thank you for the short explanation. The key words are "both need to be > > able to open the same image file at the same time". > > I would not have expected that. I thought qemu 1 will close and exit, and > > then qemu 2 will open the image, which means he can change the uid/gid > > right before > > This is supposed to be a safety thing. If anything goes badly wrong on the > target host you need to be able to rollback & continue running on the source > VMs. So the source VM doesn't want to close the disk images, until the target > VM has confirmed it is successfully running. This implies there is a period > of time when both have to have the disk image open. Crucially though, even > when 2 QEMUs have the disks open, only *1* QEMU has the guest CPU running > and permitting disk writes. > > > - just as in normal operation. > > Is this the reason why my failing try leaves me with a read-only fs on the > > guest? Which I would see as a bug, not? > > Turning it read-only is possibly the only way to not corrupt the fs image > > if two qemus have it open simultaneously. > > The guest sets its FS read-only when it gets an I/O error reported by > the virtual disk driver. QEMU reports an I/O error to the guest, when > it in turns gets an I/O error on the host storage. This happens because > qemu is loosing privileges to access the disks. > > > Regards, > Daniel Ok, the source VM does not want to close the image until confirmed that the destination VM is up and running. But if it is not up and running and the source VM still cannot go on - because its fs is read-only - then what's the use of keeping it open in the first place? The simple fact is: a reproducably failing migration leaves you with a non-working guest. I cannot think of any argument supporting the idea this being no bug. I mean the whole story of migration is only done for keeping up the guest... Else you could as well copy the config with the guest offline. And: the source guest VM has no I/O error. The destination guest VM cannot touch the fs image for permission reason, so the fs is still safe in the state the frozen source guest left it. -- Regards, Stephan -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] UID/GID during kvm/qemu migrate
On Tue, 30 Jul 2019 11:35:45 +0100 Daniel P. Berrangé wrote: > On Mon, Jul 29, 2019 at 02:04:14PM +0200, Stephan von Krawczynski wrote: > > Hello, > > > > is there some immanent code in libvirt that forces UID/GID of the libvirt > > standard user to be the same on two boxes migrating qemu vms against each > > other? > > The migration itself uses root obviously (password is requested). But if a > > vm xml does not contain any definition regarding UID/GID what else could > > prevent this from working? > > > > I believe I ran into such a problem trying to migrate and ending up in an > > error, a vm still working on original host but its fs (netfs pool > > (nfs/raw)) being switched to read-only... > > When migrating a VM whose image is hosted on NFS, you have 2 QEMU processes > which both need to be able to open the same image file at the same time. > QEMU runs as an unprivileged user normally, and so the disk images get > chowned to this unprivileged user by libvirt when QEMU is started. If the > QEMU on the target host is given a UID/GID that's different from the QEMU > on the source host, then the target QEMU will likely have problems opening > the image. > > Basically when using shared FS storage, the rule is to have all your hosts > configured in the same way from libvirt's POV. > > Regards, > Daniel Hello Daniel, thank you for the short explanation. The key words are "both need to be able to open the same image file at the same time". I would not have expected that. I thought qemu 1 will close and exit, and then qemu 2 will open the image, which means he can change the uid/gid right before - just as in normal operation. Is this the reason why my failing try leaves me with a read-only fs on the guest? Which I would see as a bug, not? Turning it read-only is possibly the only way to not corrupt the fs image if two qemus have it open simultaneously. -- Regards, Stephan -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] UID/GID during kvm/qemu migrate
Hello, is there some immanent code in libvirt that forces UID/GID of the libvirt standard user to be the same on two boxes migrating qemu vms against each other? The migration itself uses root obviously (password is requested). But if a vm xml does not contain any definition regarding UID/GID what else could prevent this from working? I believe I ran into such a problem trying to migrate and ending up in an error, a vm still working on original host but its fs (netfs pool (nfs/raw)) being switched to read-only... -- Regards, Stephan -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] UID/GID during kvm/qemu migrate
Hello, is there some immanent code in libvirt that forces UID/GID of the libvirt standard user to be the same on two boxes migrating qemu vms against each other? The migration itself uses root obviously (password is requested). But if a vm xml does not contain any definition regarding UID/GID what else could prevent this from working? I believe I ran into such a problem trying to migrate and ending up in an error, a vm still working on original host but its fs (netfs pool (nfs/raw)) being switched to read-only... -- Regards, Stephan -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Problem configuring selective dropping of root (SOLVED)
On Thu, 11 Jul 2019 08:06:35 +0200 Stephan von Krawczynski wrote: > On Wed, 10 Jul 2019 15:55:04 +0200 > Martin Kletzander wrote: > > > On Wed, Jul 10, 2019 at 03:44:54PM +0200, Stephan von Krawczynski wrote: > > >On Wed, 10 Jul 2019 14:48:14 +0200 > > >Martin Kletzander wrote: > > > > > >> On Tue, Jul 09, 2019 at 02:45:18PM +0200, Stephan von Krawczynski > > >> wrote: > > >> >On Tue, 9 Jul 2019 14:26:08 +0200 > > >> >Pavel Hrdina wrote: > > >> > > > >> >> On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski > > >> >> wrote: > > >> >> > On Tue, 9 Jul 2019 09:40:23 +0100 > > >> >> > Daniel P. Berrangé wrote: > > >> >> > > > >> >> > > On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski > > >> >> > > wrote: > > >> >> > > > Hello list, > > >> >> > > > > > >> >> > > > I came across a fundamental flaw in the libvirt user > > >> >> > > > configuration lately and try to find a solution now. Here is > > >> >> > > > the problem: I run several qemu instances on arch linux all > > >> >> > > > configured via libvirt. The default config as user nobody:kvm > > >> >> > > > was fine up to the day I tried to use a host filesystem via > > >> >> > > > 9p. If you want to gain all user rights on the guest inside > > >> >> > > > that fs you have to run qemu as root. So far so good. But if > > >> >> > > > you have several qemus running and only one needs to be root, > > >> >> > > > what to do? You can try to give a -runas by using . > > >> >> > > > But that does not work, qemu instantly crashes. I think this > > >> >> > > > is because to have _one_ root qemu, you have to configure > > >> >> > > > libvirt to use root user. This means all rights to fs and so > > >> >> > > > on are set to root and this is what lets qemu probably go > > >> >> > > > crazy if dropping root by -runas. The whole thing would be a > > >> >> > > > lot easier and more transparent if the user in libvirt > > >> >> > > > wouldn't be a global config, but instead be part of the domain > > >> >> > > > xml. This way every qemu started could use a different user > > >> >> > > > and have different rights. In my case all but one could be > > >> >> > > > nobody:kvm, and one root:root. This should not be to > > >> >> > > > complicated based on whats already there, is it? > > >> >> > > > > >> >> > > Libvirt needs to know about the user/group QEMU is running at in > > >> >> > > order to ensure it gets given access to the various files it > > >> >> > > needs to use. If you look at the XML of the running guest you > > >> >> > > should see a describing the user/group it is running > > >> >> > > as currently. > > >> >> > > > > >> >> > > If no is in the offline config, libvirt adds the > > >> >> > > default seclabel, but if you want a different user/group, you > > >> >> > > can add the yourself. > > >> >> > > > > >> >> > > Regards, > > >> >> > > Daniel > > >> >> > > > >> >> > Hello Daniel, > > >> >> > > > >> >> > well, tried that (as good as the docs are) by adding: > > >> >> > > > >> >> > > > >> >> > nobody:kvm > > >> >> > > > >> >> > > > >> >> > This edit worked in virsh without giving errors. > > >> >> > Starting the domain and then looking into the xml showed: > > >> >> > > > >> >> > > > >> >> > > > >> >> > Consequently qemu runs still as root. My user:group setting simply > > >> >> > vanished. > > >> >> > > > >> >> > I think at least some better docs are needed with a str
Re: [libvirt] Problem configuring selective dropping of root
On Thu, 11 Jul 2019 10:07:11 +0200 Pavel Hrdina wrote: > On Wed, Jul 10, 2019 at 11:56:38AM +0200, Stephan von Krawczynski wrote: > > On Wed, 10 Jul 2019 09:56:35 +0200 > > Pavel Hrdina wrote: > > > > > On Wed, Jul 10, 2019 at 12:01:18AM +0200, Stephan von Krawczynski > > > wrote: > > > > On Tue, 9 Jul 2019 14:26:08 +0200 > > > > Pavel Hrdina wrote: > > > > > > > > > [...] > > > > > > > > > > In addition if you would like to have only one VM as root:root you > > > > > should keep the default config as nobody:kvm and use the root:root > > > > > for that specific VM. > > > > > > > > > > Pavel > > > > > > > > Let me answer this part in another post. > > > > Generally I agree with you. But there is one question: if I configure > > > > libvirt to use nobody:kvm as user, how is it possible to start a qemu > > > > with root privileges? I thought it not to be possible that it runs a > > > > root process while its config says it should be nobody ...? > > > > > > That configuration is in /etc/libvirt/qemu.conf which configures things > > > related to QEMU process and the user:group configuration tells how the > > > QEMU process will be started. The system libvirtd daemon runs always as > > > root:root in order to have permissions to execute QEMU process under any > > > user and to configure a lot of other things when starting a VM. > > > > > > > I thought it can only _drop_ privileges from root to nobody, because > > > > its primary user is root. > > > > Or is it in fact always running as root, and only "fake-dropping" to > > > > the configured user (maybe a spawned child), while still being able to > > > > spawn other root processes? > > > > > > I'm not sure what do you mean by "fake-dropping", libvirt forks itself > > > in order to create a new process where the QEMU binary is executed and > > > the permissions are configured for that newly created process. > > > > > > All of this is true only for the system libvirt, that means if you use > > > qemu:///system connection, for the session libvirt everything runs as > > > your user and there is no session libvirt for root user. > > > > > > The XML and configuration that I've suggested should work as I've tried > > > it before sending the mail. > > > > > > Pavel > > > > Thank you for clearing things up a bit for me. > > I am using arch linux (see OP) and the libvirt version gives: > > > > virsh # version > > Kompiliert gegen die Bibliothek: libvirt 5.4.0 > > Verwende Bibliothek: libvirt 5.4.0 > > > > Verwende API: QEMU 5.4.0 > > Laufender Hypervisor: QEMU 4.0.0 > > > > Using your seclabel with root:root in libvirt/qemu.conf throws: > > > > virsh # start collabora > > Fehler: Domain collabora konnte nicht gestartet werden > > Fehler: Interner Fehler: process exited while connecting to monitor: > > 2019-07-10T09:49:52.519211Z qemu-system-x86_64: -object > > secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-18-collabora/master-key.aes: > > Unable to read /var/lib/libvirt/qemu/domain-18-collabora/master-key.aes: > > Failed to open file > > “/var/lib/libvirt/qemu/domain-18-collabora/master-key.aes”: Permission > > denied > > > > Unfortunately I cannot verify what file permissions the requested file > > has, as it is vanished as soon as the crash happens. > > I bet though that it has root:root and the correctly set qemu user > > nobody:kvm has no read rights. So obviously relabel does not work. > > As it works on your side question is what's different? You are sure that > > you did not try the other way round and seclabel to root:root for a setup > > with standard user nobody:kvm. This would explain why you do not get this > > error... I generally try not to patch around in libvirt or qemu or the > > hosts' arch system. Which makes this probably at least a bug in the > > distro... > > Since the conversation of solution for the issue continues in different > part of this thread I would like to just point out that you should try > doing it the other way around. > > You've stated in previous mails that you would like to have only one VM > as root:root and the rest of the VMs as the default nobody:kvm. In > general it's really bad idea to configure root:root as the default, as > it create
Re: [libvirt] Problem configuring selective dropping of root (SOLVED)
On Thu, 11 Jul 2019 08:06:35 +0200 Stephan von Krawczynski wrote: > On Wed, 10 Jul 2019 15:55:04 +0200 > Martin Kletzander wrote: > > > On Wed, Jul 10, 2019 at 03:44:54PM +0200, Stephan von Krawczynski wrote: > > >On Wed, 10 Jul 2019 14:48:14 +0200 > > >Martin Kletzander wrote: > > > > > >> On Tue, Jul 09, 2019 at 02:45:18PM +0200, Stephan von Krawczynski > > >> wrote: > > >> >On Tue, 9 Jul 2019 14:26:08 +0200 > > >> >Pavel Hrdina wrote: > > >> > > > >> >> On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski > > >> >> wrote: > > >> >> > On Tue, 9 Jul 2019 09:40:23 +0100 > > >> >> > Daniel P. Berrangé wrote: > > >> >> > > > >> >> > > On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski > > >> >> > > wrote: > > >> >> > > > Hello list, > > >> >> > > > > > >> >> > > > I came across a fundamental flaw in the libvirt user > > >> >> > > > configuration lately and try to find a solution now. Here is > > >> >> > > > the problem: I run several qemu instances on arch linux all > > >> >> > > > configured via libvirt. The default config as user nobody:kvm > > >> >> > > > was fine up to the day I tried to use a host filesystem via > > >> >> > > > 9p. If you want to gain all user rights on the guest inside > > >> >> > > > that fs you have to run qemu as root. So far so good. But if > > >> >> > > > you have several qemus running and only one needs to be root, > > >> >> > > > what to do? You can try to give a -runas by using . > > >> >> > > > But that does not work, qemu instantly crashes. I think this > > >> >> > > > is because to have _one_ root qemu, you have to configure > > >> >> > > > libvirt to use root user. This means all rights to fs and so > > >> >> > > > on are set to root and this is what lets qemu probably go > > >> >> > > > crazy if dropping root by -runas. The whole thing would be a > > >> >> > > > lot easier and more transparent if the user in libvirt > > >> >> > > > wouldn't be a global config, but instead be part of the domain > > >> >> > > > xml. This way every qemu started could use a different user > > >> >> > > > and have different rights. In my case all but one could be > > >> >> > > > nobody:kvm, and one root:root. This should not be to > > >> >> > > > complicated based on whats already there, is it? > > >> >> > > > > >> >> > > Libvirt needs to know about the user/group QEMU is running at in > > >> >> > > order to ensure it gets given access to the various files it > > >> >> > > needs to use. If you look at the XML of the running guest you > > >> >> > > should see a describing the user/group it is running > > >> >> > > as currently. > > >> >> > > > > >> >> > > If no is in the offline config, libvirt adds the > > >> >> > > default seclabel, but if you want a different user/group, you > > >> >> > > can add the yourself. > > >> >> > > > > >> >> > > Regards, > > >> >> > > Daniel > > >> >> > > > >> >> > Hello Daniel, > > >> >> > > > >> >> > well, tried that (as good as the docs are) by adding: > > >> >> > > > >> >> > > > >> >> > nobody:kvm > > >> >> > > > >> >> > > > >> >> > This edit worked in virsh without giving errors. > > >> >> > Starting the domain and then looking into the xml showed: > > >> >> > > > >> >> > > > >> >> > > > >> >> > Consequently qemu runs still as root. My user:group setting simply > > >> >> > vanished. > > >> >> > > > >> >> > I think at least some better docs are needed with a str
Re: [libvirt] Problem configuring selective dropping of root (SOLVED)
On Wed, 10 Jul 2019 15:55:04 +0200 Martin Kletzander wrote: > On Wed, Jul 10, 2019 at 03:44:54PM +0200, Stephan von Krawczynski wrote: > >On Wed, 10 Jul 2019 14:48:14 +0200 > >Martin Kletzander wrote: > > > >> On Tue, Jul 09, 2019 at 02:45:18PM +0200, Stephan von Krawczynski wrote: > >> >On Tue, 9 Jul 2019 14:26:08 +0200 > >> >Pavel Hrdina wrote: > >> > > >> >> On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski > >> >> wrote: > >> >> > On Tue, 9 Jul 2019 09:40:23 +0100 > >> >> > Daniel P. Berrangé wrote: > >> >> > > >> >> > > On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski > >> >> > > wrote: > >> >> > > > Hello list, > >> >> > > > > >> >> > > > I came across a fundamental flaw in the libvirt user configuration > >> >> > > > lately and try to find a solution now. Here is the problem: > >> >> > > > I run several qemu instances on arch linux all configured via > >> >> > > > libvirt. > >> >> > > > The default config as user nobody:kvm was fine up to the day I > >> >> > > > tried > >> >> > > > to use a host filesystem via 9p. If you want to gain all user > >> >> > > > rights > >> >> > > > on the guest inside that fs you have to run qemu as root. So far > >> >> > > > so > >> >> > > > good. But if you have several qemus running and only one needs to > >> >> > > > be > >> >> > > > root, what to do? You can try to give a -runas by using > >> >> > > > . > >> >> > > > But that does not work, qemu instantly crashes. I think this is > >> >> > > > because to have _one_ root qemu, you have to configure libvirt to > >> >> > > > use > >> >> > > > root user. This means all rights to fs and so on are set to root > >> >> > > > and > >> >> > > > this is what lets qemu probably go crazy if dropping root by > >> >> > > > -runas. > >> >> > > > The whole thing would be a lot easier and more transparent if the > >> >> > > > user > >> >> > > > in libvirt wouldn't be a global config, but instead be part of the > >> >> > > > domain xml. This way every qemu started could use a different > >> >> > > > user and > >> >> > > > have different rights. In my case all but one could be > >> >> > > > nobody:kvm, and > >> >> > > > one root:root. This should not be to complicated based on whats > >> >> > > > already there, is it? > >> >> > > > >> >> > > Libvirt needs to know about the user/group QEMU is running at in > >> >> > > order to > >> >> > > ensure it gets given access to the various files it needs to use. > >> >> > > If you > >> >> > > look at the XML of the running guest you should see a > >> >> > > describing the user/group it is running as currently. > >> >> > > > >> >> > > If no is in the offline config, libvirt adds the default > >> >> > > seclabel, but if you want a different user/group, you can add the > >> >> > > yourself. > >> >> > > > >> >> > > Regards, > >> >> > > Daniel > >> >> > > >> >> > Hello Daniel, > >> >> > > >> >> > well, tried that (as good as the docs are) by adding: > >> >> > > >> >> > > >> >> > nobody:kvm > >> >> > > >> >> > > >> >> > This edit worked in virsh without giving errors. > >> >> > Starting the domain and then looking into the xml showed: > >> >> > > >> >> > > >> >> > > >> >> > Consequently qemu runs still as root. My user:group setting simply > >> >> > vanished. > >> >> > > >> >> > I think at least some better docs are needed with a striking example > >>
Re: [libvirt] Problem configuring selective dropping of root
On Wed, 10 Jul 2019 14:48:14 +0200 Martin Kletzander wrote: > On Tue, Jul 09, 2019 at 02:45:18PM +0200, Stephan von Krawczynski wrote: > >On Tue, 9 Jul 2019 14:26:08 +0200 > >Pavel Hrdina wrote: > > > >> On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski wrote: > >> > On Tue, 9 Jul 2019 09:40:23 +0100 > >> > Daniel P. Berrangé wrote: > >> > > >> > > On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski > >> > > wrote: > >> > > > Hello list, > >> > > > > >> > > > I came across a fundamental flaw in the libvirt user configuration > >> > > > lately and try to find a solution now. Here is the problem: > >> > > > I run several qemu instances on arch linux all configured via > >> > > > libvirt. > >> > > > The default config as user nobody:kvm was fine up to the day I tried > >> > > > to use a host filesystem via 9p. If you want to gain all user rights > >> > > > on the guest inside that fs you have to run qemu as root. So far so > >> > > > good. But if you have several qemus running and only one needs to be > >> > > > root, what to do? You can try to give a -runas by using . > >> > > > But that does not work, qemu instantly crashes. I think this is > >> > > > because to have _one_ root qemu, you have to configure libvirt to use > >> > > > root user. This means all rights to fs and so on are set to root and > >> > > > this is what lets qemu probably go crazy if dropping root by -runas. > >> > > > The whole thing would be a lot easier and more transparent if the > >> > > > user > >> > > > in libvirt wouldn't be a global config, but instead be part of the > >> > > > domain xml. This way every qemu started could use a different user > >> > > > and > >> > > > have different rights. In my case all but one could be nobody:kvm, > >> > > > and > >> > > > one root:root. This should not be to complicated based on whats > >> > > > already there, is it? > >> > > > >> > > Libvirt needs to know about the user/group QEMU is running at in order > >> > > to > >> > > ensure it gets given access to the various files it needs to use. If > >> > > you > >> > > look at the XML of the running guest you should see a > >> > > describing the user/group it is running as currently. > >> > > > >> > > If no is in the offline config, libvirt adds the default > >> > > seclabel, but if you want a different user/group, you can add the > >> > > yourself. > >> > > > >> > > Regards, > >> > > Daniel > >> > > >> > Hello Daniel, > >> > > >> > well, tried that (as good as the docs are) by adding: > >> > > >> > > >> > nobody:kvm > >> > > >> > > >> > This edit worked in virsh without giving errors. > >> > Starting the domain and then looking into the xml showed: > >> > > >> > > >> > > >> > Consequently qemu runs still as root. My user:group setting simply > >> > vanished. > >> > > >> > I think at least some better docs are needed with a striking example of > >> > how to change user and group ... > >> > I may be biased, but how to set user and group is probably the most basic > >> > example of how to use seclabel - and I cannot find one. > >> > >> I agree that the documentation is not the best one. > >> > >> You need to use type='static' relabel='yes': > >> > >> > >> nobody:kvm > >> > >> > >> To achieve that. > >> > >> In addition if you would like to have only one VM as root:root you > >> should keep the default config as nobody:kvm and use the root:root for > >> that specific VM. > >> > >> Pavel > > > >Hello Pavel, > > > >thank you for taking up the thread, but unfortunately your suggestion does > >not > >work: > > > >virsh # start collabora > >Fehler: Domain collabora konnte nicht gestartet werden > >Fehler: Interner Fehler: process exited while connecting to monitor: > >2019-07-09T12:34:00.735392Z qemu-system
Re: [libvirt] Problem configuring selective dropping of root
On Wed, 10 Jul 2019 09:56:35 +0200 Pavel Hrdina wrote: > On Wed, Jul 10, 2019 at 12:01:18AM +0200, Stephan von Krawczynski wrote: > > On Tue, 9 Jul 2019 14:26:08 +0200 > > Pavel Hrdina wrote: > > > > > [...] > > > > > > In addition if you would like to have only one VM as root:root you > > > should keep the default config as nobody:kvm and use the root:root for > > > that specific VM. > > > > > > Pavel > > > > Let me answer this part in another post. > > Generally I agree with you. But there is one question: if I configure > > libvirt to use nobody:kvm as user, how is it possible to start a qemu with > > root privileges? I thought it not to be possible that it runs a root > > process while its config says it should be nobody ...? > > That configuration is in /etc/libvirt/qemu.conf which configures things > related to QEMU process and the user:group configuration tells how the > QEMU process will be started. The system libvirtd daemon runs always as > root:root in order to have permissions to execute QEMU process under any > user and to configure a lot of other things when starting a VM. > > > I thought it can only _drop_ privileges from root to nobody, because its > > primary user is root. > > Or is it in fact always running as root, and only "fake-dropping" to the > > configured user (maybe a spawned child), while still being able to spawn > > other root processes? > > I'm not sure what do you mean by "fake-dropping", libvirt forks itself > in order to create a new process where the QEMU binary is executed and > the permissions are configured for that newly created process. > > All of this is true only for the system libvirt, that means if you use > qemu:///system connection, for the session libvirt everything runs as > your user and there is no session libvirt for root user. > > The XML and configuration that I've suggested should work as I've tried > it before sending the mail. > > Pavel Thank you for clearing things up a bit for me. I am using arch linux (see OP) and the libvirt version gives: virsh # version Kompiliert gegen die Bibliothek: libvirt 5.4.0 Verwende Bibliothek: libvirt 5.4.0 Verwende API: QEMU 5.4.0 Laufender Hypervisor: QEMU 4.0.0 Using your seclabel with root:root in libvirt/qemu.conf throws: virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-10T09:49:52.519211Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-18-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-18-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-18-collabora/master-key.aes”: Permission denied Unfortunately I cannot verify what file permissions the requested file has, as it is vanished as soon as the crash happens. I bet though that it has root:root and the correctly set qemu user nobody:kvm has no read rights. So obviously relabel does not work. As it works on your side question is what's different? You are sure that you did not try the other way round and seclabel to root:root for a setup with standard user nobody:kvm. This would explain why you do not get this error... I generally try not to patch around in libvirt or qemu or the hosts' arch system. Which makes this probably at least a bug in the distro... -- Regards, Stephan -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Problem configuring selective dropping of root
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina wrote: > [...] > > In addition if you would like to have only one VM as root:root you > should keep the default config as nobody:kvm and use the root:root for > that specific VM. > > Pavel Let me answer this part in another post. Generally I agree with you. But there is one question: if I configure libvirt to use nobody:kvm as user, how is it possible to start a qemu with root privileges? I thought it not to be possible that it runs a root process while its config says it should be nobody ...? I thought it can only _drop_ privileges from root to nobody, because its primary user is root. Or is it in fact always running as root, and only "fake-dropping" to the configured user (maybe a spawned child), while still being able to spawn other root processes? -- Regards, Stephan -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Problem configuring selective dropping of root
On Tue, 9 Jul 2019 14:26:08 +0200 Pavel Hrdina wrote: > On Tue, Jul 09, 2019 at 02:03:15PM +0200, Stephan von Krawczynski wrote: > > On Tue, 9 Jul 2019 09:40:23 +0100 > > Daniel P. Berrangé wrote: > > > > > On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski > > > wrote: > > > > Hello list, > > > > > > > > I came across a fundamental flaw in the libvirt user configuration > > > > lately and try to find a solution now. Here is the problem: > > > > I run several qemu instances on arch linux all configured via libvirt. > > > > The default config as user nobody:kvm was fine up to the day I tried > > > > to use a host filesystem via 9p. If you want to gain all user rights > > > > on the guest inside that fs you have to run qemu as root. So far so > > > > good. But if you have several qemus running and only one needs to be > > > > root, what to do? You can try to give a -runas by using . > > > > But that does not work, qemu instantly crashes. I think this is > > > > because to have _one_ root qemu, you have to configure libvirt to use > > > > root user. This means all rights to fs and so on are set to root and > > > > this is what lets qemu probably go crazy if dropping root by -runas. > > > > The whole thing would be a lot easier and more transparent if the user > > > > in libvirt wouldn't be a global config, but instead be part of the > > > > domain xml. This way every qemu started could use a different user and > > > > have different rights. In my case all but one could be nobody:kvm, and > > > > one root:root. This should not be to complicated based on whats > > > > already there, is it? > > > > > > Libvirt needs to know about the user/group QEMU is running at in order to > > > ensure it gets given access to the various files it needs to use. If you > > > look at the XML of the running guest you should see a > > > describing the user/group it is running as currently. > > > > > > If no is in the offline config, libvirt adds the default > > > seclabel, but if you want a different user/group, you can add the > > > yourself. > > > > > > Regards, > > > Daniel > > > > Hello Daniel, > > > > well, tried that (as good as the docs are) by adding: > > > > > > nobody:kvm > > > > > > This edit worked in virsh without giving errors. > > Starting the domain and then looking into the xml showed: > > > > > > > > Consequently qemu runs still as root. My user:group setting simply > > vanished. > > > > I think at least some better docs are needed with a striking example of > > how to change user and group ... > > I may be biased, but how to set user and group is probably the most basic > > example of how to use seclabel - and I cannot find one. > > I agree that the documentation is not the best one. > > You need to use type='static' relabel='yes': > > > nobody:kvm > > > To achieve that. > > In addition if you would like to have only one VM as root:root you > should keep the default config as nobody:kvm and use the root:root for > that specific VM. > > Pavel Hello Pavel, thank you for taking up the thread, but unfortunately your suggestion does not work: virsh # start collabora Fehler: Domain collabora konnte nicht gestartet werden Fehler: Interner Fehler: process exited while connecting to monitor: 2019-07-09T12:34:00.735392Z qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-17-collabora/master-key.aes: Failed to open file “/var/lib/libvirt/qemu/domain-17-collabora/master-key.aes”: Permission denied Obviously this is because "type='static'" means that libvirt does not care about setting the user rights for qemu, which then leads to this. I did think "relabel='yes'" should do that, but does not - or I have a deep misunderstanding concerning the seclabel parameters ... Thanks for any help to solve this, if there is no bug involved. Dumpxml shows this btw: nobody:kvm which at least is what was configured. -- Regards, Stephan -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Problem configuring selective dropping of root
On Tue, 9 Jul 2019 09:40:23 +0100 Daniel P. Berrangé wrote: > On Mon, Jul 08, 2019 at 09:47:24PM +0200, Stephan von Krawczynski wrote: > > Hello list, > > > > I came across a fundamental flaw in the libvirt user configuration lately > > and try to find a solution now. Here is the problem: > > I run several qemu instances on arch linux all configured via libvirt. The > > default config as user nobody:kvm was fine up to the day I tried to use a > > host filesystem via 9p. If you want to gain all user rights on the guest > > inside that fs you have to run qemu as root. So far so good. But if you > > have several qemus running and only one needs to be root, what to do? You > > can try to give a -runas by using . But that does not work, > > qemu instantly crashes. I think this is because to have _one_ root qemu, > > you have to configure libvirt to use root user. This means all rights to > > fs and so on are set to root and this is what lets qemu probably go crazy > > if dropping root by -runas. The whole thing would be a lot easier and more > > transparent if the user in libvirt wouldn't be a global config, but > > instead be part of the domain xml. This way every qemu started could use a > > different user and have different rights. > > In my case all but one could be nobody:kvm, and one root:root. > > This should not be to complicated based on whats already there, is it? > > Libvirt needs to know about the user/group QEMU is running at in order to > ensure it gets given access to the various files it needs to use. If you > look at the XML of the running guest you should see a describing > the user/group it is running as currently. > > If no is in the offline config, libvirt adds the default > seclabel, but if you want a different user/group, you can add the > yourself. > > Regards, > Daniel Hello Daniel, well, tried that (as good as the docs are) by adding: nobody:kvm This edit worked in virsh without giving errors. Starting the domain and then looking into the xml showed: Consequently qemu runs still as root. My user:group setting simply vanished. I think at least some better docs are needed with a striking example of how to change user and group ... I may be biased, but how to set user and group is probably the most basic example of how to use seclabel - and I cannot find one. -- Regards, Stephan -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] Problem configuring selective dropping of root
Hello list, I came across a fundamental flaw in the libvirt user configuration lately and try to find a solution now. Here is the problem: I run several qemu instances on arch linux all configured via libvirt. The default config as user nobody:kvm was fine up to the day I tried to use a host filesystem via 9p. If you want to gain all user rights on the guest inside that fs you have to run qemu as root. So far so good. But if you have several qemus running and only one needs to be root, what to do? You can try to give a -runas by using . But that does not work, qemu instantly crashes. I think this is because to have _one_ root qemu, you have to configure libvirt to use root user. This means all rights to fs and so on are set to root and this is what lets qemu probably go crazy if dropping root by -runas. The whole thing would be a lot easier and more transparent if the user in libvirt wouldn't be a global config, but instead be part of the domain xml. This way every qemu started could use a different user and have different rights. In my case all but one could be nobody:kvm, and one root:root. This should not be to complicated based on whats already there, is it? -- Regards, Stephan -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
