[Bug 1633207] Re: VM fails to start with dac security driver added
Local creation with a apparmor seclabel fails the same as the migration, so we can ignore all "migration specials". To test that add: to /usr/share/uvtool/libvirt/template.xml And run uvt-kvm create again -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
This bug was fixed in the package libvirt - 2.1.0-1ubuntu9.1 --- libvirt (2.1.0-1ubuntu9.1) yakkety; urgency=medium * d/p/u/apparmor-fix-other-seclabels.patch fixes an issue parsing non apparmor security labels (LP: #1633207). -- Christian Ehrhardt Thu, 01 Dec 2016 09:44:12 +0100 ** Changed in: libvirt (Ubuntu Yakkety) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Since there was no reply to verify in a week I felt I had to clear the queue and tested it myself again. @bugproxy: In general - a.k.a. for next time - I'd really like to have 3rd party verification. Not to save me the 15 minutes, but to make sure it really addresses your issue and get further verification if anything else was broken by the SRU. If it doesn't fit with your current plans that is fine, but then let me know an estimate when you expect you get to it. ** Tags removed: verification-needed ** Tags added: verification-done -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Hello bugproxy, or anyone else affected, Accepted libvirt into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libvirt/2.1.0-1ubuntu9.1 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance! ** Changed in: libvirt (Ubuntu Yakkety) Status: Triaged => Fix Committed ** Tags added: verification-needed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Prepared SRU Template and Uploaded into the (Y) SRU review queue. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
** Description changed: + [Impact] + + * Due to an upstream change in libvirt 2.0 users of libvirt >=2.0 +(that is >=Yakkety) can't use non apparmor security labels anymore. + + * That means old guest definitions that should still work fail to start +now + + * The issue was in virt-aa-helper, the proposed fix was tested and then +brought upstream. This is a backport of the upstream accepted fix. + + [Test Case] + + * Testcase with virt-aa-helper on a minimal xml: + $ cat << EOF > /tmp/test.xml + + test-seclabel + 12345678-9abc-def1-2345-6789abcdef00 + 1 + hvm + + + + EOF + $ /usr/lib/libvirt/virt-aa-helper -d -r -p 0 \ + -u libvirt-12345678-9abc-def1-2345-6789abcdef00 < /tmp/test.xml + + Current Result: + virt-aa-helper: error: could not parse XML + virt-aa-helper: error: could not get VM definition + Expected Result is to emit a valid apparmor profile + + * The more complex test is to create a guest (whatever way you like) and + add an empty dac security label (as shown above) to then start the + guest. + + Current Result: + error: Failed to start domain yakkety-doubleseclabel + error: internal error: cannot load AppArmor profile 'libvirt-8746b00d-aad1-4346-8784-2d4331465153' + Expected Result: + properly staring the guest + + [Regression Potential] + + * The change is in the parsing of domain info in domain.conf. While no +local nor upstream tests broke anything one could think of very special +xml configuation that now might fail parsing. OTOH the new change now +skips some of the parsing, so even if we miss to consider something it +shouldn't fail, but instead "forget" to read some data correctly. The +part that we skip are seclabels which are created dynamically anyway. + + * Also the changed flag is local to virt-aa-helper.c so and guarded by +that flag in domain_conf.c so it should be a no-op to anybody but virt- +aa-helper for sure. + + [Other Info] + + * Anything else you think is useful to include + * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board + * and address these questions in advance + + ---Problem Description--- VM fails to start with dac security driver added - + ---uname output--- Linux ltc-test-ci1 4.4.0-9136-generic #55-Ubuntu SMP Fri Aug 26 05:56:24 UTC 2016 ppc64le ppc64le ppc64le GNU/Linux - - Machine Type = power 8 ppc64le - + + Machine Type = power 8 ppc64le + ---Steps to Reproduce--- - + VM fails to start with dac security driver added - 1. Define a VM with both apparmor and dac security driver( Used XML as below) + 1. Define a VM with both apparmor and dac security driver( Used XML as below) #virsh dumpxml virt-tests-vm1 - virt-tests-vm1 - 0491f0cd-eb14-4992-be4c-53a1adf1d314 - 33554432 - 33554432 - 32 - - /machine - - - hvm - - - - - - - destroy - restart - restart - - /usr/bin/kvm - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + virt-tests-vm1 + 0491f0cd-eb14-4992-be4c-53a1adf1d314 + 33554432 + 33554432 + 32 + + /machine + + + hvm + + + + + + + destroy + restart + restart + + /usr/bin/kvm + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 2. virsh start virt-tests-vm1 #virsh start virt-tests-vm1 error: Failed to start domain virt-tests-vm1 error: internal error: cannot load AppArmor profile 'libvirt-0491f0cd-eb14-4992-be4c-53a1adf1d314'--NOK - 3. After removing dac line from xml() VM started fine #virsh start virt-tests-vm1 Domain virt-tests-vm1 started + Userspace tool common name: ii libvirt-bin + 2.1.0-1ubuntu5 ppc64el programs for the + libvirt library - - Userspace tool common name: ii libvirt-bin 2.1.0-1ubuntu5 ppc64el programs for the libvirt library - - The userspace tool has the following bit modes: both + The userspace tool has the following bit modes: both Userspace package: ii libvirt-bin 2.1.0-1ubuntu5 ppc64el programs for the libvirt library ** Description changed: [Impact] - * Due to an upstream change in libvirt 2.0 users of libvirt >=2.0 -(that is >=Yakkety) can't use non apparmor security labels anymore. + * Due to an upstream change in libvirt 2.0 users of libvirt >=2.0 + (that is >=Yakkety) can't use non apparmor secur
[Bug 1633207] Re: VM fails to start with dac security driver added
The refreshed upstream accepted fix is now available in Zesty as 2.1.0-1ubuntu14 With that ready now preparing the SRU into Yakkety. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
FYI - The backport SRU to Yakkety will have to wait until we have a upstream accepted solution. ** Also affects: libvirt (Ubuntu Yakkety) Importance: Undecided Status: New ** Changed in: libvirt (Ubuntu Yakkety) Status: New => Triaged ** Changed in: libvirt (Ubuntu Yakkety) Importance: Undecided => Low -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
This bug was fixed in the package libvirt - 2.1.0-1ubuntu13 --- libvirt (2.1.0-1ubuntu13) zesty; urgency=medium * drop d/p/ubuntu/fix-ftbfs-for-gnutls-3-5-6.patch as the offending change in gnutls has been reverted (LP: #1641615) * Build depend on gnutls >= 3.5.6-4ubuntu2 to build after the gnutls fix migrated -- Christian Ehrhardt Thu, 17 Nov 2016 08:43:10 +0100 ** Changed in: libvirt (Ubuntu) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
FYI - v2 of the patch in discussion upstream https://www.redhat.com/archives/libvir-list/2016-November/msg00991.html -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
What worked last week doesn't have to this week - I ran into an FTBFS - please wait a bit until resolved. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Unfortunately upstream response is super slow on this. I think the patch is right and therefore I'm willing to put it into zesty as being a dev release in development. That will also give us more coverage if there is anything we might have missed. That said pushed it to Zesty now the way it was tested.by me and the reporter. Since it is not an issue for Xenial there is not SRU need there but for Yakkety I'd only consider an SRU once upstream discussion settled and accepted it. That said @IBM - if you want to request an SRU on this into Yakkety I'd ask you to join the upstream discussion on libvirt to give it some weight by a third party pushing for it. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
** Changed in: libvirt (Ubuntu) Status: Confirmed => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Thank you a lot for verifying the ppa. Since this isn't critically urgent I'll wait with a fix upload to the package until the upstream discussion settled (better than to revert in two weeks again). Once https://www.redhat.com/archives/libvir-list/2016-October/msg01297.html followed in November by thread https://www.redhat.com/archives/libvir-list/2016-November/msg00229.html conclude I'll go forward on this. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
FYI discussion started at https://www.redhat.com/archives/libvir- list/2016-October/msg01297.html -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Thanks a lot Guido for your feedback - it helped me better "reading the case". I see the same issue throughout latest libvirt upstream as of today. So I'm gonna submit the fix upstream for discussion as I could easily overlook something here. E.g. parseOpaque is quite close as it is passed up to virDomainDefParseXML, but I think that would be even more misuse than a new flag. If accepted there (one way or the other) I intend to create a diff to upload for latest Debian and Ubuntu and consider SRUs from there. The created domain while active has both seclabels and valid content in them just as it had back on libvirt 1.3: [...] libvirt-956134c4-d91d-417e-b68f-1d8d492419d6 libvirt-956134c4-d91d-417e-b68f-1d8d492419d6 +112:+116 +112:+116 @AGX - I'll set you on cc on that upstream discussion. @IBM - I have a new version (2.1.0-1ubuntu10~ppa5) in the ppa that works for me, It would be great if you could verify this one for you as well. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
dfbc9a83 was necessary since libvirt changed the paths of the monitor socket in a89f05ba8df095875f5ec8a9065a585af63a010b. We had to switch from VIR_DOMAIN_DEF_PARSE_INACTIVE to active since we need the domain id (ctl->def->id) as it is part of the socket path now. It would probably o.k. to skip validation but we need to parse the active domain config to get the id. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
I subscribed agx, the author of the conflicting patch upstream. Questions: agx - Could you please comment on my finding? agx - Please describe your case that let you write dfbc9a83? IBM - I don't think it helps yet, but if you can please try to verify the ppa I provide at https://launchpad.net/~paelzer/+archive/ubuntu/libvirt-bug-1633207 I'm available on freenode e.g. in #ubuntu-server as cpaelzer. This likely is an extended weekend for both of us, but please feel free to try to catch me there. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
I made an experimental fix available at https://launchpad.net/~paelzer/+archive/ubuntu/libvirt-bug-1633207 In the pure aa-helper tests that continues to work with all my usual minor tests and it fixes dac and dac+apparmor label issues. But I seem to run into issues with doing full guests: qemu-system-x86_64: -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-2-yakkety-sec-testfix/master-key.aes: Unable to read /var/lib/libvirt/qemu/domain-2-yakkety-sec-testfix/master-key.aes: Failed to open file '/var/lib/libvirt/qemu/domain-2-yakkety-sec-testfix/master-key.aes': Permission denie That is due to an apparmor deny and might be that this was the issue that was tried to be fixed with the breaking change in libvirt? I'll run more tests on it on my own. But I'd really like to coordinate with the author of the former change what the test case was that made him create the patch. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
TL;DR:
- a dac sec label is parsed
- it has no label, but due to a bug it searches one
- label can't be found for an inactive domain
- exit with Error
- expected fix is reverting part of dfbc9a83
Debug-Analysis:
Interesting part of the call chain:
get_definition -> virDomainDefParseString -> virDomainDefParse ->
virDomainDefParseNode -> virDomainDefParseXML -> virSecurityLabelDefsParseXML
-> virSecurityLabelDefParseXML
Compiled -O0 -g to see more to see where it is failing.
The code itself (of that failing function) didn't change since 1.3.1 (Xenial).
gdb ~/libvirt-2.1.0/debian/tmp/usr/lib/libvirt/virt-aa-helper
set env LD_LIBRARY_PATH
/home/ubuntu/libvirt-2.1.0/debian/tmp/usr/lib/x86_64-linux-gnu/
set solib-search-path
/home/ubuntu/libvirt-2.1.0/debian/tmp/usr/lib/x86_64-linux-gnu/
b virSecurityLabelDefsParseXML
run -d -r -p 0 -u libvirt-6e082f89-902c-413c-9d9e-f609089d3374 <
yakkety-sec-dac.xml
virSecurityLabelDefParseXML (ctxt=0x557ddaf0, flags=1024) at
../../../src/conf/domain_conf.c:6384
n (number of labels) is 1
single def parse in virSecurityLabelDefParseXML
1. type dynamic = VIR_DOMAIN_SECLABEL_DYNAMIC
2. relabel yes
3-5 useless if/jumps
6. fails at parsing the actual label
it doesn't find a label, but thinks it needs one
check:
6.1 seclabel->type == VIR_DOMAIN_SECLABEL_STATIC => it is not
6.2 !(flags & VIR_DOMAIN_DEF_PARSE_INACTIVE) &&=> true
6.3 seclabel->type != VIR_DOMAIN_SECLABEL_NONE => true
=> There is no label for the currently off machine, so it fails to find one and
goes to error path
The function does right, but the flags suggest it would be alive.
Definiton:
/* Parse only parts of the XML that would be present in an inactive libvirt
* XML. Note that the flag does not imply that ABI incompatible
* transformations can be used, since it's used to strip runtime info when
* restoring save images/migration. */
VIR_DOMAIN_DEF_PARSE_INACTIVE= 1 << 1,
The flag comes from the first in the call chain "get_definition"
ctl->def = virDomainDefParseString(xmlStr, ctl->caps, ctl->xmlopt,
VIR_DOMAIN_DEF_PARSE_SKIP_VALIDATE);
That exactly is a diff of the Ubuntu versions on that call:
ctl->def = virDomainDefParseString(xmlStr,
ctl->caps, ctl->xmlopt,
- VIR_DOMAIN_DEF_PARSE_INACTIVE);
+ VIR_DOMAIN_DEF_PARSE_SKIP_VALIDATE);
Almost all other changes do OR it in:
-int domainflags = VIR_DOMAIN_DEF_PARSE_INACTIVE;
+int domainflags = VIR_DOMAIN_DEF_PARSE_INACTIVE |
+ VIR_DOMAIN_DEF_PARSE_SKIP_VALIDATE;
Check upstream for the reasons:
commit b394af162a3871575d9f9c28f72331f198aafa25
Author: Peter Krempa
Date: Thu May 26 15:58:53 2016 +0200
conf: Add infrastructure for adding configuration validation
On the critical place before there was a 0, so setting fix was like ORing in.
Why was there a 0 and not VIR_DOMAIN_DEF_PARSE_INACTIVE like in the past?
That was the reason there was a 0 before b394af16 came in:
commit dfbc9a8382adc0495bf0e034ae6add92bed4822b
Author: Guido Günther
Date: Sat Apr 2 12:49:28 2016 +0200
apparmor: QEMU monitor socket moved
That changed the call from VIR_DOMAIN_DEF_PARSE_INACTIVE to 0 for issues
starting with apparmor but provides no further detail.
The patch to fix would be as easy as:
--- libvirt-2.1.0.orig/src/security/virt-aa-helper.c
+++ libvirt-2.1.0/src/security/virt-aa-helper.c
@@ -708,6 +708,7 @@ get_definition(vahControl * ctl, const c
ctl->def = virDomainDefParseString(xmlStr,
ctl->caps, ctl->xmlopt,
+ VIR_DOMAIN_DEF_PARSE_INACTIVE |
VIR_DOMAIN_DEF_PARSE_SKIP_VALIDATE);
if (ctl->def == NULL) {
I checked some related cases on apparmor instead of dac labels:
- if dumpxml runs on an running instance with apparmor labels it adds the label
to the output, so next load works as it can find it
- if a uuid is not yet defined it creates new labels and works
- if a uuid is defined, but no lavel in xml aa-helper fails on apparmor
seclabels with the same issue (fixed by the same patch)
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1633207
Title:
VM fails to start with dac security driver added
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Again at: sudo virsh start yakkety-doubleseclabel error: Failed to start domain yakkety-doubleseclabel error: internal error: cannot load AppArmor profile 'libvirt-8746b00d-aad1-4346-8784-2d4331465153' In the log I found the related: Okt 27 13:45:50 horsea libvirtd[10370]: internal error: Child process (LIBVIRT_LOG_OUTPUTS=3:stderr /usr/lib/libvirt/virt-aa-helper -p 0 -r -u libvirt-8746b00d-aad1-4346-8784-2d4331465153) unexpected exit status 1: 2016-10-27 13:45:20.873+: 10640: info : libvirt version: 2.1.0, package: 1ubuntu10~ppa3 (Christian Ehrhardt Mon, 24 Oct 2016 14:21:36 +0200) 2016-10-27 13:45:20.873+: 10640: info : hostname: horsea 2016-10-27 13:45:20.873+: 10640: error : virSecurityLabelDefParseXML:6473 : XML error: security label is missing virt-aa-helper: error: could not parse XML virt-aa-helper: error: could not get VM definition Okt 27 13:45:50 horsea libvirtd[10370]: internal error: cannot load AppArmor profile 'libvirt-8746b00d-aad1-4346-8784-2d4331465153' Okt 27 13:45:50 horsea virtlogd[7706]: End of file while reading data: Input/output error I also found that adding dac alone is enough to trigger: $ virsh dumpxml yakkety-doubleseclabel | grep -A 20 ' => Failing $ virsh dumpxml yakkety-sec-app | grep -A 20 seclabel => Working $ virsh dumpxml yakkety-sec-dac | grep -A 20 seclabel => Failing just as much as case 1, maybe because apparmor is default on. Trying to check the /usr/lib/libvirt/virt-aa-helper in those cases, but since it is not meant to be called directly that is a bit tricky. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
After a bit of twiddling I found a somewhat reasonable repro with the virt-aa-helper tool. diff -Naur yakkety-sec-dac.xml yakkety-sec-nodac.xml --- yakkety-sec-dac.xml 2016-10-27 14:32:39.565995840 + +++ yakkety-sec-nodac.xml 2016-10-27 14:32:45.097973456 + @@ -60,6 +60,5 @@ - So the only diff is if the dac seclabel is here or not. $ sudo /usr/lib/libvirt/virt-aa-helper -d -r -p 0 -u libvirt-6e082f89-902c-413c-9d9e-f609089d3374 < yakkety-sec-dac.xml virt-aa-helper: error: could not parse XML virt-aa-helper: error: could not get VM definition $ sudo /usr/lib/libvirt/virt-aa-helper -d -r -p 0 -u libvirt-6e082f89-902c-413c-9d9e-f609089d3374 < yakkety-sec-nodac.xml virt-aa-helper: /etc/apparmor.d/libvirt/libvirt-6e082f89-902c-413c-9d9e-f609089d3374.files virt-aa-helper: "/var/log/libvirt/**/yakkety-sec-dac.log" w, "/var/lib/libvirt/qemu/domain-yakkety-sec-dac/monitor.sock" rw, "/var/lib/libvirt/qemu/domain--1-yakkety-sec-dac/*" rw, "/var/lib/libvirt/qemu/channel/target/domain--1-yakkety-sec-dac/*" rw, "/var/run/libvirt/**/yakkety-sec-dac.pid" rwk, "/run/libvirt/**/yakkety-sec-dac.pid" rwk, "/var/run/libvirt/**/*.tunnelmigrate.dest.yakkety-sec-dac" rw, "/run/libvirt/**/*.tunnelmigrate.dest.yakkety-sec-dac" rw, "/var/lib/uvtool/libvirt/images/yakkety-sec-dac.qcow" rw, "/var/lib/uvtool/libvirt/images/x-uvt-b64-Y29tLnVidW50dS5jbG91ZC5kYWlseTpzZXJ2ZXI6MTYuMTA6YW1kNjQgMjAxNjEwMjI=" r, "/var/lib/uvtool/libvirt/images/yakkety-sec-dac-ds.qcow" rw, # for qemu guest agent channel owner "/var/lib/libvirt/qemu/channel/target/domain-yakkety-sec-dac/**" rw, /dev/vhost-net rw, New running debuild locally on xenial and yakkety libvirt to have the packaged aa-helper in a debuggable and recompilable fashion. ** Changed in: libvirt (Ubuntu) Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Once more confirmed that it worked in Xenial - adding regression-release ** Tags added: regression-release -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Ok, I found why those templates didn't get on my BM test system to begin with. They were conffiles and I had none of them modified, but some more in the same directories. So while not that clear still the usual "protect custom conffiles" mechanism that blocked me. A full purge + manual extra conffile clean + re-install made it working again. Overall feels a bit touchy atm :-/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Three way check on fresh installs: dpkg -S $((find /etc/apparmor.d/ -name '*libvirt*' && find /etc/apparmor.d/ -name '*TEMPLATE*' )| xargs) | sort X Y X-Y upgrade libvirt-bin: /etc/apparmor.d/abstractions/libvirt-lxc libvirt-daemon-system: /etc/apparmor.d/abstractions/libvirt-lxc libvirt-daemon-system: /etc/apparmor.d/abstractions/libvirt-lxc libvirt-bin: /etc/apparmor.d/abstractions/libvirt-qemu libvirt-daemon-system: /etc/apparmor.d/abstractions/libvirt-qemu libvirt-daemon-system: /etc/apparmor.d/abstractions/libvirt-qemu libvirt-bin: /etc/apparmor.d/libvirt libvirt-daemon-system: /etc/apparmor.d/libvirt libvirt-daemon-system: /etc/apparmor.d/libvirt libvirt-bin: /etc/apparmor.d/libvirt/TEMPLATE.lxc libvirt-daemon-system: /etc/apparmor.d/libvirt/TEMPLATE.lxc libvirt-daemon-system: /etc/apparmor.d/libvirt/TEMPLATE.lxc libvirt-bin: /etc/apparmor.d/libvirt/TEMPLATE.qemu libvirt-daemon-system: /etc/apparmor.d/libvirt/TEMPLATE.qemu libvirt-daemon-system: /etc/apparmor.d/libvirt/TEMPLATE.qemu libvirt-daemon-system: /etc/apparmor.d/local/usr.lib.libvirt.virt-aa-helper libvirt-daemon-system: /etc/apparmor.d/local/usr.lib.libvirt.virt-aa-helper libvirt-bin: /etc/apparmor.d/local/usr.sbin.libvirtd libvirt-daemon-system: /etc/apparmor.d/local/usr.sbin.libvirtd libvirt-daemon-system: /etc/apparmor.d/local/usr.sbin.libvirtd libvirt-bin: /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper libvirt-daemon-system: /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper libvirt-daemon-system: /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper libvirt-bin: /etc/apparmor.d/usr.sbin.libvirtd libvirt-daemon-system: /etc/apparmor.d/usr.sbin.libvirtd libvirt-daemon-system: /etc/apparmor.d/usr.sbin.libvirtd In this case Y and the X-Y upgrade was equal. Also the formerly missing TEMPLATE files were here. Note that I still have that cae on my phys box - no matter how often I reinstall. Checking content between X and Y: Equal: 5f6aa836ced6b474dabfce46a8bfb5e4 /etc/apparmor.d/libvirt/TEMPLATE.lxc b0dfa704c6297fd9a4e68f0137c6be88 /etc/apparmor.d/libvirt/TEMPLATE.qemu 7166fa490aaf905b7f71cb5407ef0696 /etc/apparmor.d/local/usr.sbin.libvirtd No functional diff (only comments/reordering): /etc/apparmor.d/abstractions/libvirt-lxc /etc/apparmor.d/abstractions/libvirt-qemu /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper New but non-functional (empty to carry overwrites) /etc/apparmor.d/local/usr.lib.libvirt.virt-aa-helper Changed: /etc/apparmor.d/usr.sbin.libvirtd added "/usr/sbin/virtlogd pix" Here the easy in container test stops, as the apparmor security driver really can't work in there. I need to go back to my BM system and understand/fix why it does run into "error: unsupported configuration: Unable to find security driver for model apparmor" now. While only partially related I still wanted to document here to find it later if needed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
While debugging I found the first level of oddities that I'll continue on and hopefully gives us a solution (or at least eliminate one roadblock). I think I found that things work with the error described in the bug on Xenial->Yakkety upgraded systems. But on all others I see: error: unsupported configuration: Unable to find security driver for model apparmor That would explain the reproducibility fuzz a bit. After realizing that I checked logs: internal error: template '/etc/apparmor.d/libvirt/TEMPLATE.qemu' does not exist internal error: template '/etc/apparmor.d/libvirt/TEMPLATE.qemu' does not exist unsupported configuration: Security driver apparmor not enabled internal error: template '/etc/apparmor.d/libvirt/TEMPLATE.qemu' does not exist Now checking for those files is even more strange. $ dpkg -S /etc/apparmor.d/libvirt/TEMPLATE.qemu libvirt-daemon-system: /etc/apparmor.d/libvirt/TEMPLATE.qemu sudo apt-get install --reinstall libvirt-daemon-system ll /etc/apparmor.d/libvirt/TEMPLATE.qemu ls: cannot access '/etc/apparmor.d/libvirt/TEMPLATE.qemu': No such file or directory I guess we have those things here: 1. no proper handling of conffile changes due to the switch to the upstream provided apparmor profiles 2. on upgraded systems old&new somehow conflict 3. on new Yakkety apparmor seclabel doesn't work at all Going on with debugging tomorrow. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
I realized that part of my former verification was caused by the kvm-in- lxd env I use to avoid needing too much metal. So I retried on x86 again as these code paths shouldn't be arch specific at all. And now I was able to recreate on x86 as well. The summary looks like this now: * - xenial - works ppc4el - yakkety - fail x86 - yakkety - fail x86 - yakkety with 4.4 kernel - fail Going on with debugging, but I'm on the Road the next few days - so it might take a bit unless someone else jumps in. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Ha - got my container trick working again. So testing on Yakkety, adding the double seclabel. Finally - able to reproduce - yeah! Looking deeper into that now... ** Changed in: libvirt (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
2nd level kvm failed me as well :-/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
I have to report that my usual trick to run KVM from inside a container doesn't work the same way on ppc64el. It might take a while for me to get a Yakkety ppc64el BM system, so more than before I'm dependent on you reporting the extended logs as I requested. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Thanks satheera for the reply. I wonder why it works for me than as I explicitly tested ppc as well just as you do ... ? It works fine on x86 with Yakkety. As well as fine on ppc64el with Xenial. I don't have a Yakkety around yet and machines are scarce. I assume the xml is how avocado creates it for you. I compared our libvirt xml files and tried to remove any remaining delta. That changes from mine to yours were: + resource partition + adding topology - cpu features + on crash destroy -> restart + adding spapr-vio scsi controller Still starting fine. >From you it would be great if you could enable debugging for libvirt service and virsh, run the failing start of the guest again and report the qemu log file and libvirt from journalctl here. See https://libvirt.org/logging.html for more. It seems I have to try getting a ppc64el on Yakkety for this test next ... working on that ... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1633207] Re: VM fails to start with dac security driver added
Hi, I tested a simple guest as created with uvt-kvm: $ uvt-kvm create --memory 2048 --cpu 4 --password=ubuntu paelzer-yakkety-test-libvirt release=yakkety arch=ppc64el label=daily plus the two lines: That works on: Xenial: ok Yakkety: ok I did the same on ppc64el, but only had a Xenial host available there. Yet this worked just fine as well. Quoting the report: "Linux ltc-test-ci1 4.4.0-9136-generic #55-Ubuntu SMP Fri Aug 26 05:56:24" Since that seems to be a pre-release yakkety, could I ask you to retest with at least the released levels and report the versions of qemu&libvirt involved for you (dpkg -l '*qemu*' '*libvirt*')? ** Changed in: libvirt (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1633207 Title: VM fails to start with dac security driver added To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1633207/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
