[Bug 1776189] Re: implement a machine type based control of host-phys-bits
This bug was fixed in the package qemu - 1:2.11+dfsg-1ubuntu7.4 --- qemu (1:2.11+dfsg-1ubuntu7.4) bionic; urgency=medium * d/p/ubuntu/machine-type-hpb.patch: add -hpb machine type for host-phys-bits=true (LP: #1776189) - add an info about this change in debian/qemu-system-x86.NEWS -- Christian Ehrhardt Wed, 13 Jun 2018 10:41:34 +0200 ** Changed in: qemu (Ubuntu Bionic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
In addition we had the same pushed through regression tests (on ppa with same content). ** Tags removed: verification-needed verification-needed-bionic ** Tags added: verification-done verification-done-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
Upgraded: qemu-block-extra/bionic-proposed 1:2.11+dfsg-1ubuntu7.4 amd64 [upgradable from: 1:2.11+dfsg-1ubuntu7.3] qemu-kvm/bionic-proposed 1:2.11+dfsg-1ubuntu7.4 amd64 [upgradable from: 1:2.11+dfsg-1ubuntu7.3] qemu-system-common/bionic-proposed 1:2.11+dfsg-1ubuntu7.4 amd64 [upgradable from: 1:2.11+dfsg-1ubuntu7.3] qemu-system-x86/bionic-proposed 1:2.11+dfsg-1ubuntu7.4 amd64 [upgradable from: 1:2.11+dfsg-1ubuntu7.3] qemu-user-static/bionic-proposed 1:2.11+dfsg-1ubuntu7.4 amd64 [upgradable from: 1:2.11+dfsg-1ubuntu7.3] qemu-utils/bionic-proposed 1:2.11+dfsg-1ubuntu7.4 amd64 [upgradable from: 1:2.11+dfsg-1ubuntu7.3] Upgrade worked fine as expected. New types look good $ kvm -M ? | grep hpb pc-i440fx-bionic-hpb Ubuntu 18.04 PC (i440FX + PIIX, +host-phys-bits=true, 1996) pc-q35-bionic-hpbUbuntu 18.04 PC (Q35 + ICH9, +host-phys-bits=true, 2009) Starting the guest with that type and 1.4 TB works (also workes with smaller mem - tested 256G). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
SRU review for Bionic. This looks reasonable to me. I'm not so sure about the hardware enablement exception, but as a new feature it seems like something we'd want for an LTS anyway, and complies with all the SRU requirements under that exception. It's an entirely new machine type which isn't expected to impact anything existing. I'm a little concerned that the feature hasn't had much time to age in Cosmic, so I suggest a longer aging period in bionic-proposed than normal for quality reasons, since normally when we backport a feature it is already well established in the development release or upstream. ** Changed in: qemu (Ubuntu Bionic) Status: Triaged => Fix Committed ** Tags added: verification-needed verification-needed-bionic -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
- Final test build and test from PPA for target release - Added SRU Template - Uploaded to review/acceptance by the SRU Team - Pushed to package repositories as we consider it proposed Waiting on SRU Team now. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
** Description changed: + [Impact] + + * Qemu supports running guests >1TB but currently the virtualization +stack above has no good way to control that (e.g. libvirt/openstack). + + * Long term we'd want to see bug 1769053 (this is where all started) +implemented in libvirt and exploited by at least the major upper +stacks. But until then we want/need to make a way available that lets +users run huge guests. + + * Following the example of CentOS (but not switching the default for +better compatibility) we provide a machine type which has the host- +phys-bit setting of qemu default-on. Since upper layers of the virt +stack already can control machine types that allows most use cases to +work until the long term solution can take over down the road. + + [Test Case] + + * You can do this on swap, but it will be so slow that you will suffer. +So better get a machine with a lot of memory - if possible >1TB + + * Spawn a guest with uvtool and ensure that you are running fine + + * 'virsh edit' the guest and increase the memory consumption to >1TB + + * When this guest is started it works at first but later on +initialization from the guest will touch unreachable adressing bits and +crash - this will appear as hard freeze to the guest and as KVM +emulation failure in the host log of guest execution. + + * With the fix you can modify the default type (which should be pc- +i440fx-bionic) to use the -hpb suffix so it would be: + pc-i440fx-bionic-hpb +Use virsh edit to do so. + + * Start the guest again, now the guest will work (be aware that +initializing so much memory can take some time) + + * This can be tested from PPA build at https://launchpad.net/~ci-train- + ppa-service/+archive/ubuntu/3292 + + + [Regression Potential] + + * We were tempted to change the default on the normal type which would +have had the easiest "ease of use" but then huge regression potential. +We decided against it so we have a new type that people need to select, +this means two things: +1. no change => no regression for users of existing types +2. users using the new type might find new issues we have never seen + (tests are good thou) but that is not a regression, but a + theoretical set of issues with the new function. + + [Other Info] + + * The SRUability in my opinion falls under BOTH paragraphs from the SRU +page + + * For Long Term Support releases we regularly want to enable new +hardware. Such changes are appropriate provided that we can ensure not +to affect upgrades on existing hardware. For example, modaliases of +newly introduced drivers must not overlap with previously shipped +drivers. This also includes updating hardware description data such as +udev's keymaps, media-player-info, mobile broadband vendors, or PCI +vendor/product list updates. +=> This is ensuring that - as machines grow - Bionuc users + can put all their ram to good use in KVM guests. + + * For Long Term Support releases we sometimes want to introduce new +features. They must not change the behaviour on existing installations +(e. g. entirely new packages are usually fine). If existing software +needs to be modified to make use of the new feature, it must be +demonstrated that these changes are unintrusive, have a minimal +regression potential, and have been tested properly. To avoid +regressions on upgrade, any such feature must then also be added to any +newer supported Ubuntu release. Once a new feature/package has been +introduced, subsequent changes to it are subject to the usual +requirements of SRUs to avoid regressions. +=> This is exactly what we achieved by adding new types instead of + modifying an existing ones. + + + --- + Hi, on the discussions and tests around bug 1769053 it was identified to long term it would be nice to expose host-pyhs-bits and phys-bits through libvirt. - - That is what bug 1769053 and the related upstream BZ will continue to track. + That is what bug 1769053 and the related upstream BZ will continue to + track. But there are two shortcomings on this for now: 1. it takes way too long to get that done for people wanting to use this right now 2. even if it would be implemented in libvirt, the stacks above do not yet exploit the theoretical pyhs-bits related attributes. To get Ubuntu Users with huge systems a rather quick access to this feature including usability in higher parts of the virtualization stack we want to somewhat follow what RedHat/CentOs do at [1] There they just switch the default to "on" in the default machine type. This has some potential drawbacks if you want to migrate across different generations of machines, therefore we don't want it to be the default in Ubuntu. Instead we would prefer to add a type with a
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
** Also affects: qemu (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: seabios (Ubuntu Bionic) Importance: Undecided Status: New ** No longer affects: seabios (Ubuntu Bionic) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
This bug was fixed in the package qemu - 1:2.11+dfsg-1ubuntu11 --- qemu (1:2.11+dfsg-1ubuntu11) cosmic; urgency=medium * d/p/ubuntu/machine-type-hpb.patch: add -hpb machine type for host-phys-bits=true (LP: #1776189) - add an info about this change in debian/qemu-system-x86.NEWS -- Christian Ehrhardt Tue, 12 Jun 2018 09:01:00 +0200 ** Changed in: qemu (Ubuntu) Status: Triaged => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/347801 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
** Merge proposal linked: https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/347796 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
We did a short brainstorm/bikeshed sesssion on the name and decided to: - use "hpb" for host-phys-bits abbreviation as the former -large could be a lot of other things - acked on not start type proliferation in general (only in very special cases) - if possible use a "+" instead of "-" in the type - Add desc in text line for the type - add a doc or news entry along the change -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
Yep, now all things fit together. - My old (swap based) tests that worked without a change - working on real >1TB machines without a change. That said the change to seabios is really not needed. Setting that task to invalid. ** Changed in: seabios (Ubuntu) Status: Triaged => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1776189] Re: implement a machine type based control of host-phys-bits
RE: bikeshedding on the name; I don't think -large or -big maps well the the feature, which is more like -cpu host; ie, give the host cpu attributes to the guest vcpu. In this case, we're asking to have host physical addressing bits applied to the guest vcpus. So maybe -hpb for HostPhysicalBits? On Mon, Jun 11, 2018 at 8:57 AM, Christian Ehrhardt <1776...@bugs.launchpad.net> wrote: > As assumed the seabios change really IS implemented differently. > That means I have the todo to restest without those. > > -- > You received this bug notification because you are subscribed to qemu in > Ubuntu. > https://bugs.launchpad.net/bugs/1776189 > > Title: > implement a machine type based control of host-phys-bits > > To manage notifications about this bug go to: > https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
As assumed the seabios change really IS implemented differently. That means I have the todo to restest without those. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
FYI on the bug as I mentione dthis being controllable in higher stacks. Libvirt is via type="" attribute. Openstack can control this globally or per image - global: https://docs.openstack.org/mitaka/config-reference/compute/config-options.html - per image: https://docs.openstack.org/image-guide/image-metadata.html - implementation: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L4125 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
So far all tests work, but they are still ongoing. While that is testing I was submitting a request for discussion upstream why the change is not integrated there. See: https://mail.coreboot.org/pipermail/seabios/2018-June/012310.html I hope that the feedback will help to evaluate if we want to carry this Delta or if there are any reasons against it. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
I'll run less case-bound tests in the form of a full pre-upload regression test on these changes. Then if safe IMHO can start pushing this to Cosmic and from there consider Bionic SRU. Note: due to the fact that Cosmic has no "own" qemu version yet the change there will also just modify/extend the Bionic types. Only later on a qemu merge for Cosmic will add cosmic named types. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1776189] Re: implement a machine type based control of host-phys-bits
I already did some tests with these changes from ppa [1]. This also includes changes to seabios which seem safe for users with less physical bits and required for those huge guests to work properly. [1]: https://launchpad.net/~canonical-server/+archive/ubuntu/large-virt ** Also affects: seabios (Ubuntu) Importance: Undecided Status: New ** Changed in: seabios (Ubuntu) Status: New => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1776189 Title: implement a machine type based control of host-phys-bits To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1776189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
