On Sun, 19 Jun 2016 19:13:17 +0300 Marcel Apfelbaum <mar...@redhat.com> wrote:
> On 06/17/2016 07:07 PM, Laszlo Ersek wrote: > > On 06/17/16 11:52, Igor Mammedov wrote: > >> On Fri, 17 Jun 2016 11:17:54 +0200 > >> Gerd Hoffmann <kra...@redhat.com> wrote: > >> > >>> On Fr, 2016-06-17 at 10:43 +0200, Paolo Bonzini wrote: > >>>> > >>>> On 17/06/2016 10:15, Dr. David Alan Gilbert wrote: > >>>>> Larger is a problem if the guest tries to map something to a > >>>>> high address that's not addressable. > >>>> > >>>> Right. It's not a problem for most emulated PCI devices (it > >>>> would be a problem for those that have large RAM BARs, but even > >>>> our emulated video cards do not have 64-bit RAM BARs, I think; > >>> > >>> qxl can be configured to have one, try "-device > >>> qxl-vga,vram64_size_mb=1024" > >>> > >>>>> 2) While we have maxmem settings to tell us the top of VM > >>>>> RAM, do we have anything that tells us the top of IO space? > >>>>> What happens when we hotplug a PCI card? > >>> > >>>> (arch/x86/kernel/setup.c) but I agree that (2) is a blocker. > >>> > >>> seabios maps stuff right above ram (possibly with a hole due to > >>> alignment requirements). > >>> > >>> ovmf maps stuff into a 32G-aligned 32G hole. Which lands at 32G > >>> and therefore is addressable with 36 bits, unless you have tons > >>> of ram (> 30G) assigned to your guest. A physical host machine > >>> where you can plug in enough ram for such a configuration likely > >>> has more than 36 physical address lines too ... > >>> > >>> qemu checks where the firmware mapped 64bit bars, then adds those > >>> ranges to the root bus pci resources in the acpi tables > >>> (see /proc/iomem). > >>> > >>>> You don't know how the guest will assign PCI BAR addresses, and > >>>> as you said there's hotplug too. > >>> > >>> Not sure whenever qemu adds some extra space for hotplug to the > >>> 64bit hole and if so how it calculates the size then. But the > >>> guest os should stick to those ranges when configuring hotplugged > >>> devices. > >> currently firmware would assign 64-bit BARs after > >> reserved-memory-end (not sure about ovmf though) > > > > OVMF does the same as well. It makes sure that the 64-bit PCI MMIO > > aperture is located above "etc/reserved-memory-end", if the latter > > exists. > > > >> but QEMU on ACPI side will add 64-bit _CRS only > >> for firmware mapped devices (i.e. no space reserved for hotplug). > >> And is I recall correctly ovmf won't map BARs if it doesn't have > >> a driver for it > > > > Yes, that's correct, generally for all UEFI firmware. > > > > More precisely, BARs will be allocated and programmed, but the MMIO > > space decoding bit will not be set (permanently) in the device's > > command register, if there is no matching driver in the firmware > > (or in the device's own oprom). > > > >> so ACPI tables won't even have a space for not mapped > >> 64-bit BARs. > > > > This used to be true, but that's not the case since > > <https://github.com/tianocore/edk2/commit/8f35eb92c419>. > > > > Namely, specifically for conforming to QEMU's ACPI generator, OVMF > > *temporarily* enables, as a platform quirk, all PCI devices present > > in the system, before triggering QEMU to generate the ACPI payload. > > > > Thus, nowadays 64-bit BARs work fine with OVMF, both for > > virtio-modern devices, and assigned physical devices. (This is very > > easy to test, because, unlike SeaBIOS, the edk2 stuff built into > > OVMF prefers to allocate 64-bit BARs outside of the 32-bit address > > space.) > > > > Devices behind PXBs are a different story, but Marcel's been looking > > into that, see > > <https://bugzilla.redhat.com/show_bug.cgi?id=1323976>. > > > >> There was another attempt to reserve more space in _CRS > >> https://lists.nongnu.org/archive/html/qemu-devel/2016-05/msg00090.html > > > > That's actually Marcel's first own patch set for addressing > > RHBZ#1323976 that I mentioned above (see it linked in > > <https://bugzilla.redhat.com/show_bug.cgi?id=1323976#c2>). > > > > It might have wider effects, but it is entirely motivated, to my > > knowledge, by PXB. If you don't have extra root bridges, and/or you > > plug all your devices with 64-bit MMIO BARs into the > > "main" (default) root bridge, then (I believe) that patch set is > > not supposed to make any difference. (I could be wrong, it's been a > > while since I looked at Marcel's work!) > > > > Patch 3 and 4 indeed are for PXB only. but patch 'pci: reserve 64 bit > MMIO range for PCI hotplug' (see > https://lists.nongnu.org/archive/html/qemu-devel/2016-05/msg00091.html) > tries to reserve [above_4g_mem_size, max_addressable_cpu_bits] range > for PCI hotplug. it should be [reserved-memory-end, max_addressable_cpu_bits] > > The implementation is not good enough because the number of > addressable bits is hard-coded. However, we have now David's wrapper > I can use. > > > Thanks, > Marcel > > > > > > > > > Thanks > > Laszlo > > >
