Re: [SeaBIOS] [RFC PATCH v3 3/3] fw/pci: Allocate IGD stolen memory
On Sat, 13 Feb 2016 19:20:32 -0500 "Kevin O'Connor" wrote: > On Sat, Feb 13, 2016 at 01:57:09PM -0700, Alex Williamson wrote: > > On Sat, 13 Feb 2016 15:05:09 -0500 > > "Kevin O'Connor" wrote: > > > On Sat, Feb 13, 2016 at 11:51:51AM -0700, Alex Williamson wrote: > > > > On Sat, 13 Feb 2016 13:18:39 -0500 > > > > "Kevin O'Connor" wrote: > > > > > On Sat, Feb 13, 2016 at 08:12:09AM -0700, Alex Williamson wrote: > > > > > > On Fri, 12 Feb 2016 21:49:04 -0500 > > > > > > "Kevin O'Connor" wrote: > > > > > > > On Fri, Feb 12, 2016 at 05:23:18PM -0700, Alex Williamson wrote: > > > > > > > > > > > > > > > Intel IGD makes use of memory allocated and marked reserved by > > > > > > > > the > > > > > > > > BIOS as a stolen memory range. For the most part, guest > > > > > > > > drivers don't > > > > > > > > make use of this, but our achilles heel is the vBIOS. The vBIOS > > > > > > > > programs the device to use the host stolen memory range and > > > > > > > > it's used > > > > > > > > in the pre-boot environment. Generally the guest won't have > > > > > > > > access to > > > > > > > > the host stolen memory area, so these accesses either land in VM > > > > > > > > memory or unassigned space and generate IOMMU faults. By > > > > > > > > allocating > > > > > > > > this range in SeaBIOS and programming it into the device, QEMU > > > > > > > > (via > > > > > > > > vfio) can make sure this guest allocated stolen memory range is > > > > > > > > used > > > > > > > > instead. > > > > > > > > > > > > > > What does "vBIOS" mean in this context? Is it the video device > > > > > > > option > > > > > > > rom or something else? > > > > > > > > > > > > vBIOS = video BIOS, you're correct, it's just the video device > > > > > > option > > > > > > ROM. > > > > > > > > > > Is the problem from when the host runs the video option rom, or is the > > > > > problem from the guest (via SeaBIOS) running the video option rom? If > > > > > the guest is running the option rom, is it the first time the option > > > > > rom has been run for the machine (ie, was the option rom not executed > > > > > on the host when the host machine first booted)? > > > > > > > > > > FWIW, many of the chromebooks use coreboot with Intel graphics and a > > > > > number of users have installed SeaBIOS (running natively) on their > > > > > machines. Running the intel video option rom more than once has been > > > > > known to cause issues. > > > > > > > > The issue is in the VM and it occurs every time the option ROM is > > > > executed. Standard VGA text mode displays fine (ex. SeaBIOS version > > > > string and boot menu), but any sort of extended graphics mode (ex. live > > > > CD boot menu) tries to make use of the host memory area which > > > > corresponds to the stolen memory area of the physical device. We're > > > > not really sure how the ROM execution arrives at these addresses (not > > > > from the device according to access traces), but we can see when the > > > > ROM is writing these addresses to the device and modify they addresses > > > > to point at a VM memory range which we've allocated. That's what this > > > > code attempts to do, allocate a buffer and tell QEMU about it via the > > > > BDSM (Base Data of Stolen Memory) register. > > > > > > Forgive me if I'm not fully understanding this. If I read what you're > > > saying then the sequence is something like: > > > > > > 1 - the host system bios (or vgabios) programs the GTT/stolen memory > > > base register at host system bootup time and reserves it in the > > > host e820 map. > > > > > > 2 - upon running qemu, the guest reruns the vga bios option rom which > > > seems to work (ie, text mode works) > > > > > > 3 - in the guest, upon running a bootloader that uses graphics mode, > > > the bootloader calls the vgabios to switch to graphics mode, and > > > the vgabios sends commands to the graphics hardware that somehow > > > reference the host stolen memory > > > > What exactly happens here isn't clear to me, but this is a plausible > > explanation. What we see in tracing access to the hardware is that a > > bunch of addresses are written to the device that fall within the host > > e820 reserved area and then the device starts generating IOMMU faults > > trying to access those addresses. > > > > > 4 - your patch causes QEMU to catch these commands with references to > > > the host stolen memory and replace them with references to the > > > guest stolen memory (which seabios creates) > > > > > > Am I understanding the above correctly? > > > > Yes. > > > > > Is the only reason to run the intel option rom in the guest for > > > bootloader graphic mode support? Last time I looked, the intel vga > > > hardware could fully emulate a legacy vga device - if the device is in > > > vga compatibility mode then it may be possible to have seavgabios > > > drive mode changes. >
Re: [SeaBIOS] [RFC PATCH v3 3/3] fw/pci: Allocate IGD stolen memory
On Sat, Feb 13, 2016 at 01:57:09PM -0700, Alex Williamson wrote: > On Sat, 13 Feb 2016 15:05:09 -0500 > "Kevin O'Connor" wrote: > > On Sat, Feb 13, 2016 at 11:51:51AM -0700, Alex Williamson wrote: > > > On Sat, 13 Feb 2016 13:18:39 -0500 > > > "Kevin O'Connor" wrote: > > > > On Sat, Feb 13, 2016 at 08:12:09AM -0700, Alex Williamson wrote: > > > > > On Fri, 12 Feb 2016 21:49:04 -0500 > > > > > "Kevin O'Connor" wrote: > > > > > > On Fri, Feb 12, 2016 at 05:23:18PM -0700, Alex Williamson wrote: > > > > > > > Intel IGD makes use of memory allocated and marked reserved by the > > > > > > > BIOS as a stolen memory range. For the most part, guest drivers > > > > > > > don't > > > > > > > make use of this, but our achilles heel is the vBIOS. The vBIOS > > > > > > > programs the device to use the host stolen memory range and it's > > > > > > > used > > > > > > > in the pre-boot environment. Generally the guest won't have > > > > > > > access to > > > > > > > the host stolen memory area, so these accesses either land in VM > > > > > > > memory or unassigned space and generate IOMMU faults. By > > > > > > > allocating > > > > > > > this range in SeaBIOS and programming it into the device, QEMU > > > > > > > (via > > > > > > > vfio) can make sure this guest allocated stolen memory range is > > > > > > > used > > > > > > > instead. > > > > > > > > > > > > What does "vBIOS" mean in this context? Is it the video device > > > > > > option > > > > > > rom or something else? > > > > > > > > > > vBIOS = video BIOS, you're correct, it's just the video device option > > > > > ROM. > > > > > > > > Is the problem from when the host runs the video option rom, or is the > > > > problem from the guest (via SeaBIOS) running the video option rom? If > > > > the guest is running the option rom, is it the first time the option > > > > rom has been run for the machine (ie, was the option rom not executed > > > > on the host when the host machine first booted)? > > > > > > > > FWIW, many of the chromebooks use coreboot with Intel graphics and a > > > > number of users have installed SeaBIOS (running natively) on their > > > > machines. Running the intel video option rom more than once has been > > > > known to cause issues. > > > > > > The issue is in the VM and it occurs every time the option ROM is > > > executed. Standard VGA text mode displays fine (ex. SeaBIOS version > > > string and boot menu), but any sort of extended graphics mode (ex. live > > > CD boot menu) tries to make use of the host memory area which > > > corresponds to the stolen memory area of the physical device. We're > > > not really sure how the ROM execution arrives at these addresses (not > > > from the device according to access traces), but we can see when the > > > ROM is writing these addresses to the device and modify they addresses > > > to point at a VM memory range which we've allocated. That's what this > > > code attempts to do, allocate a buffer and tell QEMU about it via the > > > BDSM (Base Data of Stolen Memory) register. > > > > Forgive me if I'm not fully understanding this. If I read what you're > > saying then the sequence is something like: > > > > 1 - the host system bios (or vgabios) programs the GTT/stolen memory > > base register at host system bootup time and reserves it in the > > host e820 map. > > > > 2 - upon running qemu, the guest reruns the vga bios option rom which > > seems to work (ie, text mode works) > > > > 3 - in the guest, upon running a bootloader that uses graphics mode, > > the bootloader calls the vgabios to switch to graphics mode, and > > the vgabios sends commands to the graphics hardware that somehow > > reference the host stolen memory > > What exactly happens here isn't clear to me, but this is a plausible > explanation. What we see in tracing access to the hardware is that a > bunch of addresses are written to the device that fall within the host > e820 reserved area and then the device starts generating IOMMU faults > trying to access those addresses. > > > 4 - your patch causes QEMU to catch these commands with references to > > the host stolen memory and replace them with references to the > > guest stolen memory (which seabios creates) > > > > Am I understanding the above correctly? > > Yes. > > > Is the only reason to run the intel option rom in the guest for > > bootloader graphic mode support? Last time I looked, the intel vga > > hardware could fully emulate a legacy vga device - if the device is in > > vga compatibility mode then it may be possible to have seavgabios > > drive mode changes. > > I have a SandyBridge based laptop (Lenovo W520) where the LCD panel > won't turn on without the vBIOS. This confuses me - why didn't the host system BIOS turn on the LCD panel during host bootup? >Another desktop IvyBridge system > doesn't really care about the vBIOS so long as we don't ask it to > output anything
Re: [SeaBIOS] [RFC PATCH v3 3/3] fw/pci: Allocate IGD stolen memory
On Sat, 13 Feb 2016 15:05:09 -0500 "Kevin O'Connor" wrote: > On Sat, Feb 13, 2016 at 11:51:51AM -0700, Alex Williamson wrote: > > On Sat, 13 Feb 2016 13:18:39 -0500 > > "Kevin O'Connor" wrote: > > > On Sat, Feb 13, 2016 at 08:12:09AM -0700, Alex Williamson wrote: > > > > On Fri, 12 Feb 2016 21:49:04 -0500 > > > > "Kevin O'Connor" wrote: > > > > > On Fri, Feb 12, 2016 at 05:23:18PM -0700, Alex Williamson wrote: > > > > > > Intel IGD makes use of memory allocated and marked reserved by the > > > > > > BIOS as a stolen memory range. For the most part, guest drivers > > > > > > don't > > > > > > make use of this, but our achilles heel is the vBIOS. The vBIOS > > > > > > programs the device to use the host stolen memory range and it's > > > > > > used > > > > > > in the pre-boot environment. Generally the guest won't have access > > > > > > to > > > > > > the host stolen memory area, so these accesses either land in VM > > > > > > memory or unassigned space and generate IOMMU faults. By allocating > > > > > > this range in SeaBIOS and programming it into the device, QEMU (via > > > > > > vfio) can make sure this guest allocated stolen memory range is used > > > > > > instead. > > > > > > > > > > What does "vBIOS" mean in this context? Is it the video device option > > > > > rom or something else? > > > > > > > > vBIOS = video BIOS, you're correct, it's just the video device option > > > > ROM. > > > > > > Is the problem from when the host runs the video option rom, or is the > > > problem from the guest (via SeaBIOS) running the video option rom? If > > > the guest is running the option rom, is it the first time the option > > > rom has been run for the machine (ie, was the option rom not executed > > > on the host when the host machine first booted)? > > > > > > FWIW, many of the chromebooks use coreboot with Intel graphics and a > > > number of users have installed SeaBIOS (running natively) on their > > > machines. Running the intel video option rom more than once has been > > > known to cause issues. > > > > The issue is in the VM and it occurs every time the option ROM is > > executed. Standard VGA text mode displays fine (ex. SeaBIOS version > > string and boot menu), but any sort of extended graphics mode (ex. live > > CD boot menu) tries to make use of the host memory area which > > corresponds to the stolen memory area of the physical device. We're > > not really sure how the ROM execution arrives at these addresses (not > > from the device according to access traces), but we can see when the > > ROM is writing these addresses to the device and modify they addresses > > to point at a VM memory range which we've allocated. That's what this > > code attempts to do, allocate a buffer and tell QEMU about it via the > > BDSM (Base Data of Stolen Memory) register. > > Forgive me if I'm not fully understanding this. If I read what you're > saying then the sequence is something like: > > 1 - the host system bios (or vgabios) programs the GTT/stolen memory > base register at host system bootup time and reserves it in the > host e820 map. > > 2 - upon running qemu, the guest reruns the vga bios option rom which > seems to work (ie, text mode works) > > 3 - in the guest, upon running a bootloader that uses graphics mode, > the bootloader calls the vgabios to switch to graphics mode, and > the vgabios sends commands to the graphics hardware that somehow > reference the host stolen memory What exactly happens here isn't clear to me, but this is a plausible explanation. What we see in tracing access to the hardware is that a bunch of addresses are written to the device that fall within the host e820 reserved area and then the device starts generating IOMMU faults trying to access those addresses. > 4 - your patch causes QEMU to catch these commands with references to > the host stolen memory and replace them with references to the > guest stolen memory (which seabios creates) > > Am I understanding the above correctly? Yes. > Is the only reason to run the intel option rom in the guest for > bootloader graphic mode support? Last time I looked, the intel vga > hardware could fully emulate a legacy vga device - if the device is in > vga compatibility mode then it may be possible to have seavgabios > drive mode changes. I have a SandyBridge based laptop (Lenovo W520) where the LCD panel won't turn on without the vBIOS. Another desktop IvyBridge system doesn't really care about the vBIOS so long as we don't ask it to output anything before the guest native drivers are loaded. If we could, I think we'd just enable vBIOS for laptop panel support, but that's really not an option, it's going to run as a boot option ROM as well, so we need to fix the issues that it generates there. > > > [...] > > > > The write to 0x5C is used by QEMU code that traps writes to the > > > > device I/O port BAR and replaces the host stolen memo
Re: [SeaBIOS] [RFC PATCH v3 3/3] fw/pci: Allocate IGD stolen memory
On Sat, Feb 13, 2016 at 11:51:51AM -0700, Alex Williamson wrote: > On Sat, 13 Feb 2016 13:18:39 -0500 > "Kevin O'Connor" wrote: > > On Sat, Feb 13, 2016 at 08:12:09AM -0700, Alex Williamson wrote: > > > On Fri, 12 Feb 2016 21:49:04 -0500 > > > "Kevin O'Connor" wrote: > > > > On Fri, Feb 12, 2016 at 05:23:18PM -0700, Alex Williamson wrote: > > > > > Intel IGD makes use of memory allocated and marked reserved by the > > > > > BIOS as a stolen memory range. For the most part, guest drivers don't > > > > > make use of this, but our achilles heel is the vBIOS. The vBIOS > > > > > programs the device to use the host stolen memory range and it's used > > > > > in the pre-boot environment. Generally the guest won't have access to > > > > > the host stolen memory area, so these accesses either land in VM > > > > > memory or unassigned space and generate IOMMU faults. By allocating > > > > > this range in SeaBIOS and programming it into the device, QEMU (via > > > > > vfio) can make sure this guest allocated stolen memory range is used > > > > > instead. > > > > > > > > What does "vBIOS" mean in this context? Is it the video device option > > > > rom or something else? > > > > > > vBIOS = video BIOS, you're correct, it's just the video device option > > > ROM. > > > > Is the problem from when the host runs the video option rom, or is the > > problem from the guest (via SeaBIOS) running the video option rom? If > > the guest is running the option rom, is it the first time the option > > rom has been run for the machine (ie, was the option rom not executed > > on the host when the host machine first booted)? > > > > FWIW, many of the chromebooks use coreboot with Intel graphics and a > > number of users have installed SeaBIOS (running natively) on their > > machines. Running the intel video option rom more than once has been > > known to cause issues. > > The issue is in the VM and it occurs every time the option ROM is > executed. Standard VGA text mode displays fine (ex. SeaBIOS version > string and boot menu), but any sort of extended graphics mode (ex. live > CD boot menu) tries to make use of the host memory area which > corresponds to the stolen memory area of the physical device. We're > not really sure how the ROM execution arrives at these addresses (not > from the device according to access traces), but we can see when the > ROM is writing these addresses to the device and modify they addresses > to point at a VM memory range which we've allocated. That's what this > code attempts to do, allocate a buffer and tell QEMU about it via the > BDSM (Base Data of Stolen Memory) register. Forgive me if I'm not fully understanding this. If I read what you're saying then the sequence is something like: 1 - the host system bios (or vgabios) programs the GTT/stolen memory base register at host system bootup time and reserves it in the host e820 map. 2 - upon running qemu, the guest reruns the vga bios option rom which seems to work (ie, text mode works) 3 - in the guest, upon running a bootloader that uses graphics mode, the bootloader calls the vgabios to switch to graphics mode, and the vgabios sends commands to the graphics hardware that somehow reference the host stolen memory 4 - your patch causes QEMU to catch these commands with references to the host stolen memory and replace them with references to the guest stolen memory (which seabios creates) Am I understanding the above correctly? Is the only reason to run the intel option rom in the guest for bootloader graphic mode support? Last time I looked, the intel vga hardware could fully emulate a legacy vga device - if the device is in vga compatibility mode then it may be possible to have seavgabios drive mode changes. > > [...] > > > The write to 0x5C is used by QEMU code that traps writes to the > > > device I/O port BAR and replaces the host stolen memory address > > > with the new guest address generated here. 0x5C is initialized to > > > 0x0 by kernel vfio code, so we can detect whether it has been > > > written. If not written, QEMU has no place to redirect to for > > > stolen memory and it will either overlap VM memory or an unassigned > > > area. The former may corrupt VM memory, the latter throws host > > > errors. We could in QEMU halt with a hardware error if 0x5C hasn't > > > been programmed. > > > > So, if I understand correctly, 0x5C is not a "real" register on the > > hardware, but is instead just a mechanism to give QEMU the address of > > some guest visible ram? > > It is a real register, BDSM that is virtualized by vfio turning it > effectively into a scratch register. On physical hardware the > register is read-only. > > > BTW, is 0xFC a "real" register in the hardware? How does the guest > > find the location of the "OpRegion" if SeaBIOS allocates it? > > 0xFC is the ASL Storage register, the guest finds the location of the > OpRegion using this register. This
Re: [SeaBIOS] [RFC PATCH v3 3/3] fw/pci: Allocate IGD stolen memory
On Sat, 13 Feb 2016 13:18:39 -0500 "Kevin O'Connor" wrote: > On Sat, Feb 13, 2016 at 08:12:09AM -0700, Alex Williamson wrote: > > On Fri, 12 Feb 2016 21:49:04 -0500 > > "Kevin O'Connor" wrote: > > > On Fri, Feb 12, 2016 at 05:23:18PM -0700, Alex Williamson wrote: > > > > Intel IGD makes use of memory allocated and marked reserved by the > > > > BIOS as a stolen memory range. For the most part, guest drivers don't > > > > make use of this, but our achilles heel is the vBIOS. The vBIOS > > > > programs the device to use the host stolen memory range and it's used > > > > in the pre-boot environment. Generally the guest won't have access to > > > > the host stolen memory area, so these accesses either land in VM > > > > memory or unassigned space and generate IOMMU faults. By allocating > > > > this range in SeaBIOS and programming it into the device, QEMU (via > > > > vfio) can make sure this guest allocated stolen memory range is used > > > > instead. > > > > > > What does "vBIOS" mean in this context? Is it the video device option > > > rom or something else? > > > > vBIOS = video BIOS, you're correct, it's just the video device option > > ROM. > > Is the problem from when the host runs the video option rom, or is the > problem from the guest (via SeaBIOS) running the video option rom? If > the guest is running the option rom, is it the first time the option > rom has been run for the machine (ie, was the option rom not executed > on the host when the host machine first booted)? > > FWIW, many of the chromebooks use coreboot with Intel graphics and a > number of users have installed SeaBIOS (running natively) on their > machines. Running the intel video option rom more than once has been > known to cause issues. The issue is in the VM and it occurs every time the option ROM is executed. Standard VGA text mode displays fine (ex. SeaBIOS version string and boot menu), but any sort of extended graphics mode (ex. live CD boot menu) tries to make use of the host memory area which corresponds to the stolen memory area of the physical device. We're not really sure how the ROM execution arrives at these addresses (not from the device according to access traces), but we can see when the ROM is writing these addresses to the device and modify they addresses to point at a VM memory range which we've allocated. That's what this code attempts to do, allocate a buffer and tell QEMU about it via the BDSM (Base Data of Stolen Memory) register. > [...] > > The write to 0x5C is used by QEMU code that traps writes to the > > device I/O port BAR and replaces the host stolen memory address > > with the new guest address generated here. 0x5C is initialized to > > 0x0 by kernel vfio code, so we can detect whether it has been > > written. If not written, QEMU has no place to redirect to for > > stolen memory and it will either overlap VM memory or an unassigned > > area. The former may corrupt VM memory, the latter throws host > > errors. We could in QEMU halt with a hardware error if 0x5C hasn't > > been programmed. > > So, if I understand correctly, 0x5C is not a "real" register on the > hardware, but is instead just a mechanism to give QEMU the address of > some guest visible ram? It is a real register, BDSM that is virtualized by vfio turning it effectively into a scratch register. On physical hardware the register is read-only. > BTW, is 0xFC a "real" register in the hardware? How does the guest > find the location of the "OpRegion" if SeaBIOS allocates it? 0xFC is the ASL Storage register, the guest finds the location of the OpRegion using this register. This is another register that is read-only on real hardware but virtualized through vfio so we can relocate the OpRegion into the VM address space. I've found that allocating a dummy MMIO BAR does work as an alternative for mapping space for this stolen memory into the VM address space. For a Linux guest it works to allocate BAR5 on the IGD device. Windows10 is not so happy with this, but does work if I allocate the BAR on something like the ISA bridge device. My guess is that the IGD driver in Windows freaks out at finding this strange new BAR on its device. So I'll need to come up with an algorithm for either creating a dummy PCI device to host this BAR or trying to add it to other existing devices. It's certainly a more self-contained solution this way, so I expect we'll only need patch 1/3 from this series. Thanks, Alex ___ SeaBIOS mailing list SeaBIOS@seabios.org http://www.seabios.org/mailman/listinfo/seabios
Re: [SeaBIOS] [RFC PATCH v3 3/3] fw/pci: Allocate IGD stolen memory
On Sat, Feb 13, 2016 at 08:12:09AM -0700, Alex Williamson wrote: > On Fri, 12 Feb 2016 21:49:04 -0500 > "Kevin O'Connor" wrote: > > On Fri, Feb 12, 2016 at 05:23:18PM -0700, Alex Williamson wrote: > > > Intel IGD makes use of memory allocated and marked reserved by the > > > BIOS as a stolen memory range. For the most part, guest drivers don't > > > make use of this, but our achilles heel is the vBIOS. The vBIOS > > > programs the device to use the host stolen memory range and it's used > > > in the pre-boot environment. Generally the guest won't have access to > > > the host stolen memory area, so these accesses either land in VM > > > memory or unassigned space and generate IOMMU faults. By allocating > > > this range in SeaBIOS and programming it into the device, QEMU (via > > > vfio) can make sure this guest allocated stolen memory range is used > > > instead. > > > > What does "vBIOS" mean in this context? Is it the video device option > > rom or something else? > > vBIOS = video BIOS, you're correct, it's just the video device option > ROM. Is the problem from when the host runs the video option rom, or is the problem from the guest (via SeaBIOS) running the video option rom? If the guest is running the option rom, is it the first time the option rom has been run for the machine (ie, was the option rom not executed on the host when the host machine first booted)? FWIW, many of the chromebooks use coreboot with Intel graphics and a number of users have installed SeaBIOS (running natively) on their machines. Running the intel video option rom more than once has been known to cause issues. [...] > The write to 0x5C is used by QEMU code that traps writes to the device > I/O port BAR and replaces the host stolen memory address with the new > guest address generated here. 0x5C is initialized to 0x0 by kernel > vfio code, so we can detect whether it has been written. If not > written, QEMU has no place to redirect to for stolen memory and it will > either overlap VM memory or an unassigned area. The former may corrupt > VM memory, the latter throws host errors. We could in QEMU halt with a > hardware error if 0x5C hasn't been programmed. So, if I understand correctly, 0x5C is not a "real" register on the hardware, but is instead just a mechanism to give QEMU the address of some guest visible ram? BTW, is 0xFC a "real" register in the hardware? How does the guest find the location of the "OpRegion" if SeaBIOS allocates it? -Kevin ___ SeaBIOS mailing list SeaBIOS@seabios.org http://www.seabios.org/mailman/listinfo/seabios
Re: [SeaBIOS] [RFC PATCH v3 3/3] fw/pci: Allocate IGD stolen memory
Hi Kevin,
On Fri, 12 Feb 2016 21:49:04 -0500
"Kevin O'Connor" wrote:
> On Fri, Feb 12, 2016 at 05:23:18PM -0700, Alex Williamson wrote:
> > Intel IGD makes use of memory allocated and marked reserved by the
> > BIOS as a stolen memory range. For the most part, guest drivers don't
> > make use of this, but our achilles heel is the vBIOS. The vBIOS
> > programs the device to use the host stolen memory range and it's used
> > in the pre-boot environment. Generally the guest won't have access to
> > the host stolen memory area, so these accesses either land in VM
> > memory or unassigned space and generate IOMMU faults. By allocating
> > this range in SeaBIOS and programming it into the device, QEMU (via
> > vfio) can make sure this guest allocated stolen memory range is used
> > instead.
>
> What does "vBIOS" mean in this context? Is it the video device option
> rom or something else?
vBIOS = video BIOS, you're correct, it's just the video device option
ROM.
> >
> > Signed-off-by: Alex Williamson
> > ---
> > src/fw/pciinit.c | 13 -
> > 1 file changed, 12 insertions(+), 1 deletion(-)
> >
> > diff --git a/src/fw/pciinit.c b/src/fw/pciinit.c
> > index 92170d5..c1ad5d4 100644
> > --- a/src/fw/pciinit.c
> > +++ b/src/fw/pciinit.c
> > @@ -260,7 +260,7 @@ static void ich9_smbus_setup(struct pci_device *dev,
> > void *arg)
> > static void intel_igd_opregion_setup(struct pci_device *dev, void *arg)
> > {
> > struct romfile_s *file = romfile_find("etc/igd-opregion");
> > -void *opregion;
> > +void *opregion, *bdsm;
> > u16 bdf = dev->bdf;
> >
> > if (!file || !file->size)
> > @@ -281,6 +281,17 @@ static void intel_igd_opregion_setup(struct pci_device
> > *dev, void *arg)
> >
> > dprintf(1, "Intel IGD OpRegion enabled on %02x:%02x.%x\n",
> > pci_bdf_to_bus(bdf), pci_bdf_to_dev(bdf), pci_bdf_to_fn(bdf));
> > +
> > +bdsm = memalign_high(1024 * 1024, 1024 * 1024);
> > +if (!bdsm) {
> > +warn_noalloc();
> > +return;
> > +}
>
> The "high" memory pool is not a good fit for such a large allocation.
> For so much space, I'd use memalign_tmphigh() followed by
> e820_add(addr, size, E820_RESERVED).
Ok, as you saw on IRC I was looking for alternatives, I can easily
switch to this.
> > +pci_config_writel(bdf, 0x5C, cpu_to_le32((u32)bdsm));
> > +
> > +dprintf(1, "Allocated 1MB reserved memory for Intel IGD stolen memory
> > at "
> > +"0x%08x\n", (u32)bdsm);
> > }
>
> Does this make sense to do unconditionally in the firmware whenever
> the device is present? The SeaBIOS release schedule is not in sync
> with QEMU, so changes in the firmware tend to take longer to deploy if
> something needs to be done differently in the future.
>
> What happens if this register is not set? Does anything go wrong if
> register 0xFC is set, but 0x5C is not (eg, due to allocation failure).
It's not entirely unconditional, it's tagged onto the end of the
OpRegion setup, so we know that we're at least dealing with a QEMU that
has that support. We could link it to the PCI ROM BAR for IGD being
present since this is necessary to support the code in that ROM, but
that doesn't give us any idea if the QEMU support is there. Maybe we
could do both, OpRegion support and ROM BAR present.
The write to 0x5C is used by QEMU code that traps writes to the device
I/O port BAR and replaces the host stolen memory address with the new
guest address generated here. 0x5C is initialized to 0x0 by kernel
vfio code, so we can detect whether it has been written. If not
written, QEMU has no place to redirect to for stolen memory and it will
either overlap VM memory or an unassigned area. The former may corrupt
VM memory, the latter throws host errors. We could in QEMU halt with a
hardware error if 0x5C hasn't been programmed.
One alternative I was considering was to simply use the unused BAR5
slot on IGD to expose a fake 1MB, 32bit MMIO BAR. SeaBIOS would program
this normally, QEMU would back it with its own allocation and we could
easily find the address of it when we need it. The downside being that
the PCI BAR can be disabled and remapped, but for the small window
where we need it, I'm not sure that matters. It would certainly be
more transparent to the VM BIOS. It would also make it much easier to
change the size if we found some devices need more/less/none space. I
initially thought the fake BAR wasn't a great fix, but maybe its
advantages outweigh its downsides. I'll give it a shot. Thanks,
Alex
___
SeaBIOS mailing list
SeaBIOS@seabios.org
http://www.seabios.org/mailman/listinfo/seabios
