On 06/30/2016 08:07 AM, Laszlo Ersek wrote:
Hi,
does anyone on this list have experience with $SUBJECT, using physical
UEFI firmware derived from edk2?
The problem we're seeing with OVMF is the following: PCIe hotplug works
with PCIe root ports, but it doesn't work with PCIe downstream ports.
The error messages reported by both Linux and Windows guests indicate
"lack of resources" for the device being plugged.
In my understanding:
* Each PCIe downstream port qualifies as a separate bus / bridge.
* Bridges need IO and MMIO resources for themselves. When a device is
behind a bridge at boot, the device's IO and MMIO BARs are carved out
of the bridge's IO and MMIO apertures. For IO, the standard bridge
aperture size is 4096 (byte-sized) ports.
* If no device behind a bridge needs a specific type of resource (for
example, no devices behind a bridge have IO BARs), then that kind of
aperture is not necessary for the bridge either.
A special case of this is when there are no devices behind a bridge
at all. Example: PCIe downstream port empty at boot. In this case,
the bridge needs no resources at all, at boot.
* Different firmwares seem to follow different policies in the last
case (-> bridge empty at boot). SeaBIOS for example allocates the
0x1000 IO ports nonetheless. The PCI bus driver in edk2 does not: it
allocates no apertures for bridges that have no devices behind them
at boot.
This appears to be a trade-off: if the bridge remains empty for the
entire lifetime of the OS, then any IO ports that have been allocated
in advance will be in vain (wasted). However, if the aperture is not
allocated at boot, then hotplug under the bridge may not work --
resources for the device being hot-plugged cannot be allocated from
nonexistent bridge apertures.
And that's what we are experiencing with OVMF.
Laszlo,
I believe that's an accurate summary of PCIe behavior.
My questions are:
- Is the above behavior of the edk2 PCI bus driver intentional?
I'm not familiar with the history of the code.... But any PCI bridge
which wants to support hotplug needs resources allocated to it at boot
time. As you say, that's a trade-off between wasting resources
(especially I/O ports) and allowing hotplug. The authors of the code
must have opted for avoiding waste vs. allowing hotplug.
- Does PCIe hotplug into downstream ports work with any (phys) firmware
forked from edk2? Brian, Samer, do you guys have experience with this?
Yes, we support it on our UV server line. Physically, hotplug is only
possible in expansion chassis designed for that purpose. (That's not a
problem with virtual machines, of course.) But we've had to do quite a
bit of work on the host bridge driver to allowing scaling to the sizes
of systems we support (hundreds of sockets and PCIe slots, dozens of TB
of RAM), including reserving resources for hotplug only in the specific
cases where it's possible.
- What could be the difference between root ports and downstream ports?
(Hotplug into root ports seems to work fine.) Are OSes entitled to
allocate any unused address space (MMIO and IO) right when a device is
hot-plugged into a root port?
I'm not sure what the OS rules for this are....
--
Brian
--------------------------------------------------------------------
"The national budget must be balanced. The public debt must be
reduced; the arrogance of the authorities must be moderated and
controlled. Payments to foreign governments must be reduced, if
the nation doesn't want to go bankrupt. People must again learn
to work, instead of living on public assistance."
-- Marcus Tullius Cicero, 55 BC
_______________________________________________
edk2-devel mailing list
edk2-devel@lists.01.org
https://lists.01.org/mailman/listinfo/edk2-devel
