On Tue, Jan 14, 2020 at 09:41:46AM -0500, Jason Andryuk wrote: > On Tue, Jan 14, 2020 at 5:04 AM Roger Pau Monné <roger....@citrix.com> wrote: > > > > On Mon, Jan 13, 2020 at 02:01:47PM -0500, Jason Andryuk wrote: > > > On Fri, Mar 22, 2019 at 3:43 PM Jason Andryuk <jandr...@gmail.com> wrote: > > > > > > > > On Thu, Mar 21, 2019 at 11:09 PM Roger Pau Monné <roger....@citrix.com> > > > > wrote: > > > > > > > > > > The patch below should prevent hvmloader from placing multiple BARs on > > > > > the same page, could you give it a try? > > > > > > > > > > Note that this is not going to prevent the guest from moving those > > > > > BARs around and place them in the same page, thus breaking the initial > > > > > placement done by hvmloader. > > > > > > > > > > Thanks, Roger. > > > > > > > > Hi, Roger. > > > > > > > > I've minimally tested this. Yes, this patch seems to place small BARs > > > > into separate pages. The linux stubdom and QEMU then use the spacing > > > > as provided by hvmloader. > > > > > > Roger, > > > > > > Would you mind submitting this patch to Xen? > > > > Hm, I'm half minded regarding this patch. It feels more like a bandaid > > than a proper solution. Mapping BARs not multiple of page-sizes is > > dangerous because AFAIK there's no entity that asserts there isn't any > > other BAR from a different device on the same page, and hence you > > might end up mapping some MMIO region from another device > > inadvertently. > > We have the guest, linux stubdom with qemu, & dom0. Are you concerned > that all of them need a minimum of page alignment?
No, not really. The hardware domain (dom0 in normal deployments) should be the one that makes sure there are no BARs sharing physical pages. > Linux PCI subsytem has an option resource_alignment that can be > applied to either a single device or all devices. Booting with > pci=resource_aligment=4096 will align each device to a page. Do you > think pciback should force resource_alignment=4096 for dom0? Ideally Xen should keep track of the BARs position and size and refuse to passthrough devices that have BARs sharing a page with other devices BARs. > Are > there other MMIO ranges to be concerned about adjacent to BARs? IIRC you can have two BARs of different devices in the same 4K page, BARs are only aligned to it's size, so BARs smaller than 4K are not required to be page aligned. > On my one test machine with a BAR smaller than 4096, the firmware > already sets an alignment of 4096. Linux dom0 seems to keep the > firmware BAR alignment by default. The PCI spec recommend BARs to be sized to a multiple of a page size, but sadly that's not a mandatory requirement. Will submit the patch now, thanks for the ping, I completely forgot about this TBH. Roger.
