> From: Nicolas Saenz Julienne <nsaenzjulie...@suse.de>
> Date: Mon, 15 Mar 2021 11:26:18 +0100
>
> Hi Mark!
Hi Nicolas,
> On Sat, 2021-03-13 at 10:24 +0100, Mark Kettenis wrote:
> [...]
> > Fortunately Nicolas Saenz Julienne recently introduced
> > dev_phys_to_bus() and dev_bus_to_phys() interfaces to do this. Those
> > interfaces make use of a dma-ranges property in the device tree which
> > doesn't work so well for PCI devices though.
>
> Why doesn't it work with PCI devices? Raspberry Pi 4 has a PCIe bus
> that needs DMA translations. This is handled by parsing its
> 'dma-ranges' property. Here's how rpi4's devicetree looks like[1]:
>
> pcie0: pcie@7d500000 {
> compatible = "brcm,bcm2711-pcie";
> ranges = <0x02000000 0x0 0xf8000000 0x6 0x00000000 0x0
> 0x04000000>;
> /*
> * The wrapper around the PCIe block has a bug preventing it
> * from accessing beyond the first 3GB of memory.
> */
> dma-ranges = <0x02000000 0x0 0x00000000 0x0 0x00000000 0x0
> 0xc0000000>;
> };
Does that work even though there are no device tree nodes for the PCI
devices themselves?
>
> > However, the PCI code in U-Boot already has a way to describe DMA address
> > translation through its regions support.
>
> regions contain a representation of devicetree's 'ranges' property, which
> doesn't describe the DMA address translations, but CPU's view of PCI memory.
It describes both. The DMA address translations are described by
regions that have the PCI_REGION_SYS_MEMORY flag set. Some of the PCI
host bridge drivers set up these regions, and the generic DM_PCI code
by default adds such regions for the system RAM banks. And there is a
fdt_pci_dma_ranges() function to populate a "dma-ranges" property
based on these regions.
The benefit of the PCI regions code is that it can describe more
complex setups where a single translation offset isn't enough. And
the code will print a warbing if you try to do DMA to memory that isn't
"visable" for the PCI device.
In the particular example of the M1 code I don't really want to add a
dma-ranges property to the device tree since the IOMMU setup is only
used by u-boot and not really usable by Linux or another OS as it only
covers a (relatively) small part of system memory.
> > diff --git a/include/phys2bus.h b/include/phys2bus.h
> > index 866b8b51a8..13d23ef4bb 100644
> > --- a/include/phys2bus.h
> > +++ b/include/phys2bus.h
> > @@ -23,14 +23,21 @@ static inline unsigned long bus_to_phys(unsigned long
> > bus)
> >
> >
> > #if CONFIG_IS_ENABLED(DM)
> > #include <dm/device.h>
> > +#include <pci.h>
> >
> >
> > static inline dma_addr_t dev_phys_to_bus(struct udevice *dev, phys_addr_t
> > phys)
> > {
> > + if (device_is_on_pci_bus(dev))
> > + return dm_pci_phys_to_bus(dev, phys, PCI_REGION_SYS_MEMORY);
> > +
>
> Note that this would break USB support on RPi4.
It should work if the PCI host bridge driver sets up the regions
properly based on the dma-ranges property. Looks like the current
driver doesn't do this, so I'll have to fix that.
Cheers,
Mark
