On Fri, 24 May 2024, Julien Grall wrote: > Hi Stefano, > > On 24/05/2024 03:18, Stefano Stabellini wrote: > > From: Vikram Garhwal <fnu.vik...@xilinx.com> > > > > Signed-off-by: Vikram Garhwal <fnu.vik...@xilinx.com> > > Signed-off-by: Stefano Stabellini <stefano.stabell...@xilinx.com> > > Signed-off-by: Henry Wang <xin.wa...@amd.com> > > --- > > docs/misc/arm/overlay.txt | 82 +++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 82 insertions(+) > > create mode 100644 docs/misc/arm/overlay.txt > > > > diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt > > new file mode 100644 > > index 0000000000..0a2dee951a > > --- /dev/null > > +++ b/docs/misc/arm/overlay.txt > > @@ -0,0 +1,82 @@ > > +# Device Tree Overlays support in Xen > > + > > +Xen experimentally supports dynamic device assignment to running > > +domains, i.e. adding/removing nodes (using .dtbo) to/from Xen device > > +tree, and attaching them to a running domain with given $domid. > > + > > +Dynamic node assignment works in two steps: > > + > > +## Add/Remove device tree overlay to/from Xen device tree > > + > > +1. Xen tools check the dtbo given and parse all other user provided > > arguments > > +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. > > +3. Xen hypervisor applies/removes the dtbo to/from Xen device tree. > > + > > +## Attach device from the DT overlay to domain > > + > > +1. Xen tools check the dtbo given and parse all other user provided > > arguments > > +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. > > +3. Xen hypervisor attach the device to the user-provided $domid by > > + mapping node resources in the DT overlay. > > + > > +# Examples > > + > > +Here are a few examples on how to use it. > > + > > +## Dom0 device add > > + > > +For assigning a device tree overlay to Dom0, user should firstly properly > > +prepare the DT overlay. More information about device tree overlays can be > > +found in [1]. Then, in Dom0, enter the following: > > + > > + (dom0) xl dt-overlay add overlay.dtbo > > + > > +This will allocate the devices mentioned in overlay.dtbo to Xen device > > tree. > > + > > +To assign the newly added device from the dtbo to Dom0: > > + > > + (dom0) xl dt-overlay attach overlay.dtbo 0 > > + > > +Next, if the user wants to add the same device tree overlay to dom0 > > +Linux, execute the following: > > + > > + (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay > > + (dom0) cat overlay.dtbo > > > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo > > + > > +Finally if needed, the relevant Linux kernel drive can be loaded using: > > + > > + (dom0) modprobe module_name.ko > > + > > +## DomU device add/remove > > + > > +All the nodes in dtbo will be assigned to a domain; the user will need > > +to prepare the dtb for the domU. > > s/dtb/dtbo/?
Yes, done > But I am little bit confused with the wording. I think you may > want to add *different dtbo* so it clarifies from the start (this only becomes > obvious at the end of the section) that the user is not meant to use the same > for all the commands. Yes it was confusing, I tried to clarify it. This is a doc so we could improve it further after the freeze. > For example, the `interrupt-parent` > > +property of the DomU overlay should be changed to the Xen hardcoded > > +value `0xfde8`, and the xen,reg property should be added to specify the > > +address mappings. If xen,reg is not present, it is assumed 1:1 mapping. > > Repeating an earlier comment here. I think xen,reg should be mandatory for > non-direct mapped domain. That's OK > Also, can you clarify what is the expect property layout for xen,reg? I'll add a pointer > > +Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`. > > + > > +For new domains to be created, the user will need to create the DomU > > +with below properties properly configured in the xl config file: > > +- `iomem` > > I looked at your reply in v4 and I am afraid I still don't understand why we > are mentioning 'iomem'. If we want to use the commands below, then the domain > needs to be created in advance. So you can't yet know 'iomem'. > > You could avoid "xl dt-overlay attach" but then you need the user to specify > both "irqs" and "iomem". From a user point of view, it would be easier to add > a new propery in the configuration file listing the overlays. Something like: > > dt_overlays = [ "overlay.dtbo", ... ] > > Anyway, that somewhat separate. For now, I think we want to drop 'iomem' from > the list and reword this paragraph to say that the 'passthrough' property > needs to be set if you plan to use DT overlay and devices requiring the IOMMU. OK, I did that > > > +- `passthrough` (if IOMMU is needed) > > This property is required at the start because we don't support enabling the > IOMMU lazily. Yeah. I think it is much clearer now. > > + > > +User will also need to modprobe the relevant drivers. For already > > +running domains, the user can use the xl dt-overlay attach command, > > +example: > > + > > + (dom0) xl dt-overlay add overlay.dtbo # If not executed > > before > > + (dom0) xl dt-overlay attach overlay.dtbo $domid > > + (dom0) xl console $domid # To access $domid > > console > > + > > +Next, if the user needs to modify/prepare the overlay.dtbo suitable for > > +the domU: > > + > > + (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay > > + (domU) cat overlay_domu.dtbo > > > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo > > + > > +Finally, if needed, the relevant Linux kernel drive can be probed: > > + > > + (domU) modprobe module_name.ko > > + > > +[1] https://www.kernel.org/doc/Documentation/devicetree/overlay-notes.txt
