Re: [PATCH v4 9/9] docs: Add device tree overlay documentation
On Thu, 23 May 2024, Julien Grall wrote: > Hi Henry, > > On 23/05/2024 08:40, Henry Wang wrote: > > From: Vikram Garhwal > > > > Signed-off-by: Vikram Garhwal > > Signed-off-by: Stefano Stabellini > > Signed-off-by: Henry Wang > > --- > > v4: > > - No change. > > v3: > > - No change. > > v2: > > - Update the content based on the changes in this version. > > --- > > docs/misc/arm/overlay.txt | 99 +++ > > 1 file changed, 99 insertions(+) > > create mode 100644 docs/misc/arm/overlay.txt > > > > diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt > > new file mode 100644 > > index 00..811a6de369 > > --- /dev/null > > +++ b/docs/misc/arm/overlay.txt > > @@ -0,0 +1,99 @@ > > +# Device Tree Overlays support in Xen > > + > > +Xen now supports dynamic device assignment to running domains, > > This reads as we "support" the feature. I would prefer if we write "Xen > expirementally supports..." or similar. Done > > +i.e. adding/removing nodes (using .dtbo) to/from Xen device tree, and > > +attaching/detaching them to/from a running domain with given $domid. > > + > > +Dynamic node assignment works in two steps: > > + > > +## Add/Remove device tree overlay to/from Xen device tree > > + > > +1. Xen tools check the dtbo given and parse all other user provided > > arguments > > +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. > > +3. Xen hypervisor applies/removes the dtbo to/from Xen device tree. > > + > > +## Attach/Detach device from the DT overlay to/from domain > > + > > +1. Xen tools check the dtbo given and parse all other user provided > > arguments > > +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. > > +3. Xen hypervisor attach/detach the device to/from the user-provided $domid > > by > > + mapping/unmapping node resources in the DT overlay. > > + > > +# Examples > > + > > +Here are a few examples on how to use it. > > + > > +## Dom0 device add > > + > > +For assigning a device tree overlay to Dom0, user should firstly properly > > +prepare the DT overlay. More information about device tree overlays can be > > +found in [1]. Then, in Dom0, enter the following: > > + > > +(dom0) xl dt-overlay add overlay.dtbo > > + > > +This will allocate the devices mentioned in overlay.dtbo to Xen device > > tree. > > + > > +To assign the newly added device from the dtbo to Dom0: > > + > > +(dom0) xl dt-overlay attach overlay.dtbo 0 > > + > > +Next, if the user wants to add the same device tree overlay to dom0 > > +Linux, execute the following: > > + > > +(dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay > > +(dom0) cat overlay.dtbo > > > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo > > + > > +Finally if needed, the relevant Linux kernel drive can be loaded using: > > + > > +(dom0) modprobe module_name.ko > > + > > +## Dom0 device remove > > + > > +For removing the device from Dom0, first detach the device from Dom0: > > + > > +(dom0) xl dt-overlay detach overlay.dtbo 0 > > + > > +NOTE: The user is expected to unload any Linux kernel modules which > > +might be accessing the devices in overlay.dtbo before detach the device. > > +Detaching devices without unloading the modules might result in a crash. > > + > > +Then remove the overlay from Xen device tree: > > + > > +(dom0) xl dt-overlay remove overlay.dtbo > > + > > +## DomU device add/remove > > + > > +All the nodes in dtbo will be assigned to a domain; the user will need > > +to prepare the dtb for the domU. For example, the `interrupt-parent` > > property > > +of the DomU overlay should be changed to the Xen hardcoded value `0xfde8`. > > +Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`. > > + > > +User will need to create the DomU with below properties properly configured > > +in the xl config file: > > +- `iomem` > > I don't quite understand how the user can specify the MMIO region if the > device is attached after the domain is created. I think this was meant for a domain about to be created (not already running). I clarified. > > > +- `passthrough` (if IOMMU is needed) > > + > > +User will also need to modprobe the relevant drivers. > > + > > +Example for domU device add: > > + > > +(dom0) xl dt-overlay add overlay.dtbo# If not executed > > before > > +(dom0) xl dt-overlay attach overlay.dtbo $domid > > Can how clarify how the MMIO will be mapped? Is it direct mapped? If so, > couldn't this result to clash with other part of the address space (e.g. > RAM?). Yes, it is reusing the same code as dom0, which makes the code nice but it doesn't support non-1:1 mappings. I think those should be done via the xen,reg property. My suggestion would be this: - if xen,reg is present, use it - if xen,reg is not present, fall back to 1:1 mapping based on reg For the next version of the series, I'd just document the current limitation of the implementation. I added this to patch
Re: [PATCH v4 9/9] docs: Add device tree overlay documentation
Hi Henry, On 23/05/2024 08:40, Henry Wang wrote: From: Vikram Garhwal Signed-off-by: Vikram Garhwal Signed-off-by: Stefano Stabellini Signed-off-by: Henry Wang --- v4: - No change. v3: - No change. v2: - Update the content based on the changes in this version. --- docs/misc/arm/overlay.txt | 99 +++ 1 file changed, 99 insertions(+) create mode 100644 docs/misc/arm/overlay.txt diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt new file mode 100644 index 00..811a6de369 --- /dev/null +++ b/docs/misc/arm/overlay.txt @@ -0,0 +1,99 @@ +# Device Tree Overlays support in Xen + +Xen now supports dynamic device assignment to running domains, This reads as we "support" the feature. I would prefer if we write "Xen expirementally supports..." or similar. +i.e. adding/removing nodes (using .dtbo) to/from Xen device tree, and +attaching/detaching them to/from a running domain with given $domid. + +Dynamic node assignment works in two steps: + +## Add/Remove device tree overlay to/from Xen device tree + +1. Xen tools check the dtbo given and parse all other user provided arguments +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. +3. Xen hypervisor applies/removes the dtbo to/from Xen device tree. + +## Attach/Detach device from the DT overlay to/from domain + +1. Xen tools check the dtbo given and parse all other user provided arguments +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. +3. Xen hypervisor attach/detach the device to/from the user-provided $domid by + mapping/unmapping node resources in the DT overlay. + +# Examples + +Here are a few examples on how to use it. + +## Dom0 device add + +For assigning a device tree overlay to Dom0, user should firstly properly +prepare the DT overlay. More information about device tree overlays can be +found in [1]. Then, in Dom0, enter the following: + +(dom0) xl dt-overlay add overlay.dtbo + +This will allocate the devices mentioned in overlay.dtbo to Xen device tree. + +To assign the newly added device from the dtbo to Dom0: + +(dom0) xl dt-overlay attach overlay.dtbo 0 + +Next, if the user wants to add the same device tree overlay to dom0 +Linux, execute the following: + +(dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay +(dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo + +Finally if needed, the relevant Linux kernel drive can be loaded using: + +(dom0) modprobe module_name.ko + +## Dom0 device remove + +For removing the device from Dom0, first detach the device from Dom0: + +(dom0) xl dt-overlay detach overlay.dtbo 0 + +NOTE: The user is expected to unload any Linux kernel modules which +might be accessing the devices in overlay.dtbo before detach the device. +Detaching devices without unloading the modules might result in a crash. + +Then remove the overlay from Xen device tree: + +(dom0) xl dt-overlay remove overlay.dtbo + +## DomU device add/remove + +All the nodes in dtbo will be assigned to a domain; the user will need +to prepare the dtb for the domU. For example, the `interrupt-parent` property +of the DomU overlay should be changed to the Xen hardcoded value `0xfde8`. +Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`. + +User will need to create the DomU with below properties properly configured +in the xl config file: +- `iomem` I don't quite understand how the user can specify the MMIO region if the device is attached after the domain is created. +- `passthrough` (if IOMMU is needed) + +User will also need to modprobe the relevant drivers. + +Example for domU device add: + +(dom0) xl dt-overlay add overlay.dtbo# If not executed before +(dom0) xl dt-overlay attach overlay.dtbo $domid Can how clarify how the MMIO will be mapped? Is it direct mapped? If so, couldn't this result to clash with other part of the address space (e.g. RAM?). +(dom0) xl console $domid # To access $domid console + +Next, if the user needs to modify/prepare the overlay.dtbo suitable for +the domU: + +(domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay +(domU) cat overlay_domu.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo + +Finally, if needed, the relevant Linux kernel drive can be probed: + +(domU) modprobe module_name.ko + +Example for domU overlay remove: + +(dom0) xl dt-overlay detach overlay.dtbo $domid +(dom0) xl dt-overlay remove overlay.dtbo I assume we have safety check in place to ensure we can't remove the device if it is already attached. Is that correct? Cheers, -- Julien Grall
[PATCH v4 9/9] docs: Add device tree overlay documentation
From: Vikram Garhwal Signed-off-by: Vikram Garhwal Signed-off-by: Stefano Stabellini Signed-off-by: Henry Wang --- v4: - No change. v3: - No change. v2: - Update the content based on the changes in this version. --- docs/misc/arm/overlay.txt | 99 +++ 1 file changed, 99 insertions(+) create mode 100644 docs/misc/arm/overlay.txt diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt new file mode 100644 index 00..811a6de369 --- /dev/null +++ b/docs/misc/arm/overlay.txt @@ -0,0 +1,99 @@ +# Device Tree Overlays support in Xen + +Xen now supports dynamic device assignment to running domains, +i.e. adding/removing nodes (using .dtbo) to/from Xen device tree, and +attaching/detaching them to/from a running domain with given $domid. + +Dynamic node assignment works in two steps: + +## Add/Remove device tree overlay to/from Xen device tree + +1. Xen tools check the dtbo given and parse all other user provided arguments +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. +3. Xen hypervisor applies/removes the dtbo to/from Xen device tree. + +## Attach/Detach device from the DT overlay to/from domain + +1. Xen tools check the dtbo given and parse all other user provided arguments +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. +3. Xen hypervisor attach/detach the device to/from the user-provided $domid by + mapping/unmapping node resources in the DT overlay. + +# Examples + +Here are a few examples on how to use it. + +## Dom0 device add + +For assigning a device tree overlay to Dom0, user should firstly properly +prepare the DT overlay. More information about device tree overlays can be +found in [1]. Then, in Dom0, enter the following: + +(dom0) xl dt-overlay add overlay.dtbo + +This will allocate the devices mentioned in overlay.dtbo to Xen device tree. + +To assign the newly added device from the dtbo to Dom0: + +(dom0) xl dt-overlay attach overlay.dtbo 0 + +Next, if the user wants to add the same device tree overlay to dom0 +Linux, execute the following: + +(dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay +(dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo + +Finally if needed, the relevant Linux kernel drive can be loaded using: + +(dom0) modprobe module_name.ko + +## Dom0 device remove + +For removing the device from Dom0, first detach the device from Dom0: + +(dom0) xl dt-overlay detach overlay.dtbo 0 + +NOTE: The user is expected to unload any Linux kernel modules which +might be accessing the devices in overlay.dtbo before detach the device. +Detaching devices without unloading the modules might result in a crash. + +Then remove the overlay from Xen device tree: + +(dom0) xl dt-overlay remove overlay.dtbo + +## DomU device add/remove + +All the nodes in dtbo will be assigned to a domain; the user will need +to prepare the dtb for the domU. For example, the `interrupt-parent` property +of the DomU overlay should be changed to the Xen hardcoded value `0xfde8`. +Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`. + +User will need to create the DomU with below properties properly configured +in the xl config file: +- `iomem` +- `passthrough` (if IOMMU is needed) + +User will also need to modprobe the relevant drivers. + +Example for domU device add: + +(dom0) xl dt-overlay add overlay.dtbo# If not executed before +(dom0) xl dt-overlay attach overlay.dtbo $domid +(dom0) xl console $domid # To access $domid console + +Next, if the user needs to modify/prepare the overlay.dtbo suitable for +the domU: + +(domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay +(domU) cat overlay_domu.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo + +Finally, if needed, the relevant Linux kernel drive can be probed: + +(domU) modprobe module_name.ko + +Example for domU overlay remove: + +(dom0) xl dt-overlay detach overlay.dtbo $domid +(dom0) xl dt-overlay remove overlay.dtbo + +[1] https://www.kernel.org/doc/Documentation/devicetree/overlay-notes.txt -- 2.34.1
