On 3/19/25 6:12 PM, Shameerali Kolothum Thodi wrote:
>
>> -----Original Message-----
>> From: Eric Auger <[email protected]>
>> Sent: Wednesday, March 19, 2025 5:01 PM
>> To: Jason Gunthorpe <[email protected]>; Donald Dutile
>> <[email protected]>
>> Cc: Nicolin Chen <[email protected]>; Shameerali Kolothum Thodi
>> <[email protected]>; [email protected];
>> [email protected]; [email protected];
>> [email protected]; [email protected]; [email protected];
>> [email protected]; Linuxarm <[email protected]>; Wangzhou (B)
>> <[email protected]>; jiangkunkun <[email protected]>;
>> Jonathan Cameron <[email protected]>;
>> [email protected]
>> Subject: Re: [RFC PATCH v2 03/20] hw/arm/smmuv3-accel: Add initial
>> infrastructure for smmuv3-accel device
>>
>> Hi,
>>
>>
>> On 3/19/25 1:23 AM, Jason Gunthorpe wrote:
>>> On Tue, Mar 18, 2025 at 05:22:51PM -0400, Donald Dutile wrote:
>>>
>>>> I agree with Eric that 'accel' isn't needed -- this should be
>>>> ascertained from the pSMMU that a physical device is attached to.
>>> I seem to remember the point was made that we don't actually know if
>>> accel is possible, or desired, especially in the case of hotplug.
>> that's why I think it would be better if we could instantiate a single
>> type of device that can do both accel and non accel mode.
>> Maybe that would be at the price of always enforcing MSI resv regions on
>> guest to assure MSI nesting is possible.
>>
>>> The accelerated mode has a number of limitations that the software
>>> mode does not have. I think it does make sense that the user would
>>> deliberately choose to use a more restrictive operating mode and then
>>> would have to meet the requirements - eg by creating the required
>>> number and configuration of vSMMUs.
>> To avoid any misunderstanding I am not pushing for have a single vSMMU
>> instance. I advocate for having several instances, each somehow
>> specialized for VFIO devices or emulated devices. Maybe we can opt-in
>> with accel=on but the default could be auto (the property can be
>> AUTO_ON_OFF) where the code detects if a VFIO device is translated.In
>> case incompatible devices are translated into a same vSMMU instance I
>> guess it could be detected and will fail.
>>
>> What I am pusshing for is to have a single type of QEMU device which can
>> do both accel and non accel.
>>> In general I advocate for having several vSMMU instances, each of them
>>>
>>>> Now... how does vfio(?; why not qemu?) layer determine that? --
>>>> where are SMMUv3 'accel' features exposed either: a) in the device
>>>> struct (for the smmuv3) or (b) somewhere under sysfs? ... I couldn't
>>>> find anything under either on my g-h system, but would appreciate a
>>>> ptr if there is.
>>> I think it is not discoverable yet other thatn through
>>> try-and-fail. Discoverability would probably be some bits in an
>>> iommufd GET_INFO ioctl or something like that.
>> yeah but at least we can easily detect if a VFIO device is beeing
>> translated by a vSMMU instance in which case there is no other choice to
>> turn accel on.
> Not sure, how you can handle hotplug in such a case? For example if the
> smmuv3
> dev starts with an emulated device and later try plug a vfio dev? In case of
> "accel"
> the feature bits(IIDR) is queried from the host SMMUv3 and is presented to
> to the vSMMU(See patch #16). We can't do this once Guest is booted.
if accel=auto, if smmu is attached to a bus where only emulated devices
are plugged, at cold start accel=false and then it effectively becomes
impossible to hotplug a vfio device.
if accel=auto and smmu is attached to a bus where a VFIO-PCI device is
cold-plugged, we end up with accel=on forced.
otherwise you always have the possible to opt-on for accel with accel=true
just like intel_iommu has caching_mode option.
>
> Also Daniel previously commented on RFCv1 that he would like to have explicit
> vSMMU<-->pSMMU association in Qemu command line.
> https://lore.kernel.org/qemu-devel/[email protected]/
tbh I did not understand why this explicit setting was needed and why it
can't be inferred from the HostIOMMUDevice. But I need to read this back.
>
> Though we are not there yet without a cold-plugged VFIO dev at the moment,
> having auto detection of accel is not the right approach if we want an
> explicit
> association in Qemu command line.
Maybe we shall not focus too much on auto detection at the moment.
Eric
>
> Thanks,
> Shameer
