On Fri, 2026-08-21 at 07:15 +0000, Duan, Zhenzhong wrote: > Caution: External email. Do not open attachments or click links, unless this > email comes from a known sender and you know the content is safe. > > > > > -----Original Message----- > > From: Clément MATHIEU--DRIF > > <[[email protected]](mailto:[email protected])> > > Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for passthrough > > device > > > > > > On Fri, 2026-08-21 at 02:44 +0000, Duan, Zhenzhong wrote: > > > > > Caution: External email. Do not open attachments or click links, unless > > > this email > > > > comes from a known sender and you know the content is safe. > > > > > > > > > > > Hi Clement, > > > > > > > > > > > > > -----Original Message----- > > > > From: Clément MATHIEU--DRIF <[clement.mathieu-- > > > > > > > [[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected]))> > > > > > > > > > Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for > > > > passthrough > > > > device > > > > > > > > > > > > On Wed, 2026-08-12 at 13:26 +0000, Duan, Zhenzhong wrote: > > > > > > > > > > > > > Caution: External email. Do not open attachments or click links, > > > > > unless this > > > > > > > > > > > email > > > > > > > > > > > > > comes from a known sender and you know the content is safe. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: Clément MATHIEU--DRIF <[clement.mathieu-- > > > > > > > > > > > > > > > > > > > > > > [[[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected]))](mailto:[clement.mathieu-- > > > > > > > [[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected])))> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for > > > > > > > > > > > > > > > > passthrough > > > > > > > > > > > > > > > > > > > > device > > > > > > > > > > > > > > > > > > On Mon, 2026-08-10 at 10:49 +0000, Duan, Zhenzhong wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Caution: External email. Do not open attachments or click links, > > > > > > > unless > > > > > > > > > > > > > > > > > > > > > > this > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > email > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > comes from a known sender and you know the content is safe. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Clement, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > > > From: Clément MATHIEU--DRIF <[clement.mathieu-- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [[[[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected]))](mailto:[[[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected]) > > om))](mailto:[clement.mathieu-- > > > > > > > > > > > > > > > > > > > > > > > > > > [[[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected]))](mailto:[clement.mathieu-- > > > > > > > [[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected]))))> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > passthrough > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > device > > > > > > > > > > > > > > > > Hi Zhenzhong, > > > > > > > > > > > > > > > > I will read this series early next week. > > > > > > > > thanks for the respin! > > > > > > > > > > > > > > > > btw, related to this work: have ever seen > > > > > > > > vfio_device_get_aw_bits returning 47 whereas > > > > > > > > the iommu supports 48? This leads the following > > > > > > > > aw test to fail: > > > > > > > > > > > > > > > > ``` > > > > > > > > ret = hiodc->get_cap(hiod, HOST_IOMMU_DEVICE_CAP_AW_BITS, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > errp); > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > if (ret < 0) { > > > > > > > > return false; > > > > > > > > } > > > > > > > > if (s->aw_bits > ret) { > > > > > > > > error_setg(errp, "aw-bits %d > host aw-bits %d", s->aw_bits, > > > > > > > > ret); > > > > > > > > return false; > > > > > > > > } > > > > > > > > ``` > > > > > > > > > > > > > > > > I got this when trying to nest VMs. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am trying to reproduce, what device did you passthrough? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I used edu for testing. > > > > > > An I had aw-bits=48 on both vms. > > > > > > > > > > > > > > > > > > > > > > > > > Reproduced the same with L1(fsts=on) and L2(fsts=off). > > > > > > > > > > > > > > > > My config has fsts=on on both VMs. > > > > Which kernel are you using? > > > > Mine is RHEL10.2 (6.12.0-211.7.3.el10_2.x86_64) > > > > > > > > > > > > In theory we will not run into that check if fsts=on on both VMs, on my > > > env I see: > > > > > > "qemu-system-x86_64: -device vfio-pci,host=00:04.0,iommufd=iommufd0: vfio > > > > 0000:00:04.0: Failed to allocate hwpt: Operation not supported" > > > > > > > > which is expected as we want nesting support in L1 but it only supports > > > fsts. > > > > > > Oh yep, I forgot to mention that I'm not attaching iommufd for this test. > > I agree with what you say when iommufd is present, but without it (weird > > config, > > I admit, but interesting for testing), I get: > > > > ``` > > qemu-system-x86_64: -device vfio-pci,host=00:01.0: vfio 0000:00:01.0: > > Failed to > > set vIOMMU: aw-bits 48 > host aw-bits 47 > > > Oh, I see, I reproduced the same. > > > > ``` > > > > Just wondering why such a 1bit difference. > > > host aw-bits 47 is what host kernel supports, not the pure hw limit. > For first stage page table, PT_FEAT_SIGN_EXTEND is set, > kernel limited max iova ranges to 47bit even though hw supports 48 bit, see > below: > > static __always_inline struct pt_range _pt_top_range(struct pt_common > *common, > uintptr_t top_of_table) > { > ... > /* > * The top range will default to the lower region only with sign > extend. > */ > range.max_vasz_lg2 = max_vasz_lg2; > if (pt_feature(common, PT_FEAT_SIGN_EXTEND)) > max_vasz_lg2--; > ... > } > > So qemu gets host aw-bits 47. This is strictly following vtd spec: > > Software using first-stage translation structures to translate an IO Virtual > Address (IOVA) must use > canonical addresses. Additionally, software must limit addresses to less than > the minimum of MGAW > and the lower canonical address width implied by FSPM (i.e., 47-bit when FSPM > is 4-level and 56-bit > when FSPM is 5-level).
Oh, I see, good to know! That makes sense. Thanks! cmd > > BRs, > Zhenzhong
