On 07/27/17 at 05:29pm, Joerg Roedel wrote:
> On Fri, Jul 21, 2017 at 04:59:03PM +0800, Baoquan He wrote:
> > Add function copy_dev_tables to copy the old DEV table entries of the
> > panicked
>
> Since there is only one (for now), you can name the function in
> singular: copy_dev_table() or copy_device_table().
Make sense, will change it to copy_device_table(). Thanks.
>
> > kernel to the new allocated DEV table. Since all iommus share the same DTE
> > table
> > the copy only need be done one time. Besides, we also need to:
> >
> > - Check whether all IOMMUs actually use the same device table with the
> > same size
> >
> > - Verify that the size of the old device table is the expected size.
> >
> > - Reserve the old domain id occupied in 1st kernel to avoid touching the
> > old
> > io-page tables. Then on-flight DMA can continue looking it up.
> >
> > And also define MACRO DEV_DOMID_MASK to replace magic number 0xffffULL, it
> > can be
> > reused in copy_dev_tables().
> >
> > Signed-off-by: Baoquan He <b...@redhat.com>
> > ---
> > drivers/iommu/amd_iommu.c | 2 +-
> > drivers/iommu/amd_iommu_init.c | 55
> > +++++++++++++++++++++++++++++++++++++++++
> > drivers/iommu/amd_iommu_types.h | 1 +
> > 3 files changed, 57 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c
> > index e5a03f259986..4d00f1bda900 100644
> > --- a/drivers/iommu/amd_iommu.c
> > +++ b/drivers/iommu/amd_iommu.c
> > @@ -2086,7 +2086,7 @@ static void set_dte_entry(u16 devid, struct
> > protection_domain *domain, bool ats)
> > flags |= tmp;
> > }
> >
> > - flags &= ~(0xffffUL);
> > + flags &= ~DEV_DOMID_MASK;
> > flags |= domain->id;
> >
> > amd_iommu_dev_table[devid].data[1] = flags;
> > diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c
> > index f6da5fe03b31..c58f091ce232 100644
> > --- a/drivers/iommu/amd_iommu_init.c
> > +++ b/drivers/iommu/amd_iommu_init.c
> > @@ -842,6 +842,61 @@ static int get_dev_entry_bit(u16 devid, u8 bit)
> > }
> >
> >
> > +static int copy_dev_tables(void)
> > +{
> > + struct dev_table_entry *old_devtb = NULL;
> > + u32 lo, hi, devid, old_devtb_size;
> > + phys_addr_t old_devtb_phys;
> > + u64 entry, last_entry = 0;
> > + struct amd_iommu *iommu;
> > + u16 dom_id, dte_v;
> > + static int copied;
> > +
> > + for_each_iommu(iommu) {
> > + if (!translation_pre_enabled(iommu)) {
> > + pr_err("IOMMU:%d is not pre-enabled!/n",
> > + iommu->index);
> > + return -1;
> > + }
> > +
> > + /* All IOMMUs should use the same device table with the same
> > size */
> > + lo = readl(iommu->mmio_base + MMIO_DEV_TABLE_OFFSET);
> > + hi = readl(iommu->mmio_base + MMIO_DEV_TABLE_OFFSET + 4);
> > + entry = (((u64) hi) << 32) + lo;
> > + if (last_entry && last_entry != entry) {
> > + pr_err("IOMMU:%d should use the same dev table as
> > others!/n",
> > + iommu->index);
> > + return -1;
> > + }
> > + last_entry = entry;
> > +
> > + old_devtb_size = ((entry & ~PAGE_MASK) + 1) << 12;
> > + if (old_devtb_size != dev_table_size) {
> > + pr_err("The device table size of IOMMU:%d is not
> > expected!/n",
> > + iommu->index);
> > + return -1;
> > + }
>
> I think the loop can end here. There is only one table to copy, so you
> don't need to copy it for every iommu. Just do the checks in the loop
> and the copy once after the loop.
Ok, will change. I worried that device table addr might not be stored
into iommu register correctly. I am fine we only do the check only at
the first time.
>
> > +
> > + old_devtb_phys = entry & PAGE_MASK;
> > + old_devtb = memremap(old_devtb_phys, dev_table_size,
> > MEMREMAP_WB);
> > + if (!old_devtb)
> > + return -1;
> > +
> > + if (copied)
> > + continue;
> > + for (devid = 0; devid <= amd_iommu_last_bdf; ++devid) {
> > + amd_iommu_dev_table[devid] = old_devtb[devid];
> > + dom_id = old_devtb[devid].data[1] & DEV_DOMID_MASK;
> > + dte_v = old_devtb[devid].data[0] & DTE_FLAG_V;
> > + if (dte_v && dom_id)
> > + __set_bit(dom_id, amd_iommu_pd_alloc_bitmap);
> > + }
>
> Please only copy the DTE when DTE.V is set in the old table. Also
Sorry, the sanity check about DTE.V has been done in patch 7/13. I
should rewrite the subject and git log for patch 7/13.
> don't copy any of the IOMMUv2 enablement bits from the old DTE. PRI
> faults are recoverable by design and a device shouldn't fail on a
> negative answer from the IOMMU.
Yes, this is done in patch 11/13. Saw your comment in patch 11/13.
>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
