fetch_pte() could race with increase_address_space() because it held no lock from iommu_unmap_page(). On the CPU that runs fetch_pte() it could see a stale domain->pt_root and a new increased domain->mode from increase_address_space(). As the result, it could trigger invalid accesses later on. Fix it by using a pair of smp_[w|r]mb in those places.
kernel BUG at drivers/iommu/amd_iommu.c:1704! BUG_ON(unmapped && !is_power_of_2(unmapped)); Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 07/10/2019 RIP: 0010:amd_iommu_unmap+0x1b2/0x1d0 Call Trace: <IRQ> __iommu_unmap+0x106/0x320 iommu_unmap_fast+0xe/0x10 __iommu_dma_unmap+0xdc/0x1a0 iommu_dma_unmap_sg+0xae/0xd0 scsi_dma_unmap+0xe7/0x150 pqi_raid_io_complete+0x37/0x60 [smartpqi] pqi_irq_handler+0x1fc/0x13f0 [smartpqi] __handle_irq_event_percpu+0x78/0x4f0 handle_irq_event_percpu+0x70/0x100 handle_irq_event+0x5a/0x8b handle_edge_irq+0x10c/0x370 do_IRQ+0x9e/0x1e0 common_interrupt+0xf/0xf </IRQ> Signed-off-by: Qian Cai <c...@lca.pw> --- drivers/iommu/amd_iommu.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index 20cce366e951..22328a23335f 100644 --- a/drivers/iommu/amd_iommu.c +++ b/drivers/iommu/amd_iommu.c @@ -1434,6 +1434,11 @@ static bool increase_address_space(struct protection_domain *domain, *pte = PM_LEVEL_PDE(domain->mode, iommu_virt_to_phys(domain->pt_root)); domain->pt_root = pte; + /* + * Make sure fetch_pte() will see the new domain->pt_root before it + * snapshots domain->mode. + */ + smp_wmb(); domain->mode += 1; ret = true; @@ -1460,6 +1465,8 @@ static u64 *alloc_pte(struct protection_domain *domain, *updated = increase_address_space(domain, address, gfp) || *updated; level = domain->mode - 1; + /* To pair with smp_wmb() in increase_address_space(). */ + smp_rmb(); pte = &domain->pt_root[PM_LEVEL_INDEX(level, address)]; address = PAGE_SIZE_ALIGN(address, page_size); end_lvl = PAGE_SIZE_LEVEL(page_size); @@ -1545,6 +1552,8 @@ static u64 *fetch_pte(struct protection_domain *domain, return NULL; level = domain->mode - 1; + /* To pair with smp_wmb() in increase_address_space(). */ + smp_rmb(); pte = &domain->pt_root[PM_LEVEL_INDEX(level, address)]; *page_size = PTE_LEVEL_PAGE_SIZE(level); -- 2.21.0 (Apple Git-122.2) _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu