Hello On Thu, Dec 8, 2011 at 7:47 PM, Roland Dreier <rol...@purestorage.com> wrote: > > [ 597.412843] kernel BUG at drivers/iommu/intel-iommu.c:1767! > So this is > BUG_ON(addr_width < BITS_PER_LONG && (iov_pfn + nr_pages - 1) >>> addr_width); > > in __domain_mapping() I believe. And we have: > int addr_width = agaw_to_width(domain->agaw) - VTD_PAGE_SHIFT; > How much RAM does you system have?
192 GB. A few gigabyte is reserved for 2 MB huge pages, if that matters. > I don't know too much about the low-level VT-d details, but is it > possible the setup code > is choosing a too small "guest address width" to cover all your memory? > > drivers/net/ethernet/mellanox/mlx4/main.c has > > err = pci_set_dma_mask(pdev, DMA_BIT_MASK(64)); > if (err) { > dev_warn(&pdev->dev, "Warning: couldn't set 64-bit PCI > DMA mask.\n"); > err = pci_set_dma_mask(pdev, DMA_BIT_MASK(32)); > if (err) { > dev_err(&pdev->dev, "Can't set PCI DMA mask, > aborting.\n"); > goto err_release_regions; > } > } > err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64)); > if (err) { > dev_warn(&pdev->dev, "Warning: couldn't set 64-bit " > "consistent PCI DMA mask.\n"); > err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32)); > if (err) { > dev_err(&pdev->dev, "Can't set consistent PCI > DMA mask, " > "aborting.\n"); > goto err_release_regions; > } > } > > do you see any warnings in your kernel log about setting the PCI DMA mask? > (in any case that should only affect bus addresses, not "guest addresses") I don't see anything like what you mentioned here. >From the SAS controller: mpt2sas0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (198141716 kB) Not seeing anything about PCI DMA errors. I think the BIOS has VT-d enabled. dmesg says: PCI-DMA: Intel(R) Virtualization Technology for Directed I/O I can send you a full dmesg if that would help. Regards Albert -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html