On Wed, 18 Sep 2019, Christoph Hellwig wrote: > On Tue, Sep 17, 2019 at 06:41:02PM +0000, Lendacky, Thomas wrote: > > > diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c > > > --- a/drivers/nvme/host/pci.c > > > +++ b/drivers/nvme/host/pci.c > > > @@ -1613,7 +1613,8 @@ static int nvme_alloc_admin_tags(struct nvme_dev > > > *dev) > > > dev->admin_tagset.timeout = ADMIN_TIMEOUT; > > > dev->admin_tagset.numa_node = dev_to_node(dev->dev); > > > dev->admin_tagset.cmd_size = sizeof(struct nvme_iod); > > > - dev->admin_tagset.flags = BLK_MQ_F_NO_SCHED; > > > + dev->admin_tagset.flags = BLK_MQ_F_NO_SCHED | > > > + BLK_MQ_F_BLOCKING; > > > > I think you want to only set the BLK_MQ_F_BLOCKING if the DMA is required > > to be unencrypted. Unfortunately, force_dma_unencrypted() can't be called > > from a module. Is there a DMA API that could be called to get that info? > > The DMA API must support non-blocking calls, and various drivers rely > on that. So we need to provide that even for the SEV case. If the > actual blocking can't be made to work we'll need to wire up the DMA > pool in kernel/dma/remap.c for it (and probably move it to separate > file). >
Resurrecting this thread from a couple months ago because it appears that this is still an issue with 5.4 guests. dma_pool_alloc(), regardless of whether mem_flags allows blocking or not, can always sleep if the device's DMA must be unencrypted and mem_encrypt_active() == true. We know this because vm_unmap_aliases() can always block. NVMe's setup of PRPs and SGLs uses dma_pool_alloc(GFP_ATOMIC) but when this is a SEV-enabled guest this allocation may block due to the possibility of allocating DMA coherent memory through dma_direct_alloc(). It seems like one solution would be to add significant latency by doing BLK_MQ_F_BLOCKING if force_dma_unencrypted() is true for the device but this comes with significant downsides. So we're left with making dma_pool_alloc(GFP_ATOMIC) actually be atomic even when the DMA needs to be unencrypted for SEV. Christoph's suggestion was to wire up dmapool in kernel/dma/remap.c for this. Is that necessary to be done for all devices that need to do dma_pool_alloc(GFP_ATOMIC) or can we do it within the DMA API itself so it's transparent to the driver? Thomas/Brijesh: separately, it seems the use of set_memory_encrypted() or set_memory_decrypted() must be possible without blocking; is this only an issue from the DMA API point of view or can it be done elsewhere? _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
