Re: [PATCH v2] nvme: Cache DMA descriptors to prevent corruption.
On 11/30/20 12:50 PM, Tom Roeder wrote: > On Fri, Nov 20, 2020 at 09:02:43AM +0100, Christoph Hellwig wrote: >> On Thu, Nov 19, 2020 at 05:27:37PM -0800, Tom Roeder wrote: >>> This patch changes the NVMe PCI implementation to cache host_mem_descs >>> in non-DMA memory instead of depending on descriptors stored in DMA >>> memory. This change is needed under the malicious-hypervisor threat >>> model assumed by the AMD SEV and Intel TDX architectures, which encrypt >>> guest memory to make it unreadable. Some versions of these architectures >>> also make it cryptographically hard to modify guest memory without >>> detection. >> >> I don't think this is a useful threat model, and I've not seen a >> discussion on lkml where we had any discussion on this kind of threat >> model either. > > Thanks for the feedback and apologies for the lack of context. > > I was under the impression that support for AMD SEV SNP will start showing > up in KVM soon, and my understanding of SNP is that it implies this threat > model for the guest. See the patchset for SEV-ES, which is the generation > before SNP: > https://lkml.org/lkml/2020/9/14/1168.> This doesn't get quite to the SNP > threat model, but it starts to assume > more maliciousness on the part of the hypervisor. > > You can also see the talk from David Kaplan of AMD from the 2019 Linux > Security Summit for info about SNP: > https://www.youtube.com/watch?v=yr56SaJ_0QI. > > >> >> Before you start sending patches that regress optimizations in various >> drivers (and there will be lots with this model) we need to have a >> broader discussion first. > > I've added Tom Lendacky and David Kaplan from AMD on the thread now, since > I don't think I have enough context to say where this discussion should > take place or the degree to which they think it has or hasn't. > > Tom, David: can you please comment on this? Any discussion should certainly take place in the open on the mailing lists. Further information on SEV-SNP can be found on the SEV developer web page at https://developer.amd.com/sev. There is a white paper specific to SNP: https://www.amd.com/system/files/TechDocs/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more.pdf Also, volume 2 of the AMD APM provides further information on the various SEV features (sections 15.34 to 15.36): https://www.amd.com/system/files/TechDocs/24593.pdf It is a good idea to go through the various drivers and promote changes to provide protection from a malicious hypervisor, but, as Christoph states, it needs to be discussed in order to determine the best approach. Thanks, Tom > >> >> And HMB support, which is for low-end consumer devices that are usually >> not directly assigned to VMs aren't a good starting point for this. > > I'm glad to hear that this case doesn't apply directly to cases we would > care about for assignment to guests. I'm not very familiar with this > codebase, unfortunately. Do the same kinds of issues apply for the kinds > of devices that would be assigned to guests?
Re: [PATCH v2] nvme: Cache DMA descriptors to prevent corruption.
On Fri, Nov 20, 2020 at 06:29:54AM -0800, Keith Busch wrote: On Fri, Nov 20, 2020 at 09:02:43AM +0100, Christoph Hellwig wrote: On Thu, Nov 19, 2020 at 05:27:37PM -0800, Tom Roeder wrote: > This patch changes the NVMe PCI implementation to cache host_mem_descs > in non-DMA memory instead of depending on descriptors stored in DMA > memory. This change is needed under the malicious-hypervisor threat > model assumed by the AMD SEV and Intel TDX architectures, which encrypt > guest memory to make it unreadable. Some versions of these architectures > also make it cryptographically hard to modify guest memory without > detection. I don't think this is a useful threat model, and I've not seen a discussion on lkml where we had any discussion on this kind of threat model either. Before you start sending patches that regress optimizations in various drivers (and there will be lots with this model) we need to have a broader discussion first. And HMB support, which is for low-end consumer devices that are usually not directly assigned to VMs aren't a good starting point for this. Yeah, while doing this for HMB isn't really a performance concern, this method for chaining SGL/PRP lists would be. I see that this answers a question I just asked in my reply to the previous message. Sorry about that. Can you please point me to the code in question? And perhaps more importantly, the proposed mitigation only lets the guest silently carry on from such an attack while the device is surely corrupting something. I think we'd rather free the wrong address since that may at least eventually raise an error. From a security perspective, I'd rather not free the wrong address, since that could lead to an attack on the guest (use-after-free). But I agree with the concern about fixing the problem silently. Maybe this code should instead raise an error itself in this case after comparing the cached values with the values stored in the DMA memory?
Re: [PATCH v2] nvme: Cache DMA descriptors to prevent corruption.
On Fri, Nov 20, 2020 at 09:02:43AM +0100, Christoph Hellwig wrote: On Thu, Nov 19, 2020 at 05:27:37PM -0800, Tom Roeder wrote: This patch changes the NVMe PCI implementation to cache host_mem_descs in non-DMA memory instead of depending on descriptors stored in DMA memory. This change is needed under the malicious-hypervisor threat model assumed by the AMD SEV and Intel TDX architectures, which encrypt guest memory to make it unreadable. Some versions of these architectures also make it cryptographically hard to modify guest memory without detection. I don't think this is a useful threat model, and I've not seen a discussion on lkml where we had any discussion on this kind of threat model either. Thanks for the feedback and apologies for the lack of context. I was under the impression that support for AMD SEV SNP will start showing up in KVM soon, and my understanding of SNP is that it implies this threat model for the guest. See the patchset for SEV-ES, which is the generation before SNP: https://lkml.org/lkml/2020/9/14/1168. This doesn't get quite to the SNP threat model, but it starts to assume more maliciousness on the part of the hypervisor. You can also see the talk from David Kaplan of AMD from the 2019 Linux Security Summit for info about SNP: https://www.youtube.com/watch?v=yr56SaJ_0QI. Before you start sending patches that regress optimizations in various drivers (and there will be lots with this model) we need to have a broader discussion first. I've added Tom Lendacky and David Kaplan from AMD on the thread now, since I don't think I have enough context to say where this discussion should take place or the degree to which they think it has or hasn't. Tom, David: can you please comment on this? And HMB support, which is for low-end consumer devices that are usually not directly assigned to VMs aren't a good starting point for this. I'm glad to hear that this case doesn't apply directly to cases we would care about for assignment to guests. I'm not very familiar with this codebase, unfortunately. Do the same kinds of issues apply for the kinds of devices that would be assigned to guests?
Re: [PATCH v2] nvme: Cache DMA descriptors to prevent corruption.
On Fri, Nov 20, 2020 at 09:02:43AM +0100, Christoph Hellwig wrote: > On Thu, Nov 19, 2020 at 05:27:37PM -0800, Tom Roeder wrote: > > This patch changes the NVMe PCI implementation to cache host_mem_descs > > in non-DMA memory instead of depending on descriptors stored in DMA > > memory. This change is needed under the malicious-hypervisor threat > > model assumed by the AMD SEV and Intel TDX architectures, which encrypt > > guest memory to make it unreadable. Some versions of these architectures > > also make it cryptographically hard to modify guest memory without > > detection. > > I don't think this is a useful threat model, and I've not seen a > discussion on lkml where we had any discussion on this kind of threat > model either. > > Before you start sending patches that regress optimizations in various > drivers (and there will be lots with this model) we need to have a > broader discussion first. > > And HMB support, which is for low-end consumer devices that are usually > not directly assigned to VMs aren't a good starting point for this. Yeah, while doing this for HMB isn't really a performance concern, this method for chaining SGL/PRP lists would be. And perhaps more importantly, the proposed mitigation only lets the guest silently carry on from such an attack while the device is surely corrupting something. I think we'd rather free the wrong address since that may at least eventually raise an error.
Re: [PATCH v2] nvme: Cache DMA descriptors to prevent corruption.
On Thu, Nov 19, 2020 at 05:27:37PM -0800, Tom Roeder wrote: > This patch changes the NVMe PCI implementation to cache host_mem_descs > in non-DMA memory instead of depending on descriptors stored in DMA > memory. This change is needed under the malicious-hypervisor threat > model assumed by the AMD SEV and Intel TDX architectures, which encrypt > guest memory to make it unreadable. Some versions of these architectures > also make it cryptographically hard to modify guest memory without > detection. I don't think this is a useful threat model, and I've not seen a discussion on lkml where we had any discussion on this kind of threat model either. Before you start sending patches that regress optimizations in various drivers (and there will be lots with this model) we need to have a broader discussion first. And HMB support, which is for low-end consumer devices that are usually not directly assigned to VMs aren't a good starting point for this.
