>> + pbdev->state = ZPCI_FS_ERROR; >> + s390_pci_generate_error_event(ERR_EVENT_PERMERR, >> pbdev->fh, >> + pbdev->fid, 0, 0); > > A permanent error will indicate to the guest that PCI device in > unusable, in that case would the guest drive a device re-initialization? > Would it be better to generate a ERR_EVENT_SERVAC to allow the guest to > attempt recovery and so drive the mappings again? > The problem is that if we hit this scenario, we've exhausted the vfio DMA limit for the device, so there is no way to create more mappings without freeing some other ones up. In the case of a RPCIT, we can ask the guest to try and free up all stale/invalidated mappings to make room, but in this case 1) we have no mechanism to ask the guest to do that 2) we are in the middle replaying only the valid mappings, so there are no invalid mappings to flush anyway -- so we're really in a permanent error case here -- the number of vfio-allowed concurrent DMA mappings is less than what our IOMMU wishes to replay. AFAICT this can only get solved by forcing the guest to throw everything out and start over.
I think in reality, you would only hit this in a migration scenario with a vfio-pci device (which we don't support with this series) where the vfio DMA limit is lower on the target than it was on the host. If you hit this doing IOMMU replay of a vfio-pci device without a migration involved then the limit should be the same as it was before replay and it should have been impossible to map more than the vfio DMA limit (so if you managed it, a permanent error also seems reasonable because there may be a deeper issue). Would an 0x3a trigger a total wipe of the IOMMU contents that we were trying to replay? If yes, it might be something to consider if/when we support vfio-pci migration but I'm not sure it makes sense under the current scenario(s) where we should never hit this path. Thanks, Matt
