On Tue, Apr 21, 2026 at 11:44 AM Jason Gunthorpe <[email protected]> wrote: > > On Tue, Apr 21, 2026 at 10:40:29AM -0700, David Matlack wrote: > > On Mon, Mar 23, 2026 at 4:58 PM David Matlack <[email protected]> wrote: > > > > > +static int vfio_pci_liveupdate_freeze(struct liveupdate_file_op_args > > > *args) > > > +{ > > > + struct vfio_device *device = vfio_device_from_file(args->file); > > > + struct vfio_pci_core_device *vdev; > > > + struct pci_dev *pdev; > > > + int ret; > > > + > > > + vdev = container_of(device, struct vfio_pci_core_device, vdev); > > > + pdev = vdev->pdev; > > > + > > > + guard(mutex)(&device->dev_set->lock); > > > + > > > + /* > > > + * Userspace must disable interrupts on the device prior to > > > freeze so > > > + * that the device does not send any interrupts until new > > > interrupt > > > + * handlers have been established by the next kernel. > > > + */ > > > + if (vdev->irq_type != VFIO_PCI_NUM_IRQS) { > > > + pci_err(pdev, "Freeze failed! Interrupts are still > > > enabled.\n"); > > > + return -EINVAL; > > > + } > > > + > > > + ret = pci_load_saved_state(pdev, vdev->pci_saved_state); > > > + if (ret) > > > + return ret; > > > + > > > + vfio_pci_core_try_reset(vdev); > > > > I think a call to pci_clear_master() is needed here to so the device > > stops issuing memory transactions while VFIO is trying to resetting > > it. > > That doesn't sound right, reset should always work..
pcie_flr() calls pci_wait_for_pending_transaction() which times out and prints an error message. The reset seems to be working fine though after that. > > Didn't look beyond this hunk but why is a function called 'freeze' > donig a reset anyhow? Disabling bus mastering, resetting the device, and restoring vdev->pci_saved_state in freeze() is temporary. This allows us to pass the preserved device to the next kernel in a safe idle state. The plan is to remove them once enough Live Update functionality in VFIO, the PCI core, and iommufd are working. > > Sashiko also wondered if pci_clear_master() is needed to ensure that > > the device has bus mastering disabled during the kexec > > Only devices that are not being preserved.. > > > think that is actually an issue. vfio_pci_core_enable() makes sure > > that vdev->pci_saved_state has bus mastering disabled, so > > pci_restore_state() below here should guaranteed it is clear during > > kexec. > > .. and bus mastering should not be disabled on any device expecting to > be preserved ..
