On Wed, 16 Jul 2025 at 19:07, Steve Sistare <steven.sist...@oracle.com> wrote:
>
> Close a race condition that causes cpr-transfer to lose VFIO
> interrupts on ARM.
>
> CPR stops VCPUs but does not disable VFIO interrupts, which may continue
> to arrive throughout the transition to new QEMU.
>
> CPR calls kvm_irqchip_remove_irqfd_notifier_gsi in old QEMU to force
> future interrupts to the producer eventfd, where they are preserved.
> Old QEMU then destroys the old KVM instance. However, interrupts may
> already be pending in KVM state. To preserve them, call ioctl
> KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES to flush them to guest RAM, where
> they will be picked up when the new KVM+VCPU instance is created.
>
> Signed-off-by: Steve Sistare <steven.sist...@oracle.com>
> +static int kvm_arm_gicv3_notifier(NotifierWithReturn *notifier,
> + MigrationEvent *e, Error **errp)
> +{
> + if (e->type == MIG_EVENT_PRECOPY_DONE) {
> + GICv3State *s = container_of(notifier, GICv3State, cpr_notifier);
> + return kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
> + KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES,
> + NULL, true, errp);
> + }
> + return 0;
> +}
>
> static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
> {
> @@ -883,13 +895,17 @@ static void kvm_arm_gicv3_realize(DeviceState *dev,
> Error **errp)
> GICD_CTLR)) {
> error_setg(&s->migration_blocker, "This operating system kernel does
> "
> "not support vGICv3 migration");
> - if (migrate_add_blocker(&s->migration_blocker, errp) < 0) {
> + if (migrate_add_blocker_modes(&s->migration_blocker, MIG_MODE_NORMAL,
> + MIG_MODE_CPR_TRANSFER, errp) < 0) {
Why did you change this? It's the general "if no support, can't
migrate at all" check, which seems unrelated to cpr-transfer.
> return;
> }
> }
> if (kvm_device_check_attr(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
> KVM_DEV_ARM_VGIC_SAVE_PENDING_TABLES)) {
> qemu_add_vm_change_state_handler(vm_change_state_handler, s);
> + migration_add_notifier_mode(&s->cpr_notifier,
> + kvm_arm_gicv3_notifier,
> + MIG_MODE_CPR_TRANSFER);
> }
> }
Otherwise the patch looks OK in general shape, but I know
nothing about cpr-transfer so a review from somebody on the
migration side would be helpful.
thanks
-- PMM
