I am rebasing the patchset on top of the most recent QEMU commit, especially "migration/cpr: support vhost-vsock devices".
Meanwhile, any feedback on the proposal is appreciated. So far, the main idea is to reuse as much code as possible from cpr-exec and cpr-transfer. 1. Introduce the new cpr-luo mode. 2. Use a UUID to identify a LUO session. 3. Introduce APIs to create, preserve, and retrieve sessions and fds. 4. As LUO uses the name, id, and token instead of an fd, create an independent VMStateDescription rather than reuse the one for cpr-exec and cpr-transfer. 5. Compute the token when transforming entries from the hash table into the list. 6. Preserve fds on the source. Follow the approach used in cpr-exec and cpr-transfer. 7. Retrieve fds on the target. Follow the approach used in cpr-exec and cpr-transfer. The tricky part is where to set the cpr incoming mode. 8. There is no need to modify save_fd() or load_fd(), as we only need to migrate the corresponding name and token, not the fds. 9. Handle corner-case errors. 10. Block VFIO. Thank you very much! Dongli Zhang On Mon, Oct 12, 2026 4:55:56PM -0700, Dongli Zhang wrote: > This series introduces cpr-luo, a CPR migration mode that uses the Linux Live > Update Orchestrator (LUO) and Kexec Handover (KHO) to support KVM host kernel > updates through kexec. On the outgoing side, QEMU saves CPR state in a memfd > and > preserves > > This series introduces cpr-luo, a CPR migration mode that uses the Linux > Live Update Orchestrator (LUO) and Kexec Handover (KHO) to support KVM host > kernel updates through kexec. > > On the outgoing side, QEMU saves CPR state in a memfd and preserves that fd > together with the RAMBlock backing fds in LUO. KHO carries the preserved LUO > state into the new kernel. The incoming QEMU retrieves the CPR state and > RAMBlock backing fds from LUO after kexec. > > The regular migration stream carries device vmstate. > > Although VFIO is currently blocked, I have an offline implementation that > skips cpr-luo mode in cpr_is_incoming(), skips precopy, and carefully handles > monitor_fd_param() although I never use it, so that VFIO is always > reconstructed. It works with both iommufd and legacy for mlx5_vfio_pci by > saving the VFIO and vendor specific state in vmstate as well. > > The result is that guest RAM remains available across a kexec-based host > kernel update without copying its contents through the migration stream. > > Here is the main idea of the patchset: > > 1. Introduce the new cpr-luo mode. > 2. Use a UUID to identify a LUO session. > 3. Introduce APIs to create, preserve, and retrieve sessions and fds. > 4. As LUO uses the name, id, and token instead of an fd, create an > independent VMStateDescription rather than reuse the one for cpr-exec and > cpr-transfer. > 5. Compute the token when transforming entries from the hash table into the > list. > 6. Preserve fds on the source. Follow the approach used in cpr-exec and > cpr-transfer. > 7. Retrieve fds on the target. Follow the approach used in cpr-exec and > cpr-transfer. The tricky part is where to set the cpr incoming mode. > 8. There is no need to modify save_fd() or load_fd(), as we only need to > migrate the corresponding name and token, not the fds. > 9. Handle corner-case errors. > 10. Block VFIO. > > I may need to rebase the current patchset on top of the patches below, > from which I learned quite a lot in each iteration. While working on this > patchset, I identified some cleanup and code issues in cpr-exec and > cpr-transfer. I may send fixes for them separately. > > [PATCH v16 00/22] migration/cpr: support vhost-vsock devices > https://urldefense.com/v3/__https://lore.kernel.org/all/20261012-up-vsock-vhost- > [email protected]/__;!!ACWV5N9M2RV99hQ!N58C4d74- > zpeHeXtxKWUAIRgf18-Xzk7_V7p49rzvh29deg5g2oBeAn0R5b89vJzLBPv6rkzjTo-8F0Iy16Z0oRy$ > > <https://urldefense.com/v3/__https://lore.kernel.org/all/[email protected]/__;!!ACWV5N9M2RV99hQ!N58C4d74-zpeHeXtxKWUAIRgf18-Xzk7_V7p49rzvh29deg5g2oBeAn0R5b89vJzLBPv6rkzjTo-8F0Iy16Z0oRy$> > > Example: > > Add "kho=on liveupdate=on" to host kernel cmdline. > > Create QEMU instance with memfd. > > qemu-system-x86_64 \ > -uuid 6f1b9f5a-6b92-4a5d-ae57-7a0ddabc1234 \ > -machine q35,dump-guest-core=off,memory-backend=ram0,aux-ram-share=on \ > -accel kvm -cpu host \ > -name debug-threads=on \ > -m 8G -smp 4 \ > -object memory-backend-memfd,id=ram0,size=8G,share=on \ > -device virtio-blk-pci,drive=drive0,id=virtblk0,num-queues=4 \ > -drive file=boot.qcow2,if=none,id=drive0 \ > -netdev user,id=net0,hostfwd=tcp::5028-:22 \ > -device virtio-net-pci,netdev=net0 \ > -vnc :8 -monitor stdio > > Don't terminate QEMU process after cpr-luo migration. > > (qemu) migrate_set_parameter mode cpr-luo > (qemu) migrate -d file:cpr-luo.state > (qemu) info status > VM status: paused (postmigrate) > > Use kexec to reboot the KVM host. > > host# kexec -s -l /boot/vmlinuz-$(uname -r) --initrd=/boot/initramfs-$(uname > -r).img --reuse-cmdline > host# sync > host# kexec -e > > Resume QEMU instance after kexec reboot. > > qemu-system-x86_64 \ > -uuid 6f1b9f5a-6b92-4a5d-ae57-7a0ddabc1234 \ > -machine q35,dump-guest-core=off,memory-backend=ram0,aux-ram-share=on \ > -accel kvm -cpu host \ > -name debug-threads=on \ > -m 8G -smp 4 \ > -object memory-backend-memfd,id=ram0,size=8G,share=on \ > -device virtio-blk-pci,drive=drive0,id=virtblk0,num-queues=4 \ > -drive file=boot.qcow2,if=none,id=drive0 \ > -netdev user,id=net0,hostfwd=tcp::5028-:22 \ > -device virtio-net-pci,netdev=net0 \ > -vnc :8 -monitor stdio \ > -incoming cpr-luo:defer > > (qemu) migrate_incoming file:cpr-luo.state > (qemu) info status > VM status: running > > > Dongli Zhang (21): > qapi/migration: add cpr-luo migration mode > migration/cpr: move CprFd definition to header file > scripts/update-linux-headers: Add liveupdate.h > linux-headers: import liveupdate UAPI header > migration/cpr-luo: require a non-null machine UUID > migration/cpr-luo: add LUO session management > migration/cpr-luo: add LUO fd management helpers > migration/cpr-luo: track LUO fds > migration/cpr-luo: add LUO fd VMStateDescription > migration/cpr-luo: add tokens for LUO fds > migration/cpr: move QEMUFile fd helpers to common code > migration/cpr-luo: add state fd helpers > migration/cpr: add more arguments for cpr_walk_fd() > migration/cpr-luo: add helpers to preserve and retrieve all LUO fds > migration/cpr-luo: enable cpr-luo mode for source side preservation > migration/cpr-luo: enable cpr-luo mode for target side retrieval > migration/cpr-luo: abort session when migration fails > system/physmem: add cpr-luo blocker > monitor/qmp: close LUO session when resuming source QEMU > vfio: block cpr-luo migration > migration: cpr-luo documentation > > backends/hostmem-memfd.c | 1 + > backends/hostmem-shm.c | 1 + > docs/devel/migration/CPR.rst | 64 +++++++ > hw/vfio/cpr-iommufd.c | 17 +- > hw/vfio/cpr-legacy.c | 8 + > include/hw/vfio/vfio-cpr.h | 1 + > include/migration/cpr.h | 37 +++- > include/system/iommufd.h | 1 + > include/system/system.h | 2 + > linux-headers/linux/liveupdate.h | 239 +++++++++++++++++++++++++ > migration/cpr-exec.c | 32 +--- > migration/cpr-luo-stub.c | 78 ++++++++ > migration/cpr-luo.c | 328 ++++++++++++++++++++++++++++++++++ > migration/cpr.c | 126 +++++++++++-- > migration/meson.build | 6 + > migration/migration.c | 12 +- > migration/options.c | 6 + > migration/ram.c | 1 + > monitor/qmp-cmds.c | 7 + > qapi/migration.json | 11 +- > scripts/update-linux-headers.sh | 2 +- > system/globals.c | 5 + > system/physmem.c | 9 +- > system/vl.c | 9 + > 24 files changed, 950 insertions(+), 53 deletions(-) > > base-commit: 3377670ee96b5a68361016680a95b9013709d4b7 > > Thank you very much! > > Dongli Zhang > >
