On Wed, Nov 18, 2026 10:32:35AM -0800, Fabiano Rosas wrote:
> Dongli Zhang <dongli. zhang@ oracle. com> writes: > Hi Fabiano and Peter, > >
> On
> Tue, Nov 17, 2026 11: 25: 50AM -0800, Peter Xu wrote: >> On Tue, Nov 17, 2026
> at
> 04: 01: 45PM -0300, Fabiano Rosas wrote: > Sorry for the >>
>
> Dongli Zhang <[email protected]> writes:
>
>> Hi Fabiano and Peter,
>>
>> On Tue, Nov 17, 2026 11:25:50AM -0800, Peter Xu wrote:
>>> On Tue, Nov 17, 2026 at 04: 01: 45PM -0300, Fabiano Rosas wrote: > Sorry
>>> for the
>>> delay, we're all swimming in slop these days. The only > feedback I have at
>>> the
>>> moment is that this feature needs tests. What's > your plan in this regard?
>>>
>>> On Tue, Nov 17, 2026 at 04:01:45PM -0300, Fabiano Rosas wrote:
>>>> Sorry for the delay, we're all swimming in slop these days. The only
>>>> feedback I have at the moment is that this feature needs tests. What's
>>>> your plan in this regard? Can this run with nested virtualization? Or
>>>> perhaps is there a way to mock the kernel interfaces and ensure QEMU
>>>> does the right thing even without them? We don't need much, but the code
>>>> definitely needs to be exercised somehow.
>>
>> Thank you very much for the response and suggestion. I will follow this to
>> make
>> the feature more robust.
>>
>> So far, the only way to test it is through kexec, which not only takes time
>> but
>> also isn't friendly to unit testing.
>>
>> Based on my understanding of LUO/KHO, with the latest mainline Linux kernel,
>> once the fds are preserved, there is no way to retrieve them until after
>> kexec.
>>
>> Therefore, perhaps we could implement the LUO API functions in QEMU through
>> an
>> ops interface.
>>
>> 1. The first idea is to implement an emulator so that the ops interface can
>> support both the real API and a test API. The test API would talk to the
>> emulator to emulate the preserve/retrieve procedure.
>>
>
> Where does that emulator sit exactly? Because the problem of
> transfering state will be the same as in 2 below, no? Since QEMU source
> needs to exit.
The emulator can be a daemon built from the QEMU source tree. QEMU talks to it
over a Unix domain socket.
For cpr-luo, there would be two sets of ops:
- The regular ops use /dev/liveupdate to preserve and retrieve supported fds,
including memfd and potentially VFIO in the future. I actually did some
experiments with the PCI/IOMMU/VFIO LUO patchset and was able to preserve and
retrieve a Mellanox PF from QEMU.
- The test ops use the Unix domain socket to create sessions and preserve and
retrieve fds. This simulates the workflow for QEMU.
This would let developers quickly test the QEMU side with different devices
without having to kexec or reboot.
For example, I often use live migration to a file to validate QEMU devices.
I also use live migration to a file, reboot host, and load from file to simulate
live migration between two machines with different tsc frequencies.
>
>> 2. The second idea is to implement a set of test ops, with QEMU launched by a
>> parent process. The source QEMU could preserve fds, and the target QEMU could
>> retrieve them through the parent process.
>>
>
> Something like migration-test as the parent process sounds easy enough.
>
>> 3. The third idea is to implement something in the kernel to emulate a quick
>> switch between outgoing and incoming fds so that we can retrieve them
>> immediately after preserving them.
>>
>
> Anything in-kernel would have to be part of a larger LUO testing
> approach I think. Otherwise we waste effort. I don't suppose the kernel
> side has any sort of selftests (or similar) for this?
Based on my hands-on experience, I need to do a kexec reboot to test the full
preserve-and-retrieve lifecycle.
>
> What can we get from /dev/liveupdate in the host without having to
> kexec? On a quick look, I see systemd has a test-luo program, I wonder
> if they have a way to test outside of a VM.
Based on my understanding of the latest mainline Linux code, new sessions are
added to luo_session_global.outgoing, while sessions from the previous boot are
added to luo_session_global.incoming. There's no way to switch between the two
within the same boot, so we can't retrieve a session after creating it.
I am not very familiar with the systemd code, so I asked AI to check it. It said
the luo test in systemd doesn't try to retrieve the session until after a kexec
reboot.
Thank you very much!
Dongli Zhang
>
> One other aspect aside from the normal functioning of the feature is how
> does QEMU behave when the host doesn't support LUO or the configuration
> is not quite right. Although easy to fix, those issues tend to be
> numerous when the support for a kernel feature is first implemented in
> QEMU. A test that doesn't access the host at all might miss those.
>
>> Options (1) and (2) seem better. While (2) looks easier to implement for
>> quick
>> unit testing, (1) would be better for testing with arbitrary devices and
>> configurations in cpr-luo mode.
>>
>
> Let's understand a bit better what the options are, but we don't need to
> do anything fancy, we can improve the testing as the work evolves.
>
>> Thank you very much!
>>
>> Dongli Zhang
>>
>>>
>>> Apologize from my side too on being slow, I'm going to start reading mails
>>> on the list this week, but it'll still be slow, together with other things.
>>> Just to say, this one is on my review list. Sorry again.
>>>
>>> --
>>> Peter Xu
>>>
>