On Sun, Aug 23, 2026 11:29:53PM -0700, Markus Armbruster wrote:
Dongli Zhang <[email protected]> writes:
>
>> On Fri, Aug 21, 2026 12:13:30AM -0700, Markus Armbruster wrote:
>>> Dongli Zhang <[email protected]> writes:
>>>
>>>> On Thu, Aug 20, 2026 12:09:19AM -0700, Markus Armbruster wrote:
>>>>> Dongli Zhang <[email protected]> writes:
[snip]
>>> message.
>>
>> Sure.
>>
>>>
>>> We should also help users decide whether to use the feature. I gather
>>> it's useful with migration mode cpr-exec. Any other use cases? Does it
>>> even work with other migration modes? If yes, which ones, and what
>>> happens when you try with another one?
>>
>> So far I only added support for cpr-exec. I have not tried othe modes.
>
> What happens when you try to use it with other migration modes?
If I use memfd transport without cpr-exec mode, it will be denied.
The code checks that only cpr-exec mode is supported.
(qemu) migrate -d memfd:
Error: memfd transport is only supported with cpr-exec
>
>> In my opinion, cpr-transfer could also use a memfd-backed channel, but I am
>> not
>> sure it is necessary.
>>
>> For cpr-transfer, the source and destination are separate QEMU processes. If
>> we
>> want to pass a memfd between them, we still need an IPC mechanism that can
>> transfer file descriptors, such as a Unix domain socket with SCM_RIGHTS. That
>> reduces the value of using memfd as the migration channel, because we need at
>> least one Unix domain socket anyway to pass the memfd fd and coordinate the
>> handoff.
>>
>> If a Unix domain socket is already required, the main benefit of memfd would
>> likely be performance, especially when the device state is large.
>
> We'd want to measure the performance impact to see whether it's worth
> complicating configuration and code.
I may create a VM with many virtio devices to maximize the storage overhead of
the VM device state. However, I don't think the goal is to improve performance.
>
>> cpr-exec is different. The new QEMU is started via exec(), so the memfd fd
>> can
>> be preserved across exec by clearing CLOEXEC and passing the fd number
>> through
>> an environment variable. This allows us to add a memfd-backed migration
>> channel
>> without needing an external file or socket for the device state.
>
> What are the actual user-visible benefits?
The goal is to avoid the VM state file used during live update with cpr-exec.
So, the user-visible benefit is that the file is no longer required. Live update
with cpr-exec no longer relies on a file on disk.
>
> Performance? Provide measurements, please.
Not performance.
>
> Simpler configuration? Not sure that would be worth the special case.
Yes. No file required.
"migrate -d memfd:" vs. "migrate -d file:vm.state".
"migrate_incoming memfd:" vs. "migrate_incoming file:vm.state".
>
> Anything else?
No. Writable storage is no longer required to cache the VM device state file.
That's the only explicit benefit so far.
>
>> One potential future use case is LUO/KHO support for QEMU. In that case,
>> device
>> state could be preserved in memory as a memfd and potentially survive across
>> a
>> KHO kexec. We won't need to preserve anything on storage. Indeed, I did some
>> internal tests using memfd to preserve device state across kexec although
>> using
>> different implementation, and it worked.
>>
>> For other live migration modes, or even local migration, I do not think this
>> is
>> useful. For example, saving to and loading from a file won't use memfd, as
>> the
>> QEMU processes are terminated during the this process.
>
> User documentation should spell out when memfd can be used. I recommend
> to make it usable only when it's known to improve things in at least
> some scenario that matters. Code should cleanly reject attempts to use
> memfd when it's not usable.
>
> User documentation should help users decide whether to use it.
>
> The commit message(s) may need to provide additional detail on how
> things improve.
>
Thank you very much for feedback!
Dongli Zhang
