On Wed, Mar 05, 2025 at 03:48:18PM -0700, Jim Fehlig via Devel wrote:
> Introduce support for QEMU's new mapped-ram stream format [1].
> mapped-ram can be enabled by setting the 'save_image_format'
> setting in qemu.conf to 'sparse'.
>
> To use mapped-ram with QEMU:
> - The 'mapped-ram' migration capability must be set to true
> - The 'multifd' migration capability must be set to true and
> the 'multifd-channels' migration parameter must set to 1
> - QEMU must be provided an fdset containing the migration fd
> - The 'migrate' qmp command is invoked with a URI referencing the
> fdset and an offset where to start reading or writing the data
> stream, e.g.
>
> {"execute":"migrate",
> "arguments":{"detach":true,"resume":false,
> "uri":"file:/dev/fdset/0,offset=0x11921"}}
>
> The mapped-ram stream, in conjunction with direct IO and multifd
> support provided by subsequent patches, can significantly improve
> the time required to save VM memory state. The following tables
> compare mapped-ram with the existing, sequential save stream. In
> all cases, the save and restore operations are to/from a block
> device comprised of two NVMe disks in RAID0 configuration with
> xfs (~8600MiB/s). The values in the 'save time' and 'restore time'
> columns were scraped from the 'real' time reported by time(1). The
> 'Size' and 'Blocks' columns were provided by the corresponding
> outputs of stat(1).
>
> VM: 32G RAM, 1 vcpu, idle (shortly after boot)
>
> | save | restore |
> | time | time | Size | Blocks
> -----------------------+---------+---------+--------------+--------
> legacy | 6.193s | 4.399s | 985744812 | 1925288
> -----------------------+---------+---------+--------------+--------
> mapped-ram | 5.109s | 1.176s | 34368554354 | 1774472
> -----------------------+---------+---------+--------------+--------
> legacy + direct IO | 5.725s | 4.512s | 985765251 | 1925328
> -----------------------+---------+---------+--------------+--------
> mapped-ram + direct IO | 4.627s | 1.490s | 34368554354 | 1774304
> -----------------------+---------+---------+--------------+--------
> mapped-ram + direct IO | | | |
> + multifd-channels=8 | 4.421s | 0.845s | 34368554318 | 1774312
> -------------------------------------------------------------------
>
> VM: 32G RAM, 30G dirty, 1 vcpu in tight loop dirtying memory
>
> | save | restore |
> | time | time | Size | Blocks
> -----------------------+---------+---------+--------------+---------
> legacy | 25.800s | 14.332s | 33154309983 | 64754512
> -----------------------+---------+---------+--------------+---------
> mapped-ram | 18.742s | 15.027s | 34368559228 | 64617160
> -----------------------+---------+---------+--------------+---------
> legacy + direct IO | 13.115s | 18.050s | 33154310496 | 64754520
> -----------------------+---------+---------+--------------+---------
> mapped-ram + direct IO | 13.623s | 15.959s | 34368557392 | 64662040
> -----------------------+-------- +---------+--------------+---------
> mapped-ram + direct IO | | | |
> + multifd-channels=8 | 6.994s | 6.470s | 34368554980 | 64665776
> --------------------------------------------------------------------
>
> As can be seen from the tables, one caveat of mapped-ram is the logical
> file size of a saved image is basically equivalent to the VM memory size.
> Note however that mapped-ram typically uses fewer blocks on disk, hence
> the name 'sparse' for 'save_image_format'.
>
> Also note the mapped-ram stream is incompatible with the existing stream
> format, hence mapped-ram cannot be used to restore an image saved with
> the existing format and vice versa.
>
> [1]
> https://gitlab.com/qemu-project/qemu/-/blob/master/docs/devel/migration/mapped-ram.rst?ref_type=heads
>
> Signed-off-by: Jim Fehlig <[email protected]>
> ---
> src/qemu/qemu_driver.c | 23 +++++-
> src/qemu/qemu_migration.c | 149 ++++++++++++++++++++++++++------------
> src/qemu/qemu_migration.h | 4 +-
> src/qemu/qemu_monitor.c | 34 +++++++++
> src/qemu/qemu_monitor.h | 4 +
> src/qemu/qemu_saveimage.c | 31 +++++---
> src/qemu/qemu_saveimage.h | 1 +
> src/qemu/qemu_snapshot.c | 7 +-
> 8 files changed, 191 insertions(+), 62 deletions(-)
Reviewed-by: Daniel P. Berrangé <[email protected]>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
