Re: [PATCH v8 0/6] migration: reduce time of loading non-iterable vmstate

[Chuang Xu]

Hi, Paolo,

A few months ago, Juan told me that this series requires your or someone 
familiar with memory API's feedback.

Could you please review it and provide some suggestions?

On 2023/3/17 下午4:18, Chuang Xu wrote:
In this version:

- delete useless line change.
- update comments and commit messages.

The duration of loading non-iterable vmstate accounts for a significant
portion of downtime (starting with the timestamp of source qemu stop and
ending with the timestamp of target qemu start). Most of the time is spent
committing memory region changes repeatedly.

This patch packs all the changes to memory region during the period of  
loading non-iterable vmstate in a single memory transaction. With the
increase of devices, this patch will greatly improve the performance.

Here are the test1 results:
test info:
- Host
   - Intel(R) Xeon(R) Platinum 8362 CPU
   - Mellanox Technologies MT28841
- VM
   - 32 CPUs 128GB RAM VM
   - 8 16-queue vhost-net device
   - 16 4-queue vhost-user-blk device.

        time of loading non-iterable vmstate     downtime
before           112 ms                           285 ms
after            20 ms                            194 ms


In test2, we keep the number of the device the same as test1, reduce the
number of queues per device:

Here are the test2 results:
test info:
- Host
   - Intel(R) Xeon(R) Platinum 8362 CPU
   - Mellanox Technologies MT28841
- VM
   - 32 CPUs 128GB RAM VM
   - 8 1-queue vhost-net device
   - 16 1-queue vhost-user-blk device.

        time of loading non-iterable vmstate     downtime
before           65 ms                            151 ms

after            19 ms                            100 ms


In test3, we keep the number of queues per device the same as test1, reduce
the number of devices:

Here are the test3 results:
test info:
- Host
   - Intel(R) Xeon(R) Platinum 8362 CPU
   - Mellanox Technologies MT28841
- VM
   - 32 CPUs 128GB RAM VM
   - 1 16-queue vhost-net device
   - 1 4-queue vhost-user-blk device.

        time of loading non-iterable vmstate     downtime
before           24 ms                            51 ms
after            9 ms                             36 ms


As we can see from the test results above, both the number of queues and
the number of devices have a great impact on the time of loading non-iterable
vmstate. The growth of the number of devices and queues will lead to more
mr commits, and the time consumption caused by the flatview reconstruction
will also increase.

Please review, Chuang

Thanks!