On Mon, Nov 03, 2025 at 11:57:50AM -0700, Jon Kohler wrote:
> Increase MAX_MEM_PREALLOC_THREAD_COUNT from 16 to 32. This was last
> touched in 2017 [1] and, since then, physical machine sizes and VMs
> therein have continue to get even bigger, both on average and on the
> extremes.
>
> For very large VMs, using 16 threads to preallocate memory can be a
> non-trivial bottleneck during VM start-up and migration. Increasing
> this limit to 32 threads reduces the time taken for these operations.
>
> Test results from quad socket Intel 8490H (4x 60 cores) show a fairly
> linear gain of 50% with the 2x thread count increase.
>
> ---------------------------------------------
> Idle Guest w/ 2M HugePages | Start-up time
> ---------------------------------------------
> 240 vCPU, 7.5TB (16 threads) | 2m41.955s
> ---------------------------------------------
> 240 vCPU, 7.5TB (32 threads) | 1m19.404s
> ---------------------------------------------
If we're configuring a guest with 240 vCPUs, then this implies the admin
is expecting that the guest will consume upto 240 host CPUs worth of
compute time.
What is the purpose of limiting the number of prealloc threads to a
value that is an order of magnitude less than the number of vCPUs the
guest has been given ?
Have you measured what startup time would look like with 240 prealloc
threads ? Do we hit some scaling limit before that point making more
prealloc threads counter-productive ?
I guess there could be different impact for hotadd vs cold add. With
cold startup the vCPU threads are not yet consuming CPU time, so we
can reasonably consume that resource for prealloc, where as for
hot-add any prealloc is on top of what vCPUs are already consuming.
> [1] 1e356fc14bea ("mem-prealloc: reduce large guest start-up and migration
> time.")
>
> Signed-off-by: Jon Kohler <[email protected]>
> ---
> util/oslib-posix.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/util/oslib-posix.c b/util/oslib-posix.c
> index 3c14b72665..dc001da66d 100644
> --- a/util/oslib-posix.c
> +++ b/util/oslib-posix.c
> @@ -61,7 +61,7 @@
> #include "qemu/memalign.h"
> #include "qemu/mmap-alloc.h"
>
> -#define MAX_MEM_PREALLOC_THREAD_COUNT 16
> +#define MAX_MEM_PREALLOC_THREAD_COUNT 32
>
> struct MemsetThread;
>
> --
> 2.43.0
>
>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
