Guoyi Tu <t...@chinatelecom.cn> writes:
> In the test environment, we conducted IO stress tests on all storage disks
> within a virtual machine that had five storage devices mounted.During
> testing,
> we found that the qemu process allocated a large amount of memory (~800MB)
> to handle these IO operations.
>
> When the test ended, although qemu called free() to release the allocated
> memory, the memory was not actually returned to the operating system, as
> observed via the top command.
>
> Upon researching the glibc memory management mechanism, we found that when
> small chunks of memory are allocated in user space and then released with
> free(), the glibc memory management mechanism does not necessarily return
> this memory to the operating system. Instead, it retains the memory until
> certain conditions are met for release.
Yes.
> For virtual machines that only have business operations during specific
> periods, they remain idle most of the time. However, the qemu process
> still occupies a large amount of memory resources, leading to significant
> memory resource waste.
Mitigation: the memory free()'s but not returned to the OS can be paged
out.
> To address this issue, this patch introduces an API to actively reclaim
> idle memory within the qemu process. This API effectively calls
> malloc_trim()
> to notify glibc to trim free memory. With this api, the management tool
> can monitor the virtual machine's state and call this API during idle times
> to free up the memory occupied by the virtual machine, thereby allowing more
> virtual machines to be provisioned.
How does this affect the test case you described above?
There's an existing use of malloc_trim() in util/rcu.c's
call_rcu_thread(). It's from commit 5a22ab71623:
rcu: reduce more than 7MB heap memory by malloc_trim()
Since there are some issues in memory alloc/free machenism
in glibc for little chunk memory, if Qemu frequently
alloc/free little chunk memory, the glibc doesn't alloc
little chunk memory from free list of glibc and still
allocate from OS, which make the heap size bigger and bigger.
This patch introduce malloc_trim(), which will free heap
memory when there is no rcu call during rcu thread loop.
malloc_trim() can be enabled/disabled by --enable-malloc-trim/
--disable-malloc-trim in the Qemu configure command. The
default malloc_trim() is enabled for libc.
Below are test results from smaps file.
(1)without patch
55f0783e1000-55f07992a000 rw-p 00000000 00:00 0 [heap]
Size: 21796 kB
Rss: 14260 kB
Pss: 14260 kB
(2)with patch
55cc5fadf000-55cc61008000 rw-p 00000000 00:00 0 [heap]
Size: 21668 kB
Rss: 6940 kB
Pss: 6940 kB
Signed-off-by: Yang Zhong <yang.zh...@intel.com>
Message-Id: <1513775806-19779-1-git-send-email-yang.zh...@intel.com>
Signed-off-by: Paolo Bonzini <pbonz...@redhat.com>
How would the malloc_trim() you propose interact with this one?
> Signed-off-by: Guoyi Tu <t...@chinatelecom.cn>
> Signed-off-by: dengpengcheng <dengp...@chinatelecom.cn>
> ---
> hmp-commands.hx | 13 +++++++++++++
> include/monitor/hmp.h | 1 +
> monitor/hmp-cmds.c | 14 ++++++++++++++
> monitor/qmp-cmds.c | 18 ++++++++++++++++++
> qapi/misc.json | 13 +++++++++++++
> 5 files changed, 59 insertions(+)
>
> diff --git a/hmp-commands.hx b/hmp-commands.hx
> index 06746f0afc..0fde22fc71 100644
> --- a/hmp-commands.hx
> +++ b/hmp-commands.hx
> @@ -1858,4 +1858,17 @@ SRST
> ``xen-event-list``
> List event channels in the guest
> ERST
> +
> + {
> + .name = "trim-memory",
> + .args_type = "reserved:l?",
> + .params = "[reserved]",
> + .help = "trim momory",
> + .cmd = hmp_trim_memory,
> + },
> +
> +SRST
> +``trim-memory`` *reserved*
> + try to release free memory and keep reserved bytes of free memory
> untrimmed
> +ERST
> #endif
> diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
> index 954f3c83ad..547cde0056 100644
> --- a/include/monitor/hmp.h
> +++ b/include/monitor/hmp.h
> @@ -181,5 +181,6 @@ void hmp_boot_set(Monitor *mon, const QDict *qdict);
> void hmp_info_mtree(Monitor *mon, const QDict *qdict);
> void hmp_info_cryptodev(Monitor *mon, const QDict *qdict);
> void hmp_dumpdtb(Monitor *mon, const QDict *qdict);
> +void hmp_trim_memory(Monitor *mon, const QDict *qdict);
>
> #endif
> diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
> index ea79148ee8..f842e43315 100644
> --- a/monitor/hmp-cmds.c
> +++ b/monitor/hmp-cmds.c
> @@ -460,3 +460,17 @@ void hmp_dumpdtb(Monitor *mon, const QDict *qdict)
> monitor_printf(mon, "dtb dumped to %s", filename);
> }
> #endif
> +
> +void hmp_trim_memory(Monitor *mon, const QDict *qdict)
> +{
> + int64_t reserved;
> + bool has_reserved = qdict_haskey(qdict, "reserved");
> + Error *err = NULL;
> +
> + if (has_reserved) {
> + reserved = qdict_get_int(qdict, "reserved");
> + }
> +
> + qmp_trim_memory(has_reserved, reserved, &err);
> + hmp_handle_error(mon, err);
> +}
> diff --git a/monitor/qmp-cmds.c b/monitor/qmp-cmds.c
> index f84a0dc523..878a7a646a 100644
> --- a/monitor/qmp-cmds.c
> +++ b/monitor/qmp-cmds.c
> @@ -31,6 +31,7 @@
> #include "qapi/type-helpers.h"
> #include "hw/mem/memory-device.h"
> #include "hw/intc/intc.h"
> +#include <malloc.h>
>
> NameInfo *qmp_query_name(Error **errp)
> {
> @@ -161,6 +162,23 @@ void qmp_add_client(const char *protocol, const
> char *fdname,
> }
> }
>
> +void qmp_trim_memory(bool has_reserved, int64_t reserved, Error **errp)
> +{
> +#if defined(CONFIG_MALLOC_TRIM)
> + if (!has_reserved) {
> + reserved = 1024 * 1024;
> + }
> + if (reserved < 0) {
> + error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
> + "reserved", "a >0 reserved");
> + return;
> + }
> + malloc_trim(reserved);
> +#else
> + error_setg(errp, "malloc_trim feature not configured");
Have you tried making the entire command conditional instead? Like...
> +#endif
> +}
> +
> char *qmp_human_monitor_command(const char *command_line, bool
> has_cpu_index,
> int64_t cpu_index, Error **errp)
> {
> diff --git a/qapi/misc.json b/qapi/misc.json
> index ec30e5c570..00e6f2f650 100644
> --- a/qapi/misc.json
> +++ b/qapi/misc.json
> @@ -605,3 +605,16 @@
> { 'event': 'VFU_CLIENT_HANGUP',
> 'data': { 'vfu-id': 'str', 'vfu-qom-path': 'str',
> 'dev-id': 'str', 'dev-qom-path': 'str' } }
> +
> +##
> +# @trim-memory:
> +#
> +# try to release free memory
> +#
> +# @reserved: specifies the amount of free space to leave untrimmed.
> +# default to 1MB if not specified.
> +#
> +# Since: 9.0
> +##
> +{'command': 'trim-memory',
> + 'data': {'*reserved': 'int'} }
... so:
{ 'command': 'trim-memory',
'data': {'*reserved': 'int'},
'if': 'CONFIG_MALLOC_TRIM' }
Could we do without the argument?
