Re: [Qemu-devel] [PATCH] vl: Adjust the place of calling mlockall to speedup VM's startup

[zhanghailiang]

On 2014/9/23 16:30, Michael S. Tsirkin wrote:

On Tue, Sep 23, 2014 at 03:57:47PM +0800, zhanghailiang wrote:

If we configure mlock=on and memory policy=bind at the same time,
It will consume lots of time for system to treat with memory,
especially when call mbind after mlockall.

Adjust the place of calling mlockall, calling mbind before mlockall
can remarkably reduce the time of VM's startup.

Signed-off-by: zhanghailiang <zhang.zhanghaili...@huawei.com>

The idea makes absolute sense to me:
bind after lock will force data copy of
all pages. bind before lock gives us an
indication where to put data on fault in.

Acked-by: Michael S. Tsirkin <m...@redhat.com>

Thanks for your quick reviewing..

Best Regards,
zhanghailiang

---
Hi,

Actually, for mbind and mlockall, i have made a test about the time consuming
for the different call sequence.

The results is shown below. It is obviously that mlockall called before mbind is
more time-consuming.

Besides, this patch is OK with memory hotplug.

TEST CODE:
     if (mbind_first) {
         printf("mbind --> mlockall\n");
         mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2,
               MPOL_MF_STRICT | MPOL_MF_MOVE);
         mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
               MPOL_MF_STRICT | MPOL_MF_MOVE);
         mlockall(MCL_CURRENT | MCL_FUTURE);
     } else {
         printf("mlockall --> mbind\n");
         mlockall(MCL_CURRENT | MCL_FUTURE);
         mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2 ,
               MPOL_MF_STRICT | MPOL_MF_MOVE);
         mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2,
               MPOL_MF_STRICT | MPOL_MF_MOVE);
     }

RESULT 1:
#time /home/test_mbind 10240 0
memroy size 10737418240
mlockall --> mbind

real    0m11.886s
user    0m0.004s
sys     0m11.865s
#time /home/test_mbind 10240 1
memroy size 10737418240
mbind --> mlockall

real    0m5.334s
user    0m0.000s
sys     0m5.324s

RESULT 2:
#time /home/test_mbind 4096 0
memroy size 4294967296
mlockall --> mbind

real    0m5.503s
user    0m0.000s
sys     0m5.492s
#time /home/test_mbind 4096 1
memroy size 4294967296
mbind --> mlockall

real    0m2.139s
user    0m0.000s
sys     0m2.132s

Best Regards,
zhanghailiang
---
  vl.c | 11 +++++------
  1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/vl.c b/vl.c
index dc792fe..adf4770 100644
--- a/vl.c
+++ b/vl.c
@@ -134,6 +134,7 @@ const char* keyboard_layout = NULL;
  ram_addr_t ram_size;
  const char *mem_path = NULL;
  int mem_prealloc = 0; /* force preallocation of physical target memory */
+int enable_mlock = false;
  int nb_nics;
  NICInfo nd_table[MAX_NICS];
  int autostart;
@@ -1421,12 +1422,8 @@ static void smp_parse(QemuOpts *opts)

  }

-static void configure_realtime(QemuOpts *opts)
+static void realtime_init(void)
  {
-    bool enable_mlock;
-
-    enable_mlock = qemu_opt_get_bool(opts, "mlock", true);
-
      if (enable_mlock) {
          if (os_mlock() < 0) {
              fprintf(stderr, "qemu: locking memory failed\n");
@@ -3973,7 +3970,7 @@ int main(int argc, char **argv, char **envp)
                  if (!opts) {
                      exit(1);
                  }
-                configure_realtime(opts);
+                enable_mlock = qemu_opt_get_bool(opts, "mlock", true);
                  break;
              case QEMU_OPTION_msg:
                  opts = qemu_opts_parse(qemu_find_opts("msg"), optarg, 0);
@@ -4441,6 +4438,8 @@ int main(int argc, char **argv, char **envp)

      machine_class->init(current_machine);

+    realtime_init();
+
      audio_init();

      cpu_synchronize_all_post_init();
--
1.7.12.4

.