Anthony Liguori <anth...@codemonkey.ws> wrote:
> On 11/23/2010 05:03 PM, Juan Quintela wrote:
>> From: Juan Quintela<quint...@trasno.org>
>>
>> cheking each 64 pages is a random magic number as good as any other.
>> We don't want to test too many times, but on the other hand,
>> qemu_get_clock_ns() is not so expensive either.
>>
>> Signed-off-by: Juan Quintela<quint...@trasno.org>
>> Signed-off-by: Juan Quintela<quint...@redhat.com>
>>
>
> This is just introducing a future problem.
>
> BufferedFile should hit the qemu_file_rate_limit check when the socket
> buffer gets filled up. Are you claiming that the problem is due to
> trying to send a large number of zero pages? Is this purely synthetic
> because you're testing an utilized large memory guest?
Get big memory machine
Launch a 64GB guest.
Let it idle after start
Migrate it, and you will get stalls of around 3-4 seconds (see the patch
0/6 file). I got 375000 pages sent in one go (all zero pages).
In not idle/new guests, it don't happen (you don't have several 500MB of
zeropages together in one 1GB guest).
The other solution to this problem is to just limit the number of pages
that we can sent in one go, but the problem is what number to put there.
For my test hosts, around 8000 pages are ok, but this is a 16core
machine with 128GB of RAM. I don't know how to get a value that works
for every machine.
The timout value is not trivial either. We need "clearly" that is less
than 100ms (otherwise the rate limit calculation is completely broken),
and other things in qemu don't work very well (monitor cames to mind).
I tested several values:
- no limit: migration takes 112 seconds, but we have lots of stalls of
around 3 seconds.
- 15ms (migration_max_downtime()/2). Migration total time moves to 404
seconds.
- 50ms: sweet spot, bigger value that don't have stalls. (migration
takes 150s)
- 75ms-100ms: try both of them, total speed is a bit better (~125s), but
we are back to having stalls.
Later, Juan.
> Regards,
>
> Anthony Liguori
>
>> ---
>> arch_init.c | 16 ++++++++++++++++
>> 1 files changed, 16 insertions(+), 0 deletions(-)
>>
>> diff --git a/arch_init.c b/arch_init.c
>> index d32aaae..b463798 100644
>> --- a/arch_init.c
>> +++ b/arch_init.c
>> @@ -40,6 +40,7 @@
>> #include "net.h"
>> #include "gdbstub.h"
>> #include "hw/smbios.h"
>> +#include "buffered_file.h"
>>
>> #ifdef TARGET_SPARC
>> int graphic_width = 1024;
>> @@ -218,6 +219,7 @@ int ram_save_live(Monitor *mon, QEMUFile *f, int stage,
>> void *opaque)
>> uint64_t bytes_transferred_last;
>> uint64_t t0;
>> double bwidth = 0;
>> + int i;
>>
>> if (stage< 0) {
>> cpu_physical_memory_set_dirty_tracking(0);
>> @@ -261,6 +263,7 @@ int ram_save_live(Monitor *mon, QEMUFile *f, int stage,
>> void *opaque)
>> bytes_transferred_last = bytes_transferred;
>> t0 = qemu_get_clock_ns(rt_clock);
>>
>> + i = 0;
>> while (!qemu_file_rate_limit(f)) {
>> int bytes_sent;
>>
>> @@ -269,6 +272,19 @@ int ram_save_live(Monitor *mon, QEMUFile *f, int stage,
>> void *opaque)
>> if (bytes_sent == 0) { /* no more blocks */
>> break;
>> }
>> + /* we want to check in the 1st loop, just in case it was the 1st time
>> + and we had to sync the dirty bitmap.
>> + qemu_get_clock_ns() is a bit expensive, so we only check each
>> some
>> + iterations
>> + */
>> + if ((i& 63) == 0) {
>> + uint64_t t1 = (qemu_get_clock_ns(rt_clock) - t0) / 1000000;
>> + if (t1> buffered_file_interval/2) {
>> + printf("big delay %ld milliseconds, %d iterations\n", t1,
>> i);
>> + break;
>> + }
>> + }
>> + i++;
>> }
>>
>> t0 = qemu_get_clock_ns(rt_clock) - t0;
>>
