On 11/30/2010 12:04 PM, Juan Quintela wrote:
Anthony Liguori<anth...@codemonkey.ws> wrote:
On 11/30/2010 10:32 AM, Juan Quintela wrote:
"Michael S. Tsirkin"<m...@redhat.com> wrote:
On Tue, Nov 30, 2010 at 04:40:41PM +0100, Juan Quintela wrote:
Basically our bitmap handling code is "exponential" on memory size,
I didn't realize this. What makes it exponential?
Well, 1st of all, it is "exponential" as you measure it.
stalls by default are:
1-2GB: milliseconds
2-4GB: 100-200ms
4-8GB: 1s
64GB: 59s
400GB: 24m (yes, minutes)
That sounds really exponential.
How are you measuring stalls btw?
At the end of the ram_save_live(). This was the reason that I put the
information there.
for the 24mins stall (I don't have that machine anymore) I had less
"exact" measurements. It was the amount that it "decided" to sent in
the last non-live part of memory migration. With the stalls& zero page
account, we just got to the point where we had basically infinity speed.
That's not quite guest visible.
It only is a "stall" if the guest is trying to access device emulation
and acquiring the qemu_mutex. A more accurate measurement would be
something that measured guest availability. For instance, I tight loop
of while (1) { usleep(100); gettimeofday(); } that then recorded periods
of unavailability > X.
Of course, it's critically important that a working version of pvclock
be available int he guest for this to be accurate.
Regards,
Anthony Liguori
Later, Juan.
Regards,
Anthony Liguori
Now the other thing is the cache size.
with 64GB of RAM, we basically have a 16MB bitmap size, i.e. we blow the
cache each time that we run ram_save_live().
This is one of the reasons why I don't want to walk the bitmap on
ram_save_remaining().
ram_save_remaining() is linear with memory size (previous my changes).
ram_save_live is also linear on the memory size, so we are in a worse
case of n^n (notice that this is the worst case, we normally do much
better n^2, n^3 or so).
Add to this that we are blowing the caches with big amounts of memory
(we don't do witch smaller sizes), and you can see that our behaviour is
clearly exponential.
As I need to fix them, I will work on them today/tomorrow.
Later, Juan.
