Hi Mike
I also have done some timing tests and come to the same conclusions
broadly as Martin.
On 08/03/2010 17:55, Mike Gerdts wrote:
> On Mon, Mar 8, 2010 at 11:06 AM, Martin Mueller - Sun Germany SE
> <Martin.Mueller at sun.com> wrote:
>> Hi *
>>
>> I wanted to share some timing data of LDOM V1.3 warm migrations. The
>> setup under test was:
>>
>> * two T5120 connected via switched Fast Ethernet
>> * 1.4GHz T2 Chips
>> * S10U8+latest EIS CD, LDOMs V1.3 SUNWldm software
>> * the moving LDOM was running "xclock -update 1" (to have it doing
>> something, and to monitor quiescing during migration)
>> * Timing command:
>> ptime ldm migrate -p ./rootpw<ldom name> <target host>
>> * up to six migrations back and forth to get some statistical evidence
>>
>> The result is attached:
>> * The diagram contains error bars, the trend line seems to be reliable
>> (as reliable as a sample of five to six can be ;-))
>> * the "governing law" for the run time seems to be
>>
>> 27.2 sec + 14.8 sec/GB * (RAM LDom)
>>
>> In other words: the basic duration is 27 sec plus another 14.8 sec per
>> GB RAM the guest has assigned
>
> It looks like there are two things that may need some attention:
>
> 1) What is happening in that fixed time of 27.2 seconds?
>
> The solution to this is likely somewhere where code has to be changed.
This is the startup overhead - is also depends on a number of things -
the number of cpus in the primary and now also if a mau is selected for
the primary
>
> 2) Why is it only getting 540 Mbits/sec (1 / 14.8 * 8 * 1000) of
> network throughput?
>
> During the memory copy is a CPU pegged ("mpstat 1", possibly "prstat
> -mLc -n 5 1")? If so a fix probably requires code changes to offload
> crypto. If not, there is is a pretty good chance that a bit of network
> tuning will get your throughput much closer to wire speed.
> http://unix.derkeiler.com/Newsgroups/comp.unix.solaris/2007-04/msg00439.html
> suggests several parameters that are pretty common to set - go look at
> disclosures for pretty much any benchmark published by Sun. Note that
> I haven't read that post closely, but the parameters set at the top
> are consistent with what my experience suggests is needed to get wire
> speed on gigabit NICs.
>
>>
The system is not network limited - but limited by the amount of work to
be done by the primary domain in doing memory compression and moving
bits around.
T
>> Regards
>> Martin
>
>
> Thank you very much for sharing your work!
>
>
>>
>>
>> --
>> Martin M?ller martin.mueller at sun.com
>> Systems Infrastructure Ambassador Tel.: +49 2102 45 11740
>> Sun Microsystems GmbH Fax: +49 2102 499516
>> D-40880 Ratingen Mobile: +49 172 8618483
>>
>> Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1, D-85551
>> Kirchheim-Heimstetten
>> Amtsgericht Muenchen: HRB 161028
>> Geschaeftsfuehrer: Thomas Schr?der, Wolfgang Engels, Wolf Frenkel
>> Vorsitzender des Aufsichtsrates: Martin Haering
>>
>> _______________________________________________
>> ldoms-discuss mailing list
>> ldoms-discuss at opensolaris.org
>> http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss
>>
>>
>
>
>
