On Mon, Mar 8, 2010 at 11:06 AM, Martin Mueller - Sun Germany SE
<Martin.Mueller at sun.com> wrote:
> Hi *
>
> I wanted to share some timing data of LDOM V1.3 warm migrations. The
> setup under test was:
>
> * two T5120 connected via switched Fast Ethernet
> * 1.4GHz T2 Chips
> * S10U8+latest EIS CD, LDOMs V1.3 SUNWldm software
> * the moving LDOM was running "xclock -update 1" (to have it doing
> something, and to monitor quiescing during migration)
> * Timing command:
> ptime ldm migrate -p ./rootpw <ldom name> <target host>
> * up to six migrations back and forth to get some statistical evidence
>
> The result is attached:
> * The diagram contains error bars, the trend line seems to be reliable
> (as reliable as a sample of five to six can be ;-))
> * the "governing law" for the run time seems to be
>
> 27.2 sec + 14.8 sec/GB * (RAM LDom)
>
> In other words: the basic duration is 27 sec plus another 14.8 sec per
> GB RAM the guest has assigned
It looks like there are two things that may need some attention:
1) What is happening in that fixed time of 27.2 seconds?
The solution to this is likely somewhere where code has to be changed.
2) Why is it only getting 540 Mbits/sec (1 / 14.8 * 8 * 1000) of
network throughput?
During the memory copy is a CPU pegged ("mpstat 1", possibly "prstat
-mLc -n 5 1")? If so a fix probably requires code changes to offload
crypto. If not, there is is a pretty good chance that a bit of network
tuning will get your throughput much closer to wire speed.
http://unix.derkeiler.com/Newsgroups/comp.unix.solaris/2007-04/msg00439.html
suggests several parameters that are pretty common to set - go look at
disclosures for pretty much any benchmark published by Sun. Note that
I haven't read that post closely, but the parameters set at the top
are consistent with what my experience suggests is needed to get wire
speed on gigabit NICs.
>
> Regards
> Martin
Thank you very much for sharing your work!
>
>
> --
> Martin M?ller ? ? ? ? ? ? ? ? ? ? ? ? martin.mueller at sun.com
> Systems Infrastructure Ambassador ? ? Tel.: +49 2102 45 11740
> Sun Microsystems GmbH ? ? ? ? ? ? ? ? Fax: +49 2102 499516
> D-40880 Ratingen ? ? ? ? ? ? ? ? ? ? ?Mobile: +49 172 8618483
>
> Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1, D-85551
> Kirchheim-Heimstetten
> Amtsgericht Muenchen: HRB 161028
> Geschaeftsfuehrer: Thomas Schr?der, Wolfgang Engels, Wolf Frenkel
> Vorsitzender des Aufsichtsrates: Martin Haering
>
> _______________________________________________
> ldoms-discuss mailing list
> ldoms-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss
>
>
--
Mike Gerdts
http://mgerdts.blogspot.com/
