On 09/10/2010 10:48 AM, Michael Kluge wrote: > OK, IB back to back is at 1,2 GB/s, 10GE back to back at 950 MB/s, with > additional lnet router I see 550 MB/s. Time for lnet tuning? >
This smells like latency - you doubled the number of hops and lost ~50% of your performance. Is your later email correct for this test - a single client->router->server ? I'd try adding higher concurrency or more clients with the same concurrenct to see if you can ameliorate some of that. I'd also benchmark each link separately with higher concurrency to see what the limits are. You should be able to graph concurrency/client count on the X axis and see a nice smooth curve that flattens out to some maximum performance. Cheers, Nic P.S to get more accurate performance numbers, you probably want to run 'lst stat writers 5 & sleep 30'. You might want to take the middle 3 data points and average them, discarding the first and last that could be subject to LST overhead and ramp up/down. FWIW, this is what I do when I'm benchmarking throughput rates. Nic _______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss