On Fri, 1 Dec 2017 21:32:35 +0100
Götz Waschk <goetz.was...@gmail.com> wrote:
...
> # Benchmarking Alltoall
> # #processes = 1024
> #----------------------------------------------------------------
> #bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
> 0 1000 0.04 0.09 0.05
> 1 1000 253.40 335.35 293.06
> 2 1000 266.93 346.65 306.23
> 4 1000 303.52 382.41 342.21
> 8 1000 383.89 493.56 439.34
> 16 1000 501.27 627.84 569.80
> 32 1000 1039.65 1259.70 1163.12
> 64 1000 1710.12 2071.47 1910.62
> 128 1000 3051.68 3653.44 3398.65
As a potentially interesting data point, I dug through my archive of
imb output and found an example that also showed something strange
happening at the 128 to 256 byte transition on alltoall @1024 ranks
(although in my case it didn't completely hang):
# Benchmarking Alltoall
# #processes = 1024
#----------------------------------------------------------------
#bytes #repetitions t_min[usec] t_max[usec] t_avg[usec]
1 1000 417.44 417.59 417.54
2 1000 410.50 410.72 410.67
4 1000 365.92 366.21 365.99
8 1000 583.21 583.51 583.37
16 1000 652.90 653.09 652.98
32 1000 982.09 982.42 982.28
64 1000 2090.70 2091.11 2090.90
128 1000 2590.91 2591.93 2591.44
256 93 70077.42 70219.70 70174.85
512 93 88611.39 88711.53 88672.84
My output was run on OpenMPI-1.7.6 on CentOS-6 on Mellanox FDR ib
(using the normal verbs/openib transport).
/Peter K
--
Sent from my Android device with K-9 Mail. Please excuse my brevity._______________________________________________
users mailing list
users@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users
