Thanks Andreas for your response.
I ran anotherLnet Self test with 48 concurrent processes, since the nodes have
52 physical cores and I was able to achieve same throughput (2052.71 MiB/s =
2152 MB/s).
Is it expected to lose almost 600 MB/s (2750-2150= ) due to overheads on
ethernet
The first thing to note is that lst reports results in binary units
(MiB/s) while iperf reports results in decimal units (Gbps). If you do the
conversion you get 2055.31 MiB/s = 2155 MB/s.
The other thing to check is the CPU usage. For TCP the CPU usage can
be high. You should try RoCE+o2iblnd
