Il mer 2 ott 2019, 19:05 Doron Somech <somdo...@gmail.com> ha scritto:
> > You don't need to create multiple sockets, just call connect multiple > times with same address. > Wow, really?? I wish I had known that, I already changed quite a bit of code to use multiple zmq sockets to make better use of background zmq threads!! I will try connecting multiple times... At this point I suggest modifying the benchmark utility to just do this trick and update the performance graphs in the wiki with new results! Francesco On Wed, Oct 2, 2019, 19:45 Brett Viren via zeromq-dev < > zeromq-dev@lists.zeromq.org> wrote: > >> Hi Francesco, >> >> I confirm your benchmark using two systems with the same 100 Gbps >> Mellanox NICs but with an intervening Juniper QFX5200 switch (100 Gbps >> ports). >> >> To reach ~25 Gbps with the largest message sizes required "jumbo frame" >> MTU. The default mtu=1500 allows only ~20 Gbps. I also tried two more >> doubling of zmsg size in the benchmark and these produce no significant >> increase in throughput. OTOH, pinning the receiver (local_thr) to a CPU >> gets it up to 33 Gbps. >> >> I note that iperf3 can achieve almost 40 Gbps (20 Gbps w MTU=1500). >> Multiple simultaneous iperf3 tests can, in aggregate, use 90-100 Gbps. >> >> In both the ZMQ and singular iperf3 tests, it seems that CPU is the >> bottleneck. For ZeroMQ the receiver's I/O thread is pegged at 100%. >> With iperf3 it's that of the client/sender. The other ends in both >> cases are at about 50%. >> >> The zguide suggests to use one I/O thread per GByte/s (faq says "Gbps") >> so I tried the naive thing and hacked the ZMQ remote_thr.cpp and >> local_thr.cpp so each use ten I/O threads. While I see all ten threads >> in "top -H", still only one thread uses any CPU and it remains pegged at >> 100% on the receiver (local_thr) and about 50% on the sender >> (remote_thr). I think now that I misinterpreted this advice and it's >> really relevant to the case of handling a very large number of >> connections. >> >> >> Any suggestions on how to let ZeroMQ get higher throughput at 100 Gbps? >> If so, I'll give them a try. >> >> >> Cheers, >> -Brett. >> >> Francesco <francesco.monto...@gmail.com> writes: >> >> > Hi all, >> > >> > I placed here: >> > http://zeromq.org/results:100gbe-tests-v432 >> > the results I collected using 2 Mellanox ConnectX-5 linked by 100Gbps >> > fiber cable. >> > >> > The results are not too much different from those at 10gpbs >> > (http://zeromq.org/results:10gbe-tests-v432 )... the difference in TCP >> > throughput is that >> > - even using 100kB-long messages we still cannot saturate the link >> > - latency is very much improved for messages > 10kB long >> > >> > Hopefully we will be able to improve performances in the future to >> > improve these benchmarks... >> > >> > Francesco >> > _______________________________________________ >> > zeromq-dev mailing list >> > zeromq-dev@lists.zeromq.org >> > https://lists.zeromq.org/mailman/listinfo/zeromq-dev >> _______________________________________________ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> https://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >
_______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org https://lists.zeromq.org/mailman/listinfo/zeromq-dev
