On Tue, Aug 11, 2015 at 11:18 PM, Maxim Sobolev <sobo...@freebsd.org> wrote:
> Hi folks, > > Hi, > We've trying to migrate some of our high-PPS systems to a new hardware that > has four X540-AT2 10G NICs and observed that interrupt time goes through > roof after we cross around 200K PPS in and 200K out (two ports in LACP). > The previous hardware was stable up to about 350K PPS in and 350K out. I > believe the old one was equipped with the I350 and had the identical LACP > configuration. The new box also has better CPU with more cores (i.e. 24 > cores vs. 16 cores before). CPU itself is 2 x E5-2690 v3. > 200K PPS, and even 350K PPS are very low value indeed. On a Intel Xeon L5630 (4 cores only) with one X540-AT2 (then 2 10Gigabit ports) I've reached about 1.8Mpps (fastforwarding enabled) [1]. But my setup didn't use lagg(4): Can you disable lagg configuration and re-measure your performance without lagg ? Do you let Intel NIC drivers using 8 queues for port too? In my use case (forwarding smallest UDP packet size), I obtain better behaviour by limiting NIC queues to 4 (hw.ix.num_queues or hw.ixgbe.num_queues, don't remember) if my system had 8 cores. And this with Gigabit Intel[2] or Chelsio NIC [3]. Don't forget to disable TSO and LRO too. Regards, Olivier [1] http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_an_ibm_system_x3550_m3_with_10-gigabit_intel_x540-at2#graphs [2] http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_superserver_5018a-ftn4#graph1 [3] http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_hp_proliant_dl360p_gen8_with_10-gigabit_with_10-gigabit_chelsio_t540-cr#reducing_nic_queues _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
