Thanks Jack, The switch/lacp is where I was shifting my focus to. Unfortunately it's a remote site and it'll take some time before I can take it out of the picture so I figured I might ask you first ;)
I'll report back here when I have more information. Thanks for your feedback. Highly appreciated. Cheers, Rumen Telbizov On Thu, Oct 3, 2013 at 3:38 PM, Jack Vogel <jfvo...@gmail.com> wrote: > Set the storm threshold to 0, that way you disable the check and don't > have to wonder > if its a problem :) > > Given everything you've told me I'd suspect the switch... easy enough to > test, remove it > from the equation, run two systems back-to-back and see how that effects > the numbers. > > Jack > > > > On Thu, Oct 3, 2013 at 3:24 PM, Rumen Telbizov <telbi...@gmail.com> wrote: > >> Jack, >> >> Thanks for the hints. >> >> I checked that the card is inserted into PCIe v3.0 x16 + I get no >> complaints from the driver. >> I checked for interrupt throttling messages in the logs - found nothing. I >> also set hw.intr_storm_threshold: 10000 >> then I saw hw.igb.max_interrupt_rate: 8000 and figured that I should have >> hw.ix or hw.ixgbe which seems >> to have existed in the past but I don't have it.Also tried this >> dev.ix.0.enable_aim: 0 + dev.ix.1.enable_aim: 0 >> >> I think I have enough mbufs: >> kern.ipc.nmbclusters=262144 >> kern.ipc.nmbjumbop=262144 >> >> I also tried doubling those. >> >> So after trying all those it still seems like I can't get past the >> 4-5Gbit/s barrier so I decided to ask. >> Now I start wondering if it has anything to do with the lacp/switch or >> even >> with that patch from Oleg which helped me overcome the problem with the >> flapping interface. >> >> If anything else comes to mind and there should be another knob to turn - >> feel free to suggest it. >> >> Thanks, >> Rumen Telbizov >> >> >> On Thu, Oct 3, 2013 at 3:15 PM, Jack Vogel <jfvo...@gmail.com> wrote: >> >> > Make sure you are in slots with enough lanes/bandwidth, the driver >> should >> > complain when loaded >> > if you aren't though. Make sure you aren't getting interrupts throttled, >> > and that you have sufficient >> > mbuf resources so you aren't waiting on those, that can make a BIG >> > difference. >> > >> > Jack >> > >> > >> > >> > On Thu, Oct 3, 2013 at 2:37 PM, Rumen Telbizov <telbi...@gmail.com> >> wrote: >> > >> >> Thanks Jack, >> >> >> >> So I am running a couple of FreeBSD machines with those and they are in >> >> LACP pair plus vlans on top of them. >> >> When I run iperf between the two I get about about 1.7Gbit/s with 1 >> >> thread and I max out around 4.5-5Gbp/s with 6-8 multiple parallel >> threads >> >> (tried both UDP and TCP). >> >> >> >> What kind of tuning would you recommend that I try? >> >> >> >> Thanks, >> >> Rumen Telbizov >> >> >> >> >> >> On Thu, Oct 3, 2013 at 1:35 PM, Jack Vogel <jfvo...@gmail.com> wrote: >> >> >> >>> sysctl dev.ix.0 etc... >> >>> >> >>> Jack >> >>> >> >>> >> >>> >> >>> On Thu, Oct 3, 2013 at 12:35 PM, Rumen Telbizov <telbi...@gmail.com >> >wrote: >> >>> >> >>>> Hello everyone, >> >>>> >> >>>> I am trying to tweak some of the sysctl tunables for the ix (ixgbe) >> >>>> driver >> >>>> in FreeBSD 9.2 since I am experiencing less than ideal performance >> and >> >>>> it >> >>>> seems like I can't find any: >> >>>> >> >>>> # sysctl -a | grep -i ixgbe >> >>>> device ixgbe >> >>>> >> >>>> I am running 9.2-RC4. >> >>>> >> >>>> Any input appreciated. >> >>>> >> >>>> Thanks, >> >>>> -- >> >>>> Rumen Telbizov >> >>>> Unix Systems Administrator <http://telbizov.com> >> >>>> _______________________________________________ >> >>>> freebsd-stable@freebsd.org mailing list >> >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> >>>> To unsubscribe, send any mail to " >> >>>> freebsd-stable-unsubscr...@freebsd.org" >> >>>> >> >>> >> >>> >> >> >> >> >> >> -- >> >> Rumen Telbizov >> >> Unix Systems Administrator <http://telbizov.com> >> >> >> >> > >> > >> >> >> -- >> Rumen Telbizov >> Unix Systems Administrator <http://telbizov.com> >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" >> > > -- Rumen Telbizov Unix Systems Administrator <http://telbizov.com> _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"