In Linux/Intel I would use following methodology for performance analysis: Hardware: * turbostat Look for C/P states for cores, frequencies, number of SMIs. [1] * cpufreq-info Look for current driver, frequencies, and governor. * atop Look for interrupt distribution across cores Look for context switches, interrupts. * ethtool -S for stats, look for errors, drops, overruns, missed interrupts, etc -k for offloads, enable GRO/GSO, rss/rps/rfs/xps[0] -g for ring sizes, increase -c for interrupt coalescing
Kernel: * /proc/net/softirq [2] and /proc/interrupts [3] Again, distribution, missed, delayed interrupts (optional) NUMA-affinity * perf top Look where kernel spends its time. * iptables Look if there are rules (if any) that may affect performance. * netstat -s, netstat -m Look for error counters and buffer counts * sysctl / grub So much to tweak here. Try increasing hashtable sizes, playing with memory buffers and other limits. BSD has alternatives to most of these, e.g. perf -> hwpmc, ethtool -> ifconfig, softirq -> netisr, menu.lst -> loader.conf In your case I suppose turning on interrupt coalescing along with enabling hardware offloads may help. [0] Comparing mutiqueue support Linux vs FreeBSD https://wiki.freebsd.org/201305DevSummit/NetworkReceivePerformance/ComparingMutiqueueSupportLinuxvsFreeBSD [1] You can pin your processor to a specific C-state: https://gist.github.com/SaveTheRbtz/f5e8d1ca7b55b6a7897b [2] You can analyze that data with: https://gist.github.com/SaveTheRbtz/172b2e2eb3cbd96b598d [3] You can set affinity with: https://gist.github.com/SaveTheRbtz/8875474 PS. Sorry for so many Linuxisms on freebsd-performance@ > On Feb 5, 2016, at 2:57 PM, Adrian Chadd <adrian.ch...@gmail.com> wrote: > > On 5 February 2016 at 10:15, Meyer, Wolfgang <wolfgang.me...@hob.de> wrote: >> >> >>> -----Original Message----- >>> From: owner-freebsd-performa...@freebsd.org [mailto:owner-freebsd- >>> performa...@freebsd.org] On Behalf Of Allan Jude >>> Sent: Mittwoch, 3. Februar 2016 22:50 >>> To: freebsd-performance@freebsd.org >>> Subject: Re: ixgbe: Network performance tuning (#TCP connections) >>> >>> On 2016-02-03 16:34, Adrian Chadd wrote: >>>> hi, >>>> >>>> can you share your testing program source? >>>> >>>> >>>> -a >>>> >>> >>> I have a Dual E5-2630 v3 (2x8x 2.40ghz (+HT)) with the Intel X540-AT2 that I >>> can try to replicate this one to help debug it. >>> >>> -- >>> Allan Jude >> >> I'll try to do some polishing and removal of cruft next week than I hope I >> will feel comfortable putting it to the public :-) >> >> Not that they are overly sophisticated programs, just some test set-up >> created in the past and over time one gets used to use it as a sort of >> private "benchmark". > > Please do - it'd be good to see what you're doing and figure out > what's causing the poor behaviour. > > Also having more public benchmarks for testing and reproducibility is > always good. :) > > > > -adrian > >> >> Regards, >> Wolfgang Meyer >> >> >> ________________________________ >> >> Follow HOB: >> >> - HOB: http://www.hob.de/redirect/hob.html >> - Xing: http://www.hob.de/redirect/xing.html >> - LinkedIn: http://www.hob.de/redirect/linkedin.html >> - HOBLink Mobile: http://www.hob.de/redirect/hoblinkmobile.html >> - Facebook: http://www.hob.de/redirect/facebook.html >> - Twitter: http://www.hob.de/redirect/twitter.html >> - YouTube: http://www.hob.de/redirect/youtube.html >> - E-Mail: http://www.hob.de/redirect/mail.html >> >> >> HOB GmbH & Co. KG >> Schwadermuehlstr. 3 >> D-90556 Cadolzburg >> >> Geschaeftsfuehrung: Klaus Brandstaetter, Zoran Adamovic >> >> AG Fuerth, HRA 5180 >> Steuer-Nr. 218/163/00107 >> USt-ID-Nr. DE 132747002 >> >> Komplementaerin HOB electronic Beteiligungs GmbH >> AG Fuerth, HRB 3416 >> _______________________________________________ >> freebsd-performance@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-performance >> To unsubscribe, send any mail to >> "freebsd-performance-unsubscr...@freebsd.org" > _______________________________________________ > freebsd-performance@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscr...@freebsd.org"
signature.asc
Description: Message signed with OpenPGP using GPGMail