Re: ixg(4) performances
On Tue, Aug 26, 2014 at 12:57:37PM +, Christos Zoulas wrote: > >I also found this page that tackles the same problem on Linux: > >http://dak1n1.com/blog/7-performance-tuning-intel-10gbe It seems that page describe a slightly different model. Intel 82599 datasheet is available here: http://www.intel.fr/content/www/fr/fr/ethernet-controllers/82599-10-gbe-controller-datasheet.html No reference to MMRBC in this document, but I understand "Max Read Request Size" is the same thing. Page 765 tells us about register A8, bits 12-14 that should be set to 100. pcictl /dev/pci5 read -d 0 -f 1 0x18 tells me the value 0x00092810 I tried this command: pcictl /dev/pci5 write -d 0 -f 1 0x18 0x00094810 Further pcictl read suggests it works as the new value is returned. However it gives no performance improvement. This means that I misunderstood what this register is about, or how to change it (byte order?). Or the performance are constrained by something unrelated. In the blog post cited above, the poster acheived more than 5 Gb/s before touching MMRBC, while I am stuck at 2,7 GB/s. Any new idea welcome. -- Emmanuel Dreyfus m...@netbsd.org
Re: ixg(4) performances
On Tue, Aug 26, 2014 at 04:40:25PM +, Taylor R Campbell wrote: > New version with some changes suggested by wiz@. Anyone has objection to this change being committed and pulled up to netbsd-7? -- Emmanuel Dreyfus m...@netbsd.org
Re: ixg(4) performances
What is your test setup? Do you have 2 identical boxes? Does it perform better e.g. on Linux or FreeBSD? If so, you could check how the config registers get set by that particular OS. 2014-08-28 9:26 GMT+02:00 Emmanuel Dreyfus : > On Tue, Aug 26, 2014 at 12:57:37PM +, Christos Zoulas wrote: >> >I also found this page that tackles the same problem on Linux: >> >http://dak1n1.com/blog/7-performance-tuning-intel-10gbe > > It seems that page describe a slightly different model. > Intel 82599 datasheet is available here: > http://www.intel.fr/content/www/fr/fr/ethernet-controllers/82599-10-gbe-controller-datasheet.html > > No reference to MMRBC in this document, but I understand "Max Read Request > Size" is the same thing. Page 765 tells us about register A8, bits 12-14 > that should be set to 100. > pcictl /dev/pci5 read -d 0 -f 1 0x18 tells me the value 0x00092810 > > I tried this command: > pcictl /dev/pci5 write -d 0 -f 1 0x18 0x00094810 > > Further pcictl read suggests it works as the new value is returned. > However it gives no performance improvement. This means that I > misunderstood what this register is about, or how to change it (byte order?). > > Or the performance are constrained by something unrelated. In the blog > post cited above, the poster acheived more than 5 Gb/s before touching > MMRBC, while I am stuck at 2,7 GB/s. Any new idea welcome. > > -- > Emmanuel Dreyfus > m...@netbsd.org
RE: ixg(4) performances
> Or the performance are constrained by something unrelated. In the blog > post cited above, the poster acheived more than 5 Gb/s before touching > MMRBC, while I am stuck at 2,7 GB/s. Any new idea welcome. The blog post refers to PCI-X, I'm more familiar with PCIe, but the concepts are similar. There are several possibilities, all revolving about differences between the blog poster's base system and yorus. 1) the test case is using a platform that has better PCI performance (in the PCIe world this could be: Gen3 versus Gen2 support in the slot being used; more lanes in the slot being used) 2) the test case has a root complex with a PCI controller with better performance than the one in your system; 3) the test case system has a different PCI configuration, in particular different bridging. For example, a PCI bridge or switch on your platform can change basic capabilities compared to the reference. 4) related to 3: one of the bridges on your system (between ixg and root complex) is not configured for 4K reads, and so the setting on the ixg board won't help [whereas this wasn't the case on the blog system]. 5) related to 4: one of the bridges in your system (between ixg and root complex) is not capable of 4K reads... (see 4). And of course you have to consider: 6) the writer has something else different than you have, for example silicon rev, BIOS, PCI-X where you have PCIe, etc. 7) the problem is completely unrelated to PCIe. You're in a tough situation, experimentally, because you can't take a working (5 Gbps) system and directly compare to the non-working (2.7 Gbps) situation. --Terry
Re: ixg(4) performances
In article <20140828072832.gi8...@homeworld.netbsd.org>, Emmanuel Dreyfus wrote: >On Tue, Aug 26, 2014 at 04:40:25PM +, Taylor R Campbell wrote: >> New version with some changes suggested by wiz@. > >Anyone has objection to this change being committed and pulled up to >netbsd-7? > Not me. christos
Re: ixg(4) performances
On Thu, 28 Aug 2014, Emmanuel Dreyfus wrote: On Tue, Aug 26, 2014 at 12:57:37PM +, Christos Zoulas wrote: I also found this page that tackles the same problem on Linux: http://dak1n1.com/blog/7-performance-tuning-intel-10gbe It seems that page describe a slightly different model. Intel 82599 datasheet is available here: http://www.intel.fr/content/www/fr/fr/ethernet-controllers/82599-10-gbe-controller-datasheet.html No reference to MMRBC in this document, but I understand "Max Read Request Size" is the same thing. Page 765 tells us about register A8, bits 12-14 that should be set to 100. pcictl /dev/pci5 read -d 0 -f 1 0x18 tells me the value 0x00092810 I tried this command: pcictl /dev/pci5 write -d 0 -f 1 0x18 0x00094810 Further pcictl read suggests it works as the new value is returned. However it gives no performance improvement. This means that I misunderstood what this register is about, or how to change it (byte order?). Or the performance are constrained by something unrelated. In the blog post cited above, the poster acheived more than 5 Gb/s before touching MMRBC, while I am stuck at 2,7 GB/s. Any new idea welcome. Isn't your PCIe slot constrained? I thought I remembered that you're only getting 2.5GT/s and I forget what test you're running. -- Hisashi T Fujinaka - ht...@twofifty.com BSEE + BSChem + BAEnglish + MSCS + $2.50 = coffee
Re: ixg(4) performances
On Thu, Aug 28, 2014 at 08:37:06AM -0700, Hisashi T Fujinaka wrote: > Isn't your PCIe slot constrained? I thought I remembered that you're > only getting 2.5GT/s and I forget what test you're running. I use netperf, and I now get 2.7 Gb/s. -- Emmanuel Dreyfus m...@netbsd.org
Re: RFC: IRQ affinity (aka interrupt routing)
Matt Thomas wrote: > > As I've been reading this discussion, it seems very x86 centric. > > I've thinking about adding > > void intr_distribute(void *ih, const kcpuset_t *newset, kcpuset_t *oldset) > Agree. And this would be MI interface. Just as Kengo-san already pointed out, MD may want to return ENOTSUP or whatever else. -- Mindaugas
Re: ixg(4) performances
Terry Moore wrote: > There are several possibilities, all revolving about differences between the > blog poster's base system and yorus. Do I have a way to investigate for appropriate PCI setup? Here is what dmesg says about it: pci0 at mainbus0 bus 0: configuration mode 1 pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok ppb4 at pci0 dev 14 function 0: vendor 0x10de product 0x005d (rev. 0xa3) ppb4: PCI Express 1.0 pci5 at ppb4 bus 5 pci5: i/o space, memory space enabled, rd/line, wr/inv ok ixg1 at pci5 dev 0 function 1: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.3.10 -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org
Unallocated inode
I'm currently running kernel: NetBSD slave 6.1_STABLE NetBSD 6.1_STABLE (SLAVE) #4: Fri May 23 23:42:30 EST 2014 stix@slave:/home/netbsd/netbsd-6/obj.amd64/home/netbsd/netbsd-6/src/sys/arch/amd64/compile/SLAVE amd64 Built from netbsd-6 branch synced around the build time. Over the last year, I have seen 2 instances where I've had cleared inodes, causing obvious errors: slave:ksh$ sudo find /home -xdev -ls > /dev/null find: /home/netbsd/cvsroot/pkgsrc/japanese/p5-Jcode/pkg/Attic/PLIST,v: Bad file descriptor find: /home/netbsd/cvsroot/pkgsrc/print/texlive-pdftools/patches/Attic/patch-ac,v: Bad file descriptor fsdb tells me they're "unallocated inode"s, which I can easily fix, but does anyone have any idea what might be causing them? This appears similar to issues reported previously: http://mail-index.netbsd.org/tech-kern/2013/10/19/msg015770.html My filesystem is FFSv2 with wapbl, sitting on a raidframe mirror over two SATA drives. Cheers, -- Paul Ripke "Great minds discuss ideas, average minds discuss events, small minds discuss people." -- Disputed: Often attributed to Eleanor Roosevelt. 1948.
