Re: Upgrade 9.1 - 9.2 Invalid format / BTX halted
I found, that freebsd-update install doesn't upgrade all files in /boot. At least the loader binary was zero: ls -lh boot/loader* -r-xr-xr-x 1 tom wheel 0B Oct 2 22:54 loader -r--r--r-- 1 tom wheel 0B Oct 2 22:54 loader.4th -rw-r--r-- 1 tom wheel15B Jun 5 2012 loader.conf -r--r--r-- 1 tom wheel14k Jan 3 2012 loader.help -r--r--r-- 1 tom wheel 0B Oct 2 22:54 loader.rc the / partition was not full! I copied the /boot directory from an other 9.1 machine and could boot the machine. But the server is still running FreeBSD 9.1. Any ideas what is wrong? Am 02.10.2013 23:21, schrieb Thomas Krause: Hi, I tried to upgrade one of my servers from FreeBSD 9.1 to 9.2 with freebsd-update upgrade -r 9.2-RELEASE after the first reboot the system hangs on boot with Invalid format ... BTX halted It's a Supermicro X9SCL-F mainboard. I upgraded FreeBSD from 9.0 to 9.1 REL sucessfull, when 9.1 was released. What happend with the upgrade to 9.2? Here is the screenshot http://212.78.99.130/btx.jpg Thanks for any help. Regards, Thomas. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Build failure for 9.2-RELEASE
I failed with make buildworld on 9.2, building from a 9.2 prerelease. svn revision on system to be built was 255986. uname -a shows FreeBSD amelia2 9.2-PRERELEASE FreeBSD 9.2-PRERELEASE #17 r254196: Sun Aug 11 00:36:49 UTC 2013 root@amelia2:/usr/obj/usr/src/sys/SANDY amd64 I had MODULES_WITH_WORLD=yes in /etc/make.conf to avoid the redundancy and annoyance of rebuilding modules every time I build a new kernel, like if I want a more portable kernel (for a USB-stick install) as well as my custom kernel. Point of failure seemed to be in module dtrace. I have in kernel config #optionsKDTRACE_FRAME # Ensure frames are compiled in #optionsKDTRACE_HOOKS # Kernel DTrace hooks Maybe I need to reenable these? Or is there a great hazard in building modules with world, which seems so logical? Last 32 lines of buildworld.log are === sys/modules/dtrace/sdt (depend) @ - /usr/src/sys machine - /usr/src/sys/amd64/include x86 - /usr/src/sys/x86/include awk -f @/tools/vnode_if.awk @/kern/vnode_if.src -p awk -f @/tools/vnode_if.awk @/kern/vnode_if.src -q awk -f @/tools/vnode_if.awk @/kern/vnode_if.src -h rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -I/usr/src/sys/modules/dtrace/sdt/../../../cddl/compat/opensolaris -I/usr/src/sys/modules/dtrace/sdt/../../../cddl/contrib/opensolaris/uts/common -I/usr/src/sys/modules/dtrace/sdt/../../.. -I. -I@ -I@/contrib/altq -std=iso9899:1999 /usr/src/sys/modules/dtrace/sdt/../../../cddl/dev/sdt/sdt.c /usr/src/sys/modules/dtrace/sdt/../../../cddl/dev/sdt/sdt.c:27:25: error: opt_kdtrace.h: No such file or directory mkdep: compile failed *** [.depend] Error code 1 Stop in /usr/src/sys/modules/dtrace/sdt. *** [depend] Error code 1 Stop in /usr/src/sys/modules/dtrace. *** [depend] Error code 1 Stop in /usr/src/sys/modules. *** [depend] Error code 1 Stop in /usr/src/sys. *** [sys.depend__D] Error code 1 Stop in /usr/src. *** [_depend] Error code 1 Stop in /usr/src. *** [buildworld] Error code 1 Stop in /usr/src. Tom ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
mps in GENERIC in FreeBSD 9.2R i386
Did nobody ever verify this for Ken? On Mon, Oct 01, 2012 at 23:38:33 +0530, Desai, Kashyap wrote: -Original Message- From: owner-freebsd-stable at freebsd.org [mailto:owner-freebsd- stable at freebsd.org] On Behalf Of Kenneth D. Merry Sent: Monday, October 01, 2012 8:58 PM To: John Baldwin Cc: Harald Schmalzbauer; freebsd-stable at freebsd.org Subject: Re: mps in GENERIC, only in amd64? (RELENG_9_1) On Mon, Oct 01, 2012 at 08:49:36 -0400, John Baldwin wrote: On Saturday, September 29, 2012 5:58:42 am Harald Schmalzbauer wrote: Hello, accidentally I saw that mps is included in sys/amd64/conf/GENERIC, but not in sys/i386/conf/GENERIC. Is this intended? Have you tested it on i386? From the log message, Ken (cc'd) only added it on amd64 as it hadn't been tested on i386. That was certainly the case two years ago. Since then, though, I think the LSI folks have tested it on i386. If we get reports of success using it on i386, I don't see any issue with putting it in GENERIC. YES LSI has tested i386 arch on different Released FreeBSDs of 7.x, 8.x and 9.x series. That confirms it. I'll go ahead and check it into head if someone with an i386 build environment can confirm that the driver in head builds properly on i386. Thanks, Ken It seems to compile cleanly on i386. I don't have an easy way to test it though (only compiling in a VM). ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Device timeout from mfi(9) while booting 9.2-RELEASE
We're having trouble booting 9.2-RELEASE on a system with an mfi (Drake Skinny) controller. When it tries to mount root we get: mfi0: COMMAND 0xff8000cb7220 TIMEOUT AFTER 59 SECONDS mfi0: COMMAND 0xff8000cb72a8 TIMEOUT AFTER 60 SECONDS mfi0: COMMAND 0xff8000cb7220 TIMEOUT AFTER 90 SECONDS I also get the following when mfi attaches (with bootverbose): mfi0: Drake Skinny port 0x5000-0x50ff mem 0xfbd6-0xfbd63fff,0xfbd0-0xf bd3 irq 16 at device 0.0 on pci2^M mfi0: attempting to allocate 1 MSI vectors (1 supported)^M msi: routing MSI IRQ 282 to local APIC 0 vector 77^M mfi0: using IRQ 282 for MSI^M mfi0: Using MSI^M mfi0: Megaraid SAS driver Ver 4.23 ^M mfi0: Frame 0xff85d394b280 timed out command 0x101^M error 60 in callback from mfi_send_frame^M mfi0: Frame 0xff85d394b500 timed out command 0x1040100^M error 60 in callback from mfi_send_frame^M mfi0: Frame 0xff85d394b780 timed out command 0x1040300^M error 60 in callback from mfi_send_frame^M mfi0: Error 255 fetching controller entries^M I've put full bootverbose logs from boots with hw.mfi.msi=0 and hw.mfi.msi=1 here: http://people.freebsd.org/~rstone/mfi_timeout_irq.txt http://people.freebsd.org/~rstone/mfi_timeout_msi.txt Neither boot was successful. I don't have the ability to boot from alternative media so I have a limited ability to test changes (I can re-image from a full disk image, but that requires building a new disk image which takes some time). I did see the following forum post and tried with cbb and pccard removed from GENERIC. That didn't fix it. http://forums.freebsd.org/showpost.php?p=231135postcount=4 ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Upgrade 9.1 - 9.2 Invalid format / BTX halted
03.10.2013 11:43, Thomas Krause пишет: I found, that freebsd-update install doesn't upgrade all files in /boot. At least the loader binary was zero: ls -lh boot/loader* -r-xr-xr-x 1 tom wheel 0B Oct 2 22:54 loader -r--r--r-- 1 tom wheel 0B Oct 2 22:54 loader.4th -rw-r--r-- 1 tom wheel15B Jun 5 2012 loader.conf -r--r--r-- 1 tom wheel14k Jan 3 2012 loader.help -r--r--r-- 1 tom wheel 0B Oct 2 22:54 loader.rc Tom is root... the / partition was not full! I copied the /boot directory from an other 9.1 machine and could boot the machine. But the server is still running FreeBSD 9.1. Any ideas what is wrong? I've just upgraded a box from 9.2-RC2-p1 to 9.2-RELEASE: - % ls -l /boot/loader* -r-xr-xr-x 1 root wheel 229376 Oct 3 20:54 /boot/loader -r--r--r-- 1 root wheel5884 Oct 3 20:54 /boot/loader.4th -rw-r--r-- 1 root wheel 65 Jan 4 2013 /boot/loader.conf -r--r--r-- 1 root wheel 14766 Aug 14 2012 /boot/loader.help -r--r--r-- 1 root wheel 393 Oct 3 20:54 /boot/loader.rc - Does /var/log/messages show something interesting? -- WBR, Boris Samorodov (bsam) FreeBSD Committer, http://www.FreeBSD.org The Power To Serve ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
ixgbe/ix sysctl missing in FreeBSD 9.2
Hello everyone, I am trying to tweak some of the sysctl tunables for the ix (ixgbe) driver in FreeBSD 9.2 since I am experiencing less than ideal performance and it seems like I can't find any: # sysctl -a | grep -i ixgbe deviceixgbe I am running 9.2-RC4. Any input appreciated. Thanks, -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Device timeout from mfi(9) while booting 9.2-RELEASE
On Thu, Oct 03, 2013 at 11:50:15AM -0400, Ryan Stone wrote: We're having trouble booting 9.2-RELEASE on a system with an mfi (Drake Skinny) controller. When it tries to mount root we get: mfi0: COMMAND 0xff8000cb7220 TIMEOUT AFTER 59 SECONDS mfi0: COMMAND 0xff8000cb72a8 TIMEOUT AFTER 60 SECONDS mfi0: COMMAND 0xff8000cb7220 TIMEOUT AFTER 90 SECONDS I also get the following when mfi attaches (with bootverbose): mfi0: Drake Skinny port 0x5000-0x50ff mem 0xfbd6-0xfbd63fff,0xfbd0-0xf bd3 irq 16 at device 0.0 on pci2^M mfi0: attempting to allocate 1 MSI vectors (1 supported)^M msi: routing MSI IRQ 282 to local APIC 0 vector 77^M mfi0: using IRQ 282 for MSI^M mfi0: Using MSI^M mfi0: Megaraid SAS driver Ver 4.23 ^M mfi0: Frame 0xff85d394b280 timed out command 0x101^M error 60 in callback from mfi_send_frame^M mfi0: Frame 0xff85d394b500 timed out command 0x1040100^M error 60 in callback from mfi_send_frame^M mfi0: Frame 0xff85d394b780 timed out command 0x1040300^M error 60 in callback from mfi_send_frame^M mfi0: Error 255 fetching controller entries^M I've put full bootverbose logs from boots with hw.mfi.msi=0 and hw.mfi.msi=1 here: http://people.freebsd.org/~rstone/mfi_timeout_irq.txt http://people.freebsd.org/~rstone/mfi_timeout_msi.txt Neither boot was successful. I don't have the ability to boot from alternative media so I have a limited ability to test changes (I can re-image from a full disk image, but that requires building a new disk image which takes some time). I did see the following forum post and tried with cbb and pccard removed from GENERIC. That didn't fix it. http://forums.freebsd.org/showpost.php?p=231135postcount=4 I have two Drake Skinny cards, one four-ports, one with eight ports, both not working with either version of driver in HEAD from February up to now. I tried to talk about this both with Doug Ambrisko and Desai, Kashyap kashyap.de...@lsi.com, and provided the debugging data I could think of, but no avail. There is a rumor that it might be related to the slot the card is inserted in. I only have 8x or wider PCIe slot provided by CPU north-bridge. It might be that card does not like it and only work in the south-bridge PCIe lanes. I have no idea why could it be so, neither I saw the confirmation of the speculation. pgpuCq2UR3ALv.pgp Description: PGP signature
Re: ixgbe/ix sysctl missing in FreeBSD 9.2
sysctl dev.ix.0 etc... Jack On Thu, Oct 3, 2013 at 12:35 PM, Rumen Telbizov telbi...@gmail.com wrote: Hello everyone, I am trying to tweak some of the sysctl tunables for the ix (ixgbe) driver in FreeBSD 9.2 since I am experiencing less than ideal performance and it seems like I can't find any: # sysctl -a | grep -i ixgbe deviceixgbe I am running 9.2-RC4. Any input appreciated. Thanks, -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ixgbe/ix sysctl missing in FreeBSD 9.2
Thanks Jack, So I am running a couple of FreeBSD machines with those and they are in LACP pair plus vlans on top of them. When I run iperf between the two I get about about 1.7Gbit/s with 1 thread and I max out around 4.5-5Gbp/s with 6-8 multiple parallel threads (tried both UDP and TCP). What kind of tuning would you recommend that I try? Thanks, Rumen Telbizov On Thu, Oct 3, 2013 at 1:35 PM, Jack Vogel jfvo...@gmail.com wrote: sysctl dev.ix.0 etc... Jack On Thu, Oct 3, 2013 at 12:35 PM, Rumen Telbizov telbi...@gmail.comwrote: Hello everyone, I am trying to tweak some of the sysctl tunables for the ix (ixgbe) driver in FreeBSD 9.2 since I am experiencing less than ideal performance and it seems like I can't find any: # sysctl -a | grep -i ixgbe deviceixgbe I am running 9.2-RC4. Any input appreciated. Thanks, -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ixgbe/ix sysctl missing in FreeBSD 9.2
Make sure you are in slots with enough lanes/bandwidth, the driver should complain when loaded if you aren't though. Make sure you aren't getting interrupts throttled, and that you have sufficient mbuf resources so you aren't waiting on those, that can make a BIG difference. Jack On Thu, Oct 3, 2013 at 2:37 PM, Rumen Telbizov telbi...@gmail.com wrote: Thanks Jack, So I am running a couple of FreeBSD machines with those and they are in LACP pair plus vlans on top of them. When I run iperf between the two I get about about 1.7Gbit/s with 1 thread and I max out around 4.5-5Gbp/s with 6-8 multiple parallel threads (tried both UDP and TCP). What kind of tuning would you recommend that I try? Thanks, Rumen Telbizov On Thu, Oct 3, 2013 at 1:35 PM, Jack Vogel jfvo...@gmail.com wrote: sysctl dev.ix.0 etc... Jack On Thu, Oct 3, 2013 at 12:35 PM, Rumen Telbizov telbi...@gmail.comwrote: Hello everyone, I am trying to tweak some of the sysctl tunables for the ix (ixgbe) driver in FreeBSD 9.2 since I am experiencing less than ideal performance and it seems like I can't find any: # sysctl -a | grep -i ixgbe deviceixgbe I am running 9.2-RC4. Any input appreciated. Thanks, -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ixgbe/ix sysctl missing in FreeBSD 9.2
Jack, Thanks for the hints. I checked that the card is inserted into PCIe v3.0 x16 + I get no complaints from the driver. I checked for interrupt throttling messages in the logs - found nothing. I also set hw.intr_storm_threshold: 1 then I saw hw.igb.max_interrupt_rate: 8000 and figured that I should have hw.ix or hw.ixgbe which seems to have existed in the past but I don't have it.Also tried this dev.ix.0.enable_aim: 0 + dev.ix.1.enable_aim: 0 I think I have enough mbufs: kern.ipc.nmbclusters=262144 kern.ipc.nmbjumbop=262144 I also tried doubling those. So after trying all those it still seems like I can't get past the 4-5Gbit/s barrier so I decided to ask. Now I start wondering if it has anything to do with the lacp/switch or even with that patch from Oleg which helped me overcome the problem with the flapping interface. If anything else comes to mind and there should be another knob to turn - feel free to suggest it. Thanks, Rumen Telbizov On Thu, Oct 3, 2013 at 3:15 PM, Jack Vogel jfvo...@gmail.com wrote: Make sure you are in slots with enough lanes/bandwidth, the driver should complain when loaded if you aren't though. Make sure you aren't getting interrupts throttled, and that you have sufficient mbuf resources so you aren't waiting on those, that can make a BIG difference. Jack On Thu, Oct 3, 2013 at 2:37 PM, Rumen Telbizov telbi...@gmail.com wrote: Thanks Jack, So I am running a couple of FreeBSD machines with those and they are in LACP pair plus vlans on top of them. When I run iperf between the two I get about about 1.7Gbit/s with 1 thread and I max out around 4.5-5Gbp/s with 6-8 multiple parallel threads (tried both UDP and TCP). What kind of tuning would you recommend that I try? Thanks, Rumen Telbizov On Thu, Oct 3, 2013 at 1:35 PM, Jack Vogel jfvo...@gmail.com wrote: sysctl dev.ix.0 etc... Jack On Thu, Oct 3, 2013 at 12:35 PM, Rumen Telbizov telbi...@gmail.comwrote: Hello everyone, I am trying to tweak some of the sysctl tunables for the ix (ixgbe) driver in FreeBSD 9.2 since I am experiencing less than ideal performance and it seems like I can't find any: # sysctl -a | grep -i ixgbe deviceixgbe I am running 9.2-RC4. Any input appreciated. Thanks, -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Rumen Telbizov Unix Systems Administrator http://telbizov.com -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ixgbe/ix sysctl missing in FreeBSD 9.2
Set the storm threshold to 0, that way you disable the check and don't have to wonder if its a problem :) Given everything you've told me I'd suspect the switch... easy enough to test, remove it from the equation, run two systems back-to-back and see how that effects the numbers. Jack On Thu, Oct 3, 2013 at 3:24 PM, Rumen Telbizov telbi...@gmail.com wrote: Jack, Thanks for the hints. I checked that the card is inserted into PCIe v3.0 x16 + I get no complaints from the driver. I checked for interrupt throttling messages in the logs - found nothing. I also set hw.intr_storm_threshold: 1 then I saw hw.igb.max_interrupt_rate: 8000 and figured that I should have hw.ix or hw.ixgbe which seems to have existed in the past but I don't have it.Also tried this dev.ix.0.enable_aim: 0 + dev.ix.1.enable_aim: 0 I think I have enough mbufs: kern.ipc.nmbclusters=262144 kern.ipc.nmbjumbop=262144 I also tried doubling those. So after trying all those it still seems like I can't get past the 4-5Gbit/s barrier so I decided to ask. Now I start wondering if it has anything to do with the lacp/switch or even with that patch from Oleg which helped me overcome the problem with the flapping interface. If anything else comes to mind and there should be another knob to turn - feel free to suggest it. Thanks, Rumen Telbizov On Thu, Oct 3, 2013 at 3:15 PM, Jack Vogel jfvo...@gmail.com wrote: Make sure you are in slots with enough lanes/bandwidth, the driver should complain when loaded if you aren't though. Make sure you aren't getting interrupts throttled, and that you have sufficient mbuf resources so you aren't waiting on those, that can make a BIG difference. Jack On Thu, Oct 3, 2013 at 2:37 PM, Rumen Telbizov telbi...@gmail.com wrote: Thanks Jack, So I am running a couple of FreeBSD machines with those and they are in LACP pair plus vlans on top of them. When I run iperf between the two I get about about 1.7Gbit/s with 1 thread and I max out around 4.5-5Gbp/s with 6-8 multiple parallel threads (tried both UDP and TCP). What kind of tuning would you recommend that I try? Thanks, Rumen Telbizov On Thu, Oct 3, 2013 at 1:35 PM, Jack Vogel jfvo...@gmail.com wrote: sysctl dev.ix.0 etc... Jack On Thu, Oct 3, 2013 at 12:35 PM, Rumen Telbizov telbi...@gmail.com wrote: Hello everyone, I am trying to tweak some of the sysctl tunables for the ix (ixgbe) driver in FreeBSD 9.2 since I am experiencing less than ideal performance and it seems like I can't find any: # sysctl -a | grep -i ixgbe deviceixgbe I am running 9.2-RC4. Any input appreciated. Thanks, -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Rumen Telbizov Unix Systems Administrator http://telbizov.com -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ixgbe/ix sysctl missing in FreeBSD 9.2
Thanks Jack, The switch/lacp is where I was shifting my focus to. Unfortunately it's a remote site and it'll take some time before I can take it out of the picture so I figured I might ask you first ;) I'll report back here when I have more information. Thanks for your feedback. Highly appreciated. Cheers, Rumen Telbizov On Thu, Oct 3, 2013 at 3:38 PM, Jack Vogel jfvo...@gmail.com wrote: Set the storm threshold to 0, that way you disable the check and don't have to wonder if its a problem :) Given everything you've told me I'd suspect the switch... easy enough to test, remove it from the equation, run two systems back-to-back and see how that effects the numbers. Jack On Thu, Oct 3, 2013 at 3:24 PM, Rumen Telbizov telbi...@gmail.com wrote: Jack, Thanks for the hints. I checked that the card is inserted into PCIe v3.0 x16 + I get no complaints from the driver. I checked for interrupt throttling messages in the logs - found nothing. I also set hw.intr_storm_threshold: 1 then I saw hw.igb.max_interrupt_rate: 8000 and figured that I should have hw.ix or hw.ixgbe which seems to have existed in the past but I don't have it.Also tried this dev.ix.0.enable_aim: 0 + dev.ix.1.enable_aim: 0 I think I have enough mbufs: kern.ipc.nmbclusters=262144 kern.ipc.nmbjumbop=262144 I also tried doubling those. So after trying all those it still seems like I can't get past the 4-5Gbit/s barrier so I decided to ask. Now I start wondering if it has anything to do with the lacp/switch or even with that patch from Oleg which helped me overcome the problem with the flapping interface. If anything else comes to mind and there should be another knob to turn - feel free to suggest it. Thanks, Rumen Telbizov On Thu, Oct 3, 2013 at 3:15 PM, Jack Vogel jfvo...@gmail.com wrote: Make sure you are in slots with enough lanes/bandwidth, the driver should complain when loaded if you aren't though. Make sure you aren't getting interrupts throttled, and that you have sufficient mbuf resources so you aren't waiting on those, that can make a BIG difference. Jack On Thu, Oct 3, 2013 at 2:37 PM, Rumen Telbizov telbi...@gmail.com wrote: Thanks Jack, So I am running a couple of FreeBSD machines with those and they are in LACP pair plus vlans on top of them. When I run iperf between the two I get about about 1.7Gbit/s with 1 thread and I max out around 4.5-5Gbp/s with 6-8 multiple parallel threads (tried both UDP and TCP). What kind of tuning would you recommend that I try? Thanks, Rumen Telbizov On Thu, Oct 3, 2013 at 1:35 PM, Jack Vogel jfvo...@gmail.com wrote: sysctl dev.ix.0 etc... Jack On Thu, Oct 3, 2013 at 12:35 PM, Rumen Telbizov telbi...@gmail.com wrote: Hello everyone, I am trying to tweak some of the sysctl tunables for the ix (ixgbe) driver in FreeBSD 9.2 since I am experiencing less than ideal performance and it seems like I can't find any: # sysctl -a | grep -i ixgbe deviceixgbe I am running 9.2-RC4. Any input appreciated. Thanks, -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Rumen Telbizov Unix Systems Administrator http://telbizov.com -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Build failure for 9.2-RELEASE
Thomas Mueller wrote: I failed with make buildworld on 9.2, building from a 9.2 prerelease. svn revision on system to be built was 255986. uname -a shows FreeBSD amelia2 9.2-PRERELEASE FreeBSD 9.2-PRERELEASE #17 r254196: Sun Aug 11 00:36:49 UTC 2013 root@amelia2:/usr/obj/usr/src/sys/SANDY amd64 I had MODULES_WITH_WORLD=yes in /etc/make.conf to avoid the redundancy and annoyance of rebuilding modules every time I build a new kernel, like if I want a more portable kernel (for a USB-stick install) as well as my custom kernel. Point of failure seemed to be in module dtrace. I have in kernel config #optionsKDTRACE_FRAME # Ensure frames are compiled in #optionsKDTRACE_HOOKS # Kernel DTrace hooks Maybe I need to reenable these? Or is there a great hazard in building modules with world, which seems so logical? Last 32 lines of buildworld.log are === sys/modules/dtrace/sdt (depend) @ - /usr/src/sys machine - /usr/src/sys/amd64/include x86 - /usr/src/sys/x86/include awk -f @/tools/vnode_if.awk @/kern/vnode_if.src -p awk -f @/tools/vnode_if.awk @/kern/vnode_if.src -q awk -f @/tools/vnode_if.awk @/kern/vnode_if.src -h rm -f .depend mkdep -f .depend -a -nostdinc -D_KERNEL -DKLD_MODULE -I/usr/src/sys/modules/dtrace/sdt/../../../cddl/compat/opensolaris -I/usr/src/sys/modules/dtrace/sdt/../../../cddl/contrib/opensolaris/uts/common -I/usr/src/sys/modules/dtrace/sdt/../../.. -I. -I@ -I@/contrib/altq -std=iso9899:1999 /usr/src/sys/modules/dtrace/sdt/../../../cddl/dev/sdt/sdt.c /usr/src/sys/modules/dtrace/sdt/../../../cddl/dev/sdt/sdt.c:27:25: error: opt_kdtrace.h: No such file or directory mkdep: compile failed *** [.depend] Error code 1 Stop in /usr/src/sys/modules/dtrace/sdt. *** [depend] Error code 1 Stop in /usr/src/sys/modules/dtrace. *** [depend] Error code 1 Stop in /usr/src/sys/modules. *** [depend] Error code 1 Stop in /usr/src/sys. *** [sys.depend__D] Error code 1 Stop in /usr/src. *** [_depend] Error code 1 Stop in /usr/src. *** [buildworld] Error code 1 Stop in /usr/src. Tom I don't know if anyone else already commented on this, but you could try adding a line like: SRCS+= opt_kdtrace.h to the Makefile in modules/dtrace/sdt rick ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Device timeout from mfi(9) while booting 9.2-RELEASE
Hi, We have Drake Skinny (9240-4i and 9240-8i) cards running, most recently with 9.2-RC4. We'll be testing 9.2-RELEASE in the next week or two. We have seen problems with older versions of the firmware, and interactions with some Supermicro BIOS versions. All these problems have been resolved by upgrading to the latest versions of the controller firmware and system BIOSes. We're running with with hw.mfi.msi=1. I haven't tested without that since early 2011; I should probably try it out. So: What firmware do you have in the controller? Did you update it? Regards, Jan. On 04/10/2013, at 5:47 AM, Konstantin Belousov kostik...@gmail.com wrote: On Thu, Oct 03, 2013 at 11:50:15AM -0400, Ryan Stone wrote: We're having trouble booting 9.2-RELEASE on a system with an mfi (Drake Skinny) controller. When it tries to mount root we get: mfi0: COMMAND 0xff8000cb7220 TIMEOUT AFTER 59 SECONDS mfi0: COMMAND 0xff8000cb72a8 TIMEOUT AFTER 60 SECONDS mfi0: COMMAND 0xff8000cb7220 TIMEOUT AFTER 90 SECONDS I also get the following when mfi attaches (with bootverbose): mfi0: Drake Skinny port 0x5000-0x50ff mem 0xfbd6-0xfbd63fff,0xfbd0-0xf bd3 irq 16 at device 0.0 on pci2^M mfi0: attempting to allocate 1 MSI vectors (1 supported)^M msi: routing MSI IRQ 282 to local APIC 0 vector 77^M mfi0: using IRQ 282 for MSI^M mfi0: Using MSI^M mfi0: Megaraid SAS driver Ver 4.23 ^M mfi0: Frame 0xff85d394b280 timed out command 0x101^M error 60 in callback from mfi_send_frame^M mfi0: Frame 0xff85d394b500 timed out command 0x1040100^M error 60 in callback from mfi_send_frame^M mfi0: Frame 0xff85d394b780 timed out command 0x1040300^M error 60 in callback from mfi_send_frame^M mfi0: Error 255 fetching controller entries^M I've put full bootverbose logs from boots with hw.mfi.msi=0 and hw.mfi.msi=1 here: http://people.freebsd.org/~rstone/mfi_timeout_irq.txt http://people.freebsd.org/~rstone/mfi_timeout_msi.txt Neither boot was successful. I don't have the ability to boot from alternative media so I have a limited ability to test changes (I can re-image from a full disk image, but that requires building a new disk image which takes some time). I did see the following forum post and tried with cbb and pccard removed from GENERIC. That didn't fix it. http://forums.freebsd.org/showpost.php?p=231135postcount=4 I have two Drake Skinny cards, one four-ports, one with eight ports, both not working with either version of driver in HEAD from February up to now. I tried to talk about this both with Doug Ambrisko and Desai, Kashyap kashyap.de...@lsi.com, and provided the debugging data I could think of, but no avail. There is a rumor that it might be related to the slot the card is inserted in. I only have 8x or wider PCIe slot provided by CPU north-bridge. It might be that card does not like it and only work in the south-bridge PCIe lanes. I have no idea why could it be so, neither I saw the confirmation of the speculation. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: ixgbe/ix sysctl missing in FreeBSD 9.2
There's an LACP knob that ignores doing the LACP hash if the mbuf has a flowid. Find that knob and set it to 0. Then, retry. -a On 3 October 2013 15:24, Rumen Telbizov telbi...@gmail.com wrote: Jack, Thanks for the hints. I checked that the card is inserted into PCIe v3.0 x16 + I get no complaints from the driver. I checked for interrupt throttling messages in the logs - found nothing. I also set hw.intr_storm_threshold: 1 then I saw hw.igb.max_interrupt_rate: 8000 and figured that I should have hw.ix or hw.ixgbe which seems to have existed in the past but I don't have it.Also tried this dev.ix.0.enable_aim: 0 + dev.ix.1.enable_aim: 0 I think I have enough mbufs: kern.ipc.nmbclusters=262144 kern.ipc.nmbjumbop=262144 I also tried doubling those. So after trying all those it still seems like I can't get past the 4-5Gbit/s barrier so I decided to ask. Now I start wondering if it has anything to do with the lacp/switch or even with that patch from Oleg which helped me overcome the problem with the flapping interface. If anything else comes to mind and there should be another knob to turn - feel free to suggest it. Thanks, Rumen Telbizov On Thu, Oct 3, 2013 at 3:15 PM, Jack Vogel jfvo...@gmail.com wrote: Make sure you are in slots with enough lanes/bandwidth, the driver should complain when loaded if you aren't though. Make sure you aren't getting interrupts throttled, and that you have sufficient mbuf resources so you aren't waiting on those, that can make a BIG difference. Jack On Thu, Oct 3, 2013 at 2:37 PM, Rumen Telbizov telbi...@gmail.com wrote: Thanks Jack, So I am running a couple of FreeBSD machines with those and they are in LACP pair plus vlans on top of them. When I run iperf between the two I get about about 1.7Gbit/s with 1 thread and I max out around 4.5-5Gbp/s with 6-8 multiple parallel threads (tried both UDP and TCP). What kind of tuning would you recommend that I try? Thanks, Rumen Telbizov On Thu, Oct 3, 2013 at 1:35 PM, Jack Vogel jfvo...@gmail.com wrote: sysctl dev.ix.0 etc... Jack On Thu, Oct 3, 2013 at 12:35 PM, Rumen Telbizov telbi...@gmail.com wrote: Hello everyone, I am trying to tweak some of the sysctl tunables for the ix (ixgbe) driver in FreeBSD 9.2 since I am experiencing less than ideal performance and it seems like I can't find any: # sysctl -a | grep -i ixgbe deviceixgbe I am running 9.2-RC4. Any input appreciated. Thanks, -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Rumen Telbizov Unix Systems Administrator http://telbizov.com -- Rumen Telbizov Unix Systems Administrator http://telbizov.com ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: Device timeout from mfi(9) while booting 9.2-RELEASE
On Fri, Oct 04, 2013 at 11:23:45AM +1000, Jan Mikkelsen wrote: Hi, We have Drake Skinny (9240-4i and 9240-8i) cards running, most recently with 9.2-RC4. We'll be testing 9.2-RELEASE in the next week or two. We have seen problems with older versions of the firmware, and interactions with some Supermicro BIOS versions. All these problems have been resolved by upgrading to the latest versions of the controller firmware and system BIOSes. We're running with with hw.mfi.msi=1. I haven't tested without that since early 2011; I should probably try it out. So: What firmware do you have in the controller? Did you update it? Yes, I reflashed one board to the latest blob I was able to find at that time. For the second board, I did not bothered. The info I am able to find right now in my outcoming mbox, is (this is for four-ports, was flashed, I believe) mfi0: 916 (boot + 3s/0x0020/info) - Firmware version 2.130.374-2023 mfi0: 917 (boot + 6s/0x0020/info) - Package version 20.10.1-0126 mfi0: 918 (boot + 6s/0x0020/info) - Board Revision 03A (and this is for eight-ports) mfi0: 326 (boot + 4s/0x0020/info) - Firmware version 2.130.314-1585 mfi0: 327 (boot + 5s/0x0020/info) - Package version 20.10.1-0101 mfi0: 328 (boot + 5s/0x0020/info) - Board Revision 03B I am pretty much sure that the issue is either in the m/b compatibility or mfi(4) driver. pgpEdTm8QzQ0P.pgp Description: PGP signature
Re: Device timeout from mfi(9) while booting 9.2-RELEASE
We have Drake Skinny (9240-4i and 9240-8i) cards running, most recently with 9.2-RC4. We'll be testing 9.2-RELEASE in the next week or two. We have seen problems with older versions of the firmware, and interactions with some Supermicro BIOS versions. All these problems have been resolved by upgrading to the latest versions of the controller firmware and system BIOSes. We're running with with hw.mfi.msi=1. I haven't tested without that since early 2011; I should probably try it out. So: What firmware do you have in the controller? Did you update it? Yes, I reflashed one board to the latest blob I was able to find at that time. For the second board, I did not bothered. The info I am able to find right now in my outcoming mbox, is (this is for four-ports, was flashed, I believe) mfi0: 916 (boot + 3s/0x0020/info) - Firmware version 2.130.374-2023 mfi0: 917 (boot + 6s/0x0020/info) - Package version 20.10.1-0126 mfi0: 918 (boot + 6s/0x0020/info) - Board Revision 03A (and this is for eight-ports) mfi0: 326 (boot + 4s/0x0020/info) - Firmware version 2.130.314-1585 mfi0: 327 (boot + 5s/0x0020/info) - Package version 20.10.1-0101 mfi0: 328 (boot + 5s/0x0020/info) - Board Revision 03B I am pretty much sure that the issue is either in the m/b compatibility or mfi(4) driver. OK. I haven't seen timeout errors like that for a long time, so you could be right. I don't know. This is from a running 9.2-RC4 system: mfi1: Drake Skinny port 0xc000-0xc0ff mem 0xfad9c000-0xfad9,0xfadc-0xfadf irq 30 at device 0.0 on pci2 mfi1: Using MSI mfi1: Megaraid SAS driver Ver 4.23 ... mfi1: 15228 (boot + 3s/0x0020/info) - Firmware version 2.130.384-2291 mfi1: 15229 (boot + 5s/0x0020/info) - Package version 20.11.1-0137 mfi1: 15230 (boot + 5s/0x0020/info) - Board Revision 03A Regards, Jan. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
