Kernel panic (-current) AMD64 GENERIC.MP
Hello, today one of my freshly upgraded machines after one week of normal work hanged up. Don't think it's hardware related, machine was working with 4.2- stable for last 3 months without doubt. Any idea what caused that hangup ? I saw the following on the console and could only touch reset button. After reset everything works as it should, but don't know how long. I looked through http://www.openbsd.com/plus.html but didn't find anything related to that issue. panic: pool_do_get(knotepl): free list modified: magic=765eeab8; page0xfe80735dc000; item addr 0xfe80735dc688 Starting stack trace... panic() at panic+0x136 pool_do_get() at pool_do_get+0x371 pool_get() at pool_get_+0x2a kqueue_register() at kqueue_register+0x1d8 sys_kevent() at sys_kevent+0x157 syscall() at syscall+0x2a3 --- syscall (number 270) --- end of kernel end trace frame: 0x48cac040, count: 251 0x4b0906ea: End of stack trace. syncing disks... and dmesg: OpenBSD 4.3-beta (GENERIC.MP) #1569: Wed Feb 27 13:01:06 MST 2008 [EMAIL PROTECTED]:/usr/src/sys/arch/amd64/compile/ GENERIC.MP real mem = 2145902592 (2046MB) avail mem = 2072178688 (1976MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.34 @ 0x7fee8000 (67 entries) bios0: vendor FUJITSU SIEMENS // Phoenix Technologies Ltd. version 4.06 Rev. 1.06.2300 date 05/16/2007 bios0: FUJITSU SIEMENS PRIMERGY RX200 S3 acpi0 at bios0: rev 2 acpi0: tables DSDT FACP SPCR MCFG HPET APIC BOOT acpi0: wakeup devices PE2_(S4) PXH0(S5) PE4_(S4) PE6_(S4) PXH1(S4) CBD_(S4) USB1(S4) USB2(S4) USB3(S4) USB4(S4) USB5(S4) KEYB(S4) PS2M(S4) COM1(S1) COM2(S1) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpihpet0 at acpi0: 14318179 Hz acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Xeon(R) CPU 5120 @ 1.86GHz, 1862.19 MHz cpu0: FPU ,VME ,DE ,PSE ,TSC ,MSR ,PAE ,MCE ,CX8 ,APIC ,SEP ,MTRR ,PGE ,MCA ,CMOV ,PAT ,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS- CPL,VMX,EST,TM2,CX16,xTPR,NXE,LONG cpu0: 4MB 64b/line 16-way L2 cache cpu0: apic clock running at 265MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Xeon(R) CPU 5120 @ 1.86GHz, 1861.92 MHz cpu1: FPU ,VME ,DE ,PSE ,TSC ,MSR ,PAE ,MCE ,CX8 ,APIC ,SEP ,MTRR ,PGE ,MCA ,CMOV ,PAT ,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS- CPL,VMX,EST,TM2,CX16,xTPR,NXE,LONG cpu1: 4MB 64b/line 16-way L2 cache ioapic0 at mainbus0 apid 2 pa 0xfec0, version 20, 24 pins ioapic1 at mainbus0 apid 3 pa 0xfec8, version 20, 24 pins acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (PE2_) acpiprt2 at acpi0: bus 2 (PSU_) acpiprt3 at acpi0: bus 3 (PSD1) acpiprt4 at acpi0: bus 4 (PSD2) acpiprt5 at acpi0: bus 5 (PXH0) acpiprt6 at acpi0: bus 7 (PE4_) acpiprt7 at acpi0: bus -1 (PXH1) acpiprt8 at acpi0: bus -1 (CBD_) acpiprt9 at acpi0: bus 12 (PCIH) acpicpu0 at acpi0 acpicpu1 at acpi0 acpibtn0 at acpi0: PWRB ipmi at mainbus0 not configured pci0 at mainbus0 bus 0: configuration mode 1 pchb0 at pci0 dev 0 function 0 Intel 5000P Host rev 0x92 ppb0 at pci0 dev 2 function 0 Intel 5000 PCIE x8 rev 0x92 pci1 at ppb0 bus 1 ppb1 at pci1 dev 0 function 0 Intel 6321ESB PCIE rev 0x01 pci2 at ppb1 bus 2 ppb2 at pci2 dev 0 function 0 Intel 6321ESB PCIE rev 0x01 pci3 at ppb2 bus 3 ppb3 at pci2 dev 1 function 0 Intel 6321ESB PCIE rev 0x01 pci4 at ppb3 bus 4 ppb4 at pci1 dev 0 function 3 Intel 6321ESB PCIE-PCIX rev 0x01 pci5 at ppb4 bus 5 mpi0 at pci5 dev 5 function 0 Symbios Logic SAS1068 rev 0x01: apic 3 int 0 (irq 11) scsibus0 at mpi0: 112 targets sd0 at scsibus0 targ 1 lun 0: LSILOGIC, Logical Volume, 3000 SCSI2 0/ direct fixed sd0: 75340MB, 75340 cyl, 16 head, 128 sec, 512 bytes/sec, 154296320 sec total ppb5 at pci0 dev 3 function 0 Intel 5000 PCIE rev 0x92 pci6 at ppb5 bus 6 ppb6 at pci0 dev 4 function 0 Intel 5000 PCIE rev 0x92 pci7 at ppb6 bus 7 ppb7 at pci7 dev 0 function 0 ServerWorks PCIE-PCIX rev 0xb5 pci8 at ppb7 bus 8 bge0 at pci8 dev 4 function 0 Broadcom BCM5715 rev 0xa3, BCM5715 A3 (0x9003): apic 2 int 16 (irq 11), address 00:0a:e4:83:14:c6 brgphy0 at bge0 phy 1: BCM5714 10/100/1000baseT PHY, rev. 0 bge1 at pci8 dev 4 function 1 Broadcom BCM5715 rev 0xa3, BCM5715 A3 (0x9003): apic 2 int 17 (irq 9), address 00:0a:e4:83:14:c7 brgphy1 at bge1 phy 1: BCM5714 10/100/1000baseT PHY, rev. 0 ppb8 at pci0 dev 5 function 0 Intel 5000 PCIE rev 0x92 pci9 at ppb8 bus 9 ppb9 at pci0 dev 6 function 0 Intel 5000 PCIE x8 rev 0x92 pci10 at ppb9 bus 10 ppb10 at pci0 dev 7 function 0 Intel 5000 PCIE rev 0x92 pci11 at ppb10 bus 11 pchb1 at pci0 dev 16 function 0 Intel 5000 Error Reporting rev 0x92 pchb2 at pci0 dev 16 function 1 Intel 5000 Error Reporting rev 0x92 pchb3 at pci0 dev 16 function 2 Intel 5000 Error Reporting rev 0x92 pchb4 at pci0 dev 17 function 0 Intel 5000 Reserved rev 0x92 pchb5 at pci0 dev 19 function 0 Intel 5000 Reserved rev 0x92 pchb6 at pci0 dev 21 function 0 Intel
Re: OpenBGPD MIB
On Saturday, March 24, 2007, at 23:49:12, misc@openbsd.org wrote: sophisticated montoring system with snmp,that is kind of an oxymoron, isn't it... there's no such thing as far as I am aware of. Hello Henning, it's not exactly what you think :P I've to configure such system and typed ip of one extreme box into it. After a while I saw few things which surprised me a lot: 1. cpu/mem - nothing special 2. interface status - nothing special 3. bgp peers configuration - oh... what a clever system, I thought :P I clicked into it and saw that it looks after prefix count, session up/downs and reachability of neighbor. It's very very nice, isn't it? And I've managed to do that by few clicks. When I saw that I just wanted to do the same on my bgp boxes :-) So i've instaled that clumsy net-snmp packages, configured it out, clicked into mon system and typed ip addr of openbsd box, and nothing happened - just cpu/mem and interfaces status... I googled around and found PF mibs and not only (http://www.packetmischief.ca/openbsd/snmp/) Any chances to add that to the wishlist for next releases? -- Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
Re: OpenBGPD MIB
On Sunday, March 25, 2007, at 15:40:18, Claudio Jeker wrote: You should create a port or net-snmp flavor of these changes. I even have some dirty diffs to have a terse bgpctl output usable to feed into rrdtool. I should clean them up a bit and commit it. Hello Claudio, I've talked about your response with my friends, and I've almost won (my bet was that you were working on that :P). If you have anything we can test and write/modify/add to your tools we are ready to work on it :-) -- Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
Re: GRE over IPsec
On Sunday, March 25, 2007, at 18:55:31, Chris Jones wrote: Hey all, I know that it's possible to run GRE over and IPsec tunnel but I am wondering if anyone here has seen some good documentation (besides the man pages) or a howto on setting this up. I'm trying to config my OpenBSD 4.0firewall to interop with a route-based VPN network with a mix of Fortigate and Netscreen firewalls. Fortigates and Netscreens both use GRE interaces as tunnel interfaces when creating route-based VPN tunnels. Right now all endpoints are using un-numbered (0.0.0.0/0) GRE interfaces and so I would like to use a similar configuration on the OpenBSD side but I am just wondering how to accomplish this as I am uncertain how to bind the GRE interface to a tunnel. Hello Cris, GRE is standard and works in OpenBSD as RFC says ;-) When I was running gre over ipsec tunnel between two openbsd boxes (OpenBSD 3.8 or sth like that) it worked without any problems. but it works till now, so example from config of that machine (ip changed): vpn1# cat /etc/hostname.gre0 1.1.1.1 2.2.2.2.netmask 0x carp0 !ifconfig gre0 tunnel 1.1.1.1 2.2.2.2 !route add -inet 192.168.1.0/24 2.2.2.2 few things you should be aware of: a) sysctl.conf (net.inet.gre.allow=1, net.inet.ip.mtudisc=1) b) MTU - gre is taking 24 bytes from frame (i.e. 1476 from 1500 bytes) c) IPSec uses DF bit - if you don't remember about that you can get into windowing problem (ethernet uses 1500 bytes and can't be splitted into fragments because of don't fragment bit) d) use different ip address space for your vpn-routers/concentrators and for your local networks. If you get blank paper and try to draw that (with OSI model in mind) you will make it in a few minutes :-) Good luck :) -- Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
OpenBGPD MIB
Hi guys, I've looked over for importing bgpd status to snmp to use with 'sofisticated' monitoring system. Hope somebody has similar problem. Can you give me some links or tell the way you do such things ? ps. yeah, I know I can write my own, but I hope not to be Christopher Columbus :) -- regards, Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
Re: Squid 2.6 transparent proxy with pf
On Thursday, December 21, 2006, at 15:32:24, Peter N. M. Hansteen wrote: Dominik Zalewski [EMAIL PROTECTED] writes: My question is can redirect traffic on $int_if to another machine connected to the same interface? Does this rule is corrrect ? You can redirect, but you need to let the packets from the proxy pass without redirection to the rest of the world. rdr pass on $int_if proto tcp from any to any port 80 - $squid port 8080 I would supplement this with a 'no rdr' rule for the proxy generated traffic. one thing you should notice is that if packets are generated from the same network your proxy stands in, then proxy will try to send reply packets (and packets with www pages of course) directly to machines in your LAN. And it means problems ;) What I should do is create another network (i.e. if your lan is 10.0.0.0/24 you can use 10.1.0.0/24) and attach 10.1.0.1 to OpenBSD NAT box and 10.1.0.2 to proxy. Then add NAT rules at OpenBSD NAT box and the following lines: no rdr on $int_if from 10.1.0.2 rdr on $int_if from your.lan/net to any port 80 - 10.1.0.2 port 8080 I made such config about a year ago and it worked. Maybe there are newer features in PF that will work now, but my scenario was good year ago and I haven't changed it (old IT rule: if sth works well don't touch that :-)) regards, -- Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
Re: Squid 2.6 transparent proxy with pf
On Thursday, December 21, 2006, at 14:04:34, misc@openbsd.org wrote: Dominik Zalewski [EMAIL PROTECTED] writes: I have OpenBSD 4.0 firewall and I would like to redirect all outgoing http requests to my squid web proxy. Daniel Hartmeier wrote about this a while back, his article can be found at http://www.benzedrine.cx/transquid.html However Daniel's article doesn't cover squid-2.6. Guys from squid team changed configuration options in squid.conf which you should use to make it working. Here you are working config for 2.6.STABLE5: http_port 3128 transparent #httpd_accel_host virtual #httpd_accel_port 80 #httpd_accel_with_proxy on #httpd_accel_uses_host_header on Daniel: can you change it also at your page to cover that ? -- Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
PF question
Hello all, I was looking for a ipfw looking-like statement in PF: ipfw add 10 fwd ip_proxy,proxy_port from 192.168.1.0/24 to any 25 via fxp0 Is it possible to forward packet to some destination in the same subnet without changing SRC/DST_ADDRESS ? I RTFMed but haven't found anything... -- regards, Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
Re: New Article
On Friday, November 24, 2006, at 22:43:18, Chris wrote: This site is a riot! it makes fun of all the OS's i.e. NetBSD: http://uncyclopedia.org/wiki/NetBSD NetBSD (interNET Bourne Sexual Disease) is a computer virus :-P Anyway, I think real men write their own device drivers should be motto of the next -stable release of OpenBSD :) -- Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
PF/rdr/nat
Hi all, I was looking for any idea how to tune OBSD with PF, rdr nat. I use rdr round-robin of port 80 to backend webservers using private adress space. When packets go back to clients watching webpage PF makes nat on them. Anyway, if I check it with ~100Mbps of traffic everything goes slower and slower and after few minutes clients sees that webserver is responding with very long delay to client's requests. However after ~15 seconds everything works well for another minute... I was reading OpenBSD/PF FAQ, trying to change limits in PF but problem still exists. After pfctl -x misc the following comes to logs: Nov 16 08:06:30 ungabunga /bsd: pf: BAD state: TCP 10.0.0.1:80 1.1.1.1:80 2.2.2.23:5027 [lo=1659423809 high=1659488734 win=16384 modulator=0] [lo=1312540182 high=1312540506 win=65535 modulator=0] 4:4 A seq=1312540182 ack=1659423809 len=1460 ackskew=0 pkts=3188:5511 dir=out,rev Doest anyone have an idea what I should look for to find what should be tuned up? other info: there are ~2500 state entries. TIMEOUTS: tcp.first 120s tcp.opening 30s tcp.established 86400s tcp.closing 900s tcp.finwait 45s tcp.closed 90s tcp.tsdiff 30s udp.first60s udp.single 30s udp.multiple 60s icmp.first 20s icmp.error 10s other.first 60s other.single 30s other.multiple 60s frag 15s interval 10s adaptive.start24000 states adaptive.end 48000 states src.track 0s LIMITS: stateshard limit4 src-nodes hard limit4 frags hard limit4 tableshard limit 1000 table-entries hard limit 10 -- regards, Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
Re: pf load balancing and failover
On Friday, October 27, 2006, at 12:23:24, Pete Vickers wrote:
Hi Berk,
I'm really intereted in this. I have a load of legacy tcp session
based load balancing with I'd love to migrate to an OpenBSD/pf based
solution. Do you have a patch with applies cleanly to 4.0 ?
afair this patch is applied in -current tree and we are using it for a
few weeks now and works preety well.
We are rdring all traffic between 3 servers in farm: 10.0.0.13,14,15
so we are using -k 0.0.0.0/0 :-)
#!/bin/sh
$webserver1=10.0.0.13
$webserver2=10.0.0.14
$webserver3=10.0.0.15
removeweb() (
# removeweb table ip
pfctl -t $1 -Td $2
pfctl -k 0.0.0.0/0 -k $2
)
addweb() (
# addweb table ip
pfctl -t $1 -Ta $2
)
while true ; do {
webstatus1=`curl --connect-timeout 10 $webserver1 2/dev/null`
webstatus2=`curl --connect-timeout 10 $webserver2 2/dev/null`
webstatus3=`curl --connect-timeout 10 $webserver3 2/dev/null`
if [ X$webstatus1 != XOK ]; then
removeweb wwwfarm $webserver1
else
addweb wwwfarm $webserver1
fi
if [ X$webstatus2 != XOK ]; then
removeweb wwwfarm $webserver2
else
addweb wwwfarm $webserver2
fi
if [ X$webstatus3 != XOK ]; then
removeweb wwwfarm $webserver3
else
addweb wwwfarm $webserver3
fi
} ;
sleep 5;
done
exit 0
--
Sylwester S. Biernacki [EMAIL PROTECTED]
X-NET, http://www.xnet.com.pl/
Re: pf load balancing and failover
On Sunday, October 29, 2006, at 15:43:09, Berk D. Demir wrote: We are rdring all traffic between 3 servers in farm: 10.0.0.13,14,15 so we are using -k 0.0.0.0/0 :-) If you're not using sticky addresses, you don't need the patch. If you're using them, you should use the patch and kill the lingering src-track entries with pfctl option '-K' (capital K) huh - you're right... our application working in wwwfarm is clever one and don't need sticky-address option in rdr rules:) -- Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
Intel Server Adapters (NICs)
Hello, about a month ago I wrote I'm glad about em(4) driver which works pretty well on few of my boxes. However I need to change my opinion... after what I saw today in the lab: We have connected pretty well testing box - Navtel InterWatch (www.navtelcom.com). It has one 6 slots and in one of them it has 2 GigE TX ports. We've configured the following scenario: 1) Navtel port A --- em0 --- em1 --- Navtel port B 2) Navtel port A --- em2 --- em3 --- Navtel port B em0 and em1 are built-in mainboard: em0 at pci4 dev 0 function 0 Intel PRO/1000 PT (82571EB) rev 0x06: apic 2 int 16 (irq 10), em1 at pci4 dev 0 function 1 Intel PRO/1000 PT (82571EB) rev 0x06: apic 2 int 17 (irq 11), and em2 and em3 are Intel Dual Port Server Adapter on PCI-e (4x): em2 at pci5 dev 0 function 0 Intel PRO/1000MT (82573E) rev 0x03: apic 2 int 19 (irq 10), em3 at pci6 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: apic 2 int 16 (irq 11), pf is disabled, between em0 and em1 whole traffic goes through kernel routing process (Navtel port A and em0 in one /24 network, and em1 and Navtel port B are in different /24 network) sysctl tcp.send receive space is turned to 65535 When we generate 500Mbps of traffic (1434 bytes in ethernet, which gives 1400 bytes of payload in TCP) from port A to B and from B to A (two streams, each of 500Mbps) everything works pretty well on PT chips and MT chips. When we change payload to 64bytes (called IP killer :P) and put it in scenario 1) machine gets freeze and no packet is comming to port B of Navtel device. It's rather normal, there are no ASIC-based boxes which can work with such traffic :) What was strange, when we connected Navtel to em2 and em3 which is one PCI-e (4x) dual-port card and started to generate traffic from port A to B and from B to A machine has restarted about a second after test started. We've change sysctl values to move machine to debugger if anything goes bad, but it didn't change anything. I think that it's somehow connected to chipsets of that cards and mainboard bridges which are responsible for transferring packets through the mainboard. Anyway, I started to feel badly about Intel... Second test was to generate 900Mbps of pure IP traffice (payload 1400 bytes) from port A to B and second stream from B to A. In scenario 1 machine got freeze and hasn't forward any packet from em0 to em1. When we changed em0/1 to em2/3 all traffic is comming from port A to B without any loss and machine gets 75% interrupt on uniprocessor kernel. So, we've changed to MP kernel and... scenario 1) hasn't changed at all (freeze all the time), and scenario 2 got 100% idle and 0% interrupts on both CPUs (strange, I thought that it'll be 1/2 of previous 75% :P). Anyway, when we connected anything to em0/1 ports during that test and generate more than 100Mbps (bittwist software packet generator run at the second box) our test machine got freeze again... What else? Kernel is taken from CVS -current tree. After all these tests I'm changing my opinion about Intel cards, especially when I read that PT chipsets are Intel's newest baby. Does anyone got simmilar problems ? Maybe there are other ways to tune NICs to work under such traffic (buffers on NIC?). I'm not an expert in Intel network cards so any idea will be appreciated :) Maybe you can tell about other chipsets that works fine under such heavy traffic ? -- regards, Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
Re: BGP Lookingglass
On Monday, August 7, 2006, at 12:40:40, misc@openbsd.org wrote: On 2006/08/07 11:46, Philip Olsson wrote: Im woundering if there exists a looking glass suitable for public access over http that uses the new read only socket in openbgpd ? http://null-ptr.net/sw/lg/ works - besides the mentioned files you will need: bgpd_flags=-r /var/www/var/run/bgpd.sock everything listed in `ldd /usr/sbin/bgpctl' output (ld.so, lib's) you probably have /var set as nosuid, this means you can't use ping/traceroute in the jail unless you're willing to relax that (and you can't use sudo to run them since that too is setuid). Might be worth also pointing out some SSH modification here, http://archives.neohapsis.com/archives/openbsd/2006-04/1811.html which (I haven't tested, but..) should let you separate webserver from routers and just forward the RO control socket on, which makes a certain amount of sense to me, especially on a public access setup. We have written our own, which also uses id_rsa keys and users on other route-servers. We had to give up with jailing apache cause too much problems happened and now we have the following: http://www.pl-ix.pl/tools.asp (.asp is fake of course, our programmer loves such playing, it's normal PHP file) If anyone want I can send sources or publish it somewhere ;-) -- Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
Re: bgpd error: route decision engine terminated; signal 11
On Sunday, March 19, 2006, at 19:22:25, fabioFVZ wrote: Hello, i have a problem with my openbgpd (OpenBSD 3.8 from Original CD :) ) After random time...bgpd exit with this error: [..] Any idea? Many thanks Have similar problems. Try update obgpd to current version via CVS It worked for me, and since then I firstly update to current and then ask questions ;-) regs, -- Sylwester S. Biernacki [EMAIL PROTECTED] X-NET, http://www.xnet.com.pl/
