Bill,
 
Thanks for the reply, your experience and statistics do indeed seem to be identical to our own! 
 
Originally we were on 3.4.32pre1 which we were told was unstable, so we proceeded with an upgrade to 3.4.35 which seemed to be the latest stable release in the 3.4 branch.  Please keep the list informed if your testing of a newer version(I am assume you are making the jump to 4.x?) turns out to fix the problem.  We have hammered many machines in the lab with all kinds of different traffic and cannot replicate the crash problem, but stick that same box in production and it will go down within a week or two guaranteed.
 
Thanks!
 


From: Sweeney, William- CIPS [mailto:[EMAIL PROTECTED]
Sent: Wednesday, September 27, 2006 3:16 PM
To: Clark, Kevin (IT); [email protected]
Subject: RE: IP Filter 3.4.35 crashes on Solaris?

Kevin,
we've had identical issues with ipf 3.4.35 on both Solaris 8 and 9.  Random panics and probably ~15% reboot on filter reload. Much like you, we have had to build in the 'panic on reload' issue into our procedures and mantinance timing. We've been trying various OS and ipf tweaks to stack and state size, to no avail.  I'm in the process of upgrading to the newest version of IPF, in hopes it will resolve the situation.  We haven't been able to replicate the issue in the lab either.  We do see fewer problems on devices with smaller rule-sets, but I haven't had time to quantify exact figures.
 
Sun has been of zero help on this, since as far as they're concerned, IPF on Solaris 8 or 9 is a 'third party application'. Ironicaly, they instructed us to 'contact the software vendor'.  I think its thier way of ensuring we force our vendors to support Solaris 10 in a more timely fashion. ;-) 
 
Regards,
Bill


From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Clark, Kevin (IT)
Sent: Wednesday, September 27, 2006 11:28 AM
To: [email protected]
Subject: IP Filter 3.4.35 crashes on Solaris?

Hello all,
 
We have had chronic issues with IP Filter panics on our Solaris 8 machines, I was wondering if anyone else has experienced the same symptoms and if there are any fixes or workarounds.  After looking at many coredumps it definitely is IP Filter at the root of the problems, when we sent the coredumps to Sun they agreed.   I have seen mention of problems related to fr_scanlist, and this may be the case with us as well.
 
Sometimes the machine will reboot seemingly at random, no matter whether it has low traffic or high traffic passing through it at the time.  Quite often it will panic when we push new rules to ipfilter.conf and issue a reload, so much so that this is now a feared operation.
 
Here are the details of our environment, comments appreciated.
 
SunOS xxxxx 5.8 Generic_108528-24 sun4u sparc SUNW,Ultra-80
bash-2.03# ipf -V
ipf: IP Filter: v3.4.35 (432)
Kernel: IP Filter: v3.4.35
Running: yes
Log Flags: 0 = none set
Default: pass all, Logging: available
Active list: 0
 
Example panic:
Sep 26 11:31:28 hqcsbh6 ^Mpanic[cpu1]/thread=73a7be00:
Sep 26 11:31:28 hqcsbh6 unix: [ID 799565 kern.notice] BAD TRAP: type=34 rp=404ca
fb0 addr=3b mmu_fsr=0
Sep 26 11:31:28 hqcsbh6 unix: [ID 100000 kern.notice]
Sep 26 11:31:28 hqcsbh6 unix: [ID 839527 kern.notice] fping:
Sep 26 11:31:28 hqcsbh6 unix: [ID 123557 kern.notice] alignment error:
Sep 26 11:31:28 hqcsbh6 unix: [ID 381800 kern.notice] addr=0x3b
Sep 26 11:31:28 hqcsbh6 unix: [ID 101969 kern.notice] pid=28044, pc=0x71e2dddc,
sp=0x404cb040, tstate=0x4480001e01, context=0x118a
Sep 26 11:31:28 hqcsbh6 unix: [ID 743441 kern.notice] g1-g7: 1, 404cb3c7, 39, 36
, 0, 0, 73a7be00
Sep 26 11:31:28 hqcsbh6 unix: [ID 100000 kern.notice]
Sep 26 11:31:28 hqcsbh6 unix: [ID 100000 kern.notice]
Sep 26 11:31:28 hqcsbh6 genunix: [ID 872817 kern.notice] 404cae38 unix:die+a4 (3
4, 404cafb0, 3b, 0, 404cafb0, 71368084)
Sep 26 11:31:28 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 71368100 00000
000 719108cc 00000000 00000000 00000020 00000000 00000000
Sep 26 11:31:29 hqcsbh6 genunix: [ID 872817 kern.notice] 404caea8 unix:trap+5b8
(10000, 3b, 0, 73a71618, 0, 7360f8f0)
Sep 26 11:31:29 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 000005dc 00000
000 404cafb0 00000034 0080000b 00000000 735e1590 732bc480
Sep 26 11:31:29 hqcsbh6 genunix: [ID 872817 kern.notice] 404caf50 unix:prom_rtt+
0 (73a78b8c, 732bc480, 20, 10455310, 7135ff48, 73a78b8c)
Sep 26 11:31:29 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 00000002 00001
c00 80001e01 1002b294 ff218c44 ff370e74 00000000 404cafb0
Sep 26 11:31:29 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb040 unix:putnext+1
e0 (404cb3dc, 73f72284, 404cb3c4, 73f72258, 404cb3c8, 2f)
Sep 26 11:31:29 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 00000000 fe501
002 00000004 00000000 00000000 00000000 00000000 00000020
Sep 26 11:31:29 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb1b0 ipf:fr_scanlis
t+8d0 (404cb3f0, 73f72458, 404cb3c4, 73f7242c, 404cb3c8, 73f723e0)
Sep 26 11:31:29 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 73f72220 00010
000 00000000 00000000 00000000 00000000 00000000 00000000
Sep 26 11:31:29 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 73f72220 00010
000 00000000 00000000 00000000 00000000 00000000 00000000
Sep 26 11:31:29 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb320 ipf:fr_check+6
48 (733a50fc, 14, 7017381c, 1, 404cb564, 404cb3c4)
Sep 26 11:31:29 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 71e4e400 00000
000 71e4e500 404cb3c8 404cb9f4 00000000 00000040 00000000
Sep 26 11:31:29 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb430 ipf:fr_prechec
k+ca4 (404cb680, 71368084, 404cb564, 733a50fc, 732bc480, 732bc480)
Sep 26 11:31:29 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 404cb564 00000
054 0000ff00 f36dff6f 0000006d 00000073 00000000 81010100
Sep 26 11:31:30 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb4f8 ipf:fr_qout+48
0 (71368084, 732bc480, 20, 1, 5d486, 8)
Sep 26 11:31:30 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 00000000 bde4d
3c2 7339e0e0 73a2c6a4 00000004 404cbc50 73a2c658 00000000
Sep 26 11:31:30 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb638 unix:putnext+1
e0 (719108cc, 71e32e88, 732bc480, 7077e3f4, 0, 71368084)
Sep 26 11:31:30 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 71368100 00000
000 719108cc 00000000 00000000 00000020 00000000 00000000
Sep 26 11:31:30 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb698 ip:ip_wput_ire
+790 (0, 719108cc, 732bc480, 14, ffff, 0)
Sep 26 11:31:30 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb698 ip:ip_wput_ire
+790 (0, 719108cc, 732bc480, 14, ffff, 0)
Sep 26 11:31:30 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 000005dc 733a5
0fc 00021af2 7017381c c7594601 00000000 735e1590 732bc480
Sep 26 11:31:30 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb748 ipf:ipf_ip_qin
+ac (73a78b8c, 732bc480, 20, 10455310, 7135ff48, 73a78b8c)
Sep 26 11:31:30 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 101917cc 7135f
f40 73a20b84 00000000 ff218c44 ff370e74 ff383864 00000000
Sep 26 11:31:30 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb7b8 unix:putnext+1
e0 (7135ff40, 71e33538, 732bc480, 10455310, 7135ff48, 73a78b8c)
Sep 26 11:31:30 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 71365f50 7135f
f40 73a20b84 00000000 00000000 00000000 00000000 00000000
Sep 26 11:31:30 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb818 icmp:icmp_wput
+478 (54, ffff, 14, 10484234, c7594601, 732c4044)
Sep 26 11:31:30 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 732bc480 73361
480 73a20b84 733a50fc 10424048 00000000 00000000 404cbce0
Sep 26 11:31:31 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb880 unix:putnext+1
e0 (73a78a2c, 102d0210, 73361480, 10482ba4, 0, 73a20b84)
Sep 26 11:31:31 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 713656c8 00000
000 73a78a2c 00000000 404cb9f4 00000000 00000040 00000000
Sep 26 11:31:31 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 713656c8 00000
000 73a78a2c 00000000 404cb9f4 00000000 00000040 00000000
Sep 26 11:31:31 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb8e0 genunix:strput
+22c (0, 4, 73a78a2c, 404cbab8, 0, 0)
Sep 26 11:31:31 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 73361480 404cb
c50 0000ff00 f36dff6f 0000006d 00000073 7efefeff 81010100
Sep 26 11:31:31 hqcsbh6 genunix: [ID 872817 kern.notice] 404cb9f8 genunix:kstrpu
tmsg+2d0 (735b0b70, 0, 0, 0, 0, 73a78a2c)
Sep 26 11:31:31 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 00000003 00000
000 7339e0e0 73a2c6a4 00000004 404cbc50 73a2c658 00000000
Sep 26 11:31:31 hqcsbh6 genunix: [ID 872817 kern.notice] 404cba68 sockfs:sosend_
dgram+220 (8, 73a75430, 0, 40, 0, 7339e0e0)
Sep 26 11:31:31 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 404cbc50 735b0
b70 73a75430 00000010 069e21ff ff3e8e30 00000000 00000000
Sep 26 11:31:31 hqcsbh6 genunix: [ID 872817 kern.notice] 404cbaf8 sockfs:sosendm
sg+3fc (0, 7, 8, 735b0b70, 404cbc50, 10)
Sep 26 11:31:31 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 73a75430 404cb
c34 00000000 00000000 ff381d3e 00000000 00000000 ff3e7760
Sep 26 11:31:31 hqcsbh6 genunix: [ID 872817 kern.notice] 404cbb60 sockfs:sendit+
12c (40, 735b0b70, 7360f8f0, 0, 0, 404cbc34)
Sep 26 11:31:31 hqcsbh6 genunix: [ID 872817 kern.notice] 404cbb60 sockfs:sendit+
12c (40, 735b0b70, 7360f8f0, 0, 0, 404cbc34)
Sep 26 11:31:31 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 404cbc50 00000
003 00000008 73a75430 ff218c44 ff370e74 ff383864 00000000
Sep 26 11:31:31 hqcsbh6 genunix: [ID 872817 kern.notice] 404cbbd0 sockfs:sendto+
78 (3, 277e0, 40, 0, 27638, 10)
Sep 26 11:31:32 hqcsbh6 genunix: [ID 645206 kern.notice]   %l0-7: 00000000 10129
834 00000000 00000000 00000000 1040a4c8 00000000 00000000
Sep 26 11:31:32 hqcsbh6 unix: [ID 100000 kern.notice]
Sep 26 11:31:32 hqcsbh6 genunix: [ID 672855 kern.notice] syncing file systems...
Sep 26 11:31:32 hqcsbh6 genunix: [ID 733762 kern.notice]  8
Sep 26 11:31:53 hqcsbh6 last message repeated 20 times
Sep 26 11:31:54 hqcsbh6 genunix: [ID 622722 kern.notice]  done (not all i/o comp
leted)
Sep 26 11:31:55 hqcsbh6 genunix: [ID 353387 kern.notice] dumping to /dev/dsk/c0t
0d0s1, offset 65536
Sep 26 11:32:08 hqcsbh6 genunix: [ID 409368 kern.notice] ^M100% done: 16197 page
s dumped, compression ratio 3.13,
Sep 26 11:32:08 hqcsbh6 genunix: [ID 851671 kern.notice] dump succeeded



NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.


NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.

Reply via email to