I spoke too soon when I reported this being stable for Solaris8/9.
It still crashes my Sol9 loaded DNS server. The symptoms are the same:
it is stable just as long as you don't filter by UDP ports, but as soon
as you add UDP port filter rules, it will crash after a while.
I ran ACT on the core dump, and I've included the relevant portion of the
output. Trap 0x31 is some sort of data access MMU miss in fr_derefrule().
What could cause that?
Joseph Tam <[email protected]>
Edited output of act:
act 8.5
Release: SunOS 5.9 Generic_122300-31
panic: BAD TRAP: type=%x rp=%p addr=%p mmu_fsr=%x
lbolt is 0x20fd0e2, 0t34590946
...
PANIC occurred on this thread
***
process id 363 is /var/djb/bin/dnscache, parent process is 358
uid is 0x1fb 0t507, gid is 0x1fb 0t507
thread addr 0x30003367340, proc addr 0x30003364078, lwp addr 0x30003359c30
Thread bound to cpu id 0x1
t_state is 0x4 - TS_ONPROC
Scheduling info:
t_pri is 0x0, t_epri is 0x0, t_cid is 0x1
scheduling class is: TS
t_disp_time: is 0x20fbfd2, 0t34586578
last switched: 43 secs ago on cpu 0x1
pc is 0x104d1ac, sp is 0x1452210, t_stk 0x2a100371af0
Stack trace is:
unix: setjmp ()
unix: panicsys+0x44 (0x105d2c0,0x2a1003704c8,0x1,0x1,0x8,0x8)
unix: vpanic ()
unix: panic+0x1c (0x105d2c0,0x31,0x2a100370720,0x30000c59b2c,0x0)
unix: die+0xa4 (0x31,0x2a100370720,0x30000c59b2c,0x0)
unix: trap+0x8e4 (0x2a100370720)
unix: priv_rtt
ipf: fr_derefrule+0xe8
(0x30000c59b2c,0x2a100370a5e,0x30,0x89522416,0xffffffffffffffe8,0x30)
SUNW,UltraSPARC-III+: bcopy ()
ce: ce_wput+0x264 (0x30001e21818,0x30001f4eb40)
unix: putnext+0x21c (0x300011112f0,0x30001f4eb40)
pfil: pfilmodwput+0x40 (0x300011112f0,0x30001f4eb40)
unix: putnext+0x21c (0x30001111060,0x30001f4eb40)
ip: ip_wput_frag+0xb30 (0x30009475048?,0x30001f4eb40?,0x30009475098,0x0,0x4000)
ip: ip_wput_ire_fragmentit ()
ip: ip_wput_ire+0x16ac
(0x30003290438,0x30003c44b80,0x30009475048,0x300025e5540,0x0)
ip: ip_wput_pktoptions+0x56c (0x30003290438,0x30003c44b80,0x14fa230)
ip: ip_wput+0x78 (0x30003290438,0x30003c44b80)
unix: putnext+0x21c (0x300032901a8,0x30003c44b80)
udp: udp_wput+0x6bc (0x300032901a8,0x3000413f280)
unix: putnext+0x21c (0x300032906c8,0x3000413f280)
genunix: strput+0x270
(0x3000328b348,0x3000413f280,0x2a100371a00,0x2a100371698,0x0,0x0)
genunix: kstrputmsg+0x36c
(0x30001f0ce50,0x30003d16000,0x2a100371a00,0x7c,0x0,0x4)
sockfs: sosend_dgram+0x25c (0x30001f0ce50,0x30009357c90,0x10,0x2a100371a00,0x8)
sockfs: sosendmsg+0x3f4 (0x30001f0ce50,0x2a1003719d0,0x2a100371a00)
sockfs: sendit+0x15c (0x3,0x2a1003719d0,0x2a100371a00,0x8)
sockfs: sendto+0x78 (0x3,0x69ed8,0x7c,0x0,0xffbffcb0,0x10)
sockfs: sendto32+0x3c (0x3,0x69ed8,0x7c,0x0,0xffbffcb0,0x10)
unix: syscall_trap32+0xa8 (0x3,0x69ed8,0x7c,0x0,0xffbffcb0,0x10)
...
List of modules: ### MODULES ###
Id Loadaddr Size Info Rev Module Filename
...
80 13dd635 39be - 1 pfil (pfil Streams module 2.1.13) - drv/pfil
80 13dd635 39be - 1 pfil (pfil Streams driver 2.1.13) - drv/pfil
...
114 780aa000 57311 - 1 ipf (IP Filter: v4.1.32_RC4) - /usr/kernel/drv/ipf
...
KERNEL MESSAGE BUFFER: ### MSGBUF ###
lbolt msg
...
6a8 IP Filter: v4.1.32_RC4, running.
...
7e1 IP Filter: v4.1.32_RC4 detached.
7f5 IP Filter: v4.1.32_RC4, running.
...
20fc07a panic[cpu1]/thread=30003367340:
20fc07a BAD TRAP: type=31 rp=2a100370720 addr=30000c59b2c mmu_fsr=0
20fc07a
20fc07a dnscache:
20fc07a trap type = 0x31
20fc07a addr=0x30000c59b2c
20fc07a pid=363, pc=0x118b8a8, sp=0x2a10036ffc1, tstate=0x80001600,
context=0x8a4
20fc07a g1-g7: 3000365e16c, 1, 1388, 14, 18, 10, 30003367340
20fc07a
20fc07a 000002a100370440 unix:die+a4 (31, 2a100370720, 30000c59b2c, 0,
3000413f280, 3000365e158)
20fc07a %l0-3: 0000000000000000 0000030000c59b2c 000002a100370720
000002a100370610
%l4-7: 0000000000000031 0000000000000000 0000000000000000
0000000000000008
20fc0de 000002a100370520 unix:trap+8e4 (2a100370720, 0, 10000, 10200, 300,
5751428)
20fc0de %l0-3: 0000000000000001 0000000000000000 0000030003364078
0000000000000031
%l4-7: 0000000000000006 0000000000000001 0000000000000000
0000000000000000
20fc0de 000002a100370670 unix:ktl0+48 (30005751428, 30003367340, 14acc00,
150, 1a100f29172, 780ef6c8)
20fc0de %l0-3: 0000000000000001 0000000000001400 0000000080001600
000000000102db30
%l4-7: 000003000001bad8 0000000000000020 0000000000000000
000002a100370720
20fc0de 000002a1003707c0 ipf:fr_derefrule+e8 (30000c59b2c, 2a100370a5e, 30,
89522416, ffffffffffffffe8, 30)
20fc0de %l0-3: 0000000000000000 0000000000000010 0000030000fe9a80
0000000000000000
%l4-7: 0000000000000000 0000000000000020 0000000000000000
000002a100370750
20fc0de 000002a100370980 ce:ce_wput+264 (30001e21818, 30001f4eb40, 0,
30001015510, 22, 30)
20fc0de %l0-3: 0000030000c59b14 0000000000000022 0000000000000000
0000000000000000
%l4-7: 0000000000000800 0000030005c5d7c0 000002a100370a3c
0000030000074788
20fc142 000002a100370a90 unix:putnext+21c (0, 30001f4eb40, 2, 30001e1bed8,
14, 2a100370ae0)
20fc142 %l0-3: 0000000001390d14 0000030001e21908 0000030001e21818
0000000000000000
%l4-7: 0000000000000000 000000007805b3e8 00000300011112f0
0000030001f4eb40
20fc142 000002a100370b40 pfil:pfilmodwput+40 (300011112f0, 30001e1bed8, 20,
0, bad4f2, 10)
20fc142 %l0-3: 0000030001f4eb40 00000000ff000000 0000030005c5d7c0
0000030000fe9a80
%l4-7: 0000030003c44b80 0000030005c5d7c0 0000000000000000
0000000001000000
20fc142 000002a100370c00 unix:putnext+21c (0, 30001f4eb40, 0, 8000000, 800,
ba9617670800)
20fc142 %l0-3: 00000000013df918 00000300011113e0 00000300011112f0
0000000000000000
%l4-7: 0000000000000000 000000007805d928 0000030001111060
0000030001f4eb40
20fc142 000002a100370cb0 ip:ip_wput_frag+b30 (e, 0, 10000, 30001111060,
3000413f280, 3000365e158)
20fc142 %l0-3: 000003000408ea40 0000000000000000 0000030009475048
0000000000000000
%l4-7: 0000030001f4eb40 0000000000000000 0000000000000000
0000000000000008
20fc1a6 000002a100370e10 ip:ip_wput_ire+16ac (10000, 300000742b0, 3, ffff,
89522441, 30003c44b80)
20fc1a6 %l0-3: 0000000000015afb 00000300011d8000 0000030009475048
0000030000fe9b18
%l4-7: 0000000000000000 0000000000000000 0000030001111060
0000000089522416
20fc1a6 000002a100371000 ip:ip_wput_pktoptions+56c (30009475048, 0,
89522416, 30000fe9b18, 0, 0)
20fc1a6 %l0-3: 0000030003c44b80 0000000000000000 00000000014eb800
00000300025e5540
%l4-7: 0000030009475048 0000030003290438 0000000089522416
00000300025e5550
20fc1a6 000002a1003710d0 ip:ip_wput+78 (30003290438, 30003c44b80, 20, 8, 1,
0)
20fc1a6 %l0-3: 0000030009357c90 0000000000000000 0000000000000000
0000000000000000
%l4-7: 0000030003359c30 0000000000000003 000000000000007c
000002a1003719d0
20fc1a6 000002a100371190 unix:putnext+21c (0, 30001199f80, 20, 0, 53, 10)
20fc1a6 %l0-3: 00000000012a4750 000003000119def8 0000030003290438
0000000000000000
%l4-7: 0000030001199f88 00000000014ebed8 00000300032901a8
0000030003c44b80
20fc20a 000002a100371240 udp:udp_wput+6bc (3000194cf50, ffff, 84,
30000fe9b2c, 0, 10)
20fc20a %l0-3: 0000000000000084 0000030000fe9b18 0000000089522416
0000030003c44b80
%l4-7: 0000000000000014 000000000000096a 00000300032901a8
000003000413f280
20fc20a 000002a100371320 unix:putnext+21c (0, 3000413f280, 20, 30003c44b80,
0, 10)
20fc20a %l0-3: 0000000001251118 0000030003290298 00000300032901a8
0000000000000000
%l4-7: 0000000000000000 00000000014e3f90 00000300032906c8
000003000413f280
20fc20a 000002a1003713d0 genunix:strput+270 (3000328b348, 0, 0, 2a100371698,
0, 0)
20fc20a %l0-3: 0000000000000004 00000300032906c8 0000000000000000
000003000328b348
%l4-7: 000003000413f280 000002a100371a00 0000000000020400
0000000000020400
20fc20a 000002a1003715c0 genunix:kstrputmsg+36c (30001f0ce50, 300032906c8,
0, 0, 0, 0)
20fc20a %l0-3: 000003000328b3c8 0000030003d16000 000003000328b348
0000000000000004
%l4-7: 0000000000000083 000002a100371a00 0000000000000000
0000030003d16000
20fc26e 000002a1003716a0 sockfs:sosend_dgram+25c (0, 30009357c90, 10,
2a100371a00, 8, 22)
20fc26e %l0-3: 0000000000000010 0000000001469800 000000000000007c
0000030001f0ce50
%l4-7: 0000030003d16000 0000000000000078 0000000000000000
0000000000000000
20fc26e 000002a100371790 sockfs:sosendmsg+3f4 (0, 30009357c90, 7, 20, 0, 0)
20fc26e %l0-3: 0000000000000010 0000000000000008 0000030009357c90
0000000000000000
%l4-7: 0000030001f0ce50 000002a1003719d0 000002a100371a00
0000000000000000
20fc26e 000002a100371850 sockfs:sendit+15c (2a100371a00, 8, 30001f0ce50, 8,
1, 0)
20fc26e %l0-3: 0000030009357c90 0000000000000000 0000000000000000
0000000000000000
%l4-7: 0000030003359c30 0000000000000003 000000000000007c
000002a1003719d0
20fc26e 000002a100371920 sockfs:sendto+78 (3, 69ed8, 7c, 0, ffbffcb0, 10)
20fc26e %l0-3: 000000000c3cac70 00000000ff2428e0 00000000ff2428e4
0000000000000000
%l4-7: 0000000000000000 0000000000000000 0000000000000000
0000000000000000
20fc2d2 000002a100371a40 sockfs:sendto32+3c (3, 69ed8, 7c, 0, ffbffcb0, 10)
20fc2d2 %l0-3: 00000000ff2404e8 000000000116693c 00000000ffffc000
000000000000000c
%l4-7: 000000000000006e 000000000140f8b0 0000000000069ed8
0000000000060c00
