[Bug 209259] kernel panic when using PF and NAT
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209259 Kristof Provost changed: What|Removed |Added Resolution|--- |Overcome By Events Status|New |Closed CC||k...@freebsd.org --- Comment #1 from Kristof Provost --- FreeBSD 10.3 is no longer supported. If the problem can be reproduced on 12.0 or 11.2 please re-open this bug. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
Hi, there is a function called pf_get_sport in /usr/src/sys/netpfil/pf/pf_lb.c which contains a do while loop, the guard is ! PF_AEQ(&init_addr, naddr, af)). We put a counter in this loop and we saw it spin 431728 times, this appears to coincide with a lockup. we'll continue investigating tomorrow. Regards Joe Jones From: Kristof Provost Sent: 01 March 2018 09:57:18 To: Joe Jones Cc: freebsd-pf@freebsd.org Subject: Re: Kernel Panic On 1 Mar 2018, at 15:37, Joe Jones wrote: > yes we use pfsync. Yesterday we tried with pfsync switched off, the > box still locked up but this time without a panic. > > We make the DIOCRADDADDRS ioctl on the master and the backup (we use > CARPed pairs). > Interesting. It might be related to pfsync. Is is the master that panics or the backup? Or both? Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
On Thu, Mar 1, 2018 at 9:43 AM, Joe Jones wrote: > Hi Kristo, > > It's just the master that crashed, the backup can take over. > > We think the panic we got by compiling with witness and invariant may be a > red herring. > > We are now looking rules like > > nat on $isp_if from to any -> sticky-address > > if we replace the external_napts table with a single address rather than a > block of addresses the box does not crash. > > We are following this line of investigation at the moment. > This is a known issue and should be documented somewhere, possibly man page. It source is when locking was re-designed for pf(4). https://github.com/freebsd/freebsd/blob/releng/11.1/sys/netpfil/pf/pf_lb.c#L428 * XXXGL: in the round-robin case we need to store * the round-robin machine state in the rule, thus * forwarding thread needs to modify rule. * * This is done w/o locking, because performance is assumed * more important than round-robin precision. * * In the simpliest case we just update the "rpool->cur" * pointer. However, if pool contains tables or dynamic * addresses, then "tblidx" is also used to store machine * state. Since "tblidx" is int, concurrent access to it can't * lead to inconsistence, only to lost of precision. * * Things get worse, if table contains not hosts, but * prefixes. In this case counter also stores machine state, * and for IPv6 address, counter can't be updated atomically. * Probably, using round-robin on a table containing IPv6 * prefixes (or even IPv4) would cause a panic. The fix is to add proper locking around such scenario. At minimum there would be needed a RULES_WLOCK in there or maybe reside to atomics. > Regards > Joe Jones > > > On 01/03/18 09:57, Kristof Provost wrote: > >> On 1 Mar 2018, at 15:37, Joe Jones wrote: >> >>> yes we use pfsync. Yesterday we tried with pfsync switched off, the box >>> still locked up but this time without a panic. >>> >>> We make the DIOCRADDADDRS ioctl on the master and the backup (we use >>> CARPed pairs). >>> >>> Interesting. It might be related to pfsync. Is is the master that panics >> or the backup? Or both? >> >> Regards, >> Kristof >> > > ___ > freebsd-pf@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org" > > -- > Ermal > ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
Hi Kristo, It's just the master that crashed, the backup can take over. We think the panic we got by compiling with witness and invariant may be a red herring. We are now looking rules like nat on $isp_if from to any -> sticky-address if we replace the external_napts table with a single address rather than a block of addresses the box does not crash. We are following this line of investigation at the moment. Regards Joe Jones On 01/03/18 09:57, Kristof Provost wrote: On 1 Mar 2018, at 15:37, Joe Jones wrote: yes we use pfsync. Yesterday we tried with pfsync switched off, the box still locked up but this time without a panic. We make the DIOCRADDADDRS ioctl on the master and the backup (we use CARPed pairs). Interesting. It might be related to pfsync. Is is the master that panics or the backup? Or both? Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
Hi Kristof, yes we use pfsync. Yesterday we tried with pfsync switched off, the box still locked up but this time without a panic. We make the DIOCRADDADDRS ioctl on the master and the backup (we use CARPed pairs). Regards Joe Jones On 01/03/18 03:00, Kristof Provost wrote: On 28 Feb 2018, at 9:52, Kristof Provost wrote: On 27 Feb 2018, at 20:40, Joe Jones wrote: we have a kernel panic after compiling with witness and invariant Feb 27 13:49:33 sovapn1 kernel: lock order reversal: Feb 27 13:49:33 sovapn1 kernel: 1st 0xfe000fed78b8 pf_idhash (pf_idhash) @ /usr/src/sys/netpfil/pf/pf.c:1078 Feb 27 13:49:33 sovapn1 kernel: 2nd 0xf8001e0474a8 pfsync (pfsync) @ /usr/src/sys/netpfil/pf/if_pfsync.c:1667 That’s a lock order reversal. It’s not good, but it should at worst result in a deadlock. Did the system stop after this? It also looks like a different problem from the panic you initially reported. Also, do you actively use pfsync in this setup? Does the panic happen on the box where you DIOCRADDADDRS or the other(s)? Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
On 1 Mar 2018, at 15:37, Joe Jones wrote: yes we use pfsync. Yesterday we tried with pfsync switched off, the box still locked up but this time without a panic. We make the DIOCRADDADDRS ioctl on the master and the backup (we use CARPed pairs). Interesting. It might be related to pfsync. Is is the master that panics or the backup? Or both? Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
On 28 Feb 2018, at 9:52, Kristof Provost wrote: On 27 Feb 2018, at 20:40, Joe Jones wrote: we have a kernel panic after compiling with witness and invariant Feb 27 13:49:33 sovapn1 kernel: lock order reversal: Feb 27 13:49:33 sovapn1 kernel: 1st 0xfe000fed78b8 pf_idhash (pf_idhash) @ /usr/src/sys/netpfil/pf/pf.c:1078 Feb 27 13:49:33 sovapn1 kernel: 2nd 0xf8001e0474a8 pfsync (pfsync) @ /usr/src/sys/netpfil/pf/if_pfsync.c:1667 That’s a lock order reversal. It’s not good, but it should at worst result in a deadlock. Did the system stop after this? It also looks like a different problem from the panic you initially reported. Also, do you actively use pfsync in this setup? Does the panic happen on the box where you DIOCRADDADDRS or the other(s)? Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
On 27 Feb 2018, at 20:40, Joe Jones wrote: we have a kernel panic after compiling with witness and invariant Feb 27 13:49:33 sovapn1 kernel: lock order reversal: Feb 27 13:49:33 sovapn1 kernel: 1st 0xfe000fed78b8 pf_idhash (pf_idhash) @ /usr/src/sys/netpfil/pf/pf.c:1078 Feb 27 13:49:33 sovapn1 kernel: 2nd 0xf8001e0474a8 pfsync (pfsync) @ /usr/src/sys/netpfil/pf/if_pfsync.c:1667 That’s a lock order reversal. It’s not good, but it should at worst result in a deadlock. Did the system stop after this? It also looks like a different problem from the panic you initially reported. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
Hi,
we have a kernel panic after compiling with witness and invariant
Feb 27 13:49:33 sovapn1 kernel: lock order reversal:
Feb 27 13:49:33 sovapn1 kernel: 1st 0xfe000fed78b8 pf_idhash (pf_idhash) @
/usr/src/sys/netpfil/pf/pf.c:1078
Feb 27 13:49:33 sovapn1 kernel: 2nd 0xf8001e0474a8 pfsync (pfsync) @
/usr/src/sys/netpfil/pf/if_pfsync.c:1667
Feb 27 13:49:33 sovapn1 kernel: stack backtrace:
Feb 27 13:49:33 sovapn1 kernel: #0 0x80a9efe0 at witness_debugger+0x70
Feb 27 13:49:33 sovapn1 kernel: #1 0x80a9eed3 at
witness_checkorder+0xe23
Feb 27 13:49:33 sovapn1 kernel: #2 0x80a20aad at __mtx_lock_flags+0x9d
Feb 27 13:49:33 sovapn1 kernel: #3 0x826d9f9d at
pfsync_insert_state+0x7d
Feb 27 13:49:33 sovapn1 kernel: #4 0x826ea729 at pf_state_insert+0x8b9
Feb 27 13:49:33 sovapn1 kernel: #5 0x826f444c at pf_test_rule+0x2bbc
Feb 27 13:49:33 sovapn1 kernel: #6 0x826eec58 at pf_test+0x1598
Feb 27 13:49:33 sovapn1 kernel: #7 0x826ffb2d at pf_check_in+0x1d
Feb 27 13:49:33 sovapn1 kernel: #8 0x80b488c4 at pfil_run_hooks+0xa4
Feb 27 13:49:33 sovapn1 kernel: #9 0x80ba5757 at ip_tryforward+0x157
Feb 27 13:49:33 sovapn1 kernel: #10 0x80ba7a8a at ip_input+0x38a
Feb 27 13:49:33 sovapn1 kernel: #11 0x80b47a00 at
netisr_dispatch_src+0x80
Feb 27 13:49:33 sovapn1 kernel: #12 0x80b309ce at ether_demux+0x15e
Feb 27 13:49:33 sovapn1 kernel: #13 0x80b317ee at ether_nh_input+0x31e
Feb 27 13:49:33 sovapn1 kernel: #14 0x80b47a00 at
netisr_dispatch_src+0x80
Feb 27 13:49:33 sovapn1 kernel: #15 0x80b30d62 at ether_input+0x62
Feb 27 13:49:33 sovapn1 kernel: #16 0x82671427 at igb_rxeof+0x6e7
Feb 27 13:49:33 sovapn1 kernel: #17 0x826708fe at igb_msix_que+0xee
we compiled with
options INVARIANTS
options INVARIANT_SUPPORT
options WITNESSoptions WITNESS_SKIPSPIN
options WITNESS_KDB
we went back to the 11.1 release as the relenge branch didn't compile
for us (probably our fault).
Regards
Joe Jones
On 27/02/18 04:40, Kristof Provost wrote:
On 26 Feb 2018, at 17:06, Joe Jones wrote:
Hi Kristof,
we are not updating rules during the test although in production
we will reload the rule set from time to time. We are constantly
adding and removing from tables though, using the DIOCRADDADDRS
and DIOCRDELADDRS ioctl, also DIOCKILLSTATES is being called a
lot. These are all in response to RADIUS events. We tried using
pfctl shell command rather than calling ioctl directly, to check
that it wasn't a problem with how we are calling the ioctl.
That’s interesting.
The panic leads me to suspect something’s wrong with the
kt->pfrkt_ipv4->rt, which would explain why we get the unexpected NULL
result.
My first guess at the cause would be a race condition, where it’s
being modified (through one of the ioctls you do) while the
pfr_pool_get() is walking it.
I don’t immediately see where that’d happen though, because both
DIOCRADDADDRS and DIOCRDELADDRS take the rules lock (and
pfr_pool_get() takes it too).
It might be interesting to run this with these extra asserts (and be
sure to enable INVARIANTS).
|diff --git a/sys/netpfil/pf/pf_table.c b/sys/netpfil/pf/pf_table.c
index 18342a94073..cad9b4ea89f 100644 --- a/sys/netpfil/pf/pf_table.c
+++ b/sys/netpfil/pf/pf_table.c @@ -962,6 +962,8 @@
pfr_unroute_kentry(struct pfr_ktable *kt, struct pfr_kentry *ke)
struct radix_node *rn; struct radix_head *head = NULL; +
PF_RULES_WASSERT(); + if (ke->pfrke_af == AF_INET) head =
&kt->pfrkt_ip4->rh; else if (ke->pfrke_af == AF_INET6) @@ -1855,6
+1859,8 @@ pfr_destroy_ktable(struct pfr_ktable *kt, int flushaddr) {
struct pfr_kentryworkq addrq; + PF_RULES_WASSERT(); + if (flushaddr) {
pfr_enqueue_addrs(kt, &addrq, NULL, 0); pfr_clean_node_mask(kt, &addrq); |
Regards,
Kristof
___
freebsd-pf@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
On 26 Feb 2018, at 17:06, Joe Jones wrote:
Hi Kristof,
we are not updating rules during the test although in production we
will reload the rule set from time to time. We are constantly adding
and removing from tables though, using the DIOCRADDADDRS and
DIOCRDELADDRS ioctl, also DIOCKILLSTATES is being called a lot. These
are all in response to RADIUS events. We tried using pfctl shell
command rather than calling ioctl directly, to check that it wasn't a
problem with how we are calling the ioctl.
That’s interesting.
The panic leads me to suspect something’s wrong with the
kt->pfrkt_ipv4->rt, which would explain why we get the unexpected NULL
result.
My first guess at the cause would be a race condition, where it’s
being modified (through one of the ioctls you do) while the
pfr_pool_get() is walking it.
I don’t immediately see where that’d happen though, because both
DIOCRADDADDRS and DIOCRDELADDRS take the rules lock (and pfr_pool_get()
takes it too).
It might be interesting to run this with these extra asserts (and be
sure to enable INVARIANTS).
diff --git a/sys/netpfil/pf/pf_table.c b/sys/netpfil/pf/pf_table.c
index 18342a94073..cad9b4ea89f 100644
--- a/sys/netpfil/pf/pf_table.c
+++ b/sys/netpfil/pf/pf_table.c
@@ -962,6 +962,8 @@ pfr_unroute_kentry(struct pfr_ktable *kt, struct
pfr_kentry *ke)
struct radix_node *rn;
struct radix_head *head = NULL;
+ PF_RULES_WASSERT();
+
if (ke->pfrke_af == AF_INET)
head = &kt->pfrkt_ip4->rh;
else if (ke->pfrke_af == AF_INET6)
@@ -1855,6 +1859,8 @@ pfr_destroy_ktable(struct pfr_ktable *kt, int
flushaddr)
{
struct pfr_kentryworkq addrq;
+ PF_RULES_WASSERT();
+
if (flushaddr) {
pfr_enqueue_addrs(kt, &addrq, NULL, 0);
pfr_clean_node_mask(kt, &addrq);
Regards,
Kristof
___
freebsd-pf@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
Hi Kristof,
we are not updating rules during the test although in production we will
reload the rule set from time to time. We are constantly adding and
removing from tables though, using the DIOCRADDADDRS and DIOCRDELADDRS
ioctl, also DIOCKILLSTATES is being called a lot. These are all in
response to RADIUS events. We tried using pfctl shell command rather
than calling ioctl directly, to check that it wasn't a problem with how
we are calling the ioctl.
A little background. Our production system is running on 8.4 and has
been stable for years. We are in the process of moving to 11.1 and are
having big problems with stability when we allow customer traffic into
the machine. At the moment we are using mirror ports on the switch to
play live traffic into it. We're trying to work out the simplest
configuration that causes a problem with a view to producing a good bug
report.
I have notices that the pfil interface
https://www.freebsd.org/cgi/man.cgi?query=pfil&sektion=9 has locking in
it which didn't exist in 8, I think it was introduced in 9? the locking
functions appear in the man page in 10. I don't know if that interface
is used directly by pf, but I'm guessing packet processing needs to be
thread safe in a way it didn't in 8.
Regards
Joe Jones
On 25/02/18 10:56, Kristof Provost wrote:
On 14 Feb 2018, at 19:57, Joe Jones wrote:
On 14/02/18 13:09, Kristof Provost wrote:
On 14 Feb 2018, at 23:47, Joe Jones wrote:
we are running test traffic through our system, after between 1 and
12 hours we get a kernel panic, always in the pfr_pool_get function
in /usr/src/sys/netpfil/pf/pf_table.c line 2140. After a bit of
investigation I confirmed that ke2 is set to null on line 2122.
It’d probably be interesting to know what the contents of uaddr/addr
is here.
From a very quick look at the code there’s supposed to be a route
lookup there, and I’d expect there to always be a result. The code
certainly expects it, because that looks to be what causes the panic.
(kgdb) p *uaddr
No symbol "uaddr" in current context.
(kgdb) p *addr
$1 = {
pfa = {
v4 = {
s_addr = 2016475826
},
v6 = {
__u6_addr = {
__u6_addr8 = 0xfe310d0c "��0x0\r1",
__u6_addr16 = 0xfe310d0c,
__u6_addr32 = 0xfe310d0c
}
},
addr8 = 0xfe310d0c "��0x0\r1",
addr16 = 0xfe310d0c,
addr32 = 0xfe310d0c
}
}
Interesting… That looks okay, so I have no idea why that lookup
returned NULL.
Are you modifying tables/rules at all during this test?
Am I right in thinking that's in network order.
I believe so, yes.
Regards,
Kristof
___
freebsd-pf@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: Kernel Panic
On 14 Feb 2018, at 23:47, Joe Jones wrote: Hi, we are running test traffic through our system, after between 1 and 12 hours we get a kernel panic, always in the pfr_pool_get function in /usr/src/sys/netpfil/pf/pf_table.c line 2140. After a bit of investigation I confirmed that ke2 is set to null on line 2122. It’d probably be interesting to know what the contents of uaddr/addr is here. From a very quick look at the code there’s supposed to be a route lookup there, and I’d expect there to always be a result. The code certainly expects it, because that looks to be what causes the panic. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Kernel Panic
Hi, we are running test traffic through our system, after between 1 and 12 hours we get a kernel panic, always in the pfr_pool_get function in /usr/src/sys/netpfil/pf/pf_table.c line 2140. After a bit of investigation I confirmed that ke2 is set to null on line 2122. We have tried a kernel compiled from /base/releng/11.1 and the equivalent branch of pfsense and get the same thing. The network card is a 4 port Intel I350 (igb). The test uses 2 of the 4 ports, msix with 2 queues per port. net.link.ifqmaxlen="2048" hw.igb.rx_process_limit="-1" hw.igb.num_queues="2" Next I'm going to start stripping parts out of our setup to try and get to a system that is stable under the test load. Can anyone shed any light on what is happening here. Thanks Joe Jones GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: current process= 12 (irq272: igb1:que 1) trap number= 12 panic: page fault cpuid = 3 KDB: stack backtrace: #0 0x80aadb97 at kdb_backtrace+0x67 #1 0x80a6bbf6 at vpanic+0x186 #2 0x80a6ba63 at panic+0x43 #3 0x80ee0832 at trap_fatal+0x322 #4 0x80ee0889 at trap_pfault+0x49 #5 0x80ee00c6 at trap+0x286 #6 0x80ec4d41 at calltrap+0x8 #7 0x824c65df at pf_map_addr+0x58f #8 0x824c6a11 at pf_get_translation+0x1e1 #9 0x824bade1 at pf_test_rule+0x241 #10 0x824b78bb at pf_test+0x174b #11 0x824c961d at pf_check_out+0x1d #12 0x80b71f4b at pfil_run_hooks+0x7b #13 0x80bcdb71 at ip_tryforward+0x1f1 #14 0x80bcfb88 at ip_input+0x368 #15 0x80b71140 at netisr_dispatch_src+0xa0 #16 0x80b5ac0d at ether_demux+0x16d #17 0x80b5b89b at ether_nh_input+0x31b Uptime: 4h17m34s Dumping 2778 out of 65374 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/carp.ko...Reading symbols from /usr/lib/debug//boot/kernel/carp.ko.debug...done. done. Loaded symbols for /boot/kernel/carp.ko Reading symbols from /boot/kernel/pfsync.ko...Reading symbols from /usr/lib/debug//boot/kernel/pfsync.ko.debug...done. done. Loaded symbols for /boot/kernel/pfsync.ko Reading symbols from /boot/kernel/pf.ko...Reading symbols from /usr/lib/debug//boot/kernel/pf.ko.debug...done. done. Loaded symbols for /boot/kernel/pf.ko Reading symbols from /boot/modules/mellynat.ko...done. Loaded symbols for /boot/modules/mellynat.ko Reading symbols from /boot/modules/mellycount.ko...done. Loaded symbols for /boot/modules/mellycount.ko Reading symbols from /boot/kernel/pflog.ko...Reading symbols from /usr/lib/debug//boot/kernel/pflog.ko.debug...done. done. Loaded symbols for /boot/kernel/pflog.ko Reading symbols from /boot/kernel/uhid.ko...Reading symbols from /usr/lib/debug//boot/kernel/uhid.ko.debug...done. done. Loaded symbols for /boot/kernel/uhid.ko #0 doadump (textdump=) at pcpu.h:222 222pcpu.h: No such file or directory. in pcpu.h (kgdb) backtrace #0 doadump (textdump=) at pcpu.h:222 #1 0x80a6b771 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:366 #2 0x80a6bc30 in vpanic (fmt=, ap=optimized out>) at /usr/src/sys/kern/kern_shutdown.c:759 #3 0x80a6ba63 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:690 #4 0x80ee0832 in trap_fatal (frame=0xfe31ac40, eva=153) at /usr/src/sys/amd64/amd64/trap.c:801 #5 0x80ee0889 in trap_pfault (frame=0xfe31ac40, usermode=0) at pcpu.h:222 #6 0x80ee00c6 in trap (frame=0xfe31ac40) at /usr/src/sys/amd64/amd64/trap.c:421 #7 0x80ec4d41 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236 #8 0x824dba80 in pfr_pool_get (kt=0xf80010e17000, pidx=, counter=0xf8000e925298, af=optimized out>) at /usr/src/sys/netpfil/pf/pf_table.c:2141 #9 0x824c65df in pf_map_addr (af=, r=, saddr=, naddr=optimized out>, init_addr=, sn=out>) at uma.h:363 #10 0x824c6a11 in pf_get_translation (pd=, m=, off=, direction=optimized out>, kif=, sn=0xfe31b248, skp=, nkp=, saddr=, daddr=, sport=, dport=optimized out>) at /usr/src/sys/netpfil/pf/pf_lb.c:223 #11 0x824bade1
Re: sonewconn: pru_attach() failed and kernel panic in PF
So it turns out I shouldn't commit things when jet lagged. You want r314810 in head. The other one was mistakenly done in stable/11. It needed to go there sooner or later so I'm just going to leave it. Regards, Kristof > On 5 Mar 2017, at 22:19, Kristof Provost wrote: > >> On 5 Mar 2017, at 21:42, Kristof Provost wrote: >> There’s only a couple of calls to uma_zfree() in pf_get_translations(). >> >> These are: >> * uma_zfree(V_pf_state_key_z, skp); >> * uma_zfree(V_pf_state_key_z, *nkp); >> * uma_zfree(V_pf_state_key_z, *skp); >> >> Going by the inconsistent pointer use the first one is rather suspect. >> Looking a bit deeper, pf_get_translation() is only called from one place, >> and it always passes stack variables for skp and nkp, so the first call >> ends up trying to free that, which won’t work too well. >> >> That’s a bug (and I’ll fix it), but you’re only running into it because >> pf_state_key_clone() returned NULL, which will only happen under memory >> pressure. >> > The fix is done in r314702. > > Regards, > Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: sonewconn: pru_attach() failed and kernel panic in PF
On 5 Mar 2017, at 21:42, Kristof Provost wrote: There’s only a couple of calls to uma_zfree() in pf_get_translations(). These are: * uma_zfree(V_pf_state_key_z, skp); * uma_zfree(V_pf_state_key_z, *nkp); * uma_zfree(V_pf_state_key_z, *skp); Going by the inconsistent pointer use the first one is rather suspect. Looking a bit deeper, pf_get_translation() is only called from one place, and it always passes stack variables for skp and nkp, so the first call ends up trying to free that, which won’t work too well. That’s a bug (and I’ll fix it), but you’re only running into it because pf_state_key_clone() returned NULL, which will only happen under memory pressure. The fix is done in r314702. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: sonewconn: pru_attach() failed and kernel panic in PF
On 27 Feb 2017, at 21:08, Ross wrote: Hello One of my machines panics almost every day. It is always like this: first there is a number of messages about "sonewconn: pcb 0xf80085478740: pru_attach() failed" at the same time and then panic. Here's an example: ... many lines of sonewconn ... Feb 27 13:41:43 core kernel: sonewconn: pcb 0xf8008575bcb0: pru_attach() failed Feb 27 13:41:43 core kernel: I wonder if you’re running low on memory by any chance. I think I know why you’re crashing, but I suspect your root problem is that you’re running low on memory and that’s why you’re seeing the pru_attach() failures, and eventually running into the pf panic. Feb 27 13:41:43 core kernel: KDB: stack backtrace: Feb 27 13:41:43 core kernel: #0 0x80b312c7 at kdb_backtrace+0x67 Feb 27 13:41:43 core kernel: #1 0x80ae5c92 at vpanic+0x182 Feb 27 13:41:43 core kernel: #2 0x80ae5b03 at panic+0x43 Feb 27 13:41:43 core kernel: #3 0x80fd6d51 at trap_fatal+0x351 Feb 27 13:41:43 core kernel: #4 0x80fd6f43 at trap_pfault+0x1e3 Feb 27 13:41:43 core kernel: #5 0x80fd64ec at trap+0x26c Feb 27 13:41:43 core kernel: #6 0x80fb9d61 at calltrap+0x8 Feb 27 13:41:43 core kernel: #7 0x80e4185e at uma_zfree_arg+0x4fe Feb 27 13:41:43 core kernel: #8 0x82442165 at pf_get_translation+0x2c5 There’s only a couple of calls to uma_zfree() in pf_get_translations(). These are: * uma_zfree(V_pf_state_key_z, skp); * uma_zfree(V_pf_state_key_z, *nkp); * uma_zfree(V_pf_state_key_z, *skp); Going by the inconsistent pointer use the first one is rather suspect. Looking a bit deeper, pf_get_translation() is only called from one place, and it always passes stack variables for skp and nkp, so the first call ends up trying to free that, which won’t work too well. That’s a bug (and I’ll fix it), but you’re only running into it because pf_state_key_clone() returned NULL, which will only happen under memory pressure. What should I do to fix it? You’ll need to look at your system and figure out who’s running away with all of your memory. Regards, Kristof ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
sonewconn: pru_attach() failed and kernel panic in PF
Hello One of my machines panics almost every day. It is always like this: first there is a number of messages about "sonewconn: pcb 0xf80085478740: pru_attach() failed" at the same time and then panic. Here's an example: ... many lines of sonewconn ... Feb 27 13:41:43 core kernel: sonewconn: pcb 0xf8008575bcb0: pru_attach() failed Feb 27 13:41:43 core kernel: Feb 27 13:41:43 core kernel: Feb 27 13:41:43 core kernel: Fatal trap 12: page fault while in kernel mode Feb 27 13:41:43 core kernel: cpuid = 5; apic id = 0a Feb 27 13:41:43 core kernel: fault virtual address = 0x0 Feb 27 13:41:43 core kernel: fault code = supervisor write data, page not present Feb 27 13:41:43 core kernel: instruction pointer= 0x20:0x80e45383 Feb 27 13:41:43 core kernel: stack pointer = 0x28:0xfe00d7dd7f80 Feb 27 13:41:43 core kernel: frame pointer = 0x28:0xfe00d7dd7fe0 Feb 27 13:41:43 core kernel: code segment = base 0x0, limit 0xf, type 0x1b Feb 27 13:41:43 core kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Feb 27 13:41:43 core kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Feb 27 13:41:43 core kernel: current process= 0 (em0 taskq) Feb 27 13:41:43 core kernel: trap number= 12 Feb 27 13:41:43 core kernel: panic: page fault Feb 27 13:41:43 core kernel: cpuid = 5 Feb 27 13:41:43 core kernel: KDB: stack backtrace: Feb 27 13:41:43 core kernel: #0 0x80b312c7 at kdb_backtrace+0x67 Feb 27 13:41:43 core kernel: #1 0x80ae5c92 at vpanic+0x182 Feb 27 13:41:43 core kernel: #2 0x80ae5b03 at panic+0x43 Feb 27 13:41:43 core kernel: #3 0x80fd6d51 at trap_fatal+0x351 Feb 27 13:41:43 core kernel: #4 0x80fd6f43 at trap_pfault+0x1e3 Feb 27 13:41:43 core kernel: #5 0x80fd64ec at trap+0x26c Feb 27 13:41:43 core kernel: #6 0x80fb9d61 at calltrap+0x8 Feb 27 13:41:43 core kernel: #7 0x80e4185e at uma_zfree_arg+0x4fe Feb 27 13:41:43 core kernel: #8 0x82442165 at pf_get_translation+0x2c5 Feb 27 13:41:43 core kernel: #9 0x824369d3 at pf_test_rule+0x2b3 Feb 27 13:41:43 core kernel: #10 0x82433e23 at pf_test+0x1a23 Feb 27 13:41:43 core kernel: #11 0x8244596d at pf_check_in+0x1d Feb 27 13:41:43 core kernel: #12 0x80c1e983 at pfil_run_hooks+0x83 Feb 27 13:41:43 core kernel: #13 0x80c82d7b at ip_input+0x3bb Feb 27 13:41:43 core kernel: #14 0x80c1d905 at netisr_dispatch_src+0xa5 Feb 27 13:41:43 core kernel: #15 0x80c0636a at ether_demux+0x12a Feb 27 13:41:43 core kernel: #16 0x80c06fc2 at ether_nh_input+0x322 Feb 27 13:41:43 core kernel: #17 0x80c1d905 at netisr_dispatch_src+0xa5 What should I do to fix it? PS This is FreeBSD 11.0-RELEASE ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
[Bug 209259] kernel panic when using PF and NAT
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209259 Mark Linimon changed: What|Removed |Added Assignee|freebsd-b...@freebsd.org|freebsd-pf@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
[Bug 143543] [pf] [panic] PF route-to causes kernel panic
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=143543 Kristof Provost changed: What|Removed |Added Resolution|--- |FIXED CC||k...@freebsd.org Status|In Progress |Closed --- Comment #6 from Kristof Provost --- Closing based on the report in comment #5. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
[Bug 143543] [pf] [panic] PF route-to causes kernel panic
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=143543 c...@pfsense.org changed: What|Removed |Added CC||c...@pfsense.org --- Comment #5 from c...@pfsense.org --- The examples provided here definitely work fine on all currently supported FreeBSD versions. This is safe to close unless there's some unusual edge case not detailed here. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-pf@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
[Bug 200222] [patch][pf] fix possible kernel panic on missing mtag
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200222 Gleb Smirnoff changed: What|Removed |Added Assignee|freebsd-pf@FreeBSD.org |gleb...@freebsd.org Status|New |In Progress --- Comment #2 from Gleb Smirnoff --- The patches referenced don't look like patches to FreeBSD head or stable/10. There is no pd.act.qid in FreeBSD pf. Can you please better explain the problem? -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
[Bug 200222] [patch][pf] fix possible kernel panic on missing mtag
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200222 Mark Linimon changed: What|Removed |Added Assignee|freebsd-b...@freebsd.org|freebsd-pf@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
[Bug 192426] [bpf] [panic] [patch]: Kernel panic when using BPF
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192426 Andrey V. Elsukov changed: What|Removed |Added Assignee|freebsd-pf@FreeBSD.org |a...@freebsd.org CC||a...@freebsd.org -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
[Bug 192426] [bpf] [panic] [patch]: Kernel panic when using BPF
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192426 Mark Linimon changed: What|Removed |Added Keywords||patch Assignee|freebsd-...@freebsd.org |freebsd-pf@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/124364: [pf] [panic] Kernel panic with pf + bridge
Just as a note, this is an issue especially when using bridge+carp+pf. On Tue, Sep 11, 2012 at 1:00 PM, Gleb Smirnoff wrote: > The following reply was made to PR kern/124364; it has been noted by GNATS. > > From: Gleb Smirnoff > To: Vladimir Shapkin > Cc: bug-follo...@freebsd.org > Subject: kern/124364: [pf] [panic] Kernel panic with pf + bridge > Date: Tue, 11 Sep 2012 14:51:06 +0400 > >Vladimir, > >have you tried to reproduce the problem on newer versions of FreeBSD? > > -- > Totus tuus, Glebius. > ___ > freebsd-pf@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org" -- Ermal ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/124364: [pf] [panic] Kernel panic with pf + bridge
Synopsis: [pf] [panic] Kernel panic with pf + bridge State-Changed-From-To: open->closed State-Changed-By: glebius State-Changed-When: Tue Sep 11 11:03:00 UTC 2012 State-Changed-Why: Submitter email bounces. http://www.freebsd.org/cgi/query-pr.cgi?pr=124364 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
kern/124364: [pf] [panic] Kernel panic with pf + bridge
The following reply was made to PR kern/124364; it has been noted by GNATS. From: Gleb Smirnoff To: Vladimir Shapkin Cc: bug-follo...@freebsd.org Subject: kern/124364: [pf] [panic] Kernel panic with pf + bridge Date: Tue, 11 Sep 2012 14:51:06 +0400 Vladimir, have you tried to reproduce the problem on newer versions of FreeBSD? -- Totus tuus, Glebius. ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: FreeBSD 9.0-RELEASE: VIMAGE and PF, Constant Kernel Panic
On 17. Feb 2012, at 19:42 , Robert Z wrote: > I am getting constant kernel panics with VIMAGE and PF on FreeBSD 9-RELEASE > with jails. > > I have tried diffrent PF configurations with VIMAGE + epair or VIMAGE + > netgragh and still get kernel panics. > The kernel panics stop as soon as I disable PF in rc.conf. > > Example of setup PF + VIMAGE + epair. > http://wiki.polymorf.fr/index.php/Howto:FreeBSD_jail_vnet > > Example of setup PF + VIMAGE + netgragh > http://druidbsd.sourceforge.net/vimage.shtml > > > I am attaching an example pic of said panics. > http://i40.tinypic.com/2q00etz.jpg > > > Any advice on solving this will be appreciated. VIMAGE is experimental. pf is not yet supported as are a couple of other things including most cloned interfaces etc. -- Bjoern A. Zeeb You have to have visions! It does not matter how good you are. It matters what good you do! ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
FreeBSD 9.0-RELEASE: VIMAGE and PF, Constant Kernel Panic
I am getting constant kernel panics with VIMAGE and PF on FreeBSD 9-RELEASE with jails. I have tried diffrent PF configurations with VIMAGE + epair or VIMAGE + netgragh and still get kernel panics. The kernel panics stop as soon as I disable PF in rc.conf. Example of setup PF + VIMAGE + epair. http://wiki.polymorf.fr/index.php/Howto:FreeBSD_jail_vnet Example of setup PF + VIMAGE + netgragh http://druidbsd.sourceforge.net/vimage.shtml I am attaching an example pic of said panics. http://i40.tinypic.com/2q00etz.jpg Any advice on solving this will be appreciated. Robert ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/158873: [pf] [panic] When I launch pf daemon, I have a kernel panic
Synopsis: [pf] [panic] When I launch pf daemon, I have a kernel panic Responsible-Changed-From-To: freebsd-pf->bz Responsible-Changed-By: bz Responsible-Changed-When: Wed Oct 19 11:38:23 UTC 2011 Responsible-Changed-Why: Take. Seen too late that here's another patch. http://www.freebsd.org/cgi/query-pr.cgi?pr=158873 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/160496: [pf] [patch] kernel panic with pf + VIMAGE
Synopsis: [pf] [patch] kernel panic with pf + VIMAGE Responsible-Changed-From-To: freebsd-pf->freebsd-virtualization Responsible-Changed-By: bz Responsible-Changed-When: Tue Sep 6 10:04:16 UTC 2011 Responsible-Changed-Why: Move to vritualization where we aggregate VIMAGE PRs. http://www.freebsd.org/cgi/query-pr.cgi?pr=160496 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/160496: [pf] [patch] kernel panic with pf + VIMAGE
Old Synopsis: [patch] kernel panic with pf + VIMAGE New Synopsis: [pf] [patch] kernel panic with pf + VIMAGE Responsible-Changed-From-To: freebsd-bugs->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Tue Sep 6 02:01:39 UTC 2011 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=160496 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/158873: [pf] [panic] When I launch pf daemon, I have a kernel panic
Old Synopsis: When I launch pf daemon, I have a kernel panic New Synopsis: [pf] [panic] When I launch pf daemon, I have a kernel panic Responsible-Changed-From-To: freebsd-net->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Thu Jul 14 01:55:42 UTC 2011 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=158873 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/143543: [pf] [panic] PF route-to causes kernel panic
The following reply was made to PR kern/143543; it has been noted by GNATS. From: Nick Leuta To: bug-follo...@freebsd.org, sl...@aprec.ru Cc: Subject: Re: kern/143543: [pf] [panic] PF route-to causes kernel panic Date: Sun, 14 Mar 2010 23:20:44 +0300 Hmm... Im my case "ipfw fwd" command doesn't work too - it forwards locally generated packets using the routing table (???)... but yes, it has some effect - it changes the interface where the packets are originated. PF's "route-to" command works fine, but only if the destination host is reachable... ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/143543: [pf] [panic] PF route-to causes kernel panic
The following reply was made to PR kern/143543; it has been noted by GNATS. From: =?windows-1251?B?0eL/8u7x6+Di?= To: bug-follo...@freebsd.org, sl...@aprec.ru Cc: Subject: Re: kern/143543: [pf] [panic] PF route-to causes kernel panic Date: Sun, 14 Mar 2010 19:00:25 +0300 I'm now using ipfw setfib command as workaround, PF as NAT + ipfw works fine for me. --=20 =D1 =F3=E2=E0=E6=E5=ED=E8=E5=EC, =C1=E5=EB=EE=E3=F3=F0=EE=E2 =D1=E2=FF=F2=EE=F1=EB=E0=E2 8 (81555) 7-40-99 =D0=E5=EB=E0=ED=F2, http://www.relant.ru mailto:sl...@aprec.ru ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/143543: [pf] [panic] PF route-to causes kernel panic
The following reply was made to PR kern/143543; it has been noted by GNATS. From: Nick Leuta To: bug-follo...@freebsd.org, sl...@aprec.ru Cc: Subject: Re: kern/143543: [pf] [panic] PF route-to causes kernel panic Date: Sun, 14 Mar 2010 03:34:50 +0300 I have the similar problem but in a bit different situation... the rule is: pass out quick route-to (vlan2 192.168.0.1) from 192.168.0.2 to any where 192.168.0.2 is binded to the vlan2 interface. The default gateway is 192.168.1.1 and is accessible through another interface. The "ping -S 192.168.0.2 192.168.0.1" command is used for test purposes, and (sic!) the 192.168.0.1 is unreachable (really down...). Without that rule we have: PING 192.168.0.1 (192.168.0.1) from 192.168.0.2: 56 data bytes ping: sendto: Host is down With the rule we obtain the kernel panic (in "ping" process) instead of the "ping: sendto: Host is down" message after the same timeout as in the case without rule. ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/143543: [pf] [panic] PF route-to causes kernel panic
Old Synopsis: PF route-to causes kernel panic New Synopsis: [pf] [panic] PF route-to causes kernel panic Responsible-Changed-From-To: freebsd-bugs->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Thu Feb 4 10:21:23 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=143543 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/141905: [pf] [panic] pf kernel panic on 7.2-RELEASE with empty pf.conf
Old Synopsis: pf kernel panic on 7.2-RELEASE with empty pf.conf New Synopsis: [pf] [panic] pf kernel panic on 7.2-RELEASE with empty pf.conf Responsible-Changed-From-To: freebsd-amd64->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Tue Dec 22 22:31:27 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=141905 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/130977: [netgraph][pf] kernel panic trap 12 on user connect to VPN server
Synopsis: [netgraph][pf] kernel panic trap 12 on user connect to VPN server State-Changed-From-To: open->closed State-Changed-By: mlaier State-Changed-When: Mon Apr 13 23:33:36 UTC 2009 State-Changed-Why: Fix commited to head and stable/7. Thanks. http://www.freebsd.org/cgi/query-pr.cgi?pr=130977 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/130977: [netgraph][pf] kernel panic trap 12 on user connect to VPN server
i'm tried to reproduce bug with current patch, but it's not
reproduced. as i see bug is fixed. thanks.
2009/4/11 Max Laier :
> The following reply was made to PR kern/130977; it has been noted by GNATS.
>
> From: Max Laier
> To: bug-follo...@freebsd.org,
> darki...@gmail.com
> Cc:
> Subject: Re: kern/130977: [netgraph][pf] kernel panic trap 12 on user connect
> to VPN server
> Date: Sat, 11 Apr 2009 01:11:54 +0100
>
> --Boundary-00=_LB+3JRWytgyQ4AY
> Content-Type: text/plain;
> charset="us-ascii"
> Content-Transfer-Encoding: 7bit
> Content-Disposition: inline
>
> Here is the MFC patch - if possible, please try and report back.
>
> --
> Max
>
> --Boundary-00=_LB+3JRWytgyQ4AY
> Content-Type: text/x-patch;
> charset="ISO-8859-1";
> name="mfc.ifg.patch"
> Content-Transfer-Encoding: 7bit
> Content-Disposition: attachment;
> filename="mfc.ifg.patch"
>
> Index: sys/net/if.c
> ===
> --- sys/net/if.c (revision 190905)
> +++ sys/net/if.c (working copy)
> @@ -128,6 +128,7 @@
> static void do_link_state_change(void *, int);
> static int if_getgroup(struct ifgroupreq *, struct ifnet *);
> static int if_getgroupmembers(struct ifgroupreq *);
> +static void if_delgroups(struct ifnet *);
> #ifdef INET6
> /*
> * XXX: declare here to avoid to include many inet6 related files..
> @@ -828,6 +829,7 @@
> rt_ifannouncemsg(ifp, IFAN_DEPARTURE);
> EVENTHANDLER_INVOKE(ifnet_departure_event, ifp);
> devctl_notify("IFNET", ifp->if_xname, "DETACH", NULL);
> + if_delgroups(ifp);
>
> IF_AFDATA_LOCK(ifp);
> for (dp = domains; dp; dp = dp->dom_next) {
> @@ -963,6 +965,53 @@
> }
>
> /*
> + * Remove an interface from all groups
> + */
> +static void
> +if_delgroups(struct ifnet *ifp)
> +{
> + struct ifg_list *ifgl;
> + struct ifg_member *ifgm;
> + char groupname[IFNAMSIZ];
> +
> + IFNET_WLOCK();
> + while (!TAILQ_EMPTY(&ifp->if_groups)) {
> + ifgl = TAILQ_FIRST(&ifp->if_groups);
> +
> + strlcpy(groupname, ifgl->ifgl_group->ifg_group, IFNAMSIZ);
> +
> + IF_ADDR_LOCK(ifp);
> + TAILQ_REMOVE(&ifp->if_groups, ifgl, ifgl_next);
> + IF_ADDR_UNLOCK(ifp);
> +
> + TAILQ_FOREACH(ifgm, &ifgl->ifgl_group->ifg_members, ifgm_next)
> + if (ifgm->ifgm_ifp == ifp)
> + break;
> +
> + if (ifgm != NULL) {
> + TAILQ_REMOVE(&ifgl->ifgl_group->ifg_members, ifgm,
> + ifgm_next);
> + free(ifgm, M_TEMP);
> + }
> +
> + if (--ifgl->ifgl_group->ifg_refcnt == 0) {
> + TAILQ_REMOVE(&ifg_head, ifgl->ifgl_group, ifg_next);
> + EVENTHANDLER_INVOKE(group_detach_event,
> + ifgl->ifgl_group);
> + free(ifgl->ifgl_group, M_TEMP);
> + }
> + IFNET_WUNLOCK();
> +
> + free(ifgl, M_TEMP);
> +
> + EVENTHANDLER_INVOKE(group_change_event, groupname);
> +
> + IFNET_WLOCK();
> + }
> + IFNET_WUNLOCK();
> +}
> +
> +/*
> * Stores all groups from an interface in memory pointed
> * to by data
> */
>
> --Boundary-00=_LB+3JRWytgyQ4AY--
> ___
> freebsd-pf@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
>
___
freebsd-pf@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/130977: [netgraph][pf] kernel panic trap 12 on user connect to VPN server
The following reply was made to PR kern/130977; it has been noted by GNATS.
From: Max Laier
To: bug-follo...@freebsd.org,
darki...@gmail.com
Cc:
Subject: Re: kern/130977: [netgraph][pf] kernel panic trap 12 on user connect
to VPN server
Date: Sat, 11 Apr 2009 01:11:54 +0100
--Boundary-00=_LB+3JRWytgyQ4AY
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Here is the MFC patch - if possible, please try and report back.
--
Max
--Boundary-00=_LB+3JRWytgyQ4AY
Content-Type: text/x-patch;
charset="ISO-8859-1";
name="mfc.ifg.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="mfc.ifg.patch"
Index: sys/net/if.c
===
--- sys/net/if.c (revision 190905)
+++ sys/net/if.c (working copy)
@@ -128,6 +128,7 @@
static void do_link_state_change(void *, int);
static intif_getgroup(struct ifgroupreq *, struct ifnet *);
static intif_getgroupmembers(struct ifgroupreq *);
+static void if_delgroups(struct ifnet *);
#ifdef INET6
/*
* XXX: declare here to avoid to include many inet6 related files..
@@ -828,6 +829,7 @@
rt_ifannouncemsg(ifp, IFAN_DEPARTURE);
EVENTHANDLER_INVOKE(ifnet_departure_event, ifp);
devctl_notify("IFNET", ifp->if_xname, "DETACH", NULL);
+ if_delgroups(ifp);
IF_AFDATA_LOCK(ifp);
for (dp = domains; dp; dp = dp->dom_next) {
@@ -963,6 +965,53 @@
}
/*
+ * Remove an interface from all groups
+ */
+static void
+if_delgroups(struct ifnet *ifp)
+{
+ struct ifg_list *ifgl;
+ struct ifg_member *ifgm;
+ char groupname[IFNAMSIZ];
+
+ IFNET_WLOCK();
+ while (!TAILQ_EMPTY(&ifp->if_groups)) {
+ ifgl = TAILQ_FIRST(&ifp->if_groups);
+
+ strlcpy(groupname, ifgl->ifgl_group->ifg_group, IFNAMSIZ);
+
+ IF_ADDR_LOCK(ifp);
+ TAILQ_REMOVE(&ifp->if_groups, ifgl, ifgl_next);
+ IF_ADDR_UNLOCK(ifp);
+
+ TAILQ_FOREACH(ifgm, &ifgl->ifgl_group->ifg_members, ifgm_next)
+ if (ifgm->ifgm_ifp == ifp)
+ break;
+
+ if (ifgm != NULL) {
+ TAILQ_REMOVE(&ifgl->ifgl_group->ifg_members, ifgm,
+ ifgm_next);
+ free(ifgm, M_TEMP);
+ }
+
+ if (--ifgl->ifgl_group->ifg_refcnt == 0) {
+ TAILQ_REMOVE(&ifg_head, ifgl->ifgl_group, ifg_next);
+ EVENTHANDLER_INVOKE(group_detach_event,
+ ifgl->ifgl_group);
+ free(ifgl->ifgl_group, M_TEMP);
+ }
+ IFNET_WUNLOCK();
+
+ free(ifgl, M_TEMP);
+
+ EVENTHANDLER_INVOKE(group_change_event, groupname);
+
+ IFNET_WLOCK();
+ }
+ IFNET_WUNLOCK();
+}
+
+/*
* Stores all groups from an interface in memory pointed
* to by data
*/
--Boundary-00=_LB+3JRWytgyQ4AY--
___
freebsd-pf@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/130977: [netgraph][pf] kernel panic trap 12 on user connect to VPN server
On Friday 10 April 2009 13:50:03 Mikolaj Golub wrote:
> The following reply was made to PR kern/130977; it has been noted by GNATS.
>
> From: Mikolaj Golub
> To: bug-follo...@freebsd.org,darki...@gmail.com
> Cc:
> Subject: Re: kern/130977: [netgraph][pf] kernel panic trap 12 on user
> connect to VPN server Date: Fri, 10 Apr 2009 14:42:59 +0300
>
> The problem here (as in kern/131310 and may be in some other reports) is
> that net/if.c:if_attach() when attaching interface adds it to default group
> ALL calling if_addgroup(ifp, IFG_ALL). But when interface is removed (in
> this case ng, but the same thing occurs for other interfaces too, e.g. I
> checked it for tap) the reference to it does not removed from
> ifgl_group.ifg_members list.
>
> The simple test can be used to confirm this:
>
> 1) add interface (e.g. starting mpd);
>
> 2) run kgdb and find reference to ng interface in the list
> ifnet.tqh_first.if_groups->tqh_first.ifgl_group.ifg_members
>
> E.g. in my case it is:
>
> (kgdb) p
> *ifnet.tqh_first.if_groups->tqh_first.ifgl_group.ifg_members.tqh_first.ifgm
>_next.tqe_next.ifgm_next.tqe_next.ifgm_next.tqe_next.ifgm_ifp $1 = {if_softc
> = 0xc4e180c0, if_l2com = 0x0, if_link = {tqe_next = 0x0, tqe_prev =
> 0xc4264808}, if_xname = "ng0", '\0' , if_dname =
> 0xc4bd60d9 "ng", if_dunit = 0, if_addrhead = { tqh_first = 0xc4ba4e00,
> tqh_last = 0xc4ba4e60}, if_klist = {kl_list = {slh_first = 0x0}, kl_lock =
> 0xc07abb00 , kl_unlock = 0xc07abb30 ,
> ...
>
> 3) remove ng interface (e.g. stopping mpd). Check that in the list
> ifnet.tqh_first.if_groups->tqh_first.ifgl_group.ifg_members we still have
> the reference to already removed interface:
>
> (kgdb) p
> *ifnet.tqh_first.if_groups->tqh_first.ifgl_group.ifg_members.tqh_first.ifgm
>_next.tqe_next.ifgm_next.tqe_next.ifgm_next.tqe_next.ifgm_ifp $2 = {if_softc
> = 0xdeadc0de, if_l2com = 0xdeadc0de, if_link = {tqe_next = 0xdeadc0de,
> tqe_prev = 0xdeadc0de}, if_xname = "ÞÀÞÞÀÞÞÀÞÞÀÞ", if_dname =
> 0xdeadc0de , if_dunit =
> -559038242, if_addrhead = {tqh_first = 0xdeadc0de, tqh_last = 0xdeadc0de},
> if_klist = {kl_list = { slh_first = 0xdeadc0de}, kl_lock = 0xdeadc0de,
> kl_unlock = 0xdeadc0de, kl_locked = 0xdeadc0de,
>
> If you repeat this process many times you will have the long least of
> invalid ifgm_ifp references.
>
> pf traverses the list
> ifnet.tqh_first.if_groups->tqh_first.ifgl_group.ifg_members in
> pfi_table_update and calls pfi_instance_add() with nonvalid ifgm_ifp
> argument and the system panics trying to access invalid memory.
>
> I don't know if this correct solution but adding if_delgroup(ifp, IFG_ALL)
> to sys/net/if.c:if_detach() fixes the problem for me.
>
> --- sys/net/if.c.orig 2009-04-01 10:53:55.0 +0300
> +++ sys/net/if.c2009-04-10 12:38:14.0 +0300
> @@ -846,6 +846,7 @@ if_detach(struct ifnet *ifp)
> mtx_destroy(&ifp->if_snd.ifq_mtx);
> IF_AFDATA_DESTROY(ifp);
> splx(s);
> + if_delgroup(ifp, IFG_ALL);
> }
>
> /*
Good catch! Thank you very much. I'll commit your fix shortly after some
testing and will see that we can get it into 7.2
--
/"\ Best regards, | mla...@freebsd.org
\ / Max Laier | ICQ #67774661
X http://pf4freebsd.love2party.net/ | mla...@efnet
/ \ ASCII Ribbon Campaign | Against HTML Mail and News
___
freebsd-pf@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/130977: [netgraph][pf] kernel panic trap 12 on user connect to VPN server
The following reply was made to PR kern/130977; it has been noted by GNATS.
From: Mikolaj Golub
To: bug-follo...@freebsd.org,darki...@gmail.com
Cc:
Subject: Re: kern/130977: [netgraph][pf] kernel panic trap 12 on user connect
to VPN server
Date: Fri, 10 Apr 2009 14:42:59 +0300
The problem here (as in kern/131310 and may be in some other reports) is that
net/if.c:if_attach() when attaching interface adds it to default group ALL
calling if_addgroup(ifp, IFG_ALL). But when interface is removed (in this case
ng, but the same thing occurs for other interfaces too, e.g. I checked it for
tap) the reference to it does not removed from ifgl_group.ifg_members list.
The simple test can be used to confirm this:
1) add interface (e.g. starting mpd);
2) run kgdb and find reference to ng interface in the list
ifnet.tqh_first.if_groups->tqh_first.ifgl_group.ifg_members
E.g. in my case it is:
(kgdb) p
*ifnet.tqh_first.if_groups->tqh_first.ifgl_group.ifg_members.tqh_first.ifgm_next.tqe_next.ifgm_next.tqe_next.ifgm_next.tqe_next.ifgm_ifp
$1 = {if_softc = 0xc4e180c0, if_l2com = 0x0, if_link = {tqe_next = 0x0,
tqe_prev = 0xc4264808},
if_xname = "ng0", '\0' , if_dname = 0xc4bd60d9 "ng",
if_dunit = 0, if_addrhead = {
tqh_first = 0xc4ba4e00, tqh_last = 0xc4ba4e60}, if_klist = {kl_list =
{slh_first = 0x0},
kl_lock = 0xc07abb00 , kl_unlock = 0xc07abb30
,
...
3) remove ng interface (e.g. stopping mpd). Check that in the list
ifnet.tqh_first.if_groups->tqh_first.ifgl_group.ifg_members we still have the
reference
to already removed interface:
(kgdb) p
*ifnet.tqh_first.if_groups->tqh_first.ifgl_group.ifg_members.tqh_first.ifgm_next.tqe_next.ifgm_next.tqe_next.ifgm_next.tqe_next.ifgm_ifp
$2 = {if_softc = 0xdeadc0de, if_l2com = 0xdeadc0de, if_link = {tqe_next =
0xdeadc0de, tqe_prev = 0xdeadc0de},
if_xname = "ÞÀÞÞÀÞÞÀÞÞÀÞ", if_dname = 0xdeadc0de ,
if_dunit = -559038242, if_addrhead = {tqh_first = 0xdeadc0de, tqh_last =
0xdeadc0de}, if_klist = {kl_list = {
slh_first = 0xdeadc0de}, kl_lock = 0xdeadc0de, kl_unlock = 0xdeadc0de,
kl_locked = 0xdeadc0de,
If you repeat this process many times you will have the long least of invalid
ifgm_ifp references.
pf traverses the list
ifnet.tqh_first.if_groups->tqh_first.ifgl_group.ifg_members in
pfi_table_update and calls pfi_instance_add() with nonvalid ifgm_ifp argument
and the system panics trying to access invalid memory.
I don't know if this correct solution but adding if_delgroup(ifp, IFG_ALL) to
sys/net/if.c:if_detach() fixes the problem for me.
--- sys/net/if.c.orig 2009-04-01 10:53:55.0 +0300
+++ sys/net/if.c2009-04-10 12:38:14.0 +0300
@@ -846,6 +846,7 @@ if_detach(struct ifnet *ifp)
mtx_destroy(&ifp->if_snd.ifq_mtx);
IF_AFDATA_DESTROY(ifp);
splx(s);
+ if_delgroup(ifp, IFG_ALL);
}
/*
--
Mikolaj Golub
___
freebsd-pf@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-pf
To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/130977: [netgraph][pf] kernel panic trap 12 on user connect to VPN server
The following reply was made to PR kern/130977; it has been noted by GNATS. From: Oleg S To: bug-follo...@freebsd.org, darki...@gmail.com Cc: Subject: Re: kern/130977: [netgraph][pf] kernel panic trap 12 on user connect to VPN server Date: Sun, 15 Feb 2009 22:25:05 +0200 More detailed: in pf firewall should be rule like: pass in quick proto tcp from any to (self) port 25 flags S/SA keep state e.g. system crash in case in firewall present '(self)' macro ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: misc/130977: [netgraph][pf] kernel panic trap 12 on user connect to VPN server
Synopsis: [netgraph][pf] kernel panic trap 12 on user connect to VPN server Responsible-Changed-From-To: freebsd-bugs->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Sun Jan 25 20:33:47 UTC 2009 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=130977 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "freebsd-pf-unsubscr...@freebsd.org"
Re: kern/124364: [pf] [panic] Kernel panic with pf + bridge
Old Synopsis: Kernel panic with pf + bridge New Synopsis: [pf] [panic] Kernel panic with pf + bridge Responsible-Changed-From-To: freebsd-i386->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Sun Jun 8 21:08:36 UTC 2008 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=124364 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kern/117827: [pf] [panic] kernel panic with pf and ng
Synopsis: [pf] [panic] kernel panic with pf and ng State-Changed-From-To: feedback->closed State-Changed-By: mlaier State-Changed-When: Sat Apr 12 20:06:42 UTC 2008 State-Changed-Why: Commited to RELENG_6 and _7. Thanks for testing. http://www.freebsd.org/cgi/query-pr.cgi?pr=117827 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kern/117827: [pf] [panic] kernel panic with pf and ng
The following reply was made to PR kern/117827; it has been noted by GNATS. From: Max Laier <[EMAIL PROTECTED]> To: [EMAIL PROTECTED], [EMAIL PROTECTED] Cc: Subject: Re: kern/117827: [pf] [panic] kernel panic with pf and ng Date: Sat, 29 Mar 2008 01:56:36 +0100 Here are MFC patches for RELENG_6 and RELENG_7, please test and report back, thanks! http://people.freebsd.org/%7Emlaier/pf.dyn_altq.R6.diff http://people.freebsd.org/%7Emlaier/pf.dyn_altq.R7.diff -- Max ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kern/117827: [pf] [panic] kernel panic with pf and ng
Synopsis: [pf] [panic] kernel panic with pf and ng State-Changed-From-To: open->feedback State-Changed-By: mlaier State-Changed-When: Sat Mar 29 01:00:24 UTC 2008 State-Changed-Why: MFC patches need testing, thanks. http://www.freebsd.org/cgi/query-pr.cgi?pr=117827 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kern/117827: [pf] kernel panic with pf and ng
The following reply was made to PR kern/117827; it has been noted by GNATS. From: Florian Smeets <[EMAIL PROTECTED]> To: [EMAIL PROTECTED], [EMAIL PROTECTED] Cc: Subject: Re: kern/117827: [pf] kernel panic with pf and ng Date: Sun, 30 Dec 2007 16:27:36 +0100 Hi, Max has submitted a patch for this here: http://docs.freebsd.org/cgi/mid.cgi?200711272144.52511.max For me it works successfully on 7-BETA4 and -CURRENT. Cheers, Florian ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kern/117827: [pf] kernel panic with pf and ng
Synopsis: [pf] kernel panic with pf and ng State-Changed-From-To: feedback->open State-Changed-By: kris State-Changed-When: Sun Dec 30 11:25:42 UTC 2007 State-Changed-Why: Feedback received, thanks http://www.freebsd.org/cgi/query-pr.cgi?pr=117827 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kern/117827: [pf] kernel panic with pf and ng
Synopsis: [pf] kernel panic with pf and ng State-Changed-From-To: open->feedback State-Changed-By: kris State-Changed-When: Tue Dec 25 13:55:43 UTC 2007 State-Changed-Why: Awaiting user response with necessary debugging information http://www.freebsd.org/cgi/query-pr.cgi?pr=117827 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kern/117827: Kernel Panic
The following reply was made to PR kern/117827; it has been noted by GNATS. From: Remko Lodder <[EMAIL PROTECTED]> To: Dmitry Rybin <[EMAIL PROTECTED]> Cc: [EMAIL PROTECTED] Subject: Re: kern/117827: Kernel Panic Date: Mon, 05 Nov 2007 18:46:02 +0100 Dmitry Rybin wrote: > FreeBSD alpha.kirgudu.loc 7.0-BETA2 FreeBSD 7.0-BETA2 #0: Mon Nov 5 > 00:34:07 MSK 2007 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/ALPHA7 i386 > >> Description: > If I use pf packet filter and use ng interface, then I am destroy interface > and create it -> pfctl -f /etc/pf.conf -> kernel panic >> How-To-Repeat: > 1. > enable pf. > pf.conf > vpn_if="ng0" > > pass out quick on $vpn_if keep state > === > 2. > mpd4, create connection. > === > 3. > pfctl -f /etc/pf.conf > === > 4. > Stop and then start mpd4. > === > 5. > pfctl -f /etc/pf.conf > > KERNEL PANIC >> Fix: > I don't know > Hello, Please obtain a kernel dump and backtrace and post them in the ticket (reply-all to this message). You can read how you do that through: http://www.freebsd.org/doc/en/developers-handbook/kerneldebug.html The current information is too narrow to be useful for resolving the problem. Thanks, remko -- Kind regards, Remko Lodder ** [EMAIL PROTECTED] FreeBSD** [EMAIL PROTECTED] /* Quis custodiet ipsos custodes */ ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kern/117827: [pf] kernel panic with pf and ng
Old Synopsis: Kernel Panic New Synopsis: [pf] kernel panic with pf and ng Responsible-Changed-From-To: freebsd-bugs->freebsd-pf Responsible-Changed-By: linimon Responsible-Changed-When: Mon Nov 5 02:53:10 UTC 2007 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=117827 ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Kernel panic with PF
Daniel Hartmeier wrote: > On Fri, Jul 21, 2006 at 10:57:28AM +0200, Michal Mertl wrote: > > > The proxy in fact runs in parallel (according to "pfctl -s info" it did > > about 50 inserts and removal in the state table per second - some 10Mbit > > of traffic, probably mostly HTTP) and it is quite possible that your > > explanation is correct. I will forward your suspicion to the vendor. > > This functionality of the software (using PF with anchors) is quite new > > - they used different mechanisms in previous versions so it may well > > have some bugs. > > Anchors were introduced for this purpose, i.e. splitting the ruleset > into separate pieces, over each of which a single process can have > authority, so different processes don't stomp on each other's toes with > ruleset modifications. They (the Kernun authors) run multiple processes for each proxy. Originally they used slightly modified Apached core for their proxies I believe. Thus there are probably more processes using the same anchor. I don't really understand what they do inside - I would think that when there are no traffic blocking rules, there's no point in doing anything with PF except initial setting of the rdr rule to the proxy. > Ask them if they really need to still use DIOCCHANGERULE, as the idea > with anchors is generally to only operate within one anchor, and usually > flush or replace the (smaller) ruleset within. > > Each anchor has its own ticket, so if you're seeing ticket mismatches, > that means there are concurrent operations on the same anchor, even. I see. It would be better if they were part of this communication because I don't know the internals (although I have the source code). I have problems reaching them at the moment though. > Daniel > ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Kernel panic with PF
On Fri, Jul 21, 2006 at 10:57:28AM +0200, Michal Mertl wrote: > The proxy in fact runs in parallel (according to "pfctl -s info" it did > about 50 inserts and removal in the state table per second - some 10Mbit > of traffic, probably mostly HTTP) and it is quite possible that your > explanation is correct. I will forward your suspicion to the vendor. > This functionality of the software (using PF with anchors) is quite new > - they used different mechanisms in previous versions so it may well > have some bugs. Anchors were introduced for this purpose, i.e. splitting the ruleset into separate pieces, over each of which a single process can have authority, so different processes don't stomp on each other's toes with ruleset modifications. Ask them if they really need to still use DIOCCHANGERULE, as the idea with anchors is generally to only operate within one anchor, and usually flush or replace the (smaller) ruleset within. Each anchor has its own ticket, so if you're seeing ticket mismatches, that means there are concurrent operations on the same anchor, even. Daniel ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Kernel panic with PF
Daniel Hartmeier wrote: > On Fri, Jul 21, 2006 at 02:05:45AM +0200, Max Laier wrote: > > > Which proxies are you using? The "pool_ticket: 1429 != 1430" messages you > > quote below indicate a synchronization problem within the app talking to pf > > via ioctl's. Tickets are used to ensure atomic commits for operations that > > require more than one ioctl. If your proxy app runs in parallel it might > > screw up the internal state and thus leave it undefined afterwards. I give > > you that this shouldn't cause a kernel problem, but if we could fix the app > > we can probably find the right sanity check more easily. > > This looks like a bug in pf_ioctl.c pfioctl() DIOCCHANGERULE > > if (newrule->action == PF_NAT) || > (newrule->action == PF_RDR) || > (newrule->action == PF_BINAT) || > (newrule->rt > PF_FASTROUTE)) && > - !pcr->anchor[0])) && > + !newrule->anchor)) && > (TAILQ_FIRST(&newrule->rpool.list) == NULL)) > error = EINVAL; > > i.e. the pool must not be empty for routing and translation rules, > except for translation rules that are actually anchor _calls_. > > The confusion is between translation rules within anchors > (pcr->anchor[0] != '\0') and calls to anchors' translation rules > (rule->anchor != NULL). > > If the proxy is using DIOCCHANGERULE (it must be the proxy, pfctl isn't > using it at all), AND is trying to add/update a rule that requires at > least one replacement address but contains an empty list, then this > would cause the panic seen when that rule later matches a packet. > > This needs fixing in OpenBSD as well. > > Michal, can you please confirm that the patch above fixes the panic? > The proxy will still misbehave and cause the log messages (one more > EINVAL in this case ;), but the kernel shouldn't crash anymore. I am afraid I can't test it at the moment. I am going to get one of the machines to my lab and will experiment with it there. I am afraid I will have problems generating enough traffic for the problem to appear but I will try. > Thanks for the excellent bug report! Thank you. I don't think is was that good as I now see that you had to guess there are anchors used. The rules look like this (except the rules seen by 'pfctl -s nat' they are generated by the proxies when they start): fw1#pfctl -s rule fw1#pfctl -s nat nat-anchor "/kernun/*" all rdr-anchor "/kernun/*" all fw1#pfctl -s Anchors -v kernun kernun/4026 kernun/4039 kernun/4088 kernun/4112 kernun/4134 kernun/4164 kernun/4197 kernun/4257 kernun/4296 kernun/4338 kernun/4383 kernun/4431 kernun/4482 kernun/4590 kernun/4649 fw1# pfctl -a kernun/4039 -s nat rdr on em0 inet proto tcp from any to any port = http label "HTTP" -> 127.0.0.1 When the system was under load I saw ~5000 states in 'pfctl -s state'. Thank you again. I will let you know when I get a chance to test your patch and or find out anything new. Michal ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Kernel panic with PF
Max Laier píše v pá 21. 07. 2006 v 02:05 +0200: > [CC'ing -pf] > > On Thursday 20 July 2006 17:53, Michal Mertl wrote: > > Hello, > > > > I am deploying FreeBSD based application proxies' based firewall > > (www.kernun.com, but not much English there) and am having frequent > > panics of RELENG_6_1 under load. The server has IP forwarding disabled. > > > > I've got two machines in a carp cluster and the transparent proxies use > > PF to get the data. > > Which proxies are you using? The "pool_ticket: 1429 != 1430" messages you > quote below indicate a synchronization problem within the app talking to pf > via ioctl's. Tickets are used to ensure atomic commits for operations that > require more than one ioctl. If your proxy app runs in parallel it might > screw up the internal state and thus leave it undefined afterwards. I give > you that this shouldn't cause a kernel problem, but if we could fix the app > we can probably find the right sanity check more easily. The proxy in fact runs in parallel (according to "pfctl -s info" it did about 50 inserts and removal in the state table per second - some 10Mbit of traffic, probably mostly HTTP) and it is quite possible that your explanation is correct. I will forward your suspicion to the vendor. This functionality of the software (using PF with anchors) is quite new - they used different mechanisms in previous versions so it may well have some bugs. Thanks Michal ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Kernel panic with PF
On Fri, Jul 21, 2006 at 02:05:45AM +0200, Max Laier wrote: > Which proxies are you using? The "pool_ticket: 1429 != 1430" messages you > quote below indicate a synchronization problem within the app talking to pf > via ioctl's. Tickets are used to ensure atomic commits for operations that > require more than one ioctl. If your proxy app runs in parallel it might > screw up the internal state and thus leave it undefined afterwards. I give > you that this shouldn't cause a kernel problem, but if we could fix the app > we can probably find the right sanity check more easily. This looks like a bug in pf_ioctl.c pfioctl() DIOCCHANGERULE if (newrule->action == PF_NAT) || (newrule->action == PF_RDR) || (newrule->action == PF_BINAT) || (newrule->rt > PF_FASTROUTE)) && - !pcr->anchor[0])) && + !newrule->anchor)) && (TAILQ_FIRST(&newrule->rpool.list) == NULL)) error = EINVAL; i.e. the pool must not be empty for routing and translation rules, except for translation rules that are actually anchor _calls_. The confusion is between translation rules within anchors (pcr->anchor[0] != '\0') and calls to anchors' translation rules (rule->anchor != NULL). If the proxy is using DIOCCHANGERULE (it must be the proxy, pfctl isn't using it at all), AND is trying to add/update a rule that requires at least one replacement address but contains an empty list, then this would cause the panic seen when that rule later matches a packet. This needs fixing in OpenBSD as well. Michal, can you please confirm that the patch above fixes the panic? The proxy will still misbehave and cause the log messages (one more EINVAL in this case ;), but the kernel shouldn't crash anymore. Thanks for the excellent bug report! Daniel ___ freebsd-pf@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-pf To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Kernel panic with PF
[CC'ing -pf]
On Thursday 20 July 2006 17:53, Michal Mertl wrote:
> Hello,
>
> I am deploying FreeBSD based application proxies' based firewall
> (www.kernun.com, but not much English there) and am having frequent
> panics of RELENG_6_1 under load. The server has IP forwarding disabled.
>
> I've got two machines in a carp cluster and the transparent proxies use
> PF to get the data.
Which proxies are you using? The "pool_ticket: 1429 != 1430" messages you
quote below indicate a synchronization problem within the app talking to pf
via ioctl's. Tickets are used to ensure atomic commits for operations that
require more than one ioctl. If your proxy app runs in parallel it might
screw up the internal state and thus leave it undefined afterwards. I give
you that this shouldn't cause a kernel problem, but if we could fix the app
we can probably find the right sanity check more easily.
> I don't know much about kernel internals and PF but from the following
> backtrace I understand that the crash happens because rpool->cur on line
> 2158 in src/sys/contrib/pf/net/pf.c is NULL and is dereferenced. It
> probably shouldn't happen yet it does.
>
> The machines are SMP and were running SMP kernel. The only places where
> pool.cur (or pool->cur) is assigned to are in pf_ioctl.c. It seems there
> are some lock operations though so it is probably believed that the
> coder is properly locked.
>
> I have been running with kern.smp.disabled=1 for a moment before I put
> the old firewall in place and haven't seen the panic but the time was
> deffinitely too short to make me believe it fixes the issue. Can setting
> debug.mpsafenet to 0 possibly also help?
>
> I could probably bandaid this particular failure mode by returning
> failure instead of panicing but the bug is probably elsewhere.
>
> I've lost the debug kernel from which this backtrace is and can't
> therefore continue much :-(. Unfortunately so far I can only reproduce
> the problem in production and for obvious reasons I can't put it there.
>
> Fatal trap 12: page fault while in kernel mode
> cpuid = 0; apic id = 00
> fault virtual address = 0x28
> fault code = supervisor read, page not present
> instruction pointer = 0x8:0x801ab528
> stack pointer = 0x10:0xb1ade650
> frame pointer = 0x10:0xff004cc7cc30
> code segment= base 0x0, limit 0xf, type 0x1b
> = DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags= interrupt enabled, resume, IOPL = 0
> current process = 15 (swi1: net)
> trap number = 12
> panic: page fault
>
> #0 doadump () at pcpu.h:172
> #1 0x0004 in ?? ()
> #2 0x803d5137 in boot (howto=260)
> at ../../../kern/kern_shutdown.c:402
> #3 0x803d58a1 in panic (fmt=0xff007ba32000 "@\223{")
> at ../../../kern/kern_shutdown.c:558
> #4 0x80543b3f in trap_fatal (frame=0xff007ba32000,
> eva=18446742976272241472) at ../../../amd64/amd64/trap.c:660
> #5 0x80543e5f in trap_pfault (frame=0xb1ade5a0,
> usermode=0)
> at ../../../amd64/amd64/trap.c:573
> #6 0x80544113 in trap (frame=
> {tf_rdi = 2, tf_rsi = -1098223465792, tf_rdx = -1098439497700,
> tf_rcx = -1
> 314002464, tf_r8 = 0, tf_r9 = -1314002776, tf_rax = 0, tf_rbx = 0,
> tf_rbp = -109
> 8223465424, tf_r10 = 1, tf_r11 = 257, tf_r12 = -1098439497700, tf_r13 =
> -1314002
> 776, tf_r14 = 2, tf_r15 = -1314002464, tf_trapno = 12, tf_addr = 40,
> tf_flags =
> 216171684640539392, tf_err = 0, tf_rip = -214576, tf_cs = 8,
> tf_rflags = 661
> 18, tf_rsp = -1314003360, tf_ss = 16})
> at ../../../amd64/amd64/trap.c:352
> #7 0x8052feab in calltrap ()
> at ../../../amd64/amd64/exception.S:168
> #8 0x801ab528 in pf_map_addr (af=2 '\002',
> r=0xff004cc7cac0,
> saddr=0xff003fe7681c, naddr=0xb1ade9e0, init_addr=0x0,
> sn=0xb1ade8a8) at ../../../contrib/pf/net/pf.c:2163
> #9 0x801acab6 in pf_get_translation (pd=0xb1ade9c0,
> m=0xff0042ede900, off=20, direction=1, kif=0xff007b038a00,
> sn=0xb1ade8a8, saddr=0xff003fe7681c, sport=0,
> daddr=0xff003fe76820, dport=50881, naddr=0xb1ade9e0,
> nport=0xb1ade8b6) at ../../../contrib/pf/net/pf.c:2618
> #10 0x801b315b in pf_test_tcp (rm=0xb1ade960,
> sm=0xb1ade950, direction=1, kif=0xff007b038a00,
> m=0xff0042ede900, off=20, h=0xff003fe76810,
> pd=0xb1ade9c0, am=0xb1ade968,
> rsm=0xb1ade970,
> ifq=0x2, inp=0x0) at ../../../contrib/pf/net/pf.c:3013
> #11 0x801b5694 in pf_test (dir=1, ifp=0xffbee800,
> m0=0xb1adeaa0, eh=0xb1ade97e, inp=0x0)
> at ../../../contrib/pf/net/pf.c:6449
> #12 0x801bafb2 in pf_check_in (arg=0x2, m=0xb1adeaa0,
> ifp=0xff004cc7cac0, dir=-1314002464, inp=0xb1ade9e0)
>
