https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=284606
--- Comment #10 from [email protected] --- Seen exactly the same panic with net/amnezia-kmod instead of if_wg, it is based on if_wg, svery possible that root case is the same. In may case - caused by following ipfw rule: ipfw add tcp-setmss 1380 tcp from any to any out xmit wg0 tcpflags syn I have a kernel cash dump: without debug it is very similar to this one: Fatal trap 12: page fault while in kernel mode ... --- trap 0xc, rip = 0xffffffff80cd9ac4, rsp = 0xfffffe0159ac9c20, rbp = 0xfffffe0159ac9cb0 --- ip_tryforward() at ip_tryforward+0x274/frame 0xfffffe0159ac9cb0 ip_input() at ip_input+0x321/frame 0xfffffe0159ac9d10 netisr_dispatch_src() at netisr_dispatch_src+0x9f/frame 0xfffffe0159ac9d60 wg_deliver_in() at wg_deliver_in+0x3ad/frame 0xfffffe0159ac9e40 gtaskqueue_run_locked() at gtaskqueue_run_locked+0x14e/frame 0xfffffe0159ac9ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc2/frame 0xfffffe0159ac9ef0 ... (kgdb) fr 8 #8 0xffffffff80cd9ac4 in ip_tryforward (m=0x0) at /usr/src/sys/netinet/ip_fastfwd.c:416 416 ip = mtod(m, struct ip *); (kgdb) p m $1 = (struct mbuf *) 0x0 (kgdb) fr 9 #9 0xffffffff80cdc251 in ip_input (m=0xfffff8020f13a500) at /usr/src/sys/netinet/ip_input.c:587 587 if ((m = ip_tryforward(m)) == NULL) (kgdb) p m $2 = (struct mbuf *) 0xfffff8020f13a500 (kgdb) so, that means that pfil_mbuf_fwd() returned with m == NULL and PFIL_PASS here: https://github.com/freebsd/freebsd-src/blob/main/sys/netinet/ip_fastfwd.c#L407 which is not expected then I have reproduced it with debug kernel - it died earlier, in firewall: Unread portion of the kernel message buffer: panic: ipfw_check_packet: m0 is NULL cpuid = 9 time = 1761568018 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0164266950 vpanic() at vpanic+0x161/frame 0xfffffe0164266a80 panic() at panic+0x43/frame 0xfffffe0164266ae0 ipfw_check_packet() at ipfw_check_packet+0x6ba/frame 0xfffffe0164266bd0 pfil_mbuf_out() at pfil_mbuf_out+0x58/frame 0xfffffe0164266c00 ip_tryforward() at ip_tryforward+0x2a5/frame 0xfffffe0164266ca0 ip_input() at ip_input+0x3af/frame 0xfffffe0164266d00 netisr_dispatch_src() at netisr_dispatch_src+0xb4/frame 0xfffffe0164266d60 wg_deliver_in() at wg_deliver_in+0x3ad/frame 0xfffffe0164266e40 gtaskqueue_run_locked() at gtaskqueue_run_locked+0x18e/frame 0xfffffe0164266ec0 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xd3/frame 0xfffffe0164266ef0 fork_exit() at fork_exit+0x82/frame 0xfffffe0164266f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0164266f30 --- trap 0xdeadc0de, rip = 0xdeadc0dedeadc0de, rsp = 0xdeadc0dedeadc0de, rbp = 0xdeadc0dedeadc0de --- (kgdb) bt #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 #1 doadump (textdump=textdump@entry=1) at /usr/src/sys/kern/kern_shutdown.c:405 #2 0xffffffff80b0d420 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:523 #3 0xffffffff80b0d939 in vpanic (fmt=0xffffffff835056aa "%s: m0 is NULL", ap=ap@entry=0xfffffe0164266ac0) at /usr/src/sys/kern/kern_shutdown.c:967 #4 0xffffffff80b0d6c3 in panic (fmt=<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:891 #5 0xffffffff834e666a in ipfwlog_clone_create (ifc=<optimized out>, unit=0, params=<optimized out>) at /usr/src/sys/netpfil/ipfw/ip_fw_bpf.c:134 #6 0xffffffff80c71d18 in pfil_mbuf_common (pch=<optimized out>, m=0x2, m@entry=0xfffffe0164266b98, ifp=0xfffffe0164266c38, flags=131072, inp=0xfffff80004f2e000, inp@entry=0x0) at /usr/src/sys/net/pfil.c:212 #7 pfil_mbuf_out (head=<optimized out>, m=0x2, m@entry=0xfffffe0164266c38, ifp=0xfffffe0164266c38, inp=0xfffff80004f2e000, inp@entry=0x0) at /usr/src/sys/net/pfil.c:233 #8 0xffffffff80cf1e75 in ip_tryforward (m=0x0) at /usr/src/sys/netinet/ip_fastfwd.c:409 #9 0xffffffff80cf4dff in ip_input (m=0xfffff8020c324700) at /usr/src/sys/netinet/ip_input.c:587 #10 0xffffffff80c6dea4 in netisr_dispatch_src (proto=1, source=0, m=0xfffff8020c324700) at /usr/src/sys/net/netisr.c:1152 #11 0xffffffff8352663d in wg_deliver_in () from /boot/modules/if_amn.ko #12 0xffffffff80b5c7ce in gtaskqueue_run_locked (queue=0x2, queue@entry=0xfffff8021de61900) at /usr/src/sys/kern/subr_gtaskqueue.c:369 #13 0xffffffff80b5c503 in gtaskqueue_thread_loop (arg=arg@entry=0xfffffe016409f0e0) at /usr/src/sys/kern/subr_gtaskqueue.c:545 #14 0xffffffff80ac0022 in fork_exit (callout=0xffffffff80b5c430 <gtaskqueue_thread_loop>, arg=0xfffffe016409f0e0, frame=0xfffffe0164266f40) at /usr/src/sys/kern/kern_fork.c:1153 #15 <signal handler called> #16 0xdeadc0dedeadc0de in ?? () Backtrace stopped: Cannot access memory at address 0xdeadc0dedeadc0de (kgdb) bt #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 #1 doadump (textdump=textdump@entry=1) at /usr/src/sys/kern/kern_shutdown.c:405 #2 0xffffffff80b0d420 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:523 #3 0xffffffff80b0d939 in vpanic (fmt=0xffffffff835056aa "%s: m0 is NULL", ap=ap@entry=0xfffffe0164266ac0) at /usr/src/sys/kern/kern_shutdown.c:967 #4 0xffffffff80b0d6c3 in panic (fmt=<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:891 #5 0xffffffff834e666a in ipfwlog_clone_create (ifc=<optimized out>, unit=0, params=<optimized out>) at /usr/src/sys/netpfil/ipfw/ip_fw_bpf.c:134 #6 0xffffffff80c71d18 in pfil_mbuf_common (pch=<optimized out>, m=0x2, m@entry=0xfffffe0164266b98, ifp=0xfffffe0164266c38, flags=131072, inp=0xfffff80004f2e000, inp@entry=0x0) at /usr/src/sys/net/pfil.c:212 #7 pfil_mbuf_out (head=<optimized out>, m=0x2, m@entry=0xfffffe0164266c38, ifp=0xfffffe0164266c38, inp=0xfffff80004f2e000, inp@entry=0x0) at /usr/src/sys/net/pfil.c:233 #8 0xffffffff80cf1e75 in ip_tryforward (m=0x0) at /usr/src/sys/netinet/ip_fastfwd.c:409 #9 0xffffffff80cf4dff in ip_input (m=0xfffff8020c324700) at /usr/src/sys/netinet/ip_input.c:587 #10 0xffffffff80c6dea4 in netisr_dispatch_src (proto=1, source=0, m=0xfffff8020c324700) at /usr/src/sys/net/netisr.c:1152 #11 0xffffffff8352663d in wg_deliver_in () from /boot/modules/if_amn.ko #12 0xffffffff80b5c7ce in gtaskqueue_run_locked (queue=0x2, queue@entry=0xfffff8021de61900) at /usr/src/sys/kern/subr_gtaskqueue.c:369 #13 0xffffffff80b5c503 in gtaskqueue_thread_loop (arg=arg@entry=0xfffffe016409f0e0) at /usr/src/sys/kern/subr_gtaskqueue.c:545 #14 0xffffffff80ac0022 in fork_exit (callout=0xffffffff80b5c430 <gtaskqueue_thread_loop>, arg=0xfffffe016409f0e0, frame=0xfffffe0164266f40) at /usr/src/sys/kern/kern_fork.c:1153 #15 <signal handler called> #16 0xdeadc0dedeadc0de in ?? () Backtrace stopped: Cannot access memory at address 0xdeadc0dedeadc0de (kgdb) fr 9 #9 0xffffffff80cf4dff in ip_input (m=0xfffff8020c324700) at /usr/src/sys/netinet/ip_input.c:587 587 if ((m = ip_tryforward(m)) == NULL) (kgdb) p m $1 = (struct mbuf *) 0xfffff8020c324700 (kgdb) p *m $2 = {{m_next = 0xdeadc0dedeadc0de, m_slist = {sle_next = 0xdeadc0dedeadc0de}, m_stailq = {stqe_next = 0xdeadc0dedeadc0de}}, {m_nextpkt = 0xdeadc0dedeadc0de, m_slistpkt = {sle_next = 0xdeadc0dedeadc0de}, m_stailqpkt = {stqe_next = 0xdeadc0dedeadc0de}}, m_data = 0xdeadc0dedeadc0de <error: Cannot access memory at address 0xdeadc0dedeadc0de>, m_len = -559038242, m_type = 222, m_flags = 14593472, {{{m_pkthdr = {{ snd_tag = 0xdeadc0dedeadc0de, rcvif = 0xdeadc0dedeadc0de, {rcvidx = 49374, rcvgen = 57005}}, {leaf_rcvif = 0xdeadc0dedeadc0de, {leaf_rcvidx = 49374, leaf_rcvgen = 57005}}, tags = {slh_first = 0xdeadc0dedeadc0de}, len = -559038242, flowid = 3735929054, csum_flags = 3735929054, fibnum = 49374, numa_domain = 173 '\255', rsstype = 222 '\336', {rcv_tstmp = 16045693110842147038, {l2hlen = 222 '\336', l3hlen = 192 '\300', l4hlen = 173 '\255', l5hlen = 222 '\336', inner_l2hlen = 222 '\336', inner_l3hlen = 192 '\300', inner_l4hlen = 173 '\255', inner_l5hlen = 222 '\336'}}, PH_per = {eight = "\336\300\255\336\336\300\255", <incomplete sequence \336>, sixteen = {49374, 57005, 49374, 57005}, thirtytwo = {3735929054, 3735929054}, sixtyfour = {16045693110842147038}, unintptr = {16045693110842147038}, ptr = 0xdeadc0dedeadc0de}, {PH_loc = {eight = "\336\300\255\336\336\300\255", <incomplete sequence \336>, sixteen = {49374, 57005, 49374, 57005}, thirtytwo = {3735929054, 3735929054}, sixtyfour = {16045693110842147038}, unintptr = {16045693110842147038}, ptr = 0xdeadc0dedeadc0de}, memlen = 3735929054}}, {m_epg_npgs = 222 '\336', m_epg_nrdy = 192 '\300', m_epg_hdrlen = 173 '\255', m_epg_trllen = 222 '\336', m_epg_1st_off = 49374, m_epg_last_len = 57005, m_epg_flags = 222 '\336', m_epg_record_type = 192 '\300', __spare = "\255", <incomplete sequence \336>, m_epg_enc_cnt = -559038242, m_epg_tls = 0xdeadc0dedeadc0de, m_epg_so = 0xdeadc0dedeadc0de, m_epg_seqno = 16045693110842147038, m_epg_stailq = {stqe_next = 0xdeadc0dedeadc0de}}}, {m_ext = {{ext_count = 3735929054, ext_cnt = 0xdeadc0dedeadc0de}, ext_size = 3735929054, ext_type = 222, ext_flags = 14593472, {{ext_buf = 0xdeadc0dedeadc0de <error: Cannot access memory at address 0xdeadc0dedeadc0de>, ext_arg2 = 0xdeadc0dedeadc0de}, {extpg_pa = {16045693110842147038, 16045693110842147038, 16045693110842147038, 16045693110842147038, 16045693110842147038}, extpg_trail = "\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255", <incomplete sequence \336>, extpg_hdr = "\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255"}}, ext_free = 0xdeadc0dedeadc0de, ext_arg1 = 0xdeadc0dedeadc0de}, m_pktdat = 0xfffff8020c324760 "\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255", <incomplete sequence \336>...}}, m_dat = 0xfffff8020c324720 "\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255", <incomplete sequence \336>...}} (kgdb) fr 8 #8 0xffffffff80cf1e75 in ip_tryforward (m=0x0) at /usr/src/sys/netinet/ip_fastfwd.c:409 warning: Source file is more recent than executable. 409 if (pfil_mbuf_out(V_inet_pfil_head, &m, nh->nh_ifp, (kgdb) p m $3 = (struct mbuf *) 0x0 So, sounds like if firewall updates packet something get broken. Additional considiration, on other system I have pf firewall updating MSS and it does not breaks. But I can't say for sure, as far as I have no exact reproduction, it just fires few times a day with some wg clients. -- You are receiving this mail because: You are the assignee for the bug.
