https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=275798
Bug ID: 275798 Summary: panic: sackhint bytes rtx >= 0 Product: Base System Version: CURRENT Hardware: Any OS: Any Status: New Severity: Affects Some People Priority: --- Component: kern Assignee: b...@freebsd.org Reporter: ddan...@nvidia.com During IPSEC regression, got Kernel Panic when ran traffic over tunnel mode and did ipsec restart in parallel Unread portion of the kernel message buffer: [8118.158276] panic: sackhint bytes rtx >= 0 [8118.169026] cpuid = 8 [8118.183306] time = 1702046494 [8118.197593] KDB: stack backtrace: [8118.212342] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01579543b0 [8118.227297] vpanic() at vpanic+0x132/frame 0xfffffe01579544e0 [8118.241552] panic() at panic+0x43/frame 0xfffffe0157954540 [8118.255964] tcp_sack_doack() at tcp_sack_doack+0x83c/frame 0xfffffe01579545d0 [8118.270235] tcp_do_segment() at tcp_do_segment+0x20f2/frame 0xfffffe01579546b0 [8118.284532] tcp_input_with_port() at tcp_input_with_port+0x1157/frame 0xfffffe0157954800 [8118.298806] tcp_input() at tcp_input+0xb/frame 0xfffffe0157954810 [8118.313084] ip_input() at ip_input+0x2ab/frame 0xfffffe0157954870 [8118.327340] netisr_dispatch_src() at netisr_dispatch_src+0xad/frame 0xfffffe01579548d0 [8118.341622] gif_input() at gif_input+0x241/frame 0xfffffe0157954920 [8118.355939] in_gif_input() at in_gif_input+0x74/frame 0xfffffe0157954960 [8118.370225] encap_input() at encap_input+0xd7/frame 0xfffffe01579549d0 [8118.384523] encap4_input() at encap4_input+0x26/frame 0xfffffe0157954a00 [8118.398803] ip_input() at ip_input+0x2ab/frame 0xfffffe0157954a60 [8118.413058] netisr_dispatch_src() at netisr_dispatch_src+0xad/frame 0xfffffe0157954ac0 [8118.427347] ether_demux() at ether_demux+0x17a/frame 0xfffffe0157954af0 [8118.441647] ether_nh_input() at ether_nh_input+0x39f/frame 0xfffffe0157954b40 [8118.455928] netisr_dispatch_src() at netisr_dispatch_src+0xad/frame 0xfffffe0157954ba0 [8118.470195] ether_input() at ether_input+0xd9/frame 0xfffffe0157954c00 [8118.484537] tcp_lro_flush_all() at tcp_lro_flush_all+0xdc/frame 0xfffffe0157954c50 [8118.498580] mlx5e_rx_cq_comp() at mlx5e_rx_cq_comp+0x1241/frame 0xfffffe0157954d70 [8118.512841] mlx5_cq_completion() at mlx5_cq_completion+0x74/frame 0xfffffe0157954dd0 [8118.527130] mlx5_eq_int() at mlx5_eq_int+0x28a/frame 0xfffffe0157954e30 [8118.541408] mlx5_msix_handler() at mlx5_msix_handler+0x15/frame 0xfffffe0157954e40 [8118.555698] lkpi_irq_handler() at lkpi_irq_handler+0x29/frame 0xfffffe0157954e60 [8118.570117] ithread_loop() at ithread_loop+0x266/frame 0xfffffe0157954ef0 [8118.584389] fork_exit() at fork_exit+0x82/frame 0xfffffe0157954f30 [8118.598289] fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0157954f30 [8118.611898] --- trap 0, rip = 0, rsp = 0, rbp = 0 --- [8118.626197] Uptime: 2h15m18s [8118.663701] Dumping 1731 out of 32701 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% __curthread () at /usr/kernel_git/sys/amd64/include/pcpu_aux.h:57 57 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct pcpu, (kgdb) #0 __curthread () at /usr/kernel_git/sys/amd64/include/pcpu_aux.h:57 #1 doadump (textdump=textdump@entry=1) at /usr/kernel_git/sys/kern/kern_shutdown.c:403 #2 0xffffffff80b3b390 in kern_reboot (howto=260) at /usr/kernel_git/sys/kern/kern_shutdown.c:521 #3 0xffffffff80b3b88f in vpanic (fmt=0xffffffff8114668e "sackhint bytes rtx >= 0", ap=ap@entry=0xfffffe0157954520) at /usr/kernel_git/sys/kern/kern_shutdown.c:965 #4 0xffffffff80b3b633 in panic (fmt=<unavailable>) at /usr/kernel_git/sys/kern/kern_shutdown.c:889 #5 0xffffffff80d3ba6c in tcp_sack_doack (tp=<optimized out>, to=0xfffffe01579545e8, th_ack=<optimized out>) at /usr/kernel_git/sys/netinet/tcp_sack.c:779 #6 0xffffffff80d2cbe2 in tcp_do_segment (tp=0xfffff8022bcf0000, tp@entry=<error reading variable: value is not available>, m=0xfffff80368fe5000, m@entry=<error reading variable: value is not available>, th=0xfffff80368fe5098, th@entry=<error reading variable: value is not available>, drop_hdrlen=64, drop_hdrlen@entry=<error reading variable: value is not available>, tlen=<optimized out>, tlen@entry=<error reading variable: value is not available>, iptos=<unavailable>, iptos@entry=<error reading variable: value is not available>) at /usr/kernel_git/sys/netinet/tcp_input.c:2501 #7 0xffffffff80d2a0a7 in tcp_input_with_port (mp=<optimized out>, offp=<optimized out>, proto=<optimized out>, port=port@entry=0) at /usr/kernel_git/sys/netinet/tcp_input.c:1160 #8 0xffffffff80d2aadb in tcp_input (mp=<unavailable>, offp=<unavailable>, proto=<unavailable>) at /usr/kernel_git/sys/netinet/tcp_input.c:1492 #9 0xffffffff80d1888b in ip_input (m=0x0, m@entry=<error reading variable: value is not available>) at /usr/kernel_git/sys/netinet/ip_input.c:861 #10 0xffffffff80c9393d in netisr_dispatch_src (proto=proto@entry=1, source=source@entry=0, m=0xfffff80368fe5000) at /usr/kernel_git/sys/net/netisr.c:1152 #11 0xffffffff80c93c7f in netisr_dispatch (proto=<unavailable>, proto@entry=1, m=<unavailable>) at /usr/kernel_git/sys/net/netisr.c:1243 #12 0xffffffff80c77da1 in gif_input (m=<unavailable>, m@entry=0xfffff80368fe5000, ifp=ifp@entry=0xfffff8000b26c800, proto=proto@entry=4, ecn=<optimized out>, ecn@entry=0 '\000') at /usr/kernel_git/sys/net/if_gif.c:587 #13 0xffffffff80d092c4 in in_gif_input (m=0xfffff80368fe5000, off=20, proto=4, arg=<optimized out>) at /usr/kernel_git/sys/netinet/in_gif.c:315 #14 0xffffffff80d15717 in encap_input (head=<optimized out>, m=0xfffff80368fe5000, off=20, proto=proto@entry=4) at /usr/kernel_git/sys/netinet/ip_encap.c:328 #15 0xffffffff80d15606 in encap4_input (mp=0xfffffe0157954a30, offp=0xfffffe0157954a2c, proto=4) at /usr/kernel_git/sys/netinet/ip_encap.c:370 #16 0xffffffff80d1888b in ip_input (m=0xfffff80368fe5000, m@entry=<error reading variable: value is not available>) at /usr/kernel_git/sys/netinet/ip_input.c:861 #17 0xffffffff80c9393d in netisr_dispatch_src (proto=proto@entry=1, source=source@entry=0, m=0xfffff80368fe5000) at /usr/kernel_git/sys/net/netisr.c:1152 #18 0xffffffff80c93c7f in netisr_dispatch (proto=<unavailable>, proto@entry=1, m=<unavailable>) at /usr/kernel_git/sys/net/netisr.c:1243 #19 0xffffffff80c7610a in ether_demux (ifp=ifp@entry=0xfffff80112341800, m=<unavailable>) at /usr/kernel_git/sys/net/if_ethersubr.c:952 #20 0xffffffff80c7779f in ether_input_internal (ifp=0xfffff80112341800, m=<unavailable>) at /usr/kernel_git/sys/net/if_ethersubr.c:716 #21 ether_nh_input (m=<unavailable>, m@entry=<error reading variable: value is not available>) at /usr/kernel_git/sys/net/if_ethersubr.c:746 #22 0xffffffff80c9393d in netisr_dispatch_src (proto=proto@entry=5, source=source@entry=0, m=m@entry=0xfffff80368fe5000) at /usr/kernel_git/sys/net/netisr.c:1152 #23 0xffffffff80c93c7f in netisr_dispatch (proto=<unavailable>, proto@entry=5, m=<unavailable>, m@entry=0xfffff80368fe5000) at /usr/kernel_git/sys/net/netisr.c:1243 #24 0xffffffff80c76569 in ether_input (ifp=0xfffff80112341800, ifp@entry=<error reading variable: value is not available>, m=0xfffff80368fe5000, m@entry=<error reading variable: value is not available>) at /usr/kernel_git/sys/net/if_ethersubr.c:857 #25 0xffffffff80d33f4c in tcp_lro_flush_all (lc=0xfffffe0158ae5dc0) at /usr/kernel_git/sys/netinet/tcp_lro.c:1255 #26 0xffffffff82778971 in mlx5_cq_arm (cq=0xfffffe0158ae5ce0, cmd=0, uar_page=<unavailable>, doorbell_lock=0x0, cons_index=<optimized out>) --Type <RET> for more, q to quit, c to continue without paging-- at ../../dev/mlx5/cq.h:149 #27 mlx5e_cq_arm (cq=0xfffffe0158ae5cc0, dblock=0x0) at ../../dev/mlx5/mlx5_en/en.h:1273 #28 mlx5e_rx_cq_comp (mcq=<optimized out>, eqe=<optimized out>) at /tmp/ofed_freebsd_3_0/sys/modules/mlx5en/../../dev/mlx5/mlx5_en/mlx5_en_rx.c:672 #29 0xffffffff826a80a4 in mlx5_cq_completion (dev=0xfffffe01581e6000, eqe=0xfffffe0158316bc0) at /tmp/ofed_freebsd_3_0/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_cq.c:73 #30 0xffffffff826ab2ba in mlx5_eq_int (dev=0xfffffe01581e6000, eq=0xfffff800056a5700) at /tmp/ofed_freebsd_3_0/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_eq.c:275 #31 0xffffffff826aab35 in kfree (ptr=0xfffff80003c2de00) at ../../compat/linuxkpi/common/include/linux/slab.h:201 #32 mlx5_create_map_eq (dev=0xfffff8004e10e740, eq=0xfffff80003c2de58, vecidx=<optimized out>, nent=<optimized out>, mask=8590000128) at /tmp/ofed_freebsd_3_0/sys/modules/mlx5/../../dev/mlx5/mlx5_core/mlx5_eq.c:498 #33 0xffffffff826f3a99 in ?? () at /tmp/ofed_freebsd_3_0/sys/modules/linuxkpi/../../compat/linuxkpi/common/src/linux_interrupt.c:110 from /boot/kernel/linuxkpi.ko #34 0x0000000000000000 in ?? () (kgdb) PANIC: r-freeb08.mtr.labs.mlnx dumped core - see /var/crash/vmcore.6 Fri Dec 8 16:44:17 IST 2023 FreeBSD r-freeb08.mtr.labs.mlnx 15.0-CURRENT FreeBSD 15.0-CURRENT #142 ipsec-n266961-b5163c1700a8: Fri Dec 8 14:14:09 IST 2023 r...@r-freeb08.mtr.labs.mlnx:/usr/obj/usr/kernel_git/amd64.amd64/sys/LATEST amd64 -- You are receiving this mail because: You are the assignee for the bug.