[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
Hi, sorry to ask non related questions. But could youe please teach how did you identity there is a negative refcnts from the dmesg? thanks... -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1786752 Title: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release) Status in linux package in Ubuntu: Confirmed Bug description: Since upgrading from 14.04 to 18.04, we see very frequent warnings about negative refcnts in dst_release: [ 3117.882227] WARNING: CPU: 6 PID: 0 at /build/linux-I4R9hO/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882229] Modules linked in: xt_policy cls_u32 sch_sfq ip_vti ip_tunnel authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key nfnetlink_queue nfnetlink_log sch_htb xt_TPROXY xt_multiport veth nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo br_netfilter bridge overlay macvlan 8021q garp mrp stp llc bonding algif_skcipher af_alg xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ip6t_REJECT nf_reject_ipv6 xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 dm_crypt iptable_nat nf_nat_ipv4 xt_DSCP xt_dscp xt_mark iptable_mangle xt_limit xt_tcpudp xt_addrtype intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp nf_conntrack_ipv4 nf_defrag_ipv4 kvm_intel xt_conntrack kvm joydev input_leds ipt_REJECT nf_reject_ipv4 ftdi_sio usbserial ipmi_si [ 3117.882265] irqbypass intel_cstate mei_me mei ioatdma acpi_pad intel_rapl_perf shpchp ipmi_devintf ipmi_msghandler acpi_power_meter lpc_ich mac_hid ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp sch_fq_codel nf_conntrack iptable_filter ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mgag200 ttm crct10dif_pclmul igb ixgbe crc32_pclmul drm_kms_helper hid_generic ghash_clmulni_intel syscopyarea i2c_algo_bit dca sysfillrect usbhid pcbc sysimgblt fb_sys_fops aesni_intel aes_x86_64 crypto_simd ahci glue_helper ptp hid mxm_wmi cryptd [ 3117.882309] drm libahci megaraid_sas pps_core mdio wmi [ 3117.882315] CPU: 6 PID: 0 Comm: swapper/6 Tainted: GW 4.15.0-30-generic #32-Ubuntu [ 3117.882316] Hardware name: Intel Corporation S2600WT2R/S2600WT2R, BIOS SE5C610.86B.01.01.0022.062820171903 06/28/2017 [ 3117.882319] RIP: 0010:nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882320] RSP: 0018:88d77f3839f0 EFLAGS: 00010246 [ 3117.882322] RAX: RBX: 90de4000 RCX: 1924 [ 3117.882323] RDX: RSI: 88d749ae6400 RDI: 88d655e96600 [ 3117.882324] RBP: 88d77f383a68 R08: 88d7493dc000 R09: 0018 [ 3117.882324] R10: 0001 R11: 88e770dedc00 R12: 88d655e96600 [ 3117.882325] R13: 88d77f383ae8 R14: 88d7799ff200 R15: 88d7493dc000 [ 3117.882327] FS: () GS:88d77f38() knlGS: [ 3117.882327] CS: 0010 DS: ES: CR0: 80050033 [ 3117.882328] CR2: ff600400 CR3: 00193de0a002 CR4: 003606e0 [ 3117.882329] DR0: DR1: DR2: [ 3117.882330] DR3: DR6: fffe0ff0 DR7: 0400 [ 3117.882331] Call Trace: [ 3117.882332] [ 3117.882337] ? nf_nat_ipv4_fn+0x15b/0x200 [nf_nat_ipv4] [ 3117.882339] nf_nat_ipv4_out+0xc5/0xe0 [nf_nat_ipv4] [ 3117.882342] iptable_nat_ipv4_out+0x15/0x20 [iptable_nat] [ 3117.882347] nf_hook_slow+0x48/0xc0 [ 3117.882353] ip_output+0xd2/0xe0 [ 3117.882355] ? ip_fragment.constprop.44+0x80/0x80 [ 3117.882357] ip_forward_finish+0x49/0x70 [ 3117.882359] ip_forward+0x366/0x440 [ 3117.882361] ? ip_frag_mem+0x20/0x20 [ 3117.882362] ip_rcv_finish+0x129/0x430 [ 3117.882364] ip_rcv+0x28f/0x3a0 [ 3117.882366] ? inet_del_offload+0x40/0x40 [ 3117.882372] __netif_receive_skb_core+0x432/0xb40 [ 3117.882379] ? handle_edge_irq+0x7c/0x190 [ 3117.882384] ? irq_exit+0x67/0xc0 [ 3117.882391] ? do_IRQ+0x82/0xd0 [ 3117.882393] __netif_receive_skb+0x18/0x60 [ 3117.882395] ? __netif_receive_skb+0x18/0x60 [ 3117.882397] netif_receive_skb_internal+0x37/0xd0 [ 3117.882398] napi_gro_receive+0xc5/0xf0 [ 3117.882407] ixgbe_clean_rx_irq+0x446/0xe30 [ixgbe] [ 3117.882411] ixgbe_poll+0x256/0x710 [ixgbe] [ 3117.882413] ? do_IRQ+0x82/0xd0 [ 3117.882415] net_rx_action+0x140/0x3a0 [ 3117.882418] __do_softirq+0xdf/0x2b2 [ 3117.882419] irq_exit+0xb6/0xc0 [ 3117.882421] do_IRQ+0x82/0xd0 [ 3117.882423] common_interrupt+0x84/0x84 [ 3117.882424] [ 3117.882427] RIP:
[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
We have upgraded to kernel v4.20.15 and its going to be more than a week, we are not seeing any more call traces in this kernel logs. We will watch for few more days and i will report back here. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1786752 Title: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release) Status in linux package in Ubuntu: Confirmed Bug description: Since upgrading from 14.04 to 18.04, we see very frequent warnings about negative refcnts in dst_release: [ 3117.882227] WARNING: CPU: 6 PID: 0 at /build/linux-I4R9hO/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882229] Modules linked in: xt_policy cls_u32 sch_sfq ip_vti ip_tunnel authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key nfnetlink_queue nfnetlink_log sch_htb xt_TPROXY xt_multiport veth nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo br_netfilter bridge overlay macvlan 8021q garp mrp stp llc bonding algif_skcipher af_alg xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ip6t_REJECT nf_reject_ipv6 xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 dm_crypt iptable_nat nf_nat_ipv4 xt_DSCP xt_dscp xt_mark iptable_mangle xt_limit xt_tcpudp xt_addrtype intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp nf_conntrack_ipv4 nf_defrag_ipv4 kvm_intel xt_conntrack kvm joydev input_leds ipt_REJECT nf_reject_ipv4 ftdi_sio usbserial ipmi_si [ 3117.882265] irqbypass intel_cstate mei_me mei ioatdma acpi_pad intel_rapl_perf shpchp ipmi_devintf ipmi_msghandler acpi_power_meter lpc_ich mac_hid ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp sch_fq_codel nf_conntrack iptable_filter ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mgag200 ttm crct10dif_pclmul igb ixgbe crc32_pclmul drm_kms_helper hid_generic ghash_clmulni_intel syscopyarea i2c_algo_bit dca sysfillrect usbhid pcbc sysimgblt fb_sys_fops aesni_intel aes_x86_64 crypto_simd ahci glue_helper ptp hid mxm_wmi cryptd [ 3117.882309] drm libahci megaraid_sas pps_core mdio wmi [ 3117.882315] CPU: 6 PID: 0 Comm: swapper/6 Tainted: GW 4.15.0-30-generic #32-Ubuntu [ 3117.882316] Hardware name: Intel Corporation S2600WT2R/S2600WT2R, BIOS SE5C610.86B.01.01.0022.062820171903 06/28/2017 [ 3117.882319] RIP: 0010:nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882320] RSP: 0018:88d77f3839f0 EFLAGS: 00010246 [ 3117.882322] RAX: RBX: 90de4000 RCX: 1924 [ 3117.882323] RDX: RSI: 88d749ae6400 RDI: 88d655e96600 [ 3117.882324] RBP: 88d77f383a68 R08: 88d7493dc000 R09: 0018 [ 3117.882324] R10: 0001 R11: 88e770dedc00 R12: 88d655e96600 [ 3117.882325] R13: 88d77f383ae8 R14: 88d7799ff200 R15: 88d7493dc000 [ 3117.882327] FS: () GS:88d77f38() knlGS: [ 3117.882327] CS: 0010 DS: ES: CR0: 80050033 [ 3117.882328] CR2: ff600400 CR3: 00193de0a002 CR4: 003606e0 [ 3117.882329] DR0: DR1: DR2: [ 3117.882330] DR3: DR6: fffe0ff0 DR7: 0400 [ 3117.882331] Call Trace: [ 3117.882332] [ 3117.882337] ? nf_nat_ipv4_fn+0x15b/0x200 [nf_nat_ipv4] [ 3117.882339] nf_nat_ipv4_out+0xc5/0xe0 [nf_nat_ipv4] [ 3117.882342] iptable_nat_ipv4_out+0x15/0x20 [iptable_nat] [ 3117.882347] nf_hook_slow+0x48/0xc0 [ 3117.882353] ip_output+0xd2/0xe0 [ 3117.882355] ? ip_fragment.constprop.44+0x80/0x80 [ 3117.882357] ip_forward_finish+0x49/0x70 [ 3117.882359] ip_forward+0x366/0x440 [ 3117.882361] ? ip_frag_mem+0x20/0x20 [ 3117.882362] ip_rcv_finish+0x129/0x430 [ 3117.882364] ip_rcv+0x28f/0x3a0 [ 3117.882366] ? inet_del_offload+0x40/0x40 [ 3117.882372] __netif_receive_skb_core+0x432/0xb40 [ 3117.882379] ? handle_edge_irq+0x7c/0x190 [ 3117.882384] ? irq_exit+0x67/0xc0 [ 3117.882391] ? do_IRQ+0x82/0xd0 [ 3117.882393] __netif_receive_skb+0x18/0x60 [ 3117.882395] ? __netif_receive_skb+0x18/0x60 [ 3117.882397] netif_receive_skb_internal+0x37/0xd0 [ 3117.882398] napi_gro_receive+0xc5/0xf0 [ 3117.882407] ixgbe_clean_rx_irq+0x446/0xe30 [ixgbe] [ 3117.882411] ixgbe_poll+0x256/0x710 [ixgbe] [ 3117.882413] ? do_IRQ+0x82/0xd0 [ 3117.882415] net_rx_action+0x140/0x3a0 [ 3117.882418] __do_softirq+0xdf/0x2b2 [ 3117.882419] irq_exit+0xb6/0xc0 [ 3117.882421] do_IRQ+0x82/0xd0 [ 3117.882423] common_interrupt+0x84/0x84 [
[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
Seems like it's fixed by this commit:
commit 542fbda0f08f1cbbc250f9e59f7537649651d0c8
Author: Florian Westphal
Date: Tue Dec 11 07:45:29 2018 +0100
netfilter: nat: can't use dst_hold on noref dst
The dst entry might already have a zero refcount, waiting on rcu list
to be free'd. Using dst_hold() transitions its reference count to 1, and
next dst release will try to free it again -- resulting in a double free:
WARNING: CPU: 1 PID: 0 at include/net/dst.h:239
nf_xfrm_me_harder+0xe7/0x130 [nf_nat]
RIP: 0010:nf_xfrm_me_harder+0xe7/0x130 [nf_nat]
Code: 48 8b 5c 24 60 65 48 33 1c 25 28 00 00 00 75 53 48 83 c4 68 5b 5d
41 5c c3 85 c0 74 0d 8d 48 01 f0 0f b1 0a 74 86 85 c0 75 f3 <0f> 0b e9 7b ff ff
ff 29 c6 31 d2 b9 20 00 48 00 4c 89 e7 e8 31 27
Call Trace:
nf_nat_ipv4_out+0x78/0x90 [nf_nat_ipv4]
nf_hook_slow+0x36/0xd0
ip_output+0x9f/0xd0
ip_forward+0x328/0x440
ip_rcv+0x8a/0xb0
Use dst_hold_safe instead and bail out if we cannot take a reference.
Fixes: a4c2fd7f7891 ("net: remove DST_NOCACHE flag")
Reported-by: Martin Zaharinov
Signed-off-by: Florian Westphal
Signed-off-by: Pablo Neira Ayuso
Try kernel >= v4.20 to verify.
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1786752
Title:
Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
Status in linux package in Ubuntu:
Confirmed
Bug description:
Since upgrading from 14.04 to 18.04, we see very frequent warnings
about negative refcnts in dst_release:
[ 3117.882227] WARNING: CPU: 6 PID: 0 at
/build/linux-I4R9hO/linux-4.15.0/include/net/dst.h:256
nf_xfrm_me_harder+0x127/0x140 [nf_nat]
[ 3117.882229] Modules linked in: xt_policy cls_u32 sch_sfq ip_vti ip_tunnel
authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm4_tunnel tunnel4
ipcomp xfrm_ipcomp esp4 ah4 af_key nfnetlink_queue nfnetlink_log sch_htb
xt_TPROXY xt_multiport veth nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo
br_netfilter bridge overlay macvlan 8021q garp mrp stp llc bonding
algif_skcipher af_alg xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6
ip6t_REJECT nf_reject_ipv6 xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4
dm_crypt iptable_nat nf_nat_ipv4 xt_DSCP xt_dscp xt_mark iptable_mangle
xt_limit xt_tcpudp xt_addrtype intel_rapl sb_edac x86_pkg_temp_thermal
intel_powerclamp coretemp nf_conntrack_ipv4 nf_defrag_ipv4 kvm_intel
xt_conntrack kvm joydev input_leds ipt_REJECT nf_reject_ipv4 ftdi_sio usbserial
ipmi_si
[ 3117.882265] irqbypass intel_cstate mei_me mei ioatdma acpi_pad
intel_rapl_perf shpchp ipmi_devintf ipmi_msghandler acpi_power_meter lpc_ich
mac_hid ip6table_filter ip6_tables nf_conntrack_netbios_ns
nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp sch_fq_codel
nf_conntrack iptable_filter ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp
libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs
zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor
async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mgag200 ttm
crct10dif_pclmul igb ixgbe crc32_pclmul drm_kms_helper hid_generic
ghash_clmulni_intel syscopyarea i2c_algo_bit dca sysfillrect usbhid pcbc
sysimgblt fb_sys_fops aesni_intel aes_x86_64 crypto_simd ahci glue_helper ptp
hid mxm_wmi cryptd
[ 3117.882309] drm libahci megaraid_sas pps_core mdio wmi
[ 3117.882315] CPU: 6 PID: 0 Comm: swapper/6 Tainted: GW
4.15.0-30-generic #32-Ubuntu
[ 3117.882316] Hardware name: Intel Corporation S2600WT2R/S2600WT2R, BIOS
SE5C610.86B.01.01.0022.062820171903 06/28/2017
[ 3117.882319] RIP: 0010:nf_xfrm_me_harder+0x127/0x140 [nf_nat]
[ 3117.882320] RSP: 0018:88d77f3839f0 EFLAGS: 00010246
[ 3117.882322] RAX: RBX: 90de4000 RCX:
1924
[ 3117.882323] RDX: RSI: 88d749ae6400 RDI:
88d655e96600
[ 3117.882324] RBP: 88d77f383a68 R08: 88d7493dc000 R09:
0018
[ 3117.882324] R10: 0001 R11: 88e770dedc00 R12:
88d655e96600
[ 3117.882325] R13: 88d77f383ae8 R14: 88d7799ff200 R15:
88d7493dc000
[ 3117.882327] FS: () GS:88d77f38()
knlGS:
[ 3117.882327] CS: 0010 DS: ES: CR0: 80050033
[ 3117.882328] CR2: ff600400 CR3: 00193de0a002 CR4:
003606e0
[ 3117.882329] DR0: DR1: DR2:
[ 3117.882330] DR3: DR6: fffe0ff0 DR7:
0400
[ 3117.882331] Call Trace:
[ 3117.882332]
[ 3117.882337] ? nf_nat_ipv4_fn+0x15b/0x200 [nf_nat_ipv4]
[ 3117.882339] nf_nat_ipv4_out+0xc5/0xe0 [nf_nat_ipv4]
[ 3117.882342] iptable_nat_ipv4_out+0x15/0x20 [iptable_nat]
[ 3117.882347] nf_hook_slow+0x48/0xc0
[ 3117.882353]
[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
# lspci 00:00.0 Host bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DMI2 (rev 01) 00:01.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D PCI Express Root Port 1 (rev 01) 00:02.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D PCI Express Root Port 2 (rev 01) 00:02.2 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D PCI Express Root Port 2 (rev 01) 00:03.0 PCI bridge: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D PCI Express Root Port 3 (rev 01) 00:04.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 0 (rev 01) 00:04.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 1 (rev 01) 00:04.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 2 (rev 01) 00:04.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 3 (rev 01) 00:04.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 4 (rev 01) 00:04.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 5 (rev 01) 00:04.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 6 (rev 01) 00:04.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 7 (rev 01) 00:05.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Map/VTd_Misc/System Management (rev 01) 00:05.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D IIO Hot Plug (rev 01) 00:05.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D IIO RAS/Control Status/Global Errors (rev 01) 00:05.4 PIC: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D I/O APIC (rev 01) 00:11.0 Unassigned class [ff00]: Intel Corporation C610/X99 series chipset SPSR (rev 05) 00:11.1 SMBus: Intel Corporation C610/X99 series chipset MS SMBus 0 (rev 05) 00:11.4 SATA controller: Intel Corporation C610/X99 series chipset sSATA Controller [AHCI mode] (rev 05) 00:14.0 USB controller: Intel Corporation C610/X99 series chipset USB xHCI Host Controller (rev 05) 00:16.0 Communication controller: Intel Corporation C610/X99 series chipset MEI Controller #1 (rev 05) 00:16.1 Communication controller: Intel Corporation C610/X99 series chipset MEI Controller #2 (rev 05) 00:1a.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #2 (rev 05) 00:1c.0 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #1 (rev d5) 00:1c.3 PCI bridge: Intel Corporation C610/X99 series chipset PCI Express Root Port #4 (rev d5) 00:1d.0 USB controller: Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #1 (rev 05) 00:1f.0 ISA bridge: Intel Corporation C610/X99 series chipset LPC Controller (rev 05) 00:1f.2 SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05) 00:1f.3 SMBus: Intel Corporation C610/X99 series chipset SMBus Controller (rev 05) 01:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS-3 3108 [Invader] (rev 02) 02:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01) 02:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01) 04:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) 04:00.3 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) 06:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01) 06:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01) 09:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200e [Pilot] ServerEngines (SEP1) (rev 05) 7f:08.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D QPI Link 0 (rev 01) 7f:08.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D QPI Link 0 (rev 01) 7f:08.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D QPI Link 0 (rev 01) 7f:09.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D QPI Link 1 (rev 01) 7f:09.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D QPI Link 1 (rev 01) 7f:09.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D QPI Link 1 (rev 01) 7f:0b.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) 7f:0b.1 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1 (rev 01) 7f:0b.2 Performance counters: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D
[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
We have upgraded the latest kernel and even occasionally we facing this call traces and CPU tainted, # uname -rn 4.15.0-45-generic # uname -a Linux 4.15.0-45-generic #48-Ubuntu SMP Tue Jan 29 16:28:13 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux # dmesg -T | grep "Fri Mar 1.*CPU" [Fri Mar 1 10:28:42 2019] WARNING: CPU: 33 PID: 0 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 10:28:42 2019] CPU: 33 PID: 0 Comm: swapper/33 Tainted: GW OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 10:29:56 2019] WARNING: CPU: 12 PID: 0 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 10:29:56 2019] CPU: 12 PID: 0 Comm: swapper/12 Tainted: GW OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 10:53:01 2019] WARNING: CPU: 13 PID: 0 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 10:53:01 2019] WARNING: CPU: 1 PID: 18428 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 10:53:01 2019] WARNING: CPU: 5 PID: 18424 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 10:53:01 2019] CPU: 1 PID: 18428 Comm: haproxy Tainted: GW OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 10:53:01 2019] CPU: 5 PID: 18424 Comm: haproxy Tainted: GW OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 10:53:01 2019] CPU: 13 PID: 0 Comm: swapper/13 Tainted: GW OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 10:53:50 2019] WARNING: CPU: 5 PID: 0 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 10:53:50 2019] CPU: 5 PID: 0 Comm: swapper/5 Tainted: GW OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 12:29:32 2019] WARNING: CPU: 22 PID: 0 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 12:29:32 2019] CPU: 22 PID: 0 Comm: swapper/22 Tainted: GW OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 13:01:20 2019] WARNING: CPU: 31 PID: 20932 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 13:01:20 2019] CPU: 31 PID: 20932 Comm: thread.rb:70 Tainted: G W OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 13:01:23 2019] WARNING: CPU: 14 PID: 18429 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 13:01:23 2019] CPU: 14 PID: 18429 Comm: haproxy Tainted: GW OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 13:01:23 2019] WARNING: CPU: 31 PID: 7532 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 13:01:23 2019] CPU: 31 PID: 7532 Comm: telegraf Tainted: GW OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 13:16:28 2019] WARNING: CPU: 32 PID: 7842 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 13:16:28 2019] CPU: 32 PID: 7842 Comm: haproxy Tainted: GW OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 13:48:17 2019] WARNING: CPU: 13 PID: 0 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 13:48:17 2019] CPU: 13 PID: 0 Comm: swapper/13 Tainted: GW OE4.15.0-45-generic #48-Ubuntu [Fri Mar 1 13:48:20 2019] WARNING: CPU: 5 PID: 41 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 13:48:20 2019] CPU: 5 PID: 41 Comm: ksoftirqd/5 Tainted: GW OE4.15.0-45-generic #48-Ubuntu # dmesg -T | grep "Fri Mar 1" [Fri Mar 1 10:28:42 2019] WARNING: CPU: 33 PID: 0 at /build/linux-uQJ2um/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [Fri Mar 1 10:28:42 2019] Modules linked in: sctp xt_recent xfrm4_tunnel tunnel4 nfnetlink_queue nfnetlink_log ipcomp xfrm_ipcomp esp4 ah4 af_key tcp_diag udp_diag inet_diag sch_htb xt_TPROXY xt_multiport veth nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo br_netfilter bridge aufs overlay macvlan 8021q garp mrp stp llc bonding wireguard(OE) ip6_udp_tunnel udp_tunnel algif_skcipher af_alg xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ip6t_REJECT nf_reject_ipv6 xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 xt_DSCP xt_dscp xt_mark dm_crypt iptable_mangle xt_limit xt_tcpudp xt_addrtype intel_rapl nf_conntrack_ipv4 nf_defrag_ipv4 sb_edac x86_pkg_temp_thermal intel_powerclamp xt_conntrack coretemp ipt_REJECT nf_reject_ipv4 kvm_intel kvm shpchp ftdi_sio usbserial irqbypass lpc_ich joydev input_leds [Fri Mar 1 10:28:42 2019] intel_cstate intel_rapl_perf mei_me mei ipmi_si ipmi_devintf ipmi_msghandler ioatdma ip6table_filter ip6_tables mac_hid acpi_power_meter acpi_pad nf_conntrack_netbios_ns
[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
Just to follow up , on behalf of Juergen. Do we have any progress with this bug ? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1786752 Title: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release) Status in linux package in Ubuntu: Confirmed Bug description: Since upgrading from 14.04 to 18.04, we see very frequent warnings about negative refcnts in dst_release: [ 3117.882227] WARNING: CPU: 6 PID: 0 at /build/linux-I4R9hO/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882229] Modules linked in: xt_policy cls_u32 sch_sfq ip_vti ip_tunnel authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key nfnetlink_queue nfnetlink_log sch_htb xt_TPROXY xt_multiport veth nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo br_netfilter bridge overlay macvlan 8021q garp mrp stp llc bonding algif_skcipher af_alg xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ip6t_REJECT nf_reject_ipv6 xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 dm_crypt iptable_nat nf_nat_ipv4 xt_DSCP xt_dscp xt_mark iptable_mangle xt_limit xt_tcpudp xt_addrtype intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp nf_conntrack_ipv4 nf_defrag_ipv4 kvm_intel xt_conntrack kvm joydev input_leds ipt_REJECT nf_reject_ipv4 ftdi_sio usbserial ipmi_si [ 3117.882265] irqbypass intel_cstate mei_me mei ioatdma acpi_pad intel_rapl_perf shpchp ipmi_devintf ipmi_msghandler acpi_power_meter lpc_ich mac_hid ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp sch_fq_codel nf_conntrack iptable_filter ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mgag200 ttm crct10dif_pclmul igb ixgbe crc32_pclmul drm_kms_helper hid_generic ghash_clmulni_intel syscopyarea i2c_algo_bit dca sysfillrect usbhid pcbc sysimgblt fb_sys_fops aesni_intel aes_x86_64 crypto_simd ahci glue_helper ptp hid mxm_wmi cryptd [ 3117.882309] drm libahci megaraid_sas pps_core mdio wmi [ 3117.882315] CPU: 6 PID: 0 Comm: swapper/6 Tainted: GW 4.15.0-30-generic #32-Ubuntu [ 3117.882316] Hardware name: Intel Corporation S2600WT2R/S2600WT2R, BIOS SE5C610.86B.01.01.0022.062820171903 06/28/2017 [ 3117.882319] RIP: 0010:nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882320] RSP: 0018:88d77f3839f0 EFLAGS: 00010246 [ 3117.882322] RAX: RBX: 90de4000 RCX: 1924 [ 3117.882323] RDX: RSI: 88d749ae6400 RDI: 88d655e96600 [ 3117.882324] RBP: 88d77f383a68 R08: 88d7493dc000 R09: 0018 [ 3117.882324] R10: 0001 R11: 88e770dedc00 R12: 88d655e96600 [ 3117.882325] R13: 88d77f383ae8 R14: 88d7799ff200 R15: 88d7493dc000 [ 3117.882327] FS: () GS:88d77f38() knlGS: [ 3117.882327] CS: 0010 DS: ES: CR0: 80050033 [ 3117.882328] CR2: ff600400 CR3: 00193de0a002 CR4: 003606e0 [ 3117.882329] DR0: DR1: DR2: [ 3117.882330] DR3: DR6: fffe0ff0 DR7: 0400 [ 3117.882331] Call Trace: [ 3117.882332] [ 3117.882337] ? nf_nat_ipv4_fn+0x15b/0x200 [nf_nat_ipv4] [ 3117.882339] nf_nat_ipv4_out+0xc5/0xe0 [nf_nat_ipv4] [ 3117.882342] iptable_nat_ipv4_out+0x15/0x20 [iptable_nat] [ 3117.882347] nf_hook_slow+0x48/0xc0 [ 3117.882353] ip_output+0xd2/0xe0 [ 3117.882355] ? ip_fragment.constprop.44+0x80/0x80 [ 3117.882357] ip_forward_finish+0x49/0x70 [ 3117.882359] ip_forward+0x366/0x440 [ 3117.882361] ? ip_frag_mem+0x20/0x20 [ 3117.882362] ip_rcv_finish+0x129/0x430 [ 3117.882364] ip_rcv+0x28f/0x3a0 [ 3117.882366] ? inet_del_offload+0x40/0x40 [ 3117.882372] __netif_receive_skb_core+0x432/0xb40 [ 3117.882379] ? handle_edge_irq+0x7c/0x190 [ 3117.882384] ? irq_exit+0x67/0xc0 [ 3117.882391] ? do_IRQ+0x82/0xd0 [ 3117.882393] __netif_receive_skb+0x18/0x60 [ 3117.882395] ? __netif_receive_skb+0x18/0x60 [ 3117.882397] netif_receive_skb_internal+0x37/0xd0 [ 3117.882398] napi_gro_receive+0xc5/0xf0 [ 3117.882407] ixgbe_clean_rx_irq+0x446/0xe30 [ixgbe] [ 3117.882411] ixgbe_poll+0x256/0x710 [ixgbe] [ 3117.882413] ? do_IRQ+0x82/0xd0 [ 3117.882415] net_rx_action+0x140/0x3a0 [ 3117.882418] __do_softirq+0xdf/0x2b2 [ 3117.882419] irq_exit+0xb6/0xc0 [ 3117.882421] do_IRQ+0x82/0xd0 [ 3117.882423] common_interrupt+0x84/0x84 [ 3117.882424] [ 3117.882427] RIP: 0010:cpuidle_enter_state+0xa7/0x2f0 [ 3117.882428] RSP:
[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
Hi Do you find fix for this bug I use latest kernel 4.19.8 and have same problem see down i use pppoe with 1k+ users if i activate shaper with hfsc and imq machine crash and reboot after stop shaper for test mashine only send in dmesg bug error mesg. This is a Kernel BUG and need to by fix if any need more info or debug i will send you. [93408.910441] WARNING: CPU: 1 PID: 0 at include/net/dst.h:239 nf_xfrm_me_harder+0xe7/0x130 [nf_nat] [93408.945500] Modules linked in: udp_diag unix_diag af_packet_diag sch_hfsc iptable_filter xt_IMQ iptable_mangle xt_addrtype xt_nat ipt_MASQUERADE iptable_nat nf_nat_ipv4 ip_tables bpfilter sch_fq_codel netconsole r8169 tg3 libphy igb i2c_algo_bit ixgb nf_nat_pptp nf_nat_proto_gre nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_tftp nf_conntrack_tftp nf_nat_ftp nf_conntrack_ftp nf_nat nf_conntrack nf_defrag_ipv4 pppoe pptp gre pppox sha1_mb mcryptd sha1_ssse3 sha1_generic arc4 ppp_mppe ppp_generic slhc megaraid_sas [last unloaded: imq] [93409.103093] CPU: 1 PID: 0 Comm: PDS/1 Tainted: G O 4.19.6 #1 [93409.124009] Hardware name: Supermicro Super Server/X11SSZ-F, BIOS 2.2b 02/12/2018 [93409.166624] RIP: 0010:nf_xfrm_me_harder+0xe7/0x130 [nf_nat] [93409.188447] Code: 48 8b 5c 24 60 65 48 33 1c 25 28 00 00 00 75 53 48 83 c4 68 5b 5d 41 5c c3 85 c0 74 0d 8d 48 01 f0 0f b1 0a 74 86 85 c0 75 f3 <0f> 0b e9 7b ff ff ff 29 c6 31 d2 b9 20 00 48 00 4c 89 e7 e8 31 27 [93409.254096] RSP: 0018:888454c83ce0 EFLAGS: 00010246 [93409.275791] RAX: RBX: 81e83f40 RCX: 98a2 [93409.297628] RDX: RSI: 88843c229480 RDI: 0038 [93409.319099] RBP: R08: 0038 R09: 8883e2811000 [93409.340547] R10: 0009 R11: R12: 8883e68d7a00 [93409.361901] R13: 888454c83db8 R14: 888454c9f64c R15: 888454c9f650 [93409.383419] FS: () GS:888454c8() knlGS: [93409.427294] CS: 0010 DS: ES: CR0: 80050033 [93409.449919] CR2: 7f864054e730 CR3: 01e0a006 CR4: 001606e0 [93409.472746] DR0: DR1: DR2: [93409.495480] DR3: DR6: fffe0ff0 DR7: 0400 [93409.518035] Call Trace: [93409.539897] [93409.561039] ? nf_nat_ipv4_manip_pkt+0x54/0x100 [nf_nat_ipv4] [93409.582440] nf_nat_ipv4_out+0x78/0x90 [nf_nat_ipv4] [93409.603476] nf_hook_slow+0x36/0xd0 [93409.623865] ip_output+0x9f/0xd0 [93409.643603] ? ip_fragment.constprop.5+0x70/0x70 [93409.663095] ip_forward+0x328/0x440 [93409.681987] ? ip_defrag.cold.3+0x22/0x22 [93409.700405] ip_rcv+0x8a/0xb0 [93409.718201] ? ip_rcv_finish_core.isra.0+0x340/0x340 [93409.735914] __netif_receive_skb_one_core+0x4b/0x70 [93409.753308] process_backlog+0x95/0x130 [93409.770199] net_rx_action+0x122/0x2d0 [93409.786744] __do_softirq+0xba/0x206 [93409.802880] irq_exit+0xae/0xf0 [93409.818424] call_function_single_interrupt+0xf/0x20 [93409.834147] [93409.849337] RIP: 0010:mwait_idle+0x50/0x80 [93409.864491] Code: 0f ba e2 27 72 3a 31 d2 65 48 8b 04 25 40 4c 01 00 48 89 d1 0f 01 c8 48 8b 00 48 c1 e8 03 83 e0 01 75 2b 48 89 c1 fb 0f 01 c9 <65> 8b 05 69 b2 9e 7e 65 48 8b 04 25 40 4c 01 00 f0 80 60 02 df c3 [93409.913067] RSP: 0018:c90b3ed0 EFLAGS: 0246 ORIG_RAX: ff04 [93409.946216] RAX: RBX: 0001 RCX: [93409.963281] RDX: RSI: RDI: 888454c9ef80 [93409.980003] RBP: 81e943a0 R08: R09: 0001e540 [93409.996513] R10: fefbc9e70705 R11: 0001 R12: [93410.013011] R13: R14: R15: [93410.029583] do_idle+0x1c7/0x1e0 [93410.045584] cpu_startup_entry+0x6a/0x70 [93410.061835] start_secondary+0x165/0x190 [93410.077838] secondary_startup_64+0xa4/0xb0 [93410.093605] ---[ end trace 2bdcfd51408010e6 ]--- [93410.109465] dst_release: dst:aa09d44e refcnt:-1 [167366.275640] WARNING: CPU: 1 PID: 0 at include/net/dst.h:239 nf_xfrm_me_harder+0xe7/0x130 [nf_nat] [167366.309111] Modules linked in: udp_diag unix_diag af_packet_diag sch_hfsc iptable_filter xt_IMQ iptable_mangle xt_addrtype xt_nat ipt_MASQUERADE iptable_nat nf_nat_ipv4 ip_tables bpfilter sch_fq_codel netconsole r8169 tg3 libphy igb i2c_algo_bit ixgb nf_nat_pptp nf_nat_proto_gre nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_tftp nf_conntrack_tftp nf_nat_ftp nf_conntrack_ftp nf_nat nf_conntrack nf_defrag_ipv4 pppoe pptp gre pppox sha1_mb mcryptd sha1_ssse3 sha1_generic arc4 ppp_mppe ppp_generic slhc megaraid_sas [last unloaded: imq] [167366.464130] CPU: 1 PID: 0 Comm: PDS/1 Tainted: GW O 4.19.6 #1 [167366.485631] Hardware name: Supermicro Super Server/X11SSZ-F, BIOS 2.2b 02/12/2018 [167366.528653] RIP: 0010:nf_xfrm_me_harder+0xe7/0x130 [nf_nat]
[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
The issue still is visible with the latest mainline kernel; see attached. ** Attachment added: "dmesg from 4.18" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1786752/+attachment/5174879/+files/dmesg_4.18.txt ** Changed in: linux (Ubuntu) Status: Incomplete => Confirmed ** Tags added: kernel-bug-exists-upstream -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1786752 Title: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release) Status in linux package in Ubuntu: Confirmed Bug description: Since upgrading from 14.04 to 18.04, we see very frequent warnings about negative refcnts in dst_release: [ 3117.882227] WARNING: CPU: 6 PID: 0 at /build/linux-I4R9hO/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882229] Modules linked in: xt_policy cls_u32 sch_sfq ip_vti ip_tunnel authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key nfnetlink_queue nfnetlink_log sch_htb xt_TPROXY xt_multiport veth nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo br_netfilter bridge overlay macvlan 8021q garp mrp stp llc bonding algif_skcipher af_alg xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ip6t_REJECT nf_reject_ipv6 xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 dm_crypt iptable_nat nf_nat_ipv4 xt_DSCP xt_dscp xt_mark iptable_mangle xt_limit xt_tcpudp xt_addrtype intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp nf_conntrack_ipv4 nf_defrag_ipv4 kvm_intel xt_conntrack kvm joydev input_leds ipt_REJECT nf_reject_ipv4 ftdi_sio usbserial ipmi_si [ 3117.882265] irqbypass intel_cstate mei_me mei ioatdma acpi_pad intel_rapl_perf shpchp ipmi_devintf ipmi_msghandler acpi_power_meter lpc_ich mac_hid ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp sch_fq_codel nf_conntrack iptable_filter ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mgag200 ttm crct10dif_pclmul igb ixgbe crc32_pclmul drm_kms_helper hid_generic ghash_clmulni_intel syscopyarea i2c_algo_bit dca sysfillrect usbhid pcbc sysimgblt fb_sys_fops aesni_intel aes_x86_64 crypto_simd ahci glue_helper ptp hid mxm_wmi cryptd [ 3117.882309] drm libahci megaraid_sas pps_core mdio wmi [ 3117.882315] CPU: 6 PID: 0 Comm: swapper/6 Tainted: GW 4.15.0-30-generic #32-Ubuntu [ 3117.882316] Hardware name: Intel Corporation S2600WT2R/S2600WT2R, BIOS SE5C610.86B.01.01.0022.062820171903 06/28/2017 [ 3117.882319] RIP: 0010:nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882320] RSP: 0018:88d77f3839f0 EFLAGS: 00010246 [ 3117.882322] RAX: RBX: 90de4000 RCX: 1924 [ 3117.882323] RDX: RSI: 88d749ae6400 RDI: 88d655e96600 [ 3117.882324] RBP: 88d77f383a68 R08: 88d7493dc000 R09: 0018 [ 3117.882324] R10: 0001 R11: 88e770dedc00 R12: 88d655e96600 [ 3117.882325] R13: 88d77f383ae8 R14: 88d7799ff200 R15: 88d7493dc000 [ 3117.882327] FS: () GS:88d77f38() knlGS: [ 3117.882327] CS: 0010 DS: ES: CR0: 80050033 [ 3117.882328] CR2: ff600400 CR3: 00193de0a002 CR4: 003606e0 [ 3117.882329] DR0: DR1: DR2: [ 3117.882330] DR3: DR6: fffe0ff0 DR7: 0400 [ 3117.882331] Call Trace: [ 3117.882332] [ 3117.882337] ? nf_nat_ipv4_fn+0x15b/0x200 [nf_nat_ipv4] [ 3117.882339] nf_nat_ipv4_out+0xc5/0xe0 [nf_nat_ipv4] [ 3117.882342] iptable_nat_ipv4_out+0x15/0x20 [iptable_nat] [ 3117.882347] nf_hook_slow+0x48/0xc0 [ 3117.882353] ip_output+0xd2/0xe0 [ 3117.882355] ? ip_fragment.constprop.44+0x80/0x80 [ 3117.882357] ip_forward_finish+0x49/0x70 [ 3117.882359] ip_forward+0x366/0x440 [ 3117.882361] ? ip_frag_mem+0x20/0x20 [ 3117.882362] ip_rcv_finish+0x129/0x430 [ 3117.882364] ip_rcv+0x28f/0x3a0 [ 3117.882366] ? inet_del_offload+0x40/0x40 [ 3117.882372] __netif_receive_skb_core+0x432/0xb40 [ 3117.882379] ? handle_edge_irq+0x7c/0x190 [ 3117.882384] ? irq_exit+0x67/0xc0 [ 3117.882391] ? do_IRQ+0x82/0xd0 [ 3117.882393] __netif_receive_skb+0x18/0x60 [ 3117.882395] ? __netif_receive_skb+0x18/0x60 [ 3117.882397] netif_receive_skb_internal+0x37/0xd0 [ 3117.882398] napi_gro_receive+0xc5/0xf0 [ 3117.882407] ixgbe_clean_rx_irq+0x446/0xe30 [ixgbe] [ 3117.882411] ixgbe_poll+0x256/0x710 [ixgbe] [ 3117.882413] ? do_IRQ+0x82/0xd0 [ 3117.882415] net_rx_action+0x140/0x3a0 [ 3117.882418]
[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.18 kernel[0]. If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'. If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'. Once testing of the upstream kernel is complete, please mark this bug as "Confirmed". Thanks in advance. [0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.18 ** Changed in: linux (Ubuntu) Importance: Undecided => Medium ** Changed in: linux (Ubuntu) Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1786752 Title: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release) Status in linux package in Ubuntu: Incomplete Bug description: Since upgrading from 14.04 to 18.04, we see very frequent warnings about negative refcnts in dst_release: [ 3117.882227] WARNING: CPU: 6 PID: 0 at /build/linux-I4R9hO/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882229] Modules linked in: xt_policy cls_u32 sch_sfq ip_vti ip_tunnel authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key nfnetlink_queue nfnetlink_log sch_htb xt_TPROXY xt_multiport veth nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo br_netfilter bridge overlay macvlan 8021q garp mrp stp llc bonding algif_skcipher af_alg xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ip6t_REJECT nf_reject_ipv6 xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 dm_crypt iptable_nat nf_nat_ipv4 xt_DSCP xt_dscp xt_mark iptable_mangle xt_limit xt_tcpudp xt_addrtype intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp nf_conntrack_ipv4 nf_defrag_ipv4 kvm_intel xt_conntrack kvm joydev input_leds ipt_REJECT nf_reject_ipv4 ftdi_sio usbserial ipmi_si [ 3117.882265] irqbypass intel_cstate mei_me mei ioatdma acpi_pad intel_rapl_perf shpchp ipmi_devintf ipmi_msghandler acpi_power_meter lpc_ich mac_hid ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp sch_fq_codel nf_conntrack iptable_filter ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mgag200 ttm crct10dif_pclmul igb ixgbe crc32_pclmul drm_kms_helper hid_generic ghash_clmulni_intel syscopyarea i2c_algo_bit dca sysfillrect usbhid pcbc sysimgblt fb_sys_fops aesni_intel aes_x86_64 crypto_simd ahci glue_helper ptp hid mxm_wmi cryptd [ 3117.882309] drm libahci megaraid_sas pps_core mdio wmi [ 3117.882315] CPU: 6 PID: 0 Comm: swapper/6 Tainted: GW 4.15.0-30-generic #32-Ubuntu [ 3117.882316] Hardware name: Intel Corporation S2600WT2R/S2600WT2R, BIOS SE5C610.86B.01.01.0022.062820171903 06/28/2017 [ 3117.882319] RIP: 0010:nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882320] RSP: 0018:88d77f3839f0 EFLAGS: 00010246 [ 3117.882322] RAX: RBX: 90de4000 RCX: 1924 [ 3117.882323] RDX: RSI: 88d749ae6400 RDI: 88d655e96600 [ 3117.882324] RBP: 88d77f383a68 R08: 88d7493dc000 R09: 0018 [ 3117.882324] R10: 0001 R11: 88e770dedc00 R12: 88d655e96600 [ 3117.882325] R13: 88d77f383ae8 R14: 88d7799ff200 R15: 88d7493dc000 [ 3117.882327] FS: () GS:88d77f38() knlGS: [ 3117.882327] CS: 0010 DS: ES: CR0: 80050033 [ 3117.882328] CR2: ff600400 CR3: 00193de0a002 CR4: 003606e0 [ 3117.882329] DR0: DR1: DR2: [ 3117.882330] DR3: DR6: fffe0ff0 DR7: 0400 [ 3117.882331] Call Trace: [ 3117.882332] [ 3117.882337] ? nf_nat_ipv4_fn+0x15b/0x200 [nf_nat_ipv4] [ 3117.882339] nf_nat_ipv4_out+0xc5/0xe0 [nf_nat_ipv4] [ 3117.882342] iptable_nat_ipv4_out+0x15/0x20 [iptable_nat] [ 3117.882347] nf_hook_slow+0x48/0xc0 [ 3117.882353] ip_output+0xd2/0xe0 [ 3117.882355] ? ip_fragment.constprop.44+0x80/0x80 [ 3117.882357] ip_forward_finish+0x49/0x70 [ 3117.882359] ip_forward+0x366/0x440 [ 3117.882361] ? ip_frag_mem+0x20/0x20 [ 3117.882362] ip_rcv_finish+0x129/0x430 [ 3117.882364] ip_rcv+0x28f/0x3a0 [ 3117.882366] ? inet_del_offload+0x40/0x40 [ 3117.882372] __netif_receive_skb_core+0x432/0xb40 [ 3117.882379] ? handle_edge_irq+0x7c/0x190 [ 3117.882384] ? irq_exit+0x67/0xc0 [ 3117.882391] ? do_IRQ+0x82/0xd0 [ 3117.882393] __netif_receive_skb+0x18/0x60 [ 3117.882395] ?
[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
** Attachment added: "dmesg" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1786752/+attachment/5174664/+files/dmesg.log -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1786752 Title: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release) Status in linux package in Ubuntu: Confirmed Bug description: Since upgrading from 14.04 to 18.04, we see very frequent warnings about negative refcnts in dst_release: [ 3117.882227] WARNING: CPU: 6 PID: 0 at /build/linux-I4R9hO/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882229] Modules linked in: xt_policy cls_u32 sch_sfq ip_vti ip_tunnel authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key nfnetlink_queue nfnetlink_log sch_htb xt_TPROXY xt_multiport veth nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo br_netfilter bridge overlay macvlan 8021q garp mrp stp llc bonding algif_skcipher af_alg xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ip6t_REJECT nf_reject_ipv6 xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 dm_crypt iptable_nat nf_nat_ipv4 xt_DSCP xt_dscp xt_mark iptable_mangle xt_limit xt_tcpudp xt_addrtype intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp nf_conntrack_ipv4 nf_defrag_ipv4 kvm_intel xt_conntrack kvm joydev input_leds ipt_REJECT nf_reject_ipv4 ftdi_sio usbserial ipmi_si [ 3117.882265] irqbypass intel_cstate mei_me mei ioatdma acpi_pad intel_rapl_perf shpchp ipmi_devintf ipmi_msghandler acpi_power_meter lpc_ich mac_hid ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp sch_fq_codel nf_conntrack iptable_filter ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mgag200 ttm crct10dif_pclmul igb ixgbe crc32_pclmul drm_kms_helper hid_generic ghash_clmulni_intel syscopyarea i2c_algo_bit dca sysfillrect usbhid pcbc sysimgblt fb_sys_fops aesni_intel aes_x86_64 crypto_simd ahci glue_helper ptp hid mxm_wmi cryptd [ 3117.882309] drm libahci megaraid_sas pps_core mdio wmi [ 3117.882315] CPU: 6 PID: 0 Comm: swapper/6 Tainted: GW 4.15.0-30-generic #32-Ubuntu [ 3117.882316] Hardware name: Intel Corporation S2600WT2R/S2600WT2R, BIOS SE5C610.86B.01.01.0022.062820171903 06/28/2017 [ 3117.882319] RIP: 0010:nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882320] RSP: 0018:88d77f3839f0 EFLAGS: 00010246 [ 3117.882322] RAX: RBX: 90de4000 RCX: 1924 [ 3117.882323] RDX: RSI: 88d749ae6400 RDI: 88d655e96600 [ 3117.882324] RBP: 88d77f383a68 R08: 88d7493dc000 R09: 0018 [ 3117.882324] R10: 0001 R11: 88e770dedc00 R12: 88d655e96600 [ 3117.882325] R13: 88d77f383ae8 R14: 88d7799ff200 R15: 88d7493dc000 [ 3117.882327] FS: () GS:88d77f38() knlGS: [ 3117.882327] CS: 0010 DS: ES: CR0: 80050033 [ 3117.882328] CR2: ff600400 CR3: 00193de0a002 CR4: 003606e0 [ 3117.882329] DR0: DR1: DR2: [ 3117.882330] DR3: DR6: fffe0ff0 DR7: 0400 [ 3117.882331] Call Trace: [ 3117.882332] [ 3117.882337] ? nf_nat_ipv4_fn+0x15b/0x200 [nf_nat_ipv4] [ 3117.882339] nf_nat_ipv4_out+0xc5/0xe0 [nf_nat_ipv4] [ 3117.882342] iptable_nat_ipv4_out+0x15/0x20 [iptable_nat] [ 3117.882347] nf_hook_slow+0x48/0xc0 [ 3117.882353] ip_output+0xd2/0xe0 [ 3117.882355] ? ip_fragment.constprop.44+0x80/0x80 [ 3117.882357] ip_forward_finish+0x49/0x70 [ 3117.882359] ip_forward+0x366/0x440 [ 3117.882361] ? ip_frag_mem+0x20/0x20 [ 3117.882362] ip_rcv_finish+0x129/0x430 [ 3117.882364] ip_rcv+0x28f/0x3a0 [ 3117.882366] ? inet_del_offload+0x40/0x40 [ 3117.882372] __netif_receive_skb_core+0x432/0xb40 [ 3117.882379] ? handle_edge_irq+0x7c/0x190 [ 3117.882384] ? irq_exit+0x67/0xc0 [ 3117.882391] ? do_IRQ+0x82/0xd0 [ 3117.882393] __netif_receive_skb+0x18/0x60 [ 3117.882395] ? __netif_receive_skb+0x18/0x60 [ 3117.882397] netif_receive_skb_internal+0x37/0xd0 [ 3117.882398] napi_gro_receive+0xc5/0xf0 [ 3117.882407] ixgbe_clean_rx_irq+0x446/0xe30 [ixgbe] [ 3117.882411] ixgbe_poll+0x256/0x710 [ixgbe] [ 3117.882413] ? do_IRQ+0x82/0xd0 [ 3117.882415] net_rx_action+0x140/0x3a0 [ 3117.882418] __do_softirq+0xdf/0x2b2 [ 3117.882419] irq_exit+0xb6/0xc0 [ 3117.882421] do_IRQ+0x82/0xd0 [ 3117.882423] common_interrupt+0x84/0x84 [ 3117.882424] [ 3117.882427] RIP:
[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
** Attachment added: "version.log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1786752/+attachment/5174665/+files/version.log -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1786752 Title: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release) Status in linux package in Ubuntu: Confirmed Bug description: Since upgrading from 14.04 to 18.04, we see very frequent warnings about negative refcnts in dst_release: [ 3117.882227] WARNING: CPU: 6 PID: 0 at /build/linux-I4R9hO/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882229] Modules linked in: xt_policy cls_u32 sch_sfq ip_vti ip_tunnel authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key nfnetlink_queue nfnetlink_log sch_htb xt_TPROXY xt_multiport veth nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo br_netfilter bridge overlay macvlan 8021q garp mrp stp llc bonding algif_skcipher af_alg xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ip6t_REJECT nf_reject_ipv6 xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 dm_crypt iptable_nat nf_nat_ipv4 xt_DSCP xt_dscp xt_mark iptable_mangle xt_limit xt_tcpudp xt_addrtype intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp nf_conntrack_ipv4 nf_defrag_ipv4 kvm_intel xt_conntrack kvm joydev input_leds ipt_REJECT nf_reject_ipv4 ftdi_sio usbserial ipmi_si [ 3117.882265] irqbypass intel_cstate mei_me mei ioatdma acpi_pad intel_rapl_perf shpchp ipmi_devintf ipmi_msghandler acpi_power_meter lpc_ich mac_hid ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp sch_fq_codel nf_conntrack iptable_filter ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mgag200 ttm crct10dif_pclmul igb ixgbe crc32_pclmul drm_kms_helper hid_generic ghash_clmulni_intel syscopyarea i2c_algo_bit dca sysfillrect usbhid pcbc sysimgblt fb_sys_fops aesni_intel aes_x86_64 crypto_simd ahci glue_helper ptp hid mxm_wmi cryptd [ 3117.882309] drm libahci megaraid_sas pps_core mdio wmi [ 3117.882315] CPU: 6 PID: 0 Comm: swapper/6 Tainted: GW 4.15.0-30-generic #32-Ubuntu [ 3117.882316] Hardware name: Intel Corporation S2600WT2R/S2600WT2R, BIOS SE5C610.86B.01.01.0022.062820171903 06/28/2017 [ 3117.882319] RIP: 0010:nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882320] RSP: 0018:88d77f3839f0 EFLAGS: 00010246 [ 3117.882322] RAX: RBX: 90de4000 RCX: 1924 [ 3117.882323] RDX: RSI: 88d749ae6400 RDI: 88d655e96600 [ 3117.882324] RBP: 88d77f383a68 R08: 88d7493dc000 R09: 0018 [ 3117.882324] R10: 0001 R11: 88e770dedc00 R12: 88d655e96600 [ 3117.882325] R13: 88d77f383ae8 R14: 88d7799ff200 R15: 88d7493dc000 [ 3117.882327] FS: () GS:88d77f38() knlGS: [ 3117.882327] CS: 0010 DS: ES: CR0: 80050033 [ 3117.882328] CR2: ff600400 CR3: 00193de0a002 CR4: 003606e0 [ 3117.882329] DR0: DR1: DR2: [ 3117.882330] DR3: DR6: fffe0ff0 DR7: 0400 [ 3117.882331] Call Trace: [ 3117.882332] [ 3117.882337] ? nf_nat_ipv4_fn+0x15b/0x200 [nf_nat_ipv4] [ 3117.882339] nf_nat_ipv4_out+0xc5/0xe0 [nf_nat_ipv4] [ 3117.882342] iptable_nat_ipv4_out+0x15/0x20 [iptable_nat] [ 3117.882347] nf_hook_slow+0x48/0xc0 [ 3117.882353] ip_output+0xd2/0xe0 [ 3117.882355] ? ip_fragment.constprop.44+0x80/0x80 [ 3117.882357] ip_forward_finish+0x49/0x70 [ 3117.882359] ip_forward+0x366/0x440 [ 3117.882361] ? ip_frag_mem+0x20/0x20 [ 3117.882362] ip_rcv_finish+0x129/0x430 [ 3117.882364] ip_rcv+0x28f/0x3a0 [ 3117.882366] ? inet_del_offload+0x40/0x40 [ 3117.882372] __netif_receive_skb_core+0x432/0xb40 [ 3117.882379] ? handle_edge_irq+0x7c/0x190 [ 3117.882384] ? irq_exit+0x67/0xc0 [ 3117.882391] ? do_IRQ+0x82/0xd0 [ 3117.882393] __netif_receive_skb+0x18/0x60 [ 3117.882395] ? __netif_receive_skb+0x18/0x60 [ 3117.882397] netif_receive_skb_internal+0x37/0xd0 [ 3117.882398] napi_gro_receive+0xc5/0xf0 [ 3117.882407] ixgbe_clean_rx_irq+0x446/0xe30 [ixgbe] [ 3117.882411] ixgbe_poll+0x256/0x710 [ixgbe] [ 3117.882413] ? do_IRQ+0x82/0xd0 [ 3117.882415] net_rx_action+0x140/0x3a0 [ 3117.882418] __do_softirq+0xdf/0x2b2 [ 3117.882419] irq_exit+0xb6/0xc0 [ 3117.882421] do_IRQ+0x82/0xd0 [ 3117.882423] common_interrupt+0x84/0x84 [ 3117.882424] [ 3117.882427] RIP:
[Kernel-packages] [Bug 1786752] Re: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release)
** Attachment added: "lspci" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1786752/+attachment/5174666/+files/lspci-vnvn.log -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1786752 Title: Reference counter issue in 4.15 (nf_xfrm_me_harder / dst_release) Status in linux package in Ubuntu: Confirmed Bug description: Since upgrading from 14.04 to 18.04, we see very frequent warnings about negative refcnts in dst_release: [ 3117.882227] WARNING: CPU: 6 PID: 0 at /build/linux-I4R9hO/linux-4.15.0/include/net/dst.h:256 nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882229] Modules linked in: xt_policy cls_u32 sch_sfq ip_vti ip_tunnel authenc echainiv xfrm6_mode_tunnel xfrm4_mode_tunnel xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key nfnetlink_queue nfnetlink_log sch_htb xt_TPROXY xt_multiport veth nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo br_netfilter bridge overlay macvlan 8021q garp mrp stp llc bonding algif_skcipher af_alg xt_hl ip6t_rt nf_conntrack_ipv6 nf_defrag_ipv6 ip6t_REJECT nf_reject_ipv6 xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 dm_crypt iptable_nat nf_nat_ipv4 xt_DSCP xt_dscp xt_mark iptable_mangle xt_limit xt_tcpudp xt_addrtype intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp nf_conntrack_ipv4 nf_defrag_ipv4 kvm_intel xt_conntrack kvm joydev input_leds ipt_REJECT nf_reject_ipv4 ftdi_sio usbserial ipmi_si [ 3117.882265] irqbypass intel_cstate mei_me mei ioatdma acpi_pad intel_rapl_perf shpchp ipmi_devintf ipmi_msghandler acpi_power_meter lpc_ich mac_hid ip6table_filter ip6_tables nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp sch_fq_codel nf_conntrack iptable_filter ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mgag200 ttm crct10dif_pclmul igb ixgbe crc32_pclmul drm_kms_helper hid_generic ghash_clmulni_intel syscopyarea i2c_algo_bit dca sysfillrect usbhid pcbc sysimgblt fb_sys_fops aesni_intel aes_x86_64 crypto_simd ahci glue_helper ptp hid mxm_wmi cryptd [ 3117.882309] drm libahci megaraid_sas pps_core mdio wmi [ 3117.882315] CPU: 6 PID: 0 Comm: swapper/6 Tainted: GW 4.15.0-30-generic #32-Ubuntu [ 3117.882316] Hardware name: Intel Corporation S2600WT2R/S2600WT2R, BIOS SE5C610.86B.01.01.0022.062820171903 06/28/2017 [ 3117.882319] RIP: 0010:nf_xfrm_me_harder+0x127/0x140 [nf_nat] [ 3117.882320] RSP: 0018:88d77f3839f0 EFLAGS: 00010246 [ 3117.882322] RAX: RBX: 90de4000 RCX: 1924 [ 3117.882323] RDX: RSI: 88d749ae6400 RDI: 88d655e96600 [ 3117.882324] RBP: 88d77f383a68 R08: 88d7493dc000 R09: 0018 [ 3117.882324] R10: 0001 R11: 88e770dedc00 R12: 88d655e96600 [ 3117.882325] R13: 88d77f383ae8 R14: 88d7799ff200 R15: 88d7493dc000 [ 3117.882327] FS: () GS:88d77f38() knlGS: [ 3117.882327] CS: 0010 DS: ES: CR0: 80050033 [ 3117.882328] CR2: ff600400 CR3: 00193de0a002 CR4: 003606e0 [ 3117.882329] DR0: DR1: DR2: [ 3117.882330] DR3: DR6: fffe0ff0 DR7: 0400 [ 3117.882331] Call Trace: [ 3117.882332] [ 3117.882337] ? nf_nat_ipv4_fn+0x15b/0x200 [nf_nat_ipv4] [ 3117.882339] nf_nat_ipv4_out+0xc5/0xe0 [nf_nat_ipv4] [ 3117.882342] iptable_nat_ipv4_out+0x15/0x20 [iptable_nat] [ 3117.882347] nf_hook_slow+0x48/0xc0 [ 3117.882353] ip_output+0xd2/0xe0 [ 3117.882355] ? ip_fragment.constprop.44+0x80/0x80 [ 3117.882357] ip_forward_finish+0x49/0x70 [ 3117.882359] ip_forward+0x366/0x440 [ 3117.882361] ? ip_frag_mem+0x20/0x20 [ 3117.882362] ip_rcv_finish+0x129/0x430 [ 3117.882364] ip_rcv+0x28f/0x3a0 [ 3117.882366] ? inet_del_offload+0x40/0x40 [ 3117.882372] __netif_receive_skb_core+0x432/0xb40 [ 3117.882379] ? handle_edge_irq+0x7c/0x190 [ 3117.882384] ? irq_exit+0x67/0xc0 [ 3117.882391] ? do_IRQ+0x82/0xd0 [ 3117.882393] __netif_receive_skb+0x18/0x60 [ 3117.882395] ? __netif_receive_skb+0x18/0x60 [ 3117.882397] netif_receive_skb_internal+0x37/0xd0 [ 3117.882398] napi_gro_receive+0xc5/0xf0 [ 3117.882407] ixgbe_clean_rx_irq+0x446/0xe30 [ixgbe] [ 3117.882411] ixgbe_poll+0x256/0x710 [ixgbe] [ 3117.882413] ? do_IRQ+0x82/0xd0 [ 3117.882415] net_rx_action+0x140/0x3a0 [ 3117.882418] __do_softirq+0xdf/0x2b2 [ 3117.882419] irq_exit+0xb6/0xc0 [ 3117.882421] do_IRQ+0x82/0xd0 [ 3117.882423] common_interrupt+0x84/0x84 [ 3117.882424] [ 3117.882427] RIP:
