Public bug reported: While running automated tests of our application, we hit this kernel panic on average 1-2x in a 30-hour run on Ubuntu 24.04 only. 22.04 and 25.04 don't seem to hit the same problem. Our app is a container netowrking and security plugin for Kubernetes, the tests involve setting up lots of simulated clusters in docker containers and then tearing them down again. This test was run in BPF mode, so attaching BPF programs to the interfaces within the containers could be a factor.
The backtrace is not always the same; I'll post another example in a follow-on comment. [ 241.450488] BUG: kernel NULL pointer dereference, address: 0000000000000000 [ 241.458239] #PF: supervisor read access in kernel mode [ 241.463494] #PF: error_code(0x0000) - not-present page [ 241.468823] PGD 277e75067 P4D 0 [ 241.472159] Oops: Oops: 0000 [#1] SMP NOPTI 4 7t64i6m8e]= "CPU: 1 UID: 0 PID: 10106 Comm: runc Not tainted 6.14.0-1020-gcp #21~24.04.1-Ubuntu [ 241.485203] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/24/2025 [ 241.494532] RIP: 0010:l3mdev_fib_table_rcu+0x1e/0x60 [ 241.499605] Code: 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 48 85 ff 74 49 55 8b 07 48 89 e5 a9 00 00 04 00 74 1c 48 8b 87 f8 02 00 00 <48> 8b 00 48 85 c0 74 14 ff d0 0f 1f 00 5d 31 ff c3 cc cc cc cc a9 [ 241.518492] RSP: 0018:ff620cde40114b18 EFLAGS: 00010206 28-6039]T 1R2:07AX: 0000000000000000 RBX: ff26e4a10f0acc00 RCX: 0000000000000000 [ 241.531110] RDX: 00000000060011ac RSI: ff26e4a3e2c8d000 RDI: ff26e4a3e2c8d000 [ 241.538359] RBP: ff620cde40114b18 R08: 0000000000000000 R09: 0000000000000000 [ 241.545592] R10: 0000000000000000 R11: ff26e4a284bb4240 R12: 00000000060011ac [ 241.552827] R13: ff620cde40114cb0 R14: ff26e4a3e2c8d000 R15: ff26e4a284bb4240 [ 241.560583] FS: 00007922a5b69740(0000) GS:ff26e4a3efc80000(0000) knlGS:0000000000000000 [ 241.568773] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 241.574620] CR2: 0000000000000000 CR3: 00000002dec69004 CR4: 0000000000371ef0 [ 241.581858] Call Trace: [ 241.584405] <IRQ> [ 241.586520] inet_addr_type_dev_table+0x1a/0x50 [ 241.591167] icmp_route_lookup.constprop.0+0x182/0x450 [ 241.596412] ? enqueue_dl_entity+0x502/0xc50 [ 241.600786] ? psi_group_change+0x1f8/0x4c0 [ 241.605070] __icmp_send+0x47a/0x800 [ 241.608777] ipv4_send_dest_unreach+0xd3/0x1a0 [ 241.613323] ipv4_link_failure+0x12/0x70 [ 241.617409] arp_error_report+0x36/0x60 212365284]2 Z " neigh_invalidate+0x7d/0x140 [ 241.625400] ? __pfx_neigh_timer_handler+0x10/0x10 [ 241.630297] neigh_timer_handler+0x2df/0x330 [ 241.634670] ? __pfx_neigh_timer_handler+0x10/0x10 [ 241.639578] call_timer_fn+0x2c/0x120 [ 241.643879] __run_timers+0x22d/0x290 [ 241.648163] run_timer_softirq+0x93/0x120 [ 241.652276] handle_softirqs+0xe4/0x310 [ 241.656220] __irq_exit_rcu+0x10e/0x130 [ 241.660159] irq_exit_rcu+0xe/0x20 [ 241.663669] sysvec_apic_timer_interrupt+0xa0/0xc0 [ 241.668584] </IRQ> [ 241.670785] <TASK> [ 241.672984] asm_sysvec_apic_timer_interrupt+0x1b/0x20 [ 241.678225] RIP: 0010:__memcg_slab_post_alloc_hook+0x127/0x3e0 [ 241.684168] Code: 0f 82 c6 02 00 00 48 c7 c1 00 00 00 80 48 2b 0d 17 19 5e 01 48 01 ca 48 c1 ea 0c 48 c1 e2 06 48 03 15 f5 18 5e 01 48 8b 4a 08 <49> 89 d4 f6 c1 01 0f 85 d8 01 00 00 0f 1f 44 00 00 31 d2 41 80 7c 03f0o4 5m]s gR="SP: 0018:ff620cde8308b6f8 EFLAGS: 00000286 [ 241.708374] RAX: ff26e4a1da0cfcf8 RBX: ff620cde8308b770 RCX: ff26e4a0c01aac00 [ 241.715612] RDX: ff98c974086833c0 RSI: 0000000000000000 RDI: 0000000000000000 [ 241.722860] RBP: ff620cde8308b740 R08: 0000000000000000 R09: 0000000000000000 [ 241.730183] R10: 0000000000000001 R11: 0000000000000000 R12: ff620cde8308b770 r3 7f4ai2l6e]d R13: 0000000000000000 R14: ff26e4a0c0195900 R15: ff26e4a0c01aac00 to exit within [ 241.744673] ? __memcg_slab_post_alloc_hook+0xc9/0x3e0 [ 241.751306] ? vm_area_dup+0x4f/0x120 [ 241.755075] kmem_cache_alloc_noprof+0x35a/0x390 [ 241.759801] vm_area_dup+0x4f/0x120 i3g39n8a]l 9 -__split_vma+0x80/0x340 [ 241.767033] vma_modify+0xbf/0xe0 7h0e4 6f0]o r cevma_modify_flags+0xb1/0xf0 " container=7a3b[ 241.774437] mprotect_fixup+0x117/0x370 [ 241.779960] do_mprotect_pkey+0x3d3/0x5d0 [ 241.784164] __x64_sys_mprotect+0x1f/0x30 [ 241.788279] x64_sys_call+0xb8f/0x2650 [ 241.792135] do_syscall_64+0x7e/0x170 [ 241.795902] ? xas_load+0x17/0x100 f9e0cd50ea62edb8[ 241.799430] ? filemap_get_entry+0x66/0x1a0 [ 241.805121] ? __mod_memcg_lruvec_state+0xf4/0x250 [ 241.810015] ? __lruvec_stat_mod_folio+0x79/0xd0 [ 241.814741] ? set_ptes.isra.0+0x3b/0x90 [ 241.818763] ? set_pte_range+0xfb/0x210 [ 241.822704] ? queued_spin_unlock+0x9/0x20 [ 241.826901] ? finish_fault+0x41d/0x480 [ 241.830844] ? put_page+0x23/0xa0 4442a686]e 7 236? unlock_page+0x18/0x70 [ 241.837950] ? do_fault+0x1ea/0x220 b1f5c5509]3 f ae? handle_pte_fault+0x99/0x1f0 [ 241.847146] ? __handle_mm_fault+0x62c/0x770 [ 241.851517] ? __count_memcg_events+0xd8/0x1a0 [ 241.856061] ? count_memcg_events.constprop.0+0x2a/0x50 [ 241.861397] ? handle_mm_fault+0x1cd/0x2b0 d53349007c38 [ 241.865619] ? do_user_addr_fault+0x597/0x840 [ 241.871304] ? arch_exit_to_user_mode_prepare.isra.0+0x22/0x120 [ 241.877334] ? irqentry_exit_to_user_mode+0x2d/0x1d0 [ 241.882408] ? irqentry_exit+0x21/0x40 [ 241.886279] ? exc_page_fault+0x96/0x1a0 [ 241.890320] entry_SYSCALL_64_after_hwframe+0x76/0x7e [ 241.895479] RIP: 0033:0x7922a5bbedbb [ 241.899182] Code: 8d 66 0f 1f 44 00 00 f7 d8 89 05 00 35 01 00 48 c7 c0 ff ff ff ff eb 8b 0f 1f 80 00 00 00 00 f3 0f 1e fa b8 0a 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8d 0d d5 34 01 00 f7 d8 89 01 48 83 [ 241.918051] RSP: 002b:00007fffa0e949e8 EFLAGS: 00000206 ORIG_RAX: 000000000000000a [ 241.925724] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007922a5bbedbb [ 241.932961] RDX: 0000000000000001 RSI: 0000000000481000 RDI: 000056a9378a4000 [ 241.940194] RBP: 00007fffa0e94b10 R08: 00007922a5bd22e0 R09: 00007922a5926e90 [ 241.947475] R10: 00007922a5bd22e0 R11: 0000000000000206 R12: 00007fffa0e94ae0 [ 241.954717] R13: 0000000000000000 R14: 00007922a5bd22e0 R15: 0000000000000000 [ 241.961961] </TASK> [ 241.964252] Modules linked in: xt_CT ipt_REJECT nf_reject_ipv4 xt_connmark iptable_nat iptable_raw ip_set wireguard curve25519_x86_64 libchacha20poly1305 chacha_x86_64 poly1305_x86_64 ip6_udp_tunnel udp_tunnel libcurve25519_generic libchacha xt_mark iptable_mangle ip6table_mangle ip6_tables iptable_filter xt_comment veth tls ipip tunnel4 ip_tunnel xt_conntrack nft_chain_nat xt_MASQUERADE nf_nat bridge xfrm_user xfrm_algo xt_addrtype nft_compat nf_tables overlay 8021q garp mrp stp llc binfmt_misc nls_iso8859_1 polyval_clmulni polyval_generic ghash_clmulni_intel sha256_ssse3 sha1_ssse3 aesni_intel crypto_simd cryptd psmouse gve input_leds serio_raw sch_fq_codel dm_multipath nvme_fabrics efi_pstore dmi_sysfs virtio_rng ip_tables x_tables autofs4 [ 242.030135] CR2: 0000000000000000 [ 242.033555] ---[ end trace 0000000000000000 ]--- [ 242.172512] RIP: 0010:l3mdev_fib_table_rcu+0x1e/0x60 [ 242.177604] Code: 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 48 85 ff 74 49 55 8b 07 48 89 e5 a9 00 00 04 00 74 1c 48 8b 87 f8 02 00 00 <48> 8b 00 48 85 c0 74 14 ff d0 0f 1f 00 5d 31 ff c3 cc cc cc cc a9 [ 242.196494] RSP: 0018:ff620cde40114b18 EFLAGS: 00010206 [ 242.201831] RAX: 0000000000000000 RBX: ff26e4a10f0acc00 RCX: 0000000000000000 [ 242.209077] RDX: 00000000060011ac RSI: ff26e4a3e2c8d000 RDI: ff26e4a3e2c8d000 [ 242.216314] RBP: ff620cde40114b18 R08: 0000000000000000 R09: 0000000000000000 [ 242.223555] R10: 0000000000000000 R11: ff26e4a284bb4240 R12: 00000000060011ac [ 242.230818] R13: ff620cde40114cb0 R14: ff26e4a3e2c8d000 R15: ff26e4a284bb4240 [ 242.238061] FS: 00007922a5b69740(0000) GS:ff26e4a3efc80000(0000) knlGS:0000000000000000 [ 242.246497] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 242.252353] CR2: 0000000000000000 CR3: 00000002dec69004 CR4: 0000000000371ef0 [ 242.259600] Kernel panic - not syncing: Fatal exception in interrupt [ 242.266446] Kernel Offset: 0x37600000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff) [ 242.399948] Rebooting in 10 seconds.. ProblemType: Bug DistroRelease: Ubuntu 24.04 Package: linux-gcp 6.14.0-1020.21~24.04.1 ProcVersionSignature: User Name 6.14.0-1020.21~24.04.1-gcp 6.14.11 Uname: Linux 6.14.0-1020-gcp x86_64 AlsaDevices: total 0 crw-rw---- 1 root audio 116, 1 Dec 9 12:08 seq crw-rw---- 1 root audio 116, 33 Dec 9 12:08 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.28.1-0ubuntu3.8 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A CasperMD5CheckResult: unknown CloudArchitecture: x86_64 CloudBuildName: server CloudID: gce CloudName: gce CloudPlatform: gce CloudRegion: europe-west3 CloudSerial: 20251205 CloudSubPlatform: metadata (http://metadata.google.internal/computeMetadata/v1/) CurrentDmesg: Error: command ['dmesg'] failed with exit code 1: dmesg: read kernel buffer failed: Operation not permitted Date: Tue Dec 9 13:12:28 2025 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: Lsusb-t: Lsusb-v: Error: command ['lsusb', '-v'] failed with exit code 1: MachineType: Google Google Compute Engine PciMultimedia: ProcFB: ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-6.14.0-1020-gcp root=PARTUUID=957f60dc-de5a-4346-8819-8cd2a0c0fcfc ro console=ttyS0,115200 panic=-1 RelatedPackageVersions: linux-restricted-modules-6.14.0-1020-gcp N/A linux-backports-modules-6.14.0-1020-gcp N/A linux-firmware N/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux-gcp-6.14 UpgradeStatus: No upgrade log present (probably fresh install) acpidump: dmi.bios.date: 09/24/2025 dmi.bios.release: 1.0 dmi.bios.vendor: Google dmi.bios.version: Google dmi.board.asset.tag: 4887391B-10AF-DD33-3455-F220D8C99428 dmi.board.name: Google Compute Engine dmi.board.vendor: Google dmi.chassis.type: 1 dmi.chassis.vendor: Google dmi.modalias: dmi:bvnGoogle:bvrGoogle:bd09/24/2025:br1.0:svnGoogle:pnGoogleComputeEngine:pvr:rvnGoogle:rnGoogleComputeEngine:rvr:cvnGoogle:ct1:cvr:sku: dmi.product.name: Google Compute Engine dmi.sys.vendor: Google ** Affects: linux-gcp-6.14 (Ubuntu) Importance: Undecided Status: New ** Tags: amd64 apport-bug cloud-image noble -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2134472 Title: Kernel panic, NULL pointer in l3mdev_fib_table_rcu To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-gcp-6.14/+bug/2134472/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
