Package: src:linux
Version: 6.1.192-2
Severity: critical
Justification: breaks the whole system

Dear Maintainer,

After the ext4 regression in 6.1.192-1 (#1154101, #1154151), I upgraded
to 6.1.192-2 (linux-image-6.1.0-56-amd64). The ext4 mballoc crash is gone
(the create files / drop_caches / delete reproducer passes), but the system
still locked up about 9 minutes after boot, with a different oops in the
page cache path.

* Hardware / setup
  - Intel DN2800MT (Atom N2800), 2 GB RAM, single SATA HDD
  - ext4 root filesystem, swap partition
  - Docker (overlay2) workload: MariaDB, PostgreSQL, Apache/PHP,
    reverse proxy, file sync service, WireGuard

* Kernel history on this machine
  - 6.1.0-54-amd64 (6.1.190-1): stable, 8+ hours with the same
    workload, currently used as workaround
  - 6.1.0-55-amd64 (6.1.192-1): lockups from a few minutes to ~4.5 hours
    after boot (consistent with #1154101)
  - 6.1.0-56-amd64 (6.1.192-2): oops below ~9 minutes after boot

* Symptoms on 6.1.192-2
  - Oops in xas_load called from filemap_map_pages, in a page fault from
    an apache2 process
  - The task dies inside an RCU read-side critical section
    ("Voluntary context switch within RCU read-side critical section!")
    and then blocks in exit_mmap on the mmap rwsem
  - Followed by repeated RCU expedited stalls on that task, every minute
  - All disk writes then hang, SSH login hangs right after authentication,
    the machine still answers ping; a hard reset is required

My guess (not verified) is another NULL vs ERR_PTR mismatch introduced by
the folio backports, similar to the ext4 one, this time in the page cache
fault-around path.

* Oops (hostname removed)

BUG: kernel NULL pointer dereference, address: 0000000000000000
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 5fd4067 P4D 5fd4067 PUD 0
Oops: 0000 [#1] PREEMPT SMP NOPTI
CPU: 2 PID: 2537 Comm: apache2 Not tainted 6.1.0-56-amd64 #1  Debian
6.1.192-2
Hardware name:  /DN2800MT, BIOS MTCDT10N.86A.0158.2012.0808.1446 08/08/2012
RIP: 0010:xas_load+0x26/0x40
Code: 00 00 00 00 e8 3b ff ff ff 48 89 c2 83 e2 03 48 83 fa 02 75 08 48 3d
00 10 00 00 77 05 c3 cc cc cc cc 0f b6 4f 10 48 8d 70 fe <38> 48 fe 72 ee
e8 90 fd ff ff 80 3e 00 75 d0 c3 cc cc cc cc 66 0f
RSP: 0000:ffffd2d0c133bce8 EFLAGS: 00010202
RAX: 0000000000000002 RBX: ffffd2d0c133bd40 RCX: 0000000000000048
RDX: 0000000000000003 RSI: 0000000000000000 RDI: fffff68d806f1f80
RBP: ffff8cc9cac91778 R08: 000000004056f000 R09: 000000004856efff
R10: 00000000000008c0 R11: 00000000000008b1 R12: 00000000000008c0
R13: fffff68d806f1fb4 R14: ffff8cc9cac91778 R15: ffff8cc9c5fd4008
FS:  00007ff390e38c00(0000) GS:ffff8cca3c100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 000000000612c000 CR4: 00000000000006e0
Call Trace:
 <TASK>
 ? filemap_map_pages+0xb0/0x720
 ? do_fault+0x1a4/0x410
 ? __handle_mm_fault+0x660/0xfa0
 ? handle_mm_fault+0xdb/0x2d0
 ? do_user_addr_fault+0x191/0x570
 ? exc_page_fault+0x70/0x170
 ? asm_exc_page_fault+0x22/0x30
 </TASK>
Modules linked in: wireguard libchacha20poly1305 chacha_x86_64
poly1305_x86_64 curve25519_x86_64 libcurve25519_generic libchacha
ip6_udp_tunnel udp_tunnel nf_conntrack_netlink xt_nat xt_tcpudp veth
xt_conntrack xt_MASQUERADE bridge stp llc xfrm_user xfrm_algo xt_set ip_set
nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_addrtype
ipt_REJECT nf_reject_ipv4 xt_multiport nft_compat nf_tables nfnetlink
overlay cfg80211 rfkill isofs cdrom binfmt_misc intel_powerclamp
drm_kms_helper coretemp evdev sha512_ssse3 sha512_generic ppdev iTCO_wdt
sha256_ssse3 sha1_ssse3 i2c_algo_bit at24 intel_pmc_bxt iTCO_vendor_support
parport_pc watchdog parport sg button acpi_cpufreq drm dm_mod efi_pstore
fuse loop configfs ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 btrfs
blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy
async_pq async_xor async_tx xor raid6_pq libcrc32c crc32c_generic raid1
raid0 multipath linear md_mod sd_mod t10_pi crc64_rocksoft crc64 crc_t10dif
crct10dif_generic crct10dif_common ahci libahci libata scsi_mod e1000e
uhci_hcd ehci_pci ehci_hcd usbcore scsi_common i2c_i801 i2c_smbus lpc_ich
usb_common video wmi
CR2: 0000000000000000
---[ end trace 0000000000000000 ]---

------------[ cut here ]------------
Voluntary context switch within RCU read-side critical section!
WARNING: CPU: 2 PID: 2537 at kernel/rcu/tree_plugin.h:320
rcu_note_context_switch+0x4ee/0x650
CPU: 2 PID: 2537 Comm: apache2 Tainted: G      D            6.1.0-56-amd64
#1  Debian 6.1.192-2
Hardware name:  /DN2800MT, BIOS MTCDT10N.86A.0158.2012.0808.1446 08/08/2012
RIP: 0010:rcu_note_context_switch+0x4ee/0x650
Call Trace:
 <TASK>
 __schedule+0xae/0xa00
 schedule+0x5a/0xd0
 rwsem_down_write_slowpath+0x315/0x6f0
 exit_mmap+0xf6/0x310
 __mmput+0x3e/0x130
 do_exit+0x326/0xb10
 ? do_user_addr_fault+0x191/0x570
 make_task_dead+0x8d/0x90
 rewind_stack_and_make_dead+0x17/0x20
 </TASK>

Then, every minute until the hard reset:
rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P2537 }
85122 jiffies s: 1505 root: 0x0/T

So far this happened once on 6.1.192-2. I can provide the full kernel log
of that boot or test patches/kernels if needed.

Thanks for your work.

-- 
Regards.
Max

Reply via email to