On Mon, May 25, 2026 at 4:59 PM Sergey Bugaev <[email protected]> wrote:
> How does that work? Even if you don't use x29 as a frame pointer
> (which you normally do on AArch64, so I doubt that my GCC had that
> off), surely you still save/restore at least x30?
>
> Maybe that could still happen if zero_out_bss got inlined into
> c_boot_entry, and so did not have its x30 saved?
The inlining is right, zero_out_bss does get inlined into c_boot_entry.
But the LR overwrite is one frame deeper, in memset itself, which is
a leaf function and normally doesn't save x30.
I should have been more specific about the flag in my last reply. The
actual trigger is -mno-omit-leaf-frame-pointer, not -fno-omit-frame-pointer
alone, the general flag still lets GCC omit the prologue on leaves.
My nix cross-gcc has both on by default, and the leaf one is what
makes memset push x30. Same source, same compiler, only the flag
differs:
With -mno-omit-leaf-frame-pointer (nix default):
memset:
stp x29, x30, [sp, #-16]! ; saves x30 into the boot stack
<body: zeroes [__bss_start, __bss_end)>
ldp x29, x30, [sp], #16 ; reloads x30 = 0 (slot was wiped)
ret ; branches to PC=0
Without it (stock cross-gcc):
memset:
<body: zeroes [__bss_start, __bss_end)>
ret ; x30 stayed in the register
c_boot_entry has SP on the boot stack, the boot stack lives in .bss,
so the stp lands inside the about-to-be-wiped range. qemu cpu_reset
confirms it: PC=0, all GPRs 0, undefined-instruction loop at
vector_base + 0x200.
I tried this on your branch wip-aarch64 a845e3dc + the same
nix gcc, byte-identical fault. So the crash has been latent on
wip-aarch64 since the boot stack moved into .bss, it just needed a
toolchain that emits -mno-omit-leaf-frame-pointer to trip.
nix's cross-gcc is the unusual one for wrapping every invocation with
those flags, but the broader trend matters: distro build systems are
moving toward frame-pointer defaults for the packages they ship.
Fedora 38 added exactly the pair we're tripping to its RPM build
flags:
https://fedoraproject.org/wiki/Changes/fno-omit-frame-pointer
Ubuntu 24.04 LTS did the same for 64-bit platforms:
https://ubuntu.com/blog/ubuntu-performance-engineering-with-frame-pointers-by-default
With PATCH 08 the boot stack sits above __bss_end, so the saved x30
lands outside the wipe range and ret returns cleanly. With the flag
off, memset doesn't push x30 in the first place. Either way, no
crash, regardless of which toolchain the next person uses. v2 will
rewrite the commit body around this mechanism instead of the
boot-route speculation.
Paulo
