On 2026/9/15 20:53, Breno Leitao wrote: > Problem: > ======== > > When a page is hard-offlined due to an uncorrectable memory error (multi > bit ECC), memory_failure() sets PG_hwpoison, unmaps it, removes it from > the buddy allocator. This information is not carried to the next kernel > that is kexeced. The new kernel kexecs and trip over that bad memory > bank _again_. > > Why now: > ======== > > Several industry trends make this increasingly important: > > 1) DRAM is getting more expensive > 2) soldered / on-package memory (LPDDR, HBM) is becoming more common, so a > failing part can no longer simply be swapped; > 3) memory is kept in service far longer (at Meta, DRAM lifetime is being > drastically extended). > 4) It is more and more common to kexec instead of full reboot > 5) Increase of memory per system with CXL > > What was done already: > ====================== > > In order make linux deal better with the problem above, I've done > already fixed a bunch of stuff in this area, such as: > > 1) Panic on unrecoverable errors, instead of "printk and carry over": > https://lore.kernel.org/all/[email protected]/ > > 2) Respect poisoned memory at kexec time > https://lore.kernel.org/all/[email protected]/ > > Now, the natural follow up is to carry the poisoned memory information > to the next kexec kernel, avoiding tripping over a known "bad page". > > Proposed Solution: > ================== > > Carry the poisoned frames to the next kernel in a new EFI configuration > table, LINUX_EFI_POISONED_MEMORY. > > The table is a bitmap with one bit per 2MB of physical memory, modeled > on LINUX_EFI_UNACCEPTED_MEMORY. The stub sizes it from the EFI memory > map and installs it empty while boot services are up -- a running kernel > cannot install a configuration table, it can only flip bits -- and a > table inherited from an earlier boot is reused as-is. That is one > fixed-size allocation, 64KB per TiB of the span the memory map describes, > with no list to grow at runtime and no chain to trust at parse time. > > The mechanism is architecture independent, so x86 and arm64 use the same > code. > > Each hard offline sets the bit for its unit. Soft-offlined pages are not > recorded: they are still functional, and were offlined predictively. > > The next kernel takes the table into use from efi_config_parse_tables(), > which vets the inherited header and hands the table's pages to memblock. > It is EFI ACPI reclaim memory, which x86 leaves out of memblock and so > out of the direct map; the unaccepted memory table is handled the same > way for the same reason. > > The frames themselves are poisoned in __free_pages_core(), as each block > reaches the buddy allocator. That is the one point every producer passes > through -- memblock_free_pages() early, deferred_free_pages() once the > deferred struct pages are up, and generic_online_page() at any later > hotplug -- and it is already where the unaccepted memory table is > consulted. The recorded frames are held back and the rest of the block is > freed, so they never enter the allocator instead of being taken back out > of it. They end up in the state a frame poisoned by this kernel would be > in, so everything that already understands PG_hwpoison covers them -- > including the kexec segment placement check from the series linked above, > which keeps the segments a kexec places off these frames. > > Granularity is the trade-off: one bad 4KB frame costs a whole 2MB unit in > every later kernel of the chain. In exchange, a row or column fault -- > roughly a quarter of the DRAM faults reported in [1], and potentially > thousands of 4KB pages scattered over gigabytes -- collapses into a bit > or two. > > A bit is never cleared, which is a known limitation: it stands for a > whole unit, so an unpoison of one frame cannot tell whether the unit as a > whole is good again.
Thanks for your patches. I have a question about unpoison: If a bit is never cleared, after we do some memory-failure+unpoison tests, kexec will lose the tested memory without reboot? Thanks. .
