On 11/1/24 10:37, Jan Beulich wrote:
On 11.01.2024 03:29, Patrick Plenefisch wrote:
Hi,
I ran into a memory allocation issue, I think. It is the same as
https://github.com/QubesOS/qubes-issues/issues/8791 and I saw at the end it
was recommended (by marmarek) that the issue reporter forward the issue to
this list. I searched the list, but as I didn't see it in the list already,
I'm doing that now.
Hardware:
I have an AMD Threadripper 7960X on a ASRock TRX50 WS motherboard. The
Qubes reporter had a Threadripper 3970X on an ASUS Prime TRX40-Pro
Motherboard. I saw a 3rd issue report of a similar issue on another
Threadripper, so I think this may be Threadripper-specific.
Setup:
The QuebesOS reporter was using Qubes Installer.
My install was that I had a fresh install of Debian 12 (no gui), and then
did `apt install xen-system-amd64` and rebooted.
The issue:
Any boot of Xen on the hardware results in a halted machine. When
monitoring the logs with `vga=,keep`, we get:
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
(XEN) Freed 644kB init memory
mapping kernel into physical memory
about to get started…
xen hypervisor allocated kernel memory conflicts with E820
So first of all (the title doesn't say it) this is a Linux Dom0 issue.
Whether or not needing addressing in Xen is unknown at this point.
(XEN) Hardware Dom0 halted: halting machine
None of the settings I or the Qubes reporter have tried have been able to
get past this failure.
I am happy to provide debugging support.
Well, the crucial piece of data initially is going to be: What's the
E820 map Xen gets to see, what's the E820 map Dom0 gets to see, and
what address range is the conflict detected for? The first question
is possible to answer by supplying a serial log. The second question
likely means adding some debugging code to either Xen or Linux. The
answer to third question may be possible to infer from the other
data, but would likely be better to obtain explicitly by adjusting /
amending the message Linux emits.
We 've already hit similar issue because xen doesn't take into account
the reserved memory regions when loading the dom0 kernel (even if it is
relocatable). It can be worked around by changing accordingly
CONFIG_PHYSICAL_START in kernel config.
Let me provide more details on how to get the info Jan requested:
1) in the xen cmdline add: e820-verbose=true console_to_ring
2) in the dom0 kernel cmdline add: earlyprintk=xen
3) change the xen log message emitted by the linux kernel to print the
conflicting address, like below
diff --git a/arch/x86/xen/setup.c b/arch/x86/xen/setup.c
index cfa99e8f054b..ad88b700d58e 100644
--- a/arch/x86/xen/setup.c
+++ b/arch/x86/xen/setup.c
@@ -717,7 +717,7 @@ static void __init xen_reserve_xen_mfnlist(void)
xen_relocate_p2m();
memblock_phys_free(start, size);
}
-
+void xen_raw_printk(const char *fmt, ...);
/**
* xen_memory_setup - Hook for machine specific memory setup.
**/
@@ -853,7 +853,8 @@ char * __init xen_memory_setup(void)
*/
if (xen_is_e820_reserved(__pa_symbol(_text),
__pa_symbol(__bss_stop) - __pa_symbol(_text))) {
- xen_raw_console_write("Xen hypervisor allocated kernel
memory conflicts with E820 map\n");
+ xen_raw_printk("Xen hypervisor allocated kernel memory
conflicts with E820 map: %#lx - %#lx\n",
+ __pa_symbol(_text),
__pa_symbol(__bss_stop));
BUG();
}
Jan
