Hi all,
On 05.12.22 15:41, Ralf Ramsauer wrote:
[Adding Andrej]
Hi Karim,
On 05/12/2022 13:30, Karim Manaouil wrote:
Hi Ralf,
I am trying to boot a Linux guest (based on
configs/x86/linux-x86-demo.c).
I tried to debug and solve this issue for a while but no success so far.
The cell is created, and the guest starts booting, but then fails when
initialising the APIC. A write to an APIC register is intercepted by
Jailhouse and it decides that the guest is trying to set an invalid LVT
delivery mode. I checked the x86 documentation, it seems that it
should not be invalid, but I am not knowledgeable enough.
Sound familiar… I remember that Andrej and I have hit this one on an AMD
machine some years ago:
https://groups.google.com/g/jailhouse-dev/c/1wRKIiGN0GA/m/_p_NSIBpDwAJ
Andrej, do you know how we finally (quick?)-fixed that issue back then?
Did we really have hardware misbehavior? It's been a while, but as far
as I remember that was the case.
Afaik it was a CPU bug.
We had to apply the attached patch to the kernel to make it boot as guest.
Thanks,
Andrej
Thanks,
Ralf
Here is, pasted below, the log output from Jailhouse and the booting
guest kernel.
PS: I am following the steps in Documentation/non-root-linux.txt and
using the patched kernel in queues/jailhouse branch.
Cheers
Karim
---
Jailhouse:
Cell "linux-x86-demo" can be loaded
Started cell "linux-x86-demo"
CPU 2 received SIPI, vector 100
CPU 3 received SIPI, vector 100
FATAL: Setting invalid LVT delivery mode (reg 35, value 00000700)
FATAL: Unhandled APIC access, offset 848, is_write: 1
RIP: 0xffffffffaf84fb92 RSP: 0xffffffffb1003e80 FLAGS: 246
RAX: 0xffffffffaf84fb90 RBX: 0x0000000000000180 RCX: 0x0000000000000000
RDX: 0x0000000000000000 RSI: 0x0000000000000700 RDI: 0x0000000000000350
CS: 10 BASE: 0x0000000000000000 AR-BYTES: 29b EFER.LMA 1
CR0: 0x0000000080050033 CR3: 0x000000003ae0c000 CR4: 0x00000000000406b0
EFER: 0x0000000000001d01
Parking CPU 2 (Cell: "linux-x86-demo")
Cell "linux-x86-demo" can be loaded
Closing cell "linux-x86-demo"
Page pool usage after cell destruction: mem 2972/32211, remap
65703/131072
CPU 2 received SIPI, vector 96
CPU 3 received SIPI, vector 96
Linux demo guest (last few lines from kernel boot log on the serial
console):
init, 1176K bss, 73676K reserved, 0K cma-reserved)
[ 2.960440] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 3.040332] Dynamic Preempt: voluntary
[ 3.083151] rcu: Preemptible hierarchical RCU implementation.
[ 3.151749] rcu: RCU event tracing is enabled.
[ 3.205830] rcu: RCU restricting CPUs from NR_CPUS=64 to
nr_cpu_ids=2.
[ 3.284872] Trampoline variant of Tasks RCU enabled.
[ 3.345191] rcu: RCU calculated value of scheduler-enlistment delay
is 100 jiffies.
[ 3.436710] rcu: Adjusting geometry for rcu_fanout_leaf=16,
nr_cpu_ids=2
[ 3.518471] NR_IRQS: 4352, nr_irqs: 424, preallocated irqs: 0
[ 3.585483] rcu: srcu_init: Setting srcu_struct sizes based on
contention.
[ 3.667665] Console: colour dummy device 80x25
[ 3.720639] Enabling UART0 (port 0x3f8)
[ 3.766423] printk: console [ttyS0] enabled
[ 3.766423] printk: console [ttyS0] enabled
[ 3.866333] printk: bootconsole [earlyser0] disabled
[ 3.866333] printk: bootconsole [earlyser0] disabled
[ 3.985019] APIC: Switch to symmetric I/O mode setup
[ 4.046377] Switched APIC routing to physical flat.
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336. Is e buidheann
carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an Alba,
àireamh clàraidh SC005336.
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to jailhouse-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/b48f1252-a6b7-a183-7f37-080b95043ad6%40st.oth-regensburg.de.
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 0428ad289899..836436ce05ba 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1699,6 +1699,14 @@ static void setup_local_APIC(void)
value |= SPURIOUS_APIC_VECTOR;
apic_write(APIC_SPIV, value);
+ // HACK: some CPUs (e.g. the AMD Ryzen family) fail to reset LVT_LINT
+ // registers according to specification, so we help them do that
+ if (((value = apic_read(APIC_LVT0)) & APIC_LVT_MASKED) == 0)
+ apic_write(APIC_LVT0, value | APIC_LVT_MASKED);
+
+ if (((value = apic_read(APIC_LVT1)) & APIC_LVT_MASKED) == 0)
+ apic_write(APIC_LVT1, value | APIC_LVT_MASKED);
+
perf_events_lapic_init();
/*
