Hi,
On 16/12/2020 14:34, Jan Kiszka wrote:
> On 10.12.20 16:20, Bram Hooimeijer wrote:
>> Dear Jailhouse community,
>>
>> Over the last days I have been trying to get Jailhouse running on real
>> hardware. I ran into quite some issues, but managed to get it running
>> eventually.
>> For some issues, I was able to find a solution somewhere in the mailing
>> archives, but not for all of them. So I thought I'd share my experiences
>> here for future references.
>> There's also some solutions which are not entirely clear. If you have any
>> reference on that it would be welcome, but I understand these might be very
>> system specific as well.
>>
>> ** Installing Jailhouse **
>> * Installing Jailhouse on generic Ubuntu (2.10) gives the following error:
>>> ERROR: modpost: "lapic_timer_period"
>>> [/data/ecseqm/jailhouse/510_siemens_jailhouse/driver/jailhouse.ko]
>>> undefined!
>>> ERROR: modpost: "__get_vm_area_caller"
>>> [/data/ecseqm/jailhouse/510_siemens_jailhouse/driver/jailhouse.ko]
>>> undefined!
>>> ERROR: modpost: "ioremap_page_range"
>>> [/data/ecseqm/jailhouse/510_siemens_jailhouse/driver/jailhouse.ko]
>>> undefined!
>> I believe some of the kernel symbols have been renamed in I believe 5.8. As
>> a solution, I switched to 2.04 LTS (Kernel 5.4), where it installed without
>> issues.
>>
>
> That used to work by luck for some kernels via ksymall, but that is
> disabled in other kernels for security reasons - and now even removed
> from upstream. You need some patches from the github.com/siemens/linux
> jailhouse-enabling/5.4 queue (or queues/jailhouse for the head queue,
> that's on git.kiszka.org).
>
>> ** Enabling Jailhouse
>> * enabling a compiled sysconfig.cell results in:
>>> JAILHOUSE_ENABLE: Invalid argument
>> with dmesg listing:
>>> jailhouse: Not a system configuration
>> This issue was already reported on the mailing list, but I'd like to note
>> that it was not a singular case. Switching from GCC-9 to GCC-7 solved the
>> issue for me too. I guess GCC-9 takes the freedom to move the header away
>> from the initial memory location, which results in Jailhouse failing to
>> verify the .cell to be a system configuration description.
>>
>
> If that isn't solved in current next, we should address it. The proper
> fix is moving away from gcc to a different way of compiling, but that's
> another story.
If it's the same issue that was reported in October ("Jailhouse in qemu and
ubuntu"), then maybe the attached patch could help.
Thanks,
Andrea
>
>> * enabling results in:
>>> FATAL: Unhandled MSR write: c8f
>> Ubuntu distributions have CONFIG_X86_CPU_RESCTRL=y kernel config set by
>> default. I do not know how widespread this is. Apparently, this conflicts
>> with jailhouse, as such a Linux kernel tends to write to 0xc8f:
>> IA32_PQR_ASSOC. Disabling rdt over the kernel commandline does not solve the
>> issue. Perhaps it might be useful to extend the hardware check to check for
>> this flag? Rebuilding the kernel without RESCTRL fixed the issue.
>>
>> * enabling results in:
>>> FATAL: Invalid MMIO/RAM read, addr: 0x0000000087a6e070 size: 8
>>> FATAL: Invalid MMIO/RAM write, addr: 0x0000000087a6e018 size: 0
>> These memory accesses are
>>> 86dff000-924fefff : Reserved
>>> 89f5b018-89f5b06f : APEI ERST
>>> 89f5b070-89f5d017 : APEI ERST
>> The APEI ERST are included by jailhouse by default, however the remainder of
>> the reserved region is not. Passing this to jailhouse solves the issue. Does
>> anybody have any idea where this region could be used for? I expect some
>> bios driver? Is there a way to detect which driver is interfering here?
>>
>> * Similarly, a violation was found while destroying a non-root cell:
>>> FATAL: Invalid PIO read, port: 500 size: 2
>> Which jailhouse identifies as:
>>> /* Port I/O: 0510-0515 : ACPI CPU throttle */
>>> PIO_RANGE(0x510, 0x6),
>> Again, passing the port solves the issue. Does anybody have a clue why the
>> CPU throttle is required to destroy a non-root cell? Is it an idea to
>> include these PIO ports by default in the config creation?
>>
>> The last issues were described to some extend in Jan's ELCE talk in 2016,
>> "Tutorial: Bootstrapping the Partitioning Hypervisor Jailhouse". I really
>> liked the introduction and think it is an excellent talk to get started with
>> Jailhouse.
>> However, it is not linked in e.g. the readme or the FAQ. This might be
>> beneficial.
>>
>> If anyone has any ideas about the PIO/Memory requirements, that would be
>> really welcome.
>>
>> Thanks for all the efforts put in Jailhouse.
>>
>
> Thanks for this detailed feedback! In general, we are happy to take
> contributions also to the README or other docs where you see gaps.
> That's best provided by new users as it's too easy for us oversee
> details that we know but that aren't sufficiently documented yet.
>
> Jan
>
--
You received this message because you are subscribed to the Google Groups
"Jailhouse" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jailhouse-dev/64c38e90-8dc5-9b2e-804d-3a098825f7d8%40tum.de.
>From 8dc5f24394f8e7bf9e08618a10a425b502a3319c Mon Sep 17 00:00:00 2001
From: Andrea Bastoni <[email protected]>
Date: Wed, 16 Dec 2020 16:57:57 +0100
Subject: [PATCH] configs, inmates: Makefile: remove .note.gnu.property section
during objcopy
It seems that the .note.gnu.property section is interfering with objcpy and
causes the "JHSYST" identification to be removed from the .cell.
Removing the section while copying the .o fixes the issue:
objcopy --version
GNU objcopy (GNU Binutils for Ubuntu) 2.34
readelf -a jailhouse/configs/x86/qemu-x86.o
...
Displaying notes found in: .note.gnu.property
Owner Data size Description
GNU 0x00000010 NT_GNU_PROPERTY_TYPE_0
Properties: x86 feature: IBT, SHSTK
hexdump -C jailhouse/configs/x86/qemu-x86.cell
00000000 04 00 00 00 10 00 00 00 05 00 00 00 47 4e 55 00 |............GNU.|
objcopy -O binary --remove-section=.note.gnu.property
jailhouse/configs/x86/qemu-x86.o jailhouse/configs/x86/qemu-x86.cell
00000000 4a 48 53 59 53 54 0d 00 01 00 00 00 00 00 00 3a |JHSYST.........:|
Maybe related to:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=906414
Signed-off-by: Andrea Bastoni <[email protected]>
---
configs/Makefile | 2 +-
inmates/Makefile | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/configs/Makefile b/configs/Makefile
index 513b0a9b..0e617e4e 100644
--- a/configs/Makefile
+++ b/configs/Makefile
@@ -23,7 +23,7 @@ ifneq ($(wildcard $(obj)/../include/jailhouse/config.h),)
KBUILD_CFLAGS += -include $(obj)/../include/jailhouse/config.h
endif
-OBJCOPYFLAGS := -O binary
+OBJCOPYFLAGS := -O binary --remove-section=.note.gnu.property
CONFIGS = $(shell cd $(src); ls $(SRCARCH)/*.c)
diff --git a/inmates/Makefile b/inmates/Makefile
index 7d3fafa0..a4ab7903 100644
--- a/inmates/Makefile
+++ b/inmates/Makefile
@@ -36,7 +36,7 @@ KBUILD_CFLAGS += -march=armv7ve
KBUILD_AFLAGS += -march=armv7ve
endif
-OBJCOPYFLAGS := -O binary
+OBJCOPYFLAGS := -O binary --remove-section=.note.gnu.property
# prior to 4.19
LDFLAGS += --gc-sections -T
# since 4.19
--
2.29.2
