[osv-dev] [PATCH] x64: fix direct kernel mode boot on qemu

[Waldemar Kozaczuk]

I cannot pretend I fully understand all details, but at some point
the gnu assembler (possibly starting with 2.36) on Fedora started
generating ELF notes named note.gnu.property.* that somehow when
linked produce an loader.elf that both readelf and QEMU report
has malformed notes. The readelf reports this:

readelf -Wn build/release/loader-strippef.elf
Displaying notes found in: .note
  OwnerData sizeDescription
  GNU  0x0020   NT_GNU_PROPERTY_TYPE_0
Properties: x86 feature used: x86, x87, MMX, XMM, FXSR, XSAVE, x86 ISA u
  Xen  0x0008   NT_VERSION (version)   description 
data: 00 10 31 40 00 00 00 00 
  Xen  0x0008   NT_ARCH (architecture) description 
data: 00 00 31 40 00 00 00 00 
  Xen  0x0008   Unknown note type: (0x0003)
description data: 00 00 00 40 00 00 00 00 
  Xen  0x0008   Unknown note type: (0x0005)
description data: 78 65 6e 2d 33 2e 30 00 
  Xen  0x0004   Unknown note type: (0x0006)
description data: 6f 73 76 00 
  ?.?  0x0007   Unknown note type: (0x006e6558)
description data: 04 00 00 00 08 00 00 
readelf: build/release/loader-stripped.elf: Warning: note with invalid namesz 
and/or descsz found at offset 0xc0
readelf: build/release/loader-stripped.elf: Warning:  type: 0x656e6567, 
namesize: 0x0008, descsize: 0x006e6558, alignment: 8

QEMU on other hand fails to run OSv in the direct kernel mode like so:
./script/run.py -k
qemu-system-x86_64: Error loading uncompressed kernel without PVH ELF Note
qemu failed.

The logic in QEMU to load kernel in direct mode relies on the ELF note
XEN_ELFNOTE_PHYS32_ENTRY that specifies an address of hvm_xen_start used
to jump to when booting OSv. This ELF note as well as other XEN notes
are defined in assembly in arch/x64/entry-xen.S. Because something is
wrong with those notes, QEMU can not boot OSv.

After some research I realized that the entry-xen object file has one
extra note - GNU in .note.gnu.property section:

readelf -nW build/release/arch/x64/entry-xen.o 

Displaying notes found in: .note.xen
  OwnerData sizeDescription
  Xen  0x0008   NT_VERSION (version)   description 
data: 00 10 00 00 00 00 00 00 
  Xen  0x0008   NT_ARCH (architecture) description 
data: 00 00 00 00 00 00 00 00 
  Xen  0x0008   Unknown note type: (0x0003)
description data: 00 00 00 40 00 00 00 00 
  Xen  0x0008   Unknown note type: (0x0005)
description data: 78 65 6e 2d 33 2e 30 00 
  Xen  0x0004   Unknown note type: (0x0006)
description data: 6f 73 76 00 
  Xen  0x0004   Unknown note type: (0x0007)
description data: 3f 2e 3f 00 
  Xen  0x0008   Unknown note type: (0x0008)
description data: 67 65 6e 65 72 69 63 00 
  Xen  0x0016   Unknown note type: (0x000a)
description data: 21 77 72 69 74 61 62 6c 65 5f 70 61 67 65 5f 74 61 62 6c 65 
73 00 
  Xen  0x0004   Unknown note type: (0x000b)
description data: 79 65 73 00 
  Xen  0x0008   Unknown note type: (0x0012)
description data: 1c 10 00 c0 ff ff ff ff 

Displaying notes found in: .note.gnu.property
  OwnerData sizeDescription
  GNU  0x0020   NT_GNU_PROPERTY_TYPE_0
Properties: x86 ISA used: , x86 feature used: x86

I have also found somebody reported similar issue -
https://sourceware.org/bugzilla/show_bug.cgi?id=27753 - which can be
fixed by adding DISCARD to the linker script to filter out unwanted
section from the target ELF.

Besides adding DISCARD rule to the linker script this patch also
cleans some issue with the assembly code to add the elf notes in entry-xen.S.

Signed-off-by: Waldemar Kozaczuk 
---
 arch/x64/entry-xen.S | 4 +++-
 arch/x64/loader.ld   | 5 +
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/arch/x64/entry-xen.S b/arch/x64/entry-xen.S
index 451d35e4..0146a799 100644
--- a/arch/x64/entry-xen.S
+++ b/arch/x64/entry-xen.S
@@ -8,6 +8,7 @@
 
 #define elfnote(type, valtype, value) \
 .pushsection .note.xen, "a", @note; \
+.align 4; \
 .long 2f-1f; \
 .long 3f-2f; \
 .long type; \
@@ -17,7 +18,8 @@
 2: \
 valtype value; \
 3: \
-.align 4
+.align 4; \
+.popsection
 
 #define elfnote_val(type, value) elfnote(type, .quad, value)
 #define elfnote_str(type, value) elfnote(type, .asciz, value)
diff --git a/arch/x64/loader.ld b/arch/x64/loader.ld
index debdbf15..65f88d5b 100644
--- a/arch/x64/loader.ld
+++ b/arch/x64/loader.ld
@@ -57,6 +57,11 @@ SECTIONS
 HIDDEN(memcpy_decode_end = .);
 KEEP(*(.mem

[osv-dev] [PATCH] iconv: use stubs if iconv functions not used by an app

[Waldemar Kozaczuk]

The GCC libstdc++ commit 
https://github.com/gcc-mirror/gcc/commit/c0ace69ec677d1f85f6a433c8fae2d4df6f75714
which I believe fixes the bug 
https://gcc.gnu.org/bugzilla/show_bug.cgi?format=multiple&id=64132
makes the C++ standard library depend on libc iconv* functions. The
implementation of these functions in musl employs many large arrays
used to translate characters from one encoding to another and inflates
the kernel by around 156K. After some research to see where exactly
the related code - _M_initialize_numpunct() and _M_initialize_moneypunct()
is executed it turns out that OSv itself internally does not trigger
the code path leading to the execution of iconv functions because
it uses a default C locale. This observation in general would be a mute
point as by default OSv provides iconv* functions as part of libc anyway so C++ 
standard
library using it does not really change anything. Except when one builds
custom kernel to remove all symbols but what application specifically
needs like so (for details see d19ccb1cde100ab4a5c8e6db9a0d69560cabbd04):

./scripts/build fs=rofs conf_hide_symbols=1 image=native-example \
 conf_version_script=build/last/app_version_script

In this case we would like to optimize kernel size and replace
real iconv functions which would be referenced by _M_initialize_numpunct and
_M_initialize_moneypunct with simple stubs in libc/locale/iconv_stubs.cc.
This optimization allows us to reduce kernel size by another 156K.

Signed-off-by: Waldemar Kozaczuk 
---
 Makefile   | 10 --
 libc/locale/iconv_stubs.cc | 26 ++
 2 files changed, 34 insertions(+), 2 deletions(-)
 create mode 100644 libc/locale/iconv_stubs.cc

diff --git a/Makefile b/Makefile
index d0971a16..dd495e83 100644
--- a/Makefile
+++ b/Makefile
@@ -1142,8 +1142,6 @@ musl += locale/catgets.o
 libc += locale/catopen.o
 libc += locale/duplocale.o
 libc += locale/freelocale.o
-musl += locale/iconv.o
-musl += locale/iconv_close.o
 libc += locale/intl.o
 libc += locale/langinfo.o
 musl += locale/localeconv.o
@@ -2085,8 +2083,16 @@ endif
 endif
 linker_archives_options = --no-whole-archive $(libstdc++.a) $(libgcc.a) 
$(libgcc_eh.a) $(boost-libs) \
   --exclude-libs libstdc++.a --gc-sections
+ifneq ($(shell grep -c iconv $(out)/version_script),0)
+musl += locale/iconv.o
+musl += locale/iconv_close.o
+else
+libc += locale/iconv_stubs.o
+endif
 else
 linker_archives_options = --whole-archive $(libstdc++.a) $(libgcc_eh.a) 
$(boost-libs) --no-whole-archive $(libgcc.a)
+musl += locale/iconv.o
+musl += locale/iconv_close.o
 endif
 
 $(out)/default_version_script: exported_symbols/*.symbols 
exported_symbols/$(arch)/*.symbols
diff --git a/libc/locale/iconv_stubs.cc b/libc/locale/iconv_stubs.cc
new file mode 100644
index ..4c7b7d7d
--- /dev/null
+++ b/libc/locale/iconv_stubs.cc
@@ -0,0 +1,26 @@
+/*
+ * Copyright (C) 2022 Waldemar Kozaczuk
+ *
+ * This work is open source software, licensed under the terms of the
+ * BSD license as described in the LICENSE file in the top-level directory.
+ */
+
+#include 
+#include 
+#include 
+
+iconv_t iconv_open(const char *to, const char *from) {
+WARN_STUBBED();
+errno = EINVAL;
+return (iconv_t)-1;
+}
+
+size_t iconv(iconv_t cd, char **in, size_t *inb, char **out, size_t *outb) {
+WARN_STUBBED();
+return 0l;
+}
+
+int iconv_close(iconv_t cd) {
+WARN_STUBBED();
+return 0l;
+}
-- 
2.35.1

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/20220627185608.37930-1-jwkozaczuk%40gmail.com.