[osv-dev] [RFC] Make ZFS optional as a shared library

2021-12-10 Thread Waldemar Kozaczuk
ge. Fixes #1009 Signed-off-by: Waldemar Kozaczuk --- Makefile | 51 +++ bootfs.manifest.skel | 1 + bsd/init.cc | 7 --- bsd/porting/shrinker.cc | 30 +--- core/osv_c_wrappers.cc | 17 +++ core/pagecache.cc

[osv-dev] [PATCH] aarch64 linker: enforce some sections kept when garbage collection on

2021-12-06 Thread Waldemar Kozaczuk
collection on. This changes to not have any effect on the linker result when garbage collection is off - the default. Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/loader.ld | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/aarch64/loader.ld b/arch

[osv-dev] [PATCH] aarch64: add list files of exported symbols

2021-12-06 Thread Waldemar Kozaczuk
This patch adds files with lists of symbols to be exported by kernel. The files have been generated on Ubuntu 20.10 using the scripts/extract_symbols.sh. These files will be fed to the process to generate version script. Signed-off-by: Waldemar Kozaczuk --- .../aarch64/osv_ld-linux-aarch64.so

[osv-dev] [PATCH V2] scripts: add new generate_version_script.sh

2021-12-06 Thread Waldemar Kozaczuk
This patch adds new script that generates version script based on the symbol files from exported_symbols//*.symbols Comparing to the 1st version, this one will work when building both x64 and aarch64 versions of kernel. Signed-off-by: Waldemar Kozaczuk --- scripts/generate_version_script.sh

[osv-dev] [PATCH] makefile: enable garbage collection of code and data when hiding symbols

2021-12-05 Thread Waldemar Kozaczuk
h is obnly enabled when conf_hide_symbols is ON. Using version script (--version-script) plays some role in futher reducing number of symbols down to what exactly we want to export in exported//*symbols. Signed-off-by: Waldemar Kozaczuk --- Makefile | 19 +-- conf/base.mk | 1

[osv-dev] [PATCH] scripts: add new generate_version_script.sh

2021-12-05 Thread Waldemar Kozaczuk
This patch adds new script that generates version script based on the symbol files from exported_symbols//*.symbols Signed-off-by: Waldemar Kozaczuk --- scripts/generate_version_script.sh | 27 +++ 1 file changed, 27 insertions(+) create mode 100755 scripts

[osv-dev] [PATCH] x64: add list files of exported symbols

2021-12-05 Thread Waldemar Kozaczuk
This patch adds files with lists of symbols to be exported by kernel. The files have been generated on Fedora 33 using the scripts/extract_symbols.sh. These files will be fed to the process to generate version script. Signed-off-by: Waldemar Kozaczuk --- .../x64/osv_ld-linux-x86-64.so.2

[osv-dev] [PATCH] linker: enforce some sections kept when garbage collection on

2021-12-05 Thread Waldemar Kozaczuk
the linker result when garbage collection is off - the default. Signed-off-by: Waldemar Kozaczuk --- arch/x64/loader.ld | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/x64/loader.ld b/arch/x64/loader.ld index 78a3c939..debdbf15 100644 --- a/arch/x64

[osv-dev] [PATCH] Makefile: add option to build kernel with most symbols hidden

2021-12-02 Thread Waldemar Kozaczuk
like httpserver.so rely on OSv specific API symbols and they will not work either. To address this, we will need to expose some of the OSv C++ API as C. It is not clear if this patch fully addresses the issue #97. We could however close it and open smaller ones to address remaining gaps. Ref

[osv-dev] [PATCH] scripts: add new script extract_symbols.sh

2021-12-02 Thread Waldemar Kozaczuk
the script source and the README file. Signed-off-by: Waldemar Kozaczuk --- exported_symbols/README.md | 29 + scripts/extract_symbols.sh | 128 + 2 files changed, 157 insertions(+) create mode 100644 exported_symbols/README.md create mode 100755

[osv-dev] [PATCH] expose inet_aton and inet_ntoa when kernel built with most symbols hidden

2021-12-02 Thread Waldemar Kozaczuk
this patch exposes the bsd versions of inet_aton and inet_ntoa so that when kernel is built with most symbols hidden, these two ones are visible. Refs #1178 Signed-off-by: Waldemar Kozaczuk --- bsd/sys/libkern/inet_aton.c | 3 ++- bsd/sys/libkern/inet_ntoa.c | 3 ++- 2 files changed, 4 insert

[osv-dev] [PATCH V2 2/3] prepare to hide kernel symbols: expose libc symbols in certain files

2021-11-27 Thread Waldemar Kozaczuk
ble with Linux - moves some of the OSV_*API annotation from header files to the source files Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/feexcept.cc | 7 +- arch/x64/feexcept.cc | 4 + bsd/sys/kern/uipc_syscalls_wrap.cc | 37 - bsd/sys/xen/xenstore/x

[osv-dev] [PATCH V2 1/3] prepare to hide kernel symbols: add macros to expose and hide symbols

2021-11-26 Thread Waldemar Kozaczuk
the header file for more details. Signed-off-by: Waldemar Kozaczuk --- include/osv/export.h | 41 + 1 file changed, 41 insertions(+) create mode 100644 include/osv/export.h diff --git a/include/osv/export.h b/include/osv/export.h new file mode 100644 inde

[osv-dev] [PATCH V2] libc: remove/add some aliases

2021-11-26 Thread Waldemar Kozaczuk
(alias) statements from runtime.cc. Finally we also remove obsolete ___fprintf_chk alias. Signed-off-by: Waldemar Kozaczuk --- libc/aliases.ld| 11 ++- libc/stdio/__fprintf_chk.c | 1 - runtime.cc | 10 -- 3 files changed, 6 insertions(+), 16 deletions

[osv-dev] [PATCH V2] unit tests: change to use standard glibc and STD C++ constructs

2021-11-24 Thread Waldemar Kozaczuk
hese 5 unit tests compilable and runnable on Linux and allows for verification if these tests pass on Linux. Version 2 that fixes the corrupt part of the original version. Signed-off-by: Waldemar Kozaczuk --- tests/tst-af-local.cc | 24 ++--- tests/tst-bsd-tcp1.cc

[osv-dev] [PATCH 3/3] prepare to hide kernel symbols: hide some symbols

2021-11-21 Thread Waldemar Kozaczuk
This last patch in the series annotates some symbols in few files that need to be hidden regardless if we compile kernel with non-glibc symbols hidden or not. For that we use OSV_HIDDEN_API macro. Signed-off-by: Waldemar Kozaczuk --- libc/libc.cc| 2 +- libc/libc.hh| 3

[osv-dev] [PATCH 2/3] prepare to hide kernel symbols: expose libc symbols in certain files

2021-11-21 Thread Waldemar Kozaczuk
ned-off-by: Waldemar Kozaczuk --- bsd/porting/sync_stub.h| 11 +- bsd/sys/kern/uipc_syscalls_wrap.cc | 37 +++--- bsd/sys/netinet/in.h | 11 +- bsd/sys/rpc/xdr.h | 75 +++-- bsd/sys/sys/libkern.h | 3 +- bsd/sys

[osv-dev] [PATCH 1/3] prepare to hide kernel symbols: add macros to expose and hide symbols

2021-11-21 Thread Waldemar Kozaczuk
the header file for more details. Signed-off-by: Waldemar Kozaczuk --- include/osv/export.h | 39 +++ 1 file changed, 39 insertions(+) create mode 100644 include/osv/export.h diff --git a/include/osv/export.h b/include/osv/export.h new file mode 100644 inde

[osv-dev] [PATCH] libc: remove/add some aliases

2021-11-21 Thread Waldemar Kozaczuk
This patch removes some entries from aliases.ld for symbols that are not present in glibc shared libraries OSv dynamic linker advertises. It also adds new entries for symbols that are present in the same libraries. Signed-off-by: Waldemar Kozaczuk --- libc/aliases.ld| 11

[osv-dev] [PATCH] libc: hide internal musl and libc symbols

2021-11-21 Thread Waldemar Kozaczuk
it. Signed-off-by: Waldemar Kozaczuk --- include/api/internal_musl_headers/errno.h| 2 +- include/api/internal_musl_headers/sys/mman.h | 4 +- include/api/internal_musl_headers/time.h | 2 +- libc/internal/locale_impl.h | 2 + libc/signal/block.c

[osv-dev] [PATCH] getifaddrs: make some internal functions static

2021-11-20 Thread Waldemar Kozaczuk
Signed-off-by: Waldemar Kozaczuk --- libc/network/getifaddrs.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/libc/network/getifaddrs.c b/libc/network/getifaddrs.c index 4cdf4d6c..435c93b3 100644 --- a/libc/network/getifaddrs.c +++ b/libc/network/getifaddrs.c @@ -115,7

[osv-dev] [PATCH] libc: remove wrong implementation of setpgid()

2021-11-20 Thread Waldemar Kozaczuk
: Waldemar Kozaczuk --- Makefile | 1 - libc/unistd/setsid.cc | 9 - 2 files changed, 10 deletions(-) delete mode 100644 libc/unistd/setsid.cc diff --git a/Makefile b/Makefile index 91cd6887..354b3411 100644 --- a/Makefile +++ b/Makefile @@ -1710,7 +1710,6 @@ libc += unistd

[osv-dev] [PATCH] libc: add missing sched_getcpu() declaration

2021-11-20 Thread Waldemar Kozaczuk
Signed-off-by: Waldemar Kozaczuk --- include/api/sched.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/api/sched.h b/include/api/sched.h index 925ea986..1893efc1 100644 --- a/include/api/sched.h +++ b/include/api/sched.h @@ -127,6 +127,7 @@ inline static size_t CPU_ALLOC_SIZE

[osv-dev] [PATCH] libtsm: hide all symbols

2021-11-20 Thread Waldemar Kozaczuk
The libtsm is a library used internally to implement the VGA driver. There is no need to expose its symbols so this patch hides all of them. Signed-off-by: Waldemar Kozaczuk --- drivers/libtsm/libtsm_int.hh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/libtsm

[osv-dev] [PATCH] unit tests: change to use standard glibc and STD C++ constructs

2021-11-18 Thread Waldemar Kozaczuk
hese 5 unit tests compilable and runnable on Linux and allows for verification if these tests pass on Linux. Signed-off-by: Waldemar Kozaczuk --- include/api/sched.h | 1 + tests/tst-af-local.cc | 24 ++--- tests/tst-bsd-tcp1.cc

[osv-dev] [PATCH] unit tests: replace OSv debug() with printf

2021-11-18 Thread Waldemar Kozaczuk
This patch replaces the calls to OSv specific debug() with standard printf(). These makes these 3 unit tests compilable and runnable on Linux and allows for verification if these tests pass on Linux. Signed-off-by: Waldemar Kozaczuk --- tests/tst-except.cc | 8 tests/tst-readdir.cc

[osv-dev] [PATCH] hide symbols defined in linker script

2021-11-18 Thread Waldemar Kozaczuk
The linker scripts for both x64 and aarch64 define various symbols especially those ending with '_start' and '_end'. These do not need to be visible so we hiding them by using HIDDEN and PROVIDE_HIDDEN directive. Signed-off-by: Waldemar Kozaczuk --- Makefile |

[osv-dev] [PATCH] hide symbols: use .hidden to hide symbols in .s|.S files

2021-11-18 Thread Waldemar Kozaczuk
The compiler flag '-fvisibility=hidden' does not hide symbols defined in the assembly files. None of the symbols should be visible to any app so we are hiding them by using '.hidden' directive. Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/boot.S| 5

[osv-dev] [PATCH] scripts: update trace.py and prof.py to support generating flame graphs

2021-11-12 Thread Waldemar Kozaczuk
M_invoke(std::_Any_data const&);virtio::net::receiver() 342 17592192727762;17592191756991;17592192722413;17592192722076 118 18446744069951476397 130 sched::thread::main();__invoke_r&>;__invoke_impl&>;std::_Function_handler::_M_invoke(std::_Any_data const&);virtio::net::receiver();class

[osv-dev] [PATCH] scripts/osv/prof.py: update unimportant_prefixes

2021-11-12 Thread Waldemar Kozaczuk
Update unimportant_prefixes to match recent stack traces generated when sampling an app using cpu_sampler enabled with '--sampler=FREQUENCY'. Signed-off-by: Waldemar Kozaczuk --- scripts/osv/prof.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/scripts/osv

[osv-dev] [PATCH] devices: expose device type

2021-09-12 Thread Waldemar Kozaczuk
such case. Signed-off-by: Waldemar Kozaczuk --- drivers/device.hh| 8 drivers/pci-function.hh | 1 + drivers/virtio-mmio.hh | 1 + drivers/virtio-pci-device.hh | 3 +++ 4 files changed, 13 insertions(+) diff --git a/drivers/device.hh b/drivers/device.hh inde

[osv-dev] [PATCH V2] socket: implement Linux flavor of SO_REUSE_PORT option

2021-08-22 Thread Waldemar Kozaczuk
of(inpcblbgroup::il_inp[0]) * size; Fixes #1170 Signed-off-by: Waldemar Kozaczuk --- bsd/sys/compat/linux/linux.h | 1 + bsd/sys/compat/linux/linux_socket.cc | 2 + bsd/sys/netinet/in_pcb.cc| 285 +++ bsd/sys/netinet/in_pcb.h | 32 ++

[osv-dev] socket: implement Linux flavor of SO_REUSE_PORT option

2021-08-21 Thread Waldemar Kozaczuk
d/or SO_REUSEPORT flags drive same address and/or port collision logic. Some articles about SO_REUSE_PORT: - https://lwn.net/Articles/542629/ - https://linuxjedi.co.uk/2020/04/25/socket-so_reuseport-and-kernel-implementations/ Fixes #1170 Signed-off-by: Waldemar Kozaczuk --- bsd/sys/compat/linu

[osv-dev] in_pcb: extract union in_dependaddr

2021-08-21 Thread Waldemar Kozaczuk
group struct defined as part of the forth coming patch. Signed-off-by: Waldemar Kozaczuk --- bsd/sys/netinet/in_pcb.h | 25 +++-- bsd/sys/netinet/tcp_input.cc | 4 ++-- 2 files changed, 13 insertions(+), 16 deletions(-) diff --git a/bsd/sys/netinet/in_pcb.h b/bsd/sys

[osv-dev] [PATCH] vfs: harden drele() to check for null v_op and vop_inactive

2021-07-12 Thread Waldemar Kozaczuk
mount(): No error information(-1) ``` Fixes #813 Signed-off-by: Waldemar Kozaczuk --- fs/vfs/vfs_vnode.cc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/vfs/vfs_vnode.cc b/fs/vfs/vfs_vnode.cc index a292344f..0918d0d9 100644 --- a/fs/vfs/vfs_vnode.cc +++ b/fs/vfs

[osv-dev] [PATCH] tests: disable java tests when cross-compiling to aarch64

2021-06-24 Thread Waldemar Kozaczuk
Although it might be possible to download aarch64 version of JDK and cross-compile java-base and related binaries, for now let us disable java-tests in such scenario. We still support building and running java tests on native aarch64 machine. Signed-off-by: Waldemar Kozaczuk --- modules/tests

[osv-dev] [PATCH] acpi: capture ACPI root address when booting in PVH mode

2021-06-24 Thread Waldemar Kozaczuk
size=128M --rng \ --cmdline='--verbose --ip=eth0,172.16.0.2,255.255.255.252 --defaultgw=172.16.0.1 --nameserver=172.16.0.1 /hello' \ --net "tap=fc_tap0,mac=,ip=172.16.0.1,mask=255.255.255.252" Signed-off-by: Waldemar Kozaczuk --- arch/x64/arch-setup.cc | 3 +++ arch/x6

[osv-dev] [PATCH] futex: implement FUTEX_WAIT_BITSET

2021-06-20 Thread Waldemar Kozaczuk
tst-async.cc and tst-promise.cc pass. Fixes #1147 Signed-off-by: Waldemar Kozaczuk --- linux.cc | 29 + 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/linux.cc b/linux.cc index 522ed8ba..adeb89f0 100644 --- a/linux.cc +++ b/linux.cc @@ -62,23 +62,44

[osv-dev] [PATCH] aarch64: make java-base and related modules compile

2021-06-13 Thread Waldemar Kozaczuk
cuted from modules/java-tests/test_commands, generated during build time. Signed-off-by: Waldemar Kozaczuk --- modules/cloud-init/Makefile | 43 - modules/cloud-init/module.py | 1 - modules/golang/Makefile | 27 +++--- modules/

[osv-dev] [PATCH] openjdk8: make it work with aarch64

2021-06-13 Thread Waldemar Kozaczuk
This patch modifies existing openjdk8-from-host module to make it work on aarch64 Linux host. Signed-off-by: Waldemar Kozaczuk --- modules/openjdk8-from-host/Makefile | 12 modules/openjdk8-from-host/module.py | 28 +++- 2 files changed, 27 insertions

[osv-dev] [PATCH] java: add OpenJDK module for java 9 and above

2021-06-13 Thread Waldemar Kozaczuk
This patch adds new module to allow building an image with JDK 9 or newer from the files found on a Linux host. It does so by locating javac executable found in the PATH and then navigating the corresponding directory to collect all required files. Signed-off-by: Waldemar Kozaczuk --- modules

[osv-dev] [PATCH] sigaction: handle null sa_sigaction as SIG_DFL

2021-05-28 Thread Waldemar Kozaczuk
#1047 Signed-off-by: Waldemar Kozaczuk --- libc/signal.cc | 6 +- modules/tests/Makefile | 3 ++- tests/tst-sigaction.cc | 43 ++ 3 files changed, 50 insertions(+), 2 deletions(-) create mode 100644 tests/tst-sigaction.cc diff --git a/libc

[osv-dev] [PATCH 2/3] syscalls: minimally implement tgkill

2021-05-24 Thread Waldemar Kozaczuk
to kill(). Lastly this patch enhances tst-kill.cc to test raise() and pthread_kill(). Refs #1047 Signed-off-by: Waldemar Kozaczuk --- libc/pthread.cc | 13 - linux.cc | 17 + tests/tst-kill.cc | 5 + 3 files changed, 34 insertions(+), 1 deletion

[osv-dev] [PATCH 3/3] libc: add brk()/sbrk() stubs

2021-05-24 Thread Waldemar Kozaczuk
Some applications like keydb that use jmalloc, call sbrk() on startup even though later fall back to mmap(). So this patch provides simple stubs of those 2 functions that set errno to ENOMEM and return error. Signed-off-by: Waldemar Kozaczuk --- libc/mman.cc | 15 +++ 1 file

[osv-dev] [PATCH 1/3] libc: add strerrordesc_np

2021-05-24 Thread Waldemar Kozaczuk
Signed-off-by: Waldemar Kozaczuk --- libc/aliases.ld | 1 + 1 file changed, 1 insertion(+) diff --git a/libc/aliases.ld b/libc/aliases.ld index 3db1dec6..14dac146 100644 --- a/libc/aliases.ld +++ b/libc/aliases.ld @@ -48,6 +48,7 @@ __strndup = strndup; __stpcpy = stpcpy; __stpncpy = stpncpy;

[osv-dev] [PATCH V2 3/3] aarch64: make synch_thread/synch_port work with weak memory model

2021-05-15 Thread Waldemar Kozaczuk
tween CPUs. Signed-off-by: Waldemar Kozaczuk --- bsd/porting/synch.cc | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/bsd/porting/synch.cc b/bsd/porting/synch.cc index 59e82b7b..1641ac29 100644 --- a/bsd/porting/synch.cc +++ b/bsd/porting/synch.cc @@ -26,7 +26,7 @@

[osv-dev] [PATCH V2 2/3] aarch64: make queue_mpsc work with weak memory model

2021-05-15 Thread Waldemar Kozaczuk
run on different CPUs and therefore need to SEE the latest value of this variable after the write on a different CPUs. For that reason we make poplist an atomic and use acquire/release semantics to archieve cross-CPU visibility. Signed-off-by: Waldemar Kozaczuk --- include/lockfree/queue

[osv-dev] [PATCH V2 1/3] aarch64: make wait_record with weak memory model

2021-05-15 Thread Waldemar Kozaczuk
consistency. The ARM architecture comes with weak memory model where each CPUs in essence has it own seperate view of memory synced via common interconnect. For example, the memory system does not guarantee that a write becomes visible to all other CPUs at the same time (this behavior is called write no

[osv-dev] [PATCH V2 0/3] aarch64: make wait_record, queue_mpsc and synch_thread/synch_port work with weak memory model

2021-05-15 Thread Waldemar Kozaczuk
weak memory model architectures like ARM. These two constitute the building blocks of the very critical synchronization mechanism in OSv - lock-free mutex. Similarly the 3rd patch modifies synch_thread/synch_port to make it work correctly with weak memory model. Fixes #1123 Waldemar Kozaczuk (3

[osv-dev] [PATCH V2] aarch64: make RCU work correctly in SMP mode

2021-05-12 Thread Waldemar Kozaczuk
could execute the same test over 13K times. Fixes #1134 Signed-off-by: Waldemar Kozaczuk --- core/rcu.cc | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/core/rcu.cc b/core/rcu.cc index d6f587b8..2bc7a75f 100644 --- a/core/rcu.cc +++ b/core/rcu.cc @@ -73,7 +73,7

[osv-dev] [PATCH] aarch64: handle AT_HWCAP in getauxval

2021-05-09 Thread Waldemar Kozaczuk
representing relevant CPU features. This new function is called by __getauxval() for aarch64. Besides making the annoying warning go away, this patch also makes OSv take advantage of Armv8.1-A platforms that implement CAS functions based on LSE instructions. Fixes #1129 Signed-off-by: Waldemar

[osv-dev] [PATCH] aarch64: make RCU work correctly in SMP mode

2021-05-09 Thread Waldemar Kozaczuk
memory barrier instructions generated around those. Before this patch, the tst-evenfd.so test would hang under 100 runs with 2 vCPUs with the symptoms described by #1134. After this patch I could execute the same test over 13K times. Fixes #1134 Signed-off-by: Waldemar Kozaczuk --- core/rcu.cc

[osv-dev] [PATCH] mempool: resize l1_pool_stats once

2021-05-03 Thread Waldemar Kozaczuk
ual to the number of cpus in the system we resize the l1_pool_stats vector. We also do it before incrementing the smp_allocator_cnt. Signed-off-by: Waldemar Kozaczuk --- core/mempool.cc | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/core/mempool.cc b/core/mempool.cc inde

[osv-dev] [PATCH] aarch64: make wait_record and queue_mpsc work with weak memory model

2021-05-01 Thread Waldemar Kozaczuk
each CPUs in essence has it own seperate view of memory synced via common interconnect. For example, the memory system does not guarantee that a write becomes visible to all other CPUs at the same time (this behavior is called write non-atomicity). So the ARM is way less forgiving. Fixes #1123

[osv-dev] [PATCH V2] dynamic linker: properly handle STT_NOTYPE symbols

2021-04-29 Thread Waldemar Kozaczuk
resolve such symbols at the relative offset as specified by st_value and cause page fault. This patch fixes this logic by correctly adding the base of the ELF to the st_value. This patch effectively makes it possibly to run latest Node.JS 12 and 14 on OSv. Signed-off-by: Waldemar Kozaczuk

[osv-dev] [PATCH] aarch64: unmask clock event interrupt on secondary CPUs

2021-04-28 Thread Waldemar Kozaczuk
is registered). Fixes #1132 Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/arm-clock.cc | 24 +++- arch/aarch64/arm-clock.hh | 10 ++ arch/aarch64/gic.cc | 6 ++ 3 files changed, 27 insertions(+), 13 deletions(-) diff --git a/arch/aarch64/arm-clock.cc b

[osv-dev] [PATCH] dynamic linker: properly handle STT_NOTYPE symbols

2021-04-27 Thread Waldemar Kozaczuk
resolve such symbols at the relative offset as specified by st_value and cause page fault. This patch fixes this logic by correctly adding the base of the ELF to the st_value. This patch effectively makes it possibly to run latest Node.JS 12 and 14 on OSv. Signed-off-by: Waldemar Kozaczuk

[osv-dev] [PATCH V2] aarch64: implement minimal PL031 support

2021-03-29 Thread Waldemar Kozaczuk
functionality that is only accurate up to a second. This patch effectively fixes 2 unit tests - tst-time.cc and tst-timerfd.cc. Fixes #1091 Signed-off-by: Waldemar Kozaczuk --- Makefile | 1 + arch/aarch64/arch-dtb.cc | 19 + arch/aarch64/arch-dtb.hh | 7 + arch

[osv-dev] [PATCH V2] aarch64: mask interrupt and disable timer in the handler

2021-03-29 Thread Waldemar Kozaczuk
the method set() to make it set new timer only if calculated number of ticks (tval) is greater than 0. Fixes #1127 Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/arm-clock.cc | 49 ++- 1 file changed, 43 insertions(+), 6 deletions(-) diff --git a/arch/aarch

[osv-dev] [PATCH] loader.py: improve "osv info threads"

2021-03-29 Thread Waldemar Kozaczuk
slightly re-arranged layout that moves the widest 'location' column to the very right and added header line that lets us save some real estate for new column Signed-off-by: Waldemar Kozaczuk --- scripts/loader.py | 90 ++- 1 file changed, 57 inse

[osv-dev] [PATCH] aarch64: add CFI directives to the page fault and interrupt handler assembly

2021-03-29 Thread Waldemar Kozaczuk
.h:291 #15 0x402e72b4 in sched::thread::main (this=0xa000408eea48) at core/sched.cc:1267 #16 sched::thread_main_c (t=0xffffa000408eea48) at arch/aarch64/arch-switch.hh:162 #17 0x4020b788 in thread_main () at arch/aarch64/entry.S:114 Refs #1128 Signed-off-by: Waldemar

[osv-dev] [PATCH] aarch64: add thread_main to improve stacktraces

2021-03-29 Thread Waldemar Kozaczuk
:load_balance ( this=0x407009b8 , std::allocator >+24>) at core/sched.cc:730 #6 0x402e72b4 in sched::thread::main (this=0xa000408a39a0) at core/sched.cc:1267 #7 sched::thread_main_c (t=0xa000408a39a0) at arch/aarch64/arch-switch.hh:162 #8 0x00004020b788 in thread_main (

[osv-dev] [PATCH] aarch64: mask interrupt and disable timer in the handler

2021-03-19 Thread Waldemar Kozaczuk
is greater than 0. Fixes #1127 Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/arm-clock.cc | 39 +-- 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/arch/aarch64/arm-clock.cc b/arch/aarch64/arm-clock.cc index 7d11e454..d32f58b1 100644 --- a/a

[osv-dev] [PATCH] aarch64: implement minimal PL031 support

2021-03-19 Thread Waldemar Kozaczuk
functionality that is only accurate up to a second. This patch effectively fixes 2 unit tests - tst-time.cc and tst-timerfd.cc. Fixes #1091 Signed-off-by: Waldemar Kozaczuk --- Makefile | 1 + arch/aarch64/arch-dtb.cc | 19 + arch/aarch64/arch-dtb.hh | 7 + arch

[osv-dev] [PATCH v1] aarch64: implement reschedule_from_interrupt() in assembly

2021-03-14 Thread Waldemar Kozaczuk
x27; OSv v0.55.0-205-g7e7c49f7 getauxval() stubbed eth0: 192.168.122.15 Booted up in 58.20 ms Cmdline: /tests/misc-ctxsw.so 683 colocated 690 apart 690 nopin Fixes #1121 Fixes #1124 Signed-off-by: Waldemar Kozaczuk --- Makefile | 1 + arch/aarch64/arch-switc

[osv-dev] [PATCH v0] aarch64: implement reschedule_from_interrupt() in assembly

2021-03-03 Thread Waldemar Kozaczuk
apart 665 nopin AFTER: taskset -c 2-5 ./scripts/run.py -c 1 -e '/tests/misc-ctxsw.so' OSv v0.55.0-205-g7e7c49f7 getauxval() stubbed eth0: 192.168.122.15 Booted up in 58.20 ms Cmdline: /tests/misc-ctxsw.so 683 colocated 690 apart 690 nopin Fixes #1121 Fixes

[osv-dev] [PATCH] aarch64: use memory barrier to force completion of PTE writes

2021-03-01 Thread Waldemar Kozaczuk
the real culprit of the issue this patch is fixing. Fixes #1117 Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/arch-mmu.hh | 22 ++ arch/x64/arch-mmu.hh | 6 ++ core/mmu.cc | 7 +++ 3 files changed, 35 insertions(+) diff --git a/arch/aarch64/a

[osv-dev] [PATCH] aarch64: fix TLS kernel variables access from apps

2021-02-26 Thread Waldemar Kozaczuk
:ldp x29, x30, [sp], #16 0x4041df70 <+32>:ldr x0, [x1, x0] 0x4041df74 <+36>:mov x16, x0 0x00004041df78 <+40>:br x16 ``` Fixes #1122 Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/arch-elf.cc | 4 ++-- scripts/test.p

[osv-dev] [PATCH] aarch64: align up size of the kernel TLS to 64 bytes

2021-02-24 Thread Waldemar Kozaczuk
the 1st thread in similar way it is done in setup_tcb(). Fixes #1120 Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/arch-setup.cc | 2 +- core/elf.cc| 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/aarch64/arch-setup.cc b/arch/aarch64/arch-setup.cc

[osv-dev] [PATCH V3] scripts: download and build arbitrary version of boost

2021-02-16 Thread Waldemar Kozaczuk
main makefile and modules/common.gmk. Signed-off-by: Waldemar Kozaczuk --- scripts/download_and_build_boost.sh | 56 + 1 file changed, 56 insertions(+) create mode 100755 scripts/download_and_build_boost.sh diff --git a/scripts/download_and_build_boost.sh b/scripts

[osv-dev] [PATCH V2] scripts: download and build arbitrary version of boost

2021-02-16 Thread Waldemar Kozaczuk
main makefile and modules/common.gmk. Signed-off-by: Waldemar Kozaczuk --- scripts/download_and_build_boost.sh | 56 + 1 file changed, 56 insertions(+) create mode 100755 scripts/download_and_build_boost.sh diff --git a/scripts/download_and_build_boost.sh b/scripts

[osv-dev] [PATCH V2] scripts: allow building QEMU from sources

2021-02-16 Thread Waldemar Kozaczuk
Add new script download_and_build_qemu.sh that allows building QEMU from sources. It installs all necessary packages to build QEMU from sources, downloads QEMU sources and builds both x86_64 and aarch64 versions of it. Signed-off-by: Waldemar Kozaczuk --- scripts/download_and_build_qemu.sh | 61

[osv-dev] [PATCH] setup: support CentOS 7

2021-02-08 Thread Waldemar Kozaczuk
with gcc 9 (c, c++) and other binaries. There is no package repo with qemu (I guess we could install one from Fedora) so instead we provide a script to build qemu from sources (see other patch). Signed-off-by: Waldemar Kozaczuk --- scripts/setup.py | 53

[osv-dev] [PATCH] sched: enforce tls register refresh after thread switch

2021-02-07 Thread Waldemar Kozaczuk
erminating thread to a new class method destroy_current_cpu_thread() to enforce fresh reading of the TLS register. Fixes #1121 Signed-off-by: Waldemar Kozaczuk --- core/sched.cc| 19 ++- include/osv/sched.hh | 1 + 2 files changed, 15 insertions(+), 5 deletions(-) di

[osv-dev] [PATCH] scripts: allow building QEMU from sources

2021-02-07 Thread Waldemar Kozaczuk
Add new script download_and_build_qemu.sh that allows building QEMU from sources. It installs all necessary packages to build QEMU from sources, downloads QEMU sources and builds both x86_64 and aarch64 versions of it. Signed-off-by: Waldemar Kozaczuk --- scripts/download_and_build_qemu.sh | 61

[osv-dev] [PATCH] tests: handle more generic CROSS_PREFIX for aarch64 builds

2021-02-07 Thread Waldemar Kozaczuk
e ldd find locations of boost binaries even where tests are built against arbitrary version of boost specified by the boost_base parameter. Signed-off-by: Waldemar Kozaczuk --- modules/tests/Makefile | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/modules/tests/Makefile

[osv-dev] [PATCH 2/2] boost: enhance modules/common.gmk to build against arbitrary boost

2021-02-07 Thread Waldemar Kozaczuk
parameter - boost_base - that points to the base of the folder where boost was built. Here is an example of how kernel and the tests modules can be built against arbitrary boost: boost_base=./build/downloaded_packages/x86_64/boost/install ./scripts/build -j4 image=tests Signed-off-by: Waldemar

[osv-dev] [PATCH 1/2] boost: enhance main makefile to build against arbitrary boost

2021-02-07 Thread Waldemar Kozaczuk
the base of the folder where boost was built. Here is an example of how kernel can be built against arbitrary boost: boost_base=./build/downloaded_packages/x86_64/boost/install ./scripts/build -j4 image=native-example Signed-off-by: Waldemar Kozaczuk --- Makefile | 48

[osv-dev] [PATCH] aarch64: make dl_tests makefile build for aarch64

2021-02-07 Thread Waldemar Kozaczuk
This patch changes dl_tests makefile to delegate to modules/common.gmk to support building dl_tests module for both x64 and aarch64. This patch also effectively makes tst-dlfcn.so to pass on aarch64. Signed-off-by: Waldemar Kozaczuk --- modules/dl_tests/Makefile | 41

[osv-dev] [PATCH V2] scripts/run.py: use QEMU_PATH to use arbitrary qemu executable

2021-02-07 Thread Waldemar Kozaczuk
o not pass it to run.py. So instead, we allow one set an environment variable QEMU_PATH which would be used if set or exported in the shell invoking run.py or other scripts calling it. Signed-off-by: Waldemar Kozaczuk --- scripts/run.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) di

[osv-dev] [PATCH] scripts/run.py: use QEMU_PATH to use arbitrary qemu executable

2021-02-07 Thread Waldemar Kozaczuk
o not pass it to run.py. So instead, we allow one set an environment variable QEMU_PATH which would be used if set or exported in the shell invoking run.py or other scripts calling it. Signed-off-by: Waldemar Kozaczuk --- scripts/run.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) di

[osv-dev] [PATCH] aarch64: handle R_AARCH64_COPY relocations

2021-02-07 Thread Waldemar Kozaczuk
This patch enhances dynamic linker to handle COPY relocations for aarch64 which we do exactly same way for x86_64. This is enough to support running Python 3 on OSv ARM. Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/arch-elf.cc | 6 -- 1 file changed, 4 insertions(+), 2 deletions

[osv-dev] [PATCH] aarch64: fix loader.py to use correct IP register

2021-02-07 Thread Waldemar Kozaczuk
proper PC register. Signed-off-by: Waldemar Kozaczuk --- scripts/loader.py | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/scripts/loader.py b/scripts/loader.py index 4b6aa32f..5b224e34 100644 --- a/scripts/loader.py +++ b/scripts/loader.py @@ -747,7 +747,7 @@ class

[osv-dev] [PATCH] scripts: download and build arbitrary version of boost

2021-02-05 Thread Waldemar Kozaczuk
the build/downloaded_packages//boost/install/ directory and will automatically be used where cross-compiling aarch64 OSv kernel. In order to use x64 version of boost built using this script we still need to make changes to the main makefile and modules/common.gmk. Signed-off-by: Waldemar Kozaczuk

[osv-dev] [PATCH] tests/tst-threadcomplete.cc: disable part testing pinning when cpu == 1

2021-02-02 Thread Waldemar Kozaczuk
: Waldemar Kozaczuk --- tests/tst-threadcomplete.cc | 14 ++ 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/tests/tst-threadcomplete.cc b/tests/tst-threadcomplete.cc index 71c8af1f..67fb12ec 100644 --- a/tests/tst-threadcomplete.cc +++ b/tests/tst-threadcomplete.cc

[osv-dev] [PATCH] tests: simplify makefile by including new modules/common.gmk

2021-01-21 Thread Waldemar Kozaczuk
This patch simplifies the tests makefile by including new modules/common.gmk which identifies common compiler flags depending on the architecture and possibly cross-compiler prefix and locates common shared libraries. Signed-off-by: Waldemar Kozaczuk --- modules/tests/Makefile | 95

[osv-dev] [PATCH] libtools: include common.gmk

2021-01-21 Thread Waldemar Kozaczuk
/release./modules/libtools/ instead of modules/libtools. Signed-off-by: Waldemar Kozaczuk --- modules/libtools/.gitignore | 1 + modules/libtools/Makefile | 53 +++ modules/libtools/usr.manifest | 2 -- 3 files changed, 17 insertions(+), 39 deletions

[osv-dev] [PATCH] httpserver-api: include common.gmk

2021-01-21 Thread Waldemar Kozaczuk
build/release./modules/httpserver-api/ instead of modules/httpserver-api. Signed-off-by: Waldemar Kozaczuk --- modules/httpserver-api/.gitignore | 2 -- modules/httpserver-api/Makefile | 58 --- modules/httpserver-api/module.py | 2 +- 3 files changed, 24

[osv-dev] [PATCH] httpserver-monitoring-api: include common.gmk

2021-01-21 Thread Waldemar Kozaczuk
artifacts go to build/release./modules/httpserver-monitoring-api/ instead of modules/httpserver-monitoring-api. Signed-off-by: Waldemar Kozaczuk --- modules/httpserver-monitoring-api/.gitignore | 2 - modules/httpserver-monitoring-api/Makefile| 58 --- .../add_aarch64_libraries.sh

[osv-dev] [PATCH] modules: add common makefile to allow cross-compiling

2021-01-21 Thread Waldemar Kozaczuk
include headers are set in the COMMON variable. Fixes #1105 Signed-off-by: Waldemar Kozaczuk --- modules/common.gmk | 172 + 1 file changed, 172 insertions(+) create mode 100644 modules/common.gmk diff --git a/modules/common.gmk b/modules/common.gmk new

[osv-dev] [PATCH] include: support compiling OSv on CentOS 7

2021-01-20 Thread Waldemar Kozaczuk
when compiling sources using the math header, we get errors caused by a clash with isinf and isnan macros. To solve this problem we simply undef both _GLIBCXX_HAVE_OBSOLETE_ISINF and _GLIBCXX_HAVE_OBSOLETE_ISNAN in OSv math.h header. Signed-off-by: Waldemar Kozaczuk --- include/api/math.h | 3

[osv-dev] [PATCH] threads: allocate stacks lazily to save memory

2021-01-16 Thread Waldemar Kozaczuk
41 83 45 00 01 addl $0x1,%fs:0x0(%r13) 4035fdfc: e8 3f f6 ff ff callq 4035f440 ``` As an example of memory saving, tomcat using ~ 300 thread ended up 365MB instead of 620MB before the patch. Fixes #143 Signed-off-by: Matthew Pabst Signed-off-by: Waldemar Kozaczuk --- Makefil

[osv-dev] [PATCH] aarch64: support compiling kernel natively on aarch64 host

2021-01-05 Thread Waldemar Kozaczuk
lways point the header includes and *so patch to build/downloaded_packages/aarch64/boost/install. Fixes #1106 Signed-off-by: Waldemar Kozaczuk --- Makefile | 119 ++ scripts/build | 10 - 2 files changed, 79 insertions(+), 50 deletions(-) diff --gi

[osv-dev] [PATCH] aarch64: support downloading aarch64 packages on Ubuntu x64

2021-01-05 Thread Waldemar Kozaczuk
++ headers and all other gcc libraries' binaries like libstdc++.so.6 and libgcc_s.so, so we only need to download the aarch64 versions of the boost libraries. Signed-off-by: Waldemar Kozaczuk --- scripts/download_aarch64_packages.py | 78 +++ sc

[osv-dev] [PATCH] setup: install aarch64 C++ cross-compile packages for x86_64 hosts only

2021-01-03 Thread Waldemar Kozaczuk
e this case. Signed-off-by: Waldemar Kozaczuk --- scripts/setup.py | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/scripts/setup.py b/scripts/setup.py index 3cf45eac..56b405f1 100755 --- a/scripts/setup.py +++ b/scripts/setup.py @@ -6,6 +6,8 @@ import sys, argparse i

[osv-dev] [PATCH] aarch64: make boot messages quieter and consistent with x86_64

2021-01-03 Thread Waldemar Kozaczuk
using '#if CONF_logger_debug' macro so that it would still get printed in debug build. This also makes the console output more consistent with what we print on x86_64 with '--verbose' flag on or off. Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/arch-setup.cc | 2 ++ arch/

[osv-dev] [PATCH] aarch64: implement processor_to_nano in armclock

2021-01-03 Thread Waldemar Kozaczuk
properly on aarch64. Partially addresses #1091 Signed-off-by: Waldemar Kozaczuk --- arch/aarch64/arm-clock.cc | 8 1 file changed, 8 insertions(+) diff --git a/arch/aarch64/arm-clock.cc b/arch/aarch64/arm-clock.cc index d942af6b..c1f4b277 100644 --- a/arch/aarch64/arm-clock.cc +++ b/arch

[osv-dev] [PATCH] bootchart: do not print empty entries

2021-01-03 Thread Waldemar Kozaczuk
ly changes boot_time_chart::print_one_time() to ignore empty entries. Signed-off-by: Waldemar Kozaczuk --- core/chart.cc | 3 +++ 1 file changed, 3 insertions(+) diff --git a/core/chart.cc b/core/chart.cc index c3355907..131ce9c1 100644 --- a/core/chart.cc +++ b/core/chart.cc @@ -12,6 +12,9

[osv-dev] [PATCH] aarch64: do not disable rofs cache in run.py

2021-01-03 Thread Waldemar Kozaczuk
It is not clear which commit has fixed this issue (possibly musl upgrade), but for a while now, ROFS works well with both cache on or off on aarch64. So this patch removes unnecessary logic from run.py Signed-off-by: Waldemar Kozaczuk --- scripts/run.py | 3 --- 1 file changed, 3 deletions

[osv-dev] [PATCH V2] aarch64: synchronize data and instruction caches after loading code into memory

2020-12-27 Thread Waldemar Kozaczuk
memory. To achieve it delegates to an obscure built-in - __clear_cache(). This logic is actually no-op in x86-64 port, as this architecture has strong coherency between instruction and data caches and there is no need to do anything special in this case. Fixes #1100 Signed-o

<    1   2   3   4   5   6   7   >