[osv-dev] [PATCH v2 2/2] Add script to download aarch64 cross compiler toolchain

['Stewart Hildebrand' via OSv Development]

This is a source-able script that downloads the aarch64 toolchain from
developer.arm.com, extracts it, and, when being sourced, adds the path to PATH.
If ccache is installed, it will also add the ccache directory to the front of
PATH.

Add a check in the OSv build script to source the toolchain if arch=aarch64 and
the toolchain has been downloaded to the build directory.

Add "centos" as an option in download_aarch64_packages.py to download the
toolchain. I did not change the logic for Ubuntu here due to the possibility of
ending up with unvetted combinations of this GCC version with various Ubuntu
system libraries that could be built with different versions of GCC. This
toolchain can still be used on Ubuntu by sourcing it directly, but in this case
I recommend building Boost and other dependent libraries from source to ensure
they are built with the same toolchain that is being used to build OSv and your
apps/modules.

This toolchain should work on any distribution that ARM supports, which is
Ubuntu 16.04 LTS or later, or RHEL 7 or later. I've tested this on CentOS 7 and
Ubuntu 18.04.

My primary motivation for this is to support the aarch64 build on CentOS 7.
While the EPEL repository for CentOS 7 does have an aarch64 cross compiler
toolchain available, it is quite an old version (4.8.5). On Ubuntu, the aarch64
cross compiler toolchain conflicts with the gcc-multilib package. Using the ARM
toolchain allows us to avoid these particular issues.

This toolchain does not provide Boost, so Boost will need to be built. Boost can
be built with the script download_and_build_boost.sh in the scripts/ directory,
or it can be built manually. Here are instructions I used for building Boost
from source. Ensure that the current working directory is the root of the OSv
repository, then run the following commands:

  $ source scripts/download_aarch64_toolchain.sh
  $ mkdir -p build/downloaded_packages/aarch64/boost
  $ pushd build/downloaded_packages/aarch64/boost
  $ wget 
https://dl.bintray.com/boostorg/release/1.70.0/source/boost_1_70_0.tar.gz
  $ tar -xf boost_1_70_0.tar.gz
  $ cd boost_1_70_0/
  $ BOOST_DIR=$(pwd)
  $ ./bootstrap.sh
  $ echo "using gcc : arm : aarch64-none-linux-gnu-g++ ;" > user-config.jam
  $ ./b2 --user-config=${BOOST_DIR}/user-config.jam toolset=gcc-arm 
architecture=arm address-model=64 -j$(nproc)

After building Boost, we need to set up some symlinks so that OSv can find the
Boost headers and libraries. Ensure your current working directory is the Boost
source directory BOOST_DIR (it should be already), then run the following
commands:

  $ ln -s boost_1_70_0/stage ../install
  $ mkdir -p stage/usr/include
  $ ln -s ../../../boost stage/usr/include/boost
  $ popd

After building Boost and setting up the symlinks, we should be ready to build
OSv with the aarch64 toolchain:

  $ scripts/build arch=aarch64 fs=ramfs --create-disk image=native-example 
-j$(nproc)

Signed-off-by: Stewart Hildebrand 

---

v2:
  expand arch with a trailing dash in recognition that it may be unset
  update patch description to mention Waldek's boost build script
---
 scripts/build |  5 +++
 scripts/download_aarch64_packages.py  |  2 +
 scripts/download_aarch64_toolchain.sh | 55 +++
 3 files changed, 62 insertions(+)
 create mode 100644 scripts/download_aarch64_toolchain.sh

diff --git a/scripts/build b/scripts/build
index 250d5360..5fdcc7a8 100755
--- a/scripts/build
+++ b/scripts/build
@@ -88,6 +88,11 @@ do
esac
 done
 
+if [[ "${arch-}" == "aarch64" && -d 
"build/downloaded_packages/aarch64/toolchain/" ]]; then
+   . scripts/download_aarch64_toolchain.sh
+   export CROSS_PREFIX=aarch64-none-linux-gnu-
+fi
+
 make "${args[@]}" ${stage1_args} | tee build.out
 # check exit status of make
 status=${PIPESTATUS[0]}
diff --git a/scripts/download_aarch64_packages.py 
b/scripts/download_aarch64_packages.py
index 58a3cf7d..931decf9 100755
--- a/scripts/download_aarch64_packages.py
+++ b/scripts/download_aarch64_packages.py
@@ -78,6 +78,8 @@ elif name.lower() == 'ubuntu':
 print("Cound not find boost version from neither main nor universe 
ports index!")
 sys.exit(1)
 commands_to_download = ubuntu_download_commands(boost_version)
+elif name.lower() == "centos":
+commands_to_download = [ 'bash -eu 
%s/scripts/download_aarch64_toolchain.sh' % osv_root ]
 else:
 print("The distribution %s is not supported for cross-compiling aarch64 
version of OSv" % name)
 sys.exit(1)
diff --git a/scripts/download_aarch64_toolchain.sh 
b/scripts/download_aarch64_toolchain.sh
new file mode 100644
index ..0403712c
--- /dev/null
+++ b/scripts/download_aarch64_toolchain.sh
@@ -0,0 +1,55 @@
+#!/bin/bash -eu
+
+# Copyright (c) 2021, DornerWorks, Ltd.
+# Author: Stewart Hildebrand
+# SPDX-License-Identifier: BSD-3-Clause OR MIT
+
+# Usage:
+# $ source download_aarch64_toolchain.sh
+# or
+# $ . download_aarch64_toolchain.sh
+
+# 
https://developer.ar

[osv-dev] [PATCH v2 0/2] Add script to download aarch64 cross compiler toolchain

['Stewart Hildebrand' via OSv Development]

Stewart Hildebrand (2):
  build: set arch in arg-parsing loop
  Add script to download aarch64 cross compiler toolchain

 scripts/build | 14 +--
 scripts/download_aarch64_packages.py  |  2 +
 scripts/download_aarch64_toolchain.sh | 55 +++
 3 files changed, 68 insertions(+), 3 deletions(-)
 create mode 100644 scripts/download_aarch64_toolchain.sh

-- 
2.30.0

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/20210208184545.97466-1-stewart.hildebrand%40dornerworks.com.

[osv-dev] [PATCH v2 1/2] build: set arch in arg-parsing loop

['Stewart Hildebrand' via OSv Development]

Add a case in the initial argument parsing loop to both set the variable and
append it to the args array.

In the follow up patch, there will be a need to know if we are doing a cross
build. In this case, passing arch=aarch64 is required, thus the arch= variable
will be set in the initial arg parsing loop.

If arch= isn't given on the build command line, it will be unset initially.
That's okay, since we only need the arch variable early if it's explicitly given
on the build command line. The arch variable will still be set later in the
script regardless.

While we're here, use a cleaner syntax for appending to an array and fixup an
outdated comment.

Signed-off-by: Stewart Hildebrand 

---

v2:
  Don't change the default arch logic
  Fixup comment
---
 scripts/build | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/scripts/build b/scripts/build
index b494141c..250d5360 100755
--- a/scripts/build
+++ b/scripts/build
@@ -22,7 +22,7 @@ usage() {
 
Options:
  --help|-h Print this help message
- arch=x64|aarch64  Specify the build architecture; default 
is x64
+ arch=x64|aarch64  Specify the build architecture; default 
is the same as build host arch
  mode=release|debugSpecify the build mode; default is 
release
  export=none|selected|all  If 'selected' or 'all' export the app 
files to 
  export_dir=  The directory to export the files to; 
default is build/export
@@ -80,8 +80,11 @@ do
image=*|modules=*|fs=*|usrskel=*|check|--append-manifest|--create-disk) 
;;
clean)
stage1_args=clean ;;
-   *)  # yuck... Is there a prettier way to append to array?
-   args[${#args[@]}]="$i" ;;
+   arch=*)
+   eval ${i}
+   args+=("$i") ;;
+   *)
+   args+=("$i") ;;
esac
 done
 
-- 
2.30.0

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/20210208184545.97466-2-stewart.hildebrand%40dornerworks.com.

[osv-dev] Re: [PATCH] scripts: download and build arbitrary version of boost

['Stewart Hildebrand' via OSv Development]

On Friday, February 5, 2021 at 4:22:33 PM UTC-5 jwkoz...@gmail.com wrote:

> On some distributions, like CentOS, the default version of boost 
> is really old. So the best option is to build a newer one from source. 
> Also even on Fedora or Ubuntu it might be useful to point to 
> different version of boost other than what ./scripts/setup.py installs. 
>
> This script automates process of downloading the sources of specified 
> version 
> of boost and building it using default or specified toolchain. 
> The last CROSS_AARCH64_TOOLSET parameter (gcc or gcc-arm) is intended to 
> cross-compile 
> the aarch64 version of boost on x86_64 machine. 
>
> The built version of the boost is symlinked into the 
> build/downloaded_packages//boost/install/ directory and will 
> automatically be used where cross-compiling aarch64 OSv kernel. 
> In order to use x64 version of boost built using this script 
> we still need to make changes to the main makefile and 
> modules/common.gmk. 
>
> Signed-off-by: Waldemar Kozaczuk  
> --- 
> scripts/download_and_build_boost.sh | 63 + 
> 1 file changed, 63 insertions(+) 
> create mode 100755 scripts/download_and_build_boost.sh 
>
> diff --git a/scripts/download_and_build_boost.sh 
> b/scripts/download_and_build_boost.sh 
> new file mode 100755 
> index ..7181f5d4 
> --- /dev/null 
> +++ b/scripts/download_and_build_boost.sh 
> @@ -0,0 +1,63 @@ 
> +#!/bin/bash 
> +# 
> +# Copyright (C) 2020 Waldemar Kozaczuk 
> +# 
> +# This work is open source software, licensed under the terms of the 
> +# BSD license as described in the LICENSE file in the top-level 
> directory. 
> +# 
> + 
> +# Usage: 
> +# ./scripts/download_and_build_boost.sh   
>  
> +# 
> + 
> +OSV_ROOT=$(dirname $(readlink -e $0))/.. 
> +echo $OSV_ROOT 
> + 
> +BOOST_VERSION=$1 
> +if [[ "${BOOST_VERSION}" == "" ]]; then 
> + BOOST_VERSION="1.70.0" 
> +fi 
> +BOOST_VERSION2=${BOOST_VERSION//[.]/_} 
> + 
> +ARCH=$2 
> +if [[ "${ARCH}" == "" ]]; then 
> + ARCH=$(uname -m)


This is nitpicky, but uname -m wouldn't correspond to the arch of HOST_CXX 
(g++) when chrooting into a aarch64 sysroot. This is a nitpick (not an 
objection) because it's probably an unusual situation to find yourself in. 
If really want to chase this down, I recommend replicating the logic from 
the main Makefile.
 

>
> +fi 
> + 
> +#If CROSS_AARCH64_TOOLSET is set or passed we assume we are 
> crosscompiling aarch64 on Intel 
> +CROSS_AARCH64_TOOLSET=$3 
> + 
> +rm -rf %s/boost/install 
> + 
> +TAR_DOWNLOAD_DIR=${OSV_ROOT}/build/downloaded_packages/${ARCH}/boost/upstream/
>  
>
> +mkdir -p ${TAR_DOWNLOAD_DIR} 
> + 
> +BOOST_URL=
> https://dl.bintray.com/boostorg/release/${BOOST_VERSION}/source/boost_${BOOST_VERSION2}.tar.gz
>  
> +wget -c -O ${TAR_DOWNLOAD_DIR}/boost_${BOOST_VERSION2}.tar.gz 
> ${BOOST_URL} 
> + 
> +pushd ${TAR_DOWNLOAD_DIR} 
> +rm -rf ./boost_${BOOST_VERSION2} && tar -xf 
> ./boost_${BOOST_VERSION2}.tar.gz 
> +cd ./boost_${BOOST_VERSION2} 
> +./bootstrap.sh 
> --with-libraries=system,thread,test,chrono,regex,date_time,filesystem,locale,random,atomic,log,program_options
>  
>
> + 
> +BOOST_DIR=$(pwd) 
> +case $CROSS_AARCH64_TOOLSET in 
> + gcc) 
> + echo "using gcc : arm : aarch64-linux-gnu-g++ ;" > user-config.jam ;; 
> + gcc-arm) 
> + echo "using gcc : arm : aarch64-none-linux-gnu-g++ ;" > user-config.jam 
> ;; 
> +esac


There is a possibility of many more combinations of aarch64-*-g++. 
aarch64-none-elf-, aarch64-xilinx-linux-, to name a couple. Instead of 
trying to predict all possible combinations, I recommend using CROSS_PREFIX 
instead.


> + 
> +B2_OPTIONS="threading=multi" 
> +if [[ "${CROSS_AARCH64_TOOLSET}" != "" ]]; then 
> + B2_OPTIONS="${B2_OPTIONS} --user-config=${BOOST_DIR}/user-config.jam 
> toolset=${CROSS_AARCH64_TOOLSET} architecture=arm address-model=64"


I'm not convinced that the toolset= option should correspond to a 
particular CROSS_PREFIX. Wouldn't we always want toolset=gcc-arm for 
aarch64 builds?


> +fi 
> +./b2 ${B2_OPTIONS} -j$(nproc) 
> + 
> +# 
> +# Create symlinks to install boost and make it visible to OSv makefile 
> +rm -f ../../install 
> +ln -s upstream/boost_${BOOST_VERSION2}/stage ../../install 
> +mkdir -p stage/usr/include 
> +ln -s ../../../boost stage/usr/include/boost 
> +popd 
> -- 
> 2.29.2 
>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/63b192b5-3448-464a-9510-69b2a0ea38f8n%40googlegroups.com.

[osv-dev] [COMMIT osv master] aarch64: make dl_tests makefile build for aarch64

[Commit Bot]

From: Waldemar Kozaczuk 
Committer: Waldemar Kozaczuk 
Branch: master

aarch64: make dl_tests makefile build for aarch64

This patch changes dl_tests makefile to delegate to modules/common.gmk
to support building dl_tests module for both x64 and aarch64.

This patch also effectively makes tst-dlfcn.so to pass on aarch64.

Signed-off-by: Waldemar Kozaczuk 

---
diff --git a/modules/dl_tests/Makefile b/modules/dl_tests/Makefile
--- a/modules/dl_tests/Makefile
+++ b/modules/dl_tests/Makefile
@@ -1,30 +1,15 @@
-src := $(OSV_BASE)
-out := $(OSV_BUILD_PATH)
-arch := $(ARCH)
-fs_type := $(fs_type)
-bionic_test_libs := $(src)/modules/dl_tests/bionic/tests/libs
-
-quiet = $(if $V, $1, @echo " $2"; $1)
-very-quiet = $(if $V, $1, @$1)
-makedir = $(call very-quiet, mkdir -p $(dir $@))
-
-autodepend = -MD -MT $@ -MP
+include ../common.gmk
 
-INCLUDES = -I$(src)/arch/$(ARCH) -I$(src) -I$(src)/include \
-   -I$(src)/arch/common -isystem $(src)/include/glibc-compat \
-   $(shell $(CXX) -E -xc++ - -v &1 | awk '/^End/ {exit} /^ 
.*c\+\+/ {print "-isystem" $$0}') \
-   -isystem $(src)/include/api -isystem $(src)/include/api/$(ARCH) \
-   -isystem $(out)/gen/include
+bionic_test_libs := $(src)/modules/dl_tests/bionic/tests/libs
 
-COMMON = $(autodepend) $(INCLUDES) -g -O2 -fPIC -DBOOST_TEST_DYN_LINK \
+COMMON += -g -O2 -fPIC -DBOOST_TEST_DYN_LINK \
-U _FORTIFY_SOURCE -D_KERNEL -D__OSV__ -DCONF_debug_memory=0 \
-Wall -Wno-pointer-arith -Wformat=0 -Wno-format-security
 
-LIBS =
+LIBS = $(libgcc_s_dir)/libgcc_s.so.1
 
 CXXFLAGS = -std=gnu++11 $(COMMON)
-CXXFLAGS2 = -Wl,--no-as-needed -L=$(out)/dl_tests -L=/usr/lib -Wl,-rpath . 
-Wl,-rpath / -Wl,-rpath /usr/lib $(CXXFLAGS)
-
+CXXFLAGS2 = $(CXXFLAGS) -Wl,--no-as-needed -L$(out)/dl_tests/ -Wl,-rpath . 
-Wl,-rpath / -Wl,-rpath /usr/lib
 CFLAGS = -std=gnu99 $(COMMON)
 
 tests := libtest_simple.so libtest_empty.so 
libtest_dlsym_from_this_grandchild.so \
@@ -37,39 +22,39 @@ module: usr.manifest $(all_tests)
 
 $(out)/dl_tests/libtest_simple.so: 
$(bionic_test_libs)/dlopen_testlib_simple.cpp
$(makedir)
-   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS) -D__SHARED_OBJECT__=1 
-shared -o $@ $<, CXX dl_tests/libtest_simple.so)
+   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS) $(LDFLAGS) 
-D__SHARED_OBJECT__=1 -shared -o $@ $<, CXX dl_tests/libtest_simple.so)
 
 $(out)/dl_tests/libtest_empty.so: $(bionic_test_libs)/empty.cpp
$(makedir)
-   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS) -D__SHARED_OBJECT__=1 
-shared -o $@ $<, CXX dl_tests/libtest_empty.so)
+   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS) $(LDFLAGS) 
-D__SHARED_OBJECT__=1 -shared -o $@ $<, CXX dl_tests/libtest_empty.so)
 
 $(out)/dl_tests/libtest_check_rtld_next_from_library.so: 
$(bionic_test_libs)/check_rtld_next_from_library.cpp
$(makedir)
-   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS) -D__SHARED_OBJECT__=1 
-shared -o $@ $<, CXX dl_tests/libtest_check_rtld_next_from_library.so)
+   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS) $(LDFLAGS) 
-D__SHARED_OBJECT__=1 -shared -o $@ $<, CXX 
dl_tests/libtest_check_rtld_next_from_library.so)
 
 $(out)/dl_tests/libtest_dlsym_from_this_grandchild.so: 
$(bionic_test_libs)/dlsym_from_this_symbol2.cpp
$(makedir)
-   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS) -D__SHARED_OBJECT__=1 
-shared -o $@ $<, CXX dl_tests/libtest_dlsym_from_this_grandchild.so)
+   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS) $(LDFLAGS) 
-D__SHARED_OBJECT__=1 -shared -o $@ $<, CXX 
dl_tests/libtest_dlsym_from_this_grandchild.so)
 
 $(out)/dl_tests/libtest_dlsym_from_this_child.so: 
$(bionic_test_libs)/dlsym_from_this_functions.cpp \
$(out)/dl_tests/libtest_dlsym_from_this_grandchild.so
$(makedir)
-   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS2) 
-ltest_dlsym_from_this_grandchild -D__SHARED_OBJECT__=1 -shared -o $@ $<, CXX 
dl_tests/libtest_dlsym_from_this_child.so)
+   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS2) $(LDFLAGS) 
-ltest_dlsym_from_this_grandchild -D__SHARED_OBJECT__=1 -shared -o $@ $<, CXX 
dl_tests/libtest_dlsym_from_this_child.so)
 
 $(out)/dl_tests/libtest_dlsym_from_this.so: 
$(bionic_test_libs)/dlsym_from_this_symbol.cpp \
$(out)/dl_tests/libtest_dlsym_from_this_child.so
$(makedir)
-   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS2) 
-ltest_dlsym_from_this_child -D__SHARED_OBJECT__=1 -shared -o $@ $<, CXX 
dl_tests/libtest_dlsym_from_this.so)
+   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS2) $(LDFLAGS) 
-ltest_dlsym_from_this_child -D__SHARED_OBJECT__=1 -shared -o $@ $<, CXX 
dl_tests/libtest_dlsym_from_this.so)
 
 $(out)/dl_tests/libdlext_test.so: $(bionic_test_libs)/dlext_test_library.cpp \
$(out)/dl_tests/libtest_simple.so
$(makedir)
-   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS2) -ltest_simple 
-D__SHARED_OBJECT__=1 -shared -o $@ $<, CXX dl_tests/libdlext_test.so)
+   $(call quiet, cd $(out); $(CXX) $(CXXFLAGS2) $(LDFLAGS) -ltest_simple 
-D

[osv-dev] [PATCH] setup: support CentOS 7

[Waldemar Kozaczuk]

This patch modifies setup.py to support installing necessary
packages to build and test OSv on CentOS 7.

Please note that unlike for other distributions, we do
not use default packages for gcc or qemu as these are pretty old.
Instead we enable SCL repo and install Developet Toolset 9 which comes
with gcc 9 (c, c++) and other binaries.

There is no package repo with qemu (I guess we could install one from
Fedora) so instead we provide a script to build qemu from sources (see
other patch).

Signed-off-by: Waldemar Kozaczuk 
---
 scripts/setup.py | 53 ++--
 1 file changed, 51 insertions(+), 2 deletions(-)

diff --git a/scripts/setup.py b/scripts/setup.py
index 2289e100..7413db7b 100755
--- a/scripts/setup.py
+++ b/scripts/setup.py
@@ -126,7 +126,7 @@ class Fedora(object):
 versions = [Fedora_27, Fedora_28, Fedora_29, Fedora_30, Fedora_31, 
Fedora_32, Fedora_33]
 
 class RHELbased(Fedora):
-name = ['Scientific Linux', 'NauLinux', 'CentOS Linux', 'Red Hat 
Enterprise Linux', 'Oracle Linux']
+name = ['Scientific Linux', 'NauLinux', 'Red Hat Enterprise Linux', 
'Oracle Linux']
 
 class RHELbased_70(object):
 packages = []
@@ -336,12 +336,55 @@ class LinuxMint(Ubuntu):
 
 versions = [LinuxMint_18_03, LinuxMint_19]
 
+class CentOS(object):
+name = 'CentOS Linux'
+install = 'yum -y install'
+packages = [
+'ant',
+'autoconf',
+'automake',
+'curl',
+'git',
+'gnutls-utils',
+'libtool',
+'maven',
+'maven-shade-plugin',
+'openssl',
+'openssl-libs',
+'openssl-devel',
+'p11-kit',
+'patch',
+'python3-requests',
+'qemu-img',
+'tcpdump',
+'unzip',
+'wget',
+'yaml-cpp-devel',
+'pax-utils',
+'java-1.8.0-openjdk',
+'devtoolset-9-toolchain',
+ ]
+
+test_packages = ['openssl-devel']
+
+class CentOS_7(object):
+pre_install = 'yum -y install epel-release centos-release-scl 
centos-release-scl-rh 
https://packages.endpoint.com/rhel/7/os/x86_64/endpoint-repo-1.7-1.x86_64.rpm'
+packages = []
+ec2_packages = []
+test_packages = []
+ec2_post_install = None
+post_install = 'echo "---> Run \'scl enable devtoolset-9 bash\' or add 
\'source /opt/rh/devtoolset-9/enable\' to ~/.bashrc or ~/.bash_profile" to 
enable GCC 9 before building OSv !'
+version = '7'
+
+versions = [CentOS_7]
+
 distros = [
Debian(),
Fedora(),
Ubuntu(),
LinuxMint(),
-   RHELbased()
+   RHELbased(),
+   CentOS()
]
 
 parser = argparse.ArgumentParser(prog='setup')
@@ -366,6 +409,8 @@ for distro in distros:
 if dver.version == version or version.startswith(dver.version+'.'):
 if hasattr(distro, 'pre_install'):
 subprocess.check_call(distro.pre_install, shell=True)
+if hasattr(dver, 'pre_install'):
+subprocess.check_call(dver.pre_install, shell=True)
 pkg = distro.packages + dver.packages
 if cmdargs.ec2:
 pkg += distro.ec2_packages + dver.ec2_packages
@@ -377,6 +422,10 @@ for distro in distros:
 subprocess.check_call(distro.ec2_post_install, 
shell=True)
 if dver.ec2_post_install:
 subprocess.check_call(dver.ec2_post_install, 
shell=True)
+if hasattr(distro, 'post_install'):
+subprocess.check_call(distro.post_install, shell=True)
+if hasattr(dver, 'post_install'):
+subprocess.check_call(dver.post_install, shell=True)
 sys.exit(0)
 print ('Your distribution %s version %s is not supported by this 
script' % (name, version))
 sys.exit(1)
-- 
2.29.2

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/20210208172309.1341773-1-jwkozaczuk%40gmail.com.

Re: [osv-dev] [PATCH] sched: enforce tls register refresh after thread switch

[Waldek Kozaczuk]

On Monday, February 8, 2021 at 9:24:25 AM UTC-5 Nadav Har'El wrote:

> On Mon, Feb 8, 2021 at 7:55 AM Waldemar Kozaczuk  
> wrote:
>
>> On RISC architecture like aarch64 the thread register tpidr_el0 that
>> holds an address of thread local storage memory for current thread,
>> is never accessed directly in one instruction like on x86_64. Instead
>> compiler generates machone code that copies value of the tpidr_el0
>> to another generic register when reading and does the opposite
>> when writing.
>>
>> As the issue #1121 explains, the reschedule_from_interrupt() method
>> uses thread local variable 'current_cpu' to detect any pending thread to
>> de destroyed for current cpu after switching from old thread
>> to a new one. It must use TLS variable instead of any stack variables
>> because those would be invalid at this point as they would be on the
>> stack of the old thread switched from.
>>
>> This all works fine on x86_64, where every read or write of
>> thread register fs is performed in one instruction like so:
>>
>> mov%fs:0xf948,%rax
>>
>> On aarch64, the value of the thread register tpidr_el0 is copied
>> once to another register x21 before access of first TLS variable in this
>> method and never refreshed after that. When reschedule_from_interrupt()
>> calls switch_to() to switch to new thread, the update of the tpidr_el0
>> register is done in assembly and compiler has no idea that we are
>> changing TLS register. So after return to reschedule_from_interrupt()
>> before it calls cpu::current(), the register x21 still holds old
>> value of tpidr_el0 register.
>>
>> In most cases this does not cause any harm, as typically the old
>> and new thread holds the same current_cpu address. But it leads to very
>> ugly crashes when sched::pin() is used which calls
>> reschedule_from_interrupt() after changing its current_cpu to a new one
>> it is pinning the current thread to.
>>
>> So in order to fix this problem, this patch delegates the code to
>> destroy current cpu terminating thread to a new class method
>> destroy_current_cpu_thread() to enforce fresh reading of the TLS
>> register.
>>
>> Fixes #1121
>>
>> Signed-off-by: Waldemar Kozaczuk 
>> ---
>>  core/sched.cc| 19 ++-
>>  include/osv/sched.hh |  1 +
>>  2 files changed, 15 insertions(+), 5 deletions(-)
>>
>> diff --git a/core/sched.cc b/core/sched.cc
>> index 06f849d1..45427cac 100644
>> --- a/core/sched.cc
>> +++ b/core/sched.cc
>> @@ -229,6 +229,13 @@ void cpu::schedule()
>>  }
>>  }
>>
>> +void __attribute__ ((noinline)) cpu::destroy_current_cpu_thread() {
>> +if (cpu::current()->terminating_thread) {
>> +cpu::current()->terminating_thread->destroy();
>> +cpu::current()->terminating_thread = nullptr;
>> +}
>> +}
>>
>
> Nice catch, but it's really sad we need to call another function, which 
> slows down the context switch (even if a bit) also on x86 where this isn't 
> needed at all.
> I think there must be a better solution:
>
> You are saying that the problem arises because switch_to() set one 
> register, but didn't set its copy to the same value.
> So, can't we change switch_to() - on aarch64 only - to also make this 
> additional copy?
>

The problem is that we do not know which generic register compiler is going 
to use. And it decides it in reschedule_from_interrupt() before this line:

thread* p = thread::current(); 

which uses TLS access. So switch_to() (which is not even inlined function) 
would have no way of knowing which register to copy to.

Another alternative is to make this code ARCH dependant with #ifdef so we 
can keep the x86_64 code and change aarch64 only.

Ideally, if there was a compiler directive to enforce re-read of the 
tpidr_el0 register.


> +
>>  void cpu::reschedule_from_interrupt(bool called_from_yield,
>>  thread_runtime::duration 
>> preempt_after)
>>  {
>> @@ -338,15 +345,17 @@ void cpu::reschedule_from_interrupt(bool 
>> called_from_yield,
>>  }
>>  n->switch_to();
>>
>> -// Note: after the call to n->switch_to(), we should no longer use 
>> any of
>> +// Note 1: after the call to n->switch_to(), we should no longer use 
>> any of
>>  // the local variables, nor "this" object, because we just switched 
>> to n's
>>  // stack and the values we can access now are those that existed in 
>> the
>>  // reschedule call which scheduled n out, and will now be returning.
>>  // So to get the current cpu, we must use cpu::current(), not "this".
>> -if (cpu::current()->terminating_thread) {
>> -cpu::current()->terminating_thread->destroy();
>> -cpu::current()->terminating_thread = nullptr;
>> -}
>> +// Note 2: in addition, in order to destroy any pending thread to be 
>> destroyed
>> +// for current CPU, we delegate to a class static method 
>> destroy_current_cpu_thread
>> +// to enforce that TLS register is refreshed before we reference 
>> cu

[osv-dev] Re: Nginx failing after many requests

[Waldek Kozaczuk]

Hi,

On Wednesday, February 3, 2021 at 11:47:13 AM UTC-5 Matthew Kenigsberg 
wrote:

>
> Hi,
>
> I'm trying to benchmark nginx with weighttp, but it appears to crash after 
> too many requests are sent. As soon as requests start failing, it seems all 
> following requests will fail. Any ideas why?
>
Not really but I think I may have seen similar behavior with nginx. 

>  
>

> $ curl ip
> 
> 
> 
> Welcome to nginx!
> ...
> $ weighttp -t 1 -n 1000 -c 1 http://ip
> ...
> requests: 1000 total, 1000 started, 1000 done, 1000 succeeded, 0 failed, 0 
> errored
> ...
>
> $ weighttp -t 1 -n 1000 -c 1 http://ip
> ...
> requests: 1000 total, 1000 started, 1000 done, 22 succeeded, 978 failed, 0 
> errored
> ...
>
> $ curl 129.114.109.194
> curl: (52) Empty reply from server
> $ sleep 60
> $ curl 129.114.109.194
> curl: (52) Empty reply from server
>
> Not sure if this is related to either of: 
> https://groups.google.com/u/1/g/osv-dev/c/jclM9NhO7eg/m/BXV20hJUAQAJ
> https://github.com/cloudius-systems/osv/issues/889
>
Maybe.
It would be best if you could connect with gdb to OSv when it crashes/hangs 
and send us a thread list and ideally stack traces of interesting threads 
(most likely those part of TCP stack). I suggest you build and run ROFS 
image as it will have fewer threads. 
See https://github.com/cloudius-systems/osv/wiki/Debugging-OSv.

Does this crash happen with single cpu or with many?
 

>
>
> Thanks,
> Matthew
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/906a7d15-dcc7-44cf-9cc2-867de8657d88n%40googlegroups.com.

Re: [osv-dev] [PATCH] sched: enforce tls register refresh after thread switch

[Nadav Har'El]

On Mon, Feb 8, 2021 at 7:55 AM Waldemar Kozaczuk 
wrote:

> On RISC architecture like aarch64 the thread register tpidr_el0 that
> holds an address of thread local storage memory for current thread,
> is never accessed directly in one instruction like on x86_64. Instead
> compiler generates machone code that copies value of the tpidr_el0
> to another generic register when reading and does the opposite
> when writing.
>
> As the issue #1121 explains, the reschedule_from_interrupt() method
> uses thread local variable 'current_cpu' to detect any pending thread to
> de destroyed for current cpu after switching from old thread
> to a new one. It must use TLS variable instead of any stack variables
> because those would be invalid at this point as they would be on the
> stack of the old thread switched from.
>
> This all works fine on x86_64, where every read or write of
> thread register fs is performed in one instruction like so:
>
> mov%fs:0xf948,%rax
>
> On aarch64, the value of the thread register tpidr_el0 is copied
> once to another register x21 before access of first TLS variable in this
> method and never refreshed after that. When reschedule_from_interrupt()
> calls switch_to() to switch to new thread, the update of the tpidr_el0
> register is done in assembly and compiler has no idea that we are
> changing TLS register. So after return to reschedule_from_interrupt()
> before it calls cpu::current(), the register x21 still holds old
> value of tpidr_el0 register.
>
> In most cases this does not cause any harm, as typically the old
> and new thread holds the same current_cpu address. But it leads to very
> ugly crashes when sched::pin() is used which calls
> reschedule_from_interrupt() after changing its current_cpu to a new one
> it is pinning the current thread to.
>
> So in order to fix this problem, this patch delegates the code to
> destroy current cpu terminating thread to a new class method
> destroy_current_cpu_thread() to enforce fresh reading of the TLS
> register.
>
> Fixes #1121
>
> Signed-off-by: Waldemar Kozaczuk 
> ---
>  core/sched.cc| 19 ++-
>  include/osv/sched.hh |  1 +
>  2 files changed, 15 insertions(+), 5 deletions(-)
>
> diff --git a/core/sched.cc b/core/sched.cc
> index 06f849d1..45427cac 100644
> --- a/core/sched.cc
> +++ b/core/sched.cc
> @@ -229,6 +229,13 @@ void cpu::schedule()
>  }
>  }
>
> +void __attribute__ ((noinline)) cpu::destroy_current_cpu_thread() {
> +if (cpu::current()->terminating_thread) {
> +cpu::current()->terminating_thread->destroy();
> +cpu::current()->terminating_thread = nullptr;
> +}
> +}
>

Nice catch, but it's really sad we need to call another function, which
slows down the context switch (even if a bit) also on x86 where this isn't
needed at all.
I think there must be a better solution:

You are saying that the problem arises because switch_to() set one
register, but didn't set its copy to the same value.
So, can't we change switch_to() - on aarch64 only - to also make this
additional copy?

+
>  void cpu::reschedule_from_interrupt(bool called_from_yield,
>  thread_runtime::duration
> preempt_after)
>  {
> @@ -338,15 +345,17 @@ void cpu::reschedule_from_interrupt(bool
> called_from_yield,
>  }
>  n->switch_to();
>
> -// Note: after the call to n->switch_to(), we should no longer use
> any of
> +// Note 1: after the call to n->switch_to(), we should no longer use
> any of
>  // the local variables, nor "this" object, because we just switched
> to n's
>  // stack and the values we can access now are those that existed in
> the
>  // reschedule call which scheduled n out, and will now be returning.
>  // So to get the current cpu, we must use cpu::current(), not "this".
> -if (cpu::current()->terminating_thread) {
> -cpu::current()->terminating_thread->destroy();
> -cpu::current()->terminating_thread = nullptr;
> -}
> +// Note 2: in addition, in order to destroy any pending thread to be
> destroyed
> +// for current CPU, we delegate to a class static method
> destroy_current_cpu_thread
> +// to enforce that TLS register is refreshed before we reference
> current cpu
> +// using new thread TLS variable we switched to. This is necessary
> for architectures
> +// like aarch64 where tpidr_el0 register is copied to other one
> before use.
> +destroy_current_cpu_thread();
>  }
>
>  void cpu::timer_fired()
> diff --git a/include/osv/sched.hh b/include/osv/sched.hh
> index 0fb44f77..78fed622 100644
> --- a/include/osv/sched.hh
> +++ b/include/osv/sched.hh
> @@ -886,6 +886,7 @@ struct cpu : private timer_base::client {
>   */
>  void reschedule_from_interrupt(bool called_from_yield = false,
>  thread_runtime::duration preempt_after =
> thyst);
> +static void destroy_current_cpu_thread();
>  void enqueue(thread& t);
>  void init_

Re: [osv-dev] GDB stub

[Glauber Costa]

On Mon., Feb. 8, 2021, 3:00 a.m. Nadav Har'El,  wrote:

> On Wed, Feb 3, 2021 at 1:18 AM Waldek Kozaczuk 
> wrote:
>
>> Currently, it is only possible to connect from gdb to a running instance
>> of OSv on QEMU. But ideally, it would be nice to have a gdb stub in OSv
>> that would let one connect from gdb to OSv running on any hypervisor.
>>
>> Per this fragment of this Wiki -
>> https://github.com/cloudius-systems/osv/wiki/Debugging-OSv
>>
>> *"The fact that our gdb support works with the unit of cpus, not OSV
>> threads, makes it convenient to debug OSV itself, but less convenient to
>> debug application code running inside OSV. Particularly problematic is
>> single stepping support which isn’t currently supported well: The problem
>> is that when you try to look at a single OSV thread (see the next section),
>> a single-step operation (“s” or “n”) makes one step inside the vcpu, not
>> the osv thread*, and this step might include switching to a different
>> thread. One way to work around this problem is to compile OSV with
>> preemption disabled (not recommended, but can be a convenient trick in some
>> debugging situations). *A better approach would have been to include a
>> gdb stub into OSV itself, to allow gdb to see OSV threads directly and
>> single-step them. We’ve made some progress in implementing such a stub, but
>> it isn’t ready for prime-time yet.*"
>>
>> it looks like somebody may have implemented an initial version of the gdb
>> stub. Is there a working patch of it somewhere?
>>
>
> I have a vague recollection - it was many years ago - that Glauber (CCed)
> was the one who tried doing it.
> Glauber, if it was you, do you remember what you tried?
> If it wasn't you, sorry for the spam :-)
>

I don't think it was me. I have no memory of ever doing this.

But no need to be sorry, it's good to hear from you!

>
>
>> Regards,
>> Waldek
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "OSv Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to osv-dev+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/osv-dev/a3ca8d68-5994-4b51-94c4-539e2024f12dn%40googlegroups.com
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAA6-i6rAWgYqkCOPnGiQABpEfC%3DKqD1jCmmw82LEmrCNrHWyqA%40mail.gmail.com.

[osv-dev] Re: [PATCH 1/2] build: set arch earlier

['Stewart Hildebrand' via OSv Development]

On Friday, February 5, 2021 at 3:23:19 PM UTC-5 jwkoz...@gmail.com wrote:

> On Thursday, February 4, 2021 at 12:48:56 PM UTC-5 
> stewart.h...@dornerworks.com wrote:
>
>> Add a case in the initial argument parsing loop to both set the variable 
>> and 
>> append it to the args array. 
>>
>> While we're here, use a cleaner syntax for appending to an array. 
>>
>> If arch= isn't given on the build command line, explicitly set it to the 
>> default x64. 
>>
>> Signed-off-by: Stewart Hildebrand  
>> --- 
>> scripts/build | 10 +++--- 
>> 1 file changed, 7 insertions(+), 3 deletions(-) 
>>
>> diff --git a/scripts/build b/scripts/build 
>> index b494141c..d859202a 100755 
>> --- a/scripts/build 
>> +++ b/scripts/build 
>> @@ -80,11 +80,16 @@ do 
>> image=*|modules=*|fs=*|usrskel=*|check|--append-manifest|--create-disk) 
>> ;; 
>> clean) 
>> stage1_args=clean ;; 
>> - *) # yuck... Is there a prettier way to append to array? 
>> - args[${#args[@]}]="$i" ;; 
>> + arch=*) 
>> + eval ${i} 
>> + args+=("$i") ;; 
>> + *) 
>> + args+=("$i") ;; 
>> esac 
>> done 
>>
>> +arch=${arch:-x64}
>
> Should we really default to x64 or the host arch (uname -m)? I build often 
> on my Odroid machine natively.
> In reality, it probably should be like that:
>
> arch=$(uname -m)
>> if [ "${arch}" == "x86_64" ]; then
>> arch="x64"
>> fi
>>
>
I forgot to consider this case. I was misled by a comment further above in 
the build script that says "default is x64" (I'll update the comment in 
v2). Taking a step back, I don't actually need to set arch earlier. I only 
need to know if it's a cross build.

Assuming these are the supported build configs:
1. build==x64, target == x64 (arch may be omitted/unset)
2. build==x64, target == aarch64 (arch=aarch64)
3. build==aarch64, target==aarch64 (arch may be omitted/unset)

The only time we'd want to use the cross compiler toolchain is if build == 
x64 and target == aarch64. In this case, specifying arch=aarch64 is 
required, and it'll be set in the case I added to the arg-parsing loop 
above. For v2, I'll simply omit the default-setting line here.

+ 
>> make "${args[@]}" ${stage1_args} | tee build.out 
>> # check exit status of make 
>> status=${PIPESTATUS[0]} 
>> @@ -162,7 +167,6 @@ fs_size=${vars[fs_size]-$(($fs_size_mb*1024*1024))} 
>> fs_size=$((fs_size - (fs_size & 511))) 
>>
>> SRC=`pwd` 
>> -arch=`expr $OUT : '.*\.\(.*\)'`
>
> So I guess we are deciding to default to the host arch (or x64)  instead 
> of whatever the last build was, are we?
>

I'll leave this line in for v2.
 

>
>> mode=`expr $OUT : '.*/\(.*\)\..*'` 
>>
>> # Set "modules" according to the image= or modules= paramters, or some 
>> -- 
>> 2.30.0 
>>
>>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/2507e240-ba88-4c1d-85ed-8947f4f12109n%40googlegroups.com.

Re: [osv-dev] [PATCH] scripts: allow building QEMU from sources

[Nadav Har'El]

On Mon, Feb 8, 2021 at 5:12 AM Waldemar Kozaczuk 
wrote:

> Add new script download_and_build_qemu.sh that allows building QEMU from
> sources. It installs all necessary packages to build QEMU from sources,
> downloads QEMU sources and builds both x86_64 and aarch64 versions of
> it.
>
> Signed-off-by: Waldemar Kozaczuk 
> ---
>  scripts/download_and_build_qemu.sh | 61 ++
>  1 file changed, 61 insertions(+)
>  create mode 100755 scripts/download_and_build_qemu.sh
>
> diff --git a/scripts/download_and_build_qemu.sh
> b/scripts/download_and_build_qemu.sh
> new file mode 100755
> index ..711882ec
> --- /dev/null
> +++ b/scripts/download_and_build_qemu.sh
> @@ -0,0 +1,61 @@
> +#!/bin/bash
> +#
> +# Copyright (C) 2020 Waldemar Kozaczuk
> +#
> +# This work is open source software, licensed under the terms of the
> +# BSD license as described in the LICENSE file in the top-level directory.
> +#
> +
> +# This scripts installs all necessary packages to build QEMU from sources,
> +# downloads QEMU sources and builds both x86_64 and aarch64 versions of it
> +#
> +# Usage:
> +#  ./scripts/download_and_build_qemu.sh 
> +#
> +
> +#Install tools
> +USER_ID=$(id -u)
> +LINUX_DIST_ID=$(python3 -c "import distro; print(distro.id())")
>

I guess that's ok, but personally I prefer something simpler, like using
"test -f /etc/redhat-release" or "test -f /etc/fedora-release" to check if
this is a redhat-like system (including Fedora) and specifically fedora.

+
> +case ${LINUX_DIST_ID} in
> +  fedora|centos)
> +PACKAGES="ninja-build cmake3 glib2-devel libfdt-devel pixman-devel
> zlib-devel libaio-devel libcap-devel libiscsi-devel"
> +if [[ "${LINUX_DIST_ID}" == "fedora" ]]; then
> +  PACKAGE_MANAGER=dnf
> +else
> +  PACKAGE_MANAGER=yum
> +fi ;;
> +  ubuntu)
> +PACKAGES="ninja-build cmake libglib2.0-dev libfdt-dev libpixman-1-dev
> zlib1g-dev libaio-dev libcap-dev libnfs-dev libiscsi-dev"
> +PACKAGE_MANAGER="apt-get" ;;
> +  *)
> +echo "Unsupported distribution!" && exit 1
> +esac
> +
> +if [[ "${USER_ID}" == "0" ]]; then
> +  echo "Installing all necessary packages!" && ${PACKAGE_MANAGER} -y
> install ${PACKAGES}
>

I personally prefer not to add the "-y". I think it's a good idea to allow
the user, who has no idea that this script is even planning to install
anything, say ok if anything needs to be installed.

+else
> +  echo "Needs super user access to install all necessary packages!" &&
> sudo ${PACKAGE_MANAGER} -y install ${PACKAGES}
>
+fi
> +
> +#Download and build qemu
> +QEMU_VERSION=$1
> +QEMU_VERSION=${QEMU_VERSION:-v5.2.0}
> +
> +OSV_ROOT=$(dirname $(readlink -e $0))/..
>
+BUILD_DIR=${OSV_ROOT}/build/downloaded_packages
> +
> +mkdir -p ${BUILD_DIR}
>

Please use double quotes on $BUILD_DIR here an in most lines below. If the
current directory's name has a space in it, all this code will break.

+pushd ${BUILD_DIR}
> +
> +if [[ ! -d qemu ]]; then
> +  git clone --depth 1 --branch ${QEMU_VERSION} git://
> git.qemu.org/qemu.git
> +fi
> +cd qemu
> +mkdir -p build && cd build
> +
> +../configure --target-list=x86_64-softmmu,aarch64-softmmu
> +make -j$(nproc)
> +popd
> +
> +echo "Built QEMU at ${BUILD_DIR}/qemu/build/qemu-system-x86_64 and
> ${BUILD_DIR}/qemu/build/qemu-system-aarch64"
> --
> 2.29.2
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/20210208031209.1266843-1-jwkozaczuk%40gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CANEVyjsZqQ7rJXnpR1ibup6R_BSJ1yXKV5KNFV_p9c6PQwryzw%40mail.gmail.com.

[osv-dev] [COMMIT osv master] tests: handle more generic CROSS_PREFIX for aarch64 builds

[Commit Bot]

From: Waldemar Kozaczuk 
Committer: Nadav Har'El 
Branch: master

tests: handle more generic CROSS_PREFIX for aarch64 builds

Instead of the gcc specific 'aarch64-linux-gnu-' CROSS_PREFIX,
we look for more generic one that starts with aarch64. This makes
the tests makefile function properly with toolchains other than gcc.

Also we explicitly set LD_LIBRARY_PATH to boost-lib-dir set by
modules/common.gmk to make ldd find locations of boost binaries
even where tests are built against arbitrary version of boost specified
by the boost_base parameter.

Signed-off-by: Waldemar Kozaczuk 
Message-Id: <20210208012512.1167449-1-jwkozac...@gmail.com>

---
diff --git a/modules/tests/Makefile b/modules/tests/Makefile
--- a/modules/tests/Makefile
+++ b/modules/tests/Makefile
@@ -229,10 +229,10 @@ build_all_tests: $(all_tests:%=$(out)/%)
 usr.manifest: build_all_tests $(lastword $(MAKEFILE_LIST)) usr.manifest.skel 
FORCE
@echo "  generating modules/tests/usr.manifest"
@cat $@.skel > $@
-   @if [ "$(CROSS_PREFIX)" = "aarch64-linux-gnu-" ]; then \
+   @if [[ "$(CROSS_PREFIX)" == aarch64* ]]; then \
./add_aarch64_boost_libraries.sh $(OSV_BASE) >> $@; \
else \
-   ldd $(addprefix $(out)/tests/,$(boost-tests)) | grep libboost | 
sed 's/ *[^ ] *\(.*\) => \(.*\) .*/\/usr\/lib\/\1: \2/' | sort | uniq >> $@; \
+   LD_LIBRARY_PATH=$(boost-lib-dir) ldd $(addprefix 
$(out)/tests/,$(boost-tests)) | grep libboost | sed 's/ *[^ ] *\(.*\) => \(.*\) 
.*/\/usr\/lib\/\1: \2/' | sort | uniq >> $@; \
fi
@echo $(all_tests) | tr ' ' '\n' | grep -v "tests/rofs/tst-.*.so" | awk 
'{print "/" $$0 ": ./" $$0}' >> $@
@echo $(all_tests) | tr ' ' '\n' | grep "tests/rofs/tst-.*.so" | sed 
's/\.so//' | awk 'BEGIN { FS = "/" } ; { print "/tests/" $$3 "-rofs.so: 
./tests/" $$2 "/" $$3 ".so"}' >> $@
@@ -243,10 +243,10 @@ FORCE:
 common.manifest: build_all_tests $(lastword $(MAKEFILE_LIST)) 
usr.manifest.skel FORCE
@echo "  generating modules/tests/common.manifest"
@cat usr.manifest.skel > $@
-   @if [ "$(CROSS_PREFIX)" = "aarch64-linux-gnu-" ]; then \
+   @if [[ "$(CROSS_PREFIX)" == aarch64* ]]; then \
./add_aarch64_boost_libraries.sh $(OSV_BASE) >> $@; \
else \
-   ldd $(addprefix $(out)/tests/,$(boost-tests)) | grep libboost | 
sed 's/ *[^ ] *\(.*\) => \(.*\) .*/\/usr\/lib\/\1: \2/' | sort | uniq >> $@; \
+   LD_LIBRARY_PATH=$(boost-lib-dir) ldd $(addprefix 
$(out)/tests/,$(boost-tests)) | grep libboost | sed 's/ *[^ ] *\(.*\) => \(.*\) 
.*/\/usr\/lib\/\1: \2/' | sort | uniq >> $@; \
fi
@echo $(common-tests) $(solaris-tests) | tr ' ' '\n' | awk '{print 
"/tests/" $$0 ": ./tests/" $$0}' >> $@
 

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/bc41e405bad095e0%40google.com.

[osv-dev] [COMMIT osv master] scripts/run.py: use QEMU_PATH to use arbitrary qemu executable

[Commit Bot]

From: Waldemar Kozaczuk 
Committer: Nadav Har'El 
Branch: master

scripts/run.py: use QEMU_PATH to use arbitrary qemu executable

Sometimes it is desirable to use arbitrary version of QEMU executable
when running OSv on it. The scripts/run.py takes the '--qemu-path'
already but there are other scripts like upload_manifest.py or scripts/build
that delegate to run.py but either do not accept qemu path argument or do not
pass it to run.py.

So instead, we allow one set an environment variable QEMU_PATH
which would be used if set or exported  in the shell invoking
run.py or other scripts calling it.

Signed-off-by: Waldemar Kozaczuk 
Message-Id: <20210207192452.995-1-jwkozac...@gmail.com>

---
diff --git a/scripts/run.py b/scripts/run.py
--- a/scripts/run.py
+++ b/scripts/run.py
@@ -271,7 +271,7 @@ def start_osv_qemu(options):
 qemu_env = os.environ.copy()
 
 qemu_env['OSV_BRIDGE'] = options.bridge
-qemu_path = options.qemu_path or ('qemu-system-%s' % options.arch)
+qemu_path = options.qemu_path or qemu_env.get('QEMU_PATH') or 
('qemu-system-%s' % options.arch)
 cmdline = [qemu_path] + args
 if options.dry_run:
 print(format_args(cmdline))

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/6de0d605bacfc8ed%40google.com.

[osv-dev] [COMMIT osv master] aarch64: handle R_AARCH64_COPY relocations

[Commit Bot]

From: Waldemar Kozaczuk 
Committer: Nadav Har'El 
Branch: master

aarch64: handle R_AARCH64_COPY relocations

This patch enhances dynamic linker to handle COPY relocations
for aarch64 which we do exactly same way for x86_64. This is enough
to support running Python 3 on OSv ARM.

Signed-off-by: Waldemar Kozaczuk 
Message-Id: <20210207145747.1093420-1-jwkozac...@gmail.com>

---
diff --git a/arch/aarch64/arch-elf.cc b/arch/aarch64/arch-elf.cc
--- a/arch/aarch64/arch-elf.cc
+++ b/arch/aarch64/arch-elf.cc
@@ -47,9 +47,11 @@ bool object::arch_relocate_rela(u32 type, u32 sym, void 
*addr,
 case R_AARCH64_ABS64:
 *static_cast(addr) = symbol(sym).relocated_addr() + addend;
 break;
-case R_AARCH64_COPY:
-abort();
+case R_AARCH64_COPY: {
+symbol_module sm = symbol_other(sym);
+memcpy(addr, sm.relocated_addr(), sm.size());
 break;
+}
 case R_AARCH64_GLOB_DAT:
 case R_AARCH64_JUMP_SLOT:
 *static_cast(addr) = symbol(sym).relocated_addr() + addend;

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/1e23c605bacf6e63%40google.com.

[osv-dev] [COMMIT osv master] aarch64: fix loader.py to use correct IP register

[Commit Bot]

From: Waldemar Kozaczuk 
Committer: Nadav Har'El 
Branch: master

aarch64: fix loader.py to use correct IP register

The scripts/loader.py used by gdb to provide extra OSv specific
capabilities uses wrong register x30 to reference IP - Instruction
Pointer. The x30 is actually used as a LR - Link Register that holds
an address to return to when branching with BL.

So instead this patch changes loader.py to use proper PC register.

Signed-off-by: Waldemar Kozaczuk 
Message-Id: <20210207144947.1092704-1-jwkozac...@gmail.com>

---
diff --git a/scripts/loader.py b/scripts/loader.py
--- a/scripts/loader.py
+++ b/scripts/loader.py
@@ -747,7 +747,7 @@ def __init__(self, cpu_thread):
 def set_pointers(self):
 self.sp = ulong(gdb.parse_and_eval('$sp'))  #stack pointer
 self.fp = ulong(gdb.parse_and_eval('$x29')) #frame pointer
-self.ip = ulong(gdb.parse_and_eval('$x30')) #instruction pointer
+self.ip = ulong(gdb.parse_and_eval('$pc'))  #instruction pointer
 
 def template_arguments(gdb_type):
 n = 0
@@ -893,7 +893,7 @@ def __save_old_pointers(self):
 self.old_rbp = gdb.parse_and_eval('$rbp').cast(ulong_type)
 else:
 self.old_sp = gdb.parse_and_eval('$sp').cast(ulong_type)
-self.old_x30 = gdb.parse_and_eval('$x30').cast(ulong_type)
+self.old_pc = gdb.parse_and_eval('$pc').cast(ulong_type)
 self.old_x29 = gdb.parse_and_eval('$x29').cast(ulong_type)
 def __save_new_pointers(self, thread):
 if arch == 'x64':
@@ -902,7 +902,7 @@ def __save_new_pointers(self, thread):
 self.new_rbp = thread['_state']['rbp'].cast(ulong_type)
 else:
 self.new_sp = thread['_state']['sp'].cast(ulong_type)
-self.new_x30 = thread['_state']['pc'].cast(ulong_type)
+self.new_pc = thread['_state']['pc'].cast(ulong_type)
 self.new_x29 = thread['_state']['fp'].cast(ulong_type)
 def __reset_old_pointers(self):
 if arch == 'x64':
@@ -911,7 +911,7 @@ def __reset_old_pointers(self):
 gdb.execute('set $rbp = %s' % self.old_rbp)
 else:
 gdb.execute('set $sp = %s' % self.old_sp)
-gdb.execute('set $x30 = %s' % self.old_x30)
+gdb.execute('set $pc = %s' % self.old_pc)
 gdb.execute('set $x29 = %s' % self.old_x29)
 def __reset_new_pointers(self):
 if arch == 'x64':
@@ -920,7 +920,7 @@ def __reset_new_pointers(self):
 gdb.execute('set $rbp = %s' % self.new_rbp)
 else:
 gdb.execute('set $sp = %s' % self.new_sp)
-gdb.execute('set $x30 = %s' % self.new_x30)
+gdb.execute('set $pc = %s' % self.new_pc)
 gdb.execute('set $x29 = %s' % self.new_x29)
 
 def exit_thread_context():

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/fdd2e305bacf63e5%40google.com.

Re: [osv-dev] [PATCH] scripts: download and build arbitrary version of boost

[Nadav Har'El]

On Fri, Feb 5, 2021 at 11:22 PM Waldemar Kozaczuk 
wrote:

> On some distributions, like CentOS, the default version of boost
> is really old. So the best option is to build a newer one from source.
> Also even on Fedora or Ubuntu it might be useful to point to
> different version of boost other than what ./scripts/setup.py installs.
>
> This script automates process of downloading the sources of specified
> version
> of boost and building it using default or specified toolchain.
> The last CROSS_AARCH64_TOOLSET parameter (gcc or gcc-arm) is intended to
> cross-compile
> the aarch64 version of boost on x86_64 machine.
>
> The built version of the boost is symlinked into the
> build/downloaded_packages//boost/install/ directory and will
> automatically be used where cross-compiling aarch64 OSv kernel.
> In order to use x64 version of boost built using this script
> we still need to make changes to the main makefile and
> modules/common.gmk.
>
> Signed-off-by: Waldemar Kozaczuk 
> ---
>  scripts/download_and_build_boost.sh | 63 +
>  1 file changed, 63 insertions(+)
>  create mode 100755 scripts/download_and_build_boost.sh
>
> diff --git a/scripts/download_and_build_boost.sh
> b/scripts/download_and_build_boost.sh
> new file mode 100755
> index ..7181f5d4
> --- /dev/null
> +++ b/scripts/download_and_build_boost.sh
> @@ -0,0 +1,63 @@
> +#!/bin/bash
> +#
> +# Copyright (C) 2020 Waldemar Kozaczuk
> +#
> +# This work is open source software, licensed under the terms of the
> +# BSD license as described in the LICENSE file in the top-level directory.
> +#
> +
> +# Usage:
> +#  ./scripts/download_and_build_boost.sh  
> 
> +#
> +
> +OSV_ROOT=$(dirname $(readlink -e $0))/..
> +echo $OSV_ROOT
> +
> +BOOST_VERSION=$1
> +if [[ "${BOOST_VERSION}" == "" ]]; then
> +  BOOST_VERSION="1.70.0"
> +fi
> +BOOST_VERSION2=${BOOST_VERSION//[.]/_}
> +
> +ARCH=$2
> +if [[ "${ARCH}" == "" ]]; then
> +  ARCH=$(uname -m)
> +fi
> +
> +#If CROSS_AARCH64_TOOLSET is set or passed we assume we are
> crosscompiling aarch64 on Intel
> +CROSS_AARCH64_TOOLSET=$3
> +
> +rm -rf %s/boost/install
> +
>
> +TAR_DOWNLOAD_DIR=${OSV_ROOT}/build/downloaded_packages/${ARCH}/boost/upstream/
> +mkdir -p ${TAR_DOWNLOAD_DIR}
> +
> +BOOST_URL=
> https://dl.bintray.com/boostorg/release/${BOOST_VERSION}/source/boost_${BOOST_VERSION2}.tar.gz
> +wget -c -O ${TAR_DOWNLOAD_DIR}/boost_${BOOST_VERSION2}.tar.gz ${BOOST_URL}
> +
> +pushd ${TAR_DOWNLOAD_DIR}
> +rm -rf ./boost_${BOOST_VERSION2} && tar -xf
> ./boost_${BOOST_VERSION2}.tar.gz
>

Nitpicks on code which scares me (even if in this particular case it's
probably benign):

1. "pushd" may fail, and if it does, the script just continues in the wrong
directory. In such scripts I usually use "set -e" to make sure that any
failed command stops the entire script. But you can also add checks to
individual commands if you prefer, e.g., "pushd ... || echo oops; exit 1"

2. If $BOOST_VERSION2 contains a space the above command can rm -rf
anything! Never try it with BOOST_VERSION2="xyz /"... If you don't believe
me try the following non-harmful example in bash:

$ a="xyz /"
$ ls x${a}
ls: cannot access 'xxyz': No such file or directory
/:
   boot  etc   lib ...

The solution is to put quotes around the $a:

   $ a = "xyz /"
   $ ls "x$a"
   ls: cannot access 'xxyz /': No such file or directory

Similarly below any time you use a variable.

>
> +cd ./boost_${BOOST_VERSION2}
> +./bootstrap.sh
> --with-libraries=system,thread,test,chrono,regex,date_time,filesystem,locale,random,atomic,log,program_options
> +
> +BOOST_DIR=$(pwd)
> +case $CROSS_AARCH64_TOOLSET in
> +  gcc)
> +echo "using gcc : arm : aarch64-linux-gnu-g++ ;" > user-config.jam ;;
> +  gcc-arm)
> +echo "using gcc : arm : aarch64-none-linux-gnu-g++ ;" >
> user-config.jam ;;
> +esac
> +
> +B2_OPTIONS="threading=multi"
> +if [[ "${CROSS_AARCH64_TOOLSET}" != "" ]]; then
> +  B2_OPTIONS="${B2_OPTIONS} --user-config=${BOOST_DIR}/user-config.jam
> toolset=${CROSS_AARCH64_TOOLSET} architecture=arm address-model=64"
> +fi
> +./b2 ${B2_OPTIONS} -j$(nproc)
> +
> +#
> +# Create symlinks to install boost and make it visible to OSv makefile
> +rm -f ../../install
> +ln -s upstream/boost_${BOOST_VERSION2}/stage ../../install
> +mkdir -p stage/usr/include
> +ln -s ../../../boost stage/usr/include/boost
> +popd
>

The popd at the end of the script does nothing (and also pushd instead of
just cd wasn't needed), but I guess it's harmless.


> --
> 2.29.2
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/2021020521.1038474-1-jwkozaczuk%40gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.

[osv-dev] [COMMIT osv master] tests/tst-threadcomplete.cc: disable part testing pinning when cpu == 1

[Commit Bot]

From: Waldemar Kozaczuk 
Committer: Nadav Har'El 
Branch: master

tests/tst-threadcomplete.cc: disable part testing pinning when cpu == 1

On aarch64 when running tst-threadcomplete.cc on single cpu, OSv crashes with
the page fault when accessing 2nd cpu to pin some threads to.

Therefore this patch changes this test (just like tst-pin.cc) to disable
and warn the part exercising pinning cannot be run on single CPU.

Signed-off-by: Waldemar Kozaczuk 
Message-Id: <20210203041449.945828-1-jwkozac...@gmail.com>

---
diff --git a/tests/tst-threadcomplete.cc b/tests/tst-threadcomplete.cc
--- a/tests/tst-threadcomplete.cc
+++ b/tests/tst-threadcomplete.cc
@@ -129,10 +129,16 @@ int main(int ac, char** av)
 do_test(true);
 std::cerr << "Starting join tests...\n";
 do_test(false);
-std::cerr << "Starting thread-on-heap tests... (no load)\n";
-do_heap_test(false);
-std::cerr << "Starting thread-on-heap tests... (loaded)\n";
-do_heap_test(true);
+if (sched::cpus.size() >= 2) {
+std::cerr << "Starting thread-on-heap tests... (no load)\n";
+do_heap_test(false);
+std::cerr << "Starting thread-on-heap tests... (loaded)\n";
+do_heap_test(true);
+} else {
+// This test cannot be run on only one CPU, because we want to see
+// the effect of pinning threads to different CPUs.
+std::cout << "Cannot run the thread-on-heap tests with only one 
CPU.\n";
+}
 std::cerr << "Passed\n";
 }
 

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/d378ea05bacea3aa%40google.com.

Re: [osv-dev] GDB stub

[Nadav Har'El]

On Wed, Feb 3, 2021 at 1:18 AM Waldek Kozaczuk  wrote:

> Currently, it is only possible to connect from gdb to a running instance
> of OSv on QEMU. But ideally, it would be nice to have a gdb stub in OSv
> that would let one connect from gdb to OSv running on any hypervisor.
>
> Per this fragment of this Wiki -
> https://github.com/cloudius-systems/osv/wiki/Debugging-OSv
>
> *"The fact that our gdb support works with the unit of cpus, not OSV
> threads, makes it convenient to debug OSV itself, but less convenient to
> debug application code running inside OSV. Particularly problematic is
> single stepping support which isn’t currently supported well: The problem
> is that when you try to look at a single OSV thread (see the next section),
> a single-step operation (“s” or “n”) makes one step inside the vcpu, not
> the osv thread*, and this step might include switching to a different
> thread. One way to work around this problem is to compile OSV with
> preemption disabled (not recommended, but can be a convenient trick in some
> debugging situations). *A better approach would have been to include a
> gdb stub into OSV itself, to allow gdb to see OSV threads directly and
> single-step them. We’ve made some progress in implementing such a stub, but
> it isn’t ready for prime-time yet.*"
>
> it looks like somebody may have implemented an initial version of the gdb
> stub. Is there a working patch of it somewhere?
>

I have a vague recollection - it was many years ago - that Glauber (CCed)
was the one who tried doing it.
Glauber, if it was you, do you remember what you tried?
If it wasn't you, sorry for the spam :-)


> Regards,
> Waldek
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/a3ca8d68-5994-4b51-94c4-539e2024f12dn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CANEVyjs6jduwbHOr_MhoJqtq8%3DcctrT95qcWTDTxHHLpPtdqEw%40mail.gmail.com.