[osv-dev] Re: Another FOSDEM talk about OSv being used to build real-world server-less platform by Genezio

[Waldek Kozaczuk]

I have come across another interesting research paper that evaluates OSv - 
"Are Unikernels Ready for Serverless on the Edge?"

https://arxiv.org/html/2403.00515v1

Regards,
Waldek

On Thursday, February 8, 2024 at 2:36:27 PM UTC-5 Waldek Kozaczuk wrote:

> Very interesting use case of using OSv to build a fast and very 
> resource-efficient server-less platform - 
> https://fosdem.org/2024/schedule/event/fosdem-2024-3269-run-node-js-in-a-unikernel-reliably/
> .
>
> Waldek
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/94553916-a230-49d3-bc18-5104182f9ba9n%40googlegroups.com.

[osv-dev] Re: I am running OSv in Ubuntu, how can I display this picture? Thanks！！

[Waldek Kozaczuk]

Hi,

You need to build an image with the 'httpserver-html5-gui' module - 

scripts/build image=...,httpserver-html5-gui

Regards,
Waldek
On Thursday, March 21, 2024 at 4:13:13 AM UTC-4 tyh249...@gmail.com wrote:

> [image: {FA65C1F6-24AF-4fe8-A5CE-ACCEF6FB37B5}.png]
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/e1866f2e-5cd2-4c53-b878-ba7de8396335n%40googlegroups.com.

[osv-dev] Re: qemu-system-x86_64: -drive file=/home/yuhang/osv/build/last/usr.img,if=none,id=hd0,cache=none,aio=native: Unable to use native AIO: failed to create linux AIO context: Function not imple

[Waldek Kozaczuk]

Can you give us more context? How do you build the image (full build 
command), how do you run it, etc?

On Tuesday, March 19, 2024 at 4:49:40 AM UTC-4 tyh249...@gmail.com wrote:

> why this error？i can not solve it.[image: 
> {8D254E09-51CC-475d-82D3-6A7398A8635F}.png]

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/aeb61d20-4d70-456d-9183-b65fec67ec92n%40googlegroups.com.

Re: [osv-dev] Virtio-net poor parallel latency

[Waldek Kozaczuk]

Hi,

I would add that the *java-httpserver *is NOT the best representative of 
Java http server examples. It uses the internal JDK HTTP server - 
com.sun.net.httpserver.HttpServer. I suggest you try other java example 
like jetty, tomcat, akka-http (scala app on JVM).

In my experience Java networking apps performed quite well compared to 
Linux guest (please see slide 23 
of 
https://www.p99conf.io/session/osv-unikernel-optimizing-guest-os-to-run-stateless-and-serverless-apps-in-the-cloud/).

Regards,
Waldek 

On Monday, March 18, 2024 at 5:54:18 PM UTC-4 דור לאור wrote:

> On Mon, Mar 18, 2024 at 8:29 PM Darren L  wrote:
>
>> Hello!
>>
>> Thank you for the suggestions. My testing environment is a i9-13900H, 
>> which has 20 total threads, of which I am allocating the first 8 to OSv and 
>> the next 4 to the client. These 12 in-use threads exist on 6 hyperthreaded 
>> cores.
>>
>> I wasn't sure how to measure the vmexits and/or process scheduling on the 
>> host.
>>
>
> https://access.redhat.com/solutions/6994095
> Guest/hypervisor efficiency is many times a function of how many times the 
> guest
> exits to the host. Lower is better
>  
>
>>
>> I didn't see netperf on recent versions of OSv in the /tools, it seems to 
>> have existed in OSv v0.5 then afterwards was removed. I did run a similar 
>> benchmark on Python 3.10 and ran the same parallel tests, and I received 
>> much lower latency numbers compared to the Java 8 version. In this case, I 
>> received latency numbers of 45ms (min-max 18-63, std: 12) compared to a 
>> possibly confusing measurement for Linux VM of 837ms (min-max 39-1903, std: 
>> 817). If this does suggest that the JVM is the issue, what steps should I 
>> take to debug this problem? The application I am using must use Java 8 to 
>> run; it cannot be run on any other platform.
>>
>
> Eliminate Java is one option. Another is to use a recent JVM (17) and ZGC 
> and hopefully there wouldn't
> be GC events (not sure it's a real issue here)
>  
>
>>
>> I know these are not the exact details you requested, but I am more than 
>> happy to learn how I can capture the other details, if necessary. Thank you!
>>
>> On Wednesday, March 13, 2024 at 5:23:53 PM UTC-4 דור לאור wrote:
>>
>>> Lots of good details. It's not simple to figure out what's the issue 
>>> since 
>>> you have hypervisor, host, OS and JVM variables. 
>>>
>>> How many threads does the host have? Make sure there are enough hardware 
>>> threads for the 
>>> guest, virtio on the host and the client. This way all 
>>> OSv's runable threads will be schedulable. 
>>> You can also measure the amount of vmexits and process scheduling on the 
>>> host.
>>> There is a chance the JVM is an issue too, can you do the same with 
>>> netperf?
>>>
>>>
>>> On Wed, Mar 13, 2024 at 9:14 PM Darren L  wrote:
>>>
 Hello!

 I was wondering if I could get any pointers on why I am receiving 
 significant latency issues using the virtio-net driver when processing 
 multiple parallel clients. Hopefully I can explain my issue enough to be 
 replicated.

 *Testing environment:*
 - Comparison: Ubuntu Server (Linux) VM and OSv (used the option "-nv" 
 in the run.py script for tap networking)
 - In common: 4 CPU cores, 4GB of RAM, QEMU KVM, used "taskset" to pin 
 to the same cores
 - Program: *java-httpserver* program from the apps directory, java8
 - What was sent: data of varying sizes (1KB to 1MB, 4MB, 8MB...) on the 
 same machine to the VMs

 *Observations:*
 - With single-threaded requests and low data sizes, I was able to 
 measure a latency on OSv that is lower than the Linux VM latency
 - example: for 32KB I measured ~4ms for OSv and 9.8ms for the Linux 
 VM
 - At high data sizes (256KB+), OSv started to measure a higher latency 
 than the Linux VM
 - When I sent *multiple requests* at the same time, OSv suffered a 
 much larger average latency penalty
 - example, at 1MB data size and 16 parallel requests, average 
 latency was:
 - OSv: 120ms (min-max 14-225ms, std: 62ms)
 - Linux VM: 82ms (min-max 24-144ms, std: 34ms) for the Linux VM

 *Other notes:*
 - I've been using the OSv profiling tools and have seen that the hot 
 spots typically were in virtio::virtio_driver::wait_for_queue and 
 virtio::net::receiver, but I was unable to identify the exact issue on why 
 this latency is the case
 - I also noticed when tracing the network layer (
 https://github.com/cloudius-systems/osv/wiki/Trace-analysis-using-trace.py#tracing-network-layer),
  
 there were a lot of *net_packet_handling* lines; about as much as 
 there were *net_packet_in* lines for 1MB, which might indicate that 
 the packets are not being processed fast enough and are delayed because it 
 is put in a queue?

 Hope this is clear enough! I am hoping to understand whether I am 
 

[osv-dev] Another FOSDEM talk about OSv being used to build real-world server-less platform by Genezio

[Waldek Kozaczuk]

Very interesting use case of using OSv to build a fast and very 
resource-efficient server-less platform 
- 
https://fosdem.org/2024/schedule/event/fosdem-2024-3269-run-node-js-in-a-unikernel-reliably/.

Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/cd543013-e7ba-4931-abe6-42bcfd185befn%40googlegroups.com.

[osv-dev] Re: FOSDEM24 Presentation

[Waldek Kozaczuk]

And the slides if you do not have time to watch the presentation 
- 
https://fosdem.org/2024/events/attachments/fosdem-2024-3483-support-dynamically-linked-executables-via-linux-ld-so-and-implement-ena-driver-to-expand-application-of-osv/slides/21593/OSv_FOSDEM_24_5NmcHjr.pdf

On Monday, February 5, 2024 at 1:10:44 PM UTC-5 Waldek Kozaczuk wrote:

> In case you cannot play video using this direct link, here is an 
> alternative - 
> https://fosdem.org/2024/schedule/event/fosdem-2024-3483-support-dynamically-linked-executables-via-linux-ld-so-and-implement-ena-driver-to-expand-application-of-osv/
>
> On Monday, February 5, 2024 at 12:54:22 PM UTC-5 Waldek Kozaczuk wrote:
>
>> In case you missed it, the recording of my FOSDEM talk about the most 
>> recent OSv enhancements has been just released:
>>
>>
>> https://video.fosdem.org/2024/ud2208/fosdem-2024-3483-support-dynamically-linked-executables-via-linux-ld-so-and-implement-ena-driver-to-expand-application-of-osv.av1.webm
>>
>> Tune in to learn about how OSv can now run statically linked executables 
>> and the dynamic ones via Linux dynamic linker and how I use Xconfig to 
>> build a custom kernel of 788K in size that runs an app with only 1.2MB of 
>> memory!
>>
>> Waldek
>> On Thursday, February 1, 2024 at 8:11:54 AM UTC-5 Waldek Kozaczuk wrote:
>>
>>> If you happen to be at #FOSDEM 
>>> <https://twitter.com/hashtag/FOSDEM?src=hashtag_click> in Brussels this 
>>> coming weekend and are interested in recent improvements to the OSv , 
>>> please come to watch my presentation - 
>>> https://fosdem.org/2024/schedule/event/fosdem-2024-3483-support-dynamically-linked-executables-via-linux-ld-so-and-implement-ena-driver-to-expand-application-of-osv/
>>>
>>> "From the beginning, OSv has been designed to implement a subset of the 
>>> Linux flavor of POSIX API. Initially, most Linux applications had to be 
>>> re-compiled from source as shared libraries. Starting with the release 
>>> 0.54.0, it has become possible to run unmodified Linux position-independent 
>>> executables (so-called "PIEs") and position-dependent executables "as-is".
>>>
>>> In my presentation, I will talk about how OSv has been enhanced to 
>>> support running statically linked executables. Not only that, the same 
>>> improvements make it possible to execute dynamically linked executables via 
>>> Linux dynamic linker. I will also briefly describe a newly added 
>>> strace-like capability aimed to help analyze system calls on the fly.
>>>
>>> In addition, I will explain how we have implemented the ENA networking 
>>> driver and thus made it possible to run OSv on AWS Nitro EC2 instances. 
>>> Finally, I will briefly cover the upcoming build configuration tool, 
>>> based on Xconfig, aimed to let the user select OSv kernel components to be 
>>> included or excluded and various parameters to be configured."
>>>
>>> Cheers,
>>> Waldek
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/c1c6da57-3273-4f28-874b-b9e672ae0088n%40googlegroups.com.

[osv-dev] Re: FOSDEM24 Presentation

[Waldek Kozaczuk]

In case you cannot play video using this direct link, here is an 
alternative - 
https://fosdem.org/2024/schedule/event/fosdem-2024-3483-support-dynamically-linked-executables-via-linux-ld-so-and-implement-ena-driver-to-expand-application-of-osv/

On Monday, February 5, 2024 at 12:54:22 PM UTC-5 Waldek Kozaczuk wrote:

> In case you missed it, the recording of my FOSDEM talk about the most 
> recent OSv enhancements has been just released:
>
>
> https://video.fosdem.org/2024/ud2208/fosdem-2024-3483-support-dynamically-linked-executables-via-linux-ld-so-and-implement-ena-driver-to-expand-application-of-osv.av1.webm
>
> Tune in to learn about how OSv can now run statically linked executables 
> and the dynamic ones via Linux dynamic linker and how I use Xconfig to 
> build a custom kernel of 788K in size that runs an app with only 1.2MB of 
> memory!
>
> Waldek
> On Thursday, February 1, 2024 at 8:11:54 AM UTC-5 Waldek Kozaczuk wrote:
>
>> If you happen to be at #FOSDEM 
>> <https://twitter.com/hashtag/FOSDEM?src=hashtag_click> in Brussels this 
>> coming weekend and are interested in recent improvements to the OSv , 
>> please come to watch my presentation - 
>> https://fosdem.org/2024/schedule/event/fosdem-2024-3483-support-dynamically-linked-executables-via-linux-ld-so-and-implement-ena-driver-to-expand-application-of-osv/
>>
>> "From the beginning, OSv has been designed to implement a subset of the 
>> Linux flavor of POSIX API. Initially, most Linux applications had to be 
>> re-compiled from source as shared libraries. Starting with the release 
>> 0.54.0, it has become possible to run unmodified Linux position-independent 
>> executables (so-called "PIEs") and position-dependent executables "as-is".
>>
>> In my presentation, I will talk about how OSv has been enhanced to 
>> support running statically linked executables. Not only that, the same 
>> improvements make it possible to execute dynamically linked executables via 
>> Linux dynamic linker. I will also briefly describe a newly added 
>> strace-like capability aimed to help analyze system calls on the fly.
>>
>> In addition, I will explain how we have implemented the ENA networking 
>> driver and thus made it possible to run OSv on AWS Nitro EC2 instances. 
>> Finally, I will briefly cover the upcoming build configuration tool, 
>> based on Xconfig, aimed to let the user select OSv kernel components to be 
>> included or excluded and various parameters to be configured."
>>
>> Cheers,
>> Waldek
>>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/c48fa5b4-87c0-4d05-9686-31965fed03c2n%40googlegroups.com.

[osv-dev] Re: FOSDEM24 Presentation

[Waldek Kozaczuk]

In case you missed it, the recording of my FOSDEM talk about the most 
recent OSv enhancements has been just released:

https://video.fosdem.org/2024/ud2208/fosdem-2024-3483-support-dynamically-linked-executables-via-linux-ld-so-and-implement-ena-driver-to-expand-application-of-osv.av1.webm

Tune in to learn about how OSv can now run statically linked executables 
and the dynamic ones via Linux dynamic linker and how I use Xconfig to 
build a custom kernel of 788K in size that runs an app with only 1.2MB of 
memory!

Waldek
On Thursday, February 1, 2024 at 8:11:54 AM UTC-5 Waldek Kozaczuk wrote:

> If you happen to be at #FOSDEM 
> <https://twitter.com/hashtag/FOSDEM?src=hashtag_click> in Brussels this 
> coming weekend and are interested in recent improvements to the OSv , 
> please come to watch my presentation - 
> https://fosdem.org/2024/schedule/event/fosdem-2024-3483-support-dynamically-linked-executables-via-linux-ld-so-and-implement-ena-driver-to-expand-application-of-osv/
>
> "From the beginning, OSv has been designed to implement a subset of the 
> Linux flavor of POSIX API. Initially, most Linux applications had to be 
> re-compiled from source as shared libraries. Starting with the release 
> 0.54.0, it has become possible to run unmodified Linux position-independent 
> executables (so-called "PIEs") and position-dependent executables "as-is".
>
> In my presentation, I will talk about how OSv has been enhanced to support 
> running statically linked executables. Not only that, the same improvements 
> make it possible to execute dynamically linked executables via Linux 
> dynamic linker. I will also briefly describe a newly added strace-like 
> capability aimed to help analyze system calls on the fly.
>
> In addition, I will explain how we have implemented the ENA networking 
> driver and thus made it possible to run OSv on AWS Nitro EC2 instances. 
> Finally, I will briefly cover the upcoming build configuration tool, based 
> on Xconfig, aimed to let the user select OSv kernel components to be 
> included or excluded and various parameters to be configured."
>
> Cheers,
> Waldek
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/8c148bdf-d1df-4593-821d-f8beae248e5dn%40googlegroups.com.

Re: [osv-dev] FOSDEM24 Presentation

[Waldek Kozaczuk]

Great idea!

However, I have only 20 minutes to present so I am not sure if I will be 
able to do any of this.

Maybe I should start some youtube channel where I can demo all this.

Waldek

On Thursday, February 1, 2024 at 9:38:17 AM UTC-5 דור לאור wrote:

> Waldek, do you have some standard Linux apps you tested that can be 
> just loaded to OSv? Will be nice to showcase OSv values, with either 
> performance or VM convenience when you take an app and quickly boot 
> it with OSv and it just works. 
>
>
> On Thu, Feb 1, 2024 at 3:11 PM Waldek Kozaczuk  wrote:
>
>> If you happen to be at #FOSDEM 
>> <https://twitter.com/hashtag/FOSDEM?src=hashtag_click> in Brussels this 
>> coming weekend and are interested in recent improvements to the OSv , 
>> please come to watch my presentation - 
>> https://fosdem.org/2024/schedule/event/fosdem-2024-3483-support-dynamically-linked-executables-via-linux-ld-so-and-implement-ena-driver-to-expand-application-of-osv/
>>
>> "From the beginning, OSv has been designed to implement a subset of the 
>> Linux flavor of POSIX API. Initially, most Linux applications had to be 
>> re-compiled from source as shared libraries. Starting with the release 
>> 0.54.0, it has become possible to run unmodified Linux position-independent 
>> executables (so-called "PIEs") and position-dependent executables "as-is".
>>
>> In my presentation, I will talk about how OSv has been enhanced to 
>> support running statically linked executables. Not only that, the same 
>> improvements make it possible to execute dynamically linked executables via 
>> Linux dynamic linker. I will also briefly describe a newly added 
>> strace-like capability aimed to help analyze system calls on the fly.
>>
>> In addition, I will explain how we have implemented the ENA networking 
>> driver and thus made it possible to run OSv on AWS Nitro EC2 instances. 
>> Finally, I will briefly cover the upcoming build configuration tool, 
>> based on Xconfig, aimed to let the user select OSv kernel components to be 
>> included or excluded and various parameters to be configured."
>>
>> Cheers,
>> Waldek
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "OSv Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to osv-dev+u...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/osv-dev/877de531-b8f4-4680-a065-4965528951a2n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/osv-dev/877de531-b8f4-4680-a065-4965528951a2n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/8505a5cf-f0c1-4d82-85d8-bb89acfc81c6n%40googlegroups.com.

[osv-dev] FOSDEM24 Presentation

[Waldek Kozaczuk]

If you happen to be at #FOSDEM 
 in Brussels this 
coming weekend and are interested in recent improvements to the OSv , 
please come to watch my presentation 
- 
https://fosdem.org/2024/schedule/event/fosdem-2024-3483-support-dynamically-linked-executables-via-linux-ld-so-and-implement-ena-driver-to-expand-application-of-osv/

"From the beginning, OSv has been designed to implement a subset of the 
Linux flavor of POSIX API. Initially, most Linux applications had to be 
re-compiled from source as shared libraries. Starting with the release 
0.54.0, it has become possible to run unmodified Linux position-independent 
executables (so-called "PIEs") and position-dependent executables "as-is".

In my presentation, I will talk about how OSv has been enhanced to support 
running statically linked executables. Not only that, the same improvements 
make it possible to execute dynamically linked executables via Linux 
dynamic linker. I will also briefly describe a newly added strace-like 
capability aimed to help analyze system calls on the fly.

In addition, I will explain how we have implemented the ENA networking 
driver and thus made it possible to run OSv on AWS Nitro EC2 instances. 
Finally, I will briefly cover the upcoming build configuration tool, based 
on Xconfig, aimed to let the user select OSv kernel components to be 
included or excluded and various parameters to be configured."

Cheers,
Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/877de531-b8f4-4680-a065-4965528951a2n%40googlegroups.com.

Re: [osv-dev] Increasing 2GB limit for OSV_KERNEL_VM_SHIFT

[Waldek Kozaczuk]

Hi,

Sorry for the late reply.

I did manage to replicate your issue. It turns out that when building so 
large images, one needs more memory to run the so-called ZFS builder. After 
increasing the memory from 512M to 1G I was able to build the image 
successfully.

diff --git a/scripts/upload_manifest.py b/scripts/upload_manifest.py
index a3796f95..65e91a9c 100755
--- a/scripts/upload_manifest.py
+++ b/scripts/upload_manifest.py
@@ -164,7 +164,7 @@ def main():
 console = '--console=serial'
 zfs_builder_name = 'zfs_builder-stripped.elf'
 
-osv = subprocess.Popen('cd ../..; scripts/run.py -k --kernel-path 
build/release/%s --arch=%s --vnc none -m 512 -c1 -i "%s" 
--block-device-cache unsafe -s -e "%s --norandom --nomount --noinit 
--preload-zfs-library /tools/mkfs.so; /tools/cpiod.so --prefix /zfs/zfs/; 
/zfs.so set compression=off osv" --forward tcp:127.0.0.1:%s-:1' % 
(zfs_builder_name,arch,image_path,console,upload_port), shell=True, 
stdout=subprocess.PIPE)
+osv = subprocess.Popen('cd ../..; scripts/run.py -k --kernel-path 
build/release/%s --arch=%s --vnc none -m 1G -c1 -i "%s" 
--block-device-cache unsafe -s -e "%s --norandom --nomount --noinit 
--preload-zfs-library /tools/mkfs.so; /tools/cpiod.so --prefix /zfs/zfs/; 
/zfs.so set compression=off osv" --forward tcp:127.0.0.1:%s-:1' % 
(zfs_builder_name,arch,image_path,console,upload_port), shell=True, 
stdout=subprocess.PIPE)
 
 upload(osv, manifest, depends, upload_port)
 
In case you see a similar error when running the image, also try to bump up 
memory like - run.py -m 1G

We should probably detect this condition in core/mmu.cc accordingly and 
better handle it possibly by informing user there is not enough physical 
memory:

 181 
 182 // For now, only allow non-mmaped areas.  Later, we can either
 183 // bounce such addresses, or lock them in memory and translate
 184 assert(virt >= phys_mem);
 185 return reinterpret_cast(virt) & (mem_area_size - 1);
 186 }
On Wednesday, December 13, 2023 at 10:57:13 PM UTC-5 Darren L wrote:

> Hi Waldek,
>
> I am able to make the issue appear again by creating a big file (i.e. dd 
> if=/dev/urandom of=1GB.bin bs=64M count=16 iflag=fullblock), creating a 
> new app within the apps folder called big_file, setting the usr.manifest to 
> include /big_file/**: ${MODULE_DIR}/**, and running:
>
> ./scripts/build fs_size_mb=8192 image=python-from-host,java8,big_file
>
> I updated the buf.h file, and the issue remains.
>
> When I used fs=rofs instead, the issue returns but it doesn't appear until 
> I run the run.py script. Example: the last few lines of the build state:
>
> First block: 5242316, blocks count: 5127
> Directory entries count 43078
> Symlinks count 10
> Inodes count 43079
>
> But when I run
>
> ./scripts/run.py -e "/python3.10"
>
> I get the following message:
>
> OSv v0.57.0-86-g873cb55a
>
> Assertion failed: virt >= phys_mem (core/mmu.cc: virt_to_phys: 184)
>
> [backtrace]
> 0x40244a7c <__assert_fail+28>
> 0x402b85fc 
> 0x402ef3e2  unsigned int, virtio::vring_desc::flags)::{lambda(unsigned long, unsigned 
> long)#1}>(void*, unsigned long, virtio::vring::add_sg(void*, unsigned int, 
> virtio::vring_desc::flags)::{lambda(unsigned long, unsigned long)#1})+34>
> 0x402ef148 
> 0x403bfd75 
> 0x403cacbb  long, void*)+107>
> 0x403c91fd 
> 0x403c1476 
> 0x403beadb 
> 0x40239491 
> 0x403e4e69 
> 0x4037db2d 
> 0x4030b361 
>
> When I use Virtio-FS, the error does not seem to appear, but this seems to 
> cause different errors possibly. When trying to run Python in the 
> unikernel, I get:
>
> sudo PATH=/usr/lib/qemu:$PATH ./scripts/run.py --virtio-fs-tag=myfs 
> --virtio-fs-dir=$(pwd)/build/export -e "/python3.10"
> OSv v0.57.0-86-g873cb55a
> eth0: 192.168.122.15
> Booted up in 109.01 ms
> Cmdline: /python3.10
> Fatal Python error: _Py_HashRandomization_Init: failed to get random 
> numbers to initialize Python
> Python runtime state: preinitialized
>
> Hope this is enough to replicate the issue. Thank you!
>
> On Monday, December 11, 2023 at 3:03:01 PM UTC-5 jwkoz...@gmail.com wrote:
>
>> Hi,
>>
>> On Thursday, December 7, 2023 at 6:43:00 PM UTC-5 Darren L wrote:
>>
>> Hi Waldek,
>>
>> Thanks for the quick response. For more details, I am trying to run a 
>> research prototype that requires both Python 3 and Java 8 to run correctly, 
>> and I placed the prototype's executable files (which includes large amounts 
>> of static data required to run the program) as an image in the apps 
>> directory to be linked to the image. The individual files are not huge (up 
>> to a few hundred MB each), but rather there are a lot of files that need to 
>> be included to run the prototype (totalling altogether 2-3 GB). I wasn't 
>> sure if this was the best approach. It seems that I could also dynamically 
>> link it in the .img file after the fact, since I only need the 

Re: [osv-dev] Increasing 2GB limit for OSV_KERNEL_VM_SHIFT

[Waldek Kozaczuk]

Hi,

On Thursday, December 7, 2023 at 6:43:00 PM UTC-5 Darren L wrote:

Hi Waldek,

Thanks for the quick response. For more details, I am trying to run a 
research prototype that requires both Python 3 and Java 8 to run correctly, 
and I placed the prototype's executable files (which includes large amounts 
of static data required to run the program) as an image in the apps 
directory to be linked to the image. The individual files are not huge (up 
to a few hundred MB each), but rather there are a lot of files that need to 
be included to run the prototype (totalling altogether 2-3 GB). I wasn't 
sure if this was the best approach. It seems that I could also dynamically 
link it in the .img file after the fact, since I only need the Python and 
Java images to run the program correctly?

The error I am getting occurs when I run the following command: 
`./scripts/build 
fs_size_mb=8192 image=python-from-host,java8,prototype` where prototype 
contains my large executable files. The error I receive is during the build 
process. It will state:

```
Assertion failed: virt >= phys_mem (core/mmu.cc: virt_to_phys: 184)

[backtrace]
0x40244a7c <__assert_fail+28>
0x402b85fc 
0x402ef3e2 (void*, unsigned long, virtio::vring::add_sg(void*, unsigned int, 
virtio::vring_desc::flags)::{lambda(unsigned long, unsigned long)#1})+34>
0x402ef1e6 
0x1006b1e5 
0x1009620a 
0x1006e476 
0x1009620a 
0x40260e1e 
0x40260ef2 
0x4037db2d 
0x4030b361 
```

I do not think this error has anything to do with the ELF layout or the 
kernel shift. I think this issue is different and it happens when OSv runs 
during the build process (ZFS builder) to create the ZFS disk and upload 
all files.
I am still very interested in replicating and fixing it.

My wild guess is that it may be caused by the bug somebody else discovered 
and fixed in his pull request 
- https://github.com/cloudius-systems/osv/pull/1284/files. Can you only 
update the include/osv/buf.h file to see if your problem goes away? 
Otherwise, I have to be able to replicate it somehow.

Also, can you try to build a ROFS image (add fs=rofs to your build command) 
and run it?
 
There is also an option to use Virtio-FS 
(see https://github.com/cloudius-systems/osv/wiki/virtio-fs).

I'm using the most recent OSv pulled from Github. I'm not using node but 
Python. To my best understanding, this is what I got using readelf for Java 
and Python. I used the files within my own machine that were then 
transferred into OSv. I wasn't sure how to use `readelf` within the 
unikernel itself, so I might need guidance on using readelf within OSv if 
this is not sufficient.

openjdk-8-zulu-full's java binary:

```
Elf file type is EXEC (Executable file)
Entry point 0x400570
There are 9 program headers, starting at offset 64


Program Headers:
  Type   Offset   VirtAddr   PhysAddr   FileSiz 
 MemSiz   Flg Align
  PHDR   0x40 0x00400040 0x00400040 0x0001f8 
0x0001f8 R E 0x8
  INTERP 0x000238 0x00400238 0x00400238 0x1c 
0x1c R   0x1
  [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
  LOAD   0x00 0x0040 0x0040 0x0008c4 
0x0008c4 R E 0x20
  LOAD   0x000d98 0x00600d98 0x00600d98 0x00027c 
0x000290 RW  0x20
  DYNAMIC0x000dd0 0x00600dd0 0x00600dd0 0x000210 
0x000210 RW  0x8
  NOTE   0x000254 0x00400254 0x00400254 0x44 
0x44 R   0x4
  GNU_EH_FRAME   0x000818 0x00400818 0x00400818 0x24 
0x24 R   0x4
  GNU_STACK  0x00 0x 0x 0x00 
0x00 RW  0x8
  GNU_RELRO  0x000d98 0x00600d98 0x00600d98 0x000268 
0x000268 R   0x1

 Section to Segment mapping:
  Segment Sections...
   00 
   01 .interp 
   02 .interp .note.ABI-tag .note.gnu.build-id .hash .gnu.hash .dynsym 
.dynstr .gnu.version .gnu.version_d .gnu.version_r .rela.dyn .rela.plt 
.init .plt .text .fini .rodata .eh_frame_hdr .eh_frame 
   03 .ctors .dtors .jcr .data.rel.ro .dynamic .got .got.plt .data .bss 
   04 .dynamic 
   05 .note.ABI-tag .note.gnu.build-id 
   06 .eh_frame_hdr 
   07 
   08 .ctors .dtors .jcr .data.rel.ro .dynamic .got 
```

python3.10.12:

```
Elf file type is DYN (Position-Independent Executable file)
Entry point 0x22cb80

There are 13 program headers, starting at offset 64

Program Headers:
  Type   Offset   VirtAddr   PhysAddr   FileSiz 
 MemSiz   Flg Align
  PHDR   0x40 0x0040 0x0040 0x0002d8 
0x0002d8 R   0x8
  INTERP 0x000318 0x0318 0x0318 0x1c 
0x1c R   0x1
  [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
  LOAD   0x00 0x 0x 0x06c228 

Re: [osv-dev] ENA driver

[Waldek Kozaczuk]

 

Even though this driver implements stateless offloads - TXCSUM, RXCSUM, 
TSO, LRO - (just like the original FreeBSD one - 
https://github.com/amzn/amzn-drivers/tree/master/kernel/fbsd/ena#stateless-offloads),
 
the underlying ENA device does NOT implement RXCSUM nor TSO (see 
amzn/amzn-drivers#29 <https://github.com/amzn/amzn-drivers/issues/29>). It 
also looks like the LRO logic never gets activated based on the observed 
values of relevant tracepoints. We use netchannels so maybe it does not 
matter as much.

OSv when running on t3nano reports this:
D/22 ena]: Elastic Network Adapter (ENA)ena v2.6.3
[D/22 ena]: LLQ is not supported. Using the host mode policy.
[D/22 ena]: ena_attach: set max_num_io_queues to 2
[D/22 ena]: Enable only 3 MSI-x (out of 9), reduce the number of queues
[D/22 ena]: device offloads (caps): TXCSUM=2, TXCSUM_IPV6=0, TSO4=0, 
TSO6=0, RXCSUM=0, RXCSUM_IPV6=0, LRO=1, JUMBO_MTU=1
...
[D/22 ena]: ena_update_hwassist: CSUM_IP=1, CSUM_UDP=4, CSUM_TCP=2, 
CSUM_UDP_IPV6=0, CSUM_TCP_IPV6=0, CSUM_TSO=0

Can anyone confirm if this indeed is the case? The issue I am citing above 
was opened in 2017 and never updated. If so how much of the performance 
impact does it have?

Relatedly, I have since improved the driver a bit. Mainly I have changed 
the "cleanup" logic (mostly handling RX) to make the worker threads and 
corresponding MSIX vectors pin to a single vCPU. That seems to reduce # of 
IPIs and in some workflows I see performance improve by 5-10%.

I have also run more tests with iperf3 and netperf:

netperf -H 172.31.89.238 -t TCP_STREAM -l 5 -- -m 65536
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 
172.31.89.238 () port 0 AF_INET : demo
Recv   SendSend  
Socket Socket  Message  Elapsed  
Size   SizeSize Time Throughput  
bytes  bytes   bytessecs.10^6bits/sec  

 65536  16384  655365.01 3776.40  

iperf3 -t 5 -c 172.31.93.118
Connecting to host 172.31.93.118, port 5201
[  5] local 172.31.90.167 port 55674 connected to 172.31.93.118 port 5201
[ ID] Interval   Transfer Bitrate Retr  Cwnd
[  5]   0.00-1.00   sec   444 MBytes  3.72 Gbits/sec  5901199 KBytes   

[  5]   1.00-2.00   sec   421 MBytes  3.54 Gbits/sec  5529147 KBytes   

[  5]   2.00-3.00   sec   464 MBytes  3.89 Gbits/sec  5923157 KBytes   

[  5]   3.00-4.00   sec   440 MBytes  3.69 Gbits/sec  6117158 KBytes   

[  5]   4.00-5.00   sec   450 MBytes  3.78 Gbits/sec  6686260 KBytes   

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval   Transfer Bitrate Retr
[  5]   0.00-5.00   sec  2.17 GBytes  3.72 Gbits/sec  30156 
sender
[  5]   0.00-5.03   sec  2.16 GBytes  3.69 Gbits/sec 
 receiver 

With iperf3 I typically see a relatively high number of retries. Do you 
think it indicates some sort of bottleneck on OSv side? 

Relatedly, with both iperf and netperf I never see OSv exceed the 4Gbits/s 
barrier and it never approaches the NIC bandwidth limit (the maximum of 
t3nano is 5Gbits).

Finally, all the tests I have been running we conducted with clients (wrk, 
iperf, netperf, etc) on the t3micro Ubuntu instance deployed in the same 
availability zone (us-east-1f) and same VPC. I have also had no chance to 
compare it to Linux guest so no idea if these results are half decent or 
not.

Any input is highly appreciated.

On Thursday, November 30, 2023 at 2:57:48 AM UTC-5 דור לאור wrote:

> Sweat! 
>
> On Thu, Nov 30, 2023 at 7:52 AM Waldek Kozaczuk  
> wrote:
>
>> Hi,
>>
>> I have just created a pull request that implements OSv ENA driver by 
>> porting the FreeBSD version - 
>> https://github.com/cloudius-systems/osv/pull/1283. If you have bandwidth 
>> and know-how please feel free to review this PR.
>>
>> This is enough to run OSv on a nitro instance like t3 nano with ramfs 
>> image (ideally we would like NVMe driver). I have only tested it with 
>> simple golang httpserver and OSv http monitoring module and only on t3 
>> nano. It seems to be functional and stable but I still need to run more 
>> tests to get a better feel.
>>
>> Regards,
>> Waldek
>> [image: i-060143e9381af3d55.jpg]
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "OSv Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to osv-dev+u...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/osv-dev/32a2dfa7-9407-4a1a-a4fc-5711401dfec8n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/osv-dev/32a2dfa7-9407-4a1a-a4fc-5711401dfec8n%40googlegroups.com?utm_medium=email_source=footer>
>> .
&

Re: [osv-dev] Increasing 2GB limit for OSV_KERNEL_VM_SHIFT

[Waldek Kozaczuk]

Hi,

The 2GB limit and the commit you are referring to should only limit the
size of the position-dependent executables (these executables typically
want to be loaded in a place where OSv kernel used to be before this
commit).

Are your executables larger than 2GB in size? Can you run 'readelf -W -l'
against java and node like in this example:

readelf -W -l /usr/lib/jvm/java-8-openjdk-amd64/bin/java

Elf file type is DYN (Position-Independent Executable file)
Entry point 0x10b0
There are 13 program headers, starting at offset 64

Program Headers:
  Type   Offset   VirtAddr   PhysAddr   FileSiz
 MemSiz   Flg Align
  PHDR   0x40 0x0040 0x0040 0x0002d8
0x0002d8 R   0x8
  INTERP 0x000318 0x0318 0x0318 0x1c
0x1c R   0x1
  [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]
  LOAD   0x00 0x 0x 0x000750
0x000750 R   0x1000
  LOAD   0x001000 0x1000 0x1000 0x0001a9
0x0001a9 R E 0x1000
  LOAD   0x002000 0x2000 0x2000 0x00011c
0x00011c R   0x1000
  LOAD   0x002d48 0x3d48 0x3d48 0x0002c8
0x0002d0 RW  0x1000
  DYNAMIC0x002d58 0x3d58 0x3d58 0x000260
0x000260 RW  0x8
  NOTE   0x000338 0x0338 0x0338 0x30
0x30 R   0x8
  NOTE   0x000368 0x0368 0x0368 0x44
0x44 R   0x4
  GNU_PROPERTY   0x000338 0x0338 0x0338 0x30
0x30 R   0x8
  GNU_EH_FRAME   0x002038 0x2038 0x2038 0x34
0x34 R   0x4
  GNU_STACK  0x00 0x 0x 0x00
0x00 RW  0x10
  GNU_RELRO  0x002d48 0x3d48 0x3d48 0x0002b8
0x0002b8 R   0x1

Can you give us more details about your use case and the error you are
getting with the existing limit of 2GB (I am assuming there is a reason you
want to increase it).

Regards,
Waldek

On Thu, Dec 7, 2023 at 3:52 AM Darren Lim  wrote:

> Hello,
>
> Per the patch here (
> https://github.com/cloudius-systems/osv/commit/2a1795db8a22b0b963a64d068f5d8acc93e5785d),
> I was hoping to get help with making the changes to increase the kernel
> limit from 2GB to a larger size.
>
> For context, I am trying to load a large project (~3GB) into the
> unikernel, along with decently large languages (Java, Python) by creating a
> custom image for the build script. It currently complains in mmu.cc,
> stating:
>
> Assertion failed: virt >= phys_mem (core/mmu.cc: virt_to_phys: 184)
>
> Thank you!
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/a5c2844a-e03e-4ac4-8e0a-de81f575889fn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfMM94CWqL8RtnN7BZ2ycCty3P%2BZbsaM-zTPgMf0EzbtiQ%40mail.gmail.com.

[osv-dev] Spam

[Waldek Kozaczuk]

Hi,

Does anybody have admin access to remove these spam accounts that have been 
spamming recently?

Regards,
Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/c48827a3-a1ce-4d49-908d-b6cca9e82dfcn%40googlegroups.com.

[osv-dev] ENA driver

[Waldek Kozaczuk]

Hi,

I have just created a pull request that implements OSv ENA driver by 
porting the FreeBSD version 
- https://github.com/cloudius-systems/osv/pull/1283. If you have bandwidth 
and know-how please feel free to review this PR.

This is enough to run OSv on a nitro instance like t3 nano with ramfs image 
(ideally we would like NVMe driver). I have only tested it with simple 
golang httpserver and OSv http monitoring module and only on t3 nano. It 
seems to be functional and stable but I still need to run more tests to get 
a better feel.

Regards,
Waldek
[image: i-060143e9381af3d55.jpg]

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/32a2dfa7-9407-4a1a-a4fc-5711401dfec8n%40googlegroups.com.

Re: [osv-dev] Some questions about OSv

[Waldek Kozaczuk]

On Tue, Nov 28, 2023 at 3:04 PM Yueyang Pan  wrote:

> Hi Nadav and Waldek,
> Thanks a lot for very detailed answers from both of you. I have some
> updates on this.
> For the first question, I ended up implementing my own adhoc stat class
> where I can measure the total time (Or total count) of a function and
> calculate the average. I am still struggling to make the perf work. I got
> this error when using perf kvm as shown here
> https://github.com/cloudius-systems/osv/wiki/Debugging-OSv#profiling
> *Couldn't record guest kernel [0]'s reference relocation symbol*.
> From perf. Have you ever encountered this problem when you were developing?
>

I have never seen it but I will try to dig a bit deeper once I have time.

>
> For the second question, I ended up removing the global tlb_flush_mutex
> and introduced linux-like design where you have percpu call_function_data
> which contains a percpu array of call_single_data. Each CPU has its own
> call_single_queue where the call_single_data is enqueued or dequeued. If
> you don’t mind, I can arrange the code a bit and send the patch. Then you
> can review it. I am not sure how the developing process works for OSv and I
> will appreciate it very much if you can give me some guide.
>
Feel free to create PR on github.

Do you see significant improvement with your change to use percpu
call_function_data? OSv has its one percpu structures concept
(see include/osv/percpu.hh) so I wonder if you can leverage it.

I wonder how this Linux-like solution helps given that the point of the
mmu::flush_tlb_all() (where tlb_flush_mutex is used) is to coordinate the
flushing of TLB and make sure all CPUs do it so the virtual/physical
mapping is in sync across all CPUs. How do you achieve it in your solution?
Is potential speed improvement gained from avoiding IPIs which are known to
be slow?


> For the scheduling part, I am reading the paper now and the doc. Thanks
> for the resources. I need sometime to digest because I found that
> preempt_lock matters a lot for performance of my code.
>
> Best Wishes
> Pan
>
>
> On 28 Nov 2023, at 08:29, Nadav Har'El  wrote:
>
> On Tue, Nov 28, 2023 at 8:20 AM Waldek Kozaczuk 
> wrote:
>
>> Hi,
>>
>> It is great to hear from you. Please see my answers below.
>>
>> I hope you also do not mind I reply to the group so others may add
>> something extra or refine/correct my answers as I am not an original
>> developer/designer of OSv.
>>
>> On Fri, Nov 24, 2023 at 8:50 AM Yueyang Pan  wrote:
>>
>>> Dear Waldemar Kozaczuk,
>>> I am Yueyang Pan from EPFL. Currently I am working on a project
>>> about remote memory and trying to develop a prototype based on OSv. I am
>>> the guy who raised the questions on the google group several days ago as
>>> well. For that question, I made a workaround by adding my own stats class
>>> which record the sum and count because I need is the average number. Now I
>>> have some further questions. Probably they are a bit dumb for you but I
>>> will be very grateful if you could spend a little bit of time to give me
>>> some suggestions.
>>>
>>
>> The tracepoints use ring buffers of fixed size so eventually, all old
>> tracepoints would be overwritten by new ones. I think you can either
>> increase the size or use the approach used by the script *freq.py*
>>
>
> Exactly. OSv's tracepoints have two modes. One is indeed to save them in a
> ring buffer - so you'll see the last N traced events when you read that
> buffer - but other is a mode that just counts the events. What freq.py does
> is to retrieve the count at one second, then retrieve the count the next
> second - and the subtraction is the average number of this even per second.
>
> If you want instead of counting the event, to have a sum of, say, integers
> that come from the event (e.g., sum of packet lengths), we don't have
> support for this at the moment - we only increment the count by 1. It could
> be added as a feature, I guess. But you can always do something ad-hoc like
> maintain a global variable which you add.
>
>
>> (you need to add the module *httpserver-monitoring-api)*. There is also
>> newly added (experimental though) strace-like functionality (see
>> https://github.com/cloudius-systems/osv/commit/7d7b6d0f1261b87b678c572068e39d482e2103e4).
>> Finally, you may find the comments on this issue relevant -
>> https://github.com/cloudius-systems/osv/issues/1261#issuecomment-1722549524.
>> I am also sure you have come across this wiki page -
>> https://github.com/cloudius-systems/osv/wiki/Trace-analysis-using-trace.py
>> .
>>
>> Now after my profiling, I fo

[osv-dev] Re: Some questions about OSv

[Waldek Kozaczuk]

On Tuesday, November 28, 2023 at 1:20:06 AM UTC-5 Waldek Kozaczuk wrote:

Hi,

It is great to hear from you. Please see my answers below. 

I hope you also do not mind I reply to the group so others may add 
something extra or refine/correct my answers as I am not an original 
developer/designer of OSv.

On Fri, Nov 24, 2023 at 8:50 AM Yueyang Pan  wrote:

Dear Waldemar Kozaczuk,
I am Yueyang Pan from EPFL. Currently I am working on a project about 
remote memory and trying to develop a prototype based on OSv. I am the guy 
who raised the questions on the google group several days ago as well. For 
that question, I made a workaround by adding my own stats class which 
record the sum and count because I need is the average number. Now I have 
some further questions. Probably they are a bit dumb for you but I will be 
very grateful if you could spend a little bit of time to give me some 
suggestions.


The tracepoints use ring buffers of fixed size so eventually, all old 
tracepoints would be overwritten by new ones. I think you can either 
increase the size or use the approach used by the script *freq.py* (you 
need to add the module *httpserver-monitoring-api)*. There is also newly 
added (experimental though) strace-like functionality (see 
https://github.com/cloudius-systems/osv/commit/7d7b6d0f1261b87b678c572068e39d482e2103e4).
 
Finally, you may find the comments on this issue relevant - 
https://github.com/cloudius-systems/osv/issues/1261#issuecomment-1722549524. 
I am also sure you have come across this wiki page - 
https://github.com/cloudius-systems/osv/wiki/Trace-analysis-using-trace.py.

Now after my profiling, I found the mutex in global tib_flush_mutex to 
be hot in my benchmark so I am trying to remove it but it turns to be a bit 
hard without understanding the thread model of OSv. So I would like to ask 
whether there is any high-level doc that describes what the scheduling 
policy of OSv is, how the priority of the threads are decided, whether we 
can disable preemption or not (the functionality of preempt_lock) and the 
design of synchronisation primitives (for example why it is not allowed to 
have preemption disabled inside lockfree::mutex). I am trying to understand 
by reading the code directly but it can be really helpful if there is some 
material which describes the design.


If indeed your "hot" spot is around tlb_flush_mutex (used by  
flush_tlb_all()) then I am guessing your program does a lot of mmap/unmap 
(see *unpopulate* class in core/memory.cc that uses *tlb_gather*). I am not 
familiar with details of what it tlb_gather exactly does it probably forces 
TLB (Translation Lookaway Buffer) to flush old virtual/physical memory 
mapping entries after unmapping. The mmu::*flush_tlb_all*() is actually 
used in more places.

My wild suggestion would be to try to convert the tlb_flush_mutex to 
spinlock (see include/osv/spinlock.h and core/spinlock.cc). It is a bit 
controversial idea as OSv prides itself on lock-less structures and almost 
no spinklocks used (the console initialization is the only place left). But 
in some places (see 
https://github.com/cloudius-systems/osv/issues/853#issuecomment-279215964) 
and 
https://github.com/cloudius-systems/osv/commit/f8866c0dfd7ca1fcb4b2d9a280946878313a75d3
 
and https://groups.google.com/g/osv-dev/c/4wMAHCs7_dk/m/1LHdvmoeBwAJ we may 
benefit from those.

Please note the lock-less *sched::thread::wait_until* in the end of the 
flush_tlb_all would need to be replaced with "busy" wait/sleep.

Or instead of spinlock you can use the Nadav's "mutex with spinning" 
- https://groups.google.com/g/osv-dev/c/4wMAHCs7_dk/m/1LHdvmoeBwAJ - it may 
be a good fit here.  


As far as the information on mutexes and scheduling, the best information 
you can find in the original OSv paper - 
https://www.usenix.org/conference/atc14/technical-sessions/presentation/kivity. 
See also https://github.com/cloudius-systems/osv/wiki/Components-of-OSv and 
many other Wikis. 

Your preemption question - the lock-free mutex needs to have preemption on 
- imagine if we have a single CPU and the mutex ends up getting into the 
wait state to acquire a lock the thread would need to be eventually 
switched to another one that would release the lock. But if the preemption 
is off, then the scheduler will keep switching to the same waiting thread 
for each timer event and our original thread would never acquire the lock.

I hope all this helps.

Waldek

Thanks in advance for any advice you could provide. The questions may 
be a bit dumb so pardon me if I disturb you. 
Best Wishes
Pan

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/27b99d15-3471-4237-8c62-a5d69fc81f58n%40googlegroups.com.

[osv-dev] Re: Some questions about OSv

[Waldek Kozaczuk]

Hi,

It is great to hear from you. Please see my answers below.

I hope you also do not mind I reply to the group so others may add
something extra or refine/correct my answers as I am not an original
developer/designer of OSv.

On Fri, Nov 24, 2023 at 8:50 AM Yueyang Pan  wrote:

> Dear Waldemar Kozaczuk,
> I am Yueyang Pan from EPFL. Currently I am working on a project about
> remote memory and trying to develop a prototype based on OSv. I am the guy
> who raised the questions on the google group several days ago as well. For
> that question, I made a workaround by adding my own stats class which
> record the sum and count because I need is the average number. Now I have
> some further questions. Probably they are a bit dumb for you but I will be
> very grateful if you could spend a little bit of time to give me some
> suggestions.
>

The tracepoints use ring buffers of fixed size so eventually, all old
tracepoints would be overwritten by new ones. I think you can either
increase the size or use the approach used by the script *freq.py* (you
need to add the module *httpserver-monitoring-api)*. There is also newly
added (experimental though) strace-like functionality (see
https://github.com/cloudius-systems/osv/commit/7d7b6d0f1261b87b678c572068e39d482e2103e4).
Finally, you may find the comments on this issue relevant -
https://github.com/cloudius-systems/osv/issues/1261#issuecomment-1722549524.
I am also sure you have come across this wiki page -
https://github.com/cloudius-systems/osv/wiki/Trace-analysis-using-trace.py.

Now after my profiling, I found the mutex in global tib_flush_mutex to
> be hot in my benchmark so I am trying to remove it but it turns to be a bit
> hard without understanding the thread model of OSv. So I would like to ask
> whether there is any high-level doc that describes what the scheduling
> policy of OSv is, how the priority of the threads are decided, whether we
> can disable preemption or not (the functionality of preempt_lock) and the
> design of synchronisation primitives (for example why it is not allowed to
> have preemption disabled inside lockfree::mutex). I am trying to understand
> by reading the code directly but it can be really helpful if there is some
> material which describes the design.


If indeed your "hot" spot is around tlb_flush_mutex (used by
flush_tlb_all()) then I am guessing your program does a lot of mmap/unmap
(see *unpopulate* class in core/memory.cc that uses *tlb_gather*). I am not
familiar with details of what it tlb_gather exactly does it probably forces
TLB (Translation Lookaway Buffer) to flush old virtual/physical memory
mapping entries after unmapping. The mmu::*flush_tlb_all*() is actually
used in more places.

My wild suggestion would be to try to convert the tlb_flush_mutex to
spinlock (see include/osv/spinlock.h and core/spinlock.cc). It is a bit
controversial idea as OSv prides itself on lock-less structures and almost
no spinklocks used (the console initialization is the only place left). But
in some places (see
https://github.com/cloudius-systems/osv/issues/853#issuecomment-279215964)
and
https://github.com/cloudius-systems/osv/commit/f8866c0dfd7ca1fcb4b2d9a280946878313a75d3
and https://groups.google.com/g/osv-dev/c/4wMAHCs7_dk/m/1LHdvmoeBwAJ we may
benefit from those.

Please note the lock-less *sched::thread::wait_until* in the end of the
flush_tlb_all would need to be replaced with "busy" wait/sleep.

As far as the information on mutexes and scheduling, the best information
you can find in the original OSv paper -
https://www.usenix.org/conference/atc14/technical-sessions/presentation/kivity.
See also https://github.com/cloudius-systems/osv/wiki/Components-of-OSv and
many other Wikis.

Your preemption question - the lock-free mutex needs to have preemption on
- imagine if we have a single CPU and the mutex ends up getting into the
wait state to acquire a lock the thread would need to be eventually
switched to another one that would release the lock. But if the preemption
is off, then the scheduler will keep switching to the same waiting thread
for each timer event and our original thread would never acquire the lock.

I hope all this helps.

Waldek

Thanks in advance for any advice you could provide. The questions may
> be a bit dumb so pardon me if I disturb you.
> Best Wishes
> Pan
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfOPXt-0pz0tU2ftDdu6Ptcq_ObfT4OjVC8LKYyLXouHoQ%40mail.gmail.com.

[osv-dev] Re: NVMe driver

[Waldek Kozaczuk]

Hi Cyril,

Do you happen to still have a copy of that branch somewhere? I clicked on 
the github link and I got 404.

I would like to continue your work if you do not mind and have no copyright 
reservations.

Regards,
Waldek
On Wednesday, September 9, 2015 at 2:32:10 PM UTC-4 cyril@gmail.com 
wrote:

> Hello,
>
> I've developed this driver as an OSv learning exercise and want to solicit 
> some feedback. It uses only basic capabilities of the NVMe interface and 
> can be improved much further. However, all the basic functionality is in 
> place and it runs IO happily in my QEMU.
> I wouldn't consider it production quality yet, so I won't submit it for 
> integration. I would like to people to try it nonetheless. I especially 
> interested in seeing it working with NVMe 1.2 compliant hardware if 
> available.
>
> Instructions to test it under QEMU.
> 1. build OSv from my nvme branch [1]
> 2. Run ./scripts/run.py -VE -e "./zpool.so create data /dev/nvme1; 
> ./zpool.so status -v"
>
> scripts/run.py in this branch updated to create NVMe drive when -E flag is 
> used. I also build and run debug image - it spits quite a lot of debug 
> output to the screen.
>
> Have fun !
>
>
> [1] https://github.com/imp/osv/tree/nvme 
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/79ce4ba5-7ab5-437c-9485-b382f3cb20efn%40googlegroups.com.

[osv-dev] Re: [COMMIT osv master] Merge 'Dynamic linker: support loading and processing statically linked executables' from WALDEMAR KOZACZUK

[Waldek Kozaczuk]

Hi,

Could you please run 'readelf -W -l ' against this executable? Maybe it is 
a statically linked executable.

It is determined by this expression:

bool is_statically_linked() { return !_is_dynamically_linked_executable && 
_ehdr.e_entry; }

and _is_dynamically_linked_executable is set to true if there is PT_INTERP 
header.

On Sunday, September 10, 2023 at 5:24:43 AM UTC-4 n...@scylladb.com wrote:

> When I try to run the default scripts/build (the Lua shell) on Fedora 38 
> (with https://github.com/cloudius-systems/osv/pull/1257),
> I get the error message added in this patch:
>
> $ scripts/run.py 
> OSv v0.57.0-61-g72ed41c6
> eth0: 192.168.122.15
> Booted up in 156.82 ms
> Cmdline: /cli
>
> Statically linked executables are not supported yet!
>
> Why does it think the Lua executable is statically linked?
>
> --
> Nadav Har'El
> n...@scylladb.com
>
>
> On Thu, Aug 31, 2023 at 9:51 AM Commit Bot  
> wrote:
>
>> From: Nadav Har'El 
>> Committer: Nadav Har'El 
>> Branch: master
>>
>> Merge 'Dynamic linker: support loading and processing statically linked 
>> executables' from WALDEMAR KOZACZUK
>>
>> The two commits provide necessary modifications to the OSv dynamic linker 
>> to support loading and processing statically linked executables.
>>
>> Please note these changes are NOT enough to make OSv run statically 
>> linked executables.
>>
>> Closes #1253
>>
>> * github.com:cloudius-systems/osv:
>>   dynamic linker: support loading and processing static ELF
>>   dynamic linker: rename is_executable to is_dynamically_linked_executable
>>
>> ---
>> diff --git a/arch/aarch64/arch-elf.cc b/arch/aarch64/arch-elf.cc
>> --- a/arch/aarch64/arch-elf.cc
>> +++ b/arch/aarch64/arch-elf.cc
>> @@ -66,7 +66,7 @@ bool object::arch_relocate_rela(u32 type, u32 sym, void 
>> *addr,
>>  if (sym) {
>>  auto sm = symbol(sym);
>>  ulong tls_offset;
>> -if (sm.obj->is_executable()) {
>> +if (sm.obj->is_dynamically_linked_executable()) {
>>  // If this is an executable (pie or position-dependant 
>> one)
>>  // then the variable is located in the reserved slot of 
>> the TLS
>>  // right where the kernel TLS lives
>> @@ -119,7 +119,7 @@ void object::arch_relocate_tls_desc(u32 sym, void 
>> *addr, Elf64_Sxword addend)
>>  ulong tls_offset;
>>  if (sym) {
>>  auto sm = symbol(sym);
>> -if (sm.obj->is_executable() || sm.obj->is_core()) {
>> +if (sm.obj->is_dynamically_linked_executable() || 
>> sm.obj->is_core()) {
>>  // If this is an executable (pie or position-dependant one)
>>  // then the variable is located in the reserved slot of the 
>> TLS
>>  // right where the kernel TLS lives
>> @@ -163,7 +163,7 @@ void object::prepare_initial_tls(void* buffer, size_t 
>> size,
>>
>>  void object::prepare_local_tls(std::vector& offsets)
>>  {
>> -if (!_static_tls && !is_executable()) {
>> +if (!_static_tls && !is_dynamically_linked_executable()) {
>>  return;
>>  }
>>
>> diff --git a/arch/aarch64/arch-switch.hh b/arch/aarch64/arch-switch.hh
>> --- a/arch/aarch64/arch-switch.hh
>> +++ b/arch/aarch64/arch-switch.hh
>> @@ -120,7 +120,7 @@ void thread::setup_tcb()
>>  assert(obj);
>>  user_tls_size = obj->initial_tls_size();
>>  user_tls_data = obj->initial_tls();
>> -if (obj->is_executable()) {
>> +if (obj->is_dynamically_linked_executable()) {
>> executable_tls_size = obj->get_tls_size();
>>  }
>>  }
>> diff --git a/arch/x64/arch-elf.cc b/arch/x64/arch-elf.cc
>> --- a/arch/x64/arch-elf.cc
>> +++ b/arch/x64/arch-elf.cc
>> @@ -138,7 +138,7 @@ bool object::arch_relocate_rela(u32 type, u32 sym, 
>> void *addr,
>>  if (sym) {
>>  auto sm = symbol(sym);
>>  ulong tls_offset;
>> -if (sm.obj->is_executable()) {
>> +if (sm.obj->is_dynamically_linked_executable()) {
>>  // If this is an executable (pie or position-dependant 
>> one)
>>  // then the variable is located in the reserved slot of 
>> the TLS
>>  // right where the kernel TLS lives
>> @@ -202,7 +202,7 @@ void object::prepare_initial_tls(void* buffer, size_t 
>> size,
>>
>>  void object::prepare_local_tls(std::vector& offsets)
>>  {
>> -if (!_static_tls && !is_executable()) {
>> +if (!_static_tls && !is_dynamically_linked_executable()) {
>>  return;
>>  }
>>
>> diff --git a/arch/x64/arch-switch.hh b/arch/x64/arch-switch.hh
>> --- a/arch/x64/arch-switch.hh
>> +++ b/arch/x64/arch-switch.hh
>> @@ -210,7 +210,7 @@ void thread::setup_tcb()
>>  assert(obj);
>>  user_tls_size = obj->initial_tls_size();
>>  user_tls_data = obj->initial_tls();
>> -if (obj->is_executable()) {
>> +if (obj->is_dynamically_linked_executable()) {
>> executable_tls_size = obj->get_tls_size();
>> 

Re: [osv-dev] Unhandled relocation type of 'R_X86_64_IRELATIVE'

[Waldek Kozaczuk]

Hi,

I have created a pull request that should address this 
- https://github.com/cloudius-systems/osv/pull/1245. Please give it a try 
and let me know.

Waldek

On Thursday, August 10, 2023 at 5:59:13 PM UTC-4 Waldek Kozaczuk wrote:

> Hi,
>
>
> Yes indeed OSv does not support this relocation at this time. 
>
> However I am working on the changes to support static executables that 
> include adding support of these relocations. I should be sending 
> corresponding PR soon. 
>
> Regards,
> Waldek
>
> On Thu, Aug 10, 2023 at 02:30 Deepak Krishnan  wrote:
>
>> Hi,
>>
>> I was playing around with OSv and came across an .so file with a 
>> ''R_X86_64_IRELATIVE' entry in PLTGOT - which seems to be crashing OSv 
>> because of this assert in core/elf.cc:relocate_pltgot() : assert(type == 
>> ARCH_JUMP_SLOT || type == ARCH_TLSDESC)
>>
>> The binary itself seems to have been built using GCC 7.3.0 and isn't 
>> anything special as far as I can see. What's the reason behind the assert 
>> there? Is an IRELATIVE type relocation not expected or is it simply 
>> unhandled?
>>
>> Thank You in advance.
>>
>> -Deepak 
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "OSv Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to osv-dev+u...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/osv-dev/3ef4fbbb-2d8a-4aa5-8254-54eb742e613fn%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/osv-dev/3ef4fbbb-2d8a-4aa5-8254-54eb742e613fn%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/b0675763-2318-4a7f-a28c-746c6d506071n%40googlegroups.com.

Re: [osv-dev] Unhandled relocation type of 'R_X86_64_IRELATIVE'

[Waldek Kozaczuk]

Hi,


Yes indeed OSv does not support this relocation at this time.

However I am working on the changes to support static executables that
include adding support of these relocations. I should be sending
corresponding PR soon.

Regards,
Waldek

On Thu, Aug 10, 2023 at 02:30 Deepak Krishnan  wrote:

> Hi,
>
> I was playing around with OSv and came across an .so file with a
> ''R_X86_64_IRELATIVE' entry in PLTGOT - which seems to be crashing OSv
> because of this assert in core/elf.cc:relocate_pltgot() : assert(type ==
> ARCH_JUMP_SLOT || type == ARCH_TLSDESC)
>
> The binary itself seems to have been built using GCC 7.3.0 and isn't
> anything special as far as I can see. What's the reason behind the assert
> there? Is an IRELATIVE type relocation not expected or is it simply
> unhandled?
>
> Thank You in advance.
>
> -Deepak
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/3ef4fbbb-2d8a-4aa5-8254-54eb742e613fn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfNuRTWsL9wr7Bzekq3GXA0WTig3L7S69d7g-k8XpPzHRw%40mail.gmail.com.

[osv-dev] Pull requests

[Waldek Kozaczuk]

Hi,

I am about to create a number of pull requests to address various things I 
have discovered when working on adding support for static executables in 
OSv. Most of those are minimal or fairly simple and I wonder if I should 
create one large pull request with many commits or if I shall create PR per 
each fixed issue/commit? Preferences?

Regards,
Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/3be2ab22-dbc0-46d8-b08b-066635e38487n%40googlegroups.com.

[osv-dev] OSv and Genezio presentations

[Waldek Kozaczuk]

Hi,

I am back after a family summer break vacation so you should hear from me 
more often hopefully.

I would also like to share links to the OSv presentation I gave a month ago 
on the Unikernel Alliance forum:
- 
recording: 
https://www.youtube.com/watch?v=KnzRnKTCmjc=PLnqfImRjARAjX5veufPxCkGYntfsl8Eb_=3
- slides PDF: 
https://drive.google.com/file/d/1XPzjLPRnq58HzW84yn98Ut7GVbOp6_cq/view?pli=1
- slides on Google 
docs: 
https://docs.google.com/presentation/d/1-dlart1tFslsdjZVajkKpbyREN0UKGoIo55HvZtAl9s/edit?usp=sharing

I would also like to share the presentation by the Genezio team that has 
been building their serverless platform based on OSv:

- 
recording: 
https://www.youtube.com/watch?v=xRmRXCY_h6I=PLnqfImRjARAjX5veufPxCkGYntfsl8Eb_=4
- slides 
PDF: 
https://drive.google.com/file/d/1G_41wvszW8WAgOW1m108_cAP1SZ5lkOC/view?usp=drive_link

Enjoy,
Waldek Kozaczuk

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/853de34d-67aa-470b-96c1-16f6e8fb7589n%40googlegroups.com.

[osv-dev] Changes to support static executables and dynamically linked executables via Linux dynamic linker

[Waldek Kozaczuk]

Hi,

For the last 2 months or more, I have been quietly working on the changes 
to OSv to support static executables and dynamically linked executables via 
Linux dynamic linker (aka ld-linux-x86-64.so.2).

I think I am at the point where I can start submitting the changes. And I 
am not sure whether I should use the traditional approach of sending the 
patches to this mailing list or create pull requests on Git Hub? 

I think the changes can be grouped into the following sets:

   - enhance OSv dynamic linker to support statically linked executables 
   (this would 3-5 commits/patches)
   - implement arch_prctl
   - implement FS register switch on syscall and interrupt/exception 
   entry/exit and other places
   - add all extra necessary syscalls to linux.cc
  - most of the 40-50 new ones are straightforward mappings to function 
  calls but some like clone and other require a bit of extra implementation 
- 
  so maybe 3-5 commits/patches
   - improve vdso: mostly add logic to handle FS register switch 
   - scripts like loader.py and build to support statically linked 
   executables
   - various identified related improvements/fixed - 5 -7 small patches

I have been testing my changes mostly by running various apps and unit 
tests via Linux dynamic linker (like ./scripts/run.py -e 
'/lib64/ld-linux-x86-64.so.2 /tests/tst-hello' where tst-hello is a 
dynamically linked pie) and I got following ones to work:

   - 86 unit tests (most of the ones that use standard glibc interface - 
   the tst-async.so and tst-app.so do not fit this bill)
   - nginx
   - lighhtpd
   - rust-pie-example
   - rust-pie-httpserver
   - golang-pie-example
   - golang-pie-httpserver
   - iperf3
   - simple executables from Linux host like ls/find/tree
   - lua
   - graalvm-hello, graalvm-httpserver
   - sqlite
   - redis
   - java including http server
   - node js
   - python
   - possibly many others

I have tested also the statically linked hello world executable (the 
ld-linux-x86-64.so.2  is also statically linked). Interestingly enough the 
Golang statically linked one does not work yet.

There are still some outstanding issues I have identified but I feel that 
most of the changes are ready to go.

Regards,
Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/d1503e6f-90ea-46a2-89b7-c6aefcaa2775n%40googlegroups.com.

[osv-dev] Re: New user questions ...

[Waldek Kozaczuk]

Hi,

Sorry for the late reply.

On Friday, June 30, 2023 at 5:06:44 AM UTC-4 ni...@descartes.co.uk wrote:

On Thursday, 29 June 2023 at 04:22:51 UTC+1 jwkoz...@gmail.com wrote:

Regarding your specific problem (I am assuming you are running the latest 
version 1.0 which I released a month or two ago), I would need to test and 
investigate it. How do you exactly build the image using Capstan? What 
packages/repo do you use?


I've tried 1.0 and 0.5, hacking away at a number of the examples to no 
avail, all from the main Git repro and docs.

If you have an example with the package & run.yaml that work using the 
mpm's from Git that may inform me better to investigate your suggestions 
below.

But overall, it's not a specifc problem for me as I appear to be able to 
get things tested with the OSv scripts so I don't appear to need Capstan.


I have researched the '*run.yaml*' - related functionality and it looks 
like it is only useful if you use *runtimes*.  The runtime is another 
concept in Capstan that was meant I think to simplify building Node, 
Python, and Java apps by implicitly adding the corresponding language 
runtime - Node.js or Python interpreter or JDK to the OSv image filesystem. 
In essence, it is a special package with extra implicit behavior. 
Unfortunately, the runtimes have to be ready to be fetched from an S3 
bucket or GitHub (there are some but super old) or prepared by the user on 
a local filesystem. Again, this is rather tedious and I do not have a 
recipe handy on how to create one.


 

I must admit that I do not use Capstan often myself to run images because 
run.py is so much more versatile. Please note you can always build images 
using Capstan which is pretty easy and advisable and you can run those 
images with run.py using the "-i " option. You can also use 
-dry-run option which will output a shell script that you can use instead.

./scripts/run.py -i 
~/.capstan/repository/test-golang-example-rofs/test-golang-example-rofs.qemu


I've looked in the repros & online but this test-golang-example-rofs 
doesn't seem readily available.
 

PS. This actively maintained shell script 
- scripts/tests/compose_and_test_selected_apps.sh - may be somewhat useful 
to see an example of how Capstan is used to build various images and run.py 
to run it.


As above & from my first post, happy to transcribe notes I have in to 
something that gets people going, if only to help with my aging 
magentic-core era wetware memory. Only you can say how helpful it would be 
right now to debug Capstan - I know you know that the docs are suffering 
badly from bitrot so I'm sure you won't be surprised that just getting to a 
compiled & working OSv with some of the examples working requires some 
determination!

However it strikes me that if I need to provide a mechanism for end users 
to inject their own configuration or code snippets, that Capstan will help 
streamline an automated server build and potentially provide the end user 
with a local test setup.

I'm unlikely to be useful in writing new components - I can see what's 
missing that causes the SQLite example to fail (no __fgets_chk) but 
implementing it would be a long voyage of discovery. I've not yet figured 
out where to tell the usr.manifest where to put a test.db for SQLite to use 
or if that's irrelevant due to the missing call. But I have managed to get 
a database in to the Python3x example (as well as getting Python3x to 
build) and can run the img and type SQLite related Python and it works.

I can be used for testing, some debugging, implementation examples and 
notes if that would be useful.


I am very interested in your help and suggestions on how we can improve 
Capstan and the use of OSv by users like you. 

Would you have some time to have a video chat on Zoom or something like 
this to discuss it?

Regards,
Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/fabe0bb7-0ccf-48ab-b213-55a798b18912n%40googlegroups.com.

[osv-dev] Fwd: [unikernel-alliance] Re: Next Meeting: OSv + Genezio, Thursday, June 29, 2023, 6pm CEST, online, Zoom

[Waldek Kozaczuk]

FYI
-- Forwarded message -
From: Răzvan Deaconescu 
Date: Wed, Jun 28, 2023 at 11:54 PM
Subject: [unikernel-alliance] Re: Next Meeting: OSv + Genezio, Thursday,
June 29, 2023, 6pm CEST, online, Zoom
To: Unikernel Alliance 


On Mon, Jun 26, 2023 at 10:58 AM Razvan Deaconescu 
wrote:
> Hi, everyone.
>
> We meet on Thursday, June 29, 2023, at 6pm CEST for our next Unikraft
> Alliance meeting. The meeting will take place online, on Zoom[1] (thanks
> for the link, Thomas):
>
> We will feature talks from
> - Waldek Kozaczuk: OSv
> - Bogdan Vlad: Genezio
>
> Full agenda is:
> * Next Events
> * Unikernel Alliance Administrative / Logistics
> * Talks:
>   * OSv: Waldek
>   * Genezio: Bogdan
> * Others
>
> We will record the talks for those who can't make it.
>
> We will keep the meeting summary on Google Drive[2].
>
> Meet you on Thursday!

Hi, everyone.

Reminder that we meet today, Thursday, June 29, 2023, at 6pm CEST for
our next Unikraft Alliance meeting. The meeting will take place
online, on Zoom[1].We will feature talks from:

- Waldek Kozaczuk: OSv
- Bogdan Vlad: Genezio

> [1]
https://us06web.zoom.us/j/83474439883?pwd=VVROZ1YrTmJkWS9lNVRrRnhPOVhPUT09
> [2]
https://docs.google.com/document/d/1yfC16igXi8dOx4byhNVOQTbf3X9ZP1ErsP6PtePgsyA/edit#heading=h.4lga7uz29mr

Razvan

-- 
You received this message because you are subscribed to the Google Groups
"Unikernel Alliance" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to unikernel-alliance+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/unikernel-alliance/CAFD6GprscqfD-6W8R%2BiCpGy9Jue5kQpmLWce5ycGMSQnbkAWaA%40mail.gmail.com
.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfNMY-T%2BKfc9Qxpabk6djC52J8JySVWwjBnMZuajnGMvYw%40mail.gmail.com.

[osv-dev] Re: New user questions ...

[Waldek Kozaczuk]

Hi,

Welcome to the OSv community!

As you have already realized OSv images can be built and run using these 
two methods:
- scripts/build and scripts/run.py from OSv repo
- Capstan

Unfortunately, Capstan has always been somewhat "behind" the 1st method in 
terms of filesystems (RoFS, ZFS, VirtioFS) and hypervisors it can run on. 
To make this even more confusing Capstan support 2 way of building images - 
with Capstanfile and with packages. We have plans to improve Capstan (see 
https://github.com/cloudius-systems/capstan/wiki/Capstan-2.0) but it may 
not happen until later this year.

Regarding your specific problem (I am assuming you are running the latest 
version 1.0 which I released a month or two ago), I would need to test and 
investigate it. How do you exactly build the image using Capstan? What 
packages/repo do you use?

I must admit that I do not use Capstan often myself to run images because 
run.py is so much more versatile. Please note you can always build images 
using Capstan which is pretty easy and advisable and you can run those 
images with run.py using the "-i " option. You can also use 
-dry-run option which will output a shell script that you can use instead.

./scripts/run.py -i 
~/.capstan/repository/test-golang-example-rofs/test-golang-example-rofs.qemu 

OSv v0.57.0
eth0: 192.168.122.15
Booted up in 172.55 ms
Cmdline: runscript /run/default;
runscript expand fn='/run/default' line=0 '/go.so /hello.so'
Hello, 世界
Go version: go1.12.6

Regards,
Waldek

PS. This actively maintained shell script 
- scripts/tests/compose_and_test_selected_apps.sh - may be somewhat useful 
to see an example of how Capstan is used to build various images and run.py 
to run it.
On Wednesday, June 28, 2023 at 6:52:27 AM UTC-4 ni...@descartes.co.uk wrote:

> Hello!
>
> I've a use case of having lots of small pockets of data with a REST api, 
> isolated as the user base have their own unique set of columns in the 
> primary table as well as user security concerns / paranoia.
>
> I've got as far as running OSv native, Python3 (some fixes required) & Go 
> examples using the .scripts/build & run, I can see the quick launch (sub 
> 200ms) and low disk & memory use which is exactly what would be needed to 
> start on-demand & quit when dormant.
>
> I haven't been able to get Capstan to run - it composes but appears to 
> ignore the run.yaml and tries /run/default. Even if the yaml has default 
> defined that doesn't work either.
>
> I can clean up my notes as a sort of quick start doc but it would be 
> useful to know who/how OSv is being used lest I end up going down the wrong 
> avenue.
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/460c9068-2a8a-4a58-b48c-bb91928e156fn%40googlegroups.com.

[osv-dev] Fwd: [unikernel-alliance] Next meeting: Thursday, June 29, 2023, 6pm CEST: OSv + Genezio

[Waldek Kozaczuk]

FYI

-- Forwarded message -
From: Razvan Deaconescu 
Date: Wed, Jun 7, 2023 at 12:15 AM
Subject: [unikernel-alliance] Next meeting: Thursday, June 29, 2023, 6pm
CEST: OSv + Genezio
To: Unikernel Alliance 


Hi, everyone.

We organize our next Unikernel Alliance meeting on Thursday, June 29,
2023, 6pm CEST. We will use Zoom[1], same as last time - thanks,
Thomas. I updated the details in the calendar invite.

Please distribute the announcement in your communities. Everyone is
welcome to join.

We will feature talks from the OSv project and from Genezio. Who will
give the talks? Waldek for OSv? Bogdan / Andreia for Genezio? Each talk
should fit into a 20 minutes timeslot to allow time for questions and
discussions. Similar to the last meeting, we will record the talks and
provide the recordings afterward for those who can't make it.

Full agenda of the meeting:
* Next Events
* Unikernel Alliance Administrative / Logistics
* Talks
  * OSv: Waldek
  *Genezio: Bogdan / Andreia
* Others

We keep the meeting summary in the "Unikernel Alliance: Meetings"
document, part of the "Unikernel Alliance" Google Drive directory[2].

[1]
https://us06web.zoom.us/j/83474439883?pwd=VVROZ1YrTmJkWS9lNVRrRnhPOVhPUT09
[2]
https://drive.google.com/drive/folders/1k4epAn8kMrypj5X1D13tOCKW2OgW4olf?usp=share_link

Razvan

-- 
You received this message because you are subscribed to the Google Groups
"Unikernel Alliance" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to unikernel-alliance+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/unikernel-alliance/86o7lrdi1u.fsf%40drone.cs.pub.ro
.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfNsw5EEABMrGQ9D85xVbdRyeYu7cFu2H%2B3_JfzoSUybLw%40mail.gmail.com.

Re: [osv-dev] [V2 3/4] libc: add RT sched API

[Waldek Kozaczuk]

BTW I have just pushed Nadav's latest low-level real-time scheduler MR so 
you no longer have to send it along with others (please rebase accordingly 
if necessary). 

I saw some of you mentioned OSv pid. It used to be 0 but now it is 2 (see 
constant in include/osv/pid.h). Changing from 0 to 2 helps us run some apps 
(like Dotnet) that do not like to run with pid 0.

Regards,
Waldek
On Monday, May 22, 2023 at 12:23:46 PM UTC-4 Luca Abeni wrote:

> Hi,
>
> On Mon, 22 May 2023 at 14:30, Nadav Har'El  wrote:
> >
> > On Wed, May 10, 2023 at 12:47 PM Nadav Har'El  wrote:
> >>
> >>
> >> sched::thread *t;
> >> if (pid == 0) {
> >> t = sched::thread::current();
> >> } else {
> >> t = sched::thread::find_by_id(pid);
> >> if (!t) {
> >> errno = ESRCH;
> >> return -1;
> >> }
> >> // TODO: After the thread was found, if it exits the code below
> >> // may crash. Perhaps we should have a version of find_by_id(),
> >> // with_thread_by_id(pid, func), which holds thread_map_mutex while
> >> // func runs.
> >> }
> >>
> >> I think you can just use exactly that code, as is (I guess with the 
> same TODO, which is still valid).
> >
> >
> > I looked again, and we actually have a function exactly like I wished 
> for in the TODO :-)
> > with_thread_by_id() from include/osv/sched.hh. I added it in commit 
> f142ccf42f5d4465e827e3ff0a330b62d070755d
> > six years ago - I guess I fulfilled my own wish but then forgot to go 
> back to where I wanted to use it.
> >
> > So you should probably use the with_thread_by_id() function.
>
> OK, I am looking at this API right now... So, the setscheduler
> function would look like
> int sched_setscheduler(pid_t pid, int policy, const struct sched_param 
> *param)
> {
> sched::thread *t;
> int res = -1;
>
> if (!param) {
> errno = EINVAL;
> return -1;
> }
>
> if (pid == 0) {
> t = sched::thread::current();
> res = sched_setparam_aux(t, policy, param->sched_priority);
> } else {
> sched::with_thread_by_id(pid, [param, policy, ] (sched::thread *t) {
> if (t) res = sched_setparam_aux(t, policy, param->sched_priority);
> });
> if (res == -1) {
> errno = ESRCH;
> }
> }
>
> return res;
> }
>
> ...Right? For the moment, this is only compile-tested; if you agree
> that this is the right thing to do, I'll test it and I will update all
> the parts of the patch where "getpid()" was erroneously called.
> (note: I left the "pid == 0" check to have an efficient fastpath; If
> this is a useless optimization, I'll remove the if() - the alternative
> would be to always call something like "with_thread_by_id(pid ? pid:
> gettid(), ...))
>
>
> Luca
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/c86f74eb-179c-4a6f-b716-6831e82eb47en%40googlegroups.com.

Re: [osv-dev] Patches for running cyclictest on OSv

[Waldek Kozaczuk]

Luca,

I am not sure you have seen newly updated PR by Nadav that adds low level
support -
https://github.com/cloudius-systems/osv/pull/1223. You may want to use it
update your branch.

I have reviewed his PR and it looks good to me. But ideally I want to run
more tests on my side and I would like you to run some tests on your side
that involve the high level code before I merge this PR into master. It
obviously touches very critical part of the kernel - thread scheduler.

Regards and thanks for your work Luca.

Waldek

On Wed, May 3, 2023 at 14:04 Luca Abeni  wrote:

> Hi again,
>
> On Mon, 17 Apr 2023 at 10:30, Nadav Har'El  wrote:
> [...]
> >> > 6. Please verify the changes to clock_gettime() and nonsleep() to
> return EINVAL when given a null pointer is the same in Linux (i.e., that it
> doesn't return EFAULT) in that case. It's a bit sad we need to waste time
> on these checks :-(
> >>
> >> I just tested, and on Linux a program passing NULL to clock_gettime()
> >> just segfaults...
> >
> >
> > Interesting. Maybe even though clock_gettime() is in section 2 of the
> manual, it isn't a real system call
> > and is a C wrapper - which segfaults? Real system calls are supposed to
> return EFAULT, not segfault.
>
> I verified this happens because clock_gettime() is a vsyscall
> implemented through the vDSO. This is the backtrace:
> Program received signal SIGSEGV, Segmentation fault.
> do_hres (ts=, clk=, vd=)
> at
> /build/linux-lowlatency-UVnJHF/linux-lowlatency-6.2.0/include/vdso/math64.h:21
> Downloading source file
> /build/linux-lowlatency-UVnJHF/linux-lowlatency-6.2.0/include/vdso/math64.h
> 21
> /build/linux-lowlatency-UVnJHF/linux-lowlatency-6.2.0/include/vdso/math64.h:
> Directory not empty.
> (gdb) bt
> #0  do_hres (ts=, clk=, vd=)
> at
> /build/linux-lowlatency-UVnJHF/linux-lowlatency-6.2.0/include/vdso/math64.h:21
> #1  __cvdso_clock_gettime_common (ts=0x0, clock=1, vd=)
> at
> /build/linux-lowlatency-UVnJHF/linux-lowlatency-6.2.0/arch/x86/entry/vdso/../../../../lib/vdso/gettimeofday.c:251
> #2  __cvdso_clock_gettime_data (clock=1, ts=0x0, vd=)
> at
> /build/linux-lowlatency-UVnJHF/linux-lowlatency-6.2.0/arch/x86/entry/vdso/../../../../lib/vdso/gettimeofday.c:258
> #3  0x00421f66 in clock_gettime ()
> #4  0x00401791 in test (ts=0x0) at gettime_test.c:11
> #5  0x00401802 in main () at gettime_test.c:23
>
> And the kernel documentation explains that
> "
> ...if you pass a bad pointer to a vDSO function, you might get SIGSEGV
> instead of -EFAULT.
> "
>
>
> So, I guess we can remove the NULL check :)
>
> Luca
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/CAJfkoWq7HqAv1drjf54vr0buwfavY3mLT%2BUQfFrfLTntNXcg5w%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfN2fiDrF96tqVHL87XZHsBjb5O9dkOo-Ci-ysGk-6S7ag%40mail.gmail.com.

[osv-dev] OSv can run dynamically linked executables via Linux dynamic linker - ld-linux-x86-64.so.2

[Waldek Kozaczuk]

Yesterday I tweaked OSv further to get it to run a dynamically linked 
executable 'Hello World' via ld-linux-x86-64.so.2 instead of OSv's built-in 
dynamic linker. In essence, I built upon the work to get the static 
executable to work and modified core/elf.cc to handle 
ld-linux-x86-64.so.2 as a program interpreter to launch an executable. I 
also had to enhance the startup code to prepare the program stack 
accordingly.

OSv v0.57.0-41-gf27b35b9
Booted up in 4.57 ms
Cmdline: /usr/lib/ld-linux-x86-64.so.2 /hello 
 -> syscall: 012
 -> syscall: 158
 -> syscall: 257
 -> syscall: 000
 -> syscall: 017
 -> syscall: 017
 -> syscall: 009
 -> syscall: 009
 -> syscall: 009
 -> syscall: 009
 -> syscall: 003
 -> syscall: 063
 -> syscall: 021
 -> syscall: 257
 -> syscall: 257
 -> syscall: 262
 -> syscall: 257
 -> syscall: 262
 -> syscall: 257
 -> syscall: 262
 -> syscall: 257
 -> syscall: 262
 -> syscall: 257
 -> syscall: 262
 -> syscall: 257
 -> syscall: 262
 -> syscall: 257
 -> syscall: 262
 -> syscall: 257
 -> syscall: 262
 -> syscall: 257
 -> syscall: 262
 -> syscall: 257
 -> syscall: 000
 -> syscall: 017
 -> syscall: 017
 -> syscall: 017
 -> syscall: 262
 -> syscall: 017
 -> syscall: 009
 -> syscall: 009
 -> syscall: 009
 -> syscall: 009
 -> syscall: 009
 -> syscall: 003
 -> syscall: 009
 -> syscall: 158
 -> syscall: 218
 -> syscall: 273
 -> syscall: 010
 -> syscall: 010
 -> syscall: 010
 -> syscall: 302
 -> syscall: 262
 -> syscall: 016
 -> syscall: 318
 -> syscall: 228
 -> syscall: 228
 -> syscall: 012
 -> syscall: 012
 -> syscall: 001
Hello from C code
 -> syscall: 231

and the memory map from gdb:
(gdb) osv mmap
0x 0x [0.0 kB] flags=none 
perm=none
0x1000 0x10001000 [4.0 kB] flags=fmF 
 perm=roffset=0x path=/libvdso.so
0x10001000 0x10002000 [4.0 kB] flags=fmF 
 perm=rx   offset=0x1000 path=/libvdso.so
0x10002000 0x10003000 [4.0 kB] flags=fmF 
 perm=roffset=0x2000 path=/libvdso.so
0x10003000 0x10004000 [4.0 kB] flags=fmF 
 perm=roffset=0x2000 path=/libvdso.so
0x10004000 0x10005000 [4.0 kB] flags=fmF 
 perm=rw   offset=0x3000 path=/libvdso.so
0x10005000 0x10007000 [8.0 kB] flags=fmF 
 perm=roffset=0x path=/usr/lib/ld-linux-x86-64.so.2
0x10007000 0x1002d000 [152.0 kB]   flags=fmF 
 perm=rx   offset=0x2000 path=/usr/lib/ld-linux-x86-64.so.2
0x1002d000 0x10038000 [44.0 kB]flags=fmF 
 perm=roffset=0x00028000 path=/usr/lib/ld-linux-x86-64.so.2
0x10039000 0x1003b000 [8.0 kB] flags=fmF 
 perm=roffset=0x00033000 path=/usr/lib/ld-linux-x86-64.so.2
0x1003b000 0x1003d000 [8.0 kB] flags=fmF 
 perm=rw   offset=0x00035000 path=/usr/lib/ld-linux-x86-64.so.2
0x2000 0x20001000 [4.0 kB] flags=p   
 perm=none
0x20001000 0x20002000 [4.0 kB] flags=p   
 perm=none
0x20002000 0x20101000 [1020.0 kB]  flags=p   
 perm=rw  
0x20101000 0x20102000 [4.0 kB] flags=p   
 perm=none
0x20102000 0x20201000 [1020.0 kB]  flags=p   
 perm=rw  
0x20201000 0x20301000 [1024.0 kB]  flags=none 
perm=rw  
0x20301000 0x20302000 [4.0 kB] flags=mF   
perm=roffset=0x path=/hello
0x20302000 0x20303000 [4.0 kB] flags=fmF 
 perm=rx   offset=0x1000 path=/hello
0x20303000 0x20304000 [4.0 kB] flags=fmF 
 perm=roffset=0x2000 path=/hello
0x20304000 0x20305000 [4.0 kB] flags=fmF 
 perm=roffset=0x2000 path=/hello
0x20305000 0x20306000 [4.0 kB] flags=fmF 
 perm=rw   offset=0x3000 path=/hello
0x20306000 0x20308000 [8.0 kB] flags=none 
perm=rw  
0x2040 0x20428000 [160.0 kB]   flags=mF   
perm=roffset=0x path=/lib64/libc.so.6
0x20428000 0x2059c000 [1488.0 kB]  flags=fmF 
 perm=rx   offset=0x00028000 path=/lib64/libc.so.6
0x2059c000 0x205f4000 [352.0 kB]   flags=fmF 
 perm=roffset=0x0019c000 path=/lib64/libc.so.6
0x205f4000 0x205f8000 [16.0 kB]flags=fmF 
 perm=roffset=0x001f3000 path=/lib64/libc.so.6
0x205f8000 0x205fa000 [8.0 kB] flags=fmF 
 perm=rw   offset=0x001f7000 path=/lib64/libc.so.6
0x205fa000 0x20607000 [52.0 kB]flags=f   
 perm=rw  
0x4000 0x4000 [0.0 kB] flags=none 
perm=none

As you can see we have to explicitly tell OSv to run ld-linux-x86-64.so.2 
as a statically linked PIE and then give control over to it to 

[osv-dev] Measuring cost of the syscall on OSv and Linux

[Waldek Kozaczuk]

Hi,

I wrote a simple program that calls the gettid syscall 50 million times in 
a loop to measure the cost of syscall. I am interested if my test program 
makes sense and can be improved. More importantly if it correctly measures 
what I want to measure.

Here is the program:

#define _GNU_SOURCE
#include 
#include 
#include 
#include 
#include 
#include 
#include 

#ifdef __OSV__
long gettid();
#endif

void call_gettid_syscall()
{
// errors are returned).
unsigned long syscall_nr = __NR_gettid;
long tid = 0;

#ifdef __x86_64__
asm ("movq %[syscall_no], %%rax\n"
 "syscall\n"
 "movq %%rax, %[tid]\n"
 : [tid]"=m" (tid)
 : [syscall_no]"m" (syscall_nr)
 : "rax", "rdi");
#endif

#ifdef __aarch64__
asm ("mov x8, %[syscall_no]\n"
 "svc #0\n"
 "mov %[tid], x0\n"
 : [tid]"=r" (tid)
 : [syscall_no]"r" (syscall_nr)
 : "x0", "x8");
#endif

assert(tid >= 0);
}

uint64_t nstime()
{
struct timeval tv;
gettimeofday(, NULL);
uint64_t mul = 10, mul2 = 1000;
return tv.tv_sec * mul + tv.tv_usec * mul2;
}

int main(int argc, char **argv)
{
long count = 5000;
long loop = count;
uint64_t start = nstime();

while (loop--) {
call_gettid_syscall();
}

uint64_t end = nstime();

printf("%lu ns elapsed %.2f sec %s\n", (end - start) / count, (end - 
start) / 10.0, " gettid syscall");

loop = count;
start = nstime();

long tid = 0;
while (loop--) {
tid = gettid();
assert(tid >=0);
}

end = nstime();

printf("%lu ns elapsed %.2f sec %s\n", (end - start) / count, (end - 
start) / 10.0, " gettid local");
}

Here are the results on OSv (3 runs) with single cpu:

./scripts/run.py -e /tests/misc-syscall-perf.so -c 1
106 ns elapsed 5.33 sec  gettid syscall
 2 ns elapsed 0.14 sec  gettid local

107 ns elapsed 5.38 sec  gettid syscall
2 ns elapsed 0.15 sec  gettid local

107 ns elapsed 5.37 sec  gettid syscall
 2 ns elapsed 0.15 sec  gettid local

and on Linux:

taskset -c 1-1 /tmp/misc-syscall-perf

367 ns elapsed 18.38 sec  gettid syscall
367 ns elapsed 18.37 sec  gettid local

367 ns elapsed 18.39 sec  gettid syscall
366 ns elapsed 18.32 sec  gettid local

367 ns elapsed 18.40 sec  gettid syscall
366 ns elapsed 18.30 sec  gettid local

So clearly the local function calls on OSv are the winners but even the 
syscalls on OSv are almost 4 times faster than on Linux (BTW I did not see 
much difference with the static executable on OSv).

Regards,
Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/5f9e8c9e-8bb7-4da1-884e-1bb22ba5e36dn%40googlegroups.com.

Re: [osv-dev] OSv can run statically linked executable

[Waldek Kozaczuk]

On Monday, April 24, 2023 at 2:08:26 PM UTC-4 Waldek Kozaczuk wrote:

On Mon, Apr 24, 2023 at 4:34 AM Nadav Har'El  wrote:

On Mon, Apr 24, 2023 at 6:26 AM Waldek Kozaczuk  wrote:

Hi,

Over the recent week, I have been working to get OSv to run a simple "hello 
world" app (aka native-example) built as a position-dependent statically 
linked executable. 


Nice!
 


   1. Support the arch_prctl syscall that sets the app TLS 
  - this was by far the most complicated element that required changing 
  OSv to store new per-pcpu data pointer in GS register and enhancing both 
  syscall handler and interrupt/page fault handler to detect and switch if 
  needed the FS base to the kernel TLS on entry and back to the app one on 
  exit (see 
  
https://github.com/cloudius-systems/osv/issues/1137#issuecomment-1512315880
  ) 
   

If this has noticeable overhead, perhaps it makes sense to make it optional?

I have not measured it in any formal way. But when testing some of the 
earlier versions of the code, I could see the context switch time (the 
colocated one measured by misc-ctxsw) go up from 313 to 362 ns caused by 
adding this line: 

processor::wrmsr(msr::IA32_FS_KERNEL_BASE, reinterpret_cast(_tcb));
which may indirectly measure the cost of the code to change GS or FS base 
using the MSR instruction at ~ 50 ns (yikes). I would think the FSGSBASE 
instruction should be faster.


BTW I have measured indirectly the cost of the MSR and wrgsbase indirectly 
by modifying the thread::switch() like this and running misc-ctxsw (I 
assume the cost of wrfsbase would be identical):
 
+uint32_t IA32_GS_BASE = 0xc101;
 void thread::switch_to()
 {
 thread* old = current();
@@ -81,6 +82,8 @@ void thread::switch_to()
 // barriers
 barrier();
 set_fsbase(reinterpret_cast(_tcb));
+//asm volatile("wrgsbase %0" : : "r"(reinterpret_cast(_tcb)));
+//processor::wrmsr(IA32_GS_BASE, reinterpret_cast(_tcb));
 barrier();
 auto c = _detached_state->_cpu;
 old->_state.exception_stack = c->arch.get_exception_stack();

With uncommented wrgsbase the cost of colocated context switch barely 
budged. On average, I could see a maximum of 1-2 ns difference if any. 
Sometimes the times were identical. So it seems the wrgsbase is pretty 
cheap though we should avoid calling it in the interrupt/page fault/syscall 
handler.

On the other hand, with uncommented wrmsr code the cost of the context 
switch bumped by ~50ns so this instruction is very expensive. That is also 
why we need to especially avoid it if wrgsbase is not available.


Here is a subset of the changes I had to make to the context-switching code 
and the interrupt/syscall handler:

1. Add 2 new fields to the thread control block:

unsigned long app_tcb;   //holds address of the address the app passed 
to arch_prctl

long kernel_tcb_counter; //if 0 means we have to do an app/kernel/app 
FS base switch

2. Setup new per-cpu data intended to hold a pointer to the tcb:

--- a/arch/x64/arch-cpu.hh

+++ b/arch/x64/arch-cpu.hh


+struct tcb_data {

+u64 kernel_tcb;

+u64 tmp[2];

+};

+

 struct arch_cpu {

 arch_cpu();

 processor::aligned_task_state_segment atss;

@@ -46,6 +52,7 @@ struct arch_cpu {

 u32 apic_id;

 u32 acpi_id;

 u64 gdt[nr_gdt];

+tcb_data _tcb_data;

 void init_on_cpu();

 void set_ist_entry(unsigned ist, char* base, size_t size);

 char* get_ist_entry(unsigned ist);

@@ -181,6 +188,8 @@ inline void arch_cpu::init_on_cpu()

 processor::init_fpu();

 

 processor::init_syscall();

+

+processor::wrmsr(msr::IA32_GS_BASE, 
reinterpret_cast(&_tcb_data.kernel_tcb));

 }



3. Change kernel fs pointer on each context switch.

--- a/arch/x64/arch-switch.hh

+++ b/arch/x64/arch-switch.hh


@@ -81,11 +81,13 @@ void thread::switch_to()

...

 c->arch.set_exception_stack(_state.exception_stack);

+c->arch._tcb_data.kernel_tcb = reinterpret_cast(_tcb); //This 
should be very fast

 auto fpucw = processor::fnstcw();

...

@@ -258,6 +260,7 @@ void thread::setup_tcb()

 else {

 _tcb->syscall_stack_top = 0;

 }

+_tcb->kernel_tcb_counter = 1; //By default disable fs base switch

 }

4. Handle fs switch if necessary on entry/exit of syscall/exception/page 
fault handler:

This is just a code change around syscall entry but we have to do the 
opposite for exit and similar for page fault/interrupt handler (possibly 
signal handler as well) 

@@ -174,6 +214,26 @@ syscall_entry:

 .cfi_register rip, rcx # rcx took previous rip value

 .cfi_register rflags, r11 # r11 took previous rflags value

 # There is no ring transition and rflags are left unchanged.

+#

+# app->kernel tcb switch

+movq %rax, %gs:8  # save register rax so we can restore it later

+movq %gs:0, %rax  # copy address of kernel tcb to the temp register rax

+#1. Check 

Re: [osv-dev] OSv can run statically linked executable

[Waldek Kozaczuk]

On Mon, Apr 24, 2023 at 4:34 AM Nadav Har'El  wrote:

> On Mon, Apr 24, 2023 at 6:26 AM Waldek Kozaczuk 
> wrote:
>
>> Hi,
>>
>> Over the recent week, I have been working to get OSv to run a simple
>> "hello world" app (aka native-example) built as a position-dependent
>> statically linked executable.
>>
>
> Nice!
>
>
>>
>>1. Support the arch_prctl syscall that sets the app TLS
>>   - this was by far the most complicated element that required
>>   changing OSv to store new per-pcpu data pointer in GS register and
>>   enhancing both syscall handler and interrupt/page fault handler to 
>> detect
>>   and switch if needed the FS base to the kernel TLS on entry and back 
>> to the
>>   app one on exit (see
>>   
>> https://github.com/cloudius-systems/osv/issues/1137#issuecomment-1512315880
>>   )
>>
>>
> If this has noticeable overhead, perhaps it makes sense to make it
> optional?
>
I have not measured it in any formal way. But when testing some of the
earlier versions of the code, I could see the context switch time (the
colocated one measured by misc-ctxsw) go up from 313 to 362 ns caused by
adding this line:

processor::wrmsr(msr::IA32_FS_KERNEL_BASE, reinterpret_cast(_tcb));

which may indirectly measure the cost of the code to change GS or FS base
using the MSR instruction at ~ 50 ns (yikes). I would think the FSGSBASE
instruction should be faster.

Here is a subset of the changes I had to make to the context-switching code
and the interrupt/syscall handler:

1. Add 2 new fields to the thread control block:

unsigned long app_tcb;   //holds address of the address the app passed
to arch_prctl

long kernel_tcb_counter; //if 0 means we have to do an app/kernel/app
FS base switch

2. Setup new per-cpu data intended to hold a pointer to the tcb:

--- a/arch/x64/arch-cpu.hh

+++ b/arch/x64/arch-cpu.hh


+struct tcb_data {

+u64 kernel_tcb;

+u64 tmp[2];

+};

+

 struct arch_cpu {

 arch_cpu();

 processor::aligned_task_state_segment atss;

@@ -46,6 +52,7 @@ struct arch_cpu {

 u32 apic_id;

 u32 acpi_id;

 u64 gdt[nr_gdt];

+tcb_data _tcb_data;

 void init_on_cpu();

 void set_ist_entry(unsigned ist, char* base, size_t size);

 char* get_ist_entry(unsigned ist);

@@ -181,6 +188,8 @@ inline void arch_cpu::init_on_cpu()

 processor::init_fpu();



 processor::init_syscall();

+

+processor::wrmsr(msr::IA32_GS_BASE,
reinterpret_cast(&_tcb_data.kernel_tcb));

 }



3. Change kernel fs pointer on each context switch.

--- a/arch/x64/arch-switch.hh

+++ b/arch/x64/arch-switch.hh


@@ -81,11 +81,13 @@ void thread::switch_to()

...

 c->arch.set_exception_stack(_state.exception_stack);

+c->arch._tcb_data.kernel_tcb = reinterpret_cast(_tcb); //This
should be very fast

 auto fpucw = processor::fnstcw();

...

@@ -258,6 +260,7 @@ void thread::setup_tcb()

 else {

 _tcb->syscall_stack_top = 0;

 }

+_tcb->kernel_tcb_counter = 1; //By default disable fs base switch

 }

4. Handle fs switch if necessary on entry/exit of syscall/exception/page
fault handler:

This is just a code change around syscall entry but we have to do the
opposite for exit and similar for page fault/interrupt handler (possibly
signal handler as well)

@@ -174,6 +214,26 @@ syscall_entry:

 .cfi_register rip, rcx # rcx took previous rip value

 .cfi_register rflags, r11 # r11 took previous rflags value

 # There is no ring transition and rflags are left unchanged.

+#

+# app->kernel tcb switch

+movq %rax, %gs:8  # save register rax so we can restore it later

+movq %gs:0, %rax  # copy address of kernel tcb to the temp register rax

+#1. Check if kernel_tcb_counter 0 and jump over to 3 if not (no need
to do fsbase switch)

+cmpq $0, 40(%rax)

+jne on_kernel_tcb

+

+#2. If zero set fs MSR to kernel tcb

+movq %rbx, %gs:16  # save register rbx so we can restore it later

+movq (%rax), %rbx # set kernel tcb

+wrfsbase %rbx //TODO: In reality we need to check if wrfsbase is
available and use wrmsr if not

+movq %gs:16, %rbx

+

+on_kernel_tcb:

+#3. Increment counter (for nested case)

+incq 40(%rax)

+#4. Restore %rax

+movq %gs:8, %rax

+

 #

 # Unfortunately the mov instruction cannot be used to dereference an
address

 # on syscall stack pointed by address in TCB (%fs:16) - double memory
dereference.


I did measure that the context switch code is not affected in any way. But
I am sure the syscall/page fault/interrupt handler is affected but
hopefully by a tiny bit for all the cases except when the application
thread (of the static elf) gets interrupted, triggers page fault, or makes
a syscall call. In other words, I hope that kernel threads and normal
(non-

[osv-dev] OSv can run statically linked executable

[Waldek Kozaczuk]

Hi,

Over the recent week, I have been working to get OSv to run a simple "hello 
world" app (aka native-example) built as a position-dependent statically 
linked executable. In essence, I picked up where Pekka Enberg left over 8 
years ago (see https://github.com/cloudius-systems/osv/tree/static-elf). 
Obviously, given these days OSv has pretty robust support of over 70 
syscalls (and 60 more that should be trivial to add), and the remaining 
work is much more manageable.

./scripts/firecracker.py -e /hello
OSv v0.57.0-37-g0de155a4
Booted up in 5.23 ms
Cmdline: /hello 
 -> syscall: 107
 -> syscall: 102
 -> syscall: 108
 -> syscall: 104
 -> syscall: 158
 -> syscall: 012
 -> syscall: 012
 -> syscall: 158
 -> syscall: 218
 -> syscall: 273
 -> syscall: 063
 -> syscall: 302
 -> syscall: 089
 -> syscall: 318
 -> syscall: 228
 -> syscall: 228
 -> syscall: 012
 -> syscall: 012
 -> syscall: 010
 -> syscall: 262
 -> syscall: 016
 -> syscall: 001
Hello from C code
 -> syscall: 231

I will be sending a series of proper patches later after I clean some 
issues but in essence here is a list of things I had to do including what 
Pekka started:

   1. Tweak dynamic linker to support static executable:
  - Handle missing DT_SYMTAB, DT_STRTAB and DT_NEEDED.
  - Handle ET_EXEC
  - Support statically-linked executable base address
   2. Add basic handling of static ELF entry point and initial stack state 
   setup (see figure 3.9 ("Initial Process Stack") of the x86-64 ABI 
   specification - https://refspecs.linuxbase.org/elf/x86_64-abi-0.99.pdf
  - make sure the RDX registers in zeroed and the basic AUX vector with 
  AT_RANDOM is set up
  - more is left to do to fully support argv and full aux vector
   3. Add support of the brk() syscall (see issue 1138)
   4. Add dummy support of sys_set_robust_list and set_tid_address syscalls 
   (possibly needs something more for multithreaded apps).
   5. Support the arch_prctl syscall that sets the app TLS 
  - this was by far the most complicated element that required changing 
  OSv to store new per-pcpu data pointer in GS register and enhancing both 
  syscall handler and interrupt/page fault handler to detect and switch if 
  needed the FS base to the kernel TLS on entry and back to the app one on 
  exit 
  (see 
https://github.com/cloudius-systems/osv/issues/1137#issuecomment-1512315880) 
   6. Fixing a potential bug in handling TCGETS in the console driver.
   7. Implement sys_prlimit
   8. Enable the readlink, geteuid and getegid

This was enough to run a single-threaded app but we will need to implement 
the clone syscall to support multi-threaded apps. In addition, we would 
want to support the static pies as well which I hope should not be very 
difficult.

Regards,
Waldek 

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/3a561595-e072-4980-8375-3b742717dd6dn%40googlegroups.com.

[osv-dev] Re: Build failed in Jenkins: osv-build #1093

[Waldek Kozaczuk]

I have downloaded the same version of the latest JDK 17 (openjdk 17.0.6 
2023-01-17) and both Java unit tests pass just fine.

So I am at a loss for what might be happening on this build machine. Maybe 
still somehow the files from multiple JDKs get mixed up (would it be 
possible to see build/last/usr.manifest?).

It looks like the java-perms (1st test) possibly crashes for some reason 
and leaves qemu running and eventually gets killed by Jenkins. Maybe we 
should try to run ./test.py with -v (verbose) to see the output. I hate to 
push another commit just to see that.

Waldek
On Wednesday, April 12, 2023 at 8:03:16 PM UTC-4 jen...@scylladb.com wrote:

> See <
> https://jenkins.scylladb.com/job/osv-build/1093/display/redirect?page=changes
> >
>
> Changes:
>
> [jwkozaczuk] loader.py: use with statement when writing trace file
>
> [jwkozaczuk] prof.py: remove extra noise from profiler stack traces
>
> [jwkozaczuk] trace.py: inspect backtrace filename in a safe manner
>
> [jwkozaczuk] trace: add mechanism to resolve application symbols
>
>
> --
> [...truncated 147.45 KB...]
> Adding /usr/share/zoneinfo/Asia/Ust-Nera...
> Adding /usr/share/zoneinfo/Asia/Vientiane...
> Adding /usr/share/zoneinfo/Asia/Vladivostok...
> Adding /usr/share/zoneinfo/Asia/Yakutsk...
> Adding /usr/share/zoneinfo/Asia/Yangon...
> Adding /usr/share/zoneinfo/Asia/Manila...
> Adding /usr/share/zoneinfo/Asia/Srednekolymsk...
> Adding /usr/share/zoneinfo/Asia/Sakhalin...
> Adding /usr/share/zoneinfo/Asia/Samarkand...
> Adding /usr/share/zoneinfo/Asia/Shanghai...
> Adding /usr/share/zoneinfo/Asia/Thimbu...
> Adding /usr/share/zoneinfo/Asia/Thimphu...
> Adding /usr/share/zoneinfo/Asia/Tashkent...
> Adding /usr/share/zoneinfo/Asia/Tomsk...
> Adding /usr/share/zoneinfo/Asia/Ulaanbaatar...
> Adding /usr/share/zoneinfo/Asia/Ulan_Bator...
> Adding /usr/share/zoneinfo/Asia/Urumqi...
> Adding /usr/share/zoneinfo/Asia/Aden...
> Adding /usr/share/zoneinfo/Asia/Almaty...
> Adding /usr/share/zoneinfo/Asia/Amman...
> Adding /usr/share/zoneinfo/Asia/Anadyr...
> Adding /usr/share/zoneinfo/Asia/Aqtau...
> Adding /usr/share/zoneinfo/Asia/Aqtobe...
> Adding /usr/share/zoneinfo/Asia/Ashgabat...
> Adding /usr/share/zoneinfo/Asia/Ashkhabad...
> Adding /usr/share/zoneinfo/Asia/Atyrau...
> Adding /usr/share/zoneinfo/Asia/Baghdad...
> Adding /usr/share/zoneinfo/Asia/Bahrain...
> Adding /usr/share/zoneinfo/Asia/Baku...
> Adding /usr/share/zoneinfo/Asia/Bangkok...
> Adding /usr/share/zoneinfo/Asia/Barnaul...
> Adding /usr/share/zoneinfo/Asia/Beirut...
> Adding /usr/share/zoneinfo/Asia/Bishkek...
> Adding /usr/share/zoneinfo/Asia/Brunei...
> Adding /usr/share/zoneinfo/Asia/Calcutta...
> Adding /usr/share/zoneinfo/Asia/Chita...
> Adding /usr/share/zoneinfo/Asia/Colombo...
> Adding /usr/share/zoneinfo/Asia/Dushanbe...
> Adding /usr/share/zoneinfo/Asia/Katmandu...
> Adding /usr/share/zoneinfo/Asia/Krasnoyarsk...
> Adding /usr/share/zoneinfo/Asia/Macao...
> Adding /usr/share/zoneinfo/Asia/Magadan...
> Adding /usr/share/zoneinfo/Asia/Novokuznetsk...
> Adding /usr/lib/libboost_atomic.so.1.78.0...
> Adding /usr/lib/libboost_filesystem.so.1.78.0...
> Adding /usr/lib/libboost_program_options.so.1.78.0...
> Adding /usr/lib/libboost_unit_test_framework.so.1.78.0...
> Adding /tests/tst-pthread.so...
> Adding /tests/misc-ramdisk.so...
> Adding /tests/tst-vblk.so...
> Adding /tests/tst-bsd-evh.so...
> Adding /tests/misc-bsd-callout.so...
> Adding /tests/tst-bsd-kthread.so...
> Adding /tests/tst-bsd-taskqueue.so...
> Adding /tests/tst-fpu.so...
> Adding /tests/tst-preempt.so...
> Adding /tests/tst-tracepoint.so...
> Adding /tests/tst-hub.so...
> Adding /tests/misc-console.so...
> Adding /tests/misc-leak.so...
> Adding /tests/misc-readbench.so...
> Adding /tests/misc-mmap-anon-perf.so...
> Adding /tests/tst-mmap-file.so...
> Adding /tests/misc-mmap-big-file.so...
> Adding /tests/tst-mmap.so...
> Adding /tests/tst-huge.so...
> Adding /tests/tst-elf-permissions.so...
> Adding /tests/misc-mutex.so...
> Adding /tests/misc-sockets.so...
> Adding /tests/tst-condvar.so...
> Adding /tests/tst-queue-mpsc.so...
> Adding /tests/tst-af-local.so...
> Adding /tests/tst-pipe.so...
> Adding /tests/tst-yield.so...
> Adding /tests/misc-ctxsw.so...
> Adding /tests/tst-read.so...
> Adding /tests/tst-symlink.so...
> Adding /tests/tst-openat.so...
> Adding /tests/tst-eventfd.so...
> Adding /tests/tst-remove.so...
> Adding /tests/misc-wake.so...
> Adding /tests/tst-epoll.so...
> Adding /tests/misc-lfring.so...
> Adding /tests/misc-fsx.so...
> Adding /tests/tst-sleep.so...
> Adding /tests/tst-resolve.so...
> Adding /tests/tst-except.so...
> Adding /tests/misc-tcp-sendonly.so...
> Adding /tests/tst-tcp-nbwrite.so...
> Adding /tests/misc-tcp-hash-srv.so...
> Adding /tests/misc-loadbalance.so...
> Adding /tests/misc-scheduler.so...
> Adding /tests/tst-console.so...
> Adding /tests/tst-app.so...
> Adding /tests/misc-setpriority.so...
> Adding /tests/misc-timeslice.so...

[osv-dev] Unikernel Group: Kick-off meeting from Razvan Deaconescu

[Waldek Kozaczuk]

Hi,

I am forwarding an email received from Razvan about upcoming virtual 
meeting about Unikernels:

"As discussed, we want to organize a group of people / parties interested
in unikernels and unikernel-related topics. The goal would be to capture
most (all) unikernel / libOS into periodic meetings and to have common
resources that we can use to advertise unikernel technologies.

Our first meeting would be one to get to know each other and agree on
next steps, communication channels and resources to use.

More concretely, the proposed agenda is:

- short presentation of participants: people & projects / communities
- recurrence of meetings
- community name (Unikommunity, Unikernel Consortium, Unikernel Task
  Force?)
- resources (communication channels, website, social media)
- governance of group
- Aachen face-to-face meeting, July 7, 2023

The meeting will take place online. We would use Google Meet, if
everyone is OK with that.

Please fill out this Doodle form[1] so we can arrange a suitable
time. We'll keep a meeting summary so you can be up-to-date with the
discussions, even if you can't make it.

Also, feel free to distribute the link and ask other people who you see
relevant for this group. It's supposed to be an open room for open
discussions regarding unikernels / libOSes.

[1] https://doodle.com/meeting/participate/id/aKLAOKMb;

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/d42243b8-e307-4459-a73f-d5f6ab5bcce8n%40googlegroups.com.

[osv-dev] Syscalls

[Waldek Kozaczuk]

Hi,

I have just documented how OSv handles systems calls, which ones are 
implemented and which ones should be trivial to implement (or expose) in 
this new wiki page - https://github.com/cloudius-systems/osv/wiki/Syscalls.

As of this moment there are 73 implemented systems calls and 60 ones that 
should be trivial to add (there are libc functions that we hope would map 
one-2-one). 

Regards,
Waldek

PS. I wonder if we would need to optimize the syscall() code in linux.cc by 
implementing some sort of call table instead of using a switch statement as 
it is now once we have ~ 130 system calls.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/52ee0a15-f236-415c-813d-1a4e777e3a6an%40googlegroups.com.

Re: [osv-dev] Patches for running cyclictest on OSv

[Waldek Kozaczuk]

Hi,

Thanks for the patches.

Could you please follow
https://github.com/cloudius-systems/osv/wiki/Formatting-and-sending-patches
and submit them using `git send-email` so that they are easier to review? I
think you can skip the 0001-* - we already have a github pull request -
https://github.com/cloudius-systems/osv/pull/1223 - where I left some of my
comments. Alternatively, you can create pull requests for 0002-* and 0003-*.

Thanks,
Waldemar Kozaczuk

On Mon, Apr 10, 2023 at 6:17 AM Luca Abeni  wrote:

> Hi all,
>
> I attach the patches I am using to run a simple real-time test
> (cyclictest) on OSv.
> The patches are based on github issue 386
> (https://github.com/cloudius-systems/osv/issues/386) and are a forward
> port of patches by Nadav Har'El and Claudio Fontana; I only applied a
> few changes to them.
>
> With these patches, cyclictest can start and measure some latencies...
> In the next few days, I'll run some more accurate tests, and I'll
> report on the latencies introduced by OSv.
>
>
> Luca
>
> P.S.: please cc me if you reply because I am not subscribed to the mailing
> list
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/CAJfkoWpFDkukjNb%3DR2kxugjo2YBqeMgpoCuM26QiVdwaB6k5pw%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfMV9HcfqnhavQfkuyM6kFNuZq19H%2B3OsVug6qzV%2BQZ5Xg%40mail.gmail.com.

[osv-dev] Github Discussions and Roadmap Priorities

[Waldek Kozaczuk]

Hi,

I have started using the GitHub discussions - 
https://github.com/cloudius-systems/osv/discussions. It seems like it has 
some interesting tools to help us discuss, collaborate and seek feedback. I 
am not suggesting we stop using Google emailing list but hope the Github 
discussion to supplement it.

For example, I have created a new poll 
- https://github.com/cloudius-systems/osv/discussions/1226 - to seek 
feedback on what things to work on next. The polls have a limitation of 8 
items so I could put only that many items from the roadmap. But if you 
would like us to work on something else please put it in the comments 
(ideally reference to an existing issue).

Regards,
Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/17309d79-420a-4021-bd89-bc8acb2ded89n%40googlegroups.com.

[osv-dev] ChatGPT and OSv

[Waldek Kozaczuk]

Here is an excerpt of my funny conversation with ChatGTP 3 (free version) 
about OSv. Please not how good it is about making things up which sound 
almost true except they are not:-) The best one is about virtio-fs.

Regards,
Waldek

"Does OSv implement SYSCALL instruction?

No, OSv does not implement the SYSCALL instruction. OSv is designed to run 
unikernels, which are single-application operating systems that are 
compiled with only the necessary components to run a specific application. 
Since unikernels are designed to be lightweight and efficient, they 
typically do not include a full system call interface like a traditional 
operating system. Instead, they rely on a narrow interface provided by the 
hypervisor or host operating system for accessing hardware resources and 
other system services.
[image: Waldek Kozaczuk]
Does OSv implement SYSCALL instruction handler?

As far as I know, OSv does not implement a SYSCALL instruction handler 
since it does not provide a full system call interface like a traditional 
operating system. Instead, OSv relies on a narrow interface provided by the 
hypervisor or host operating system for accessing hardware resources and 
other system services. If you need more information, I recommend looking 
into the OSv source code and documentation to see how it implements system 
calls and interacts with the underlying host operating system or hypervisor.
[image: Waldek Kozaczuk]
Unfortunately you are incorrect. I does.

I apologize for the mistake in my previous response. You are correct that 
OSv does implement a SYSCALL instruction handler. The SYSCALL instruction 
is used to transfer control to the operating system kernel to perform 
system calls. In OSv, the SYSCALL instruction is handled by the kernel to 
provide system call functionality to user space applications.
[image: Waldek Kozaczuk]
How is user space application separated from kernel in OSv?

In OSv, there is no traditional concept of a kernel and user space. 
Instead, the OS is designed as a single unified system, where each 
application runs in its own isolated sandbox environment. This sandboxing 
is achieved using a combination of hardware virtualization, memory 
protection mechanisms, and software isolation techniques.

Each application is compiled into a self-contained bootable image, called a 
"unikernel", that includes all the necessary libraries, dependencies, and 
system services required to run the application. This unikernel is then 
loaded directly into a virtual machine (VM) or container, and executed as a 
standalone OS instance.

Thus, in OSv, the user space and kernel are both contained within the 
unikernel, and are not separated as in traditional OS architectures. This 
approach allows for a highly efficient and lightweight system design, with 
minimal overhead and improved performance.
[image: Waldek Kozaczuk]
Does OSv implement the SVC instruction handler?

No, OSv does not use the SVC (Supervisor Call) instruction as it is not 
based on a traditional operating system architecture with a kernel and user 
space. Instead, OSv uses a unikernel architecture where the application and 
the operating system are compiled and linked into a single binary image, 
and there is no strict separation between user space and kernel. System 
calls are typically implemented as function calls within the same address 
space as the application.
[image: Waldek Kozaczuk]
Does OSv implement virtio-fs driver?

Yes, OSv has implemented the virtio-fs driver. Virtio-fs is a shared file 
system for virtual machines, which allows a virtual machine to access a 
directory on the host file system and enables efficient data transfer 
between the guest and the host. OSv supports the virtio-fs driver to 
provide a fast and efficient file system for virtualized applications.
[image: Waldek Kozaczuk]
How does OSv implement the virtio-fs driver?

OSv implements the virtio-fs driver as a library that runs in user-space. 
The library communicates with the kernel driver (which is usually provided 
by the host operating system) using a standard file system interface, such 
as FUSE.

The virtio-fs driver library in OSv takes care of communicating with the 
virtio-fs device in the guest operating system, and provides a 
POSIX-compliant file system interface to user-space applications running on 
OSv. This allows OSv applications to access files on the host system using 
a fast, shared memory-based communication mechanism, rather than slower 
network-based protocols like NFS or SMB.

When a user-space application in OSv makes a system call to read or write a 
file, the virtio-fs library intercepts the call and translates it into the 
appropriate virtio-fs protocol messages to communicate with the virtio-fs 
device in the guest operating system. The device driver in the guest 
operating system then translates these messages into file system operations 
on the host system, and returns the results to the virtio-fs library

[osv-dev] Re: Build failed in Jenkins: osv-build-nightly #2967

[Waldek Kozaczuk]

It looks like it always fails like this:

"Appending 

 
to usr.manifest
Appending 

 
to usr.manifest
Appending 

 
to usr.manifest
Appending 

 
to usr.manifest
Appending 

 
to usr.manifest
Preparing bootfs.manifest
Saving command line to 

No apps selected
make[1]: Entering directory 
'
Building into build/release.x64
  GEN gen/include/osv/version.h
  GEN gen/include/osv/drivers_config.h
make[1]: Leaving directory 
'
qemu-system-x86_64: -gdb tcp::1234,server,nowait: Failed to find an 
available port: Address already in use
qemu failed.
Traceback (most recent call last):
  File 
"
 
line 179, in 
main()
  File 
"
 
line 169, in main
upload(osv, manifest, depends, upload_port)
  File 
"
 
line 31, in upload
s.connect(("127.0.0.1", port))
ConnectionRefusedError: [Errno 111] Connection refused
+ cd 
+ exec ./scripts/test.py
  TEST java-perms qemu-system-x86_64: -gdb 
tcp::1234,server,nowait: Failed to find an available port: Address already 
in use
qemu failed.
Test java-perms FAILED
Traceback (most recent call last):
  File 
" 
line 191, in 
main()
  File 
" 
line 147, in main
run_tests()
  File 
" 
line 136, in run_tests
run(tests_to_run)
  File 
" 
line 109, in run
run_test(test)
  File 
" 
line 84, in run_test
test.run()
  File 
"
 
line 38, in run
run_command_in_guest(self.command, hypervisor=self.hypervisor, 
run_py_args=self.run_py_args).join()
  File 
"
 
line 184, in join
raise Exception('Guest failed (returncode=%d)' % 
self.process.returncode)
Exception: Guest failed (returncode=1)
make: *** [Makefile:162: check] Error 1
Build step 'Execute shell' marked build as failure"

So it fails to build the ZFS test image it seems and then fails to run the 
1st test java-perms because the gdb port 1234 is taken? Why would it be?

Could it be possible there is already another qemu running on the same 
machine because there is another build happening at the same time? Or maybe 
some old qemu process still lingering from the old failed run?

I can build and run all tests on my Fedora 37.
 
On Saturday, February 18, 2023 at 3:00:32 PM UTC-5 jen...@scylladb.com 
wrote:

> See <
> https://jenkins.scylladb.com/job/osv-build-nightly/2967/display/redirect>
>
> Changes:
>
>
> --
> Started by timer
> Running as SYSTEM
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> [EnvInject] - Loading node environment variables.
> [EnvInject] - Preparing an environment for the build.
> [EnvInject] - Keeping Jenkins system variables.
> [EnvInject] - Keeping Jenkins build variables.
> [EnvInject] - Injecting as environment variables the properties content 
> PATH=/usr/local/bin:/usr/bin:/usr/sbin
> http_proxy=10.0.0.4:3128
>
> [EnvInject] - Variables injected successfully.
> [EnvInject] - Injecting contributions.
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> Building remotely on muninn (pack-no-dg packager-amd64 artifact-testers 
> packager pack-no-sif) in workspace <
> https://jenkins.scylladb.com/job/osv-build-nightly/ws/>
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> The recommended git tool is: NONE
> No credentials specified
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> > git 

Re: [osv-dev] OSv talk

[Waldek Kozaczuk]

The video of the talk has been just released and you can watch it here 
- https://fosdem.org/2023/schedule/event/osvevolution/ (the play button 
should play it). There are also slides attached on that page below. 

If you want to see the version of the slides with presenter notes, here you 
go -  
https://docs.google.com/presentation/d/1xcKLI-txmS0_T9QUgo6msRaOGzreU22x-Gzn9beh46I/edit?usp=sharing.
 
At the end of the talk I ran out of time so that way you can read 
everything I was supposed to say there. 

I hope you will enjoy my talk if you have not watched it live.

Nadav,
You are right! We might want to jump to a version like 1.0 - maybe it is 
finally time for it anyway.

Waldek

On Sunday, February 5, 2023 at 8:24:30 AM UTC-5 n...@scylladb.com wrote:
On Sun, Feb 5, 2023 at 10:23 AM Nadav Har'El  wrote:

On Sun, Feb 5, 2023 at 9:56 AM Waldek Kozaczuk  wrote:
If you are going to happen to have some time today from 12AM - 1PM Brussels 
time, please watch my talk here at 
https://fosdem.org/2023/schedule/event/osvevolution/. It should be 
streamable live.

Regards,
Waldemar Kozaczuk

Good luck!

I'll try to watch live, I'm sure it will be interesting.

Indeed it was interesting!
Version "0.57"  was for sure a bigger accomplishment than the version 
number suggests.
And thanks for mentioning me :-)

I hope they will have the recording available somewhere later, in case 
someone missed the live feed.

Nadav.

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/4ec729c1-9a4d-40dd-a27b-a2132ecaf31cn%40googlegroups.com.

[osv-dev] OSv talk

[Waldek Kozaczuk]

If you are going to happen to have some time today from 12AM - 1PM Brussels 
time, please watch my talk here 
at https://fosdem.org/2023/schedule/event/osvevolution/. It should be 
streamable live.

Regards,
Waldemar Kozaczuk

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/794a3706-33c9-40b7-8898-8c7d3a2bec11n%40googlegroups.com.

[osv-dev] Re: [PATCH v2] test/tst-feexcept.cc: Fix test for integer division by zero

[Waldek Kozaczuk]

Thanks, Nadav!

Indeed I can confirm that all tests also work on Fedora 37. I have just
committed your patches.

Regards,
Waldek

On Wed, Jan 11, 2023 at 6:58 AM Nadav Har'El  wrote:

> Waldek, after my two patches from today, all the tests pass on my Fedora
> 36 build machine :-)
>
> I can also try Fedora 37 later.
>
> --
> Nadav Har'El
> n...@scylladb.com
>
>
> On Wed, Jan 11, 2023 at 1:55 PM Nadav Har'El  wrote:
>
>> tst-feexcept.cc checks several aspects of SIGFPE. One of the things
>> we noticed in the past, and tested, is that division by *integer* zero
>> also generates SIGFPE. We had a test for this using C code, but
>> unfortunately C and C++ chose to make division by zero "undefined",
>> and starting with Gcc 12, 1/0 started to return 0 instead of SIGFPE
>> which broke this test.
>>
>> Changed the test to do division by zero in assembly-language, which
>> still causes a SIGFPE.
>>
>> Fixes #1213
>>
>> Signed-off-by: Nadav Har'El 
>> ---
>>  tests/tst-feexcept.cc | 8 ++--
>>  1 file changed, 6 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/tst-feexcept.cc b/tests/tst-feexcept.cc
>> index 5d5ddc1f..403185b6 100644
>> --- a/tests/tst-feexcept.cc
>> +++ b/tests/tst-feexcept.cc
>> @@ -78,8 +78,12 @@ int main(int argc, char **argv)
>>  // Test that fegetexcept() does not return a negative number
>>  expect(fegetexcept() >= 0, true);
>>
>> -// Test that *integer* division by zero generates, ironically, a
>> SIGFPE
>> -expect(sig_check([] { printf("%d\n", 1 / zero_i()); }, SIGFPE),
>> true);
>> +// Test that *integer* division by zero generates, ironically, a
>> SIGFPE.
>> +// Starting with Gcc 12, division by zero in C no longer generates a
>> +// SIGFPE and just returns 0. So to check this we need to use
>> assembly
>> +// directly.
>> +//expect(sig_check([] { printf("%d\n", 1 / zero_i()); }, SIGFPE),
>> true);
>> +expect(sig_check([] { asm("movl $1,%eax; movl $0,%ebx; movl $0,
>> %edx; divl %ebx;"); }, SIGFPE), true);
>>
>>  // While, continuing the irony, by default a floating-point division
>> by
>>  // zero does NOT generate a SIGFPE signal, but rather inf or nan:
>> --
>> 2.39.0
>>
>>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfMJUMSKxjPE8vs5QHoDGnoK1YZ-zPoMGNrTi-Tv9T6ksQ%40mail.gmail.com.

[osv-dev] Re: Build failed in Jenkins: osv-build-nightly #2933

[Waldek Kozaczuk]

The failing java 17 test on fedora 37 seems to be affected by a bug in 
memmove, probably specific to gcc 12. I have described the issue here 
- https://github.com/cloudius-systems/osv/issues/1212.

On Saturday, December 31, 2022 at 3:00:27 PM UTC-5 jen...@scylladb.com 
wrote:

> See <
> https://jenkins.scylladb.com/job/osv-build-nightly/2933/display/redirect>
>
> Changes:
>
>
> --
> Started by timer
> Running as SYSTEM
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> [EnvInject] - Loading node environment variables.
> [EnvInject] - Preparing an environment for the build.
> [EnvInject] - Keeping Jenkins system variables.
> [EnvInject] - Keeping Jenkins build variables.
> [EnvInject] - Injecting as environment variables the properties content 
> PATH=/usr/local/bin:/usr/bin:/usr/sbin
> http_proxy=10.0.0.4:3128
>
> [EnvInject] - Variables injected successfully.
> [EnvInject] - Injecting contributions.
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> Building remotely on muninn (pack-no-dg packager-amd64 artifact-testers 
> packager pack-no-sif) in workspace <
> https://jenkins.scylladb.com/job/osv-build-nightly/ws/>
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> The recommended git tool is: NONE
> No credentials specified
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> > git rev-parse --resolve-git-dir <
> https://jenkins.scylladb.com/job/osv-build-nightly/ws/.git> # timeout=10
> Fetching changes from the remote Git repository
> > git config remote.origin.url https://github.com/cloudius-systems/osv # 
> timeout=10
> Fetching upstream changes from https://github.com/cloudius-systems/osv
> > git --version # timeout=10
> > git --version # 'git version 2.38.1'
> > git fetch --tags --force --progress -- 
> https://github.com/cloudius-systems/osv 
> +refs/heads/*:refs/remotes/origin/* # timeout=60
> > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> Checking out Revision 07a8b81e9496690957fb471462aba8840341925e 
> (refs/remotes/origin/master)
> > git config core.sparsecheckout # timeout=10
> > git checkout -f 07a8b81e9496690957fb471462aba8840341925e # timeout=10
> Commit message: "readme: update with more up to date information"
> > git rev-list --no-walk 07a8b81e9496690957fb471462aba8840341925e # 
> timeout=10
> > git remote # timeout=10
> > git submodule init # timeout=10
> > git submodule sync # timeout=10
> > git config --get remote.origin.url # timeout=10
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> > git submodule init # timeout=10
> > git config -f .gitmodules --get-regexp ^submodule\.(.+)\.url # timeout=10
> > git config --get submodule.external/x64/acpica.url # timeout=10
> > git config -f .gitmodules --get submodule.external/x64/acpica.path # 
> timeout=10
> > git config --get submodule.apps.url # timeout=10
> > git config -f .gitmodules --get submodule.apps.path # timeout=10
> > git config --get submodule.modules/httpserver/swagger-ui.url # timeout=10
> > git config -f .gitmodules --get 
> submodule.modules/httpserver/swagger-ui.path # timeout=10
> > git config --get submodule.modules/httpserver/osv-gui.url # timeout=10
> > git config -f .gitmodules --get 
> submodule.modules/httpserver/osv-gui.path # timeout=10
> > git config --get submodule.musl_0.9.12.url # timeout=10
> > git config -f .gitmodules --get submodule.musl_0.9.12.path # timeout=10
> > git config --get submodule.musl_1.1.24.url # timeout=10
> > git config -f .gitmodules --get submodule.musl_1.1.24.path # timeout=10
> > git submodule update --init --recursive external/x64/acpica # timeout=10
> > git submodule update --init --recursive apps # timeout=10
> > git submodule update --init --recursive 
> modules/httpserver-html5-gui/swagger-ui # timeout=10
> > git submodule update --init --recursive 
> modules/httpserver-html5-gui/osv-gui # timeout=10
> > git submodule update --init --recursive musl_0.9.12 # timeout=10
> > git submodule update --init --recursive musl_1.1.24 # timeout=10
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> [EnvInject] - Executing scripts and injecting environment variables after 
> the SCM step.
> ERROR: Not all environment variables could be successfully injected. Check 
> for similarly-named environment variables.
> [osv-build-nightly] $ /bin/sh -xe /tmp/jenkins9780950073216863458.sh
> + git submodule update --init
> ERROR: Not all environment variables could be successfully injected. 

[osv-dev] Evolution of OSv: Towards Greater Modularity and Composability

[Waldek Kozaczuk]

Hi,

If you happen to be in Brussels at the beginning of February, I will be 
giving a talk at FOSDEM 2023 about recent improvements to OSv. Feel free to 
stop by to listen or watch it after:

https://fosdem.org/2023/schedule/event/osvevolution/

Regards,
Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/3158f9c9-8f47-4ee6-bf83-3162cf850d3cn%40googlegroups.com.

[osv-dev] OSv 0.57.0 (nickname Magpie) just been recently released

[Waldek Kozaczuk]

Hi,

I have just published another release of OSv - 
https://github.com/cloudius-systems/osv/releases/tag/v0.57.0. 

I am also enclosing the release notes down here:

"Highlights

This new release brings many new and exciting features and improvements 
including:

   - New build mode to *hide* the non-glibc symbols and libstdc++
   - ZFS code extracted out of the kernel in form of a dynamically linked 
   library libsolaris.so
   - Build mode intended to tailor the kernel to a set of specific device 
   drivers - *driver profiles*
   - Mechanism to build a version of the kernel with a subset of glibc 
   symbols needed to support a specific application
   - Experimental *lazy stack* support
   - Major improvements to building and running ZFS images
   - Major improvements to the *aarch64* port like dedicated exception 
   stack and support to handle system calls and signals
   - Support new system calls
   - Support the SO_REUSE_PORT socket option
   - Minimal rnetlink support
   - Various VFS enhancements

Enhancements and Improvements in Detail
   
   - Hide most symbols and standard C++ library
  - hide most symbols using -fvisibility=hidden and expose the public 
  ones using (__visibility__("default")) - 2df8652 
  

  , 317d259 
  

  - add lists of exported symbols 
  

  - add option to build kernel with most symbols hidden - af2d371 
  

  - enable garbage collection of code and data when hiding symbols - 
  730fd82 
  

   - Improve modularity
  - extract ZFS code as external dynamic library libsolaris.so - d7a6274 
  

  - httpserver-monitoring-api: stop using kernel internal C++ API - 
  891bfcd 
  

  - add new C-wrappers to expose the module API - 182499c 
  

   - Driver Profiles
  - support driver profiles - ffb91b1 
  

   - App-specific kernel
  - support app version script - d19ccb1 
  

   - Enhance networking stack
  - socket: implement the Linux flavor of SO_REUSE_PORT option - 985d4d7 
  

  - routecache: handle the case when sent to non-loopback address - 
  a0251df 
  

   - Aarch64 enhancements
  - move kernel to 63rd GB of virtual memory - 850ce0e 
  

  - handle system calls - 519 
  

  - handle exceptions on dedicated stack - a6fbc36 
  

  - force TLB flush when mprotect changes permission - fd24359 
  

  - implement signal handler - d8d2719 
  

  - fix ZFS support - ea9cb44 
  

   - Improve building and running ZFS images 
   
  - new zfs_builder to create ZFS images - f935ca1 
  

  - vfs: support mounting ZFS from non-root partition - 9cc49b3 
  

  - support building rofs+zfs image and second ZFS disk - f6e09c0 
  

  - allow mounting and building on host - 6aa1053 
  

   - New syscalls
  - add getgid, getuid, lseek and statfs - b9c3210 
  

Re: [osv-dev] Build failed in Jenkins: osv-build #1085

[Waldek Kozaczuk]

I wonder if that one failed because previous failures left corrupt
usr.manifest under modules/openjdk9_1x-from-host and it needs to be
manually deleted.

I think I saw a similar issue while working on the libsunec.so problem.

On Thu, Dec 29, 2022 at 7:37 PM jenkins via OSv Development <
osv-dev@googlegroups.com> wrote:

> See <
> https://jenkins.scylladb.com/job/osv-build/1085/display/redirect?page=changes
> >
>
> Changes:
>
> [Nadav Har'El] modules/httpserver-api: add missing include
>
> [jwkozaczuk] OpenJDK 9 and above: do not add libsunec.so if does not exist
>
>
> --
> [...truncated 132.13 KB...]
>   LD tests/tst-fstatat.so
>   CXX tests/misc-reboot.cc
>   LD tests/misc-reboot.so
>   CXX tests/tst-fcntl.cc
>   LD tests/tst-fcntl.so
>   CXX tests/payload-namespace.cc
>   LD tests/payload-namespace.so
>   CXX tests/tst-namespace.cc
>   LD tests/tst-namespace.so
>   CXX tests/tst-without-namespace.cc
>   LD tests/tst-without-namespace.so
>   CXX tests/payload-env.cc
>   LD tests/payload-env.so
>   CXX tests/payload-merge-env.cc
>   LD tests/payload-merge-env.so
>   CC tests/misc-execve.c
>   LD tests/misc-execve.so
>   CC tests/misc-execve-payload.c
>   LD tests/misc-execve-payload.so
>   CXX tests/misc-mutex2.cc
>   LD tests/misc-mutex2.so
>   CXX tests/tst-pthread-setcancelstate.cc
>   LD tests/tst-pthread-setcancelstate.so
>   CXX tests/tst-pin.cc
>   LD tests/tst-pin.so
>   CXX tests/tst-run.cc
>   LD tests/tst-run.so
>   CXX tests/tst-pthread-affinity-inherit.cc
>   LD tests/tst-pthread-affinity-inherit.so
>   CXX tests/tst-sem-timed-wait.cc
>   LD tests/tst-sem-timed-wait.so
>   CC tests/tst-ttyname.c
>   LD tests/tst-ttyname.so
>   CXX tests/tst-pthread-barrier.cc
>   LD tests/tst-pthread-barrier.so
>   CXX tests/tst-feexcept.cc
>   LD tests/tst-feexcept.so
>   CXX tests/tst-math.cc
>   LD tests/tst-math.so
>   CXX tests/tst-sigaltstack.cc
> :
> In function ‘int endless_recursive()’:
> :64:5:
> warning: infinite recursion detected [-Winfinite-recursion]
>64 | int endless_recursive() {
>   | ^
> :65:29:
> note: recursive call
>65 | return endless_recursive() + endless_recursive();
>   |~^~
>   LD tests/tst-sigaltstack.so
>   CXX tests/tst-fread.cc
>   LD tests/tst-fread.so
>   CXX tests/tst-tcp-cork.cc
>   LD tests/tst-tcp-cork.so
>   CXX tests/tst-tcp-v6.cc
>   LD tests/tst-tcp-v6.so
>   CXX tests/tst-calloc.cc
>   LD tests/tst-calloc.so
>   CC tests/tst-crypt.c
>   LD tests/tst-crypt.so
>   CXX tests/tst-non-fpic.cc
>   LD tests/tst-non-fpic.so
> /usr/bin/ld: <
> https://jenkins.scylladb.com/job/osv-build/ws/build/release.x64/tests/tst-non-fpic.o>:
> warning: relocation in read-only section `.text.startup'
> /usr/bin/ld: warning: creating DT_TEXTREL in a shared object
>   CXX tests/tst-small-malloc.cc
>   LD tests/tst-small-malloc.so
>   CXX tests/tst-getopt.cc
>   LD tests/tst-getopt.so
>   CXX tests/tst-getopt-pie.cc
>   LD tests/tst-getopt-pie.so
>   LD tests/tst-non-pie.so
>   CC tests/tst-semaphore.c
>   LD tests/tst-semaphore.so
>   CXX tests/tst-elf-init.cc
>   LD tests/tst-elf-init.so
>   CXX tests/tst-realloc.cc
>   LD tests/tst-realloc.so
>   CXX tests/tst-setjmp.cc
>   LD tests/tst-setjmp.so
>   CXX tests/libtls.cc
>   LD tests/libtls.so
>   LD tests/libtls_gold.so
>   CXX tests/tst-tls.cc
>   CXX tests/tst-tls.cc
>   CXX tests/tst-tls-pie.cc
>   LD tests/tst-tls-pie.so
>   CXX tests/tst-sigaction.cc
>   LD tests/tst-sigaction.so
>   CXX tests/tst-syscall.cc
>   LD tests/tst-syscall.so
>   CXX tests/tst-ifaddrs.cc
>   LD tests/tst-ifaddrs.so
>   CXX tests/tst-getdents.cc
>   LD tests/tst-getdents.so
>   CC tests/tst-netlink.c
>   LD tests/tst-netlink.so
>   CXX tests/misc-zfs-io.cc
>   LD tests/misc-zfs-io.so
>   CXX tests/misc-zfs-arc.cc
>   LD tests/misc-zfs-arc.so
>   CXX tests/tst-pthread-create.cc
>   LD tests/tst-pthread-create.so
>   CXX tests/misc-futex-perf.cc
>   LD tests/misc-futex-perf.so
>   CXX tests/tst-mmx-fpu.cc
>   LD tests/tst-mmx-fpu.so
>   CXX tests/testrunner.cc
>   LD tests/testrunner.so
>   CXX tests/tst-vfs.cc
>   LD tests/tst-vfs.so
>   CXX tests/tst-libc-locking.cc
>   LD tests/tst-libc-locking.so
>   CXX tests/misc-fs-stress.cc
>   LD tests/misc-fs-stress.so
>   CXX tests/misc-bdev-write.cc
>   LD tests/misc-bdev-write.so
>   CXX tests/misc-bdev-wlatency.cc
>   LD tests/misc-bdev-wlatency.so
>   CXX tests/misc-bdev-rw.cc
>   LD tests/misc-bdev-rw.so
>   CXX tests/tst-promise.cc
>   LD tests/tst-promise.so
>   CXX tests/tst-dlfcn.cc
>   LD tests/tst-dlfcn.so
>   CXX tests/tst-stat.cc
>   LD tests/tst-stat.so
>   CXX tests/tst-wait-for.cc
>   LD tests/tst-wait-for.so
>   CXX tests/tst-bsd-tcp1.cc
>   LD tests/tst-bsd-tcp1.so
>   CXX 

Re: [osv-dev] [FOSDEM'23] Submit talk for the "Microkernel and Component-based OS" devroom

[Waldek Kozaczuk]

Hi,

Thanks for the invitation!

I think we have enough new things coming up on the OSv side to have an
interesting presentation. I will try to provide a submission.

Is the deadline end of today (Dec 2nd) according to which time zone?

Regards,
Waldek

On Thu, Dec 1, 2022 at 12:15 PM 'Razvan Deaconescu' via OSv Development <
osv-dev@googlegroups.com> wrote:

> Hi.
>
> I am Razvan Deaconescu, I'm from University POLITEHNICA of Bucharest
> (working on Unikraft unikernel) and I'm managing the "Microkernel and
> Component-based OS" devroom[2] at FOSDEM'2023[1].
>
> The devroom will take place in person, at FOSDEM'23, in Brussels, on
> Sunday, February 5, 2023.
>
> If you have an interesting topic to talk about it would be great if you
> made a submission. This is something easy to do, takes about 15-20
> minutes on Pentabarf[3].
>
> The deadline for submission is tomorrow, Friday, December 2, 2022.
>
> [1] https://fosdem.org/2023/
> [2]
> https://fosdem.org/2023/schedule/track/microkernel_and_component_based_os/
> [3] https://penta.fosdem.org/submission/
>
> Razvan
>
> --
> You received this message because you are subscribed to the Google Groups
> "OSv Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/86sfhzyris.fsf%40drone.cs.pub.ro
> .
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfMgLPXNjNr8T4AfC9qbyYS7c1hYmhHdhT6tXwCw6LrqZg%40mail.gmail.com.

[osv-dev] Re: Please add me to the mailing list

[Waldek Kozaczuk]

Hi,

I think you can add yourself, no? Otherwise, can someone that has admin 
permissions do that?

Regards,
Waldek

On Tuesday, November 15, 2022 at 8:10:40 AM UTC-5 Kyle Jackson wrote:

> Thanks
>
> R,
>
> Kyle Jackson
> (757) 362-7328
> DataSource Inc
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/f3c7e40c-2c09-417f-9411-fde6339a070bn%40googlegroups.com.

[osv-dev] Coverity scan report

[Waldek Kozaczuk]

I have just submitted a coverity scan of the latest OSv scan and here are 
some of the findings with the link to view full report:

Cheers,
Waldek

PS. Looking to automate the submission through Travis (by following this 
guide - https://scan.coverity.com/travis_ci). But somehow the "Configure 
Travis CI" button does no show up). Any ideas would help.

"Please find the latest report on new defect(s) introduced to OSv found 
with Coverity Scan.

300 new defect(s) introduced to OSv found with Coverity Scan.
213 defect(s), reported by Coverity Scan earlier, were marked fixed in the 
recent build analyzed by Coverity Scan.

New defect(s) Reported-by: Coverity Scan
Showing 20 of 300 defect(s)


** CID 1499387:  Memory - illegal accesses  (UNINIT)
/fs/vfs/vfs_lookup.cc: 351 in lookup()



*** CID 1499387:  Memory - illegal accesses  (UNINIT)
/fs/vfs/vfs_lookup.cc: 351 in lookup()
345 /*
346  * Get the vnode for directory
347  */
348 if ((error = namei(dir, )) != 0) {
349 return error;
350 }
>>> CID 1499387:  Memory - illegal accesses  (UNINIT)
>>> Using uninitialized value "dp".
351 if (dp->d_vnode->v_type != VDIR) {
352 drele(dp);
353 return ENOTDIR;
354 }
355 
356 *dpp = dp;

** CID 1499386:  Memory - illegal accesses  (UNINIT)
/fs/vfs/vfs_syscalls.cc: 907 in sys_link()



*** CID 1499386:  Memory - illegal accesses  (UNINIT)
/fs/vfs/vfs_syscalls.cc: 907 in sys_link()
901 oldpath, newpath));
902 
903 /* File from oldpath must exist */
904 if ((error = namei(oldpath, )) != 0)
905 return error;
906 
>>> CID 1499386:  Memory - illegal accesses  (UNINIT)
>>> Using uninitialized value "olddp".
907 vp = olddp->d_vnode;
908 vn_lock(vp);
909 
910 if (vp->v_type == VDIR) {
911 error = EPERM;
912 goto out;

** CID 1499385:(UNREACHABLE)
/arch/x64/arch-trace.hh: 28 in ::tracepointv<(unsigned int)14, 
std::tuple (long), (_assign)>::operator ()(long)()
/arch/x64/arch-trace.hh: 28 in ::tracepointv<(unsigned int)17, 
std::tuple (long), (_assign)>::operator ()(long)()
/arch/x64/arch-trace.hh: 28 in ::tracepointv<(unsigned int)20, 
std::tuple (long), (_assign)>::operator ()(long)()
/arch/x64/arch-trace.hh: 28 in ::tracepointv<(unsigned int)11, 
std::tuple (long), (_assign)>::operator ()(long)()



*** CID 1499385:(UNREACHABLE)
/arch/x64/arch-trace.hh: 28 in ::tracepointv<(unsigned int)14, 
std::tuple (long), (_assign)>::operator ()(long)()
22 ".quad %c[type] \n\t"
23 ".quad 1b \n\t"
24 ".quad %l[slow_path] \n\t"
25 ".popsection"
26 : : [type]"i"((*this)), [id]"i"(_id) : : 
slow_path);
27 return;
>>> CID 1499385:(UNREACHABLE)
>>> This code cannot be reached: "slow_path:
auto data = ((st...".
28 slow_path:
29 // We don't want register shuffling and function calls here, so 
pretend
30 // to the compiler that the slow path just stores some data into 
local
31 // memory and executes an instruction that clobbers just a few 
registers
32 // (instead of lots of registers and all of memory):
33 auto data = assign(as...);
/arch/x64/arch-trace.hh: 28 in ::tracepointv<(unsigned int)17, 
std::tuple (long), (_assign)>::operator ()(long)()
22 ".quad %c[type] \n\t"
23 ".quad 1b \n\t"
24 ".quad %l[slow_path] \n\t"
25 ".popsection"
26 : : [type]"i"((*this)), [id]"i"(_id) : : 
slow_path);
27 return;
>>> CID 1499385:(UNREACHABLE)
>>> This code cannot be reached: "slow_path:
auto data = ((st...".
28 slow_path:
29 // We don't want register shuffling and function calls here, so 
pretend
30 // to the compiler that the slow path just stores some data into 
local
31 // memory and executes an instruction that clobbers just a few 
registers
32 // (instead of lots of registers and all of memory):
33 auto data = assign(as...);
/arch/x64/arch-trace.hh: 28 in ::tracepointv<(unsigned int)20, 
std::tuple (long), (_assign)>::operator ()(long)()
22 ".quad %c[type] \n\t"
23 ".quad 1b \n\t"
24 ".quad %l[slow_path] \n\t"
25 ".popsection"
26 : : [type]"i"((*this)), [id]"i"(_id) : : 
slow_path);
27 return;
>>> CID 1499385:(UNREACHABLE)
>>> This code cannot be reached: "slow_path:
auto data 

[osv-dev] Re: About JVM in aarch64

[Waldek Kozaczuk]

Hi,

I am not sure if you are still interested in it but I have just finally 
found some time to work on it and it seems I got it work. I will be sending 
a proper patch soon but here is an inlined one you can apply
manually to the current master:

diff --git a/arch/aarch64/arch-setup.cc b/arch/aarch64/arch-setup.cc
index 24e007c4..89dceae9 100644
--- a/arch/aarch64/arch-setup.cc
+++ b/arch/aarch64/arch-setup.cc
@@ -36,12 +36,11 @@
 
 void setup_temporary_phys_map()
 {
-// duplicate 1:1 mapping into phys_mem
+// duplicate 1:1 mapping into the lower part of phys_mem
 u64 *pt_ttbr0 = reinterpret_cast(processor::read_ttbr0());
-u64 *pt_ttbr1 = reinterpret_cast(processor::read_ttbr1());
 for (auto&& area : mmu::identity_mapped_areas) {
 auto base = reinterpret_cast(get_mem_area_base(area));
-pt_ttbr1[mmu::pt_index(base, 3)] = pt_ttbr0[0];
+pt_ttbr0[mmu::pt_index(base, 3)] = pt_ttbr0[0];
 }
 mmu::flush_tlb_all();
 }
diff --git a/core/mmu.cc b/core/mmu.cc
index 007d4331..33ae8407 100644
--- a/core/mmu.cc
+++ b/core/mmu.cc
@@ -78,7 +78,7 @@ public:
 };
 
 constexpr uintptr_t lower_vma_limit = 0x0;
-constexpr uintptr_t upper_vma_limit = 0x8000;
+constexpr uintptr_t upper_vma_limit = 0x4000;
 
 typedef boost::intrusive::set,
diff --git a/include/osv/mmu-defs.hh b/include/osv/mmu-defs.hh
index 18edf441..fd6a85a6 100644
--- a/include/osv/mmu-defs.hh
+++ b/include/osv/mmu-defs.hh
@@ -46,12 +46,12 @@ constexpr uintptr_t mem_area_size = uintptr_t(1) << 44;
 
 constexpr uintptr_t get_mem_area_base(mem_area area)
 {
-return 0x8000 | uintptr_t(area) << 44;
+return 0x4000 | uintptr_t(area) << 44;
 }
 
 static inline mem_area get_mem_area(void* addr)
 {
-return mem_area(reinterpret_cast(addr) >> 44 & 7);
+return mem_area(reinterpret_cast(addr) >> 44 & 3);
 }
 
 constexpr void* translate_mem_area(mem_area from, mem_area to, void* addr)
diff --git a/scripts/loader.py b/scripts/loader.py
index 6878a7a3..0ce782d0 100755
--- a/scripts/loader.py
+++ b/scripts/loader.py
@@ -27,7 +27,7 @@ class status_enum_class(object):
 pass
 status_enum = status_enum_class()
 
-phys_mem = 0x8000
+phys_mem = 0x4000
 
 def pt_index(addr, level):
 return (addr >> (12 + 9 * level)) & 511

Best regards,
Waldek

On Friday, September 2, 2022 at 1:26:42 PM UTC-4 Waldek Kozaczuk wrote:

> Yeah, this is not as trivial as changing this single line (that is why it 
> is not done yet :-)). But it may not be that difficult.
>
> I will try to take a stab at it and see what can be done.
>
> Waldek
>
> PS. It would be nice to get Java run on ARM.
>
> On Friday, September 2, 2022 at 12:44:36 AM UTC-4 iron...@gmail.com wrote:
>
>> Hi, I have a ongoing project about integrating OSV and JVM applications 
>> with aarch64.
>>
>> However, as you addressed, I encountered same issue when executing Java 
>> applications on osv.
>>
>> https://github.com/cloudius-systems/osv/issues/1196
>>
>> I am not quite getting a detail but you mentioned that changing 
>> *phys_mem* worked for these issues.
>>
>> After I saw your thread, I changed  *phys_mem *value
>>  from:0x8000 -> to:0x8000 
>>
>>   in get_mem_area_base at *include/osv/mmu-defs.hh, *but osv did not 
>> print any message in boot process  :/
>>
>> I will be appreciated if you offer the patch or commit  you did.
>>
>> Thank you in advance.
>>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/64fc1b28-c98d-4a08-8fe7-f0e56bd5e099n%40googlegroups.com.

[osv-dev] Re: About JVM in aarch64

[Waldek Kozaczuk]

Yeah, this is not as trivial as changing this single line (that is why it 
is not done yet :-)). But it may not be that difficult.

I will try to take a stab at it and see what can be done.

Waldek

PS. It would be nice to get Java run on ARM.

On Friday, September 2, 2022 at 12:44:36 AM UTC-4 iron...@gmail.com wrote:

> Hi, I have a ongoing project about integrating OSV and JVM applications 
> with aarch64.
>
> However, as you addressed, I encountered same issue when executing Java 
> applications on osv.
>
> https://github.com/cloudius-systems/osv/issues/1196
>
> I am not quite getting a detail but you mentioned that changing *phys_mem* 
> worked 
> for these issues.
>
> After I saw your thread, I changed  *phys_mem *value
>  from:0x8000 -> to:0x8000 
>
>   in get_mem_area_base at *include/osv/mmu-defs.hh, *but osv did not 
> print any message in boot process  :/
>
> I will be appreciated if you offer the patch or commit  you did.
>
> Thank you in advance.
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/560b452c-3c72-4e7d-8b87-ff27d49591d8n%40googlegroups.com.

Re: [osv-dev] [PATCH] vfs: support mounting ZFS from non-root partition

[Waldek Kozaczuk]

I have updated this Wiki page with the latest improvement that makes the 
build script add relevant mountpoint line to /etc/fstab when "mixing" RoFS 
with ZFS. I have also added extra info on how mount, inspect, modify and 
unmount ZFS images on Linux host.

On Monday, July 18, 2022 at 1:51:06 AM UTC-4 Waldek Kozaczuk wrote:

> Here is a new wiki page (still work in progress) that should help a bit - 
> https://github.com/cloudius-systems/osv/wiki/Filesystems#zfs.
>
> On Sunday, July 17, 2022 at 10:59:28 AM UTC-4 Waldek Kozaczuk wrote:
>
>> Done.
>>
>> Yes, I will write a detailed Wiki page about it. For sure it deserves it.
>>
>> On Sun, Jul 17, 2022 at 4:22 AM Nadav Har'El  wrote:
>>
>>> Hi, I tried to commit this patch but it did not apply, probably I got 
>>> the wrong order. Can you please apply this and the other related patches 
>>> yourself, in the right order? Thanks.
>>>
>>> --
>>> Nadav Har'El
>>> n...@scylladb.com
>>>
>>>
>>> On Sun, Jul 17, 2022 at 8:28 AM Waldemar Kozaczuk  
>>> wrote:
>>>
>>>> This patch mainly enhances VFS layer to support dynamically loading
>>>> the ZFS library libsolaris.so from other root filesystem like ROFS and
>>>> mounting ZFS filesystem from devices different than /dev/vblk0.1. The
>>>> supported scenarios include specifying a loader option '--mount-fs' or
>>>> adding an entry to /etc/fstab.
>>>>
>>>> In this patch we take advantage of the existing logic in VFS
>>>> pivot_rootfs() that implicitly loads shared libraries from the 
>>>> directory /usr/lib/fs
>>>> which is where we place libsolaris.so in the image. This was done as
>>>> part of the commit 4ffb0fa9329849cd587d62f91b6979bc0e0ce6d1 to support
>>>> dynamically loading NFS filesystem library. To support similar scenario
>>>> with ZFS we need to on top of this enhance the mount_fs() to detect
>>>> ZFS case and call zfsdev_init() and initialize BSD shrinker. We also
>>>> enhance unmount_rootfs() to make it unmount ZFS from non-root mount
>>>> points.
>>>>
>>>> This patch also add new module - zfs - which is intended to be used when
>>>> building two types of images as described by #1200. Please note the next
>>>> patch will enhance the build script to support building such images.
>>>>
>>>> 1. Run OSv from a single disk with two partitions: ROFS + ZFS (on 
>>>> /dev/vblk0.2)
>>>>   ./scripts/run.py --execute='--mount-fs=zfs,/dev/vblk0.2,/data 
>>>> /zpool.so list'
>>>>
>>>> 2. Run OSv with 2 disks: 1st one with ROFS and second one with ZFS 
>>>> (/dev/vblk1.1):
>>>>   ./scripts/run.py --execute='--mount-fs=zfs,/dev/vblk1.1,/data 
>>>> /zpool.so list' --second-disk-image build/release/zfs_disk.img
>>>>
>>>> Refs #1200
>>>>
>>>> Signed-off-by: Waldemar Kozaczuk 
>>>> ---
>>>>  Makefile   |  2 +-
>>>>  bootfs.manifest.skel   |  2 +-
>>>>  drivers/zfs.cc | 12 -
>>>>  exported_symbols/osv_libsolaris.so.symbols |  1 +
>>>>  fs/vfs/main.cc | 52 +++---
>>>>  fs/zfs/zfs_initialize.c|  6 +++
>>>>  fs/zfs/zfs_null_vfsops.cc  |  5 ++-
>>>>  loader.cc  | 34 ++
>>>>  modules/zfs-tools/usr.manifest |  1 -
>>>>  modules/zfs/usr.manifest   |  2 +
>>>>  scripts/upload_manifest.py |  2 +-
>>>>  usr_rofs.manifest.skel |  1 +
>>>>  zfs_builder_bootfs.manifest.skel   |  2 +-
>>>>  13 files changed, 100 insertions(+), 22 deletions(-)
>>>>  create mode 100644 modules/zfs/usr.manifest
>>>>
>>>> diff --git a/Makefile b/Makefile
>>>> index 6ac9c792..0625524b 100644
>>>> --- a/Makefile
>>>> +++ b/Makefile
>>>> @@ -2383,7 +2383,7 @@ 
>>>> $(out)/bsd/cddl/contrib/opensolaris/lib/libzfs/common/zprop_common.o: 
>>>> bsd/sys/cd
>>>>
>>>>  $(out)/libzfs.so: $(libzfs-objects) $(out)/libuutil.so 
>>>> $(out)/libsolaris.so
>>>> $(makedir)
>>>> -   $(call quiet, $(CC) $(CFLAGS) -o $@ $(libzfs-objects) -L$(out) 
>>&g

Re: [osv-dev] Build failed in Jenkins: osv-build #1077

[Waldek Kozaczuk]

Right. I think this is the right fix:

diff --git a/Makefile b/Makefile
index 0625524b..dab704af 100644
--- a/Makefile
+++ b/Makefile
@@ -529,6 +529,9 @@ $(out)/preboot.bin: $(out)/preboot.elf
 edata = $(shell readelf --syms $(out)/loader.elf | grep "\.edata" | awk 
'{print "0x" $$2}')
 image_size = $$(( $(edata) - $(kernel_vm_base) ))
 
+builder_edata = $(shell readelf --syms $(out)/zfs_builder.elf | grep 
"\.edata" | awk '{print "0x" $$2}')
+builder_image_size = $$(( $(builder_edata) - $(kernel_vm_base) ))
+
 $(out)/loader.img: $(out)/preboot.bin $(out)/loader-stripped.elf
$(call quiet, dd if=$(out)/preboot.bin of=$@ > /dev/null 2>&1, DD 
$@ preboot.bin)
$(call quiet, dd if=$(out)/loader-stripped.elf of=$@ conv=notrunc 
obs=4096 seek=16 > /dev/null 2>&1, DD $@ loader-stripped.elf)
@@ -538,7 +541,7 @@ $(out)/loader.img: $(out)/preboot.bin 
$(out)/loader-stripped.elf
 $(out)/zfs_builder.img: $(out)/preboot.bin $(out)/zfs_builder-stripped.elf
$(call quiet, dd if=$(out)/preboot.bin of=$@ > /dev/null 2>&1, DD 
$@ preboot.bin)
$(call quiet, dd if=$(out)/zfs_builder-stripped.elf of=$@ 
conv=notrunc obs=4096 seek=16 > /dev/null 2>&1, DD $@ 
zfs_builder-stripped.elf)
-   $(call quiet, scripts/imgedit.py setsize_aarch64 "-f raw $@" 
$(image_size), IMGEDIT $@)
+   $(call quiet, scripts/imgedit.py setsize_aarch64 "-f raw $@" 
$(builder_image_size), IMGEDIT $@)
$(call quiet, scripts/imgedit.py setargs "-f raw $@" $(cmdline), 
IMGEDIT $@)
 
 endif # aarch64


Could you please see if this works for you?

./scripts/build image=native-example fs=zfs arch=aarch64 -j$(nproc)
./scripts/run.py --arch=aarch64 (will complain of wrong arch of the 
native-example)

On Sunday, July 24, 2022 at 11:25:03 AM UTC-4 n...@scylladb.com wrote:

> On second thought, why does zfs_builder.img depend on the size of 
> loader.elf and not zfs_builder.elf?
> Are you familiar with the Makefile code ("git blame" suggests you wrote 
> it)? Can you please have a look?
> Maybe this use of "image_size" in that specific code is a mistake?
>
> --
> Nadav Har'El
> n...@scylladb.com
>
>
> On Sun, Jul 24, 2022 at 6:21 PM Nadav Har'El  wrote:
>
>> I think the missing dependency is here (it's in a aarch64-specific 
>> section):
>>
>> $(out)/zfs_builder.img: $(out)/preboot.bin $(out)/zfs_builder-stripped.elf
>>  $(call quiet, dd if=$(out)/preboot.bin of=$@ > /dev/null 2>&1, DD $@ 
>> preboot.bin)
>>  $(call quiet, dd if=$(out)/zfs_builder-stripped.elf of=$@ 
>> conv=notrunc obs=4096 seek=16 > /dev/null 2>&1, DD $@ 
>> zfs_builder-stripped.elf)
>>  $(call quiet, scripts/imgedit.py setsize_aarch64 "-f raw $@" 
>> $(image_size), IMGEDIT $@)
>>  $(call quiet, scripts/imgedit.py setargs "-f raw $@" $(cmdline), 
>> IMGEDIT $@)
>>
>> Note how it needs image_size, which runs readelf on loader.elf, but 
>> doesn't depend on loader.elf.
>>
>> I'll prepare a patch but am not sure I can reproduce it...
>>
>>
>> --
>> Nadav Har'El
>> n...@scylladb.com
>>
>>
>> On Sun, Jul 24, 2022 at 6:09 PM Nadav Har'El  wrote:
>>
>>> On Mon, Jul 18, 2022 at 6:42 AM Waldek Kozaczuk  
>>> wrote:
>>>
>>>> It is weird. I cannot reproduce it. I assume that the aarch64 build 
>>>> failed. Is it 'scripts/build -j$(nproc) arch=aarch64' in the Jenkins 
>>>> machine?
>>>>
>>>
>>> Sort of, you can see in 
>>> https://jenkins.scylladb.com/blue/rest/organizations/jenkins/pipelines/osv-build/runs/1077/log/?start=0
>>> it is:
>>>   
>>>
>>> make -j4 ARCH=aarch64
>>>
>>>
>>> The failure is:
>>>
>>>
>>>   LINK zfs_builder.elf
>>>   STRIP zfs_builder.elf -> zfs_builder-stripped.elf 
>>> readelf: build/release.aarch64/loader.elf: Error: Not an ELF file - it has 
>>> the wrong magic bytes at the start
>>>
>>>
>>>> I wonder if we have some missing dependency in the makefile. Possibly 
>>>> this line a symptom of some issue:
>>>>
>>>
>>> Yes, I'm guessing this is the case. I'll take a look.
>>>  
>>>
>>>>
>>>> readelf: build/release.aarch64/loader.elf: Error: Not an ELF file - it 
>>>> has the wrong magic bytes at the start
>>>>
>>>>
>>>>
>>>> On Sun, Jul 17, 2022 at 7:46 PM  wrote:
>>>>
>>>>> See <
>>>>> https://jen

Re: [osv-dev] [PATCH] routecache: handle case when sent to our non-loopback address

[Waldek Kozaczuk]

On Sunday, July 24, 2022 at 10:21:48 AM UTC-4 n...@scylladb.com wrote:

> On Sun, Jul 24, 2022 at 5:11 PM Waldek Kozaczuk  
> wrote:
>
>> BTW there are more routecache related patches that I think we can bring 
>> in from Spirent fork which I documented here - 
>> https://github.com/cloudius-systems/osv/issues/1197.
>> This one looks like a bug fix  - 
>> https://github.com/SpirentOrion/osv/commit/38df7da9fbe2f171297f41ecae58edc9bd436617
>>
>
> Yes, looks correct to me.
>  
>
>> This one more of an ehnancement - 
>> https://github.com/SpirentOrion/osv/commit/a20c112bf47a4a36b1fff17313f9d1b2e3ce919c
>> .
>>
>
> This patch seems to suggest our cache was missing rt_gateway and it needs 
> it. But why didn't I ever notice this?
> Unfortunately the patch doesn't say what kind of real-life scenario it was 
> trying to fix. 
>
>
>> I'm not sure this is the *best* way to tell these two cases apart, but I 
>>> probably agree it's a "good enough" way. As you can
>>> see in the comments I wrote in routecache.hh (which I should have 
>>> copy-edited more carefully, I see they have a lot of
>>> silly mistakes...), routecache.hh is all about the art of the "good 
>>> enough", so it makes sense to continue this.
>>>
>>  
>> In nutshell, this new version of the logic checks the device associated 
>> with the cached (present) route entry to determine what kind of match it 
>> should do - just netmask if non-loopback or
>> more involved in a loopback one.
>>
>
> Yes, but consider for example that even on the real loopback (not an 
> optimization where the outside address is "converted" to loopback) you can 
> have 127.0.0.1 and 127.0.0.2
> both on the same loopback. The new code will force it to use two different 
> routing table entries instead of one.
>
> I think it will use one entry. Consider this part of the new code:

135 if (is_loopback_net(address)) {
136 return 
137 }
138 // We shouldn't use this entry on IP addresses just 
because they're
139 // on the same network as our non-loopback address. So 
match the entire
140 // address.
141 if (e.address == address) {
142 return 
143 }

The 1st if statement will qualify both 127.0.0.1 and 127.0.0.2 as loopback, 
no? If that is the case, it will return the same entry for both. 
 

> It's not a huge problem - most people never use multiple loopback 
> addresses. But someone who will use twenty of these addresses (I'm 
> sometimes doing this to run multiple servers on the same host for testing), 
> will suffer because the route cache only has room for 10 routes.
>
> I'm tending to accept this patch, but I have a feeling that the "real" fix 
> would be something else. But this fix is probably "good enough".
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/6a0f1686-7b42-4788-98f5-f1a0f67e703fn%40googlegroups.com.

Re: [osv-dev] [PATCH] routecache: handle case when sent to our non-loopback address

[Waldek Kozaczuk]

BTW there are more routecache related patches that I think we can bring in 
from Spirent fork which I documented here 
- https://github.com/cloudius-systems/osv/issues/1197.
This one looks like a bug fix  
- 
https://github.com/SpirentOrion/osv/commit/38df7da9fbe2f171297f41ecae58edc9bd436617
This one more of an ehnancement 
- 
https://github.com/SpirentOrion/osv/commit/a20c112bf47a4a36b1fff17313f9d1b2e3ce919c.
What do you think about those above?

The patch we are discussing here addresses "real" bug I have come accross 
myself.

On Sunday, July 24, 2022 at 6:02:22 AM UTC-4 n...@scylladb.com wrote:

> Hi, this code is probably correct given that both you and the Spirent guys 
> gave it a lot of thought, but there's something I'm not sure I understand:
>
> If I understand correctly (but please correct me if I'm wrong), the 
> problem is that OSv has somewhere an "optimization" that
> internal connections to the node's external IP address are handled 
> internally (loopback) instead of being sent to the (virtual) network
> card, so if we cache this knowledge, then next time if the connection is 
> of the "other" type (external connection after we cached the
> internal route, or internal connection after the external route) the 
> result will be either not working (passing external connection
> through loopback) or inefficient (passing internal connection through the 
> real device).
>

Right, that is exactly what I am thinking as well. I am not an expert here, 
but I think the if_simloop() function called from ether_output() is where
that optimization happens. For illustration see this:

"local call (from itself on 192.168.122.15)"
0x8119b040 /httpclient.so   015.368111615 net_packet_in 
   b'IP 192.168.122.15.46890 > 192.168.122.15.8000: Flags ÆSÅ, seq 
744996567, win 65535, options Æmss 1460,nop,wscale 6,sackOK,TS val 15368 
ecr 0Å, length 0'
  log_packet_in(mbuf*, int) core/net_trace.cc:143
  netisr_queue_workstream(netisr_workstream*, unsigned int, netisr_work*, 
mbuf*, int*) Æclone .constprop.0Å bsd/sys/net/netisr.cc:633
  netisr_queue_src bsd/sys/net/netisr.cc:684
  netisr_queue_src bsd/sys/net/netisr.cc:712
  *if_simloop bsd/sys/net/if_loop.cc:291*
  *ether_output bsd/sys/net/if_ethersubr.cc:256*
 * ip_output(mbuf*, mbuf*, route*, int, ip_moptions*, inpcb*) 
bsd/sys/netinet/ip_output.cc:621*
  tcp_output bsd/sys/netinet/tcp_output.cc:1385
  tcp_usr_connect bsd/sys/netinet/tcp_usrreq.cc:465
  tcp_usr_connect bsd/sys/netinet/tcp_usrreq.cc:436
  kern_connect bsd/sys/kern/uipc_syscalls.cc:374

"external call"
0x807e0040 virtio-net-rx0 4.284705130 net_packet_out   
b'IP 192.168.122.15.8000 > 192.168.122.1.45818: Flags Æ.Å, ack 512095, 
win 65534, length 0'
  log_packet_out(mbuf*, int) core/net_trace.cc:148
  *ether_output bsd/sys/net/if_ethersubr.cc:397 //This should really 
be ether_output_frame*
  *ether_output bsd/sys/net/if_ethersubr.cc:366*
  *ip_output(mbuf*, mbuf*, route*, int, ip_moptions*, inpcb*) 
bsd/sys/netinet/ip_output.cc:621*
  tcp_twrespond bsd/sys/netinet/tcp_timewait.cc:553
  tcp_twstart bsd/sys/netinet/tcp_timewait.cc:281
  tcp_do_segment(mbuf*, tcphdr*, socket*, tcpcb*, int, int, unsigned char, 
int, bool&) bsd/sys/netinet/tcp_input.cc:2511
  tcp_input bsd/sys/netinet/tcp_input.cc:956
  ip_input bsd/sys/netinet/ip_input.cc:774
  netisr_dispatch_src bsd/sys/net/netisr.cc:769

BTW in case of the local call the qualification "net_packet_in" vs 
"net_packet_out" might be confusing. So this if_simloop() might be what is 
used to handle this kind of "loops" to itself.


> If I understand correctly your patch works because external connections 
> happen to look up the gateway (*.1)
> instead of the actual IP address, so it makes it easy to tell these two 
> cases apart. Is this true?
>
I think these are actually the "responses" back from 192.168.122.15 to the 
"world outside" (pointed by the gateway 192.168.122.1) when
the routecache is used as well. 

>
> I'm not sure this is the *best* way to tell these two cases apart, but I 
> probably agree it's a "good enough" way. As you can
> see in the comments I wrote in routecache.hh (which I should have 
> copy-edited more carefully, I see they have a lot of
> silly mistakes...), routecache.hh is all about the art of the "good 
> enough", so it makes sense to continue this.
>
 
In nutshell, this new version of the logic checks the device associated 
with the cached (present) route entry to determine what kind of match it 
should do - just netmask if non-loopback or
more involved in a loopback one.

>
>
>
> --
> Nadav Har'El
> n...@scylladb.com
>
>
> On Sat, Jul 23, 2022 at 9:15 PM Waldemar Kozaczuk  
> wrote:
>
>> This patch modifies the OSv routecache to handle the case when a route
>> entry gets added for a non-loopback address and loopback device before
>> packets start coming from outside of the local network. The practical
>> example is an app that on startup calls itself on its HTTP endpoint but
>> 

Re: [osv-dev] [PATCH] vfs: support mounting ZFS from non-root partition

[Waldek Kozaczuk]

Here is a new wiki page (still work in progress) that should help a bit 
- https://github.com/cloudius-systems/osv/wiki/Filesystems#zfs.

On Sunday, July 17, 2022 at 10:59:28 AM UTC-4 Waldek Kozaczuk wrote:

> Done.
>
> Yes, I will write a detailed Wiki page about it. For sure it deserves it.
>
> On Sun, Jul 17, 2022 at 4:22 AM Nadav Har'El  wrote:
>
>> Hi, I tried to commit this patch but it did not apply, probably I got the 
>> wrong order. Can you please apply this and the other related patches 
>> yourself, in the right order? Thanks.
>>
>> --
>> Nadav Har'El
>> n...@scylladb.com
>>
>>
>> On Sun, Jul 17, 2022 at 8:28 AM Waldemar Kozaczuk  
>> wrote:
>>
>>> This patch mainly enhances VFS layer to support dynamically loading
>>> the ZFS library libsolaris.so from other root filesystem like ROFS and
>>> mounting ZFS filesystem from devices different than /dev/vblk0.1. The
>>> supported scenarios include specifying a loader option '--mount-fs' or
>>> adding an entry to /etc/fstab.
>>>
>>> In this patch we take advantage of the existing logic in VFS
>>> pivot_rootfs() that implicitly loads shared libraries from the directory 
>>> /usr/lib/fs
>>> which is where we place libsolaris.so in the image. This was done as
>>> part of the commit 4ffb0fa9329849cd587d62f91b6979bc0e0ce6d1 to support
>>> dynamically loading NFS filesystem library. To support similar scenario
>>> with ZFS we need to on top of this enhance the mount_fs() to detect
>>> ZFS case and call zfsdev_init() and initialize BSD shrinker. We also
>>> enhance unmount_rootfs() to make it unmount ZFS from non-root mount
>>> points.
>>>
>>> This patch also add new module - zfs - which is intended to be used when
>>> building two types of images as described by #1200. Please note the next
>>> patch will enhance the build script to support building such images.
>>>
>>> 1. Run OSv from a single disk with two partitions: ROFS + ZFS (on 
>>> /dev/vblk0.2)
>>>   ./scripts/run.py --execute='--mount-fs=zfs,/dev/vblk0.2,/data 
>>> /zpool.so list'
>>>
>>> 2. Run OSv with 2 disks: 1st one with ROFS and second one with ZFS 
>>> (/dev/vblk1.1):
>>>   ./scripts/run.py --execute='--mount-fs=zfs,/dev/vblk1.1,/data 
>>> /zpool.so list' --second-disk-image build/release/zfs_disk.img
>>>
>>> Refs #1200
>>>
>>> Signed-off-by: Waldemar Kozaczuk 
>>> ---
>>>  Makefile   |  2 +-
>>>  bootfs.manifest.skel   |  2 +-
>>>  drivers/zfs.cc | 12 -
>>>  exported_symbols/osv_libsolaris.so.symbols |  1 +
>>>  fs/vfs/main.cc | 52 +++---
>>>  fs/zfs/zfs_initialize.c|  6 +++
>>>  fs/zfs/zfs_null_vfsops.cc  |  5 ++-
>>>  loader.cc  | 34 ++
>>>  modules/zfs-tools/usr.manifest |  1 -
>>>  modules/zfs/usr.manifest   |  2 +
>>>  scripts/upload_manifest.py |  2 +-
>>>  usr_rofs.manifest.skel |  1 +
>>>  zfs_builder_bootfs.manifest.skel   |  2 +-
>>>  13 files changed, 100 insertions(+), 22 deletions(-)
>>>  create mode 100644 modules/zfs/usr.manifest
>>>
>>> diff --git a/Makefile b/Makefile
>>> index 6ac9c792..0625524b 100644
>>> --- a/Makefile
>>> +++ b/Makefile
>>> @@ -2383,7 +2383,7 @@ 
>>> $(out)/bsd/cddl/contrib/opensolaris/lib/libzfs/common/zprop_common.o: 
>>> bsd/sys/cd
>>>
>>>  $(out)/libzfs.so: $(libzfs-objects) $(out)/libuutil.so 
>>> $(out)/libsolaris.so
>>> $(makedir)
>>> -   $(call quiet, $(CC) $(CFLAGS) -o $@ $(libzfs-objects) -L$(out) 
>>> -luutil -lsolaris, LINK libzfs.so)
>>> +   $(call quiet, $(CC) $(CFLAGS) -o $@ $(libzfs-objects) -L$(out) 
>>> -luutil, LINK libzfs.so)
>>>
>>>  #include $(src)/bsd/cddl/contrib/opensolaris/cmd/zpool/build.mk:
>>>  zpool-cmd-file-list = zpool_iter  zpool_main  zpool_util  zpool_vdev
>>> diff --git a/bootfs.manifest.skel b/bootfs.manifest.skel
>>> index bab4c606..5e71fb3b 100644
>>> --- a/bootfs.manifest.skel
>>> +++ b/bootfs.manifest.skel
>>> @@ -1,2 +1,2 @@
>>>  [manifest]
>>> -/libsolaris.so: libsolaris.so
>>> +/usr/lib/fs/libsolaris.so: libsolaris.so
>>> diff --

Re: [osv-dev] Build failed in Jenkins: osv-build #1077

[Waldek Kozaczuk]

It is weird. I cannot reproduce it. I assume that the aarch64 build failed.
Is it 'scripts/build -j$(nproc) arch=aarch64' in the Jenkins machine?

I wonder if we have some missing dependency in the makefile. Possibly this
line a symptom of some issue:

readelf: build/release.aarch64/loader.elf: Error: Not an ELF file - it has
the wrong magic bytes at the start



On Sun, Jul 17, 2022 at 7:46 PM  wrote:

> See <
> https://jenkins.scylladb.com/job/osv-build/1077/display/redirect?page=changes
> >
>
> Changes:
>
> [jwkozaczuk] zfs: use spa_dev_path instead of defaulting to /dev/vblk0.1
>
> [jwkozaczuk] zfs: refactor loader to handle loading libsolaris.so
>
> [jwkozaczuk] devfs: print details of mounted partition
>
> [jwkozaczuk] vfs: support mounting ZFS from non-root partition
>
> [jwkozaczuk] zfs: support building rofs+zfs image and second ZFS disk
>
> [jwkozaczuk] tests: add misc-zfs-io to all images
>
> [jwkozaczuk] zpool import: do not try devices where ZFS is mounted
>
> [jwkozaczuk] tests: fix misc-zfs-arc.cc
>
> [jwkozaczuk] zfs: allow mounting and building on host
>
> [jwkozaczuk] Fix dependabot alert in jolokia module
>
>
> --
> [...truncated 683.59 KB...]
>   CC musl/src/math/log2l.c
>   CC musl/src/math/logb.c
>   CC musl/src/math/logbf.c
>   CC musl/src/math/logbl.c
>   CC musl/src/math/logf.c
>   CC musl/src/math/logl.c
>   CC musl/src/math/lrint.c
>   CC musl/src/math/lround.c
>   CC musl/src/math/lroundf.c
>   CC musl/src/math/lroundl.c
>   CC musl/src/math/modf.c
>   CC musl/src/math/modff.c
>   CC musl/src/math/modfl.c
>   CC musl/src/math/nan.c
>   CC musl/src/math/nanf.c
>   CC musl/src/math/nanl.c
>   CC musl/src/math/nearbyint.c
>   CC musl/src/math/nearbyintf.c
>   CC musl/src/math/nearbyintl.c
>   CC musl/src/math/nextafter.c
>   CC musl/src/math/nextafterf.c
>   CC musl/src/math/nextafterl.c
>   CC musl/src/math/nexttoward.c
>   CC musl/src/math/nexttowardf.c
>   CC musl/src/math/nexttowardl.c
>   CC musl/src/math/pow.c
>   CC musl/src/math/powf.c
>   CC musl/src/math/powf_data.c
>   CC musl/src/math/powl.c
>   CC musl/src/math/remainder.c
>   CC musl/src/math/remainderf.c
>   CC musl/src/math/remainderl.c
>   CC musl/src/math/remquo.c
>   CC musl/src/math/remquof.c
>   CC musl/src/math/remquol.c
>   CC musl/src/math/rint.c
>   CC musl/src/math/rintf.c
>   CC musl/src/math/rintl.c
>   CC musl/src/math/round.c
>   CC musl/src/math/roundf.c
>   CC musl/src/math/roundl.c
>   CC musl/src/math/scalb.c
>   CC musl/src/math/scalbf.c
>   CC musl/src/math/scalbln.c
>   CC musl/src/math/scalblnf.c
>   CC musl/src/math/scalblnl.c
>   CC musl/src/math/scalbn.c
>   CC musl/src/math/scalbnf.c
>   CC musl/src/math/scalbnl.c
>   CC musl/src/math/signgam.c
>   CC musl/src/math/significand.c
>   CC musl/src/math/significandf.c
>   CC musl/src/math/sin.c
>   CC musl/src/math/sincos.c
>   CC musl/src/math/sincosf.c
>   CC musl/src/math/sinf.c
>   CC musl/src/math/sinh.c
>   CC musl/src/math/sinhf.c
>   CC musl/src/math/sinhl.c
>   CC musl/src/math/sinl.c
>   CC musl/src/math/sqrt.c
>   CC musl/src/math/sqrtf.c
>   CC musl/src/math/sqrtl.c
>   CC musl/src/math/tan.c
>   CC musl/src/math/tanf.c
>   CC musl/src/math/tanh.c
>   CC musl/src/math/tanhf.c
>   CC musl/src/math/tanhl.c
>   CC musl/src/math/tanl.c
>   CC musl/src/math/tgamma.c
>   CC musl/src/math/tgammaf.c
>   CC musl/src/math/tgammal.c
>   CC musl/src/math/trunc.c
>   CC musl/src/math/truncf.c
>   CC musl/src/math/truncl.c
>   CC musl/src/misc/a64l.c
>   CC musl/src/misc/get_current_dir_name.c
>   CC musl/src/misc/getsubopt.c
>   CC musl/src/multibyte/btowc.c
>   CC musl/src/multibyte/internal.c
>   CC musl/src/multibyte/mblen.c
>   CC musl/src/multibyte/mbrtowc.c
>   CC musl/src/multibyte/mbsrtowcs.c
>   CC musl/src/multibyte/mbstowcs.c
>   CC musl/src/multibyte/mbtowc.c
>   CC musl/src/multibyte/wcrtomb.c
>   CC musl/src/multibyte/wcstombs.c
>   CC musl/src/multibyte/wctob.c
>   CC musl/src/multibyte/wctomb.c
>   CC musl/src/network/gethostbyaddr.c
>   CC musl/src/network/resolvconf.c
>   CC musl/src/network/res_msend.c
>   CC musl/src/network/lookup_ipliteral.c
>   CC musl/src/network/lookup_name.c
>   CC musl/src/network/lookup_serv.c
>   CC musl/src/network/inet_aton.c
>   CC musl/src/network/gai_strerror.c
>   CC musl/src/network/getservbyname_r.c
>   CC musl/src/network/getservbyport_r.c
>   CC musl/src/network/getifaddrs.c
>   CC musl/src/network/if_nameindex.c
>   CC musl/src/network/if_freenameindex.c
>   CC musl/src/prng/rand.c
>   CC musl/src/prng/rand_r.c
>   CC musl/src/prng/drand48.c
>   CC musl/src/prng/lcong48.c
>   CC musl/src/prng/lrand48.c
>   CC musl/src/prng/mrand48.c
>   CC musl/src/prng/seed48.c
>   CC musl/src/prng/srand48.c
>   CC musl/src/search/tsearch.c
>   CC musl/src/stdio/__fclose_ca.c
>   CC musl/src/stdio/__overflow.c
>   CC musl/src/stdio/__stdio_close.c
>   CC musl/src/stdio/__stdio_exit.c
>   CC musl/src/stdio/__stdio_seek.c
>   CC musl/src/stdio/__stdio_write.c
>   CC 

Re: [osv-dev] [PATCH] vfs: support mounting ZFS from non-root partition

[Waldek Kozaczuk]

Done.

Yes, I will write a detailed Wiki page about it. For sure it deserves it.

On Sun, Jul 17, 2022 at 4:22 AM Nadav Har'El  wrote:

> Hi, I tried to commit this patch but it did not apply, probably I got the
> wrong order. Can you please apply this and the other related patches
> yourself, in the right order? Thanks.
>
> --
> Nadav Har'El
> n...@scylladb.com
>
>
> On Sun, Jul 17, 2022 at 8:28 AM Waldemar Kozaczuk 
> wrote:
>
>> This patch mainly enhances VFS layer to support dynamically loading
>> the ZFS library libsolaris.so from other root filesystem like ROFS and
>> mounting ZFS filesystem from devices different than /dev/vblk0.1. The
>> supported scenarios include specifying a loader option '--mount-fs' or
>> adding an entry to /etc/fstab.
>>
>> In this patch we take advantage of the existing logic in VFS
>> pivot_rootfs() that implicitly loads shared libraries from the directory
>> /usr/lib/fs
>> which is where we place libsolaris.so in the image. This was done as
>> part of the commit 4ffb0fa9329849cd587d62f91b6979bc0e0ce6d1 to support
>> dynamically loading NFS filesystem library. To support similar scenario
>> with ZFS we need to on top of this enhance the mount_fs() to detect
>> ZFS case and call zfsdev_init() and initialize BSD shrinker. We also
>> enhance unmount_rootfs() to make it unmount ZFS from non-root mount
>> points.
>>
>> This patch also add new module - zfs - which is intended to be used when
>> building two types of images as described by #1200. Please note the next
>> patch will enhance the build script to support building such images.
>>
>> 1. Run OSv from a single disk with two partitions: ROFS + ZFS (on
>> /dev/vblk0.2)
>>   ./scripts/run.py --execute='--mount-fs=zfs,/dev/vblk0.2,/data /zpool.so
>> list'
>>
>> 2. Run OSv with 2 disks: 1st one with ROFS and second one with ZFS
>> (/dev/vblk1.1):
>>   ./scripts/run.py --execute='--mount-fs=zfs,/dev/vblk1.1,/data /zpool.so
>> list' --second-disk-image build/release/zfs_disk.img
>>
>> Refs #1200
>>
>> Signed-off-by: Waldemar Kozaczuk 
>> ---
>>  Makefile   |  2 +-
>>  bootfs.manifest.skel   |  2 +-
>>  drivers/zfs.cc | 12 -
>>  exported_symbols/osv_libsolaris.so.symbols |  1 +
>>  fs/vfs/main.cc | 52 +++---
>>  fs/zfs/zfs_initialize.c|  6 +++
>>  fs/zfs/zfs_null_vfsops.cc  |  5 ++-
>>  loader.cc  | 34 ++
>>  modules/zfs-tools/usr.manifest |  1 -
>>  modules/zfs/usr.manifest   |  2 +
>>  scripts/upload_manifest.py |  2 +-
>>  usr_rofs.manifest.skel |  1 +
>>  zfs_builder_bootfs.manifest.skel   |  2 +-
>>  13 files changed, 100 insertions(+), 22 deletions(-)
>>  create mode 100644 modules/zfs/usr.manifest
>>
>> diff --git a/Makefile b/Makefile
>> index 6ac9c792..0625524b 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -2383,7 +2383,7 @@
>> $(out)/bsd/cddl/contrib/opensolaris/lib/libzfs/common/zprop_common.o:
>> bsd/sys/cd
>>
>>  $(out)/libzfs.so: $(libzfs-objects) $(out)/libuutil.so
>> $(out)/libsolaris.so
>> $(makedir)
>> -   $(call quiet, $(CC) $(CFLAGS) -o $@ $(libzfs-objects) -L$(out)
>> -luutil -lsolaris, LINK libzfs.so)
>> +   $(call quiet, $(CC) $(CFLAGS) -o $@ $(libzfs-objects) -L$(out)
>> -luutil, LINK libzfs.so)
>>
>>  #include $(src)/bsd/cddl/contrib/opensolaris/cmd/zpool/build.mk:
>>  zpool-cmd-file-list = zpool_iter  zpool_main  zpool_util  zpool_vdev
>> diff --git a/bootfs.manifest.skel b/bootfs.manifest.skel
>> index bab4c606..5e71fb3b 100644
>> --- a/bootfs.manifest.skel
>> +++ b/bootfs.manifest.skel
>> @@ -1,2 +1,2 @@
>>  [manifest]
>> -/libsolaris.so: libsolaris.so
>> +/usr/lib/fs/libsolaris.so: libsolaris.so
>> diff --git a/drivers/zfs.cc b/drivers/zfs.cc
>> index fb335340..02c011ac 100644
>> --- a/drivers/zfs.cc
>> +++ b/drivers/zfs.cc
>> @@ -61,16 +61,24 @@ zfs_device::~zfs_device()
>>  device_destroy(_zfs_dev);
>>  }
>>
>> +static bool zfsdev_initialized = false;
>> +
>>  void zfsdev_init(void)
>>  {
>> -new zfs_device();
>> +if (!zfsdev_initialized) {
>> +new zfs_device();
>> +zfsdev_initialized = true;
>> +}
>>  }
>>
>>  }
>>
>>  extern "C" OSV_LIBSOLARIS_API void zfsdev_init()
>>  {
>> -new zfsdev::zfs_device();
>> +if (!zfsdev::zfsdev_initialized) {
>> +new zfsdev::zfs_device();
>> +zfsdev::zfsdev_initialized = true;
>> +}
>>  }
>>
>>
>> diff --git a/exported_symbols/osv_libsolaris.so.symbols
>> b/exported_symbols/osv_libsolaris.so.symbols
>> index c115cb02..9b959675 100644
>> --- a/exported_symbols/osv_libsolaris.so.symbols
>> +++ b/exported_symbols/osv_libsolaris.so.symbols
>> @@ -91,4 +91,5 @@ vrele
>>  vttoif_tab
>>  wakeup
>>  zfsdev_init
>> +zfs_driver_initialized
>>  zfs_update_vfsops
>> diff --git a/fs/vfs/main.cc b/fs/vfs/main.cc
>> index 

[osv-dev] Re: [COMMIT osv master] libc: implement getentropy()

[Waldek Kozaczuk]

Excellent! Now I can upgrade to Fedora 36 as well.

Do unit tests build and pass as well?

On Sunday, June 26, 2022 at 6:28:34 AM UTC-4 Commit Bot wrote:

> From: Nadav Har'El 
> Committer: Nadav Har'El 
> Branch: master
>
> libc: implement getentropy()
>
> Starting in gcc commit
> https://gcc.gnu.org/g:3439657b02869299685d259c3a77aa38714565b7, libstdc++
> started to use the getentropy() function, so the OSv kernel cannot be
> linked without implementing it.
>
> It's easy to implement getentropy() by calling getrandom().
>
> Musl also has its own getentropy.c but it has a silly compilation error
> (when len=0, it returns an uninitialized "ret") so let's just write
> one of our own and later reconsider taking Musl's one.
>
> After this patch, OSv can finally build on Fedora 36 with Gcc 12.1.1.
>
> Fixes #1198.
>
> Signed-off-by: Nadav Har'El 
>
> ---
> diff --git a/include/api/unistd.h b/include/api/unistd.h
> --- a/include/api/unistd.h
> +++ b/include/api/unistd.h
> @@ -179,6 +179,7 @@ char *getusershell(void);
> int acct(const char *);
> long syscall(long, ...);
> long __syscall(long, ...);
> +int getentropy(void *, size_t);
> #endif
>
> #ifdef _GNU_SOURCE
> diff --git a/libc/random.c b/libc/random.c
> --- a/libc/random.c
> +++ b/libc/random.c
> @@ -1,5 +1,6 @@
> /*
> * Copyright (C) 2018 Waldemar Kozaczuk
> + * Copyright (C) 2022 Nadav Har'El
> *
> * This work is open source software, licensed under the terms of the
> * BSD license as described in the LICENSE file in the top-level directory.
> @@ -49,3 +50,14 @@ ssize_t getrandom(void *buf, size_t count, unsigned int 
> flags)
> close(fd);
> return read;
> }
> +
> +int getentropy(void *buf, size_t len)
> +{
> + if (len > 256) {
> + errno = EIO;
> + return -1;
> + } else if (len == 0) {
> + return 0;
> + }
> + return getrandom(buf, len, 0) >= 0;
> +}
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/36d2b09a-c689-446e-83c0-1222c6d9bd52n%40googlegroups.com.

[osv-dev] Bug fixes and enhancements from the Spirent fork of OSv

[Waldek Kozaczuk]

Hi,

Recently I found some time to research and curate a list of bug fixes and 
enhancements from the Spirent fork of OSv - 
https://github.com/SpirentOrion/osv. I have listed and described all my 
findings as the issue https://github.com/cloudius-systems/osv/issues/1197.

I think that some of them can be merged as is. But some require more 
analysis and testing. And some may provides unique functionality that might 
be of little interest to most OSv users.

If anyone is interested in bringing in any of the features, feel free to 
add your comments to the issue 1197 or respond to this email.

Waldek




-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/c9cfedbb-4dc0-45c1-8b74-f22ec2c7294fn%40googlegroups.com.

Re: [osv-dev] [PATCH] vfs: refactor the *at() functions

[Waldek Kozaczuk]

Thanks for the review.

On Wednesday, May 18, 2022 at 2:45:34 AM UTC-4 Nadav Har'El wrote:

> On Wed, May 18, 2022 at 6:25 AM Waldemar Kozaczuk  
> wrote:
>
>> The VFS functions like openat(), faccessat() and others alike
>> take a directory descriptor argument intended to make a filesystem
>> action happen relative to that directory.
>>
>> The implemetations of those function repeat almost the same code
>> over and over. So this patch makes vfs more DRY by introducing 
>> a helper function - vfs_fun_at() - which implements common
>> logic and executed a lambda function specific to given
>> VFS action.
>>
>> This patch also adds some unit tests around readlinkat() and mkdirat().
>>
>> Signed-off-by: Waldemar Kozaczuk 
>> ---
>>  fs/vfs/main.cc   | 188 +++
>>  tests/tst-readdir.cc |   9 +++
>>  tests/tst-symlink.cc |  12 ++-
>>  3 files changed, 66 insertions(+), 143 deletions(-)
>>
>> diff --git a/fs/vfs/main.cc b/fs/vfs/main.cc
>> index ca357cc8..a72042b2 100644
>> --- a/fs/vfs/main.cc
>> +++ b/fs/vfs/main.cc
>> @@ -160,21 +160,8 @@ int open(const char *pathname, int flags, ...)
>>
>>  LFS64(open);
>>
>> -OSV_LIBC_API
>> -int openat(int dirfd, const char *pathname, int flags, ...)
>> +static int vfs_fun_at(int dirfd, const char *pathname, 
>> std::function fun)
>>
>
> Nitpick: I think if you want zero runtime overhead for this function, you 
> can use a template
> parameter for this "fun" instead of std::function. But I don't know, maybe 
> the compiler is actually
> smart enough to optimize all this std::function stuff away when inlining 
> vfs_fun_at calls?
> And it's probably negligible anyway compared to all those strlcat calls :-)
>
> I think I will stick with the regular function. As you point out it comes 
with a overhead of 2 function calls
but on other hand it makes a binary smaller unlike templates. Plus these 
functions are normally not called thousand of times
by normal apps. But we can always change it later.
 

>  {
>
>
>> -mode_t mode = 0;
>> -if (flags & O_CREAT) {
>> -va_list ap;
>> -va_start(ap, flags);
>> -mode = apply_umask(va_arg(ap, mode_t));
>> -va_end(ap);
>> -}
>> -
>> -if (pathname[0] == '/' || dirfd == AT_FDCWD) {
>> -return open(pathname, flags, mode);
>> -}
>>
>
> Unless I'm misunderstanding something, I think this if() should remain 
> (and of
> course call fun(), not open()). More on this below.
>

> -
>>  struct file *fp;
>>  int error = fget(dirfd, );
>>  if (error) {
>> @@ -191,16 +178,38 @@ int openat(int dirfd, const char *pathname, int 
>> flags, ...)
>>  /* build absolute path */
>>  strlcpy(p, fp->f_dentry->d_mount->m_path, PATH_MAX);
>>  strlcat(p, fp->f_dentry->d_path, PATH_MAX);
>> -strlcat(p, "/", PATH_MAX);
>> -strlcat(p, pathname, PATH_MAX);
>> +if (pathname) {
>>
>
> Is any *at() function actually allowed to pass a null pathname? (which is 
> not the same as an empty string).
> If Linux allows this, we should have a test for it.
>
I need to handle it. 

>
> +strlcat(p, "/", PATH_MAX);
>> +strlcat(p, pathname, PATH_MAX);
>> +}
>>
>> -error = open(p, flags, mode);
>> +error = fun(p);
>>
>>  vn_unlock(vp);
>>  fdrop(fp);
>>
>>  return error;
>>  }
>> +
>> +OSV_LIBC_API
>> +int openat(int dirfd, const char *pathname, int flags, ...)
>> +{
>> +mode_t mode = 0;
>> +if (flags & O_CREAT) {
>> +va_list ap;
>> +va_start(ap, flags);
>> +mode = apply_umask(va_arg(ap, mode_t));
>> +va_end(ap);
>> +} 
>>
> +
>> +if (pathname[0] == '/' || dirfd == AT_FDCWD) {
>> +return open(pathname, flags, mode);
>> +}
>>
>
> I think this if() should be part of the general vfs_fun_at()  code.
> I looked at openat(2), mkdirat(2), fdaccessat(2) for example - and they
> all support AT_FDCWD. Are there functions which don't?
>
I agree with your idea - I will improve it in the next patch. 

>
> +
>> +return vfs_fun_at(dirfd, pathname, [flags, mode](const char 
>> *absolute_path) {
>> +return open(absolute_path, flags, mode);
>> +});
>> +}
>>  LFS64(openat);
>>
>>  // open() has an optional third argument, "mode", which is only needed in
>> @@ -611,35 +620,14 @@ int __fxstatat(int ver, int dirfd, const char 
>> *pathname, struct stat *st,
>>  return fstat(dirfd, st);
>>  }
>>
>> -struct file *fp;
>> -int error = fget(dirfd, );
>> -if (error) {
>> -errno = error;
>> -return -1;
>> -}
>> -
>> -struct vnode *vp = fp->f_dentry->d_vnode;
>> -vn_lock(vp);
>> -
>> -std::unique_ptr up (new char[PATH_MAX]);
>> -char *p = up.get();
>> -/* build absolute path */
>> -strlcpy(p, fp->f_dentry->d_mount->m_path, PATH_MAX);
>> -strlcat(p, fp->f_dentry->d_path, PATH_MAX);
>> -strlcat(p, "/", PATH_MAX);
>> -strlcat(p, pathname, PATH_MAX);
>> -
>> -if (flags & AT_SYMLINK_NOFOLLOW) 

[osv-dev] Re: Missing _dl_find_object()

[Waldek Kozaczuk]

I think I have a good handle on what needs to be done and I have been able 
to successfully test it somehow.

Should be sending some patches soon.

Regards,
Waldek

On Saturday, May 7, 2022 at 12:07:51 PM UTC-4 Waldek Kozaczuk wrote:

> And here is how gcc changed to use _dl_find_object() - 
> https://github.com/gcc-mirror/gcc/commit/790854ea7670f11c14d431c102a49181d2915965.
>  
> It seems to be used during unwinding.
>
> On Saturday, May 7, 2022 at 1:32:10 AM UTC-4 Waldek Kozaczuk wrote:
>
>> After upgrading to newest Ubuntu 2022.04 I started seeing following 
>> crashes of programs using libgcc_s.so:
>>
>> #0  _dl_find_object (pc1=0x1003f5ca <_Unwind_Resume+58>, 
>> result=0x20200180) at linux.cc:542
>> #1  0x100410f6 in _Unwind_Find_FDE ()
>> #2  0x1003d833 in ?? ()
>> #3  0x1003ead0 in ?? ()
>> #4  0x1003f5cb in _Unwind_Resume ()
>> #5  0x10062daa in ?? ()
>> #6  0x10075b5c in 
>> boost::execution_monitor::vexecute(boost::function const&) ()
>> #7  0x1007f0a9 in boost::unit_test::framework::init(bool (*)(), 
>> int, char**) ()
>> #8  0x1009254d in boost::unit_test::unit_test_main(bool (*)(), 
>> int, char**) ()
>> #9  0x4039d001 in osv::application::run_main 
>> (this=0xa0bd8c10) at core/app.cc:416
>> #10 0x4039d20d in operator() (app=, __closure=0x0) 
>> at core/app.cc:236
>> #11 _FUN () at core/app.cc:238
>> #12 0x403d084a in operator() (__closure=0xa0d57800) at 
>> libc/pthread.cc:116
>> #13 std::__invoke_impl> (*)(void*), void*, sigset_t, const 
>> pthread_private::thread_attr*)::&> (__f=...)
>> at /usr/include/c++/11/bits/invoke.h:61
>> #14 std::__invoke_r> (*)(void*), void*, sigset_t, const 
>> pthread_private::thread_attr*)::&> (__fn=...)
>> at /usr/include/c++/11/bits/invoke.h:154
>> #15 std::_Function_handler> pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const 
>> pthread_private::thread_attr*):: >::_M_invoke(const 
>> std::_Any_data &) (__functor=...) at 
>> /usr/include/c++/11/bits/std_function.h:290
>> #16 0x4036b58e in sched::thread::main (this=0x80f6a040) 
>> at core/sched.cc:1267
>> #17 sched::thread_main_c (t=0x80f6a040) at 
>> arch/x64/arch-switch.hh:325
>> #18 0x402fda03 in thread_main () at arch/x64/entry.S:116
>>
>> After some digging I discovered this new function was added as per this 
>> patch to libc - 
>> https://www.mail-archive.com/gcc-p...@gcc.gnu.org/msg275305.html 
>> <https://www.mail-archive.com/gcc-patches@gcc.gnu.org/msg275305.html>. 
>> And there s this description:
>>
>> http://www.gnu.org/software/libc/manual/html_node/Dynamic-Linker-Introspection.html
>>
>> It is interesting that libgcc_s.so on pretty new Fedora 35 with newer gcc 
>> does not need this.
>>
>> I tried to stub something but this is not enough:
>>
>> struct dl_find_object
>> {
>>   __extension__ unsigned long long int dlfo_flags;
>>   void *dlfo_map_start;/* Beginning of mapping containing 
>> address.  */
>>   void *dlfo_map_end;/* End of mapping.  */
>>   struct link_map *dlfo_link_map;
>>   void *dlfo_eh_frame;/* Exception handling data of the object. 
>>  */
>>   __extension__ unsigned long long int __dflo_reserved[7];
>> };
>>
>> #include 
>> extern "C" int
>> _dl_find_object (void *pc1, dl_find_object* result)
>> {
>>result->dlfo_eh_frame = 0;
>>result->dlfo_link_map = 0; // where would this come from?
>>auto eo = elf::get_program()->object_containing_addr(pc1);
>>result->dlfo_map_start = (void*)(eo->base());
>>result->dlfo_map_end = (void*)(eo->end());
>>return 0;
>> }
>>
>> Any suggestions would be welcome.
>>
>> Regards,
>> Waldek
>>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/bf71b9f7-161a-4e17-ba0c-958519051055n%40googlegroups.com.

[osv-dev] Missing _dl_find_object()

[Waldek Kozaczuk]

After upgrading to newest Ubuntu 2022.04 I started seeing following crashes 
of programs using libgcc_s.so:

#0  _dl_find_object (pc1=0x1003f5ca <_Unwind_Resume+58>, 
result=0x20200180) at linux.cc:542
#1  0x100410f6 in _Unwind_Find_FDE ()
#2  0x1003d833 in ?? ()
#3  0x1003ead0 in ?? ()
#4  0x1003f5cb in _Unwind_Resume ()
#5  0x10062daa in ?? ()
#6  0x10075b5c in 
boost::execution_monitor::vexecute(boost::function const&) ()
#7  0x1007f0a9 in boost::unit_test::framework::init(bool (*)(), 
int, char**) ()
#8  0x1009254d in boost::unit_test::unit_test_main(bool (*)(), int, 
char**) ()
#9  0x4039d001 in osv::application::run_main 
(this=0xa0bd8c10) at core/app.cc:416
#10 0x4039d20d in operator() (app=, __closure=0x0) 
at core/app.cc:236
#11 _FUN () at core/app.cc:238
#12 0x403d084a in operator() (__closure=0xa0d57800) at 
libc/pthread.cc:116
#13 std::__invoke_impl&> (__f=...)
at /usr/include/c++/11/bits/invoke.h:61
#14 std::__invoke_r&> (__fn=...)
at /usr/include/c++/11/bits/invoke.h:154
#15 std::_Function_handler >::_M_invoke(const 
std::_Any_data &) (__functor=...) at 
/usr/include/c++/11/bits/std_function.h:290
#16 0x4036b58e in sched::thread::main (this=0x80f6a040) at 
core/sched.cc:1267
#17 sched::thread_main_c (t=0x80f6a040) at 
arch/x64/arch-switch.hh:325
#18 0x402fda03 in thread_main () at arch/x64/entry.S:116

After some digging I discovered this new function was added as per this 
patch to libc 
- https://www.mail-archive.com/gcc-patches@gcc.gnu.org/msg275305.html. And 
there s this description:
http://www.gnu.org/software/libc/manual/html_node/Dynamic-Linker-Introspection.html

It is interesting that libgcc_s.so on pretty new Fedora 35 with newer gcc 
does not need this.

I tried to stub something but this is not enough:

struct dl_find_object
{
  __extension__ unsigned long long int dlfo_flags;
  void *dlfo_map_start;/* Beginning of mapping containing address. 
 */
  void *dlfo_map_end;/* End of mapping.  */
  struct link_map *dlfo_link_map;
  void *dlfo_eh_frame;/* Exception handling data of the object.  */
  __extension__ unsigned long long int __dflo_reserved[7];
};

#include 
extern "C" int
_dl_find_object (void *pc1, dl_find_object* result)
{
   result->dlfo_eh_frame = 0;
   result->dlfo_link_map = 0; // where would this come from?
   auto eo = elf::get_program()->object_containing_addr(pc1);
   result->dlfo_map_start = (void*)(eo->base());
   result->dlfo_map_end = (void*)(eo->end());
   return 0;
}

Any suggestions would be welcome.

Regards,
Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/8a1d87f7-4d50-44d7-872a-7a94bc453397n%40googlegroups.com.

[osv-dev] Re: Image not running using capstan

[Waldek Kozaczuk]

Hi,

Which version of capstan are you using? Most likely you have to pass '-f' 
option to expose the port your app is listening to host like in this 
example:

capstan run com.example.word-finder -f 4004:4000

from 
https://github.com/cloudius-systems/capstan/blob/master/Documentation/WalkthroughNodeJS.md.

Regards,
Waldek

On Thursday, April 28, 2022 at 7:45:59 AM UTC-4 SB wrote:

> Hi All,
>
> I am trying to build one package with a simple nodejs app. I already built 
> the image and seems like it is running fine without any errors but I don't 
> see any running instances nor any service is listening on the port 3000. 
> Please find screenshots below:
>
> ubuntu@unikern:~/calc$ cat meta/package.yaml
> name: calculator.app.pkg
> title: Calculator App
> author: Shambo
> created: "2022-04-27T19:05:46Z"
> require:
>   - osv.node-js
> ubuntu@unikern:~/calc$ cat meta/run.yaml
> runtime: native
> config_set:
>
>### This is one configuration set (feel free to rename it).  ###
>
>default:
>   # REQUIRED
>   # Filepath of the NodeJS entrypoint (where server is defined).
>   # Note that package root will correspond to filesystem root (/) in 
> OSv image.
>   # Example value: /libnode.so server.js
>   bootcmd: /libnode.so server.js
>   # OPTIONAL
>   # Environment variables.
>   # A map of environment variables to be set when unikernel is run.
>   # Example value:  env:
>   # PORT: 3000
>   # HOSTNAME: www.myserver.org
>
>   #env:
>   #   PORT: 3000
>
> # Add as many named configurations as you need
>
> # OPTIONAL
> # What config_set should be used as default.
> # This value can be overwritten with --runconfig argument.
> config_set_default: default
>
> #
>
> Screenshot of running instance without error and output of capstan 
> instances 
>
> OS details:
> ubuntu@unikern:~/calc$ cat /etc/os-release
> PRETTY_NAME="Ubuntu 21.10"
> NAME="Ubuntu"
> VERSION_ID="21.10"
> VERSION="21.10 (Impish Indri)"
> VERSION_CODENAME=impish
> ID=ubuntu
> ID_LIKE=debian
> HOME_URL="https://www.ubuntu.com/;
> SUPPORT_URL="https://help.ubuntu.com/;
> BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/;
> PRIVACY_POLICY_URL="
> https://www.ubuntu.com/legal/terms-and-policies/privacy-policy;
> UBUNTU_CODENAME=impish
>
> Please help me here. Thanks.
>
> Cheers,
> SB
>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/5b123cb7-955c-4124-ba01-ead2ab794a1en%40googlegroups.com.

Re: [osv-dev] [COMMIT osv master] mmap: avoid collisions with linear map

[Waldek Kozaczuk]

On Thursday, March 31, 2022 at 8:54:24 AM UTC-4 Nadav Har'El wrote:

> On Thu, Mar 31, 2022 at 6:36 AM Commit Bot  
> wrote:
>
>> From: Waldemar Kozaczuk 
>> Committer: Waldemar Kozaczuk 
>> Branch: master
>>
>> mmap: avoid collisions with linear map
>>
>> This patch enhances virtual memory mapping code to track both
>> linear (linear_map) and non-linear (mmap()) virtual memory mappings.
>>
>> It does so by adding simple struct - vma_range and vma_range_set
>> collection to store all mappings. It then modifies mmu::find_hole()
>> implementation to use vma_range_set instead of vma_list to find next hole.
>> That way it can avoid collisions described in the issue #1135.
>>
>> Fixes #1135
>>
>> Signed-off-by: Waldemar Kozaczuk 
>>
>> ---
>> diff --git a/core/mmu.cc b/core/mmu.cc
>> --- a/core/mmu.cc
>> +++ b/core/mmu.cc
>> @@ -47,6 +47,17 @@ extern const char text_start[], text_end[];
>>
>>  namespace mmu {
>>
>> +struct vma_range_compare {
>> +bool operator()(const vma_range& a, const vma_range& b) {
>> +return a.start() < b.start();
>> +}
>> +};
>> +
>> +//Set of all vma ranges - both linear and non-linear ones
>> +__attribute__((init_priority((int)init_prio::vma_range_set)))
>> +std::set vma_range_set;
>> +rwlock_t vma_range_set_mutex;
>>
>
> Do we still need both vma_range_set and vma_list? Why?
>
I have actually thought hard about using only one set and one data 
structure to represent both non-linear and linear VMAs but this seemed a 
harder road. The information is quite different and in most places (except 
for find_hole() we either need to access the regular (non-linear) *vma_list* 
or the new *linear_vma_set* (which I added in one of the previous patches). 
The find_hole() is the ONLY place where need information about all ranges 
to avoid a collision. If we put everything in one set then in most places 
when we iterate over/access we would need to differentiate between two 
types and I reasoned it would only complicate things and possibly break 
existing code. So the solution was to add another set - vma_range_set (a 
superset) where we register ALL ranges (only start address and size) which 
we then use in the find_hole().

>  
>
>> +
>>  struct linear_vma_compare {
>>  bool operator()(const linear_vma* a, const linear_vma* b) {
>>  return a->_virt_addr < b->_virt_addr;
>> @@ -66,6 +77,9 @@ class vma_compare {
>>  }
>>  };
>>
>> +constexpr uintptr_t lower_vma_limit = 0x0;
>> +constexpr uintptr_t upper_vma_limit = 0x8000;
>> +
>>  typedef boost::intrusive::set>bi::compare,
>>bi::member_hook> @@ -78,9 +92,15 @@ struct vma_list_type : vma_list_base {
>>  vma_list_type() {
>>  // insert markers for the edges of allocatable area
>>  // simplifies searches 
>>
> -insert(*new anon_vma(addr_range(0, 0), 0, 0));
>> -uintptr_t e = 0x8000;
>> -insert(*new anon_vma(addr_range(e, e), 0, 0));
>> +auto lower_edge = new anon_vma(addr_range(lower_vma_limit, 
>> lower_vma_limit), 0, 0);
>> +insert(*lower_edge);
>> +auto upper_edge = new anon_vma(addr_range(upper_vma_limit, 
>> upper_vma_limit), 0, 0);
>> +insert(*upper_edge);
>> +
>> +WITH_LOCK(vma_range_set_mutex.for_write()) {
>> +vma_range_set.insert(vma_range(lower_edge));
>> +vma_range_set.insert(vma_range(upper_edge));
>> +}
>>  }
>>  };
>>
>> @@ -954,27 +974,41 @@ static error protect(const void *addr, size_t size, 
>> unsigned int perm)
>>  return no_error();
>>  }
>>
>> +class vma_range_addr_compare {
>> +public:
>> +bool operator()(const vma_range& x, uintptr_t y) const { return 
>> x.start() < y; }
>> +bool operator()(uintptr_t x, const vma_range& y) const { return x < 
>> y.start(); }
>> +};
>> +
>>  uintptr_t find_hole(uintptr_t start, uintptr_t size)
>>  {
>>  bool small = size < huge_page_size;
>>  uintptr_t good_enough = 0;
>>
>> -// FIXME: use lower_bound or something
>> -auto p = vma_list.begin();
>> +SCOPE_LOCK(vma_range_set_mutex.for_read());
>>
>
> Unfortunately I'm very rusy with this area of the code, so I have a 
> question that might be stupid:
>
> How come we didn't need a mutex here before, but need one now?
> Could adding a mutex here be a problem - namely that maybe we need to call 
> some of this code from non-preemptable code and this will now break?
>

Yeah, it is a good point. The intention was to guard against concurrent 
calls from linear_map() which inserts to both linear_vma_set and 
vma_range_set and find_hole(). Maybe it is not necessary as this code is 
called from allocate() which is called with the 
SCOPE_LOCK(vma_list_mutex.for_write()) from both map_anon() and map_file(). 
Could these be called from non-preemptable code?

Also I have added this code to many places in core/mmu.cc (other patch_:

WITH_LOCK(vma_range_set_mutex.for_write()) {
   

[osv-dev] Re: Crash in the FreeBSD networking stack code caused by likely bug in reference counting logic

[Waldek Kozaczuk]

BTW I have run more tests to see any performance gains and I failed to 
notice any significant differences. So maybe atomics are not that expensive 
in this case. But I think my proposal still stands.

I also noticed that if I bump the number of iterations to over 100K at some 
point the client fails to connect with EADDRNOTAVAIL on both x64 and 
aarch64:

in_pcb.cc: EADDRNOTAVAIL: 0, last - first:55535
tst-tcp1: client: connect() failed after i=111840!
[tst-tcp1: client]: Address not available

I pinpointed it to this fragment of in_pcb_lport() in 
bsd/sys/netinet/in_pcb.cc (see the lines 605 and 606 in bold):

598 if (dorandom)
 599 *lastport = first + (arc4random() % (last - first));
 600 
 601 count = last - first;
 602 
 603 do {
 604 if (count-- < 0)/* completely used? */ {
 
*605 printf("in_pcb.cc: EADDRNOTAVAIL: 0, last - 
first:%ld\n", last - first); 606 return 
(EADDRNOTAVAIL);*
 607 }
 608 ++*lastport;
 609 if (*lastport < first || *lastport > last)
 610 *lastport = first;
 611 lport = htons(*lastport);
 612 
 613 #ifdef INET6
 614 if ((inp->inp_vflag & INP_IPV6) != 0)
 615 tmpinp = in6_pcblookup_local(pcbinfo,
 616 >in6p_laddr, lport, lookupflags, 
cred);
 617 #endif
 618 #if defined(INET) && defined(INET6)
 619 else
 620 #endif
 621 #ifdef INET
 622 tmpinp = in_pcblookup_local(pcbinfo, laddr,
 623 lport, lookupflags, cred);
 624 #endif
 625 } while (tmpinp != NULL);
 626 
 627 #ifdef INET
 628 if ((inp->inp_vflag & (INP_IPV4|INP_IPV6)) == INP_IPV4)
 629 laddrp->s_addr = laddr.s_addr;
 630 #endif
 631 *lportp = lport;
 632 
 633 return (0);
 634 }

I wonder if we somehow ran out of available ports because maybe we do not 
clean/make them available fast enough?

On Monday, March 28, 2022 at 9:09:04 PM UTC-4 Waldek Kozaczuk wrote:

> Hi,
>
> As I was working on a patch to address the issue 
> https://github.com/cloudius-systems/osv/issues/1087, I discovered that 
> occasionally it would crash or hang while running a test 
> tst-bsd-tcp1-zsndrcv.cc. The crash would look like this:
>
> page fault outside application, addr: 0x
> [registers]
> PC: 0x000fc011ca5c 
> X00: 0x0012 X01: 0x90004088f4a0 X02: 0x
> X03: 0x800040f668c0 X04: 0xa00040e778e8 X05: 0x800040f66950
> X06: 0x X07: 0x X08: 0x
> X09: 0x X10: 0x X11: 0x
> X12: 0x4000 X13: 0x X14: 0x00b0
> X15: 0x X16: 0x0001 X17: 0x0001
> X18: 0x55697ee1 X19: 0xa0004088e800 X20: 0x800040f66840
> X21: 0x90004088f000 X22: 0x000fc0672838 X23: 0x
> X24: 0x X25: 0x90004088f4a0 X26: 0x0001
> X27: 0x X28: 0x0002 X29: 0x800040f667a0
> X30: 0x000fc011ca2c SP:  0x800040f667a0 ESR: 0x9606
> PSTATE: 0x2345
> Aborted
>
> [backtrace]
> 0x000fc014c2e4 
> 0x000fc017e4c8 
> 0x000fc017e254 
> 0x000fc01234dc 
> 0x000fc0124a80 
> 0x000fc0145920 
> 0x000fc0145afc 
> 0x000fc0100bb8 
> 0x000fc010de7c 
> 0x000fc010233c 
> 0x100376a0 
> 0x100377e0 
> 0x000fc0257440 
> 0x000fc017d784 
>
> -
> (gdb) bt
> #0  processor::wfi () at arch/aarch64/processor.hh:52
> #1  processor::halt_no_interrupts () at arch/aarch64/processor.hh:52
> #2  arch::halt_no_interrupts () at arch/aarch64/arch.hh:48
> #3  osv::halt () at arch/aarch64/power.cc:24
> #4  0x000fc00e9fcc in abort (fmt=fmt@entry=0xfc0481190 "Aborted\n") at 
> runtime.cc:142
> #5  0x000fc00a0654 in abort () at runtime.cc:106
> #6  0x000fc014c2e8 in mmu::vm_sigsegv (ef=0x800040f66680, addr=0) 
> at core/mmu.cc:1420
> #7  mmu::vm_sigsegv (ef=0x800040f66680, addr=0) at core/mmu.cc:1414
> #8  mmu::vm_fault (addr=0, addr@entry=1, ef=ef@entry=0x800040f66680) 
> at core/mmu.cc:1442
> #9  0x000fc017e4cc in page_fault (ef=0x800040f66680) at 
> arch/aarch64/mmu.cc:50
> #10 
> #11 0x000fc011ca5c in ifa_ifwithnet 
> (addr=addr@entry=0x800040f66840, ignore_ptp=0) at bsd/sys/net/if.cc:1434
> #12 0x000fc01234e0 in in_pcbladdr (inp=inp@entry=0xa000411c1600, 
> faddr=faddr@entry=0x800040f669d8, laddr=laddr@entry=0x800040f669d0, 

[osv-dev] Crash in the FreeBSD networking stack code caused by likely bug in reference counting logic

[Waldek Kozaczuk]

Hi,

As I was working on a patch to address the 
issue https://github.com/cloudius-systems/osv/issues/1087, I discovered 
that occasionally it would crash or hang while running a test 
tst-bsd-tcp1-zsndrcv.cc. The crash would look like this:

page fault outside application, addr: 0x
[registers]
PC: 0x000fc011ca5c 
X00: 0x0012 X01: 0x90004088f4a0 X02: 0x
X03: 0x800040f668c0 X04: 0xa00040e778e8 X05: 0x800040f66950
X06: 0x X07: 0x X08: 0x
X09: 0x X10: 0x X11: 0x
X12: 0x4000 X13: 0x X14: 0x00b0
X15: 0x X16: 0x0001 X17: 0x0001
X18: 0x55697ee1 X19: 0xa0004088e800 X20: 0x800040f66840
X21: 0x90004088f000 X22: 0x000fc0672838 X23: 0x
X24: 0x X25: 0x90004088f4a0 X26: 0x0001
X27: 0x X28: 0x0002 X29: 0x800040f667a0
X30: 0x000fc011ca2c SP:  0x800040f667a0 ESR: 0x9606
PSTATE: 0x2345
Aborted

[backtrace]
0x000fc014c2e4 
0x000fc017e4c8 
0x000fc017e254 
0x000fc01234dc 
0x000fc0124a80 
0x000fc0145920 
0x000fc0145afc 
0x000fc0100bb8 
0x000fc010de7c 
0x000fc010233c 
0x100376a0 
0x100377e0 
0x000fc0257440 
0x000fc017d784 

-
(gdb) bt
#0  processor::wfi () at arch/aarch64/processor.hh:52
#1  processor::halt_no_interrupts () at arch/aarch64/processor.hh:52
#2  arch::halt_no_interrupts () at arch/aarch64/arch.hh:48
#3  osv::halt () at arch/aarch64/power.cc:24
#4  0x000fc00e9fcc in abort (fmt=fmt@entry=0xfc0481190 "Aborted\n") at 
runtime.cc:142
#5  0x000fc00a0654 in abort () at runtime.cc:106
#6  0x000fc014c2e8 in mmu::vm_sigsegv (ef=0x800040f66680, addr=0) 
at core/mmu.cc:1420
#7  mmu::vm_sigsegv (ef=0x800040f66680, addr=0) at core/mmu.cc:1414
#8  mmu::vm_fault (addr=0, addr@entry=1, ef=ef@entry=0x800040f66680) at 
core/mmu.cc:1442
#9  0x000fc017e4cc in page_fault (ef=0x800040f66680) at 
arch/aarch64/mmu.cc:50
#10 
#11 0x000fc011ca5c in ifa_ifwithnet 
(addr=addr@entry=0x800040f66840, ignore_ptp=0) at bsd/sys/net/if.cc:1434
#12 0x000fc01234e0 in in_pcbladdr (inp=inp@entry=0xa000411c1600, 
faddr=faddr@entry=0x800040f669d8, laddr=laddr@entry=0x800040f669d0, 
cred=0x0)
at bsd/sys/netinet/in_pcb.cc:969
#13 0x000fc0124a84 in in_pcbconnect_setup 
(inp=inp@entry=0xa000411c1600, nam=nam@entry=0xa00040b046d0, 
laddrp=laddrp@entry=0x800040f66a30, 
lportp=lportp@entry=0x800040f66a2e, 
faddrp=faddrp@entry=0xa000411c1664, 
fportp=fportp@entry=0xa000411c1654, 
oinpp=oinpp@entry=0x800040f66a38, 
cred=cred@entry=0x0) at bsd/sys/netinet/in_pcb.cc:1056
#14 0x000fc0145924 in tcp_connect (tp=tp@entry=0xa00040e5e400, 
nam=nam@entry=0xa00040b046d0, td=) at 
bsd/sys/netinet/tcp_usrreq.cc:1089
#15 0x000fc0145b00 in tcp_usr_connect (td=, 
nam=0xa00040b046d0, so=) at 
bsd/sys/netinet/tcp_usrreq.cc:463
#16 tcp_usr_connect (so=, nam=0xa00040b046d0, 
td=) at bsd/sys/netinet/tcp_usrreq.cc:436
#17 0x000fc0100bbc in kern_connect (fd=fd@entry=4, 
sa=0xa00040b046d0) at bsd/sys/kern/uipc_syscalls.cc:377
#18 0x000fc010de80 in linux_connect (s=s@entry=4, 
name=name@entry=0x800040f66be8, namelen=namelen@entry=16) at 
bsd/sys/compat/linux/linux_socket.cc:684
#19 0x000fc0102340 in connect (fd=fd@entry=4, 
addr=addr@entry=0x800040f66be8, len=len@entry=16) at 
bsd/sys/kern/uipc_syscalls_wrap.cc:128
#20 0x100376a4 in test_bsd_tcp1::tcp_client (this=0x201ffa98) 
at /home/wkozaczuk/projects/osv/tests/tst-bsd-tcp1-zrcv.cc:169
#21 0x100377e4 in test_bsd_tcp1::run()::{lambda()#2}::operator()() 
const (__closure=)
at /home/wkozaczuk/projects/osv/tests/tst-bsd-tcp1-zrcv.cc:230
#22 std::__invoke_impl(std::__invoke_other, 
test_bsd_tcp1::run()::{lambda()#2}&) (__f=...)
at /usr/include/c++/10/bits/invoke.h:60
#23 std::__invoke_r(std::__is_invocable&&, 
(test_bsd_tcp1::run()::{lambda()#2}&)...) (__fn=...)
at /usr/include/c++/10/bits/invoke.h:153
#24 std::_Function_handler::_M_invoke(std::_Any_data const&) 
(__functor=...)
at /usr/include/c++/10/bits/std_function.h:291
#25 0x000fc0257444 in sched::thread::main (this=0xa00040c72e00) at 
core/sched.cc:1267
#26 sched::thread_main_c (t=0xa00040c72e00) at 
arch/aarch64/arch-switch.hh:162
#27 0x000fc017d788 in thread_main () at arch/aarch64/entry.S:116
(gdb) frame 11
#11 0x000fc011ca5c in ifa_ifwithnet 
(addr=addr@entry=0x800040f66840, ignore_ptp=0) at bsd/sys/net/if.cc:1434
1434if (ifa->ifa_addr->sa_family != af)
(gdb) p ifa->ifa_addr
$1 = (bsd_sockaddr *) 0x80004088e800
(gdb) p ifa->ifa_addr->sa_family
$2 = 232 '\350'
(gdb) p af
$3 = 2

As you can see the fault 

[osv-dev] Proposal to change virtual memory layout

[Waldek Kozaczuk]

In the last couple of weeks I have been working on various issues related 
to aarch64 port. I have managed to make good progress and I will be sending 
new patches soon.

Two issues had to do with making Java run on aarch64 
- https://github.com/cloudius-systems/osv/issues/1145 
and https://github.com/cloudius-systems/osv/issues/1157.  After exchanging 
some emails on the openjdk emailing list and researching this problem, I 
finally discovered that the problem only happens when JIT is enabled and is 
caused by the fact that the JIT compiler generates machine code to access 
arbitrary address in memory in a way that assumes all addresses are 48 
bits, meaning first 16 bits are 0. And here are the details:

"Once I got hold of the JDK debuginfo files and identified the patching 
code - MacroAssembler::pd_patch_instruction(), I was able to put a 
breakpoint in it and see something very revealing:

#0  MacroAssembler::pd_patch_instruction_size (branch=0x2879465c 
"\351\377\237\322\351\377\277\362\351\377\337\362\n\243\352\227\037",
target=0xa00042c862e0 "\020zB") at 
src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp:75
#1  0x10bc13cc in MacroAssembler::pd_patch_instruction (file=0x0, 
line=0, target=0xa00042c862e0 "\020zB", branch=)
at src/hotspot/cpu/aarch64/macroAssembler_aarch64.hpp:626
#2  NativeMovConstReg::set_data (this=0x2879465c, x=-105551995837728) 
at src/hotspot/cpu/aarch64/nativeInst_aarch64.cpp:262
#3  0x10850bd0 in CompiledIC::set_ic_destination_and_value 
(value=0xa00042c862e0,
entry_point=0x2823d290 
"(\b@\271\b\001]\322*\005@\371\037\001\n\353,\001", , this=)
at src/hotspot/share/code/compiledIC.hpp:193
#4  ICStub::finalize (this=) at 
src/hotspot/share/code/icBuffer.cpp:91
#5  ICStubInterface::finalize (this=, self=) 
at src/hotspot/share/code/icBuffer.cpp:43
#6  0x10e30958 in StubQueue::stub_finalize 
(this=0xa00041555300, s=) at 
src/hotspot/share/code/stubs.hpp:168
#7  StubQueue::remove_first (this=0xa00041555300) at 
src/hotspot/share/code/stubs.cpp:175


The corresponding crash value of X9 was this: 

0x*a00042c862e0*

vs the target argument of pd_patch_instruction() (see above in the 
backtrace):

0x*a00042c862e0*

Now given this comment:

// Move a constant pointer into r.  In AArch64 mode the virtual
// address space is 48 bits in size, so we only need three
// instructions to create a patchable instruction sequence that can
// reach anywhere.

and this fragment of pd_patch_instruction() - 
https://github.com/openjdk/jdk17u/blob/6f0f42630eac1febf562062afc523fdf3d2a920a/src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp#L152-L161

it seems that the code to load x8 register with an address gets patched 
with 0xa00042c862e0 instead of 0xa00042c862e0. It is interesting 
that this assert - 
https://github.com/openjdk/jdk17u/blob/6f0f42630eac1febf562062afc523fdf3d2a920a/src/hotspot/cpu/aarch64/macroAssembler_aarch64.cpp#L77
 
- does not get hit.

The bottom line is that the valid address 0xa00042c862e0 gets truncated 
to 0xa00042c862e0 I guess based on the assumption that in Linux all 
userspace addresses are 48-bits long (see 
https://www.kernel.org/doc/html/latest/arm64/memory.html). In OSv 
unikernel, there is no separation between user space and kernel space, and 
it happens that addresses returned by malloc fall into this range:

0xa000 - 0xafff

So I guess the only solution to fix it on the OSv side would be to tweak 
its virtual memory mapping for mallocs and make sure it never uses virtual 
addresses > 48-bits."

Currently OSv maps this part of virtual memory like so:
-- 0x  8000   phys_mem --\ | | |- Main Area - 16T -- 0x 
 9000   --X | | |- Page Area - 16T -- 0x  a000  
 --X | | |- Mempool Area - 16T -- 0x  b000   --X | | |- 
Debug Area - 80T -- 0x     --/

I wonder if this was arbitrary choice made early in OSv design and there 
was some good reason for it.

Could this be changed to this:

-- 0x  8000   phys_mem --\ | | |- Main Area - 16T -- 0x 
 9000   --X | | |- Page Area - 16T -- 0x  a000  
 --X | | |- Mempool Area - 16T -- 0x  b000   --X | | |- 
Debug Area - 80T -- 0x     --/

I did manage to hack the code for aarch64 and it seems to be working.

I also found a similar case for x86_64. The library rapidjson used by 
dotnet uses last 16 bits of addresses to compress some info into it - 
see https://github.com/Tencent/rapidjson/pull/546#issue-133623698.

Now going forward I think Linux will extend the userspace addresses 
eventually from 48 bits to 56 bits 
(see https://en.wikipedia.org/wiki/Intel_5-level_paging) or higher. And 
dotnet actually made a fix to disable this high 16-bits hack. But given 
there are Linux apps that may assume that addresses are 48-bit and take 
advantage of 

Re: [osv-dev] [PATCH 1/2] add new C-wrappers to expose module API

[Waldek Kozaczuk]

Thanks for your review. I have addressed the first two of your suggestions.

On Wednesday, January 19, 2022 at 3:47:09 AM UTC-5 Nadav Har'El wrote:

>
> Looks good. Just a couple of nitpicks below.
>
> On Mon, Jan 17, 2022 at 7:55 AM Waldemar Kozaczuk  
> wrote:
>
>> The commit af2d371a61f6ab1eb5a066a0c3e93230faf6611c introduced ability
>> to build OSv kernel with most symbols but subset of glibc hidden.
>> The regular Linux glibc apps should run fine on such kernel, but
>> unfortunately many unit tests and various internal OSv apps (so called
>> modules) do not as they have been coded to use many internal API
>> symbols. One such example is httpserver monitoring api module that
>> exposes various monitoring API REST endpoints. 
>>
>> At some point XLAB introduced C-wrappers API made of single C-style
>> osv_get_all_app_threads() functions. This patch enhances the C-wrappers 
>> API
>> by adding 9 more functions intended to be used by httpserver monitoring
>> api module.
>>
>> Please note that new C-style API will open up access to relevant 
>> functionality
>> to new apps/modules implemented in languages different than C++.
>>
>> Signed-off-by: Waldemar Kozaczuk 
>> ---
>>  core/osv_c_wrappers.cc   | 121 +++
>>  include/osv/export.h |   3 +
>>  include/osv/osv_c_wrappers.h | 105 +-
>>  3 files changed, 228 insertions(+), 1 deletion(-)
>>
>> diff --git a/core/osv_c_wrappers.cc b/core/osv_c_wrappers.cc
>> index 137f2c6f..dbda0613 100644
>> --- a/core/osv_c_wrappers.cc
>> +++ b/core/osv_c_wrappers.cc
>> @@ -1,12 +1,27 @@
>> +/*
>> + * Copyright (C) 2022 Waldemar Kozaczuk
>> + * Copyright (C) 2016 XLAB, d.o.o.
>> + *
>> + * This work is open source software, licensed under the terms of the
>> + * BSD license as described in the LICENSE file in the top-level 
>> directory.
>> + */
>>
>>  #include 
>> +#include 
>>  #include 
>>  #include 
>>  #include 
>> +#include 
>> +#include 
>> +#include 
>> +#include 
>> +#include 
>> +#include 
>>
>>  using namespace osv;
>>  using namespace sched;
>>
>> +extern "C" OSV_MODULE_API
>>  int osv_get_all_app_threads(pid_t tid, pid_t** tid_arr, size_t *len) {
>>  thread* app_thread = tid==0? thread::current(): 
>> thread::find_by_id(tid);
>>  if (app_thread == nullptr) {
>> @@ -28,3 +43,109 @@ int osv_get_all_app_threads(pid_t tid, pid_t** 
>> tid_arr, size_t *len) {
>>  }
>>  return 0;
>>  }
>> +
>> +static void free_threads_names(std::vector ) {
>> +for (auto  : threads) {
>> +if (t.name) {
>> +free(t.name);
>> +}
>> +}
>> +}
>> +
>> +static char* str_to_c_str(const std::string& str) {
>> +auto len = str.size();
>> +char *buf = static_cast(malloc(len + 1)); // This will be 
>> free()-ed in C world
>> +if (buf) {
>> +std::copy(str.begin(), str.end(), buf);
>> +buf[len] = '\0';
>> +return buf;
>> +} else {
>> +return nullptr;
>>
>
> Nitpick: you can simplify this by just returning "buf" unconditionally:
>
> if (buf) {
> std::copy(str.begin(), str.end(), buf);
> buf[len] = '\0';
> }
> return buf;
>
>  
>
>> +}
>> +}
>> +
>> +extern "C" OSV_MODULE_API
>> +int osv_get_all_threads(osv_thread** thread_arr, size_t *len) {
>> +using namespace std::chrono;
>> +std::vector threads;
>> +
>> +osv_thread thread;
>> +bool str_copy_error = false;
>> +sched::with_all_threads([&](sched::thread ) {
>> +thread.id = t.id();
>> +auto tcpu = t.tcpu();
>> +thread.cpu_id = tcpu ? tcpu->id : -1;
>> +thread.cpu_ms = 
>> duration_cast(t.thread_clock()).count();
>> +thread.switches = t.stat_switches.get();
>> +thread.migrations = t.stat_migrations.get();
>> +thread.preemptions = t.stat_preemptions.get();
>> +thread.name = str_to_c_str(t.name());
>> +if (!thread.name) {
>> +str_copy_error = true;
>> +}
>> +thread.priority = t.priority();
>> +thread.stack_size = t.get_stack_info().size;
>> +thread.status = 
>> static_cast(static_cast(t.get_status()));
>> +threads.push_back(thread);
>> +});
>> +
>> +if (str_copy_error) {
>> +goto error;
>> +}
>> +
>> +*thread_arr = (osv_thread*)malloc(threads.size()*sizeof(osv_thread));
>> +if (*thread_arr == nullptr) {
>> +goto error;
>> +}
>> +
>> +std::copy(threads.begin(), threads.end(), *thread_arr);
>> +*len = threads.size();
>> +return 0;
>> +
>> +error:
>> +free_threads_names(threads);
>> +*len = 0;
>> +return ENOMEM;
>> +}
>> +
>> +extern "C" OSV_MODULE_API
>> +char *osv_version() {
>> +return str_to_c_str(osv::version());
>
> +}
>> +
>> +extern "C" OSV_MODULE_API
>> +char *osv_cmdline() {
>> +return str_to_c_str(osv::getcmdline());
>> +}
>> +
>> +extern "C" OSV_MODULE_API
>> +char *osv_hypervisor_name() {
>> +return 

[osv-dev] Re: [PATCH] lua: change build process to download artifacts from lua binaries

[Waldek Kozaczuk]

I have tested it on Fedora 33 and Ubuntu 21.10. It would be nice to see if 
it works on newer versions of Fedora - 34,35 and possibly some older ones.

On Thursday, January 13, 2022 at 9:46:03 PM UTC-5 Waldek Kozaczuk wrote:

> As the issue #1166 explains, building lua module does not work on Fedora
> 33 and up. In short, lua module depends on a specific version 5.3 of lua
> interpreter, library and header files which may not be available on
> given version of Fedora and which may be the case with other Linux
> distributions.
>
> The issue #1166 describes at least three alternative solutions to the
> problem, but this patch solves it by changing the makefile to
> download the lua interpreter (lua executable), library and header files
> from a well maintained repository - LuaBinaries at 
> http://luabinaries.sourceforge.net/
> - logically in a similar way we download luarocks. The LuaBinaries has
> been in place since 2005 so there is good chance we can keep relying
> on it in foreseeable future.
>
> At the moment the makefile downloads fairly recent version 5.3.6 of lua 
> binaries
> which are compatible with the versions of lua modules (like socket, etc)
> and luarocks. In future we may upgrade all elements needed to build the
> module as we see fit.
>
> As the result of this patch, lua module should in theory be build-able on 
> any
> Linux distribution and version. in reality with newer versions of gcc
> one can imagine that lua modules themselves will stop compiling at which
> point we will need to upgrade those and possibly lua and luarocks itself.
> Also please note that lua module no longer depends on version of lua
> installed on host if any.
>
> Fixes #1166
>
> Signed-off-by: Waldemar Kozaczuk 
> ---
> modules/cli/Makefile | 15 +--
> modules/lua/Makefile | 63 +---
> 2 files changed, 53 insertions(+), 25 deletions(-)
>
> diff --git a/modules/cli/Makefile b/modules/cli/Makefile
> index 8a3b037e..ab648879 100644
> --- a/modules/cli/Makefile
> +++ b/modules/cli/Makefile
> @@ -1,21 +1,18 @@
> -LUA_LIB = $(shell pkg-config --libs lua53 2>/dev/null || pkg-config 
> --libs lua || echo 'ERROR: Could not find lua, please run 
> ./scripts/setup.py')
> -LUA_INCLUDES = $(shell pkg-config --cflags lua53 2>/dev/null || 
> pkg-config --cflags lua || echo 'ERROR: Could not find lua, please run 
> ./scripts/setup.py')
> +SRC = $(shell readlink -f ../..)
> +
> +LUA_DIR = $(SRC)/modules/lua/upstream/lua5.3
>
> CC=gcc
> -CFLAGS=-O2 -g -Wall -std=gnu99
> -LIBS=-ledit -ltinfo $(LUA_LIB)
> +CFLAGS=-O2 -g -Wall -std=gnu99 -I $(LUA_DIR)/include
> +LIBS=-ledit -ltinfo -ldl -L$(LUA_DIR) -llua53
> SRCS=cli.c
> MAIN=cli
>
> -INCLUDES = $(LUA_INCLUDES)
> -
> -SRC = $(shell readlink -f ../..)
> -
> module: $(MAIN)
> $(SRC)/scripts/manifest_from_host.sh $(MAIN) > usr.manifest
>
> $(MAIN): $(SRCS)
> - $(CC) $(CFLAGS) $(INCLUDES) $^ -fPIC -pie -o $@ $(LIBS)
> + $(CC) $(CFLAGS) $^ -fPIC -pie -o $@ $(LIBS)
>
> rpm: $(MAIN)
> make -C rpmbuild
> diff --git a/modules/lua/Makefile b/modules/lua/Makefile
> index e48791af..a2412894 100644
> --- a/modules/lua/Makefile
> +++ b/modules/lua/Makefile
> @@ -1,62 +1,93 @@
> SRC = $(shell readlink -f ../..)
> +
> +# This makefile orchestrates building some key lua modules used by the 
> OSv cli
> +# module. Please note that both lua binaries, header files and luarocks 
> are
> +# downloaded from internet and lua artifacts if installed on the host are 
> not used.
> +# This should make maintenance of lua module much less painful as 
> regardless
> +# of the Linux distribution and version it will use lua 5.3 and luarocks 
> 3.1.1
> +# until we specifically upgrade them by modifying this makefile.
> +
> +LUA=lua5.3
> +LUA_DIR=upstream/$(LUA)
> LUA_ROCKS=upstream/luarocks-3.1.1-linux-x86_64/luarocks
> +
> MODULES_DIR=install/lua_modules
> +LUA_ROCKS_INSTALL_MODULE := $(LUA_ROCKS) --lua-dir=$(LUA_DIR) install 
> --no-doc --tree $(MODULES_DIR)
> +
> LDIR=install/lua_modules/lib/lua/5.3
> CDIR=install/lua_modules/share/lua/5.3
>
> +# Set LUAROCKS_CONFIG to make luarocks use lua binaries downloaded in 
> upstream/lua5.3
> +export LUAROCKS_CONFIG=$(SRC)/modules/lua/upstream/config.lua
> +
> # List of Lua modules, each module has its own target
> LUA_MODULES=LuaSocket LuaJSON Lua_stdlib LuaFileSystem LuaPath LuaSec
>
> -LUA_LIBRARY := $(shell ldconfig -p | grep -Po "liblua*.5\.3.so.0" | head 
> -1)
> -ifndef LUA_LIBRARY
> - LUA_LIBRARY := $(shell ldconfig -p | grep -Po "liblua*.5\.3.so" | head 
> -1)
> -endif
> -
> module: $(LUA_MODULES)
> mkdir -p $(MODULES_DIR)
> - $(S

Re: [osv-dev] tst-tls-pie.so crashes with kernel compiled with most symbols hidden

[Waldek Kozaczuk]

Also from this 
- https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html:

"
constructor
destructor
constructor (priority)
destructor (priority)
The constructor attribute causes the function to be called automatically 
before execution enters main (). Similarly, the destructor attribute causes 
the function to be called automatically after main () completes or exit () 
is called. Functions with these attributes are useful for initializing data 
that is used implicitly during the execution of the program.

On some targets, the attributes also accept an integer argument to specify 
a priority to control the order in which constructor and destructor 
functions are run. A constructor with a smaller priority number runs before 
a constructor with a larger priority number; the opposite relationship 
holds for destructors. Note that priorities 0-100 are reserved. So, if you 
have a constructor that allocates a resource and a destructor that 
deallocates the same resource, both functions typically have the same 
priority. The priorities for constructor and destructor functions are the 
same as those specified for namespace-scope C++ objects (see C++ 
Attributes). *However, at present, the order in which constructors for C++ 
objects with static storage duration and functions decorated with attribute 
constructor are invoked is unspecified.* In mixed declarations, attribute 
init_priority can be used to impose a specific ordering.

Using the argument forms of the constructor and destructor attributes on 
targets where the feature is not supported is rejected with an error."

On Thursday, January 6, 2022 at 1:48:42 PM UTC-5 Waldek Kozaczuk wrote:

> I think I have all the answers. See below.
>
> On Thursday, January 6, 2022 at 4:55:25 AM UTC-5 Nadav Har'El wrote:
>
>> On Thu, Jan 6, 2022 at 6:56 AM Waldek Kozaczuk  
>> wrote:
>>
>>> Hi,
>>>
>>> When one compiles kernel with most symbols hidden (which also hides 
>>> completely STD C++ library), the tst-tls-pie.so crashes like so:
>>>
>>> #0  0x402da5b2 in processor::cli_hlt () at 
>>> arch/x64/processor.hh:247
>>> #1  arch::halt_no_interrupts () at arch/x64/arch.hh:48
>>> #2  osv::halt () at arch/x64/power.cc:26
>>> #3  0x402211a0 in abort (fmt=fmt@entry=0x4049aef3 "Aborted\n") 
>>> at runtime.cc:137
>>> #4  0x402211b2 in abort () at runtime.cc:103
>>> #5  0x4039cade in osv::generate_signal (siginfo=..., 
>>> ef=0x80e87068) at libc/signal.cc:130
>>> #6  0x4039cb9f in osv::handle_mmap_fault 
>>> (addr=addr@entry=18446744073709547520, sig=sig@entry=11, 
>>> ef=ef@entry=0x80e87068)
>>> at libc/signal.cc:145
>>> #7  0x4028709b in mmu::vm_sigsegv (ef=0x80e87068, 
>>> addr=18446744073709547520) at core/mmu.cc:1334
>>> #8  mmu::vm_fault (addr=18446744073709547520, 
>>> addr@entry=18446744073709551592, ef=ef@entry=0x80e87068) at 
>>> core/mmu.cc:1354
>>> #9  0x402d3d90 in page_fault (ef=0x80e87068) at 
>>> arch/x64/mmu.cc:42
>>> #10 
>>> #11 0x10156cf7 in std::ostream::sentry::sentry 
>>> (this=0x20200df0, __os=...)
>>> at 
>>> /usr/src/debug/gcc-10.3.1-1.fc33.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/ostream.tcc:51
>>>
>>
>> Since your kernel does *not* export the C++ library, where did this 
>> symbol come from? Did you build the tests with the host's C++ library?
>> I'll continue reading with the assumption that you did.
>>
>
> It turns out that even though the symbols are hidden and their names do 
> not show up in the stack trace printed by the kernel, there are still in 
> the original elf before stripping so when I connect 
> with gdb pointing to loader.elf the symbol names show up nicely.
>
> readelf -sW build/release/loader-stripped.elf | grep run_init_funcs 
> #NOTHING
>
> readelf -sW build/release/loader.elf | grep run_init_funcs
>  14789: 40297120   488 FUNCLOCAL  DEFAULT4 
> _ZN3elf6object14run_init_funcsEiPPc
>
>
>>  
>>
>>> #12 0x1015743c in std::__ostream_insert>> std::char_traits > (__out=..., __s=0x40303a "PASS", __n=__n@entry=4)
>>> at 
>>> /usr/src/debug/gcc-10.3.1-1.fc33.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/ostream_insert.h:82
>>> #13 0x00401ccf in std::operator<<  > 
>>> (__s=, __out=...)
>>> at /usr/include/c++/10/bits/char_traits.h:371
>>> #14 report (ok=, msg="v7 in init function") at 
>>> /home/w

Re: [osv-dev] tst-tls-pie.so crashes with kernel compiled with most symbols hidden

[Waldek Kozaczuk]

I think I have all the answers. See below.

On Thursday, January 6, 2022 at 4:55:25 AM UTC-5 Nadav Har'El wrote:

> On Thu, Jan 6, 2022 at 6:56 AM Waldek Kozaczuk  wrote:
>
>> Hi,
>>
>> When one compiles kernel with most symbols hidden (which also hides 
>> completely STD C++ library), the tst-tls-pie.so crashes like so:
>>
>> #0  0x402da5b2 in processor::cli_hlt () at 
>> arch/x64/processor.hh:247
>> #1  arch::halt_no_interrupts () at arch/x64/arch.hh:48
>> #2  osv::halt () at arch/x64/power.cc:26
>> #3  0x402211a0 in abort (fmt=fmt@entry=0x4049aef3 "Aborted\n") at 
>> runtime.cc:137
>> #4  0x402211b2 in abort () at runtime.cc:103
>> #5  0x4039cade in osv::generate_signal (siginfo=..., 
>> ef=0x80e87068) at libc/signal.cc:130
>> #6  0x4039cb9f in osv::handle_mmap_fault 
>> (addr=addr@entry=18446744073709547520, sig=sig@entry=11, 
>> ef=ef@entry=0x80e87068)
>> at libc/signal.cc:145
>> #7  0x4028709b in mmu::vm_sigsegv (ef=0x80e87068, 
>> addr=18446744073709547520) at core/mmu.cc:1334
>> #8  mmu::vm_fault (addr=18446744073709547520, 
>> addr@entry=18446744073709551592, ef=ef@entry=0x80e87068) at 
>> core/mmu.cc:1354
>> #9  0x402d3d90 in page_fault (ef=0x80e87068) at 
>> arch/x64/mmu.cc:42
>> #10 
>> #11 0x10156cf7 in std::ostream::sentry::sentry 
>> (this=0x20200df0, __os=...)
>> at 
>> /usr/src/debug/gcc-10.3.1-1.fc33.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/ostream.tcc:51
>>
>
> Since your kernel does *not* export the C++ library, where did this symbol 
> come from? Did you build the tests with the host's C++ library?
> I'll continue reading with the assumption that you did.
>

It turns out that even though the symbols are hidden and their names do not 
show up in the stack trace printed by the kernel, there are still in the 
original elf before stripping so when I connect 
with gdb pointing to loader.elf the symbol names show up nicely.

readelf -sW build/release/loader-stripped.elf | grep run_init_funcs #NOTHING

readelf -sW build/release/loader.elf | grep run_init_funcs
 14789: 40297120   488 FUNCLOCAL  DEFAULT4 
_ZN3elf6object14run_init_funcsEiPPc


>  
>
>> #12 0x1015743c in std::__ostream_insert> std::char_traits > (__out=..., __s=0x40303a "PASS", __n=__n@entry=4)
>> at 
>> /usr/src/debug/gcc-10.3.1-1.fc33.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/ostream_insert.h:82
>> #13 0x00401ccf in std::operator<<  > 
>> (__s=, __out=...)
>> at /usr/include/c++/10/bits/char_traits.h:371
>> #14 report (ok=, msg="v7 in init function") at 
>> /home/wkozaczuk/projects/osv-true-master/tests/tst-tls.cc:57
>> #15 0x004014b3 in before_main () at 
>> /home/wkozaczuk/projects/osv-true-master/tests/tst-tls.cc:127
>> #16 0x402972c5 in elf::object::run_init_funcs 
>> (this=0xa094a600, argc=argc@entry=1, 
>> argv=argv@entry=0xa094a400)
>> at core/elf.cc:1178
>> #17 0x40298a0b in elf::program::init_library (this=> out>, argc=1, argv=0xa094a400) at core/elf.cc:1500
>> #18 0x4020c239 in osv::application::main 
>> (this=0xa094cc10) at core/app.cc:319
>> #19 0x40365cb9 in operator() (app=, __closure=0x0) 
>> at core/app.cc:236
>> #20 _FUN () at core/app.cc:238
>> #21 0x40398a66 in operator() (__closure=0xa0abda00) at 
>> libc/pthread.cc:116
>> #22 std::__invoke_impl> (*)(void*), void*, sigset_t, const 
>> pthread_private::thread_attr*)::&> (__f=...) at 
>> /usr/include/c++/10/bits/invoke.h:60
>> #23 std::__invoke_r> (*)(void*), void*, sigset_t, const 
>> pthread_private::thread_attr*)::&> (__fn=...) at 
>> /usr/include/c++/10/bits/invoke.h:153
>> #24 std::_Function_handler> pthread_private::pthread::pthread(void* (*)(void*), void*, sigset_t, const 
>> pthread_private::thread_attr*):: >::_M_invoke(const 
>> std::_Any_data &) (__functor=...) at 
>> /usr/include/c++/10/bits/std_function.h:291
>> #25 0x403391ea in sched::thread::main (this=0x80e82040) 
>> at core/sched.cc:1267
>> #26 sched::thread_main_c (t=0x80e82040) at 
>> arch/x64/arch-switch.hh:325
>> #27 0x402d3b33 in thread_main () at arch/x64/entry.S:116
>>
>> I have been researching this for a bit and realized that when I change 
>> the report() function not use std::cout but instead use pri

[osv-dev] tst-tls-pie.so crashes with kernel compiled with most symbols hidden

[Waldek Kozaczuk]

Hi,

When one compiles kernel with most symbols hidden (which also hides 
completely STD C++ library), the tst-tls-pie.so crashes like so:

#0  0x402da5b2 in processor::cli_hlt () at arch/x64/processor.hh:247
#1  arch::halt_no_interrupts () at arch/x64/arch.hh:48
#2  osv::halt () at arch/x64/power.cc:26
#3  0x402211a0 in abort (fmt=fmt@entry=0x4049aef3 "Aborted\n") at 
runtime.cc:137
#4  0x402211b2 in abort () at runtime.cc:103
#5  0x4039cade in osv::generate_signal (siginfo=..., 
ef=0x80e87068) at libc/signal.cc:130
#6  0x4039cb9f in osv::handle_mmap_fault 
(addr=addr@entry=18446744073709547520, sig=sig@entry=11, 
ef=ef@entry=0x80e87068)
at libc/signal.cc:145
#7  0x4028709b in mmu::vm_sigsegv (ef=0x80e87068, 
addr=18446744073709547520) at core/mmu.cc:1334
#8  mmu::vm_fault (addr=18446744073709547520, 
addr@entry=18446744073709551592, ef=ef@entry=0x80e87068) at 
core/mmu.cc:1354
#9  0x402d3d90 in page_fault (ef=0x80e87068) at 
arch/x64/mmu.cc:42
#10 
#11 0x10156cf7 in std::ostream::sentry::sentry 
(this=0x20200df0, __os=...)
at 
/usr/src/debug/gcc-10.3.1-1.fc33.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/ostream.tcc:51
#12 0x1015743c in std::__ostream_insert > (__out=..., __s=0x40303a "PASS", __n=__n@entry=4)
at 
/usr/src/debug/gcc-10.3.1-1.fc33.x86_64/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/ostream_insert.h:82
#13 0x00401ccf in std::operator<<  > 
(__s=, __out=...)
at /usr/include/c++/10/bits/char_traits.h:371
#14 report (ok=, msg="v7 in init function") at 
/home/wkozaczuk/projects/osv-true-master/tests/tst-tls.cc:57
#15 0x004014b3 in before_main () at 
/home/wkozaczuk/projects/osv-true-master/tests/tst-tls.cc:127
#16 0x402972c5 in elf::object::run_init_funcs 
(this=0xa094a600, argc=argc@entry=1, 
argv=argv@entry=0xa094a400)
at core/elf.cc:1178
#17 0x40298a0b in elf::program::init_library (this=, 
argc=1, argv=0xa094a400) at core/elf.cc:1500
#18 0x4020c239 in osv::application::main (this=0xa094cc10) 
at core/app.cc:319
#19 0x40365cb9 in operator() (app=, __closure=0x0) 
at core/app.cc:236
#20 _FUN () at core/app.cc:238
#21 0x40398a66 in operator() (__closure=0xa0abda00) at 
libc/pthread.cc:116
#22 std::__invoke_impl&> (__f=...) at 
/usr/include/c++/10/bits/invoke.h:60
#23 std::__invoke_r&> (__fn=...) at 
/usr/include/c++/10/bits/invoke.h:153
#24 std::_Function_handler >::_M_invoke(const 
std::_Any_data &) (__functor=...) at 
/usr/include/c++/10/bits/std_function.h:291
#25 0x403391ea in sched::thread::main (this=0x80e82040) at 
core/sched.cc:1267
#26 sched::thread_main_c (t=0x80e82040) at 
arch/x64/arch-switch.hh:325
#27 0x402d3b33 in thread_main () at arch/x64/entry.S:116

I have been researching this for a bit and realized that when I change the 
report() function not use std::cout but instead use printf() like so:

printf("%s: %s\n", ok ? "PASS" : "FAIL", msg.c_str());
the crash goes away.

I knew something was wrong with the initialization of some objects but 
could not quite pin it down. Then I tried to run the tst-tls-pie.so on the 
Linux host (it is a pie) and it crashed with a segmentation fault. This 
made me think that maybe something is wrong with the test program itself. 

Finally, I found this on the internet 
- https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94810 - where somebody tried 
to compile and run a program with a similar construct: std::cout used in 
the __constructor__ annotated function. The stack trace looks very similar 
as well. The bug was rejected with the explanation that the 
std::ios_base::Init object may not be initialized yet if used in the 
__constructor__ annotated function.

What is interesting this test happens to work with kernel exposing all 
symbols including stdc++ because the kernel copy of std::ios_base::Init was 
already initialized.

So I think the tst-tls.cc needs to be changed to not to use std:cout in the 
report function or uses different report() from before_main(). 

Waldek

PS1. Regarding using std:cout in various __constructor__  kernel functions 
like parse_madt() (which calls debug()) and called by smp_init(), I wonder 
if we are just lucky that std::ios_base::Init is already initialized and we 
do not see similar crashes.

PS2: This printout with dynamic linker info suggests that all global 
objects in stdc++.so should have been initialized before calling 
before_main(). So why is std::ios_base::Init  not initialized yet?

ELF [tid:26, mod:5, /usr/lib/libstdc++.so.6]: Executing DT_INIT function
ELF [tid:26, mod:5, /usr/lib/libstdc++.so.6]: Finished executing DT_INIT 
function
ELF [tid:26, mod:5, /usr/lib/libstdc++.so.6]: Executing 12 DT_INIT_ARRAYSZ 
functions
ELF [tid:26, mod:5, /usr/lib/libstdc++.so.6]: Finished executing 12 

[osv-dev] Re: Build failed in Jenkins: osv-build-nightly #2627

[Waldek Kozaczuk]

Now we have an unrelated issue with this 
- https://github.com/HewlettPackard/netperf/issues/42. We probably need to 
create a patch and apply it to the netperf app that is built as part of the 
nighly build

On Wednesday, December 22, 2021 at 3:04:23 PM UTC-5 jen...@scylladb.com 
wrote:

> See <
> https://jenkins.scylladb.com/job/osv-build-nightly/2627/display/redirect?page=changes
> >
>
> Changes:
>
> [jwkozaczuk] aarch64: revert changes to make mkfs and cpiod with stdc++
>
>
> --
> [...truncated 606.50 KB...]
> checking netinet/in.h usability... yes
> checking netinet/in.h presence... yes
> checking for netinet/in.h... yes
> checking netinet/sctp.h usability... yes
> checking netinet/sctp.h presence... yes
> checking for netinet/sctp.h... yes
> checking sched.h usability... yes
> checking sched.h presence... yes
> checking for sched.h... yes
> checking signal.h usability... yes
> checking signal.h presence... yes
> checking for signal.h... yes
> checking for stdlib.h... (cached) yes
> checking for string.h... (cached) yes
> checking for strings.h... (cached) yes
> checking syscall.h usability... yes
> checking syscall.h presence... yes
> checking for syscall.h... yes
> checking sys/ioctl.h usability... yes
> checking sys/ioctl.h presence... yes
> checking for sys/ioctl.h... yes
> checking sys/mman.h usability... yes
> checking sys/mman.h presence... yes
> checking for sys/mman.h... yes
> checking sys/param.h usability... yes
> checking sys/param.h presence... yes
> checking for sys/param.h... yes
> checking sys/socket.h usability... yes
> checking sys/socket.h presence... yes
> checking for sys/socket.h... yes
> checking for sys/stat.h... (cached) yes
> checking sys/time.h usability... yes
> checking sys/time.h presence... yes
> checking for sys/time.h... yes
> checking for sys/wait.h... (cached) yes
> checking sys/ipc.h usability... yes
> checking sys/ipc.h presence... yes
> checking for sys/ipc.h... yes
> checking sys/sockio.h usability... no
> checking sys/sockio.h presence... no
> checking for sys/sockio.h... no
> checking sys/sysinfo.h usability... yes
> checking sys/sysinfo.h presence... yes
> checking for sys/sysinfo.h... yes
> checking for sys/wait.h... (cached) yes
> checking for stdlib.h... (cached) yes
> checking for unistd.h... (cached) yes
> checking netinet/in_systm.h usability... yes
> checking netinet/in_systm.h presence... yes
> checking for netinet/in_systm.h... yes
> checking netinet/ip.h usability... yes
> checking netinet/ip.h presence... yes
> checking for netinet/ip.h... yes
> checking for main in -lsocket... no
> checking for main in -lmach... no
> checking for an ANSI C-conforming const... (cached) yes
> checking for off_t... yes
> checking for size_t... yes
> checking for socklen_t equivalent... unsigned
> checking for h_errno declaration in netdb.h... yes
> checking for struct sockaddr_storage... yes
> checking whether time.h and sys/time.h may both be included... yes
> checking for stdbool.h that conforms to C99... yes
> checking for _Bool... yes
> checking if sockaddr struct has sa_len member... no
> checking for pid_t... yes
> checking vfork.h usability... no
> checking vfork.h presence... no
> checking for vfork.h... no
> checking for fork... yes
> checking for vfork... yes
> checking for working fork... yes
> checking for working vfork... (cached) yes
> checking for stdlib.h... (cached) yes
> checking for unistd.h... (cached) yes
> checking for sys/param.h... (cached) yes
> checking for getpagesize... yes
> checking for working mmap... yes
> checking sys/select.h usability... yes
> checking sys/select.h presence... yes
> checking for sys/select.h... yes
> checking for sys/socket.h... (cached) yes
> checking types of arguments for select... int,fd_set *,struct timeval *
> checking whether setpgrp takes no argument... yes
> checking return type of signal handlers... void
> checking for alarm... yes
> checking for bzero... yes
> checking for daemon... yes
> checking for gethostbyname... yes
> checking for gethrtime... no
> checking for gettimeofday... yes
> checking for inet_ntoa... yes
> checking for memset... yes
> checking for memcpy... yes
> checking for munmap... yes
> checking for select... yes
> checking for setsid... yes
> checking for socket... yes
> checking for sqrt... yes
> checking for strcasecmp... yes
> checking for strchr... yes
> checking for strstr... yes
> checking for strtoul... yes
> checking for uname... yes
> checking for toupper... yes
> checking for getnameinfo... yes
> checking for getaddrinfo... yes
> checking for inet_ntop... yes
> checking for getifaddrs... yes
> checking for sendfile... yes
> checking for uname... (cached) yes
> checking for mpctl... no
> checking for processor_bind... no
> checking for sched_setaffinity... yes
> checking for bind_to_cpu_id... no
> checking for bindprocessor... no
> checking whether to include histogram support... no
> checking whether to include dirty support... no
> 

Re: [osv-dev] Build failed in Jenkins: osv-build #1050

[Waldek Kozaczuk]

This is just a warning, the error is with the linker (I forgot to test this
with cross-compiling to aarch64):

/usr/bin/aarch64-linux-gnu-ld: cannot find -lstdc++
collect2: error: ld returned 1 exit status
make: *** [Makefile:2040: build/release.aarch64/tools/mkfs/mkfs.so] Error 1
make: *** Waiting for unfinished jobs
make failed. Exiting from build script

On Wed, Dec 22, 2021 at 8:23 AM Nadav Har'El  wrote:

> The build is now failing on a real problem, not garbage in the build
> directory:
>
> https://jenkins.scylladb.com/view/osv/job/osv-build/1050/consoleText
>
>   CXX bsd/porting/mnttab.cc
> bsd/cddl/contrib/opensolaris/cmd/zpool/zpool_vdev.c: In function 
> ‘split_mirror_vdev’:
> bsd/cddl/contrib/opensolaris/cmd/zpool/zpool_vdev.c:1411:1: note: parameter 
> passing for argument of type ‘splitflags_t’ {aka ‘struct splitflags’} changed 
> in GCC 9.1
>  1411 | split_mirror_vdev(zpool_handle_t *zhp, char *newname, nvlist_t *props,
>   | ^
>
> I don't know if this failure has anything to do with your changes or it's 
> some bad luck that the compiler on Jenkins changed on the same day...
>
>
> --
> Nadav Har'El
> n...@scylladb.com
>
>
> On Wed, Dec 22, 2021 at 3:16 PM  wrote:
>
>> See 
>>
>> Changes:
>>
>>
>> --
>> [...truncated 2.90 MB...]
>>   CXX bsd/sys/netinet/arpcache.cc
>>   CXX bsd/sys/xen/evtchn.cc
>>   CC bsd/sys/dev/random/hash.c
>>   CXX bsd/sys/dev/random/randomdev_soft.cc
>>   CXX bsd/sys/dev/random/yarrow.cc
>>   CXX bsd/sys/dev/random/random_harvestq.cc
>>   CXX bsd/sys/dev/random/harvest.cc
>>   CXX bsd/sys/dev/random/live_entropy_sources.cc
>>   CXX core/mmu.cc
>>   CXX arch/aarch64/early-console.cc
>>   CXX drivers/console.cc
>>   CXX drivers/console-multiplexer.cc
>>   CXX drivers/console-driver.cc
>>   CXX drivers/line-discipline.cc
>>   CXX drivers/clock.cc
>>   CXX drivers/clock-common.cc
>>   CXX drivers/clockevent.cc
>>   CXX drivers/isa-serial-base.cc
>>   CXX core/elf.cc
>>   CXX drivers/random.cc
>>   CXX drivers/zfs.cc
>>   CXX drivers/null.cc
>>   CXX drivers/device.cc
>>   CXX drivers/pci-generic.cc
>>   CXX drivers/pci-device.cc
>>   CXX drivers/pci-function.cc
>>   CXX drivers/pci-bridge.cc
>>   CXX drivers/driver.cc
>>   CXX drivers/mmio-isa-serial.cc
>>   CXX drivers/pl011.cc
>>   CXX drivers/pl031.cc
>>   CXX drivers/cadence-uart.cc
>>   CXX drivers/xenconsole.cc
>>   CXX drivers/virtio.cc
>>   CXX drivers/virtio-pci-device.cc
>>   CXX drivers/virtio-mmio.cc
>>   CXX drivers/virtio-vring.cc
>>   CXX drivers/virtio-rng.cc
>>   CXX drivers/virtio-blk.cc
>>   CXX drivers/virtio-net.cc
>>   CXX drivers/virtio-fs.cc
>>   CXX arch/aarch64/arch-trace.cc
>>   CXX arch/aarch64/arch-setup.cc
>>   CXX arch/aarch64/signal.cc
>>   CXX arch/aarch64/arch-cpu.cc
>>   CXX arch/aarch64/backtrace.cc
>>   CXX arch/aarch64/smp.cc
>>   CXX arch/aarch64/mmu.cc
>>   LINK build/release.aarch64/dummy-shlib.so
>>   CXX tools/mkfs/mkfs.cc
>>   CC bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_changelist.c
>>   CC bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_config.c
>>   CC bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c
>>   CC bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_diff.c
>>   CC bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_import.c
>> In file included from
>> bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_dataset.c:55:
>> bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h: In function
>> ‘zfs_rename’:
>> bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h:574:12: note:
>> parameter passing for argument of type ‘renameflags_t’ {aka ‘struct
>> renameflags’} changed in GCC 9.1
>>   574 | extern int zfs_rename(zfs_handle_t *, const char *, const char *,
>>   |^~
>>   CC bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_iter.c
>>   CC bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_mount.c
>>   CC bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c
>>   CC bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c
>>   CC bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_util.c
>>   CC bsd/cddl/contrib/opensolaris/lib/libzpool/common/util.c
>>   CC bsd/cddl/contrib/opensolaris/lib/libzpool/common/kernel.c
>> bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c: In function
>> ‘zpool_valid_proplist’:
>> bsd/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c:419:1: note:
>> parameter passing for argument of type ‘prop_flags_t’ {aka ‘struct
>> prop_flags’} changed in GCC 9.1
>>   419 | zpool_valid_proplist(libzfs_handle_t *hdl, const char *poolname,
>>   | ^~~~
>>   CC bsd/cddl/compat/opensolaris/misc/mkdirp.c
>>   CC bsd/cddl/compat/opensolaris/misc/zmount.c
>>   CC bsd/sys/cddl/contrib/opensolaris/common/zfs/zfs_prop.c
>>   CC bsd/sys/cddl/contrib/opensolaris/common/zfs/zprop_common.c
>>   CC 

Re: [osv-dev] [PATCH] libc: implement posix_madvise

[Waldek Kozaczuk]

On Tuesday, December 21, 2021 at 5:57:52 PM UTC-5 Nadav Har'El wrote:

> On Wed, Dec 22, 2021 at 12:42 AM Waldek Kozaczuk  
> wrote:
>
>> The OSV_LIBC_API macro makes sure that during the compilation phase the 
>> posix_madvise symbol is left as GLOBAL DEFAULT in the libc/mman.o where all 
>> other symbols are left hidden. The presence of posix_madvise in those 4 
>> symbols files is used to feed the version script that is used by the linker 
>> to eliminate all other symbols but the ones listed in the sum of the 
>> symbols files for given architecture - aarch64 or x64. The reason we have 
>> given symbol occur almost always in 4 of those files [(x64 + aarch64) times 
>> 2] is because most symbols are also present in ld-musl-x86_64.so.1 and our 
>> dynamic linker advertises this library as well. And each of the symbol 
>> files was originally generated using scripts/extract_symbols.sh which 
>> identifies symbols to be exported against a version of kernel.elf built 
>> with all symbols exported and given Linux library file from the host. At 
>> the end of the day, it does not matter which file the given symbol is part 
>> of as we collapse them all into a single version script. 
>>
>> The OSV_LIBC_API used during the compilation phase and the symbol files 
>> used by the linker may be seen as a duplicate mechanism and frankly, the 
>> version script would have been enough to export (keep) symbols we want to. 
>> However OSV_LIBC_API + -fvisibility-hidden "*lets the optimizer produce 
>> better code*" (see https://gcc.gnu.org/wiki/Visibility) and this 
>> mechanism but itself is also not enough because there are still some 
>> symbols left (C++ typeinfo and other symbols, possibly originating from 
>> libstdc++.a) what we need to eliminate using the version script and linker.
>>
>
> Thanks for the explanation.
>
> So, my next question would be if we can automatically produce the linker 
> scripts, in one of two ways:
> 1. Recreate them every time on the user's build machine like you said you 
> build them for the first time, or 
>
 
This would be sensitive to the host. In the older version of Linux with 
older libc library, some newer libc symbols would not be present. I thought 
about it before and I came to the conclusion that maintaining these symbols 
file manually is better. These symbols lists just need to be supersets 
ideally across all distributions.

2. Create at least parts of it from automatic grepping of OSV_LIBC_API et 
> al. in the source files.
>

That would be difficult to achieve as we could only do that against OSv 
native code with OSV_LIBC_API in it. Musl sources from musl/ use different 
scheme - compile everything without -fvisiblity-hidden and annotate all 
symbols to be hidden with their hidden macro. 

>
> The goal is that you won't need to remember to add a symbols in 6 
> different places for it to work.
> Keep in mind that we'll note really ave comprehensive tests for all 
> symbols in all cases, so if I forget to add new_function() to libdl, or to 
> aarch64, I might not notice this in my tests.
>
> If the answer is no, then my question would be if we could at least not 
> duplicate the linker scripts. Can't x86 and aarch have the same list of 
> exported functions? And if two libraries contain the same symbols, could we 
> somehow automatically copy symbols between them?
>

At least we can improve it by keeping common lists (for x64 and aacth64) 
list - > 99% is the same. We can also add more ways to automatically verify 
symbols do not get lost, etc. The scipts/extract_symbols.sh has a 
'--verify" option that can be used from makefile to run a check to verify 
we are not losing any symbols from the list files. We can have another 
check that verifies that symbols with OSV_LIBC_API are also present in 
those lists. I will try to come up with some improvements.

>
>
>> On Tuesday, December 21, 2021 at 4:45:42 PM UTC-5 Nadav Har'El wrote:
>>
>>> Can you please remind me why we need both OSV_LIBC_API and putting the 
>>> function name in 5 different files?
>>> Isn't there a big risk that later when all this export stuff is less 
>>> fresh in our mind, we forget one of these 6 places and create a mess?
>>>
>>> --
>>> Nadav Har'El
>>> n...@scylladb.com
>>>
>>>
>>> On Tue, Dec 21, 2021 at 8:26 PM Waldemar Kozaczuk  
>>> wrote:
>>>
>>>> Some apps/runtimes like dotnet call posix_madvise
>>>> which we do not implement. This patch adds a simple
>>>> implementation of it based on madvise with a difference
>>>> that only POSIX_MADV_DONTNEED is supported. 

Re: [osv-dev] [PATCH] libc: implement posix_madvise

[Waldek Kozaczuk]

The OSV_LIBC_API macro makes sure that during the compilation phase the 
posix_madvise symbol is left as GLOBAL DEFAULT in the libc/mman.o where all 
other symbols are left hidden. The presence of posix_madvise in those 4 
symbols files is used to feed the version script that is used by the linker 
to eliminate all other symbols but the ones listed in the sum of the 
symbols files for given architecture - aarch64 or x64. The reason we have 
given symbol occur almost always in 4 of those files [(x64 + aarch64) times 
2] is because most symbols are also present in ld-musl-x86_64.so.1 and our 
dynamic linker advertises this library as well. And each of the symbol 
files was originally generated using scripts/extract_symbols.sh which 
identifies symbols to be exported against a version of kernel.elf built 
with all symbols exported and given Linux library file from the host. At 
the end of the day, it does not matter which file the given symbol is part 
of as we collapse them all into a single version script. 

The OSV_LIBC_API used during the compilation phase and the symbol files 
used by the linker may be seen as a duplicate mechanism and frankly, the 
version script would have been enough to export (keep) symbols we want to. 
However OSV_LIBC_API + -fvisibility-hidden "*lets the optimizer produce 
better code*" (see https://gcc.gnu.org/wiki/Visibility) and this mechanism 
but itself is also not enough because there are still some symbols left 
(C++ typeinfo and other symbols, possibly originating from libstdc++.a) 
what we need to eliminate using the version script and linker.

On Tuesday, December 21, 2021 at 4:45:42 PM UTC-5 Nadav Har'El wrote:

> Can you please remind me why we need both OSV_LIBC_API and putting the 
> function name in 5 different files?
> Isn't there a big risk that later when all this export stuff is less fresh 
> in our mind, we forget one of these 6 places and create a mess?
>
> --
> Nadav Har'El
> n...@scylladb.com
>
>
> On Tue, Dec 21, 2021 at 8:26 PM Waldemar Kozaczuk  
> wrote:
>
>> Some apps/runtimes like dotnet call posix_madvise
>> which we do not implement. This patch adds a simple
>> implementation of it based on madvise with a difference
>> that only POSIX_MADV_DONTNEED is supported. On top of
>> this as required posix_madvise() returns an error without
>> setting errno.
>>
>> Signed-off-by: Waldemar Kozaczuk 
>> ---
>>  .../aarch64/osv_ld-musl-aarch64.so.1.symbols   |  1 +
>>  exported_symbols/aarch64/osv_libc.so.6.symbols |  1 +
>>  .../x64/osv_ld-musl-x86_64.so.1.symbols|  1 +
>>  exported_symbols/x64/osv_libc.so.6.symbols |  1 +
>>  libc/mman.cc   | 14 ++
>>  5 files changed, 18 insertions(+)
>>
>> diff --git a/exported_symbols/aarch64/osv_ld-musl-aarch64.so.1.symbols 
>> b/exported_symbols/aarch64/osv_ld-musl-aarch64.so.1.symbols
>> index c463131a..4c284fba 100644
>> --- a/exported_symbols/aarch64/osv_ld-musl-aarch64.so.1.symbols
>> +++ b/exported_symbols/aarch64/osv_ld-musl-aarch64.so.1.symbols
>> @@ -739,6 +739,7 @@ posix_fadvise
>>  posix_fadvise64
>>  posix_fallocate
>>  posix_fallocate64
>> +posix_madvise
>>  posix_memalign
>>  pow
>>  pow10
>> diff --git a/exported_symbols/aarch64/osv_libc.so.6.symbols 
>> b/exported_symbols/aarch64/osv_libc.so.6.symbols
>> index 000191b7..ea0bfcdc 100644
>> --- a/exported_symbols/aarch64/osv_libc.so.6.symbols
>> +++ b/exported_symbols/aarch64/osv_libc.so.6.symbols
>> @@ -667,6 +667,7 @@ posix_fadvise
>>  posix_fadvise64
>>  posix_fallocate
>>  posix_fallocate64
>> +posix_madvise
>>  posix_memalign
>>  ppoll
>>  prctl
>> diff --git a/exported_symbols/x64/osv_ld-musl-x86_64.so.1.symbols 
>> b/exported_symbols/x64/osv_ld-musl-x86_64.so.1.symbols
>> index b3f87859..d88e98ed 100644
>> --- a/exported_symbols/x64/osv_ld-musl-x86_64.so.1.symbols
>> +++ b/exported_symbols/x64/osv_ld-musl-x86_64.so.1.symbols
>> @@ -721,6 +721,7 @@ posix_fadvise
>>  posix_fadvise64
>>  posix_fallocate
>>  posix_fallocate64
>> +posix_madvise
>>  __posix_getopt
>>  posix_memalign
>>  pow
>> diff --git a/exported_symbols/x64/osv_libc.so.6.symbols 
>> b/exported_symbols/x64/osv_libc.so.6.symbols
>> index 6635cabb..07b5368b 100644
>> --- a/exported_symbols/x64/osv_libc.so.6.symbols
>> +++ b/exported_symbols/x64/osv_libc.so.6.symbols
>> @@ -596,6 +596,7 @@ posix_fadvise
>>  posix_fadvise64
>>  posix_fallocate
>>  posix_fallocate64
>> +posix_madvise
>>  __posix_getopt
>>  posix_memalign
>>  ppoll
>> diff --git a/libc/mman.cc b/libc/mman.cc
>> index 9dd6429a..75a94eb0 100644
>> --- a/libc/mman.cc
>> +++ b/libc/mman.cc
>> @@ -257,3 +257,17 @@ void *sbrk(intptr_t increment)
>>  errno = ENOMEM;
>>  return (void *)-1;
>>  }
>> +
>> +static unsigned posix_madvise_to_advise(int advice)
>> +{
>> +if (advice == POSIX_MADV_DONTNEED) {
>> +return mmu::advise_dontneed;
>> +}
>> +return 0;
>> +}
>> +
>> +OSV_LIBC_API
>> +int posix_madvise(void *addr, size_t len, int 

Re: [osv-dev] Roadmap going forward

[Waldek Kozaczuk]

Hi,

On Mon, Dec 20, 2021 at 03:15 Nadav Har'El  wrote:

>
> On Mon, Dec 20, 2021 at 4:23 AM Waldek Kozaczuk 
> wrote:
>
>> Hi,
>>
>> It has been a while since the last release but I have only recently found
>> some time and energy to update the roadmap -
>> https://github.com/cloudius-systems/osv/wiki/Roadmap. It is meant to be
>> a "living" document that more-less captures what we (me?) would like to
>> work on in near future. So it is more-less my wish list but I would like to
>> use this occasion to seek your input as well. I would like to know what you
>> would want to be implemented/improved/changed etc. Feel free to respond to
>> this email or directly update the roadmap wiki or add comments. Also, all
>> OSv issues are here - https://github.com/cloudius-systems/osv/issues and
>> if you find any interesting ones please feel free to add them to the wiki.
>>
>> In order to facilitate the discussion I am enclosing the roadmap here
>> with some of my extra comments:
>>
>> Modularization/"Librarization"
>>
>>- Hide most symbols and C++ standard library (Partially addressed by
>>this
>>
>> <https://github.com/cloudius-systems/osv/commit/af2d371a61f6ab1eb5a066a0c3e93230faf6611c>
>>  and
>>other commits)
>>   - Issue 97
>>   
>> <https://github.com/cloudius-systems/osv/issues/97#issuecomment-643833296> -
>>   Be more selective on symbols exported from the kernel
>>   - The research results with numbers described in this discussion
>>   <https://groups.google.com/d/msg/osv-dev/fs-jmyaXY_E/vWUk-wDDAAAJ>
>>and the older one
>>   <https://groups.google.com/d/msg/osv-dev/JsJfnhTX_EY/PKUvYm9-AQAJ> seem
>>   to indicate that both C++ standard library and other non-public symbols
>>   should be hidden in the kernel to make this goal achievable
>>- Make ZFS optional as a shared library
>><https://github.com/cloudius-systems/osv/issues/1009> - the patches
>><https://groups.google.com/g/osv-dev/c/utWNzLGBwog> to address this
>>issue were sent a couple of days ago
>>- Create toolchain to optionally build custom kernel tailored to
>>specific hypervisor or app
>><https://github.com/cloudius-systems/osv/issues/1110>
>>   - See this original discussion
>>   <https://groups.google.com/d/msg/osv-dev/BHwN6Stm3n4/arvdOwl3AAAJ>
>>- Reduce kernel size
>>   - Shrinking kernel described in this early discussion
>>   <https://groups.google.com/d/msg/osv-dev/hCYGRzytaJ4/D23S_ibNAgAJ> 
>> would
>>   really be a result of hiding C++ library and most symbols (see above) 
>> but
>>   also conditionally compiling in/out various modules (see the first 
>> bullet
>>   in the Modularization/"Librarization")
>>- Expose C++ APIs as C to hide C++ completely
>>
>>
> You've made some really nice progress on this!
>
>
>> <https://github.com/cloudius-systems/osv/wiki/Roadmap#support-statically-linked-executables>Support
>> statically linked executables
>>
>>- See
>>
>> https://github.com/cloudius-systems/osv/issues?q=is%3Aissue+is%3Aopen+label%3Astatic-elf
>>
>>
> What's the motivation for this? What uses statically linked executables?
>

For example ffmpeg is distributed as static executable. Also the motivation
is to limit number of symbols we keep adding for new kinds of apps. The
performance hit might not be that bad.

>
> <https://github.com/cloudius-systems/osv/wiki/Roadmap#improve-tooling>Improve
>> tooling
>>
>>- Improve capstan (see Capstan "2.0" Wiki
>><https://github.com/cloudius-systems/capstan/wiki/Capstan-2.0>)
>>   - *THIS is a project on its own, somewhat independent but I think
>>   it is very important to improve building and running OSv images; it 
>> would
>>   be nice to fine a volunteer that would focus on it entirely*
>>- Create IntelliJ, Maven and or Gradle plugins to deploy Java, Node,
>>etc apps
>>- Run Docker images on OSv (possibly musl-based like Alpine are
>>better)
>>   - There is already an existing example -
>>   
>> https://github.com/cloudius-systems/osv-apps/tree/master/openjdk12-jre-from-docker
>>
>>
>> <https://github.com/cloudius-systems/osv/wiki/Roadmap#merge-parts-of-ipv6-branch-into-master>Merge
>> parts of ipv6 branch into master
>>
>>- Port partial netlink support
>>
>&

[osv-dev] Roadmap going forward

[Waldek Kozaczuk]

Hi,

It has been a while since the last release but I have only recently found 
some time and energy to update the roadmap 
- https://github.com/cloudius-systems/osv/wiki/Roadmap. It is meant to be a 
"living" document that more-less captures what we (me?) would like to work 
on in near future. So it is more-less my wish list but I would like to use 
this occasion to seek your input as well. I would like to know what you 
would want to be implemented/improved/changed etc. Feel free to respond to 
this email or directly update the roadmap wiki or add comments. Also, all 
OSv issues are here - https://github.com/cloudius-systems/osv/issues and if 
you find any interesting ones please feel free to add them to the wiki.

In order to facilitate the discussion I am enclosing the roadmap here with 
some of my extra comments:

Modularization/"Librarization"
   
   - Hide most symbols and C++ standard library (Partially addressed by this 
   

 and 
   other commits)
  - Issue 97 
  
 - 
  Be more selective on symbols exported from the kernel
  - The research results with numbers described in this discussion 
  
   and the older one 
   seem 
  to indicate that both C++ standard library and other non-public symbols 
  should be hidden in the kernel to make this goal achievable
   - Make ZFS optional as a shared library 
    - the patches 
    to address this 
   issue were sent a couple of days ago
   - Create toolchain to optionally build custom kernel tailored to 
   specific hypervisor or app 
   
  - See this original discussion 
  
   - Reduce kernel size
  - Shrinking kernel described in this early discussion 
   would 
  really be a result of hiding C++ library and most symbols (see above) but 
  also conditionally compiling in/out various modules (see the first bullet 
  in the Modularization/"Librarization")
   - Expose C++ APIs as C to hide C++ completely

Support
 
statically linked executables
   
   - See 
   
https://github.com/cloudius-systems/osv/issues?q=is%3Aissue+is%3Aopen+label%3Astatic-elf

Improve 
tooling
   
   - Improve capstan (see Capstan "2.0" Wiki 
   )
  - *THIS is a project on its own, somewhat independent but I think it 
  is very important to improve building and running OSv images; it would be 
  nice to fine a volunteer that would focus on it entirely*
   - Create IntelliJ, Maven and or Gradle plugins to deploy Java, Node, etc 
   apps
   - Run Docker images on OSv (possibly musl-based like Alpine are better)
  - There is already an existing example - 
  
https://github.com/cloudius-systems/osv-apps/tree/master/openjdk12-jre-from-docker
   
Merge
 
parts of ipv6 branch into master
   
   - Port partial netlink support 
   

 in 
   order to fix libc: Fix if_indextoname(), if_nametoindex() 
   

   - Also, reach out to Spirent to see if any more networking related bugs 
   fixed can be brought over from the https://github.com/SpirentOrion/osv
fork

Others
   
   - Help review and merge the "lazy-allocated stack" patch 
   
   - Fix cli/lua on newer Fedora (see 
   https://github.com/cloudius-systems/osv/issues/1166)
   - Consider implementing a read-write filesystem alternative to ZFS - 
ext2/3/4 
   ? 
   - Improve gdb support 
   
   - #1069  - Add 
   vsock support
  - *Would there really be real-world application of that? Maybe to 
  provide better OSv-host configuration capabilities?*
   - Support other 

Re: [osv-dev] [PATCH V2 1/2] Better handle exceptions thrown by dynamic linker:

[Waldek Kozaczuk]

This seems to be the exact same patch as you sent last time.

On Thu, Dec 9, 2021 at 2:15 AM AubreyTurner  wrote:

> Use _init_called bool variable in elf object, initialize to false, set to
> true
> at end of run_init_funcs(), in run_fini_funcs() only call fini functions if
> _init_called has been set to true. Avoids page fault case by only calling
> fini
> functions for modules who's init functions were called for. Instead throws
> corresponding exception and error message is shown.
>
> Signed-off-by: AubreyTurner 
> ---
>  core/elf.cc| 6 --
>  include/osv/elf.hh | 1 +
>  2 files changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/core/elf.cc b/core/elf.cc
> index 80bfd841..418aa10f 100644
> --- a/core/elf.cc
> +++ b/core/elf.cc
> @@ -123,6 +123,7 @@ object::object(program& prog, std::string pathname)
>  , _dynamic_table(nullptr)
>  , _module_index(_prog.register_dtv(this))
>  , _is_executable(false)
> +, _init_called(false)
>  , _visibility_thread(nullptr)
>  , _visibility_level(VisibilityLevel::Public)
>  {
> @@ -1178,12 +1179,13 @@ void object::run_init_funcs(int argc, char** argv)
>  }
>  elf_debug("Finished executing %d DT_INIT_ARRAYSZ functions\n",
> nr);
>  }
> +_init_called = true;
>  }
>
>  // Run the object's static destructors or similar finalization
>  void object::run_fini_funcs()
>  {
> -if (dynamic_exists(DT_FINI_ARRAY)) {
> +if (dynamic_exists(DT_FINI_ARRAY) && _init_called) {
>  auto funcs = dynamic_ptr(DT_FINI_ARRAY);
>  auto nr = dynamic_val(DT_FINI_ARRAYSZ) / sizeof(*funcs);
>  elf_debug("Executing %d DT_FINI_ARRAYSZ functions\n", nr);
> @@ -1193,7 +1195,7 @@ void object::run_fini_funcs()
>  }
>  elf_debug("Finished executing %d DT_FINI_ARRAYSZ functions\n",
> nr);
>  }
> -if (dynamic_exists(DT_FINI)) {
> +if (dynamic_exists(DT_FINI) && _init_called) {
>  auto func = dynamic_ptr(DT_FINI);
>  if (func) {
>  elf_debug("Executing DT_FINI function\n");
> diff --git a/include/osv/elf.hh b/include/osv/elf.hh
> index ae6ff786..afc9c9a5 100644
> --- a/include/osv/elf.hh
> +++ b/include/osv/elf.hh
> @@ -435,6 +435,7 @@ protected:
>  std::unique_ptr _section_names_cache;
>  bool _is_executable;
>  bool is_core();
> +bool _init_called;
>
>  std::unordered_map _cached_symbols;
>
> --
> 2.30.2
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "OSv Development" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/osv-dev/XvZMmvDZNsE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> osv-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/osv-dev/20211209071458.35108-1-aubreyt1231%40gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfPC4OUrrO7XMdaKZXd7Q%2BDYWzYkXLbemPk0t%3DzNnjC0og%40mail.gmail.com.

[osv-dev] Re: [PATCH] Better handle exceptions thrown by dynamic linker:

[Waldek Kozaczuk]

Hi,

Your patch logically looks good and works fine. I have tested a case when 
one of the shared libraries was of the wrong arch. 

There is a nitpick I am describing below. Anyway, please resend a new 
version (V2) also with an updated commit description to make it 
automatically close defect #1116. Add a line like this:

Fixes #1116

Thanks for the patch,
Waldek

On Tuesday, December 7, 2021 at 1:21:27 AM UTC-5 AubreyTurner wrote:

> Use _init_called bool variable in elf object, initialize to false, set to 
> true 
> at end of run_init_funcs(), in run_fini_funcs() only call fini functions 
> if 
> _init_called has been set to true. Avoids page fault case by only calling 
> fini 
> functions for modules who's init functions were called for. Instead throws 
> corresponding exception and error message is shown. 
>
> Signed-off-by: AubreyTurner  
> --- 
> core/elf.cc | 6 -- 
> include/osv/elf.hh | 1 + 
> 2 files changed, 5 insertions(+), 2 deletions(-) 
>
> diff --git a/core/elf.cc b/core/elf.cc 
> index 80bfd841..418aa10f 100644 
> --- a/core/elf.cc 
> +++ b/core/elf.cc 
> @@ -123,6 +123,7 @@ object::object(program& prog, std::string pathname) 
> , _dynamic_table(nullptr) 
> , _module_index(_prog.register_dtv(this)) 
> , _is_executable(false) 
> + , _init_called(false) 
> , _visibility_thread(nullptr) 
> , _visibility_level(VisibilityLevel::Public) 
> { 
> @@ -1178,12 +1179,13 @@ void object::run_init_funcs(int argc, char** argv) 
> } 
> elf_debug("Finished executing %d DT_INIT_ARRAYSZ functions\n", nr); 
> } 
> + _init_called = true; 
> } 
>
> // Run the object's static destructors or similar finalization 
> void object::run_fini_funcs() 
> {

You may want to simplify this by simply returning from this method if 
!_init_called instead of checking it in those two if statements separately 
below:

if (!_init_called) {
   return;
}


> - if (dynamic_exists(DT_FINI_ARRAY)) { 
> + if (dynamic_exists(DT_FINI_ARRAY) && _init_called) { 
> auto funcs = dynamic_ptr(DT_FINI_ARRAY); 
> auto nr = dynamic_val(DT_FINI_ARRAYSZ) / sizeof(*funcs); 
> elf_debug("Executing %d DT_FINI_ARRAYSZ functions\n", nr); 
> @@ -1193,7 +1195,7 @@ void object::run_fini_funcs() 
> } 
> elf_debug("Finished executing %d DT_FINI_ARRAYSZ functions\n", nr); 
> } 
> - if (dynamic_exists(DT_FINI)) { 
> + if (dynamic_exists(DT_FINI) && _init_called) { 
> auto func = dynamic_ptr(DT_FINI); 
> if (func) { 
> elf_debug("Executing DT_FINI function\n"); 
> diff --git a/include/osv/elf.hh b/include/osv/elf.hh 
> index ae6ff786..afc9c9a5 100644 
> --- a/include/osv/elf.hh 
> +++ b/include/osv/elf.hh 
> @@ -435,6 +435,7 @@ protected: 
> std::unique_ptr _section_names_cache; 
> bool _is_executable; 
> bool is_core(); 
> + bool _init_called; 
>
> std::unordered_map _cached_symbols; 
>
> -- 
> 2.30.2 
>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/31cc1df5-fedd-4de8-a53f-d4fc126d3eb0n%40googlegroups.com.

Re: [osv-dev] Another filesystem, maybe ext2/3/4?

[Waldek Kozaczuk]

Created an issue for this 
- https://github.com/cloudius-systems/osv/issues/1179

On Thursday, October 21, 2021 at 11:06:04 PM UTC-4 Waldek Kozaczuk wrote:

> Hi,
>
> On Wednesday, October 13, 2021 at 9:32:18 AM UTC-4 Nadav Har'El wrote:
>
>> On Wed, Oct 13, 2021 at 12:16 AM Gregory Burd  wrote:
>>
>>> Hello OSv-ers,
>>>
>>> I'm a huge fan of ZFS, it's an amazing bit of work and I'm thrilled it's 
>>> a core component in OSv.  That said, it's not a great choice in all cases, 
>>> the overhead of ZFS can outweigh the benefits.  I've heard many references 
>>> to "adding another filesystem" into the mix in different contexts, most 
>>> recently in the (amazing) talk given at p99conf by Waldek.
>>>
>>> So, how about ext2 pulled straight from the BSD tree?
>>> https://github.com/freebsd/freebsd-src/tree/main/sys/fs/ext2fs
>>>
>>> Why ext2 and not ?  Well, it's not my 
>>> favorite filesystem either but it is popular and well known.  It's easy for 
>>> Linux users to get comfortable with and the tools are generally installed 
>>> by default on most distros.  I would imagine that the BSD code is fairly 
>>> complete and supported and I believe it supports ext2, 3, and 4 (
>>> https://wiki.freebsd.org/Ext2fs).
>>>
>>> anyone have thoughts?
>>>
>>
>> I think it makes sense, but only if it's something that you personally 
>> care about for some reason - e.g., that it's important for you for OSv to 
>> be smaller and you believe that replacing ZFS will make it smaller. Or that 
>> some other advantage of ext2 over zfs is interesting for you.
>>
>> Something worth keeping in mind is that one of the claimed advantages of 
>> OSv over, say, Linux, is that OSv does not need to support a gazillion 
>> different drivers and filesystems. It's not like anyone will ever plug a 
>> ext2-formatted disk or ntfs-formatted or whatever into OSv - so we don't 
>> *need* to support any of these filesystems. If we do want to support them, 
>> it should be out of some expected benefit - not out of necessity. So let's 
>> just spell out in advance what this benefit might be over the filesystems 
>> we already have (zfs, ramfs and rofs).
>>
>
> If we go ahead with implementing ext2 support, we should define a minimal 
> subset of it we want to implement (do we need extents, large files, etc?). 
> We should probably also not make the same mistake as with ZFS and NOT 
> implement tools equivalent to zpool.so, mkfs.so, etc. Let us make it so for 
> all admin functionality we would delegate to the toolset on host OS.  
>
>>
>> Also, I don't remember if Waldek did this or only partially (?), but if 
>> you're adding ext2 to reduce the kernel size, we first need to compile a 
>> kernel without zfs. We could add a build-time feature of removing zfs (see 
>> https://github.com/cloudius-systems/osv/issues/1110) or build it into a 
>> shared library that doesn't need to be loaded (
>> https://github.com/cloudius-systems/osv/issues/1009). This would be 
>> similar to the Linux build system - which allows keeping some parts of the 
>> kernel out of the build, but also keeping some parts of the kernel in the 
>> build but as separate modules (sort of shared libraries).
>>  
>>
>>> anyone have suggestions on where to start?
>>>
>>
>> I would start with making (at least to myself) the case of what the 
>> benefit of adding ext2 would be.
>> If you think it is the code size, I would start by trying to estimate how 
>> much smaller the kernel would be without ZFS. For that I would start adding 
>> to our build system an option to compile without ZFS - or compile ZFS into 
>> a shared library.
>>
>
> I think in my presentation (slides 9/10) I claimed that I was able to trim 
> the size of the kernel by ~0.7MB (after enabling GC). Also adding an option 
> to build OSv without ZFS is something that I was planning to prepare proper 
> patches for. However, this would come in the order described in the 
> presentation - the 3rd step, but does not have to: we can work on the 
> ability to compile ZFS out independently. I think I might have something 
> ready for steps 1 and 2 (hide C++ std, enable GC) in the next 2-4 weeks. 
> Meanwhile, my WIP branch - 
> https://github.com/wkozaczuk/osv/commits/minimize_kernel_size - has all 
> code changes I made for the presentation including adding commenting out 
> ZFS in the right places so you can start experimenting with it. This 
> particular commit deals with ZFS - 
> https://github.com/wkozaczuk/osv/commit/df

[osv-dev] Re: Resolver, res_* and other related functions

[Waldek Kozaczuk]

On Friday, October 29, 2021 at 8:25:17 PM UTC-4 Waldek Kozaczuk wrote:

> Hi,
>
> As I have been working on hiding standard library C++ symbols and most 
> others that are not glibc, I have come across some resolver functions 
> (please https://linux.die.net/man/3/res_mkquery). On Linux they are 
> typically defined as part of libresolv.so.2 and OSv "advertises" per 
> core/elf.cc. The commit 
> https://github.com/cloudius-systems/osv/commit/ddb9ef9dce9a22037d2e2ca6b7dfa42b806b613c
>  
> added the "libresolv.so.2" in order to support krb library.
>
> However as I was trying to establish which exact symbols out from around 
> 90 ones in libresolv.so.2 OSv kernel exposes, I discovered that there are 
> none.  This command lists 91 symbols on my Fedora 33:
>
> nm -C --dynamic --defined-only --extern-only $(gcc --print-file-name 
> libresolv.so.2) | grep -vP '\d{16,16} A' | grep -v GLIBC_PRIVATE | cut -c 
> 20- | cut --delimiter=@ -f 1 | sort | uniq
>
> And intersecting it with what loader.elf exposes gives nothing.
>
> I am not sure how important it is that we do not implement or expose some 
> of these symbols. But after some digging, it turns out that we do include 
> some of them from musl/src/network however their glibc version starts with 
> a double underscore (__) and those are hidden. Let me list all the relevant 
> functions that we do include and how they are used, exposed and what I 
> think we should do about it if anything:
>
>
>- *dn_expand* - we use the musl implementation in 
>musl_1.1.24/src/network/dn_expand.c as-is, however only 'dn_expand' is 
>exposed and libresolv.so lists '__dn_expand'
>   - this needs to be fixed by updating one of the internal musl 
>   headers by removing 'hidden' from the declaration I think or in another 
> way
>   - dn_expand() is internally used in 
>   musl/src/network/lookup_name.c which we include per the main Makefile
>  - the musl 1.1.24 also uses it in network/getnameinfo.c but OSv 
>  has its own version of it under libc/network which is based on older 
> musl 
>  0.9.12 and modified to read '/etc/hosts' to speed dns resolution by 
>  delegating to OSv dns api. I*s it worth taking getnameinfo.c 
>  from 1.1.24 and re-apply same OSv changes*?
>   - I wonder how important this function is, so far we have not come 
>   across any apps that complain about '__dn_expand' missing
>
> The __dn_expand is now exposed with the commit 
b6e7d185454d707ae2d66fcad40546e5f5747062.

>
>- *res_mkquery* -  we use the musl implementation in 
>musl_1.1.24/src/network/res_mkquery.c as-is, however only 'res_mkquery' is 
>exposed and libresolv.so lists '__res_mkquery' (the ld-musl-x86_64.so.1 
>exports res_mkquery)
>   - this needs to be fixed by updating one of the internal musl 
>   headers by removing 'hidden' from the declaration I think or in another 
> way
>   - __res_mkquery() is internally used in 
>   musl/src/network/lookup_name.c which we include per the main Makefile 
> (as 
>   dn_expand(), the __res_mkquery is also used in musl getnameinfo.c)
>   - As with dn_expand, I wonder how important this function is, so 
>   far we have not come across any apps that complain about it missing
>- *res_msend* - this is NOT actually a symbol from glibc but the 
>internal function __res_msend_rc() is used by musl network/lookup_name.c 
>which in turn is used by musl network/gethostbyname2_r.c
>   - The lookup_name.c is new in musl 1.1.24 which is a result of 
>   major refactoring of musl resolver logic after 0.9.12. I wonder if 
>   lookup_name.c executes similar logic as getnameinfo and we need to 
> adjust 
>   it to read /etc/hosts as well?
>
> Regards,
> Waldek
>
> PS. The file 
> https://github.com/cloudius-systems/osv/blob/master/libc/network/README.md 
> has some extra info about potentially unfinished upgrade from 0.9.12 to 
> 1.1.24 and may help shed some light.
>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/f8d60ce0-7370-48f1-b04a-ca4b269b670en%40googlegroups.com.

Re: [osv-dev] Hiding some symbols defined in bsd/* tree

[Waldek Kozaczuk]

Created new issue for it 
- https://github.com/cloudius-systems/osv/issues/1178

On Sunday, November 28, 2021 at 9:18:34 AM UTC-5 Nadav Har'El wrote:

> On Sun, Nov 28, 2021 at 3:15 PM Waldek Kozaczuk  
> wrote:
>
>> As Nadav noted when reviewing my patch to hide/expose some libc and other 
>> libraries related symbols, some of those defined in the bsd/ part of the 
>> tree which originate from FreeBSD, may not necessarily "do" what their 
>> Linux equivalent "do" even though their share the same name. 
>
>
> On one hand, I agree - we have entire directories, like bsd/, where 
> nothing in there is supposed to be a glibc ABI so we don't need to 
> advertise.
>
> On the other hand, I worry what will happen if later want to go more into 
> the direction of a "modular" kernel, where pieces of the kernel are separe 
> shared objects and loaded separately. If we ever want to do that, it is 
> quite possible that some of these symbols from one shared object should be 
> visible to other shared objects - but not to the application. I'm worried 
> maybe the "hidden" mechanism will not be delicate enough.
>
> Maybe we will need something more delicate than just "hidden" (I'm afraid 
> I don't remember enough about the dynamic linker to recommend something 
> specific): We will need some symbols to be exported by the kernel or pieces 
> of it compiled as shared objects, but change the dynamic linker to make 
> certain symbols visible only to specific shared objects (i.e., other kernel 
> pieces) and not others (i.e., user application).
>
> Of course, we don't have to do it now.
>
>
>
>> Since then I have sent new version of the patch that would hide all these 
>> symbols BUT only if we enable hiding symbols in kernel which will be an 
>> optional build mechanism feature. But I wonder if we should hide those 
>> symbols proactively no matter what?
>>
>> Here are the symbols:
>>
>>- inet_aton, inet_ntoa, inet_ntoa_r - this is most serious one 
>>actually because on one hand we have those implemented in 
>>bsd/sys/libkern/[inet_aton.c, inet_ntoa.c] and we also have the musl 
>>versions in musl_1.1.24/src/network/[inet_aton.c, inet_ntoa.c] with __ 
>>version which then have aliases defined for those. I am afraid in the end 
>>we are exposing the FreeBSD versions of these 3 functions not the musl 
>> ones 
>>as we should.
>>- mtx_lock and related mtx_* ones which are defined and used 
>>internally in bsd/ subtree and happen to collide with C11 mtx_* symbols 
>>(see https://en.cppreference.com/w/c/thread)
>>- some symbols that are part of libbsd.so library on Linux (see 
>>https://gitlab.freedesktop.org/libbsd/libbsd) because any app that 
>>depends on libbsd.so will see OSv version (should it?):
>>   - 
>>   - arc4random
>>   - MD5Final
>>   - MD5Init
>>   - MD5Update
>>- RPC related xdr_* functions - they may not have same implementation 
>>os the thier Linux counterpart
>>
>>
> Good list.
>
>
>> What do you think?
>> Waldek
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "OSv Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to osv-dev+u...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/osv-dev/7ef8c6b1-2201-48c3-a606-2dca3bc37335n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/osv-dev/7ef8c6b1-2201-48c3-a606-2dca3bc37335n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/fc44afdf-410d-4bcd-89d8-fc17dd731024n%40googlegroups.com.

Re: [osv-dev] Re: [PATCH v2] gdb: fix unique_ptr_get

[Waldek Kozaczuk]

Shall we apply this patch but only mark it fixing issue #1176?

On Wednesday, November 24, 2021 at 9:14:45 PM UTC-5 Waldek Kozaczuk wrote:

> On Wednesday, November 24, 2021 at 5:07:20 PM UTC-5 Nadav Har'El wrote:
>
>> On Wed, Nov 24, 2021 at 5:00 PM Waldek Kozaczuk  
>> wrote:
>>
>>> So I applied your patch on my Fedora 33 laptop and I get this error when 
>>> running trace test:
>>>
>>>   TEST tracing_smoke_test warning: Remote gdbserver does 
>>> not support determining executable automatically.
>>> RHEL <=6.8 and <=7.2 versions of gdbserver do not support such automatic 
>>> executable detection.
>>> The following versions of gdbserver support it:
>>> - Upstream version of gdbserver (unsupported) 7.10 or later
>>> - Red Hat Developer Toolset (DTS) version of gdbserver from DTS 4.0 or 
>>> later (only on x86_64)
>>> - RHEL-7.3 versions of gdbserver (on any architecture)
>>> sched::cpu::do_idle (this=0x8001e040) at core/sched.cc:454
>>> 454 handle_incoming_wakeups();
>>> Saving traces to tracefile ...
>>> Python Exception  'utf-8' codec can't decode 
>>> byte 0xc6 in position 0: invalid continuation byte: 
>>> Error occurred in Python: 'utf-8' codec can't decode byte 0xc6 in 
>>> position 0: invalid continuation byte
>>>
>>
>> This is the same error that I'm getting in Fedora 34, *after* my patch 
>> fixed previous failures.
>> I assumed it was a pre-existing bug and my patch just allowed me to get 
>> to it, but maybe I was wrong.
>> Do you not get this error before my patch?
>>
> I only have Fedora 33 and I do not get this error before your patch. 
>
>>
>> Can you please check if "osv info threads" works with my patch (this is 
>> what I was trying to fix).
>>
> The "osv info threads" still works on my Fedora 33 if that is what you are 
> asking.
>
>>  
>>
>>> [Inferior 1 (process 1) detached]
>>>
>>> Test tracing_smoke_test FAILED
>>> Traceback (most recent call last):
>>>   File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
>>> 200, in 
>>> main()
>>>   File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
>>> 155, in main
>>> run_tests()
>>>   File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
>>> 144, in run_tests
>>> run(tests_to_run)
>>>   File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
>>> 117, in run
>>> run_test(test)
>>>   File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
>>> 92, in run_test
>>> test.run()
>>>   File 
>>> "/home/wkozaczuk/projects/osv-true-master/scripts/tests/testing.py", line 
>>> 52, in run
>>> self.f()
>>>   File 
>>> "/home/wkozaczuk/projects/osv-true-master/scripts/tests/test_tracing.py", 
>>> line 28, in tracing_smoke_test
>>> assert(subprocess.call([trace_script, 'extract']) == 0)
>>> AssertionError
>>>
>>> My gcc:
>>> gcc --version
>>> gcc (GCC) 10.3.1 20210422 (Red Hat 10.3.1-1)
>>> Copyright (C) 2020 Free Software Foundation, Inc.
>>>
>>> So there is something weird going on.
>>> On Monday, November 22, 2021 at 7:56:26 AM UTC-5 Nadav Har'El wrote:
>>>
>>>> In my setup (Fedora 34, gcc 11.2.1), suddenly the gdb scripts' 
>>>> unique_ptr_get() stopped working in "osv info threads" and also in one 
>>>> of the tests - see issues #1175 and #1176. 
>>>>
>>>> The following patch, "inspired" by gcc's "printers.py" fixes it on 
>>>> my setup. However, I can't explain why this particular change is 
>>>> suddenly needed. printers.py is similar to what I did but not quite 
>>>> the same, and the old code should have continued to work. I'll 
>>>> appreciate it if someone with a different compiler version can check 
>>>> if this patch doesn't break "osv info threads" on different compiler 
>>>> versions. 
>>>>
>>>> Fixes #1175 
>>>> Fixes #1176 (but the test fails in a new place, I guess I need to fix 
>>>> another problem). 
>>>>
>>>> Signed-off-by: Nadav Har'El  
>>>> --- 
>>>> scripts/loader.py | 13 ++--- 
>>>> 1

[osv-dev] Hiding some symbols defined in bsd/* tree

[Waldek Kozaczuk]

As Nadav noted when reviewing my patch to hide/expose some libc and other 
libraries related symbols, some of those defined in the bsd/ part of the 
tree which originate from FreeBSD, may not necessarily "do" what their 
Linux equivalent "do" even though their share the same name. 

Since then I have sent new version of the patch that would hide all these 
symbols BUT only if we enable hiding symbols in kernel which will be an 
optional build mechanism feature. But I wonder if we should hide those 
symbols proactively no matter what?

Here are the symbols:

   - inet_aton, inet_ntoa, inet_ntoa_r - this is most serious one actually 
   because on one hand we have those implemented in 
   bsd/sys/libkern/[inet_aton.c, inet_ntoa.c] and we also have the musl 
   versions in musl_1.1.24/src/network/[inet_aton.c, inet_ntoa.c] with __ 
   version which then have aliases defined for those. I am afraid in the end 
   we are exposing the FreeBSD versions of these 3 functions not the musl ones 
   as we should.
   - mtx_lock and related mtx_* ones which are defined and used internally 
   in bsd/ subtree and happen to collide with C11 mtx_* symbols 
   (see https://en.cppreference.com/w/c/thread)
   - some symbols that are part of libbsd.so library on Linux 
   (see https://gitlab.freedesktop.org/libbsd/libbsd) because any app that 
   depends on libbsd.so will see OSv version (should it?):
  - 
  - arc4random
  - MD5Final
  - MD5Init
  - MD5Update
   - RPC related xdr_* functions - they may not have same implementation os 
   the thier Linux counterpart


What do you think?
Waldek

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/7ef8c6b1-2201-48c3-a606-2dca3bc37335n%40googlegroups.com.

Re: [osv-dev] [PATCH] libc: remove/add some aliases

[Waldek Kozaczuk]

On Monday, November 22, 2021 at 9:20:48 AM UTC-5 Nadav Har'El wrote:

> On Sun, Nov 21, 2021 at 10:38 AM Waldemar Kozaczuk  
> wrote:
>
>> This patch removes some entries from aliases.ld for symbols
>> that are not present in glibc shared libraries OSv dynamic linker
>> advertises. It also adds new entries for symbols that are present in the
>> same libraries.
>>
>> Signed-off-by: Waldemar Kozaczuk 
>> ---
>>  libc/aliases.ld| 11 ++-
>>  libc/stdio/__fprintf_chk.c |  1 -
>>  2 files changed, 6 insertions(+), 6 deletions(-)
>>
>> diff --git a/libc/aliases.ld b/libc/aliases.ld
>> index 14dac146..09ed4b70 100644
>> --- a/libc/aliases.ld
>> +++ b/libc/aliases.ld
>> @@ -29,7 +29,6 @@ __finitel = finitel;
>>
>>  /* locale */
>>  __toupper_l = toupper_l;
>> -__setlocale = setlocale;
>>
>
> I am guessing we didn't insert those aliases for fun - I'm guessing we saw 
> something actually using it. But if you see that this alias doesn't exist 
> in gcc I think it's safe to remove it.
>

Yeah, I was the one that populated this file mostly as part of the effort 
when migrating to the newer musl 1.1.24. And I may have done some things 
wrong. I justify the removal of some of those aliases on the fact tha tI 
could not find the __setlocale in any shared libraries advertised by 
core/elf.cc. I also could not find __setlocale in any header files on the 
host. But maybe I should crosscheck Ubuntu.

>
>
>>  /* multibyte */
>>  __mbrlen = mbrlen;
>> @@ -50,11 +49,7 @@ __stpncpy = stpncpy;
>>  __strdup = strdup;
>>  strerrordesc_np = strerror;
>>
>> -/* temp */
>> -__mktemp = mktemp;
>> -
>>  /* time */
>> -__clock_gettime = clock_gettime;
>>
>
> Again, I'm guessing some old versions of libraries or something used these 
> aliases,
> because a Google search for __clock_gettime returns stuff.
> I'm fine with removing those aliases if they aren't used in modern code.
>
I think it is BSD/MacOS-ism. I could not find it the Linux host shared 
libraries nor headers.

>  
>
>>  __gmtime_r = gmtime_r;
>>  __strftime_l = strftime_l;
>>
>> @@ -72,3 +67,9 @@ fstatat64 = fstatat;
>>
>>  __lseek = lseek;
>>  __getauxval = getauxval;
>> +
>> +__getpagesize = getpagesize;
>>
>
> We already have such an alias in runtime.cc, so please remove it from 
> there to add it here.
>
Sure. 

>  
>
>> +newlocale = __newlocale;
>>
>
> We already have such an alias in runtime.cc, so please remove it from 
> there to add it here.
>
Sure. 

>
> +_exit = exit;
>>
> +_Exit = exit;
>>
>
> Again, we already had those aliases, in runtime.cc. 
> Please remove the aliases from there, and also copy the comment to 
> aliases.ld.
>
> After you remove all the ALIAS calls from runtime.cc you can remove the 
> definition of ALIAS :-)
>  
>
>> +__dn_expand = dn_expand;
>> diff --git a/libc/stdio/__fprintf_chk.c b/libc/stdio/__fprintf_chk.c
>> index 2d7e42d4..68447c63 100644
>> --- a/libc/stdio/__fprintf_chk.c
>> +++ b/libc/stdio/__fprintf_chk.c
>> @@ -15,4 +15,3 @@ __fprintf_chk(FILE *fp, int flag, const char *format, 
>> ...)
>>
>>  return ret;
>>  }
>> -weak_alias(__fprintf_chk, ___fprintf_chk);
>> -- 
>> 2.31.1
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "OSv Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to osv-dev+u...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/osv-dev/20211121083757.22214-1-jwkozaczuk%40gmail.com
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/7e644d0a-d349-48a4-ab8b-af90705067ban%40googlegroups.com.

Re: [osv-dev] [PATCH 3/3] prepare to hide kernel symbols: hide some symbols

[Waldek Kozaczuk]

On Monday, November 22, 2021 at 10:27:20 AM UTC-5 Nadav Har'El wrote:

> On Sun, Nov 21, 2021 at 11:09 AM Waldemar Kozaczuk  
> wrote:
>
>> This last patch in the series annotates some symbols in few
>> files that need to be hidden regardless if we compile kernel with
>> non-glibc symbols hidden or not.
>
>
> I see only a couple of these cases. Why did you need to do those specific 
> cases?
>
To hide those specific symbols as they should not be visible to the apps. 

>
>  
>
>> For that we use OSV_HIDDEN_API macro.
>>
>
> OSV_HIDDEN.
>
Right. 

>
>
>> Signed-off-by: Waldemar Kozaczuk 
>> ---
>>  libc/libc.cc| 2 +-
>>  libc/libc.hh| 3 ++-
>>  libc/locale/uselocale.c | 2 +-
>>  libc/misc/uname.c   | 3 ++-
>>  4 files changed, 6 insertions(+), 4 deletions(-)
>>
>> diff --git a/libc/libc.cc b/libc/libc.cc
>> index b75b3867..4fb7d574 100644
>> --- a/libc/libc.cc
>> +++ b/libc/libc.cc
>> @@ -43,7 +43,7 @@ int libc_error(int err)
>>
>>  #undef errno
>>
>> -int __thread errno;
>> +OSV_HIDDEN int __thread errno;
>>
>>  int* __errno_location()
>>  {
>> diff --git a/libc/libc.hh b/libc/libc.hh
>> index cf345ce8..bba4a0a6 100644
>> --- a/libc/libc.hh
>> +++ b/libc/libc.hh
>> @@ -10,8 +10,9 @@
>>
>>  #include 
>>  #include "internal/libc.h" // for the macros
>> +#include 
>>
>> -int libc_error(int err);
>> +OSV_HIDDEN int libc_error(int err);
>>
>>  template 
>>  T* libc_error_ptr(int err);
>> diff --git a/libc/locale/uselocale.c b/libc/locale/uselocale.c
>> index 93e65e21..bbdae028 100644
>> --- a/libc/locale/uselocale.c
>> +++ b/libc/locale/uselocale.c
>> @@ -1,7 +1,7 @@
>>  #include "locale_impl.h"
>>  #include "libc.h"
>>
>> -__thread locale_t __current_locale;
>> +OSV_HIDDEN __thread locale_t __current_locale;
>>
>>  locale_t __uselocale(locale_t l)
>>  {
>> diff --git a/libc/misc/uname.c b/libc/misc/uname.c
>> index f7770ddf..3f1bf754 100644
>> --- a/libc/misc/uname.c
>> +++ b/libc/misc/uname.c
>> @@ -7,6 +7,7 @@
>>
>>  #include 
>>  #include 
>> +#include "osv/export.h"
>>
>>  // The Linux version we're pretending to be
>>  #define LINUX_MAJOR 3
>> @@ -23,7 +24,7 @@ _Static_assert(KERNEL_VERSION(LINUX_MAJOR, LINUX_MINOR, 
>> LINUX_PATCH)
>>  #define str(s) #s
>>  #define str2(s) str(s)
>>
>> -struct utsname utsname = {
>> +struct utsname utsname OSV_HIDDEN = {
>> .sysname= "Linux",  /* lie, to avoid confusing the 
>> payload. */
>> .nodename   = "osv.local",
>> .release= str2(LINUX_MAJOR) "." str2(LINUX_MINOR) "." 
>> str2(LINUX_PATCH),
>> -- 
>> 2.31.1
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "OSv Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to osv-dev+u...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/osv-dev/20211121090925.22630-3-jwkozaczuk%40gmail.com
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/2f52edf2-2e9b-4f78-8734-e9bf6fff406dn%40googlegroups.com.

Re: [osv-dev] [PATCH 1/3] prepare to hide kernel symbols: add macros to expose and hide symbols

[Waldek Kozaczuk]

On Monday, November 22, 2021 at 9:48:04 AM UTC-5 Nadav Har'El wrote:

> On Sun, Nov 21, 2021 at 11:09 AM Waldemar Kozaczuk  
> wrote:
>
>> This patch is the first one in the series that annotates relevant
>> symbols across number of source files in order to expose or hide them
>> when kernel is compiled to hide non-glibc symbols. Please note that none
>> of this patches neither hides nor exposes any symbols. In essense it will
>> only have an effect once we make changes to the makefile to allow 
>> compiling
>> the kernel with most symbols hidden.
>>
>> This patch adds new header - export.h - with new macros that either
>> expose (__visibility__("default")) or hide (__visibility__("hidden"))
>> the symbols. Please read the comments in the header file for more
>> details.
>>
>> Signed-off-by: Waldemar Kozaczuk 
>> ---
>>  include/osv/export.h | 39 +++
>>  1 file changed, 39 insertions(+)
>>  create mode 100644 include/osv/export.h
>>
>> diff --git a/include/osv/export.h b/include/osv/export.h
>> new file mode 100644
>> index ..e6266fcc
>> --- /dev/null
>> +++ b/include/osv/export.h
>> @@ -0,0 +1,39 @@
>> +/*
>> + * Copyright (C) 2021 Waldemar Kozaczuk
>> + *
>> + * This work is open source software, licensed under the terms of the
>> + * BSD license as described in the LICENSE file in the top-level 
>> directory.
>> + */
>> +
>> +#ifndef EXPORT_H
>> +#define EXPORT_H
>> +
>> +//
>> +// Please note that the macros below are used in the source files and
>> +// are intended to expose the annotated symbols as public when the kernel
>> +// files are compiled with the flag '-fvisibility=hidden'. When the 
>> kernel
>> +// is compiled without compatibility flag, these macro do not have any 
>> affect
>>
>
> You refer to "compatibility flag" a few times in this comment, and I 
> didn't understand
> what it means. Compatibility with what?
>
 
I meant the "visibility" instead. I will update the comments in the new 
version of the patch. 

>  
>
>> +// as all symbols in this case are exposed as public. So either way, the 
>> symbols
>> +// annotated with these macros yield desired effect.
>> +// We do not really need to define a macro for each Linux glibc library 
>> and we could
>> +// have had single OSV_GLIBC_API macro instead of eight ones below. 
>> However by
>> +// having a macro for each library shared file where a symbol is part of,
>> +// we automatically self-document the code and in future could 
>> auto-generate some
>> +// docs.
>> +#define OSV_LIBAIO_API __attribute__((__visibility__("default")))
>> +#define OSV_LIBC_API __attribute__((__visibility__("default")))
>> +#define OSV_LIBM_API __attribute__((__visibility__("default")))
>> +#define OSV_LIBBSD_API __attribute__((__visibility__("default")))
>> +#define OSV_LIBPTHREAD_API __attribute__((__visibility__("default")))
>> +#define OSV_LIBUTIL_API __attribute__((__visibility__("default")))
>> +#define OSV_LIBXENSTORE_API __attribute__((__visibility__("default")))
>> +#define OSV_LD_LINUX_x86_64_API 
>> __attribute__((__visibility__("default")))
>>
>
I think I should have explained more explicitly what really each of those 
above is. Each macro above does the same thing but by putting a specific 
one next to a given symbol, we mean that this symbol is defined in one of 
the Linux library files which are advertised in core/elf.cc:

  "libresolv.so.2"
  "libc.so.6"
  "libm.so.6"
  "ld-linux-x86-64.so.2" // or ld-linux-aarch64.so.1"
  "libc.musl-x86_64.so.1"
  "libboost_system.so.1.55.0"
  "libpthread.so.0"
  "libdl.so.2"
  "librt.so.1"
  "libstdc++.so.6"
  "libaio.so.1"
  "libxenstore.so.3.0"
  "libcrypt.so.1"

So symbol annotated with OSV_LIBXENSTORE_API is physically located in 
"libxenstore.so.3.0",  OSV_LIBC_AP in  "libc.so.6" shared library on Linux, 
etc. I hope this will address some of your questions/concerns in other 
related patches.
 

> +
>> +// In some very few cases, when source files are compiled without 
>> compatibility
>> +// flag in order to expose most symbols in the corresponding file, there 
>> are some specific
>> +// symbols in the same file that we want to hide and this is where we 
>> use this macro.
>> +// Regardless if we hide most symbols in the kernel or not, the 
>> annotated symbols would
>> +// be always hidden.
>> +#define OSV_HIDDEN __attribute__((__visibility__("hidden")))
>>
>
> In commit 815398537f665fca72b4c2040a6f70262d597ac5 you added, following 
> musl,
> the macro "hidden", in lowercase. Maybe we should use that instead of the 
> uglier OSV_HIDDEN?
> Or maybe "hidden" is too short/lowercase? But then again, we already have 
> it...
>
You are correct that I added "hidden" macro in that commit. But this is 
really a musl macro defined in mush src/include/features.h:

#define hidden __attribute__((__visibility__("hidden")))

which I copied into OSv 

Re: [osv-dev] Re: [PATCH v2] gdb: fix unique_ptr_get

[Waldek Kozaczuk]

On Wednesday, November 24, 2021 at 5:07:20 PM UTC-5 Nadav Har'El wrote:

> On Wed, Nov 24, 2021 at 5:00 PM Waldek Kozaczuk  
> wrote:
>
>> So I applied your patch on my Fedora 33 laptop and I get this error when 
>> running trace test:
>>
>>   TEST tracing_smoke_test warning: Remote gdbserver does 
>> not support determining executable automatically.
>> RHEL <=6.8 and <=7.2 versions of gdbserver do not support such automatic 
>> executable detection.
>> The following versions of gdbserver support it:
>> - Upstream version of gdbserver (unsupported) 7.10 or later
>> - Red Hat Developer Toolset (DTS) version of gdbserver from DTS 4.0 or 
>> later (only on x86_64)
>> - RHEL-7.3 versions of gdbserver (on any architecture)
>> sched::cpu::do_idle (this=0x8001e040) at core/sched.cc:454
>> 454 handle_incoming_wakeups();
>> Saving traces to tracefile ...
>> Python Exception  'utf-8' codec can't decode 
>> byte 0xc6 in position 0: invalid continuation byte: 
>> Error occurred in Python: 'utf-8' codec can't decode byte 0xc6 in 
>> position 0: invalid continuation byte
>>
>
> This is the same error that I'm getting in Fedora 34, *after* my patch 
> fixed previous failures.
> I assumed it was a pre-existing bug and my patch just allowed me to get to 
> it, but maybe I was wrong.
> Do you not get this error before my patch?
>
I only have Fedora 33 and I do not get this error before your patch. 

>
> Can you please check if "osv info threads" works with my patch (this is 
> what I was trying to fix).
>
The "osv info threads" still works on my Fedora 33 if that is what you are 
asking.

>  
>
>> [Inferior 1 (process 1) detached]
>>
>> Test tracing_smoke_test FAILED
>> Traceback (most recent call last):
>>   File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
>> 200, in 
>> main()
>>   File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
>> 155, in main
>> run_tests()
>>   File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
>> 144, in run_tests
>> run(tests_to_run)
>>   File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
>> 117, in run
>> run_test(test)
>>   File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
>> 92, in run_test
>> test.run()
>>   File 
>> "/home/wkozaczuk/projects/osv-true-master/scripts/tests/testing.py", line 
>> 52, in run
>> self.f()
>>   File 
>> "/home/wkozaczuk/projects/osv-true-master/scripts/tests/test_tracing.py", 
>> line 28, in tracing_smoke_test
>> assert(subprocess.call([trace_script, 'extract']) == 0)
>> AssertionError
>>
>> My gcc:
>> gcc --version
>> gcc (GCC) 10.3.1 20210422 (Red Hat 10.3.1-1)
>> Copyright (C) 2020 Free Software Foundation, Inc.
>>
>> So there is something weird going on.
>> On Monday, November 22, 2021 at 7:56:26 AM UTC-5 Nadav Har'El wrote:
>>
>>> In my setup (Fedora 34, gcc 11.2.1), suddenly the gdb scripts' 
>>> unique_ptr_get() stopped working in "osv info threads" and also in one 
>>> of the tests - see issues #1175 and #1176. 
>>>
>>> The following patch, "inspired" by gcc's "printers.py" fixes it on 
>>> my setup. However, I can't explain why this particular change is 
>>> suddenly needed. printers.py is similar to what I did but not quite 
>>> the same, and the old code should have continued to work. I'll 
>>> appreciate it if someone with a different compiler version can check 
>>> if this patch doesn't break "osv info threads" on different compiler 
>>> versions. 
>>>
>>> Fixes #1175 
>>> Fixes #1176 (but the test fails in a new place, I guess I need to fix 
>>> another problem). 
>>>
>>> Signed-off-by: Nadav Har'El  
>>> --- 
>>> scripts/loader.py | 13 ++--- 
>>> 1 file changed, 10 insertions(+), 3 deletions(-) 
>>>
>>> diff --git a/scripts/loader.py b/scripts/loader.py 
>>> index 7706ba38..63334e19 100644 
>>> --- a/scripts/loader.py 
>>> +++ b/scripts/loader.py 
>>> @@ -1000,10 +1000,17 @@ def find_or_give_last(predicate, seq): 
>>>
>>> def unique_ptr_get(u): 
>>> try: 
>>> - # Since gcc 7 
>>> - return u['_M_t']['_M_t']['_M_head_impl'] 
>>> + tuple_member = u['_M_t']['_M_t'] 
>

[osv-dev] Re: [PATCH v2] gdb: fix unique_ptr_get

[Waldek Kozaczuk]

So I applied your patch on my Fedora 33 laptop and I get this error when 
running trace test:

  TEST tracing_smoke_test warning: Remote gdbserver does 
not support determining executable automatically.
RHEL <=6.8 and <=7.2 versions of gdbserver do not support such automatic 
executable detection.
The following versions of gdbserver support it:
- Upstream version of gdbserver (unsupported) 7.10 or later
- Red Hat Developer Toolset (DTS) version of gdbserver from DTS 4.0 or 
later (only on x86_64)
- RHEL-7.3 versions of gdbserver (on any architecture)
sched::cpu::do_idle (this=0x8001e040) at core/sched.cc:454
454 handle_incoming_wakeups();
Saving traces to tracefile ...
Python Exception  'utf-8' codec can't decode 
byte 0xc6 in position 0: invalid continuation byte: 
Error occurred in Python: 'utf-8' codec can't decode byte 0xc6 in position 
0: invalid continuation byte
[Inferior 1 (process 1) detached]

Test tracing_smoke_test FAILED
Traceback (most recent call last):
  File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
200, in 
main()
  File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
155, in main
run_tests()
  File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
144, in run_tests
run(tests_to_run)
  File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 
117, in run
run_test(test)
  File "/home/wkozaczuk/projects/osv-true-master/scripts/test.py", line 92, 
in run_test
test.run()
  File "/home/wkozaczuk/projects/osv-true-master/scripts/tests/testing.py", 
line 52, in run
self.f()
  File 
"/home/wkozaczuk/projects/osv-true-master/scripts/tests/test_tracing.py", 
line 28, in tracing_smoke_test
assert(subprocess.call([trace_script, 'extract']) == 0)
AssertionError

My gcc:
gcc --version
gcc (GCC) 10.3.1 20210422 (Red Hat 10.3.1-1)
Copyright (C) 2020 Free Software Foundation, Inc.

So there is something weird going on.
On Monday, November 22, 2021 at 7:56:26 AM UTC-5 Nadav Har'El wrote:

> In my setup (Fedora 34, gcc 11.2.1), suddenly the gdb scripts'
> unique_ptr_get() stopped working in "osv info threads" and also in one
> of the tests - see issues #1175 and #1176.
>
> The following patch, "inspired" by gcc's "printers.py" fixes it on
> my setup. However, I can't explain why this particular change is
> suddenly needed. printers.py is similar to what I did but not quite
> the same, and the old code should have continued to work. I'll
> appreciate it if someone with a different compiler version can check
> if this patch doesn't break "osv info threads" on different compiler
> versions.
>
> Fixes #1175
> Fixes #1176 (but the test fails in a new place, I guess I need to fix
> another problem).
>
> Signed-off-by: Nadav Har'El 
> ---
> scripts/loader.py | 13 ++---
> 1 file changed, 10 insertions(+), 3 deletions(-)
>
> diff --git a/scripts/loader.py b/scripts/loader.py
> index 7706ba38..63334e19 100644
> --- a/scripts/loader.py
> +++ b/scripts/loader.py
> @@ -1000,10 +1000,17 @@ def find_or_give_last(predicate, seq):
>
> def unique_ptr_get(u):
> try:
> - # Since gcc 7
> - return u['_M_t']['_M_t']['_M_head_impl']
> + tuple_member = u['_M_t']['_M_t']
> + tuple_impl_type = tuple_member.type.fields()[0].type # _Tuple_impl
> + tuple_head_type = tuple_impl_type.fields()[1].type # _Head_base
> + head_field = tuple_head_type.fields()[0]
> + return tuple_member.cast(head_field.type)
> except:
> - return u['_M_t']['_M_head_impl']
> + try:
> + # Since gcc 7
> + return u['_M_t']['_M_t']['_M_head_impl']
> + except:
> + return u['_M_t']['_M_head_impl']
>
> def thread_cpu(t):
> d = unique_ptr_get(t['_detached_state'])
> -- 
> 2.31.1
>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/4ee39ca8-575e-4b19-ac75-ca0f536faaecn%40googlegroups.com.

[osv-dev] Re: First Contributions

[Waldek Kozaczuk]

Hi and welcome,

Any of 
the 
https://github.com/cloudius-systems/osv/issues?q=is%3Aopen+is%3Aissue+label%3Agood-for-newcomers
 
would be a good one to start. Maybe #1116 could be a nice start. This 
comment 
- https://github.com/cloudius-systems/osv/issues/1116#issuecomment-755395318 
- provides a good blueprint of a solution.

Regards,
Waldek

On Monday, November 15, 2021 at 12:10:43 AM UTC-5 Aubrey turner wrote:

> Hello! I'm a student at the University of Texas studying CS. For my 
> virtualization class I have an assignment involving contributing to an open 
> source project. I have been looking through the easy and good-for-newcomers 
> tagged issues and was wondering if there was a specific issue that would be 
> good/helpful for me to work on as my first issue since I am new to open 
> source?
>
> Thanks,
> Aubrey
>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/06007d0f-8443-4b8d-ab68-2f1a9f2c4018n%40googlegroups.com.

Re: [osv-dev] [PATCH] scripts/osv/prof.py: update unimportant_prefixes

[Waldek Kozaczuk]

Well it is because the stack trace looks different, possibly because the
new GCC generates slightly different code.

On Sun, Nov 14, 2021 at 4:08 AM Nadav Har'El  wrote:

> I committed this because I assumed you know what you are doing, but I am
> curious: Why did you remove
> sched::timer_base::expire() from the list? Has this specific function
> become more interesting because of recent changes?
>
> On Sat, Nov 13, 2021 at 9:46 AM Waldemar Kozaczuk 
> wrote:
>
>> Update unimportant_prefixes to match recent stack traces generated
>> when sampling an app using cpu_sampler enabled with
>> '--sampler=FREQUENCY'.
>>
>> Signed-off-by: Waldemar Kozaczuk 
>> ---
>>  scripts/osv/prof.py | 4 +++-
>>  1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/scripts/osv/prof.py b/scripts/osv/prof.py
>> index 3a13..05a06e86 100644
>> --- a/scripts/osv/prof.py
>> +++ b/scripts/osv/prof.py
>> @@ -74,9 +74,11 @@ unimportant_prefixes = [
>>   'log',
>>   'trace_slow_path',
>>   'operator()',
>> + '_FUN',
>> + 'prof::cpu_sampler::timer_fired()',
>>   'prof::cpu_sampler::timer_fired()',
>> - 'sched::timer_base::expire()',
>>   'sched::timer_list::fired()',
>> + 'std::function::operator()() const',
>>   'interrupt_descriptor_table::invoke_interrupt(unsigned int)',
>>   'interrupt',
>>   'interrupt_entry_common'),
>> --
>> 2.31.1
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "OSv Development" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to osv-dev+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/osv-dev/2023074641.5098-1-jwkozaczuk%40gmail.com
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups "OSv 
Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to osv-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/osv-dev/CAL9cFfP-dg4d5f%2BQVifQ%2BLdanY4FnArVKcDbgFxGPiTHduMB_w%40mail.gmail.com.

Re: [osv-dev] Hiding C++ standard library other non-glibc symbols and its implications to the tests and OSv apps and modules

[Waldek Kozaczuk]

On Thu, Nov 4, 2021 at 3:22 AM Nadav Har'El  wrote:

>
> On Thu, Nov 4, 2021 at 2:47 AM Waldek Kozaczuk 
> wrote:
>
>> Hi,
>>
>> I have been pretty quiet on this list since the P99 conference. Partly
>> because I was taking some time off and partly because I have been working
>> quietly on designing and implementing the changes to the OSv build system
>> to accommodate hiding C++ std and other non-glibc symbols as described in
>> my P99 presentation.
>>
>> This email will be pretty long so bear with me. My objective here is to
>> describe in more detail what my exact plan is, what I have done so far and
>> what is left. As you will see not everything is as easy as I hoped for. Or
>> it is not easy to decide what the right thing to do is and this is where I
>> am seeking your advice. Obviously, any feedback regarding my plan and what
>> I have done so far is also very welcome.
>>
>> The goal of this exercise is not only to lower the kernel size and
>> therefore reduce memory utilization and to make it more secure by exposing
>> glibc/musl libc symbols only as described by the issue #97 ("*Be more
>> selective on symbols exported from the kernel*" -
>> https://github.com/cloudius-systems/osv/issues/97). Hiding the symbols
>> would also address other issues as well I think:
>>
>>- #821 - "Combining pre-compiled OSv kernel with pre-compiled
>>executable"
>>- #1161 - "Support running .NET apps on OSv" (depends on hiding std
>>C++ to allow adding different version of it to the image)
>>- #1110 - "Modularization/"Librarization" - create toolchain to
>>optionally build custom kernel tailored to specific hypervisor or app
>>- #1009 - "Make ZFS optional as a shared library"
>>
>> So doing it right will benefit the users of OSv on many fronts. But I
>> think there is some painful road ahead of us to accomplish that. And maybe
>> we will have to make some compromises.
>>
>> Roughly my original plan was this:
>>
>>1. Link libstdc++.a with '-no-whole-archive' flag.
>>2. Expose the glibc symbols and hide everything else using a version
>>script that lists symbols we want to expose.
>>3. Enable garbage collection using the compiler flags
>>(-ffunction-sections, -fdata-sections) and a linker flag "--gc-sections"
>>
>> The new plan is slightly different with an extra initial step that I had
>> thought originally would be too tedious to achieve:
>>
>>1. Hide most symbols in compilation phase using the*
>>'-fvisibility=hidden' and '-fvisibility-inlines-hidden' flags* for
>>most non-musl and non-'./libc/*' sources and use newly defines macros to
>>enforce specific symbols as public (aka default visibility for some) or as
>>hidden. This actually seems to work pretty well and was not as tedious as 
>> I
>>expected.
>>
>>
> This seems a good idea, but if I understand correctly, it means you will
> need to add some macro in front of each function we want to export which
> will export it.
>
> I'm not worried that this would be "tedious", but what does worry me is
> how we can verify that we don't miss a lot of the symbols we were supposed
> to export.
> Over the years, we tested OSv with many different Linux applications, and
> slowly slowly found more and more symbols we need to define. If we now
> forget to
> export one of those, we may break one of these applications which we made
> to work on OSv some five years ago, and aren't using today. Our test suite
> does
> not, unfortunately, have a long list of applications that we test continue
> to work.
>
> What I propose you could do - and reading what you wrote below I'm not
> sure whether you were planning to do - is to write a script (like you
> wanted to write anyway)
> that gets a list of symbols from some library (e.g., glibc) and then looks
> in OSv for symbols which 1. exist in OSv, but 2. are *not* exported.
> Symbols found thus are a sure sign we forgot to export.
> Note that I'm not proposing that we look for symbols which exist in the
> library but not in OSv - there may be many of those. Rather, I'm proposing
> we look for library symbols which *are* in OSv, but we forgot to export
> them.
>

This is a valid concern and only now I realized that I have not adequately
explained the mechanics of what I am doing here. What you are suggesting
above is more less what the new script ./scripts/extract_symbols.sh does or
allows one to do (see below with a hyperlink to the copy of it in my
branch). In essence as I was wor

[osv-dev] Hiding C++ standard library other non-glibc symbols and its implications to the tests and OSv apps and modules

[Waldek Kozaczuk]

Hi,

I have been pretty quiet on this list since the P99 conference. Partly 
because I was taking some time off and partly because I have been working 
quietly on designing and implementing the changes to the OSv build system 
to accommodate hiding C++ std and other non-glibc symbols as described in 
my P99 presentation.

This email will be pretty long so bear with me. My objective here is to 
describe in more detail what my exact plan is, what I have done so far and 
what is left. As you will see not everything is as easy as I hoped for. Or 
it is not easy to decide what the right thing to do is and this is where I 
am seeking your advice. Obviously, any feedback regarding my plan and what 
I have done so far is also very welcome.

The goal of this exercise is not only to lower the kernel size and 
therefore reduce memory utilization and to make it more secure by exposing 
glibc/musl libc symbols only as described by the issue #97 ("*Be more 
selective on symbols exported from the kernel*" - 
https://github.com/cloudius-systems/osv/issues/97). Hiding the symbols 
would also address other issues as well I think:

   - #821 - "Combining pre-compiled OSv kernel with pre-compiled executable"
   - #1161 - "Support running .NET apps on OSv" (depends on hiding std C++ 
   to allow adding different version of it to the image)
   - #1110 - "Modularization/"Librarization" - create toolchain to 
   optionally build custom kernel tailored to specific hypervisor or app
   - #1009 - "Make ZFS optional as a shared library"

So doing it right will benefit the users of OSv on many fronts. But I think 
there is some painful road ahead of us to accomplish that. And maybe we 
will have to make some compromises.

Roughly my original plan was this:

   1. Link libstdc++.a with '-no-whole-archive' flag.
   2. Expose the glibc symbols and hide everything else using a version 
   script that lists symbols we want to expose.
   3. Enable garbage collection using the compiler flags 
   (-ffunction-sections, -fdata-sections) and a linker flag "--gc-sections"

The new plan is slightly different with an extra initial step that I had 
thought originally would be too tedious to achieve:

   1. Hide most symbols in compilation phase using the* 
   '-fvisibility=hidden' and '-fvisibility-inlines-hidden' flags* for most 
   non-musl and non-'./libc/*' sources and use newly defines macros to enforce 
   specific symbols as public (aka default visibility for some) or as hidden. 
   This actually seems to work pretty well and was not as tedious as I 
   expected. 
   2. Link libstdc++.a, *libgcc_eh.a and libboost_system.a* with 
   '-no-whole-archive' flag and add extra linker flag '--exclude-libs 
   libstdc++.a' to hide most std C++ symbols (to solve the problem of 5K 
   symbols left with original approach).
   3. Expose the glibc symbols and hide everything else using a version 
   script that lists symbols we want to expose - this should be very close to 
   what is left after compiler hiding symbols in step 2 (an is based on my 
   experiments).
   4. Enable garbage collection using the compiler flags 
   (-ffunction-sections, -fdata-sections) and a linker flag "--gc-sections"

Why do I think it is a good idea to compile with '-fvisibility=hidden' 
which seems to be redundant with the step 3 (use version script)? It is 
because the compiler may be able to produce better machine code knowing 
that all or most symbols are hidden. This effect may be even more profound 
if we decide to enable -flto/lto optimizations later.

So far I have reached a point close to complete on step 1 and 2 (the step 3 
and 4 is fairly easy to do). You can see all my work so far on my 
branch https://github.com/wkozaczuk/osv/tree/hide_stdcxx_and_most_symbols. 
But here a specific list with more details (so you can comment):

   - Create new script *extract_symbols.sh* to generate lists of symbols to 
   be exported by OSv kernel for each library advertised in core/elf.cc (see 
   
https://github.com/wkozaczuk/osv/blob/hide_stdcxx_and_most_symbols/scripts/extract_symbols.sh)
 
   - the script uses '*nm -C --dynamic --defined-only --extern-only*' 
  against given library on the host and then intersects them with symbols 
  exported by loader-stripped.elf; this obviously means the lists might be 
  slightly different on each distribution/version of Linux host as glibc 
  keeps expanding
  - for now the script stores the list files at 
  ./exported_symbols/osv_{lib}.symbols but eventually we would actually 
  "freeze" those as they would become an input to assemble a version script 
  used in the step 3 (see above); we would refresh every so often
   - Update *conf/release.mk* to add 'conf-hide = -fvisibility=hidden' and 
   'conf-hide-cxx = -fvisibility-inlines-hidden' to allow turning hiding of 
   symbols on or off.
   - Update *main Makefile*