Re: [PATCH v2 00/20] aarch64-gnu port & GNU/Hurd on AArch64 progress

2024-03-23 Thread Samuel Thibault
Hello,

Sergey Bugaev, le sam. 23 mars 2024 20:32:41 +0300, a ecrit:
> This is v2 of my work on the aarch64-gnu port, aka GNU/Hurd on 64-bit
> ARM. v1 is here [0].

Thanks!

I applied the easy parts: patches 1-6 and 18 for now.

Samuel



[PATCH v2 00/20] aarch64-gnu port & GNU/Hurd on AArch64 progress

2024-03-23 Thread Sergey Bugaev
Hello!

This is v2 of my work on the aarch64-gnu port, aka GNU/Hurd on 64-bit
ARM. v1 is here [0].

[0]: https://sourceware.org/pipermail/libc-alpha/2024-January/153675.html

Last time, Joseph Myers has pointed out that the aarch64-gnu port can
not be merged into glibc until aarch64-gnu support is upstream in all of
glibc's build-time dependencies. That is still not the case, so this
patchset can not be merged yet; but otherwise it should be in a fairly
mergeable state. It does not yet anything to NEWS or
build-many-glibcs.py, though.

I'm porting this, again, to gather some feedback (hopefully more than
last time...), and at Maxim's request, so Linaro can test this on their
CI and ensure this doesn't break existing ports.

The upstreaming status of various aarch64-gnu components is,
specifically:

* Binutils patch to add the aarch64-gnu target is upstream and in the
  2.42 release;
* GCC patches to add aarch64-gnu target (& libgcc host) have been
  reviewed by ARM and Hurd port maintainers and should land upstream
  very soon;
* initial Hurd patches are upstream;
* glibc patches (this patchset) are not yet upstream;
* GNU Mach changes are not upstream, and upstreaming story is unclear;
* GNU MIG needs no changes, it just works.

Last time, there was no AArch64 port of GNU Mach, and so the only
testing I have done was running a simple statically-linked executable on
Linux under GDB -- which, nevertheless, helped me identify and fix a
number of issues.

Since then, however, I have been (some may say, relentlessly) working on
filling in the missing piece, namely porting gnumach (with important
help & contributions by Luca D.). I am happy to report that we now have
an experimental port of gnumach that builds and works on AArch64! While
that may sound impressive, note that various things about it are in an
extremely basic, proof-of-concept state rather than being seriously
production-ready; and also that Mach is a small kernel (indeed, a
microkernel), and it was designed from the start (back in the 80s) to be
portable, so most of the "buisness logic" functionality (virtual memory,
IPC, tasks/threads/scheduler) is explicitly arch-independent.

Despite the scary "WIP proof-of-concept" status, there is enough
functionality in Mach to run userland code, handle exceptions and
syscalls, interact with the MMU to implement all the expected virtual
memory semantics, schedule/switch tasks and threads, and so on.
Moreover, all of gnumach's userspace self-tests pass!

This meant there was enough things in place for me to try running glibc
on it, and the amazing thing is my simple test executable, the same one
I previously tested on Linux w/ GDB, just worked on real Mach without me
having to make any additional changes to the glibc side, or even
recompile it.

But I did not stop there, and got several of the core Hurd servers
working! Namely, these are ext2fs, exec, startup, auth, and proc
servers. All of them but ext2fs are dynamically linked; ld-aarch64.so.1
sucessfully locates and maps the programs themselves and their required
dependencies, and Mach pages in code and data pages from ext2fs as they
are accessed, transparently to the program, just as one would expect it
to.

It turned out that Mach on i386 and x86_64 did not enforce the (lack of)
execute permission on pages, i.e. even pages mapped without
VM_PROT_EXECUTE were executable in practice. This caused a number of
bugs related to mapping executable stacks to go unnoticed, there were
issues in all of Mach, glibc, and the Hurd's exec server related to
not creating executable stacks as actually executable. As I implemented
the execute permission properly in Mach on AArch64 (indeed, I even
support execute-only pages when the hardware implements FEAT_EPAN), I
have encountered all of those oversights one by one when trying to run
progressively more code, and have hopefully fixed them all. Hopefully
we'll stop requiring executable stacks for glibc and Hurd libraries some
time, and then we'll get working non-executable stacks on AArch64.

As expected, I have done some tweaks to the AArch64-specific Mach APIs
(primarily thread state and exception code definitions) compared to the
"preliminary sketches" of them that I posted in January, but they were
actually rather small. I've got some more confidence in the APIs now
after having implemented support for them from both sides now, and
having tested that it works in practice. No more backwards-incompatible
changes to AArch64-specific Mach APIs are expected (by me anyway); we'll
definetely want to add more things later (aarch64_debug_state for GDB,
PAC RPCs, and more), but those should be purely additive.

I have added a new Mach syscall (trap), thread_set_self_state (), to
implement sigreturn () on top of. I have originally hoped that it would
be possible to use the regular thread_set_state (mach_thread_self ())
call for it (special-casing it on AArch64 to allow setting the calling
thread's state), and indeed have initially