[PATCH] Hurd/HurdDevelopers.mdwn: added Svante's contributions
Sorry for another tiny change. Svante asked me to include his contributions as well. --- Hurd/HurdDevelopers.mdwn | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Hurd/HurdDevelopers.mdwn b/Hurd/HurdDevelopers.mdwn index 5328e37a..18222e13 100644 --- a/Hurd/HurdDevelopers.mdwn +++ b/Hurd/HurdDevelopers.mdwn @@ -10,13 +10,14 @@ License|/fdl]]."]]"""]] The Hurd has very few people working on it. This is a list of people who are large contributors to the project. + * Samuel Thibault - Our fearless leader * Joshua Branson - Developer * Sergey Bugaev - 64 bit port and security fixes * Flávio Cruz - Mach IPC improvements * Almudena Garcia - SMP work and hardware tester * Joan Lledó - PCI arbiter and lwip port -* Svante Signell - pfinet/fakeroot fixes +* Svante Signell - pfinet/fakeroot fixes, file record locking, ported ada/gnat * Damien Zammit - ACPI arbiter, SMP, and rumpkernel Here is a list of people who were large contributors: -- 2.42.0
Re: [PATCH v3] news/2023-q4.mdwn: new qoth for q4 of 2023.
"Dr. Arne Babenhauserheide" writes: > Samuel Thibault writes: > >> Applied, thanks! >> >> jbra...@dismail.de, le dim. 07 janv. 2024 11:18:20 -0500, a ecrit: >>> This is version 3 of the last qoth of 2023. I've incorporated Dr. >>> Arne's comments. > > Are you already spreading this somewhere? The changes are pretty > awesome! Looks like it's already on the wiki: https://www.gnu.org/software/hurd/index.html > > Best wishes, > Arne -- Joshua Branson Sent from the Hurd
Re: [PATCH v3] news/2023-q4.mdwn: new qoth for q4 of 2023.
Samuel Thibault writes: > Applied, thanks! > > jbra...@dismail.de, le dim. 07 janv. 2024 11:18:20 -0500, a ecrit: >> This is version 3 of the last qoth of 2023. I've incorporated Dr. >> Arne's comments. Are you already spreading this somewhere? The changes are pretty awesome! Best wishes, Arne -- Unpolitisch sein heißt politisch sein, ohne es zu merken. draketo.de signature.asc Description: PGP signature
Re: [PATCH] open_issues/bcachefs.mdwn: new file.
Applied, thanks! jbra...@dismail.de, le sam. 06 janv. 2024 14:59:40 -0500, a ecrit: > Well, we might as well document our conversation with Kent about bachchefs. > > --- > open_issues/bcachefs.mdwn | 326 ++ > 1 file changed, 326 insertions(+) > create mode 100644 open_issues/bcachefs.mdwn > > diff --git a/open_issues/bcachefs.mdwn b/open_issues/bcachefs.mdwn > new file mode 100644 > index ..aa39bce0 > --- /dev/null > +++ b/open_issues/bcachefs.mdwn > @@ -0,0 +1,326 @@ > +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software > Foundation, > +Inc."]] > + > +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable > +id="license" text="Permission is granted to copy, distribute and/or modify > this > +document under the terms of the GNU Free Documentation License, Version 1.2 > or > +any later version published by the Free Software Foundation; with no > Invariant > +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the > license > +is included in the section entitled [[GNU Free Documentation > +License|/fdl]]."]]"""]] > + > +[[!tag open_issue_hurd]] > + > +The Hurd's primary filesystem is ext2, which works but lacks modern > +features. With ext2, Hurd users reguarly deal with filesystem > +corruption. Ext2 does not have a journal, so Hurd users occasionally > +have to deal with filesystem corruption. `fsck` can fix most of the > +issues (with loss of random data), but without a proper journal the > +Hurd currently is not a good a OS for long-term data storage. > + > +Bcachefs is a modern COW (copy-on-write) open source filesystem for > +Linux, which intends to replace Btrfs and ZFS while having the > +performance of ext4 or XFS. It is almost 100,000 lines of code. > +Btrfs is 150,000 lines of code. Bcachefs is structured as a > +filesystem built on top of a database. There is a clean small > +database transaction layer. That core database library is maybe > +25,000 lines of code. > + > +Some Hurd developers recently [[talked with > +Bcachefs|https://youtube.com/watch?v=bcWsrYvc5Fg]] author Kent > +Overstreat about porting bcachefs to the Hurd. There are currently no > +concrete plans to do so due to lack of developer man power. > + > +90% of the Bcachefs filesystem code builds and runs in userspace. It > +uses a shim layer that makes maps kernel locking primatives to > +pthreads, the kernel io API is mapped to AIO, etc. Bcachefs does > +intend to eventually rewrite most or all of its current codebase into > +rust. > + > +Kent is ok with us merging a shim layer for libstore that maps to the > +Unix filesystem API. That would be a header file that goes into the > +bcachefs code. > + > +There is a somewhat working FUSE port of bcachefs, but Kent is not > +certain that is a good way to run bcachefs in userspace. Kent wants > +to use the FUSE port to help in debbugging. Suppose bcachefs starts > +acting up, then you could switch to running it in userspace and attach > +GDB to the running process. This is currently not possible. > + > +We could port bcachefs to the Hurd's native filesystem API: libdiskfs. > + > +One interesting aspect of the conversation was Kent's goal of re-using > +kernel code in userspace. The Linux kernel hashtable code is high > +performance, resizeable, lockless, and builds and runs in userspace. > +As long as you have liburcu, then you can use the kernel hashtable in > +userspace on the Hurd. This might be useful to use on the Hurd. > + > +Bcachefs is liscensed as GPLv2, and many of Kent's previous employers > +own the patents, including Google. Kent is ok with potentially making > +the license GPLv2+, as long as there was not a promise to keep > +bcachefs GPLv2 only. > + > +# IRC logs > + > +https://logs.guix.gnu.org/hurd/2023-09-26.log > + > +maybe I'm wrong though, do you know much about fuse? or > file systems? > + no i dont know much about filesystems > + what is bcachefs? > +see? :D > + I agree that someone intimate in the Mach pager api, libdiskfs > and fuse would be great at that meeting > +I do kind of understand Mach VM / paging, I must say > +from the looks of it, I even understand it best among > those who have looked at it recently > +and I mostly understand libdiskfs > + so go to the meeting > + what is fuse? do we even need it for hurd? > + file systems in userspace > +FUSE is "filesystem in user space", it's both the name > for the concept, and the name of Linux's specific mechanism, of offloading fs > to userland > + yeah, i think it may be unneeded for filesystem on hurd > +it's basically a giant hack that pretends to be a > fileystem implementation to the rest of the kernel, and then sends requests > and receives responses from a userland program that _actually_ implements the > fs > +on the Hurd, *of course* filesystems are implemented in > userland,
Re: [PATCH] added Sergey as a Hurd Developer
Updated, thanks! jbra...@dismail.de, le sam. 06 janv. 2024 15:22:23 -0500, a ecrit: > I also added Almudena Garcia as well. > > * Hurd/HurdDevelopers.mdwn: reorganized and added Sergey as a Hurd Developer. > --- > Hurd/HurdDevelopers.mdwn | 27 ++- > 1 file changed, 14 insertions(+), 13 deletions(-) > > diff --git a/Hurd/HurdDevelopers.mdwn b/Hurd/HurdDevelopers.mdwn > index 908ebb04..5328e37a 100644 > --- a/Hurd/HurdDevelopers.mdwn > +++ b/Hurd/HurdDevelopers.mdwn > @@ -9,27 +9,28 @@ License|/fdl]]."]]"""]] > [[!meta title="Hurd Developers"]] > > The Hurd has very few people working on it. This is a list of people who are > large contributors to the project. > - > > -* Thomas Schwinge - Maintainer > -* Samuel Thibault - Developer > -* Joshual Branson - Developer > -* Justus Winter - Developer > +* Samuel Thibault - Our fearless leader > +* Joshua Branson - Developer > +* Sergey Bugaev - 64 bit port and security fixes > +* Flávio Cruz - Mach IPC improvements > +* Almudena Garcia - SMP work and hardware tester > * Joan Lledó - PCI arbiter and lwip port > -* Damien Zammit - ACPI arbiter > * Svante Signell - pfinet/fakeroot fixes > -* Flávio Cruz - fixes > -* Manolis Ragkousis - Hurd support for Guix > +* Damien Zammit - ACPI arbiter, SMP, and rumpkernel > > Here is a list of people who were large contributors: > > -* Zheng Da - netdde > -* Jeremie Koenig - Debian installer > -* Marcus Brinkman - Co-author of the Critique > -* Neal H. Walfield - Co-author of the Critique > -* Richard Braun - Owner of darnassus.sceen.net > * Miles Bader - Original developer > +* Richard Braun - Owner of darnassus.sceen.net > +* Marcus Brinkman - Co-author of the Critique > * Thomas Bushnell - Original maintainer > +* Zheng Da - netdde > +* Jeremie Koenig - Debian installer > * Roland McGrath - Original maintainer > +* Manolis Ragkousis - Hurd support for Guix > +* Thomas Schwinge - Previous Maintainer > +* Neal H. Walfield - Co-author of the Critique > +* Justus Winter - Developer > > [See the full list at > Savannah](https://savannah.gnu.org/project/memberlist.php?group=hurd) > -- > 2.42.0 > > -- Samuel --- Pour une évaluation indépendante, transparente et rigoureuse ! Je soutiens la Commission d'Évaluation de l'Inria.
Re: [PATCH v3] news/2023-q4.mdwn: new qoth for q4 of 2023.
Applied, thanks! jbra...@dismail.de, le dim. 07 janv. 2024 11:18:20 -0500, a ecrit: > This is version 3 of the last qoth of 2023. I've incorporated Dr. > Arne's comments. > > --- > news/2023-q4.mdwn | 121 ++ > 1 file changed, 121 insertions(+) > create mode 100644 news/2023-q4.mdwn > > diff --git a/news/2023-q4.mdwn b/news/2023-q4.mdwn > new file mode 100644 > index ..294b6b22 > --- /dev/null > +++ b/news/2023-q4.mdwn > @@ -0,0 +1,121 @@ > +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] > + > +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable > +id="license" text="Permission is granted to copy, distribute and/or modify > this > +document under the terms of the GNU Free Documentation License, Version 1.2 > or > +any later version published by the Free Software Foundation; with no > Invariant > +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the > license > +is included in the section entitled [[GNU Free Documentation > +License|/fdl]]."]]"""]] > + > +[[!meta date="2024-01-05 22:22 UTC"]] > + > +Hello! Welcome to a new qoth. This qoth covers new and interesting GNU/Hurd > +developments in Q4 of 2023! > +[[!if test="included()" then="""[[!toggle id=full_news > +text="Details."]][[!toggleable id=full_news text="[[!paste > id=full_news]]"]]""" > +else=" > +[[!paste id=full_news]]"]] > + > +[[!cut id="full_news" text=""" > + > +Samuel Thibault fixed gcc's Hurd's default pie and [[added static pie > +support|https://lists.gnu.org/archive/html/bug-hurd/2023-10/msg00061.html]]. > +He also added a [[whatis > +command|https://lists.gnu.org/archive/html/bug-hurd/2023-10/msg7.html]] > +to GNU Mach's kernel debugger, which can determine what an > +address points to (a stack? a port? some kalloc? ...). He also added > +[[hurd-amd64 support to > +GCC|https://lists.debian.org/debian-hurd/2023/11/msg00039.html]]. > + > +Samuel requested that the Hurd team set up a [[continuous > +integration,|https://lists.gnu.org/archive/html/bug-hurd/2023-12/msg7.html]] > +so that when developers make code changes, they can be certain that > +they did not break anything. It turns out that the Hurd supports > +several different environments: 32 bit, 64 bit, 32-on-64 bit, ACPI, > +non-ACPI, SMP, non-SMP, Xen, etc. Apparently Flavio has a [[personal > +CI|https://github.com/flavioc/cross-hurd/actions/runs/7080757561]], > +but it is set up in a Debian independent way. If you are interested in > +helping the Hurd project set up a CI, then please get in touch! > + > +Luca Dariz worked on adding [[some simple GNU Mach user-space tests > +|https://lists.gnu.org/archive/html/bug-hurd/2023-10/msg00031.html]]. > +With a working MiG, a GNU/Linux machine can run make check in the GNU > +Mach source code, which will launch qemu to ensure that 32 bit (PAE > +and non PAE), 32 on 64 bit, and full 64 bit GNU Mach works. We > +currently do this testing on GNU/Linux, because qemu does not run on > +the Hurd. > + > +Many people worked on the Hurd's new [[x86_64 bit > +support|https://lists.gnu.org/archive/html/bug-hurd/2023-10/msg00036.html]]. > +A 64-bit debian bbuild is set up, and we can bootstrap a chroot! The > +hurd-amd64 wanna-build infrastructure is also set up. We are having > +issues reliably building packages on a 64-bit Hurd, which lead Samuel > +to uncover and fix [[a proc > +leak|https://lists.gnu.org/archive/html/bug-hurd/2023-11/msg00035.html]]. > + > +Flavio Cruz [[improved GNU Mach's > +IPC|https://lists.gnu.org/archive/html/bug-hurd/2023-11/msg00033.html]] > +by reordering `mach_msg_type_t` fields to byte align `msgt_name` and > +`msgt_size`. He also created a patch series to [[avoid message > +resizing for > +x86_64|https://lists.gnu.org/archive/html/bug-hurd/2023-11/msg00028.html]]. > +He also [[removed untyped mach RPC > +code|https://lists.gnu.org/archive/html/bug-hurd/2023-11/msg00026.html]]. > +GNU Mach uses typed IPC. The Hurd could support both typed and > +untyped, but it appears that the Hurd only uses typed RPC. So it > +seems best to remove any untyped RPC code. > + > +Sergey Bugaev added [[GNU Mach entry re-coalescing > +support|https://darnassus.sceen.net/~hurd-web/open_issues/gnumach_vm_map_entry_forward_merging/]]. > +Essentially, Mach is not always able to merge two vm entries that are > +made next to each other, which slows down ext2 and bash. Sergey > +allowed GNU Mach to merge entries in the common cases, which greatly > +helps ext2fs. > + > +Sergey is also working on [[porting the Ladybird web > +browser|https://lists.gnu.org/archive/html/bug-hurd/2023-11/msg00013.html]] > +to the Hurd. The author of this post uses the [[netsurf web > +browser|https://www.netsurf-browser.org/]] on the Hurd, which works on > +simple websites like wikipedia, but it badly renders javascript heavy > +websites, which makes many websites un-useable. If Sergey is > +successful in porting
Re: [PATCH] open_issues/arm_port.mdwn: documented Sergeys AArch64 port status.
Applied, thanks!
jbra...@dismail.de, le mar. 09 janv. 2024 15:20:47 -0500, a ecrit:
> I figure that we might as well document the AArch64 port status on the wiki.
>
> ---
> open_issues/arm_port.mdwn | 172 +-
> 1 file changed, 134 insertions(+), 38 deletions(-)
>
> diff --git a/open_issues/arm_port.mdwn b/open_issues/arm_port.mdwn
> index 26e0b770..8a2bc27f 100644
> --- a/open_issues/arm_port.mdwn
> +++ b/open_issues/arm_port.mdwn
> @@ -9,56 +9,152 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts.
> A copy of the license
> is included in the section entitled [[GNU Free Documentation
> License|/fdl]]."]]"""]]
>
> -Several people have expressed interested in a port of GNU/Hurd for the ARM
> -architecture.
> +Several people have expressed interested in a port of GNU/Hurd for the
> +ARM architecture. Luckily a userspace port of the Hurd servers and
> +glibc is underway. As early as January 1, 2024 an AArch64 port is
> +making some progress. Sergey did some hacking on glibc, binutils,
> +GCC, and added some headers to GNU Mach. He was able to build the
> +core Hurd servers: ext2fs, proc, exec, and auth.
>
> +One would think that he would need to port GNU Mach to run the
> +binaries, but Sergey ran a statically linked hello world executable on
> +GNU/Linux, under GDB, being careful to skip over and emulate syscalls
> +and RPCs. The glibc port has the TLS setup, hwcaps / cpu-features,
> +and ifuncs.
>
> -# IRC, freenode, #hurd, 2012-07-28
> +Now to some of the more technical things:
>
> - Has anyone heard about porting hurd and gnu/mach to arm
> - architecture?
> - mcsim: i think so
> - mcsim: why are you asking ?
> - I found an article where author stated that he has ported hurd to
> - arm, but I have never met this information before.
> - He wrote ethernet driver and managed to use ping command
> - author's name is Sartakov Vasily
> - well that's possible, a long time ago
> - and it was probably not complete enough to be merged upstream
> - like many other attempts at many other things
> - Not so long. Article is dated by June 2011.
> - do you have a link ?
> - Yes, but it is in Russian.
> - oh
> - well i don't remember him sharing that with us
> - mcsim: he did some work on porting Mach, but AIUI never got it
> - nearly finished
> - nowadays he does L4 stuff
> - was also at FOSDEM
> +- The TLS implementation is basically complete and working. We're
> +using `tpidr_el0` for the thread pointer (as can be seen in the listing
> +above), like GNU/Linux and unlike Windows (which uses x18, apparently)
> +and macOS (which uses `tpidrro_el0`). We're using "Variant I" layout, as
> +described in "ELF Handling for Thread-Local Storage", again same as
> +GNU/Linux, and unlike what we do on both x86 targets. This actually
> +ends up being simpler than what we had for x86! The other cool thing
> +is that we can do `msr tpidr_el0, x0` from userspace without any
> +gnumach involvement, so that part of the implementation is quite a bit
> +simpler too.
>
> +- Conversely, while on x86 it is possible to perform "cpuid" and
> +identify CPU features entirely in user space, on AArch64 this requires
> +access to some EL1-only registers. On Linux and the BSDs, the kernel
> +exposes info about the CPU features via `AT_HWCAP` (and more recently,
> +`AT_HWCAP2`) auxval entries. Moreover, Linux allows userland to read
> +some otherwise EL1-only registers (notably for us, `midr_el1`) by
> +catching the trap that results from the EL0 code trying to do that,
> +and emulating its effect. Also, Linux exposes `midr_el1` and
> +`revidr_el1` values through procfs.
>
> -## IRC, freenode, #hurd, 2012-10-09
> +- The Hurd does not use `auxval`, nor is gnumach involved in `execve` anyway.
> +So I thought the natural way to expose this info would be with an RPC,
> +and so in `mach_aarch64.defs` I have an `aarch64_get_hwcaps` routine that
> +returns the two hwcaps values (using the same bits as `AT_HWCAP{,2}`) and
> +the values of `midr_el1`/`revidr_el1`. This is hooked to `init_cpu_features`
> +in glibc, and used to initialize `GLRO(dl_hwcap)` / `GLRO(dl_hwcap2)` and
> +eventually to pick the appropriate ifunc implementations.
>
> - bootinfdsds: There was an unfinished port to arm, if you're
> - interested.
> - mcsim: Has that ever been published?
> - tschwinge: I don't think so. But I have an email of that person
> and
> - I think that this could be discussed with him.
> +- The page size (or rather, paging granularity) is notoriously not
> +necessarily 4096 on ARM, and the best practice is for userland not to
> +assume any specific page size and always query it dynamically. GNU
> +Mach will (probably) have to be built support for some specific page
> +size, but I've cleaned up a few places in glibc where things were
> +relying on a statically defined page size.
>
> +-
[PATCH] open_issues/arm_port.mdwn: documented Sergeys AArch64 port status.
I figure that we might as well document the AArch64 port status on the wiki.
---
open_issues/arm_port.mdwn | 172 +-
1 file changed, 134 insertions(+), 38 deletions(-)
diff --git a/open_issues/arm_port.mdwn b/open_issues/arm_port.mdwn
index 26e0b770..8a2bc27f 100644
--- a/open_issues/arm_port.mdwn
+++ b/open_issues/arm_port.mdwn
@@ -9,56 +9,152 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A
copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-Several people have expressed interested in a port of GNU/Hurd for the ARM
-architecture.
+Several people have expressed interested in a port of GNU/Hurd for the
+ARM architecture. Luckily a userspace port of the Hurd servers and
+glibc is underway. As early as January 1, 2024 an AArch64 port is
+making some progress. Sergey did some hacking on glibc, binutils,
+GCC, and added some headers to GNU Mach. He was able to build the
+core Hurd servers: ext2fs, proc, exec, and auth.
+One would think that he would need to port GNU Mach to run the
+binaries, but Sergey ran a statically linked hello world executable on
+GNU/Linux, under GDB, being careful to skip over and emulate syscalls
+and RPCs. The glibc port has the TLS setup, hwcaps / cpu-features,
+and ifuncs.
-# IRC, freenode, #hurd, 2012-07-28
+Now to some of the more technical things:
- Has anyone heard about porting hurd and gnu/mach to arm
- architecture?
- mcsim: i think so
- mcsim: why are you asking ?
- I found an article where author stated that he has ported hurd to
- arm, but I have never met this information before.
- He wrote ethernet driver and managed to use ping command
- author's name is Sartakov Vasily
- well that's possible, a long time ago
- and it was probably not complete enough to be merged upstream
- like many other attempts at many other things
- Not so long. Article is dated by June 2011.
- do you have a link ?
- Yes, but it is in Russian.
- oh
- well i don't remember him sharing that with us
- mcsim: he did some work on porting Mach, but AIUI never got it
- nearly finished
- nowadays he does L4 stuff
- was also at FOSDEM
+- The TLS implementation is basically complete and working. We're
+using `tpidr_el0` for the thread pointer (as can be seen in the listing
+above), like GNU/Linux and unlike Windows (which uses x18, apparently)
+and macOS (which uses `tpidrro_el0`). We're using "Variant I" layout, as
+described in "ELF Handling for Thread-Local Storage", again same as
+GNU/Linux, and unlike what we do on both x86 targets. This actually
+ends up being simpler than what we had for x86! The other cool thing
+is that we can do `msr tpidr_el0, x0` from userspace without any
+gnumach involvement, so that part of the implementation is quite a bit
+simpler too.
+- Conversely, while on x86 it is possible to perform "cpuid" and
+identify CPU features entirely in user space, on AArch64 this requires
+access to some EL1-only registers. On Linux and the BSDs, the kernel
+exposes info about the CPU features via `AT_HWCAP` (and more recently,
+`AT_HWCAP2`) auxval entries. Moreover, Linux allows userland to read
+some otherwise EL1-only registers (notably for us, `midr_el1`) by
+catching the trap that results from the EL0 code trying to do that,
+and emulating its effect. Also, Linux exposes `midr_el1` and
+`revidr_el1` values through procfs.
-## IRC, freenode, #hurd, 2012-10-09
+- The Hurd does not use `auxval`, nor is gnumach involved in `execve` anyway.
+So I thought the natural way to expose this info would be with an RPC,
+and so in `mach_aarch64.defs` I have an `aarch64_get_hwcaps` routine that
+returns the two hwcaps values (using the same bits as `AT_HWCAP{,2}`) and
+the values of `midr_el1`/`revidr_el1`. This is hooked to `init_cpu_features`
+in glibc, and used to initialize `GLRO(dl_hwcap)` / `GLRO(dl_hwcap2)` and
+eventually to pick the appropriate ifunc implementations.
- bootinfdsds: There was an unfinished port to arm, if you're
- interested.
- mcsim: Has that ever been published?
- tschwinge: I don't think so. But I have an email of that person and
- I think that this could be discussed with him.
+- The page size (or rather, paging granularity) is notoriously not
+necessarily 4096 on ARM, and the best practice is for userland not to
+assume any specific page size and always query it dynamically. GNU
+Mach will (probably) have to be built support for some specific page
+size, but I've cleaned up a few places in glibc where things were
+relying on a statically defined page size.
+- There are a number of hardware hardening features available on AArch64
+(PAC, BTI, MTE — why do people keep adding more and more workarounds,
+including hardware ones, instead of rewriting software in a properly
+memory-safe language...). Those are not really supported right now; all
