Re: [PATCH] open_issues/bcachefs.mdwn: new file.
Applied, thanks! jbra...@dismail.de, le sam. 06 janv. 2024 14:59:40 -0500, a ecrit: > Well, we might as well document our conversation with Kent about bachchefs. > > --- > open_issues/bcachefs.mdwn | 326 ++ > 1 file changed, 326 insertions(+) > create mode 100644 open_issues/bcachefs.mdwn > > diff --git a/open_issues/bcachefs.mdwn b/open_issues/bcachefs.mdwn > new file mode 100644 > index ..aa39bce0 > --- /dev/null > +++ b/open_issues/bcachefs.mdwn > @@ -0,0 +1,326 @@ > +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software > Foundation, > +Inc."]] > + > +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable > +id="license" text="Permission is granted to copy, distribute and/or modify > this > +document under the terms of the GNU Free Documentation License, Version 1.2 > or > +any later version published by the Free Software Foundation; with no > Invariant > +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the > license > +is included in the section entitled [[GNU Free Documentation > +License|/fdl]]."]]"""]] > + > +[[!tag open_issue_hurd]] > + > +The Hurd's primary filesystem is ext2, which works but lacks modern > +features. With ext2, Hurd users reguarly deal with filesystem > +corruption. Ext2 does not have a journal, so Hurd users occasionally > +have to deal with filesystem corruption. `fsck` can fix most of the > +issues (with loss of random data), but without a proper journal the > +Hurd currently is not a good a OS for long-term data storage. > + > +Bcachefs is a modern COW (copy-on-write) open source filesystem for > +Linux, which intends to replace Btrfs and ZFS while having the > +performance of ext4 or XFS. It is almost 100,000 lines of code. > +Btrfs is 150,000 lines of code. Bcachefs is structured as a > +filesystem built on top of a database. There is a clean small > +database transaction layer. That core database library is maybe > +25,000 lines of code. > + > +Some Hurd developers recently [[talked with > +Bcachefs|https://youtube.com/watch?v=bcWsrYvc5Fg]] author Kent > +Overstreat about porting bcachefs to the Hurd. There are currently no > +concrete plans to do so due to lack of developer man power. > + > +90% of the Bcachefs filesystem code builds and runs in userspace. It > +uses a shim layer that makes maps kernel locking primatives to > +pthreads, the kernel io API is mapped to AIO, etc. Bcachefs does > +intend to eventually rewrite most or all of its current codebase into > +rust. > + > +Kent is ok with us merging a shim layer for libstore that maps to the > +Unix filesystem API. That would be a header file that goes into the > +bcachefs code. > + > +There is a somewhat working FUSE port of bcachefs, but Kent is not > +certain that is a good way to run bcachefs in userspace. Kent wants > +to use the FUSE port to help in debbugging. Suppose bcachefs starts > +acting up, then you could switch to running it in userspace and attach > +GDB to the running process. This is currently not possible. > + > +We could port bcachefs to the Hurd's native filesystem API: libdiskfs. > + > +One interesting aspect of the conversation was Kent's goal of re-using > +kernel code in userspace. The Linux kernel hashtable code is high > +performance, resizeable, lockless, and builds and runs in userspace. > +As long as you have liburcu, then you can use the kernel hashtable in > +userspace on the Hurd. This might be useful to use on the Hurd. > + > +Bcachefs is liscensed as GPLv2, and many of Kent's previous employers > +own the patents, including Google. Kent is ok with potentially making > +the license GPLv2+, as long as there was not a promise to keep > +bcachefs GPLv2 only. > + > +# IRC logs > + > +https://logs.guix.gnu.org/hurd/2023-09-26.log > + > +maybe I'm wrong though, do you know much about fuse? or > file systems? > + no i dont know much about filesystems > + what is bcachefs? > +see? :D > + I agree that someone intimate in the Mach pager api, libdiskfs > and fuse would be great at that meeting > +I do kind of understand Mach VM / paging, I must say > +from the looks of it, I even understand it best among > those who have looked at it recently > +and I mostly understand libdiskfs > + so go to the meeting > + what is fuse? do we even need it for hurd? > + file systems in userspace > +FUSE is "filesystem in user space", it's both the name > for the concept, and the name of Linux's specific mechanism, of offloading fs > to userland > + yeah, i think it may be unneeded for filesystem on hurd > +it's basically a giant hack that pretends to be a > fileystem implementation to the rest of the kernel, and then sends requests > and receives responses from a userland program that _actually_ implements the > fs > +on the Hurd, *of course* filesystems are implemented in > userland, that
[PATCH] open_issues/bcachefs.mdwn: new file.
Well, we might as well document our conversation with Kent about bachchefs. --- open_issues/bcachefs.mdwn | 326 ++ 1 file changed, 326 insertions(+) create mode 100644 open_issues/bcachefs.mdwn diff --git a/open_issues/bcachefs.mdwn b/open_issues/bcachefs.mdwn new file mode 100644 index ..aa39bce0 --- /dev/null +++ b/open_issues/bcachefs.mdwn @@ -0,0 +1,326 @@ +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +The Hurd's primary filesystem is ext2, which works but lacks modern +features. With ext2, Hurd users reguarly deal with filesystem +corruption. Ext2 does not have a journal, so Hurd users occasionally +have to deal with filesystem corruption. `fsck` can fix most of the +issues (with loss of random data), but without a proper journal the +Hurd currently is not a good a OS for long-term data storage. + +Bcachefs is a modern COW (copy-on-write) open source filesystem for +Linux, which intends to replace Btrfs and ZFS while having the +performance of ext4 or XFS. It is almost 100,000 lines of code. +Btrfs is 150,000 lines of code. Bcachefs is structured as a +filesystem built on top of a database. There is a clean small +database transaction layer. That core database library is maybe +25,000 lines of code. + +Some Hurd developers recently [[talked with +Bcachefs|https://youtube.com/watch?v=bcWsrYvc5Fg]] author Kent +Overstreat about porting bcachefs to the Hurd. There are currently no +concrete plans to do so due to lack of developer man power. + +90% of the Bcachefs filesystem code builds and runs in userspace. It +uses a shim layer that makes maps kernel locking primatives to +pthreads, the kernel io API is mapped to AIO, etc. Bcachefs does +intend to eventually rewrite most or all of its current codebase into +rust. + +Kent is ok with us merging a shim layer for libstore that maps to the +Unix filesystem API. That would be a header file that goes into the +bcachefs code. + +There is a somewhat working FUSE port of bcachefs, but Kent is not +certain that is a good way to run bcachefs in userspace. Kent wants +to use the FUSE port to help in debbugging. Suppose bcachefs starts +acting up, then you could switch to running it in userspace and attach +GDB to the running process. This is currently not possible. + +We could port bcachefs to the Hurd's native filesystem API: libdiskfs. + +One interesting aspect of the conversation was Kent's goal of re-using +kernel code in userspace. The Linux kernel hashtable code is high +performance, resizeable, lockless, and builds and runs in userspace. +As long as you have liburcu, then you can use the kernel hashtable in +userspace on the Hurd. This might be useful to use on the Hurd. + +Bcachefs is liscensed as GPLv2, and many of Kent's previous employers +own the patents, including Google. Kent is ok with potentially making +the license GPLv2+, as long as there was not a promise to keep +bcachefs GPLv2 only. + +# IRC logs + +https://logs.guix.gnu.org/hurd/2023-09-26.log + + maybe I'm wrong though, do you know much about fuse? or file systems? + no i dont know much about filesystems + what is bcachefs? + see? :D +I agree that someone intimate in the Mach pager api, libdiskfs and fuse would be great at that meeting + I do kind of understand Mach VM / paging, I must say + from the looks of it, I even understand it best among those who have looked at it recently + and I mostly understand libdiskfs + so go to the meeting + what is fuse? do we even need it for hurd? + file systems in userspace + FUSE is "filesystem in user space", it's both the name for the concept, and the name of Linux's specific mechanism, of offloading fs to userland + yeah, i think it may be unneeded for filesystem on hurd + it's basically a giant hack that pretends to be a fileystem implementation to the rest of the kernel, and then sends requests and receives responses from a userland program that _actually_ implements the fs + on the Hurd, *of course* filesystems are implemented in userland, that's the only and tnhe natural way everything works + but that's where the similarities end + you cannot just take a linux fuse fs, using libfuse, and run it on the Hurd + there has been a project make a library that would have the same API as libfus
