Hello waddlesplash, thank you for responding and the additional information you provided.
I have not received any feedback via the NetBSD mailing list unfortunately, so I kept working on this to come back with something more substantial. The following milestones have been reached: 1) There are now kernel drivers for ports, semaphores and areas. These were developed and tested carefully, so they should not be too bad. https://github.com/stephanwib/src/blob/trunk/sys/kern/uipc_port.c https://github.com/stephanwib/src/blob/trunk/sys/kern/uipc_hsem.c https://github.com/stephanwib/src/blob/trunk/sys/kern/uipc_area.c 2) libroot was also extended a lot. It provides the interfaces to the said kernel subsystems as Haiku/BeOS expect them. Also, the thread interface was introduced on top of NetBSDs libpthread, which was quite enjoyable as it provides some (non-standard) extensions which were necessary to get it working right. I also made a little change to libpthread to obtain kernel-level LWP IDs. A bunch of other functions, especially the get_team/thread/system_info functions were written based on the sysctl interface or libkvm. I have also included some files from Cosmoe or Haiku implementing certain other parts - these are only compile-tested at the moment. (As a side note, work on Cosmoe has been resumed recently, see https://gitlab.com/haydentech) 3) I have used tests from the Haiku source where available and there are no obvious issues anymore. (I have also some custom test programs which are not in the repo). Also, certain CLI programs work like listsem, listports, listarea and ps. 4) The process of integrating Haiku code has now started. https://github.com/stephanwib/haiku In this fork, I am implementing Makefiles to build libbe, which is already done except for the storage kit. There were only some small modifications needed to get the files compiled. Regards, Stephan Am Fr., 1. Dez. 2023 um 06:34 Uhr schrieb waddlesplash <waddlespl...@gmail.com>: > > Greetings, > > Haiku developer here. > > It's neat to see the BSDs looking at whether they can borrow anything > from us, when we've gained much by borrowing from the BSDs over the > years (from FreeBSD and OpenBSD for WiFi drivers, and NetBSD for our > DNS resolver.) > > However, there are a few misconceptions and some potential hurdles I > figured I might point out. > > On Mon, 27 Nov 2023 14:38:13 +0100, Stephan wrote: > > What makes it so special is that it is extremely integrated > > A significant amount of that integration is not just between the > various parts of the GUI and "desktop environment", though, but > between the userland and the kernel; not just in things like the > native IPC mechanisms, but also in things like the file manager, > network manager, partition editor, Terminal, etc. Porting just the > Application and Interface Kits to other platforms won't be too hard > (and has indeed been done before, as you note); porting the whole > desktop environment in a usable way is another matter. > > > It has a zero-latency, pixel-perfect looking GUI > > Well, I'd like for both of those to be true, but of course they're > not! Certainly Haiku has a lot lower latency than a standard Linux or > BSD GUI setup, but it's far from zero. > > More to the point, the lower latency is in large part due to the input > handling setup. That's a pretty good example of the system's overall > tight coupling: the input_server talks directly to the various kernel > drivers (PS/2, USB HID, etc., all of which are Haiku-native and > implement Haiku-specific interfaces) and then hands off processed > input to the app_server, which distributes it to application(s) > depending on what has focus. > > Some of the snappiness is due to architectural choices that will > follow the Interface Kit wherever it goes (e.g. the > every-window-is-a-thread paradigm), but some of it will either be lost > or impaired by the usual ways the stack has previously been ported > elsewhere (e.g. if you run the Interface Kit as "just another toolkit" > on top of X11/Wayland, or even run app_server as a client server like > XWayland does on top of Wayland.) > > If you want to run app_server as the actual display server + window > manager directly working with the system framebuffer, modesetting, > etc. that may retain more; but will also require a lot more work. > > > That is why I always found it interesting to rebase it on > > a mature unixish operating system, especially NetBSD. > > While the Be engineers may have incorporated some concepts of Unix > > into their OS, it is not so much a Unix or POSIX system. > > I know this point has been debated, but at least I am firmly in the > camp that Haiku itself *is* a "unixish" operating system (and so was > BeOS, to a lesser extent.) Both BeOS and Haiku use the fork/exec-based > process model (though fork() in a GUI application indeed doesn't make > much sense, so there were ways of spawning processes without fork/exec > even before posix_spawn came along.) BeOS lacked mmap, pthreads, and > other such "UNIXy" staples, but it had others like "unistd.h" and the > like. > > Haiku, meanwhile, is "natively POSIX" in a pretty serious way. Yes, > there's still a distinction between the Haiku-native threads API and > then pthreads, but that's a userspace-only distinction and more an > implementation detail of the libroot than anything else (the only > important difference is having a pthread structure in a TLS slot, > AFAIK.) Meanwhile POSIX mmap() on Haiku and then the Be/Haiku "areas" > API are just two ways to interact with the same thing (one can call > "delete_area" on a mmap'ed region, or "munmap" on a create_area'd > region, and they'll both work, because to the kernel it's all the same > thing in the end.) > > Likewise, the networking layer, file permissions, and so much more is > all pretty POSIXy, and has been for a long time. The number of > non-implemented POSIX APIs is pretty small (I'm not sure how many > there even are; probably they're mostly obscure or optional things > that aren't too important, seeing as we've ported some pretty large > applications like QtWebEngine/Chromium.) > > > For example, it has its own error codes > > Doesn't everyone? I'd be surprised if there was any OS that really had > no more errors besides the ones specified in POSIX. > > The real oddity is that Be/Haiku errors are always negative. You'll > never see e.g. "-EINVAL" on Haiku, because EINVAL is itself a negative > number. That feature is relied upon pretty heavily throughout the API, > and it's one of the reasons why no Haiku-native API ever needs errno, > because it'll just return its error codes directly (for example, > create_area() either returns an area_id > 0 on success, or an error > code < 0 on failure.) > > Replicating that will be important, or else there's a lot of code > stashed in hard-to-find places that will subtly misbehave... > > > IPC > > The main IPC mechanism (ports) is more than a little analogous to > AF_UNIX SOCK_DGRAM, and while it's not 1:1 (it's mostly > "connectionless", for instance), you could probably manage to > implement it on top of that (plus a few extensions) if you needed or > wanted to (though it may not be the most efficient.) > > > data types and generic lower-level C functions > > I think this is actually the bit that will give you the most trouble > to reimplement, not IPC or anything like that. > > BeOS and Haiku make serious use of extended filesystem attributes, to > the point where the OS will not function properly without them (it may > not even boot, depending.) Specifically these attributes need to be > *typed* (they aren't always strings; a 4-byte type code is stored > along with the attribute), they need to be of arbitrary lengths (the > total size of attributes on some files will easily reach a few KB; one > cannot actually build Haiku on e.g. ext4 using native ext4 extended > attributes because of this), and they need to be easily enumerable > (note the functions `fs_open_attr_dir` and friends; these allow for > enumerating attributes on a file like `readdir` for a directory). > > That's just the basics; there are some other features of attributes > that will be even trickier to implement (for example: it's possible on > Haiku for symlinks to have attributes, separate from the file they > refer to.) I don't think any other filesystem save BeFS even supports > all these features natively (though some you could implement.) > > Beyond that, there's the "query" system, which is also used for key > functionality (including "Open with" to find supporting applications) > and also isn't supported by anything except BeFS (well, and Haiku's > RAMFS, but that doesn't really count, and also packagefs.) > > There's other things beyond that, but those are the major bits, I think. > > > No, for example there was a project called cosmoe with the aim of > > making the Haiku userland run on top of a POSIX OS. ... I tried > > this on (I think) NetBSD 6 and got it working to some extent but it > > was very unstable. > > It's also very old; it's not from 2014, that repository is a source > import of a much older project (2004, maybe?) > > So in addition to being written for a different era of the Linux/BSDs, > it's also using a base of Haiku sources that pre-date even R1/alpha1 > (2009). Haiku in those days was far from being the relatively stable > and usable platform it is now, and I'd imagine that shows in Cosmoe > itself. > > Probably of interest to you is a much more recent project called "HyClone": > > https://github.com/trungnt2910/hyclone > > This project can run Haiku's ELF binaries directly on Linux (by > implementing the syscall hooks, ioctls, etc. and implementing those > directly; it's not especially Linux specific, but as the README notes, > it should be portable to other SysV OSes, including Haiku itself.) > It's sophisticated enough that it can run Haiku's own app_server (in > headless mode) as well as client applications, as the screenshot in > its README demonstrates. > > > All of this should be considered a concept or prototype at the moment. > > Anyhow, I was able to compile the original Haiku port IPC test > > programs and they all ran successfully. > > This is definitely one area where I would recommend reading the Haiku > kernel source, because chances are there are going to be subtleties > there that the tests do not capture (our test suite is also not > especially complete, for that matter.) As most of Haiku is under the > MIT license (including the core kernel itself), licensing shouldn't be > a problem. > > The source code for the ports implementation, specifically, is > primarily in this file: > > https://github.com/haiku/haiku/blob/master/src/system/kernel/port.cpp > > (Note that all the functions beginning _user_ are the syscall hooks, > so e.g. _user_get_port_info is the syscall implementation; in libroot > its hook will be called _kern_get_port_info.) > > Hopefully some of this information is informative and/or helpful to > you. At any rate, if you intend to continue working on this, feel free > to hang out in our IRC channel (OFTC#haiku) and ask questions about > our sources, internals, etc. > > -waddlesplash
