Hi Guy, I finally found time to read through your notes (rtld and plan B). great writeup and clarify lots of things. let me go straight:
two impression: the reason to plan B is mainly we want to make the user shell prompt milestone; It is the right approach for the milestone, but is it right for the port? second, lots of things (like thread) in the kernel are not ready for moving to userland port. we still need to stay in kernel for a while. I hope i am wrong. yet if some makes sense, what is the right approach for this if not considering the milestones (none software project is on time, always:). I like the libc-lite plan too, if something in the kernel not ready, go back to kernel. we may also need a init-lite. also the debugger is a must-have. thanks Noah On Feb 6, 2008 9:03 PM, Guy Shaw <Guy.Shaw at sun.com> wrote: > In my last status message, I described some of the problems with > getting rtld+libc up and running, and said that I felt compelled > to fall back on Plan B, which involves statically linking. > Now, I should detail Plan B, as if I knew what I was doing. > I have started work on it. Some preliminaries have already > been done. > > First, I want to make it clear that what I am talking about > applies strictly to userland. I am not proposing that we must > abandon the dynamically linked kernel and krtld. rtld (ld.do.1) > and krtld are built using much of the same underlying libraries, > but are different beasts. krtld is capable of doing dynamic > loading of the kernel and loadable modules, but it does not > involve processing shared objects with PIC code, a procedure > linkage table (PLT) and a global offset table (GOT). krtld has > been running pretty solidly for some time. > > So, Plan B is to have a dynamically linked kernel, but for now, > settle for statically linked userland programs. That is one > aspect of Solaris where it is best to just back off and scale > down expectations. > > You might think that this is all due to the severe time crunch, > and the limited time horizon that goes with it, and warps ones > perspective and forces priorities to change. And you would > be right. But that is not all there is to it. > > Even without any of those considerations, there is a lot of > merit to scaling back to a situation where things are as > simple as possible, as long as it is understood that it a > first step in a series of progressively more complex tests, > and as long as it is not done in a way that wastes effort, > due to unnecessary rework. It makes for good science; that > is, for good design of experiments, because fewer things are > modified at once. > > A statically linked, self-contained hello world program is > about as primitive as it gets, and it is just the ticket > for the first step. But that is old news. We have had that > for a while. But, instead of going from static hello-world > directly to single user prompt, I am taking smaller steps. > Now, there are multiple flavors of hello-* in usr/src/cmd/hello. > There is hello-kprintf, which is statically linked, and uses > nothing but the kprintf() system call. This is actually even > more primitive than the original hello world, which for my > purposes is a good thing. > > Then, there is hello-write, which is pretty much the original > hello program. I maintain distinct names, so that /sbin > will be populated with the various flavors of hello-*. > They can coexist, and I have a chance of remembering which > one does what. I want to keep them around as part of the > design-for-port initiative. > > So, then do we go to real single user prompt? > > I don't think so, because I not only want to back off on > rtld+libc, I want to bypass /sbin/init and go to run a shell, > directly. This is much simpler, because init does a lot of > stuff before you ever get to run the shell. Out of the box, > it comes with many other shared library dependencies. > > Watch this: > > % ldd /sbin/init > libpam.so.1 => /lib/libpam.so.1 > libbsm.so.1 => /lib/libbsm.so.1 > libcontract.so.1 => /lib/libcontract.so.1 > libscf.so.1 => /lib/libscf.so.1 > libc.so.1 => /lib/libc.so.1 > libcmd.so.1 => /lib/libcmd.so.1 > libsocket.so.1 => /lib/libsocket.so.1 > libnsl.so.1 => /lib/libnsl.so.1 > libmd.so.1 => /lib/libmd.so.1 > libsecdb.so.1 => /lib/libsecdb.so.1 > libtsol.so.2 => /lib/libtsol.so.2 > libnvpair.so.1 => /lib/libnvpair.so.1 > libuutil.so.1 => /lib/libuutil.so.1 > libmp.so.2 => /lib/libmp.so.2 > libm.so.2 => /lib/libm.so.2 > > Yow! What is all that? Oh yeah, there is all sorts of naming > services, and the ability to switch among them, and there is > the PAM (Pluggable Authentication Module) stuff. We must go > through the login process. Some of that authentication relies > on encryption from some other libraries. And so on. > > Besides all that, there are several things that have not > really been put to the test, and ought to be tested under > more controlled, more isolated conditions. For example, we > know that the kernel is doing a pretty good job of launching > the first program, /sbin/init, and that has a great deal of > logic in common with the code to do userland fork() and exec() > system calls, but that is not quite the same thing. So, before > we run a shell, we might want to start with a simpler unit > test program, which is more than hello, but less than shell. > > That is where usr/src/cmd/test comes in. It is a relatively > recent addition. It fits with design-for-port, so it is > a keeper, if I have my say. It starts off with a few more > exercises of some simple system calls, then progresses to > exec() of /sbin/hello-kprintf. That works, nicely. OK, so > how about fork() and then exec() of /sbin/hello-kprintf from > the child process. > > Oops, there is a problem, there. I will be working on that > today. I am not too terribly worried, because so much of the > fork and exec code is common, but there is a processor-specific > portion, and I knew that needed to be exercised. And, it does > not get exercised, as a matter of course, as long as I am > stuck in rtld+libc. Those code paths don't test themselves. > > So far, we have, in order of complexity: > /sbin/hello-kprintf > /sbin/hello-write > /sbin/test > > In parallel with work on fork()+exec() comes sash, the > stand-alone shell. It can do many things without even needing > fork and exec. > > The hello-* programs and /sbin/test all are so primitive that > they do not even rely on there being a device tree or any > pre-arranged open file descriptors, like stdin and stdout. > To get a shell working, even sash, I will have to cons up a > bit of run-time environment that establishes file descriptors 0, > 1, and 2 with direct connections to the console, and I may > need to push some terminal streams modules on pty's associated > with those file descriptors. > > After sash would come an attempt at statically linked Bourne > shell, still run directly, bypassing /sbin/init. That would > involve getting what it needs from libc.a, rather than the > completely self-contained /sbin/hello-* and /sbin/test programs. > That has been tested. It works. > > It brings in a lot of code, though. With a bit of work, > it could be made much smaller. There is a small number of > functions that are the key, because they are the ones that > bring in big chunks of functionality, due to requirements of > some standards or legacy code. For example, if my program > calls write(), that brings in a big wad, because libc:write > is not just a naked write() system call. If I tweaked libc.a > to relax just a requirement or two, say involving XPG4, or > something like that, then the resulting program built from > libc-lite.a could easily be twice as small. > > > Makefile.init > ------------- > As I mentioned above, /sbin/init does much work and depends > on many shared libraries. Some of you may have noticed the > introduction of the file, usr/src/lib/Makefile.init. > Well, maybe not. I was going to comment on more on init > and its dependencies and rant about how subsets should > work and how we must avoid error message fatigue, even > in early stages of development. But, then I looked at the > file, and read my own comments in there, in the source > code, checked into the repository, where it belongs. > > I think it covers what I wanted to say. I have not changed > my mind about that subject. If I had, I would change the > comments in Makefile.init. > > Here is an excerpt: > > # usr/src/lib/Makefile.init is a program slice of usr/src/lib/Makefile, > # specialized for building only those libraries that are needed > # to support running /sbin/init. It would, of course, only be of > # immediate interest to someone doing a port to a new platform, > # since, once a Solaris port has matured, all libraries are built. > # > # Let us put the case that you are working on a port of Solaris to > # a new platform, and just got to the point where the kernel is far > # enough along, and you are ready to build /sbin/init and the libraries > # that it depends on. > # > # The problem with usr/src/lib/Makefile is that it is pretty much made > # for building all-or-nothing. Sure, you can descend into individual > # subdirectories and build them each, by hand. But then, you have to > # know which libraries are needed. Running 'ldd /sbin/init' can tell > # you that. But, then you still need to manually direct the process, > # keeping in mind the proper build order and doing it sequentially, > # or reverse-engineering or re-engineering the logic for dependencies > # and .WAIT targets. > # > # You can also use "make -k" to build everything and ignore errors > # related to libraries you don't care about. But that quickly leads to > # error message overload, and then to ignoring all errors. And, it is > # time consuming to scan through the make logs to verify whether the subset > # you are interested in has failed to build. Maybe you can semi-automate > # the scanning of make logs, but that takes work, and the reliability of > # the results of the error filtering has to be verified, etc. > # > # This makefile retains the order of subdirectories, the explicit > dependencies > # and the .WAIT targets that are in usr/src/lib/Makefile, but for a subset > # of subdirectories. Running 'make -f Makefile.init' should build that > subset, > # according to all the same constraints that apply to building all > libraries. > # No manual intervention is required. It should not be necessary to use > # the dangerous '-k' flag to bypass errors in some targets, because if this > # subset does not build entirely and cleanly, then you are in trouble and > # should pay serious attention to any and all errors. > # > # I would like to be able to say that the order and dependencies contained > # in this makefile were generated from scratch by visiting all the > # subdirectories and applying rules to arrive at the structure of the > # makefile needed to build the libraries correctly. But, I can't say that. > # Not yet. I do not understand why the original usr/src/lib/Makefile > # has any .WAIT targets at all, let alone where they are located. > # I would have thought that explicit dependencies alone would be > # sufficient to enforce any partial order that might be needed. But then, > # I don't even understand why we can't just build all libraries in > # parallel, and then do a second pass to cross-check library dependencies. > # > # But, for now, I just take that which is given, and preserve all the > # relevant constraints of the whole and apply them to the subset. > # > # Design for Port > # --------------- > # At least in the Polaris project source code, this file will be kept > # long after the single user prompt milestone has been achieved, > # because it is part of Sun Laboratories' Design for Port program. > # Design for Port is an attempt to minimize the loss of scaffolding > # used during the process of porting Solaris. Normally, a great deal > # of temporary scaffolding gets jettisoned shortly after it serves its > # original purpose, or never even makes it into the gate; it is all > # discarded like a lower-class upbringing of which we are not proud. > # Later, some poor slob has to re-engineer the scaffolding, because > # none of it ever "arrived" to become part of Sun's institutional memory. > # > # Merely not deleting some files from the workspace is not quite enough. > # If it does not get exercised, it will suffer from software rot. > # So, making sure that Makefile.init works will be part of a build > # validation suite. > # > # -- Guy Shaw > _______________________________________________ > powerpc-discuss mailing list > powerpc-discuss at opensolaris.org >
