On Mon Mar 21 2011 at 12:47:24 -0700, Sanjay Lal wrote:
> I was curious about running NetBSD kernel code in user land run on kernels
> other than NetBSD? For instance, running the NetBSD networking stack in
> userland on Linux etc? What would be required to get such a system going?
This is probably a FAQ, but I can't remember answering it except in
private, so one more go is useful. If anyone investigates further,
some notes on the wiki might be in order (the same can be said about
how to use NetBSD-current kernel drivers on a release).
The short version is that it runs just fine, but there's always a ...
1) building
Back in 2007 when I originally ran our FFS on Linux, I built the code
with some concoctions of build.sh, nbmake USETOOLS=no and copypasted
command lines. It's not very difficult, but takes some tinkering.
Since then, Arnaud Ysmal's most excellent fs-utils required rump
support in pkgsrc. The package makes compilation trivial, but it
tends to go out-of-date. In fact, currently it's really out-of-date,
and at least I don't really know how to update it.
So, what I think would be required is something like bootstrap-rump.sh,
which would be included in the source tree and would compile the
kernel and rumpuser with the right compiler flags. The same script
could be used to install -current rump onto a release.
2) host resource access
Networking is not very exciting unless you can also access a network.
On NetBSD, there's a few different way to do this:
a) virt(4), which accesses the host network via a tap-like device
from /dev/tap using read/write
b) shmif(4), which uses a memory-mapped file as an ethernet bus. as an
implication, communication is limited to other networking stacks
running on the same host (unless one of them routes traffic to the
outside world)
c) "sockin", which actually skips the networking stack and just emulates
the inet protocol family on top of kernel sockets. it's most useful
when networking is _not_ the thing you're interested in virtualizing
(think about e.g. the nfs client)
You're probably interested in "a". Adjusting virt(4) to talk to a
Linux tap interface (or equivalent) is most likely not very difficult.
3) application interface
This is where the gloves come off. In a normal scenario the userland
and kernel are in agreement about the ABI. If you try to run kernel
code on a foreign platform, the kernel code will have its own idea
of the ABI and the userland will have its own idea. I first noticed
this when running FFS on Linux and trying to pass an off_t pointer
in the hypercall interface ...
You need userland applications at least to configure the networking
stack. This includes ioctl and r/w on the routing socket. The problem
is that there is no "NetBSD" namespace, so e.g. the name "struct
msghdr" may have a collision between Linux and NetBSD, PF_INET may
have a different value, etc.
Part of the issue is that we cannot process our headers automatically
to create a "NetBSD" namespace. Incidentally, the "xmlif" project
proposed on this list yesterday would solve part of the issue. Even if
you cannot autogenerate types properly, you can still do reasonable
hacks with e.g. sed. See src/sys/rump/include/makerumpdefs.sh (which
is mostly written for file systems, but can be augmented).
Finally, you must translate the types. This is not difficult, but
slightly boring.
Depending on what your application is, you might not have to do much
in this department. E.g. a router runs pretty much without a userland,
so you'd just have to implement the types necessary for configuration.
If you want totally transparent application use, there's some more
work involved with making librumphijack target the right syscall names.
4) hypercall interface
The rumpuser hypercall interface should be clean of ABI assumptions
(apart from the bits necessary to implement "sockin"). However,
since I haven't really tested it on non-NetBSD in years, there might
be a few snafus lurking around.
In total, I'd estimate getting the NetBSD TCP/IP stack working on Linux
to be a day or two of work, and full application support would require
some more days to get all the translation code written. Apart from that,
it's good to reserve some time for "unforeseen issues".
- antti
--
älä karot toivorikkauttas, kyl rätei ja lumpui piisaa
