On 10/8/2018 2:10 PM, Brett Lymn wrote:
On Sun, Oct 07, 2018 at 06:47:58PM -0700, Don NetBSD wrote:
I don't think so. E.g., anything that relies on (or supports) remote
clients/services would require some explicit action at a remote node to
exercise those services.
For your purposes surely either just testing on loopback or on the local
interface from the machine would be sufficient? You just wanted to make
sure things start.
Yes, but you have to arrange for all of those "things" to actually GET
started. And, the bigger problem, if they <gag> when invoked, there's
very little to help you figure out what they might be missing.
You can look at the .so's that are linked to a (non-static) binary
and arrange for them to be present. But, if they exec something else,
in turn, then you have to arrange for those binaries (and their
dependencies) to be present.
Ideally, I'd like something like the "required to run" explicit dependency
specification in pkgsrc.
[Yes, I realize this is asking a lot. So, I'm looking for a reliable way
of MANUALLY ascertaining these dependencies without having to rely on an
intimate knowledge of "how things work"]
E.g., Will ftpd(8) get "tested" if it is invoked via inetd.conf(5)?
(Ditto everything else therein).
ftp localhost
though you probably want something a bit more robust that fails within a
timeout so the tests move on.
If you want to really test externally then you could use qemu to build a
test virtual machine and use the host at the "external" machine.
I'm only looking to "test" to the extent that nothing SIGSEGV's or
otherwise crashes because I failed to include something that it needed
in the target filesystem. I assume that if I've got everything in
place, it "will work".
I've been "manually" invoking everything that I want/need to run and
capturing any errors logged to sort out what might be missing.
Right - atf can automate this bit for you. If you are doing a
customised build then you will want to do this agin if/when you update
to make sure things are not broken afterwards. It means you can
validate things in a consistent and repeatable manner.
I think that's more than I need. I'm going to pick *a* release
and stick with it (for a very long time -- updates will be difficult).
The bigger concern is deciding that I need to add some particular
binary to this "distribution". Adding shouldn't break anything
that already works but could require additional dependencies that
haven't been present, prior to that point.
The
errors alone don't tell you *what* you need to add -- they just tell you
that something is apparently "missing" or "not working properly". It
then boils down to either having a familiarity with how each piece of
code works *or* digging through the sources to see what MIGHT be the
problem.
Right, this bit will still need to be done manually but at least you can
automate the running of the tests.
Anyway, up to you how you do this. Just saying that this is the sort of
stuff that atf is meant for
To repeat/summarize, my approach, thus far, has been to PXE boot a kernel
and let it NFS mount a remote filesystem that is initially empty. Then,
examine the errors that are generated and start adding stuff to that
file system to eliminate each error -- which inevitably exposes another
error, etc.
Once the system made it to single user, I continued the process after telling
it to go multiuser.
Once it could correctly get to multiuser, THEN I started looking at the
additional programs that I wanted present -- adding each, individually;
invoking them, manually; then pacifying the errors that were thrown.
Because the order that I resolve errors has an impact on which NEW errors
might (or might not!) arise, what I learn from creating one "distribution"
(bad choice of terms) doesn't directly translate to creating a different
distribution.
E.g., an error thrown by PROGRAMA may be fixed by dragging libc into the
target filesystem. Some time later, PROGRAMB may need something else -- but
it ALSO needed libc and didn't complain about that because libc was already
in place from PROGRAMA's needs! If the "next" distribution omits PROGRAMA,
then my notes regarding PROGRAMB's needs (part of the new distribution)
will be incomplete.
