On Tue, May 19, 2020 at 03:16:51AM +0100, Ken Moffat via lfs-dev wrote:
>
> But now I'm running in chroot I'm looking at each testsuite in the
> hope that I can get everything to complete (brief note - glibc no
> failures on my haswell).
And now, the full results of the testsuites - a couple of these seem
to have failures that I'm the only one seeing, or maybe everyone
else has just learned to keep quiet about failures ;-)
This build is nominally LFS from 20200513, on my i7 haswell.
Unfortunately, it is long.
Where we have noted problems with testsuites, I've tried to see if
they can be made to work. Unfortunately, I did not always succeed.
If we say there is no testsuite, I accepted that.
Also, these are normally with my own CFLAGS,CXXFLAGS and hardening:
-O3 -march=native -D_FORTIFY_SOURCE=2 -fstack-protector-strong
-D_GLIBCXX_ASSERTIONS although where I was still getting failures
(e.g. util-linux) I also tried without those, but no difference.
I suppose I'd better note that I seem to have been running a 5.7-rc2
kernel while doing at least the latter of these (accidentally
rebooted when waking up from suspend)
dejagnu-1.6.2 (chapter 5) - ok
glibc-2.31 (my host system was LFS-svn-20200307) - no failures
I see we claim that misc/tst-ttyname and net/tst-idna_name_classify
"are known to fail in the LFS chroot environment" which I interpret
as "are expected to fail". If so, progress.
zlib-1.2.11 - ok
xz-5.2.5 - ok
file-5.38 - ok
m4-1.4.18 - ok
bc-2.7.2 - ok
binutils-2.34 - no reported failures, but the tests ended in error,
so here are the details:
=== binutils Summary ===
# of expected passes 267
# of unsupported tests 2
Testsuite summary for gold 0.1
============================================================================
# TOTAL: 4
# PASS: 4
# SKIP: 0
# XFAIL: 0
# FAIL: 0
# XPASS: 0
# ERROR: 0
============================================================================
=== gas Summary ===
# of expected passes 1358
=== ld Summary ===
# of expected passes 2409
# of expected failures 57
# of untested testcases 1
# of unsupported tests 39
I see there have been comments about gold - all my gold tests claim
to have passed.
gmp-6.2.0 - ok
mpfr-4.0.2 - ok
mpc-1.1.0 - ok
attr-2.4.48 - ok
libcap-2.34 - ok
gcc-10.1.0 (from the summary, similar to the last few releases)
=== gcc tests ===
Running target unix
FAIL: gcc.c-torture/compile/limits-exprparen.c -O0 (internal compiler error)
FAIL: gcc.c-torture/compile/limits-exprparen.c -O0 (test for excess errors)
FAIL: gcc.c-torture/compile/limits-exprparen.c -O1 (internal compiler error)
FAIL: gcc.c-torture/compile/limits-exprparen.c -O1 (test for excess errors)
FAIL: gcc.c-torture/compile/limits-exprparen.c -O2 (internal compiler error)
FAIL: gcc.c-torture/compile/limits-exprparen.c -O2 (test for excess errors)
FAIL: gcc.c-torture/compile/limits-exprparen.c -O3 -g (internal compiler
error)
FAIL: gcc.c-torture/compile/limits-exprparen.c -O3 -g (test for excess
errors)
FAIL: gcc.c-torture/compile/limits-exprparen.c -Os (internal compiler error)
FAIL: gcc.c-torture/compile/limits-exprparen.c -Os (test for excess errors)
FAIL: gcc.c-torture/compile/limits-exprparen.c -O2 -flto
-fno-use-linker-plugin -flto-partition=none (internal compiler error)
FAIL: gcc.c-torture/compile/limits-exprparen.c -O2 -flto
-fno-use-linker-plugin -flto-partition=none (test for excess errors)
FAIL: gcc.c-torture/compile/limits-exprparen.c -O2 -flto -fuse-linker-plugin
-fno-fat-lto-objects (internal compiler error)
FAIL: gcc.c-torture/compile/limits-exprparen.c -O2 -flto -fuse-linker-plugin
-fno-fat-lto-objects (test for excess errors)
=== gcc Summary ===
# of expected passes 149088
# of unexpected failures 14
=== libstdc++ tests ===
Running target unix
FAIL: 20_util/unsynchronized_pool_resource/allocate.cc execution test
FAIL: 22_locale/time_get/get_time/char/2.cc execution test
FAIL: 22_locale/time_get/get_time/char/wrapped_env.cc execution test
FAIL: 22_locale/time_get/get_time/char/wrapped_locale.cc execution test
FAIL: 22_locale/time_get/get_time/wchar_t/2.cc execution test
FAIL: 22_locale/time_get/get_time/wchar_t/wrapped_env.cc execution test
FAIL: 22_locale/time_get/get_time/wchar_t/wrapped_locale.cc execution test
FAIL: experimental/net/internet/resolver/ops/lookup.cc execution test
=== libstdc++ Summary ===
# of expected passes 14097
# of unexpected failures 8
pkg-config-0.29.2 - ok
ncurses-6.22 - I agree the testsuite can only be run after
installing, but it doesn't see particularly useful (the tests need to
be individually invoked, and they don't produce any obvious pass or
fail output - some give coloured outputs, others ran for longer than
my patience and were terminated with prejudice).
sed-4.8 - (See my post from 19 May)
The testsuite.panic-tests.sh fails because of running the tests as
root. Chowning the build tree and running as nobody, that passes.
Added to my mental todo list.
gettext-0.20.2 - ok
bison-3.6 - ok.
I saw that 3.6.1 and then 3.6.2 had been released, apparently related
to test failures on some platforms, so I also tested those, with the
same successes.
In the book we say 'Fourteen tests fail in the "Diagnostics" section,
probably because of missing locales.' so as Pierre suggested, this
cross-chap5 approach does work here.
flex-2.6.4 - ok
grep-3.4 - ok
bash-5.0 -
Inconclusive, /dev/tty errors, and run-trap reports a series of
signal errors -
7d6
< trap -- '' SIGFPE
17d15
< trap -- '' SIGFPE
[clipped]
For the /dev/tty errors, I think Pierre has now got to the bottom
of the problem!
libtool-2.4.6 - 5 unexpected failures, as the book says, when the
test is run here.
gdbm-1.18.1 - ok
gperf-3.1 - ok
expat-2.2.9 - ok
inetutils-1.9.14
The book says: One test, libls.sh, may fail in the initial chroot
environment but will pass if the test is rerun after the LFS system
is complete. One test, ping-localhost.sh, will fail if the host
system does not have ipv6 capability.
My machine has the ipv6 options compiled in to its kernel.
Previously I've had '|| true' after running the tests for this
package, which is a PITA because I never notice when results change.
So, for the first time I ran manually and everything passed.
Removed '|| true', ran the script, exited bacause libls.sh failed.
Reinstated '|| true', but this time libls.sh passed anyway.
I conclude that libls.sh randomly fails and the book should say
something like 'libls.sh may randomly fail for unknown reasons'.
perl-5.30.2 - ok
XML::Parser-2.46 - ok
intltool-0.51.0 - ok
autoconf-2.69 -
"The test suite is currently broken by bash-5 and libtool-2.4.3"
Well, we are now on a later libtool, but yes, it is still very
broken.
automake-1.16.2 -
I suggested a sed to replicate an upstream commit which causes
tags-lisp-space.sh to be skipped if etags is not present. Bruce
has suggested a better sed, I have not yet got around to that.
Anyway, by fixing that, all tests pass.
libtool-2.4.6, tests after automake - ok
libelf (elfutils-0.179) - this is one of those packages where I
normally expect several test failures because I remove -g from the
CFLAGS. That often means I don't notice different failures.
On this occasion I added -g to my CFLAGS. No failures.
The book says: "One test, run-elfclassify.sh, is known to fail."
but for me it PASSed. Again, maybe a benefit from this approach.
libffi-3.3 -
The book says "Six tests, all related to test-callback.c, are known
to fail." but all were ok :
=== libffi Summary ===
# of expected passes 2284
openssl-1.1.1g - ok
Python-3.8.3 (I upgraded to this). As the book says,
test_normalization failed because it needs the network to be
configured so that it can retrieve a file.
ninja-1.10.0 - ok
coreutils-8.32 - ok
acl-2.2.53 (after coreutils) - I get failures in
FAIL: test/root/permissions.test
FAIL: test/root/setfacl.test
The latter is obviously because I don't mount with the acl option,
but I now wonder if the first one might be failing because it is
run as root. I don't know, and to be honest I don't care about
this package.
check-0.14.0 -
For several years I've had two failures here, check_check_export
and check_check. I've now had a suggestion from upstream that on my
machines sigfpe is not being returned. See also my comments on the
bash tests. No idea why my builds are so different from everyone
else's. I mean, these tests do pass for you guys, don't they ?
diffutils-3.7.0 - ok
gawk-5.1.0 - ok
findutils-4.7.0 -
"Two tests are known to fail in the chroot environment:
sv-bug-54171.old-O3 and sv-bug-54171.new-O3."
In fact, that is another reason not to run tests as root!
Chowning the build tree to nobody and running as that user, all
tests pass. I guess I also need to commit that to trunk.
gzip-1.10 - ok
iproute2-5.6.0 -
We say: This package does not have a working test suite.
I do not yet have a view on whether or not that is correct. It
certainly requires libmna, so I have installed that on this build
(but won't be installing it in the future), but then it failed
because sudo was missing. It wanted sudo to run 'rmmod'!
I think we should say (assuming that the tests can work in a
completed system - to be proved), "The test suite SHOULD NOT be
run in chroot, so we do not provide the dependencies it requires."
Not quite sure about the wording, but I think RFC 2119 sums it up.
kbd-2.2.0 - ok
libpipeline-1.5.2 - ok
make-4.3 - ok
patch-2.7.6 - ok
man-db-2.9.1 - ok
The failure in man-missing-locales no longer happens with this
approach.
tar-1.32 - ok
texinfo-6.7 - ok
vim-8.2.0716 - ok
(after my issues mentioned on support: in my case I need to pass
TERM=linux when I'm running the tests on a graphical term where the
dafault setting is not usable with vim once that has been installed)
eudev-3.2.9 - ok
procps-ng-3.3.16 -
I removed the sed to disable failing tests, all passed so another
improvement.
util-linux-2.35.1 -
This had me tearing my hair out, and trying 2.35.2 (same results),
but I can't get past 3 failures -
ipcs: mk-rm-msg ... FAILED
(ipcs/mk-rm-msg)
ipcs: mk-rm-sem ... FAILED
(ipcs/mk-rm-sem)
ipcs: mk-rm-shm ... FAILED
(ipcs/mk-rm-shm)
I've given up on this one, it had a different failure with gcc-9
(column: invalid multibyte) but the tests themselves have not changed
since last August. I did wonder if using my own CFLAGS and CXXFLAGS
might be the cause, but a manual build without setting those gave the
same failures.
e2fsprogs-1.45.6 - ok
Conclusion: I don't like building the *temporary* chroot items in
the same chapter as the the non-chroot items, but the new approach is
a lot cleaner, and also educational (e.g. DESTDIR installs). I
commend this approach to everyone, and hope that we will move to a
variant of it (e.g. a separate chapter for the temporary build in
chroot, so that we end up with an extra chapter).
And my congratulations to Pierre for taking the time to come up with
this.
ĸen
--
Do you not know that, what you belittle by the name tree is but the
mere four-dimensional analogue of a whole multidimensional universe
which - no, I can see you do not. -- Druellae (a Dryad)
--
http://lists.linuxfromscratch.org/listinfo/lfs-dev
FAQ: http://www.linuxfromscratch.org/faq/
Unsubscribe: See the above information page
