Re: GNU make 4.3.91 on Debian 9.1
Paul Smith wrote: > *** Testing failed! Details saved in: makeerror-x86_64-pc-linux-gnu.tar.gz > *** Please report to > > make[3]: *** [Makefile:1537: check-regression] Error 1 > make[2]: *** [Makefile:1271: check-am] Error 2 > make[1]: *** [Makefile:973: check-recursive] Error 1 > make: *** [Makefile:1192: distcheck] Error 1 > > The tar file is named with the host triplet, and all the things I might > want (hopefully!!) are there. It exists in the working directory. Thank you! That should make the next reports by Nelson and me much more useful to you. Bruno
Re: GNU make 4.3.91 on Debian 9.1
On Wed, 2022-10-19 at 15:59 +0200, Bruno Haible wrote: > > Maybe just augmenting the check target to do more? > > Yes. Instead of printing > See .diff* files in work dir for details it could > - generate a summary file with details (like test-suite.log, described > above), and/or > - generate tarball with useful info to send, and/or > - (if you prefer receiving mails with several attachments instead of a > tarball attachment) print a list of files to send. > And ideally, also print the email address for the reports. I sent my reports > to bug-make@; Nelson sent his to platform-testers@. I've implemented changes to the "make check" target so that if it fails you'll see output like this: <...> 1 Test in 1 Category Failed (See .diff* files in work dir for details) :-( *** Testing failed! Details saved in: makeerror-x86_64-pc-linux-gnu.tar.gz *** Please report to make[3]: *** [Makefile:1537: check-regression] Error 1 make[2]: *** [Makefile:1271: check-am] Error 2 make[1]: *** [Makefile:973: check-recursive] Error 1 make: *** [Makefile:1192: distcheck] Error 1 The tar file is named with the host triplet, and all the things I might want (hopefully!!) are there. It exists in the working directory. This will be in the next release. I've also gone through the previous results and adjusted the regression tests to hopefully resolve at least some of the failures. Dmitry provided fixes for a few tests as well. I'm not sure what to do about the mingw and cygwin tests. I will see what I can figure out but these might have to wait for the next release. > > the src/config.h file > > The src/config.h is redundant IMO, since it is derived from > config.status. That's true for "configure"-capable systems but on other systems it might be copied from a known-good config.h. But, I guess the platform-testers wouldn't ever be testing those other systems.
Re: GNU make 4.3.91 on Debian 9.1
> make light of the one that isn't (far as I see) a regression I looked into it further and concluded that failure is indeed expected on Debian 9, where it's a harmless documentation of the legacy $(wildcard) behavior with dangling symlinks there. I've posted an explanation of that to https://savannah.gnu.org/bugs/index.php?52018#comment6 in the hope that a web search will lead me from symptom to cause more quickly next time. Having built more times now, I’m intermittently seeing Bruno’s temp_stdin failure too but I see Frank and Dmitry have gotten further with that under separate cover. That means that the only one left to be understood in this thread is Bruno's: *** work/features/output-sync.base.15 Wed Oct 19 01:39:11 2022 --- work/features/output-sync.log.15 Wed Oct 19 01:39:11 2022 *** *** 0 --- 1 + make: *** [work/features/output-sync.mk.15:3: all] Terminated I haven't seen that one, though it bears quite a resemblance to the temp_stdin symptom. From: Martin Dorey Sent: Tuesday, October 18, 2022 22:05 To: Bruno Haible ; bug-make@gnu.org ; psm...@gnu.org Subject: Re: GNU make 4.3.91 on Debian 9.1 >> On Debian 9.1.0 / x86 (a machine with glibc 2.24) ... >> 3 tests fail For me on Debian 9.13 amd64 (a machine with glibc 2.24) it remains just the one failure, the one I alluded to in https://savannah.gnu.org/bugs/?63016, so it's not very new. Indeed, output below suggests it's several years and releases old. > Is there any possibility of seeing the output Certainly, but I still don't want to spam the list to death with the whole thing and I didn't spot anything interesting. make check-local make[2]: Entering directory '/home/martind/download/make' cd tests && perl -w ./run_make_tests.pl -srcdir /home/martind/download/make -make ../make Name "main::testee_version" used only once: possible typo at ./run_make_tests.pl line 598. Name "main::testee" used only once: possible typo at ./run_make_tests.pl line 465. Name "main::tmpfilesuffix" used only once: possible typo at ./run_make_tests.pl line 467. Name "main::profile" used only once: possible typo at ./run_make_tests.pl line 432. Name "main::testname" used only once: possible typo at ./run_make_tests.pl line 715. Name "main::vos" used only once: possible typo at ./run_make_tests.pl line 432. Name "main::osname" used only once: possible typo at ./run_make_tests.pl line 569. Name "CORE::GLOBAL::rmdir" used only once: possible typo at ./run_make_tests.pl line 96. Name "main::workdir" used only once: possible typo at ./run_make_tests.pl line 254. Name "main::cwddir" used only once: possible typo at ./run_make_tests.pl line 42. --- Running tests for GNU Make on linux GNU Make 4.3.91 --- Clearing work... Finding tests... features/archives ... ok (12 passed) features/comments ... ok (1 passed) features/conditionals ... ok (5 passed) ... functions/warning ... ok (5 passed) functions/wildcard .. FAILED (9/10 passed) functions/word .. ok (18 passed) ... variables/special ... ok (6 passed) variables/undefine .. ok (5 passed) vms/library . N/A 1 Test in 1 Category Failed (See .diff* files in work dir for details) :-( make[2]: *** [Makefile:1533: check-regression] Error 1 make[2]: Leaving directory '/home/martind/download/make' make[1]: *** [Makefile:1268: check-am] Error 2 make[1]: Leaving directory '/home/martind/download/make' make: *** [Makefile:970: check-recursive] Error 1 martind@sirius:~/download/make$ find tests/work/ | grep diff tests/work/functions/wildcard.diff.9 martind@sirius:~/download/make$ ... the contents of which was as per Bruno. Debian make-3.81 (on x86 aka ia32 aka 32 bit) said: (ia32)martind@sirius:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' | /usr/bin/make -f /dev/stdin __ldir (ia32)martind@sirius:~/tmp/wildcard-9$ ls -l total 0 lrwxrwxrwx 1 martind software 6 Oct 19 04:47 __ldir -> __rdir (ia32)martind@sirius:~/tmp/wildcard-9$ But Debian make-4.0 in the same directory said: martind@zoot:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' | /usr/bin/make -f /dev/stdin martind@zoot:~/tmp/wildcard-9$ ... up to the latest git: martind@sirius:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' | ~/download/make/make -f /
Re: GNU make 4.3.91 on Debian 9.1
Hi Paul, > For example, for the failures that appear to be > timeouts sometimes the test suite will generate some information about > that, to stdout/stderr. > > Maybe there's some make target or script that, if it exists, the > platform testing scripts will run on failure to generate the tar file? Some testers (like Nelson Beebe) have more evolved "platform testing scripts" than others. I use some "platform testing routine", that involves looking at the "make check" output and guess what info would be useful to send back. > With an error I'd like to get various other files such as the > src/config.h file, the config.status file if it exists, plus the > test/work directory and also the output of the configure/build/test > run. It would be useful to formalize this, so that the platform testers' feedback becomes more useful to you. For example, Nelson's summary today [1] shows that he thought it would be useful to report the part of the "make check" output that contains the failures. I thought — seeing the line 3 Tests in 3 Categories Failed (See .diff* files in work dir for details) — that it would be good to do cd tests; tar cvfz $platform-failures.tar.gz `find work -name '*.diff*'` and send that. It turns out that both are insufficient for your analysis. If you tell us what to return as part of the feedback, we can accommodate it. Many GNU packages create a 'test-suite.log' file, that contains the list of failed or skipped tests, plus the log file (stdout+stderr) for each. This makes it simple: As a tester, I simply attach this test-suite.log (or the parts I judge relevant). > the config.status file if it exists Yes, the config.status file is useful to attach, especially when the compilation failed, but probably also when the compilation succeeded and some tests failed. > the src/config.h file The src/config.h is redundant IMO, since it is derived from config.status. > plus the test/work directory All of it? Or a selected part of it? > and also the output of the configure/build/test run. Nelson and I do keep these outputs, as part of our build scripts. If you want it, we can send it. We only need to know what to send. > I'm happy to add something to GNU make to collect "useful stuff" if > there's a standard way that testers would like it. > > Maybe just augmenting the check target to do more? Yes. Instead of printing See .diff* files in work dir for details it could - generate a summary file with details (like test-suite.log, described above), and/or - generate tarball with useful info to send, and/or - (if you prefer receiving mails with several attachments instead of a tarball attachment) print a list of files to send. And ideally, also print the email address for the reports. I sent my reports to bug-make@; Nelson sent his to platform-testers@. Assuming what you want is a tarball with config.status `find tests/work -name '*.diff*'` `find tests/work -name '*.diff*' | sed -e 's/\.diff/.log/'` that's what I'm sending now. Bruno [1] https://lists.gnu.org/archive/html/platform-testers/2022-10/msg5.html debian91-failures.tar.gz Description: application/compressed-tar
Re: GNU make 4.3.91 on Debian 9.1
On Wed, 2022-10-19 at 05:05 +, Martin Dorey wrote: > > Is there any possibility of seeing the output > > Certainly, but I still don't want to spam the list to death with the > whole thing and I didn't spot anything interesting. I meant in general. For example, for the failures that appear to be timeouts sometimes the test suite will generate some information about that, to stdout/stderr. Maybe there's some make target or script that, if it exists, the platform testing scripts will run on failure to generate the tar file? With an error I'd like to get various other files such as the src/config.h file, the config.status file if it exists, plus the test/work directory and also the output of the configure/build/test run. I'm happy to add something to GNU make to collect "useful stuff" if there's a standard way that testers would like it. Maybe just augmenting the check target to do more?
Re: GNU make 4.3.91 on Debian 9.1
>> On Debian 9.1.0 / x86 (a machine with glibc 2.24)
...
>> 3 tests fail
For me on Debian 9.13 amd64 (a machine with glibc 2.24) it remains just the one
failure, the one I alluded to in https://savannah.gnu.org/bugs/?63016, so it's
not very new. Indeed, output below suggests it's several years and releases
old.
> Is there any possibility of seeing the output
Certainly, but I still don't want to spam the list to death with the whole
thing and I didn't spot anything interesting.
make check-local
make[2]: Entering directory '/home/martind/download/make'
cd tests && perl -w ./run_make_tests.pl -srcdir /home/martind/download/make
-make ../make
Name "main::testee_version" used only once: possible typo at
./run_make_tests.pl line 598.
Name "main::testee" used only once: possible typo at ./run_make_tests.pl line
465.
Name "main::tmpfilesuffix" used only once: possible typo at ./run_make_tests.pl
line 467.
Name "main::profile" used only once: possible typo at ./run_make_tests.pl line
432.
Name "main::testname" used only once: possible typo at ./run_make_tests.pl line
715.
Name "main::vos" used only once: possible typo at ./run_make_tests.pl line 432.
Name "main::osname" used only once: possible typo at ./run_make_tests.pl line
569.
Name "CORE::GLOBAL::rmdir" used only once: possible typo at ./run_make_tests.pl
line 96.
Name "main::workdir" used only once: possible typo at ./run_make_tests.pl line
254.
Name "main::cwddir" used only once: possible typo at ./run_make_tests.pl line
42.
---
Running tests for GNU Make on linux
GNU Make 4.3.91
---
Clearing work...
Finding tests...
features/archives ... ok (12 passed)
features/comments ... ok (1 passed)
features/conditionals ... ok (5 passed)
...
functions/warning ... ok (5 passed)
functions/wildcard .. FAILED (9/10 passed)
functions/word .. ok (18 passed)
...
variables/special ... ok (6 passed)
variables/undefine .. ok (5 passed)
vms/library . N/A
1 Test in 1 Category Failed (See .diff* files in work dir for details) :-(
make[2]: *** [Makefile:1533: check-regression] Error 1
make[2]: Leaving directory '/home/martind/download/make'
make[1]: *** [Makefile:1268: check-am] Error 2
make[1]: Leaving directory '/home/martind/download/make'
make: *** [Makefile:970: check-recursive] Error 1
martind@sirius:~/download/make$ find tests/work/ | grep diff
tests/work/functions/wildcard.diff.9
martind@sirius:~/download/make$
... the contents of which was as per Bruno.
Debian make-3.81 (on x86 aka ia32 aka 32 bit) said:
(ia32)martind@sirius:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' |
/usr/bin/make -f /dev/stdin
__ldir
(ia32)martind@sirius:~/tmp/wildcard-9$ ls -l
total 0
lrwxrwxrwx 1 martind software 6 Oct 19 04:47 __ldir -> __rdir
(ia32)martind@sirius:~/tmp/wildcard-9$
But Debian make-4.0 in the same directory said:
martind@zoot:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' |
/usr/bin/make -f /dev/stdin
martind@zoot:~/tmp/wildcard-9$
... up to the latest git:
martind@sirius:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' |
~/download/make/make -f /dev/stdin
martind@sirius:~/tmp/wildcard-9$
martind@sirius:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' |
strace -f ~/download/make/make -f /dev/stdin 2>&1 | grep __ldir
read(3, "all: ; @echo $(wildcard __ldir)\n", 4096) = 32
stat("./__ldir", 0x7ffeefaa24d0)= -1 ENOENT (No such file or directory)
martind@sirius:~/tmp/wildcard-9$ echo 'all: ; @echo $(wildcard __ldir)' |
ltrace -f ~/download/make/make -f /dev/stdin 2>&1 | grep __ldir
...
[pid 1994] strchr("__ldir", '(') = nil
[pid 1994] __xstat(1, "./__ldir", 0x7ffe07b2f710) = -1
martind@sirius:~/tmp/wildcard-9$
None of which is to make light of the two failures Bruno sees that I don't.
Perhaps it's to make light of the one that isn't (far as I see) a regression.
________
From: Bug-make on behalf of
Paul Smith
Sent: Tuesday, October 18, 2022 19:58
To: Bruno Haible ; bug-make@gnu.org
Subject: Re: GNU make 4.3.91 on Debian 9.1
* EXTERNAL EMAIL *
On Wed, 2022-10-19 at 01:43 +0200, Bruno Haible wrote:
> On Debian 9.1.0 / x86 (a machine with glibc 2.24), the compilation
> succeeds but 3 tests fail.
Is there any possibility of seeing the output generated during these
runs, in addition to the work directory?
The output generated often has useful information.
Re: GNU make 4.3.91 on Debian 9.1
On Wed, 2022-10-19 at 01:43 +0200, Bruno Haible wrote: > On Debian 9.1.0 / x86 (a machine with glibc 2.24), the compilation > succeeds but 3 tests fail. Is there any possibility of seeing the output generated during these runs, in addition to the work directory? The output generated often has useful information.
GNU make 4.3.91 on Debian 9.1
On Debian 9.1.0 / x86 (a machine with glibc 2.24), the compilation succeeds but 3 tests fail. debian91-failures.tar.gz Description: application/compressed-tar
