Hi Martin, >> * Besides, the test outcomes are not generic message facilities but are >> supposed to follow a common format: >> <result>: <testname> [<subtest>] >> with <testname> the pathname to the test relative to (in this case) >> gcc/testsuite. In this case, this might be something like >> UNSUPPORTED: g++.dg/gcov/pr98273.C run-gcov-pytest >> Currently, you don't have the pathname in run-gcov-pytest, though. > > All right, now one will see: > > UNSUPPORTED: g++.dg/gcov/pr98273.C run-gcov-pytest could not find Python > interpreter and (or) pytest module
please shorten this quite a bit: maybe ... run-gcov-pytest python3 pytest missing >> * If we now have an (even optional) dependency on python/pytest, this >> (with the exact versions and use) needs to be documented in >> install.texi. > > Done that. +be installed. Some optional tests also require Python3 and pytest module. It would be better to be more specific here. Or would Python 3.0 and pytest 2.0.0 do ;-) >> * On to the implementation: your test for the presence of pytest is >> wrong: >> set result [remote_exec host "pytest -m pytest --version"] >> has nothing to do with what you actually use later: on all of Fedora >> 29, Ubuntu 20.04, and Solaris 11.4 (with a caveat) pytest is Python >> 2.7 based, but you don't check that. It is well possible that pytest >> for 2.7 is installed, but pytest for Python 3.x isn't. >> Besides, while Solaris 11.4 does bundle pytest, they don't deliver >> pytest, but only py.test due to a conflict with a different pytest from >> logilab-common, cf. https://github.com/pytest-dev/pytest/issues/1833. >> This is immaterial, however, since what you actually run is >> spawn -noecho python3 -m pytest --color=no -rA -s --tb=no >> $srcdir/$subdir/$pytest_script >> So you should just run python3 -m pytest --version instead to check >> for the presence of the version you're going to use. >> Btw., there's a mess with pytest on Fedora 29: running the above gives > > I must confirm this is mess. I definitely don't want to support Python2 and > I think > the best way would be to use 'env python3', hope it's portable enough. > @David: What do you think? As I mentioned, it's not: Solaris 11.3 has no python3, only (for the 3.x series) python3.4. However, I don't understand what you expect to gain from running $ env python3 rather than just $ python3 (or a suitable Python 3.x version by any name)? I just had a quick look and the autoconf-archive has AX_PYTHON which claims to do that: https://www.gnu.org/software/autoconf-archive/ax_python.html Unfortunately, it doesn't know about Python 3.8+ yet. >> When running the test on Solaris 11.4 (with the bundled pytest 4.4.0), >> I get >> ============================= test session starts >> ============================== >> platform sunos5 -- Python 3.7.9, pytest-4.4.0, py-1.8.0, pluggy-0.9.0 >> rootdir: /vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov >> collected 2 items >> ../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py >> .. >> =========================== 2 passed in 0.04 seconds >> =========================== >> while 4.6.9 on Linux gives >> ============================= test session starts >> ============================== >> platform linux -- Python 3.8.2, pytest-4.6.9, py-1.8.1, pluggy-0.13.0 >> rootdir: /vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov >> collected 2 items >> ../../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py >> .. >> =========================== short test summary info >> ============================ >> PASSED >> ../../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py::test_basics >> PASSED >> ../../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py::test_lines >> =========================== 2 passed in 0.17 seconds >> =========================== >> Obviously pytest -rA was introduced only after 4.4.0 and the 'A' is >> silently ignored. Fortunately, I can just use -rap instead which >> works with both versions. > > This will be fixed by this: > env python3 -m pytest --color=no -rA -s --tb=no --version No, as I already wrote: pytest 4.4.0 silently ignores -rA and doesn't print the PASSED (or FAILED) lines. With both versions, pytest -rap worked for me instead. >> After this has been processed by gcov.exp, g++.sum contains >> PASS: >> ../../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py::test_basic >> PASS: >> ../../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py::test_line >> which is again completely wrong in light of what I wrote above on the >> format of test names: it lacks the testname part completely and >> contains absolute pathnames which makes it impossible to compare >> testresults from different systems. Instead, there should be some >> sort of tag, perhaps patterned after what the various scan-* functions >> do. > > Fixed that too. Thanks; I'll give the patch a fresh whirl tonight. Rainer -- ----------------------------------------------------------------------------- Rainer Orth, Center for Biotechnology, Bielefeld University
