Hi Martin, > On 1/6/21 12:36 AM, Jeff Law wrote: >> unresolved "could not find python interpreter $testcase" in >> run-gcov-pytest if you find the right magic in the output of your spawn. > > Achieved that with the updated patch. > > Ready for master?
unfortunately, your patch has a large number of problems: * On targets where run-gcov-pytest decides that pytest isn't available (incorrectly in some cases), mail-report.log is cluttered with UNRESOLVED: could not find Python interpreter and (or) pytest module for pr98273.C I fear you've been misled by David and Jeff here: UNRESOLVED isn't appropriate for cases like this. Please read the DejaGnu manual for the semantics of the various test outcomes. If anything (we often just silently skip testcases that cannot be run on some target), use UNSUPPORTED instead. * Besides, the test outcomes are not generic message facilities but are supposed to follow a common format: <result>: <testname> [<subtest>] with <testname> the pathname to the test relative to (in this case) gcc/testsuite. In this case, this might be something like UNSUPPORTED: g++.dg/gcov/pr98273.C run-gcov-pytest Currently, you don't have the pathname in run-gcov-pytest, though. * If we now have an (even optional) dependency on python/pytest, this (with the exact versions and use) needs to be documented in install.texi. * Speaking of documenting, the new run-gcov-pytest needs to be documented in sourcebuild.texi. * On to the implementation: your test for the presence of pytest is wrong: set result [remote_exec host "pytest -m pytest --version"] has nothing to do with what you actually use later: on all of Fedora 29, Ubuntu 20.04, and Solaris 11.4 (with a caveat) pytest is Python 2.7 based, but you don't check that. It is well possible that pytest for 2.7 is installed, but pytest for Python 3.x isn't. Besides, while Solaris 11.4 does bundle pytest, they don't deliver pytest, but only py.test due to a conflict with a different pytest from logilab-common, cf. https://github.com/pytest-dev/pytest/issues/1833. This is immaterial, however, since what you actually run is spawn -noecho python3 -m pytest --color=no -rA -s --tb=no $srcdir/$subdir/$pytest_script So you should just run python3 -m pytest --version instead to check for the presence of the version you're going to use. Btw., there's a mess with pytest on Fedora 29: running the above gives [...] pluggy.PluginValidationError: Plugin 'benchmark' could not be loaded: (pytest 3.6.4 (/usr/lib/python3.7/site-packages), Requirement.parse('pytest>=3.8'))! Seems the packagers have broken things there. On top of all this, I wonder why you insist on a particular Python version here: I tried your single testcase and it PASSes just as well with Python 2.7!? One reason I'm asking is that Solaris 11.3 bundles both Python 2.7 and 3.4, but (unlike Linux and Solaris 11.4) don't have /usr/bin/python3, just python (which is 2.7), python2.7, and python3.4. Not that it matters too much, but you should be aware of the issue. When running the test on Solaris 11.4 (with the bundled pytest 4.4.0), I get ============================= test session starts ============================== platform sunos5 -- Python 3.7.9, pytest-4.4.0, py-1.8.0, pluggy-0.9.0 rootdir: /vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov collected 2 items ../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py .. =========================== 2 passed in 0.04 seconds =========================== while 4.6.9 on Linux gives ============================= test session starts ============================== platform linux -- Python 3.8.2, pytest-4.6.9, py-1.8.1, pluggy-0.13.0 rootdir: /vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov collected 2 items ../../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py .. =========================== short test summary info ============================ PASSED ../../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py::test_basics PASSED ../../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py::test_lines =========================== 2 passed in 0.17 seconds =========================== Obviously pytest -rA was introduced only after 4.4.0 and the 'A' is silently ignored. Fortunately, I can just use -rap instead which works with both versions. After this has been processed by gcov.exp, g++.sum contains PASS: ../../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py::test_basic PASS: ../../../../../../../../../../vol/gcc/src/hg/master/local/gcc/testsuite/g++.dg/gcov/test-pr98273.py::test_line which is again completely wrong in light of what I wrote above on the format of test names: it lacks the testname part completely and contains absolute pathnames which makes it impossible to compare testresults from different systems. Instead, there should be some sort of tag, perhaps patterned after what the various scan-* functions do. Please fix. Rainer -- ----------------------------------------------------------------------------- Rainer Orth, Center for Biotechnology, Bielefeld University