https://bugs.kde.org/show_bug.cgi?id=503969
--- Comment #6 from Mark Wielaard <m...@klomp.org> --- Hi Martin, (In reply to mcermak from comment #5) > (In reply to Mark Wielaard from comment #2) > > - ORIG_PWD isn't used anymore and is replace by SCRIPT_SRC=$(dirname $0) > > - This makes it possible to invoke the script from anywhere > > My original goal was to allow for a build directory separate from the source > tree. To be clear I mean doing something like ../valgrind/configure > --prefix .... This appears to partly work, but specifically the > vg-in-place script still ends up being created within the source tree. So, > I gave up. This relict change however looks good to me although it's not > strictly needed for our use-case. Yeah, the change is good. Lets keep it. Sadly vg-in-place and make regtest don't work when srcdir != builddir (although make, make check, make dist[check] do). > > - LOGDIR is redefined to no longer use the valgrind-ltp-logs but a more > > generic tests subdir > > - The buildbot relies on the old name and so should be adjusted to deal > > with "tests" > > My intent in this case was to align the structure of the logs with the one > coming from my RHEL valgrind-ltp tester here: > https://builder.sourceware.org/testruns/ > ?op_mode=search_exact&has_keyvalue_k=testrun. > gitdescribe&has_keyvalue_op=glob&has_keyvalue_v=*ltp* Aha, ok, so those have results in a top-level 'tests' dir. But I think it makes sense to upload the results of both regtests and ltpchecks together. The buildbot script moves and renames the dir anyway. We could move/rename it to ltp/tests ? > > - The summary.log isn't generated anymore > > Yes, that's intentional. This ltp tester isn't actually automake based. > But the automake log format is the one I want to use for bunsen upload. So > this patch mimics the automake log format. That said, for every testcase > two log files are synthesized: a .log file containing the actual test log > details, and a .trs file that encodes the test result (that's echo > ":test-result: $rv" > $LOGDIR/$exe.trs). There is one more thing the bunsen > parser needs to successfully recognize our set of logs as automake based: > the test-suite.log file. No matter it is empty. But it's there and with > that bunsen can parse our logs as shown above in Comment #1. This > hopefully explains most of the following notes too ... Yes it does. Could we put something inside the test-suite.log file? So that people who find it know why it is there and where to find the actual logs? Maybe simply something like: echo "See *.log files for details on each test in this directory." > test-suite.log One thing that is different from how the regtests do this is that they put a test-suite.log file in each directory where there are test .log/.trs files? Why is that necessary there and not here? > > - myLog isn't used anymore and is replaced by echo -e which means > > - That backslash escapes are now handled (no more literal \n in the > > output) > > - The individual test summary files aren't generated anymore > > - Inside the LOGDIR a couple of files are generated: > > - test-suite.log which is kept empty? Was this intended to be the new > > summary.log? > > - Various .bunsen* files. Are these necessary when the buildbot does the > > bunsen uploading? > > ... except for this ^^. These .bunsen files are there to provide more test > run context to bunsen. These need to be generated on the test system. But > it doesn't necessarily need to be part of valgrind source. Another option > is to make it part of the buildbot scripts responsible for executing the > testsuite and uploading the test results to bunsen. Moving this part out of > this patch towards the build bot scripts is possible. Lets do that. If we upload everything at the same time, bunsen would also get the config.log file to determine the configuration. > > - Two automake style testlog files are generated > > - the $exe.log which contains something on FAIL, but is empty on PASS (is > > that intended?) > > - a $exe.trs file that contains the PASS or FAIL status. > > > > Please review my understanding and update whether this is expected > > behaviour. > > Thank you for the review. Please let me know if my comments don't answer > your questions. The only unanswered question is whether the $exe.log file should be empty when the test PASSes. Don't we want some logs even if something passes? > Do you prefer to move generation of these .bunsen* files to the buildbot > scripts? Yes. -- You are receiving this mail because: You are watching all bug changes.
