On 5/13/10 1:11 PM, "Karl Hammar" <k...@aspodata.se> wrote:
> Graham:
>> On Thu, May 13, 2010 at 10:06:45AM +0200, Karl Hammar wrote:
>>> How do one do a regtest?
>>
>> Regression check; by compiling stuff.
>>
>>> 8.1 Introduction to regression tests
>>>
>>> The regression tests are automatically compiled using special `make'
>>> targets. The output of the regression tests is also automatically
>>>
>>> So, what targets?
>>
>> They might be "make baseline-test", followed by applying a patch,
>> followed by "make check", but I'm not certain. It's explained
>> somewhere in Contributing 2.
>
> Ok, found something in "3.6.3 Testing LilyPond" (though nothing in the
> chapter on regression tests).
>
> * Initial test:
>
> make [-jX]
> make test-baseline
> make [-jX CPU_COUNT=X] check
>
> * Edit/compile/test cycle:
>
> _## edit source files, then..._
>
> make clean _## only if needed (see below)_
> make [-jX] _## only if needed (see below)_
> make test-redo _## redo files differing from
> baseline_
> make [-jX CPU_COUNT=X] check _## CPU_COUNT here?_
>
> Hmm, the make check seems redundant since test-redo already does it:
>
> $ find . -name GNUmakefile | xargs grep -A 10 test-redo
> ...
> ./GNUmakefile:test-redo:
> ./GNUmakefile- for a in `cat $(RESULT_DIR)/changed.txt` ; do \
> ./GNUmakefile- echo removing $$a* ; \
> ./GNUmakefile- rm -f $$a* ;\
> ./GNUmakefile- done
> ./GNUmakefile- $(MAKE) check
> ...
>
> ./scripts/build/out/output-distance seems to be the workhorse of the
> regression tests. I cannot find any useful documentation of it with:
>
> find . -type f | xargs grep output-distance
>
> except the source code itself.
>
> But if I already have a known good result from the code tracker,
> how do I compare it with the new result?
What do you mean by "if I already have a known good result from the code
tracker"?
make test-baseline
followed by
make check
will do exactly what you want. You should then have, as described in
Contributors' 3.6.3, a file
out/test-results/index.html
that shows the results of the regression tests.
If the only snippet listed in out/test-results/index.html is
output-distance.ly, then that means you haven't broken any previous
regressions.
output-distance.ly is a snippet that has random content, so it will change
every time it's run. That means that you will *always* get a difference
when you run the regression tests.
Ideally, in this case, you'd see changes in output-distance.ly. And you'd
add a regression test for the current bug, so there would also be an entry
for the current bug. Other than that, it would say that there is a large
number of snippets with no differences.
HTH,
Carl
_______________________________________________
lilypond-devel mailing list
lilypond-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/lilypond-devel