Siarhei Siamashka <siarhei.siamas...@gmail.com> writes: > On Thu, Apr 8, 2010 at 2:39 AM, Søren Sandmann <sandm...@daimi.au.dk> wrote: > > From: Søren Sandmann Pedersen <s...@redhat.com> > > > > This serves to make sure we don't accidentally reintroduce old bugs. > > The problem (or a feature) of the current tests is that crc32 can > change any time and such lists of known old failure cases of may be > invalidated any time in the future.
Is this any different what we have now? If something changes, we have to update the CRC values. Adding a list of old failures just means there are more CRC values that would have to be updated, but this is a fairly mechanical change. > A proper solution would be to generate reproducible traces for the > failed testcases (record all the sequence of function calls, their > parameters and input data, plus store the expected output data). Sure, in fact a pixman-trace program similar to cairo-trace would be useful for a lot of things. I agree tests would ideally be done this way. However, there is still value in a simple table of regressions that have failed previously. For example the MMX bug is the kind of thing that could easily be reintroduced in other backends and not be found in the real world since it only shows up with solid alpha masks that have an RGB format and a non-zero color component. > I have a work-in-progress branch here intended to improve pixman > regression testing (and messes up all the checksums): > http://cgit.freedesktop.org/~siamashka/pixman/log/?h=tests-refactoring That looks mostly good, especially the fuzzer_main() function. I don't see how it improves *regression* testing, though. (Regression testing meaning making sure that old bugs are not reintroduced). > The current problems if the regression tests include: > 1. missing a8b8g8r8 format for mask in blitters-test > 2. scaling-test coverage is actually not good enough (it's > embarassing, but it did not take into account that the random number > generator has a limited range for generated values). > > Also this branch tries to utilize multiple CPU cores via OpenMP to > speed up the tests. This seems to be a very simple change, but works > fine and is supposed to be portable. Will this just automatically work with GCC or do you have to compile it in a special way? Soren _______________________________________________ Pixman mailing list Pixman@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/pixman