------- Comment #7 from joseph at codesourcery dot com 2009-04-29 15:56 ------- Subject: Re: Inadequate gcc.dg/compat/struct-layout-1_generate.c
On Wed, 29 Apr 2009, hjl dot tools at gmail dot com wrote: > So the issues are > > 1. We want to test as many different combination as possible at random. > 2. We want each test reproducible. > > Can we generate tests at random in such a way that they are reproducible? > Can we do > > 1. Put the random seed in generated file. > 2. Change struct-layout-1_generate.c to take a random seed. > > so that if there is a failure, we can run struct-layout-1_generate.c > with the known random seed on the same platform? If you wish to contribute extra testing of these tests you can certainly do such things. The default should be results that are comparable between different runs of the same test (the thing after "PASS: " or "FAIL: ") at different times by the same person or by different people for the same target, so that when results showing a failure appear on gcc-testresults it is unambiguous from the FAIL lines there exactly what was failing and the next run for that target will show the same lines if the bug is not fixed. If you wish to do such extra testing it would be useful for you to file bug reports with reduced testcases showing the failures. Such reports are useful however the tests were generated (including tests generated by tools not included in GCC, if the licenses on those tools permit the distribution of the tests); certainly some bugs people report appear to have tests generated by unspecified tools they may have developed. -- http://gcc.gnu.org/bugzilla/show_bug.cgi?id=39952
