On 12/07/2017 11:03 AM, Martin Sebor wrote: > On 12/07/2017 09:55 AM, Jakub Jelinek wrote: >> On Wed, Dec 06, 2017 at 05:30:53PM +0100, Jakub Jelinek wrote: >>> On Wed, Dec 06, 2017 at 09:20:15AM -0700, Martin Sebor wrote: >>>> Attached is a patch with the comment updated/simplified. >>>> The tests do the job they need to do today so I just removed >>>> the useless attribute but otherwise left them unchanged. If >>>> you would like to enhance them in some way please feel free. >>> >>> Ok for trunk, with a minor nit. I'll tweak the tests incrementally >>> when it is in. >> >> So here is the fix for those testcases. >> >> They didn't test what they meant to test, because they didn't FAIL >> without the patch. That is because the bug was that the -W* option >> affected code generation, so with -O2 -Wno-stringop-overflow it didn't >> trigger it. > > Doh! I suppose that underscores that the right way to write test > cases for optimization bugs is to prune warnings out of their output > rather than suppressing them via -Wno-foo. I've done the former > for this very reason a number of times but clearly fell into the > trap of suppressing the warnings in this instance. Thanks for > pointing it out! > >> I've changed the tests to test both in a separate noipa function where >> it doesn't know about the aliasing and string lengths from the caller, >> in that case it does more verifications, including the content of the >> whole buffer, and the individual values of the lengths, >> and what you did before. > > I don't have an opinion on the rest of these changes. I do want > to make one comment about runtime tests. I fairly regularly run > tests with cross-compilers on the build machine. This lets me > verify that compile-only tests pass but it doesn't do anything > for tests that need to run. In fact, with the current mixture > of all kinds of tests in the same directory, it pretty much rules > out drawing any conclusions from test results in this setup. So > while I appreciate the additional testing done by the runtime > tests, I think ideally, having compile time only tests would be > the baseline requirement and runtime tests would be a separate > layer that would provide additional validation when possible. So note if you set up a deeper cross compilation environment (using sysroot) you can cross-compile down to an executable. Then you ought to be able to use qemu's user emulation to run the resulting binary.
Alternately, for many embedded targets if you have a cross compilation environment (including newlib) you can then use the old ISA simulator to run execution tests. We don't do those things much anymore, but we certainly have in the past and knowing how to do so is valuable. jeff
