On Wed, 24 Jan 2007 17:26:32 -0500 (EST), Andrew Pinski <[EMAIL PROTECTED]> said:
>> That's largely because individual tests in the test suite are too >> long, which in turn is because the tests are testing code at a >> per-binary granularity: you have to run all of gcc, or all of one >> of the programs invoked by gcc, to do a single test. (Is that true? >> Please correct me if I'm wrong.) > No, they are almost all 1 or 2 functions long, nothing more than 20 > lines. Sorry, I should have been clearer: by "long" I meant long in test execution time, not in textual representation of the test. (Though having the latter be short is important, too!) There's a big different between tests that take, say, .1 to 1 second each and tests that take, say, .001 to .01 second each, especially if you have tens of thousands of test cases. > You really cannot do unit testing for parsing cases, sorry to say > that. I agree that, when testing parsing, it's frequently desirable to run through the entire parser: so this is more complicated than, say, testing a simple class interface. But I bet you can get a big speedup even here. Am I correct in thinking that you run gcc on each input test case, even if all you really want is the parser? Or at least a significant chunk of gcc? In that case, you should be able to speed this up significantly by just calling the parser itself and directly checking the resulting parse tree (or just checking what you care about, e.g. that no errors are emitted). No codegen necessary, no writing any output files, and you can probably do tricks to significantly reduce the time spent reading input files as well. I don't, offhand, see any barrier to testing the parsing of a 20 line chunk of test source code in just a few milliseconds. >> A C++ example, which is probably closest to your situation: > That is just for a simple C++ code. Our unit testing will be over > something like a million times larger than most unit testing which > case unit testing falls down. Sure, that example is testing a library that's only a couple thousand lines long. And the test coverage isn't quite as good as it should be; pretty close, though. Even so, I would be impressed if GCC is really a million times larger than that. :-) I have hands-on experience with unit testing C++ code bases of about a half-million lines of code. (Which started off as legacy code, and I'm sure it was in worse shape than GCC's.) Unit testing works there, and I see any obvious boundary in sight. David Carlton [EMAIL PROTECTED]
