On 09/14/2016 02:37 AM, Richard Biener wrote:
Note that while the "snippets" may largely work (not sure how many
you tried to come up with) I see the issue that a lot of RTL "unit
tests" would need some trees set up, like to properly form MEM_EXPRs
or REG_DECL or even SYMBOL_REFs. So I fear that the scope of
unit-tests we can implement with the proposed scheme is very limited
(you may also need other stuff setup, like alias analysis or parts of
IRA or cost analysis parts). So I agree a separate testing backend
is a good way to make unit-testing more stable on the target side we
also need a way to provide input on some of the global state that is
currently set up by frontends.
Agreed across the board. I think we're still early in the learning
phase on this stuff. I shudder when I think about the amount of state
that we depend on, but which is not represented in the RTL dumps.
But I do think there are some things we can test for in an RTL self
testing framework and that having one would be a significant step forward.
So I think we have two big questions.
First, does David's work have value as a way to directly test pieces of
the RTL pipeline without having to generate all the RTL bits by hand. I
think the answer is yes.
Second, will David's work help identify internal state that is not
expressed in the RTL dumps or poor modularity (ie, cases like trees
embedded within the RTL structures). I think the answer to this is yes
as well.
Third, is a framework like Bernd's useful as well and can it be mated
with David's work. And I think the answer is yes & yes as well with the
result being more useful than either Bernd or David's work in isolation.
But my biggest worry is with putting unit-tests into cc1 itself --
even more so with RTL unit tests of this kind than with all the other
ones we have. We'll quickly have 99% of a source file comprised of
RTL unit tests rather than source (and cc1 object size as well).
Hardly something we want to have (not even mentioning bootstrap time
issues).
I can live revisiting this -- I always expected we would after we
started building out the framework.
Yes, putting the unit-tests in source files makes us not require
exporting an interface to the parts we are testing. But that's about
the only advantage I can see. You didn't show that it isn't possible
to put the small test you were writing into a RTL-frontendish test
which starts compiling the function with the test with the pass you
are about to unit-test.
The other advantage is tests which use the internal APIs are easy to
identify/fix when an internal API is changed.
Jeff
