Hi Akim, * Akim Demaille wrote on Fri, Oct 17, 2008 at 12:15:42PM CEST: >> Consequently, there is little point in showing users how to write such >> a rule. > > I strongly disagree here.
And your position is quite founded, if but a bit hard to swallow at first. ;-) There are several aspects to this whole issue; I will try to keep them as separate as possible. First, I don't want to (have to) tell users about $(am__check_pre) and $(am__check_post). If I did, I'd call them $(check_prepare) and $(check_postprocess), but I don't in the first place, as: - this either nails down the implementation quite severely, to the point that we may not easily be able to fix bugs or inconsistencies in it, - allowing users not to use them may prevent consistent introduction of post-test cleanup macros, hard errors, and similar, - it is simply an Ugly API[tm]. (It essentially would amount to documenting $p, $tst, $dir, $f, $log, possible even how and what to write to $log; tty colors etc.) But if I understand you correctly, then that isn't the grip you have with this at all. The missing bit is: you want to be able to provide a command executing tests; moreover, different commands for different sets of tests. And that part is a Good Idea[tm]. I'd even go as far as: both the developer and the user might want to be able to have a word here. (Think valgrind, think simulator (don't think about Libtool issues just yet), ...) But I don't see the real need for users to specify the whole rule. > I wrote check.mk with a clear and simple model in mind: we are compiling > tests, and they can be written in several "languages". This fits nicely > the Make model and gives all the rest (parallelism etc.) for free. *And* > it provides an opportunity to present a uniform interface a la Automake, > using > > TESTS_SUITES = foo-suite.log bar-suite.log > foo_suite_SOURCES = 1.foo 2.foo > bar_suite_SOURCES = a.bar b.bar > > foo_COMPILER = ./foo > bar_COMPILER = ../bin/bar --verbose --warnings=all Which is a nice idea. I like it. I even tried to go this way when I first started. It turned out that a number of purely practical issues needed to be addressed: - if you still want to be able to easily limit the number of tests run at 'make' run time, you have to go through hoops to ensure that *_OBJECTS is a direct dependent of *_SOURCES (resp. TESTS), - *for this language*, the mapping from sources to objects should *not* be an automake internal detail, - the current code looks not really adapted to reset known extensions and languages per Makefile.am, so this could be a bit of work (if only to check that nothing silently breaks), - then, I'm quite a fan of backward compatibility, with the aim that most users would get a working parallel testsuite by simply adding AUTOMAKE_OPTIONS = parallel-tests to their Makefile.am. Cf. testsuite addition that tests just this. - In your proposal, it would IMVHO still be necessary to specify the set of extensions that tests can have (in order to support @substed@ tests at least, and in order to avoid ambiguities and per-target rules). I decided that this was more than I could implement in the time that I had for this, and it is probably more than I will have time for in the near future. Alternatively, is there a volunteer for implementing it? I'm currently trying to gauge whether the question is: - drop my 'parallel-tests' patches now, try for or wait for something better? or - add my 'parallel-tests' bits now, and go on? Adding the patch set now would likely mean some incompatibility in the future when your proposed semantics are implemented, which would be bad. Comments appreciated. And please don't get me wrong, I'm not attached to this patch set, but I do want to provide parallel tests eventually. Thanks, Ralf