On 1 Nov 2007, at 15:44, A. Pagaltzis wrote:
* Andy Armstrong <[EMAIL PROTECTED]> [2007-11-01 15:50]:
The config needs to be dynamic at test time - so it might as
well be a script that runs and outputs a description of which
tests to run, right?
But it only needs to be dynamic in a minority of cases. So it
seems to me it should be the other way around – rather than run
a script to emit the config, have a config that has a directive
to run a script and include its output as part of the config.
Well if it doesn't need to be dynamic it doesn't need to exist at all
- you just have the tests you want to run.
That makes it easier for other tools to analyse the test configs.
So isn't that nearly TSP? :)
In some ways. I just don’t think the harness should look at the
output of tests to check if it’s TSP rather than TAP. The list of
tests to run should be determined up front
Well that's desirable certainly - but isn't it also valuable to be
able to dynamically include include or exclude large numbers of tests
when you need to? And to do so using a well defined protocol?
, and the test scripts
should only ever output TAP.
Yeah, I have sympathy with that. I really don't want to complicate
things gratuitously.
Mixing them with scripts that output TSP feels like a confusion
of concerns to me.
I can see that too. In that sense the existing "Bail Out" directive
feels like an anomaly too.
For the configs there are also a number of issues like dealing
well with multiple possible configurations and resolution of
relative paths (esp. in compound configs).
Yup - not impossible though. You can use HTML-like semantics -
relative paths are resolved relative to the config, abs paths relative
to some well defined project root.
I think this needs a little bit more design. And since YAML is
making its way into TAP anyway, maybe it should be YAML-based
rather than a completely ad-hoc TAPish format?
Oh sure - that'd be fine. The choice of representation is somewhat
orthogonal to whether it's a good idea in the first place though.
--
Andy Armstrong, Hexten
