chromatic wrote:
On the other hand, it doesn't do anything differently with TAP as currently defined, and I share Schwern's case of howling fantods at recommending that people look in Test::Builder's test results list. Out of process interpreting and reporting feels so much nicer.
An addendum on rereading this part above:
This is another example of the coupling. Even if out-of-process is nicer (and I agree, for the most part), why not put all the information acquired during testing into the test results list and then serialize that over the output stream? There's pros and cons to any alternative to TAP, of course, but as long as Test::Builder is tightly coupled to TAP and as long as so many Test modules are written to use Test::Builder, it makes experimentation or migration to alternatives rather difficult.
The features that I think we're looking at are these:
* Provide a way for a controller to execute a test script and retreive data over a given channel
* Provide various test functions to a test script from one or more test modules
* Provide a way for test scripts or test modules to designate a distinct location where subsequent results are recorded
* Provide a way test modules to record the results and details of a test function to the previously designated location
* Provide a way for test modules to serialize the results of one or more locations in a given protocol over a given channel
* Provide a way for a controller to interpret results in a given protocol
* Provide a user-interpretable test report
Just to be playful with concepts:
* Test::Answer -- holds the details of a particular test (pass/fail/skip/todo)
* Test::Scoresheet -- holds a collection of Test::Answers
* Test::Booklet -- sets up a default Test::ScoreSheet object to hold results, creates and maintains new ones as requested and manages an "active" ScoreSheet for recording Answers. On exit, serializes all Scoresheets. The serialization method is abstract and subclasses must implement particular protocols with helper modules (e.g. Test::Protocol::TAP).
* Test::Proctor -- fires up a test script with a particular subclass of Test::Booklet (e.g. Test::Booklet::TAP) and records the response. (perl -MTest::Booklet::TAP test.t)
* Test::ReportCard -- parses results received from Test::Proctor. Subclasses parse particular protocols with helper modules and produce a human/machine readable report. Sub-subclasses could offer alternate report formats.
(Ideally, we'd even abstract the channel in there so we could do it in a way other than stdout).
IMO, the more we can keep these functions uncoupled and generic the better -- and we can let specific implementations provide the details. It certainly sounds like chromatic and Michael have got things headed in the right direction and I look forward to seeing what we wind up with.
David
