Michael G Schwern wrote:
On Sat, Oct 08, 2005 at 06:26:46PM +0100, Nik Clayton wrote:
Define a new version of TAP with a single change.
Specifically, emit a version number in the TAP output that describes the
version of TAP that's being emitted.
While this may be an interesting idea, its irrelevant to the TBT problem
because...
Not exactly...
This would then make it possible to
a) Add new features/change the format without breaking tools that read it.
...the diagnostics emitted by Test::Builder are TAP comments, if
TAP even sees them at all since they go to STDERR which most TAP parsers
ignore.
There was no protocol change here because there never was a protocol.
Test::Builder::Tester parses comments! BAD! EVIL! WRONG!
First, there's not a lot T::B::T can do in this situation. One of the
things you want to test when making sure that your Test:: module works
is that the diagnostics for failed tests are correct. That means
screen-scraping the results and comparing them against a stored string.
Second, in the TAP documentation, they're not called 'comments', they're
called 'diagnostics'. There's nothing in the documentation that
indicates that the harness is allowed to alter the format of the
diagnostics (beyond preprending '# ' and appending "\n" to parameters to
the diag() function (which, tellingly, is not called the "comment()
function").
It's an unfortunate accident of history that the diagnostics currently
look a lot like the comments in many other output formats. This has
probably led to the (IMHO) erroneous assumption that T::B can alter the
format of the diagnostics without affecting over modules.
As this issue shows, you can't.
If:
a) TAP output had an explicit version number.
b) The version number had been incremented after you made the change to
Test::Builder.
c) Test::Harness supported an "Only emit TAP corresponding to version
'x'" option (which would get passed down to Test::Builder)
d) Test::Builder::Tester (and Test::Tester, for that matter) forced the
users of those modules to explicitly state the TAP version number that
they were testing against.
then this issue wouldn't (and couldn't) have happened.
N
