On Sun, Oct 09, 2005 at 10:11:45PM +0100, Nik Clayton wrote: > >There was no protocol change here because there never was a protocol. > >Test::Builder::Tester parses comments! BAD! EVIL! WRONG! > > First, there's not a lot T::B::T can do in this situation. One of the > things you want to test when making sure that your Test:: module works > is that the diagnostics for failed tests are correct. That means > screen-scraping the results and comparing them against a stored string.
There's plenty T::B::T can do and has done. The problem is the diagnostic output from a test module which uses Test::Builder (let's call it Test::Foo) is actually an amalgamation of Test::Builder's diagnostics and Test::Foo's. There's nothing wrong with Test::Foo testing its own diagnostics, that's under its control, but Test::Builder's can change out from under Test::Foo just like any other unspecified feature of a module you depend on. test_fail() handles this problem decently by encapsulating the matching of the diagnostics output by Test::Builder. Now that TBT is shipped with Test::Builder they can be kept in sync. This way Test::Foo only has to hard code its own output in its own tests, not Test::Builder's. This is ok. There's further extentions which can be done to Test::Builder to allow TBT to supress Test::Builder from outputing any of its own diagnostics (or to more easily ignore them) thus leaving only Test::Foo's output in the stream to be scraped. > Second, in the TAP documentation, they're not called 'comments', they're > called 'diagnostics'. I don't care if they're called "Truman Capote", they're lines whose contents are to be ignored. The harness ignores them. They're comments. They're called "diagnostics" because "comments" is too generic and because it lines up with the whole "d" theme (description, diagnostic, directive). Don't put so much weight on labels. > There's nothing in the documentation that > indicates that the harness is allowed to alter the format of the > diagnostics (beyond preprending '# ' and appending "\n" to parameters to > the diag() function Nobody is or was or has suggested that the harness alter the format of the diagnostics. I don't understand where that came from. Test::Builder->diag doesn't alter them either (except, as mentioned, with the # and the \n). I think you're a bit confused about where Test::Builder ends and the harness begins. The diag() function is Test::Builder which is not the harness. TAP has nothing to say about the behavior of diag() except that it outputs a valid diagnostic (that being a line starting with #). TAP has nothing to say about Test::Builder except that it outputs valid TAP. Test::Builder and TAP are different things. > (which, tellingly, is not called the "comment() function"). Its not called comment() because I felt "diagnostics" are something more strongly associated with an error or failure and "comment" is too generic. That and its the name Mark Fowler suggested. What about "diagnostics" implies formatted data, anyway? > It's an unfortunate accident of history that the diagnostics currently > look a lot like the comments in many other output formats. This has > probably led to the (IMHO) erroneous assumption that T::B can alter the > format of the diagnostics without affecting over modules. Let's make this crystal clear. Beyond being a line which starts with #, TAP diagnostics have no format. They never have. TAP ignores their contents. Test::Builder does not alter the format of diagnostics. It just changed the ones that it emits from its own ok() method. Its allowed to do that, the diagnostics output by ok() were never specified in Test::Builder and still are not. But because it had remained the same for so long people started to rely on it. Shit like this happens. TAP is really not involved here. -- Michael G Schwern [EMAIL PROTECTED] http://www.pobox.com/~schwern Life is like a sewer - what you get out of it depends on what you put into it. - Tom Lehrer
