On Wed, Jan 26, 2005 at 05:55:24PM -0500, David Golden wrote:
While this simple example just has the test function shove a reference to a hash onto a global array @{$Tester::RESULTS{$filename}}, the test function could just as easily create an object (Test::Object::Ok, Test::Object::Is, etc.) that holds relevant data for that type of test and push that onto the array. E.g. for "is" it would hold the "got" and "expected" results.
Test::Harness does not have any of this information. It only has the test output which is if the test passed or failed and the name.
Within the test program you can get at some of this information already using Test::Builder. It does not store the arguments of each individual testing function and I'm not really sure its possible to do so in any sort of universal fashion.
What you're suggesting requires both the test running and test output to be done by the same process or for the test to output a whole lot more information. Both would require a radical alteration to the way tests are run.
You're completely right. It's not just a rewrite of Test::Harness (er, Straps), but all of Test::More (er, Builder) as well. It's a whole parallel approach to the existing system, I'll admit. I wouldn't try to do it as a drop in replacement, for sure.
But is there any reason they can't be run in the same process? Right now, T::H::Straps just executes the test file by calling perl and handing it to the system, and then iterates through the output, parsing for test info. If it's in the same process, we eliminate the need for parsing. If we can't do it in the same process, (e.g. maybe having to do with @INC, switches, environmental variables, Devel::Cover -- as I said, the devil in the details), we could still have tests that returned structured output (e.g. YAML) instead of just ok/not ok and write something to parse that. Or have them write the usual ok/not ok to stdout and write detail in a structured form to some other process through a socket or to a log file.
Either way, it's all still pretty much a separate approach from what we have today with Test::Harness and Test::Builder. (Which work fine for me -- I'm not necessarily advocating for change, rather just brainstorming some really different alternatives.)
Regards, David Golden
