On 17/07/06, demerphq <[EMAIL PROTECTED]> wrote:
On 7/17/06, Fergal Daly <[EMAIL PROTECTED]> wrote:
> On 17/07/06, demerphq <[EMAIL PROTECTED]> wrote:
> > On 7/17/06, Torsten Schoenfeld <[EMAIL PROTECTED]> wrote:
> > > On Mon, 2006-07-17 at 11:39 +0200, demerphq wrote:
> > >
> > > > Test names shouldnt be optional.
> > >
> > > I disagree. I would find it cumbersome to have to come up with a
> > > description for each and every test.
> >
> > I dont think its that cumbersome at all. Even stuff like
> >
> > "Fnorble 1"
> > "Fnorble 2"
> >
> > is sufficient.
> >
> > > > Finding a particular test in a file by its number can be quite
> > > > difficult, especially in test files where you dont have stuff like
> > > >
> > > > 'ok 26'.
> > > >
> > > > When ok() and is() are silently incrementing the counter and test
> > > > names arent used how is one supposed to find the failing test? As you
> > > > probably know it can be quite difficult.
> > >
> > > Well, if the test passes, there's no need to know where exactly it's
> > > located. If it fails, the diagnostics contain the line number:
> > >
> > > not ok 6
> > > # Failed test in t/xxx.t at line 26.
> > >
> > > I've never seen incorrect line numbers.
> >
> > I have. Lots and lots and lots of times. I could do a survey but IMO
> > it would be a waste of time.
> >
> > Anytime you need to do testing that doesnt exactly fit into the
> > provided tools from the Test::Builder suite you either need to design
> > a Test::Builder style module, or you get bogus line numbers because
> > the wrapper routines around the tests report the wrong thing.
> > Basically determining where the test originated is determined by
> > heuristic (much as Carp does its thing by heuristic). And as anybody
> > with comp-sci background knows heuristics are called that and not
> > algorithms because they are not provably correct. They get things
> > wrong.
>
> It's not really a heuristic, it's perfectly reliable.
The fact that it does the same thing every time for a given set of
input doesnt mean that it does the RIGHT thing. And i dont see how its
possible to automatically do the right thing every time. Therefore if
you need a custom wrapper you need to teach it to do the right thing.
The way it works is that Test::Builder expects there to be 1 layer of
stack between the call from the test script and the call into
Test::Builder->ok(). If your library is going to put more layers in
there then it has to tell Test::Builder about it. The standard way to
do this is to put
local $Test::Builder::Level = $Test::Builder::Level + 1;
at the top of every function in your library that does not actually
call ->ok() but does call something that will call it (you have to do
a bit more work if you want to call ->ok() directly and indirectly
from the same function). So Test::Builder knows how many levels of
stack to ignore to get out of the library and back to the test script.
Doing the right thing is not difficult, it's just involves some copy
and paste tedium. If you do the right thing though, you are guaranteed
to get the right result.
> Tthe problem is
> that it requires action on the part of the Test::XXX author to get it
> correct.
And often the testfile author as well if there isnt an "off-the-shelf"
Test:: module appropriate to the task at hand.
> There's a $Test::Builder::Level variable which should be
> incremented every time you go further down the stack inside the
> library, if you don't change it correctly then the user will get
> incorrect line numbers.
But what happens if you are using something like
sub do_test {
my ($testname,$testvalue,$code)[EMAIL PROTECTED];
my $expect=somefunc($testvalue);
is ($expect,$code->($testvalue),$testname." somefunc");
is( ....);
is( ....);
}
then you end up with the tests reporting that the wrapper sub failed.
Or you have to rewrite the wrapper as a Test::Builder module. (Blech).
Absolutely, unless Test::Builder and TAP support nested blocks, we're
stuck in a world where structured programming is difficult (TAP
without nested blocks considered harmful :).
However in the case of the subroutine above, you could solve some of
the line number issues by have Test::Builder give a "full" stacktrace
rather than just reporting a single line number.
> A quick look on CPAN, showed up several Test::* modules that don't
> bother to test it, probably because Test::Builder::Tester does not
> make it particularly pleasant to do. Test::* modules that use
> Test::Tester get it automatically because it doesn't depend on
> scraping the diagnostic string.
So there are routes to make this easier, but it seems to me they are
likely to be a lot harder than providing a proper identification
string.
This doesn't make it easier to correctly do it, it just makes it
easier to test that you've correctly done it (free in fact).
F
> > A string in a test file is trivial to find. Open the test file in an
> > editor and do a search for the string, and presto you have the failing
> > test.
>
> Test names are great. Line number are useless inside a loop,
Exaclty. Stuff like:
my @tests=( [ "B<Foo>", "*Foo*", "ASCII Bold" ] );
is easily located.
cheers,
Yves
--
perl -Mre=debug -e "/just|another|perl|hacker/"
