On 7/17/06, Fergal Daly <[EMAIL PROTECTED]> wrote:
On 17/07/06, demerphq <[EMAIL PROTECTED]> wrote:
> On 7/17/06, Torsten Schoenfeld <[EMAIL PROTECTED]> wrote:
> > On Mon, 2006-07-17 at 11:39 +0200, demerphq wrote:
> >
> > > Test names shouldnt be optional.
> >
> > I disagree. I would find it cumbersome to have to come up with a
> > description for each and every test.
>
> I dont think its that cumbersome at all. Even stuff like
>
> "Fnorble 1"
> "Fnorble 2"
>
> is sufficient.
>
> > > Finding a particular test in a file by its number can be quite
> > > difficult, especially in test files where you dont have stuff like
> > >
> > > 'ok 26'.
> > >
> > > When ok() and is() are silently incrementing the counter and test
> > > names arent used how is one supposed to find the failing test? As you
> > > probably know it can be quite difficult.
> >
> > Well, if the test passes, there's no need to know where exactly it's
> > located. If it fails, the diagnostics contain the line number:
> >
> > not ok 6
> > # Failed test in t/xxx.t at line 26.
> >
> > I've never seen incorrect line numbers.
>
> I have. Lots and lots and lots of times. I could do a survey but IMO
> it would be a waste of time.
>
> Anytime you need to do testing that doesnt exactly fit into the
> provided tools from the Test::Builder suite you either need to design
> a Test::Builder style module, or you get bogus line numbers because
> the wrapper routines around the tests report the wrong thing.
> Basically determining where the test originated is determined by
> heuristic (much as Carp does its thing by heuristic). And as anybody
> with comp-sci background knows heuristics are called that and not
> algorithms because they are not provably correct. They get things
> wrong.
It's not really a heuristic, it's perfectly reliable.
The fact that it does the same thing every time for a given set of
input doesnt mean that it does the RIGHT thing. And i dont see how its
possible to automatically do the right thing every time. Therefore if
you need a custom wrapper you need to teach it to do the right thing.
Tthe problem is
that it requires action on the part of the Test::XXX author to get it
correct.
And often the testfile author as well if there isnt an "off-the-shelf"
Test:: module appropriate to the task at hand.
There's a $Test::Builder::Level variable which should be
incremented every time you go further down the stack inside the
library, if you don't change it correctly then the user will get
incorrect line numbers.
But what happens if you are using something like
sub do_test {
my ($testname,$testvalue,$code)[EMAIL PROTECTED];
my $expect=somefunc($testvalue);
is ($expect,$code->($testvalue),$testname." somefunc");
is( ....);
is( ....);
}
then you end up with the tests reporting that the wrapper sub failed.
Or you have to rewrite the wrapper as a Test::Builder module. (Blech).
A quick look on CPAN, showed up several Test::* modules that don't
bother to test it, probably because Test::Builder::Tester does not
make it particularly pleasant to do. Test::* modules that use
Test::Tester get it automatically because it doesn't depend on
scraping the diagnostic string.
So there are routes to make this easier, but it seems to me they are
likely to be a lot harder than providing a proper identification
string.
> A string in a test file is trivial to find. Open the test file in an
> editor and do a search for the string, and presto you have the failing
> test.
Test names are great. Line number are useless inside a loop,
Exaclty. Stuff like:
my @tests=( [ "B<Foo>", "*Foo*", "ASCII Bold" ] );
is easily located.
cheers,
Yves
--
perl -Mre=debug -e "/just|another|perl|hacker/"
