Re: Bad test functions in Test::Exception
On 30 Jan 2007, at 18:19, A. Pagaltzis wrote: That could easily be accomodated by having `throws_ok` accept a sub ref as its condition argument. Then Test::Exception could pass the value of $@ to this sub as the first argument, and clear $@ to force people to use that argument instead of $@ itself; handy because $@ is maddeningly difficult to correctly preserve for testing. Nice idea. I'll add it to the todo list (patches, with tests, welcome :-) Cheers, Adrian
Re: Bad test functions in Test::Exception
On 30 Jan 2007, at 19:48, Eric Wilhelm wrote:
# from Nadim Khemir
# on Tuesday 30 January 2007 09:17 am:
# all Test::Exceptions subroutines are guaranteed to preserve the
state # of $@ so you can do things like this after throws_ok and
dies_ok like $@, 'what the stringified exception should look like';
This wouldn't be needed if dies_ok didn't exist.
I think the arguments in favor of dies_ok are good. But, I think it
would be better to return the $@ rather than trying to preserve it.
[snip]
This breaks the convention that Test::More of test methods returning
the test failure status so you can have things like:
lives_ok { $o-fribble } or diag Hmmm... unexpected failure,
Dumper( $o );
I quite like this, and it's too late to change anyway :-)
Cheers,
Adrian
Re: Bad test functions in Test::Exception
Hi Nadim,
On 30 Jan 2007, at 17:17, Nadim Khemir wrote:
[snip]
The bad guys:
# Check that something died
dies_ok { $foo-method1 } 'expecting to die';
Am I the only one who got this to pass, to find out later that what
cause the
error had nothing to do with the message I displayed.
[snip]
For me this is like somebody complaining that:
ok $o-answer, 'answer is 42'
passes when answer() return -1 :-) It's our job as developers to use
the appropriate tests for the things we're interested in.
If you are interested in what has been specifically thrown use
throws_ok. If you don't care use dies_ok.
Checking that something
dies is _not_ good enough. One should check more thorowly. Yes, I
know I can
use throws_ok but why have dies_ok ? I'd say that dies_ok might
make your
testing worse.
Emphasis on the might. There are times when you don't know what
kind of exception you'll be getting. There are times when you don't
want to bind yourself to a particular exception format that's under
the control of a third party piece of code. There are times when you
know you'll be tweaking the exception format / classes during
development so do not want to over specify a test. Sometimes you /do/
just want to know whether something dies.
At least I do ;-)
Use only the superior throws_ok.
# Check that something did not die
lives_ok { $foo-method2 } 'expecting to live';
Is the above realy a test? Ok but testing what? why wouldn't we
wrap all our
test in lives_ok? No, I don't think lives_ok makes any sense. I'd
be very
happy to see real examples of lives_ok that add anything to a test
suite.
Okay.
One of the useful features of Perl's test framework is that we don't
bail out on the first test failure so we can test a whole bunch of
things at once like this:
foreach my $arg ( - .. ) {
lives_ok { $o-fribble( $arg ) } fribble worked with $arg;
}
If we don't use lives_ok (or similar) then the script dies at the
first failure, rather than running the rest of the tests.
# Check that a test runs without an exception
lives_and { is $foo-method, 42 } 'method is 42';
Isn't this equivalent to is($foo-method, 42 , 'method is 42') ?
The test
framework will catch the error if any. It's just weird to attempt
to catch
something when the expected result is to pass.
Consider, like lives_ok, the cases where you don't want to abort the
whole script in the case of failure.
# all Test::Exceptions subroutines are guaranteed to preserve the
state
# of $@ so you can do things like this after throws_ok and dies_ok
like $@, 'what the stringified exception should look like';
This wouldn't be needed if dies_ok didn't exist.
Yes it would. It allows you to interrogate the exception after the
test. For example:
throws_ok { $o-read_file( 'unreadable.txt' ) } 'UnreadableFile';
SKIP: {
skip no exception = 1 unless my $e = $@;
is $e-filename, 'unreadable.txt', 'exception captured filename';
}
I postulate that Test::Exception would be even better if we removed
the bad
guys.
As you can probably guess I disagree :-)
Cheers,
Adrian
Re: Bad test functions in Test::Exception
On 1/31/07, Adrian Howard [EMAIL PROTECTED] wrote:
On 30 Jan 2007, at 20:11, Joshua ben Jore wrote:
Interestingly, this has caused me to wonder how well Test::Exception
handles the corner cases where $@ is clobbered during the scope ending
of eval{} and related.
It doesn't. It's been on my list for some time, but I'm too lazy :-)
I've just filed a bug against it at
http://rt.cpan.org/Ticket/Display.html?id=24678. The overall moral is
when using eval{} you have to test the return value as well as [EMAIL
PROTECTED] $@
can be empty under errors which is kind of a bummer.
My problem with making T::E do the right thing (for certain
definitions of right) is that it may not help folk - since the edge
case behaviour is something that the majority of people never
encounter so while the test will work their code won't.
Well... what I was hoping was that by making T::E exceptionally picky
that undetected errors would become visible. Right now you can have
some code that runs in T::E which says it passed but maybe it didn't
really and we just didn't notice. If T::E begins taking notice of
these corner cases then they also get promoted to a level that the
user is aware of.
I want to get to the world where these corner cases have enough bright
lights on them so they're fixed or at least compensated for. Then
there's the *next* set of corner cases ... If T::E doesn't change to
try to catch them then I see no change in the world and the corner
case continues to go undiagnosed.
dies_ok { $o-annoying_corner_case } 'exception thrown';
do the SIG{__DIE__} dance make the tester write
dies_ok { $o-annoying_corner_case; 1 } 'exception thrown';
If the SIG{__DIE__} dance happens entirely in T::E (as I suggested in
my patch) then the user's tests do not require any change. I thought
it'd be good to test all three things: that eval returned something
other than undef, that $@ is empty, and SIG{__DIE__} wasn't called.
Josh
