Re: Blurring the line between assertions and tests
On Sat, Aug 02, 2003 at 05:02:58PM +0100, Adrian Howard wrote:
> >Or even more trivially, take Test::AtRuntime and swap
> >out Test::Builder::ok() with something that dies on failure.
> [snip]
>
> I was thinking about the ability to have an assertion block - so you
> could do (something like):
>
> ASSERT {
> isa_ok($_[0], 'Foo');
> ok( $_[1], 'non-false value');
> };
>
> and get a single die reporting all failing tests in the block. So it
> would need something a little more sneaky than overriding T::B::ok.
If an ASSERT just translates into:
{
my $test_num = Test::Builder->current_test;
isa_ok($_[0], 'Foo');
ok( $_[1], 'non-false value' );
assert(0) if anything_has_failed_since($test_num);
}
well then there ya go.
--
Don't be a lover, girl, I'd like to shamelessly hump your inner child.
Re: Blurring the line between assertions and tests
On Sat, Aug 02, 2003 at 11:43:26AM -0700, Ovid wrote:
> This does mean, though, that it won't play nicely with versions of Perl < 5.6.0. Is
> that trade
> off acceptable?
I'll throw in the fallback "if DEBUG" style
TEST {
my $sky = look('up');
is( $sky, 'blue' );
} if DEBUG;
which will work without filters, attributes or magic herrings.
--
Stupid am I? Stupid like a fox!
Re: Blurring the line between assertions and tests
On Sat, Aug 02, 2003 at 11:22:43AM -0700, chromatic wrote: > Could these instead be subroutine attributes? I can see a lot of > advantages there. I know very little about subroutine attributes, so you're going to have to investigate that one. Keep in mind, though, that we want the *whole call to disappear*. Not to call a subroutine and have that quickly return, even if its in a clever attribute. -- Once is a prank. Twice is a nuisance. But NINE TIMES is a TRADITION. -- Mark-Jason Dominus in <[EMAIL PROTECTED]>
Re: Blurring the line between assertions and tests
--- chromatic <[EMAIL PROTECTED]> wrote:
> On Friday, August 1, 2003, at 05:03 PM, Michael G Schwern wrote:
>
> > Ooooh! I just had a great idea. Use "TEST { ... }" instead of "TEST:
> > { ... }"
> > in Test::AtRuntime.
>
> Could these instead be subroutine attributes? I can see a lot of
> advantages there.
That would be much nicer. I am so used to writing SKIP: {} and TODO: {} that I
scratch my head
and wonder why my cleanup doesn't work in END: {}. The presence or absence of that
little colon
causes me enough grief that I suspect having such a subtle difference as TEST {}
versus TEST: {}
is begging for trouble.
This does mean, though, that it won't play nicely with versions of Perl < 5.6.0. Is
that trade
off acceptable?
Cheers,
Ovid
=
Silence is Evilhttp://users.easystreet.com/ovid/philosophy/indexdecency.htm
Ovid http://www.perlmonks.org/index.pl?node_id=17000
Web Programming with Perl http://users.easystreet.com/ovid/cgi_course/
__
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
Re: Blurring the line between assertions and tests
On Friday, August 1, 2003, at 05:03 PM, Michael G Schwern wrote:
Ooooh! I just had a great idea. Use "TEST { ... }" instead of "TEST:
{ ... }"
in Test::AtRuntime. If the user has Filter::Simple, use that to strip
out
the TEST blocks. Otherwise, its a function call to TEST() passing in
a code
ref which it would run or not run based on if we're testing or not.
Except
now there's a dependency on Sub::Uplevel. :(
Could these instead be subroutine attributes? I can see a lot of
advantages there.
-- c
Re: Blurring the line between assertions and tests
On Saturday, August 2, 2003, at 01:03 am, Michael G Schwern wrote:
Class::Contract has always bothered me as way too much Kool-Aid to
drink in one sitting.
I completely agree. Having the ability to apply pre/post/invariant
functionality to normal Perl classes has been on my to do list for
years. Still waiting for a visit from the spare time fairy ;)
Cutting DBC down to its Best Trick: inheritable
invariants. And an invariant is just a test before or after a function
call. All the rest of Class::Contract, the enforcement, the new OO
syntax...
that can all be thrown out. If you want all that, use a different
module.
So what you're left with is...
use Test::Contract;
use Test::More;
sub add {
pre { is( @_, 2, 'got two arguments' ) }
my $sum = $_[0] + $_[1];
return $sum;
post { unlike( $ret, qr/\D/ ) }
}
...and the rest is a little clever filtering.
Ah. The infamous "trivial matter of programming" :-)
I've thought it might be nice to be able to separate out the pre-post
conditions from the actual subroutines in question and apply them in an
AOPish manner but as I said before --- never had the time to think
about it in detail.
Ooooh! I just had a great idea. Use "TEST { ... }" instead of "TEST:
{ ... }"
in Test::AtRuntime. If the user has Filter::Simple, use that to strip
out
the TEST blocks. Otherwise, its a function call to TEST() passing in
a code
ref which it would run or not run based on if we're testing or not.
Except
now there's a dependency on Sub::Uplevel. :(
Neato - but I have to admit I'm in the "source filters are evil" camp
:-) Roll on Perl6 and decent macros!
- I think the idea of being able to run tests as assertions is a cute
one worth exploring. Just having a T::B subclass that died rather than
log anything would be a boon. Giving us all the goodness of the T::B
based functions for normal assertions.
Carp::Assert::More.
The advantage would be to avoid a parallel evolution of similar groups
of functionality in the Test:: and Carp::Assert:: hierarchies.
Or even more trivially, take Test::AtRuntime and swap
out Test::Builder::ok() with something that dies on failure.
[snip]
I was thinking about the ability to have an assertion block - so you
could do (something like):
ASSERT {
isa_ok($_[0], 'Foo');
ok( $_[1], 'non-false value');
};
and get a single die reporting all failing tests in the block. So it
would need something a little more sneaky than overriding T::B::ok.
Adrian
Re: Blurring the line between assertions and tests
On Fri, Aug 01, 2003 at 11:02:19PM -, Rafael Garcia-Suarez wrote: > Michael G Schwern wrote in perl.qa : > > The only part missing is the ability to shut the tests off once you've > > released it to production. > > You could perhaps use the assertion feature of perl >= 5.9.0 > (assertion.pm and -A switch -- yes I know it lacks docs.) Backwards compatibility is desired. -- The plot seems complicated at first but with a little study it becomes hopelessly confused. - Peter Schickele, "Hansel and Gretel and Ted and Alice"
Re: Blurring the line between assertions and tests
On Sat, Aug 02, 2003 at 12:09:22AM +0100, Adrian Howard wrote:
> - Rather than running tests at live time, I'm more often doing the
> opposite. I have assertions that I only want to switch on at testing
> time since that is when I'm exercising things that might break.
>
> - This sort of thing always makes me think of things
> design-by-contract... I'm sure there is some useful intersection
> between automated tests and DBC - but I've yet to feel bright enough to
> work it out.
For my next trick...
Class::Contract has always bothered me as way too much Kool-Aid to
drink in one sitting. Cutting DBC down to its Best Trick: inheritable
invariants. And an invariant is just a test before or after a function
call. All the rest of Class::Contract, the enforcement, the new OO syntax...
that can all be thrown out. If you want all that, use a different module.
So what you're left with is...
use Test::Contract;
use Test::More;
sub add {
pre { is( @_, 2, 'got two arguments' ) }
my $sum = $_[0] + $_[1];
return $sum;
post { unlike( $ret, qr/\D/ ) }
}
...and the rest is a little clever filtering.
Ooooh! I just had a great idea. Use "TEST { ... }" instead of "TEST: { ... }"
in Test::AtRuntime. If the user has Filter::Simple, use that to strip out
the TEST blocks. Otherwise, its a function call to TEST() passing in a code
ref which it would run or not run based on if we're testing or not. Except
now there's a dependency on Sub::Uplevel. :(
> - I think the idea of being able to run tests as assertions is a cute
> one worth exploring. Just having a T::B subclass that died rather than
> log anything would be a boon. Giving us all the goodness of the T::B
> based functions for normal assertions.
Carp::Assert::More. Or even more trivially, take Test::AtRuntime and swap
out Test::Builder::ok() with something that dies on failure.
> - You'd probably want an option to pop a stack trace next to the test
> output (maybe only for failing tests?)
Definately only for failing tests. Good idea, though.
> - Option just to log failing tests might be useful?
Yep, that's in the todo.
--
Here's hoping you don't become a robot!
Re: Blurring the line between assertions and tests
On Friday, August 1, 2003, at 09:07 pm, Michael G Schwern wrote: [snip] I was thinking about inline testing, Test::Class and such and how it would be nice if we could just write test functions right in our code, like assertions. Like Carp::Assert::More, but I want all the Test:: stuff available. [snip] Eeen-teresting... random thoughts... - Rather than running tests at live time, I'm more often doing the opposite. I have assertions that I only want to switch on at testing time since that is when I'm exercising things that might break. - This sort of thing always makes me think of things design-by-contract... I'm sure there is some useful intersection between automated tests and DBC - but I've yet to feel bright enough to work it out. - I think the idea of being able to run tests as assertions is a cute one worth exploring. Just having a T::B subclass that died rather than log anything would be a boon. Giving us all the goodness of the T::B based functions for normal assertions. - You'd probably want an option to pop a stack trace next to the test output (maybe only for failing tests?) - Option just to log failing tests might be useful? Nice idea! Adrian
Re: Blurring the line between assertions and tests
Michael G Schwern wrote in perl.qa : > The only part missing is the ability to shut the tests off once you've > released it to production. You could perhaps use the assertion feature of perl >= 5.9.0 (assertion.pm and -A switch -- yes I know it lacks docs.)
Re: Blurring the line between assertions and tests
Make that... http://www.pobox.com/~schwern/src/Test-AtRuntime-0.01.tar.gz -- I knew right away that my pants and your inner child could be best friends.
Re: Blurring the line between assertions and tests
On Fri, Aug 01, 2003 at 01:07:15PM -0700, Michael G Schwern wrote:
> Another way is to use a TEST: block
> and have Filter::Simple strip them out.
>
> TEST: {
> cmp_ok( ... );
> }
> Questions? Comments? Approval?
Hell, why wait for wiser heads?
http://www.pobox.com/~schwern/Test-AtRuntime-0.01.tar.gz
The only problem is it requires 5.8.1. Something in either Test::Builder
or Filter::Simple (or a combination) causes the tests to segfault with
anything eariler. :(
http://www.iki.fi/jhi/perl-5.8.1-RC4.tar.bz2
http://www.iki.fi/jhi/perl-5.8.1-RC4.tar.gz
--
Stupid am I? Stupid like a fox!
Blurring the line between assertions and tests
I had an idea yesterday. On more than one occassion, a I've been asked
about running tests against a live site. My usual waffle is to talk about
assertions or to build a seperate test suite which is explicitly non-modifying.
Or something Skud came up with which was to tag blocks of tests in the suite
as "modifying" and not run them against a live site.
That's always felt kind of artificial. Tests are supposed to tell you
something's wrong. Having to run a suite against a live site is more like
running the tests *after* you've found out something's wrong. And then
I was thinking about inline testing, Test::Class and such and how it would
be nice if we could just write test functions right in our code, like
assertions. Like Carp::Assert::More, but I want all the Test:: stuff
available.
I think this can be done with a simple Test::Builder subclass and
attribute changes. Turn numbering off (since you don't know what order
they'll get run in, so its kind of meaningless, or if the thing will fork).
Set no_plan, since the tests would come continuously. Redirect output to a
log file... and that should be it. Then you can write:
sub day_of_week {
...
cmp_ok( $dow, '<=', 7 );
cmp_ok( $dow, '>=', 1 );
return $dow;
}
Every time day_of_week() is run those two tests get run and go to a log file.
The only part missing is the ability to shut the tests off once you've
released it to production. That's tricky. The cheap way to do it is
what Carp::Assert uses, a constant.
cmp_ok( ... ) if DEBUG;
Its wonky, but it works without fail. Another way is to use a TEST: block
and have Filter::Simple strip them out.
TEST: {
cmp_ok( ... );
}
Filtering makes me nervous, but the syntax is clean and the code will be
100% removed (at some startup cost).
Finally, there could be a TEST function which gets passed a subroutine
reference and use Sub::Uplevel to trick it into thinking its getting called
one level up...
TEST {
cmp_ok( ... );
}
This'll work, but the TEST() function will always get called, even if its
just "return if $Not_Testing" that can get expensive in hot loops.
I could do all three. :)
Questions? Comments? Approval?
--
Beer still cheaper than crack!
