Look at Test::Tester, it hijacks the Test::Builder object and replaces with
another that sometimes delegates to the real Test::Builder object and other
times delegates to a custom Test::Builder object that just collects results
without outputting anything or affecting the "real" test results.
You can probably do what you need just with Test::Tester::run_tests. This
returns are an array of test results including, pass/fail, diagnostics and
all the other stuff that's mentioned under details in the Test::Builder
docs, so for example
my @results = run_tests(
sub {
for my $i (1..1000)
{
# these test calls get collected for later analysis
ok(test_for_i($i), "i=$i"); # name is "i=$i"
}
}
}
my @bad = grep {$_->{ok} == 0} @results;
if (@bad)
{
@values = join("\n", map {$_->{name}} @bad);
# these calls go to the real Test::Builder
fail();
diag(@bad." tests failed, value were\n$values\n");
}
F
On Sat, Feb 26, 2005 at 06:00:02PM -0500, Tom Moertel wrote:
> Is there any good way to temporarily suspend or, even better, take
> control of a Test::Builder testing session?
>
> The reason I ask is because I am working on a specification-based
> testing system for Perl that utilizes random-testing methods in which
> a single property check can involve thousands of random test cases.
> It would be great if testers could take advantage of Test::More
> and other Test::Builder-based testing modules in their property
> declarations, but right now that is tricky.
>
> The problem is that when somebody uses a function such as like() or
> is_deeply() within a property, it ends up getting called thousands of
> times when the property is checked. Test::Builder naturally thinks
> each one of these calls represents a complete test case, and it
> generates thousands of "ok" outputs and so on. Oops.
>
> What I want is for each property check to be seen as a single test.
>
> How I do do this now is somewhat hackish. For the duration of a
> property check, I redefine some Test::Builder internals like so:
>
> sub check_property {
> no warnings 'redefine';
> my $property = shift;
> my $diags = [];
> local *Test::Builder::ok = sub { $_[1] ? 1 : 0 };
> local *Test::Builder::diag = sub { shift; push @$diags, @_; 0 };
> return ( $diags,
> Test::LectroTest::TestRunner->new(@_)->run($property) );
> }
>
> The idea is to sever the part of Test::Builder that reports back to
> the caller (which I want) from the part that reports back to the test
> harness (which I do not want). It seems to work fine, but mucking
> around with another module's internals carries with it an element of
> risk that I don't like, and I would rather have a cleaner option.
>
> Is there a cleaner option? If not, can/should we create one?
>
> Cheers,
> Tom
>
>
> For more background, see the following:
>
> http://community.moertel.com/LectroTest
> http://search.cpan.org/dist/Test-LectroTest/lib/Test/LectroTest/Compat.pm
