On Mon, Sep 30, 2013 at 08:52:04AM +0200, Jens Rehsack wrote:
>
> Am 29.09.2013 um 21:26 schrieb Tim Bunce <tim.bu...@pobox.com>:
>
> > The DBI adopted an approach towards testing 'variants' by creating extra
> > test files that act as wrappers around existing test files.
> >
> > The wrappers modify the process in some way, such as setting environment
> > variables, before executing the original test file.
> >
> > It works well, and the DBIT has adopted the same approach.
> >
> > I think the approach is good and more widely applicable than just the DBI.
> >
> > I'd like to propose that we factor it out into a separate distribution.
> >
> > Thoughts?
>
>
> I had the same idea - even several separate distributions, but I rejected
> myself there for following reasons:
>
> 1) The variants of the driver result in an n choose k combination
> 2) The driver attributes (eg. list of dbm backends, list of mldbm serializers)
> result in a cross product of those vectors.
>
> When making separate distributions, 1st approach would be to create a really
> generic combinatorics distribution. There are some, but all currently have
> drawbacks. This distribution would likely develop an own life (combinatoric
> problems tend to become complex and require lot of performance speed ups).
>
> Based on that Math::Fast::Combi (*gg*) Test::MakeVariantTesTFiles would
> require exactly to know what shall be done where.
I think it just requires a callback to prune combinations that aren't wanted.
> I think - from the current point of view, this would result in an over
> complex API. Maybe sourcing out the combinatoric algorithms would clean
> up the code a bit and later when we know better we can additionally create
> a distributions for computing variants (combinations, variations,
> permutations,
> cross products, …) and based on that ?
I think a clean simple API could be designed. Keeping the test rewriting
logic separate simplifies the rest of DBIT. It's a Separation Of Concerns.
Here's a rough suggestion for an API:
write_variants(
# required:
test_files => [
't/foo.t', ...
],
variants => {
wibble => { ... },
wobble => { ... },
},
# optional:
filter => sub {
my ($test_file, $variant_set) = @_;
delete $variant_set->{wibble} if ...;
return [ %$variant_set ]
}
);
The write_variants sub would call the filter callback for every combination
of variants (in this example: wibble alone, wobble alone, wibble + wobble).
The filter can return zero or more array refs where each array contains
the variants to be applied, in order. Returning nothing means don't
create variants for that test. Returning multiple items makes it easy
for the filter to generate extra permutations if appropriate for some tests.
What's missing?
Tim.
