Comparing Data Structures Slopply
So...
In a previous thread I was talking about how eq_set() should be really
called eq_bag(). Now, not that I've got too much on my plate already (see
my fun todo) but I do have an itch to scratch...
I'm fed up applying too much logic in my test suites. This is bad
because:
a) I spend too much time writing tests
b) There's a high chance I'll make a mistake in my test suite.
The kind of problems I'm facing are that I want something like this to
match:
wanted got
[[
{ a => "foo", b => "bar" }, { a => "baz", b => "qux" },
{ a => "baz", b => "qux" }, { a => "foo", b => "bar" },
]]
There's a lot of other problems like that. So I was thinking of writing
Test::Sloppy (aka Test::Fuzzy, aka...)
Quick qustions before I start
a) Is this a good idea?
b) Is there something else that already does this?
c) Should I (those modules authors willing) attempt to merge it into
other modules?
Later.
Mark.
--
s'' Mark Fowler London.pm Bath.pm
http://www.twoshortplanks.com/ [EMAIL PROTECTED]
';use Term'Cap;$t=Tgetent Term'Cap{};print$t->Tputs(cl);for$w(split/ +/
){for(0..30){$|=print$t->Tgoto(cm,$_,$y)." $w";select$k,$k,$k,.03}$y+=2}
Re: Comparing Data Structures Slopply
On Tue, Apr 09, 2002 at 03:26:21PM +0100, Mark Fowler wrote: > There's a lot of other problems like that. So I was thinking of writing > Test::Sloppy (aka Test::Fuzzy, aka...) What would it do? (I can show you lots of sloppy tests if you like. :) -- Michael G. Schwern <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/ Perl Quality Assurance <[EMAIL PROTECTED]> Kwalitee Is Job One Here's some scholarly-ass opinions...
Re: Comparing Data Structures Slopply
On Tue, 9 Apr 2002, Michael G Schwern wrote:
> On Tue, Apr 09, 2002 at 03:26:21PM +0100, Mark Fowler wrote:
> > There's a lot of other problems like that. So I was thinking of writing
> > Test::Sloppy (aka Test::Fuzzy, aka...)
>
> What would it do?
>
> (I can show you lots of sloppy tests if you like. :)
Sorry, maybe it wasn't clear from the example. sort of like eq_set meets
is_deeply.
Let's drag the example out again:
wanted got
[[
{ a => "foo", b => "bar" }, { a => "baz", b => "qux" },
{ a => "baz", b => "qux" }, { a => "foo", b => "bar" },
]]
This will of course fail with eq_set which just uses eq to compare items
(which won't work for hashes), so although it'll cope happly with the fact
that the elements aren't in the same order, it'll still fail.
(I got around this in my actual tests I just wrote by freezing the hashes
to flatten them then comparing both lists with eq_set)
Sloppy/Fuzzy would attempt to do away with such problems by sorting all
lists down to a defined level first. Off the top of my head
list_cmp_fuzzy($list1, $list2, 3)
would ignore all lists down to 3 levels.
Added other functionality to be introduced at a later date:
a) Treating lists as hashes (i.e. they can be reordered at will but
elements must be paired)
b) Doing proper set comparison (not bags)
c) Better handling of overloaded objects
And more ideas, if I can think of them. Or more importantly, if you can.
Later.
Mark.
--
s'' Mark Fowler London.pm Bath.pm
http://www.twoshortplanks.com/ [EMAIL PROTECTED]
';use Term'Cap;$t=Tgetent Term'Cap{};print$t->Tputs(cl);for$w(split/ +/
){for(0..30){$|=print$t->Tgoto(cm,$_,$y)." $w";select$k,$k,$k,.03}$y+=2}
Re: Comparing Data Structures Slopply
On Tue 09 Apr 2002 18:02, Mark Fowler <[EMAIL PROTECTED]> wrote:
> On Tue, 9 Apr 2002, Michael G Schwern wrote:
>
> > On Tue, Apr 09, 2002 at 03:26:21PM +0100, Mark Fowler wrote:
> > > There's a lot of other problems like that. So I was thinking of writing
> > > Test::Sloppy (aka Test::Fuzzy, aka...)
> >
> > What would it do?
> >
> > (I can show you lots of sloppy tests if you like. :)
Don't *overtest*.
If you go backporting this into core tests, think not only of the ease of new
methods of comparing, but also about the time it takes. Smoke testing has
proved to be vital for 5.8.0, and longer test runs (without added value)
reduce the number of configurations to be tested in the given time slot.
> Sorry, maybe it wasn't clear from the example. sort of like eq_set meets
> is_deeply.
>
> Let's drag the example out again:
>
> wanted got
>
> [[
>{ a => "foo", b => "bar" }, { a => "baz", b => "qux" },
>{ a => "baz", b => "qux" }, { a => "foo", b => "bar" },
> ]]
>
> This will of course fail with eq_set which just uses eq to compare items
> (which won't work for hashes), so although it'll cope happly with the fact
> that the elements aren't in the same order, it'll still fail.
>
> (I got around this in my actual tests I just wrote by freezing the hashes
> to flatten them then comparing both lists with eq_set)
>
> Sloppy/Fuzzy would attempt to do away with such problems by sorting all
> lists down to a defined level first. Off the top of my head
>
>list_cmp_fuzzy($list1, $list2, 3)
>
> would ignore all lists down to 3 levels.
>
> Added other functionality to be introduced at a later date:
>
> a) Treating lists as hashes (i.e. they can be reordered at will but
> elements must be paired)
> b) Doing proper set comparison (not bags)
> c) Better handling of overloaded objects
>
> And more ideas, if I can think of them. Or more importantly, if you can.
>
> Later.
>
> Mark.
>
> --
> s'' Mark Fowler London.pm Bath.pm
> http://www.twoshortplanks.com/ [EMAIL PROTECTED]
> ';use Term'Cap;$t=Tgetent Term'Cap{};print$t->Tputs(cl);for$w(split/ +/
> ){for(0..30){$|=print$t->Tgoto(cm,$_,$y)." $w";select$k,$k,$k,.03}$y+=2}
>
--
H.Merijn BrandAmsterdam Perl Mongers (http://amsterdam.pm.org/)
using perl-5.6.1, 5.7.3 & 631 on HP-UX 10.20 & 11.00, AIX 4.2, AIX 4.3,
WinNT 4, Win2K pro & WinCE 2.11. Smoking perl CORE: [EMAIL PROTECTED]
http:[EMAIL PROTECTED]/ [EMAIL PROTECTED]
send smoke reports to: [EMAIL PROTECTED], QA: http://qa.perl.org
Re: Comparing Data Structures Slopply
On Tue, Apr 09, 2002 at 05:02:32PM +0100, Mark Fowler wrote: > On Tue, 9 Apr 2002, Michael G Schwern wrote: > > > On Tue, Apr 09, 2002 at 03:26:21PM +0100, Mark Fowler wrote: > > > There's a lot of other problems like that. So I was thinking of writing > > > Test::Sloppy (aka Test::Fuzzy, aka...) > > > > What would it do? > > > > (I can show you lots of sloppy tests if you like. :) > > Sorry, maybe it wasn't clear from the example. sort of like eq_set meets > is_deeply. I think "Sloppy" or "Fuzzy" is the wrong term for this. It implies you're doing some sort of approximate matching like String::Approx. Sounds like you just want better tools for testing multi-level data structures. -- Michael G. Schwern <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/ Perl Quality Assurance <[EMAIL PROTECTED]> Kwalitee Is Job One Don't ask me lady, I live in beer.
Test::Exception... comments?
Hi all,
I've been refactoring a bunch of old tests with Test::More and some
convenience routines for testing exceptions dropped out (along with some
class base testing and mock object modules which still need cleaning up into
something sane.)
dies_ok BLOCK TEST_NAME
Tests to see that BLOCK exits by dying, rather than by exiting
normally. For example:
dies_ok { div(1, 0) } 'divide by zero detected';
lives_ok BLOCK TEST_NAME
Tests to see that BLOCK exits normally, and doesn't die. For
example:
lives_ok { $file = read_file('test.txt') } 'file read';
throws_ok BLOCK REGEX, TEST_NAME
throws_ok BLOCK CLASS, TEST_NAME
Tests to see that BLOCK throws a specific exception.
In the first form the test passes if the stringified exception
matches the give regular expression. For example:
throws_ok {
read_file('test.txt')
} qr/No such file/, 'no file';
In the second form the test passes if the exception is of the same
class as the one supplied, or a subclass of that class. For example:
throws_ok {$foo->bar} "Error::Simple", 'simple error';
(you can find the code at
if you want to
play)
Any comments before I throw it at CPAN? Sound vaguely sane?
Cheers,
Adrian
Re: Test::Exception... comments?
On Tue, Apr 09, 2002 at 10:05:49PM +0100, Adrian Howard wrote:
> Hi all,
>
> I've been refactoring a bunch of old tests with Test::More and some
> convenience routines for testing exceptions dropped out (along with some
> class base testing and mock object modules which still need cleaning up into
> something sane.)
>
> dies_ok BLOCK TEST_NAME
> Tests to see that BLOCK exits by dying, rather than by exiting
> normally. For example:
>
> dies_ok { div(1, 0) } 'divide by zero detected';
You probably want to guarantee that $@ will be how it died so you can do:
dies_ok { div(1,0) } 'div by zero';
like( $@, qr/^Illegal division by zero/ );
Even though you can use throws_ok(), the dies_ok() + $@ combo is more
flexible more processing than just a regex needs to be done on $@.
--
Michael G. Schwern <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance <[EMAIL PROTECTED]> Kwalitee Is Job One
Death follows me like a wee followey thing.
-- Quakeman
Re: Test::Exception... comments?
On Tue, 9 Apr 2002, Adrian Howard wrote: > lives_ok BLOCK TEST_NAME > Tests to see that BLOCK exits normally, and doesn't die. I'm not sure exactly what the purpose of this is; your test will still fail if it dies even when not in a lives_ok block, right? Not that I have any real objection to this, but I can't see where I'd ever use it. The rest looks great, however. Ideally it should end up in Test::More at some point, I would think. cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC
Re: Comparing Data Structures Slopply
On Tue, 9 Apr 2002, Mark Fowler wrote: > On Tue, 9 Apr 2002, Michael G Schwern wrote: > > > What would it do? > > (I can show you lots of sloppy tests if you like. :) > > Sorry, maybe it wasn't clear from the example. sort of like eq_set meets > is_deeply. Well, I think he was just being a bit of a smartass, and it came out sounding a bit too serious on the list. (Assuming I'm interpreting this correctly, of course.) I quite agree with what I percieve to be his point, which is that the name sucks. However, the test itself is a great idea. Just the other day I found myself writing a method that, given an array, returns a reference to a sorted version of that array. This was just so I didn't have to worry about the order of things within a set-type object I was testing. > a) Treating lists as hashes (i.e. they can be reordered at will but > elements must be paired) Also an excellent idea. > b) Doing proper set comparison (not bags) This I'm not sure about. You demonstrated some set comparison along these lines a while back, did you not? set_equals(\["one", "two"], \["one", "two", "two"]); (Sorry, I don't recall the method name right off-hand now.) You pointed out that it fails, and that you felt it's actually doing a bag comparison. Well, my opinion of the matter is that it *ought* to fail, because that second parameter is not a set. Sets can't have multiple instances of a particular value in them. If I iterate over a set using the common idiom for the language and data structure, I should never get multiple instances of the same value back. I think the problem in this case is not that the test method you want doesn't exist, but that you want to be using some sort of Set object and you're trying to use an array instead. cjs -- Curt Sampson <[EMAIL PROTECTED]> +81 90 7737 2974 http://www.netbsd.org Don't you know, in this new Dark Age, we're all light. --XTC
Re: Test::Exception... comments?
On Wed, Apr 10, 2002 at 10:24:32AM +0900, Curt Sampson wrote:
> On Tue, 9 Apr 2002, Adrian Howard wrote:
>
> > lives_ok BLOCK TEST_NAME
> > Tests to see that BLOCK exits normally, and doesn't die.
>
> I'm not sure exactly what the purpose of this is; your test will
> still fail if it dies even when not in a lives_ok block, right?
It'll fail and take the whole rest of the test program with it. Some
testing systems like to abort the test script on failure. Perl's
doesn't. Aegis, in particular, pretty much requires tests to always
exit gracefully, passing or not.
It's not uncommon to do this:
ok( eval { ...some code...; 1; } );
which would be really all that lives_ok() would be.
> Not that I have any real objection to this, but I can't see where
> I'd ever use it.
>
> The rest looks great, however. Ideally it should end up in Test::More
> at some point, I would think.
Nope. No more functions in Test::More, too much in there already.
I want to encourage the idea of having more than one Test::* module in
use in a test.
--
Michael G. Schwern <[EMAIL PROTECTED]>http://www.pobox.com/~schwern/
Perl Quality Assurance <[EMAIL PROTECTED]> Kwalitee Is Job One
HA HA HA You're all so ridiculous! But thanks for the money!
Re: Test::Exception... comments?
on 9/4/02 10:16 pm, Michael G Schwern at [EMAIL PROTECTED] wrote:
[snip]
> You probably want to guarantee that $@ will be how it died so you can do:
>
> dies_ok { div(1,0) } 'div by zero';
> like( $@, qr/^Illegal division by zero/ );
>
> Even though you can use throws_ok(), the dies_ok() + $@ combo is more
> flexible more processing than just a regex needs to be done on $@.
[snip]
Good idea. Done.
Fixed a couple of other silly mistakes. New code at
