Re: Unit vs. use case/functional testing
On 3/1/07 1:05 PM, "Eric Wilhelm" <[EMAIL PROTECTED]> wrote: >> although it's taken care of by a module >> that we can assume is fully tested (details will be forthcoming at >> some point, methinks). > > I would like to see that. Please keep us posted. Will do, it'll be on CPAN, although I'm not the author. Andrew -- Andrew Gianni - Lead Programmer Analyst University at Buffalo, State University of New York Computing and Information Technology / Administrative Computing Services 215 MFAC, Ellicott Complex, Buffalo, NY 14261-0026 716.645.3587x7124 - AIM: andrewsgianni - http://schoolof.info/agianni
Re: Unit vs. use case/functional testing
On 3/1/07 1:42 PM, "chromatic" <[EMAIL PROTECTED]> wrote: > On Thursday 01 March 2007 10:36, Andy Armstrong wrote: > >> If you get it right it'll also be a great help to the people writing >> the rules. You can give them a tool which allows them to start with >> an assertion and work backwards to a rule that implements it. They'll >> be able to do their own testing on the rule outside of the live >> application. > > That sounds much like FIT. Does Test::Fit look helpful? That looks like roughly what I was looking for. I'm thinking we might be able to write a general fixture sub-class for the business rule framework and all of our actual test fixtures could base class off of that, limiting further the amount of work an individual test writer would need to do. I *definitely* like the idea that it would potentially give us a tool to use with the customer when specing out the rules for an application. Even so, I would have to figure out a way to organize my fixtures effectively. There are 139 fields in the current application and while they're not all relevant to each test case, we'll have to keep track of the dependencies between them all when putting together the individual test cases. Andrew -- Andrew Gianni - Lead Programmer Analyst University at Buffalo, State University of New York Computing and Information Technology / Administrative Computing Services 215 MFAC, Ellicott Complex, Buffalo, NY 14261-0026 716.645.3587x7124 - AIM: andrewsgianni - http://schoolof.info/agianni
Re: Unit vs. use case/functional testing
On 3/1/07 12:18 PM, "Andy Armstrong" <[EMAIL PROTECTED]> wrote: > To test the rules you could maybe extend the rule notation to allow > assertions to be expressed as part of the rule set and write a test > harness that uses your (now proved) rules engine to test the assertions. That's sort of what I was thinking. There's no point in going through the CGI interface to test the rules, or even the application framework interface, minus CGI. The rules engine is general enough that we can probably write a relatively straight forward interface that allows us to pass data directly to the engine and simply ensure that the results are what we expect. Is that basically what you were suggesting? Andrew -- Andrew Gianni - Lead Programmer Analyst University at Buffalo, State University of New York Computing and Information Technology / Administrative Computing Services 215 MFAC, Ellicott Complex, Buffalo, NY 14261-0026 716.645.3587x7124 - AIM: andrewsgianni - http://schoolof.info/agianni
Unit vs. use case/functional testing
We're starting to write some rather complex business applications that
include a lot of business rules and I've come up against a bit of a
conundrum: how to best test things that aren't unit-testable. Let me
explain.
We've gotten rather efficient at writing unit tests, using Devel::Cover
(understanding its shortcomings) to track our progress, although we try our
best to write our tests along side our code. When we were implementing
relatively simple business rules, a la:
if ( $some_condition ){
&require_this_field();
}
else{
&require_that_field();
}
...we can basically cover that functionality with a unit test that goes
through and tests both branches effectively. However, our business rules
have gotten complicated enough that we are no longer writing them that way
explicitly in the code. In the last application we built, we put the rules
in a database and the appropriate ones were pulled based on circumstances
(using generalized code) and run. Now we're embarking on something different
that allows us to essentially write our business rules declaratively,
although it's taken care of by a module that we can assume is fully tested
(details will be forthcoming at some point, methinks).
In this situation, we would still like something in place to ensure that
altering the construction of the business rules doesn't cause regression in
the application, but we can't (or I'd certainly rather not) simply write
unit tests for. It's also not something that really needs to be done by the
developers per se, as they can essentially black box the code, testing for
output based on input. I'm thinking that it's about time we started
formalizing having staff with QA duties who write functional tests against
the code.
Does this sound sensible? If so, what tools could you recommend? Should we
still just use Test::More but in a more simple manner? Should we write mech
tests instead (these are web apps)? Or are there other tools that would be
useful? It seems to me that it would be rather laborious to write all of
these tests by hand.
Any insight appreciated. Recommendations on good books on general testing
philosophy would also be helpful (I've already got the developer's
notebook).
Andrew
--
Andrew Gianni - Lead Programmer Analyst
University at Buffalo, State University of New York
Computing and Information Technology / Administrative Computing Services
215 MFAC, Ellicott Complex, Buffalo, NY 14261-0026
716.645.3587x7124 - AIM: andrewsgianni - http://schoolof.info/agianni
Re: Running individual unit tests with Test::Class???
Cool, it's manageable for now and our current testing -- using sometimes large, monolithic Test::More .t files -- is no more flexible so there's no loss in the move, but the flexibility will be appreciated when you get around to it. Thanks for putting Test::Class together Adrian! Andrew On 5/31/06 10:55 AM, "Adrian Howard" <[EMAIL PROTECTED]> wrote: > [apologies to andrew for a dupe - didn't notice it went to perl-qa] > > On 31 May 2006, at 14:35, Andrew Gianni wrote: > >> Let me start by admitting that I don't know a whole lot about xUnit >> testing. >> In fact, using Test::Class is really my first exposure to the idea, so >> perhaps I'm asking for something that doesn't make sense; please >> bear with >> me in case I am. >> > [excellent description snipped] > > In short > * Yes running one test method at a time is a sensible things to do. > * No - there currently isn't a simple way of doing this > * Good news - Ovid has submitted a patch to make it easy > * Bad news - I've been too bone idle to apply it > > Hopefully I will become less lazy soon :-) Should be in the next > release, which is well overdue. > > Cheers, > > Adrian > > Andrew Gianni Administrative Computing Services Computing and Information Technology University at Buffalo 215 MFAC, Ellicott Complex Buffalo, NY 14261-0026 716.645.3587x7124
Re: Running individual unit tests with Test::Class???
Yeah, but it's so convenient to have my test classes reflect the classes of my application. It makes it so easy to see what's going on when I name my test methods the same thing as the methods in my application. Andrew On 5/31/06 10:47 AM, "A. Pagaltzis" <[EMAIL PROTECTED]> wrote: > Hi Andrew, > > * Andrew Gianni <[EMAIL PROTECTED]> [2006-05-31 15:40]: >> Is there an easy way to just run that one test method (as well >> as the startup, setup, teardown and shutdown methods) while I'm >> working on it rather than running the whole batch? > > just write a base test class with all your scaffolding methods > and then put the individual test units in individual subclasses > that inherit from. You can then easily run just the one set of > tests that you are interested in. > > Regards, Andrew Gianni Administrative Computing Services Computing and Information Technology University at Buffalo 215 MFAC, Ellicott Complex Buffalo, NY 14261-0026 716.645.3587x7124
Running individual unit tests with Test::Class???
Let me start by admitting that I don't know a whole lot about xUnit testing. In fact, using Test::Class is really my first exposure to the idea, so perhaps I'm asking for something that doesn't make sense; please bear with me in case I am. Test::Class seems like a great idea for so many reasons, but the one thing that seems like it ought to be easy to do, but I can't figure out how to accomplish, is running one unit test (or set of unit tests) by just running one test method. For example, say I have a test class with fifteen test methods, but I really only care about one of them right now, they're atomic after all. Is there an easy way to just run that one test method (as well as the startup, setup, teardown and shutdown methods) while I'm working on it rather than running the whole batch? I can vaguely see how I could do it manually by writing a script to do it for me, but is there some other, easier way to do this that's built into the module? Andrew Andrew Gianni Administrative Computing Services Computing and Information Technology University at Buffalo 215 MFAC, Ellicott Complex Buffalo, NY 14261-0026 716.645.3587x7124
Re: Is_deeply and closure-driven coderefs
Thanks, I'm going to give D::D::S a try, as I think it'll do what I'm looking for. Since I'm testing a subroutine that returns a data-structure (that happens to contain code-refs), I really don't care about the return values of the code-refs because those are tested elsewhere (through testing the closure subroutines) I just want to make sure that the structures are identical. Thanks much for the suggestions! Andrew On 5/31/06 4:48 AM, "demerphq" <[EMAIL PROTECTED]> wrote: > On 5/30/06, chromatic <[EMAIL PROTECTED]> wrote: >> On Tuesday 30 May 2006 12:08, Nicholas Perez wrote: >> >>> Why not compare signatures? Is that not feasible? >> >> Which signatures? Is it important that the code comes from the same place >> (check the CV properties) or that the code has bound to the same lexicals >> (PadWalker) or that the lexicals are in the same state (more PadWalker)? > > And as Schwern said, if you are going to do all that stuff you might > as well use DDS to do it, as its much easier, (which is hardly > surprising as the Padwalker closure code was specifically written for > DDS'es requirements) > > Cheers, > Yve Andrew Gianni Administrative Computing Services Computing and Information Technology University at Buffalo 215 MFAC, Ellicott Complex Buffalo, NY 14261-0026 716.645.3587x7124
Is_deeply and closure-driven coderefs
I have a subroutine that populates a hash of arrays with coderefs by calling
closures. I'm trying to call Test::More::is_deeply to compare two structures
that should be identical and I'm running into trouble. When none of the
closures take arguments, everything is fine, but when one of the closure
subroutines takes an argument to generate the code, is_deeply fails. I put
together a simple test case:
sub closure_no_args{
return sub{
return 1;
}
}
sub closure_args{
my $arg = shift;
return sub{
return $arg;
}
}
my $ds1 = {
one => &closure_no_args,
two => &closure_args(15),
};
my $ds2 = {
one => &closure_no_args,
two => &closure_args(15),
};
is_deeply( $ds1, $ds2, "they're the same" );
yields:
not ok 1 - they're the same
# Failed test 'they're the same'
# in test.t at line 29.
# Structures begin differing at:
# $got->{two} = CODE(0x13ef94)
# $expected->{two} = CODE(0x152c28)
1..1
# Looks like you failed 1 test of 1.
I suppose that the important thing is not the content of the coderefs, but
that their return values are equal, but I'm not sure what would really make
a valid test in this case or what the best means of comparing those would
be. Any assistance would be appreciated.
Andrew
Andrew Gianni
Administrative Computing Services
Computing and Information Technology
University at Buffalo
215 MFAC, Ellicott Complex
Buffalo, NY 14261-0026
716.645.3587x7124
Use case testing of Web apps with Perl?
I'm exploring my options for use case testing of Web apps in Perl. Does any have any experience, recommendations or resources to suggest on the topic? We're working on some pretty complicated Web apps (written in Perl) and while we have our unit testing well under control, our development of mech tests leaves a great deal to be desired. We'd like to be a bit more programmatic about writing our mech tests to test use-case driven test-cases. I'm wondering if there are any tools or ideas out there to ease the process so we don't have to manually write the numerous mech tests individually or develop our own framework for this. Any recommendations are appreciated. Andrew Andrew Gianni Administrative Computing Services Computing and Information Technology University at Buffalo 215 MFAC, Ellicott Complex Buffalo, NY 14261-0026 716.645.3587x7124
Warnings from Devel::Cover/Test::Harness
I'm trying to capture coverage data for a large project I'm working on. I'm
running all of my unit tests through Test::Harness and I'm trying to utilize
Devel::Cover within the harness. Honestly this is mostly a nitpicky thing
because I can get Devel::Cover to work, but I'm seeing warnings and I want
to make sure I'm not loosing anything because of them.
My harness looks roughly like this:
use Test::Harness;
$Test::Harness::switches = '-MDevel::Cover';
my @tests_to_run = qw(
test1.t
test2.t
test3.t
...
);
runtests(@tests_to_run);
...about as simple as it gets. When I run this way I get warnings like this:
Devel::Cover: Can't find file "blib/lib/Net/SSLeay.pm": ignored.
Devel::Cover: Can't find file "blib/lib/Storable.pm": ignored.
Devel::Cover: Can't find file "../../lib/POSIX.pm": ignored.
Interestingly enough if I change the $Test::Harness::switches assignment to
this:
$ENV{HARNESS_PERL_SWITCHES} = '-MDevel::Cover';
I no longer see the warnings. This seems like a kludge, but I also don't
want to deal with the warnings. Any thoughts as to what's going on?
Note that I do use Storable and POSIX in places, but I don't use
Net::SSLeay, at least not directly.
Andrew
Andrew Gianni
Administrative Computing Services
Computing and Information Technology
University at Buffalo
215 MFAC, Ellicott Complex
Buffalo, NY 14261-0026
716.645.3587x7124
