Re: Generating test data and testing it

[Shlomi Fish]

Hi Ovid!

On 2/20/07, Ovid <[EMAIL PROTECTED]> wrote:

Here's a common enough problem that I assume someone else has solved
it.

Tools in question:

  Test::Class
  Test::WWW::Mechanize

We want to test creating users (pseudo-code, but close to what we
actually use):

  sub add_user : Tests(5) {
  my $test = shift;
  my $mech = $test->mech;
  $mech->get_ok( $add_user_page, 'get the "Add User" page' );
  $mech->submit_form(
form_name => 'add_user',
fields=> { name => $some_name }
  );
  $mech->content_like( qr/User added/ );
  $mech->back;
  $mech->submit_form(
form_name => 'add_user',
fields=> { name => $some_other_name }
  );
  $mech->content_like( qr/User added/ );
  $mech->get_ok( $user_page, 'get the "List User" page' );
  $mech->content_like( qr/2 users found/ );
  }

So that's all well and good, but another class has another method where
we to add accounts, but we must have a customer backing each account
successfully added.  We'd rather not simply duplicate the above logic
and since the code's already written, I had the (quite possibly stupid)
idea of rewriting the above like this:

  sub add_user : Tests(5) {
  my $test = shift;
  my $mech = $test->mech;

  my $override = override_if_fixture(
  test => $test,
  mech => $mech,
  overrides => {
  get_ok   => sub {
  my ( $mech, $page ) = @_;
  $mech->get($page);
  },
  content_like => sub {},
  }
  );


Maybe I'm missing something, but in your second example, you declare
$override as a lexical and don't use it anywhere. May I inquire what
was your real intention?

Regards,

   Shlomi Fish


  $mech->get_ok( $add_user_page, 'get the "Add User" page' );
  $mech->submit_form(
form_name => 'add_user',
fields=> { name => $some_name }
  );
  $mech->content_like( qr/User added/ );
  $mech->back;
  $mech->submit_form(
form_name => 'add_user',
fields=> { name => $some_other_name }
  );
  $mech->content_like( qr/User added/ );
  $mech->get_ok( $user_page, 'get the "List User" page' );
  $mech->content_like( qr/2 users found/ );
  }

The idea is, in this scope, we'd convert the 'get_ok' and
'content_like' methods to *not* be tests for the current scope.  This
allows us to use our tests as test fixtures rather than duplicate this
logic.

I think this is a horribly clumsy idea, but the above is a simplified
example and some test data could get hairy to setup and if we change
our Web pages, I'd hate to find all of the duplicated logic for setting
up this test data and testing that it actually works (note that we
could be testing a remote server and not always be in a position to
access the database directly).

Thoughts?

Cheers,
Ovid

--

Buy the book -- http://www.oreilly.com/catalog/perlhks/
Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/




--
--
Shlomi Fish http://www.shlomifish.org/

If his programming is anything like his philosophising, he
would find 10 imaginary bugs in the "Hello World" program.

Re: Generating test data and testing it

[Fergal Daly]

Test::Tester allows you to temporarily divert test results to another
test result collector (which you can then throw away). It diverts the
results at the method call level so it's like the diverted tests never
ran (it does NOT capture STDOUT, Test::Tester sticks to the documented
interface, rather than hoping the Test::Builder never changes it's
output).

There's no obligation on you to actually use the captured test
results. The interface isn't set up for doing what you're talking
about but the following demonstrates how to do it and it could be
hidden away behind a function. In the code below, the test subroutine
is called 3 times but only 2 of them produce test resuls, the middle
call produces no output and has no effect on the test numbering etc.

use strict;
use warnings;

use Test::Tester qw(no_plan run_tests);
use Test::More;

my $delegator = Test::Tester->new_new();
my $capture = Test::Tester->capture();

my $i = 1;
sub some_tests
{
 my $really_test = shift;
 local $delegator->{Object} = $really_test ? $delegator->{Object} : $capture;

 # depending on $really_test these tests may or may not be logged
 pass("pass $i");
 $i++;
}

some_tests(1);
some_tests(0);
some_tests(1);

F


On 20/02/07, Ovid <[EMAIL PROTECTED]> wrote:

Here's a common enough problem that I assume someone else has solved
it.

Tools in question:

  Test::Class
  Test::WWW::Mechanize

We want to test creating users (pseudo-code, but close to what we
actually use):

  sub add_user : Tests(5) {
  my $test = shift;
  my $mech = $test->mech;
  $mech->get_ok( $add_user_page, 'get the "Add User" page' );
  $mech->submit_form(
form_name => 'add_user',
fields=> { name => $some_name }
  );
  $mech->content_like( qr/User added/ );
  $mech->back;
  $mech->submit_form(
form_name => 'add_user',
fields=> { name => $some_other_name }
  );
  $mech->content_like( qr/User added/ );
  $mech->get_ok( $user_page, 'get the "List User" page' );
  $mech->content_like( qr/2 users found/ );
  }

So that's all well and good, but another class has another method where
we to add accounts, but we must have a customer backing each account
successfully added.  We'd rather not simply duplicate the above logic
and since the code's already written, I had the (quite possibly stupid)
idea of rewriting the above like this:

  sub add_user : Tests(5) {
  my $test = shift;
  my $mech = $test->mech;

  my $override = override_if_fixture(
  test => $test,
  mech => $mech,
  overrides => {
  get_ok   => sub {
  my ( $mech, $page ) = @_;
  $mech->get($page);
  },
  content_like => sub {},
  }
  );
  $mech->get_ok( $add_user_page, 'get the "Add User" page' );
  $mech->submit_form(
form_name => 'add_user',
fields=> { name => $some_name }
  );
  $mech->content_like( qr/User added/ );
  $mech->back;
  $mech->submit_form(
form_name => 'add_user',
fields=> { name => $some_other_name }
  );
  $mech->content_like( qr/User added/ );
  $mech->get_ok( $user_page, 'get the "List User" page' );
  $mech->content_like( qr/2 users found/ );
  }

The idea is, in this scope, we'd convert the 'get_ok' and
'content_like' methods to *not* be tests for the current scope.  This
allows us to use our tests as test fixtures rather than duplicate this
logic.

I think this is a horribly clumsy idea, but the above is a simplified
example and some test data could get hairy to setup and if we change
our Web pages, I'd hate to find all of the duplicated logic for setting
up this test data and testing that it actually works (note that we
could be testing a remote server and not always be in a position to
access the database directly).

Thoughts?

Cheers,
Ovid

--

Buy the book -- http://www.oreilly.com/catalog/perlhks/
Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/

Re: Generating test data and testing it

[Jonathan Rockway]

Shlomi Fish wrote:
> Maybe I'm missing something, but in your second example, you declare
> $override as a lexical and don't use it anywhere. May I inquire what
> was your real intention?

I'm pretty sure he was trying to do this:

add_user.t
$testing = 1;
add_user(); # prints TAP

do_something_that_needs_a_user.t:
$testing = 0;
add_user(); # doesn't print TAP, but adds the user
ok({do_some_stuff});

The idea being that writing the add_user mech stuff twice is messy.

Suppressing the TAP seems unclean on the surface, but lacking access to
the database (so you can do a proper add_user to the data source), it
seems like a good idea to me.  The code is written exactly once, and
since it's its own test, you know it works!  (Self-testing code)++

-- 
package JAPH;use Catalyst qw/-Debug/;($;=JAPH)->config(name => do {
$,.=reverse qw[Jonathan tsu rehton lre rekca Rockway][$_].[split //,
";$;"]->[$_].q; ;for 1..4;$,=~s;^.;;;$,});$;->setup;

Re: Generating test data and testing it

[Ovid]

--- Shlomi Fish <[EMAIL PROTECTED]> wrote:
> >
> >   sub add_user : Tests(5) {
> >   my $test = shift;
> >   my $mech = $test->mech;
> >
> >   my $override = override_if_fixture(
> >   test => $test,
> >   mech => $mech,
> >   overrides => {
> >   get_ok   => sub {
> >   my ( $mech, $page ) = @_;
> >   $mech->get($page);
> >   },
> >   content_like => sub {},
> >   }
> >   );
> 
> Maybe I'm missing something, but in your second example, you declare
> $override as a lexical and don't use it anywhere. May I inquire what
> was your real intention?

It holds the overrides in scope so that when $override is DESTROYed, we
can roll back the changes to &get_ok and &content_like.

Cheers,
Ovid

--

Buy the book -- http://www.oreilly.com/catalog/perlhks/
Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/

Re: Generating test data and testing it

[Chris Dolan]

On Feb 20, 2007, at 9:12 AM, Ovid wrote:


Here's a common enough problem that I assume someone else has solved
it.


[snip]


The idea is, in this scope, we'd convert the 'get_ok' and
'content_like' methods to *not* be tests for the current scope.  This
allows us to use our tests as test fixtures rather than duplicate this
logic.

I think this is a horribly clumsy idea, but the above is a simplified
example and some test data could get hairy to setup and if we change
our Web pages, I'd hate to find all of the duplicated logic for  
setting

up this test data and testing that it actually works (note that we
could be testing a remote server and not always be in a position to
access the database directly).

Thoughts?

Cheers,
Ovid


Clever and scary.  The following is a much more straightforward  
alternative, but heavier weight:


  sub add_user : Tests(5) {
  my $test = shift;
  my $mech = $test->mech;
  $mech->get( $add_user_page );
  ok($mech->success, 'get the "Add User" page') if $testing;
  ...
  }

That is, unwrap the Test::WWW::Mech stuff for this case.  That's  
duplicating some T::W::M code, but it won't make anybody's head  
explode trying to read the test code a couple years down the road.


Chris


--
Chris Dolan, Equilibrious LLC, http://equilibrious.net/
Public key: http://chrisdolan.net/public.key
vCard: http://chrisdolan.net/ChrisDolan.vcf

Re: Generating test data and testing it

[Fergal Daly]

Here's an improved interface that lets you do

maybe_test {
 your($code);
 goes($here};
} $testing_on_or_off;

--
use strict;
use warnings;

use Test::Tester;
use Test::More 'no_plan';

my $delegator = Test::Tester->new_new();
my $capture = Test::Tester->capture();

sub maybe_test(&$) {
 my $sub = shift;
 my $really_test = shift;
 local $delegator->{Object} = $really_test ? $delegator->{Object} : $capture;
 &$sub()
}

# example usage

my $i = 1;
sub some_tests
{
 my $testing = shift;
 maybe_test {
   # depending on $really_test these tests may or may not be logged
   print $testing ? "testing $i\n" : "not testing $i\n";
   pass("pass");
   $i++;
 } $testing;
}

some_tests(1);
some_tests(0);
some_tests(1);


On 20/02/07, Fergal Daly <[EMAIL PROTECTED]> wrote:

Test::Tester allows you to temporarily divert test results to another
test result collector (which you can then throw away). It diverts the
results at the method call level so it's like the diverted tests never
ran (it does NOT capture STDOUT, Test::Tester sticks to the documented
interface, rather than hoping the Test::Builder never changes it's
output).

There's no obligation on you to actually use the captured test
results. The interface isn't set up for doing what you're talking
about but the following demonstrates how to do it and it could be
hidden away behind a function. In the code below, the test subroutine
is called 3 times but only 2 of them produce test resuls, the middle
call produces no output and has no effect on the test numbering etc.

use strict;
use warnings;

use Test::Tester qw(no_plan run_tests);
use Test::More;

my $delegator = Test::Tester->new_new();
my $capture = Test::Tester->capture();

my $i = 1;
sub some_tests
{
  my $really_test = shift;
  local $delegator->{Object} = $really_test ? $delegator->{Object} : $capture;

  # depending on $really_test these tests may or may not be logged
  pass("pass $i");
  $i++;
}

some_tests(1);
some_tests(0);
some_tests(1);

F


On 20/02/07, Ovid <[EMAIL PROTECTED]> wrote:
> Here's a common enough problem that I assume someone else has solved
> it.
>
> Tools in question:
>
>   Test::Class
>   Test::WWW::Mechanize
>
> We want to test creating users (pseudo-code, but close to what we
> actually use):
>
>   sub add_user : Tests(5) {
>   my $test = shift;
>   my $mech = $test->mech;
>   $mech->get_ok( $add_user_page, 'get the "Add User" page' );
>   $mech->submit_form(
> form_name => 'add_user',
> fields=> { name => $some_name }
>   );
>   $mech->content_like( qr/User added/ );
>   $mech->back;
>   $mech->submit_form(
> form_name => 'add_user',
> fields=> { name => $some_other_name }
>   );
>   $mech->content_like( qr/User added/ );
>   $mech->get_ok( $user_page, 'get the "List User" page' );
>   $mech->content_like( qr/2 users found/ );
>   }
>
> So that's all well and good, but another class has another method where
> we to add accounts, but we must have a customer backing each account
> successfully added.  We'd rather not simply duplicate the above logic
> and since the code's already written, I had the (quite possibly stupid)
> idea of rewriting the above like this:
>
>   sub add_user : Tests(5) {
>   my $test = shift;
>   my $mech = $test->mech;
>
>   my $override = override_if_fixture(
>   test => $test,
>   mech => $mech,
>   overrides => {
>   get_ok   => sub {
>   my ( $mech, $page ) = @_;
>   $mech->get($page);
>   },
>   content_like => sub {},
>   }
>   );
>   $mech->get_ok( $add_user_page, 'get the "Add User" page' );
>   $mech->submit_form(
> form_name => 'add_user',
> fields=> { name => $some_name }
>   );
>   $mech->content_like( qr/User added/ );
>   $mech->back;
>   $mech->submit_form(
> form_name => 'add_user',
> fields=> { name => $some_other_name }
>   );
>   $mech->content_like( qr/User added/ );
>   $mech->get_ok( $user_page, 'get the "List User" page' );
>   $mech->content_like( qr/2 users found/ );
>   }
>
> The idea is, in this scope, we'd convert the 'get_ok' and
> 'content_like' methods to *not* be tests for the current scope.  This
> allows us to use our tests as test fixtures rather than duplicate this
> logic.
>
> I think this is a horribly clumsy idea, but the above is a simplified
> example and some test data could get hairy to setup and if we change
> our Web pages, I'd hate to find all of the duplicated logic for setting
> up this test data and testing that it actually works (note that we
> could be testing a remote server and not always be in a position to
> access the database directly).
>
> Thoughts?
>
> Cheers,
> Ovid
>
> --
>
> Buy the book -- http://www.oreilly.com/catalog/perlhks/
> Perl and CGI -- http://users.easystreet.com/ovid/cgi_course/
>

Re: Generating test data and testing it

[Michael G Schwern]

Ovid wrote:
> The idea is, in this scope, we'd convert the 'get_ok' and
> 'content_like' methods to *not* be tests for the current scope.  This
> allows us to use our tests as test fixtures rather than duplicate this
> logic.
> 
> I think this is a horribly clumsy idea, but the above is a simplified
> example and some test data could get hairy to setup and if we change
> our Web pages, I'd hate to find all of the duplicated logic for setting
> up this test data and testing that it actually works (note that we
> could be testing a remote server and not always be in a position to
> access the database directly).

Ick.

I see where you're coming from but this idea sounds too much like...

sub foo {
my(%args) = @_;

if( $args{flag} ) {
...
}
else {
...
}
}

That is, you have one routine doing two different things depending on if a flag 
is passed in.  Meaning you really have two routines.  Your routine is doing too 
much work and its name is wrong.  Its not "add_user" but 
"add_two_users_and_list_them".  The fact that its a test doesn't change this.  
Break it down into smaller more flexible pieces.

  sub add_user : Tests(2) {
  my $test = shift;
  my $user_name = shift;

  my $mech = $test->mech;
  $mech->get_ok( $add_user_page, 'get the "Add User" page' );
  $mech->submit_form(
form_name => 'add_user',
fields=> { name => $user_name }
  );
  $mech->content_like( qr/User added/ );
  $mech->back;
  }

  sub check_num_users: Test(2) {
  my $test = shift;
  my $num_users = shift;

  $mech->get_ok( $user_page, 'get the "List User" page' );
  $mech->content_like( qr/$num_users users found/ );
  $mech->back;
  }

  $test->add_user("Joe Foo");
  $test->add_user("Jim Bar");
  $test->check_num_users(2);

(You might be even better served by checking that those two specific users 
actually exist rather than just checking the number of users.  If you do that 
you can tack that test onto the end of add_user().)

Now you have two handy testing utility routines where before you had one 
single-purpose routine.  Use as desired.

  sub add_account {
  my $test = shift;
  my $account_num = shift;

  $test->add_user("Account Test User");

  ...add the account...
  }

Finally, why turn off the tests?  Why not just let them run?  What's the harm?  
Worried about messing up the count?  Use no_plan, its preferable to special 
hackery.

Re: Generating test data and testing it

[A. Pagaltzis]

* Michael G Schwern <[EMAIL PROTECTED]> [2007-02-20 20:55]:
> Finally, why turn off the tests? Why not just let them run?

That was my first instinct also.

Regards,
-- 
Aristotle Pagaltzis //