Re: IETF - Let's Get Started
Hi everybody, 2008/10/8 Ovid [EMAIL PROTECTED]: We understand what a lot of basic issues are, so let's get some traction on this. As the next thing we need is a group charter (if we already have one, and I just haven't noticed, ignore this e-mail), I've copied the charter format from RFC 2418 to make a skeleton charter at testanything.org: http://testanything.org/wiki/index.php/IETF_WG_Proposed_Charter Two small and one big question: 1. Who is/are going to be the chair(s)? 2. Which area of IETF are we under? You can see the list of areas at http://www.ietf.org/html.charters/wg-dir.html. 3. We need to write a description of the problem being solved, and the objectives of this WG. I'll try coming up with a description tonight or tomorrow, but please feel free to beat me to it. cheers, Gaurav
Re: Generic test database
--- On Wed, 8/10/08, Philippe Bruhat (BooK) [EMAIL PROTECTED] wrote: Is there any way for a test suite to use the preferred database of a tester? Something like a TEST_DATABASE_DSN, TEST_DATABASE_USERNAME, TEST_DATABASE_PASSWORD triplet that CPAN testers could set to enable testing on different databases? A colleague of mine is working on a very interesting module, but even though he could release a test suite relying on sqlite, I thought it would be better to use any configured database that was available on the tester's system. Is there a recommended way to do that? I think I understand what you're trying to do, but be warned that this can be very problematic. If you run your tests against a database different from the deployment database, you're taking a huge risk as different databases behave differently. I've learned this the hard way while trying to use SQLite for tests even though the code was targeting MySQL. So long as the user understands why this is, though, I don't see a problem with this (and there are certainly advantages). Cheers, Ovid -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Tech blog- http://use.perl.org/~Ovid/journal/ Twitter - http://twitter.com/OvidPerl Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
Re: Generic test database
On Wed, Oct 08, 2008 at 05:34:52AM -0700, Ovid wrote: A colleague of mine is working on a very interesting module, but even though he could release a test suite relying on sqlite, I thought it would be better to use any configured database that was available on the tester's system. Is there a recommended way to do that? I think I understand what you're trying to do, but be warned that this can be very problematic. If you run your tests against a database different from the deployment database, you're taking a huge risk as different databases behave differently. I've learned this the hard way while trying to use SQLite for tests even though the code was targeting MySQL. Actually, I'm not trying to test a module/application that is supposed to use a generic subset of SQL assumed to work everywhere. It's more intended to support modules that explicitely work on several database engines (maybe thanks to phrasebooks or ORMs or anything) and want to run their test suite on any available database (assuming an empty database at the beginning of the first test), or even several of them if they are available. -- Philippe Bruhat (BooK) The right answer is worthless with the wrong question! (Moral from Groo The Wanderer #88 (Epic))
Re: Generic test database
On Wed, Oct 08, 2008 at 10:56:27AM +0200, Philippe Bruhat (BooK) wrote: Is there any way for a test suite to use the preferred database of a tester? Something like a TEST_DATABASE_DSN, TEST_DATABASE_USERNAME, TEST_DATABASE_PASSWORD triplet that CPAN testers could set to enable testing on different databases? A colleague of mine is working on a very interesting module, but even though he could release a test suite relying on sqlite, I thought it would be better to use any configured database that was available on the tester's system. Is there a recommended way to do that? If you use a pre-existing database, you risk polluting your tests with any rubbish that's already in there - perhaps left behind by the previous module to be tested. So I recommend creating a new database for your tests, and clearing up afterwards. This is easy for SQLite - just use File::Temp: http://search.cpan.org/src/DCANTRELL/Class-DBI-ClassGenerator-1.01/t/sqlite_create_db.pl For MySQL it's a little more involved, but only a little bit: http://search.cpan.org/src/DCANTRELL/Class-DBI-ClassGenerator-1.01/t/mysql.t http://search.cpan.org/src/DCANTRELL/Class-DBI-ClassGenerator-1.01/t/mysql_create_db.pl I've not (yet) bothered with PG, Oracle etc. When I need them, I'll probably extract it out into a nice neat Test::something module with a bunch of database-specific back-ends. -- David Cantrell | Nth greatest programmer in the world NANOG makes me want to unplug everything and hide under the bed -- brian d foy
IETF - Let's Get Started
I'm a bit disappointed that after all of our effort to get the TAP IETF process started, we have let it languish. I'd like to restart this effort. We understand what a lot of basic issues are, so let's get some traction on this. Background links: Initial proposal discussion: http://testanything.org/wiki/index.php/Initial_IETF_proposal_discussion Notes on the IETF process: http://testanything.org/wiki/index.php/TAP_at_IETF:_Notes Sign up for the IETF mailing list: https://www.ietf.org/mailman/listinfo/tap TAP: http://en.wikipedia.org/wiki/Test_Anything_Protocol Cheers, Ovid -- Buy the book - http://www.oreilly.com/catalog/perlhks/ Tech blog- http://use.perl.org/~Ovid/journal/ Twitter - http://twitter.com/OvidPerl Official Perl 6 Wiki - http://www.perlfoundation.org/perl6
Re: Generic test database
On Wed, Oct 08, 2008 at 01:17:16PM +0100, David Cantrell wrote:
On Wed, Oct 08, 2008 at 10:56:27AM +0200, Philippe Bruhat (BooK) wrote:
Is there any way for a test suite to use the preferred database of a
tester? Something like a TEST_DATABASE_DSN, TEST_DATABASE_USERNAME,
TEST_DATABASE_PASSWORD triplet that CPAN testers could set to enable
testing on different databases?
A colleague of mine is working on a very interesting module, but even
though he could release a test suite relying on sqlite, I thought it
would be better to use any configured database that was available on the
tester's system.
Is there a recommended way to do that?
If you use a pre-existing database, you risk polluting your tests with
any rubbish that's already in there - perhaps left behind by the
previous module to be tested. So I recommend creating a new database
for your tests, and clearing up afterwards.
The idea was that the test database would exist, and being a test
database would accept being wiped out by any script that would connect
to it.
This is easy for SQLite - just use File::Temp:
http://search.cpan.org/src/DCANTRELL/Class-DBI-ClassGenerator-1.01/t/sqlite_create_db.pl
For MySQL it's a little more involved, but only a little bit:
http://search.cpan.org/src/DCANTRELL/Class-DBI-ClassGenerator-1.01/t/mysql.t
http://search.cpan.org/src/DCANTRELL/Class-DBI-ClassGenerator-1.01/t/mysql_create_db.pl
I've not (yet) bothered with PG, Oracle etc. When I need them, I'll
probably extract it out into a nice neat Test::something module with a
bunch of database-specific back-ends.
The main issue we have with databases other than sqlite are always
the same: it needs to be configured, and the test script needs some
credentials. Connecting as 'root' on 'localhost' will of course not work
everywhere.
It should be fairly easy for willing CPAN testers to setup any database
they like, and provide some connection information for throwaway tables
and data (assuming the test script WILL probably drop all tables in
there and dump its own crap there).
Maybe a module with a configuration file à la CPAN.pm would be
sufficient. Something like:
use Test::Database;# not a good name
my (@handles) = Test::Database-test_handles();
my ($mysql_handle) = Test::Database-test_handles('mysql');
for my $handle (@handles) {
diag Testing with . $handle-driver();
my $dbh = $handle-dbh();# automatically connects us
# also available:
# - username()
# - password()
# - dsn()
# test away!
}
For simple stuff like SQLite or the DBD::CSV driver, the module would
create the appropriate file in a temporary directory/file, but it should
be possible to require a semi-permanent one (e.g. by having a setup
method that would create the needed stuff if missing and just return
the appropriate handler object if the stuff is already configured and
available).
After seeing your code, I suppose it could also be somewhat easy to try
out a few series of basic/default credentials on localhost for engines
like MySQL and Postgres, and try to setup a test database from there.
Does that sound like an interesting tool to have?
--
Philippe Bruhat (BooK)
The faster the climb, the swifter the fall.
(Moral from Groo The Wanderer #87 (Epic))
Generic test database
Hi, Is there any way for a test suite to use the preferred database of a tester? Something like a TEST_DATABASE_DSN, TEST_DATABASE_USERNAME, TEST_DATABASE_PASSWORD triplet that CPAN testers could set to enable testing on different databases? A colleague of mine is working on a very interesting module, but even though he could release a test suite relying on sqlite, I thought it would be better to use any configured database that was available on the tester's system. Is there a recommended way to do that? -- Philippe Bruhat (BooK) The greatest enemy is the one you do not know. (Moral from Groo #8 (Image))
Re: Generic test database
It should be fairly easy for willing CPAN testers to setup any database they like, and provide some connection information for throwaway tables and data (assuming the test script WILL probably drop all tables in there and dump its own crap there). This seems of dubious usefulness, as the intersection between the number of CPAN testers that would bother to set this up, and the number of modules that would make use of it, would be very small. After seeing your code, I suppose it could also be somewhat easy to try out a few series of basic/default credentials on localhost for engines like MySQL and Postgres, and try to setup a test database from there. Does that sound like an interesting tool to have? That sounds more interesting. The DBD::Pg strategy is to try and use DBI_DSN if is defined. Even when defined, it often fails due to permission issues, so we fall back to creating a brand new database cluster from scratch, which ensures that we can control all the parameters, paths, and permissions ourselves. Feel free to crib from the code in the link below. If I was doing it all over again, I might even jump straight to clean creation (via initdb) first. http://git.postgresql.org/?p=dbdpg.git;a=blob;f=trunk/t/dbdpg_test_setup.pl -- Greg Sabino Mullane [EMAIL PROTECTED] End Point Corporation signature.asc Description: PGP signature
Re: Generic test database
On Wed, Oct 08, 2008 at 10:17:31AM -0400, Greg Sabino Mullane wrote: It should be fairly easy for willing CPAN testers to setup any database they like, and provide some connection information for throwaway tables and data (assuming the test script WILL probably drop all tables in there and dump its own crap there). This seems of dubious usefulness, as the intersection between the number of CPAN testers that would bother to set this up, and the number of modules that would make use of it, would be very small. As far as I know, nothing exists yet for this kind of generic testing. Everyone rolls their own. DBIx::Class for example, relies on three environment variables per database engine, which probably means that most of the database-specific tests are not run because, just like you, I doubt CPANT testers bother to set those variables either. On the other hand, this list is the perfect place to reach CPAN testers if I can come up with a decent implementation for such a scheme. ;-) Being able to automagically setup a database by creating the appropriate temporary files (for sqlite) or using the default credentials (for MySQL and others) or even creating the whole database cluster (as you explained for Pg), and failing that, using a configuration file setup by the tester (only once, for all modules that will make us of this), seems interesting in the long term. Especially if CPAN testers are ready to set this up. If the discussion here can help me come up with a decent interface and help me avoid the obvious problems (I'm no database expert), then I think we can have a nice addition to the Perl test framework. After seeing your code, I suppose it could also be somewhat easy to try out a few series of basic/default credentials on localhost for engines like MySQL and Postgres, and try to setup a test database from there. Does that sound like an interesting tool to have? That sounds more interesting. Thanks for the encouragements (and criticism). :-) The DBD::Pg strategy is to try and use DBI_DSN if is defined. Aha! So DBI_DSN is an environment variable I can look at when trying to find a configured database. Even when defined, it often fails due to permission issues, so we fall back to creating a brand new database cluster from scratch, which ensures that we can control all the parameters, paths, and permissions ourselves. Feel free to crib from the code in the link below. If I was doing it all over again, I might even jump straight to clean creation (via initdb) first. http://git.postgresql.org/?p=dbdpg.git;a=blob;f=trunk/t/dbdpg_test_setup.pl Thanks for the link. Expect me to bother you with questions when I reach the point of trying to automagically setup a Pg database. :-) I'll try to come up with something for creating test databases on demand. Having some configuration module is a nice fallback for people who secured their database setup or want to have more control on what they provide. -- Philippe Bruhat (BooK) The best thing about being apart is getting together again. (Moral from Groo The Wanderer #39 (Epic))
Re: Generic test database
Greg Sabino Mullane wrote: It should be fairly easy for willing CPAN testers to setup any database they like, and provide some connection information for throwaway tables and data (assuming the test script WILL probably drop all tables in there and dump its own crap there). This seems of dubious usefulness, as the intersection between the number of CPAN testers that would bother to set this up, and the number of modules that would make use of it, would be very small. I think it's very useful. There's plenty of modules which need a database, and they all have to be configured differently and they're always a PITA when you first install and each and every time they upgrade. User setup can be dealt with by making Test::Database a build dependency. As part of Test::Database's install process it walks the user through the configuration process. Once it's done, it writes out a config file and then it's done for good. -- emacs -- THAT'S NO EDITOR... IT'S AN OPERATING SYSTEM!
