David Golden wrote:
Adam Kennedy wrote:
What about a special environment variable, like RUN_PRIVATE_TESTS?
I've been working on a concept of taggable tests on some of my larger
commercial stuff, integrating with the Test::More skip() function, and
some form of environment variables does indeed seem the best way to do
such a thing.
Isn't Test::Manifest sort of geared to do this kind of thing? (Or could
be extended to do so?)
No. The manifest solution seems to work very well for specifying order,
but I wouldn't want to use it for finding testing subsets.
For starters, the tags need to be orthogonal. So if I use the tags
NEEDS_DNS and NEEDS_DATABASE and NEEDS_APACHE then you need to be able
to enable or disable the tags/flags individually.
Also, doing it at the manifest level means you can only deal with things
at a per-file level, which what is much more useful is to do it at a
sub-test level.
If you are testing something, you can have both it's normal tests and
just wrap the tests that need the database or DNS or whatever in a skip.
Here's an example of how the output ends up. Excuse the failures, I'm
working on something :)
[EMAIL PROTECTED]:~/GeoSol/trunk/cgi-bin/test$ wprove -rl
Password:
00_legacy/20050200.......................ok
00_legacy/20050300.......................ok
00_legacy/20050900.......................ok
./01_compile.............................ok
./02_integrity...........................ok
./03_database_integrity..................ok
456/689 skipped: Skipping: TEST_TAG_DB_RO is disabled
./04_library_apis........................ok
./06_code_symbols........................NOK 421
# Failed test 'GeoSol::Interface::Role::File->can('DataToView')'
# in ./06_code_symbols.t at line 176.
# GeoSol::Interface::Role::File->can('DataToView') failed
# Looks like you failed 1 test of 798.
./06_code_symbols........................dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 421
Failed 1/798 tests, 99.87% okay
./07_templates...........................ok
./08_images..............................ok
auto/geosol__entity......................ok
auto/geosol__view........................ok
auto/geosol_base.........................ok
auto/geosol_cachemanager.................ok
auto/geosol_config.......................ok
auto/geosol_data__simple.................ok
12/14 skipped: Skipping: TEST_TAG_DB_RO is disabled
auto/geosol_data__string.................ok
22/25 skipped: Skipping: TEST_TAG_DB_RO is disabled
auto/geosol_data_boolean.................ok
auto/geosol_data_datetime................ok
auto/geosol_data_integer.................ok
auto/geosol_data_list....................ok
auto/geosol_data_longstring..............ok
auto/geosol_data_shortstring.............ok
auto/geosol_db...........................ok
13/16 skipped: Skipping: TEST_TAG_DB_RO is disabled
auto/geosol_db_config....................ok
auto/geosol_db_schema....................ok
9/12 skipped: Skipping: TEST_TAG_DB_RO is disabled
auto/geosol_email........................ok
auto/geosol_entity_application...........ok
auto/geosol_entity_message...............ok
auto/geosol_entity_report................ok
auto/geosol_entity_resume................ok
auto/geosol_entity_session...............ok
auto/geosol_entity_user..................ok
auto/geosol_field........................ok
auto/geosol_html.........................ok
etc etc...
Adam K
