I'm a big fan of testing units in isolation. It's easier, faster, and tends to
more comprehensive and detailed tests. The problem is, if the unit being
tested has a legitimate dependency on another unit in the suite, isolation
testing cannot identify interface changes.
This has bothered me for a while. There had to be a way to identify these
dependencies without substantially increasing the complexity of the tests. It
should be as automated as possible while still allowing a sufficient level of
detail.
I've come up with a scheme that could be an 80% solution. It's called
Test::Depend. It's based on Test::Builder. Here's my preliminary feature list:
- wrap around use_ok() to find out which module is being tested
- save information about module being tested
- version (if present)
- number of tests: total, passed, failed, skipped
- last modified timestamp
- look in %INC to see which modules may have been mocked
- values that aren't paths are probably mocked
- these represent dependencies
- write a test (999zzz.t) to explore dependencies
- must run after all tests
- compares saved file information with that from the most recent run
- allow finer grained dependency checking ?
- mark sections in tests AS dependencies
- 'Foo::open'
Taking an average test suite as an example, we could have 'foo.t' and 'bar.t',
testing Foo.pm and Bar.pm respectively. Bar depends on Foo, and bar.t mocks a
couple of methods of Foo.
If bar.t uses Test::Depend, it'll pick up several things:
- the module being tested (assumed from a use_ok() or require_ok() call)
- the module being mocked (assumed from an entry in %INC)
If there's no dependency information, it'll use the current version of Foo.pm,
the last modified timestamp of the test file, and the test status of foo.t (if
it also uses Test::Depend) to keep track of any changes to Foo.pm.
If this dependency information changes, it'll fail a test (or maybe warn)
because there's a potential interface change that Bar.pm may need to know.
Like I said earlier, I'm aiming for an 80% solution. There are some
assumptions and some fragilities here, but it represents a great improvement
over the current system, in my mind.
I welcome any comments.
-- c
