Hi all.
This email aims to start a discussion around clarifying the goals, scope
and requirements of a Common DBI Driver Test Suite, including learning
from the great work already done on DBI::Test.
Once we've kicked these around to the point were there's reasonable
agreement, then we'll start a new thread to discuss design/implementation.
For this thread though, goals, scope, and requirements are the focus.
First, let's spell out some basics...
Stakeholders: (who is affected by this work)
- driver developers
- cpantesters
- end users
Terminology:
* We'll use DBIT as the generic term for this thing, whatever it
turns out to look like or be called.
Goals: ("how will we know if this project is or is not a success")
G1. driver developers improve their compliance with the DBI API
and so improve consistency and portability of code.
This is what it's all about!
G2. driver developers adopt DBIT as a free test suite with good
coverage of the DBI API. This is the pay-back _for_ developers.
G3. driver developers write and share reusable generic tests
(they'll still need to write their own driver-specific tests).
This is the pay-back _from_ developers.
G4. end users won't be affected by DBIT evolution causing install failures
i.e., DBIT runs as 'author testing', *at least for now*.
This is our promise to end users not to ruin their day.
G5. be usable for the widest range of DBI drivers, including non-SQL.
This is a promise to be inclusive.
G6. enable testing with proxies like DBD::Gofer.
This is a promise to be flexible.
G7. enable testing with DBI::PurePerl for pure-perl drivers.
This is a promise to support fatpackers and perl6 ;)
G8. Provide tools that drivers can use/extend for themselves.
I'm thinking specifically here of the creation of test files
with combinations of env vars and other settings.
E.g., test DBD::CSV with Text::CSV and Text::CSV_XS
G9. end users can find the level of DBI API compliance of a driver
i.e., by looking at the test configuration for the driver
to see what areas of functionality are configured to be skipped.
[I thought of adding an extra goal: "other code can determine the
limitations/quirks of a driver i.e., by looking at the test
configuration for the driver to see how it's configured." but to be
generic I think that would require a connected $dbh to determine server
version/config. It's worth keeping in mind though in terms of having an
object that represents the server that can be queried for info needed by
tests. Perhaps that'll end up as extensions to $dbh->get_info().]
Scope: (the boundaries and deliverables of the project)
S1. the DBIT will be a separate distribution.
S3. the DBI won't have a mandatory dependency on DBIT,
for now at least, driver developers are the priority.
S4. DBIT is not meant to test the underlying database.
If a driver implements a database itself then it'll
need it's own tests to provide good coverage of that.
[Regarding S4: We could enable individual distros to install modules
that add DBIT tests that'll be included in DBIT runs in certain cases,
e.g., for certain drivers. I want to avoid running lots of SQL tests
for drivers where that doesn't make sense. I'm sure we can find a way to
do that but still allow sharing of SQL tests for SqlEngine based DBDs.]
Requirements:
R1. be easy for driver developers to adopt,
so existing drivers migrate to using it.
R2. be easy for driver developers to extend/contribute to,
so driver developers contribute to it.
R3. work well with cpantesters,
so we get good coverage (perhaps extend Test::Database)
So, any thoughts on these goals, scope, and requirements?
(Please avoid design and implementation details for now. If you have any
concerns that relate to design and implementation that you feel need to
be considered now, try to express them in terms of the goals, scope, or
requirements that would lead to that design.)
Tim.
