On Wed, 25 Sep 2013 17:28:04 +0100, Tim Bunce <tim.bu...@pobox.com>
wrote:
> Hi all.
>
> This email aims to start a discussion around clarifying the goals, scope
> and requirements of a Common DBI Driver Test Suite, including learning
> from the great work already done on DBI::Test.
>
> Once we've kicked these around to the point were there's reasonable
> agreement, then we'll start a new thread to discuss design/implementation.
> For this thread though, goals, scope, and requirements are the focus.
>
> First, let's spell out some basics...
>
> Stakeholders: (who is affected by this work)
>
> - driver developers
> - cpantesters
> - end users
>
> Terminology:
>
> * We'll use DBIT as the generic term for this thing, whatever it
> turns out to look like or be called.
>
> Goals: ("how will we know if this project is or is not a success")
>
> G1. driver developers improve their compliance with the DBI API
> and so improve consistency and portability of code.
> This is what it's all about!
Yes, my primary goal
> G2. driver developers adopt DBIT as a free test suite with good
> coverage of the DBI API. This is the pay-back _for_ developers.
Yes, also my secondary goal. Going the right way here :) :)
> G3. driver developers write and share reusable generic tests
> (they'll still need to write their own driver-specific tests).
> This is the pay-back _from_ developers.
Most likely. To be honest I never thought about it that way
> G4. end users won't be affected by DBIT evolution causing install
> failures i.e., DBIT runs as 'author testing', *at least for now*.
> This is our promise to end users not to ruin their day.
100% agree. for now it is optional
> G5. be usable for the widest range of DBI drivers, including non-SQL.
> This is a promise to be inclusive.
+1
> G6. enable testing with proxies like DBD::Gofer.
> This is a promise to be flexible.
+1
> G7. enable testing with DBI::PurePerl for pure-perl drivers.
> This is a promise to support fatpackers and perl6 ;)
+2
> G8. Provide tools that drivers can use/extend for themselves.
> I'm thinking specifically here of the creation of test files
> with combinations of env vars and other settings.
> E.g., test DBD::CSV with Text::CSV and Text::CSV_XS
As a goal, this is fine. For DBD::CSV, this falls under G7: It will use
Text::CSV_PP (from Text::CSV) under DBI::PurePerl and Text::CSV_XS
under DBI (XS)
> G9. end users can find the level of DBI API compliance of a driver
> i.e., by looking at the test configuration for the driver
> to see what areas of functionality are configured to be skipped.
I have a hard time parsing that.
> [I thought of adding an extra goal: "other code can determine the
> limitations/quirks of a driver i.e., by looking at the test
> configuration for the driver to see how it's configured." but to be
> generic I think that would require a connected $dbh to determine server
> version/config. It's worth keeping in mind though in terms of having an
> object that represents the server that can be queried for info needed by
> tests. Perhaps that'll end up as extensions to $dbh->get_info().]
>
>
> Scope: (the boundaries and deliverables of the project)
>
> S1. the DBIT will be a separate distribution.
+1. It already is, and there has never been the intention to bundle it
in any other distribution
> S3. the DBI won't have a mandatory dependency on DBIT,
> for now at least, driver developers are the priority.
+100
> S4. DBIT is not meant to test the underlying database.
> If a driver implements a database itself then it'll
> need it's own tests to provide good coverage of that.
DBIT will offer the interface to test the underlying database. What is
important here, as DBIT will offer the possibility to test under
different circumstances (like with SQL::Nano or under DBI::Proxy)
without having to rewrite all tests over and over again.
These test will reside in the test suite of the DBD, not in the test
suite of DBIT
> [Regarding S4: We could enable individual distros to install modules
> that add DBIT tests that'll be included in DBIT runs in certain cases,
> e.g., for certain drivers. I want to avoid running lots of SQL tests
> for drivers where that doesn't make sense. I'm sure we can find a way to
> do that but still allow sharing of SQL tests for SqlEngine based DBDs.]
>
>
> Requirements:
>
> R1. be easy for driver developers to adopt,
> so existing drivers migrate to using it.
+1
> R2. be easy for driver developers to extend/contribute to,
> so driver developers contribute to it.
+1
> R3. work well with cpantesters,
> so we get good coverage (perhaps extend Test::Database)
+1
> So, any thoughts on these goals, scope, and requirements?
>
> (Please avoid design and implementation details for now. If you have any
> concerns that relate to design and implementation that you feel need to
> be considered now, try to express them in terms of the goals, scope, or
> requirements that would lead to that design.)
>
> Tim.
Thanks for the summary
--
H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/
using perl5.00307 .. 5.19 porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/
http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/
