> My initial input would be: > > Ensure that we have proper goals!!! > > Is this going to be a DBI conformance test or a driver test suite? I belive > those are two different (but not incompatible) goals. A conformance test > would be more simple, in that it would test the interface to the driver not > necessarily that it's functioning correctly. It would and determine which > DBI features the driver supplies vs what DBI allows. Ideally, this would be > updated every time a new feature is added to DBI and would produce a report > showing which features were implemented by the driver, which by DBI (e.g. > the default DBI implementation of array handling vs. the driver implementing > it in a "better" fashion). and which are not implemented. > > A DBD Driver test, in my terms, would be something that reads and writes the > data to the database and VERIFIES that it's getting what is expected. In my > experience, with DBD::ODBC, this takes a bit of effort. Does this driver > supply SQL_INTEGER, SQL_SMALLINT, etc? DBD::ODBC uses a primitive method to > work this out and there are probably much better approaches. That said: > DBD::ODBC tests, for the most part, pass against Access, SQL Server, DB2 and > Oracle (and, when no DBI_DSN is set, which is important for any automated > builds). The "Driver" test would, IMHO, be VERY dependant upon the > conformance tests, to know what to test and to know how to approach inserts, > deletes, etc. > > I, actually, think the "Driver" test would be more useful for Driver writers > and the conformance test more useful for end-users -- and both would be > valuable, but I think we should define the goals first. I think I'm not > sure which direction this is going. I'm an advocate of covering both > "conformance" and "performance" -- but I think those goals should be spelled > out first. > > Just my initial thought. An intended SCOPE would be nice for those of us > who would like to contribute, but don't have a lot of time to do it. > > Jeff
My initial purpose for suggesting such a test suite was the initial difficulty I had when I first wrote a DBD: docs were pretty thin, and there was a lot of painful trial and error. And I'm certain I'm *still* not in conformance in a lot of places (esp. as DBI API changes roll out). If each DBI release contained updated conformance tests, DBD authors could simply rerun the latest conformance test against their latest driver and (hopefully) find out empirically where any non-conf's occur. Also, did you mean ..."conformance" and "functional"... rather than ..."conformance and "performance"... The latter has implications that aren't neccesarily relevant to conformance/correctness (Tho I still believe in 'Damn the metadata, FULLSPEED AHEAD!') Conformance can be pretty simplistic, tho I'd like to see thorough coverage (hopefully better coverage than some other conformance suites I've used, e.g., JDBC). Since DBI itself wraps much of the processing of underlying DBDs in its own processing, perhaps 2 conformance suites are needed: 1. DBI layer conformance: to test that the end-to-end functionality is correct. This would likely be written entirely in std. DBI/perl doing the usual DBI API calls. 2. DBD conformance: this may require more work, but actually be a smaller set of tests. Essentially, it creates a "pseudo" DBI layer that just exersizes the DBD i/f calls that the DBI layer (as opposed to a DBI app) makes. The latter may be more useful for driver writers, while the former is more useful to end-users. Also, some instrumentation for performance might be useful, esp. as driver writers roll out new releases: rerun the conformance test(s), and see where things slowed down or sped up. My 2 cents, Dean Arnold Presicient Corp. www.presicient.com
