On Mon, Sep 27, 2010 at 7:22 PM, Russell Keith-Magee <russ...@keith-magee.com> wrote: > On Tue, Sep 28, 2010 at 1:51 AM, Mark Bucciarelli <mkb...@gmail.com> wrote: >> On Sat, Sep 25, 2010 at 10:21:06PM +0800, Russell Keith-Magee wrote: >>> >>> The number 4 wasn't actually the important bit - it was that I want >>> to see a range of noSQL approaches represented. I don't want to see >>> 4 key-value stores; I want to see a key-value store, and a document >>> store, and a column store, etc. I need to see that the API is rich >>> enough to cover all noSQL bases. >>> >> >> I've seen references to Django's test suite on this list in the past, >> and I'm curious why the criteria can't simple be: "a clean run on our >> test suite." >> >> I have a backend [1] for MonetDB (a column store DB). They expose a SQL >> wrapper around their non-SQL engine, which makes it pretty easy to >> support. >> >> On my todo list was to see how it fares with the Django test-suite. >> I guess the test suite must not cover all the basees, making it a >> necessary but not sufficient criteria for inclusion? >> >> Correct? > > Well, sort of. > > The problem is that NoSQL stores don't necessarily have feature parity > with relational databases. For example, the concept of a join is > pretty fuzzy when it comes to stores like MongoDB and GAE. You *could* > emulate join with a series of lookups, but that could get pretty > spectacularly inefficient, so we're not going to force stores to > emulate behaviors that aren't extensions of their native capabilities. > Given that there isn't feature parity, Django's test suite won't be > able to pass completely. > > We're in the process of migrating Django's test suite to use > unittest2, which, in turn, means we have access to the @skipif and > @skipunless decorators; we're going to be annotating a bunch of tests > to make them 'feature aware' (e.g., don't run this test if the backend > doesn't support transactions). A logical extension of this is to add > other feature checks like "backend supports joins", and annotate the > test suite appropriately. This would allow the noSQL backend to pass > the suite, albeit with a lot of test skips. > > My (mild) concern with this approach is that a feature that this will > make it too easy to get a suite 'passing', and we'll miss some detail > in the process. For example, if we make it easy to skip the aggregates > tests, and a backend actually does support aggregates, then it becomes > easy to say "Oh, we *could* do aggregates, but we just haven't yet". > Now, this may be entirely true, but it could also be the case that > Django's backend API is structured in such as way that aggregates are > impossible to implement. Making it easy to skip tests masks the > complications that may exist in the API. > > Yours, > Russ Magee %-) > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-develop...@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > >
This doesn't seem like much of a regression to me. While developing my backend I basically ignored the existing tests because it was impossible to learn anything from 700 test failures, of which some are "expected and never will pass" (e.g. ones testing that we generate only one JOIN), other are "this is something I just haven't implemented yet, and really don't need to be told". In fact, I view being able to gradually increase the portion of the API you claim to implement, as shown in your testing, as being a tremendous boon. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Voltaire "The people's good is the highest law." -- Cicero "Code can always be simpler than you think, but never as simple as you want" -- Me -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
