On Jul 14, 2011, at 1:31 AM, Simon Riggs wrote: > On Sun, Jul 10, 2011 at 3:27 PM, Jim Dalton <jim.dal...@gmail.com> wrote: >> On Jul 10, 2011, at 3:13 AM, Simon Riggs wrote: >> >>> Maintaining the property of deferrable constraints seems important >>> here, so changing the deferrability of constraints, or overriding it >>> using the SET CONSTRAINTS command at the top of the transaction might >>> not be what we want. >> >> Well, that's just it. We want tests to behave as much like production code >> as possible, so we actually *don't* want constraint checks to be deferred. >> >> When you're working with DB normally in production, i.e. outside of a >> transaction, constraints are checked immediately. It's only when you get >> into a transaction that they are deferred. Each of our tests run inside of a >> transaction, which is an artificial construct. So to emulate production, I'm >> arguing that this property is not something we want (*unless* we are testing >> a transaction or transaction like behavior, in which case it does make sense >> to temporarily suspend constraint checks). > > My role here is to help with Postgres details, so any comments I make > are just attempts at providing useful information.
Thank you, btw, for offering your input in that capacity. I found it quite reassuring when I read your first reply that someone with your credentials was giving feedback on this issue. :) > > It sounds like there is a slight confusion about the way PostgreSQL > works. Everything is a transaction, so there is no difference between > inside/outside a transaction. > You can choose whether you wish immediate or deferred constraint > checking. It's completely user selectable, though the default is > immediate. If you're seeing deferred checks, they can be changed. > > ISTM that the tests should work the same way as things do in > production, and it looks possible to make that happen. Okay, this is interesting -- and thank you for clarifying this point, which I had grossly oversimplified. I think you actually shed a huge amount of light on part of the problem here (at least for me). So in normal, everyday use, Django opens a connection to Postgresql with an open transaction. Per the docs, it commits this transaction immediately whenever you do an INSERT or UPDATE etc. At that point Postgresql would run its constraint checks and rollback the transaction on error or else commit. During a test, we open up a big transaction during setUp and then *disable* the transaction commit and rollback operations, i.e. a call to either does nothing. Since we can't nest transactions in Postgresql (right?) this has been the only sensible way to allow tests to proceed. The problem has been -- I am positing -- that we're getting away with a lot of stuff as a result, because constraint checks are *never* checked during testing. The small slew of bugs I found is a demonstration of that, to me. My solution up to now, however, is also "unlifelike", which your comment has helped me to realize. It works kind of similar to the way Django works normally because in some ways Django sort of emulates immediate checks in its default behavior. But I think my present solution is a kludge in the other direction and I agree is not close enough to reality. I'm thinking this might not be too hard to solve though. I'm thinking that rather than *only* enabling constraint checks immediately as I was suggesting before, we could instead focus on the transaction-related methods that we are presently just blanking out (by this I mean when tests are set up, transaction.commit() transaction.rollback() etc. are being monkey patched with a function that does nothing). Even though we can't nest a transaction explicitly, we can better emulate the behavior of normal Django operation. We can keep things in deferred mode as they are in production, but then do a temporary flip to IMMEDIATE at commit() which will fire the constraint checks. Presumably, we could even do a SAVEPOINT, as you suggested, at the start so that the test could be recovered and continue on if e.g. the IntegrityError needed to be caught during testing. I imagine that this will still flush out most if not all of the little errors I had discovered (and prevent us from writing test cases going forward with similar problems) but at the same time will very closely emulate production. The challenge will be wrangling all this together with the other DBs (i.e. MySQL and SQLite) which don't share this execution model. Should be workable though. Anyhow, thank you once again Simon for shedding light on this for me. Cheers Jim -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
