On May 9, 12:24 pm, Karen Tracey <kmtra...@gmail.com> wrote: > On Mon, May 9, 2011 at 12:56 AM, Adam Seering <aseer...@mit.edu> wrote: > > On May 8, 10:53 pm, Karen Tracey <kmtra...@gmail.com> wrote:> > > > The change you have noticed is documented in the 1.3 > > > backwards-incompatibility list: > >http://docs.djangoproject.com/en/1.3/releases/1.3/#use-of-custom-sql-... > > > Actually, that's a different issue: I'm not using the custom-SQL > > loader, I'm using Python code with the post_syncdb hook. This one > > isn't in the backwards-compatibility page as far as I can tell. > > > (Was that comment intended to cover this case as well? Maybe a typo > > somewhere?) > > Hmm, you are right, this change is not supposed to affect data added via > post_syncdb handlers. Specifically the change here > is:https://code.djangoproject.com/changeset/15239. On an initial read it looks > like it might cause post_syncdb-added data to get thrown away, since the > post_syncdb signal is sent during the call_command('syncdb'). However, the > call_command('flush', ...) that was added after the call_command('syncdb', > ...) will also cause that signal to get sent.
Ah, I see what's going on. Apologies; it sounds like this was a mess- up / misunderstanding on my part: Our syncdb hooks aren't safely reentrant or re-runnable; they're not idempotent, and they depend for correctness on having a clear django.core.cache to run queries against and to push stuff into (or at least a cache that we haven't already stuffed rows into that have since been deleted from the database but not the cache; we have a fancy system that listens to post_delete to keep the cache in sync with the database, but it looks like "flush" doesn't fire post_delete?). Most were written prior to Django's handy "dispatch_uid" deduplication mechanism, so they have a simple one-off global latch that only allows them to run once. So they ran once for the syncdb, then the "flush" cleared out all the database data and they didn't run again the second time around. My mistake was to look at the queries being sent to the database, see the data being loaded and the following TRUNCATE, and assume that things were supposed to stop there. (Out of curiosity, any idea why this was done this way? It feels a bit odd to me to generate data, throw it out, and generate it again... Could the SQL-loader just be turned off during "./manage.py test" runs?) This mostly leads to a nice clean obvious solution: Make our loading functions be safely reentrant and go start using dispatch_uid. One new problem, though: We'll have to do something to flush the cache whenever "flush" is called, so that stale data doesn't get left behind. Should be quite doable, if a little clunky. > > Right now, we're working on adding tests, and on encouraging > > developers to write tests for new parts of the code, new apps/code > > that are being developed, etc. Right now, as I understand it with > > Django's current behavior and our post_syncdb hook setup, we want to > > re-load the data in three cases: (1) The current test is the first > > test, (2) all prior tests are TestCase (as opposed to > > TransactionTestCase) instances, or (3) we are a TransactionTestCase > > instance. > > (...) In fact > Django will run all TestCase tests before any TransactionTestCase tests (as > noted in the > dochttp://docs.djangoproject.com/en/1.3/topics/testing/#testcase), precisely > because the mechanism used to reset the database for these different kinds > of tests can't ensure that a TestCase run after a TransactionTestCase will > see a "clean" database on start. Ah, ok: Because that text listed "(e.g. doctests)" as the counterexample, and since it seemed to be comparing both django.test.TestCase and django.test.TransactionTestCase, I took the first sentence of that paragraph to mean "A <tt>unittest.TestCase</ tt>, on the other hand, ...". If all the TransactionTestCases are at the end as well, then that does in fact get rid of the weird edge cases that I cited, modulo doctests as you noted (which we do use occasionally but not very often). Thanks, Adam -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
