Minor correction: I placed Atomic.__exit__ to verify - the transaction is
commited every time *starting* on the second object (the third stack
printed in the previous post) - it happens at
https://github.com/django/django/blob/master/django/db/transaction.py#L288.
On Sunday, June 23, 2013 7:24:40 PM UTC-4, Yo-Yo Ma wrote:
>
> Hi again Russell,
>
> I did a little digging. I'm not sure, but I may have uncovered the
> problem. A transaction block (using `commit_on_success_unless_managed`) is
> entered and exited during each fixture object loaded, due to the calls to
> the aforementioned method that exist in various model methods (namely,
> `save_base`, in this case). Because of this, the transaction is committed
> immedately after each object is loaded, despite the attempt to wrap
> `commit_on_success_unless_managed` around the context of the `loaddata`
> call in the management command.
>
> The following are the results of my placing print statements (I know
> that's old-school - pdb is just too time consuming) inside
> `commit_on_success_unless_managed`. In each call, I added:
>
> print 'AUTOCOMMIT', connection.autocommit
> print 'IN ATOMIC BLOCK', connection.in_atomic_block
> for frame in inspect.stack():
> print frame[1], frame[3], frame[2]
>
> as well as a print after the stack of whether atomic() was returned or
> _transaction_func() was returned (for easier reading):
>
>
> AUTOCOMMIT False
> IN ATOMIC BLOCK False
>
> django/db/transaction.py commit_on_success_unless_managed 492
> django/core/management/commands/loaddata.py handle 53
> django/core/management/base.py execute 283
> django/core/management/base.py run_from_argv 240
> django/core/management/__init__.py execute 392
> django/core/management/__init__.py execute_from_command_line 399
> manage.py <module> 10
>
> ----RETURNING TRANSACTION FUNC
>
> ===========================================================
>
> AUTOCOMMIT False
> IN ATOMIC BLOCK False
>
> django/db/transaction.py commit_on_success_unless_managed 492
> django/db/models/base.py save_base 573
> django/core/serializers/base.py save 165
> django/core/management/commands/loaddata.py process_dir 225
> django/core/management/commands/loaddata.py load_label 169
> django/core/management/commands/loaddata.py loaddata 102
> django/core/management/commands/loaddata.py handle 54
> django/core/management/base.py execute 283
> django/core/management/base.py run_from_argv 240
> django/core/management/__init__.py execute 392
> django/core/management/__init__.py execute_from_command_line 399
> manage.py <module> 10
>
> ----RETURNING TRANSACTION FUNC
>
> ===========================================================
>
> SAVEPOINT False
> AUTOCOMMIT True
> IN ATOMIC BLOCK False
>
> |||||||||||||||||||||||||||||||||||||||||||||||
> django/db/transaction.py commit_on_success_unless_managed 492
> django/db/models/base.py save_base 573
> django/core/serializers/base.py save 165
> django/core/management/commands/loaddata.py process_dir 225
> django/core/management/commands/loaddata.py load_label 169
> django/core/management/commands/loaddata.py loaddata 102
> django/core/management/commands/loaddata.py handle 54
> django/core/management/base.py execute 283
> django/core/management/base.py run_from_argv 240
> django/core/management/__init__.py execute 392
> django/core/management/__init__.py execute_from_command_line 399
> manage.py <module> 10
>
> ----RETURNING ATOMIC
>
> ===========================================================
>
>
> The remaining calls were exactly like call 3 (including "IN ATOMIC BLOCK
> False", despite the 3rd call having returned `atomic()`). My prima facie
> opinion is that `with atomic()` is needed in `loaddata`, instead of `with
> commit_on_success_unless_managed`, since the latter acts funky when nested
> calls occur (as see in save_base in the stacks printed above). However, the
> issue might be something that needs to be resolved in the
> transitioning-to-atomic code. I don't fully understand all of this yet, but
> it's a start.
>
>
> On Friday, June 21, 2013 4:34:14 PM UTC-4, Yo-Yo Ma wrote:
>>
>> Pardon one typo: I meant `python manage.py loaddata test_data` in my
>> previous post.
>>
>> On Friday, June 21, 2013 4:32:33 PM UTC-4, Yo-Yo Ma wrote:
>>>
>>> Hi Russel,
>>>
>>> Thanks for taking the time to explain that. I tried that same day to
>>> reproduce the issue in a testing env with the simplified models I typed
>>> above, but my hosting provider had some erroneous networking nonsense that
>>> ruined my test after I spent a couple hours setting everything up. I
>>> figured I'm come back to it... and here I am.
>>>
>>> I didn't set up an entire test env and test app this time, just made a
>>> fresh database and ran my apps fixtures on it, but I did test my app again,
>>> using a fresh database without any data. The models and fixtures for which
>>> are as follows (minus most of the decimals, chars, and other non-FK-type
>>> fields, none of which should be related to this problem):
>>>
>>>
>>> # account/models.py
>>> class Account(models.Model):
>>> name = models.CharField(_(u'name'), max_length=255)
>>>
>>>
>>> # orders/models.py
>>> class Order(models.Model):
>>> account = models.ForeignKey('account.Account',
>>> verbose_name=_(u'account'))
>>> number = models.IntegerField(_(u'number'))
>>> bill_address = models.OneToOneField(
>>> 'orders.OrderAddress',
>>> null=True,
>>> on_delete=models.SET_NULL,
>>> related_name='bill_address_order',
>>> verbose_name=_(u'bill to address')
>>> )
>>>
>>> class OrderAddress(models.Model):
>>> account = models.ForeignKey('account.Account',
>>> verbose_name=_(u'account'))
>>> order = models.ForeignKey('orders.Order', verbose_name=_(u'order'))
>>> country = models.CharField(_(u'country'), max_length=2)
>>>
>>>
>>> // orders/fixtures/test_data.json
>>> [
>>> {
>>> "model": "orders.order",
>>> "pk": 1,
>>> "fields": {
>>> "account": 1,
>>> "number": 1,
>>> "bill_address": 1
>>> }
>>> },
>>> {
>>> "model": "orders.orderaddress",
>>> "pk": 1,
>>> "fields": {
>>> "account": 1,
>>> "order": 1,
>>> "country": "US",
>>> }
>>> }
>>> ]
>>>
>>>
>>> (an Account with the primary key of 1 already exists at the time of
>>> ``loaddata``)
>>>
>>>
>>> The error I get with `python manage.py loaddata test_data orders` is:
>>>
>>> django.db.utils.IntegrityError: Problem installing fixture
>>> '/opt/myproject/apps/orders/fixtures/test_data.json': Could not load
>>> orders.Order(pk=1): insert or update on table "orders_order" violates
>>> foreign key constraint "bill_address_id_refs_id_3a4d3fef"
>>> DETAIL: Key (bill_address_id)=(1) is not present in table
>>> "orders_orderaddress".
>>>
>>>
>>> The above fixtures load locally, and they load during test running (with
>>> Postgres) a number of times, but for some reason I get that error when
>>> using `manage.py loaddata ...`.
>>>
>>>
>>> On Sunday, June 16, 2013 7:40:02 PM UTC-4, Russell Keith-Magee wrote:
>>>>
>>>>
>>>> Circular dependencies *shouldn't* be a problem on PostgreSQL because
>>>> all constraints are set DEFERABLE INITIALLY DEFERRED; that means no
>>>> constrain checks should be performed are performed until the transaction
>>>> boundary, so all circular references shouldn't be a problem.
>>>>
>>>> Ticket #3615 exists because MySQL's implementation of DEFERABLE
>>>> INITIALLY DEFERRED under InnoDB is, to use the technical term, "Broken".
>>>> It's unrelated to any problem you may have found in PostgreSQL, because
>>>> PostgreSQL gets the underlying behaviour right.
>>>>
>>>> Beyond that, we need a specific test case to take this any further. As
>>>> it stands, I'm not aware of any problems loading fixtures into PostgreSQL.
>>>> If you are able to construct and provide a set of models (which you have
>>>> done) and simple fixture (which you haven't) that fails reliably, we have
>>>> a
>>>> new bug on our hands, and you should open a ticket with all the details
>>>> you
>>>> can provide. Confirming whether this is a problem with the alpha, or an
>>>> ongoing problem would also be helpful.
>>>>
>>>> Yours,
>>>> Russ Magee %-)
>>>>
>>>> On Mon, Jun 17, 2013 at 6:22 AM, Yo-Yo Ma <baxters...@gmail.com> wrote:
>>>>
>>>>> There doesn't appear to be a way to load fixtures from JSON (using
>>>>> Postgres - works fine in sqlite3) for the following models:
>>>>>
>>>>>
>>>>> class Collection(models.Model):
>>>>> main_thing = models.OneToOneField(
>>>>> 'things.Thing',
>>>>> null=True,
>>>>> on_delete=models.SET_NULL
>>>>> )
>>>>>
>>>>> class Thing(models.Model):
>>>>> collection = models.ForeignKey(
>>>>> 'collections.Collection'
>>>>> )
>>>>>
>>>>>
>>>>> Here is the exception:
>>>>>
>>>>> Problem installing fixture 'my_fixture.json': Could not load
>>>>> collections.Collection(pk=1): insert or update on table
>>>>> "collections_collection" violates foreign key constraint
>>>>> "main_thing_id_refs_id_3a4d3fef"
>>>>> DETAIL: Key (main_thing_id)=(1) is not present in table
>>>>> "things_thing".
>>>>>
>>>>> I'm not sure if the issue is due to the unique constraint implied by a
>>>>> OneToOneField, or if it's just related to this issue:
>>>>> https://code.djangoproject.com/ticket/3615 (seems like that ticket
>>>>> and related ones have been closed for years, so possibly not related).
>>>>>
>>>>> Any thoughts?
>>>>>
>>>>> Note: I'm using @1.6a1
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Django developers" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to django-develop...@googlegroups.com.
>>>>> To post to this group, send email to django-d...@googlegroups.com.
>>>>> Visit this group at http://groups.google.com/group/django-developers.
>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>
>>>>>
>>>>>
>>>>
>>>>
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
For more options, visit https://groups.google.com/groups/opt_out.
