Re: failure to load fixtures during unit tests

2016-05-09 Thread Rich Rauenzahn
FYI, I was finally able to resolve this.  I had an assumption that 
TestCase's began with a freshly created database between TestCase classes. 
 I asserted this in _fixture_setup() and found that assumption to be false. 
 Upon further research I found a setUpClass() in another TestCase that 
created objects that conflicted with the fixtures loaded in other TestCase 
classes.  Somehow Django-nose's FastFixtureTestCase hid this problem.

What I'm not sure about (and need to investigate) is if Django-nose's 
TestRunner is the optimizing away fresh databases between TestCase classes, 
or if that is standard.

Rich

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f74ac850-26d4-4db2-8aef-1d595f8279b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: failure to load fixtures during unit tests

2016-05-05 Thread Rich Rauenzahn

Thanks, Tim.

Unfortunately I can't move past Django 1.7 yet -- dependencies.  I've been 
marching my way up one revision at a time hopefully up to 1.9 as a way to 
keep the scope of what breaks under control as I move through each major 
revision and stabilize my project.  Then I attack replacing dependencies.

I really think I've found a bug here ... which I hope to suggest a patch 
for and submit, hence the post to the developers channel, but I can go back 
to the users group for now... My recent experience with that list doesn't 
bode well, however, and I don't have high hopes with anyone there able to 
respond at the internals level I may need to track down the issue.  I've 
almost rewritten my tests to just load raw sql, but if there is a bug here 
I'd like to help find it rather than work around/ignore it.

As I step through the code, it really looks like the _save_table() method 
in Model is trying to insert a row even though the object has already been 
restored/inserted.  At the moment, I'm reproducing it with the auth.User 
Model.

I'm getting closer to seeing what is happening 

I have a user, rich, which expects to be pk=1 per the fixture.  

> 
/opt/perfcat/virtualenv-2.7.11/lib/python2.7/site-packages/django/db/models/base.py(686)_save_table()uest_by_build_workflow_fail
 
  
685 import ipdb; ipdb.set_trace()
--> 686 if not updated:
687 if meta.order_with_respect_to:

ipdb> self.id
1
ipdb> self.__class__.objects.all()
[]
ipdb> self.__class__.objects.all()[0].id
5
ipdb> self.username
u'rich'
ipdb> 


But In this particular run I'm currently tracing, rich is already in the db 
(as the only entry) as pk=5 (via fixture loading process).   For one, this 
tells me the sequence generators aren't always resetting between fixture 
loads/tests.

So I think the code is trying to reassign it to pk=1.  

We did drop into the update code,

ipdb> pk_set and not force_insert
True

But updated is False

ipdb> updated
False

So now it tries to drop into an insert, but it is going to get an Integrity 
error because username has to be unique.

Not sure what this means, yet, but my current step through looks like this:

ipdb> 
IntegrityError: Integrit...sts.\n',)
> 
/opt/perfcat/virtualenv-2.7.11/lib/python2.7/site-packages/django/db/models/base.py(700)_save_table()
699 update_pk = bool(meta.has_auto_field and not pk_set)
--> 700 result = self._do_insert(cls._base_manager, using, 
fields, update_pk, raw)
701 if update_pk:

ipdb> update_pk
False
ipdb> meta.has_auto_field
True
ipdb> pk_set
True
ipdb> 

...if we don't need to update the pk, and it is set .. why are we inserting 
it? 

Walking through a second time with this knowledge ... and stepping into 
_do_update(),

I end up with filtered = base_qs.filter(pk=pk_val) being equal to [] 
because the entry in the db has a pk=5, and it is filtering for pk=1

So when return filtered._update(values) > 0 returns, it returns false 
because nothing was updated because the pk's didn't match.

Where I am stuck at now is not understanding how fixture loading manages 
the pks... 

Rich

On Wednesday, May 4, 2016 at 4:22:33 PM UTC-7, Tim Graham wrote:
>
> Hi Rich, django-users is the appropriate place to ask "is it a bug?" type 
> questions. We try not to use this mailing list as a second level support 
> channel, otherwise it'd get really noisy. Thanks for understanding.
>
> By the way, Django 1.7 is no longer supported. Please make sure you can 
> reproduce the issue on Django master so we don't spend time debugging 
> issues that have since been fixed.
>
> On Wednesday, May 4, 2016 at 7:13:42 PM UTC-4, Rich Rauenzahn wrote:
>>
>>
>> I'm in the middle of trying to track down a problem with loading fixtures 
>> during unit tests -- I'm hesitant to call it a bug in Django 1.7, but the 
>> inconsistent behavior is really stumping me.
>>
>> Essentially I've made a fixture via
>>
>>manage dumpdata --indent=3 -e sessions -e admin -e contenttypes -e 
>> auth.Permission > test-fixtures.json
>>
>> If I add that fixtures to my TestCase, it sometimes works if I run each 
>> test individually (using Django Nose)  -- 
>>
>>manage test --failfast test_it:TestClass.test_detail
>>manage test --failfast test_it:TestClass.test_list
>>
>> But if I run them together, 
>>
>>manage test --failfast test_it:TestClass
>>
>> I get errors about duplicate/unique problems.  Essentially a row is 
>> attempted to be added twice. 
>>
>> IntegrityError: Problem installing fixture 'test-fixtures.json': 
>> Could not load app.Branch(pk=1): duplicate key value violates unique 
>> constraint "app_branch_name_49810fc21046d2e2_uniq"
>> DETAIL:  Key (name)=(mock) already exists.
>>
>> (I've also posted this earlier today on django-users, where I also 
>> included some postgres output).  The tests within the TestCase (or 
>> TransactionTestCase) can 

failure to load fixtures during unit tests

2016-05-04 Thread Rich Rauenzahn

I'm in the middle of trying to track down a problem with loading fixtures 
during unit tests -- I'm hesitant to call it a bug in Django 1.7, but the 
inconsistent behavior is really stumping me.

Essentially I've made a fixture via

   manage dumpdata --indent=3 -e sessions -e admin -e contenttypes -e 
auth.Permission > test-fixtures.json

If I add that fixtures to my TestCase, it sometimes works if I run each 
test individually (using Django Nose)  -- 

   manage test --failfast test_it:TestClass.test_detail
   manage test --failfast test_it:TestClass.test_list

But if I run them together, 

   manage test --failfast test_it:TestClass

I get errors about duplicate/unique problems.  Essentially a row is 
attempted to be added twice. 

IntegrityError: Problem installing fixture 'test-fixtures.json': Could 
not load app.Branch(pk=1): duplicate key value violates unique constraint 
"app_branch_name_49810fc21046d2e2_uniq"
DETAIL:  Key (name)=(mock) already exists.

(I've also posted this earlier today on django-users, where I also included 
some postgres output).  The tests within the TestCase (or 
TransactionTestCase) can be empty ("pass") and still reproduce.

As best I can tell it doesn't only happen when combined -- sometimes I can 
get it to happen in a class with a single TestCase.  And it isn't always 
the same model that has the conflict.

Has anyone seen anything like this behavior before?  It's as if sometimes 
the fixtures are installed in different order each time, which makes me 
think of some dict.keys() that doesn't return the same order every time.

Rich



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/37075277-2390-476c-9d32-3b3e594d2e4f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.