Re: [Django] #25546: Duplicate queries on nested Prefetch objects with custom queries

[Django]

#25546: Duplicate queries on nested Prefetch objects with custom queries
-+-
 Reporter:  Benjamin Wohlwend|Owner:  François
 Type:   |  Freitag
  Cleanup/optimization   |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  fixed
 Keywords:  prefetch,| Triage Stage:  Ready for
  prefetch_related   |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Tim Graham ):

 In [changeset:"ec05ff086c0349e50c71b63c940a00a1d3f0385a" ec05ff08]:
 {{{
 #!CommitTicketReference repository=""
 revision="ec05ff086c0349e50c71b63c940a00a1d3f0385a"
 Refs #25546 -- Added detailed comments for prefetch_related test.
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.097adbeb37f1e3fc808b0f7b5effb35b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25546: Duplicate queries on nested Prefetch objects with custom queries

[Django]

#25546: Duplicate queries on nested Prefetch objects with custom queries
-+-
 Reporter:  piquadrat|Owner:
 Type:   |  francoisfreitag
  Cleanup/optimization   |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  fixed
 Keywords:  prefetch,| Triage Stage:  Ready for
  prefetch_related   |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham ):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [changeset:"bdbe50a491ca41e7d4ebace47bfe8abe50a58211" bdbe50a4]:
 {{{
 #!CommitTicketReference repository=""
 revision="bdbe50a491ca41e7d4ebace47bfe8abe50a58211"
 Fixed #25546 -- Prevented duplicate queries with nested
 prefetch_related().
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.fd156331d8338e7564165d050365b2cd%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25546: Duplicate queries on nested Prefetch objects with custom queries

[Django]

#25546: Duplicate queries on nested Prefetch objects with custom queries
-+-
 Reporter:  piquadrat|Owner:
 Type:   |  francoisfreitag
  Cleanup/optimization   |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch,| Triage Stage:  Ready for
  prefetch_related   |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by charettes):

 * needs_better_patch:  1 => 0
 * stage:  Accepted => Ready for checkin


Comment:

 Patch LGTM.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.53a2c12fae76be5366b766706b7ad5cd%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25546: Duplicate queries on nested Prefetch objects with custom queries

[Django]

#25546: Duplicate queries on nested Prefetch objects with custom queries
-+-
 Reporter:  piquadrat|Owner:
 Type:   |  francoisfreitag
  Cleanup/optimization   |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch,| Triage Stage:  Accepted
  prefetch_related   |
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by charettes):

 * needs_better_patch:  0 => 1
 * has_patch:  0 => 1


Comment:

 [https://github.com/django/django/pull/6002 PR]

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.96fd36d569075c89fd24a6e981d21397%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25546: Duplicate queries on nested Prefetch objects with custom queries

[Django]

#25546: Duplicate queries on nested Prefetch objects with custom queries
-+-
 Reporter:  piquadrat|Owner:
 Type:   |  francoisfreitag
  Cleanup/optimization   |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch,| Triage Stage:  Accepted
  prefetch_related   |
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by francoisfreitag):

 * owner:  nobody => francoisfreitag
 * status:  new => assigned


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.13180fcbab4eca02b261aab995dccf20%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25546: Duplicate queries on nested Prefetch objects with custom queries

[Django]

#25546: Duplicate queries on nested Prefetch objects with custom queries
-+-
 Reporter:  piquadrat|Owner:  nobody
 Type:   |   Status:  new
  Cleanup/optimization   |
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch,| Triage Stage:  Accepted
  prefetch_related   |
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by patryk4815):

 * cc: bux.patryk@… (added)


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.6bb98565745fc920d543c8afcbc26a9e%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25546: Duplicate queries on nested Prefetch objects with custom queries

[Django]

#25546: Duplicate queries on nested Prefetch objects with custom queries
-+-
 Reporter:  piquadrat|Owner:  nobody
 Type:   |   Status:  new
  Cleanup/optimization   |
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch,| Triage Stage:  Accepted
  prefetch_related   |
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by timgraham):

 * type:  Bug => Cleanup/optimization
 * stage:  Unreviewed => Accepted


Comment:

 Accepting for further investigation.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.d1d062a8f2bb3d4c632c4a1493a250b4%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25546: Duplicate queries on nested Prefetch objects with custom queries

[Django]

#25546: Duplicate queries on nested Prefetch objects with custom queries
-+-
 Reporter:  piquadrat|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch,| Triage Stage:
  prefetch_related   |  Unreviewed
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by timgraham):

 * needs_better_patch:   => 0
 * needs_tests:   => 0
 * needs_docs:   => 0


Comment:

 Any chance you can suggest a fix (or at least provide a minimal test that
 reuses models in Django's test suite)? Otherwise, it might take me a while
 to triage the issue to determine if it's valid and something we can fix.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.ef1d5a6b891679bf79a318f22eebc108%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25546: Duplicate queries on nested Prefetch objects with custom queries

[Django]

#25546: Duplicate queries on nested Prefetch objects with custom queries
--+
 Reporter:  piquadrat |  Owner:  nobody
 Type:  Bug   | Status:  new
Component:  Database layer (models, ORM)  |Version:  master
 Severity:  Normal| Resolution:
 Keywords:  prefetch, prefetch_related|   Triage Stage:  Unreviewed
Has patch:  0 |  Easy pickings:  0
UI/UX:  0 |
--+
Changes (by piquadrat):

 * Attachment "prefetch_test.tar.gz" added.

 test project

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.2e868b22f12382b6a0d4a7e0cbe50d79%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

[Django] #25546: Duplicate queries on nested Prefetch objects with custom queries

[Django]

#25546: Duplicate queries on nested Prefetch objects with custom queries
-+-
 Reporter:  piquadrat|  Owner:  nobody
 Type:  Bug  | Status:  new
Component:  Database layer   |Version:  master
  (models, ORM)  |   Keywords:  prefetch,
 Severity:  Normal   |  prefetch_related
 Triage Stage:  Unreviewed   |  Has patch:  0
Easy pickings:  0|  UI/UX:  0
-+-
 Assuming these models

 {{{
 from django.db import models

 class FastfoodChain(models.Model):
 name = models.CharField(primary_key=True, max_length=100)


 class Restaurant(models.Model):
 name = models.CharField(primary_key=True, max_length=100)
 chain = models.ForeignKey(FastfoodChain)


 class RestaurantProfile(models.Model):
 user = models.ForeignKey('auth.User')
 restaurant = models.ForeignKey(Restaurant)

 is_favorite = models.BooleanField(default=False)
 }}}

 the following test fails

 {{{
 from django.contrib.auth.models import User
 from django.db.models.query import Prefetch
 from django.test import TestCase

 from fastfood import models

 class PrefetchTestCase(TestCase):
 def test_prefetch(self):
 user = User.objects.create(username='test')
 chain1, chain2, chain3, chain4 = [
 models.FastfoodChain.objects.create(name='chain1'),
 models.FastfoodChain.objects.create(name='chain2'),
 models.FastfoodChain.objects.create(name='chain3'),
 models.FastfoodChain.objects.create(name='chain4'),
 ]
 restaurant11, restaurant12, restaurant21, restaurant31,
 restaurant32, restaurant41 = [
 models.Restaurant.objects.create(chain=chain1,
 name='restaurant11'),
 models.Restaurant.objects.create(chain=chain1,
 name='restaurant12'),
 models.Restaurant.objects.create(chain=chain2,
 name='restaurant21'),
 models.Restaurant.objects.create(chain=chain3,
 name='restaurant31'),
 models.Restaurant.objects.create(chain=chain3,
 name='restaurant32'),
 models.Restaurant.objects.create(chain=chain4,
 name='restaurant41'),
 ]
 p1, p2, p3, p4 = [
 models.RestaurantProfile.objects.create(restaurant=restaurant11,
 user=user, is_favorite=True),
 models.RestaurantProfile.objects.create(restaurant=restaurant12,
 user=user, is_favorite=False),
 models.RestaurantProfile.objects.create(restaurant=restaurant21,
 user=user, is_favorite=True),
 models.RestaurantProfile.objects.create(restaurant=restaurant31,
 user=user, is_favorite=True),
 ]
 with self.assertNumQueries(3):
 chains = models.FastfoodChain.objects.filter(
 name__in=['chain1', 'chain2', 'chain4']
 ).prefetch_related(
 Prefetch(
 'restaurant_set',
 models.Restaurant.objects.prefetch_related(
 Prefetch(
 'restaurantprofile_set',
 models.RestaurantProfile.objects.filter(
 user=user,
 is_favorite=True,
 ),
 to_attr='profiles',
 )
 ),
 to_attr='restaurants',
 ),
 )
 result_chain_1, result_chain_2, result_chain_4 = list(chains)
 self.assertListEqual(result_chain_1.restaurants,
 [restaurant11, restaurant12])
 self.assertListEqual(result_chain_2.restaurants,
 [restaurant21])
 self.assertListEqual(result_chain_4.restaurants,
 [restaurant41])

 self.assertListEqual(result_chain_1.restaurants[0].profiles,
 [p1])
 self.assertListEqual(result_chain_1.restaurants[1].profiles,
 [])
 self.assertListEqual(result_chain_2.restaurants[0].profiles,
 [p3])
 self.assertListEqual(result_chain_4.restaurants[0].profiles,
 [])
 }}}

 With this output

 {{{
 ==
 FAIL: test_prefetch (fastfood.tests.PrefetchTestCase)
 --
 Traceback (most recent call last):
   File "/Users/benjaminwohlwend/projects/prefetch_test/fastfood/tests.py",
 line 57, in test_prefetch
 self.assertListEqual(result_chain_4.restaurants[0].profiles, [])
   File "/Users/benjaminwohlwend/.virtualenvs/prefetch-test/lib/python2.7
 /site-packages/django/test/testcases.py", line 93, in __exit__
 query['sql'] for query in self.captured_queries
 AssertionError: 4 queries executed, 3 expected
 Captured queries were:
 SELECT "fastfood_fastfoodchain"."name" FROM "fastfood_fastfoodchain" WHERE