Re: [Django] #13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)

[Django]

#13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)
-+-
 Reporter:  chadc|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  contrib.admin|  Version:  master
 Severity:  Normal   |   Resolution:
 Keywords:  list_editable,   | Triage Stage:  Accepted
  admin, ForeignKey, admin   |
  efficiency |
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Aymeric Augustin):

 Using "autocomplete_fields" brings complexity down from O(m * n) to O(n).
 It still makes n database queries, but these queries return only 1 row
 instead of m.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/063.dd95a379c7924f6c724b148271d62e92%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)

[Django]

#13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)
-+-
 Reporter:  chadc|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  contrib.admin|  Version:  master
 Severity:  Normal   |   Resolution:
 Keywords:  list_editable,   | Triage Stage:  Accepted
  admin, ForeignKey, admin   |
  efficiency |
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by kissgyorgy):

 * version:  1.2 => master


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/063.b857aaa5eeb1041f859c1af88b687ff8%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)

[Django]

#13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)
-+-
 Reporter:  chadc|Owner:  nobody
 Type:  New feature  |   Status:  new
Component:  contrib.admin|  Version:  1.2
 Severity:  Normal   |   Resolution:
 Keywords:  list_editable,   | Triage Stage:  Accepted
  admin, ForeignKey, admin   |
  efficiency |
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by kissgyorgy):

 * cc: kissgyorgy@… (added)


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/063.a7bc8e9821a898d1b90d1a4be6601559%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)

[Django]

#13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)
-+-
   Reporter:  chadc  |  Owner:  nobody
   Type:  New| Status:  new
  feature|  Component:  contrib.admin
  Milestone: |   Severity:  Normal
Version:  1.2|   Keywords:  list_editable,
 Resolution: |  admin, ForeignKey, admin efficiency
   Triage Stage:  Accepted   |  Has patch:  0
Needs documentation:  0  |Needs tests:  0
Patch needs improvement:  0  |  Easy pickings:  0
-+-

Comment (by jdunck):

 Replying to [comment:5 uriel.bertoche]:
 > It also happens when you try to add a object that has a foreign key with
 too many possible values.
 > Like, if I have a table users, with 700 entries, and a models that has
 users as a foreign key, when I try adding an entry for this models, it
 will make +700 sql queries to fill the select.
 >
 > Is there any fix for this yet?

 That is fairly unrelated -- use raw_id_fields.  This issue is about
 list_editable.  The problem you're describing is on a different admin
 screen.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)

[Django]

#13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)
-+-
   Reporter:  chadc  |  Owner:  nobody
   Type:  New| Status:  new
  feature|  Component:  contrib.admin
  Milestone: |   Severity:  Normal
Version:  1.2|   Keywords:  list_editable,
 Resolution: |  admin, ForeignKey, admin efficiency
   Triage Stage:  Accepted   |  Has patch:  0
Needs documentation:  0  |Needs tests:  0
Patch needs improvement:  0  |  Easy pickings:  0
-+-
Changes (by uriel.bertoche):

 * easy:   => 0


Comment:

 It also happens when you try to add a object that has a foreign key with
 too many possible values.
 Like, if I have a table users, with 700 entries, and a models that has
 users as a foreign key, when I try adding an entry for this models, it
 will make +700 sql queries to fill the select.

 Is there any fix for this yet?

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)

[Django]

#13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)
-+-
   Reporter:  chadc  |Owner:  nobody
   Type:  New|   Status:  new
  feature|Component:  contrib.admin
  Milestone: | Severity:  Normal
Version:  1.2| Keywords:  list_editable, admin,
 Resolution: |  ForeignKey, admin efficiency
   Triage Stage:  Accepted   |Has patch:  0
Needs documentation:  0  |  Needs tests:  0
Patch needs improvement:  0  |
-+-
Changes (by graham_king):

 * type:   => New feature
 * severity:   => Normal


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)

[Django]

#13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)
---+
  Reporter:  chadc | Owner:  nobody 
   
Status:  new   | Milestone: 
   
 Component:  django.contrib.admin  |   Version:  1.2
   
Resolution:|  Keywords:  list_editable, 
admin, ForeignKey, admin efficiency
 Stage:  Accepted  | Has_patch:  0  
   
Needs_docs:  0 |   Needs_tests:  0  
   
Needs_better_patch:  0 |  
---+
Changes (by joni):

  * stage:  Unreviewed => Accepted

Comment:

 I can confirm this is actually happening.

 Regarding the additional filtering, maybe you need to filter the hosts
 list by some property of each Account.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)

[Django]

#13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)
---+
  Reporter:  chadc | Owner:  nobody 
   
Status:  new   | Milestone: 
   
 Component:  django.contrib.admin  |   Version:  1.2
   
Resolution:|  Keywords:  list_editable, 
admin, ForeignKey, admin efficiency
 Stage:  Unreviewed| Has_patch:  0  
   
Needs_docs:  0 |   Needs_tests:  0  
   
Needs_better_patch:  0 |  
---+
Comment (by chadc):

 Hmm. I had just intended to use CachedSelect as a temporary measure -- for
 my own purposes -- until the actual problem is addressed. I included it
 for the sake of anyone else who might be interested in a temporary fix. As
 I mentioned above, Jeremy seemed to indicate that there is some larger
 issue here of which I am not fully aware. I hope that someone with more
 knowledge of the problem might be willing to take a look.

 In any case:

 1. How could you implement additional filtering? I simply use
 formfield_overrides = {models.ForeignKey:{'widget':CachedSelect()}} on any
 AdminForm with an editable ForeignKey. I have no idea how any additional
 filtering could be implemented, so please let me know. Also, would it use
 the same naming convention?

 2. The cache is invalidated when it hits name == 'form-0-'.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)

[Django]

#13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)
---+
  Reporter:  chadc | Owner:  nobody 
   
Status:  new   | Milestone: 
   
 Component:  django.contrib.admin  |   Version:  1.2
   
Resolution:|  Keywords:  list_editable, 
admin, ForeignKey, admin efficiency
 Stage:  Unreviewed| Has_patch:  0  
   
Needs_docs:  0 |   Needs_tests:  0  
   
Needs_better_patch:  0 |  
---+
Changes (by patrys):

  * needs_better_patch:  => 0
  * needs_tests:  => 0
  * needs_docs:  => 0

Comment:

 Keep in mind the formset is using forms that can apply additional
 filtering to the data sets thus making each choice set unique. If you want
 to introduce some sort of caching, you pretty much have to cache by
 serialized query params. Also you need to make sure the cache is thread-
 safe and does not outlive the formset (your CachedSelect does not seem to
 ever invalidate the cache).

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

[Django] #13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)

[Django]

#13871: contrib.admin:list_editable - ForeignKey performance is O(m*n)
+---
 Reporter:  chadc   |   Owner:  
nobody
   Status:  new |   Milestone:  
  
Component:  django.contrib.admin| Version:  
1.2   
 Keywords:  list_editable, admin, ForeignKey, admin efficiency  |   Stage:  
Unreviewed
Has_patch:  0   |  
+---
 '''Overview'''

 Including ForeignKey fields in list_editable results in m*n database
 queries where m is the number of ForeignKey fields in list_editable and n
 is the number of rows in the changelist.

 '''Description'''

 I have been experiencing poor performance with the admin changelist when
 list_editable includes ForeignKey fields. In particular, rendering the
 changelist requires O(m*n) database queries where m is the number of
 ForeignKey fields included in list_editable and n is the number of rows in
 the changelist. The problem, as I understand it, stems from the fact that
 the choices for the ForeignKey widgets are not cached. So, when each
 ForeignKey widget is rendered, it queries the database to retrieve the
 list of possible values.

 My solution to this problem has been to override the Select widget with a
 CachedSelect widget in the admin model. However, as Jeremy Dunck noted in
 django-developers (link below), this may stem from a more general problem
 with ModelFormSet. I have ticketed this under contrib.admin for now, but I
 hope that Jeremy will update it as any larger issues become clear.

 '''Example'''

 {{{
 class Host(models.Model):
 name = models.CharField(max_length=128, unique=True)

 class Account(models.Model):
 host = models.ForeignKey(Host, related_name="accounts")
 name = models.CharField(max_length=128)

 class AccountAdmin(admin.ModelAdmin):
 list_display = ('name', 'host')
 list_editable = ('host',)
 list_per_page = 25
 admin.site.register(Account, AccountAdmin)
 }}}


 Rendering the ForeignKey widgets in this example requires n*m=25*1=25
 database queries:

 SELECT "hosts_host"."id",  "hosts_host"."name" FROM "hosts_host" ORDER BY
 "hosts_host"."name" ASC
 Total time: 330 ms
 Numer of queries: 25

 '''Related Links'''

 django-developers: http://groups.google.ca/group/django-
 developers/browse_thread/thread/76066baed6ba70dc

 django-users: http://groups.google.ca/group/django-
 users/browse_thread/thread/7b63fd40c891ec19

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-upda...@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.