Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  Julien Hartmann  |Owner:  Ian Foote
 Type:   |   Status:  closed
  Cleanup/optimization   |
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  fixed
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Nick Pope):

 The fix to this resulted in a regression where parameters passed to `__in`
 were reordered due to the use of `set()`. See #29503.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.a5436911a8486d19fe7d1164b2703d94%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  closed
  Cleanup/optimization   |
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:  fixed
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham ):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [changeset:"86eccdc8b67728d84440a46e5bf62c78f2eddf6d" 86eccdc8]:
 {{{
 #!CommitTicketReference repository=""
 revision="86eccdc8b67728d84440a46e5bf62c78f2eddf6d"
 Fixed #25544 -- Removed duplicate ids in prefetch_related() queries.
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.2626c35a0639bfec8fc8f71b64319a19%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  master
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by charettes):

 * needs_better_patch:  1 => 0
 * version:  1.8 => master


Comment:

 The comments have been addressed.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.fac4e404b9ebe9350de00712e2191ccb%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by timgraham):

 * needs_better_patch:  0 => 1
 * stage:  Ready for checkin => Accepted


Comment:

 Latest comments on the PR indicates a few more tests should be added.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.98dea3cadebed4bfe70f6b6c46347972%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by carljm):

 * stage:  Accepted => Ready for checkin


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.3871836bbaad8315b14f602c69e3f1e9%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Ian-Foote):

 * needs_better_patch:  1 => 0


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.d486f0bcee1f54d8c3fc8e2abe8d0137%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by charettes):

 Might be worth pinging Mark about this.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.6638ffcdf43bf0d61415e0974aecb33d%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by spectras):

 I do not know if it is related, but  ̀ArrayField.get_db_prep_value` always
 converts tuples to lists.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.f5c2b3556c9c48aeb2c3b53cc3f99ea2%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by akaariai):

 A possibility is to try-except the deduping of the values. If it doesn't
 succeed, just use the original values.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.d8860d914be6e9587e9a3b8eea64c112%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by carljm):

 @charettes I think that's a good idea! As long as we're confident that a)
 there are no other valid cases outside ArrayField where an unhashable
 value might be passed in an `__in` lookup rhs, and b) every value passed
 to an ArrayField lookup should always be iterable. But I can't come up
 with any counter-examples to either of those.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.399a1b59147e57594809a3bbc284358a%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by charettes):

 @carljm, what do you think of simply special casing `ArrayField`'s `__in`
 lookup which is something we already do for `__exact` anyway.

 Something like the following comes to mind.

 {{{#!python
 @ArrayField.register_lookup
 class ArrayInLookup(InLookup):
 def get_prep_lookup(self):
 values = super(ArrayInLookup, self).get_prep_lookup()
 return [tuple(value) for value in values]
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.1388c76c21d434353f274753c0496eb0%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-

Comment (by carljm):

 @IanFoote Oops, my fault. I didn't consider the possibility that non-
 hashable values (e.g. lists for an ArrayField) could be passed to `__in`
 lookups. I don't really see a way to preserve this fix in the face of that
 possibility (going through the value and trying to convert everything to
 something hashable doesn't seem like a good plan), so unless you have any
 ideas I'm not seeing, I think we may need to revert to your previous fix
 at the `prefetch_related` layer.

 Sorry for the bad direction!

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.82e206f9a6a9f0967a5a114c199aab9f%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by timgraham):

 * needs_better_patch:  0 => 1
 * has_patch:  0 => 1
 * stage:  Ready for checkin => Accepted


Comment:

 `postgres_tests.test_array.TestQuerying.test_in` fails.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.fcfd84f86c329a512b6ebbd09ae2889d%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Ready for
 |  checkin
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by carljm):

 * stage:  Accepted => Ready for checkin


Comment:

 Patch is ready, IMO. Waiting for actual merge until the CI catches up...

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.19addf9ba46c3f8614dceb5f413cc1ce%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  Ian-Foote
 Type:   |   Status:  assigned
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Ian-Foote):

 * owner:  nobody => Ian-Foote
 * status:  new => assigned


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.a43b7a135b84fad0ff8b44cc82292e41%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #25544: prefetch_related sends duplicate ids to database

[Django]

#25544: prefetch_related sends duplicate ids to database
-+-
 Reporter:  spectras |Owner:  nobody
 Type:   |   Status:  new
  Cleanup/optimization   |
Component:  Database layer   |  Version:  1.8
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:  prefetch duplicate   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by timgraham):

 * needs_better_patch:   => 0
 * stage:  Unreviewed => Accepted
 * type:  Uncategorized => Cleanup/optimization
 * needs_tests:   => 0
 * needs_docs:   => 0


Comment:

 I'm not too familiar with this, but accepting on the basis for further
 investigation.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.70dab7b5a48fd16ab850384a6a6df393%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.