Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2023-12-08 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  Yashas
 |  Donthi
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Yashas Donthi):

 * owner:  (none) => Yashas Donthi
 * status:  new => assigned


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018c4d68448f-bbdc3bd7-65e6-45e1-9fae-e8ceadf62e6e-00%40eu-central-1.amazonses.com.

Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2023-03-21 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  (none)
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Mariusz Felisiak):

 * owner:  Raphael Michel => (none)
 * status:  assigned => new


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/01070187038253d5-243c6227-c24a-4b21-9b05-def2c8067ddc-00%40eu-central-1.amazonses.com.

Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2017-06-22 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  Raphael
 |  Michel
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham):

 * needs_better_patch:  0 => 1


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/070.ab9c44bc28aa5e0f5236b9847eb5e1ad%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2017-06-21 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  Raphael
 |  Michel
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by François Freitag):

 * stage:  Ready for checkin => Accepted


Comment:

 As mentioned in the PR, I think there is one major issue:
 {{{prefetch_related}}} expects the results of the prefetch query to be
 unique.
 Splitting the query in batches do not enforce uniqueness and can lead to
 incorrect results. In short, if related objects are shared between several
 instances, when instances are split into batches, nothing prevents a
 related objects from appearing twice in the results. For example:

 {{{
 # Suppose there are 2 instances (A, B) and 2 related objects (1, 2)
 A -> 1
 B -> 2
 A -> 1
 }}}

 Prefetching in batches of 2 results in prefetch queries for: {{{[A, B]}}}
 then {{{[C]}}}.
 Which gives the following mapping of related objects: {{{[(A, 1), (B,
 2)]}}} then {{{[(A, 1)]}}}. The {{{rel_obj_cache}}} would then be:
 {{{#!python
 {
 (1,): [A, A],  # A should only be present once
 (2,): [B]
 }
 }}}

 A more detailed example is available on the PR.

 Maybe the way forward is to use a set or a dict to store the related
 queries' results (either for {{{all_related_objects}}} or for
 {{{rel_obj_cache}}}).

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/070.f55012750bf9db94abbbe57782d8dd29%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2017-06-20 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  Raphael
 |  Michel
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham):

 * stage:  Accepted => Ready for checkin


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/070.a627f0e6d016becb064e2d67a96e371f%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2017-06-14 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  Raphael
 |  Michel
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Raphael Michel):

 * needs_better_patch:  1 => 0


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/070.cf38f0ac76a36426b9aba9da19f4c267%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2017-06-13 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  Raphael
 |  Michel
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham):

 * needs_better_patch:  0 => 1


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/070.f1ccae55e74a134841d62356376f2070%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2017-04-17 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  Raphael
 |  Michel
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Raphael Michel):

 * needs_better_patch:  1 => 0


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/070.78102eae8c43b83a54c071358171c5fd%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2017-04-16 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  Raphael
 |  Michel
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by François Freitag):

 * needs_better_patch:  0 => 1


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/070.4d06012026a7bfd72aa6c84d6cf4cfb9%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2017-04-06 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  Raphael
 |  Michel
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Raphael Michel):

 * has_patch:  0 => 1


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/070.629ab1c15e6e1842b728036064edb394%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2017-04-06 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  Raphael
 |  Michel
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Raphael Michel):

 * status:  new => assigned
 * owner:  nobody => Raphael Michel


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/070.cb87e0aa0ae5c612a59cc487a2d12c14%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2017-02-13 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
 Reporter:  Jason Barnabe|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.10
  (models, ORM)  |
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham):

 * stage:  Unreviewed => Accepted


Old description:

> This is described in
> https://code.djangoproject.com/ticket/16937#comment:3, but I don't see
> any issue filed for it.
>
> "[prefetch_related] will not work on some backends if you have a lot of
> objects in your queryset. For example the SQLite backend has a limitation
> of 999 parameters to single SQL query, so if you have 1000 objects,
> prefetch_related will fail as you need to supply 1000 id values to the
> query."'
>
> Batch it up like in #16426 and #17788, which dealt with the same
> limitation?

New description:

 This is described in ticket:16937#comment:3, but I don't see any issue
 filed for it.

 "[prefetch_related] will not work on some backends if you have a lot of
 objects in your queryset. For example the SQLite backend has a limitation
 of 999 parameters to single SQL query, so if you have 1000 objects,
 prefetch_related will fail as you need to supply 1000 id values to the
 query."'

 Batch it up like in #16426 and #17788, which dealt with the same
 limitation?

--

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/070.3dd5d46747f3938ed19a9ec2d28cf4a8%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

[Django] #27833: prefetch_related fails with SQLite when used with 1000 parent records

2017-02-13 Thread Django 
#27833: prefetch_related fails with SQLite when used with 1000 parent records
-+-
   Reporter:  Jason  |  Owner:  nobody
  Barnabe|
   Type:  Bug| Status:  new
  Component:  Database   |Version:  1.10
  layer (models, ORM)|
   Severity:  Normal |   Keywords:
   Triage Stage: |  Has patch:  0
  Unreviewed |
Needs documentation:  0  |Needs tests:  0
Patch needs improvement:  0  |  Easy pickings:  0
  UI/UX:  0  |
-+-
 This is described in
 https://code.djangoproject.com/ticket/16937#comment:3, but I don't see any
 issue filed for it.

 "[prefetch_related] will not work on some backends if you have a lot of
 objects in your queryset. For example the SQLite backend has a limitation
 of 999 parameters to single SQL query, so if you have 1000 objects,
 prefetch_related will fail as you need to supply 1000 id values to the
 query."'

 Batch it up like in #16426 and #17788, which dealt with the same
 limitation?

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/055.a65effa9d9107acc72200ad9b8f99441%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.