#26063: Regression in Django 1.9: SQLite can no longer handle more than 2000
values
in a "foo__in" filter
----------------------------------------------+--------------------
Reporter: rhertzog | Owner: nobody
Type: Bug | Status: new
Component: Database layer (models, ORM) | Version: 1.9
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
----------------------------------------------+--------------------
Commit 4f6a7663bcddffb114f2647f9928cbf1fdd8e4b5 introduced a regression in
Django 1.9 because you can no longer have a queryset of the form {{{
Model.objects.filter(foo__in=array) }}} with more than 2000 items in
{{{array}}}.
The above changes generates a “SELECT QUOTE(?), ..., QUOTE(?)” query with
as many values as items in the array. This will hit the SQLite limit
SQLITE_MAX_COLUMN which defaults to 2000 and can only be increased up to
32767.
Before this change we were only limited by SQLITE_MAX_VARIABLE_NUMBER
which is lower by default (999) but which can be increased to a much
higher value. In Debian for example we use a value of 250000.
While the latter limit was unreasonably low (it just costs a bit of memory
and you can expect to have many parameters), the former limit make much
more sense IMO and I don't expect distributions to override the upstream
value.
I'm not sure what's the best way forward. Possibly process parameters by
batch of 2000.
(This bug has been initially reported in https://bugs.debian.org/809211 )
--
Ticket URL: <https://code.djangoproject.com/ticket/26063>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/051.1c1ce8d386cffc0052854803ce1caa60%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
