#17788: bulk_create() "too many SQL variables" error
-------------------------------------+-------------------------------------
Reporter: alpar | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version:
(models, ORM) | 1.4-beta-1
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by akaariai):
Yes, it is a somewhat dangerous operation. It will seem to work in testing
but crash in production. For that reason it would be good to have some
(large) limits for other backends which have query size limits, too. MySQL
has no practical limit (16MB by default), Oracle has a limit of 64K which
you could actually hit if you try hard enough.
It is notable that this same problem is present when you do
`qs.filter(pk__in=[large_list])` and that one can not be fixed in any
transparent way.
It could be possible to have a simple fix for sqlite3: just split the
inserts so that they have less than 1000 parameters each. No changes for
other backends, no ability to override the batch size. It would fix this
ticket's immediate problem, and could perhaps get into 1.4.
I asked pgsql-performance if there is any wisdom about preferred batch
sizes. I suspect there is no point at which the performance will
predictably collapse, so there is no need for default batch size for
PostgreSQL. But lets see if something pops up. The thread is this one:
http://archives.postgresql.org/pgsql-performance/2012-02/msg00265.php
Maybe I should open another ticket for the bigger changes? That way this
ticket could be dedicated to just fixing the SQLite3 problem.
--
Ticket URL: <https://code.djangoproject.com/ticket/17788#comment:6>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
