#17788: bulk_create() "too many SQL variables" error
-------------------------------------+-------------------------------------
Reporter: alpar | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version:
(models, ORM) | 1.4-beta-1
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by akaariai):
Attached is a patch that fixes the bulk_create issue. I used
get_batch_size which can inspect the fields and the objects in the query.
The patch also implements a default limit of 100000 objects in one batch.
Dropping one zero from the end would be wise, or if you prefer no limit,
thats OK too for me. It would be trivial to implement batch_size kwarg for
bulk_create if that is wanted down the road.
In addition to the .delete() there is also prefetch_related which doesn't
work if the inner sets are large enough (note: haven't actually tested).
My opinion is that these aren't critical issues, but it would be nice to
abstract the backend differences away if possible. However there is going
to be always one issue: `id__in` queries can not be abstracted (splitting
the query breaks sorting), so there will always be cases where SQLite
doesn't work correctly.
I noticed one issue with the bulk_create feature which might need some
attention. It uses some effort to allow mixing of pk set / pk not set
objects in one batch. In my opinion that is downright dangerous to allow,
as the pk field must be autofield for the insert to work. Saving self-
defined values to an autofield is something to be avoided, as that can
break the sequence. It is fine to do bulk inserts with autofield PK set.
Assume the user knows what he is doing, maybe he is going to reset the PK
sequence after the operation or something. But I can't see any valid use
case for mixing PK set and PK not set objects in one batch. The last added
test in the patch doesn't work on SQLite for the very reason that mixing
things in this way isn't safe. So, my question is, should this be
disallowed? After 1.4 is released that can't be done. This isn't anything
critical IMHO, but it would be good to discourage unsafe coding practices.
--
Ticket URL: <https://code.djangoproject.com/ticket/17788#comment:17>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-updates?hl=en.
