#33647: bulk_update silently truncating values for size limited fields
-------------------------------------+-------------------------------------
Reporter: jerch | Owner: Akash
| Kumar Sen
Type: Bug | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
> I believe this is now also happening with `bulk_create` after changing
the Postgres strategy to use `UNNEST`
That would make sense as
[https://github.com/django/django/blob/bbabbac936caf6db129427e6e65f03b6d0a68f62/django/db/backends/postgresql/compiler.py#L56
we use] `Field.db_type` in the same way to generate the vector of data. I
think this could qualify as a backport for 5.2 given it can result in
silent data losses but I'd like to hear a fellow on that.
> I would be interested it taking on this issue if that would be
appropriate.
Sure thing, make sure you're fully up-to-date with the discussions here
though. One way to approach the problem would be to change `Field.db_type`
signature to something like `(connection, parametrized=True)` and have
both `bulk_update` and the PostgreSQL `UNNEST` strategy for `bulk_create`
call `db_type` with `parametrized=False`.
Obviously we can't backport such a change to 5.2 and 6.0 so what we could
do in a potential backport at first is adjust `assemble_as_sql` to do
something like `field.db_type(connection).split("(")[0]`
[https://github.com/django/django/blob/bbabbac936caf6db129427e6e65f03b6d0a68f62/django/db/backends/postgresql/compiler.py#L56
when building] `db_types`.
We could then make adjusting in a following commit targeting `main` by
introducing a `parametrized=False` kwarg in `db_type` with a deprecation
period and use that in `bulk_update` and Postgres `bulk_create`.
FWIW you might also be interested in #14094 and #34887 which respectively
added support for unparametrized `varchar` (unlimited length) on Postgres
and SQLite and the more recent #24920 which added the same for
`DecimalField` (backed by `NUMERIC` which is affected by the same root
problem as the one discussed here for `CharField`) on all backends but
MySQL. I bringing the `DecimalField` example because on MySQL `DECIMAL`
without parameters [https://dbfiddle.uk/aI0yqMQA appears to be] an alias
for `DECIMAL(10, 0)` so we likely will have to special case some cases
here and there to have `parametrized=False` mean ''the loosest type''.
--
Ticket URL: <https://code.djangoproject.com/ticket/33647#comment:28>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/0107019b068c8a95-ecd57d84-23b9-4934-8d1c-b37ba1274aa1-000000%40eu-central-1.amazonses.com.
