#25930: Django-migrations & sqlite3 fail when migration-history is non-linear
-------------------------------+--------------------
Reporter: vanschelven | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 1.8
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------
1. We started seeing problems in our tests (which were the only place
where we used sqlite3), when running the tests with all migrations (as is
the default). The problem as it showed up: lots of tests failing,
complaining that a certain column wasn't existing in the DB. Despite the
fact that a migration existed to create the column, and despite the fact
that the migrations seemed fine.
2. As [stated in the
documentation](https://docs.djangoproject.com/en/1.9/topics/migrations/#sqlite),
Django doesnt' actually run alter table statements when altering sqlite
tables via migrations. Rather, it fully recreates the tables based on it
3. Consider the following graph (oldest migration at the bottom):
{{{
M
|
/ \
B C
\ /
|
A
}}}
For example's sake, let's assume `B` and `C` add columns to some table `b`
and `c` respectively. `M` does nothing, it's only there to provide a
single sink/target for django-migrations. I.e. it's a migration which only
consists of dependencies. Let's assume that Django-migrations chooses the
ordering `A` -> `B` -> `C` -> `M` (this is non-essential, the analogous
problem will show up with the other possible ordering (C before B).
4. As far as I understand, django-migrations keeps track of an internal
representation of the table after each migration; after migration `B` this
contains `b` but not `c`; after migration `C` this contains `c` but not
`b`; after migration `M` this contains both `b` and `c`.
5. Migrations `B` and `C` contain alterations, triggering a full
recreation of the table. (This is the behavior as stated in the
documentation, and I've also seen this in practice by printing all
generated queries)
6. At migration `C` this means: a full recreation of the table, but
without the column `b`. This is the source of problem (the problem being:
the complaints about column `b` not existing)
7. Migration `M` contains nothing, triggering nothing, even though django-
migrations has the right view of what the table should look like happen at
this point. (it has the merged view of both paths). This is the point in
time where the problem isn't solved, even though it could be.
8. Another look at point '7' reveals that it can be used for a workaround:
add a statement that amounts to "alter to the status quo", i.e. a
statement that looks like an alter statement but doesn't actually alter
anything. This statement will force a full recreation of the table,
containing both columns `b` and `c`. Even though I haven't deeply inpected
how django-migrations works, the workaround works. This causes me to think
that the hypothesis is indeed correct.
I've observed the above on Django 1.7; I have not checked other
installations for similar problems.
Observed in the wild, including the workaround; I do not currently have
the time available to provide an executable clean test which does not
refer to our production-code.
Still, I hope the write-up of the problem as I understand it is useful
--
Ticket URL: <https://code.djangoproject.com/ticket/25930>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/054.6c4de2f64364d2847afad0c6814f0550%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
