W dniu sobota, 5 listopada 2016 18:40:24 UTC+1 użytkownik Shai Berger napisał: > > > 2. Dependency resolution is only stable as long as the migration set is > > frozen. Sometimes introducing a new migration is enough to break > existing > > migrations by causing them to execute in a slightly different order. We > > often have to backtrack and edit existing migrations and enforce a > strict > > resolution order by introducing arbitrary dependencies. > > > > So, you say you really have implicit dependencies between migrations -- > dependencies in substance, which aren't recorded as dependencies. This > seems > to indicate that you have a lot of manually-written migrations (data > migrations?), since the automatically-written ones do include relevant > dependencies. This seems odd -- it sounds like you're doing something out > of > the ordinary. > > This would also explain some of your bad experience with squashing -- > indeed, > if you have many data migrations, squashing can become much less > effective. >
Let's not come to conclusions prematurely. Django only supports predicate dependencies. You can say "not earlier than after these are applied" but that does not mean "immediately after they are applied". Sometimes Django tries to run the migration much later. If you have your models scattered across a large number of applications (we use apps to gateway entire classes of related features) sometimes the late migrations tries to reference a column in another model that was long since removed by a much later added migration in its respective app. > 3. Removing an app from a project is a nightmare. You can't migrate to > zero > > state unless the app is still there. There is no way to add "revert all > > migrations for app X" to the migration graph, it's something you need to > > run manually. There is no clean way to remove an app that was ever > > references in a relation. We were forced to do all kinds of hacks to get > > around this. Sometimes it's necessary to create an empty eggshell app > with > > the same name and copy all migrations there then add necessary data > > migrations and finally migrations that remove all the models, indices, > > procedures etc. Sometimes people just leave a dead application in > > INSTALLED_APPS to not have to deal with this. > > Clear out (maybe even remove) models.py and type "makemigrations", and you > get > a migration that deletes everything. The answer to getting rid of the > historical migrations is squashing, but of course you first need squashing > to > work properly. > I cannot clear out anything from an app that came from PyPI. That's why I mentioned creating fake empty apps that are just containers for their migration history. Squashing does nothing to help with that if you have another application reference any of those models. Squashing only helps you have fewer migrations. If the migrations were always in the correct order, the migration engine could collapse them automatically at execution time. > 4. Squashing migrations is wonky at best. If you create a model in one > > migration, alter one of its fields in another and then finally drop the > > model sometime later, the squashed migration will have Django try to > > execute the alter first and complain about the table not being there. > Also > > the only reason we need to squash migrations is to prevent problem 1 > above > > from becoming exponentially worse. If migrations were only as slow as > the > > underlying SQL commands, we'd likely never squash them. > > > > If that's so, it's a bug you should report; it's also an issue you can > work- > around by editing the migration to remove the redundant operation. There > are > issues with squashing, to be sure, but I don't think this is one of the > serious ones. > It's a bug that I will report at some point but I mostly encounter it in environments where I can't afford the time needed to properly debug. > 6. Conflict detection and resolution (migrate --merge) is a make-believe > > solution. It just trains people to execute the command without > > investigating whether their migration history still makes sense. > > It could be smarter, assuming it understood the content of migrations. We > could probably improve it to a point where, for most cases, it would > either > know to merge automatically or know that there really is a conflict. This > would > probably not help you if you have a lot of RunPython's in your migrations. > Depends on the project. I really don't care about the framework trying to reason about the results of a git merge. Django does not have enough understanding of the code and the version control history to do the job of the person responsible for the merge. Getting stuff right 60% of the time is not reliable enough to depend on it but is reliable enough to get some people lazy. > Some of these I need to dig deeper into and probably file proper tickets. > > For example I have an idea on how to fix 4 but it would make 1 even > slower. > > > > I took some time to get a good long look at what other ORMs are doing. > The > > graph-based dependency solving approach is rather uncommon. Most systems > > treat migrations as part of the project rather than the packages it > uses. > > > > > > Possible solution (or "how I'd build it today if there was no existing > code > > in Django core"): > > > > a. Make migrations part of the project and not individual apps. This > takes > > care of problem 3 above. > > > > So, there'd be no reason to link a migration to a specific app; quite the > contrary, it would become much more logical to have one migration include > operations for many apps. That could make the process of making an app > reusable while developing it in a project quite painful. > It's already possible to do whatever you like in a migration. They're only limited to modifying stuff within an app by convenience. You can also indent Python code using three tabs and two spaces but it does not automatically make it a good idea :) > b. Prefix individual migration files with a UTC timestamp > > (20161105151023_add_foo) to provide a strict sorting order. This removes > > the depsolving requirement and takes care of 1 and 2. By eliminating > those > > it makes 4 kind of obsolete as squashing migrations would become > pointless. > > > 4: No, on large databases, squashing migrations is not pointless. > > 1&2: Strict order has its issues: Currently, if I find a problem with the > last > migration of app A, I roll it back, fix it, and roll forward. With strict > order, I would have to roll back the project, not the app. > If anything depends on app A then you're already in the exact same situation. > c. Have reusable apps provide migration templates that Django then copies > > to my project when "makemigrations" is run. > > > > I'd like to see some more details about how this works; they would need to > include the development process of reusable apps. > See my other responses for some ideas. Again, I am not proposing a particular solution with readily available code and a DEP to support it. > > d. Maintain a separate directory for each database connection. > > Seems wrong as a blanket idea -- really depends on how the databases are > used. > I wouldn't want to find myself maintaining copies of migrations which are > supposed to run on more than one database. > See other responses for how that could be solved with changesets. > > e. Execute all migrations in alphabetical order (which means by > timestamp > > first). When an unapplied migration is followed by an applied one, ask > > whether to attempt to just apply it or if the user wants to first > unapply > > migrations that came after it. To me this would work better than 6. > > This sounds like a good way to create data losses. > That's a vague statement at best. > f. Migrating to a timestamp solves 5. > > Not really. Not with a team, since the timestamps will indicate not the > real > logical order, but the order of development. You'd need empty "tag" > migrations > to set points you want to migrate to... > I don't think it's a problem that spans multiple computers. You only switch feature branches in your own local checkout. Cheers, -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/508397b7-9c91-4bb3-8dbd-98a5a28187c2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
