As far as how well apps without migrations would work compared to 1.8, a couple things come to mind:
1. There's no management commands to get the SQL for these apps. 2. There's no more support for initial data/custom SQL. On Monday, August 31, 2015 at 10:48:25 AM UTC-4, Tim Graham wrote: > > In the absence of a more sophisticated implementation, here's a possible > simple solution: https://github.com/django/django/pull/5212 > > Maybe we can come up with a better name for the migrate option (currently > --run-syncdb). > > On Friday, August 28, 2015 at 3:08:26 AM UTC-4, Markus Holtermann wrote: >> >> Hey Anssi, >> >> thanks for the ideas. A few comments inline >> >> On Friday, August 28, 2015 at 3:25:34 PM UTC+10, Anssi Kääriäinen wrote: >>> >>> Could we allow applications to have fake migration files? I'm thinking >>> of something like operations.AllModelsUpdated(). After this operation >>> is ran, Django will use the present models for that app. You could >>> place this as your last migration, and run direct SQL in previous >>> migrations (or use some completely different mechanism) . Direct SQL >>> without model state changes will be fast. The last migration would >>> supposedly be fast, as it only needs to construct up to date images of >>> the current models. >>> >> >> That is something you can already do by using `SeparateDatabaseAndState` >> https://docs.djangoproject.com/en/1.8/ref/migration-operations/#django.db.migrations.operations.SeparateDatabaseAndState. >> >> Use RunSQL for the Database part and CreateModel for the state part. >> >> The problem is that I'm not at all sure how this will mix with >>> migrations from other applications, so this proposal is perhaps >>> technically impossible to do. But if this is feasible, then it should >>> be possible to mix migrations and manual database changes pretty much >>> seamlessly. This one would work much better if we had project level >>> migrations - implementing a migration operation that tells Django that >>> all of the project's models are up to date is almost trivial. >>> >> >> I don't see how project level migrations would make that easier. Unless >> you have only changes to the data in the database Django won't change any >> SQL and will happily reuse the latest models up to that state. >> >> I believe a fake migration would work pretty well for the use case >>> where you want to use manual database control for your project, but >>> use 3rd party applications with migrations. >>> >>> Another idea I've been thinking is that maybe we could have snapshots >>> for migrations. The reason I believe this would work is that building >>> the model state is expensive, but in many cases it is completely >>> redundant to build the model state from scratch. >>> >>> The snapshots would essentially do the following operation: after >>> migrations X, Y and Z are applied the model state will be as presented >>> in this migration. The snapshots would only contain model state, not >>> any operations. When you run manage.py migrate, the app state builder >>> will be able to skip to latest snapshot that has all dependencies >>> already applied, and then continue building model state from there. >>> When performance begins to degrade, you can create a new snapshot. The >>> snapshots would probably have to be project level to work properly. >>> This would seriously speed up migrations, except for the case where >>> you run the migrations against fresh database. >>> >> >> You are only half right: the performance problem is the *rendering* of >> the models from the model states. However I don't see how having >> checkpoints would help, reusing the state. It would help to not need to >> create the state up until that point, but the performance issue there got >> resolved I think. The main problem is models having 'object references' to >> other models (`models.ForeignKey(User)` vs >> `models.ForeignKey('auth.User')`). In order to have consistent references >> between two model classes (i.e. >> `ModelA._meta.get_field('fk_to_modelb').related_model is ModelB`) all >> related models have to be rerendered once one changes: >> https://github.com/django/django/blob/master/django/db/migrations/state.py#L32 >> >> >>> - Anssi >>> >> >> /Markus >> >> >>> On Fri, Aug 28, 2015 at 3:18 AM, Andrew Godwin <and...@aeracode.org> >>> wrote: >>> > I think I agree with Markus that we do not yet have a good enough >>> > performance story to make migrations mandatory. If you want to ditch >>> them >>> > and do your own database management, we should let you - which we >>> could do >>> > with some kind of MIGRATIONS_MODULES opt-out that basically sets >>> everything >>> > to pseudo-unmanaged, but I see no reason not to have the syncdb-like >>> > solution keep working as it at least runs off of SchemaEditor now. >>> > >>> > Andrew >>> > >>> > On Thu, Aug 27, 2015 at 5:14 PM, Markus Holtermann >>> > <in...@markusholtermann.eu> wrote: >>> >> >>> >> The general benefit of *not* having migrations is the incredible >>> better >>> >> performance to setup the tables. Especially for larger projects this >>> is >>> >> something we should at least keep in mind (see >>> >> https://code.djangoproject.com/ticket/24743). I spent a fair bit of >>> time on >>> >> the issue and have not yet found a satisfying solution. >>> >> >>> >> Supporting apps without migrations would essentially continue to >>> support >>> >> the old `syncdb` behavior, though not as a standalone command, but as >>> a flag >>> >> for `manage.py migrate`. It takes the models from those apps and >>> calls >>> >> `schema_editor.create_model()`, taking the deferred SQL and running >>> it after >>> >> all models of apps w/o migrations have been created (current behavior >>> [in >>> >> tests] I think). This would continue to raise issues if models in >>> apps w/o >>> >> migrations refer to models in apps w/ migrations (missing >>> constraints), but >>> >> that is a limitation we had since 1.7 (silent failure; 1.8 fails >>> loud). >>> >> >>> >> While migrations should provide *the* way to maintain your database I >>> see >>> >> the points that have been made in other recent discussions. Although >>> Django >>> >> doesn't force you to use migrations there are some things (e.g. not >>> create >>> >> the `django_migrations` table) we probably should not do unless you >>> actually >>> >> use migrations. >>> >> >>> >> We might want to consider looking at the `MIGRATION_MODULES` settings >>> >> variable and allowing to set values to `None` in order to disable >>> migrations >>> >> for that app, but that's a different story. >>> >> >>> >> /Markus >>> >> >>> >> >>> >> On Friday, August 28, 2015 at 4:26:19 AM UTC+10, Tim Graham wrote: >>> >>> >>> >>> It would be great if we could decide this by next Friday (September >>> 4) or >>> >>> so, so that we have some time to do the necessary implementations >>> and >>> >>> testing before alpha. (For example, running manage.py test without >>> >>> migrations currently gives "no such table ...". It would be nice to >>> give a >>> >>> friendlier message if we require migrations for running tests >>> >>> (https://github.com/django/django/pull/5100 does this to some >>> extent, but >>> >>> could be improved I think). >>> >>> >>> >>> I know at least Markus said he would like to present an argument for >>> >>> keeping support for apps without migrations. >>> >>> >>> >>> On Thursday, August 13, 2015 at 1:56:05 PM UTC-4, Tim Graham wrote: >>> >>>> >>> >>>> It's a one line change in the code to keep the status quo from 1.8. >>> The >>> >>>> main drawback I see is continued support tickets about dependencies >>> between >>> >>>> apps with and without migrations. I already removed the >>> documentation >>> >>>> describing the problem and "won't fixed" tickets that suggested >>> doing more >>> >>>> to detect and throw errors about these cases. >>> >>>> >>> >>>> I guess I'd like here why migrations could be a hindrance "for >>> quick >>> >>>> prototyping and initial development." I'd argue that starting with >>> >>>> migrations from the start (manage.py makemigrations && manage.py >>> migrate) >>> >>>> will save the need to learn about --fake-initial if you decide to >>> add >>> >>>> migrations at some point later (and of course avoid dependency >>> issues). Is >>> >>>> "manage.py makemigrations && manage.py migrate" more difficult that >>> the >>> >>>> proposed ""manage.py migrate --allow-apps-without-migrations"? Am I >>> missing >>> >>>> a motive behind the request? >>> >>>> >>> >>>> diff --git a/django/core/management/commands/migrate.py >>> >>>> b/django/core/management >>> >>>> index 2f04824..dec86e1 100644 >>> >>>> --- a/django/core/management/commands/migrate.py >>> >>>> +++ b/django/core/management/commands/migrate.py >>> >>>> @@ -133,7 +133,7 @@ class Command(BaseCommand): >>> >>>> targets = executor.loader.graph.leaf_nodes() >>> >>>> >>> >>>> plan = executor.migration_plan(targets) >>> >>>> - run_syncdb = options.get('run_syncdb') and >>> >>>> executor.loader.unmigrated_a >>> >>>> + run_syncdb = bool(executor.loader.unmigrated_apps) >>> >>>> >>> >>>> # Print some useful info >>> >>>> if self.verbosity >= 1: >>> >>>> >>> >>>> On Thursday, August 6, 2015 at 9:50:10 PM UTC-4, Markus Holtermann >>> >>>> wrote: >>> >>>>> >>> >>>>> Instead of rushing to a decision if and of how we continue to >>> support >>> >>>>> "apps w/o migrations" I would argue that, since Django's test >>> suite >>> >>>>> does >>> >>>>> infact create tables for apps w/o migrations, we could add a >>> >>>>> commandline >>> >>>>> flag (e.g. --run-syncdb or whatever) to the migrate command and >>> warn >>> >>>>> the >>> >>>>> user if there are apps w/o migrations an the user doesn't >>> supplied: >>> >>>>> >>> >>>>> >>> https://github.com/django/django/blob/master/django/core/management/commands/migrate.py#L136 >>> >>> >>>>> >>> >>>>> /Markus >>> >>>>> >>> >>>>> On Thu, Aug 06, 2015 at 05:26:50PM -0700, Andrew Godwin wrote: >>> >>>>> >Do we know exactly what "support for apps without migrations" >>> would >>> >>>>> > consist >>> >>>>> >of at this point? I have that half-done code for replicating >>> syncdb >>> >>>>> > with >>> >>>>> >the autodetector stuck onto the migration executor, but it's not >>> >>>>> > especially >>> >>>>> >speedy and would need some work to make it a sensible speed >>> before it >>> >>>>> > goes >>> >>>>> >forward. >>> >>>>> > >>> >>>>> >Alternately, we could modify the autodetector to have a "create >>> only" >>> >>>>> > mode >>> >>>>> >that's more efficient or just write a new generator that is quick >>> >>>>> > because >>> >>>>> >it knows it's only making nonmigrated apps' migrations. >>> >>>>> > >>> >>>>> >Andrew >>> >>>>> > >>> >>>>> >On Thu, Aug 6, 2015 at 4:56 PM, Tim Graham <timog...@gmail.com> >>> wrote: >>> >>>>> > >>> >>>>> >> With the alpha for 1.9 coming up in 6 weeks, we need to decide >>> >>>>> >> whether or >>> >>>>> >> not to continue support for apps without migrations (currently >>> in >>> >>>>> >> master no >>> >>>>> >> tables are created for such apps (as the deprecation timeline >>> says), >>> >>>>> >> but it >>> >>>>> >> might be appropriate to add a warning or error message for this >>> case >>> >>>>> >> if >>> >>>>> >> things don't change). >>> >>>>> >> >>> >>>>> >> Is anyone sufficiently interested in continuing support for >>> apps >>> >>>>> >> without >>> >>>>> >> migrations to commit to completing it by the alpha? If not, do >>> you >>> >>>>> >> consider >>> >>>>> >> it a release blocker that I should spend time on? >>> >>>>> >> >>> >>>>> >> Related tickets: >>> >>>>> >> >>> >>>>> >> https://code.djangoproject.com/ticket/25144 - No way to create >>> >>>>> >> tables for >>> >>>>> >> apps without migrations >>> >>>>> >> https://code.djangoproject.com/ticket/24919 - Add an option >>> not to >>> >>>>> >> run >>> >>>>> >> migrations when running tests >>> >>>>> >> https://code.djangoproject.com/ticket/24481 - Improve >>> sqlmigrate to >>> >>>>> >> be >>> >>>>> >> more flexible and allow bypassing migrations on disk >>> >>>>> >> https://code.djangoproject.com/ticket/24901 - makemigrations >>> should >>> >>>>> >> create empty migrations dir for any installed app without it >>> >>>>> >> https://code.djangoproject.com/ticket/24588 - Improve handling >>> apps >>> >>>>> >> without migrations while running migrate command. >>> >>>>> >> >>> >>>>> >> On Sunday, January 18, 2015 at 5:11:53 PM UTC-5, Markus >>> Holtermann >>> >>>>> >> wrote: >>> >>>>> >>> >>> >>>>> >>> Creating in-memory migrations for all apps that don't have >>> >>>>> >>> migration >>> >>>>> >>> files seems to be an option to solve the dependency problem. >>> This >>> >>>>> >>> would >>> >>>>> >>> even allow apps without migrations to depend on those with >>> >>>>> >>> migrations. >>> >>>>> >>> >>> >>>>> >>> We have to consider though, that there are tens of apps and >>> >>>>> >>> hundreds of >>> >>>>> >>> models in our own test suite, and generating all migrations >>> during >>> >>>>> >>> start >>> >>>>> >>> seems to be quite an expensive task. And I'm not even talking >>> about >>> >>>>> >>> the >>> >>>>> >>> migration optimizer which probably needs to get a lot smarter >>> if we >>> >>>>> >>> take >>> >>>>> >>> this road. >>> >>>>> >>> >>> >>>>> >>> syncdb, which leaves a developer with a database scheme that >>> cannot >>> >>>>> >>> be >>> >>>>> >>> altered automatically, is something we should get rid of as >>> soon as >>> >>>>> >>> possible, especially since Django has a out-of-the-box >>> migration >>> >>>>> >>> system. >>> >>>>> >>> >>> >>>>> >>> /Markus >>> >>>>> >>> >>> >>>>> >>> On Sun, Jan 18, 2015 at 01:50:14PM -0800, Andrew Godwin wrote: >>> >>>>> >>> >My main argument for removing them entirely was the >>> dependency >>> >>>>> >>> > issues >>> >>>>> >>> >between apps with and without migrations. Having syncdb use >>> >>>>> >>> > SchemaEditor >>> >>>>> >>> is >>> >>>>> >>> >a big step and one I'm happy we've got to, but the only >>> advantage >>> >>>>> >>> > of >>> >>>>> >>> >keeping syncdb is for the test suite and I'd rather we >>> approach >>> >>>>> >>> > that >>> >>>>> >>> more >>> >>>>> >>> >as "migrations made and run at runtime as a special case" >>> rather >>> >>>>> >>> > than >>> >>>>> >>> >"syncdb". >>> >>>>> >>> > >>> >>>>> >>> >If nothing else, I'd like to see the end-developer-facing >>> parts, >>> >>>>> >>> > like >>> >>>>> >>> the >>> >>>>> >>> >syncdb command itself, gone. >>> >>>>> >>> > >>> >>>>> >>> >Andrew >>> >>>>> >>> > >>> >>>>> >>> >On Sun, Jan 18, 2015 at 1:43 PM, Claude Paroz < >>> cla...@2xlibre.net> >>> >>>>> >>> wrote: >>> >>>>> >>> > >>> >>>>> >>> >> Tim recently did a fabulous job of removing deprecated code >>> for >>> >>>>> >>> >> the >>> >>>>> >>> >> future 1.9 on master. Thanks for that. >>> >>>>> >>> >> >>> >>>>> >>> >> However, one thing he removed was support for apps without >>> >>>>> >>> >> migrations. >>> >>>>> >>> >> >>> >>>>> >>> >> >>> >>>>> >>> >>> >>>>> >>> >>> https://github.com/django/django/commit/7e8cf74dc74539f40f4cea53c1e8bba82791fcb6 >>> >>> >>>>> >>> >> >>> >>>>> >>> >> Considering that we have to keep internal support for >>> Django's >>> >>>>> >>> >> own >>> >>>>> >>> test >>> >>>>> >>> >> suite anyway, I wonder if we should remove that support at >>> all >>> >>>>> >>> >> for >>> >>>>> >>> >> "normal" projects. I think one of Andrew's motivation was >>> not to >>> >>>>> >>> >> have >>> >>>>> >>> to >>> >>>>> >>> >> keep two schema editing code bases. But now that the old >>> syncdb >>> >>>>> >>> >> also >>> >>>>> >>> >> uses the new schema editor, I think that this argument >>> doesn't >>> >>>>> >>> >> stand. >>> >>>>> >>> >> >>> >>>>> >>> >> So what about keeping support for apps without migrations >>> in the >>> >>>>> >>> longer >>> >>>>> >>> >> term. Of course, the fact that those apps cannot depend on >>> >>>>> >>> >> migrated >>> >>>>> >>> apps >>> >>>>> >>> >> limits its use, but I think that for quick prototyping and >>> >>>>> >>> >> initial >>> >>>>> >>> >> developement, migrations could be more of a hindrance. >>> Would you >>> >>>>> >>> >> see >>> >>>>> >>> >> major drawbacks with keeping this support? >>> >>>>> >>> >> >>> >>>>> >>> >> Opinions welcome. >>> >>>>> >>> >> >>> >>>>> >>> >> Claude >>> >>>>> >>> >> -- >>> >>>>> >>> >> www.2xlibre.net >>> >>>>> >>> >>> >>>>> >> -- >>> >>>>> >> You received this message because you are subscribed to the >>> Google >>> >>>>> >> Groups >>> >>>>> >> "Django developers (Contributions to Django itself)" group. >>> >>>>> >> To unsubscribe from this group and stop receiving emails from >>> it, >>> >>>>> >> send an >>> >>>>> >> email to django-develop...@googlegroups.com. >>> >>>>> >> To post to this group, send email to >>> django-d...@googlegroups.com. >>> >>>>> >> Visit this group at >>> >>>>> >> http://groups.google.com/group/django-developers. >>> >>>>> >> To view this discussion on the web visit >>> >>>>> >> >>> >>>>> >> >>> https://groups.google.com/d/msgid/django-developers/89839628-6b11-4c61-86ed-4d9a076e9e3d%40googlegroups.com >>> >>> >>>>> >> >>> >>>>> >> < >>> https://groups.google.com/d/msgid/django-developers/89839628-6b11-4c61-86ed-4d9a076e9e3d%40googlegroups.com?utm_medium=email&utm_source=footer> >>> >>> >>> >>>>> >> . >>> >>>>> >> >>> >>>>> >> For more options, visit https://groups.google.com/d/optout. >>> >>>>> >> >>> >>>>> > >>> >>>>> >-- >>> >>>>> >You received this message because you are subscribed to the >>> Google >>> >>>>> > Groups "Django developers (Contributions to Django itself)" >>> group. >>> >>>>> >To unsubscribe from this group and stop receiving emails from it, >>> send >>> >>>>> > an email to django-develop...@googlegroups.com. >>> >>>>> >To post to this group, send email to django-d...@googlegroups.com. >>> >>> >>>>> >Visit this group at >>> http://groups.google.com/group/django-developers. >>> >>>>> >To view this discussion on the web visit >>> >>>>> > >>> https://groups.google.com/d/msgid/django-developers/CAFwN1urojSqXkaDnjVdrPHrs5TKpQC-baz9TZLtvOZ_3UJ%2BM2Q%40mail.gmail.com. >>> >>> >>> >>>>> >For more options, visit https://groups.google.com/d/optout. >>> >>>>> >>> >>>>> -- >>> >> >>> >> -- >>> >> You received this message because you are subscribed to the Google >>> Groups >>> >> "Django developers (Contributions to Django itself)" group. >>> >> To unsubscribe from this group and stop receiving emails from it, >>> send an >>> >> email to django-develop...@googlegroups.com. >>> >> To post to this group, send email to django-d...@googlegroups.com. >>> >> Visit this group at http://groups.google.com/group/django-developers. >>> >>> >> To view this discussion on the web visit >>> >> >>> https://groups.google.com/d/msgid/django-developers/b079f398-92fb-43e4-8ab5-6c1c8dbc9b09%40googlegroups.com. >>> >>> >>> >> >>> >> For more options, visit https://groups.google.com/d/optout. >>> > >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> Groups >>> > "Django developers (Contributions to Django itself)" group. >>> > To unsubscribe from this group and stop receiving emails from it, send >>> an >>> > email to django-develop...@googlegroups.com. >>> > To post to this group, send email to django-d...@googlegroups.com. >>> > Visit this group at http://groups.google.com/group/django-developers. >>> > To view this discussion on the web visit >>> > >>> https://groups.google.com/d/msgid/django-developers/CAFwN1ur_yiBmRuj%2BY26bFNJZe7meZpVsQnTuZLRWB9dj6GBqNg%40mail.gmail.com. >>> >>> >>> > >>> > For more options, visit https://groups.google.com/d/optout. >>> >> -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5d8714b7-b2d4-4a0b-adbe-852ddc6e6d8b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
