[Django] #34841: Reverse migrations model state rendering slow with moderate to large migrations

[Django]

#34841: Reverse migrations model state rendering slow with moderate to large
migrations
------------------------------------------------+------------------------
               Reporter:  David Sanders         |          Owner:  nobody
                   Type:  Cleanup/optimization  |         Status:  new
              Component:  Migrations            |        Version:  4.2
               Severity:  Normal                |       Keywords:
           Triage Stage:  Unreviewed            |      Has patch:  0
    Needs documentation:  0                     |    Needs tests:  0
Patch needs improvement:  0                     |  Easy pickings:  0
                  UI/UX:  0                     |
------------------------------------------------+------------------------
 Relevant discussion: https://forum.djangoproject.com/t/migrating-
 backwards-takes-orders-of-magnitude-longer-than-migrating-forwards/20873/2

 Investigations revealed 2 things.

 1. this block of code in `db/migrations/excecutor.py` was responsible for
 a large portion of processing time:
 
https://github.com/django/django/blob/b8b2f7451201f3ff60891b6ce55f177400700d7a/django/db/migrations/executor.py#L222-L232

 In my small project with only a moderate number of migrations, a simple
 reverse migrate of the last migration on the main app took ~20s. The block
 below was responsible for ~16s.

 {{{
         # Generate the post migration state by starting from the state
 before
         # the last migration is unapplied and mutating it to include all
 the
         # remaining applied migrations.
         last_unapplied_migration = plan[-1][0]
         state = states[last_unapplied_migration]
         for index, (migration, _) in enumerate(full_plan):
             if migration == last_unapplied_migration:
                 for migration, _ in full_plan[index:]:
                     if migration in applied_migrations:
                         migration.mutate_state(state, preserve=False)
                 break
 }}}

 2. these 2 lines appeared to be responsible for the slowdown, commenting
 them out made the block above run almost instantly:
 
https://github.com/django/django/blob/b8b2f7451201f3ff60891b6ce55f177400700d7a/django/db/migrations/executor.py#L203-L204

 {{{
                 if "apps" not in state.__dict__:
                     state.apps  # Render all -- performance critical
 }}}

 I have a small patch to move record the unapplied state before these 2
 lines and clone it, removing the `state.apps` attribute.

-- 
Ticket URL: <https://code.djangoproject.com/ticket/34841>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018a94dcdbae-91edbd0a-bdbe-4fb9-bb1e-d38a5a281415-000000%40eu-central-1.amazonses.com.