#33583: sqlmigrate crashes on replaces to nonexistent migration.
-------------------------------------+-------------------------------------
Reporter: Amin Shah Gilani | Owner: nobody
Type: Bug | Status: closed
Component: Migrations | Version: 3.2
Severity: Normal | Resolution: invalid
Keywords: sqlmigrate, | Triage Stage:
migrate, squash | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> Hello,
>
> This is my first bug report here so please ask for any additional details
> you need. I would also appreciate a workaround. If it is obvious, it
> escapes me.
>
> == Summary
>
> When an app is renamed with its migrations including a `replaces`
> reference to the old app and migration name (such as by squashing
> migrations), the behavior between `migrate` and `sqlmigrate` becomes
> inconsistent. `migrate` happily applies all migrations, but `sqlmigrate`
> reports an error.
>
> == Other information:
>
> * This bug was first observed in v3.2, but my demo shows it also affects
> v4.0.3.
> * This bug affects installations using the [https://github.com/python-
> social-auth/social-app-django social-app-django] package, which renamed
> its app multiple times,
> * I have a demo available [https://github.com/amingilani/django-bug-PoC
> here].
>
> == Steps to reproduce:
>
> **Step 1:** Create a migration file, in an app and then rename the app
> and migration. This will result in a migration similar to this:
>
> {{{#!python
>
> # alpha/migrations/0001_initial.py
> class Migration(migrations.Migration):
>
> replaces = [
> ('first', '0001_initial'),
> ]
>
> }}}
>
> **Step 2:** Ensure you have migrations in another app, with the first
> migration the second app referencing the old migration:
>
> {{{#!python
>
> class Migration(migrations.Migration):
>
> dependencies = [
> ('first', '0001_initial'),
> ]
>
> }}}
>
> **Step 3:** Note that `migrate` will happily apply all migrations:
>
> {{{#!shell
> python3 manage.py migrate
> Operations to perform:
> Apply all migrations: admin, alpha, auth, beta, contenttypes, sessions
> Running migrations:
> Applying contenttypes.0001_initial... OK
> Applying auth.0001_initial... OK
> Applying admin.0001_initial... OK
> Applying admin.0002_logentry_remove_auto_add... OK
> Applying admin.0003_logentry_add_action_flag_choices... OK
> Applying alpha.0001_initial... OK
> Applying contenttypes.0002_remove_content_type_name... OK
> Applying auth.0002_alter_permission_name_max_length... OK
> Applying auth.0003_alter_user_email_max_length... OK
> Applying auth.0004_alter_user_username_opts... OK
> Applying auth.0005_alter_user_last_login_null... OK
> Applying auth.0006_require_contenttypes_0002... OK
> Applying auth.0007_alter_validators_add_error_messages... OK
> Applying auth.0008_alter_user_username_max_length... OK
> Applying auth.0009_alter_user_last_name_max_length... OK
> Applying auth.0010_alter_group_name_max_length... OK
> Applying auth.0011_update_proxy_permissions... OK
> Applying auth.0012_alter_user_first_name_max_length... OK
> Applying beta.0001_initial... OK
> Applying beta.0002_auto_20220318_0245... OK
> Applying beta.0003_auto_20220318_0247... OK
> Applying sessions.0001_initial... OK
> }}}
>
> **Step 4:** However, `sqlmigrate` will raise an error:
>
> {{{#!shell
> python3 manage.py sqlmigrate beta 0003
> Traceback (most recent call last):
> File "/Users/amin/sandbox/mysite/manage.py", line 22, in <module>
> main()
> File "/Users/amin/sandbox/mysite/manage.py", line 18, in main
> execute_from_command_line(sys.argv)
> File "/usr/local/lib/python3.9/site-
> packages/django/core/management/__init__.py", line 446, in
> execute_from_command_line
> utility.execute()
> File "/usr/local/lib/python3.9/site-
> packages/django/core/management/__init__.py", line 440, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
> File "/usr/local/lib/python3.9/site-
> packages/django/core/management/base.py", line 414, in run_from_argv
> self.execute(*args, **cmd_options)
> File "/usr/local/lib/python3.9/site-
> packages/django/core/management/commands/sqlmigrate.py", line 38, in
> execute
> return super().execute(*args, **options)
> File "/usr/local/lib/python3.9/site-
> packages/django/core/management/base.py", line 460, in execute
> output = self.handle(*args, **options)
> File "/usr/local/lib/python3.9/site-
> packages/django/core/management/commands/sqlmigrate.py", line 46, in
> handle
> loader = MigrationLoader(connection, replace_migrations=False)
> File "/usr/local/lib/python3.9/site-
> packages/django/db/migrations/loader.py", line 58, in __init__
> self.build_graph()
> File "/usr/local/lib/python3.9/site-
> packages/django/db/migrations/loader.py", line 276, in build_graph
> self.graph.validate_consistency()
> File "/usr/local/lib/python3.9/site-
> packages/django/db/migrations/graph.py", line 198, in
> validate_consistency
> [n.raise_error() for n in self.node_map.values() if isinstance(n,
> DummyNode)]
> File "/usr/local/lib/python3.9/site-
> packages/django/db/migrations/graph.py", line 198, in <listcomp>
> [n.raise_error() for n in self.node_map.values() if isinstance(n,
> DummyNode)]
> File "/usr/local/lib/python3.9/site-
> packages/django/db/migrations/graph.py", line 60, in raise_error
> raise NodeNotFoundError(self.error_message, self.key,
> origin=self.origin)
> django.db.migrations.exceptions.NodeNotFoundError: Migration
> beta.0001_initial dependencies reference nonexistent parent node
> ('first', '0001_initial')
> }}}
>
> **Note:** I have a demo available to try [https://github.com/amingilani
> /django-bug-PoC here].
New description:
Hello,
This is my first bug report here so please ask for any additional details
you need. I would also appreciate a workaround. If it is obvious, it
escapes me.
== Summary
When an app is renamed with its migrations including a `replaces`
reference to the old app and migration name (such as by squashing
migrations), the behavior between `migrate` and `sqlmigrate` becomes
inconsistent. `migrate` happily applies all migrations, but `sqlmigrate`
reports an error.
== Other information:
* This bug was first observed in v3.2, but my demo shows it also affects
v4.0.3.
* This bug affects installations using the [https://github.com/python-
social-auth/social-app-django social-app-django] package, which renamed
its app multiple times,
* I have a demo available [https://github.com/amingilani/django-bug-PoC
here].
== Steps to reproduce:
**Step 1:** Create a migration file, in an app and then rename the app and
migration. This will result in a migration similar to this:
{{{#!python
# alpha/migrations/0001_initial.py
class Migration(migrations.Migration):
replaces = [
('first', '0001_initial'),
]
}}}
**Step 2:** Ensure you have migrations in another app, with the first
migration the second app referencing the old migration:
{{{#!python
class Migration(migrations.Migration):
dependencies = [
('first', '0001_initial'),
]
}}}
**Step 3:** Note that `migrate` will happily apply all migrations:
{{{#!shell
python3 manage.py migrate
Operations to perform:
Apply all migrations: admin, alpha, auth, beta, contenttypes, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying admin.0003_logentry_add_action_flag_choices... OK
Applying alpha.0001_initial... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying auth.0008_alter_user_username_max_length... OK
Applying auth.0009_alter_user_last_name_max_length... OK
Applying auth.0010_alter_group_name_max_length... OK
Applying auth.0011_update_proxy_permissions... OK
Applying auth.0012_alter_user_first_name_max_length... OK
Applying beta.0001_initial... OK
Applying beta.0002_auto_20220318_0245... OK
Applying beta.0003_auto_20220318_0247... OK
Applying sessions.0001_initial... OK
}}}
**Step 4:** However, `sqlmigrate` will raise an error:
{{{#!shell
python3 manage.py sqlmigrate beta 0003
Traceback (most recent call last):
File "/Users/amin/sandbox/mysite/manage.py", line 22, in <module>
main()
File "/Users/amin/sandbox/mysite/manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.9/site-
packages/django/core/management/__init__.py", line 446, in
execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.9/site-
packages/django/core/management/__init__.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.9/site-
packages/django/core/management/base.py", line 414, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.9/site-
packages/django/core/management/commands/sqlmigrate.py", line 38, in
execute
return super().execute(*args, **options)
File "/usr/local/lib/python3.9/site-
packages/django/core/management/base.py", line 460, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.9/site-
packages/django/core/management/commands/sqlmigrate.py", line 46, in
handle
loader = MigrationLoader(connection, replace_migrations=False)
File "/usr/local/lib/python3.9/site-
packages/django/db/migrations/loader.py", line 58, in __init__
self.build_graph()
File "/usr/local/lib/python3.9/site-
packages/django/db/migrations/loader.py", line 276, in build_graph
self.graph.validate_consistency()
File "/usr/local/lib/python3.9/site-
packages/django/db/migrations/graph.py", line 198, in validate_consistency
[n.raise_error() for n in self.node_map.values() if isinstance(n,
DummyNode)]
File "/usr/local/lib/python3.9/site-
packages/django/db/migrations/graph.py", line 198, in <listcomp>
[n.raise_error() for n in self.node_map.values() if isinstance(n,
DummyNode)]
File "/usr/local/lib/python3.9/site-
packages/django/db/migrations/graph.py", line 60, in raise_error
raise NodeNotFoundError(self.error_message, self.key,
origin=self.origin)
django.db.migrations.exceptions.NodeNotFoundError: Migration
beta.0001_initial dependencies reference nonexistent parent node ('first',
'0001_initial')
}}}
**Note:** I have a demo available to try [https://github.com/amingilani
/django-bug-PoC here].
== Actual Behavior
`sqlmirate` raises an error but `migrate` processes the migrations without
a problem
== Expected Behavior
Both commands would consistently either raise an error on these
migrations, or run without raising an error
--
Comment (by Amin Shah Gilani):
Replying to [comment:1 Mariusz Felisiak]:
> Thanks for the report, this behavior was changed in
d88365708c554efe3c786c3be6da1d9de916360f. However, `replaces` is
documented and supported only for
[https://docs.djangoproject.com/en/stable/topics/migrations/#squashing-
migrations squashing migrations] so described flow was never officially
supported.
Thank you for your response. Would it then be correct to say that the
`migrate` command needs to be updated as well to raise an error? As it
stands the behavior is inconsistent. I've added existing and expected
behavior to the original ticket in order to clarify this problem.
Additionally, for applications that depend on relocated migrations in
third-party packages, will it be fine to update the now outdated pointer
to the new location of the migration?
In the context of the example above, is this fine?
{{{#!diff
-('first', '0001_initial'),
+('alpha, '0001_initial'),
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/33583#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107017f9d8c41c8-36050d91-4a35-486c-a864-51ffeddc4578-000000%40eu-central-1.amazonses.com.
