I also ran into this exact same issue. We have mysql as the backend. And the datetime type is not timezone aware. Neither "--verbose" and "-- debug" are effective for the online_data_migrations command as they are not being taken into consideration. I ended up manually printing out the traceback by adding these two lines
import sys, traceback traceback.print_exc(file=sys.stdout) here https://github.com/openstack/nova/blob/stable/newton/nova/cmd/manage.py#L897 And able to see more information on the failure. Here's a sample traceback. Running batches of 50 until complete -------------- 2017-05-19 23:34:43+00:00 ----------------- Error attempting to run <function migrate_flavors at 0x7f5a0d758e60> Traceback (most recent call last): File "/opt/stack/venv/nova-20170728T171245Z/lib/python2.7/site-packages/nova/cmd/manage.py", line 892, in _run_migration found, done = migration_meth(ctxt, count) File "/opt/stack/venv/nova-20170728T171245Z/lib/python2.7/site-packages/nova/objects/flavor.py", line 717, in migrate_flavors flavor._flavor_create(ctxt, flavor_values) File "/opt/stack/venv/nova-20170728T171245Z/lib/python2.7/site-packages/nova/objects/flavor.py", line 463, in _flavor_create return _flavor_create(context, updates) File "/opt/stack/venv/nova-20170728T171245Z/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 824, in wrapper return fn(*args, **kwargs) File "/opt/stack/venv/nova-20170728T171245Z/lib/python2.7/site-packages/nova/objects/flavor.py", line 166, in _flavor_create raise db_exc.DBError(e) DBError: (_mysql_exceptions.OperationalError) (1292, "Incorrect datetime value: '2017-05-19 23:34:43+00:00' for column 'created_at' at row 1") [SQL: u'INSERT INTO flavors (created_at, updated_at, id, name, memory_mb, vcpus, root_gb, ephemeral_gb, flavorid, swap, rxtx_factor, vcpu_weight, disabled, is_public) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)'] [parameters: (datetime.datetime(2017, 5, 19, 23, 34, 43, tzinfo=<iso8601.Utc>), None, 16, 'foo', 1024, 1, 2, 0, '50aae60f-ba2a-40d8-a0c3-2117ba0dd2a6', 0, 1.0, 0, 0, 0)] Looks like migrations code is attempting to insert a timezone aware datetime field into mysql. ** Changed in: nova Status: Expired => Confirmed -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1696417 Title: nova-manage db online_data_migrations can fail when upgrading to newton under certain conditions Status in OpenStack Compute (nova): Confirmed Bug description: Reproduceable under conditions: * Mitaka upgraded to Newton * online_data_migrations have not yet migrated flavors and/or aggregates from the nova database, to the nova-api database * One or more of the datetime-fields (created_at, updated_at, deleted_at) are set. ** Because a custom flavor has been created ** Because a flavor has been updated ** Because a flavor has been deleted (deleted flavors are probably not relevant, as the new table have no deleted flag, it just removes them altogether) Steps to reproduce: * Run 'nova-manage db online_data_migrations' It throws an error message like: Error attempting to run <function migrate_flavors at 0x488f398> Workaround: * Set created_at,updated_at and deleted_at to NULL * Run migration I have done quite a bit of troubleshooting, but haven't managed to write a patch so far. As far as I can tell, inserting a flavor or aggregate to the new tables fail due to the datetime fields including a timezone. There exists code for stripping away the timezone in nova/db/sqlalchemy/api.py (convert_objects_related_datetimes) - but the timezone reappears in nova/objects/flavor.py To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1696417/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp