[Yahoo-eng-team] [Bug 1538227] [NEW] Failed `nova-manage db sync` returns exitcode of 0
Public bug reported: We're trying to upgrade to liberty using SaltStack (just for context, same issue running the command from a shell). At one point `nova-manage db sync` is executed. Because of #1511466 we get a *critical* error but the command still returns an exitcode of 0. In a somewhat POSIX environment this means "everything is fine" so the deployment just continues with an outdated database schema. ID: nova-manage db sync Function: cmd.run Name: nova-manage db sync; sleep 15 Result: True Comment: Command "nova-manage db sync; sleep 15" run Started: 18:07:09.511897 Duration: 17747.739 ms Changes: -- pid: 18000 retcode: 0 stderr: No handlers could be found for logger "oslo_config.cfg" 2016-01-26 18:07:12.092 18001 DEBUG migrate.versioning.repository [-] Loading repository /usr/lib/python2.7/dist-packages/nova/db/sql alchemy/migrate_repo... __init__ /usr/lib/python2.7/dist-packages/migrate/versioning/repository.py:76 2016-01-26 18:07:12.093 18001 DEBUG migrate.versioning.script.base [-] Loading script /usr/lib/python2.7/dist-packages/nova/db/sqlalc hemy/migrate_repo/versions/216_havana.py... __init__ /usr/lib/python2.7/dist-packages/migrate/versioning/script/base.py:27 [...] [...] [...] 2016-01-26 18:07:12.157 18001 INFO migrate.versioning.api [-] 290 -> 291... 2016-01-26 18:07:12.167 18001 CRITICAL nova [-] ValidationError: There are still 3 unmigrated flavor records. Migration cannot conti$ ue until all instance flavor records have been migrated to the new format. Please run `nova-manage db migrate_flavor_data' first. 2016-01-26 18:07:12.167 18001 ERROR nova Traceback (most recent call last): 2016-01-26 18:07:12.167 18001 ERROR nova File "/usr/bin/nova-manage", line 10, in 2016-01-26 18:07:12.167 18001 ERROR nova sys.exit(main()) 2016-01-26 18:07:12.167 18001 ERROR nova File "/usr/lib/python2.7/dist-packages/nova/cmd/manage.py", line 1443, in main 2016-01-26 18:07:12.167 18001 ERROR nova ret = fn(*fn_args, **fn_kwargs) 2016-01-26 18:07:12.167 18001 ERROR nova File "/usr/lib/python2.7/dist-packages/nova/cmd/manage.py", line 910, in sync 2016-01-26 18:07:12.167 18001 ERROR nova return migration.db_sync(version) 2016-01-26 18:07:12.167 18001 ERROR nova File "/usr/lib/python2.7/dist-packages/nova/db/migration.py", line 26, in db_sync 2016-01-26 18:07:12.167 18001 ERROR nova return IMPL.db_sync(version=version, database=database) 2016-01-26 18:07:12.167 18001 ERROR nova File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/migration.py", line 106, in db_$ ync 2016-01-26 18:07:12.167 18001 ERROR nova version) 2016-01-26 18:07:12.167 18001 ERROR nova File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 186, in upgrade 2016-01-26 18:07:12.167 18001 ERROR nova return _migrate(url, repository, version, upgrade=True, err=err, **opts) 2016-01-26 18:07:12.167 18001 ERROR nova File "", line 2, in _migrate 2016-01-26 18:07:12.167 18001 ERROR nova File "/usr/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py", line 160, in with_engine 2016-01-26 18:07:12.167 18001 ERROR nova return f(*a, **kw) 2016-01-26 18:07:12.167 18001 ERROR nova return migration.db_sync(version) 2016-01-26 18:07:12.167 18001 ERROR nova File "/usr/lib/python2.7/dist-packages/nova/db/migration.py", line 26, in db_sync 2016-01-26 18:07:12.167 18001 ERROR nova return IMPL.db_sync(version=version, database=database) 2016-01-26 18:07:12.167 18001 ERROR nova File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/migration.py", line 106, in db_s ync 2016-01-26 18:07:12.167 18001 ERROR nova version) 2016-01-26 18:07:12.167 18001 ERROR nova File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 186, in upgrade 2016-01-26 18:07:12.167 18001 ERROR nova return _migrate(url, repository, version, upgrade=True, err=err, **opts) 2016-01-26 18:07:12.167 18001 ERROR nova File "", line 2, in _migrate 2016-01-26 18:07:12.167 18001 ERROR nova File "/usr/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py", line 160, in with_engine 2016-01-26 18:07:12.167 18001 ERROR nova return f(*a, **kw) 2016-01-26 18:07:12.167 18001 ERROR nova File
[Yahoo-eng-team] [Bug 1464290] [NEW] UnboundLocalError in neutron/db/l3_db.py (Icehouse)
Public bug reported: Hi, working on my SaltStack-modules (outdated versions [0] and [1]) for managing subnets in Icehouse-Neutron I managed to cause this error in the neutron-server on Ubuntu trusty: 2015-06-11 16:49:33.636 10605 DEBUG neutron.openstack.common.rpc.amqp [req-f47d6292-09bb-4f03-999b-cd1458c3828b None] UNIQUE_ID is 5fada601c2ca49c5a777f690b0426a45. _add_unique_id /usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/amqp.py:342 2015-06-11 16:49:33.641 10605 ERROR neutron.api.v2.resource [req-f47d6292-09bb-4f03-999b-cd1458c3828b None] add_router_interface failed 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource Traceback (most recent call last): 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource File /usr/lib/python2.7/dist-packages/neutron/api/v2/resource.py, line 87, in resource 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource result = method(request=request, **args) 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource File /usr/lib/python2.7/dist-packages/neutron/api/v2/base.py, line 200, in _handle_action 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource return getattr(self._plugin, name)(*arg_list, **kwargs) 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource File /usr/lib/python2.7/dist-packages/neutron/db/l3_db.py, line 362, in add_router_interface 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource 'tenant_id': subnet['tenant_id'], 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource UnboundLocalError: local variable 'subnet' referenced before assignment 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource 2015-06-11 16:49:33.650 10605 INFO neutron.wsgi [req-f47d6292-09bb-4f03-999b-cd1458c3828b None] 192.168.122.85 - - [11/Jun/2015 16:49:33] PUT /v2.0/routers/8afd9ee7-dd37-47f3-b2e1-42805e984a61/add_router_interface.json HTTP/1.1 500 296 0.065534 Installed neutron-packages: root@controller:~# dpkg -l neutron\* | grep ^ii ii neutron-common 1:2014.1.4-0ubuntu2 all Neutron is a virtual network service for Openstack - common ii neutron-dhcp-agent 1:2014.1.4-0ubuntu2 all Neutron is a virtual network service for Openstack - DHCP agent ii neutron-l3-agent1:2014.1.4-0ubuntu2 all Neutron is a virtual network service for Openstack - l3 agent ii neutron-metadata-agent 1:2014.1.4-0ubuntu2 all Neutron is a virtual network service for Openstack - metadata agent ii neutron-plugin-ml2 1:2014.1.4-0ubuntu2 all Neutron is a virtual network service for Openstack - ML2 plugin ii neutron-plugin-openvswitch-agent1:2014.1.4-0ubuntu2 all Neutron is a virtual network service for Openstack - Open vSwitch plug in agent ii neutron-server 1:2014.1.4-0ubuntu2 all Neutron is a virtual network service for Openstack - server More details tomorrow, when I've added some more debugging to my code. Regards, Florian [0] https://github.com/fraunhoferfokus/openstack-formula/blob/master/_modules/neutron.py [1] https://github.com/fraunhoferfokus/openstack-formula/blob/master/_states/neutron_subnet.py ** Affects: neutron Importance: Undecided Status: New ** Tags: in-stable-icehouse -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1464290 Title: UnboundLocalError in neutron/db/l3_db.py (Icehouse) Status in OpenStack Neutron (virtual network service): New Bug description: Hi, working on my SaltStack-modules (outdated versions [0] and [1]) for managing subnets in Icehouse-Neutron I managed to cause this error in the neutron-server on Ubuntu trusty: 2015-06-11 16:49:33.636 10605 DEBUG neutron.openstack.common.rpc.amqp [req-f47d6292-09bb-4f03-999b-cd1458c3828b None] UNIQUE_ID is 5fada601c2ca49c5a777f690b0426a45. _add_unique_id /usr/lib/python2.7/dist-packages/neutron/openstack/common/rpc/amqp.py:342 2015-06-11 16:49:33.641 10605 ERROR neutron.api.v2.resource [req-f47d6292-09bb-4f03-999b-cd1458c3828b None] add_router_interface failed 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource Traceback (most recent call last): 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource File /usr/lib/python2.7/dist-packages/neutron/api/v2/resource.py, line 87, in resource 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource result = method(request=request, **args) 2015-06-11 16:49:33.641 10605 TRACE neutron.api.v2.resource File
[Yahoo-eng-team] [Bug 1433595] [NEW] Keystone writes Access denied for DB-Connection to its configured logfile
Public bug reported:
So my Keystone doesn't work because it can't connect to it's DB,
understandable.
But why is its configured logfile empty and the error is only logged to
stderr*?
root@hw-ctrl:~# grep log /etc/keystone/logging.conf
[loggers]
[logger_root]
args=('/var/log/keystone/keystone.log', 'a')
root@hw-ctrl:~# grep Access denied /var/log/upstart/keystone.log | tail -n 1
2015-03-18 14:31:38.633 9577 TRACE keystone.common.wsgi OperationalError:
(OperationalError) (1045, Access denied for user 'keystone'@'192.168.2.10'
(using password: YES)) None None
root@hw-ctrl:~# grep Access denied /var/log/upstart/keystone.log | wc -l
56
root@hw-ctrl:~# ls -al /var/log/keystone/
total 8
drwx-- 2 keystone keystone 4096 Mär 18 12:17 .
drwxrwxr-x 14 root syslog 4096 Mär 18 12:17 ..
*) and thus ends up in /var/log/upstart/keystone.log on Ubuntu
** Affects: keystone
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Keystone.
https://bugs.launchpad.net/bugs/1433595
Title:
Keystone writes Access denied for DB-Connection to its configured
logfile
Status in OpenStack Identity (Keystone):
New
Bug description:
So my Keystone doesn't work because it can't connect to it's DB,
understandable.
But why is its configured logfile empty and the error is only logged
to stderr*?
root@hw-ctrl:~# grep log /etc/keystone/logging.conf
[loggers]
[logger_root]
args=('/var/log/keystone/keystone.log', 'a')
root@hw-ctrl:~# grep Access denied /var/log/upstart/keystone.log | tail -n 1
2015-03-18 14:31:38.633 9577 TRACE keystone.common.wsgi OperationalError:
(OperationalError) (1045, Access denied for user 'keystone'@'192.168.2.10'
(using password: YES)) None None
root@hw-ctrl:~# grep Access denied /var/log/upstart/keystone.log | wc -l
56
root@hw-ctrl:~# ls -al /var/log/keystone/
total 8
drwx-- 2 keystone keystone 4096 Mär 18 12:17 .
drwxrwxr-x 14 root syslog 4096 Mär 18 12:17 ..
*) and thus ends up in /var/log/upstart/keystone.log on Ubuntu
To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1433595/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1418635] [NEW] Neutron API reference is inconsistent and differs from implementation
Public bug reported:
I'm implementing modules for SaltStack (not yet on GitHub) to create entities
in OpenStack Neutron and came across quite a few problems
with the API documentation:
* Link Filtering and Column Selection to
http://docs.openstack.org/api/openstack-network/2.0/content/filtering.html
404s
* Section http://developer.openstack.org/api-ref-
networking-v2.html#subnets
* POST /v2.0/subnets aka Create subnet
- 'id' is listed to be an optional parameter but the Neutron-API in
Icehouse refuses to set a user-selected ID
- parameters 'dns_nameservers' and 'host_routes' missing from
documentation (undocumented extensions?)
* GET /v2.0/subnets aka List subnets
- can't filter by allocation_pools or enable_dhcp
* PUT /v2.0/subnets/{subnet_id} aka Update subnet
- parameters allocation_pools, network_id, tenant_id, id listed
as optional request parameters but Neutron-API in Icehouse returns
Cannot update read-only attribute $PARAMETER
* Section http://developer.openstack.org/api-ref-
networking-v2.html#networks
* GET /v2.0/networks aka List networks
- parameter shared is ignored as a filter
* Section http://developer.openstack.org/api-ref-
networking-v2.html#layer3
* POST /v2.0/routers aka Create router
- Description states router:external and external_gateway_info are
valid request parameters but they're not listed in the table of
request parameters
- What's the parameter router described as A router object. supposed
to be? A router object in JSON/XML notation inside a router object in
JSON/XML notation?
I'll probably add more when implementing functions for managing routers
in Neutron.
** Affects: neutron
Importance: Undecided
Status: New
** Tags: api documentation in-stable-icehouse
** Description changed:
- I'm implementing modules for SaltStack (not yet on GitHub) to create entities
in OpenStack Neutron and came across
- quite a few problems with the API documentation:
+ I'm implementing modules for SaltStack (not yet on GitHub) to create entities
in OpenStack Neutron and came across quite a few problems
+ with the API documentation:
- * Link Filtering and Column Selection to
http://docs.openstack.org/api/openstack-network/2.0/content/filtering.html
-404s
+ * Link Filtering and Column Selection to
+ http://docs.openstack.org/api/openstack-network/2.0/content/filtering.html
+ 404s
* Section http://developer.openstack.org/api-ref-
networking-v2.html#subnets
- * POST /v2.0/subnets aka Create subnet
- - 'id' is listed to be an optional parameter but the Neutron-API in
Icehouse refuses to set a user-selected ID
- - parameters 'dns_nameservers' and 'host_routes' missing from
documentation (undocumented extensions?)
+ * POST /v2.0/subnets aka Create subnet
+ - 'id' is listed to be an optional parameter but the Neutron-API in
+ Icehouse refuses to set a user-selected ID
+ - parameters 'dns_nameservers' and 'host_routes' missing from
+ documentation (undocumented extensions?)
- * GET /v2.0/subnets aka List subnets
- - can't filter by allocation_pools or enable_dhcp
+ * GET /v2.0/subnets aka List subnets
+ - can't filter by allocation_pools or enable_dhcp
- * PUT /v2.0/subnets/{subnet_id} aka Update subnet
- - parameters allocation_pools, network_id, tenant_id, id listed
as optional request parameters but
- Neutron-API in Icehouse returns Cannot update read-only attribute
$PARAMETER
+ * PUT /v2.0/subnets/{subnet_id} aka Update subnet
+ - parameters allocation_pools, network_id, tenant_id, id listed
+ as optional request parameters but Neutron-API in Icehouse returns
+ Cannot update read-only attribute $PARAMETER
* Section http://developer.openstack.org/api-ref-
networking-v2.html#networks
- * GET /v2.0/networks aka List networks
- - parameter shared is ignored as a filter
+ * GET /v2.0/networks aka List networks
+ - parameter shared is ignored as a filter
* Section http://developer.openstack.org/api-ref-
networking-v2.html#layer3
- * POST /v2.0/routers aka Create router
- - Description states router:external and external_gateway_info are
valid request parameters but they're not
- listed in the table of request parameters
- - What's the parameter router described as A router object. supposed to
be? A router object in JSON/XML
- notation inside a router object in JSON/XML notation?
+ * POST /v2.0/routers aka Create router
+ - Description states router:external and external_gateway_info are
+ valid request parameters but they're not listed in the table of
+ request parameters
+ - What's the parameter router described as A router object. supposed
+ to be? A router object in JSON/XML notation inside a router object in
+ JSON/XML notation?
I'll probably add more when
