Public bug reported:
Description
===========
If a Keystone token issued for a admin user (e.g. ceilometer) is expired
or revoked right after it's been validated by
keystoneauthtoken_middleware in nova-api, but before it's validated by
the very same middleware in neutron-server, nova-api will respond with
400 Bad Request instead of expected 401 Unauthorized, so that the
original request can be properly retried after re-authentication.
Steps to reproduce
==================
The condition described above is easy to reproduce synthetically by
putting breakpoints into Nova code and revoking a token. One can
reproduce the very same problem in real life by running enough
ceilometer polling agents.
Make sure you use credentials of an admin user (e.g. admin or ceilometer
in Devstack) and have at least 1 instance running (so that `nova list`
triggers an HTTP request to neutron-server).
1. Put a breakpoint on entering get_client() nova/network/neutronv2/api.py
2. Do `nova list`
3. Revoke the the issued token with `openstack token revoke $token` (you may
also need to restart memcached to make sure token validation result is not
cached)
4. Continue execution of nova-api
Expected result
===============
As token is now invalid (expired or revoked), it's expected that nova-
api responds with 401 Unauthorized, so that a client can handle this,
re-authenticate and retry the original request.
Actual result
=============
nova-api responds with 400 Bad Request and outputs the following error
into logs
2017-01-19 15:02:09.952 595 ERROR nova.network.neutronv2.api
[req-0c1558f5-9cc8-4411-9fb1-2fe7cb232725 admin admin] Neutron client was not
able
to generate a valid admin token, please verify Neutron admin credential
located in nova.conf
Environment
===========
Devstack, master (Ocata), nova HEAD at
da54487edad28c87accbf6439471e7341b52ff48
** Affects: nova
Importance: Undecided
Assignee: Roman Podoliaka (rpodolyaka)
Status: In Progress
** Tags: api neutron
** Changed in: nova
Assignee: (unassigned) => Roman Podoliaka (rpodolyaka)
** Tags added: api neutron
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1657774
Title:
Nova does not re-raise 401 Unauthorized received from Neutron for
admin users
Status in OpenStack Compute (nova):
In Progress
Bug description:
Description
===========
If a Keystone token issued for a admin user (e.g. ceilometer) is
expired or revoked right after it's been validated by
keystoneauthtoken_middleware in nova-api, but before it's validated by
the very same middleware in neutron-server, nova-api will respond with
400 Bad Request instead of expected 401 Unauthorized, so that the
original request can be properly retried after re-authentication.
Steps to reproduce
==================
The condition described above is easy to reproduce synthetically by
putting breakpoints into Nova code and revoking a token. One can
reproduce the very same problem in real life by running enough
ceilometer polling agents.
Make sure you use credentials of an admin user (e.g. admin or
ceilometer in Devstack) and have at least 1 instance running (so that
`nova list` triggers an HTTP request to neutron-server).
1. Put a breakpoint on entering get_client() nova/network/neutronv2/api.py
2. Do `nova list`
3. Revoke the the issued token with `openstack token revoke $token` (you may
also need to restart memcached to make sure token validation result is not
cached)
4. Continue execution of nova-api
Expected result
===============
As token is now invalid (expired or revoked), it's expected that nova-
api responds with 401 Unauthorized, so that a client can handle this,
re-authenticate and retry the original request.
Actual result
=============
nova-api responds with 400 Bad Request and outputs the following error
into logs
2017-01-19 15:02:09.952 595 ERROR nova.network.neutronv2.api
[req-0c1558f5-9cc8-4411-9fb1-2fe7cb232725 admin admin] Neutron client was not
able
to generate a valid admin token, please verify Neutron admin credential
located in nova.conf
Environment
===========
Devstack, master (Ocata), nova HEAD at
da54487edad28c87accbf6439471e7341b52ff48
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1657774/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
