[Yahoo-eng-team] [Bug 1523646] Re: Nova/Cinder Key Manager for Barbican Uses Stale Cache
** Changed in: nova/liberty Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1523646 Title: Nova/Cinder Key Manager for Barbican Uses Stale Cache Status in castellan: Fix Released Status in Cinder: Fix Released Status in Cinder liberty series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) liberty series: Fix Released Status in OpenStack Security Notes: Fix Released Bug description: The Key Manger for Barbican, implemented in Nova and Cinder, caches a value of barbican_client to save extra calls to Keystone for authentication. However, the cached value of barbican_client is only valid for the current context. A check needs to be made to ensure the context has not changed before using the saved value. The symptoms for using a stale cache value include getting the following error message when creating an encrypted volume. From CLI: --- openstack volume create --size 1 --type LUKS encrypted_volume The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-aea6be92-020e-41ed-ba88-44a1f5235ab0) In cinder.log --- 2015-12-03 09:09:03.648 TRACE cinder.volume.api Traceback (most recent call last): 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _exe cute_task 2015-12-03 09:09:03.648 TRACE cinder.volume.api result = task.execute(**arguments) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 409, in execute 2015-12-03 09:09:03.648 TRACE cinder.volume.api source_volume) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 338, in _get_encryption_key_ id 2015-12-03 09:09:03.648 TRACE cinder.volume.api encryption_key_id = key_manager.create_key(context) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/keymgr/barbican.py", line 147, in create_key 2015-12-03 09:09:03.648 TRACE cinder.volume.api LOG.exception(_LE("Error creating key.")) …. 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 502, in post 2015-12-03 09:09:03.648 TRACE cinder.volume.api return self.request(url, 'POST', **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner 2015-12-03 09:09:03.648 TRACE cinder.volume.api return func(*args, **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 402, in request 2015-12-03 09:09:03.648 TRACE cinder.volume.api raise exceptions.from_response(resp, method, url) 2015-12-03 09:09:03.648 TRACE cinder.volume.api Unauthorized: The request you have made requires authentication. (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-d2c52e0b-c16d-43ec-a7a0-763f1270) To manage notifications about this bug go to: https://bugs.launchpad.net/castellan/+bug/1523646/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1523646] Re: Nova/Cinder Key Manager for Barbican Uses Stale Cache
This issue has been published as OSSN-0063 on the mailing lists and wiki: https://wiki.openstack.org/wiki/OSSN/OSSN-0063 ** Changed in: ossn Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1523646 Title: Nova/Cinder Key Manager for Barbican Uses Stale Cache Status in castellan: Fix Released Status in Cinder: Fix Released Status in Cinder liberty series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) liberty series: Fix Committed Status in OpenStack Security Notes: Fix Released Bug description: The Key Manger for Barbican, implemented in Nova and Cinder, caches a value of barbican_client to save extra calls to Keystone for authentication. However, the cached value of barbican_client is only valid for the current context. A check needs to be made to ensure the context has not changed before using the saved value. The symptoms for using a stale cache value include getting the following error message when creating an encrypted volume. From CLI: --- openstack volume create --size 1 --type LUKS encrypted_volume The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-aea6be92-020e-41ed-ba88-44a1f5235ab0) In cinder.log --- 2015-12-03 09:09:03.648 TRACE cinder.volume.api Traceback (most recent call last): 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _exe cute_task 2015-12-03 09:09:03.648 TRACE cinder.volume.api result = task.execute(**arguments) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 409, in execute 2015-12-03 09:09:03.648 TRACE cinder.volume.api source_volume) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 338, in _get_encryption_key_ id 2015-12-03 09:09:03.648 TRACE cinder.volume.api encryption_key_id = key_manager.create_key(context) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/keymgr/barbican.py", line 147, in create_key 2015-12-03 09:09:03.648 TRACE cinder.volume.api LOG.exception(_LE("Error creating key.")) …. 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 502, in post 2015-12-03 09:09:03.648 TRACE cinder.volume.api return self.request(url, 'POST', **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner 2015-12-03 09:09:03.648 TRACE cinder.volume.api return func(*args, **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 402, in request 2015-12-03 09:09:03.648 TRACE cinder.volume.api raise exceptions.from_response(resp, method, url) 2015-12-03 09:09:03.648 TRACE cinder.volume.api Unauthorized: The request you have made requires authentication. (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-d2c52e0b-c16d-43ec-a7a0-763f1270) To manage notifications about this bug go to: https://bugs.launchpad.net/castellan/+bug/1523646/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1523646] Re: Nova/Cinder Key Manager for Barbican Uses Stale Cache
** Tags removed: liberty-backport-potential ** Also affects: nova/liberty Importance: Undecided Status: New ** Changed in: nova/liberty Status: New => In Progress ** Changed in: nova/liberty Assignee: (unassigned) => Matt Riedemann (mriedem) ** Changed in: nova/liberty Importance: Undecided => High -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1523646 Title: Nova/Cinder Key Manager for Barbican Uses Stale Cache Status in castellan: Fix Released Status in Cinder: Fix Released Status in Cinder liberty series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) liberty series: In Progress Status in OpenStack Security Notes: Confirmed Bug description: The Key Manger for Barbican, implemented in Nova and Cinder, caches a value of barbican_client to save extra calls to Keystone for authentication. However, the cached value of barbican_client is only valid for the current context. A check needs to be made to ensure the context has not changed before using the saved value. The symptoms for using a stale cache value include getting the following error message when creating an encrypted volume. From CLI: --- openstack volume create --size 1 --type LUKS encrypted_volume The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-aea6be92-020e-41ed-ba88-44a1f5235ab0) In cinder.log --- 2015-12-03 09:09:03.648 TRACE cinder.volume.api Traceback (most recent call last): 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _exe cute_task 2015-12-03 09:09:03.648 TRACE cinder.volume.api result = task.execute(**arguments) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 409, in execute 2015-12-03 09:09:03.648 TRACE cinder.volume.api source_volume) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 338, in _get_encryption_key_ id 2015-12-03 09:09:03.648 TRACE cinder.volume.api encryption_key_id = key_manager.create_key(context) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/keymgr/barbican.py", line 147, in create_key 2015-12-03 09:09:03.648 TRACE cinder.volume.api LOG.exception(_LE("Error creating key.")) …. 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 502, in post 2015-12-03 09:09:03.648 TRACE cinder.volume.api return self.request(url, 'POST', **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner 2015-12-03 09:09:03.648 TRACE cinder.volume.api return func(*args, **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 402, in request 2015-12-03 09:09:03.648 TRACE cinder.volume.api raise exceptions.from_response(resp, method, url) 2015-12-03 09:09:03.648 TRACE cinder.volume.api Unauthorized: The request you have made requires authentication. (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-d2c52e0b-c16d-43ec-a7a0-763f1270) To manage notifications about this bug go to: https://bugs.launchpad.net/castellan/+bug/1523646/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1523646] Re: Nova/Cinder Key Manager for Barbican Uses Stale Cache
** Also affects: cinder/liberty Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1523646 Title: Nova/Cinder Key Manager for Barbican Uses Stale Cache Status in castellan: Fix Released Status in Cinder: Fix Released Status in Cinder liberty series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) liberty series: In Progress Status in OpenStack Security Notes: Confirmed Bug description: The Key Manger for Barbican, implemented in Nova and Cinder, caches a value of barbican_client to save extra calls to Keystone for authentication. However, the cached value of barbican_client is only valid for the current context. A check needs to be made to ensure the context has not changed before using the saved value. The symptoms for using a stale cache value include getting the following error message when creating an encrypted volume. From CLI: --- openstack volume create --size 1 --type LUKS encrypted_volume The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-aea6be92-020e-41ed-ba88-44a1f5235ab0) In cinder.log --- 2015-12-03 09:09:03.648 TRACE cinder.volume.api Traceback (most recent call last): 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _exe cute_task 2015-12-03 09:09:03.648 TRACE cinder.volume.api result = task.execute(**arguments) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 409, in execute 2015-12-03 09:09:03.648 TRACE cinder.volume.api source_volume) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 338, in _get_encryption_key_ id 2015-12-03 09:09:03.648 TRACE cinder.volume.api encryption_key_id = key_manager.create_key(context) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/keymgr/barbican.py", line 147, in create_key 2015-12-03 09:09:03.648 TRACE cinder.volume.api LOG.exception(_LE("Error creating key.")) …. 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 502, in post 2015-12-03 09:09:03.648 TRACE cinder.volume.api return self.request(url, 'POST', **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner 2015-12-03 09:09:03.648 TRACE cinder.volume.api return func(*args, **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 402, in request 2015-12-03 09:09:03.648 TRACE cinder.volume.api raise exceptions.from_response(resp, method, url) 2015-12-03 09:09:03.648 TRACE cinder.volume.api Unauthorized: The request you have made requires authentication. (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-d2c52e0b-c16d-43ec-a7a0-763f1270) To manage notifications about this bug go to: https://bugs.launchpad.net/castellan/+bug/1523646/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1523646] Re: Nova/Cinder Key Manager for Barbican Uses Stale Cache
** Changed in: cinder/liberty Status: New => Fix Released ** Changed in: cinder Importance: Undecided => High ** Changed in: cinder/liberty Importance: Undecided => High ** Changed in: cinder/liberty Assignee: (unassigned) => Dave McCowan (dave-mccowan) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1523646 Title: Nova/Cinder Key Manager for Barbican Uses Stale Cache Status in castellan: Fix Released Status in Cinder: Fix Released Status in Cinder liberty series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) liberty series: In Progress Status in OpenStack Security Notes: Confirmed Bug description: The Key Manger for Barbican, implemented in Nova and Cinder, caches a value of barbican_client to save extra calls to Keystone for authentication. However, the cached value of barbican_client is only valid for the current context. A check needs to be made to ensure the context has not changed before using the saved value. The symptoms for using a stale cache value include getting the following error message when creating an encrypted volume. From CLI: --- openstack volume create --size 1 --type LUKS encrypted_volume The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-aea6be92-020e-41ed-ba88-44a1f5235ab0) In cinder.log --- 2015-12-03 09:09:03.648 TRACE cinder.volume.api Traceback (most recent call last): 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _exe cute_task 2015-12-03 09:09:03.648 TRACE cinder.volume.api result = task.execute(**arguments) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 409, in execute 2015-12-03 09:09:03.648 TRACE cinder.volume.api source_volume) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 338, in _get_encryption_key_ id 2015-12-03 09:09:03.648 TRACE cinder.volume.api encryption_key_id = key_manager.create_key(context) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/keymgr/barbican.py", line 147, in create_key 2015-12-03 09:09:03.648 TRACE cinder.volume.api LOG.exception(_LE("Error creating key.")) …. 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 502, in post 2015-12-03 09:09:03.648 TRACE cinder.volume.api return self.request(url, 'POST', **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner 2015-12-03 09:09:03.648 TRACE cinder.volume.api return func(*args, **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 402, in request 2015-12-03 09:09:03.648 TRACE cinder.volume.api raise exceptions.from_response(resp, method, url) 2015-12-03 09:09:03.648 TRACE cinder.volume.api Unauthorized: The request you have made requires authentication. (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-d2c52e0b-c16d-43ec-a7a0-763f1270) To manage notifications about this bug go to: https://bugs.launchpad.net/castellan/+bug/1523646/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1523646] Re: Nova/Cinder Key Manager for Barbican Uses Stale Cache
Reviewed: https://review.openstack.org/254358 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=676a53ce44a5624a553e80bcff339300802d5494 Submitter: Jenkins Branch:master commit 676a53ce44a5624a553e80bcff339300802d5494 Author: Dave McCowan Date: Mon Dec 7 14:28:52 2015 -0500 Check context before returning cached value The key manager caches the value of barbican client to be reused, saving an extra call to keystone. The cached value is only applicable to the current context, so the context must be checked before returning the cached value. Closes-Bug: #1523646 Change-Id: I7cd7f1ba8a749b230c611e4fb20ccf4127354c35 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1523646 Title: Nova/Cinder Key Manager for Barbican Uses Stale Cache Status in castellan: Fix Released Status in Cinder: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Security Notes: Confirmed Bug description: The Key Manger for Barbican, implemented in Nova and Cinder, caches a value of barbican_client to save extra calls to Keystone for authentication. However, the cached value of barbican_client is only valid for the current context. A check needs to be made to ensure the context has not changed before using the saved value. The symptoms for using a stale cache value include getting the following error message when creating an encrypted volume. From CLI: --- openstack volume create --size 1 --type LUKS encrypted_volume The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-aea6be92-020e-41ed-ba88-44a1f5235ab0) In cinder.log --- 2015-12-03 09:09:03.648 TRACE cinder.volume.api Traceback (most recent call last): 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _exe cute_task 2015-12-03 09:09:03.648 TRACE cinder.volume.api result = task.execute(**arguments) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 409, in execute 2015-12-03 09:09:03.648 TRACE cinder.volume.api source_volume) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 338, in _get_encryption_key_ id 2015-12-03 09:09:03.648 TRACE cinder.volume.api encryption_key_id = key_manager.create_key(context) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/keymgr/barbican.py", line 147, in create_key 2015-12-03 09:09:03.648 TRACE cinder.volume.api LOG.exception(_LE("Error creating key.")) …. 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 502, in post 2015-12-03 09:09:03.648 TRACE cinder.volume.api return self.request(url, 'POST', **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner 2015-12-03 09:09:03.648 TRACE cinder.volume.api return func(*args, **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 402, in request 2015-12-03 09:09:03.648 TRACE cinder.volume.api raise exceptions.from_response(resp, method, url) 2015-12-03 09:09:03.648 TRACE cinder.volume.api Unauthorized: The request you have made requires authentication. (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-d2c52e0b-c16d-43ec-a7a0-763f1270) To manage notifications about this bug go to: https://bugs.launchpad.net/castellan/+bug/1523646/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1523646] Re: Nova/Cinder Key Manager for Barbican Uses Stale Cache
Reviewed: https://review.openstack.org/255323 Committed: https://git.openstack.org/cgit/openstack/castellan/commit/?id=43efbf1d5fc9b3ebfef38f2e4fa016c247fb15b1 Submitter: Jenkins Branch:master commit 43efbf1d5fc9b3ebfef38f2e4fa016c247fb15b1 Author: Dave McCowan Date: Wed Dec 9 10:37:24 2015 -0500 Move line of code to ensure context and client stay in sync If the barbican_client.Client() throws an exception, then self._current_context will not match self._barbican_client. This fix moves a line of code down to ensure they will match. Change-Id: I4e6291d98d9b2d37b3d5063b9b20fbb093d254d4 Closes-bug: #1523646 ** Changed in: castellan Status: New => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1523646 Title: Nova/Cinder Key Manager for Barbican Uses Stale Cache Status in castellan: Fix Released Status in Cinder: Fix Released Status in OpenStack Compute (nova): In Progress Status in OpenStack Security Notes: New Bug description: The Key Manger for Barbican, implemented in Nova and Cinder, caches a value of barbican_client to save extra calls to Keystone for authentication. However, the cached value of barbican_client is only valid for the current context. A check needs to be made to ensure the context has not changed before using the saved value. The symptoms for using a stale cache value include getting the following error message when creating an encrypted volume. From CLI: --- openstack volume create --size 1 --type LUKS encrypted_volume The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-aea6be92-020e-41ed-ba88-44a1f5235ab0) In cinder.log --- 2015-12-03 09:09:03.648 TRACE cinder.volume.api Traceback (most recent call last): 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _exe cute_task 2015-12-03 09:09:03.648 TRACE cinder.volume.api result = task.execute(**arguments) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 409, in execute 2015-12-03 09:09:03.648 TRACE cinder.volume.api source_volume) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 338, in _get_encryption_key_ id 2015-12-03 09:09:03.648 TRACE cinder.volume.api encryption_key_id = key_manager.create_key(context) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/keymgr/barbican.py", line 147, in create_key 2015-12-03 09:09:03.648 TRACE cinder.volume.api LOG.exception(_LE("Error creating key.")) …. 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 502, in post 2015-12-03 09:09:03.648 TRACE cinder.volume.api return self.request(url, 'POST', **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner 2015-12-03 09:09:03.648 TRACE cinder.volume.api return func(*args, **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 402, in request 2015-12-03 09:09:03.648 TRACE cinder.volume.api raise exceptions.from_response(resp, method, url) 2015-12-03 09:09:03.648 TRACE cinder.volume.api Unauthorized: The request you have made requires authentication. (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-d2c52e0b-c16d-43ec-a7a0-763f1270) To manage notifications about this bug go to: https://bugs.launchpad.net/castellan/+bug/1523646/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1523646] Re: Nova/Cinder Key Manager for Barbican Uses Stale Cache
** Changed in: nova Assignee: yuntongjin (yuntongjin) => Dave McCowan (dave-mccowan) ** Also affects: ossn Importance: Undecided Status: New ** Changed in: ossn Assignee: (unassigned) => Dave McCowan (dave-mccowan) ** Changed in: castellan Assignee: (unassigned) => Dave McCowan (dave-mccowan) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1523646 Title: Nova/Cinder Key Manager for Barbican Uses Stale Cache Status in castellan: New Status in Cinder: Fix Released Status in OpenStack Compute (nova): In Progress Status in OpenStack Security Notes: New Bug description: The Key Manger for Barbican, implemented in Nova and Cinder, caches a value of barbican_client to save extra calls to Keystone for authentication. However, the cached value of barbican_client is only valid for the current context. A check needs to be made to ensure the context has not changed before using the saved value. The symptoms for using a stale cache value include getting the following error message when creating an encrypted volume. From CLI: --- openstack volume create --size 1 --type LUKS encrypted_volume The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-aea6be92-020e-41ed-ba88-44a1f5235ab0) In cinder.log --- 2015-12-03 09:09:03.648 TRACE cinder.volume.api Traceback (most recent call last): 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _exe cute_task 2015-12-03 09:09:03.648 TRACE cinder.volume.api result = task.execute(**arguments) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 409, in execute 2015-12-03 09:09:03.648 TRACE cinder.volume.api source_volume) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 338, in _get_encryption_key_ id 2015-12-03 09:09:03.648 TRACE cinder.volume.api encryption_key_id = key_manager.create_key(context) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/keymgr/barbican.py", line 147, in create_key 2015-12-03 09:09:03.648 TRACE cinder.volume.api LOG.exception(_LE("Error creating key.")) …. 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 502, in post 2015-12-03 09:09:03.648 TRACE cinder.volume.api return self.request(url, 'POST', **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner 2015-12-03 09:09:03.648 TRACE cinder.volume.api return func(*args, **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 402, in request 2015-12-03 09:09:03.648 TRACE cinder.volume.api raise exceptions.from_response(resp, method, url) 2015-12-03 09:09:03.648 TRACE cinder.volume.api Unauthorized: The request you have made requires authentication. (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-d2c52e0b-c16d-43ec-a7a0-763f1270) To manage notifications about this bug go to: https://bugs.launchpad.net/castellan/+bug/1523646/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1523646] Re: Nova/Cinder Key Manager for Barbican Uses Stale Cache
Reviewed: https://review.openstack.org/254357 Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=0832a0355381229ece235440a9c5de1301e51d07 Submitter: Jenkins Branch:master commit 0832a0355381229ece235440a9c5de1301e51d07 Author: Dave McCowan Date: Mon Dec 7 14:25:24 2015 -0500 Check context before returning cached value The key manager caches the value of barbican client to be reused, saving an extra call to keystone. The cached value is only applicable to the current context, so the context must be checked before returning the cached value. Change-Id: Ib10909a098fb2cd070129c239b6d3b95edc8fea0 Closes-Bug: #1523646 ** Changed in: cinder Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1523646 Title: Nova/Cinder Key Manager for Barbican Uses Stale Cache Status in castellan: New Status in Cinder: Fix Released Status in OpenStack Compute (nova): In Progress Bug description: The Key Manger for Barbican, implemented in Nova and Cinder, caches a value of barbican_client to save extra calls to Keystone for authentication. However, the cached value of barbican_client is only valid for the current context. A check needs to be made to ensure the context has not changed before using the saved value. The symptoms for using a stale cache value include getting the following error message when creating an encrypted volume. From CLI: --- openstack volume create --size 1 --type LUKS encrypted_volume The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-aea6be92-020e-41ed-ba88-44a1f5235ab0) In cinder.log --- 2015-12-03 09:09:03.648 TRACE cinder.volume.api Traceback (most recent call last): 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _exe cute_task 2015-12-03 09:09:03.648 TRACE cinder.volume.api result = task.execute(**arguments) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 409, in execute 2015-12-03 09:09:03.648 TRACE cinder.volume.api source_volume) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 338, in _get_encryption_key_ id 2015-12-03 09:09:03.648 TRACE cinder.volume.api encryption_key_id = key_manager.create_key(context) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/keymgr/barbican.py", line 147, in create_key 2015-12-03 09:09:03.648 TRACE cinder.volume.api LOG.exception(_LE("Error creating key.")) …. 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 502, in post 2015-12-03 09:09:03.648 TRACE cinder.volume.api return self.request(url, 'POST', **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner 2015-12-03 09:09:03.648 TRACE cinder.volume.api return func(*args, **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 402, in request 2015-12-03 09:09:03.648 TRACE cinder.volume.api raise exceptions.from_response(resp, method, url) 2015-12-03 09:09:03.648 TRACE cinder.volume.api Unauthorized: The request you have made requires authentication. (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-d2c52e0b-c16d-43ec-a7a0-763f1270) To manage notifications about this bug go to: https://bugs.launchpad.net/castellan/+bug/1523646/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1523646] Re: Nova/Cinder Key Manager for Barbican Uses Stale Cache
** Also affects: castellan Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1523646 Title: Nova/Cinder Key Manager for Barbican Uses Stale Cache Status in castellan: New Status in Cinder: In Progress Status in OpenStack Compute (nova): In Progress Bug description: The Key Manger for Barbican, implemented in Nova and Cinder, caches a value of barbican_client to save extra calls to Keystone for authentication. However, the cached value of barbican_client is only valid for the current context. A check needs to be made to ensure the context has not changed before using the saved value. The symptoms for using a stale cache value include getting the following error message when creating an encrypted volume. From CLI: --- openstack volume create --size 1 --type LUKS encrypted_volume The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-aea6be92-020e-41ed-ba88-44a1f5235ab0) In cinder.log --- 2015-12-03 09:09:03.648 TRACE cinder.volume.api Traceback (most recent call last): 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/taskflow/engines/action_engine/executor.py", line 82, in _exe cute_task 2015-12-03 09:09:03.648 TRACE cinder.volume.api result = task.execute(**arguments) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 409, in execute 2015-12-03 09:09:03.648 TRACE cinder.volume.api source_volume) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/volume/flows/api/create_volume.py", line 338, in _get_encryption_key_ id 2015-12-03 09:09:03.648 TRACE cinder.volume.api encryption_key_id = key_manager.create_key(context) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/opt/stack/cinder/cinder/keymgr/barbican.py", line 147, in create_key 2015-12-03 09:09:03.648 TRACE cinder.volume.api LOG.exception(_LE("Error creating key.")) …. 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 502, in post 2015-12-03 09:09:03.648 TRACE cinder.volume.api return self.request(url, 'POST', **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in inner 2015-12-03 09:09:03.648 TRACE cinder.volume.api return func(*args, **kwargs) 2015-12-03 09:09:03.648 TRACE cinder.volume.api File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 402, in request 2015-12-03 09:09:03.648 TRACE cinder.volume.api raise exceptions.from_response(resp, method, url) 2015-12-03 09:09:03.648 TRACE cinder.volume.api Unauthorized: The request you have made requires authentication. (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-d2c52e0b-c16d-43ec-a7a0-763f1270) To manage notifications about this bug go to: https://bugs.launchpad.net/castellan/+bug/1523646/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp