[Yahoo-eng-team] [Bug 1841486] [NEW] federation mapping debug has useless direct_maps information
Public bug reported: If you use keystone-manage mapping_engine --engine-debug to test your rules (or when debug logging is on during run time) the diagnostic output fails to emit a piece of crucial information, the contents direct map array. What you'll get instead is this: direct_maps: That's because the DirectMaps class does not have a __str__() method and Python resorts to __ref__() in the absence of __str__() and all __ref__() does is print the class name and it's memory location, not very useful. If DirectMaps had a __str__() function like this: def __str__(self): return '%s' % self._matches the debug output would include the actual direct map data like this: direct_maps: [['j...@example.com'], ['Group1', 'Group3']] ** Affects: keystone Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1841486 Title: federation mapping debug has useless direct_maps information Status in OpenStack Identity (keystone): New Bug description: If you use keystone-manage mapping_engine --engine-debug to test your rules (or when debug logging is on during run time) the diagnostic output fails to emit a piece of crucial information, the contents direct map array. What you'll get instead is this: direct_maps: That's because the DirectMaps class does not have a __str__() method and Python resorts to __ref__() in the absence of __str__() and all __ref__() does is print the class name and it's memory location, not very useful. If DirectMaps had a __str__() function like this: def __str__(self): return '%s' % self._matches the debug output would include the actual direct map data like this: direct_maps: [['j...@example.com'], ['Group1', 'Group3']] To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1841486/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1804073] Re: Keystone fails to log policy target data
** No longer affects: keystone -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1804073 Title: Keystone fails to log policy target data Status in oslo.policy: New Bug description: The Oslo Policy Enforcer requires 3 pieces of run-time information in addition to the policy rules to issue a RBAC decision: 1) the name of the rule to be evaluated (called target in the oslo-policy doc) 2) the auth context (called credentials in the oslo-policy doc) 3) the target data (resource data relevant to the rule) If you are trying to debug policy enforcement or simply validate your policy works as expect one can use the oslopolicy-checker tool. But the oslopolicy-checker tool needs the *exact* same data keystone passes to the policy enforcement engine. The fact the target data needs to be logged but isn't is captured in this comment from Henry Nash in authorize.py # TODO(henry-nash) need to log the target attributes as well https://github.com/openstack/keystone/blob/stable/rocky/keystone/common/authorization.py#L139 But that is not the best location to log, the best place is where oslo.policy is called to evaluate a policy rule, that occurs in Policy.enforce() in keystone/policy/backends/policy.py https://github.com/openstack/keystone/blob/stable/rocky/keystone/policy/backends/rules.py#L29:#L34 Here we can see it logs the rule name (e.g. action) and the auth context (credentials) msg = 'enforce %(action)s: %(credentials)s' but the target data is not logged. Besides the fact the target data is not logged is the fact the logging relies on Python's str() method to convert an object into a string representation. This has two problems, 1) all contained objects must also have __str__() methods that fully log their contents, 2) the formatting is often in Python's "representation" style which only humans and Python can parse. Since both the credential and targets parameters to the enforce method are dicts (with arbitrary complex nesting) and the fact JSON is the data format we use for data exchange and the format used by oslopolicy-checker it makes sense to log the enforcement parameters in JSON format. This way no data is lost (because there wasn't an appropriate formatter for the object) and it makes it easy import the data to another tool (again, without loss of data). To manage notifications about this bug go to: https://bugs.launchpad.net/oslo.policy/+bug/1804073/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1804073] [NEW] Keystone fails to log policy target data
Public bug reported: The Oslo Policy Enforcer requires 3 pieces of run-time information in addition to the policy rules to issue a RBAC decision: 1) the name of the rule to be evaluated (called target in the oslo-policy doc) 2) the auth context (called credentials in the oslo-policy doc) 3) the target data (resource data relevant to the rule) If you are trying to debug policy enforcement or simply validate your policy works as expect one can use the oslopolicy-checker tool. But the oslopolicy-checker tool needs the *exact* same data keystone passes to the policy enforcement engine. The fact the target data needs to be logged but isn't is captured in this comment from Henry Nash in authorize.py # TODO(henry-nash) need to log the target attributes as well https://github.com/openstack/keystone/blob/stable/rocky/keystone/common/authorization.py#L139 But that is not the best location to log, the best place is where oslo.policy is called to evaluate a policy rule, that occurs in Policy.enforce() in keystone/policy/backends/policy.py https://github.com/openstack/keystone/blob/stable/rocky/keystone/policy/backends/rules.py#L29:#L34 Here we can see it logs the rule name (e.g. action) and the auth context (credentials) msg = 'enforce %(action)s: %(credentials)s' but the target data is not logged. Besides the fact the target data is not logged is the fact the logging relies on Python's str() method to convert an object into a string representation. This has two problems, 1) all contained objects must also have __str__() methods that fully log their contents, 2) the formatting is often in Python's "representation" style which only humans and Python can parse. Since both the credential and targets parameters to the enforce method are dicts (with arbitrary complex nesting) and the fact JSON is the data format we use for data exchange and the format used by oslopolicy- checker it makes sense to log the enforcement parameters in JSON format. This way no data is lost (because there wasn't an appropriate formatter for the object) and it makes it easy import the data to another tool (again, without loss of data). ** Affects: keystone Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1804073 Title: Keystone fails to log policy target data Status in OpenStack Identity (keystone): New Bug description: The Oslo Policy Enforcer requires 3 pieces of run-time information in addition to the policy rules to issue a RBAC decision: 1) the name of the rule to be evaluated (called target in the oslo-policy doc) 2) the auth context (called credentials in the oslo-policy doc) 3) the target data (resource data relevant to the rule) If you are trying to debug policy enforcement or simply validate your policy works as expect one can use the oslopolicy-checker tool. But the oslopolicy-checker tool needs the *exact* same data keystone passes to the policy enforcement engine. The fact the target data needs to be logged but isn't is captured in this comment from Henry Nash in authorize.py # TODO(henry-nash) need to log the target attributes as well https://github.com/openstack/keystone/blob/stable/rocky/keystone/common/authorization.py#L139 But that is not the best location to log, the best place is where oslo.policy is called to evaluate a policy rule, that occurs in Policy.enforce() in keystone/policy/backends/policy.py https://github.com/openstack/keystone/blob/stable/rocky/keystone/policy/backends/rules.py#L29:#L34 Here we can see it logs the rule name (e.g. action) and the auth context (credentials) msg = 'enforce %(action)s: %(credentials)s' but the target data is not logged. Besides the fact the target data is not logged is the fact the logging relies on Python's str() method to convert an object into a string representation. This has two problems, 1) all contained objects must also have __str__() methods that fully log their contents, 2) the formatting is often in Python's "representation" style which only humans and Python can parse. Since both the credential and targets parameters to the enforce method are dicts (with arbitrary complex nesting) and the fact JSON is the data format we use for data exchange and the format used by oslopolicy-checker it makes sense to log the enforcement parameters in JSON format. This way no data is lost (because there wasn't an appropriate formatter for the object) and it makes it easy import the data to another tool (again, without loss of data). To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1804073/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help
[Yahoo-eng-team] [Bug 1776541] [NEW] Enhance PCI-DSS compliance documentation
Public bug reported: Keystone provides some documentation on PCI-DSS compliance but it's less than ideal if you're trying to answer the following questions: * What are the PCI-DSS requirements? * How does Keystone satisfy the requirements? * What release did Keystone add support for a given requirement? * How do you configure to meet the requirement? You'll discover the information is (mostly) there but it's scattered across several documents, release notes, etc. It would be good to have one document that pulls all the information listed above into one location to serve as a focal point for those needing to understand PCI- DSS compliance. I have written such a document. Rather than duplicate the information in the other documents it references the information via links where possible. This bug is mostly to have something to reference for the Gerrit review for when the doc is submitted. ** Affects: keystone Importance: Undecided Assignee: John Dennis (jdennis-a) Status: New ** Changed in: keystone Assignee: (unassigned) => John Dennis (jdennis-a) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1776541 Title: Enhance PCI-DSS compliance documentation Status in OpenStack Identity (keystone): New Bug description: Keystone provides some documentation on PCI-DSS compliance but it's less than ideal if you're trying to answer the following questions: * What are the PCI-DSS requirements? * How does Keystone satisfy the requirements? * What release did Keystone add support for a given requirement? * How do you configure to meet the requirement? You'll discover the information is (mostly) there but it's scattered across several documents, release notes, etc. It would be good to have one document that pulls all the information listed above into one location to serve as a focal point for those needing to understand PCI-DSS compliance. I have written such a document. Rather than duplicate the information in the other documents it references the information via links where possible. This bug is mostly to have something to reference for the Gerrit review for when the doc is submitted. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1776541/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1776532] [NEW] LDAP backend should support python-ldap trace logging
Public bug reported: The python-ldap library has a diagnostic and debugging feature called trace logging. The information in the trace log is crucial when trying to diagnose LDAP problems, especially connection problems. This is because what is visible at the Keystone backend is obscured by 2 other abstraction layers, the OpenStack ldappool library and the ReconnectLDAPObject implementation in python-ldap. When connection problems occur you need to be able to see what happened at the lowest level in order to understand what the upper abstraction layers are doing. Trace logging is also useful for other LDAP information besides connection issues. python-ldap controls trace logging with these two parameters: trace_level: An integer controlling the verbosity of the trace information trace_file: A Python file object used when writing trace info. Unfortunately as of today there is no way to turn on trace logging other than editing the source code to change the parameters passed into various python-ldap methods. As of python-ldap 3.1.0 you can set the environment variables PYTHON_LDAP_TRACE_LEVEL PYTHON_LDAP_TRACE_FILE (a pathname) to set these values without a code change. This version of python-ldap is very new (May 2018), however setting environment variables to turn on trace logging is not easy because of the way Keystone is deployed as an operating system service. It would be preferable to add two new configuration options to the LDAP section to control the trace_level and trace_file and have the ldap backend set these values when creating python-ldap objects. It would be good to set the trace_file to the same logging file object the rest of the backend uses so the information is contained in one place and interleaved. Also note there is already a LDAP debug level in the config, 'debug_level', which turns on debugging in the openldap C library via the OPT_DEBUG_LEVEL ldap option. python-ldap calls this library to perform many of it's operations and as such is one level below python- ldap. This debug feature is independent of the trace facility in python- ldap. We need both facilities. ** Affects: keystone Importance: Undecided Assignee: John Dennis (jdennis-a) Status: New ** Changed in: keystone Assignee: (unassigned) => John Dennis (jdennis-a) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1776532 Title: LDAP backend should support python-ldap trace logging Status in OpenStack Identity (keystone): New Bug description: The python-ldap library has a diagnostic and debugging feature called trace logging. The information in the trace log is crucial when trying to diagnose LDAP problems, especially connection problems. This is because what is visible at the Keystone backend is obscured by 2 other abstraction layers, the OpenStack ldappool library and the ReconnectLDAPObject implementation in python-ldap. When connection problems occur you need to be able to see what happened at the lowest level in order to understand what the upper abstraction layers are doing. Trace logging is also useful for other LDAP information besides connection issues. python-ldap controls trace logging with these two parameters: trace_level: An integer controlling the verbosity of the trace information trace_file: A Python file object used when writing trace info. Unfortunately as of today there is no way to turn on trace logging other than editing the source code to change the parameters passed into various python-ldap methods. As of python-ldap 3.1.0 you can set the environment variables PYTHON_LDAP_TRACE_LEVEL PYTHON_LDAP_TRACE_FILE (a pathname) to set these values without a code change. This version of python-ldap is very new (May 2018), however setting environment variables to turn on trace logging is not easy because of the way Keystone is deployed as an operating system service. It would be preferable to add two new configuration options to the LDAP section to control the trace_level and trace_file and have the ldap backend set these values when creating python-ldap objects. It would be good to set the trace_file to the same logging file object the rest of the backend uses so the information is contained in one place and interleaved. Also note there is already a LDAP debug level in the config, 'debug_level', which turns on debugging in the openldap C library via the OPT_DEBUG_LEVEL ldap option. python-ldap calls this library to perform many of it's operations and as such is one level below python- ldap. This debug feature is independent of the trace facility in python-ldap. We need both facilities. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1776532/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-
[Yahoo-eng-team] [Bug 1655182] [NEW] keystone-manage mapping_engine tester problems
Public bug reported: There are several problems with keystone-manage mapping_engine * It aborts with a backtrace because of wrong number of arguments passed to the RuleProcessor * The --engine-debug option does not work. * Error messages related to input data are cryptic and inprecise. ** Affects: keystone Importance: Undecided Assignee: John Dennis (jdennis-a) Status: In Progress -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1655182 Title: keystone-manage mapping_engine tester problems Status in OpenStack Identity (keystone): In Progress Bug description: There are several problems with keystone-manage mapping_engine * It aborts with a backtrace because of wrong number of arguments passed to the RuleProcessor * The --engine-debug option does not work. * Error messages related to input data are cryptic and inprecise. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1655182/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
