[Yahoo-eng-team] [Bug 1885002] Re: I get error 500 when trying to consume GET /os-simple-tenant-usage/{tenant_id}
[Expired for OpenStack Compute (nova) because there has been no activity
for 60 days.]
** Changed in: nova
Status: Incomplete => Expired
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1885002
Title:
I get error 500 when trying to consume GET /os-simple-tenant-
usage/{tenant_id}
Status in OpenStack Compute (nova):
Expired
Bug description:
I am trying to consume the api GET /os-simple-tenant-usage/{tenant_id}
with the start and end parameters as follows: GET /os-simple-tenant-
usage/{tenant_id}?start=2020-06-17T12:48:50=2020-06-24T12:48:50.
When I use the above, I get error 500.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1885002/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1892364] Re: L3 agent prefix delegation - adding new subnet to the router fails
Reviewed: https://review.opendev.org/747290 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=13b894288e9354b9eb0833a635fe4a624bb068a9 Submitter: Zuul Branch:master commit 13b894288e9354b9eb0833a635fe4a624bb068a9 Author: Slawek Kaplonski Date: Thu Aug 20 22:39:04 2020 +0200 Fix deletion of subnet_id from pd_subnets In the RouterInfo._process_internal_ports() method when it process old ports and port belongs to the subnet with CIDR assigned by Prefix Delegation it will try to remove subnet_id key from the pd_subnets dict. However it seems that in some case it may happen that such subnet_id key is not added to the pd_subnets dict and processing of ports fails. We shouldn't fail in such case, if there is no subnet_id key in this dict we should be good as we want to delete it simply. So this patch changes that to not raise KeyError in such case. Change-Id: I6e6d890c196716c0ef4bcc2922f1ec4c142a6e79 Closes-Bug: #1892364 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1892364 Title: L3 agent prefix delegation - adding new subnet to the router fails Status in neutron: Fix Released Bug description: Adding new subnet with prefix delegation enabled to the router fails with error like: 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent [-] Error while deleting router 7ef0983f-60a4-4389-909c-785eb6c9a51e: KeyError: '999e404c-3c33-4123-9c61-99426f2e5c16' 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent Traceback (most recent call last): 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 438, in _safe_router_removed 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self._router_removed(ri, router_id) 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 474, in _router_removed 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self.router_info[router_id] = ri 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self.force_reraise() 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent raise value 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/agent.py", line 471, in _router_removed 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent ri.delete() 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 433, in delete 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self.process_delete() 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 161, in call 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self.logger(e) 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self.force_reraise() 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent six.reraise(self.type_, self.value, self.tb) 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent raise value 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/common/utils.py", line 158, in call 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent return func(*args, **kwargs) 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent File "/usr/lib/python3.6/site-packages/neutron/agent/l3/router_info.py", line 1167, in process_delete 2020-08-20 11:22:41.356 1004285 ERROR neutron.agent.l3.agent self._process_internal_ports() 2020-08-20
[Yahoo-eng-team] [Bug 1889936] Re: Using Microsoft AD's objectGUID attribute as user_id_attribute breaks
Reviewed: https://review.opendev.org/744251 Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8bf222ac5d390e25d306d35f69bd958b18bee4d8 Submitter: Zuul Branch:master commit 8bf222ac5d390e25d306d35f69bd958b18bee4d8 Author: Lance Bragstad Date: Fri Jul 31 15:28:15 2020 -0500 Properly handle octet (byte) strings when converting LDAP responses If LDAP returns a UUID as an octet string the LDAP driver will fail to convert it to something meaningful. The error usually looks something like: ID attribute objectGUID not found in LDAP object Microsoft AD's `objectGUID` parameter is stored and transmitted as an octet string [0]. If you attempt to use the `objectGUID` to generate user or group IDs, you'll get an HTTP 404 because keystone can't decode it properly. This is unfortunate because `objectGUID` are a fixed length, UUID format, and ideal for generating IDs in keystone. As opposed to using the object's CN, which is variable length, and can generate hashes that are larger than keystone's database table limit for user IDs. [0] https://docs.microsoft.com/en-us/windows/win32/ad/reading-an-objectampaposs-objectguid-and-creating-a-string-representation-of-the-guid Change-Id: Id80b17bdff015e10340e636102576b7435bd564f Closes-Bug: 1889936 ** Changed in: keystone Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1889936 Title: Using Microsoft AD's objectGUID attribute as user_id_attribute breaks Status in OpenStack Identity (keystone): Fix Released Bug description: Microsoft AD has a default attribute in its schema for users and groups called objectGUID [0]. The attribute is assigned when new users and groups are created. If we attempt to use this attribute as a user's ID by setting user_id_attribute=objectGUID, keystone throws an HTTP 404 listing users. I was able to recreate this with using a 2016 AD server with the following keystone LDAP configuration file: [root@overcloud-controller-0 /]# cat /etc/keystone/domains/keystone.windows.conf [ldap] url=ldap://192.168.1.209 user=CN=Administrator,CN=Users,DC=mycompany,DC=local password=p@ssw0rd1 suffix=DC=mycompany,DC=local user_tree_dn=DC=mycompany,DC=local user_objectclass=user user_id_attribute=objectGUID query_scope=sub user_name_attribute=sAMAccountName [identity] driver=ldap [stack@undercloud ~]$ openstack --os-cloud overcloud user list --domain windows ID attribute objectGUID not found in LDAP object CN=Administrator,CN=Users,DC=mycompany,DC=local (HTTP 404) (Request-ID: req-cdc056fd-2ebc-4a24-81d9-8a0948fc56bf) The root of the issue is that keystone isn't properly decoding the value, which you can see from the logs as the ldap backend processes results from AD. /var/log/containers/keystone/keystone.log:2020-07-31 18:58:36.319 19 WARNING keystone.common.wsgi [req-ae63fcc8-64a1-48f5-96c9-c107df9c4be0 c4c7be68b7304ba884a0d15f3f882df4 b970c795de7749af821f1d53b51fa0d5 - default default] ID attribute objectGUID not found in LDAP object CN=Administrator,CN=Users,DC=mycompany,DC=local: NotFound: ID attribute objectGUID not found in LDAP object CN=Administrator,CN=Users,DC=mycompany,DC=local Relevant code: https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L1328-L1330 https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L950 https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L144-L183 https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L174 https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L141 https://opendev.org/openstack/keystone/src/commit/bcc751b3a24a93b5d8aab1bfb7eb8027d8499e36/keystone/identity/backends/ldap/common.py#L81-L97 We might need to consider handling this value similar what the python- ldap community suggests: https://mail.python.org/pipermail/python-ldap/2014q3/003410.html [0] https://docs.microsoft.com/en- us/windows/win32/adschema/a-objectguid To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1889936/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1890400] Re: Default gateway in HA router namespace not set if using Keepalived 1.x
Reviewed: https://review.opendev.org/745641 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=7abe0ee34c367b4abf84820048b4aed643fc1162 Submitter: Zuul Branch:master commit 7abe0ee34c367b4abf84820048b4aed643fc1162 Author: Slawek Kaplonski Date: Tue Aug 11 10:47:24 2020 +0200 Add 'keepalived_use_no_track' config option Patch [1] added option "no_track" to the keepalived's config file which is generated by L3 agent in HA mode. This was added to handle properly keepalived 2.x and interfaces which are in DOWN state in the backup nodes. But this "no_track" option is not compatible with keepalived 1.x series which is available e.g. on Ubuntu 18.04. As there is no easy way to check automatically if keepalived supports or not this config flag, this patch introduces new config option "keepalived_use_no_track". If this config option will be set to False, neutron L3 agent will not add "no_track" to the keepalived's config. As master branch is moving to gate on Ubuntu 20.04 where keepalived 2.x is already available, this new config option default value is set to True. [1] https://review.opendev.org/#/c/721799/ Change-Id: I2dfdb9f56de28d56ca0f240ff34fa7c3a12e339b Closes-Bug: #1890400 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1890400 Title: Default gateway in HA router namespace not set if using Keepalived 1.x Status in neutron: Fix Released Bug description: With Rocky commit f2d234e introduces a change making Neutron more compatible with Keepalived 2.x. A new option `no_track` is passed to Keepalived by the L3 agent which Keepalived 1.x doesn't recognize, thus the configuration is partially not applied by Keepalived 1.x. resulting in a missing default gateway. Thus instances using HA routers to communicate with the outside world are unable to reach the internet and cannot be reached via their floating IP addresses. There are some workarounds to trigger the creating of the default gateway like disable and enable the router again or disable/re-start the L3 agent which hosts the master namespace of the HA router. Steps to reproduce: 1. Create an HA router 2. Add the gateway network (now the default gateway should be set within the routers master network namespace but there is none) 3. Connect a tenant subnet to the router 4. Create an instance connected to the tenant network created in step 3 5. Try to reach the internet from within the instances created in step 4 I was able to get some log output out of Keepalived: Tue Aug 4 11:59:58 2020: Starting Keepalived v1.3.9 (10/21,2017) Tue Aug 4 11:59:58 2020: Opening file '/var/lib/neutron/ha_confs/019f8036-8730-4584-9e07-c4a6504447ab/keepalived.conf'. Tue Aug 4 11:59:58 2020: Starting VRRP child process, pid=2864 Tue Aug 4 11:59:58 2020: Registering Kernel netlink reflector Tue Aug 4 11:59:58 2020: Registering Kernel netlink command channel Tue Aug 4 11:59:58 2020: Registering gratuitous ARP shared channel Tue Aug 4 11:59:58 2020: Opening file '/var/lib/neutron/ha_confs/019f8036-8730-4584-9e07-c4a6504447ab/keepalived.conf'. Tue Aug 4 11:59:58 2020: Unknown configuration entry 'no_track' for ip address - ignoring Tue Aug 4 11:59:58 2020: Unknown configuration entry 'no_track' for ip address - ignoring Tue Aug 4 11:59:58 2020: Cannot specify scope for IPv6 addresses (fe80::f816:3eff:fe2c:6622/64) - ignoring scope Tue Aug 4 11:59:58 2020: VRRP parsed invalid IP no_track. skipping IP... Tue Aug 4 11:59:58 2020: unknown route keyword no_track Tue Aug 4 11:59:58 2020: VRRP_Instance(VR_23) removing protocol VIPs. Tue Aug 4 11:59:58 2020: VRRP_Instance(VR_23) removing protocol E-VIPs. Tue Aug 4 11:59:58 2020: Using LinkWatch kernel netlink reflector... Tue Aug 4 11:59:58 2020: VRRP_Instance(VR_23) Entering BACKUP STATE Tue Aug 4 11:59:58 2020: VRRP sockpool: [ifindex(1033), proto(112), unicast(0), fd(9,10)] You see it complaining about the 'no_track' option. We use Kolla containers with Ubuntu base. Even though Keepalived 2 was release quite a while ago, it still only provides Keepalived 1.x via the package repositories. Even Kolla in the latest version still uses Ubuntu 18.04 as base with Keepalived 1.x. Theoretically all users using Kolla containers with Ubuntu base (other base images not tested) are affected. There seems to be no apt sources for Keepalived 2.x for Ubuntu 18.04. You need to compile it from source in order to get a newer version. Maybe it should depend on the Keepalived version whether to pass the 'no_track' option or make it configurable. To manage notifications about this bug go to:
[Yahoo-eng-team] [Bug 1892741] [NEW] neutron-sriov-nic-agent cannot be disabled
Public bug reported: During a test, neutron-sriov-nic-agent was disabled (openstack network agent set --disable ) and then some operations that require the intervention of this agent were successfully performned. So, apparently the agent is not really disabled, despite showing state=DOWN. Please see the bug reproduction here: http://paste.openstack.org/show/797088/ Apparently, only DHCP and L3 agents receive the status flag. In other words: only those two agents are capable of being enabled or disabled: https://github.com/openstack/neutron/blob/master/neutron/db/agentschedulers_db.py#L53-L56 Maybe the API should answer the --disable request with an error, instead of 200 OK. ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1892741 Title: neutron-sriov-nic-agent cannot be disabled Status in neutron: New Bug description: During a test, neutron-sriov-nic-agent was disabled (openstack network agent set --disable ) and then some operations that require the intervention of this agent were successfully performned. So, apparently the agent is not really disabled, despite showing state=DOWN. Please see the bug reproduction here: http://paste.openstack.org/show/797088/ Apparently, only DHCP and L3 agents receive the status flag. In other words: only those two agents are capable of being enabled or disabled: https://github.com/openstack/neutron/blob/master/neutron/db/agentschedulers_db.py#L53-L56 Maybe the API should answer the --disable request with an error, instead of 200 OK. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1892741/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1892017] Re: Neutron server logs are too big in the gate jobs
Reviewed: https://review.opendev.org/746714 Committed: https://git.openstack.org/cgit/openstack/neutron-lib/commit/?id=024b5af64b69a6fb9eab7c5764d8f2b91f17ec2f Submitter: Zuul Branch:master commit 024b5af64b69a6fb9eab7c5764d8f2b91f17ec2f Author: Slawek Kaplonski Date: Tue Aug 18 15:13:36 2020 +0200 Log time of the apply_func only if it took more than 0.1 second It seems that logging start and end time of the neutron_lib.db.resource_extend.apply_func function is little bit too much for e.g. our gate as neutron-server logs are huge in such case. So lets use directly oslo_utils.timeutils.time_it() function and log end of such calls only if it will take more than 0.1 seconds. If function was executed in shorter time, it isn't really necessary to have it logged. Change-Id: I3931c4f7cac4df72d71403ebcaa6ec0b83756640 Closes-Bug: #1892017 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1892017 Title: Neutron server logs are too big in the gate jobs Status in neutron: Fix Released Bug description: As per email http://lists.openstack.org/pipermail/openstack-discuss/2020-August/016579.html we should not log all time-cost entries in the server logs for every job. Maybe we can do it only in the rally based jobs and disable it everywhere else? To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1892017/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1892679] [NEW] Missing Integration test for Volume Group-type
Public bug reported: Integration test for different volume group-type operations are missing. ** Affects: horizon Importance: Medium Assignee: Vishal Manchanda (vishalmanchanda) Status: Confirmed ** Tags: integration-tests ** Changed in: horizon Status: New => Confirmed ** Changed in: horizon Importance: Undecided => Medium ** Tags added: integration-tests ** Changed in: horizon Assignee: (unassigned) => Vishal Manchanda (vishalmanchanda) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1892679 Title: Missing Integration test for Volume Group-type Status in OpenStack Dashboard (Horizon): Confirmed Bug description: Integration test for different volume group-type operations are missing. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1892679/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
