[Yahoo-eng-team] [Bug 2008858] [NEW] Call the api and do not return for a long time
Public bug reported: I have an environment for using ovn. I found that sometimes when calling the api, it can return quickly, and sometimes it never returns. This phenomenon also occurs when calling the same interface. There is no error when the interface does not return. After tracing the log, we found that some of the neutral-server processes reported errors when they were just started from log, I found: when we call post_fork_initialization and add_node to db failed. After analyzing the code, we can see that the error process did not connect to ovn-nb/sb and not set _post_fork_event. However, the api process does not exit, and can receive api requests. When processing requests requires access to ovn-nb, it will always wait. It is also difficult to troubleshoot problems. It can throw a wait timeout(=cfg.CONF.ovn.ovsdb_connection_timeout) to help troubleshoot problems. error log: 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines [None req-2537f423-6d58-48ef-995f-b6f1e1bac8d9 - - - - -] Database connection was found disconnected; reconnecting: oslo_db.exception.DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') [SQL: 'SELECT 1'] (Background on this error at: http://sqlalche.me/e/e3q8) 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines Traceback (most recent call last): 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File "/var/lib/kolla/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines context) 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File "/var/lib/kolla/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines cursor.execute(statement, parameters) 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File "/var/lib/kolla/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 170, in execute 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines result = self._query(query) 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File "/var/lib/kolla/venv/lib/python3.6/site-packages/pymysql/cursors.py", line 328, in _query 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines conn.query(q) 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File "/var/lib/kolla/venv/lib/python3.6/site-packages/pymysql/connections.py", line 516, in query 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File "/var/lib/kolla/venv/lib/python3.6/site-packages/pymysql/connections.py", line 727, in _read_query_result 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines result.read() 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File "/var/lib/kolla/venv/lib/python3.6/site-packages/pymysql/connections.py", line 1066, in read 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines first_packet = self.connection._read_packet() 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File "/var/lib/kolla/venv/lib/python3.6/site-packages/pymysql/connections.py", line 656, in _read_packet 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines packet_header = self._read_bytes(4) 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File "/var/lib/kolla/venv/lib/python3.6/site-packages/pymysql/connections.py", line 702, in _read_bytes 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query') 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines The above exception was the direct cause of the following exception: 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines Traceback (most recent call last): 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File "/var/lib/kolla/venv/lib/python3.6/site-packages/oslo_db/sqlalchemy/engines.py", line 73, in _connect_ping_listener 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines connection.scalar(select([1])) 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File "/var/lib/kolla/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 880, in scalar 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines return self.execute(object, *multiparams, **params).scalar() 2023-02-18 03:40:01.845 14 ERROR oslo_db.sqlalchemy.engines File
[Yahoo-eng-team] [Bug 2006952] Re: Ambigous error when trying to boot SEV based instances from volume
Reviewed: https://review.opendev.org/c/openstack/nova/+/873388 Committed: https://opendev.org/openstack/nova/commit/54faea0196c96ae55a58cab4326277d48a59afb0 Submitter: "Zuul (22348)" Branch:master commit 54faea0196c96ae55a58cab4326277d48a59afb0 Author: Alexey Stupnikov Date: Fri Feb 10 17:14:17 2023 +0100 Fix logging in MemEncryption-related checks Currently Nova produces ambigous error when volume-backed instance is started using flavor with hw:mem_encryption extra_specs flag: ImageMeta doesn't contain name if it represents Cinder volume. This fix sligtly changes steps to get image_meta.name for some MemEncryption-related checks where it could make any difference. Closes-bug: #2006952 Change-Id: Ia69e7cb18cd862f01ecfdbdc358c87af1ab8fbf6 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2006952 Title: Ambigous error when trying to boot SEV based instances from volume Status in OpenStack Compute (nova): Fix Released Bug description: More image_meta.name use in hardware.py causing failures when presented with an empty ImageMeta object: $ openstack flavor show m1.sev_med ++--+ | Field | Value| ++--+ | OS-FLV-DISABLED:disabled | False| | OS-FLV-EXT-DATA:ephemeral | 0| | access_project_ids | None | | description| None | | disk | 2| | extra_specs| {'hw:mem_encryption': 'True'}| | id | 3952db4d-e71a-4669-9bb7-666adaef6c36 | | name | m1.sev_med | | os-flavor-access:is_public | True | | properties | hw:mem_encryption='True' | | ram| 2048 | | rxtx_factor| 1.0 | | swap | 0| | vcpus | 4| ++--+ $ openstack volume create --bootable --size 1 blank $ openstack server create --volume blank --flavor m1.sev_med --network private test Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. (HTTP 500) (Request-ID: req-c6d1a319-b4b0-4d1f-869c-dcbec2fd2554) 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi [req-c6d1a319-b4b0-4d1f-869c-dcbec2fd2554 cd40fe796ff84e3a8ba5e473a6d61f05 025f8a0d412642f693782ae20ba415ec - default default] Unexpected exception in API method: NotImplementedError: Cannot load 'name' in the base class 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/nova/api/openstack/wsgi.py", line 671, in wrapped 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi return f(*args, **kwargs) 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/nova/api/validation/__init__.py", line 110, in wrapper 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi return func(*args, **kwargs) 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/nova/api/validation/__init__.py", line 110, in wrapper 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi return func(*args, **kwargs) 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/nova/api/validation/__init__.py", line 110, in wrapper 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi return func(*args, **kwargs) 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi [Previous line repeated 9 more times] 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/nova/api/openstack/compute/servers.py", line 712, in create 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi **create_kwargs) 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi File "/usr/lib/python3.6/site-packages/nova/hooks.py", line 154, in inner 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi rv = f(*args, **kwargs) 2021-06-03 12:29:28.207 12 ERROR nova.api.openstack.wsgi File
[Yahoo-eng-team] [Bug 2008808] [NEW] Duplicate packet when ping to external with out floating ip
Public bug reported: i build openstack yoga cluster: 3 node control, 2 node network, 2 node compute. i create network and router success. i create Instance and ping to external network - when use floating ip: ok - without floating ip: packer respond is duplicate ubuntu@msql:~$ ping 1.1.1.1 PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data. 64 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=26.1 ms 64 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=26.1 ms (DUP!) 64 bytes from 1.1.1.1: icmp_seq=2 ttl=56 time=26.0 ms 64 bytes from 1.1.1.1: icmp_seq=2 ttl=56 time=26.0 ms (DUP!) 64 bytes from 1.1.1.1: icmp_seq=3 ttl=56 time=25.0 ms 64 bytes from 1.1.1.1: icmp_seq=3 ttl=56 time=25.0 ms (DUP!) 64 bytes from 1.1.1.1: icmp_seq=4 ttl=56 time=24.6 ms 64 bytes from 1.1.1.1: icmp_seq=4 ttl=56 time=24.6 ms (DUP!) 64 bytes from 1.1.1.1: icmp_seq=5 ttl=56 time=24.5 ms 64 bytes from 1.1.1.1: icmp_seq=5 ttl=56 time=24.5 ms (DUP!) ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2008808 Title: Duplicate packet when ping to external with out floating ip Status in neutron: New Bug description: i build openstack yoga cluster: 3 node control, 2 node network, 2 node compute. i create network and router success. i create Instance and ping to external network - when use floating ip: ok - without floating ip: packer respond is duplicate ubuntu@msql:~$ ping 1.1.1.1 PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data. 64 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=26.1 ms 64 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=26.1 ms (DUP!) 64 bytes from 1.1.1.1: icmp_seq=2 ttl=56 time=26.0 ms 64 bytes from 1.1.1.1: icmp_seq=2 ttl=56 time=26.0 ms (DUP!) 64 bytes from 1.1.1.1: icmp_seq=3 ttl=56 time=25.0 ms 64 bytes from 1.1.1.1: icmp_seq=3 ttl=56 time=25.0 ms (DUP!) 64 bytes from 1.1.1.1: icmp_seq=4 ttl=56 time=24.6 ms 64 bytes from 1.1.1.1: icmp_seq=4 ttl=56 time=24.6 ms (DUP!) 64 bytes from 1.1.1.1: icmp_seq=5 ttl=56 time=24.5 ms 64 bytes from 1.1.1.1: icmp_seq=5 ttl=56 time=24.5 ms (DUP!) To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2008808/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1901891] Re: Issues regarding application credentials
Reviewed: https://review.opendev.org/c/openstack/keystone/+/828595 Committed: https://opendev.org/openstack/keystone/commit/3288af579de8ee312c36fb78ac9309ce8c554827 Submitter: "Zuul (22348)" Branch:master commit 3288af579de8ee312c36fb78ac9309ce8c554827 Author: Dave Wilde (d34dh0r53) Date: Wed Feb 9 11:28:59 2022 -0600 Force algo specific maximum length The bcrypt algorithm that we use for password hashing silently length limits the size of the password that is hashed giving the user a false sense of security [0]. This patch adds a check in the verify_length_and_trunc_password function for the hash in use and updates the max_length accordingly, this will override the configured value and log a warning if the password is truncated. [0]: https://passlib.readthedocs.io/en/stable/lib/passlib.hash.bcrypt.html#security-issues Closes-bug: #1901891 Change-Id: I8d0bb2438b23227b5a66b94af6f8e198084fcd8d ** Changed in: keystone Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1901891 Title: Issues regarding application credentials Status in OpenStack Identity (keystone): Fix Released Status in OpenStack Security Advisory: Won't Fix Bug description: While looking into the application credential API we came across several issues. Since they are all closely related I will file them under this issue: - No secret strength requirements. To configure a password strength requirement for users, one can use `password_regex`. However, this is not possible for application credentials, which makes it possible to create a credentials with the secret 'a': $ openstack application credential create test-secret-strength --secret a +--+--+ | Field| Value| +--+--+ | description | None | | expires_at | None | | id | | | name | test-secret-strength | | project_id | | | roles| member reader| | secret | a| | system | None | | unrestricted | False| | user_id | | +--+--+ To attack this, you'd still need to know the ID, but combined with https://bugs.launchpad.net/keystone/+bug/1901207 the impact of this issue is increased. - No lockout feature. For normal login, the settings `lockout_failure_attempts` and `lockout_duration` are used. These do not affect the application credential API. This increases the attack surface unnecessarily in my opinion. Combined with weak secrets and https://bugs.launchpad.net/keystone/+bug/1901207 the probability of a successful attack is increased. - Only part of secret is verified. It looks like only the first 72 characters of the secret of an application credential are used to verify it. Characters after that are not used in the verification. The default length of a secret seems to be 86 characters. Even though brute forcing 72 characters is still pretty impossible, this doesn't sound like intended behaviour to me. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1901891/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1998274] Re: interface detach does not progress because libvirt does not complete the operation
Mark it at won't fix because it is a duplicate bug. ** Changed in: nova Status: New => Confirmed ** Changed in: nova Status: Confirmed => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1998274 Title: interface detach does not progress because libvirt does not complete the operation Status in OpenStack Compute (nova): Won't Fix Bug description: Description === # This might not be a nova bug but I'm wondering whether anyone in the team has any idea about $topic. Currently some tests in heat CI are consistently failing. Looking at the failures, we found detaching a port from a interface does not progress. example: https://zuul.opendev.org/t/openstack/build/301ed642a2374caf9a4f807952702a6a ~~~ Nov 29 10:14:44.823945 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: DEBUG nova.virt.libvirt.driver [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] Attempting to detach device tape01452d4-15 from instance 5cf9dfbe-e6cd-414b-a868-888aef23f733 from the persistent domain config. {{(pid=81163) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2445}} Nov 29 10:14:44.824612 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: DEBUG nova.virt.libvirt.guest [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] detach device xml: Nov 29 10:14:44.829211 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: DEBUG nova.virt.libvirt.guest [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] looking for interface given config: {{(pid=81163) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}} Nov 29 10:14:44.832316 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: DEBUG nova.virt.libvirt.guest [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] No interface of type: found in domain {{(pid=81163) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:261}} Nov 29 10:14:44.832606 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: INFO nova.virt.libvirt.driver [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] Successfully detached device tape01452d4-15 from instance 5cf9dfbe-e6cd-414b-a868-888aef23f733 from the persistent domain config. Nov 29 10:14:44.832978 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: DEBUG nova.virt.libvirt.driver [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] (1/8): Attempting to detach device tape01452d4-15 with device alias net0 from instance 5cf9dfbe-e6cd-414b-a868-888aef23f733 from the live domain config. {{(pid=81163) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2481}} Nov 29 10:14:44.833403 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: DEBUG nova.virt.libvirt.guest [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] detach device xml: Nov 29 10:14:49.838217 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: DEBUG nova.virt.libvirt.driver [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] Start waiting for the detach event from libvirt for device tape01452d4-15 with device alias net0 for instance 5cf9dfbe-e6cd-414b-a868-888aef23f733 {{(pid=81163) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2557}} Nov 29 10:15:09.840289 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: WARNING nova.virt.libvirt.driver [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] Waiting for libvirt event about the detach of device tape01452d4-15 with device alias net0 from instance 5cf9dfbe-e6cd-414b-a868-888aef23f733 is timed out. Nov 29 10:15:09.841061 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: DEBUG nova.virt.libvirt.guest [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] looking for interface given config: {{(pid=81163) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}} Nov 29 10:15:09.846547 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: DEBUG nova.virt.libvirt.driver [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] Failed to detach device tape01452d4-15 with device alias net0 from instance 5cf9dfbe-e6cd-414b-a868-888aef23f733 from the live domain config. Libvirt did not report any error but the device is still in the config. {{(pid=81163) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2499}} Nov 29 10:15:09.846792 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: DEBUG nova.virt.libvirt.driver [None req-3a1d745d-ca71-4dd3-9f75-d9adac3c5cdb demo demo] (2/8): Attempting to detach device tape01452d4-15 with device alias net0 from instance 5cf9dfbe-e6cd-414b-a868-888aef23f733 from the live domain config. {{(pid=81163) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2481}} Nov 29 10:15:09.847211 ubuntu-jammy-rax-dfw-0032320806 nova-compute[81163]: DEBUG
[Yahoo-eng-team] [Bug 2003455] Re: [ovn] MTU issues due to centralized vlan provider networks
Reviewed: https://review.opendev.org/c/openstack/neutron/+/871252 Committed: https://opendev.org/openstack/neutron/commit/acb809eea422f417d4bfb2d46918839d7d379e4c Submitter: "Zuul (22348)" Branch:master commit acb809eea422f417d4bfb2d46918839d7d379e4c Author: Luis Tomas Bolivar Date: Fri Jan 20 12:16:06 2023 +0100 [OVN] Ensure traffic for provider vlan networks is not tunneled This patch adds an extra checking to ensure the "reside-on-redirect-chassis" is set to true for the logical router port associated to vlan provider network despite having the "ovn_distributed_floating_ip" enabled or not. This is needed as there is an OVN bug [1] making it not work as expected. Note setting this to true has implications as the traffic will be centrallized (but not tunneled) through the node with the gateway port. The expected behavior of this flag, once [1] is fixed is: - reside-on-redirect-chassis flag to False: means traffic goes tunneled to the controller with the gateway port. Means it requires extra MTU reduction to work. - reside-on-redirect-chassis flag to True: means traffic is not tunneled to the controller with the gateway port, but the traffic is centralized through the controller with the gateway port. Thus it does not require extra MTU reduction. - reside-on-redirect-chassis to False, but with ovn-chassis-mac-mappings configured: means the traffic is fully distributed and it is not being tunneled, nor sent, through the controller with the gateway port. This is the preferred option as it does not require MTU reduction and it avoids the extra hop. However it is not working as expected, therefore the fallback to set reside-on-redirect-chassis to True. [1] https://bugzilla.redhat.com/show_bug.cgi?id=2162756 Closes-Bug: #2003455 Change-Id: I662cb30c842e54bb9f7dabac5519283aa7c7f8d0 ** Changed in: neutron Status: In Progress => Fix Released ** Bug watch added: Red Hat Bugzilla #2162756 https://bugzilla.redhat.com/show_bug.cgi?id=2162756 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2003455 Title: [ovn] MTU issues due to centralized vlan provider networks Status in neutron: Fix Released Bug description: After this change was added [1] the traffic gets centralized not only for vlan tenant networks, but also for vlan provider networks. This means that extra reduction on the MTU size needs to be done to account for the geneve encapsulation due to traffic going through the networker node instead of directly from the node [1] https://opendev.org/openstack/networking-ovn/commit/1440207c0d568068a37a306a7f03a81ad58e468f To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2003455/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2008461] Re: Creating RequestSpec from flavor tries to use nonexistent attribute
Thanks for reporting this bug. It is legit, but it needs to be discussed as the RequestSpec.from_primitives method is deprecated and should be removed at some point. So we need to discuss whether it is worth fixing or simply removing the method. ** Changed in: nova Status: New => Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2008461 Title: Creating RequestSpec from flavor tries to use nonexistent attribute Status in OpenStack Compute (nova): Opinion Bug description: Description === Calling RequestSpec.from_primitives, which in turn calls spec._from_flavor ends with "AttributeError: module 'nova.objects' has no attribute 'Flavor'" after the check "isinstance(flavor, objects.Flavor)" Steps to reproduce == * I directly imported RequestSpec via "from nova.objects.request_spec import RequestSpec" for testing purposes * I called RequestSpec.from_primitives * RequestSpec.from_primitives called RequestSpec._from_flavor to fill the missing information * RequestSpec._from_flavor performed a check "isinstance(flavor, objects.Flavor)", which resulted in the exception Expected result === I should have received a RequestSpec object or another error if the data provided was incorrect. Actual result = The exception "AttributeError: module 'nova.objects' has no attribute 'Flavor'" is raised. Traceback: RequestSpec.from_primitives(scheduler_connector_nova_rpc._context, dictionary_from_json, {}) File "/home/eouser/placement-simulation-tool/venv/lib/python3.10/site-packages/nova/objects/request_spec.py", line 331, in from_primitives spec._from_flavor(flavor) File "/home/eouser/placement-simulation-tool/venv/lib/python3.10/site-packages/nova/objects/request_spec.py", line 249, in _from_flavor if isinstance(flavor, objects.Flavor): AttributeError: module 'nova.objects' has no attribute 'Flavor' Environment === I use Nova 21.2.4 from PyPi, Python 3.10 on Ubuntu 20.04. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2008461/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2008223] Re: [sqlalchemy-20] The Engine.execute() method is considered legacy as of the 1.x series
Reviewed: https://review.opendev.org/c/openstack/neutron-lib/+/874866 Committed: https://opendev.org/openstack/neutron-lib/commit/444cec2ba04d12980b116086b727c7ff40db63f4 Submitter: "Zuul (22348)" Branch:master commit 444cec2ba04d12980b116086b727c7ff40db63f4 Author: Rodolfo Alonso Hernandez Date: Sat Feb 18 14:50:58 2023 +0100 [sqlalchemy-20] Use Connection.execute() instead of Engine.execute() According to the SLQAlchemy documentation, "the Engine.execute() method is considered legacy as of the 1.x series of SQLAlchemy and will be removed in 2.0. All statement execution in SQLAlchemy 2.0 is performed by the Connection.execute() method of Connection, or in the ORM by the Session.execute() method of Session." Change-Id: Ic4d7a0740f7136cac2055e57e57f66e76cb150f3 Closes-Bug: #2008223 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2008223 Title: [sqlalchemy-20] The Engine.execute() method is considered legacy as of the 1.x series Status in neutron: Fix Released Bug description: Testing patch: https://review.opendev.org/c/openstack/neutron/+/874778 Logs: https://c06c4109be1832423601-1eb2471c773c922210e88856273ba212.ssl.cf5.rackcdn.com/874778/1/check/neutron- functional-with-uwsgi/4c704c1/job-output.txt Error: 2023-02-22 20:05:21.282046 | controller | engine = engines.create_engine( 2023-02-22 20:05:21.282060 | controller | /home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-functional-gate/lib/python3.10/site-packages/neutron_lib/fixture.py:103: RemovedIn20Warning: The Engine.execute() method is considered legacy as of the 1.x series of SQLAlchemy and will be removed in 2.0. All statement execution in SQLAlchemy 2.0 is performed by the Connection.execute() method of Connection, or in the ORM by the Session.execute() method of Session. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9) 2023-02-22 20:05:21.282074 | controller | self.engine.execute("PRAGMA foreign_keys=ON") To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2008223/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2008775] [NEW] [sqlalchemy-20][vnpaas] Parent instance is not bound to a Session
*** This bug is a duplicate of bug 2008001 *** https://bugs.launchpad.net/bugs/2008001 Public bug reported: The method ``BaseIPsecVPNDriver.create_vpnservice`` is using a SQL object outside a DB context. The "vpnservice" object retrieved in [1] is used in the next line but outside the read context used to retrieve it. Logs: https://paste.opendev.org/show/bC9xTO39Www41MAKrnOy/ [1]https://github.com/openstack/neutron- vpnaas/blob/d34ff613d720c4990f41c417060e32f92f19f25a/neutron_vpnaas/services/vpn/service_drivers/base_ipsec.py#L174 ** Affects: neutron Importance: Undecided Status: New ** This bug has been marked a duplicate of bug 2008001 [neutron-vpnaas] SQL error during the "vpnservice" creation -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2008775 Title: [sqlalchemy-20][vnpaas] Parent instance is not bound to a Session Status in neutron: New Bug description: The method ``BaseIPsecVPNDriver.create_vpnservice`` is using a SQL object outside a DB context. The "vpnservice" object retrieved in [1] is used in the next line but outside the read context used to retrieve it. Logs: https://paste.opendev.org/show/bC9xTO39Www41MAKrnOy/ [1]https://github.com/openstack/neutron- vpnaas/blob/d34ff613d720c4990f41c417060e32f92f19f25a/neutron_vpnaas/services/vpn/service_drivers/base_ipsec.py#L174 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2008775/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2008772] [NEW] Ubuntu 18.04 wrong parsing of instance-data.json generating config
Public bug reported: Hello, We have an cloud service based on Openstack, we have provide IPv4 (dhcp) and IPv6 (static), on ubuntu 18.04 we detected that when is parsed instance-data.json it take info only for IPv4 and not get static records for IPv6. And related to this it not generate config for IPv6 we tested with netplan and ENI (ifupdown), all data in instance-data.json are present in the file. 2023-02-28 09:21:42,649 - util.py[DEBUG]: Reading from /sys/class/net/ens3/type (quiet=False) 2023-02-28 09:21:42,650 - util.py[DEBUG]: Read 2 bytes from /sys/class/net/ens3/type 2023-02-28 09:21:42,650 - networking.py[DEBUG]: net: all expected physical devices present 2023-02-28 09:21:42,650 - stages.py[DEBUG]: applying net config names for {'ethernets': {'ens3': {'dhcp4': True, 'set-name': 'ens3', 'match': {'macaddress': 'fa:16:3e:00:3c:33'}}}, 'version': 2} 2023-02-28 09:21:42,650 - util.py[DEBUG]: Reading from /sys/class/net/ens3/device/device (quiet=False) 2023-02-28 09:21:42,650 - util.py[DEBUG]: Read 7 bytes from /sys/class/net/ens3/device/device On ubuntu 20.04 this problem not persist. 2023-02-27 14:36:40,755 - util.py[DEBUG]: Reading from /sys/class/net/lo/type (quiet=False) 2023-02-27 14:36:40,755 - util.py[DEBUG]: Read 4 bytes from /sys/class/net/lo/type 2023-02-27 14:36:40,755 - networking.py[DEBUG]: net: all expected physical devices present 2023-02-27 14:36:40,755 - stages.py[DEBUG]: applying net config names for {'version': 1, 'config': [{'type': 'physical', 'mtu': 1500, 'accept-ra': False, 'subnets': [{'type': 'dhcp4'}, {'type': 'static6', 'netmask': ':::::', 'routes': [{'network': '::', 'netmask': '::', 'gateway': '2a10:c941:12:1::1'}], 'address': '2a10:c941:12:1::10f', 'ipv6': True}], 'mac_address': 'fa:16:3e:f3:d9:ad', 'name': 'ens3'}, {'type': 'nameserver', 'address': '8.8.8.8'}, {'type': 'nameserver', 'address': '1.1.1.1'}]} 2023-02-27 14:36:40,756 - util.py[DEBUG]: Reading from /sys/class/net/ens3/device/device (quiet=False) 2023-02-27 14:36:40,756 - util.py[DEBUG]: Read 7 bytes from /sys/class/net/ens3/device/device cloud-init version: 22.4.2-0ubuntu0~18.04.1 we tried and older version but it work same ** Affects: cloud-init Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/2008772 Title: Ubuntu 18.04 wrong parsing of instance-data.json generating config Status in cloud-init: New Bug description: Hello, We have an cloud service based on Openstack, we have provide IPv4 (dhcp) and IPv6 (static), on ubuntu 18.04 we detected that when is parsed instance-data.json it take info only for IPv4 and not get static records for IPv6. And related to this it not generate config for IPv6 we tested with netplan and ENI (ifupdown), all data in instance-data.json are present in the file. 2023-02-28 09:21:42,649 - util.py[DEBUG]: Reading from /sys/class/net/ens3/type (quiet=False) 2023-02-28 09:21:42,650 - util.py[DEBUG]: Read 2 bytes from /sys/class/net/ens3/type 2023-02-28 09:21:42,650 - networking.py[DEBUG]: net: all expected physical devices present 2023-02-28 09:21:42,650 - stages.py[DEBUG]: applying net config names for {'ethernets': {'ens3': {'dhcp4': True, 'set-name': 'ens3', 'match': {'macaddress': 'fa:16:3e:00:3c:33'}}}, 'version': 2} 2023-02-28 09:21:42,650 - util.py[DEBUG]: Reading from /sys/class/net/ens3/device/device (quiet=False) 2023-02-28 09:21:42,650 - util.py[DEBUG]: Read 7 bytes from /sys/class/net/ens3/device/device On ubuntu 20.04 this problem not persist. 2023-02-27 14:36:40,755 - util.py[DEBUG]: Reading from /sys/class/net/lo/type (quiet=False) 2023-02-27 14:36:40,755 - util.py[DEBUG]: Read 4 bytes from /sys/class/net/lo/type 2023-02-27 14:36:40,755 - networking.py[DEBUG]: net: all expected physical devices present 2023-02-27 14:36:40,755 - stages.py[DEBUG]: applying net config names for {'version': 1, 'config': [{'type': 'physical', 'mtu': 1500, 'accept-ra': False, 'subnets': [{'type': 'dhcp4'}, {'type': 'static6', 'netmask': ':::::', 'routes': [{'network': '::', 'netmask': '::', 'gateway': '2a10:c941:12:1::1'}], 'address': '2a10:c941:12:1::10f', 'ipv6': True}], 'mac_address': 'fa:16:3e:f3:d9:ad', 'name': 'ens3'}, {'type': 'nameserver', 'address': '8.8.8.8'}, {'type': 'nameserver', 'address': '1.1.1.1'}]} 2023-02-27 14:36:40,756 - util.py[DEBUG]: Reading from /sys/class/net/ens3/device/device (quiet=False) 2023-02-27 14:36:40,756 - util.py[DEBUG]: Read 7 bytes from /sys/class/net/ens3/device/device cloud-init version: 22.4.2-0ubuntu0~18.04.1 we tried and older version but it work same To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/2008772/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to :
[Yahoo-eng-team] [Bug 2008767] [NEW] [sqlalchemy-20][vnpaas] SQL execution without transaction in progress
Public bug reported: The following methods are not called from inside a DB context: * ``VPNPluginDb.get_ikepolicy`` * ``VPNPluginDb.get_ikepolicies`` * ``VPNPluginDb.get_ipsecpolicy`` * ``VPNPluginDb.get_ipsecpolicies`` Logs: https://paste.opendev.org/show/bNT4RdGsrx0DlI6ja2tS/ ** Affects: neutron Importance: High Status: In Progress ** Changed in: neutron Importance: Undecided => High ** Description changed: - ``VPNPluginDb.get_ikepolicy`` and ``VPNPluginDb.get_ikepolicies`` are - not called from inside a DB context. + The following methods are not called from inside a DB context: + * ``VPNPluginDb.get_ikepolicy`` + * ``VPNPluginDb.get_ikepolicies`` + * ``VPNPluginDb.get_ipsecpolicy`` + * ``VPNPluginDb.get_ipsecpolicies`` Logs: https://paste.opendev.org/show/bNT4RdGsrx0DlI6ja2tS/ -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2008767 Title: [sqlalchemy-20][vnpaas] SQL execution without transaction in progress Status in neutron: In Progress Bug description: The following methods are not called from inside a DB context: * ``VPNPluginDb.get_ikepolicy`` * ``VPNPluginDb.get_ikepolicies`` * ``VPNPluginDb.get_ipsecpolicy`` * ``VPNPluginDb.get_ipsecpolicies`` Logs: https://paste.opendev.org/show/bNT4RdGsrx0DlI6ja2tS/ To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2008767/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2008257] Re: OVN agent heartbeat timestamp format changed unexpectedly
Reviewed: https://review.opendev.org/c/openstack/neutron/+/874926 Committed: https://opendev.org/openstack/neutron/commit/827fbd01c306ce292e46b0d881bc3cc804202285 Submitter: "Zuul (22348)" Branch:master commit 827fbd01c306ce292e46b0d881bc3cc804202285 Author: Pierre Riteau Date: Fri Feb 24 05:34:07 2023 +0100 Normalise format of OVN agent heartbeat timestamp A recent change [1] to show the real heartbeat timestamp from OVN agents had a side effect of changing the timestamp format, which now includes a timezone: +---+--+ | Field | Value| +---+--+ | last_heartbeat_at | 2023-02-23 14:12:07.471000+00:00 | +---+--+ This unexpected format change causes some clients to fail to parse the response to GET /v2.0/agents. Normalise the format of the timestamp to remove timezone information. Also remove the microsecond part, which was not done for OVN, but is absent from other network agents. [1] https://review.opendev.org/c/openstack/neutron/+/844179 Closes-Bug: #2008257 Change-Id: I75a37fb9b49a421e4524da6b56ef8362ceb6107b ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2008257 Title: OVN agent heartbeat timestamp format changed unexpectedly Status in neutron: Fix Released Bug description: Following an upgrade from Wallaby to Yoga of a Neutron OVN deployment, I discovered that openstack-exporter would fail to return metrics for Neutron agent states with the following error: time="2023-02-23T10:49:33Z" level=error msg="Failed to collect metric for exporter: neutron, error: failed to collect metric: agent_state, error: parsing time \"2023-02-23 10:48:55.729000+00:00\": extra text: \"+00:00\"" source="exporter.go:123" I tracked it down to the following change which has also been backported to stable/branches (so a recent Wallaby would also be affected): https://review.opendev.org/c/openstack/neutron/+/844179 With this change, heartbeat timestamp includes timezone when previously it didn't, causing clients such as the gophercloud library (which openstack-exporter uses) to fail parsing of the response. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2008257/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2008764] [NEW] Data source: none, and not able to execute user-script in AWS EC2
Public bug reported: Details can be found here : https://stackoverflow.com/questions/75507635/not-able-to-run-a-user- script-on-some-ec2-instances-started-from-an-ami-of-ubunt I am launching multiple ( more than 100 EC2 instances ) from a previously set AMI. This AMI is made from an Ubuntu 20.04 AMI. Everything that is required for the work is already satisfied in AMI. Now I just want execute an user-script while launching EC2 using AMI. But the problem is coming that for 2-3% of the Ec2 instances are not able to execute the userscript , othere 97-98% of launched instances are running perfectly fine. These instances are being launched using spot requests so they are exactly similar in configurations. I am able to ssh into the faulty instances too . Last few lines of /var/log/cloud-init- output.log looks something like this: Cloud-init v. 22.2-0ubuntu1~22.04.3 running 'modules:config' at Mon, 20 Feb 2023 08:21:36 +. Up 41.66 seconds. Cloud-init v. 22.2-0ubuntu1~22.04.3 running 'modules:final' at Mon, 20 Feb 2023 08:21:39 +. Up 45.11 seconds. Cloud-init v. 22.2-0ubuntu1~22.04.3 finished at Mon, 20 Feb 2023 08:21:39 +. Datasource DataSourceNone. Up 45.51 seconds 2023-02-20 08:21:39,953 - cc_final_message.py[WARNING]: Used fallback datasource /var/lib/cloud/instance normally points to the a folder in with the name of instance id but in this faulty case , it is pointing at this -> /var/lib/cloud/instances/iid-datasource-none For Faulty instance: ubuntu@ip-172-31-21-43:/var/lib/cloud/instances/iid-datasource-none$ cat datasource DataSourceNone: DataSourceNone For healthy instance: ubuntu@ip-172-31-20-76:/var/lib/cloud/instances/i-0c964c834bf91ce81$ cat datasource DataSourceEc2Local: DataSourceEc2Local Now I am not able to figure out that why the userscript is being executed in most of the instances but these few 2-3%. ** Affects: cloud-init Importance: Undecided Status: New ** Tags: aws datasource-none ec2 user-script ** Attachment added: "cloud-init tarball" https://bugs.launchpad.net/bugs/2008764/+attachment/5650441/+files/cloud-init.tar.gz -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/2008764 Title: Data source: none, and not able to execute user-script in AWS EC2 Status in cloud-init: New Bug description: Details can be found here : https://stackoverflow.com/questions/75507635/not-able-to-run-a-user- script-on-some-ec2-instances-started-from-an-ami-of-ubunt I am launching multiple ( more than 100 EC2 instances ) from a previously set AMI. This AMI is made from an Ubuntu 20.04 AMI. Everything that is required for the work is already satisfied in AMI. Now I just want execute an user-script while launching EC2 using AMI. But the problem is coming that for 2-3% of the Ec2 instances are not able to execute the userscript , othere 97-98% of launched instances are running perfectly fine. These instances are being launched using spot requests so they are exactly similar in configurations. I am able to ssh into the faulty instances too . Last few lines of /var/log/cloud-init-output.log looks something like this: Cloud-init v. 22.2-0ubuntu1~22.04.3 running 'modules:config' at Mon, 20 Feb 2023 08:21:36 +. Up 41.66 seconds. Cloud-init v. 22.2-0ubuntu1~22.04.3 running 'modules:final' at Mon, 20 Feb 2023 08:21:39 +. Up 45.11 seconds. Cloud-init v. 22.2-0ubuntu1~22.04.3 finished at Mon, 20 Feb 2023 08:21:39 +. Datasource DataSourceNone. Up 45.51 seconds 2023-02-20 08:21:39,953 - cc_final_message.py[WARNING]: Used fallback datasource /var/lib/cloud/instance normally points to the a folder in with the name of instance id but in this faulty case , it is pointing at this -> /var/lib/cloud/instances/iid-datasource-none For Faulty instance: ubuntu@ip-172-31-21-43:/var/lib/cloud/instances/iid-datasource-none$ cat datasource DataSourceNone: DataSourceNone For healthy instance: ubuntu@ip-172-31-20-76:/var/lib/cloud/instances/i-0c964c834bf91ce81$ cat datasource DataSourceEc2Local: DataSourceEc2Local Now I am not able to figure out that why the userscript is being executed in most of the instances but these few 2-3%. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/2008764/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2008762] [NEW] size_iops_sec does behave different than mentioned in docs
Public bug reported: In documentations mentioned that size_iops_sec is "burst bucket size" https://docs.openstack.org/cinder/latest/admin/basic-volume-qos.html but in libvirt documentations is: "The optional size_iops_sec element is the size of I/O operations per second" https://libvirt.org/formatdomain.html and in qemu documentations "iops-size" described as "This setting specifies the size (in bytes) of an I/O request for accounting purposes. Larger requests will be counted proportionally to this size." https://github.com/qemu/qemu/blob/master/docs/throttle.txt so I think it is not related to burst bucket size. and of course we do not have support for *_max_length parameters for burst duration in nova. ** Affects: nova Importance: Undecided Status: New ** Summary changed: - size_iops_sec does behaves different that mentioned in docs + size_iops_sec does behave different than mentioned in docs -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2008762 Title: size_iops_sec does behave different than mentioned in docs Status in OpenStack Compute (nova): New Bug description: In documentations mentioned that size_iops_sec is "burst bucket size" https://docs.openstack.org/cinder/latest/admin/basic-volume-qos.html but in libvirt documentations is: "The optional size_iops_sec element is the size of I/O operations per second" https://libvirt.org/formatdomain.html and in qemu documentations "iops-size" described as "This setting specifies the size (in bytes) of an I/O request for accounting purposes. Larger requests will be counted proportionally to this size." https://github.com/qemu/qemu/blob/master/docs/throttle.txt so I think it is not related to burst bucket size. and of course we do not have support for *_max_length parameters for burst duration in nova. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2008762/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp