[Yahoo-eng-team] [Bug 2095167] [NEW] BW limit not applied to external/provider network
Public bug reported: The whitebox-neutron-temptest-plugin test test_dscp_bwlimit_external_network has been failing consistently since Jan 8th: https://zuul.opendev.org/t/openstack/builds?job_name=whitebox-neutron-tempest-plugin-ovn-single-thread&project=x%2Fwhitebox-neutron-tempest-plugin&pipeline=periodic&skip=0 The BW limit that the test applies to the external/provider network is not actually applied. test passes: https://03bdeb201ad5b10b1d29-c8e8f0465e65b3e97bf6a5ab87ec6f98.ssl.cf5.rackcdn.com/periodic/opendev.org/x/whitebox-neutron-tempest-plugin/master/whitebox-neutron-tempest-plugin-ovn-single-thread/dbf67fe/controller/logs/screen-q-svc.txt relevant request: req-739334e4-46ac-4806-b8b1-8b362587ebb1 when the BW limit is added to the policy (which was added to the external network before), it generates a transaction with 16 commands to the OVN NB DB, including two UpdateLSwitchPortQosOptionsCommand's test fails: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6b9/939519/1/check/whitebox-neutron-tempest-plugin-ovn-single-thread/6b9ce99/controller/logs/screen-q-svc.txt relevant request: req-75cfa10b-f983-4756-982b-239b7c4caab6 when the BW limit is added to the policy (which was added to the external network before), it generates a transaction with 13 commands to the OVN NB DB. No UpdateLSwitchPortQosOptionsCommand's at all It seems to me the following patch, which was merged on Jan 7th, could have caused this regression: https://review.opendev.org/c/openstack/neutron/+/934418 ** Affects: neutron Importance: Undecided Assignee: Rodolfo Alonso (rodolfo-alonso-hernandez) Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2095167 Title: BW limit not applied to external/provider network Status in neutron: New Bug description: The whitebox-neutron-temptest-plugin test test_dscp_bwlimit_external_network has been failing consistently since Jan 8th: https://zuul.opendev.org/t/openstack/builds?job_name=whitebox-neutron-tempest-plugin-ovn-single-thread&project=x%2Fwhitebox-neutron-tempest-plugin&pipeline=periodic&skip=0 The BW limit that the test applies to the external/provider network is not actually applied. test passes: https://03bdeb201ad5b10b1d29-c8e8f0465e65b3e97bf6a5ab87ec6f98.ssl.cf5.rackcdn.com/periodic/opendev.org/x/whitebox-neutron-tempest-plugin/master/whitebox-neutron-tempest-plugin-ovn-single-thread/dbf67fe/controller/logs/screen-q-svc.txt relevant request: req-739334e4-46ac-4806-b8b1-8b362587ebb1 when the BW limit is added to the policy (which was added to the external network before), it generates a transaction with 16 commands to the OVN NB DB, including two UpdateLSwitchPortQosOptionsCommand's test fails: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6b9/939519/1/check/whitebox-neutron-tempest-plugin-ovn-single-thread/6b9ce99/controller/logs/screen-q-svc.txt relevant request: req-75cfa10b-f983-4756-982b-239b7c4caab6 when the BW limit is added to the policy (which was added to the external network before), it generates a transaction with 13 commands to the OVN NB DB. No UpdateLSwitchPortQosOptionsCommand's at all It seems to me the following patch, which was merged on Jan 7th, could have caused this regression: https://review.opendev.org/c/openstack/neutron/+/934418 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2095167/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2045270] Re: [xena/yoga/zed] devstack-tobiko-neutron job broken
Patch merged: https://review.opendev.org/c/x/devstack-plugin-tobiko/+/902563 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2045270 Title: [xena/yoga/zed] devstack-tobiko-neutron job broken Status in neutron: Fix Released Bug description: These jobs are broken as these jobs running on focal and running tests which shouldn't on focal as per https://review.opendev.org/c/openstack/neutron/+/871982 (included in antelope+) Example failure https://d5a9a78dc7c742990ec8-242e23f01f4a3c50d38acf4a24e0c600.ssl.cf1.rackcdn.com/periodic/opendev.org/openstack/neutron/stable/yoga/devstack-tobiko-neutron/21d79ad/tobiko_results_02_create_neutron_resources_neutron.html Builds:- https://zuul.openstack.org/builds?job_name=devstack-tobiko- neutron&project=openstack%2Fneutron&branch=stable%2Fxena&branch=stable%2Fyoga&branch=stable%2Fzed&skip=0 That patch is not available before zed so for older branches need to skip those tests in some other way To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2045270/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 2027605] [NEW] [OVN][Trunk] Sometimes subport doesn't reach status ACTIVE after live-migration
Public bug reported:
this is similar to https://bugs.launchpad.net/neutron/+bug/2024160 and
actually it is reproduced with the same tempest test,
test_live_migration_with_trunk, but there are a few important
differencies.
In LP2024160, the subport remained in status DOWN when the VM was
created (before any migration!). And it happened always (the test always
failed).
With this new bug, both subport and parent port successfully reach
status=ACTIVE when the VM is created. Then, the VM is live-migrated to another
compute. The test checks the VM status successfully changes to ACTIVE.
Finally, the test fails waiting for the subport status to change to ACTIVE: it
never happens, it remains in status DOWN.
This failure doesn't happen always, so it seems it is due to a race condition.
Tempest logs when it fails (this job was retriggered/rechecked and the test
passed):
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_8ac/887220/4/check/nova-live-migration/8ace3a8/testr_results.html
How often does this fail?
https://zuul.opendev.org/t/openstack/builds?job_name=nova-live-migration&branch=master&skip=0
24 jobs run on 11th and 12th July
7 of them failed - all these failures are due to this bug
i.e. ~30%
neutron server logs:
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_8ac/887220/4/check/nova-live-migration/8ace3a8/controller/logs/screen-q-svc.txt
subport: e40925a3-4e71-4fa4-80ea-cc3a464101d6
parent port: 51e121cc-4e64-411e-b16a-e605cf967332
live-migration src host: np0034645395
live-migration dst host: np0034645398
It seems that due to a race condition, the following operations are processed
in the wrong order.
The first one sets the parent port status to ACTIVE and changes revision number
from 9 to 10:
Jul 11 15:45:48.620648 np0034645395 neutron-server[78430]: INFO
neutron.tests.unit.plugins.ml2.drivers.mechanism_logger [None
req-de6d27ae-5f07-41d8-90cb-e85d1228c53f None None] update_port_postcommit
called with port settings {'id': '51e121cc-4e64-411e-b16a-e605cf967332',
'name': 'tempest-parent-2113033899', 'network_id':
'6d3ce4b9-e66d-4137-a356-75e402377eb7', 'tenant_id':
'4e4ea23ed96240ea8108fd937f2bd12b', 'mac_address': 'fa:16:3e:e6:51:04',
'admin_state_up': True, 'status': 'ACTIVE', 'device_id':
'cb3569cf-e7e0-4b01-8d56-aaf9fbcd41a8', 'device_owner': 'compute:nova',
'standard_attr_id': 173, 'fixed_ips': [{'subnet_id':
'5ed321df-19d5-4c9b-8cf2-f27f0993fe12', 'ip_address': '10.1.0.5'}],
'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups':
['ad92c344-fc82-4f88-bdc7-1d7845432a2f'], 'description': '',
'binding:vnic_type': 'normal', 'binding:profile': {'migrating_to':
'np0034645398'}, 'binding:host_id': 'np0034645395', 'binding:vif_type': 'ovs',
'binding:vif_details
': {'port_filter': True, 'connectivity': 'l2', 'bound_drivers': {'0': 'ovn'}},
'port_security_enabled': True, 'trunk_details': {'trunk_id':
'833832a8-c2bf-4ca9-a25a-9462f63b8926', 'sub_ports': [{'segmentation_id': 42,
'segmentation_type': 'vlan', 'port_id': 'e40925a3-4e71-4fa4-80ea-cc3a464101d6',
'mac_address': 'fa:16:3e:e5:55:14'}]}, 'tags': [], 'created_at':
'2023-07-11T15:45:19Z', 'updated_at': '2023-07-11T15:45:48Z',
'revision_number': 10, 'project_id': '4e4ea23ed96240ea8108fd937f2bd12b'}
(original settings {'id': '51e121cc-4e64-411e-b16a-e605cf967332', 'name':
'tempest-parent-2113033899', 'network_id':
'6d3ce4b9-e66d-4137-a356-75e402377eb7', 'tenant_id':
'4e4ea23ed96240ea8108fd937f2bd12b', 'mac_address': 'fa:16:3e:e6:51:04',
'admin_state_up': True, 'status': 'DOWN', 'device_id':
'cb3569cf-e7e0-4b01-8d56-aaf9fbcd41a8', 'device_owner': 'compute:nova',
'standard_attr_id': 173, 'fixed_ips': [{'subnet_id':
'5ed321df-19d5-4c9b-8cf2-f27f0993fe12', 'ip_address': '10.1.0.5'}], 'allowed_
address_pairs': [], 'extra_dhcp_opts': [], 'security_groups':
['ad92c344-fc82-4f88-bdc7-1d7845432a2f'], 'description': '',
'binding:vnic_type': 'normal', 'binding:profile': {'migrating_to':
'np0034645398'}, 'binding:host_id': 'np0034645395', 'binding:vif_type': 'ovs',
'binding:vif_details': {'port_filter': True, 'connectivity': 'l2',
'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': True, 'trunk_details':
{'trunk_id': '833832a8-c2bf-4ca9-a25a-9462f63b8926', 'sub_ports':
[{'segmentation_id': 42, 'segmentation_type': 'vlan', 'port_id':
'e40925a3-4e71-4fa4-80ea-cc3a464101d6', 'mac_address': 'fa:16:3e:e5:55:14'}]},
'tags': [], 'created_at': '2023-07-11T15:45:19Z', 'updated_at':
'2023-07-11T15:45:48Z', 'revision_number': 9, 'project_id':
'4e4ea23ed96240ea8108fd937f2bd12b'}) host np0034645395 (original host
np0034645395) vif type ovs (original vif type ovs) vif details {'port_filter':
True, 'connectivity': 'l2'} (original vif details {'port_filter': True,
'connectivity': 'l2'}) bin
ding levels [{'bound_driver': 'ovn', 'bound_segment': {'id':
'21392775-0d30-44c3-9f06-aa8348d69be8', 'network_type': 'geneve',
[Yahoo-eng-team] [Bug 2024160] [NEW] [trunk ports] subport doesn't reach status ACTIVE
Public bug reported:
Test test_live_migration_with_trunk has been failing for the last two days.
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3f3/831018/30/check/nova-live-migration/3f3b065/testr_results.html
It's a test about live-migration, but it is important to notice that it
fails before any live migration happens.
The test creates a VM with a port and a subport.
The test waits until the VM status is ACTIVE -> this passes
The test waits until the subport status is ACTIVE -> this started failing two
days ago because the port status is DOWN
There was only one neutron patch merged that day[1], but I checked the
test failed during some jobs even before that patch was merged.
I compared some logs.
Neutron logs when the test passes: [2]
Neutron logs when the test fails: [3]
When it fails, I see this during the creation of the subport (and I don't see
this event when it passes):
Jun 14 18:13:43.052982 np0034303809 neutron-server[77531]: DEBUG
ovsdbapp.backend.ovs_idl.event [None req-929dd199-4247-46f5-9466-622c7d538547
None None] Matched DELETE: PortBindingUpdateVirtualPortsEvent(events=('update',
'delete'), table='Port_Binding', conditions=None, old_conditions=None),
priority=20 to row=Port_Binding(parent_port=[], mac=['fa:16:3e:93:9d:5a
19.80.0.42'], chassis=[], ha_chassis_group=[], options={'mcast_flood_reports':
'true', 'requested-chassis': ''}, type=, tag=[], requested_chassis=[],
tunnel_key=2, up=[False], logical_port=f8c707ec-ecd8-4f1e-99ba-6f8303b598b2,
gateway_chassis=[], encap=[], external_ids={'name':
'tempest-subport-2029248863', 'neutron:cidrs': '19.80.0.42/24',
'neutron:device_id': '', 'neutron:device_owner': '', 'neutron:network_name':
'neutron-5fd9faa7-ec1c-4f42-ab87-6ce19edda245', 'neutron:port_capabilities':
'', 'neutron:port_name': 'tempest-subport-2029248863', 'neutron:project_id':
'6f92a9f8e16144148026725b25711d3a', 'neutron:revision_num
ber': '1', 'neutron:security_group_ids':
'5eab41ef-c5c1-425c-a931-f5b6b4b330ad', 'neutron:subnet_pool_addr_scope4': '',
'neutron:subnet_pool_addr_scope6': '', 'neutron:vnic_type': 'normal'},
virtual_parent=[], nat_addresses=[],
datapath=3c472399-d6ee-4b7c-aa97-6777f2bc2772) old= {{(pid=77531) matches
/usr/local/lib/python3.10/dist-packages/ovsdbapp/backend/ovs_idl/event.py:43}}
...
Jun 14 18:13:49.597911 np0034303809 neutron-server[77531]: DEBUG
neutron.plugins.ml2.plugin [None req-3588521e-7878-408d-b1f8-15db562c69f8 None
None] Port f8c707ec-ecd8-4f1e-99ba-6f8303b598b2 cannot update to ACTIVE because
it is not bound. {{(pid=77531) _port_provisioned
/opt/stack/neutron/neutron/plugins/ml2/plugin.py:361}}
It seems the ovn version has changed between these jobs:
Passes [4]:
2023-06-14 10:01:46.358875 | controller | Preparing to unpack
.../ovn-common_22.03.0-0ubuntu1_amd64.deb ...
Fails [5]:
2023-06-14 17:55:07.077377 | controller | Preparing to unpack
.../ovn-common_22.03.2-0ubuntu0.22.04.1_amd64.deb ...
[1] https://review.opendev.org/c/openstack/neutron/+/883687
[2]
https://96b562ba0d2478fe5bc1-d58fbc463536b3122b4367e996d5e5b0.ssl.cf1.rackcdn.com/831018/30/check/nova-live-migration/312c2ab/controller/logs/screen-q-svc.txt
[3]
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3f3/831018/30/check/nova-live-migration/3f3b065/controller/logs/screen-q-svc.txt
[4]
https://96b562ba0d2478fe5bc1-d58fbc463536b3122b4367e996d5e5b0.ssl.cf1.rackcdn.com/831018/30/check/nova-live-migration/312c2ab/job-output.txt
[5]
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3f3/831018/30/check/nova-live-migration/3f3b065/job-output.txt
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2024160
Title:
[trunk ports] subport doesn't reach status ACTIVE
Status in neutron:
New
Bug description:
Test test_live_migration_with_trunk has been failing for the last two days.
https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3f3/831018/30/check/nova-live-migration/3f3b065/testr_results.html
It's a test about live-migration, but it is important to notice that
it fails before any live migration happens.
The test creates a VM with a port and a subport.
The test waits until the VM status is ACTIVE -> this passes
The test waits until the subport status is ACTIVE -> this started failing two
days ago because the port status is DOWN
There was only one neutron patch merged that day[1], but I checked the
test failed during some jobs even before that patch was merged.
I compared some logs.
Neutron logs when the test passes: [2]
Neutron logs when the test fails: [3]
When it fails, I see this during the creation of the subport (and I don't see
this event when it passes):
[Yahoo-eng-team] [Bug 1945299] [NEW] ovn migration create-resources is not stable
Public bug reported: Reported at: https://bugzilla.redhat.com/show_bug.cgi?id=2008425 As part of the ovs2ovn migration process, some workload is created before the migration begins. The intention is to verify that those workloads are healthy after the migration. The script that creates those workloads checks they are pingable and ssh'able. Sometimes the ssh fails because the created VMs did not get the public keys during the cloud-init phase: + ssh -i /home/stack/ovn_migration/pre_migration_resources/ovn_migration_ssh_key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null cirros@10.0.0.243 date Warning: Permanently added '10.0.0.243' (ECDSA) to the list of known hosts. Permission denied, please try again. Permission denied, please try again. cirros@10.0.0.243: Permission denied (publickey,password). + rc=255 + echo 'Done with the resource creation : exiting with 255' Done with the resource creation : exiting with 255 + exit 255 It seems the issue is related to a problem in cirros 0.4.0, which was resolved in a later release of cirros: https://github.com/cirros-dev/cirros/pull/11/commits/e40bcd2964aa496a9d03e1aaf95cf7a86938f129 So, using cirros 0.5.2 could stabilize this script. This script can be found in two different formats: https://opendev.org/openstack/neutron/src/branch/master/tools/ovn_migration/tripleo_environment/playbooks/roles/resources/create/templates/create-resources.sh.j2 https://opendev.org/openstack/neutron/src/branch/master/tools/ovn_migration/infrared/tripleo-ovn-migration/roles/create-resources/templates/create-resources.sh.j2 ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1945299 Title: ovn migration create-resources is not stable Status in neutron: New Bug description: Reported at: https://bugzilla.redhat.com/show_bug.cgi?id=2008425 As part of the ovs2ovn migration process, some workload is created before the migration begins. The intention is to verify that those workloads are healthy after the migration. The script that creates those workloads checks they are pingable and ssh'able. Sometimes the ssh fails because the created VMs did not get the public keys during the cloud-init phase: + ssh -i /home/stack/ovn_migration/pre_migration_resources/ovn_migration_ssh_key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null cirros@10.0.0.243 date Warning: Permanently added '10.0.0.243' (ECDSA) to the list of known hosts. Permission denied, please try again. Permission denied, please try again. cirros@10.0.0.243: Permission denied (publickey,password). + rc=255 + echo 'Done with the resource creation : exiting with 255' Done with the resource creation : exiting with 255 + exit 255 It seems the issue is related to a problem in cirros 0.4.0, which was resolved in a later release of cirros: https://github.com/cirros-dev/cirros/pull/11/commits/e40bcd2964aa496a9d03e1aaf95cf7a86938f129 So, using cirros 0.5.2 could stabilize this script. This script can be found in two different formats: https://opendev.org/openstack/neutron/src/branch/master/tools/ovn_migration/tripleo_environment/playbooks/roles/resources/create/templates/create-resources.sh.j2 https://opendev.org/openstack/neutron/src/branch/master/tools/ovn_migration/infrared/tripleo-ovn-migration/roles/create-resources/templates/create-resources.sh.j2 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1945299/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1938575] [NEW] Misalignment with extra-dhcp-options between neutronclient & openstackclient
Public bug reported:
The SetPort class from the openstack client does not support
--extra-dhcp-option [1]
(overcloud) $ openstack port set --extra-dhcp-option
name=mtu,value=1700,ip-version=4 port-test-202
usage: openstack port set [-h] [--description ]
[--device ] [--mac-address ]
[--device-owner ]
[--vnic-type ] [--host ]
[--dns-domain dns-domain] [--dns-name ]
[--enable | --disable] [--name ]
[--fixed-ip subnet=,ip-address=]
[--no-fixed-ip]
[--binding-profile ]
[--no-binding-profile] [--qos-policy ]
[--security-group ]
[--no-security-group]
[--enable-port-security | --disable-port-security]
[--allowed-address
ip-address=[,mac-address=]]
[--no-allowed-address]
[--data-plane-status ] [--tag ]
[--no-tag]
openstack port set: error: unrecognized arguments: --extra-dhcp-option
port-test-202
The UpdatePort class from the neutron client supports --extra-dhcp-opt [2]
This is aligned with the neutron API [3]
(overcloud) $ neutron port-update port-test-202 --extra-dhcp-opt
opt_name=mtu,opt_value=1750,ip_version=4
neutron CLI is deprecated and will be removed in the future. Use openstack CLI
instead.
Updated port: port-test-202
(overcloud) $ neutron port-show port-test-202 | grep mtu
neutron CLI is deprecated and will be removed in the future. Use openstack CLI
instead.
| | {"opt_name": "mtu", "opt_value": "1750",
"ip_version": 4}
|
[1]
https://opendev.org/openstack/python-openstackclient/src/commit/ed87f7949ef1ef580ed71b9820e16823c0466472/openstackclient/network/v2/port.py#L703
[2]
https://github.com/openstack/python-neutronclient/blob/2f047b15957308e84dcb72baee3415b8bf5a470a/neutronclient/neutron/v2_0/port.py#L305
[3]
https://docs.openstack.org/api-ref/network/v2/?expanded=update-port-detail#update-port
** Affects: neutron
Importance: Undecided
Status: New
** Description changed:
The SetPort class from the openstack client does not support
--extra-dhcp-option [1]
- (overcloud) [stack@undercloud-0 tempest-dir]$ openstack port set
--extra-dhcp-option name=mtu,value=1700,ip-version=4 port-test-202
+ (overcloud) $ openstack port set --extra-dhcp-option
name=mtu,value=1700,ip-version=4 port-test-202
usage: openstack port set [-h] [--description ]
- [--device ] [--mac-address ]
- [--device-owner ]
- [--vnic-type ] [--host ]
- [--dns-domain dns-domain] [--dns-name ]
- [--enable | --disable] [--name ]
- [--fixed-ip subnet=,ip-address=]
- [--no-fixed-ip]
- [--binding-profile ]
- [--no-binding-profile] [--qos-policy ]
- [--security-group ]
- [--no-security-group]
- [--enable-port-security | --disable-port-security]
- [--allowed-address
ip-address=[,mac-address=]]
- [--no-allowed-address]
- [--data-plane-status ] [--tag ]
- [--no-tag]
-
+ [--device ] [--mac-address ]
+ [--device-owner ]
+ [--vnic-type ] [--host ]
+ [--dns-domain dns-domain] [--dns-name ]
+ [--enable | --disable] [--name ]
+ [--fixed-ip subnet=,ip-address=]
+ [--no-fixed-ip]
+ [--binding-profile ]
+ [--no-binding-profile] [--qos-policy ]
+ [--security-group ]
+ [--no-security-group]
+ [--enable-port-security | --disable-port-security]
+ [--allowed-address
ip-address=[,mac-address=]]
+ [--no-allowed-address]
+ [--data-plane-status ] [--tag ]
+ [--no-tag]
+
openstack port set: error: unrecognized arguments: --extra-dhcp-option
port-test-202
The UpdatePort class from the neutron client supports --extra-dhcp-opt [2]
This is aligned with the neutron API [3]
(overcloud) $ neutron port-update port-test-202 --extra-dhcp-opt
opt_name=mtu,opt_value=1750,ip_version=4
neutron CLI
[Yahoo-eng-team] [Bug 1938569] [NEW] Typo in OVN SUPPORTED_DHCP_OPTS_MAPPING dictionary
Public bug reported:
SUPPORTED_DHCP_OPTS_MAPPING is a dictionary that maps neutron dhcp-
option names to ovn dhcp-option names. OVN dhcp-option ia_addr
corresponds with neutron dhcp-option names 'ia-addr' or '5'. However,
there is a type here and the corresponding OVN value is defined as
ip_addr instead of ia_addr [1]
Definition of ia_addr from OVN code: [2]
The following openstack command created a wrong entry in the OVN NBDB
DHCP_OPTIONS
$ openstack port create --network tempest-DHCPTest-980691237
--extra-dhcp-option name=ia-addr,value=2001::2001,ip-version=6 port-test-202
# ovn-nbctl list dhcp_options c9b0f6c0-c1bf-4e91-9e66-5e75491ae9e5
_uuid : c9b0f6c0-c1bf-4e91-9e66-5e75491ae9e5
cidr: "2001:db8::/64"
external_ids: {"neutron:revision_number"="0",
port_id="1188bca5-4b75-4810-a9a0-38ecc96d0274",
subnet_id="567272f0-f1b8-423c-a939-9dea6b5e26ab"}
options : {dhcpv6_stateless="true", ip_addr="2001::2001",
server_id="fa:16:3e:ce:f4:67"}
[1]
https://opendev.org/openstack/neutron/src/commit/19372a3cd8b4e6e45f707753b914e133857dd629/neutron/common/ovn/constants.py#L163
[2]
https://github.com/ovn-org/ovn/blob/1c9e46ab5c05043a8cd6c47b5fec2e1ac4c962db/lib/ovn-l7.h#L261
** Affects: neutron
Importance: Undecided
Status: In Progress
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1938569
Title:
Typo in OVN SUPPORTED_DHCP_OPTS_MAPPING dictionary
Status in neutron:
In Progress
Bug description:
SUPPORTED_DHCP_OPTS_MAPPING is a dictionary that maps neutron dhcp-
option names to ovn dhcp-option names. OVN dhcp-option ia_addr
corresponds with neutron dhcp-option names 'ia-addr' or '5'. However,
there is a type here and the corresponding OVN value is defined as
ip_addr instead of ia_addr [1]
Definition of ia_addr from OVN code: [2]
The following openstack command created a wrong entry in the OVN NBDB
DHCP_OPTIONS
$ openstack port create --network tempest-DHCPTest-980691237
--extra-dhcp-option name=ia-addr,value=2001::2001,ip-version=6 port-test-202
# ovn-nbctl list dhcp_options c9b0f6c0-c1bf-4e91-9e66-5e75491ae9e5
_uuid : c9b0f6c0-c1bf-4e91-9e66-5e75491ae9e5
cidr: "2001:db8::/64"
external_ids: {"neutron:revision_number"="0",
port_id="1188bca5-4b75-4810-a9a0-38ecc96d0274",
subnet_id="567272f0-f1b8-423c-a939-9dea6b5e26ab"}
options : {dhcpv6_stateless="true", ip_addr="2001::2001",
server_id="fa:16:3e:ce:f4:67"}
[1]
https://opendev.org/openstack/neutron/src/commit/19372a3cd8b4e6e45f707753b914e133857dd629/neutron/common/ovn/constants.py#L163
[2]
https://github.com/ovn-org/ovn/blob/1c9e46ab5c05043a8cd6c47b5fec2e1ac4c962db/lib/ovn-l7.h#L261
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1938569/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1918405] [NEW] new ovn-metadata-agent created when existing one dies
Public bug reported: This bug is reproduced during the execution of devstack-tobiko-gate-ovn job. It is not reproduced always. 1) Create resources https://1002eb5dcbc9d30cd274-231091edc31f40b3d022f1a19bae7260.ssl.cf1.rackcdn.com/776837/2/check/devstack-tobiko-gate-ovn/27904c1/tobiko_results_03_create_resources_scenario.html tobiko/tests/scenario/neutron/test_agents.py::NeutronAgentTest::test_agents_are_alive - PASS - All 2 Neutron agents are alive 2) Faults https://1002eb5dcbc9d30cd274-231091edc31f40b3d022f1a19bae7260.ssl.cf1.rackcdn.com/776837/2/check/devstack-tobiko-gate-ovn/27904c1/tobiko_results_04_faults_faults.html tobiko/tests/faults/neutron/test_agents.py::OvnControllerTest::test_restart_ovn_controller - FAIL a) Stops ovn-controller: Service 'ovn-controller' stopped on host 'ubuntu-focal-vexxhost-ca-ymq-1-0023280630' b) Start ovn-controller: Starting service 'ovn-controller' on host 'ubuntu-focal-vexxhost-ca-ymq-1-0023280630'. c) At this moment, ovn-metadata-agent is not alive (I don't understand why ovn-metadata dies when ovn-controller was restarted): "alive": true, "availability_zone": "", "binary": "ovn-controller", ... "alive": false, "availability_zone": "", "binary": "neutron-ovn-metadata-agent", d) At some point, a new ovn-metadata-agent is created - One metadata agent is alive and the other one is not (this clearly looks like a bug to me): "alive": true, "availability_zone": "", "binary": "ovn-controller", ... "alive": false, "availability_zone": "", "binary": "neutron-ovn-metadata-agent", ... "alive": true, "availability_zone": "", "binary": "neutron-ovn-metadata-agent", The test fails because of the dead metadata agent. 3) Verify resources https://1002eb5dcbc9d30cd274-231091edc31f40b3d022f1a19bae7260.ssl.cf1.rackcdn.com/776837/2/check/devstack-tobiko-gate-ovn/27904c1/tobiko_results_05_verify_resources_scenario.html tobiko/tests/scenario/neutron/test_agents.py::NeutronAgentTest::test_agents_are_alive - FAIL - due to the same reason: there are two ovn-metadata-agents and one of them is dead. ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1918405 Title: new ovn-metadata-agent created when existing one dies Status in neutron: New Bug description: This bug is reproduced during the execution of devstack-tobiko-gate- ovn job. It is not reproduced always. 1) Create resources https://1002eb5dcbc9d30cd274-231091edc31f40b3d022f1a19bae7260.ssl.cf1.rackcdn.com/776837/2/check/devstack-tobiko-gate-ovn/27904c1/tobiko_results_03_create_resources_scenario.html tobiko/tests/scenario/neutron/test_agents.py::NeutronAgentTest::test_agents_are_alive - PASS - All 2 Neutron agents are alive 2) Faults https://1002eb5dcbc9d30cd274-231091edc31f40b3d022f1a19bae7260.ssl.cf1.rackcdn.com/776837/2/check/devstack-tobiko-gate-ovn/27904c1/tobiko_results_04_faults_faults.html tobiko/tests/faults/neutron/test_agents.py::OvnControllerTest::test_restart_ovn_controller - FAIL a) Stops ovn-controller: Service 'ovn-controller' stopped on host 'ubuntu-focal-vexxhost-ca-ymq-1-0023280630' b) Start ovn-controller: Starting service 'ovn-controller' on host 'ubuntu-focal-vexxhost-ca-ymq-1-0023280630'. c) At this moment, ovn-metadata-agent is not alive (I don't understand why ovn-metadata dies when ovn-controller was restarted): "alive": true, "availability_zone": "", "binary": "ovn-controller", ... "alive": false, "availability_zone": "", "binary": "neutron-ovn-metadata-agent", d) At some point, a new ovn-metadata-agent is created - One metadata agent is alive and the other one is not (this clearly looks like a bug to me): "alive": true, "availability_zone": "", "binary": "ovn-controller", ... "alive": false, "availability_zone": "", "binary": "neutron-ovn-metadata-agent", ... "alive": true, "availability_zone": "", "binary": "neutron-ovn-metadata-agent", The test fails because of the dead metadata agent. 3) Verify resources https://1002eb5dcbc9d30cd274-231091edc31f40b3d022f1a19bae7260.ssl.cf1.rackcdn.com/776837/2/check/devstack-tobiko-gate-ovn/27904c1/tobiko_results_05_verify_resources_scenario.html tobiko/tests/scenario/neutron/test_agents.py::NeutronAgentTest::test_agents_are_alive - FAIL - due to the same reason: there are two ovn-metadata-agents and one of them is dead. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1918405/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.n
[Yahoo-eng-team] [Bug 1907548] [NEW] vlan-transparency not working with linuxbridge ml2 agent
Public bug reported:
After vlan-transparency (VT) support has been added for OVN ML2 agent,
we are creating some neutron tempest tests to cover this functionality.
Since linuxbridge (LB) agent supports VT, we intended to run those new
tests on the master LB job, but they fail.
We configured the LB agent with VT support. Devstack configures ml2
mechanism_drivers to openvswitch and linuxbridge, so we had to set it to LB
only.
https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/765002/7/zuul.d/master_jobs.yaml
New VT tests failed:
2020-12-09 13:05:37.184727 | controller | {0}
neutron_tempest_plugin.scenario.test_vlan_transparency.VlanTransparencyTest.test_vlan_transparent_allowed_address_pairs
[310.736639s] ... FAILED
2020-12-09 13:05:37.185480 | controller | Details: {'code': 500, 'created':
'2020-12-09T13:05:35Z', 'message': 'Build of instance
e0513133-3077-420e-b439-e0f61ff0fbad aborted: Failed to allocate the
network(s), not rescheduling.'}
https://zuul.opendev.org/t/openstack/build/d4f0a6a2a3824705886536e5d900a49e/log/job-output.txt#25261
nova-compute logs show errors creating instances:
Dec 09 13:05:33.788419 ubuntu-focal-rax-iad-0022130345 nova-compute[73187]:
ERROR nova.compute.manager [None req-c4f267db-cf04-4597-bbcc-e95ce67db222
tempest-VlanTransparencyTest-301394299 tempest-VlanTransparencyTest-301394299]
[instance: e0513133-3077-420e-b439-e0f61ff0fbad] Instance failed to spawn:
nova.exception.VirtualInterfaceCreateException: Virtual Interface creation
failed
https://zuul.opendev.org/t/openstack/build/d4f0a6a2a3824705886536e5d900a49e/log/controller/logs/screen-n-cpu.txt?severity=4
neutron-linuxbridge-agent shows some errors too:
Dec 09 12:27:01.234687 ubuntu-focal-rax-iad-0022130345
neutron-linuxbridge-agent[66293]: ERROR neutron.agent.linux.utils [None
req-ddbe0c50-3fb7-4834-9404-f6a8988be69b None None] Exit code: 255; Cmd:
['sysctl', '-w', 'net.ipv6.conf.vxlan-3.disable_ipv6=1']; Stdin: ; Stdout: ;
Stderr: sysctl: cannot stat /proc/sys/net/ipv6/conf/vxlan-3/disable_ipv6: No
such file or directory
https://zuul.opendev.org/t/openstack/build/d4f0a6a2a3824705886536e5d900a49e/log/controller/logs/screen-q-agt.txt?severity=4
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1907548
Title:
vlan-transparency not working with linuxbridge ml2 agent
Status in neutron:
New
Bug description:
After vlan-transparency (VT) support has been added for OVN ML2 agent,
we are creating some neutron tempest tests to cover this
functionality.
Since linuxbridge (LB) agent supports VT, we intended to run those new
tests on the master LB job, but they fail.
We configured the LB agent with VT support. Devstack configures ml2
mechanism_drivers to openvswitch and linuxbridge, so we had to set it to LB
only.
https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/765002/7/zuul.d/master_jobs.yaml
New VT tests failed:
2020-12-09 13:05:37.184727 | controller | {0}
neutron_tempest_plugin.scenario.test_vlan_transparency.VlanTransparencyTest.test_vlan_transparent_allowed_address_pairs
[310.736639s] ... FAILED
2020-12-09 13:05:37.185480 | controller | Details: {'code': 500, 'created':
'2020-12-09T13:05:35Z', 'message': 'Build of instance
e0513133-3077-420e-b439-e0f61ff0fbad aborted: Failed to allocate the
network(s), not rescheduling.'}
https://zuul.opendev.org/t/openstack/build/d4f0a6a2a3824705886536e5d900a49e/log/job-output.txt#25261
nova-compute logs show errors creating instances:
Dec 09 13:05:33.788419 ubuntu-focal-rax-iad-0022130345 nova-compute[73187]:
ERROR nova.compute.manager [None req-c4f267db-cf04-4597-bbcc-e95ce67db222
tempest-VlanTransparencyTest-301394299 tempest-VlanTransparencyTest-301394299]
[instance: e0513133-3077-420e-b439-e0f61ff0fbad] Instance failed to spawn:
nova.exception.VirtualInterfaceCreateException: Virtual Interface creation
failed
https://zuul.opendev.org/t/openstack/build/d4f0a6a2a3824705886536e5d900a49e/log/controller/logs/screen-n-cpu.txt?severity=4
neutron-linuxbridge-agent shows some errors too:
Dec 09 12:27:01.234687 ubuntu-focal-rax-iad-0022130345
neutron-linuxbridge-agent[66293]: ERROR neutron.agent.linux.utils [None
req-ddbe0c50-3fb7-4834-9404-f6a8988be69b None None] Exit code: 255; Cmd:
['sysctl', '-w', 'net.ipv6.conf.vxlan-3.disable_ipv6=1']; Stdin: ; Stdout: ;
Stderr: sysctl: cannot stat /proc/sys/net/ipv6/conf/vxlan-3/disable_ipv6: No
such file or directory
https://zuul.opendev.org/t/openstack/build/d4f0a6a2a3824705886536e5d900a49e/log/controller/logs/screen-q-agt.txt?severity=4
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1907548/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe
[Yahoo-eng-team] [Bug 1892741] [NEW] neutron-sriov-nic-agent cannot be disabled
Public bug reported: During a test, neutron-sriov-nic-agent was disabled (openstack network agent set --disable ) and then some operations that require the intervention of this agent were successfully performned. So, apparently the agent is not really disabled, despite showing state=DOWN. Please see the bug reproduction here: http://paste.openstack.org/show/797088/ Apparently, only DHCP and L3 agents receive the status flag. In other words: only those two agents are capable of being enabled or disabled: https://github.com/openstack/neutron/blob/master/neutron/db/agentschedulers_db.py#L53-L56 Maybe the API should answer the --disable request with an error, instead of 200 OK. ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1892741 Title: neutron-sriov-nic-agent cannot be disabled Status in neutron: New Bug description: During a test, neutron-sriov-nic-agent was disabled (openstack network agent set --disable ) and then some operations that require the intervention of this agent were successfully performned. So, apparently the agent is not really disabled, despite showing state=DOWN. Please see the bug reproduction here: http://paste.openstack.org/show/797088/ Apparently, only DHCP and L3 agents receive the status flag. In other words: only those two agents are capable of being enabled or disabled: https://github.com/openstack/neutron/blob/master/neutron/db/agentschedulers_db.py#L53-L56 Maybe the API should answer the --disable request with an error, instead of 200 OK. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1892741/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1861397] [NEW] Needed mechanism to differentiate pings sent in tempest tests
Public bug reported: Tempest Neutron plugin includes function check_remote_connectivity. It sends ping messages from VM instances to validate connectivity. There could be problems to correctly identify/filter those messages when several tests are running in parallel. One simple way to be able to filter those messages is using ping option "-p ". That pattern is included in the ping messages and can be filtered out when packets are captured. ** Affects: neutron Importance: Undecided Assignee: Eduardo Olivares (eolivare) Status: In Progress ** Tags: tempest -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1861397 Title: Needed mechanism to differentiate pings sent in tempest tests Status in neutron: In Progress Bug description: Tempest Neutron plugin includes function check_remote_connectivity. It sends ping messages from VM instances to validate connectivity. There could be problems to correctly identify/filter those messages when several tests are running in parallel. One simple way to be able to filter those messages is using ping option "-p ". That pattern is included in the ping messages and can be filtered out when packets are captured. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1861397/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
