** Description changed:
+ [Impact]
+ Hardware offload VXLAN tunnel flows may not be correctly offloaded and are
hence not functional in OpenStack deployments
+
+ [Test Case]
+ Deploy OpenStack configured with suitable network cards for hardware offload
of VM interfaces
+ Boot instances with hardware offloaded interfaces (capability=switchdev)
+ Instances may or may not get DHCP configured depending on how OpenStack
chooses to tag the flows associated with the VXLAN tunnel networking for the
cloud.
+
+ [Regression Potential]
+ TBC
+
+ [Original Bug Report]
OpenStack Train
OVS 2.12
Linux 5.3 from hwe edge
Connect-X 5 running latest firmware from Mellanox.
Overlay networking configured with VXLAN for project networking.
Some instances successfully DHCP, some don't.
For those that don't:
compute:
11:51:24.926581 IP bond1.2925.node-laveran.maas.58901 >
bond1.2925.node-lepaute.maas.4789: VXLAN, flags [I] (0x08), vni 1233
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from
fa:16:3e:e8:59:f3 (oui Unknown), length 300
gateway:
11:50:44.010864 IP bond1.2925.node-laveran.maas.58901 >
bond1.2925.node-lepaute.maas.4789: VXLAN, flags [I] (0x08), vni 1233
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from
fa:16:3e:e8:59:f3 (oui Unknown), length 300
11:50:44.011196 IP bond1.2925.node-lepaute.maas.55067 >
bond1.2925.node-laveran.maas.4789: VXLAN, flags [I] (0x08), vni 1233
IP 172.16.0.2.bootps > 172.16.2.236.bootpc: BOOTP/DHCP, Reply, length 328
The response is send from the gateway unit to the compute unit, but is
never seen in the tcpdump.
If hardware encap is disabled on the two underlying ports/the card,
traffic flow is seen and all instances correctly DHCP configure on boot.
- sudo ethtool -K enp3s0f0 hw-tc-offload off
- sudo ethtool -K enp3s0f1 hw-tc-offload off
+ sudo ethtool -K enp3s0f0 hw-tc-offload off
+ sudo ethtool -K enp3s0f1 hw-tc-offload off
ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: linux-image-5.3.0-23-generic 5.3.0-23.25~18.04.1
ProcVersionSignature: Ubuntu 5.3.0-23.25~18.04.1-generic 5.3.7
Uname: Linux 5.3.0-23-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.9
Architecture: amd64
Date: Fri Nov 22 12:29:20 2019
ProcEnviron:
- TERM=screen-256color
- PATH=(custom, no user)
- XDG_RUNTIME_DIR=<set>
- LANG=C.UTF-8
- SHELL=/bin/bash
+ TERM=screen-256color
+ PATH=(custom, no user)
+ XDG_RUNTIME_DIR=<set>
+ LANG=C.UTF-8
+ SHELL=/bin/bash
SourcePackage: linux-signed-hwe-edge
UpgradeStatus: No upgrade log present (probably fresh install)
- ---
+ ---
ProblemType: Bug
AlsaDevices:
- total 0
- crw-rw---- 1 root audio 116, 1 Nov 19 16:45 seq
- crw-rw---- 1 root audio 116, 33 Nov 19 16:45 timer
+ total 0
+ crw-rw---- 1 root audio 116, 1 Nov 19 16:45 seq
+ crw-rw---- 1 root audio 116, 33 Nov 19 16:45 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.9-0ubuntu7.9
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord':
'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
DistroRelease: Ubuntu 18.04
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
Lsusb:
- Bus 002 Device 002: ID 8087:8002 Intel Corp.
- Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
- Bus 001 Device 003: ID 413c:a001 Dell Computer Corp. Hub
- Bus 001 Device 002: ID 8087:800a Intel Corp.
- Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
+ Bus 002 Device 002: ID 8087:8002 Intel Corp.
+ Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
+ Bus 001 Device 003: ID 413c:a001 Dell Computer Corp. Hub
+ Bus 001 Device 002: ID 8087:800a Intel Corp.
+ Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: Dell Inc. PowerEdge R630
Package: linux (not installed)
PciMultimedia:
-
+
ProcEnviron:
- TERM=screen-256color
- PATH=(custom, no user)
- LANG=C.UTF-8
- SHELL=/bin/bash
+ TERM=screen-256color
+ PATH=(custom, no user)
+ LANG=C.UTF-8
+ SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.3.0-23-generic
root=UUID=2ff5e234-ee62-4bce-8266-cd9aa78c532f ro intel_iommu=on iommu=pt
probe_vf=0
ProcVersionSignature: Ubuntu 5.3.0-23.25~18.04.1-generic 5.3.7
RelatedPackageVersions:
- linux-restricted-modules-5.3.0-23-generic N/A
- linux-backports-modules-5.3.0-23-generic N/A
- linux-firmware 1.173.12
+ linux-restricted-modules-5.3.0-23-generic N/A
+ linux-backports-modules-5.3.0-23-generic N/A
+ linux-firmware 1.173.12
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
Tags: bionic uec-images
Uname: Linux 5.3.0-23-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:
-
+
_MarkForUpload: True
dmi.bios.date: 11/08/2016
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.3.4
dmi.board.name: 02C2CP
dmi.board.vendor: Dell Inc.
dmi.board.version: A03
dmi.chassis.type: 23
dmi.chassis.vendor: Dell Inc.
dmi.modalias:
dmi:bvnDellInc.:bvr2.3.4:bd11/08/2016:svnDellInc.:pnPowerEdgeR630:pvr:rvnDellInc.:rn02C2CP:rvrA03:cvnDellInc.:ct23:cvr:
dmi.product.name: PowerEdge R630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge R630
dmi.sys.vendor: Dell Inc.
** Summary changed:
- hw-tc-offload not reliable with VXLAN
+ [SRU] hw-tc-offload not reliable with VXLAN with OpenStack
** Description changed:
[Impact]
Hardware offload VXLAN tunnel flows may not be correctly offloaded and are
hence not functional in OpenStack deployments
[Test Case]
Deploy OpenStack configured with suitable network cards for hardware offload
of VM interfaces
Boot instances with hardware offloaded interfaces (capability=switchdev)
Instances may or may not get DHCP configured depending on how OpenStack
chooses to tag the flows associated with the VXLAN tunnel networking for the
cloud.
[Regression Potential]
- TBC
+ Low; proposed patch has been tested with OpenStack Stein and has been peer
reviewed and landed into all OVS stable branches back to v2.8
[Original Bug Report]
OpenStack Train
OVS 2.12
Linux 5.3 from hwe edge
Connect-X 5 running latest firmware from Mellanox.
Overlay networking configured with VXLAN for project networking.
Some instances successfully DHCP, some don't.
For those that don't:
compute:
11:51:24.926581 IP bond1.2925.node-laveran.maas.58901 >
bond1.2925.node-lepaute.maas.4789: VXLAN, flags [I] (0x08), vni 1233
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from
fa:16:3e:e8:59:f3 (oui Unknown), length 300
gateway:
11:50:44.010864 IP bond1.2925.node-laveran.maas.58901 >
bond1.2925.node-lepaute.maas.4789: VXLAN, flags [I] (0x08), vni 1233
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from
fa:16:3e:e8:59:f3 (oui Unknown), length 300
11:50:44.011196 IP bond1.2925.node-lepaute.maas.55067 >
bond1.2925.node-laveran.maas.4789: VXLAN, flags [I] (0x08), vni 1233
IP 172.16.0.2.bootps > 172.16.2.236.bootpc: BOOTP/DHCP, Reply, length 328
The response is send from the gateway unit to the compute unit, but is
never seen in the tcpdump.
If hardware encap is disabled on the two underlying ports/the card,
traffic flow is seen and all instances correctly DHCP configure on boot.
sudo ethtool -K enp3s0f0 hw-tc-offload off
sudo ethtool -K enp3s0f1 hw-tc-offload off
ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: linux-image-5.3.0-23-generic 5.3.0-23.25~18.04.1
ProcVersionSignature: Ubuntu 5.3.0-23.25~18.04.1-generic 5.3.7
Uname: Linux 5.3.0-23-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.9
Architecture: amd64
Date: Fri Nov 22 12:29:20 2019
ProcEnviron:
TERM=screen-256color
PATH=(custom, no user)
XDG_RUNTIME_DIR=<set>
LANG=C.UTF-8
SHELL=/bin/bash
SourcePackage: linux-signed-hwe-edge
UpgradeStatus: No upgrade log present (probably fresh install)
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Nov 19 16:45 seq
crw-rw---- 1 root audio 116, 33 Nov 19 16:45 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.9-0ubuntu7.9
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord':
'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
DistroRelease: Ubuntu 18.04
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
Lsusb:
Bus 002 Device 002: ID 8087:8002 Intel Corp.
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 413c:a001 Dell Computer Corp. Hub
Bus 001 Device 002: ID 8087:800a Intel Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: Dell Inc. PowerEdge R630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=screen-256color
PATH=(custom, no user)
LANG=C.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.3.0-23-generic
root=UUID=2ff5e234-ee62-4bce-8266-cd9aa78c532f ro intel_iommu=on iommu=pt
probe_vf=0
ProcVersionSignature: Ubuntu 5.3.0-23.25~18.04.1-generic 5.3.7
RelatedPackageVersions:
linux-restricted-modules-5.3.0-23-generic N/A
linux-backports-modules-5.3.0-23-generic N/A
linux-firmware 1.173.12
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
Tags: bionic uec-images
Uname: Linux 5.3.0-23-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:
_MarkForUpload: True
dmi.bios.date: 11/08/2016
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.3.4
dmi.board.name: 02C2CP
dmi.board.vendor: Dell Inc.
dmi.board.version: A03
dmi.chassis.type: 23
dmi.chassis.vendor: Dell Inc.
dmi.modalias:
dmi:bvnDellInc.:bvr2.3.4:bd11/08/2016:svnDellInc.:pnPowerEdgeR630:pvr:rvnDellInc.:rn02C2CP:rvrA03:cvnDellInc.:ct23:cvr:
dmi.product.name: PowerEdge R630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge R630
dmi.sys.vendor: Dell Inc.
** Description changed:
[Impact]
Hardware offload VXLAN tunnel flows may not be correctly offloaded and are
hence not functional in OpenStack deployments
[Test Case]
Deploy OpenStack configured with suitable network cards for hardware offload
of VM interfaces
Boot instances with hardware offloaded interfaces (capability=switchdev)
Instances may or may not get DHCP configured depending on how OpenStack
chooses to tag the flows associated with the VXLAN tunnel networking for the
cloud.
[Regression Potential]
- Low; proposed patch has been tested with OpenStack Stein and has been peer
reviewed and landed into all OVS stable branches back to v2.8
+ Low; proposed patch has been tested with OpenStack Stein and has been peer
reviewed and landed into all OVS stable branches back to v2.8 and will be
included in the next 2.12.x release (which would target Eoan)
[Original Bug Report]
OpenStack Train
OVS 2.12
Linux 5.3 from hwe edge
Connect-X 5 running latest firmware from Mellanox.
Overlay networking configured with VXLAN for project networking.
Some instances successfully DHCP, some don't.
For those that don't:
compute:
11:51:24.926581 IP bond1.2925.node-laveran.maas.58901 >
bond1.2925.node-lepaute.maas.4789: VXLAN, flags [I] (0x08), vni 1233
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from
fa:16:3e:e8:59:f3 (oui Unknown), length 300
gateway:
11:50:44.010864 IP bond1.2925.node-laveran.maas.58901 >
bond1.2925.node-lepaute.maas.4789: VXLAN, flags [I] (0x08), vni 1233
IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from
fa:16:3e:e8:59:f3 (oui Unknown), length 300
11:50:44.011196 IP bond1.2925.node-lepaute.maas.55067 >
bond1.2925.node-laveran.maas.4789: VXLAN, flags [I] (0x08), vni 1233
IP 172.16.0.2.bootps > 172.16.2.236.bootpc: BOOTP/DHCP, Reply, length 328
The response is send from the gateway unit to the compute unit, but is
never seen in the tcpdump.
If hardware encap is disabled on the two underlying ports/the card,
traffic flow is seen and all instances correctly DHCP configure on boot.
sudo ethtool -K enp3s0f0 hw-tc-offload off
sudo ethtool -K enp3s0f1 hw-tc-offload off
ProblemType: Bug
DistroRelease: Ubuntu 18.04
Package: linux-image-5.3.0-23-generic 5.3.0-23.25~18.04.1
ProcVersionSignature: Ubuntu 5.3.0-23.25~18.04.1-generic 5.3.7
Uname: Linux 5.3.0-23-generic x86_64
ApportVersion: 2.20.9-0ubuntu7.9
Architecture: amd64
Date: Fri Nov 22 12:29:20 2019
ProcEnviron:
TERM=screen-256color
PATH=(custom, no user)
XDG_RUNTIME_DIR=<set>
LANG=C.UTF-8
SHELL=/bin/bash
SourcePackage: linux-signed-hwe-edge
UpgradeStatus: No upgrade log present (probably fresh install)
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Nov 19 16:45 seq
crw-rw---- 1 root audio 116, 33 Nov 19 16:45 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.9-0ubuntu7.9
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord':
'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
DistroRelease: Ubuntu 18.04
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
Lsusb:
Bus 002 Device 002: ID 8087:8002 Intel Corp.
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 413c:a001 Dell Computer Corp. Hub
Bus 001 Device 002: ID 8087:800a Intel Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: Dell Inc. PowerEdge R630
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=screen-256color
PATH=(custom, no user)
LANG=C.UTF-8
SHELL=/bin/bash
ProcFB: 0 mgag200drmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.3.0-23-generic
root=UUID=2ff5e234-ee62-4bce-8266-cd9aa78c532f ro intel_iommu=on iommu=pt
probe_vf=0
ProcVersionSignature: Ubuntu 5.3.0-23.25~18.04.1-generic 5.3.7
RelatedPackageVersions:
linux-restricted-modules-5.3.0-23-generic N/A
linux-backports-modules-5.3.0-23-generic N/A
linux-firmware 1.173.12
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
Tags: bionic uec-images
Uname: Linux 5.3.0-23-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:
_MarkForUpload: True
dmi.bios.date: 11/08/2016
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 2.3.4
dmi.board.name: 02C2CP
dmi.board.vendor: Dell Inc.
dmi.board.version: A03
dmi.chassis.type: 23
dmi.chassis.vendor: Dell Inc.
dmi.modalias:
dmi:bvnDellInc.:bvr2.3.4:bd11/08/2016:svnDellInc.:pnPowerEdgeR630:pvr:rvnDellInc.:rn02C2CP:rvrA03:cvnDellInc.:ct23:cvr:
dmi.product.name: PowerEdge R630
dmi.product.sku: SKU=NotProvided;ModelName=PowerEdge R630
dmi.sys.vendor: Dell Inc.
** Also affects: cloud-archive
Importance: Undecided
Status: New
** Also affects: cloud-archive/ussuri
Importance: Undecided
Status: New
** Also affects: cloud-archive/train
Importance: Undecided
Status: New
** Changed in: cloud-archive/ussuri
Status: New => Fix Released
** Changed in: cloud-archive/ussuri
Importance: Undecided => High
** Changed in: cloud-archive/train
Importance: Undecided => High
** Changed in: cloud-archive/train
Status: New => Triaged
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1853592
Title:
[SRU] hw-tc-offload not reliable with VXLAN with OpenStack
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1853592/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
