The DPDK unit test only runs if vfio or igb_uio kernel modules are loaded: on systems with only mlx5, this test is always skipped.
Besides, the test tries to grab the first device listed by dpdk-devbind.py, regardless of the PCI device status regarding kmod binding. Remove dependency on this DPDK script and use a minimal script that reads PCI sysfs. This script is not perfect, as one can imagine PCI devices bound to vfio-pci for virtual machines. Plus, this script only tries to take over vfio-pci devices. mlx5 devices can't be taken over blindly as it could mean losing connectivity to the machine if the netdev was in use for this system. For those two reasons, add a new environment variable DPDK_PCI_ADDR for testers to select the PCI device of their liking. For consistency and grep, the temporary file PCI_ADDR is renamed to DPDK_PCI_ADDR. Reviewed-by: Maxime Coquelin <maxime.coque...@redhat.com> Acked-by: Eelco Chaudron <echau...@redhat.com> Signed-off-by: David Marchand <david.march...@redhat.com> --- Changes since v4: - separated from the original series, - rebased, - dropped mlx5 devices from the discovery script, - documented DPDK_PCI_ADDR env variable, Changes since v3: - fixed nit from Maxime, Changes since v2: - sorted logs alphabetically, --- Documentation/topics/testing.rst | 11 ++++++--- tests/automake.mk | 1 + tests/system-dpdk-find-device.py | 39 ++++++++++++++++++++++++++++++++ tests/system-dpdk-macros.at | 10 ++------ tests/system-dpdk.at | 14 ++++++------ 5 files changed, 57 insertions(+), 18 deletions(-) create mode 100755 tests/system-dpdk-find-device.py diff --git a/Documentation/topics/testing.rst b/Documentation/topics/testing.rst index 5f6940b84d..fb9b3e77b1 100644 --- a/Documentation/topics/testing.rst +++ b/Documentation/topics/testing.rst @@ -343,15 +343,20 @@ To see a list of all the available tests, run:: These tests support a `DPDK supported NIC`_. The tests operate on a wider set of environments, for instance, when a virtual port is used. -They do require proper DPDK variables (``DPDK_DIR`` and ``DPDK_BUILD``). Moreover you need to have root privileges to load the required modules and to bind -the NIC to the DPDK-compatible driver. +a PCI device to the DPDK-compatible driver. .. _DPDK supported NIC: https://core.dpdk.org/supported/#nics +The phy test will skip if no suitable PCI device is found. +It is possible to select which PCI device is used for this test by setting the +DPDK_PCI_ADDR environment variable, which is especially useful when testing +with a mlx5 device:: + + # DPDK_PCI_ADDR=0000:82:00.0 make check-dpdk + All tests are skipped if no hugepages are configured. User must look into the DPDK manual to figure out how to `Configure hugepages`_. -The phy test will skip if no compatible physical device is available. .. _Configure hugepages: https://doc.dpdk.org/guides-22.11/linux_gsg/sys_reqs.html diff --git a/tests/automake.mk b/tests/automake.mk index f8a925012d..69117514cc 100644 --- a/tests/automake.mk +++ b/tests/automake.mk @@ -192,6 +192,7 @@ SYSTEM_OFFLOADS_TESTSUITE_AT = \ SYSTEM_DPDK_TESTSUITE_AT = \ tests/system-common-macros.at \ + tests/system-dpdk-find-device.py \ tests/system-dpdk-macros.at \ tests/system-dpdk-testsuite.at \ tests/system-dpdk.at diff --git a/tests/system-dpdk-find-device.py b/tests/system-dpdk-find-device.py new file mode 100755 index 0000000000..b0997380bc --- /dev/null +++ b/tests/system-dpdk-find-device.py @@ -0,0 +1,39 @@ +#!/usr/bin/env python3 +# Copyright (c) 2021 Red Hat, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at: +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +from pathlib import Path +import os +import sys + +# The tester might want to select a PCI device, if so, trust it. +if 'DPDK_PCI_ADDR' in os.environ: + print(os.environ['DPDK_PCI_ADDR']) + sys.exit(0) + +for device in sorted(Path('/sys/bus/pci/devices').iterdir()): + class_path = device / 'class' + # Only consider Network class devices + if class_path.read_text().strip() != '0x020000': + continue + kmod_path = device / 'driver' / 'module' + kmod_name = kmod_path.resolve().name + # Only care about devices bound to vfio_pci or igb_uio. + if kmod_name not in ['vfio_pci', 'igb_uio']: + continue + print(device.resolve().name) + sys.exit(0) + +sys.exit(1) diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at index dcdfa55741..3b5a3512d4 100644 --- a/tests/system-dpdk-macros.at +++ b/tests/system-dpdk-macros.at @@ -19,14 +19,8 @@ m4_define([OVS_DPDK_PRE_PHY_SKIP], [dnl Perform the precheck OVS_DPDK_PRE_CHECK() - dnl Check if VFIO or UIO driver is loaded - AT_SKIP_IF([ ! (lsmod | grep -E "igb_uio|vfio") ], [], [stdout]) - - dnl Find PCI address candidate, skip if there is no DPDK-compatible NIC - AT_CHECK([$DPDK_DIR/usertools/dpdk-devbind.py -s | head -n +4 | tail -1], [], [stdout]) - AT_CHECK([cat stdout | cut -d" " -s -f1 > PCI_ADDR]) - AT_SKIP_IF([ ! test -s PCI_ADDR ]) - + dnl Check if a device is available for DPDK + AT_SKIP_IF([ ! $abs_top_srcdir/tests/system-dpdk-find-device.py > DPDK_PCI_ADDR ]) ]) diff --git a/tests/system-dpdk.at b/tests/system-dpdk.at index 17742d20a0..9cf0ec924d 100644 --- a/tests/system-dpdk.at +++ b/tests/system-dpdk.at @@ -63,7 +63,7 @@ OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) -AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat DPDK_PCI_ADDR)], [], [stdout], [stderr]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2 @@ -240,7 +240,7 @@ OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS and add policer AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) -AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat DPDK_PCI_ADDR)], [], [stdout], [stderr]) AT_CHECK([ovs-vsctl set interface phy0 ingress_policing_rate=10000 ingress_policing_burst=1000]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2 @@ -380,7 +380,7 @@ OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS and add egress policer AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) -AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat DPDK_PCI_ADDR)], [], [stdout], [stderr]) OVS_WAIT_UNTIL([ovs-vsctl set port phy0 qos=@newqos -- --id=@newqos create qos type=egress-policer other-config:cir=1250000 other-config:cbs=2048]) AT_CHECK([ovs-appctl -t ovs-vswitchd qos/show phy0], [], [stdout]) sleep 2 @@ -509,7 +509,7 @@ dnl First set MTU to its default value and confirm that value, then increase the dnl Add userspace bridge and attach it to OVS with default MTU value AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) -AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat DPDK_PCI_ADDR)], [], [stdout], [stderr]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2 @@ -552,7 +552,7 @@ dnl First set an increased MTU value and confirm that value, then decrease the M dnl Add userspace bridge and attach it to OVS and modify MTU value AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) -AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat DPDK_PCI_ADDR)], [], [stdout], [stderr]) AT_CHECK([ovs-vsctl set Interface phy0 mtu_request=9000]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2 @@ -677,7 +677,7 @@ OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS and set MTU value to max upper bound AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) -AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat DPDK_PCI_ADDR)], [], [stdout], [stderr]) AT_CHECK([ovs-vsctl set Interface phy0 mtu_request=9702]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2 @@ -718,7 +718,7 @@ OVS_DPDK_START() dnl Add userspace bridge and attach it to OVS and set MTU value to min lower bound AT_CHECK([ovs-vsctl add-br br10 -- set bridge br10 datapath_type=netdev]) -AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat PCI_ADDR)], [], [stdout], [stderr]) +AT_CHECK([ovs-vsctl add-port br10 phy0 -- set Interface phy0 type=dpdk options:dpdk-devargs=$(cat DPDK_PCI_ADDR)], [], [stdout], [stderr]) AT_CHECK([ovs-vsctl set Interface phy0 mtu_request=68]) AT_CHECK([ovs-vsctl show], [], [stdout]) sleep 2 -- 2.41.0 _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
