Re: [ovs-dev] [PATCH v2 3/7] ci: Run DPDK tests in GitHub Actions.

2023-08-25 Thread Eelco Chaudron



On 23 Aug 2023, at 17:34, David Marchand wrote:

> Let's enhance our coverage in the CI and run DPDK system tests.
>
> A few DPDK drivers are enabled in DPDK compilation.
>
> Put DPDK build in $PATH for dpdk-testpmd to be available.
> sudo drops PATH= updates and -E alone does not seem to preserve this
> variable.
> Pass PATH=$PATH when running the tests, as a workaround.
> Since those tests are run as root, the collection of logs is updated
> accordingly.
>
> In GHA, only two cores are available but some test rely on testpmd using
> three lcores.
> Add a DPDK_EAL_OPTIONS environment variable and use it to map all
> testpmd lcores to core 1 (and leave core 0 alone for OVS main and PMD
> threads).
>
> Signed-off-by: David Marchand 
> Acked-by: Aaron Conole 

Changes look good to me!

Acked-by: Eelco Chaudron 

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev


[ovs-dev] [PATCH v2 3/7] ci: Run DPDK tests in GitHub Actions.

2023-08-23 Thread David Marchand
Let's enhance our coverage in the CI and run DPDK system tests.

A few DPDK drivers are enabled in DPDK compilation.

Put DPDK build in $PATH for dpdk-testpmd to be available.
sudo drops PATH= updates and -E alone does not seem to preserve this
variable.
Pass PATH=$PATH when running the tests, as a workaround.
Since those tests are run as root, the collection of logs is updated
accordingly.

In GHA, only two cores are available but some test rely on testpmd using
three lcores.
Add a DPDK_EAL_OPTIONS environment variable and use it to map all
testpmd lcores to core 1 (and leave core 0 alone for OVS main and PMD
threads).

Signed-off-by: David Marchand 
Acked-by: Aaron Conole 
---
Changes since v1:
- rebased after DPDK build has been moved out of linux-build.sh,
- restored running "normal" checks in the DPDK jobs,

---
 .ci/dpdk-build.sh|  8 +---
 .ci/linux-build.sh   | 15 ++-
 .github/workflows/build-and-test.yml |  7 ---
 tests/system-dpdk-macros.at  |  2 +-
 4 files changed, 24 insertions(+), 8 deletions(-)

diff --git a/.ci/dpdk-build.sh b/.ci/dpdk-build.sh
index 02dcefef61..a754809d3d 100755
--- a/.ci/dpdk-build.sh
+++ b/.ci/dpdk-build.sh
@@ -35,9 +35,11 @@ function build_dpdk()
 DPDK_OPTS="$DPDK_OPTS -Ddeveloper_mode=disabled"
 
 # OVS compilation and "normal" unit tests (run in the CI) do not depend on
-# any DPDK driver being present.
-# We can disable all drivers to save compilation time.
-DPDK_OPTS="$DPDK_OPTS -Ddisable_drivers=*/*"
+# any DPDK driver.
+# check-dpdk unit tests requires testpmd and some net/ driver.
+# We can disable all drivers but them, in order to save compilation time.
+DPDK_OPTS="$DPDK_OPTS -Denable_apps=test-pmd"
+DPDK_OPTS="$DPDK_OPTS -Denable_drivers=net/null,net/tap,net/virtio"
 
 # Install DPDK using prefix.
 DPDK_OPTS="$DPDK_OPTS --prefix=$(pwd)/build"
diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index 99850a9434..f11a4af3ac 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -22,6 +22,9 @@ function install_dpdk()
 # Export the following path for pkg-config to find the .pc file.
 export PKG_CONFIG_PATH=$DPDK_LIB/pkgconfig/:$PKG_CONFIG_PATH
 
+# Expose dpdk binaries.
+export PATH=$(pwd)/dpdk-dir/build/bin:$PATH
+
 if [ ! -f "${VERSION_FILE}" ]; then
 echo "Could not find DPDK in $(pwd)/dpdk-dir"
 return 1
@@ -109,7 +112,7 @@ fi
 
 OPTS="${EXTRA_OPTS} ${OPTS} $*"
 
-if [ "$TESTSUITE" ]; then
+if [ "$TESTSUITE" = 'test' ]; then
 # 'distcheck' will reconfigure with required options.
 # Now we only need to prepare the Makefile without sparse-wrapped CC.
 configure_ovs
@@ -119,6 +122,16 @@ if [ "$TESTSUITE" ]; then
 TESTSUITEFLAGS=-j4 RECHECK=yes
 else
 build_ovs
+for testsuite in $TESTSUITE; do
+run_as_root=
+if [ "${testsuite##*dpdk}" != "$testsuite" ]; then
+sudo sh -c 'echo 1024 > /proc/sys/vm/nr_hugepages' || true
+[ "$(cat /proc/sys/vm/nr_hugepages)" = '1024' ]
+export DPDK_EAL_OPTIONS="--lcores 0@1,1@1,2@1"
+run_as_root="sudo -E PATH=$PATH"
+fi
+$run_as_root make $testsuite TESTSUITEFLAGS=-j4 RECHECK=yes
+done
 fi
 
 exit 0
diff --git a/.github/workflows/build-and-test.yml 
b/.github/workflows/build-and-test.yml
index 47d239f108..ba89a279a5 100644
--- a/.github/workflows/build-and-test.yml
+++ b/.github/workflows/build-and-test.yml
@@ -117,10 +117,10 @@ jobs:
 opts: --enable-shared
 
   - compiler: gcc
-testsuite:test
+testsuite:check check-dpdk
 dpdk: dpdk
   - compiler: clang
-testsuite:test
+testsuite:check check-dpdk
 dpdk: dpdk
 
   - compiler: gcc
@@ -207,7 +207,8 @@ jobs:
 mkdir logs
 cp config.log ./logs/
 cp -r ./*/_build/sub/tests/testsuite.* ./logs/ || true
-tar -czvf logs.tgz logs/
+sudo cp -r ./tests/*testsuite.* ./logs/ || true
+sudo tar -czvf logs.tgz logs/
 
 - name: upload logs on failure
   if: failure() || cancelled()
diff --git a/tests/system-dpdk-macros.at b/tests/system-dpdk-macros.at
index 01346fe6ec..108280f70d 100644
--- a/tests/system-dpdk-macros.at
+++ b/tests/system-dpdk-macros.at
@@ -92,7 +92,7 @@ m4_define([OVS_DPDK_CHECK_TESTPMD],
 # Start dpdk-testpmd in background.
 #
 m4_define([OVS_DPDK_START_TESTPMD],
-  [eal_options="--in-memory --single-file-segments --no-pci"
+  [eal_options="$DPDK_EAL_OPTIONS --in-memory --single-file-segments --no-pci"
options="$1"
[ "$options" != "${options%% -- *}" ] || options="$options -- "
eal_options="$eal_options ${options%% -- *}"
-- 
2.41.0

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev