Re: [ovs-dev] [PATCH v2 4/6] ci: Separate job for debs, ensure built pkg is tested.

2022-07-07 Thread Frode Nordahl 
On Fri, Jul 1, 2022 at 4:39 PM Ilya Maximets  wrote:
>
> On 6/29/22 15:52, Frode Nordahl wrote:
> > Use a separate GitHub Actions job for deb test so that we can
> > control base image for package test.
> >
> > The CI deb package test code currently attempts to use `apt` to
> > install local packages.  That may not produce the expected result.
> >
> > Explicitly install the local packages with `dpkg` after installing
> > dependencies from `apt` instead.
> >
> > Also enable test installation of ipsec deb package.
> >
> > Signed-off-by: Frode Nordahl 
> > ---
> >  .ci/linux-build.sh   | 21 --
> >  .github/workflows/build-and-test.yml | 62 +---
> >  2 files changed, 65 insertions(+), 18 deletions(-)
> >
> > diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
> > index 9963fb810..a8c437aaf 100755
> > --- a/.ci/linux-build.sh
> > +++ b/.ci/linux-build.sh
> > @@ -207,10 +207,23 @@ if [ "$DEB_PACKAGE" ]; then
> >  mk-build-deps --install --root-cmd sudo --remove debian/control
> >  dpkg-checkbuilddeps
> >  DEB_BUILD_OPTIONS='parallel=4 nocheck' fakeroot debian/rules binary
> > -# Not trying to install ipsec package as there are issues with 
> > system-wide
> > -# installed python3-openvswitch package and the pyenv used by Travis.
> > -packages=$(ls $(pwd)/../*.deb | grep -v ipsec)
> > -sudo apt install ${packages}
> > +packages=$(ls $(pwd)/../*.deb)
> > +deps=""
> > +for pkg in $packages; do
> > +_ifs=$IFS
> > +IFS=","
> > +for dep in $(dpkg-deb -f $pkg Depends); do
> > +dep_name=$(echo "$dep"|awk '{print$1}')
> > +# Don't install internal package inter-dependencies from apt
> > +echo $dep_name | grep -q openvswitch && continue
> > +deps+=" $dep_name"
> > +done
> > +IFS=$_ifs
> > +done
> > +# install package dependencies from apt
> > +echo $deps | xargs sudo apt -y install
> > +# install the locally built openvswitch packages
> > +sudo dpkg -i $packages
> >  exit 0
> >  fi
> >
> > diff --git a/.github/workflows/build-and-test.yml 
> > b/.github/workflows/build-and-test.yml
> > index ec34a9b0d..b36282ce5 100644
> > --- a/.github/workflows/build-and-test.yml
> > +++ b/.github/workflows/build-and-test.yml
> > @@ -9,13 +9,10 @@ jobs:
> >  automake libtool gcc bc libjemalloc1 libjemalloc-dev\
> >  libssl-dev llvm-dev libelf-dev libnuma-dev libpcap-dev  \
> >  ninja-build selinux-policy-dev
> > -  deb_dependencies: |
> > -linux-headers-$(uname -r) build-essential fakeroot devscripts 
> > equivs
> >AFXDP:   ${{ matrix.afxdp }}
> >ASAN:${{ matrix.asan }}
> >UBSAN:   ${{ matrix.ubsan }}
> >CC:  ${{ matrix.compiler }}
> > -  DEB_PACKAGE: ${{ matrix.deb_package }}
> >DPDK:${{ matrix.dpdk }}
> >DPDK_SHARED: ${{ matrix.dpdk_shared }}
> >KERNEL:  ${{ matrix.kernel }}
> > @@ -148,11 +145,7 @@ jobs:
> >  - name: update APT cache
> >run:  sudo apt update || true
> >  - name: install common dependencies
> > -  if:   matrix.deb_package == ''
> >run:  sudo apt install -y ${{ env.dependencies }}
> > -- name: install dependencies for debian packages
> > -  if:   matrix.deb_package != ''
> > -  run:  sudo apt install -y ${{ env.deb_dependencies }}
> >  - name: install libunbound libunwind
> >if:   matrix.m32 == ''
> >run:  sudo apt install -y libunbound-dev libunwind-dev
> > @@ -163,13 +156,6 @@ jobs:
> >  - name: build
> >run:  ./.ci/linux-build.sh
> >
> > -- name: upload deb packages
> > -  if:   matrix.deb_package != ''
> > -  uses: actions/upload-artifact@v2
> > -  with:
> > -name: deb-packages
> > -path: '/home/runner/work/ovs/*.deb'
> > -
> >  - name: copy logs on failure
> >if: failure() || cancelled()
> >run: |
> > @@ -224,3 +210,51 @@ jobs:
> >with:
> >  name: logs-osx-clang---disable-ssl
> >  path: config.log
> > +
> > +  build-linux-deb:
> > +env:
> > +  deb_dependencies: |
> > +linux-headers-$(uname -r) build-essential fakeroot devscripts 
> > equivs
> > +  DEB_PACKAGE: ${{ matrix.deb_package }}
>
> We should, porbbaly, just remove the 'matrix' from this job
> and define env variables directly. i.e.:
>
>DEB_PACKAGE: yes
>
> > +
> > +name: linux ${{ join(matrix.*, ' ') }}
>
> And here just:
>name: linux deb
> Or something along these lines.
>
> > +runs-on: ubuntu-22.04
> > +timeout-minutes: 30
> > +
> > +strategy:
> > +  fail-fast: false
> > +  matrix:
> > +include:
> > +  - compiler: gcc
> > +deb_package:  deb
>
> The 'matrix' section can be just removed this way.
> And we're not using the 'compiler' anywhere.

Ack, I'll incorporate the proposed changes. Thx

Re: [ovs-dev] [PATCH v2 4/6] ci: Separate job for debs, ensure built pkg is tested.

2022-07-01 Thread Ilya Maximets 
On 6/29/22 15:52, Frode Nordahl wrote:
> Use a separate GitHub Actions job for deb test so that we can
> control base image for package test.
> 
> The CI deb package test code currently attempts to use `apt` to
> install local packages.  That may not produce the expected result.
> 
> Explicitly install the local packages with `dpkg` after installing
> dependencies from `apt` instead.
> 
> Also enable test installation of ipsec deb package.
> 
> Signed-off-by: Frode Nordahl 
> ---
>  .ci/linux-build.sh   | 21 --
>  .github/workflows/build-and-test.yml | 62 +---
>  2 files changed, 65 insertions(+), 18 deletions(-)
> 
> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
> index 9963fb810..a8c437aaf 100755
> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
> @@ -207,10 +207,23 @@ if [ "$DEB_PACKAGE" ]; then
>  mk-build-deps --install --root-cmd sudo --remove debian/control
>  dpkg-checkbuilddeps
>  DEB_BUILD_OPTIONS='parallel=4 nocheck' fakeroot debian/rules binary
> -# Not trying to install ipsec package as there are issues with 
> system-wide
> -# installed python3-openvswitch package and the pyenv used by Travis.
> -packages=$(ls $(pwd)/../*.deb | grep -v ipsec)
> -sudo apt install ${packages}
> +packages=$(ls $(pwd)/../*.deb)
> +deps=""
> +for pkg in $packages; do
> +_ifs=$IFS
> +IFS=","
> +for dep in $(dpkg-deb -f $pkg Depends); do
> +dep_name=$(echo "$dep"|awk '{print$1}')
> +# Don't install internal package inter-dependencies from apt
> +echo $dep_name | grep -q openvswitch && continue
> +deps+=" $dep_name"
> +done
> +IFS=$_ifs
> +done
> +# install package dependencies from apt
> +echo $deps | xargs sudo apt -y install
> +# install the locally built openvswitch packages
> +sudo dpkg -i $packages
>  exit 0
>  fi
>  
> diff --git a/.github/workflows/build-and-test.yml 
> b/.github/workflows/build-and-test.yml
> index ec34a9b0d..b36282ce5 100644
> --- a/.github/workflows/build-and-test.yml
> +++ b/.github/workflows/build-and-test.yml
> @@ -9,13 +9,10 @@ jobs:
>  automake libtool gcc bc libjemalloc1 libjemalloc-dev\
>  libssl-dev llvm-dev libelf-dev libnuma-dev libpcap-dev  \
>  ninja-build selinux-policy-dev
> -  deb_dependencies: |
> -linux-headers-$(uname -r) build-essential fakeroot devscripts equivs
>AFXDP:   ${{ matrix.afxdp }}
>ASAN:${{ matrix.asan }}
>UBSAN:   ${{ matrix.ubsan }}
>CC:  ${{ matrix.compiler }}
> -  DEB_PACKAGE: ${{ matrix.deb_package }}
>DPDK:${{ matrix.dpdk }}
>DPDK_SHARED: ${{ matrix.dpdk_shared }}
>KERNEL:  ${{ matrix.kernel }}
> @@ -148,11 +145,7 @@ jobs:
>  - name: update APT cache
>run:  sudo apt update || true
>  - name: install common dependencies
> -  if:   matrix.deb_package == ''
>run:  sudo apt install -y ${{ env.dependencies }}
> -- name: install dependencies for debian packages
> -  if:   matrix.deb_package != ''
> -  run:  sudo apt install -y ${{ env.deb_dependencies }}
>  - name: install libunbound libunwind
>if:   matrix.m32 == ''
>run:  sudo apt install -y libunbound-dev libunwind-dev
> @@ -163,13 +156,6 @@ jobs:
>  - name: build
>run:  ./.ci/linux-build.sh
>  
> -- name: upload deb packages
> -  if:   matrix.deb_package != ''
> -  uses: actions/upload-artifact@v2
> -  with:
> -name: deb-packages
> -path: '/home/runner/work/ovs/*.deb'
> -
>  - name: copy logs on failure
>if: failure() || cancelled()
>run: |
> @@ -224,3 +210,51 @@ jobs:
>with:
>  name: logs-osx-clang---disable-ssl
>  path: config.log
> +
> +  build-linux-deb:
> +env:
> +  deb_dependencies: |
> +linux-headers-$(uname -r) build-essential fakeroot devscripts equivs
> +  DEB_PACKAGE: ${{ matrix.deb_package }}

We should, porbbaly, just remove the 'matrix' from this job
and define env variables directly. i.e.:

   DEB_PACKAGE: yes

> +
> +name: linux ${{ join(matrix.*, ' ') }}

And here just:
   name: linux deb
Or something along these lines.

> +runs-on: ubuntu-22.04
> +timeout-minutes: 30
> +
> +strategy:
> +  fail-fast: false
> +  matrix:
> +include:
> +  - compiler: gcc
> +deb_package:  deb

The 'matrix' section can be just removed this way.
And we're not using the 'compiler' anywhere.

> +
> +steps:
> +- name: checkout
> +  uses: actions/checkout@v2
> +
> +- name: update PATH
> +  run:  |
> +echo "$HOME/bin">> $GITHUB_PATH
> +echo "$HOME/.local/bin" >> $GITHUB_PATH
> +
> +- name: set up python
> +  uses: actions/setup-python@v2
> +  with:
> +python-version: '3.9'
> +
> +-

[ovs-dev] [PATCH v2 4/6] ci: Separate job for debs, ensure built pkg is tested.

2022-06-29 Thread Frode Nordahl 
Use a separate GitHub Actions job for deb test so that we can
control base image for package test.

The CI deb package test code currently attempts to use `apt` to
install local packages.  That may not produce the expected result.

Explicitly install the local packages with `dpkg` after installing
dependencies from `apt` instead.

Also enable test installation of ipsec deb package.

Signed-off-by: Frode Nordahl 
---
 .ci/linux-build.sh   | 21 --
 .github/workflows/build-and-test.yml | 62 +---
 2 files changed, 65 insertions(+), 18 deletions(-)

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index 9963fb810..a8c437aaf 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -207,10 +207,23 @@ if [ "$DEB_PACKAGE" ]; then
 mk-build-deps --install --root-cmd sudo --remove debian/control
 dpkg-checkbuilddeps
 DEB_BUILD_OPTIONS='parallel=4 nocheck' fakeroot debian/rules binary
-# Not trying to install ipsec package as there are issues with system-wide
-# installed python3-openvswitch package and the pyenv used by Travis.
-packages=$(ls $(pwd)/../*.deb | grep -v ipsec)
-sudo apt install ${packages}
+packages=$(ls $(pwd)/../*.deb)
+deps=""
+for pkg in $packages; do
+_ifs=$IFS
+IFS=","
+for dep in $(dpkg-deb -f $pkg Depends); do
+dep_name=$(echo "$dep"|awk '{print$1}')
+# Don't install internal package inter-dependencies from apt
+echo $dep_name | grep -q openvswitch && continue
+deps+=" $dep_name"
+done
+IFS=$_ifs
+done
+# install package dependencies from apt
+echo $deps | xargs sudo apt -y install
+# install the locally built openvswitch packages
+sudo dpkg -i $packages
 exit 0
 fi
 
diff --git a/.github/workflows/build-and-test.yml 
b/.github/workflows/build-and-test.yml
index ec34a9b0d..b36282ce5 100644
--- a/.github/workflows/build-and-test.yml
+++ b/.github/workflows/build-and-test.yml
@@ -9,13 +9,10 @@ jobs:
 automake libtool gcc bc libjemalloc1 libjemalloc-dev\
 libssl-dev llvm-dev libelf-dev libnuma-dev libpcap-dev  \
 ninja-build selinux-policy-dev
-  deb_dependencies: |
-linux-headers-$(uname -r) build-essential fakeroot devscripts equivs
   AFXDP:   ${{ matrix.afxdp }}
   ASAN:${{ matrix.asan }}
   UBSAN:   ${{ matrix.ubsan }}
   CC:  ${{ matrix.compiler }}
-  DEB_PACKAGE: ${{ matrix.deb_package }}
   DPDK:${{ matrix.dpdk }}
   DPDK_SHARED: ${{ matrix.dpdk_shared }}
   KERNEL:  ${{ matrix.kernel }}
@@ -148,11 +145,7 @@ jobs:
 - name: update APT cache
   run:  sudo apt update || true
 - name: install common dependencies
-  if:   matrix.deb_package == ''
   run:  sudo apt install -y ${{ env.dependencies }}
-- name: install dependencies for debian packages
-  if:   matrix.deb_package != ''
-  run:  sudo apt install -y ${{ env.deb_dependencies }}
 - name: install libunbound libunwind
   if:   matrix.m32 == ''
   run:  sudo apt install -y libunbound-dev libunwind-dev
@@ -163,13 +156,6 @@ jobs:
 - name: build
   run:  ./.ci/linux-build.sh
 
-- name: upload deb packages
-  if:   matrix.deb_package != ''
-  uses: actions/upload-artifact@v2
-  with:
-name: deb-packages
-path: '/home/runner/work/ovs/*.deb'
-
 - name: copy logs on failure
   if: failure() || cancelled()
   run: |
@@ -224,3 +210,51 @@ jobs:
   with:
 name: logs-osx-clang---disable-ssl
 path: config.log
+
+  build-linux-deb:
+env:
+  deb_dependencies: |
+linux-headers-$(uname -r) build-essential fakeroot devscripts equivs
+  DEB_PACKAGE: ${{ matrix.deb_package }}
+
+name: linux ${{ join(matrix.*, ' ') }}
+runs-on: ubuntu-22.04
+timeout-minutes: 30
+
+strategy:
+  fail-fast: false
+  matrix:
+include:
+  - compiler: gcc
+deb_package:  deb
+
+steps:
+- name: checkout
+  uses: actions/checkout@v2
+
+- name: update PATH
+  run:  |
+echo "$HOME/bin">> $GITHUB_PATH
+echo "$HOME/.local/bin" >> $GITHUB_PATH
+
+- name: set up python
+  uses: actions/setup-python@v2
+  with:
+python-version: '3.9'
+
+- name: update APT cache
+  run:  sudo apt update || true
+- name: install dependencies for debian packages
+  run:  sudo apt install -y ${{ env.deb_dependencies }}
+
+- name: prepare
+  run:  ./.ci/linux-prepare.sh
+
+- name: build
+  run:  ./.ci/linux-build.sh
+
+- name: upload deb packages
+  uses: actions/upload-artifact@v2
+  with:
+name: deb-packages
+path: '/home/runner/work/ovs/*.deb'
-- 
2.36.1

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

[ovs-dev] [PATCH v2 4/6] ci: Separate job for debs, ensure built pkg is tested.

2022-06-29 Thread Frode Nordahl 
Use a separate GitHub Actions job for deb test so that we can
control base image for package test.

The CI deb package test code currently attempts to use `apt` to
install local packages.  That may not produce the expected result.

Explicitly install the local packages with `dpkg` after installing
dependencies from `apt` instead.

Also enable test installation of ipsec deb package.

Signed-off-by: Frode Nordahl 
---
 .ci/linux-build.sh   | 21 --
 .github/workflows/build-and-test.yml | 62 +---
 2 files changed, 65 insertions(+), 18 deletions(-)

diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
index 9963fb810..a8c437aaf 100755
--- a/.ci/linux-build.sh
+++ b/.ci/linux-build.sh
@@ -207,10 +207,23 @@ if [ "$DEB_PACKAGE" ]; then
 mk-build-deps --install --root-cmd sudo --remove debian/control
 dpkg-checkbuilddeps
 DEB_BUILD_OPTIONS='parallel=4 nocheck' fakeroot debian/rules binary
-# Not trying to install ipsec package as there are issues with system-wide
-# installed python3-openvswitch package and the pyenv used by Travis.
-packages=$(ls $(pwd)/../*.deb | grep -v ipsec)
-sudo apt install ${packages}
+packages=$(ls $(pwd)/../*.deb)
+deps=""
+for pkg in $packages; do
+_ifs=$IFS
+IFS=","
+for dep in $(dpkg-deb -f $pkg Depends); do
+dep_name=$(echo "$dep"|awk '{print$1}')
+# Don't install internal package inter-dependencies from apt
+echo $dep_name | grep -q openvswitch && continue
+deps+=" $dep_name"
+done
+IFS=$_ifs
+done
+# install package dependencies from apt
+echo $deps | xargs sudo apt -y install
+# install the locally built openvswitch packages
+sudo dpkg -i $packages
 exit 0
 fi
 
diff --git a/.github/workflows/build-and-test.yml 
b/.github/workflows/build-and-test.yml
index ec34a9b0d..b36282ce5 100644
--- a/.github/workflows/build-and-test.yml
+++ b/.github/workflows/build-and-test.yml
@@ -9,13 +9,10 @@ jobs:
 automake libtool gcc bc libjemalloc1 libjemalloc-dev\
 libssl-dev llvm-dev libelf-dev libnuma-dev libpcap-dev  \
 ninja-build selinux-policy-dev
-  deb_dependencies: |
-linux-headers-$(uname -r) build-essential fakeroot devscripts equivs
   AFXDP:   ${{ matrix.afxdp }}
   ASAN:${{ matrix.asan }}
   UBSAN:   ${{ matrix.ubsan }}
   CC:  ${{ matrix.compiler }}
-  DEB_PACKAGE: ${{ matrix.deb_package }}
   DPDK:${{ matrix.dpdk }}
   DPDK_SHARED: ${{ matrix.dpdk_shared }}
   KERNEL:  ${{ matrix.kernel }}
@@ -148,11 +145,7 @@ jobs:
 - name: update APT cache
   run:  sudo apt update || true
 - name: install common dependencies
-  if:   matrix.deb_package == ''
   run:  sudo apt install -y ${{ env.dependencies }}
-- name: install dependencies for debian packages
-  if:   matrix.deb_package != ''
-  run:  sudo apt install -y ${{ env.deb_dependencies }}
 - name: install libunbound libunwind
   if:   matrix.m32 == ''
   run:  sudo apt install -y libunbound-dev libunwind-dev
@@ -163,13 +156,6 @@ jobs:
 - name: build
   run:  ./.ci/linux-build.sh
 
-- name: upload deb packages
-  if:   matrix.deb_package != ''
-  uses: actions/upload-artifact@v2
-  with:
-name: deb-packages
-path: '/home/runner/work/ovs/*.deb'
-
 - name: copy logs on failure
   if: failure() || cancelled()
   run: |
@@ -224,3 +210,51 @@ jobs:
   with:
 name: logs-osx-clang---disable-ssl
 path: config.log
+
+  build-linux-deb:
+env:
+  deb_dependencies: |
+linux-headers-$(uname -r) build-essential fakeroot devscripts equivs
+  DEB_PACKAGE: ${{ matrix.deb_package }}
+
+name: linux ${{ join(matrix.*, ' ') }}
+runs-on: ubuntu-22.04
+timeout-minutes: 30
+
+strategy:
+  fail-fast: false
+  matrix:
+include:
+  - compiler: gcc
+deb_package:  deb
+
+steps:
+- name: checkout
+  uses: actions/checkout@v2
+
+- name: update PATH
+  run:  |
+echo "$HOME/bin">> $GITHUB_PATH
+echo "$HOME/.local/bin" >> $GITHUB_PATH
+
+- name: set up python
+  uses: actions/setup-python@v2
+  with:
+python-version: '3.9'
+
+- name: update APT cache
+  run:  sudo apt update || true
+- name: install dependencies for debian packages
+  run:  sudo apt install -y ${{ env.deb_dependencies }}
+
+- name: prepare
+  run:  ./.ci/linux-prepare.sh
+
+- name: build
+  run:  ./.ci/linux-build.sh
+
+- name: upload deb packages
+  uses: actions/upload-artifact@v2
+  with:
+name: deb-packages
+path: '/home/runner/work/ovs/*.deb'
-- 
2.36.1

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev