[ovs-discuss] PVP Performance on Cascade Lake server

[Malvika Gupta]

Hi everyone,

I am testing the PVP performance of OvS-DPDK on the Cascade Lake server. It is 
running the latest Ubuntu 20.04 and Linux kernel version 5.4.0-39-generic. I am 
using DPDK version 19.11 and the latest OvS master branch from Github. The grub 
command line parameters is as follows:
ro default_hugepagesz=1G hugepagesz=1G hugepages=24 isolcpus=12-71 
nohz_full=12-71 rcu_nocbs=12-71 intel_iommu=on intel_pstate=disable 
intel_idle.max_cstate=0 processor.max_cstate=0 security=selinux selinux=1 
vt.handoff=1

The test environment setup for PVP scenario is as follows:
DPDK configuration for OvS
sudo ./ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
sudo ./ovs-vsctl --no-wait set Open_vSwitch . other_config:hw-offload=false
sudo ./ovs-vsctl --no-wait set Open_vSwitch . other_config:max-idle=50
sudo ./ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-lcore-mask=0x02
sudo ./ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem=2048
sudo ./ovs-vsctl --no-wait set Open_vSwitch . other_config:n-rxq=1
sudo ./ovs-vsctl --no-wait set Open_vSwitch . other_config:n-txq=1
sudo ./ovs-vsctl --no-wait set Open_vSwitch . other_config:pmd-cpu-mask=0x1100 
[This is an isolated CPU]

I am performing the PVP testing under two scenarios, EMC Disabled and EMC 
insertion for every flow and configure the setup accordingly:
sudo ./ovs-vsctl --no-wait set Open_vSwitch . other_config:emc-insert-inv-prob=0
sudo ./ovs-vsctl --no-wait set Open_vSwitch . other_config:emc-insert-inv-prob=1

PHY-VM-PHY Configuration
I have configured it exactly as shown in Using Open vSwitch with 
DPDK guide. The Guest VM 
installed is Ubuntu 18.04 and the XML file used to launch the VM is as follows:



  virt_ubuntu_vm

  80849065-dfc1-4f98-bc8a-794cf2566999

  

http://libosinfo.org/xmlns/libvirt/domain/1.0";>

  http://ubuntu.com/ubuntu/20.04"/>



  

  4194304

  4194304

  



  





  

  4

  

4096











  

  

hvm



  

  





  

  





  



  

  destroy

  restart

  destroy

  

/usr/bin/qemu-system-x86_64



  

  

  

  







  

  

  

  



  

  





  

  

  

  



  

  



  



Once the VM has booted, I follow the steps of the DPDK vHost User Ports 
guide - allocate 
hugepages to the Guest VM, install DPDK, bind the vhost-user interfaces to a 
vfio-pci driver (as opposed to the uio driver shown in the guide) and run the 
IO forwarding mode in the testpmd application.

I had 2 questions mainly:

  1.  For 1 flow, 1K flows an 10K flows, what performance numbers should I 
expect to see with my current system and OvS-DPDK configuration?
  2.  Is this configuration correct to achieve the best (high throughput) PVP 
performance on the Cascade Server? If not, then what should I do differently in 
order to achieve it?

I would really appreciate any input or suggestions from Intel folks as well as 
other community members. Please let me know if you need any more information 
from my side.

Thank you for your time,
Malvika
IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you.
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

[ovs-discuss] OVS-DPDK runtime issues on Octeon-Tx

[Malvika Gupta]

Hello,

I am new to OVS and I am trying to setup OVS using DPDK data path on the 
Octeon-Tx Board. The Octeon-Tx runs a custom kernel version:
# uname -a
Linux octeon-tx-1 4.9.0-OCTEONTX_SDK_6_2_0_p2_build_35 #2 SMP PREEMPT Fri Jan 
18 10:47:21 CET 2019 aarch64 aarch64 aarch64 GNU/Linux

My DPDK version is 18.11, OVS version is 2.11 (latest one) and Ubuntu version 
is 16.04.3 which is a custom kernel version 4.9.0. These are the steps I took 
to build DPDK on Octeon TX, as documented in
Octeon-Tx Board Support Package 
page<https://doc.dpdk.org/guides/platform/octeontx.html>:
# cd /lib/modules/$(uname -r)/source
# make menuconfig
# make
# cd ~
# wget http://fast.dpdk.org/rel/dpdk-18.11.tar.xz
# tar xf dpdk-18.11.tar.xz
# cd dpdk-18.11
# make config T=arm64-thunderx-linuxapp-gcc
# export RTE_SDK=~/dpdk-18.11
# export RTE_TARGET=arm64-thunderx-linuxapp-gcc
# make -j32 install T=$DPDK_TARGET DESTDIR=install

Then I proceeded to build OVS on DPDK by configuring to build it with the flag 
--with-dpdk. The build was successful. These are the steps I took:
# cd ~
# git clone https://github.com/openvswitch/ovs.git
# cd ovs
# ./boot.sh
# export DPDK_BUILD=$RTE_SDK/$RTE_TARGET
# ./configure --with-dpdk=$DPDK_BUILD --prefix=$HOME/usr 
--localstatedir=$HOME/var --sysconfdir=$HOME/etc --disable-ssl
# sudo make -j32
# sudo make -j32 install

I am able to run ovsdb-server but when I try to run the ovs-vswitchd daemon, 
it's not able to find the openvswitch.ko kernel module under 
/lib/modules/`uname -r`/build:
# export PATH=$PATH:$HOME/usr/share/openvswitch/scripts
# sudo mkdir -p $HOME/etc/openvswitch
# sudo mkdir -p $HOME/var/run/openvswitch
# sudo ./ovsdb-tool create $HOME/etc/openvswitch/conf.db 
$HOME/usr/share/openvswitch/vswitch.ovsschema
# Modify the /etc/ld.conf file to add the path to the DPDK library
# sudo ./ovsdb-server --remote=punix:$HOME/var/run/openvswitch/db.sock 
--remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile -detach
# sudo ./ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
# export DB_SOCK=$HOME/var/run/openvswitch/db.sock
# sudo ./ovs-ctl --no-ovsdb-server --db-sock="$DB_SOCK" start
modprobe: FATAL: Module openvswitch not found in directory 
/lib/modules/4.9.0-OCTEONTX_SDK_6_2_0_p2_build_35
* Inserting openvswitch module
#

>From what I understand, the OVS kernel module is not present by default in my 
>custom kernel and doesn't get built either when I compile OVS with the DPDK 
>datapath. According to the documentation, to build the OVS kernel module, the 
>configuration must be done with the flag --with-linux. But, OVS is NOT 
>building at all with this flag enabled. I am getting numerous errors and I 
>think these maybe related the various checks performed by the config file?

# ./configure --with-linux=/lib/modules/`uname -r`/build --prefix=$HOME/usr 
--localstatedir=$HOME/var --sysconfdir=$HOME/etc --disable-ssl
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes

checking whether nf_hookfn.*nf_hook_ops matches in 
/lib/modules/4.9.0-OCTEONTX_SDK_6_2_0_p2_build_35/build/include/linux/netfilter.h...
 no

#make -j32
make[3]: Entering directory '/home/malvika/ovs/datapath/linux'
make -C /lib/modules/4.9.0-OCTEONTX_SDK_6_2_0_p2_build_35/build  
M=/home/malvika/ovs/datapath/linux modules
.
In file included from ./include/net/netfilter/nf_conntrack_tuple.h:13:0,
 from 
/home/malvika/ovs/datapath/linux/compat/include/net/netfilter/nf_conntrack_count.h:19,
 from /home/malvika/ovs/datapath/linux/../compat.h:29,
 from /home/malvika/ovs/datapath/linux/../datapath.h:31,
 from /home/malvika/ovs/datapath/linux/compat/gso.h:5,
 from /home/malvika/ovs/datapath/linux/ip_gre.c:63:
./include/linux/netfilter/x_tables.h:429:64: error: unknown type name 
'nf_hookfn'
struct nf_hook_ops *xt_hook_ops_alloc(const struct xt_table *, nf_hookfn *);

^
./include/net/netfilter/nf_conntrack_tuple.h: In function 
'__nf_ct_tuple_src_equal':
./include/net/netfilter/nf_conntrack_tuple.h:126:10: error: implicit 
declaration of function 'nf_inet_addr_cmp' 
[-Werror=implicit-function-declaration]
  return (nf_inet_addr_cmp(&t1->src.u3, &t2->src.u3) &&
^

Is OVS designed to work on an Octeon-Tx board? Am I missing a step or am I 
doing wrong? I would be grateful if anyone could help me resolve this issue. 
Thank you for your time and consideration.

Best,
Malvika Gupta

IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the content