Hello,
I am currently trying to run a UPF application via VPP, I decided to use
DPDK and Kubernetes but the speeds I am getting with iperf is not
realistic, I am getting 0 Mbps when I use the VPP as a gateway, if I remove
the VPP as gateway the throughput speed goes around 4Gbps
iperf3 -c 10.0.7.167 -i 1 -t 10
Connecting to host 10.0.7.167, port 5201
[ 5] local 10.0.4.48 port 40466 connected to 10.0.7.167 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 323 KBytes 2.65 Mbits/sec 7 8.74 KBytes
[ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 1 8.74 KBytes
[ 5] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
[ 5] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec 1 8.74 KBytes
[ 5] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
[ 5] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
[ 5] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec 1 8.74 KBytes
[ 5] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
[ 5] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
[ 5] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 323 KBytes 265 Kbits/sec 10 sender
[ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec receiver
The kubernetes worker node is running on AWS (c5.4xlarge), I used sriov-dp
to add the DPDK interfaces into the pods.
Below is my startup.conf
unix {
nodaemon
log /tmp/vpp.log
full-coredump
gid vpp
interactive
cli-listen /run/vpp/cli.sock
exec /openair-upf/etc/init.conf
}
api-trace {
on
}
dpdk {
uio-driver vfio-pci
dev default {
num-rx-queues 1
num-rx-queues 1
}
dev 0000:00:06.0
dev 0000:00:07.0
dev 0000:00:08.0
}
api-segment {
gid vpp
}
plugins {
path /usr/lib/x86_64-linux-gnu/vpp_plugins/
plugin dpdk_plugin.so { enable }
plugin gtpu_plugin.so { disable }
plugin upf_plugin.so { enable }
}
Init conf:
ip table add 1
ip table add 2
set interface ip table VirtualFunctionEthernet0/6/0 1
set interface mtu 9001 VirtualFunctionEthernet0/6/0
set interface ip address VirtualFunctionEthernet0/6/0 10.0.4.11/24
set interface state VirtualFunctionEthernet0/6/0 up
set interface ip table VirtualFunctionEthernet0/7/0 0
set interface mtu 9001 VirtualFunctionEthernet0/7/0
set interface ip address VirtualFunctionEthernet0/7/0 10.0.6.11/24
set interface state VirtualFunctionEthernet0/7/0 up
set interface ip table VirtualFunctionEthernet0/8/0 2
set interface mtu 9001 VirtualFunctionEthernet0/8/0
set interface ip address VirtualFunctionEthernet0/8/0 10.0.7.11/24
set interface state VirtualFunctionEthernet0/8/0 up
ip route add 0.0.0.0/0 table 2 via 10.0.7.167 VirtualFunctionEthernet0/8/0
trace add dpdk-input 100
First of all I noticed the following errors in the container logs:
interface [error ]: hw_add_del_mac_address:
dpdk_add_del_mac_address: mac address add/del failed: -95
interface [error ]: hw_add_del_mac_address:
dpdk_add_del_mac_address: mac address add/del failed: -95
interface [error ]: hw_add_del_mac_address:
dpdk_add_del_mac_address: mac address add/del failed: -95
interface [error ]: hw_add_del_mac_address:
dpdk_add_del_mac_address: mac address add/del failed: -95
interface [error ]: hw_add_del_mac_address:
dpdk_add_del_mac_address: mac address add/del failed: -95
interface [error ]: hw_add_del_mac_address:
dpdk_add_del_mac_address: mac address add/del failed: -95
format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95
format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95
format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95
format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95
format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95
format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95
format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95
format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95
format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95
I am a bit confused about the errors because I am able to see the correct
MAC address for all the DPDK interfaces in vpp console, is it safe not to
worry about them?
vpp# show hardware-interfaces
Name Idx Link Hardware
VirtualFunctionEthernet0/6/0 1 up VirtualFunctionEthernet0/6/0
Link speed: unknown
*Ethernet address 02:3d:a7:51:90:bc*
AWS ENA VF
carrier up full duplex mtu 9001
flags: admin-up pmd rx-ip4-cksum
VirtualFunctionEthernet0/7/0 2 up VirtualFunctionEthernet0/7/0
Link speed: unknown
* Ethernet address 02:87:8d:0f:e2:20*
AWS ENA VF
VirtualFunctionEthernet0/8/0 3 up VirtualFunctionEthernet0/8/0
Link speed: unknown
* Ethernet address 02:20:55:04:c1:76*
AWS ENA VF
carrier up full duplex mtu 9001
flags: admin-up pmd rx-ip4-cksum
[ec2-user@ip-10-0-0-53 ~]$ kubectl -n oai exec -ti
oai-vpp-upf-57d4fbdcb5-xnslq bin/vppctl show version
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future
version. Use kubectl kubectl exec [POD] -- [COMMAND] instead.
vpp v21.01.1-release built by root on 4e3c1ed23a8e at 2021-11-25T16:47:55
[image: image.png]
Ping tests are working normally on the VPP..
Also will the *Link speed: unknown* have affect it also?
Any idea what could be causing the low iperf speed?
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20557): https://lists.fd.io/g/vpp-dev/message/20557
Mute This Topic: https://lists.fd.io/mt/87388694/21656
Group Owner: [email protected]
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
