Hi Team,
I’ve enabled Rx multi-queue, but it looks like only one Rx queue is
actually active, even though the Tx queues are distributing fine. We are
testing with 1k subscribers, and I see that sessions are distributed across
threads. Can you please check the below details and see if I am missing
anything?
FYI
https://github.com/travelping/upg-vpp/issues/416
vpp# show interface rx-placement
Thread 1 (vpp_wk_0):
node dpdk-input:
n4 queue 0 (polling)
n3 queue 0 (polling)
n6 queue 0 (polling)
Thread 2 (vpp_wk_1):
n4 queue 1 (polling)
n3 queue 1 (polling)
n6 queue 1 (polling)
Thread 3 (vpp_wk_2):
n4 queue 2 (polling)
n3 queue 2 (polling)
n6 queue 2 (polling)
Thread 4 (vpp_wk_3):
n4 queue 3 (polling)
n3 queue 3 (polling)
n6 queue 3 (polling)
Thread 5 (vpp_wk_4):
n4 queue 4 (polling)
n3 queue 4 (polling)
n6 queue 4 (polling)
Thread 6 (vpp_wk_5):
n4 queue 5 (polling)
n3 queue 5 (polling)
n6 queue 5 (polling)
Thread 7 (vpp_wk_6):
n4 queue 6 (polling)
n3 queue 6 (polling)
n6 queue 6 (polling)
FYI, UPF sessions distributed across all threads.
#vppctl show upf session limit 10000 > /tmp/t.1
grep "thread=1" /tmp/t.1 | wc -l
126
grep "thread=2" /tmp/t.1 | wc -l
152
grep "thread=3" /tmp/t.1 | wc -l
148
grep "thread=4" /tmp/t.1 | wc -l
144
grep "thread=5" /tmp/t.1 | wc -l
158
grep "thread=6" /tmp/t.1 | wc -l
126
grep "thread=7" /tmp/t.1 | wc -l
146
vpp# show hardware-interfaces
Name Idx Link Hardware
local0 0 down local0
Link speed: unknown
local
n3 2 up n3
Link speed: unknown
RX Queues:
queue thread mode
0 vpp_wk_0 (1) polling
1 vpp_wk_1 (2) polling
2 vpp_wk_2 (3) polling
3 vpp_wk_3 (4) polling
4 vpp_wk_4 (5) polling
5 vpp_wk_5 (6) polling
6 vpp_wk_6 (7) polling
TX Queues:
TX Hash: [name: hash-eth-l34 priority: 50 description: Hash ethernet L34
headers]
queue shared thread(s)
0 no 0
1 no 1
2 no 2
3 no 3
4 no 4
5 no 5
6 no 6
7 no 7
Ethernet address fa:16:3e:ad:64:7a
Red Hat Virtio
carrier up full duplex max-frame-size 9022
flags: admin-up maybe-multiseg int-supported
Devargs:
rx: queues 7 (max 8), desc 256 (min 32 max 32768 align 1)
tx: queues 8 (max 8), desc 256 (min 32 max 32768 align 1)
pci: device 1af4:1000 subsystem 1af4:0001 address 0000:00:05.00 numa 0
max rx packet len: 9728
promiscuous: unicast off all-multicast on
vlan offload: strip off filter off qinq off
rx offload avail: vlan-strip udp-cksum tcp-cksum tcp-lro vlan-filter
scatter
rx offload active: scatter
tx offload avail: vlan-insert multi-segs
tx offload active: multi-segs
rss avail: none
rss active: none
tx burst function: (not available)
rx burst function: (not available)
tx frames ok 15889
tx bytes ok 1365526
rx frames ok 127148
rx bytes ok 11176301
extended stats:
rx_good_packets 127148
tx_good_packets 15889
rx_good_bytes 11176301
tx_good_bytes 1365526
* rx_q2_packets 191 rx_q2_bytes
15096 **rx_q3_packets
126957***
rx_q3_bytes 11161205
tx_q0_packets 1
tx_q0_bytes 42
tx_q3_packets 14
tx_q3_bytes 784
tx_q4_packets 15874
tx_q4_bytes 1364700
rx_q2_good_packets 191
rx_q2_good_bytes 15096
rx_q2_multicast_packets 162
rx_q2_undersize_packets 29
rx_q2_size_65_127_packets 162
rx_q3_good_packets 126957
rx_q3_good_bytes 11161205
rx_q3_multicast_packets 121640
rx_q3_broadcast_packets 5292
rx_q3_undersize_packets 21177
rx_q3_size_65_127_packets 105780
tx_q0_good_packets 1
tx_q0_good_bytes 42
tx_q0_broadcast_packets 1
tx_q0_undersize_packets 1
tx_q3_good_packets 14
tx_q3_good_bytes 784
tx_q3_undersize_packets 14
tx_q4_good_packets 15874
tx_q4_good_bytes 1364700
tx_q4_broadcast_packets 1
tx_q4_undersize_packets 15
tx_q4_size_65_127_packets 15859
cat /etc/vpp/startup.conf
unix {
nodaemon
log /tmp/vpp.log
full-coredump
cli-listen /run/vpp/cli.sock
gid vpp
exec /etc/vpp/init.conf
}
api-trace {
on
}
api-segment {
gid vpp
}
socksvr {
default
}
memory {
main-heap-size 4G
}
cpu {
main-core 0
corelist-workers 1-7
}
buffers {
default data-size 8092
page-size default-hugepage
}
dpdk {
dev 0000:00:05.0 {
name n3
num-rx-queues 7
rss-queues 1-7
}
dev 0000:00:04.0 {
name n4
num-rx-queues 7
}
dev 0000:00:06.0 {
name n6
num-rx-queues 7
rss-queues 1-7
}
uio-driver vfio-pci
no-tx-checksum-offload
}
plugins {
path /usr/lib/x86_64-linux-gnu/vpp_plugins
plugin default { disable }
plugin dpdk_plugin.so { enable }
plugin upf_plugin.so {enable}
plugin nat_plugin.so { enable }
plugin ping_plugin.so { enable }
}
--
Thanks,
Chandra
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#26959): https://lists.fd.io/g/vpp-dev/message/26959
Mute This Topic: https://lists.fd.io/mt/118836466/21656
Group Owner: [email protected]
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/14379924/21656/631435203/xyzzy
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
