From: <vpp-dev@lists.fd.io> on behalf of "ravinder.ya...@hughes.com"
<ravinder.ya...@hughes.com>
Date: Tuesday, February 25, 2020 at 7:27 AM
To: "vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io>
Subject: [vpp-dev] vmxnet3 rx-queue error "vmxnet3 failed to activate dev error
1" #vmxnet3 #ipsec
[Edited Message Follows]
VPP IPsec responder on ESXI VM RHEL 7.6
Is there a limitation on the number of num-tx-queues and num-rx-queues we can
associate with vmxnet3? I have 12 worker threads running but when i try to
associate more than 4 num-rx-queues i get error saying "vmxnet3 failed to
activate dev error 1" ....
Setup Details:
I have vpp running on 16 vCPUs and set the worker thread to use vCPU (2-15) i.e
14 worker threads.
Error:
I run into vmxnet3 error when i try to create vmxnet interface with:
* num-tx-queue set to greater than 8 (Error #1 below)
<SVL> The limit is 8 for TX. </SVL>
* num-rx-queues set to greater than 4 (Error #2 below)
<SVL>
The above statement is not entirely true. I am able to set num-rx-queues
greater than 4 in two different cases as below.
DBGvpp# create interface vmxnet3 0000:13:00.0 rx-queue-size 2048 tx-queue-size
2048 num-tx-queues 8 num-rx-queues 8
create interface vmxnet3 0000:13:00.0 rx-queue-size 2048 tx-queue-size 2048
num-tx-queues 8 num-rx-queues 8
DBGvpp#
DBGvpp# create interface vmxnet3 0000:13:00.0 num-tx-queues 8 num-rx-queues 8
create interface vmxnet3 0000:13:00.0 num-tx-queues 8 num-rx-queues 8
DBGvpp#
However, some odd combinations like (tx=4, rx=5) and (tx=4, rx=6) are unwelcome
by the ESXi driver. I don’t know why yet.
</SVL>
ERROR #1: vCPUs = 16 and Worker Threads = 14
vpp# create interface vmxnet3 0000:13:00.0 rx-queue-size 2048 tx-queue-size
2048 num-tx-queues 12 num-rx-queues 4 bind
ERROR: create interface vmxnet3: number of tx queues must be <= 8 and <= number
of CPU's assigned to VPP
Works fine when num-tx-queues 8
<SVL> Isn’t the error obvious to you? The limit for tx queues is 8.
Steven
</SVL>
ERROR #2:
vpp# create interface vmxnet3 0000:13:00.0 rx-queue-size 2048 tx-queue-size
2048 num-tx-queues 8 num-rx-queues 5 bind
ERROR: create interface vmxnet3: error on activating device rc (1)
Works fine when num-rx-queues 4
vpp# create interface vmxnet3 0000:13:00.0 rx-queue-size 2048 tx-queue-size
2048 num-tx-queues 8 num-rx-queues 4 bind
vpp# sh int rx-placement
Thread 1 (vpp_wk_0):
node vmxnet3-input:
vmxnet3-0/13/0/0 queue 0 (polling)
Thread 2 (vpp_wk_1):
node vmxnet3-input:
vmxnet3-0/13/0/0 queue 1 (polling)
Thread 3 (vpp_wk_2):
node vmxnet3-input:
vmxnet3-0/13/0/0 queue 2 (polling)
Thread 4 (vpp_wk_3):
node vmxnet3-input:
vmxnet3-0/13/0/0 queue 3 (polling)
Reference:
https://vpp.flirble.org/master/db/df1/clicmd_src_plugins_vmxnet3.html
create interface vmxnet3 <pci-address> [rx-queue-size <size>] [tx-queue-size
<size>] [num-tx-queues <number>] [num-rx-queues <number>] [bind] [gso].
Thank you,
Ravin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#15525): https://lists.fd.io/g/vpp-dev/message/15525
Mute This Topic: https://lists.fd.io/mt/71527703/21656
Mute #vmxnet3: https://lists.fd.io/mk?hashtag=vmxnet3&subid=1480452
Mute #ipsec: https://lists.fd.io/mk?hashtag=ipsec&subid=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
