Ok, figured it out. I need a larger num-mbufs when RSS is enabled…
Regards,
Yichen
From: <vpp-dev-boun...@lists.fd.io> on behalf of "Yichen Wang (yicwang)"
<yicw...@cisco.com>
Date: Monday, February 13, 2017 at 17:23
To: "John Lo (loj)" <l...@cisco.com>, "vpp-dev@lists.fd.io"
<vpp-dev@lists.fd.io>
Cc: "Ian Wells (iawells)" <iawe...@cisco.com>
Subject: Re: [vpp-dev] Use RSS in VPP 17.01
Hi, John/all,
Thanks for your pointer, and I am able to bring up my VPP with multiple queues!
I am doing a PVP test, which basically I am expecting Traffic Generator -> VPP
on Host -> Loopback VM (testpmd) -> VPP on Host -> Traffic Generator. I can see
the packets are delivered to the Loopback VM with no problem, but:
(1) VPP shows all packets are dropped:
VirtualEthernet0/0/2 8 up tx packets
3237064
tx bytes
194223840
drops 3237051
But I did check testpmd and it got all packets, and does its job correctly by
forwarding the packets to the other interfaces;
(2) VPP show err:
Count Node Reason
692521171 vhost-user-input no available buffer
Why it says “no available buffer”? It works pretty well without RSS,. Did I
miss anything?
Thanks very much for your helps!
Regards,
Yichen
From: "John Lo (loj)" <l...@cisco.com>
Date: Thursday, February 9, 2017 at 20:09
To: "Yichen Wang (yicwang)" <yicw...@cisco.com>, "vpp-dev@lists.fd.io"
<vpp-dev@lists.fd.io>
Cc: "Ian Wells (iawells)" <iawe...@cisco.com>
Subject: RE: Use RSS in VPP 17.01
For VPP, the number of queues on a device can be specified in the DPDK portion
of the startup config, which default to 1. This is usually documented in as
comments in the startup.conf template file when installing VPP rpm/deb on the
target Linux OS. Following is the dpdk portion from the startup.conf in
/etc/vpp/ directory after installing the vpp deb packages on my Ubuntu server:
dpdk {
## Change default settings for all intefaces
# dev default {
## Number of receive queues, enables RSS
## Default is 1
# num-rx-queues 3
## Number of transmit queues, Default is equal
## to number of worker threads or 1 if no workers treads
# num-tx-queues 3
## Number of descriptors in transmit and receive rings
## increasing or reducing number can impact performance
## Default is 1024 for both rx and tx
# num-rx-desc 512
# num-tx-desc 512
## VLAN strip offload mode for interface
## Default is off
# vlan-strip-offload on
# }
## Whitelist specific interface by specifying PCI address
# dev 0000:02:00.0
## Whitelist specific interface by specifying PCI address and in
## addition specify custom parameters for this interface
# dev 0000:02:00.1 {
# num-rx-queues 2
# }
## Change UIO driver used by VPP, Options are: uio_pci_generic, vfio-pci
## and igb_uio (default)
# uio-driver uio_pci_generic
## Disable mutli-segment buffers, improves performance but
## disables Jumbo MTU support
# no-multi-seg
## Increase number of buffers allocated, needed only in scenarios with
## large number of interfaces and worker threads. Value is per CPU
socket.
## Default is 32768
# num-mbufs 128000
## Change hugepages allocation per-socket, needed only if there is need
for
## larger number of mbufs. Default is 256M on each detected CPU socket
# socket-mem 2048,2048
}
Regards,
John
From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On
Behalf Of Yichen Wang (yicwang)
Sent: Thursday, February 09, 2017 10:38 PM
To: vpp-dev@lists.fd.io
Cc: Ian Wells (iawells) <iawe...@cisco.com>
Subject: [vpp-dev] Use RSS in VPP 17.01
Hi, VPP folks,
From what I saw on the VPP docs, there are some places do mention that VPP
supports RSS. Like the example given in the bottom of the link, we do see two
queues per interfaces are shown:
https://wiki.fd.io/view/VPP/Using_VPP_In_A_Multi-thread_Model
I want to try exactly the same thing on both Cisco VIC (enic driver) and Intel
X710 (i40e), but could not get it working straight, and need some helps from
you guys! ☺
On Cisco VIC, I went to CIMC and configure 1 TX Queue and 4 RX Queue, and
enabled all RSS related features. From the RHEL OS, I can clearly see all 5
queues are shown in /proc/interrupts. However, when I bind the interface to
VPP, I can only see “TenGigabitEthernet9/0/0 queue 0” from “show dpdk interface
placement”, but not other queues, am I doing something wrong?
On the Intel X710 side, I did the same type of check of Cisco VIC, and I see 16
TxRx queues were shown, which makes sense because my server has 16 CPU cores.
When I bind the interfaces to VPP, same I could only see 1 queue are shown in
the VPP. How can I configure VPP/DPDK to use RSS? I looked online, but there
were really limited documents from RHEL/Intel/VPP, which doesn’t seem to be
very helpful. Only reference I found which is on OVS-DPDK and they are doing
the RSS configuration from OVS context.
Very looking forward to your helps, and thanks in advance!
Regards,
Yichen
_______________________________________________
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev
