Hi , If do not enable dpdk, plian ovs can pass about 120,000 pps but once I make ovs to use dpdk the main reason in doing this is to get more performance I am seeing that it is passing only 1000 pps. So I guess the drop from 120,000 to just 1000 means I am missing something. Regards, Ravi
-----Original Message----- From: Traynor, Kevin [mailto:kevin.tray...@intel.com] Sent: Friday, May 15, 2015 12:02 PM To: Rao, Ravi; therbert at redhat.com; dev at dpdk.org Subject: RE: [dpdk-dev] performance issue with ovs + dpdk2.0 with vhost > -----Original Message----- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Ravi Rao > Sent: Friday, May 15, 2015 5:12 PM > To: therbert at redhat.com; dev at dpdk.org > Subject: Re: [dpdk-dev] performance issue with ovs + dpdk2.0 with > vhost > > Hi, > This is what I am trying to Do. > Below is the setup.. > > | +----------------------+ | > | guest | | > | | | > | | | guest > | eth0 L3fwd eth1 | | > | | | | | > +---+--------------+---+ __| > ^ : > | | > : v __ > +-----------------+--------------+-----------------+ | > | | ovs-br0 | | > | +--------------+ | | > | ^ : | | > | +----------+ +---------+ | | host > | : v | | > | +--------------+ +--------------+ | | > | | dpdk0 | ovs-dpdk | dpdk1 | | | > +---+--------------+------------+--------------+---+ __| > ^ : > | | > : v > +--------------------------------------------------+ > | | > | traffic generator | > | | > +--------------------------------------------------+| > > > Step1: Use the latest ovs and dpdk2.0 to get the ovs running with 2 > dpdk interfaces that are bound to 2 10GB physical interfaces > #** Inser the required Modules > cd /root/dpdk-2.0.0 > modprobe uio > modprobe cuse > rmmod igb_uio > rmmod rte_kni > insmod x86_64-native-linuxapp-gcc/kmod/igb_uio.ko > > #**** Assign the dpdk capable interfaces to igb_uio driver > tools/dpdk_nic_bind.py --status tools/dpdk_nic_bind.py -b igb_uio > 0000:02:00.0 tools/dpdk_nic_bind.py -b igb_uio 0000:02:00.1 > tools/dpdk_nic_bind.py --status > > #--- Setup the openVswitch > cd /root/ovs > pkill -9 ovs > mkdir -p /usr/local/etc/openvswitch > mkdir -p /usr/local/var/run/openvswitch rm -rf > /usr/local/etc/openvswitch/conf.db > ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db > vswitchd/vswitch.ovsschema > > #Start ovsdb-server > ovsdb/ovsdb-server > --remote=punix:/usr/local/var/run/openvswitch/db.sock > --remote=db:Open_vSwitch,Open_vSwitch,manager_options --pidfile > --detach utilities/ovs-vsctl --no-wait init > > #Start vswitchd: > export DB_SOCK=/usr/local/var/run/openvswitch/db.sock > rm /dev/vhost-net > vswitchd/ovs-vswitchd --dpdk -c 0x3 -n 4 --socket-mem 1024,0 -- > unix:$DB_SOCK --pidfile --detach > > #Add bridge & ports > utilities/ovs-vsctl add-br ovs-br0 -- set bridge ovs-br0 > datapath_type=netdev utilities/ovs-vsctl add-port ovs-br0 dpdk0 -- set > Interface dpdk0 type=dpdk utilities/ovs-vsctl add-port ovs-br0 dpdk1 > -- set Interface dpdk1 type=dpdk > > Step2: Create the dpdkvhost interfaces and bring up the guestVM using > QEMU export DPDK_DIR=/root/dpdk-2.0.0 insmod > $DPDK_DIR/lib/librte_vhost/eventfd_link/eventfd_link.ko > cd /root/ovs > utilities/ovs-vsctl add-port ovs-br0 dpdkvhost0 -- set Interface > dpdkvhost0 type=dpdkvhost > utilities/ovs-vsctl add-port ovs-br0 dpdkvhost1 -- set Interface > dpdkvhost1 type=dpdkvhost > > #**** Start the guest ubuntu VM1 from a terminal that is logged in as > root > qemu-system-x86_64 --enable-kvm -k fr -m 1G \ > -cpu host -smp cores=2,threads=1,sockets=1 \ > -serial telnet::4444,server,nowait -monitor > telnet::5555,server,nowait \ > -hda /root/VMs/images/ubuntu-14.04-template.qcow2 \ > -object > memory-backend-file,id=mem,size=1G,mem-path=/mnt/huge_1GB,share=on \ > -numa node,memdev=mem \ > -netdev > type=tap,id=dpdkvhost0,script=no,downscript=no,ifname=dpdkvhost0,vhost=on \ > -device > virtio-net- > pci,netdev=dpdkvhost0,mac=52:54:00:12:34:56,csum=off,gso=off,guest_tso > 4=off,g > uest_tso6=off,guest_ecn=off > \ > -netdev > type=tap,id=dpdkvhost1,script=no,downscript=no,ifname=dpdkvhost1,vhost=on \ > -device > virtio-net- > pci,netdev=dpdkvhost1,mac=52:54:00:12:34:57,csum=off,gso=off,guest_tso > 4=off,g > uest_tso6=off,guest_ecn=off > \ > -device ne2k_pci,mac=DE:AD:DE:01:02:03,netdev=user.0 -netdev > user,id=user.0,hostfwd=tcp::2222-:22 & > > # **** Add flows between ports > utilities/ovs-ofctl del-flows ovs-br0 > utilities/ovs-ofctl add-flow ovs-br0 in_port=1,action=output:3 > utilities/ovs-ofctl add-flow ovs-br0 in_port=2,action=output:4 > utilities/ovs-ofctl add-flow ovs-br0 in_port=3,action=output:1 > utilities/ovs-ofctl add-flow ovs-br0 in_port=4,action=output:2 > > Once I complete the above settings. I log into the VM and enable ipv4 > forwarding so that it could do l3 fwd between eth0 and eth1 > > The issue I am seeing is when I start pumping packets on IXIA port > connected to physical port dpdk0 I see lots of tx_errors on dpdk0. > I can only pass about 1000 pps without getting any errors. Is there > anything I am doing wrong or missing in the above setup. >From what you are saying, I take it that it is functionally working but the >rates are low. In that case I would check/set core affinities with top -H and >taskset to see if there's contention occurring. You could also test with >multiple PMDs (hence cores) in OVS to spread the switching workload - if you >wanted to test with 4 PMDs you could use ovs-vsctl --timeout 10 set >Open_vSwitch . other_config:pmd-cpu-mask=3c > > Thanks & Regards, > Ravi.. > > On 05/15/15 09:41, Thomas F Herbert wrote: > > On 5/15/15 9:55 AM, Rao, Ravi wrote: > >> Hi All, > >> I am trying to get a Guest VM connected to the dpdkvhost > >> interface on a host which has the ovs running from the latest ovs > >> git and dpdk2.0. Looks like I am missing something as 95% of > >> traffic is not getting to the VM. Can one of you please let me know > >> which mailing list I should be posting the details for getting a > >> resolution. Is it this dpdk list OR would it be the openvswitch list? > > You could also try the openvswitch discuss list but this list is a > > good starting point. I am working with vhost/dpdk2.0 and ovs and > > others are too. > > Could you please send more details about what you are seeing? Are > > you using the Linux stack in the guest? Do you see packets on the > > input IF in the guest? > >> Regards, > >> Ravi.. > > > >