[ovs-discuss] ovs-dpdk performance not stable
Hi Everyone,
I would greatly appreciate any input.
The setting that i am working with is a host with ovs-dpdk connected to a
VM.
What i see when i do performance test is that after about a minute or two
suddenly i have many drops as if the cache was full and was dumped
improperly.
I tried to play with the settings of the n-rxq and n_txq values, which
helps but only probably until the cache is filled and then i have drops.
The things is that sometimes, rarely, as if by chance the performance
continues.
My settings is as follows:
OVS Version. 2.6.1
DPDK Version. 16.07.2
NIC Model. Ethernet controller: Intel Corporation Ethernet Connection I354
(rev 03)
pmd-cpu-mask. on core 1 mask=0x2
lcore mask. core zeor "dpdk-lcore-mask=1"
Port "dpdk0"
Interface "dpdk0"
type: dpdk
options: {n_rxq="8", n_rxq_desc="2048", n_txq="9",
n_txq_desc="2048"}
ovs-appctl dpif-netdev/pmd-rxq-show
pmd thread numa_id 0 core_id 1:
isolated : false
port: dpdk0 queue-id: 0 1 2 3 4 5 6 7
port: dpdk1 queue-id: 0 1 2 3 4 5 6 7
Thanks,
Michael
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] OVN vm on vlan network using geneve tunnel for external traffic
I played adding some flows for forwarding as vlan packets and it worked(i.e compute and gateway communicated through vlan packets on both sides) I created a vm on vlan network 172.16.1.1/24 with vlan id 20 and an external network 10.0.0.0/24 to reach internet. In the current code, when I tired to reach external network(8.8.8.8) from this vm, below physical flows will be run for routing and redirecting the packet through geneve tunnel. table=15 ip,metadata=0x2 actions=dec_ttl(),load:0xa01->NXM_NX_XXREG0[96..127],load:0xa65->NXM_NX_XXREG0[64..95],set_field:fa:16:3e:49:bb:60->eth_src,set_field:0x3->reg15,load:0x1->NXM_NX_REG10[0],resubmit(,16) //routing table=17 reg15=0x3,metadata=0x2 actions=set_field:0x4->reg15,resubmit(,18) //redirect table=32, priority=100,reg15=0x4,metadata=0x2 actions=load:0x2->NXM_NX_TUN_ID[0..23],set_field:0x4->tun_metadata0,move:NXM_NX_REG14[0..14]->NXM_NX_TUN_METADATA0[16..30],bundle(eth_src,0,active_backup,ofport,slaves:3,4) When I manually added below flows (in same tables) with higher priority for vlans, packet is forwarded with vlan tag instead of tunneling. sudo ovs-ofctl add-flow br-int "table=15,ip,priority=49,metadata=0x2,dl_dst=fa:16:3e:8c:02:5d,actions=dec_ttl(),load:0xac100201->NXM_NX_REG0[],load:0x3->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0],resubmit(,16)" sudo ovs-ofctl add-flow br-int "table=16,priority=100,reg0=0xac100201,reg15=0x3,metadata=0x2,action=mod_dl_dst:fa:16:3e:8c:02:5d,resubmit(,17)" sudo ovs-ofctl add-flow br-int "table=32,priority=200,reg0=0xac100201,reg15=0x4,metadata=0x2,action=mod_vlan_vid:20,output:9,strip_vlan" Note: Above table 15 flow is added by ovn-controller for each vlan network(fa:16:3e:8c:02:5d is 172.16.1.1 MAC) I will work on ovn-controller changes to have these flows. Thanks Anil On Tue, Apr 17, 2018 at 2:20 AM, Ben Pfaff wrote: > On Fri, Apr 13, 2018 at 09:22:44PM -0400, Russell Bryant wrote: > > On Fri, Apr 13, 2018 at 9:01 PM, Russell Bryant wrote: > > > On Fri, Apr 13, 2018 at 5:27 PM, Ben Pfaff wrote: > > >> On Wed, Apr 11, 2018 at 07:44:25PM +0530, Anil Venkata wrote: > > >>> vm created on a vlan tenant network is using geneve tunnel(between > compute > > >>> and gateway nodes) to reach external network. Because of this, we > need to > > >>> consider tunnelling overhead while assigning MTU for vlan network. > Can we > > >>> improve OVN to avoid tunnelling in this case. > > >> > > >> When OVN tunnels packets, the tunnel metadata includes information on > > >> the logical network, logical input port, and logical output port. The > > >> logical input port is only used for egress ACLs, so it could be > omitted > > >> if egress ACLs are constrained not to match on the logical input port. > > >> The logical network and logical output port are still needed, though, > so > > >> to encode that in a VLAN they would have to add up to 12 bits or less. > > >> That's pretty constraining. Do you have some idea for how to do it? > > > > > > I don't think ACLs are a factor here because it's actually the logical > > > router pipeline forwarded the packet over a tunnel. The only logical > > > switches involved are VLAN networks (a switch with a localnet port). > > > > > > The unexpected behavior here is that despite using all VLAN networks, > > > a Geneve tunnel is used when the packet is sent to the L3 gateway node > > > that's doing SNAT. Note that the type of router configured here is > > > the hybrid-type, where routing is fully distributed in all cases > > > except when NAT is required, then it gets redirected to a central > > > point. That redirect is what we're seeing here. > > > > > > I've thought of two ways out of this: > > > > > > 1) In this scenario, if you really don't want any tunneling in use, > > > configure a fully centralized router instead. The downside is that > > > East-West routing will be centralized, as well. > > > > > > 2) Use two routers. ls1 (VLAN) with all ports for VMs <-> distributed > > > east-west router <-> ls2 (VLAN) used just to interconnect the routers > > > <-> centralized router for SNAT. The downside here is that you still > > > lose the ability to bind floating IPs directly to compute nodes like > > > today. > > > > > > 3) Figure out a way for OVN to do this redirect to the gateway host > > > over a VLAN network. I suspect this isn't trivial and honestly > > > haven't spent the time to figure out what it would take, but this does > > > seem like the ideal behavior. > > > > > > -- > > > Russell Bryant > > > > After an internal conversation on this topic, I wrote the following > > doc to summarize what was observed and to capture ideas for next > > steps: > > > > https://docs.google.com/document/d/1JecGIXPH0RAqfGvD0nmtBdEU1zflH > ACp8WSRnKCFSgg/edit?usp=sharing > > I wonder whether this should be added to the OVN FAQ I starting writing > earlier today: > https://patchwork.ozlabs.org/patch/898868/ > ___ discuss mailing list
Re: [ovs-discuss] ovs-dpdk performance not stable
Hi Michael,
Are you using dpdk vhostuser ports in this deployment?
I would expect to see them listed in the output of ovs-appctl
dpif-netdev/pmd-rxq-show you posted below.
Can you describe the expected traffic flow ( Is it North/South using DPDK phy
devices as well as vhost devices or east/west between vm interfaces only).
OVS 2.6 has the ability to isolate and pin rxq queues for dpdk devices to
specific PMDs also. This can help provide more stable throughput and defined
behavior. Without doing this I believe the distribution of rxqs was dealt with
in a round robin manner which could change between deployments. This could
explain what you are seeing i.e. sometimes the traffic runs without drops.
You could try to examine ovs-appctl dpif-netdev/pmd-rxq-show when traffic is
dropping and then again when traffic is passing without issue. This output
along with the flows in each case might provide a clue as to what is happening.
If there is a difference between the two you could investigate pinning the rxqs
to the specific setup although you will only benefit from this when have at
least 2 PMDs instead of 1.
Also OVS 2.6 and DPDK 16.07 aren’t the latest releases of OVS & DPDK, have you
tried the same tests using the latest OVS 2.9 and DPDK 17.11?
Ian
From: ovs-discuss-boun...@openvswitch.org
[mailto:ovs-discuss-boun...@openvswitch.org] On Behalf Of michael me
Sent: Tuesday, April 17, 2018 10:42 AM
To: ovs-discuss@openvswitch.org
Subject: [ovs-discuss] ovs-dpdk performance not stable
Hi Everyone,
I would greatly appreciate any input.
The setting that i am working with is a host with ovs-dpdk connected to a VM.
What i see when i do performance test is that after about a minute or two
suddenly i have many drops as if the cache was full and was dumped improperly.
I tried to play with the settings of the n-rxq and n_txq values, which helps
but only probably until the cache is filled and then i have drops.
The things is that sometimes, rarely, as if by chance the performance continues.
My settings is as follows:
OVS Version. 2.6.1
DPDK Version. 16.07.2
NIC Model. Ethernet controller: Intel Corporation Ethernet Connection I354 (rev
03)
pmd-cpu-mask. on core 1 mask=0x2
lcore mask. core zeor "dpdk-lcore-mask=1"
Port "dpdk0"
Interface "dpdk0"
type: dpdk
options: {n_rxq="8", n_rxq_desc="2048", n_txq="9",
n_txq_desc="2048"}
ovs-appctl dpif-netdev/pmd-rxq-show
pmd thread numa_id 0 core_id 1:
isolated : false
port: dpdk0 queue-id: 0 1 2 3 4 5 6 7
port: dpdk1 queue-id: 0 1 2 3 4 5 6 7
Thanks,
Michael
___
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] Fwd: Regarding Matching Flows in EMC Cache
Hello people, I apologise for my stupid questions. I am learning to my best. Let me re frame my question in a better way. I want to know about the granularity of flow matching in the EMC. Here I have observed and read some facts about the matching flow. Can you please help me correct if I am wrong. *1. The below packet arrived, from these header field's miniflow will be extracted, right?* * |dpif_netdev(pmd8)|DBG|packet: tcp,vlan_tci=0x,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,nw_src=10.0.0.3,nw_dst=10.0.0.4,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=35963, tp_dst=45629,tcp_flags=ack tcp_csum:f470* *2. I am confused a bit on this point, miniflow_extract() will output a map by setting 1s for the fields extracted. Fields extracted in this case will be which fields? Are those dl_src,dl_dst, nw_src, nw_dst, nw_ttl, tp_src, tp_dst,tcp_flags tcp_csum , omitting nw_tos and nw_ecn? If yes, EMC Cache has an entry which is referencing to the flows containing all these fields?* Please help me with this, I want to confirm and clear this before I share my knowledge with anyone else as I don't wanna provide my wrong input to anyone and this mailing list is the only hope. Thank you so much. -- Forwarded message -- From: Krish Date: Mon, Apr 16, 2018 at 1:05 PM Subject: Regarding Matching Flows in EMC Cache To: ovs-discuss@openvswitch.org Hi Everyone >From Debug information of EMC Processing , I can see the packets have IPs as well as Ports. Packet information is below : *|dpif_netdev(pmd8)|DBG|packet: tcp,vlan_tci=0x,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:02,nw_src=10.0.0.3,nw_dst=10.0.0.4,nw_tos=0,nw_ecn=0,nw_ttl=64,tp_src=35963,tp_dst=45629,tcp_flags=ack tcp_csum:f470* >From the above packet, are all the fields that are displayed will be used to search in EMC Cache including checksum? Please correct my knowledge. Regards and Thanks ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
