Re: [ovs-discuss] How can I OVS run in multi-core?
On Thu, 31 Aug 2017 16:04:22 +0900 Heung Sik Choi wrote: > Hi, > > I want to measure the performance of OVS in multi core. > > I have a server on two Xeon 2630 v2 (NUMA) with 4 dual port 10GbE NIC. the > server with 12 cores and 8 physical ports run OVS. 4ports are for receiving > 40Gbps traffic and the other 4 ports are for sending 40Gbps traffic. > > I use DPDK pktgen on other machine to make intensive traffic. using this > maching, I sent 12 type flow traffic to OVS server. At this experiment I > expected that OVS run in using 12 cores fully. However, It didn't work . In order to have multiple CPUs processing packets in parallel with the kernel datapath, you need multiple queues with their interrupts spread among all relevant CPUs. And in order to the NIC to spread the traffic to different queues, you need different streams (5-tuple). That is called RSS. For example, stream A would go to queue#1 which is mapped to CPU#1 and stream B would go to queue#2 which is mapped to CPU#2 and so on. That is how Linux kernel works in general (as standalone NIC, or using Linux bridge or OVS). Here is a doc explaining it in more details: https://www.kernel.org/doc/Documentation/networking/scaling.txt -- Flavio ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] How can I OVS run in multi-core?
Hi, I want to measure the performance of OVS in multi core. I have a server on two Xeon 2630 v2 (NUMA) with 4 dual port 10GbE NIC. the server with 12 cores and 8 physical ports run OVS. 4ports are for receiving 40Gbps traffic and the other 4 ports are for sending 40Gbps traffic. I use DPDK pktgen on other machine to make intensive traffic. using this maching, I sent 12 type flow traffic to OVS server. At this experiment I expected that OVS run in using 12 cores fully. However, It didn't work . I can not find any information(such as slide, pdf and so on) that any one do experiment of OVS performance using multi core. Please let me know if you have any insights about how OVS runs on multi-core. ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] How can I OVS run in multi-core?
Do you have 40+gbs of bandwidth on the bus available? Yes. First, I'm sorry to confuse you, because I misrepresented my server specification. I have a server on two Xeon 2630 v2 (NUMA) with* 4 (not 8)* dual port 10GbE NIC. This machine used for measuring linux and DPDK forwarding performance on 40G Line. In addition, 29Gbps was achived on the machine in some experiment. It is weird that when OVS' server receive overall 4 type flows(per port only 1 flow), OVS runs using 4 cores. However, the server receive overall 16 type flows( per port 4 type flows), OVS can not use even four cores. It is very weird. If you have some experience running multi core on OVS, please let me know about how to set or some insights. 2017-08-30 11:50 GMT+09:00 Blue Lang : > Do you have 40+gbs of bandwidth on the bus available? > > On Tue, Aug 29, 2017 at 10:03 PM, Heung Sik Choi > wrote: > >> Hi, >> >> I want to measure the performance of OVS in multi core. >> >> I have a server on two Xeon 2630 v2 (NUMA) with 8 dual port 10GbE NIC. >> the server run OVS. Also there are other two server, which are that one is >> DPDK pktgen TX, the other is DPDK pktgen RX. Pktgen TX send to OVS server >> on 40G line(4* 10G Line), and then OVS send the traffic to the Pktgen RX >> machine.the OVS' rule is as below: >> >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip,n >> w_dst=192.160.110.115,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip,n >> w_dst=192.160.110.116,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip,n >> w_dst=192.160.110.117,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip,n >> w_dst=192.160.110.118,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 >> >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip,n >> w_dst=192.161.110.119,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip,n >> w_dst=192.161.110.120,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip,n >> w_dst=192.161.110.121,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip,n >> w_dst=192.161.110.122,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 >> >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip,n >> w_dst=192.162.110.123,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip,n >> w_dst=192.162.110.124,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip,n >> w_dst=192.162.110.125,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip,n >> w_dst=192.162.110.126,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 >> >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip,n >> w_dst=192.163.110.127,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip,n >> w_dst=192.163.110.128,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip,n >> w_dst=192.163.110.129,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 >> ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip,n >> w_dst=192.163.110.130,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 >> >> >> Each port(1,2,3,4) process 4 different flows, and I thought that it makes >> OVS run using full CPU core. However, it didn't!! >> >> Please let me know if you have any insights about how OVS runs on >> multi-core(also all CPU cores full utilization). >> >> ___ >> discuss mailing list >> disc...@openvswitch.org >> https://mail.openvswitch.org/mailman/listinfo/ovs-discuss >> >> > > > -- > Blue Lang > PM *| *Veracity > > 3423 Piedmont Rd NE > > Suite 350 > > Atlanta, GA 30305 > Cell: (770) 265-1381 <+17702651381> > https://www.linkedin.com/in/bluelang/ > b...@veracity.io > www.veracity.io > ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
Re: [ovs-discuss] How can I OVS run in multi-core?
Do you have 40+gbs of bandwidth on the bus available? On Tue, Aug 29, 2017 at 10:03 PM, Heung Sik Choi wrote: > Hi, > > I want to measure the performance of OVS in multi core. > > I have a server on two Xeon 2630 v2 (NUMA) with 8 dual port 10GbE NIC. the > server run OVS. Also there are other two server, which are that one is DPDK > pktgen TX, the other is DPDK pktgen RX. Pktgen TX send to OVS server on 40G > line(4* 10G Line), and then OVS send the traffic to the Pktgen RX > machine.the OVS' rule is as below: > > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip, > nw_dst=192.160.110.115,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip, > nw_dst=192.160.110.116,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip, > nw_dst=192.160.110.117,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip, > nw_dst=192.160.110.118,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 > > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip, > nw_dst=192.161.110.119,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip, > nw_dst=192.161.110.120,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip, > nw_dst=192.161.110.121,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip, > nw_dst=192.161.110.122,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 > > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip, > nw_dst=192.162.110.123,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip, > nw_dst=192.162.110.124,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip, > nw_dst=192.162.110.125,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip, > nw_dst=192.162.110.126,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 > > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip, > nw_dst=192.163.110.127,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip, > nw_dst=192.163.110.128,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip, > nw_dst=192.163.110.129,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 > ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip, > nw_dst=192.163.110.130,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 > > > Each port(1,2,3,4) process 4 different flows, and I thought that it makes > OVS run using full CPU core. However, it didn't!! > > Please let me know if you have any insights about how OVS runs on > multi-core(also all CPU cores full utilization). > > ___ > discuss mailing list > disc...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-discuss > > -- Blue Lang PM *| *Veracity 3423 Piedmont Rd NE Suite 350 Atlanta, GA 30305 Cell: (770) 265-1381 <+17702651381> https://www.linkedin.com/in/bluelang/ b...@veracity.io www.veracity.io ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
[ovs-discuss] How can I OVS run in multi-core?
Hi, I want to measure the performance of OVS in multi core. I have a server on two Xeon 2630 v2 (NUMA) with 8 dual port 10GbE NIC. the server run OVS. Also there are other two server, which are that one is DPDK pktgen TX, the other is DPDK pktgen RX. Pktgen TX send to OVS server on 40G line(4* 10G Line), and then OVS send the traffic to the Pktgen RX machine.the OVS' rule is as below: ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip,nw_dst=192.160.110.115,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip,nw_dst=192.160.110.116,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip,nw_dst=192.160.110.117,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.160.110.101,ip,nw_dst=192.160.110.118,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:5 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip,nw_dst=192.161.110.119,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip,nw_dst=192.161.110.120,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip,nw_dst=192.161.110.121,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.161.110.114,ip,nw_dst=192.161.110.122,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:6 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip,nw_dst=192.162.110.123,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip,nw_dst=192.162.110.124,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip,nw_dst=192.162.110.125,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.162.110.114,ip,nw_dst=192.162.110.126,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:7 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip,nw_dst=192.163.110.127,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip,nw_dst=192.163.110.128,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip,nw_dst=192.163.110.129,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 ovs-ofctl add-flow ovs-br1 ip,nw_src=192.163.110.114,ip,nw_dst=192.163.110.130,tcp,tp_src=42349,tcp,tp_dst=42349,actions=output:8 Each port(1,2,3,4) process 4 different flows, and I thought that it makes OVS run using full CPU core. However, it didn't!! Please let me know if you have any insights about how OVS runs on multi-core(also all CPU cores full utilization). ___ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
