Hi, all, I have one followup question about performance of ovs. I am running my experiment (12 switches and 12 hosts in Mininet) with a 32 core machine. I track the CPU/MEM usage of all threads of ovs-vswitchd (handler/revalidator), all of them are below 60%. However, I find that there are packets lost in the ovs switches. I am wondering whether that's because the buffer size of ports is not big enough (How can I configure the buffer size)? Or the queue size between kernel and userspace, as I configure to send all packets from kernel space to userspace (according to Ben's advice).
Thanks, Xuemei On Sat, Jul 25, 2015 at 6:15 PM, Xuemei Liu <[email protected]> wrote: > I found that problem in the VM of my pc. When I run the experiments on one > physical server, the CPU usage is not that low. > > On Fri, Jul 24, 2015 at 1:24 PM, Ben Pfaff <[email protected]> wrote: > >> It doesn't sound like you have a bottleneck, if CPU is less than 10%, so >> I don't understand the question. >> >> On Fri, Jul 24, 2015 at 09:42:05AM -0700, Xuemei Liu wrote: >> > Hi, Ben, >> > >> > I have one follow-up question. I setup a topology with 12 hosts and 12 >> > switches in Mininet. The hosts (acting as the packets generator to the >> > network) should send packets around 30 pps in order to make the switches >> > not drop packets. However, I find that the CPU/MEM usage of >> ovs-vswitchd is >> > not high (both < 10%). According to your experience, where do you think >> is >> > the bottleneck? How can I improve the performance of packet processing >> > speed in userspace in Mininet? >> > >> > Thanks, >> > Xuemei >> > >> > On Fri, Jul 24, 2015 at 9:30 AM, Xuemei Liu <[email protected]> >> wrote: >> > >> > > Thanks very much Ben. >> > > >> > > On Fri, Jul 24, 2015 at 9:25 AM, Ben Pfaff <[email protected]> wrote: >> > > >> > >> Any SLOW_* constant indicates why a packet can't be processed in the >> > >> fast path (e.g. in the kernel). Such packets always have to be >> handled >> > >> in userspace. Thus, tagging all flow translations with any SLOW_* >> > >> constant causes them all to be sent to userspace. >> > >> >> > >> On Thu, Jul 23, 2015 at 05:07:58PM -0700, Xuemei Liu wrote: >> > >> > Hi, Ben, >> > >> > >> > >> > After I decreased the sending rate at h1, I find s1 userspace/h2 >> can >> > >> > receive all packets. What does SLOW_ACTION mean? Why this will >> make the >> > >> > packets forwarded from kernel space to user space? >> > >> > >> > >> > Thanks, >> > >> > Xuemei >> > >> > >> > >> > On Thu, Jul 23, 2015 at 4:46 PM, Ben Pfaff <[email protected]> wrote: >> > >> > >> > >> > > Well, yes, there are performance problems, as I predicted. >> > >> > > >> > >> > > On Thu, Jul 23, 2015 at 04:42:17PM -0700, Xuemei Liu wrote: >> > >> > > > Let me try to describe it. Suppose the topology is h1-s1-h2, >> where >> > >> h1, h2 >> > >> > > > are hosts, and s1 is the ovs switch. I add policy in s1 to >> forward >> > >> > > packets >> > >> > > > with dstip of h2 to h2. Then I test in two scenarios. >> > >> > > > 1. h1 sends 9 packets (3 packet for each of 3 different flows) >> to >> > >> h2. >> > >> > > With >> > >> > > > your methods, userspace in s1 can accept all the packets, and >> h2 can >> > >> > > > receive all the 9 packets. Work perfect. >> > >> > > > 2. h1 sends 10000 packets to h2. In this case, h2 receives the >> same >> > >> > > packets >> > >> > > > as userspace in s1 does. However, the number of packets >> received is >> > >> much >> > >> > > > less than 10000. Which means many packets are lost in s1. >> > >> > > > Is this clear now? >> > >> > > > >> > >> > > > Thanks, >> > >> > > > Xuemei >> > >> > > > >> > >> > > > On Thu, Jul 23, 2015 at 4:18 PM, Ben Pfaff <[email protected]> >> wrote: >> > >> > > > >> > >> > > > > That's too vague for me to guess. What packets are getting >> lost? >> > >> > > > > >> > >> > > > > On Thu, Jul 23, 2015 at 04:10:14PM -0700, Xuemei Liu wrote: >> > >> > > > > > Hi, Ben, >> > >> > > > > > >> > >> > > > > > I tried your method, but the switch seems to drop some >> packets, >> > >> as it >> > >> > > > > does >> > >> > > > > > not output the expected packets that I forward to send to >> it. >> > >> Any >> > >> > > advice? >> > >> > > > > > >> > >> > > > > > Thanks, >> > >> > > > > > Xuemei >> > >> > > > > > >> > >> > > > > > On Thu, Jul 23, 2015 at 10:19 AM, Xuemei Liu < >> > >> [email protected]> >> > >> > > > > wrote: >> > >> > > > > > >> > >> > > > > > > Hi, Ben, >> > >> > > > > > > >> > >> > > > > > > Thanks for your response. >> > >> > > > > > > "You realize that this will be terrible for performance, >> > >> right?" >> > >> > > > > > > In fact, I have not got all packets sent to user space. >> That >> > >> is the >> > >> > > > > > > problem I am facing now. I think performance might be >> another >> > >> > > problem >> > >> > > > > after >> > >> > > > > > > I can receive all packet in user space. >> > >> > > > > > > >> > >> > > > > > > Thanks, >> > >> > > > > > > Xuemei >> > >> > > > > > > >> > >> > > > > > > On Thu, Jul 23, 2015 at 10:13 AM, Ben Pfaff < >> [email protected]> >> > >> >> > >> > > wrote: >> > >> > > > > > > >> > >> > > > > > >> On Thu, Jul 23, 2015 at 10:04:23AM -0700, Xuemei Liu >> wrote: >> > >> > > > > > >> > I am new to ovs, and I am trying to send all packets >> from >> > >> kernel >> > >> > > > > space >> > >> > > > > > >> to >> > >> > > > > > >> > user space. I comment the "unlikely(!flow)" in >> > >> > > datapath/datapath.c. >> > >> > > > > > >> > However, it seems I just receive the first packet of >> each >> > >> flow >> > >> > > (the >> > >> > > > > > >> first >> > >> > > > > > >> > packet that match one forwarding rule in the bridge) >> in >> > >> user >> > >> > > space. >> > >> > > > > > >> Could >> > >> > > > > > >> > anyone tell me why? and is there other way to achieve >> my >> > >> goal? >> > >> > > > > > >> >> > >> > > > > > >> You realize that this will be terrible for performance, >> > >> right? >> > >> > > > > > >> >> > >> > > > > > >> It's kind of a waste to modify the kernel module for >> this. >> > >> I'd >> > >> > > just >> > >> > > > > > >> modify userspace to send all packets to userspace, >> something >> > >> like >> > >> > > > > this: >> > >> > > > > > >> >> > >> > > > > > >> diff --git a/ofproto/ofproto-dpif-xlate.c >> > >> > > > > b/ofproto/ofproto-dpif-xlate.c >> > >> > > > > > >> index 52395a7..a98406a 100644 >> > >> > > > > > >> --- a/ofproto/ofproto-dpif-xlate.c >> > >> > > > > > >> +++ b/ofproto/ofproto-dpif-xlate.c >> > >> > > > > > >> @@ -4777,7 +4777,7 @@ xlate_actions(struct xlate_in >> *xin, >> > >> struct >> > >> > > > > > >> xlate_out *xout) >> > >> > > > > > >> >> > >> > > > > > >> ctx.xin = xin; >> > >> > > > > > >> ctx.xout = xout; >> > >> > > > > > >> - ctx.xout->slow = 0; >> > >> > > > > > >> + ctx.xout->slow = SLOW_ACTION; >> > >> > > > > > >> ctx.xout->has_learn = false; >> > >> > > > > > >> ctx.xout->has_normal = false; >> > >> > > > > > >> ctx.xout->has_fin_timeout = false; >> > >> > > > > > >> >> > >> > > > > > > >> > >> > > > > > > >> > >> > > > > >> > >> > > >> > >> >> > > >> > > >> > >
_______________________________________________ discuss mailing list [email protected] http://openvswitch.org/mailman/listinfo/discuss
