Hello All Let me ask additional questions regarding performance and counters. I'm seeing intermittent traffic drops in my environment.
Can anyone elaborate regarding what meaning following conuters and what is the relation to each other and to HOLD queue. - ds_flow_queue_limit_exceeded - ds_flow_unusable - flows.exceptions How I can monitor number of packets in Hold queue and how often I reach to itslimit ? For now I can only find this information by using "flow -s" command It seems that one of my problem may be udp services like dns and logging via udp. They cause vrouter to create new flow for each packet. Already I have lowered value for flow age time. Here on specific vrouter where is udp receiver I can see large spikes of values in ds_flow_unusable like 10k or more. However at the same time the counter ds_flow_queue_limit_exceeded reads small values . For this vrouter "flow -s " shows around 80-90k of flow entries, new flows 3k and hold at 120. On other vrouters with mainly tcp services I can see only rather small values in ds_flow_queue_limit_exceeded. But I'm not seeing any values in ds_flow_unusable counter. Does it mean that vrouter may drops packets also in this case? My version of the vrouter is 2.20(build64) Can anyone share knowledge how to tune vrouter parameters to achieve better performance. Regards Piotr Pieprzycki On 20 October 2015 at 11:25, Édouard Thuleau <[email protected]> wrote: > Hi Anand, > > I'm giving you my test results. > The strange behavior I found is when the limit is reached, the flow rate > completely decrease to zero. Do you have any idea why I saw that? > > Regards, > Édouard. > > On Thu, Oct 15, 2015 at 1:58 PM, Édouard Thuleau < > [email protected]> wrote: > >> Hi Anand, >> >> I'm happy to heard your work on that. >> By next release you mean the 2.22 or the 3.0? >> Do you have a bug or blueprint associated to that work? And any design >> documentations on how you will solve that? >> >> Did you measure the same results that mines? Just to be sure I configured >> correctly my vrouters and I don't loose any flow/s on my platform. >> >> Regards, >> Édouard. >> >> On Thu, Oct 15, 2015 at 10:48 AM, Anand H Krishnan <[email protected]> >> wrote: >> >>> Hello Edouard, >>> >>> >>> We are improving the flow setup rate and you should see a vastly >>> improved >>> >>> rate in the next release. >>> >>> >>> Thanks, >>> >>> Anand >>> >>> >>> ------------------------------ >>> *From:* Dev <[email protected]> on behalf of Édouard >>> Thuleau <[email protected]> >>> *Sent:* Thursday, October 15, 2015 1:34 PM >>> *To:* Rajagopalan Sivaramakrishnan >>> *Cc:* [email protected] >>> *Subject:* Re: [opencontrail-dev] Flow unsusable over new 1000req/s >>> >>> Hi Raja, >>> >>> Thanks for that explanation. >>> So with that mechanism, the vrouter is not able to manage more than 1k >>> new flow per seconds? It's not very high, modern HTTP server can handle >>> 500k req/s on a bare metal server [1] (that seems be done on the loopback >>> interface of the web server, not through a real network, no NIC. It's more >>> around 50k from an external node). >>> Did you measure the same result with the vrouter? Do you know a way to >>> improve that? >>> >>> [1] >>> https://lowlatencyweb.wordpress.com/2012/03/20/500000-requestssec-modern-http-servers-are-fast/ >>> >>> <https://lowlatencyweb.wordpress.com/2012/03/20/500000-requestssec-modern-http-servers-are-fast/> >>> 500,000 requests/sec – Modern HTTP servers are fast | The ... >>> A modern HTTP server running on somewhat recent hardware is capable of >>> servicing a huge number of requests with very low latency. Here's a plot >>> showing ... >>> Read more... >>> <https://lowlatencyweb.wordpress.com/2012/03/20/500000-requestssec-modern-http-servers-are-fast/> >>> >>> >>> Édouard. >>> >>> >>> On Thu, Oct 15, 2015 at 7:11 AM, Rajagopalan Sivaramakrishnan < >>> [email protected]> wrote: >>> >>>> Hi Edouard, >>>> When a new flow is created, the first packet of the flow is sent to >>>> the vrouter agent to lookup policy. While the flow is for the agent to >>>> resolve it, it is marked as a “HOLD" flow. >>>> Vrouter limits the number of such flows to 4096. Packets for any new >>>> flows are dropped by vrouter (and the “Flow Unusable” counter is >>>> incremented) when this limit has been crossed. >>>> This should be an ephemeral condition and new flows will be created >>>> once the agent has resolved the older flows. >>>> >>>> Raja >>>> >>>> From: Dev <[email protected]> on behalf of Édouard >>>> Thuleau <[email protected]> >>>> Date: Wednesday, October 14, 2015 at 9:42 AM >>>> To: "[email protected]" <[email protected]> >>>> Subject: [opencontrail-dev] Flow unsusable over new 1000req/s >>>> >>>> Hi, >>>> >>>> I'm doing some network stress tests on Contrail. >>>> One of them is to measure at which rate a vrouter can create new >>>> ephemeral TCP sessions. >>>> >>>> To do that test, I'm using a tsung generator distributed on different >>>> VM (five) and a nginx server that simply return an HTTP 204 code. I setted >>>> a floating IP on the nginx server and tsung clients use it as HTTP >>>> endpoint. I also had to tune some system and nginx parameters. >>>> >>>> On the Contrail side, I just tested it on 1.10 branch for the moment. >>>> The kernel module is loaded with 'vr_flow_entries' setted to 2097152 and >>>> the vrouter agent 'FLOWS.max_vm_flows' to 20%. >>>> >>>> The result is the vrouter starts to drop flow with reason "Flow >>>> Unusable" when the tsung client try to send 1000 HTTP requests per seconds. >>>> >>>> Do you think my results are correct? Do you see any mistake in my test >>>> bed? >>>> Someone already tried that type of test? If yes, what's your results? >>>> >>>> Regards, >>>> Édouard. >>>> >>>> >>> >> > > _______________________________________________ > Dev mailing list > [email protected] > http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org > >
_______________________________________________ Dev mailing list [email protected] http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
