在 2021-05-07 00:50:06，"Ben Pfaff" <[email protected]> 写道： >On Thu, May 06, 2021 at 09:58:57AM +0800, taoyunupt wrote: >> 在 2021-05-06 03:26:46，"Ben Pfaff" <[email protected]> 写道： >> >> >On Fri, Apr 30, 2021 at 06:10:43PM +0800, taoyunupt wrote: >> >> >> >> >> >> >> >> At 2021-04-29 06:39:11, "Ben Pfaff" <[email protected]> wrote: >> >> >On Wed, Apr 28, 2021 at 08:12:06PM +0800, taoyunupt wrote: >> >> >> Hi， >> >> >> Recently I encountered a TCP connection performance problem, the >> >> >> test tool is Apache benchmark. >> >> >> The OVS in my environment is set for hardware offload solution. >> >> >> The "Requests per second" is about 6000/s， it closed to non-offload >> >> >> solution. >> >> >> >> >> >> >> >> >> "flow-lmit" has a dynamic balance in udpif_revalidator, it will >> >> >> modify by the OVS condition（which is pind to "duration"）. In the >> >> >> revalidate function, when the number of flows is greater than twice >> >> >> the "flow-limit" , the delete flow operation will be triggered to >> >> >> delete all flows; when the number of flows is greater than the >> >> >> "flow-limit", the aging time will be adjusted to 0.1s, Slowly delete >> >> >> flow. >> >> >> >> >> >> >> >> >> >> >> >> I found that the reason for the poor performance is that when the >> >> >> number of flows in the datapath increases and the processing power of >> >> >> OVS decreases, a large number of flow deletions are generated. >> >> >> As we know, In the hardware offloading scenario, although there >> >> >> are a lot of flows, in fact, apart from the first packet, there is no >> >> >> need to process subsequent packets. >> >> >> In my opinion, the dynamic balance mechanism is very necessary, >> >> >> but we need to increase the value of “duration”, or provide some new >> >> >> switches for some high-performance scenarios, such as hardware >> >> >> offloading. >> >> >> Do we still need to restrict the number of flows so strictly? By >> >> >> the way, do you have another solution to resolve this? >> >> > >> >> >It's been a long time since I worked on this, but I recall two reasons >> >> >for the flow limit. First, each flow takes up memory. Second, each >> >> >flow must be revalidated periodically, meaning that it uses CPU as >> >> >well. >> >> > >> >> >I don't, off-hand, remember the real reasons why the logic for deleting >> >> >flows works as it does. It might be in the comments or the commit >> >> >messages. But, I suspect, it is because above the flow-limit we want to >> >> >try to reduce the amount of memory and CPU time dedicated to the cache >> >> >and, if we arrive at twice the flow limit, we conclude that that try >> >> >failed and that we must have a large number of very short flows so that >> >> >caching is not very valuable anyhow. >> >> > >> >> >In a hardware offload scenario, we get rid of some costs (the cost of >> >> >processing and forwarding packets and perhaps the memory cost in the >> >> >datapath) but we still have the cost of revalidating them. When there >> >> >are many flows, we add the extra cost of balancing flows between >> >> >software and the offload hardware. >> >> > >> >> >Because of the remaining cost and the added ones when there is hardware >> >> >offload, it's not obvious to me that we can stop limiting the number of >> >> >flows. I think that experimentation and measurements would be needed. >> >> >Perhaps this would be an adjustment to the dynamic algorithm, rather >> >> >> >> >than a removal of it. >> >> >> >> >> >> I think we can increase the init `flow_limit` in udpif_create,10000 is a >> >> small number for current server and OS, and if 'duration' is small ,we >> >> should increase faster by a lager number not `flow_limit += 1000;`. >> >> I have not better idea for this situation. Do you have some suggestion? I >> >> am very glad to do this change. >> > >> >What kind of number are you thinking about? I'd like to come up with a >> >rationale for choosing it. It might be even better to come up with an >> >> >algorithm or a heuristic for choosing it. >> >> >> I think we could set the initial value to 200,000, and adjust the >> increase to 20,000 each time. Can you describe the rationale >> algorithm you meationed in detailed ? > >I'd expect that whoever is changing it would propose the rationale. > >I believe that part of the current rationale is to keep the limit at a >level such that revalidation takes no more than 1 second. That's an >important aspect too. I agree with you about revalidation needs take no more than 1 second. But, in case of the number of flows is 65535, and the max_revalidator_default is 0.5s, we need about 32s ((65535-1000/1000)*0.5s) to get to the number(65535). If the number get larger, it will cost more time . The difference between offloaded and non-offloaded scenarios is whether OVS needs to process subsequent packages. In the offloaded scenario, even if the number of flows is large, it will not bring continuous pressure to OVS and CPU to process packets. May be we can add a switch before the processing of init and change the `flow-limit`, to suit different situations. What do you think ?
_______________________________________________ discuss mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
