On 01/23/2018 11:42 AM, Kevin Traynor wrote:
> On 01/17/2018 07:48 PM, Venkatesan Pradeep wrote:
>> Hi,
>>
>> Assuming that all ports use the same MTU,  in OVS2.8 and earlier, a single 
>> mempool of 256K buffers (MAX_NB_MBUF = 4096 * 64) will be created and shared 
>> by all the ports
>>
>> With the OVS2.9 mempool patches, we have port specific allocation and the 
>> number of mbufs created for each port is based on the following formula 
>> (with a lower limit of MIN_NB_MBUF = 4096*4)
>>        n_mbufs = dev->requested_n_rxq * dev->requested_rxq_size
>>               + dev->requested_n_txq * dev->requested_txq_size
>>               + MIN(RTE_MAX_LCORE, dev->requested_n_rxq) * NETDEV_MAX_BURST
>>               + MIN_NB_MBUF;
>>
>> Using minimal value (1) for n_rxq and n_rxq and default value (2048) for 
>> requested_rxq_size and requested_txq_size, the above translates to
>>       n_mbufs = 1*2048 + 1*2048 + 1*32 + 4096*4  = 20512
>>
>> Assuming all ports have the same MTU, this means that approximately 13 ports 
>> in OVS2.9 will consume as much memory as the single mempool shared by all 
>> ports in OVS2.8 (256*1024 / 20512) .
>>
>> When a node is upgraded from OVS2.8 to OVS2.9 it is quite possible that the 
>> memory set aside for OVS may be insufficient. I'm not sure if this aspect 
>> has been discussed previously and wanted to bring this up for discussion.
>>
> 
> Hi Pradeep, I don't think it has been discussed. I guess the thinking
> was that with a giant shared mempool, it was over provisioning when
> there was a few ports, and in the case where there was a lot of ports
> there could be some starvation at run time. It also meant if you had a
> mix of different MTU's you had multiple giant shared mempools and could
> run out of memory very quickly at config or run time then also.
> 
> So I can see the argument for having a mempool per port, as it is more
> fine grained and if you are going to run short of memory, it will at
> least be at config time. The problem is if you give some over provision
> to each port and you have a lot of ports, you hit the situation you are
> seeing.
> 
> I think some amount of over provision per port is needed because you
> don't want to be cutting it so fine that you run into memory issues at
> run time about local mbuf caches on cores running out, or even if
> someone used dpdk rings to send the mbuf somewhere else for a time.
> There may be other corner cases too. Perhaps as compromise the min size
> could be reduce from 4096*4 to 4096*2 or 4096.
> 
> Thoughts?
> 

I just sent a compile tested only RFC here
https://mail.openvswitch.org/pipermail/ovs-dev/2018-January/343581.html

> Kevin.
> 
>> Regards,
>>
>> Pradeep
>>
>>
>> _______________________________________________
>> dev mailing list
>> d...@openvswitch.org
>> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>>
> 
> _______________________________________________
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> 

_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss

Reply via email to