On 01/23/2018 11:42 AM, Kevin Traynor wrote: > On 01/17/2018 07:48 PM, Venkatesan Pradeep wrote: >> Hi, >> >> Assuming that all ports use the same MTU, in OVS2.8 and earlier, a single >> mempool of 256K buffers (MAX_NB_MBUF = 4096 * 64) will be created and shared >> by all the ports >> >> With the OVS2.9 mempool patches, we have port specific allocation and the >> number of mbufs created for each port is based on the following formula >> (with a lower limit of MIN_NB_MBUF = 4096*4) >> n_mbufs = dev->requested_n_rxq * dev->requested_rxq_size >> + dev->requested_n_txq * dev->requested_txq_size >> + MIN(RTE_MAX_LCORE, dev->requested_n_rxq) * NETDEV_MAX_BURST >> + MIN_NB_MBUF; >> >> Using minimal value (1) for n_rxq and n_rxq and default value (2048) for >> requested_rxq_size and requested_txq_size, the above translates to >> n_mbufs = 1*2048 + 1*2048 + 1*32 + 4096*4 = 20512 >> >> Assuming all ports have the same MTU, this means that approximately 13 ports >> in OVS2.9 will consume as much memory as the single mempool shared by all >> ports in OVS2.8 (256*1024 / 20512) . >> >> When a node is upgraded from OVS2.8 to OVS2.9 it is quite possible that the >> memory set aside for OVS may be insufficient. I'm not sure if this aspect >> has been discussed previously and wanted to bring this up for discussion. >> > > Hi Pradeep, I don't think it has been discussed. I guess the thinking > was that with a giant shared mempool, it was over provisioning when > there was a few ports, and in the case where there was a lot of ports > there could be some starvation at run time. It also meant if you had a > mix of different MTU's you had multiple giant shared mempools and could > run out of memory very quickly at config or run time then also. > > So I can see the argument for having a mempool per port, as it is more > fine grained and if you are going to run short of memory, it will at > least be at config time. The problem is if you give some over provision > to each port and you have a lot of ports, you hit the situation you are > seeing. > > I think some amount of over provision per port is needed because you > don't want to be cutting it so fine that you run into memory issues at > run time about local mbuf caches on cores running out, or even if > someone used dpdk rings to send the mbuf somewhere else for a time. > There may be other corner cases too. Perhaps as compromise the min size > could be reduce from 4096*4 to 4096*2 or 4096. > > Thoughts? >
I just sent a compile tested only RFC here https://mail.openvswitch.org/pipermail/ovs-dev/2018-January/343581.html > Kevin. > >> Regards, >> >> Pradeep >> >> >> _______________________________________________ >> dev mailing list >> d...@openvswitch.org >> https://mail.openvswitch.org/mailman/listinfo/ovs-dev >> > > _______________________________________________ > dev mailing list > d...@openvswitch.org > https://mail.openvswitch.org/mailman/listinfo/ovs-dev > _______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
