David Marchand <david.march...@redhat.com> writes: > On Thu, Jul 1, 2021 at 7:46 PM Ilya Maximets <i.maxim...@ovn.org> wrote: >> >> On 6/30/21 10:48 PM, Rosemarie O'Riorden wrote: >> > From: Rosemarie O'Riorden <rorio...@redhat.com> >> > >> > Currently, there is a default value of 1024 for socket-mem if not >> > configured. socket-limit automatically takes on the value of socket-mem >> > unless otherwise specified. With these changes, memory allocation will >> > be dynamically managed by DPDK, meaning that by default, no memory will >> > be pre-allocated on startup, and there will be no limit to how much >> > memory can be used. Either or both of these values can be set by the >> > user. >> > >> > The EAL arguments will look like this: >> > >> > - dpdk-socket-mem=<not set>, dpdk-socket-limit=<not set> >> > current: "--scket-mem=1024,1024 --socket-limit=1024,1024" >> > patch 1: "" >> > patch 2: "" >> > >> > - dpdk-socket-mem=<MEM>, dpdk-socket-limit=<not set> >> > current: "--scket-mem=MEM --socket-limit=MEM" >> > patch 1: "--scket-mem=MEM --socket-limit=MEM" >> > patch 2: "--scket-mem=MEM" >> > >> > - dpdk-socket-mem=<not set>, dpdk-socket-limit=<LIMIT> >> > current: "--scket-mem=1024,1024 --socket-limit=LIMIT" >> > patch 1: "--socket-limit=LIMIT" >> > patch 2: "--socket-limit=LIMIT" >> > >> > - dpdk-socket-mem=<MEM>, dpdk-socket-limit=<LIMIT> >> > current: "--scket-mem=MEM --socket-limit=LIMIT" >> > patch 1: "--scket-mem=MEM --socket-limit=LIMIT" >> > patch 2: "--scket-mem=MEM --socket-limit=LIMIT" >> > >> > Rosemarie O'Riorden (2): >> > dpdk: Remove default values for socket-mem and limit. >> > dpdk: Stop configuring socket-limit with the value of socket-mem. >> > >> > Documentation/intro/install/dpdk.rst | 3 +-- >> > NEWS | 4 ++++ >> > lib/dpdk.c | 6 +----- >> > vswitchd/vswitch.xml | 13 ++++++------- >> > 4 files changed, 12 insertions(+), 14 deletions(-) >> > >> >> Hi, Ian, everyone. >> >> This is the series I barely mentioned on yesterday's public >> meeting. It seems like a good thing to have in 2.16 release, >> but we definitely need an input from different sides on this >> kind of small, but user-visible change. >> >> In general, I think, it's good to stop configuring these >> unnecessary defaults that only limit users. 1GB of memory is >> barely enough for a very small setup and, most likely, most >> of users are setting dpdk-socket-mem anyway and doesn't rely >> on the default option (AFAICT, Michael is checking this with >> OpenStack folks). In any case, the upgrade path should not >> be hard, as it's enough to just set current values to the >> database before OVS upgrade. > > Afaik, setting memory configuration is mandatory in OSP. > Here, the changes mean that OSP would have to set a limit too, as they > were relying on this so far (some users are pick about how much memory > is available for the rest of the system and for vms). > > But otherwise the current parameter changes seems ok. > > >> >> Historically, it was there just to allow OVS run with DPDK >> "out-of-the-box" with a very minimal manual configuration, >> but current DPDK works without any memory configuration, so >> the original purpose of these defaults is not relevant anymore. > > Just a warning. > > Explicitely setting the memory configuration gives determinism. > Hugepages allocations are done once at EAL init, and then OVS picks > into them for building mempools and for rte_malloc's. > As long as those operations are done in control path, this is not > really a concern. > > But iirc, we have rte_malloc calls in the datapath for TSO support > (external buffer) and here, it means syscalls in the datapath when > growing "DPDK heap". > I agree, this might not happen often with 1G hugepages and once > reaching a steady state.
I guess it will be important to inform OSP folks about this change. There's no reason OSP cannot also start configuring the socket-limit already (since the code will use whatever OSP configured rather than providing a value). CC'd Assaf, Terry, and Jakub. _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev