Hello, On Fri, Jul 15, 2022 at 5:35 PM Dave Houser <davehous...@gmail.com> wrote: > DPDK version: 21.11.1 > OVS version: 2.17.2 > OS: ubuntu 18.04 > Kernel: 4.15.0-188-generic > > DPDK and OVS were compiled from source. > > I am running into a number of issues trying to get OVS to work with DPDK / > Mellanox ConnectX6, can someone assist with the following problems below?
ETOOMANYINFO and it seems you skipped some part of the documentations. For now, here is some help to get a working OVS + DPDK setup. > > 1) After restarting the openvswitch-switch service I see the following > warning: > > ``` > 2022-07-15T14:41:39.548Z|00023|dpdk|WARN|EAL: DPDK is running on a NUMA > system, but is compiled without NUMA support. > 2022-07-15T14:41:39.548Z|00024|dpdk|WARN|EAL: This will have adverse > consequences for performance and usability. > 2022-07-15T14:41:39.548Z|00025|dpdk|WARN|EAL: Please use --legacy-mem option, > or recompile with NUMA support. > ``` > > This is a DPDK warning, but it's not clear on how I can fix this. I can't > find any documentation on how to compile with the "--legacy-mem" option. Is > this a meson or ninja parameter? Or how do I "recompile with NUMA support", > this is not clear and the DPDK documentation is not clear either. Or can I > just ignore this? https://doc.dpdk.org/guides-21.11/linux_gsg/sys_reqs.html Read the requirements, related to NUMA: """ Library for handling NUMA (Non Uniform Memory Access). - numactl-devel in RHEL/Fedora; - libnuma-dev in Debian/Ubuntu; - numactl-dev in Alpine Linux """ You need to install libnuma-dev, then reconfigure your DPDK build directory (or erase your build directory and start from scratch), and finally recompile DPDK. Once done, relink your OVS binaries against this fresh DPDK. > > 2) I am trying to follow these instructions on how to deploy on Ubuntu > (https://ubuntu.com/server/docs/openvswitch-dpdk), however the Open_vSwitch > table settings presented cause OVS to crash when starting up. > > ``` > # Allocate 2G huge pages (not Numa node aware) > ovs-vsctl set Open_vSwitch . "other_config:dpdk-alloc-mem=2048" This param makes OVS ask DPDK to preallocate 2G of hugepages. However, starting 2.17, OVS can ask memory (to DPDK) when needed and you probably don't need this setting. I suggest removing it... > # limit to one whitelisted device > ovs-vsctl set Open_vSwitch . > "other_config:dpdk-extra=--pci-whitelist=0000:04:00.0" ... and this one too. By default, OVS lets DPDK scan all available devices. OVS then associates one OVS port (via this port options:dpdk-devargs) to the available DPDK resources. But, In any case, your main issue is probably a lack of hugepages, see below. > ``` > > 2.1) Trying to run the first command (ovs-vsctl set Open_vSwitch . > "other_config:dpdk-alloc-mem=2048") gives me this error: > > ``` > 2022-07-15T15:14:05.549Z|00026|dpdk|ERR|EAL: Not enough memory available! > Requested: 2048MB, available: 118MB > ``` > But the system has lots of memory: > ``` > $ free -g > total used free shared buff/cache > available > Mem: 251 89 113 0 47 > 159 > Swap: 7 0 7 > ``` > Why am I getting this error? I have 113GB of memory free. Why does DPDK show > I only have 118MB? DPDK expects hugepage memory. >From this error message (which comes with a DPDK init failure, not a crash), I suspect you don't have enough hugepages declared. The DPDK (and OVS) documentation has examples of how to configure this. Like for example: https://github.com/openvswitch/ovs/blob/master/Documentation/intro/install/dpdk.rst -- David Marchand _______________________________________________ discuss mailing list disc...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
