On Tue, Jul 24, 2018 at 01:03:28PM +0000, Shahaf Shuler wrote: > Tuesday, July 24, 2018 3:56 PM, Adrien Mazarguil: > > Subject: Re: [PATCH] mk: fix application compilation with lmnl and mlx5 > > > > On Tue, Jul 24, 2018 at 11:21:52AM +0000, Shahaf Shuler wrote: > > > Tuesday, July 24, 2018 12:29 PM, Nelio Laranjeiro: > > > > Subject: [PATCH] mk: fix application compilation with lmnl and mlx5 > > > > > > > > When Mellanox MLX5 PMD is compiled with > > > > CONFIG_RTE_LIBRTE_MLX5_DLOPEN_DEPS=y, the external dependency > > on > > > > libmln is missing. > > > > > > > > Fixes: 4d5cce06231a ("net/mlx5: lay groundwork for switch offloads") > > > > Cc: adrien.mazarg...@6wind.com > > > > > > > > Signed-off-by: Nelio Laranjeiro <nelio.laranje...@6wind.com> > > > > --- > > > > mk/rte.app.mk | 2 +- > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > diff --git a/mk/rte.app.mk b/mk/rte.app.mk index > > > > f4d28c2da..ff39d37aa > > > > 100644 > > > > --- a/mk/rte.app.mk > > > > +++ b/mk/rte.app.mk > > > > @@ -149,7 +149,7 @@ else > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_MLX4_PMD) += -lrte_pmd_mlx4 - > > > > libverbs -lmlx4 > > > > endif > > > > ifeq ($(CONFIG_RTE_LIBRTE_MLX5_DLOPEN_DEPS),y) > > > > -_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += -lrte_pmd_mlx5 -ldl > > > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += -lrte_pmd_mlx5 -ldl > > - > > > > lmnl > > > > > > This issue raise some more basic question. > > > The DLOPEN mode was introduced to run in systems which don't have > > verbs/mlx5 libs installed, because those were the only dependencies for the > > PMD back then. > > > Now we have the libmnl, which is external dependency just like rdma-core, > > and following your fix, hard linked also in case of DLOPEN option. > > > It means the whole DPDK binary/lib will be depended on libmnl and this is > > not what we want with DLOPEN. > > > > > > Can we consider different options: > > > 1. always statically link libmnl > > > 2. dlopen libmnl just like we do for verbs/mlx5 > > > > Regarding 2, unlike rdma-core/MLNX_OFED, libmnl should be available pretty > > much everywhere iproute2 can be found. The minimal version supported > > (1.0.3) was released in 2012. > > > > Using the glue approach for such a small library seems overkill; should we > > choose this path, we must also consider to get rid of it entirely since > > doing so > > would require more glue code than what mlx5 needs from this library. > > > > So with the current approach, either the application or the PMD inherits a > > dependency to libmnl, depending on whether > > CONFIG_RTE_BUILD_SHARED_LIB is respectively disabled or enabled. > > > > If disabled, applications that want static linkage can specify -static as > > part of > > their compilation flags to let the compiler automatically look for libmnl.a > > as > > needed. > > Can you confirm static compilation w/ libmnl is working w/o any issues?
Challenge accepted. Using -static is not something DPDK applications usually do and needs a few tweaks to compile successfully though (unless you meant something else by "static compilation"?) First you need to force rdma-core to generate static versions of libmlx4/libmlx5 and libiberbs as it's not the default: $ cmake -DENABLE_STATIC=1 . Next, patch DPDK to remove references to -lgcc_s, -fPIC and -export-dynamic: $ sed -i 's/[[:space:]]*-lgcc_s//' $(git grep -l -- -lgcc_s) $ sed -i 's/[[:space:]]*-fPIC//' $(git grep -l -- -fPIC) $ sed -i 's/[[:space:]]*-export-dynamic//' $(git grep -l -- -export-dynamic) Then append rdma-core dependencies to libmnl users (mlx5) in order to avoid undefined references to libnl/libm stuff normally pulled by libibverbs: $ sed -i 's/-lmnl/-lmnl -lnl-route-3 -lnl-3 -lm/' $(git grep -l -- -lmnl) Configure DPDK with mlx5 enabled: $ make config T=x86_64-native-linuxapp-gcc $ sed -i 's/^\(CONFIG_RTE_LIBRTE_MLX5_PMD\)=.*/\1=y/' build/.config Finally add -static to $CC (the easiest method I could find) while compiling DPDK: $ make CC='gcc -static' You can ignore warnings about statically linking "getaddrinfo" and friends. What matters is we now have a testpmd binary with no dependencies: $ readelf -d build/app/testpmd | grep NEEDED $ $ ldd build/app/testpmd not a dynamic executable $ Now start testpmd with a mlx5 adapter and a couple of representors for fun: # ./build/app/testpmd -w 06:00.0,representor=[0-1] -n 4 -c 0x80 -- -i --rxq=1 --txq=1 EAL: Detected 32 lcore(s) EAL: Detected 2 NUMA nodes EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: No free hugepages reported in hugepages-1048576kB EAL: Probing VFIO support... EAL: PCI device 0000:06:00.0 on NUMA socket 0 EAL: probe driver: 15b3:1017 net_mlx5 [...] Configuring Port 0 (socket 0) Port 0: 24:8A:07:8D:AE:F6 Configuring Port 1 (socket 0) Port 1: A6:41:D9:89:DB:12 Configuring Port 2 (socket 0) Port 2: FE:A8:15:80:DE:C0 Checking link statuses... Done testpmd> Phew! > To put this in perspective, this also applies to all other dependencies > > it will collect while compiling DPDK (libz, libdl, libpcap, libnuma to name > > a > > few). > > > > In my opinion, the purpose of *_DLOPEN_DEPS is to deal with large, > > nonstandard libraries where versioning issues are commonplace. This doesn't > > apply to libmnl, which shouldn't be a maintenance nightmare to package > > maintainers. I suggest to leave things as is. -- Adrien Mazarguil 6WIND