Tuesday, July 24, 2018 6:14 PM, Adrien Mazarguil: > Subject: Re: [PATCH] mk: fix application compilation with lmnl and mlx5 > On Tue, Jul 24, 2018 at 01:03:28PM +0000, Shahaf Shuler wrote: > > Tuesday, July 24, 2018 3:56 PM, Adrien Mazarguil: > > > Subject: Re: [PATCH] mk: fix application compilation with lmnl and > > > mlx5
[...] > > > > Can we consider different options: > > > > 1. always statically link libmnl > > > > 2. dlopen libmnl just like we do for verbs/mlx5 > > > > > > Regarding 2, unlike rdma-core/MLNX_OFED, libmnl should be available > > > pretty much everywhere iproute2 can be found. The minimal version > > > supported > > > (1.0.3) was released in 2012. > > > > > > Using the glue approach for such a small library seems overkill; > > > should we choose this path, we must also consider to get rid of it > > > entirely since doing so would require more glue code than what mlx5 > needs from this library. > > > > > > So with the current approach, either the application or the PMD > > > inherits a dependency to libmnl, depending on whether > > > CONFIG_RTE_BUILD_SHARED_LIB is respectively disabled or enabled. > > > > > > If disabled, applications that want static linkage can specify > > > -static as part of their compilation flags to let the compiler > > > automatically look for libmnl.a as needed. > > > > Can you confirm static compilation w/ libmnl is working w/o any issues? > > Challenge accepted. 😊 Using -static is not something DPDK applications usually > do and needs a few tweaks to compile successfully though (unless you > meant something else by "static compilation"?) Yes this is what I meant. However I was under the impression we can statically link to libmnl while keeping the rdma-core w/ dynamic linkage. > > First you need to force rdma-core to generate static versions of > libmlx4/libmlx5 and libiberbs as it's not the default: > > $ cmake -DENABLE_STATIC=1 . > > Next, patch DPDK to remove references to -lgcc_s, -fPIC and -export- > dynamic: > > $ sed -i 's/[[:space:]]*-lgcc_s//' $(git grep -l -- -lgcc_s) $ sed -i > 's/[[:space:]]*- > fPIC//' $(git grep -l -- -fPIC) $ sed -i 's/[[:space:]]*-export-dynamic//' > $(git > grep -l -- -export-dynamic) It seems the fPIC and export-dynamic are defined only when compiling the DPDK as shared library. The gcc_s is from eal. Do you think it is correct to put it only when compiling as shared library? It seems this approach was taken in the rte.vars.mk files. > > Then append rdma-core dependencies to libmnl users (mlx5) in order to > avoid undefined references to libnl/libm stuff normally pulled by libibverbs: > > $ sed -i 's/-lmnl/-lmnl -lnl-route-3 -lnl-3 -lm/' $(git grep -l -- -lmnl) > > Configure DPDK with mlx5 enabled: > > $ make config T=x86_64-native-linuxapp-gcc $ sed -i > 's/^\(CONFIG_RTE_LIBRTE_MLX5_PMD\)=.*/\1=y/' build/.config > > Finally add -static to $CC (the easiest method I could find) while compiling > DPDK: > > $ make CC='gcc -static' > > You can ignore warnings about statically linking "getaddrinfo" and friends. > What matters is we now have a testpmd binary with no dependencies: > > $ readelf -d build/app/testpmd | grep NEEDED $ > > $ ldd build/app/testpmd > not a dynamic executable > $ > > Now start testpmd with a mlx5 adapter and a couple of representors for fun: > > # ./build/app/testpmd -w 06:00.0,representor=[0-1] -n 4 -c 0x80 -- -i --rxq=1 > --txq=1 > EAL: Detected 32 lcore(s) > EAL: Detected 2 NUMA nodes > EAL: Multi-process socket /var/run/dpdk/rte/mp_socket > EAL: No free hugepages reported in hugepages-1048576kB > EAL: Probing VFIO support... > EAL: PCI device 0000:06:00.0 on NUMA socket 0 > EAL: probe driver: 15b3:1017 net_mlx5 > [...] > Configuring Port 0 (socket 0) > Port 0: 24:8A:07:8D:AE:F6 > Configuring Port 1 (socket 0) > Port 1: A6:41:D9:89:DB:12 > Configuring Port 2 (socket 0) > Port 2: FE:A8:15:80:DE:C0 > Checking link statuses... > Done > testpmd> > > Phew! Nicely done. So if indeed it is possible w/ some fixes on the DPDK to fully support static linkage of both rdma-core and libmnl, maybe we should consider such compilation flag for mlx drivers. This can be very good alternative to the current DLOPEN approach. > > > To put this in perspective, this also applies to all other > > dependencies > > > it will collect while compiling DPDK (libz, libdl, libpcap, libnuma > > > to name a few). > > > > > > In my opinion, the purpose of *_DLOPEN_DEPS is to deal with large, > > > nonstandard libraries where versioning issues are commonplace. This > > > doesn't apply to libmnl, which shouldn't be a maintenance nightmare > > > to package maintainers. I suggest to leave things as is. > > -- > Adrien Mazarguil > 6WIND
