On Mon, 22 Jul 2019 10:06:11 +0100 Bruce Richardson <bruce.richard...@intel.com> wrote:
> On Mon, Jul 22, 2019 at 09:38:27AM +0200, Thomas Monjalon wrote: > > 19/07/2019 22:55, Stephen Hemminger: > > > On Tue, 16 Jul 2019 09:46:04 +0100 > > > Bruce Richardson <bruce.richard...@intel.com> wrote: > > > > > > > On Mon, Jul 15, 2019 at 05:19:12PM -0700, Stephen Hemminger wrote: > > > > > On Mon, 15 Jul 2019 16:41:36 -0700 > > > > > Stephen Hemminger <step...@networkplumber.org> wrote: > > > > > > > > > > > If DPDK is built as a shared library, then any application linked > > > > > > with rte.app.mk will not find any PCI devices. When the application > > > > > > is started no ethernet devices are found. > > > > > > > > > > > > This is because the link order of libraries on the command line > > > > > > matters. > > > > > > And PCI is before EAL. That causes there to be no dependency on PCI > > > > > > so linker ignores linking the library. > > > > > > Swapping the order fixes this. > > > > > > > > > > > > Fixes: c752998b5e2e ("pci: introduce library and driver") > > > > > > Cc: sta...@dpdk.org > > > > > > Signed-off-by: Stephen Hemminger <step...@networkplumber.org> > > > > > > --- > > > > > > mk/rte.app.mk | 2 +- > > > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > > > > > diff --git a/mk/rte.app.mk b/mk/rte.app.mk > > > > > > index a277c808ed8e..470b92e4d73e 100644 > > > > > > --- a/mk/rte.app.mk > > > > > > +++ b/mk/rte.app.mk > > > > > > @@ -90,8 +90,8 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_STACK) += > > > > > > -lrte_stack > > > > > > _LDLIBS-$(CONFIG_RTE_DRIVER_MEMPOOL_RING) += -lrte_mempool_ring > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_OCTEONTX2_MEMPOOL) += > > > > > > -lrte_mempool_octeontx2 > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_RING) += -lrte_ring > > > > > > -_LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_EAL) += -lrte_eal > > > > > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_PCI) += -lrte_pci > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_CMDLINE) += -lrte_cmdline > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_REORDER) += -lrte_reorder > > > > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_SCHED) += -lrte_sched > > > > > > > > > > It still happens with 19.08. Testpmd works but only because it is > > > > > linked with so many things. But l3fwd fails... > > > > > > > > > > # ./examples/l3fwd/build/l3fwd -n4 -l0-3 -w 02:00.0 > > > > > EAL: Detected 8 lcore(s) > > > > > EAL: Detected 1 NUMA nodes > > > > > EAL: failed to parse device "02:00.0" > > > > > EAL: Unable to parse device '02:00.0' > > > > > EAL: Error - exiting with code: 1 > > > > > Cause: Invalid EAL parameters > > > > > > > > I don't think the position of these is going to be the cause here, the > > > > more > > > > likely cause is that the pci bus driver - and all other drivers - are > > > > not > > > > linked into apps for shared library builds. You always need to pass "-d" > > > > parameter to load drivers at init time (or have them installed in the > > > > correct driver path). For example, for me with a shared library build > > > > the > > > > following gives a no ports error: > > > > > > > > sudo ./build/l2fwd -c F00000 -- -p 3 > > > > > > > > while this succeeds and runs fine > > > > > > > > sudo ./build/l2fwd -c F00000 -d > > > > $RTE_SDK/$RTE_TARGET/lib/librte_pmd_i40e.so -- -p 3 > > > > > > The root cause is that recent gcc won't run constructor on unused > > > libraries. > > > Testing a patch to take --as-needed off of PCI library. > > > > > > See: > > > https://stackoverflow.com/questions/11631161/force-to-link-against-unused-shared-library > > > > > > > The constructor is run when calling dlopen, right? > > > > Note: dlopen with -d is a feature. > > The original idea was to be able to specify which driver we want to use. > > If we want an automatic dlopen, like modprobe, then we need more scripts. > > But I understand you are against the whole dlopen idea. > > > > This issue is more of a problem for development systems where we EAL path > is not really usable for finding the drivers. For a properly deployed > system where we use DPDK installed to /usr/local or /usr, the EAL PMD path > will be correctly configured and properly probe all drivers. The problem is that bus drivers register themselves in constructors and these construtors are not run with as-needed. One part of fixing this is: diff --git a/mk/rte.app.mk b/mk/rte.app.mk index df917f946497..46bdff8ec5e8 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -130,6 +130,9 @@ ifeq ($(CONFIG_RTE_LIBRTE_FSLMC_BUS),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_COMMON_FSLMC) += -lrte_common_fslmc endif +# Bus devices register in constructor so always link +_LDLIBS-y += --no-as-needed + _LDLIBS-$(CONFIG_RTE_LIBRTE_PCI_BUS) += -lrte_bus_pci _LDLIBS-$(CONFIG_RTE_LIBRTE_VDEV_BUS) += -lrte_bus_vdev _LDLIBS-$(CONFIG_RTE_LIBRTE_DPAA_BUS) += -lrte_bus_dpaa @@ -137,6 +140,8 @@ ifeq ($(CONFIG_RTE_EAL_VFIO),y) _LDLIBS-$(CONFIG_RTE_LIBRTE_FSLMC_BUS) += -lrte_bus_fslmc endif +_LDLIBS-y += --as-needed + ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),n) # plugins (link only if static libraries)