[dpdk-dev] [PATCH 0/2] Mellanox ConnectX-3 PMD
How fast does this driver perform? Is it capable for sending/receiving 64B packets at 40G line rate? I'm using another version of user driver and it is not scalable for line rate, regardless of the number of TX cores. Keunhong. 2015. 1. 30. ?? 12:22? "Adrien Mazarguil" ?? ??: > This PMD adds support for Mellanox ConnectX-3-based adapters through the > verbs framework. It relies on external libraries (libibverbs and user space > driver libmlx4) and kernel support to do so. > > While these libraries and kernel modules are available on OpenFabrics > Alliance's website [1] and provided by package managers on most > distributions, this PMD requires Ethernet extensions that may not be > supported at the moment (this is a work in progress). > > Mellanox OFED [2] includes the necessary support and should be used in the > meantime. For DPDK, only libibverbs, libmlx4 and mlnx-ofed-kernel packages > are required from that distribution. > > The following kernel modules must be loaded before using this PMD: > > - mlx4_core (hardware driver, does global initialization) > - mlx4_en (Ethernet device driver) > - mlx4_ib (InfiniBand device driver) > - ib_uverbs (user space driver for verbs) > > Actual documentation will be added in V2. > > [1] https://www.openfabrics.org/ > [2] > http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers > > Adrien Mazarguil (2): > scripts: add auto-config-h.sh > mlx4: new poll mode driver > > config/common_bsdapp | 11 + > config/common_linuxapp | 11 + > lib/Makefile |1 + > lib/librte_pmd_mlx4/Makefile | 119 ++ > lib/librte_pmd_mlx4/mlx4.c | 4739 > ++ > lib/librte_pmd_mlx4/mlx4.h | 166 ++ > mk/rte.app.mk|8 + > scripts/auto-config-h.sh | 137 ++ > 8 files changed, 5192 insertions(+) > create mode 100755 lib/librte_pmd_mlx4/Makefile > create mode 100755 lib/librte_pmd_mlx4/mlx4.c > create mode 100644 lib/librte_pmd_mlx4/mlx4.h > create mode 100755 scripts/auto-config-h.sh > > -- > 2.1.0 > >
[dpdk-dev] [PATCH] pci: save list of detached devices, and re-probe during driver unload
Can you refer me to the logging facilities you are referring to for this
Freebsd driver?
device_probe_and_attach is an API in Freebsd kernel which is called during
boot for finding the relevant driver for each device.
I added manual call to it in the driver unload for re-probing and
re-matching the devices that were detached by the driver.
Before this change, you had to reboot in order to get the devices back.
-Original Message-
From: Neil Horman [mailto:nhor...@tuxdriver.com]
Sent: 01 March 2015 15:48
To: Raz Amir
Cc: dev at dpdk.org
Subject: Re: [dpdk-dev] [PATCH] pci: save list of detached devices, and
re-probe during driver unload
On Thu, Feb 26, 2015 at 06:33:20AM +, Raz Amir wrote:
> Added code that saves the pointers to the detached devices, during
> driver loading, and during driver unloading, go over the list,
> and re-attach them by calling device_probe_and_attach
> on each device.
>
> Signed-off-by: Raz Amir
> ---
> lib/librte_eal/bsdapp/nic_uio/nic_uio.c | 26 +-
> 1 file changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> index 5ae8560..7d702a5 100644
> --- a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> +++ b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> @@ -55,6 +55,9 @@ __FBSDID("$FreeBSD$");
>
> #define MAX_BARS (PCIR_MAX_BAR_0 + 1)
>
> +#define MAX_DETACHED_DEVICES 128
> +static device_t detached_devices[MAX_DETACHED_DEVICES] = {};
> +static int last_detached = 0;
>
> struct nic_uio_softc {
> device_tdev_t;
> @@ -291,14 +294,35 @@ nic_uio_load(void)
> if (dev != NULL)
> for (i = 0; i < NUM_DEVICES; i++)
> if (pci_get_vendor(dev) == devices[i].vend
&&
> - pci_get_device(dev) ==
devices[i].dev)
> + pci_get_device(dev) ==
devices[i].dev) {
> + if (last_detached+1
< MAX_DETACHED_DEVICES) {
> +
printf("nic_uio_load: detaching and storing dev=%p\n", dev);
No printfs. Use the logging facilities
> +
detached_devices[last_detached++] = dev;
> + }
> + else {
> +
printf("nic_uio_load: reached MAX_DETACHED_DEVICES=%d. dev=%p won't be
reattached\n",
> +
MAX_DETACHED_DEVICES, dev);
Dittto
> + }
> +
> device_detach(dev);
> + }
> }
> }
>
> static void
> nic_uio_unload(void)
> {
> + int i;
> + printf("nic_uio_unload: entered ... \n");
> +
> + for (i = 0; i < last_detached; i++) {
> + printf("nic_uio_unload: calling to device_probe_and_attach
for dev=%p...\n",
> + detached_devices[i]);
> + device_probe_and_attach(detached_devices[i]);
Where is this defined? It doesn't appear to be in the latest dpdk.
> + printf("nic_uio_unload: done.\n");
> + }
> +
> + printf("nic_uio_unload: leaving ... \n");
> }
>
> static int
> --
> 2.1.2
>
>
[dpdk-dev] [PATCH v3 0/3] Mellanox ConnectX-3 PMD
On Fri, Feb 27, 2015 at 07:38:59PM +0100, Adrien Mazarguil wrote: > On Thu, Feb 26, 2015 at 03:49:07PM +0200, Gleb Natapov wrote: > > On Thu, Feb 26, 2015 at 02:36:27PM +0100, Thomas Monjalon wrote: > > > 2015-02-26 13:51, Gleb Natapov: > > > > Did git pull today. After enabling mlnx pmd compilation fails with: > > > > > > > > dpdk/lib/librte_pmd_mlx4/mlx4.c: In function ?mlx4_pci_devinit?: > > > > dpdk/lib/librte_pmd_mlx4/mlx4.c:4636:14: error: too few arguments to > > > > function ?rte_eth_dev_allocate? > > > > eth_dev = rte_eth_dev_allocate(name); > > > > > > Yes, thanks for reporting. > > > I didn't test the disabled mlx4 after hotplug integration: > > > dpdk.org/browse/dpdk/commit/?id=9f1653e7b7e1746e7c > > > > > > Clearly, I have to improve my sanity checks. > > > Sorry for the inconvenience. > > No problem, I fixed that locally, but now I see another issue. I have > > several PMDs statically compiled in with my application and I expect > > dpdk to choose correct one depending on available HW, but mlnx pmd does > > not behave nicely, if its initialization fails it kills entire > > application: > > > > EAL: PCI device :03:00.0 on NUMA socket 0 > > EAL: probe driver: 15b3:1003 librte_pmd_mlx4 > > EAL: Error - exiting with code: 1 > > Cause: Requested device :03:00.0 cannot be used > > Forgot to set in-reply-to, but I just sent a patch to work around that > issue and make mlx4 nicer: > > http://dpdk.org/dev/patchwork/patch/3796/ > Works for me, thanks! May be better to change: "cannot use device, are drivers up to date? To "cannot use device, are drivers and fw up to date?". On one of my machines that was the case. I see that some features are missing from the PMD though. Some of them are nice to have for good performance like providing rss hash value in mbuf (PKT_RX_RSS_HASH), but others are absolutely required for my application to work: setting (or at least getting) rss key and redirection table. Are there any plans to support those? -- Gleb.
[dpdk-dev] [PATCH] pci: save list of detached devices, and re-probe during driver unload
On Sun, Mar 01, 2015 at 04:21:10PM +0200, Raz Amir wrote:
> Can you refer me to the logging facilities you are referring to for this
> Freebsd driver?
> device_probe_and_attach is an API in Freebsd kernel which is called during
rte_log and friends.
> boot for finding the relevant driver for each device.
> I added manual call to it in the driver unload for re-probing and
> re-matching the devices that were detached by the driver.
> Before this change, you had to reboot in order to get the devices back.
>
my bad, I didn't see it defined and was thinking it was removed from an earlier
version of the dpdk.
Neil
> -Original Message-
> From: Neil Horman [mailto:nhorman at tuxdriver.com]
> Sent: 01 March 2015 15:48
> To: Raz Amir
> Cc: dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] pci: save list of detached devices, and
> re-probe during driver unload
>
> On Thu, Feb 26, 2015 at 06:33:20AM +, Raz Amir wrote:
> > Added code that saves the pointers to the detached devices, during
> > driver loading, and during driver unloading, go over the list,
> > and re-attach them by calling device_probe_and_attach
> > on each device.
> >
> > Signed-off-by: Raz Amir
> > ---
> > lib/librte_eal/bsdapp/nic_uio/nic_uio.c | 26 +-
> > 1 file changed, 25 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> > index 5ae8560..7d702a5 100644
> > --- a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> > +++ b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> > @@ -55,6 +55,9 @@ __FBSDID("$FreeBSD$");
> >
> > #define MAX_BARS (PCIR_MAX_BAR_0 + 1)
> >
> > +#define MAX_DETACHED_DEVICES 128
> > +static device_t detached_devices[MAX_DETACHED_DEVICES] = {};
> > +static int last_detached = 0;
> >
> > struct nic_uio_softc {
> > device_tdev_t;
> > @@ -291,14 +294,35 @@ nic_uio_load(void)
> > if (dev != NULL)
> > for (i = 0; i < NUM_DEVICES; i++)
> > if (pci_get_vendor(dev) == devices[i].vend
> &&
> > - pci_get_device(dev) ==
> devices[i].dev)
> > + pci_get_device(dev) ==
> devices[i].dev) {
> > + if (last_detached+1
> < MAX_DETACHED_DEVICES) {
> > +
> printf("nic_uio_load: detaching and storing dev=%p\n", dev);
> No printfs. Use the logging facilities
>
> > +
> detached_devices[last_detached++] = dev;
> > + }
> > + else {
> > +
> printf("nic_uio_load: reached MAX_DETACHED_DEVICES=%d. dev=%p won't be
> reattached\n",
> > +
> MAX_DETACHED_DEVICES, dev);
> Dittto
>
> > + }
> > +
> > device_detach(dev);
> > + }
> > }
> > }
> >
> > static void
> > nic_uio_unload(void)
> > {
> > + int i;
> > + printf("nic_uio_unload: entered ... \n");
> > +
> > + for (i = 0; i < last_detached; i++) {
> > + printf("nic_uio_unload: calling to device_probe_and_attach
> for dev=%p...\n",
> > + detached_devices[i]);
> > + device_probe_and_attach(detached_devices[i]);
> Where is this defined? It doesn't appear to be in the latest dpdk.
>
> > + printf("nic_uio_unload: done.\n");
> > + }
> > +
> > + printf("nic_uio_unload: leaving ... \n");
> > }
> >
> > static int
> > --
> > 2.1.2
> >
> >
>
>
[dpdk-dev] [PATCH] pci: save list of detached devices, and re-probe during driver unload
On Thu, Feb 26, 2015 at 06:33:20AM +, Raz Amir wrote:
> Added code that saves the pointers to the detached devices, during
> driver loading, and during driver unloading, go over the list,
> and re-attach them by calling device_probe_and_attach
> on each device.
>
> Signed-off-by: Raz Amir
> ---
> lib/librte_eal/bsdapp/nic_uio/nic_uio.c | 26 +-
> 1 file changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> index 5ae8560..7d702a5 100644
> --- a/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> +++ b/lib/librte_eal/bsdapp/nic_uio/nic_uio.c
> @@ -55,6 +55,9 @@ __FBSDID("$FreeBSD$");
>
> #define MAX_BARS (PCIR_MAX_BAR_0 + 1)
>
> +#define MAX_DETACHED_DEVICES 128
> +static device_t detached_devices[MAX_DETACHED_DEVICES] = {};
> +static int last_detached = 0;
>
> struct nic_uio_softc {
> device_tdev_t;
> @@ -291,14 +294,35 @@ nic_uio_load(void)
> if (dev != NULL)
> for (i = 0; i < NUM_DEVICES; i++)
> if (pci_get_vendor(dev) == devices[i].vend &&
> - pci_get_device(dev) ==
> devices[i].dev)
> + pci_get_device(dev) ==
> devices[i].dev) {
> + if (last_detached+1 <
> MAX_DETACHED_DEVICES) {
> +
> printf("nic_uio_load: detaching and storing dev=%p\n", dev);
No printfs. Use the logging facilities
> +
> detached_devices[last_detached++] = dev;
> + }
> + else {
> +
> printf("nic_uio_load: reached MAX_DETACHED_DEVICES=%d. dev=%p won't be
> reattached\n",
> +
> MAX_DETACHED_DEVICES, dev);
Dittto
> + }
> +
> device_detach(dev);
> + }
> }
> }
>
> static void
> nic_uio_unload(void)
> {
> + int i;
> + printf("nic_uio_unload: entered ... \n");
> +
> + for (i = 0; i < last_detached; i++) {
> + printf("nic_uio_unload: calling to device_probe_and_attach for
> dev=%p...\n",
> + detached_devices[i]);
> + device_probe_and_attach(detached_devices[i]);
Where is this defined? It doesn't appear to be in the latest dpdk.
> + printf("nic_uio_unload: done.\n");
> + }
> +
> + printf("nic_uio_unload: leaving ... \n");
> }
>
> static int
> --
> 2.1.2
>
>
