Re: [dpdk-dev] [PATCH v1 2/3] net/hyperv: implement core functionality

2017-12-19 Thread Nelio Laranjeiro
Hi Keith, On Mon, Dec 18, 2017 at 06:43:35PM +, Wiles, Keith wrote: > > > > On Dec 18, 2017, at 11:59 AM, Adrien Mazarguil > > wrote: > > >> Not to criticize style, but a few blank lines could help in > >> readability for these files IMHO. Unless blank lines are illegal > >> :-) > > > >

Re: [dpdk-dev] [RFC] eventdev: add crypto adapter API header

2017-12-19 Thread Jerin Jacob
-Original Message- > Date: Mon, 18 Dec 2017 16:33:53 + > From: "Eads, Gage" > To: Jerin Jacob > CC: "Gujjar, Abhinandan S" , "dev@dpdk.org" > , "Vangati, Narender" , "Rao, > Nikhil" , "hemant.agra...@nxp.com" > , "Doherty, Declan" , > "nidadavolu.mur...@cavium.com" , > "nithin.da

Re: [dpdk-dev] [PATCH v2] igb_uio: allow multi-process access

2017-12-19 Thread Tiwei Bie
Hi, On Tue, Dec 19, 2017 at 07:42:20AM -0800, Xiao Wang wrote: > @@ -336,6 +337,10 @@ struct rte_uio_pci_dev { > struct pci_dev *dev = udev->pdev; > int err; > > + atomic_inc(&udev->refcnt); > + if (atomic_read(&udev->refcnt) > 1) The "inc and read" should be atomic. Otherwi

[dpdk-dev] [PATCH v3] igb_uio: allow multi-process access

2017-12-19 Thread Xiao Wang
In some case, one device are accessed by different processes via different BARs, so one uio device may be opened by more than one process, for this case we just need to enable interrupt once, and pci_clear_master only when the last process closed. Fixes: 5f6ff30dc507 ("igb_uio: fix interrupt enabl

Re: [dpdk-dev] [PATCH v2] igb_uio: allow multi-process access

2017-12-19 Thread Wang, Xiao W
Thanks for pointing it out. Fix it in v3. BRs, Xiao > -Original Message- > From: Bie, Tiwei > Sent: Tuesday, December 19, 2017 5:05 PM > To: Wang, Xiao W > Cc: Yigit, Ferruh ; dev@dpdk.org; > step...@networkplumber.org > Subject: Re: [dpdk-dev] [PATCH v2] igb_uio: allow multi-process acc

[dpdk-dev] [PATCH] doc: update IPSec Multi-buffer lib versioning

2017-12-19 Thread Pablo de Lara
IPSec Multi-buffer library v0.48 has been released, which includes, among other features, support for AES-CCM. Signed-off-by: Pablo de Lara --- doc/guides/cryptodevs/aesni_gcm.rst | 4 ++-- doc/guides/cryptodevs/aesni_mb.rst | 9 + 2 files changed, 7 insertions(+), 6 deletions(-) diff

Re: [dpdk-dev] [PATCH] ethdev: fix setting of MAC address

2017-12-19 Thread Olivier MATZ
Hi, On Mon, Dec 18, 2017 at 02:38:55PM +0300, Andrew Rybchenko wrote: > On 12/18/2017 01:53 PM, Igor Ryzhov wrote: > > > > On Mon, Dec 18, 2017 at 1:35 PM, Andrew Rybchenko > > mailto:arybche...@solarflare.com>> wrote: > > > > On 12/14/2017 08:15 PM, Olivier Matz wrote: > > > > From

Re: [dpdk-dev] [PATCH] ethdev: fix setting of MAC address

2017-12-19 Thread Andrew Rybchenko
On 12/19/2017 12:29 PM, Olivier MATZ wrote: Hi, On Mon, Dec 18, 2017 at 02:38:55PM +0300, Andrew Rybchenko wrote: On 12/18/2017 01:53 PM, Igor Ryzhov wrote: On Mon, Dec 18, 2017 at 1:35 PM, Andrew Rybchenko mailto:arybche...@solarflare.com>> wrote: On 12/14/2017 08:15 PM, Olivier Matz wr

Re: [dpdk-dev] [PATCH v1 2/3] net/hyperv: implement core functionality

2017-12-19 Thread Bruce Richardson
On Mon, Dec 18, 2017 at 09:23:41PM +0100, Adrien Mazarguil wrote: > On Mon, Dec 18, 2017 at 10:34:12AM -0800, Stephen Hemminger wrote: > > On Mon, 18 Dec 2017 17:46:23 +0100 > > Adrien Mazarguil wrote: > > > > > +static int > > > +hyperv_iface_is_netvsc(const struct if_nameindex *iface) > > > +{

Re: [dpdk-dev] [PATCH v1 1/3] net/hyperv: introduce MS Hyper-V platform driver

2017-12-19 Thread Adrien Mazarguil
On Mon, Dec 18, 2017 at 01:17:51PM -0800, Stephen Hemminger wrote: > On Mon, 18 Dec 2017 20:54:16 +0100 > Thomas Monjalon wrote: > > > > > +#endif /* RTE_LIBRTE_HYPERV_DEBUG */ > > > > + > > > > +#define DEBUG(...) PMD_DRV_LOG(DEBUG, __VA_ARGS__) > > > > +#define INFO(...) PMD_DRV_LOG(INFO, __VA_

Re: [dpdk-dev] [PATCH v4 2/4] change root makefile license to SPDX tag

2017-12-19 Thread Bruce Richardson
On Tue, Dec 19, 2017 at 10:59:26AM +0530, Hemant Agrawal wrote: > On 12/18/2017 10:00 PM, Thomas Monjalon wrote: > > 18/12/2017 16:52, Bruce Richardson: > > > On Mon, Dec 18, 2017 at 06:09:02PM +0530, Hemant Agrawal wrote: > > > > --- a/GNUmakefile > > > > +++ b/GNUmakefile > > > > @@ -1,33 +1,6 @@

Re: [dpdk-dev] [PATCH v1 2/3] net/hyperv: implement core functionality

2017-12-19 Thread Adrien Mazarguil
On Mon, Dec 18, 2017 at 03:59:46PM -0800, Stephen Hemminger wrote: > On Mon, 18 Dec 2017 17:46:23 +0100 > Adrien Mazarguil wrote: > > > +static int > > +ether_addr_from_str(struct ether_addr *eth_addr, const char *str) > > +{ > > + static const uint8_t conv[0x100] = { > > + ['0'] = 0x

Re: [dpdk-dev] [PATCH] net/virtio-user: specify MAC address for tap port

2017-12-19 Thread Ning Li
Hi Tiwei, > Hi Ning, > > On Thu, Dec 14, 2017 at 07:38:14PM +0800, Ning Li wrote: > > When use virtio_user as exception path, we need to specify a MAC > > address for the tap port. > > Is this a fix? Did you meet any issue? If so, please describe > the issue and add a fixline. Specify the MAC a

Re: [dpdk-dev] [PATCH 0/6] next-build: create both static and shared libs

2017-12-19 Thread Bruce Richardson
On Mon, Dec 18, 2017 at 01:05:14PM -0500, Aaron Conole wrote: > Luca Boccassi writes: > > > On Tue, 2017-12-12 at 17:14 +, Bruce Richardson wrote: > >> On Tue, Dec 12, 2017 at 04:59:34PM +, Bruce Richardson wrote: > >> > This patchset changes the meson+ninja build system to always create

Re: [dpdk-dev] [PATCH 1/2] mbuf: support PPPoE and L2TP in software packet type parser

2017-12-19 Thread Olivier MATZ
Hi Beilei, On Tue, Nov 28, 2017 at 06:12:55PM +0800, Beilei Xing wrote: > Add support of PPPoE and L2TP in rte_net_get_ptype(). > > Signed-off-by: Beilei Xing > --- > lib/librte_mbuf/rte_mbuf_ptype.c | 2 ++ > lib/librte_mbuf/rte_mbuf_ptype.h | 26 ++ > 2 files changed,

[dpdk-dev] [PATCH v5 2/4] change root makefile license to SPDX tag

2017-12-19 Thread Hemant Agrawal
Signed-off-by: Hemant Agrawal --- GNUmakefile | 32 ++-- Makefile| 32 ++-- 2 files changed, 4 insertions(+), 60 deletions(-) diff --git a/GNUmakefile b/GNUmakefile index 45b7fbb..594f8cb 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -1

Re: [dpdk-dev] [PATCH v1 2/3] net/hyperv: implement core functionality

2017-12-19 Thread Adrien Mazarguil
On Tue, Dec 19, 2017 at 09:53:27AM +, Bruce Richardson wrote: > On Mon, Dec 18, 2017 at 09:23:41PM +0100, Adrien Mazarguil wrote: > > On Mon, Dec 18, 2017 at 10:34:12AM -0800, Stephen Hemminger wrote: > > > On Mon, 18 Dec 2017 17:46:23 +0100 > > > Adrien Mazarguil wrote: > > > > > > > > +sta

[dpdk-dev] [PATCH v5 1/4] Introducing SPDX License Identifiers

2017-12-19 Thread Hemant Agrawal
The DPDK uses the Open Source BSD-3-Clause license for the core libraries and drivers. The kernel components are naturally GPLv2 licensed. Many of the files in the DPDK source code contain the full text of the applicable license. For example, most of the BSD-3-Clause files contain a full copy of t

[dpdk-dev] [PATCH v5 4/4] change NXP dpaa code License text to SPDX tags

2017-12-19 Thread Hemant Agrawal
Signed-off-by: Hemant Agrawal --- config/defconfig_arm64-dpaa-linuxapp-gcc | 34 +++- doc/guides/cryptodevs/dpaa_sec.rst| 31 +++--- doc/guides/nics/dpaa.rst | 31 +++--- doc/guides/prog_guide/rte_security.r

Re: [dpdk-dev] [PATCH] net/pcap: convert license headers to SPDX tags

2017-12-19 Thread Bruce Richardson
On Mon, Dec 18, 2017 at 01:18:33PM -0800, Ferruh Yigit wrote: > On 12/18/2017 1:06 PM, Ferruh Yigit wrote: > > Signed-off-by: Ferruh Yigit > > <...> > > > + * SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(c) 2010-2016 Intel Corporation. All rights reserved. > > + * Copyright(c) 2014 6W

Re: [dpdk-dev] [PATCH v5 2/4] change root makefile license to SPDX tag

2017-12-19 Thread Bruce Richardson
On Tue, Dec 19, 2017 at 03:44:39PM +0530, Hemant Agrawal wrote: > Signed-off-by: Hemant Agrawal > --- Acked-by: Bruce Richardson FYI: if doing a V6, you can drop the "All rights reserved" from the copyright line. If no V6, it's fine as-is (or it can be removed on apply).

Re: [dpdk-dev] [PATCH 1/2] mempool: indicate the usages of multi memzones

2017-12-19 Thread Olivier MATZ
Hi Hemant, On Wed, Dec 06, 2017 at 06:01:12PM +0530, Hemant Agrawal wrote: > This is required for the optimizations w.r.t hw mempools. > They will use different kind of optimizations if the buffers > are from single contiguous memzone. > > Signed-off-by: Hemant Agrawal > --- > lib/librte_mempoo

Re: [dpdk-dev] [PATCH v3 7/8] app/eventdev: add service core configuration

2017-12-19 Thread Van Haaren, Harry
> From: Pavan Nikhilesh [mailto:pbhagavat...@caviumnetworks.com] > Add service core configuration for Rx adapter. The configuration picks > the least used service core to run the service on. > > Signed-off-by: Pavan Nikhilesh > --- > app/test-eventdev/evt_common.h | 41 ---

Re: [dpdk-dev] [PATCH 1/3] lib/cmdline: add echo support in batch loading from file

2017-12-19 Thread Olivier MATZ
Hi Xueming, On Wed, Nov 15, 2017 at 11:45:43PM +0800, Xueming Li wrote: > Add echo option to echo commandline to screen when running loaded > scripts from file. > > Signed-off-by: Xueming Li > --- > lib/librte_cmdline/cmdline_socket.c | 5 +++-- > lib/librte_cmdline/cmdline_socket.h | 3 ++- >

Re: [dpdk-dev] [PATCH 1/2] mempool: indicate the usages of multi memzones

2017-12-19 Thread Hemant Agrawal
Hi Olivier, On 12/19/2017 3:54 PM, Olivier MATZ wrote: Hi Hemant, On Wed, Dec 06, 2017 at 06:01:12PM +0530, Hemant Agrawal wrote: This is required for the optimizations w.r.t hw mempools. They will use different kind of optimizations if the buffers are from single contiguous memzone. Signed-o

Re: [dpdk-dev] [PATCH] net/virtio: fix Rx and Tx handler selection for arm32

2017-12-19 Thread Olivier MATZ
Hi Tiwei, On Mon, Dec 18, 2017 at 10:50:47AM +0800, Tiwei Bie wrote: > On Thu, Dec 14, 2017 at 03:32:13PM +0100, Olivier Matz wrote: > > From: Samuel Gauthier > > > > On arm32, we were always selecting the simple handler, but it is only > > available if neon is present. > > > > This is due to a

[dpdk-dev] [PATCH 1/2] build: remove architecture flag as default C flag

2017-12-19 Thread Bruce Richardson
Any flags added to the project args are automatically added to all builds, both native and cross-compiled. This is not what we want for the -march flag as a valid -march for the cross-compile is not valid for pmdinfogen which is a native-build tool. Instead we store the march flag as a variable, a

[dpdk-dev] [PATCH 2/2] net/pcap: fix cross compilation

2017-12-19 Thread Bruce Richardson
The detection of pcap as a dependency involves invoking pcap-config to get parameters - something not possible in a cross-compilation environment. Therefore we need to just look for the presence of the library in a cross-compilation environment and assume if the library is present we can compile an

[dpdk-dev] [PATCH 0/2] Enable cross-compilation of DPDK with meson

2017-12-19 Thread Bruce Richardson
While meson has built-in support for cross-compilation, there can be things inside the meson.build files which cause problems in a cross-compile environment. This patchset fixes a number of issues found when doing test builds for arm architecture. NOTE: this patchset only contains the fixes made t

[dpdk-dev] [RFC PATCH] RFC build: prototype support for ARM builds

2017-12-19 Thread Bruce Richardson
Add some skeleton files to enable compiling for ARM target. This has been tested by doing a cross-compile for armv8-a type using the linaro gcc toolchain. meson arm-build --cross-file aarch64_cross.txt ninja -C arm-build where aarch64_cross.txt contained the following [bi

Re: [dpdk-dev] [PATCH 1/2] mempool: indicate the usages of multi memzones

2017-12-19 Thread Olivier MATZ
On Tue, Dec 19, 2017 at 04:16:33PM +0530, Hemant Agrawal wrote: > Hi Olivier, > > On 12/19/2017 3:54 PM, Olivier MATZ wrote: > > Hi Hemant, > > > > On Wed, Dec 06, 2017 at 06:01:12PM +0530, Hemant Agrawal wrote: > > > This is required for the optimizations w.r.t hw mempools. > > > They will use d

[dpdk-dev] [RFC 01/23] eal/memory: move get_virtual_area out of linuxapp eal_memory.c

2017-12-19 Thread Anatoly Burakov
Move get_virtual_area out of linuxapp EAL memory and make it common to EAL, so that other code could reserve virtual areas as well. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_common_memory.c | 70 ++ lib/librte_eal/common/eal_private.h | 29 +++

[dpdk-dev] [PATCH 00/23] Dynamic memory allocation for DPDK

2017-12-19 Thread Anatoly Burakov
This patchset introduces a prototype implementation of dynamic memory allocation for DPDK. It is intended to start a conversation and build consensus on the best way to implement this functionality. The patchset works well enough to pass all unit tests, and to work with traffic forwarding, provided

[dpdk-dev] [PATCH 02/23] eal: add function to report number of detected sockets

2017-12-19 Thread Anatoly Burakov
At the moment, we always rely on scanning everything for every socket up until RTE_MAX_NUMA_NODES and checking if there's a memseg associated with each socket if we want to find out how many sockets we actually have. This becomes a problem when we may have memory on socket but it's not allocated ye

[dpdk-dev] [PATCH 01/23] eal: move get_virtual_area out of linuxapp eal_memory.c

2017-12-19 Thread Anatoly Burakov
Move get_virtual_area out of linuxapp EAL memory and make it common to EAL, so that other code could reserve virtual areas as well. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_common_memory.c | 70 ++ lib/librte_eal/common/eal_private.h | 29 +++

[dpdk-dev] [PATCH 04/23] eal: move all locking to heap

2017-12-19 Thread Anatoly Burakov
Down the line, we will need to do everything from the heap as any alloc or free may trigger alloc/free OS memory, which would involve growing/shrinking heap. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/malloc_elem.c | 16 ++-- lib/librte_eal/common/malloc_heap.c | 36 +++

[dpdk-dev] [RFC 02/23] eal/lcore: add function to report number of detected sockets

2017-12-19 Thread Anatoly Burakov
At the moment, we always rely on scanning everything for every socket up until RTE_MAX_NUMA_NODES and checking if there's a memseg associated with each socket if we want to find out how many sockets we actually have. This becomes a problem when we may have memory on socket but it's not allocated ye

[dpdk-dev] [RFC 05/23] eal/malloc: protect malloc heap stats with a lock

2017-12-19 Thread Anatoly Burakov
This does not change the public API, as this API is not meant to be called directly. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/malloc_heap.c | 7 ++- lib/librte_eal/common/malloc_heap.h | 2 +- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/commo

[dpdk-dev] [PATCH 06/23] eal: make malloc a doubly-linked list

2017-12-19 Thread Anatoly Burakov
As we are preparing for dynamic memory allocation, we need to be able to handle holes in our malloc heap, hence we're switching to doubly linked list, and prepare infrastructure to support it. Since our heap is now aware where are our first and last elements, there is no longer any need to have a

[dpdk-dev] [PATCH 10/23] eal: read hugepage counts from node-specific sysfs path

2017-12-19 Thread Anatoly Burakov
For non-legacy memory init mode, instead of looking at generic sysfs path, look at sysfs paths pertaining to each NUMA node for hugepage counts. Note that per-NUMA node path does not provide information regarding reserved pages, so we might not get the best info from these paths, but this saves us

[dpdk-dev] [PATCH 08/23] eal: add "single file segments" command-line option

2017-12-19 Thread Anatoly Burakov
For now, this option does nothing, but it will be useful in dynamic memory allocation down the line. Currently, DPDK stores all pages as separate files in hugetlbfs. This option will allow storing all pages in one file (one file per socket, per page size). Signed-off-by: Anatoly Burakov --- lib/

[dpdk-dev] [RFC 04/23] eal/malloc: move all locking to heap

2017-12-19 Thread Anatoly Burakov
Down the line, we will need to do everything from the heap as any alloc or free may trigger alloc/free OS memory, which would involve growing/shrinking heap. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/malloc_elem.c | 16 ++-- lib/librte_eal/common/malloc_heap.c | 36 +++

[dpdk-dev] [PATCH 07/23] eal: make malloc_elem_join_adjacent_free public

2017-12-19 Thread Anatoly Burakov
We need this function to join newly allocated segments with the heap. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/malloc_elem.c | 6 +++--- lib/librte_eal/common/malloc_elem.h | 3 +++ 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/librte_eal/common/malloc_elem.c

[dpdk-dev] [RFC 15/23] eal/memalloc: add function to check if memory is physically contiguous

2017-12-19 Thread Anatoly Burakov
This will be helpful down the line when we implement support for allocating physically contiguous memory. We can no longer guarantee physically contiguous memory unless we're in IOVA_AS_VA mode, but we can certainly try and see if we succeed. In addition, this would be useful for e.g. PMD's who may

[dpdk-dev] [PATCH 15/23] eal: add API to check if memory is physically contiguous

2017-12-19 Thread Anatoly Burakov
This will be helpful down the line when we implement support for allocating physically contiguous memory. We can no longer guarantee physically contiguous memory unless we're in IOVA_AS_VA mode, but we can certainly try and see if we succeed. In addition, this would be useful for e.g. PMD's who may

[dpdk-dev] [RFC 10/23] eal: populate hugepage counts from socket-specific sysfs path

2017-12-19 Thread Anatoly Burakov
For non-legacy memory init mode, instead of looking at generic sysfs path, look at sysfs paths pertaining to each NUMA node for hugepage counts. Note that per-NUMA node path does not provide information regarding reserved pages, so we might not get the best info from these paths, but this saves us

[dpdk-dev] [PATCH 09/23] eal: add "legacy memory" option

2017-12-19 Thread Anatoly Burakov
This adds a "--legacy-mem" command-line switch. It will be used to go back to the old memory behavior, one where we can't dynamically allocate/free memory (the downside), but one where the user can get physically contiguous memory, like before (the upside). For now, nothing but the legacy behavior

[dpdk-dev] [RFC 14/23] eal/memory: add support for dynamic unmapping of pages

2017-12-19 Thread Anatoly Burakov
This isn't used anywhere yet, but the support is now there. Also, adding cleanup to allocation procedures, so that if we fail to allocate everything we asked for, we can free all of it back. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_memalloc.h | 3 + lib/librte_eal/lin

[dpdk-dev] [PATCH 03/23] eal: add rte_fbarray

2017-12-19 Thread Anatoly Burakov
rte_fbarray is a simple resizable array, not unlike vectors in higher-level languages. Rationale for its existence is the following: since we are going to map memory page-by-page, there could be quite a lot of memory segments to keep track of (for smaller page sizes, page count can easily reach tho

[dpdk-dev] [RFC 06/23] eal/malloc: make malloc a doubly-linked list

2017-12-19 Thread Anatoly Burakov
As we are preparing for dynamic memory allocation, we need to be able to handle holes in our malloc heap, hence we're switching to doubly linked list, and prepare infrastructure to support it. Since our heap is now aware where are our first and last elements, there is no longer any need to have a

[dpdk-dev] [PATCH 05/23] eal: protect malloc heap stats with a lock

2017-12-19 Thread Anatoly Burakov
This does not change the public API, as this API is not meant to be called directly. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/malloc_heap.c | 7 ++- lib/librte_eal/common/malloc_heap.h | 2 +- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/commo

[dpdk-dev] [RFC 07/23] eal/malloc: make malloc_elem_join_adjacent_free public

2017-12-19 Thread Anatoly Burakov
We need this function to join newly allocated segments with the heap. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/malloc_elem.c | 6 +++--- lib/librte_eal/common/malloc_elem.h | 3 +++ 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/librte_eal/common/malloc_elem.c

[dpdk-dev] [PATCH 19/23] eal: enable reserving physically contiguous memzones

2017-12-19 Thread Anatoly Burakov
This adds a new set of _contig API's to rte_memzone. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_common_memzone.c | 44 lib/librte_eal/common/include/rte_memzone.h | 158 2 files changed, 202 insertions(+) diff --git a/lib/librte_eal/comm

[dpdk-dev] [PATCH 12/23] eal: add support for dynamic memory allocation

2017-12-19 Thread Anatoly Burakov
Nothing uses that code yet. The bulk of it is copied from old memory allocation stuff (eal_memory.c). We provide an API to allocate either one page or multiple pages, guaranteeing that we'll get contiguous VA for all of the pages that we requested. Signed-off-by: Anatoly Burakov --- lib/librte_e

[dpdk-dev] [PATCH 20/23] eal: make memzones use rte_fbarray

2017-12-19 Thread Anatoly Burakov
We greatly expand memzone list, and it makes some operations faster. Plus, it's there, so we might as well use it. As part of this commit, a potential memory leak is fixed (when we allocate a memzone but there's no room in config, we don't free it back), and there's a compile fix for ENA driver.

[dpdk-dev] [PATCH 14/23] eal: add support for dynamic unmapping of pages

2017-12-19 Thread Anatoly Burakov
This isn't used anywhere yet, but the support is now there. Also, adding cleanup to allocation procedures, so that if we fail to allocate everything we asked for, we can free all of it back. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_memalloc.h | 3 + lib/librte_eal/lin

[dpdk-dev] [RFC 13/23] eal/memory: make use of dynamic memory allocation for init

2017-12-19 Thread Anatoly Burakov
Add a new (non-legacy) memory init path for EAL. It uses the new dynamic allocation facilities, although it's only being run at startup. If no -m or --socket-mem switches were specified, the new init will not allocate anything, whereas if those switches were passed, appropriate amounts of pages wo

[dpdk-dev] [PATCH 18/23] eal: add rte_malloc support for allocating contiguous memory

2017-12-19 Thread Anatoly Burakov
This adds a new set of _contig API's to rte_malloc. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/include/rte_malloc.h | 181 + lib/librte_eal/common/rte_malloc.c | 63 ++ 2 files changed, 244 insertions(+) diff --git a/lib/librte_eal/comm

[dpdk-dev] [RFC 12/23] eal/memalloc: add support for dynamic memory allocation

2017-12-19 Thread Anatoly Burakov
Nothing uses that code yet. The bulk of it is copied from old memory allocation stuff (eal_memory.c). We provide an API to allocate either one page or multiple pages, guaranteeing that we'll get contiguous VA for all of the pages that we requested. Signed-off-by: Anatoly Burakov --- lib/librte_e

[dpdk-dev] [RFC 20/23] eal/memzone: make memzones use rte_fbarray

2017-12-19 Thread Anatoly Burakov
We greatly expand memzone list, and it makes some operations faster. Plus, it's there, so we might as well use it. As part of this commit, a potential memory leak is fixed (when we allocate a memzone but there's no room in config, we don't free it back), and there's a compile fix for ENA driver.

[dpdk-dev] [RFC 18/23] eal/malloc: add rte_malloc support for allocating contiguous memory

2017-12-19 Thread Anatoly Burakov
This adds a new set of _contig API's to rte_malloc. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/include/rte_malloc.h | 181 + lib/librte_eal/common/rte_malloc.c | 63 ++ 2 files changed, 244 insertions(+) diff --git a/lib/librte_eal/comm

[dpdk-dev] [RFC 16/23] eal/malloc: enable dynamic memory allocation/free on malloc/free

2017-12-19 Thread Anatoly Burakov
This set of changes enables rte_malloc to allocate and free memory as needed. The way it works is, first malloc checks if there is enough memory already allocated to satisfy user's request. If there isn't, we try and allocate more memory. The reverse happens with free - we free an element, check it

[dpdk-dev] [PATCH 13/23] eal: make use of dynamic memory allocation for init

2017-12-19 Thread Anatoly Burakov
Add a new (non-legacy) memory init path for EAL. It uses the new dynamic allocation facilities, although it's only being run at startup. If no -m or --socket-mem switches were specified, the new init will not allocate anything, whereas if those switches were passed, appropriate amounts of pages wo

[dpdk-dev] [PATCH 17/23] eal: add backend support for contiguous memory allocation

2017-12-19 Thread Anatoly Burakov
No major changes, just add some checks in a few key places, and a new parameter to pass around. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_common_memzone.c | 16 +++-- lib/librte_eal/common/malloc_elem.c| 105 +++-- lib/librte_eal/common/malloc_

[dpdk-dev] [RFC 17/23] eal/malloc: add backend support for contiguous memory allocation

2017-12-19 Thread Anatoly Burakov
No major changes, just add some checks in a few key places, and a new parameter to pass around. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_common_memzone.c | 16 +++-- lib/librte_eal/common/malloc_elem.c| 105 +++-- lib/librte_eal/common/malloc_

[dpdk-dev] [PATCH 11/23] eal: replace memseg with memseg lists

2017-12-19 Thread Anatoly Burakov
Before, we were aggregating multiple pages into one memseg, so the number of memsegs was small. Now, each page gets its own memseg, so the list of memsegs is huge. To accommodate the new memseg list size and to keep the under-the-hood workings sane, the memseg list is now not just a single list, bu

[dpdk-dev] [RFC 23/23] eal/memalloc: register/unregister memory with VFIO when alloc/free pages

2017-12-19 Thread Anatoly Burakov
Signed-off-by: Anatoly Burakov --- lib/librte_eal/linuxapp/eal/eal_memalloc.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/lib/librte_eal/linuxapp/eal/eal_memalloc.c b/lib/librte_eal/linuxapp/eal/eal_memalloc.c index 13172a0..8b3f219 100755 --- a/lib/librte_eal/linuxapp/eal/e

[dpdk-dev] [PATCH 23/23] eal: map/unmap memory with VFIO when alloc/free pages

2017-12-19 Thread Anatoly Burakov
Signed-off-by: Anatoly Burakov --- lib/librte_eal/linuxapp/eal/eal_memalloc.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/lib/librte_eal/linuxapp/eal/eal_memalloc.c b/lib/librte_eal/linuxapp/eal/eal_memalloc.c index 13172a0..8b3f219 100755 --- a/lib/librte_eal/linuxapp/eal/e

[dpdk-dev] [PATCH 22/23] vfio: allow to map other memory regions

2017-12-19 Thread Anatoly Burakov
Currently it is not possible to use memory that is not owned by DPDK to perform DMA. This scenarion might be used in vhost applications (like SPDK) where guest send its own memory table. To fill this gap provide API to allow registering arbitrary address in VFIO container. Signed-off-by: Pawel Wod

[dpdk-dev] [PATCH 21/23] mempool: add support for the new memory allocation methods

2017-12-19 Thread Anatoly Burakov
If a user has specified that the zone should have contiguous memory, use the new _contig allocation API's instead of normal ones. Otherwise, account for the fact that unless we're in IOVA_AS_VA mode, we cannot guarantee that the pages would be physically contiguous, so we calculate the memzone size

[dpdk-dev] [RFC 21/23] lib/mempool: add support for the new memory allocation methods

2017-12-19 Thread Anatoly Burakov
If a user has specified that the zone should have contiguous memory, use the new _contig allocation API's instead of normal ones. Otherwise, account for the fact that unless we're in IOVA_AS_VA mode, we cannot guarantee that the pages would be physically contiguous, so we calculate the memzone size

[dpdk-dev] [RFC 19/23] eal/memzone: add support for reserving physically contiguous memzones

2017-12-19 Thread Anatoly Burakov
This adds a new set of _contig API's to rte_memzone. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_common_memzone.c | 44 lib/librte_eal/common/include/rte_memzone.h | 158 2 files changed, 202 insertions(+) diff --git a/lib/librte_eal/comm

[dpdk-dev] [PATCH 16/23] eal: enable dynamic memory allocation/free on malloc/free

2017-12-19 Thread Anatoly Burakov
This set of changes enables rte_malloc to allocate and free memory as needed. The way it works is, first malloc checks if there is enough memory already allocated to satisfy user's request. If there isn't, we try and allocate more memory. The reverse happens with free - we free an element, check it

Re: [dpdk-dev] [PATCH 00/11] net/vhostpci: A new vhostpci PMD supporting VM2VM scenario

2017-12-19 Thread Maxime Coquelin
Hi Zhiyong, On 11/30/2017 10:46 AM, Zhiyong Yang wrote: Vhostpci PMD is a new type driver working in guest OS which has ability to drive the vhostpci modern pci device, which is a new virtio device. The following linking is about vhostpci design: An initial device design is presented at KVM Fo

[dpdk-dev] [RFC v2 00/23] Dynamic memory allocation for DPDK

2017-12-19 Thread Anatoly Burakov
This patchset introduces a prototype implementation of dynamic memory allocation for DPDK. It is intended to start a conversation and build consensus on the best way to implement this functionality. The patchset works well enough to pass all unit tests, and to work with traffic forwarding, provided

[dpdk-dev] [RFC v2 05/23] eal: protect malloc heap stats with a lock

2017-12-19 Thread Anatoly Burakov
This does not change the public API, as this API is not meant to be called directly. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/malloc_heap.c | 7 ++- lib/librte_eal/common/malloc_heap.h | 2 +- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/lib/librte_eal/commo

[dpdk-dev] [RFC v2 01/23] eal: move get_virtual_area out of linuxapp eal_memory.c

2017-12-19 Thread Anatoly Burakov
Move get_virtual_area out of linuxapp EAL memory and make it common to EAL, so that other code could reserve virtual areas as well. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_common_memory.c | 70 ++ lib/librte_eal/common/eal_private.h | 29 +++

[dpdk-dev] [RFC v2 08/23] eal: add "single file segments" command-line option

2017-12-19 Thread Anatoly Burakov
For now, this option does nothing, but it will be useful in dynamic memory allocation down the line. Currently, DPDK stores all pages as separate files in hugetlbfs. This option will allow storing all pages in one file (one file per socket, per page size). Signed-off-by: Anatoly Burakov --- lib/

[dpdk-dev] [RFC v2 02/23] eal: add function to report number of detected sockets

2017-12-19 Thread Anatoly Burakov
At the moment, we always rely on scanning everything for every socket up until RTE_MAX_NUMA_NODES and checking if there's a memseg associated with each socket if we want to find out how many sockets we actually have. This becomes a problem when we may have memory on socket but it's not allocated ye

[dpdk-dev] [RFC v2 04/23] eal: move all locking to heap

2017-12-19 Thread Anatoly Burakov
Down the line, we will need to do everything from the heap as any alloc or free may trigger alloc/free OS memory, which would involve growing/shrinking heap. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/malloc_elem.c | 16 ++-- lib/librte_eal/common/malloc_heap.c | 36 +++

[dpdk-dev] [RFC v2 07/23] eal: make malloc_elem_join_adjacent_free public

2017-12-19 Thread Anatoly Burakov
We need this function to join newly allocated segments with the heap. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/malloc_elem.c | 6 +++--- lib/librte_eal/common/malloc_elem.h | 3 +++ 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/librte_eal/common/malloc_elem.c

[dpdk-dev] [RFC v2 09/23] eal: add "legacy memory" option

2017-12-19 Thread Anatoly Burakov
This adds a "--legacy-mem" command-line switch. It will be used to go back to the old memory behavior, one where we can't dynamically allocate/free memory (the downside), but one where the user can get physically contiguous memory, like before (the upside). For now, nothing but the legacy behavior

[dpdk-dev] [RFC v2 14/23] eal: add support for dynamic unmapping of pages

2017-12-19 Thread Anatoly Burakov
This isn't used anywhere yet, but the support is now there. Also, adding cleanup to allocation procedures, so that if we fail to allocate everything we asked for, we can free all of it back. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_memalloc.h | 3 + lib/librte_eal/lin

[dpdk-dev] [RFC v2 12/23] eal: add support for dynamic memory allocation

2017-12-19 Thread Anatoly Burakov
Nothing uses that code yet. The bulk of it is copied from old memory allocation stuff (eal_memory.c). We provide an API to allocate either one page or multiple pages, guaranteeing that we'll get contiguous VA for all of the pages that we requested. Signed-off-by: Anatoly Burakov --- lib/librte_e

[dpdk-dev] [RFC v2 15/23] eal: add API to check if memory is physically contiguous

2017-12-19 Thread Anatoly Burakov
This will be helpful down the line when we implement support for allocating physically contiguous memory. We can no longer guarantee physically contiguous memory unless we're in IOVA_AS_VA mode, but we can certainly try and see if we succeed. In addition, this would be useful for e.g. PMD's who may

[dpdk-dev] [RFC v2 10/23] eal: read hugepage counts from node-specific sysfs path

2017-12-19 Thread Anatoly Burakov
For non-legacy memory init mode, instead of looking at generic sysfs path, look at sysfs paths pertaining to each NUMA node for hugepage counts. Note that per-NUMA node path does not provide information regarding reserved pages, so we might not get the best info from these paths, but this saves us

[dpdk-dev] [RFC v2 18/23] eal: add rte_malloc support for allocating contiguous memory

2017-12-19 Thread Anatoly Burakov
This adds a new set of _contig API's to rte_malloc. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/include/rte_malloc.h | 181 + lib/librte_eal/common/rte_malloc.c | 63 ++ 2 files changed, 244 insertions(+) diff --git a/lib/librte_eal/comm

[dpdk-dev] [RFC v2 13/23] eal: make use of dynamic memory allocation for init

2017-12-19 Thread Anatoly Burakov
Add a new (non-legacy) memory init path for EAL. It uses the new dynamic allocation facilities, although it's only being run at startup. If no -m or --socket-mem switches were specified, the new init will not allocate anything, whereas if those switches were passed, appropriate amounts of pages wo

[dpdk-dev] [RFC v2 21/23] mempool: add support for the new memory allocation methods

2017-12-19 Thread Anatoly Burakov
If a user has specified that the zone should have contiguous memory, use the new _contig allocation API's instead of normal ones. Otherwise, account for the fact that unless we're in IOVA_AS_VA mode, we cannot guarantee that the pages would be physically contiguous, so we calculate the memzone size

[dpdk-dev] [RFC v2 03/23] eal: add rte_fbarray

2017-12-19 Thread Anatoly Burakov
rte_fbarray is a simple resizable array, not unlike vectors in higher-level languages. Rationale for its existence is the following: since we are going to map memory page-by-page, there could be quite a lot of memory segments to keep track of (for smaller page sizes, page count can easily reach tho

[dpdk-dev] [RFC v2 11/23] eal: replace memseg with memseg lists

2017-12-19 Thread Anatoly Burakov
Before, we were aggregating multiple pages into one memseg, so the number of memsegs was small. Now, each page gets its own memseg, so the list of memsegs is huge. To accommodate the new memseg list size and to keep the under-the-hood workings sane, the memseg list is now not just a single list, bu

[dpdk-dev] [RFC v2 06/23] eal: make malloc a doubly-linked list

2017-12-19 Thread Anatoly Burakov
As we are preparing for dynamic memory allocation, we need to be able to handle holes in our malloc heap, hence we're switching to doubly linked list, and prepare infrastructure to support it. Since our heap is now aware where are our first and last elements, there is no longer any need to have a

[dpdk-dev] [RFC v2 20/23] eal: make memzones use rte_fbarray

2017-12-19 Thread Anatoly Burakov
We greatly expand memzone list, and it makes some operations faster. Plus, it's there, so we might as well use it. As part of this commit, a potential memory leak is fixed (when we allocate a memzone but there's no room in config, we don't free it back), and there's a compile fix for ENA driver.

[dpdk-dev] [RFC v2 17/23] eal: add backend support for contiguous memory allocation

2017-12-19 Thread Anatoly Burakov
No major changes, just add some checks in a few key places, and a new parameter to pass around. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_common_memzone.c | 16 +++-- lib/librte_eal/common/malloc_elem.c| 105 +++-- lib/librte_eal/common/malloc_

[dpdk-dev] [RFC v2 23/23] eal: map/unmap memory with VFIO when alloc/free pages

2017-12-19 Thread Anatoly Burakov
Signed-off-by: Anatoly Burakov --- lib/librte_eal/linuxapp/eal/eal_memalloc.c | 11 +++ 1 file changed, 11 insertions(+) diff --git a/lib/librte_eal/linuxapp/eal/eal_memalloc.c b/lib/librte_eal/linuxapp/eal/eal_memalloc.c index 13172a0..8b3f219 100755 --- a/lib/librte_eal/linuxapp/eal/e

[dpdk-dev] [RFC v2 19/23] eal: enable reserving physically contiguous memzones

2017-12-19 Thread Anatoly Burakov
This adds a new set of _contig API's to rte_memzone. Signed-off-by: Anatoly Burakov --- lib/librte_eal/common/eal_common_memzone.c | 44 lib/librte_eal/common/include/rte_memzone.h | 158 2 files changed, 202 insertions(+) diff --git a/lib/librte_eal/comm

[dpdk-dev] [RFC v2 16/23] eal: enable dynamic memory allocation/free on malloc/free

2017-12-19 Thread Anatoly Burakov
This set of changes enables rte_malloc to allocate and free memory as needed. The way it works is, first malloc checks if there is enough memory already allocated to satisfy user's request. If there isn't, we try and allocate more memory. The reverse happens with free - we free an element, check it

[dpdk-dev] [RFC v2 22/23] vfio: allow to map other memory regions

2017-12-19 Thread Anatoly Burakov
Currently it is not possible to use memory that is not owned by DPDK to perform DMA. This scenarion might be used in vhost applications (like SPDK) where guest send its own memory table. To fill this gap provide API to allow registering arbitrary address in VFIO container. Signed-off-by: Pawel Wod

Re: [dpdk-dev] [PATCH 00/23] Dynamic memory allocation for DPDK

2017-12-19 Thread Burakov, Anatoly
On 19-Dec-17 11:04 AM, Anatoly Burakov wrote: This patchset introduces a prototype implementation of dynamic memory allocation for DPDK. It is intended to start a conversation and build consensus on the best way to implement this functionality. The patchset works well enough to pass all unit test

Re: [dpdk-dev] [RFC 01/23] eal/memory: move get_virtual_area out of linuxapp eal_memory.c

2017-12-19 Thread Burakov, Anatoly
On 19-Dec-17 11:04 AM, Anatoly Burakov wrote: Move get_virtual_area out of linuxapp EAL memory and make it common to EAL, so that other code could reserve virtual areas as well. Signed-off-by: Anatoly Burakov --- My apologies, this patchset was sent erroneously. Please look at the version ta

  1   2   3   >