Re: [lng-odp] [API-NEXT PATCH v2 05/10] linux-gen: pool: reimplement pool with ring

[Bill Fischofer]

On Fri, Oct 21, 2016 at 3:36 PM, Bill Fischofer 
wrote:

>
>
> On Fri, Oct 21, 2016 at 3:25 PM, Bill Fischofer  > wrote:
>
>>
>>
>> On Fri, Oct 21, 2016 at 6:15 AM, Petri Savolainen <
>> petri.savolai...@nokia.com> wrote:
>>
>>> Used the ring data structure to implement pool. Also
>>> buffer structure was simplified to enable future driver
>>> interface. Every buffer includes a packet header, so each
>>> buffer can be used as a packet head or segment. Segmentation
>>> was disabled and segment size was fixed to a large number
>>> (64kB) to limit the number of modification in the commit.
>>>
>>> Signed-off-by: Petri Savolainen 
>>> ---
>>>  .../include/odp/api/plat/pool_types.h  |6 -
>>>  .../linux-generic/include/odp_buffer_inlines.h |  160 +--
>>>  .../linux-generic/include/odp_buffer_internal.h|  104 +-
>>>  .../include/odp_classification_datamodel.h |2 +-
>>>  .../linux-generic/include/odp_config_internal.h|   34 +-
>>>  .../linux-generic/include/odp_packet_internal.h|   13 +-
>>>  platform/linux-generic/include/odp_pool_internal.h |  270 +---
>>>  .../linux-generic/include/odp_timer_internal.h |4 -
>>>  platform/linux-generic/odp_buffer.c|8 -
>>>  platform/linux-generic/odp_classification.c|   25 +-
>>>  platform/linux-generic/odp_crypto.c|4 +-
>>>  platform/linux-generic/odp_packet.c|   99 +-
>>>  platform/linux-generic/odp_pool.c  | 1441
>>> 
>>>  platform/linux-generic/odp_timer.c |1 +
>>>  platform/linux-generic/pktio/socket.c  |   16 +-
>>>  platform/linux-generic/pktio/socket_mmap.c |   10 +-
>>>  test/common_plat/performance/odp_pktio_perf.c  |2 +-
>>>  test/common_plat/performance/odp_scheduling.c  |8 +-
>>>  test/common_plat/validation/api/packet/packet.c|8 +-
>>>  19 files changed, 746 insertions(+), 1469 deletions(-)
>>>
>>> diff --git a/platform/linux-generic/include/odp/api/plat/pool_types.h
>>> b/platform/linux-generic/include/odp/api/plat/pool_types.h
>>> index 1ca8f02..4e39de5 100644
>>> --- a/platform/linux-generic/include/odp/api/plat/pool_types.h
>>> +++ b/platform/linux-generic/include/odp/api/plat/pool_types.h
>>> @@ -39,12 +39,6 @@ typedef enum odp_pool_type_t {
>>> ODP_POOL_TIMEOUT = ODP_EVENT_TIMEOUT,
>>>  } odp_pool_type_t;
>>>
>>> -/** Get printable format of odp_pool_t */
>>> -static inline uint64_t odp_pool_to_u64(odp_pool_t hdl)
>>> -{
>>> -   return _odp_pri(hdl);
>>> -}
>>> -
>>>  /**
>>>   * @}
>>>   */
>>> diff --git a/platform/linux-generic/include/odp_buffer_inlines.h
>>> b/platform/linux-generic/include/odp_buffer_inlines.h
>>> index 2b1ab42..2f5eb88 100644
>>> --- a/platform/linux-generic/include/odp_buffer_inlines.h
>>> +++ b/platform/linux-generic/include/odp_buffer_inlines.h
>>> @@ -18,43 +18,20 @@ extern "C" {
>>>  #endif
>>>
>>>  #include 
>>> -#include 
>>>
>>> -static inline odp_buffer_t odp_buffer_encode_handle(odp_buffer_hdr_t
>>> *hdr)
>>> -{
>>> -   odp_buffer_bits_t handle;
>>> -   uint32_t pool_id = pool_handle_to_index(hdr->pool_hdl);
>>> -   struct pool_entry_s *pool = get_pool_entry(pool_id);
>>> +odp_event_type_t _odp_buffer_event_type(odp_buffer_t buf);
>>> +void _odp_buffer_event_type_set(odp_buffer_t buf, int ev);
>>> +int odp_buffer_snprint(char *str, uint32_t n, odp_buffer_t buf);
>>>
>>> -   handle.handle = 0;
>>> -   handle.pool_id = pool_id;
>>> -   handle.index = ((uint8_t *)hdr - pool->pool_mdata_addr) /
>>> -   ODP_CACHE_LINE_SIZE;
>>> -   handle.seg = 0;
>>> -
>>> -   return handle.handle;
>>> -}
>>> +void *buffer_map(odp_buffer_hdr_t *buf, uint32_t offset, uint32_t
>>> *seglen,
>>> +uint32_t limit);
>>>
>>>  static inline odp_buffer_t odp_hdr_to_buf(odp_buffer_hdr_t *hdr)
>>>  {
>>> return hdr->handle.handle;
>>>  }
>>>
>>> -static inline odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf)
>>> -{
>>> -   odp_buffer_bits_t handle;
>>> -   uint32_t pool_id;
>>> -   uint32_t index;
>>> -   struct pool_entry_s *pool;
>>> -
>>> -   handle.handle = buf;
>>> -   pool_id   = handle.pool_id;
>>> -   index = handle.index;
>>> -   pool  = get_pool_entry(pool_id);
>>> -
>>> -   return (odp_buffer_hdr_t *)(void *)
>>> -   (pool->pool_mdata_addr + (index * ODP_CACHE_LINE_SIZE));
>>> -}
>>> +odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf);
>>>
>>>  static inline uint32_t pool_id_from_buf(odp_buffer_t buf)
>>>  {
>>> @@ -64,131 +41,6 @@ static inline uint32_t pool_id_from_buf(odp_buffer_t
>>> buf)
>>> return handle.pool_id;
>>>  }
>>>
>>> -static inline odp_buffer_hdr_t *validate_buf(odp_buffer_t buf)
>>> -{
>>> -   odp_buffer_bits_t handle;
>>> -   odp_buffer_hdr_t *buf_hdr;
>>> -   handle.handle = buf;
>>> -
>>> -   /* For buffer handles, segment index must be 0 and po

Re: [lng-odp] [API-NEXT PATCH v2 05/10] linux-gen: pool: reimplement pool with ring

[Bill Fischofer]

On Fri, Oct 21, 2016 at 3:25 PM, Bill Fischofer 
wrote:

>
>
> On Fri, Oct 21, 2016 at 6:15 AM, Petri Savolainen <
> petri.savolai...@nokia.com> wrote:
>
>> Used the ring data structure to implement pool. Also
>> buffer structure was simplified to enable future driver
>> interface. Every buffer includes a packet header, so each
>> buffer can be used as a packet head or segment. Segmentation
>> was disabled and segment size was fixed to a large number
>> (64kB) to limit the number of modification in the commit.
>>
>> Signed-off-by: Petri Savolainen 
>> ---
>>  .../include/odp/api/plat/pool_types.h  |6 -
>>  .../linux-generic/include/odp_buffer_inlines.h |  160 +--
>>  .../linux-generic/include/odp_buffer_internal.h|  104 +-
>>  .../include/odp_classification_datamodel.h |2 +-
>>  .../linux-generic/include/odp_config_internal.h|   34 +-
>>  .../linux-generic/include/odp_packet_internal.h|   13 +-
>>  platform/linux-generic/include/odp_pool_internal.h |  270 +---
>>  .../linux-generic/include/odp_timer_internal.h |4 -
>>  platform/linux-generic/odp_buffer.c|8 -
>>  platform/linux-generic/odp_classification.c|   25 +-
>>  platform/linux-generic/odp_crypto.c|4 +-
>>  platform/linux-generic/odp_packet.c|   99 +-
>>  platform/linux-generic/odp_pool.c  | 1441
>> 
>>  platform/linux-generic/odp_timer.c |1 +
>>  platform/linux-generic/pktio/socket.c  |   16 +-
>>  platform/linux-generic/pktio/socket_mmap.c |   10 +-
>>  test/common_plat/performance/odp_pktio_perf.c  |2 +-
>>  test/common_plat/performance/odp_scheduling.c  |8 +-
>>  test/common_plat/validation/api/packet/packet.c|8 +-
>>  19 files changed, 746 insertions(+), 1469 deletions(-)
>>
>> diff --git a/platform/linux-generic/include/odp/api/plat/pool_types.h
>> b/platform/linux-generic/include/odp/api/plat/pool_types.h
>> index 1ca8f02..4e39de5 100644
>> --- a/platform/linux-generic/include/odp/api/plat/pool_types.h
>> +++ b/platform/linux-generic/include/odp/api/plat/pool_types.h
>> @@ -39,12 +39,6 @@ typedef enum odp_pool_type_t {
>> ODP_POOL_TIMEOUT = ODP_EVENT_TIMEOUT,
>>  } odp_pool_type_t;
>>
>> -/** Get printable format of odp_pool_t */
>> -static inline uint64_t odp_pool_to_u64(odp_pool_t hdl)
>> -{
>> -   return _odp_pri(hdl);
>> -}
>> -
>>  /**
>>   * @}
>>   */
>> diff --git a/platform/linux-generic/include/odp_buffer_inlines.h
>> b/platform/linux-generic/include/odp_buffer_inlines.h
>> index 2b1ab42..2f5eb88 100644
>> --- a/platform/linux-generic/include/odp_buffer_inlines.h
>> +++ b/platform/linux-generic/include/odp_buffer_inlines.h
>> @@ -18,43 +18,20 @@ extern "C" {
>>  #endif
>>
>>  #include 
>> -#include 
>>
>> -static inline odp_buffer_t odp_buffer_encode_handle(odp_buffer_hdr_t
>> *hdr)
>> -{
>> -   odp_buffer_bits_t handle;
>> -   uint32_t pool_id = pool_handle_to_index(hdr->pool_hdl);
>> -   struct pool_entry_s *pool = get_pool_entry(pool_id);
>> +odp_event_type_t _odp_buffer_event_type(odp_buffer_t buf);
>> +void _odp_buffer_event_type_set(odp_buffer_t buf, int ev);
>> +int odp_buffer_snprint(char *str, uint32_t n, odp_buffer_t buf);
>>
>> -   handle.handle = 0;
>> -   handle.pool_id = pool_id;
>> -   handle.index = ((uint8_t *)hdr - pool->pool_mdata_addr) /
>> -   ODP_CACHE_LINE_SIZE;
>> -   handle.seg = 0;
>> -
>> -   return handle.handle;
>> -}
>> +void *buffer_map(odp_buffer_hdr_t *buf, uint32_t offset, uint32_t
>> *seglen,
>> +uint32_t limit);
>>
>>  static inline odp_buffer_t odp_hdr_to_buf(odp_buffer_hdr_t *hdr)
>>  {
>> return hdr->handle.handle;
>>  }
>>
>> -static inline odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf)
>> -{
>> -   odp_buffer_bits_t handle;
>> -   uint32_t pool_id;
>> -   uint32_t index;
>> -   struct pool_entry_s *pool;
>> -
>> -   handle.handle = buf;
>> -   pool_id   = handle.pool_id;
>> -   index = handle.index;
>> -   pool  = get_pool_entry(pool_id);
>> -
>> -   return (odp_buffer_hdr_t *)(void *)
>> -   (pool->pool_mdata_addr + (index * ODP_CACHE_LINE_SIZE));
>> -}
>> +odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf);
>>
>>  static inline uint32_t pool_id_from_buf(odp_buffer_t buf)
>>  {
>> @@ -64,131 +41,6 @@ static inline uint32_t pool_id_from_buf(odp_buffer_t
>> buf)
>> return handle.pool_id;
>>  }
>>
>> -static inline odp_buffer_hdr_t *validate_buf(odp_buffer_t buf)
>> -{
>> -   odp_buffer_bits_t handle;
>> -   odp_buffer_hdr_t *buf_hdr;
>> -   handle.handle = buf;
>> -
>> -   /* For buffer handles, segment index must be 0 and pool id in
>> range */
>> -   if (handle.seg != 0 || handle.pool_id >= ODP_CONFIG_POOLS)
>> -   return NULL;
>> -
>> -   pool_entry_t *pool =
>> -   odp_pool_t

Re: [lng-odp] [API-NEXT PATCH v2 05/10] linux-gen: pool: reimplement pool with ring

[Bill Fischofer]

On Fri, Oct 21, 2016 at 6:15 AM, Petri Savolainen <
petri.savolai...@nokia.com> wrote:

> Used the ring data structure to implement pool. Also
> buffer structure was simplified to enable future driver
> interface. Every buffer includes a packet header, so each
> buffer can be used as a packet head or segment. Segmentation
> was disabled and segment size was fixed to a large number
> (64kB) to limit the number of modification in the commit.
>
> Signed-off-by: Petri Savolainen 
> ---
>  .../include/odp/api/plat/pool_types.h  |6 -
>  .../linux-generic/include/odp_buffer_inlines.h |  160 +--
>  .../linux-generic/include/odp_buffer_internal.h|  104 +-
>  .../include/odp_classification_datamodel.h |2 +-
>  .../linux-generic/include/odp_config_internal.h|   34 +-
>  .../linux-generic/include/odp_packet_internal.h|   13 +-
>  platform/linux-generic/include/odp_pool_internal.h |  270 +---
>  .../linux-generic/include/odp_timer_internal.h |4 -
>  platform/linux-generic/odp_buffer.c|8 -
>  platform/linux-generic/odp_classification.c|   25 +-
>  platform/linux-generic/odp_crypto.c|4 +-
>  platform/linux-generic/odp_packet.c|   99 +-
>  platform/linux-generic/odp_pool.c  | 1441
> 
>  platform/linux-generic/odp_timer.c |1 +
>  platform/linux-generic/pktio/socket.c  |   16 +-
>  platform/linux-generic/pktio/socket_mmap.c |   10 +-
>  test/common_plat/performance/odp_pktio_perf.c  |2 +-
>  test/common_plat/performance/odp_scheduling.c  |8 +-
>  test/common_plat/validation/api/packet/packet.c|8 +-
>  19 files changed, 746 insertions(+), 1469 deletions(-)
>
> diff --git a/platform/linux-generic/include/odp/api/plat/pool_types.h
> b/platform/linux-generic/include/odp/api/plat/pool_types.h
> index 1ca8f02..4e39de5 100644
> --- a/platform/linux-generic/include/odp/api/plat/pool_types.h
> +++ b/platform/linux-generic/include/odp/api/plat/pool_types.h
> @@ -39,12 +39,6 @@ typedef enum odp_pool_type_t {
> ODP_POOL_TIMEOUT = ODP_EVENT_TIMEOUT,
>  } odp_pool_type_t;
>
> -/** Get printable format of odp_pool_t */
> -static inline uint64_t odp_pool_to_u64(odp_pool_t hdl)
> -{
> -   return _odp_pri(hdl);
> -}
> -
>  /**
>   * @}
>   */
> diff --git a/platform/linux-generic/include/odp_buffer_inlines.h
> b/platform/linux-generic/include/odp_buffer_inlines.h
> index 2b1ab42..2f5eb88 100644
> --- a/platform/linux-generic/include/odp_buffer_inlines.h
> +++ b/platform/linux-generic/include/odp_buffer_inlines.h
> @@ -18,43 +18,20 @@ extern "C" {
>  #endif
>
>  #include 
> -#include 
>
> -static inline odp_buffer_t odp_buffer_encode_handle(odp_buffer_hdr_t
> *hdr)
> -{
> -   odp_buffer_bits_t handle;
> -   uint32_t pool_id = pool_handle_to_index(hdr->pool_hdl);
> -   struct pool_entry_s *pool = get_pool_entry(pool_id);
> +odp_event_type_t _odp_buffer_event_type(odp_buffer_t buf);
> +void _odp_buffer_event_type_set(odp_buffer_t buf, int ev);
> +int odp_buffer_snprint(char *str, uint32_t n, odp_buffer_t buf);
>
> -   handle.handle = 0;
> -   handle.pool_id = pool_id;
> -   handle.index = ((uint8_t *)hdr - pool->pool_mdata_addr) /
> -   ODP_CACHE_LINE_SIZE;
> -   handle.seg = 0;
> -
> -   return handle.handle;
> -}
> +void *buffer_map(odp_buffer_hdr_t *buf, uint32_t offset, uint32_t *seglen,
> +uint32_t limit);
>
>  static inline odp_buffer_t odp_hdr_to_buf(odp_buffer_hdr_t *hdr)
>  {
> return hdr->handle.handle;
>  }
>
> -static inline odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf)
> -{
> -   odp_buffer_bits_t handle;
> -   uint32_t pool_id;
> -   uint32_t index;
> -   struct pool_entry_s *pool;
> -
> -   handle.handle = buf;
> -   pool_id   = handle.pool_id;
> -   index = handle.index;
> -   pool  = get_pool_entry(pool_id);
> -
> -   return (odp_buffer_hdr_t *)(void *)
> -   (pool->pool_mdata_addr + (index * ODP_CACHE_LINE_SIZE));
> -}
> +odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf);
>
>  static inline uint32_t pool_id_from_buf(odp_buffer_t buf)
>  {
> @@ -64,131 +41,6 @@ static inline uint32_t pool_id_from_buf(odp_buffer_t
> buf)
> return handle.pool_id;
>  }
>
> -static inline odp_buffer_hdr_t *validate_buf(odp_buffer_t buf)
> -{
> -   odp_buffer_bits_t handle;
> -   odp_buffer_hdr_t *buf_hdr;
> -   handle.handle = buf;
> -
> -   /* For buffer handles, segment index must be 0 and pool id in
> range */
> -   if (handle.seg != 0 || handle.pool_id >= ODP_CONFIG_POOLS)
> -   return NULL;
> -
> -   pool_entry_t *pool =
> -   odp_pool_to_entry(_odp_cast_scalar(odp_pool_t,
> -  handle.pool_id));
> -
> -   /* If pool not created, handle is invalid */
> -   if (pool->s.poo

[lng-odp] [PATCH v2 2/2] add travis.yml for gitub automation

[Mike Holmes]

allows pushes to a github clone of ODP to automatically trigger CI builds

Signed-off-by: Mike Holmes 
---
 .travis.yml | 35 +++
 1 file changed, 35 insertions(+)
 create mode 100644 .travis.yml

diff --git a/.travis.yml b/.travis.yml
new file mode 100644
index 000..0d1add3
--- /dev/null
+++ b/.travis.yml
@@ -0,0 +1,35 @@
+# Copyright (c) 2016, Linaro Limited
+# All rights reserved.
+# SPDX-License-Identifier: BSD-3-Clause
+
+
+
+language: c
+compiler: clang
+sudo: required
+
+before_install:
+- sudo apt-get -qq update
+- sudo apt-get install automake autoconf libtool libssl-dev graphviz 
mscgen doxygen
+- gem install asciidoctor
+
+#   Install cunit for the validation tests because distro version is too 
old and fails C99 compile
+- export CUNIT_VERSION=2.1-3
+- curl -sSOL 
http://sourceforge.net/projects/cunit/files/CUnit/${CUNIT_VERSION}/CUnit-${CUNIT_VERSION}.tar.bz2
+- tar -jxf *.bz2
+- cd CUnit*
+- ./bootstrap
+- ./configure --enable-debug --enable-automated --enable-basic 
--enable-console --enable-examples --enable-test
+- make
+- sudo make install
+- cd ..
+- export LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH"
+
+install:
+- ./bootstrap
+
+script:
+- ./configure --enable-test-cpp --enable-test-vald 
--enable-test-helper --enable-test-perf --enable-user-guides 
--enable-test-perf-proc --enable-test-example
+- make check
+- make doxygen-doc
+- make distcheck
-- 
2.7.4

[lng-odp] [PATCH v2 1/2] configure: the version cannot use a script

[Mike Holmes]

Use of a script in the autotools breaks github CI
Define the API version in configure and reuse it in the code

Benefits:
removes dependence on scripts to build
removes scripts and the make file from odp/scripts
removes generated .scmversion file
places api versioning right next to lib versioning in the configure.ac
allows github to auto build odp repos

Signed-off-by: Mike Holmes 
---
 .gitignore   |  1 -
 Makefile.am  |  5 ++--
 configure.ac | 21 +++--
 include/odp/api/spec/.gitignore  |  1 +
 include/odp/api/spec/{version.h => version.h.in} |  6 ++---
 platform/Makefile.inc|  3 +--
 scripts/Makefile.am  |  1 -
 scripts/get_impl_str.sh  | 11 -
 scripts/git_hash.sh  | 29 
 9 files changed, 26 insertions(+), 52 deletions(-)
 create mode 100644 include/odp/api/spec/.gitignore
 rename include/odp/api/spec/{version.h => version.h.in} (93%)
 delete mode 100644 scripts/Makefile.am
 delete mode 100755 scripts/get_impl_str.sh
 delete mode 100755 scripts/git_hash.sh

diff --git a/.gitignore b/.gitignore
index d4e7d02..fbc0eab 100644
--- a/.gitignore
+++ b/.gitignore
@@ -12,7 +12,6 @@
 .deps/
 .dirstamp
 .libs/
-.scmversion
 CUnit-Memory-Dump.xml
 ID
 Makefile
diff --git a/Makefile.am b/Makefile.am
index 625748a..2a9a658 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -11,9 +11,8 @@ SUBDIRS = @platform_with_platform@ \
  test \
  helper/test \
  doc \
- example \
- scripts
+ example
 
 @DX_RULES@
 
-EXTRA_DIST = bootstrap $(DX_CONFIG) CHANGELOG config/README .scmversion
+EXTRA_DIST = bootstrap $(DX_CONFIG) CHANGELOG config/README
diff --git a/configure.ac b/configure.ac
index f081c51..ab19084 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,5 +1,23 @@
 AC_PREREQ([2.5])
-AC_INIT([OpenDataPlane], m4_esyscmd(./scripts/git_hash.sh .), 
[lng-odp@lists.linaro.org])
+##
+# Set correct API version
+##
+m4_define([odpapi_generation_version], [1])
+m4_define([odpapi_major_version], [11])
+m4_define([odpapi_minor_version], [0])
+m4_define([odpapi_point_version], [0])
+m4_define([odpapi_version],
+
[odpapi_generation_version.odpapi_major_version.odpapi_minor_version.odpapi_point_version])
+AC_INIT([OpenDataPlane],[odpapi_version],[lng-odp@lists.linaro.org])
+
+ODP_VERSION_API_GENERATION=odpapi_generation_version
+AC_SUBST(ODP_VERSION_API_GENERATION)
+ODP_VERSION_API_MAJOR=odpapi_major_version
+AC_SUBST(ODP_VERSION_API_MAJOR)
+ODP_VERSION_API_MINOR=odpapi_minor_version
+AC_SUBST(ODP_VERSION_API_MINOR)
+AC_CONFIG_FILES([include/odp/api/spec/version.h])
+
 AM_INIT_AUTOMAKE([1.9 tar-pax subdir-objects])
 AC_CONFIG_SRCDIR([helper/config.h.in])
 AM_CONFIG_HEADER([helper/config.h])
@@ -271,7 +289,6 @@ AC_CONFIG_FILES([Makefile
 helper/test/Makefile
 pkgconfig/libodp-linux.pc
 pkgconfig/libodphelper-linux.pc
-scripts/Makefile
 ])
 
 AC_SEARCH_LIBS([timer_create],[rt posix4])
diff --git a/include/odp/api/spec/.gitignore b/include/odp/api/spec/.gitignore
new file mode 100644
index 000..6702033
--- /dev/null
+++ b/include/odp/api/spec/.gitignore
@@ -0,0 +1 @@
+version.h
diff --git a/include/odp/api/spec/version.h b/include/odp/api/spec/version.h.in
similarity index 93%
rename from include/odp/api/spec/version.h
rename to include/odp/api/spec/version.h.in
index aa3f3ab..4b16dcc 100644
--- a/include/odp/api/spec/version.h
+++ b/include/odp/api/spec/version.h.in
@@ -37,7 +37,7 @@ extern "C" {
  * very significant changes to the API. APIs with different
  * versions are likely not backward compatible.
  */
-#define ODP_VERSION_API_GENERATION 1
+#define ODP_VERSION_API_GENERATION @ODP_VERSION_API_GENERATION@
 
 /**
  * ODP API major version
@@ -45,7 +45,7 @@ extern "C" {
  * Introduction of major new features or changes. APIs with different major
  * versions are likely not backward compatible.
  */
-#define ODP_VERSION_API_MAJOR 10
+#define  ODP_VERSION_API_MAJOR @ODP_VERSION_API_MAJOR@
 
 /**
  * ODP API minor version
@@ -54,7 +54,7 @@ extern "C" {
  * to the API. For an API with common generation and major version, but with
  * different minor numbers the two versions are backward compatible.
  */
-#define ODP_VERSION_API_MINOR 1
+#define ODP_VERSION_API_MINOR @ODP_VERSION_API_MINOR@
 
 /**
  * ODP API version string
diff --git a/platform/Makefile.inc b/platform/Makefile.inc
index a44f88f..432d1fd 100644
--- a/platform/Makefile.inc
+++ b/platform/Makefile.inc
@@ -10,8 +10,7 @@ lib_LTLIBRARIES = $(LIB)/libodp-linux.la
 
 AM_LDFLAGS += -version-number '$(ODP_LIBSO_VERSION)'
 
-GIT_DESC = `$(top_s

Re: [lng-odp] [API-NEXT PATCHv6 1/5] api: packet: add support for packet splices and references

[Bill Fischofer]

On Fri, Oct 21, 2016 at 12:07 PM, Bala Manoharan 
wrote:

> Regards,
> Bala
>
>
> On 21 October 2016 at 19:41, Bill Fischofer 
> wrote:
> >
> >
> > On Fri, Oct 21, 2016 at 8:21 AM, Bala Manoharan <
> bala.manoha...@linaro.org>
> > wrote:
> >>
> >> Regards,
> >> Bala
> >>
> >>
> >> On 11 October 2016 at 09:15, Bill Fischofer 
> >> wrote:
> >> > Introduce four new APIs that support efficient sharing of portions of
> >> > packets.
> >> >
> >> > odp_packet_splice() creates a reference to a base packet by splicing a
> >> > supplied header packet onto it at a specified offset. If multiple
> >> > splices
> >> > are created then each shares the base packet that was spliced.
> >> >
> >> > odp_packet_ref() creates a reference to a base packet by splicing a
> >> > zero-length header onto it at a specified offset. This allows an
> >> > application to "hold onto" a pointer to a packet so that it will not
> be
> >> > freed until all references to it are freed.
> >> >
> >> > odp_packet_is_a_splice() allows an application to determine whether a
> >> > packet is a splice and if so how many individual packets are contained
> >> > within it.
> >> >
> >> > odp_packet_is_spliced() allows an application to determine whether a
> >> > packet
> >> > has a splice on it and if so how many splices are based on it.
> >> >
> >> > Signed-off-by: Bill Fischofer 
> >> > ---
> >> >  include/odp/api/spec/packet.h | 103
> >> > ++
> >> >  1 file changed, 103 insertions(+)
> >> >
> >> > diff --git a/include/odp/api/spec/packet.h
> >> > b/include/odp/api/spec/packet.h
> >> > index 4a14f2d..8e147a3 100644
> >> > --- a/include/odp/api/spec/packet.h
> >> > +++ b/include/odp/api/spec/packet.h
> >> > @@ -844,6 +844,109 @@ int odp_packet_concat(odp_packet_t *dst,
> >> > odp_packet_t src);
> >> >   */
> >> >  int odp_packet_split(odp_packet_t *pkt, uint32_t len, odp_packet_t
> >> > *tail);
> >> >
> >> > +/**
> >> > + * Splice a packet
> >> > + *
> >> > + * Create a new packet by splicing a header onto an existing packet.
> >> > The
> >> > + * spliced packet consists of the header followed by a shared
> reference
> >> > to the
> >> > + * base packet receiving the splice starting at a designated offset.
> If
> >> > + * multiple splices are created on the same base packet, it is the
> >> > + * application's responsibility to coordinate any changes to the
> shared
> >> > + * segment(s) of the splice, otherwise results are undefined.
> >> > + *
> >> > + * @param hdrHandle of the header packet to be spliced onto the
> >> > base packet
> >> > + *
> >> > + * @param pktHandle of the base packet that is to receive the
> >> > splice
> >> > + *
> >> > + * @param offset Byte offset within the base packet at which the
> splice
> >> > is
> >> > + *   to be made.
> >> > + *
> >> > + * @return   Handle of the spliced packet. This may or may not be
> >> > the
> >> > + *   same as the input hdr packet. The caller should use
> >> > this
> >> > + *   for any future references to the splice. The
> original
> >> > hdr
> >> > + *   packet no longer has any independent existence.
> >> > + *
> >> > + * @retval ODP_PACKET_INVALID If the splice failed. In this case both
> >> > hdr
> >> > + *and pkt are unchanged.
> >> > + *
> >> > + * @note The base pkt remains valid after the completion of the
> splice,
> >> > + *   however changes that affect its length may yield
> unpredictable
> >> > + *   results when viewed through the returned splice handle. For
> >> > best
> >> > + *   portability and predictable behavior, applications should
> >> > regard the
> >> > + *   base packet as read only following a successful splice, with
> >> > the
> >> > + *   exception of tailroom manipulation (which still requires
> >> > application
> >> > + *   coordination if multiple splices exist on the same base
> packet
> >> > whose
> >> > + *   tail is being pushed or pulled).
> >> > + *
> >> > + * @note Either packet input to this routine may itself be a splice,
> >> > however
> >> > + *   individual implementations may impose limits on how deeply
> >> > splices
> >> > + *   may be nested and fail the attempted splice if those limits
> >> > are
> >> > + *   exceeded.
> >> > + *
> >> > + * @note The packets used to create a splice may reside in different
> >> > pools,
> >> > + *   however individual implementations may require that both
> >> > reside in
> >> > + *   the same pool and fail the attempted splice if this
> >> > restriction is
> >> > + *   not observed. Upon return odp_packet_pool() returns the pool
> >> > id of
> >> > + *   the header packet.
> >> > + *
> >> > + * @note Once successfully spliced, the base packet may be freed via
> >> > + *   odp_packet_free(), however the storage used to represent it
> >> > will not
> >> > + *   be released until all splices based on it have themselves
> been
> >> 

Re: [lng-odp] [API-NEXT PATCHv6 1/5] api: packet: add support for packet splices and references

[Bala Manoharan]

Regards,
Bala


On 21 October 2016 at 19:41, Bill Fischofer  wrote:
>
>
> On Fri, Oct 21, 2016 at 8:21 AM, Bala Manoharan 
> wrote:
>>
>> Regards,
>> Bala
>>
>>
>> On 11 October 2016 at 09:15, Bill Fischofer 
>> wrote:
>> > Introduce four new APIs that support efficient sharing of portions of
>> > packets.
>> >
>> > odp_packet_splice() creates a reference to a base packet by splicing a
>> > supplied header packet onto it at a specified offset. If multiple
>> > splices
>> > are created then each shares the base packet that was spliced.
>> >
>> > odp_packet_ref() creates a reference to a base packet by splicing a
>> > zero-length header onto it at a specified offset. This allows an
>> > application to "hold onto" a pointer to a packet so that it will not be
>> > freed until all references to it are freed.
>> >
>> > odp_packet_is_a_splice() allows an application to determine whether a
>> > packet is a splice and if so how many individual packets are contained
>> > within it.
>> >
>> > odp_packet_is_spliced() allows an application to determine whether a
>> > packet
>> > has a splice on it and if so how many splices are based on it.
>> >
>> > Signed-off-by: Bill Fischofer 
>> > ---
>> >  include/odp/api/spec/packet.h | 103
>> > ++
>> >  1 file changed, 103 insertions(+)
>> >
>> > diff --git a/include/odp/api/spec/packet.h
>> > b/include/odp/api/spec/packet.h
>> > index 4a14f2d..8e147a3 100644
>> > --- a/include/odp/api/spec/packet.h
>> > +++ b/include/odp/api/spec/packet.h
>> > @@ -844,6 +844,109 @@ int odp_packet_concat(odp_packet_t *dst,
>> > odp_packet_t src);
>> >   */
>> >  int odp_packet_split(odp_packet_t *pkt, uint32_t len, odp_packet_t
>> > *tail);
>> >
>> > +/**
>> > + * Splice a packet
>> > + *
>> > + * Create a new packet by splicing a header onto an existing packet.
>> > The
>> > + * spliced packet consists of the header followed by a shared reference
>> > to the
>> > + * base packet receiving the splice starting at a designated offset. If
>> > + * multiple splices are created on the same base packet, it is the
>> > + * application's responsibility to coordinate any changes to the shared
>> > + * segment(s) of the splice, otherwise results are undefined.
>> > + *
>> > + * @param hdrHandle of the header packet to be spliced onto the
>> > base packet
>> > + *
>> > + * @param pktHandle of the base packet that is to receive the
>> > splice
>> > + *
>> > + * @param offset Byte offset within the base packet at which the splice
>> > is
>> > + *   to be made.
>> > + *
>> > + * @return   Handle of the spliced packet. This may or may not be
>> > the
>> > + *   same as the input hdr packet. The caller should use
>> > this
>> > + *   for any future references to the splice. The original
>> > hdr
>> > + *   packet no longer has any independent existence.
>> > + *
>> > + * @retval ODP_PACKET_INVALID If the splice failed. In this case both
>> > hdr
>> > + *and pkt are unchanged.
>> > + *
>> > + * @note The base pkt remains valid after the completion of the splice,
>> > + *   however changes that affect its length may yield unpredictable
>> > + *   results when viewed through the returned splice handle. For
>> > best
>> > + *   portability and predictable behavior, applications should
>> > regard the
>> > + *   base packet as read only following a successful splice, with
>> > the
>> > + *   exception of tailroom manipulation (which still requires
>> > application
>> > + *   coordination if multiple splices exist on the same base packet
>> > whose
>> > + *   tail is being pushed or pulled).
>> > + *
>> > + * @note Either packet input to this routine may itself be a splice,
>> > however
>> > + *   individual implementations may impose limits on how deeply
>> > splices
>> > + *   may be nested and fail the attempted splice if those limits
>> > are
>> > + *   exceeded.
>> > + *
>> > + * @note The packets used to create a splice may reside in different
>> > pools,
>> > + *   however individual implementations may require that both
>> > reside in
>> > + *   the same pool and fail the attempted splice if this
>> > restriction is
>> > + *   not observed. Upon return odp_packet_pool() returns the pool
>> > id of
>> > + *   the header packet.
>> > + *
>> > + * @note Once successfully spliced, the base packet may be freed via
>> > + *   odp_packet_free(), however the storage used to represent it
>> > will not
>> > + *   be released until all splices based on it have themselves been
>> > freed.
>>
>> Do you have a reason why the 'base' packet will be freed by the
>> application after creating the slice?
>
>
> The point being made here is only that the storage behind the base packet
> will not be released until all references to it are freed. The application
> is under no obligation to issue an odp_packet_free() at any particular time.
> 

Re: [lng-odp] [PATCH v3 1/5] example: l3fwd: add missing gitignores

[Mike Holmes]

On 3 October 2016 at 07:49, Matias Elo  wrote:
> Signed-off-by: Matias Elo 

Reviewed-by: Mike Holmes 

But I wonder if we should at some point  ignore these in the root of
odp/example and odp/test to save repeating them ?

> ---
>  example/l3fwd/.gitignore | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/example/l3fwd/.gitignore b/example/l3fwd/.gitignore
> index 74e501f..3411830 100644
> --- a/example/l3fwd/.gitignore
> +++ b/example/l3fwd/.gitignore
> @@ -1 +1,3 @@
>  odp_l3fwd
> +*.log
> +*.trs
> --
> 2.7.4
>



-- 
Mike Holmes
Program Manager - Linaro Networking Group
Linaro.org │ Open source software for ARM SoCs
"Work should be fun and collaborative, the rest follows"

Re: [lng-odp] [PATCHv3] test: skip pktio_perf tests on 1 and 2 cpus machines

[Mike Holmes]

On 14 October 2016 at 07:51, Maxim Uvarov  wrote:
> Make check should skip the test instead of failing it.
> Test splits RX and TX cores for packet processing. Core
> 0 bind to control thread. So running machine should have
> at least 2 worker threads which is not enough on 1 and 2
> cpus machine. CUnit uses special value 77 to mark test as
> SKIPPED and not fail on it.
>
> Signed-off-by: Maxim Uvarov 

Reviewed-by: Mike Holmes 

> ---
>
>  v3: update if logic (found just after apply.)
>  v2: update description (Mike)
>  test/common_plat/performance/odp_pktio_perf.c | 16 +++-
>  1 file changed, 11 insertions(+), 5 deletions(-)
>
> diff --git a/test/common_plat/performance/odp_pktio_perf.c 
> b/test/common_plat/performance/odp_pktio_perf.c
> index f041b13..483f067 100644
> --- a/test/common_plat/performance/odp_pktio_perf.c
> +++ b/test/common_plat/performance/odp_pktio_perf.c
> @@ -34,6 +34,8 @@
>  #include 
>  #include 
>
> +#define TEST_SKIP 77
> +
>  #define PKT_BUF_NUM   8192
>  #define MAX_NUM_IFACES2
>  #define TEST_HDR_MAGIC0x92749451
> @@ -558,7 +560,7 @@ static int setup_txrx_masks(odp_cpumask_t *thd_mask_tx,
>gbl_args->args.cpu_count);
> if (num_workers < 2) {
> LOG_ERR("Need at least two cores\n");
> -   return -1;
> +   return TEST_SKIP;
> }
>
> if (gbl_args->args.num_tx_workers) {
> @@ -659,7 +661,7 @@ static int run_test_single(odp_cpumask_t *thd_mask_tx,
>
>  static int run_test(void)
>  {
> -   int ret = 1;
> +   int ret;
> int i;
> odp_cpumask_t txmask, rxmask;
> test_status_t status = {
> @@ -669,8 +671,9 @@ static int run_test(void)
> .warmup = 1,
> };
>
> -   if (setup_txrx_masks(&txmask, &rxmask) != 0)
> -   return -1;
> +   ret = setup_txrx_masks(&txmask, &rxmask);
> +   if (ret)
> +   return ret;
>
> printf("Starting test with params:\n");
> printf("\tTransmit workers: \t%d\n", odp_cpumask_count(&txmask));
> @@ -691,8 +694,11 @@ static int run_test(void)
> run_test_single(&txmask, &rxmask, &status);
> status.warmup = 0;
>
> -   while (ret > 0)
> +   while (1) {
> ret = run_test_single(&txmask, &rxmask, &status);
> +   if (ret <= 0)
> +   break;
> +   }
>
> return ret;
>  }
> --
> 2.7.1.250.gff4ea60
>



-- 
Mike Holmes
Program Manager - Linaro Networking Group
Linaro.org │ Open source software for ARM SoCs
"Work should be fun and collaborative, the rest follows"

Re: [lng-odp] [API-NEXT PATCHv6 1/5] api: packet: add support for packet splices and references

[Bill Fischofer]

On Fri, Oct 21, 2016 at 8:21 AM, Bala Manoharan 
wrote:

> Regards,
> Bala
>
>
> On 11 October 2016 at 09:15, Bill Fischofer 
> wrote:
> > Introduce four new APIs that support efficient sharing of portions of
> > packets.
> >
> > odp_packet_splice() creates a reference to a base packet by splicing a
> > supplied header packet onto it at a specified offset. If multiple splices
> > are created then each shares the base packet that was spliced.
> >
> > odp_packet_ref() creates a reference to a base packet by splicing a
> > zero-length header onto it at a specified offset. This allows an
> > application to "hold onto" a pointer to a packet so that it will not be
> > freed until all references to it are freed.
> >
> > odp_packet_is_a_splice() allows an application to determine whether a
> > packet is a splice and if so how many individual packets are contained
> > within it.
> >
> > odp_packet_is_spliced() allows an application to determine whether a
> packet
> > has a splice on it and if so how many splices are based on it.
> >
> > Signed-off-by: Bill Fischofer 
> > ---
> >  include/odp/api/spec/packet.h | 103 ++
> 
> >  1 file changed, 103 insertions(+)
> >
> > diff --git a/include/odp/api/spec/packet.h
> b/include/odp/api/spec/packet.h
> > index 4a14f2d..8e147a3 100644
> > --- a/include/odp/api/spec/packet.h
> > +++ b/include/odp/api/spec/packet.h
> > @@ -844,6 +844,109 @@ int odp_packet_concat(odp_packet_t *dst,
> odp_packet_t src);
> >   */
> >  int odp_packet_split(odp_packet_t *pkt, uint32_t len, odp_packet_t
> *tail);
> >
> > +/**
> > + * Splice a packet
> > + *
> > + * Create a new packet by splicing a header onto an existing packet. The
> > + * spliced packet consists of the header followed by a shared reference
> to the
> > + * base packet receiving the splice starting at a designated offset. If
> > + * multiple splices are created on the same base packet, it is the
> > + * application's responsibility to coordinate any changes to the shared
> > + * segment(s) of the splice, otherwise results are undefined.
> > + *
> > + * @param hdrHandle of the header packet to be spliced onto the
> base packet
> > + *
> > + * @param pktHandle of the base packet that is to receive the splice
> > + *
> > + * @param offset Byte offset within the base packet at which the splice
> is
> > + *   to be made.
> > + *
> > + * @return   Handle of the spliced packet. This may or may not be
> the
> > + *   same as the input hdr packet. The caller should use
> this
> > + *   for any future references to the splice. The original
> hdr
> > + *   packet no longer has any independent existence.
> > + *
> > + * @retval ODP_PACKET_INVALID If the splice failed. In this case both
> hdr
> > + *and pkt are unchanged.
> > + *
> > + * @note The base pkt remains valid after the completion of the splice,
> > + *   however changes that affect its length may yield unpredictable
> > + *   results when viewed through the returned splice handle. For
> best
> > + *   portability and predictable behavior, applications should
> regard the
> > + *   base packet as read only following a successful splice, with
> the
> > + *   exception of tailroom manipulation (which still requires
> application
> > + *   coordination if multiple splices exist on the same base packet
> whose
> > + *   tail is being pushed or pulled).
> > + *
> > + * @note Either packet input to this routine may itself be a splice,
> however
> > + *   individual implementations may impose limits on how deeply
> splices
> > + *   may be nested and fail the attempted splice if those limits are
> > + *   exceeded.
> > + *
> > + * @note The packets used to create a splice may reside in different
> pools,
> > + *   however individual implementations may require that both
> reside in
> > + *   the same pool and fail the attempted splice if this
> restriction is
> > + *   not observed. Upon return odp_packet_pool() returns the pool
> id of
> > + *   the header packet.
> > + *
> > + * @note Once successfully spliced, the base packet may be freed via
> > + *   odp_packet_free(), however the storage used to represent it
> will not
> > + *   be released until all splices based on it have themselves been
> freed.
>
> Do you have a reason why the 'base' packet will be freed by the
> application after creating the slice?
>

The point being made here is only that the storage behind the base packet
will not be released until all references to it are freed. The application
is under no obligation to issue an odp_packet_free() at any particular
time. One reason why a packet (or one of its references) may be freed is if
it is transmitted since ODP TX processing implicitly calls
odp_packet_free() once the odp_packet_t has been sent. So, for example, if
a reference to a packet is transmitted then the base packet will s

Re: [lng-odp] [PATCHv3] test: skip pktio_perf tests on 1 and 2 cpus machines

[Maxim Uvarov]

please review v3.

Maxim.

On 10/14/16 14:51, Maxim Uvarov wrote:

Make check should skip the test instead of failing it.
Test splits RX and TX cores for packet processing. Core
0 bind to control thread. So running machine should have
at least 2 worker threads which is not enough on 1 and 2
cpus machine. CUnit uses special value 77 to mark test as
SKIPPED and not fail on it.

Signed-off-by: Maxim Uvarov 
---

  v3: update if logic (found just after apply.)
  v2: update description (Mike)
  test/common_plat/performance/odp_pktio_perf.c | 16 +++-
  1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/test/common_plat/performance/odp_pktio_perf.c 
b/test/common_plat/performance/odp_pktio_perf.c
index f041b13..483f067 100644
--- a/test/common_plat/performance/odp_pktio_perf.c
+++ b/test/common_plat/performance/odp_pktio_perf.c
@@ -34,6 +34,8 @@
  #include 
  #include 
  
+#define TEST_SKIP 77

+
  #define PKT_BUF_NUM   8192
  #define MAX_NUM_IFACES2
  #define TEST_HDR_MAGIC0x92749451
@@ -558,7 +560,7 @@ static int setup_txrx_masks(odp_cpumask_t *thd_mask_tx,
   gbl_args->args.cpu_count);
if (num_workers < 2) {
LOG_ERR("Need at least two cores\n");
-   return -1;
+   return TEST_SKIP;
}
  
  	if (gbl_args->args.num_tx_workers) {

@@ -659,7 +661,7 @@ static int run_test_single(odp_cpumask_t *thd_mask_tx,
  
  static int run_test(void)

  {
-   int ret = 1;
+   int ret;
int i;
odp_cpumask_t txmask, rxmask;
test_status_t status = {
@@ -669,8 +671,9 @@ static int run_test(void)
.warmup = 1,
};
  
-	if (setup_txrx_masks(&txmask, &rxmask) != 0)

-   return -1;
+   ret = setup_txrx_masks(&txmask, &rxmask);
+   if (ret)
+   return ret;
  
  	printf("Starting test with params:\n");

printf("\tTransmit workers: \t%d\n", odp_cpumask_count(&txmask));
@@ -691,8 +694,11 @@ static int run_test(void)
run_test_single(&txmask, &rxmask, &status);
status.warmup = 0;
  
-	while (ret > 0)

+   while (1) {
ret = run_test_single(&txmask, &rxmask, &status);
+   if (ret <= 0)
+   break;
+   }
  
  	return ret;

  }

Re: [lng-odp] [PATCH 02/10] linux-gen: pktio: do not free zero packets

[Maxim Uvarov]

On 10/20/16 14:52, Savolainen, Petri (Nokia - FI/Espoo) wrote:



-Original Message-
From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of Maxim
Uvarov
Sent: Wednesday, October 19, 2016 3:49 PM
To: lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [PATCH 02/10] linux-gen: pktio: do not free zero
packets

On 10/19/16 15:09, Petri Savolainen wrote:

In some error cases, netmap and dpdk pktios were calling
odp_packet_free_multi with zero packets. Moved existing error
check to avoid a free call with zero packets.

Signed-off-by: Petri Savolainen 
---
   platform/linux-generic/pktio/dpdk.c   | 10 ++
   platform/linux-generic/pktio/netmap.c | 10 ++
   2 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/platform/linux-generic/pktio/dpdk.c b/platform/linux-

generic/pktio/dpdk.c

index 11f3509..0eb025a 100644
--- a/platform/linux-generic/pktio/dpdk.c
+++ b/platform/linux-generic/pktio/dpdk.c
@@ -956,10 +956,12 @@ static int dpdk_send(pktio_entry_t *pktio_entry,

int index,

rte_pktmbuf_free(tx_mbufs[i]);
}

-   odp_packet_free_multi(pkt_table, tx_pkts);
-
-   if (odp_unlikely(tx_pkts == 0 && __odp_errno != 0))
-   return -1;
+   if (odp_unlikely(tx_pkts == 0)) {
+   if (__odp_errno != 0)
+   return -1;

no need to modify original code for 2 ifs. One if and && usually
generates faster code.

The else branch underneath is the actual bug correction ==> call free only when 
tx_pkts != 0. Errno is not always set.

-Petri
I agree with that so that no need splitting  this if on 2. You can just 
add 'else' in the bottom.


Maxim.


+   } else {
+   odp_packet_free_multi(pkt_table, tx_pkts);
+   }

return tx_pkts;
   }
diff --git a/platform/linux-generic/pktio/netmap.c b/platform/linux-

generic/pktio/netmap.c

index 412beec..c1cdf72 100644
--- a/platform/linux-generic/pktio/netmap.c
+++ b/platform/linux-generic/pktio/netmap.c
@@ -830,10 +830,12 @@ static int netmap_send(pktio_entry_t *pktio_entry,

int index,

if (!pkt_nm->lockless_tx)
odp_ticketlock_unlock(&pkt_nm->tx_desc_ring[index].s.lock);

-   odp_packet_free_multi(pkt_table, nb_tx);
-
-   if (odp_unlikely(nb_tx == 0 && __odp_errno != 0))
-   return -1;
+   if (odp_unlikely(nb_tx == 0)) {
+   if (__odp_errno != 0)
+   return -1;

same here.

Maxim.

+   } else {
+   odp_packet_free_multi(pkt_table, nb_tx);
+   }

return nb_tx;
   }

Re: [lng-odp] [PATCH 01/10] linux-gen: ipc: disable build of ipc pktio

[Maxim Uvarov]

On 10/20/16 15:00, Savolainen, Petri (Nokia - FI/Espoo) wrote:



-Original Message-
From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of Maxim
Uvarov
Sent: Wednesday, October 19, 2016 3:59 PM
To: lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [PATCH 01/10] linux-gen: ipc: disable build of ipc
pktio

On 10/19/16 15:09, Petri Savolainen wrote:

IPC pktio implementation depends heavily on pool internals. It's
build is disabled due to pool re-implementation. IPC should be
re-implemented with a cleaner internal interface towards pool and
shm.

Signed-off-by: Petri Savolainen 
---
   platform/linux-generic/Makefile.am | 2 --
   1 file changed, 2 deletions(-)

diff --git a/platform/linux-generic/Makefile.am b/platform/linux-

generic/Makefile.am

index 0ec13d4..0fba393 100644
--- a/platform/linux-generic/Makefile.am
+++ b/platform/linux-generic/Makefile.am
@@ -111,7 +111,6 @@ noinst_HEADERS = \
  ${srcdir}/include/odp_name_table_internal.h \
  ${srcdir}/include/odp_packet_internal.h \
  ${srcdir}/include/odp_packet_io_internal.h \
- ${srcdir}/include/odp_packet_io_ipc_internal.h \
  ${srcdir}/include/odp_packet_io_queue.h \
  ${srcdir}/include/odp_packet_io_ring_internal.h \
  ${srcdir}/include/odp_packet_netmap.h \
@@ -158,7 +157,6 @@ __LIB__libodp_linux_la_SOURCES = \
   odp_packet_io.c \
   pktio/ethtool.c \
   pktio/io_ops.c \
-  pktio/ipc.c \
   pktio/pktio_common.c \
   pktio/loop.c \
   pktio/netmap.c \

you just not need to turn on  -D_ODP_PKTIO_IPC from configure. If it's
not enough add ifdefs inside pktio/ipc.c.
Or with your changes tests will be built and run.

Maxim.


Entire IPC code won't build after pool structure changes. So, entire ipc.c file 
would need to be #ifdef'ed. I think this was a cleaner way to disable entirely 
the build of IPC code.

-Petri
yes. You can ifdef it. Then we merge Christophe changes and ipc repair. 
Than your changes and I will try to fix it again.


Maxim.

Re: [lng-odp] [API-NEXT PATCHv6 1/5] api: packet: add support for packet splices and references

[Bala Manoharan]

Regards,
Bala


On 11 October 2016 at 09:15, Bill Fischofer  wrote:
> Introduce four new APIs that support efficient sharing of portions of
> packets.
>
> odp_packet_splice() creates a reference to a base packet by splicing a
> supplied header packet onto it at a specified offset. If multiple splices
> are created then each shares the base packet that was spliced.
>
> odp_packet_ref() creates a reference to a base packet by splicing a
> zero-length header onto it at a specified offset. This allows an
> application to "hold onto" a pointer to a packet so that it will not be
> freed until all references to it are freed.
>
> odp_packet_is_a_splice() allows an application to determine whether a
> packet is a splice and if so how many individual packets are contained
> within it.
>
> odp_packet_is_spliced() allows an application to determine whether a packet
> has a splice on it and if so how many splices are based on it.
>
> Signed-off-by: Bill Fischofer 
> ---
>  include/odp/api/spec/packet.h | 103 
> ++
>  1 file changed, 103 insertions(+)
>
> diff --git a/include/odp/api/spec/packet.h b/include/odp/api/spec/packet.h
> index 4a14f2d..8e147a3 100644
> --- a/include/odp/api/spec/packet.h
> +++ b/include/odp/api/spec/packet.h
> @@ -844,6 +844,109 @@ int odp_packet_concat(odp_packet_t *dst, odp_packet_t 
> src);
>   */
>  int odp_packet_split(odp_packet_t *pkt, uint32_t len, odp_packet_t *tail);
>
> +/**
> + * Splice a packet
> + *
> + * Create a new packet by splicing a header onto an existing packet. The
> + * spliced packet consists of the header followed by a shared reference to 
> the
> + * base packet receiving the splice starting at a designated offset. If
> + * multiple splices are created on the same base packet, it is the
> + * application's responsibility to coordinate any changes to the shared
> + * segment(s) of the splice, otherwise results are undefined.
> + *
> + * @param hdrHandle of the header packet to be spliced onto the base 
> packet
> + *
> + * @param pktHandle of the base packet that is to receive the splice
> + *
> + * @param offset Byte offset within the base packet at which the splice is
> + *   to be made.
> + *
> + * @return   Handle of the spliced packet. This may or may not be the
> + *   same as the input hdr packet. The caller should use this
> + *   for any future references to the splice. The original hdr
> + *   packet no longer has any independent existence.
> + *
> + * @retval ODP_PACKET_INVALID If the splice failed. In this case both hdr
> + *and pkt are unchanged.
> + *
> + * @note The base pkt remains valid after the completion of the splice,
> + *   however changes that affect its length may yield unpredictable
> + *   results when viewed through the returned splice handle. For best
> + *   portability and predictable behavior, applications should regard the
> + *   base packet as read only following a successful splice, with the
> + *   exception of tailroom manipulation (which still requires application
> + *   coordination if multiple splices exist on the same base packet whose
> + *   tail is being pushed or pulled).
> + *
> + * @note Either packet input to this routine may itself be a splice, however
> + *   individual implementations may impose limits on how deeply splices
> + *   may be nested and fail the attempted splice if those limits are
> + *   exceeded.
> + *
> + * @note The packets used to create a splice may reside in different pools,
> + *   however individual implementations may require that both reside in
> + *   the same pool and fail the attempted splice if this restriction is
> + *   not observed. Upon return odp_packet_pool() returns the pool id of
> + *   the header packet.
> + *
> + * @note Once successfully spliced, the base packet may be freed via
> + *   odp_packet_free(), however the storage used to represent it will not
> + *   be released until all splices based on it have themselves been 
> freed.

Do you have a reason why the 'base' packet will be freed by the
application after creating the slice?

> + */
> +odp_packet_t odp_packet_splice(odp_packet_t hdr,
> +  odp_packet_t pkt, uint32_t offset);

Do we have an use-case for this scenario?
I would prefer that the API take the entire second packet instead of
an 'offset' at-least in the first version and that the 'hdr' handle
becomes invalid once the packet has been sliced and the new handle
returned replaces the 'hdr' handle.

> +
> +/**
> + * Create a reference to a packet
> + *
> + * Create a (shared) reference to a base packet starting at a specified
> + * byte offset. A reference is simply a splice with a zero-length header.
> + *
> + * @param pktHandle of the base packet for which a reference is to be
> + *   created.
> + * @param offset Byte offset in t

Re: [lng-odp] [PATCH 1/2] example: odp_timer_simple: decrease timer pool resolution

[Maxim Uvarov]

Merged both patches. Second one with my review only.

Maxim.

On 10/21/16 10:10, Kevin Wang wrote:

Reviewed-by: Kevin Wang 

On 21 October 2016 at 14:17, Yi He  wrote:


Reply for Kevin Wang to add his review.

Best Regards, Yi

On 19 July 2016 at 02:29, Brian Brooks  wrote:


On 07/18 13:41:32, Mike Holmes wrote:

Is 10 milliseconds adequate for real work ?

Should this be in some way calculated depending on the platform
capabilities where it is executes since this example will also be run

on

real HW accelerated systems and for those I assume we want the better
resolution.

I think you just created a new feature request. :) Enable applications to
discover the maximum timer pool resolution supported by the platform.

This example app expires its timers every two seconds, so this patch just
tweaks it to work (it failed on my machine) with the reference
implementation
on a generic Linux platform.

Re: [lng-odp] [PATCH v3 1/5] example: l3fwd: add missing gitignores

[Maxim Uvarov]

Does somebody review test this?

Maxim.

On 10/03/16 14:49, Matias Elo wrote:

Signed-off-by: Matias Elo 
---
  example/l3fwd/.gitignore | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/example/l3fwd/.gitignore b/example/l3fwd/.gitignore
index 74e501f..3411830 100644
--- a/example/l3fwd/.gitignore
+++ b/example/l3fwd/.gitignore
@@ -1 +1,3 @@
  odp_l3fwd
+*.log
+*.trs

Re: [lng-odp] [API-NEXT PATCH] test: drv: shm: decrease test memory requirement

[Maxim Uvarov]

Merged,

Maxim.

On 10/21/16 11:21, Elo, Matias (Nokia - FI/Espoo) wrote:

Reviewed-and-tested-by: Matias Elo 


-Original Message-
From: Christophe Milard [mailto:christophe.mil...@linaro.org]
Sent: Wednesday, October 19, 2016 5:30 PM
To: bill.fischo...@linaro.org; Elo, Matias (Nokia - FI/Espoo) ; lng-odp@lists.linaro.org
Cc: Christophe Milard 
Subject: [API-NEXT PATCH] test: drv: shm: decrease test memory requirement

The memory consumption is proporsional to the number of ODP threads
available. The test failed on systems with large number of CPU, due
to outage ot pre-allocaed address space. This patch shrinks the test
requirement.

Signed-off-by: Christophe Milard 
---
  test/common_plat/validation/drv/drvshmem/drvshmem.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/common_plat/validation/drv/drvshmem/drvshmem.c
b/test/common_plat/validation/drv/drvshmem/drvshmem.c
index 5e6d2e5..559c55d 100644
--- a/test/common_plat/validation/drv/drvshmem/drvshmem.c
+++ b/test/common_plat/validation/drv/drvshmem/drvshmem.c
@@ -16,7 +16,7 @@
  #define TEST_SHARE_BAR (0xf0f0f0f)
  #define SMALL_MEM 10
  #define MEDIUM_MEM 4096
-#define BIG_MEM 16777216
+#define BIG_MEM 65536
  #define STRESS_SIZE 32/* power of 2 and <=256 */
  #define STRESS_RANDOM_SZ 5
  #define STRESS_ITERATION 5000
--
2.7.4

[lng-odp] [API-NEXT PATCH v2 05/10] linux-gen: pool: reimplement pool with ring

[Petri Savolainen]

Used the ring data structure to implement pool. Also
buffer structure was simplified to enable future driver
interface. Every buffer includes a packet header, so each
buffer can be used as a packet head or segment. Segmentation
was disabled and segment size was fixed to a large number
(64kB) to limit the number of modification in the commit.

Signed-off-by: Petri Savolainen 
---
 .../include/odp/api/plat/pool_types.h  |6 -
 .../linux-generic/include/odp_buffer_inlines.h |  160 +--
 .../linux-generic/include/odp_buffer_internal.h|  104 +-
 .../include/odp_classification_datamodel.h |2 +-
 .../linux-generic/include/odp_config_internal.h|   34 +-
 .../linux-generic/include/odp_packet_internal.h|   13 +-
 platform/linux-generic/include/odp_pool_internal.h |  270 +---
 .../linux-generic/include/odp_timer_internal.h |4 -
 platform/linux-generic/odp_buffer.c|8 -
 platform/linux-generic/odp_classification.c|   25 +-
 platform/linux-generic/odp_crypto.c|4 +-
 platform/linux-generic/odp_packet.c|   99 +-
 platform/linux-generic/odp_pool.c  | 1441 
 platform/linux-generic/odp_timer.c |1 +
 platform/linux-generic/pktio/socket.c  |   16 +-
 platform/linux-generic/pktio/socket_mmap.c |   10 +-
 test/common_plat/performance/odp_pktio_perf.c  |2 +-
 test/common_plat/performance/odp_scheduling.c  |8 +-
 test/common_plat/validation/api/packet/packet.c|8 +-
 19 files changed, 746 insertions(+), 1469 deletions(-)

diff --git a/platform/linux-generic/include/odp/api/plat/pool_types.h 
b/platform/linux-generic/include/odp/api/plat/pool_types.h
index 1ca8f02..4e39de5 100644
--- a/platform/linux-generic/include/odp/api/plat/pool_types.h
+++ b/platform/linux-generic/include/odp/api/plat/pool_types.h
@@ -39,12 +39,6 @@ typedef enum odp_pool_type_t {
ODP_POOL_TIMEOUT = ODP_EVENT_TIMEOUT,
 } odp_pool_type_t;
 
-/** Get printable format of odp_pool_t */
-static inline uint64_t odp_pool_to_u64(odp_pool_t hdl)
-{
-   return _odp_pri(hdl);
-}
-
 /**
  * @}
  */
diff --git a/platform/linux-generic/include/odp_buffer_inlines.h 
b/platform/linux-generic/include/odp_buffer_inlines.h
index 2b1ab42..2f5eb88 100644
--- a/platform/linux-generic/include/odp_buffer_inlines.h
+++ b/platform/linux-generic/include/odp_buffer_inlines.h
@@ -18,43 +18,20 @@ extern "C" {
 #endif
 
 #include 
-#include 
 
-static inline odp_buffer_t odp_buffer_encode_handle(odp_buffer_hdr_t *hdr)
-{
-   odp_buffer_bits_t handle;
-   uint32_t pool_id = pool_handle_to_index(hdr->pool_hdl);
-   struct pool_entry_s *pool = get_pool_entry(pool_id);
+odp_event_type_t _odp_buffer_event_type(odp_buffer_t buf);
+void _odp_buffer_event_type_set(odp_buffer_t buf, int ev);
+int odp_buffer_snprint(char *str, uint32_t n, odp_buffer_t buf);
 
-   handle.handle = 0;
-   handle.pool_id = pool_id;
-   handle.index = ((uint8_t *)hdr - pool->pool_mdata_addr) /
-   ODP_CACHE_LINE_SIZE;
-   handle.seg = 0;
-
-   return handle.handle;
-}
+void *buffer_map(odp_buffer_hdr_t *buf, uint32_t offset, uint32_t *seglen,
+uint32_t limit);
 
 static inline odp_buffer_t odp_hdr_to_buf(odp_buffer_hdr_t *hdr)
 {
return hdr->handle.handle;
 }
 
-static inline odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf)
-{
-   odp_buffer_bits_t handle;
-   uint32_t pool_id;
-   uint32_t index;
-   struct pool_entry_s *pool;
-
-   handle.handle = buf;
-   pool_id   = handle.pool_id;
-   index = handle.index;
-   pool  = get_pool_entry(pool_id);
-
-   return (odp_buffer_hdr_t *)(void *)
-   (pool->pool_mdata_addr + (index * ODP_CACHE_LINE_SIZE));
-}
+odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf);
 
 static inline uint32_t pool_id_from_buf(odp_buffer_t buf)
 {
@@ -64,131 +41,6 @@ static inline uint32_t pool_id_from_buf(odp_buffer_t buf)
return handle.pool_id;
 }
 
-static inline odp_buffer_hdr_t *validate_buf(odp_buffer_t buf)
-{
-   odp_buffer_bits_t handle;
-   odp_buffer_hdr_t *buf_hdr;
-   handle.handle = buf;
-
-   /* For buffer handles, segment index must be 0 and pool id in range */
-   if (handle.seg != 0 || handle.pool_id >= ODP_CONFIG_POOLS)
-   return NULL;
-
-   pool_entry_t *pool =
-   odp_pool_to_entry(_odp_cast_scalar(odp_pool_t,
-  handle.pool_id));
-
-   /* If pool not created, handle is invalid */
-   if (pool->s.pool_shm == ODP_SHM_INVALID)
-   return NULL;
-
-   uint32_t buf_stride = pool->s.buf_stride / ODP_CACHE_LINE_SIZE;
-
-   /* A valid buffer index must be on stride, and must be in range */
-   if ((handle.index % buf_stride != 0) ||
-   ((uint32_t)(handle.index / buf_stride) >= pool->s.params

[lng-odp] [API-NEXT PATCH v2 09/10] linux-gen: pool: clean up pool inlines functions

[Petri Savolainen]

Removed odp_pool_to_entry(), which was a duplicate of
pool_entry_from_hdl(). Renamed odp_buf_to_hdr() to
buf_hdl_to_hdr(), which describes more accurately the internal
function. Inlined pool_entry(), pool_entry_from_hdl() and
buf_hdl_to_hdr(), which are used often and also outside of
pool.c. Renamed odp_buffer_pool_headroom() and _tailroom() to
simply pool_headroom() and _tailroom(), since those are internal
functions (not API as previous names hint). Also moved those
into pool.c, since inlining is not needed for functions that are
called only in (netmap) init phase.

Signed-off-by: Petri Savolainen 
---
 .../linux-generic/include/odp_buffer_inlines.h |  2 -
 .../linux-generic/include/odp_packet_internal.h|  2 +-
 platform/linux-generic/include/odp_pool_internal.h | 36 ---
 platform/linux-generic/odp_buffer.c|  6 +--
 platform/linux-generic/odp_packet.c|  8 ++--
 platform/linux-generic/odp_packet_io.c |  2 +-
 platform/linux-generic/odp_pool.c  | 54 ++
 platform/linux-generic/odp_queue.c |  4 +-
 platform/linux-generic/odp_schedule_ordered.c  |  4 +-
 platform/linux-generic/odp_timer.c |  2 +-
 platform/linux-generic/pktio/loop.c|  2 +-
 platform/linux-generic/pktio/netmap.c  |  4 +-
 platform/linux-generic/pktio/socket_mmap.c |  2 +-
 13 files changed, 62 insertions(+), 66 deletions(-)

diff --git a/platform/linux-generic/include/odp_buffer_inlines.h 
b/platform/linux-generic/include/odp_buffer_inlines.h
index 2f5eb88..f8688f6 100644
--- a/platform/linux-generic/include/odp_buffer_inlines.h
+++ b/platform/linux-generic/include/odp_buffer_inlines.h
@@ -31,8 +31,6 @@ static inline odp_buffer_t odp_hdr_to_buf(odp_buffer_hdr_t 
*hdr)
return hdr->handle.handle;
 }
 
-odp_buffer_hdr_t *odp_buf_to_hdr(odp_buffer_t buf);
-
 static inline uint32_t pool_id_from_buf(odp_buffer_t buf)
 {
odp_buffer_bits_t handle;
diff --git a/platform/linux-generic/include/odp_packet_internal.h 
b/platform/linux-generic/include/odp_packet_internal.h
index 8ad664b..48f9b26 100644
--- a/platform/linux-generic/include/odp_packet_internal.h
+++ b/platform/linux-generic/include/odp_packet_internal.h
@@ -184,7 +184,7 @@ typedef struct {
  */
 static inline odp_packet_hdr_t *odp_packet_hdr(odp_packet_t pkt)
 {
-   return (odp_packet_hdr_t *)odp_buf_to_hdr((odp_buffer_t)pkt);
+   return (odp_packet_hdr_t *)buf_hdl_to_hdr((odp_buffer_t)pkt);
 }
 
 static inline void copy_packet_parser_metadata(odp_packet_hdr_t *src_hdr,
diff --git a/platform/linux-generic/include/odp_pool_internal.h 
b/platform/linux-generic/include/odp_pool_internal.h
index 278c553..f7c315c 100644
--- a/platform/linux-generic/include/odp_pool_internal.h
+++ b/platform/linux-generic/include/odp_pool_internal.h
@@ -73,23 +73,45 @@ typedef struct pool_t {
 
 } pool_t;
 
-pool_t *pool_entry(uint32_t pool_idx);
+typedef struct pool_table_t {
+   pool_tpool[ODP_CONFIG_POOLS];
+   odp_shm_t shm;
+} pool_table_t;
 
-static inline pool_t *odp_pool_to_entry(odp_pool_t pool_hdl)
+extern pool_table_t *pool_tbl;
+
+static inline pool_t *pool_entry(uint32_t pool_idx)
 {
-   return pool_entry(_odp_typeval(pool_hdl));
+   return &pool_tbl->pool[pool_idx];
 }
 
-static inline uint32_t odp_buffer_pool_headroom(odp_pool_t pool)
+static inline pool_t *pool_entry_from_hdl(odp_pool_t pool_hdl)
 {
-   return odp_pool_to_entry(pool)->headroom;
+   return &pool_tbl->pool[_odp_typeval(pool_hdl)];
 }
 
-static inline uint32_t odp_buffer_pool_tailroom(odp_pool_t pool)
+static inline odp_buffer_hdr_t *buf_hdl_to_hdr(odp_buffer_t buf)
 {
-   return odp_pool_to_entry(pool)->tailroom;
+   odp_buffer_bits_t handle;
+   uint32_t pool_id, index, block_offset;
+   pool_t *pool;
+   odp_buffer_hdr_t *buf_hdr;
+
+   handle.handle = buf;
+   pool_id   = handle.pool_id;
+   index = handle.index;
+   pool  = pool_entry(pool_id);
+   block_offset  = index * pool->block_size;
+
+   /* clang requires cast to uintptr_t */
+   buf_hdr = (odp_buffer_hdr_t *)(uintptr_t)&pool->base_addr[block_offset];
+
+   return buf_hdr;
 }
 
+uint32_t pool_headroom(odp_pool_t pool);
+uint32_t pool_tailroom(odp_pool_t pool);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/platform/linux-generic/odp_buffer.c 
b/platform/linux-generic/odp_buffer.c
index 0ddaf95..eed15c0 100644
--- a/platform/linux-generic/odp_buffer.c
+++ b/platform/linux-generic/odp_buffer.c
@@ -26,14 +26,14 @@ odp_event_t odp_buffer_to_event(odp_buffer_t buf)
 
 void *odp_buffer_addr(odp_buffer_t buf)
 {
-   odp_buffer_hdr_t *hdr = odp_buf_to_hdr(buf);
+   odp_buffer_hdr_t *hdr = buf_hdl_to_hdr(buf);
 
return hdr->addr[0];
 }
 
 uint32_t odp_buffer_size(odp_buffer_t buf)
 {
-   odp_buffer_hdr_t *hdr = odp_buf_to_hdr(buf);
+   odp_buffer_hdr_t *hdr = buf_hdl_t

[lng-odp] [API-NEXT PATCH v2 03/10] linux-gen: ring: created common ring implementation

[Petri Savolainen]

Moved scheduler ring code into a new header file, so that
it can be used also in other parts of the implementation.

Signed-off-by: Petri Savolainen 
---
 platform/linux-generic/Makefile.am |   1 +
 platform/linux-generic/include/odp_ring_internal.h | 111 +
 platform/linux-generic/odp_schedule.c  | 102 ++-
 3 files changed, 120 insertions(+), 94 deletions(-)
 create mode 100644 platform/linux-generic/include/odp_ring_internal.h

diff --git a/platform/linux-generic/Makefile.am 
b/platform/linux-generic/Makefile.am
index f63f007..2f99d4f 100644
--- a/platform/linux-generic/Makefile.am
+++ b/platform/linux-generic/Makefile.am
@@ -149,6 +149,7 @@ noinst_HEADERS = \
  ${srcdir}/include/odp_pool_internal.h \
  ${srcdir}/include/odp_posix_extensions.h \
  ${srcdir}/include/odp_queue_internal.h \
+ ${srcdir}/include/odp_ring_internal.h \
  ${srcdir}/include/odp_schedule_if.h \
  ${srcdir}/include/odp_schedule_internal.h \
  ${srcdir}/include/odp_schedule_ordered_internal.h \
diff --git a/platform/linux-generic/include/odp_ring_internal.h 
b/platform/linux-generic/include/odp_ring_internal.h
new file mode 100644
index 000..6a6291a
--- /dev/null
+++ b/platform/linux-generic/include/odp_ring_internal.h
@@ -0,0 +1,111 @@
+/* Copyright (c) 2016, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#ifndef ODP_RING_INTERNAL_H_
+#define ODP_RING_INTERNAL_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include 
+#include 
+#include 
+
+/* Ring empty, not a valid data value. */
+#define RING_EMPTY ((uint32_t)-1)
+
+/* Ring of uint32_t data
+ *
+ * Ring stores head and tail counters. Ring indexes are formed from these
+ * counters with a mask (mask = ring_size - 1), which requires that ring size
+ * must be a power of two. Also ring size must be larger than the maximum
+ * number of data items that will be stored on it (there's no check against
+ * overwriting). */
+typedef struct {
+   /* Writer head and tail */
+   odp_atomic_u32_t w_head;
+   odp_atomic_u32_t w_tail;
+   uint8_t pad[ODP_CACHE_LINE_SIZE - (2 * sizeof(odp_atomic_u32_t))];
+
+   /* Reader head and tail */
+   odp_atomic_u32_t r_head;
+   odp_atomic_u32_t r_tail;
+
+   uint32_t data[0];
+} ring_t ODP_ALIGNED_CACHE;
+
+/* Initialize ring */
+static inline void ring_init(ring_t *ring)
+{
+   odp_atomic_init_u32(&ring->w_head, 0);
+   odp_atomic_init_u32(&ring->w_tail, 0);
+   odp_atomic_init_u32(&ring->r_head, 0);
+   odp_atomic_init_u32(&ring->r_tail, 0);
+}
+
+/* Dequeue data from the ring head */
+static inline uint32_t ring_deq(ring_t *ring, uint32_t mask)
+{
+   uint32_t head, tail, new_head;
+   uint32_t data;
+
+   head = odp_atomic_load_u32(&ring->r_head);
+
+   /* Move reader head. This thread owns data at the new head. */
+   do {
+   tail = odp_atomic_load_u32(&ring->w_tail);
+
+   if (head == tail)
+   return RING_EMPTY;
+
+   new_head = head + 1;
+
+   } while (odp_unlikely(odp_atomic_cas_acq_u32(&ring->r_head, &head,
+ new_head) == 0));
+
+   /* Read queue index */
+   data = ring->data[new_head & mask];
+
+   /* Wait until other readers have updated the tail */
+   while (odp_unlikely(odp_atomic_load_acq_u32(&ring->r_tail) != head))
+   odp_cpu_pause();
+
+   /* Now update the reader tail */
+   odp_atomic_store_rel_u32(&ring->r_tail, new_head);
+
+   return data;
+}
+
+/* Enqueue data into the ring tail */
+static inline void ring_enq(ring_t *ring, uint32_t mask, uint32_t data)
+{
+   uint32_t old_head, new_head;
+
+   /* Reserve a slot in the ring for writing */
+   old_head = odp_atomic_fetch_inc_u32(&ring->w_head);
+   new_head = old_head + 1;
+
+   /* Ring is full. Wait for the last reader to finish. */
+   while (odp_unlikely(odp_atomic_load_acq_u32(&ring->r_tail) == new_head))
+   odp_cpu_pause();
+
+   /* Write data */
+   ring->data[new_head & mask] = data;
+
+   /* Wait until other writers have updated the tail */
+   while (odp_unlikely(odp_atomic_load_acq_u32(&ring->w_tail) != old_head))
+   odp_cpu_pause();
+
+   /* Now update the writer tail */
+   odp_atomic_store_rel_u32(&ring->w_tail, new_head);
+}
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/platform/linux-generic/odp_schedule.c 
b/platform/linux-generic/odp_schedule.c
index 81e79c9..86c98fe 100644
--- a/platform/linux-generic/odp_schedule.c
+++ b/platform/linux-generic/odp_schedule.c
@@ -17,12 +17,12 @@
 #include 
 #include 
 #include 
-#include 
 #include 
 #include 
 #include 
 #include 
 #include 
+#include 
 
 /* Number of priority levels  */
 #define NUM_PRIO 8
@@ -82,9 +82,

[lng-odp] [API-NEXT PATCH v2 10/10] linux-gen: pool: ptr instead of hdl in buffer_alloc_multi

[Petri Savolainen]

Improve performance by changing the first parameter of
buffer_alloc_multi() to pool pointer (from handle), to avoid
double lookup of the pool pointer. Pointer is available for
packet alloc calls already.

Signed-off-by: Petri Savolainen 
---
 platform/linux-generic/include/odp_buffer_internal.h |  4 
 platform/linux-generic/include/odp_pool_internal.h   |  4 
 platform/linux-generic/odp_packet.c  |  6 +++---
 platform/linux-generic/odp_pool.c| 16 ++--
 4 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/platform/linux-generic/include/odp_buffer_internal.h 
b/platform/linux-generic/include/odp_buffer_internal.h
index 64ba221..0ca13f8 100644
--- a/platform/linux-generic/include/odp_buffer_internal.h
+++ b/platform/linux-generic/include/odp_buffer_internal.h
@@ -105,10 +105,6 @@ struct odp_buffer_hdr_t {
 };
 
 /* Forward declarations */
-int buffer_alloc_multi(odp_pool_t pool_hdl, odp_buffer_t buf[],
-  odp_buffer_hdr_t *buf_hdr[], int num);
-void buffer_free_multi(const odp_buffer_t buf[], int num_free);
-
 int seg_alloc_head(odp_buffer_hdr_t *buf_hdr, int segcount);
 void seg_free_head(odp_buffer_hdr_t *buf_hdr, int segcount);
 int seg_alloc_tail(odp_buffer_hdr_t *buf_hdr, int segcount);
diff --git a/platform/linux-generic/include/odp_pool_internal.h 
b/platform/linux-generic/include/odp_pool_internal.h
index f7c315c..f7e951a 100644
--- a/platform/linux-generic/include/odp_pool_internal.h
+++ b/platform/linux-generic/include/odp_pool_internal.h
@@ -109,6 +109,10 @@ static inline odp_buffer_hdr_t 
*buf_hdl_to_hdr(odp_buffer_t buf)
return buf_hdr;
 }
 
+int buffer_alloc_multi(pool_t *pool, odp_buffer_t buf[],
+  odp_buffer_hdr_t *buf_hdr[], int num);
+void buffer_free_multi(const odp_buffer_t buf[], int num_free);
+
 uint32_t pool_headroom(odp_pool_t pool);
 uint32_t pool_tailroom(odp_pool_t pool);
 
diff --git a/platform/linux-generic/odp_packet.c 
b/platform/linux-generic/odp_packet.c
index 59f0c99..ff41cd0 100644
--- a/platform/linux-generic/odp_packet.c
+++ b/platform/linux-generic/odp_packet.c
@@ -81,7 +81,7 @@ int packet_alloc_multi(odp_pool_t pool_hdl, uint32_t len,
int num, i;
odp_packet_hdr_t *pkt_hdrs[max_num];
 
-   num = buffer_alloc_multi(pool_hdl, (odp_buffer_t *)pkt,
+   num = buffer_alloc_multi(pool, (odp_buffer_t *)pkt,
 (odp_buffer_hdr_t **)pkt_hdrs, max_num);
 
for (i = 0; i < num; i++) {
@@ -112,7 +112,7 @@ odp_packet_t odp_packet_alloc(odp_pool_t pool_hdl, uint32_t 
len)
if (odp_unlikely(len > pool->max_len))
return ODP_PACKET_INVALID;
 
-   ret = buffer_alloc_multi(pool_hdl, (odp_buffer_t *)&pkt, NULL, 1);
+   ret = buffer_alloc_multi(pool, (odp_buffer_t *)&pkt, NULL, 1);
if (ret != 1)
return ODP_PACKET_INVALID;
 
@@ -143,7 +143,7 @@ int odp_packet_alloc_multi(odp_pool_t pool_hdl, uint32_t 
len,
if (odp_unlikely(len > pool->max_len))
return -1;
 
-   count = buffer_alloc_multi(pool_hdl, (odp_buffer_t *)pkt,
+   count = buffer_alloc_multi(pool, (odp_buffer_t *)pkt,
   (odp_buffer_hdr_t **)pkt_hdrs, num);
 
for (i = 0; i < count; ++i) {
diff --git a/platform/linux-generic/odp_pool.c 
b/platform/linux-generic/odp_pool.c
index faea2fc..364df97 100644
--- a/platform/linux-generic/odp_pool.c
+++ b/platform/linux-generic/odp_pool.c
@@ -528,19 +528,17 @@ int odp_pool_info(odp_pool_t pool_hdl, odp_pool_info_t 
*info)
return 0;
 }
 
-int buffer_alloc_multi(odp_pool_t pool_hdl, odp_buffer_t buf[],
+int buffer_alloc_multi(pool_t *pool, odp_buffer_t buf[],
   odp_buffer_hdr_t *buf_hdr[], int max_num)
 {
-   pool_t *pool;
ring_t *ring;
uint32_t mask, i;
pool_cache_t *cache;
uint32_t cache_num, num_ch, num_deq, burst;
 
-   pool  = pool_entry_from_hdl(pool_hdl);
ring  = &pool->ring.hdr;
mask  = pool->ring_mask;
-   cache = local.cache[_odp_typeval(pool_hdl)];
+   cache = local.cache[pool->pool_idx];
 
cache_num = cache->num;
num_ch= max_num;
@@ -696,9 +694,11 @@ void buffer_free_multi(const odp_buffer_t buf[], int 
num_total)
 odp_buffer_t odp_buffer_alloc(odp_pool_t pool_hdl)
 {
odp_buffer_t buf;
+   pool_t *pool;
int ret;
 
-   ret = buffer_alloc_multi(pool_hdl, &buf, NULL, 1);
+   pool = pool_entry_from_hdl(pool_hdl);
+   ret = buffer_alloc_multi(pool, &buf, NULL, 1);
 
if (odp_likely(ret == 1))
return buf;
@@ -708,7 +708,11 @@ odp_buffer_t odp_buffer_alloc(odp_pool_t pool_hdl)
 
 int odp_buffer_alloc_multi(odp_pool_t pool_hdl, odp_buffer_t buf[], int num)
 {
-   return buffer_alloc_multi(pool_hdl, buf, NULL, num);
+   pool_t *pool;
+
+   pool = pool_entry_from_hdl(pool_hdl);
+
+   return buffer_alloc_multi(pool, b

[lng-odp] [API-NEXT PATCH v2 00/10] pool optimization

[Petri Savolainen]

Pool performance is optimized by using a ring as the global buffer storage. 
IPC build is disabled, since it needs large modifications due to dependency to 
pool internals. Old pool implementation was based on locks and linked list of 
buffer headers. New implementation maintain a ring of buffer handles, which 
enable fast, burst based allocs and frees. Also ring scales better with number 
of cpus than a list (enq and deq operations update opposite ends of the pool). 
Seqmentation was not implemented yet to limit the patch set size. Instead, each 
packet is created from single, fixed size (64kB) buffer. Segmentation and more 
efficient packet data pointer, etc operations will follow.

L2fwd link rate (%), 2 x 40GE, 64 byte packets

direct- parallel-   atomic-
cpusorigdirect  difforigparall  difforigatomic  diff
1   7 % 8 % 1 % 6 % 6 % 2 % 5.4 %   5.6 %   4 %
2   14 %15 %7 % 9 % 9 % 5 % 8 % 9 % 8 %
4   28 %30 %6 % 13 %14 %13 %12 %15 %19 %
6   42 %44 %6 % 16 %19 %19 %8 % 20 %150 %
8   46 %59 %28 %19 %23 %26 %18 %24 %34 %
10  55 %57 %3 % 20 %27 %37 %8 % 28 %264 %
12  56 %56 %-1 %22 %31 %43 %7 % 32 %357 %

Max packet rate of NICs are reached with 10-12 cpu on direct mode. Otherwise, 
all cases were improved. Especially, scheduler driven cases suffered on bad 
pool scalability.

changes in v2:
* rebased to api-next branch
* added a comment that ring size must be larger than number of items in it
* fixed clang build issue
* added parens in align macro

v1 reviews:
Reviewed-by: Brian Brooks 


Petri Savolainen (10):
  linux-gen: ipc: disable build of ipc pktio
  linux-gen: pktio: do not free zero packets
  linux-gen: ring: created common ring implementation
  linux-gen: align: added round up power of two
  linux-gen: pool: reimplement pool with ring
  linux-gen: ring: added multi enq and deq
  linux-gen: pool: use ring multi enq and deq operations
  linux-gen: pool: optimize buffer alloc
  linux-gen: pool: clean up pool inlines functions
  linux-gen: pool: ptr instead of hdl in buffer_alloc_multi

 platform/linux-generic/Makefile.am |3 +-
 .../include/odp/api/plat/pool_types.h  |6 -
 .../linux-generic/include/odp_align_internal.h |   34 +-
 .../linux-generic/include/odp_buffer_inlines.h |  160 +--
 .../linux-generic/include/odp_buffer_internal.h|  101 +-
 .../include/odp_classification_datamodel.h |2 +-
 .../linux-generic/include/odp_config_internal.h|   34 +-
 .../linux-generic/include/odp_packet_internal.h|   15 +-
 platform/linux-generic/include/odp_pool_internal.h |  290 +---
 platform/linux-generic/include/odp_ring_internal.h |  176 +++
 .../linux-generic/include/odp_timer_internal.h |4 -
 platform/linux-generic/odp_buffer.c|   14 +-
 platform/linux-generic/odp_classification.c|   25 +-
 platform/linux-generic/odp_crypto.c|4 +-
 platform/linux-generic/odp_packet.c|  109 +-
 platform/linux-generic/odp_packet_io.c |2 +-
 platform/linux-generic/odp_pool.c  | 1467 
 platform/linux-generic/odp_queue.c |4 +-
 platform/linux-generic/odp_schedule.c  |  102 +-
 platform/linux-generic/odp_schedule_ordered.c  |4 +-
 platform/linux-generic/odp_timer.c |3 +-
 platform/linux-generic/pktio/dpdk.c|   10 +-
 platform/linux-generic/pktio/loop.c|2 +-
 platform/linux-generic/pktio/netmap.c  |   14 +-
 platform/linux-generic/pktio/socket.c  |   16 +-
 platform/linux-generic/pktio/socket_mmap.c |   10 +-
 test/common_plat/performance/odp_pktio_perf.c  |2 +-
 test/common_plat/performance/odp_scheduling.c  |8 +-
 test/common_plat/validation/api/packet/packet.c|8 +-
 29 files changed, 1015 insertions(+), 1614 deletions(-)
 create mode 100644 platform/linux-generic/include/odp_ring_internal.h

-- 
2.8.1

[lng-odp] [API-NEXT PATCH v2 06/10] linux-gen: ring: added multi enq and deq

[Petri Savolainen]

Added multi-data versions of ring enqueue and dequeue operations.

Signed-off-by: Petri Savolainen 
---
 platform/linux-generic/include/odp_ring_internal.h | 65 ++
 1 file changed, 65 insertions(+)

diff --git a/platform/linux-generic/include/odp_ring_internal.h 
b/platform/linux-generic/include/odp_ring_internal.h
index 6a6291a..55fedeb 100644
--- a/platform/linux-generic/include/odp_ring_internal.h
+++ b/platform/linux-generic/include/odp_ring_internal.h
@@ -80,6 +80,45 @@ static inline uint32_t ring_deq(ring_t *ring, uint32_t mask)
return data;
 }
 
+/* Dequeue multiple data from the ring head. Num is smaller than ring size. */
+static inline uint32_t ring_deq_multi(ring_t *ring, uint32_t mask,
+ uint32_t data[], uint32_t num)
+{
+   uint32_t head, tail, new_head, i;
+
+   head = odp_atomic_load_u32(&ring->r_head);
+
+   /* Move reader head. This thread owns data at the new head. */
+   do {
+   tail = odp_atomic_load_u32(&ring->w_tail);
+
+   /* Ring is empty */
+   if (head == tail)
+   return 0;
+
+   /* Try to take all available */
+   if ((tail - head) < num)
+   num = tail - head;
+
+   new_head = head + num;
+
+   } while (odp_unlikely(odp_atomic_cas_acq_u32(&ring->r_head, &head,
+ new_head) == 0));
+
+   /* Read queue index */
+   for (i = 0; i < num; i++)
+   data[i] = ring->data[(head + 1 + i) & mask];
+
+   /* Wait until other readers have updated the tail */
+   while (odp_unlikely(odp_atomic_load_acq_u32(&ring->r_tail) != head))
+   odp_cpu_pause();
+
+   /* Now update the reader tail */
+   odp_atomic_store_rel_u32(&ring->r_tail, new_head);
+
+   return num;
+}
+
 /* Enqueue data into the ring tail */
 static inline void ring_enq(ring_t *ring, uint32_t mask, uint32_t data)
 {
@@ -104,6 +143,32 @@ static inline void ring_enq(ring_t *ring, uint32_t mask, 
uint32_t data)
odp_atomic_store_rel_u32(&ring->w_tail, new_head);
 }
 
+/* Enqueue multiple data into the ring tail. Num is smaller than ring size. */
+static inline void ring_enq_multi(ring_t *ring, uint32_t mask, uint32_t data[],
+ uint32_t num)
+{
+   uint32_t old_head, new_head, i;
+
+   /* Reserve a slot in the ring for writing */
+   old_head = odp_atomic_fetch_add_u32(&ring->w_head, num);
+   new_head = old_head + 1;
+
+   /* Ring is full. Wait for the last reader to finish. */
+   while (odp_unlikely(odp_atomic_load_acq_u32(&ring->r_tail) == new_head))
+   odp_cpu_pause();
+
+   /* Write data */
+   for (i = 0; i < num; i++)
+   ring->data[(new_head + i) & mask] = data[i];
+
+   /* Wait until other writers have updated the tail */
+   while (odp_unlikely(odp_atomic_load_acq_u32(&ring->w_tail) != old_head))
+   odp_cpu_pause();
+
+   /* Now update the writer tail */
+   odp_atomic_store_rel_u32(&ring->w_tail, old_head + num);
+}
+
 #ifdef __cplusplus
 }
 #endif
-- 
2.8.1

[lng-odp] [API-NEXT PATCH v2 08/10] linux-gen: pool: optimize buffer alloc

[Petri Savolainen]

Round up global pool allocations to a burst size. Cache any
extra buffers for future use. Prefetch buffers header which
very newly allocated from global pool and will be returned to
the caller.

Signed-off-by: Petri Savolainen 
---
 .../linux-generic/include/odp_buffer_internal.h|  3 +-
 platform/linux-generic/odp_packet.c| 16 +++--
 platform/linux-generic/odp_pool.c  | 74 --
 3 files changed, 68 insertions(+), 25 deletions(-)

diff --git a/platform/linux-generic/include/odp_buffer_internal.h 
b/platform/linux-generic/include/odp_buffer_internal.h
index abe8591..64ba221 100644
--- a/platform/linux-generic/include/odp_buffer_internal.h
+++ b/platform/linux-generic/include/odp_buffer_internal.h
@@ -105,7 +105,8 @@ struct odp_buffer_hdr_t {
 };
 
 /* Forward declarations */
-int buffer_alloc_multi(odp_pool_t pool_hdl, odp_buffer_t buf[], int num);
+int buffer_alloc_multi(odp_pool_t pool_hdl, odp_buffer_t buf[],
+  odp_buffer_hdr_t *buf_hdr[], int num);
 void buffer_free_multi(const odp_buffer_t buf[], int num_free);
 
 int seg_alloc_head(odp_buffer_hdr_t *buf_hdr, int segcount);
diff --git a/platform/linux-generic/odp_packet.c 
b/platform/linux-generic/odp_packet.c
index 03769f6..bf22443 100644
--- a/platform/linux-generic/odp_packet.c
+++ b/platform/linux-generic/odp_packet.c
@@ -77,14 +77,16 @@ static void packet_init(pool_t *pool, odp_packet_hdr_t 
*pkt_hdr,
 int packet_alloc_multi(odp_pool_t pool_hdl, uint32_t len,
   odp_packet_t pkt[], int max_num)
 {
-   odp_packet_hdr_t *pkt_hdr;
pool_t *pool = odp_pool_to_entry(pool_hdl);
int num, i;
+   odp_packet_hdr_t *pkt_hdrs[max_num];
 
-   num = buffer_alloc_multi(pool_hdl, (odp_buffer_t *)pkt, max_num);
+   num = buffer_alloc_multi(pool_hdl, (odp_buffer_t *)pkt,
+(odp_buffer_hdr_t **)pkt_hdrs, max_num);
 
for (i = 0; i < num; i++) {
-   pkt_hdr = odp_packet_hdr(pkt[i]);
+   odp_packet_hdr_t *pkt_hdr = pkt_hdrs[i];
+
packet_init(pool, pkt_hdr, len, 1 /* do parse */);
 
if (pkt_hdr->tailroom >= pkt_hdr->buf_hdr.segsize)
@@ -110,7 +112,7 @@ odp_packet_t odp_packet_alloc(odp_pool_t pool_hdl, uint32_t 
len)
if (odp_unlikely(len > pool->max_len))
return ODP_PACKET_INVALID;
 
-   ret = buffer_alloc_multi(pool_hdl, (odp_buffer_t *)&pkt, 1);
+   ret = buffer_alloc_multi(pool_hdl, (odp_buffer_t *)&pkt, NULL, 1);
if (ret != 1)
return ODP_PACKET_INVALID;
 
@@ -131,6 +133,7 @@ int odp_packet_alloc_multi(odp_pool_t pool_hdl, uint32_t 
len,
pool_t *pool = odp_pool_to_entry(pool_hdl);
size_t pkt_size = len ? len : pool->data_size;
int count, i;
+   odp_packet_hdr_t *pkt_hdrs[num];
 
if (odp_unlikely(pool->params.type != ODP_POOL_PACKET)) {
__odp_errno = EINVAL;
@@ -140,10 +143,11 @@ int odp_packet_alloc_multi(odp_pool_t pool_hdl, uint32_t 
len,
if (odp_unlikely(len > pool->max_len))
return -1;
 
-   count = buffer_alloc_multi(pool_hdl, (odp_buffer_t *)pkt, num);
+   count = buffer_alloc_multi(pool_hdl, (odp_buffer_t *)pkt,
+  (odp_buffer_hdr_t **)pkt_hdrs, num);
 
for (i = 0; i < count; ++i) {
-   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt[i]);
+   odp_packet_hdr_t *pkt_hdr = pkt_hdrs[i];
 
packet_init(pool, pkt_hdr, pkt_size, 0 /* do not parse */);
if (len == 0)
diff --git a/platform/linux-generic/odp_pool.c 
b/platform/linux-generic/odp_pool.c
index a2e5d54..7dc0938 100644
--- a/platform/linux-generic/odp_pool.c
+++ b/platform/linux-generic/odp_pool.c
@@ -562,14 +562,14 @@ int odp_pool_info(odp_pool_t pool_hdl, odp_pool_info_t 
*info)
return 0;
 }
 
-int buffer_alloc_multi(odp_pool_t pool_hdl, odp_buffer_t buf[], int max_num)
+int buffer_alloc_multi(odp_pool_t pool_hdl, odp_buffer_t buf[],
+  odp_buffer_hdr_t *buf_hdr[], int max_num)
 {
pool_t *pool;
ring_t *ring;
-   uint32_t mask;
-   int i;
+   uint32_t mask, i;
pool_cache_t *cache;
-   uint32_t cache_num;
+   uint32_t cache_num, num_ch, num_deq, burst;
 
pool  = pool_entry_from_hdl(pool_hdl);
ring  = &pool->ring.hdr;
@@ -577,28 +577,66 @@ int buffer_alloc_multi(odp_pool_t pool_hdl, odp_buffer_t 
buf[], int max_num)
cache = local.cache[_odp_typeval(pool_hdl)];
 
cache_num = cache->num;
+   num_ch= max_num;
+   num_deq   = 0;
+   burst = CACHE_BURST;
 
-   if (odp_likely((int)cache_num >= max_num)) {
-   for (i = 0; i < max_num; i++)
-   buf[i] = cache->buf[cache_num - max_num + i];
+   if (odp_unlikely(cache_num < (uint32_t)max_num)) {
+   /* Cache does not have enough buffers */
+   

[lng-odp] [API-NEXT PATCH v2 07/10] linux-gen: pool: use ring multi enq and deq operations

[Petri Savolainen]

Use multi enq and deq operations to optimize global pool
access performance. Temporary uint32_t arrays are needed
since handles are pointer size variables.

Signed-off-by: Petri Savolainen 
---
 platform/linux-generic/odp_pool.c | 32 
 1 file changed, 20 insertions(+), 12 deletions(-)

diff --git a/platform/linux-generic/odp_pool.c 
b/platform/linux-generic/odp_pool.c
index 1286753..a2e5d54 100644
--- a/platform/linux-generic/odp_pool.c
+++ b/platform/linux-generic/odp_pool.c
@@ -586,15 +586,16 @@ int buffer_alloc_multi(odp_pool_t pool_hdl, odp_buffer_t 
buf[], int max_num)
return max_num;
}
 
-   for (i = 0; i < max_num; i++) {
-   uint32_t data;
+   {
+   /* Temporary copy needed since odp_buffer_t is uintptr_t
+* and not uint32_t. */
+   int num;
+   uint32_t data[max_num];
 
-   data = ring_deq(ring, mask);
+   num = ring_deq_multi(ring, mask, data, max_num);
 
-   if (data == RING_EMPTY)
-   break;
-
-   buf[i] = (odp_buffer_t)(uintptr_t)data;
+   for (i = 0; i < num; i++)
+   buf[i] = (odp_buffer_t)(uintptr_t)data[i];
}
 
return i;
@@ -629,17 +630,24 @@ static inline void buffer_free_to_pool(uint32_t pool_id,
cache_num = cache->num;
 
if (odp_unlikely((int)(CONFIG_POOL_CACHE_SIZE - cache_num) < num)) {
+   uint32_t index;
int burst = CACHE_BURST;
 
if (odp_unlikely(num > CACHE_BURST))
burst = num;
 
-   for (i = 0; i < burst; i++) {
-   uint32_t data, index;
+   {
+   /* Temporary copy needed since odp_buffer_t is
+* uintptr_t and not uint32_t. */
+   uint32_t data[burst];
+
+   index = cache_num - burst;
+
+   for (i = 0; i < burst; i++)
+   data[i] = (uint32_t)
+ (uintptr_t)cache->buf[index + i];
 
-   index = cache_num - burst + i;
-   data  = (uint32_t)(uintptr_t)cache->buf[index];
-   ring_enq(ring, mask, data);
+   ring_enq_multi(ring, mask, data, burst);
}
 
cache_num -= burst;
-- 
2.8.1

[lng-odp] [API-NEXT PATCH v2 04/10] linux-gen: align: added round up power of two

[Petri Savolainen]

Added a macro to round up a value to the next power of two,
if it's not already a power of two. Also removed duplicated
code from the same file.

Signed-off-by: Petri Savolainen 
---
 .../linux-generic/include/odp_align_internal.h | 34 +-
 1 file changed, 7 insertions(+), 27 deletions(-)

diff --git a/platform/linux-generic/include/odp_align_internal.h 
b/platform/linux-generic/include/odp_align_internal.h
index 9ccde53..d9cd30b 100644
--- a/platform/linux-generic/include/odp_align_internal.h
+++ b/platform/linux-generic/include/odp_align_internal.h
@@ -29,24 +29,18 @@ extern "C" {
 
 /**
  * @internal
- * Round up pointer 'x' to alignment 'align'
- */
-#define ODP_ALIGN_ROUNDUP_PTR(x, align)\
-   ((void *)ODP_ALIGN_ROUNDUP((uintptr_t)(x), (uintptr_t)(align)))
-
-/**
- * @internal
- * Round up pointer 'x' to cache line size alignment
+ * Round up 'x' to alignment 'align'
  */
-#define ODP_CACHE_LINE_SIZE_ROUNDUP_PTR(x)\
-   ((void *)ODP_CACHE_LINE_SIZE_ROUNDUP((uintptr_t)(x)))
+#define ODP_ALIGN_ROUNDUP(x, align)\
+   ((align) * (((x) + (align) - 1) / (align)))
 
 /**
  * @internal
- * Round up 'x' to alignment 'align'
+ * When 'x' is not already a power of two, round it up to the next
+ * power of two value. Zero is not supported as an input value.
  */
-#define ODP_ALIGN_ROUNDUP(x, align)\
-   ((align) * (((x) + align - 1) / (align)))
+#define ODP_ROUNDUP_POWER_2(x)\
+   (1 << (((int)(8 * sizeof(x))) - __builtin_clz((x) - 1)))
 
 /**
  * @internal
@@ -82,20 +76,6 @@ extern "C" {
 
 /**
  * @internal
- * Round down pointer 'x' to 'align' alignment, which is a power of two
- */
-#define ODP_ALIGN_ROUNDDOWN_PTR_POWER_2(x, align)\
-((void *)ODP_ALIGN_ROUNDDOWN_POWER_2((uintptr_t)(x), (uintptr_t)(align)))
-
-/**
- * @internal
- * Round down pointer 'x' to cache line size alignment
- */
-#define ODP_CACHE_LINE_SIZE_ROUNDDOWN_PTR(x)\
-   ((void *)ODP_CACHE_LINE_SIZE_ROUNDDOWN((uintptr_t)(x)))
-
-/**
- * @internal
  * Round down 'x' to 'align' alignment, which is a power of two
  */
 #define ODP_ALIGN_ROUNDDOWN_POWER_2(x, align)\
-- 
2.8.1

[lng-odp] [API-NEXT PATCH v2 02/10] linux-gen: pktio: do not free zero packets

[Petri Savolainen]

In some error cases, netmap and dpdk pktios were calling
odp_packet_free_multi with zero packets. Moved existing error
check to avoid a free call with zero packets.

Signed-off-by: Petri Savolainen 
---
 platform/linux-generic/pktio/dpdk.c   | 10 ++
 platform/linux-generic/pktio/netmap.c | 10 ++
 2 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/platform/linux-generic/pktio/dpdk.c 
b/platform/linux-generic/pktio/dpdk.c
index 11f3509..0eb025a 100644
--- a/platform/linux-generic/pktio/dpdk.c
+++ b/platform/linux-generic/pktio/dpdk.c
@@ -956,10 +956,12 @@ static int dpdk_send(pktio_entry_t *pktio_entry, int 
index,
rte_pktmbuf_free(tx_mbufs[i]);
}
 
-   odp_packet_free_multi(pkt_table, tx_pkts);
-
-   if (odp_unlikely(tx_pkts == 0 && __odp_errno != 0))
-   return -1;
+   if (odp_unlikely(tx_pkts == 0)) {
+   if (__odp_errno != 0)
+   return -1;
+   } else {
+   odp_packet_free_multi(pkt_table, tx_pkts);
+   }
 
return tx_pkts;
 }
diff --git a/platform/linux-generic/pktio/netmap.c 
b/platform/linux-generic/pktio/netmap.c
index 412beec..c1cdf72 100644
--- a/platform/linux-generic/pktio/netmap.c
+++ b/platform/linux-generic/pktio/netmap.c
@@ -830,10 +830,12 @@ static int netmap_send(pktio_entry_t *pktio_entry, int 
index,
if (!pkt_nm->lockless_tx)
odp_ticketlock_unlock(&pkt_nm->tx_desc_ring[index].s.lock);
 
-   odp_packet_free_multi(pkt_table, nb_tx);
-
-   if (odp_unlikely(nb_tx == 0 && __odp_errno != 0))
-   return -1;
+   if (odp_unlikely(nb_tx == 0)) {
+   if (__odp_errno != 0)
+   return -1;
+   } else {
+   odp_packet_free_multi(pkt_table, nb_tx);
+   }
 
return nb_tx;
 }
-- 
2.8.1

[lng-odp] [API-NEXT PATCH v2 01/10] linux-gen: ipc: disable build of ipc pktio

[Petri Savolainen]

IPC pktio implementation depends heavily on pool internals. It's
build is disabled due to pool re-implementation. IPC should be
re-implemented with a cleaner internal interface towards pool and
shm.

Signed-off-by: Petri Savolainen 
---
 platform/linux-generic/Makefile.am | 2 --
 1 file changed, 2 deletions(-)

diff --git a/platform/linux-generic/Makefile.am 
b/platform/linux-generic/Makefile.am
index dc2486c..f63f007 100644
--- a/platform/linux-generic/Makefile.am
+++ b/platform/linux-generic/Makefile.am
@@ -139,7 +139,6 @@ noinst_HEADERS = \
  ${srcdir}/include/odp_name_table_internal.h \
  ${srcdir}/include/odp_packet_internal.h \
  ${srcdir}/include/odp_packet_io_internal.h \
- ${srcdir}/include/odp_packet_io_ipc_internal.h \
  ${srcdir}/include/odp_packet_io_queue.h \
  ${srcdir}/include/odp_packet_io_ring_internal.h \
  ${srcdir}/include/odp_packet_netmap.h \
@@ -189,7 +188,6 @@ __LIB__libodp_linux_la_SOURCES = \
   odp_packet_io.c \
   pktio/ethtool.c \
   pktio/io_ops.c \
-  pktio/ipc.c \
   pktio/pktio_common.c \
   pktio/loop.c \
   pktio/netmap.c \
-- 
2.8.1

Re: [lng-odp] [API-NEXT PATCHv3 05/16] api: shm: add flag to guarantee address unicity on all ODP threads

[Savolainen, Petri (Nokia - FI/Espoo)]

Hi,

I'll copy all API related comments into this response. Also I'd suggest that 
the next version has all three API patches merged into one, since those touch 
the same feature and lines in the file. Single patch is easier to review, find 
later in commit log and handle during merge, etc. Now all three patches 
conflict if you change any one of them.


> -Original Message-
> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> Christophe Milard
> Sent: Wednesday, October 19, 2016 6:30 PM
> To: mike.hol...@linaro.org; bill.fischo...@linaro.org; lng-
> o...@lists.linaro.org
> Subject: [lng-odp] [API-NEXT PATCHv3 05/16] api: shm: add flag to
> guarantee address unicity on all ODP threads
> 
> The ODP_SHM_SINGLE_VA flag is created: when set (at odp_shm_reserve()),
> this flag guarantees that all ODP threads sharing this memory
> block will see the block at the same address (regadless of ODP
> thread type -pthread vs process- or fork time)

This explanation would be good to move into the actual API spec under...

> 
> Signed-off-by: Christophe Milard 
> ---
>  include/odp/api/spec/shared_memory.h | 7 +++
>  1 file changed, 3 insertions(+), 4 deletions(-)
> 
> diff --git a/include/odp/api/spec/shared_memory.h
> b/include/odp/api/spec/shared_memory.h
> index 8c76807..fefb5d6 100644
> --- a/include/odp/api/spec/shared_memory.h
> +++ b/include/odp/api/spec/shared_memory.h
> @@ -45,10 +45,9 @@ extern "C" {
>  /*
>   * Shared memory flags
>   */
> -
> -/* Share level */
> -#define ODP_SHM_SW_ONLY 0x1 /**< Application SW only, no HW access */
> -#define ODP_SHM_PROC0x2 /**< Share with external processes */
> +#define ODP_SHM_SW_ONLY  0x1 /**< Application SW only, no HW
> access   */
> +#define ODP_SHM_PROC 0x2 /**< Share with external processes
> */

... something like this:

/**
 * Single virtual address
 *
 * When set, this flag guarantees that all ODP threads sharing this
 * memory block will see the block at the same address - regardless
 * of ODP thread type (e.g. pthread vs. process (or fork process time)).
 */
> +#define ODP_SHM_SINGLE_VA0x4
> 
>  /**
>   * Shared memory block info
> --
> 2.7.4


> +#define ODP_SHM_LOCK 0x8 /**< prevent swapping this memory */

Is this really needed? I think you can assume that a data plane application 
never wants that  memory (allocated from ODP) can be swapped to HDD. It would 
just ruin application performance and latency totally. So, let's leave the flag 
out and implementation will prevent swapping whenever possible.


> The flag ODP_SHM_EXPORT is added: when passed at odp_shm_reserve() time
> the memory block becomes visible to other ODP instances.
> The function odp_shm_reserve_exported() is added: this function enables to
> reserve block of memories exported by other ODP instances (using the
> ODP_SHM_EXPORT flag).

Again part of this git log text should be moved to the API spec.

For example:

/**
 * When set, the memory block becomes visible to other ODP instances 
 * through odp_shm_reserve_exported().
 * /
> +#define ODP_SHM_EXPORT   0x10

> +/**
> + * get and reserve a block of shared memory, exported by another ODP
> instance

Start document text with an upper case character.

> + *
> + * @param[in] remote_name  Name of the block, in the remote ODP instance

[in] is not needed, since far majority of parameters are [in]. We need to 
highlight only the exceptional [out] or [in, out] parameters.

> + * @param[in] odp_inst Remote ODP instance, as returned by
> odp_init_global()
> + * @param[in] local_name   Name given to the block, in the local ODP
> instance

This name can be optional (== NULL).

> + * @param[in] alignBlock alignment in bytes

When the block is already created (by the other instance), you would not be 
able to change the align (or size). Why this param is here? I'd just remove it.


> + * @param[in] flagsShared memory parameter flags (ODP_SHM_*).
> + * Default value is 0.

Same thing here. Flags were already defined by the other instance.

> + *
> + * @return A new handle to the block if it is found (must be freed when
> done).
> + * @retval ODP_SHM_INVALID on failure
> + */
> +odp_shm_t odp_shm_reserve_exported(const char *remote_name,
> +odp_instance_t odp_inst,
> +const char *local_name,
> +uint64_t align, uint32_t flags);

The name of the function should not include term "reserve", since this call 
does not reserve anything from the system, but returns a handle to an already 
existing memory block.

odp_shm_t odp_shm_lookup_remote(odp_instance_t remote_inst,
const char *remote_name,
const char *local_name);


-Petri

Re: [lng-odp] api-next needs update

[Maxim Uvarov]

Done.

On 21 October 2016 at 11:21, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia-bell-labs.com> wrote:

> Hi,
>
> Api-next needs a *quick* update to the latest level of master branch.
>
> There's now about 20 commit gap. For example, "Revert example: generator:
> actually use specified default" is not yet on api-next, which breaks 'sudo
> make check'. Also my pool optimizations cannot be rebased to api-next until
> is up to date. I guess also Christophe's patch set should be rebased on top
> of the updated api-next (latest development in master).
>
> -Petri
>
>

[lng-odp] api-next needs update

[Savolainen, Petri (Nokia - FI/Espoo)]

Hi,

Api-next needs a *quick* update to the latest level of master branch.

There's now about 20 commit gap. For example, "Revert example: generator: 
actually use specified default" is not yet on api-next, which breaks 'sudo make 
check'. Also my pool optimizations cannot be rebased to api-next until is up to 
date. I guess also Christophe's patch set should be rebased on top of the 
updated api-next (latest development in master).

-Petri

Re: [lng-odp] [API-NEXT PATCH] test: drv: shm: decrease test memory requirement

[Elo, Matias (Nokia - FI/Espoo)]

Reviewed-and-tested-by: Matias Elo 

> -Original Message-
> From: Christophe Milard [mailto:christophe.mil...@linaro.org]
> Sent: Wednesday, October 19, 2016 5:30 PM
> To: bill.fischo...@linaro.org; Elo, Matias (Nokia - FI/Espoo) 
>  bell-labs.com>; lng-odp@lists.linaro.org
> Cc: Christophe Milard 
> Subject: [API-NEXT PATCH] test: drv: shm: decrease test memory requirement
> 
> The memory consumption is proporsional to the number of ODP threads
> available. The test failed on systems with large number of CPU, due
> to outage ot pre-allocaed address space. This patch shrinks the test
> requirement.
> 
> Signed-off-by: Christophe Milard 
> ---
>  test/common_plat/validation/drv/drvshmem/drvshmem.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/test/common_plat/validation/drv/drvshmem/drvshmem.c
> b/test/common_plat/validation/drv/drvshmem/drvshmem.c
> index 5e6d2e5..559c55d 100644
> --- a/test/common_plat/validation/drv/drvshmem/drvshmem.c
> +++ b/test/common_plat/validation/drv/drvshmem/drvshmem.c
> @@ -16,7 +16,7 @@
>  #define TEST_SHARE_BAR (0xf0f0f0f)
>  #define SMALL_MEM 10
>  #define MEDIUM_MEM 4096
> -#define BIG_MEM 16777216
> +#define BIG_MEM 65536
>  #define STRESS_SIZE 32   /* power of 2 and <=256 */
>  #define STRESS_RANDOM_SZ 5
>  #define STRESS_ITERATION 5000
> --
> 2.7.4

Re: [lng-odp] [PATCH 03/10] linux-gen: ring: created common ring implementation

[Savolainen, Petri (Nokia - FI/Espoo)]

> -Original Message-
> From: Mike Holmes [mailto:mike.hol...@linaro.org]
> Sent: Thursday, October 20, 2016 9:17 PM
> To: Savolainen, Petri (Nokia - FI/Espoo)  labs.com>
> Cc: Yi He ; Maxim Uvarov ; lng-
> odp 
> Subject: Re: [lng-odp] [PATCH 03/10] linux-gen: ring: created common ring
> implementation
> 
> On 20 October 2016 at 08:12, Savolainen, Petri (Nokia - FI/Espoo)
>  wrote:
> >>
> >>
> >> I tested this piece of code, while en-queuing, below ring full
> condition
> >> cannot prevent overwriting, take an example, in case not any reader,
> >> r_tail
> >> will always be zero, and new_head is keeping increment, this condition
> is
> >> always false.
> >> odp_atomic_load_acq_u32(&ring->r_tail) == new_head
> >>
> >> Can be:
> >> do {
> >> /* current occupies in ring buffer */
> >> entries = new_head - odp_atomic_load_acq_u32(&ring->r_tail);
> >> } while (odp_unlikely(entries > ring_size));
> >>
> >> Best Regards, Yi
> >>
> >
> >
> > The ring implementation takes advantage of the fact that these rings are
> always larger than the number of items in them. E.g. a pool has a fixed,
> max number of events and the ring size is always selected to be larger
> than that. I'll add a comment that ring size must be always larger than
> max number of items.
> 
> Can we add an assert to that effect also ?
>

It's a matter of e.g. pool init code to create a ring that is larger than 'num' 
events. Ring itself don't have change to check it without performance 
degradation. Since it's not an API, but implementation internal structure, we 
can expect that the user knows what his doing with it (in those couple of 
places it's used).

-Petri

Re: [lng-odp] [PATCH 1/2] example: odp_timer_simple: decrease timer pool resolution

[Kevin Wang]

Reviewed-by: Kevin Wang 

On 21 October 2016 at 14:17, Yi He  wrote:

> Reply for Kevin Wang to add his review.
>
> Best Regards, Yi
>
> On 19 July 2016 at 02:29, Brian Brooks  wrote:
>
>> On 07/18 13:41:32, Mike Holmes wrote:
>> > Is 10 milliseconds adequate for real work ?
>> >
>> > Should this be in some way calculated depending on the platform
>> > capabilities where it is executes since this example will also be run
>> on
>> > real HW accelerated systems and for those I assume we want the better
>> > resolution.
>>
>> I think you just created a new feature request. :) Enable applications to
>> discover the maximum timer pool resolution supported by the platform.
>>
>> This example app expires its timers every two seconds, so this patch just
>> tweaks it to work (it failed on my machine) with the reference
>> implementation
>> on a generic Linux platform.
>>
>
>


-- 
Thanks,
Kevin