Re: [lng-odp] [API-NEXT PATCHv7 1/6] linux-gen: _ishm: adding buddy and slab allocation

[Maxim Uvarov]

Merged,
Maxim.

On 01/13/17 10:33, Yi He wrote:
> For this patch series:
> 
> Reviewed-and-tested-by: Yi He 
> 
> 
> 
> On 13 January 2017 at 15:55, Christophe Milard > wrote:
> 
>> _ishm now provides functions to create/destroy pools for buddy/slab
>> memory allocation, as well as functions to allocated/release memory
>> from the created pools.
>>
>> Signed-off-by: Christophe Milard 
>> ---
>>  platform/linux-generic/Makefile.am |   2 +
>>  platform/linux-generic/_ishm.c |  14 +-
>>  platform/linux-generic/_ishmpool.c | 811
>> +
>>  .../linux-generic/include/_ishmpool_internal.h |  56 ++
>>  4 files changed, 882 insertions(+), 1 deletion(-)
>>  create mode 100644 platform/linux-generic/_ishmpool.c
>>  create mode 100644 platform/linux-generic/include/_ishmpool_internal.h
>>
>> diff --git a/platform/linux-generic/Makefile.am b/platform/linux-generic/
>> Makefile.am
>> index 6bbe775..d615088 100644
>> --- a/platform/linux-generic/Makefile.am
>> +++ b/platform/linux-generic/Makefile.am
>> @@ -127,6 +127,7 @@ noinst_HEADERS = \
>>   ${srcdir}/include/_fdserver_internal.h \
>>   ${srcdir}/include/_ishm_internal.h \
>>   ${srcdir}/include/_ishmphy_internal.h \
>> + ${srcdir}/include/_ishmpool_internal.h \
>>   ${srcdir}/include/odp_align_internal.h \
>>   ${srcdir}/include/odp_atomic_internal.h \
>>   ${srcdir}/include/odp_buffer_inlines.h \
>> @@ -172,6 +173,7 @@ __LIB__libodp_linux_la_SOURCES = \
>>_fdserver.c \
>>_ishm.c \
>>_ishmphy.c \
>> +  _ishmpool.c \
>>odp_atomic.c \
>>odp_barrier.c \
>>odp_bitmap.c \
>> diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_
>> ishm.c
>> index 23f620d..4c2578b 100644
>> --- a/platform/linux-generic/_ishm.c
>> +++ b/platform/linux-generic/_ishm.c
>> @@ -59,6 +59,7 @@
>>  #include <_fdserver_internal.h>
>>  #include <_ishm_internal.h>
>>  #include <_ishmphy_internal.h>
>> +#include <_ishmpool_internal.h>
>>  #include 
>>  #include 
>>  #include 
>> @@ -1441,8 +1442,19 @@ int _odp_ishm_init_global(void)
>>  * is performed for the main thread... Many init_global() functions
>>  * indeed assume the availability of odp_shm_reserve()...:
>>  */
>> -   return do_odp_ishm_init_local();
>> +   if (do_odp_ishm_init_local()) {
>> +   ODP_ERR("unable to init the main thread\n.");
>> +   goto init_glob_err4;
>> +   }
>> +
>> +   /* get ready to create pools: */
>> +   _odp_ishm_pool_init();
>>
>> +   return 0;
>> +
>> +init_glob_err4:
>> +   if (_odp_ishmphy_unbook_va())
>> +   ODP_ERR("unable to unbook virtual space\n.");
>>  init_glob_err3:
>> if (munmap(ishm_ftbl, sizeof(ishm_ftable_t)) < 0)
>> ODP_ERR("unable to munmap main fragment table\n.");
>> diff --git a/platform/linux-generic/_ishmpool.c b/platform/linux-generic/_
>> ishmpool.c
>> new file mode 100644
>> index 000..df6e49e
>> --- /dev/null
>> +++ b/platform/linux-generic/_ishmpool.c
>> @@ -0,0 +1,811 @@
>> +/* Copyright (c) 2017, Linaro Limited
>> + * All rights reserved.
>> + *
>> + * SPDX-License-Identifier: BSD-3-Clause
>> + */
>> +
>> +/* This file gathers the buddy and slab allocation functionality provided
>> + * by _ishm.
>> + * _odp_ishmpool_create() can be used to create a pool for buddy/slab
>> + * allocation. _odp_ishmpool_create() will allocate a memory area using
>> + * ishm_reserve() for both the control part (needed for tracking
>> + * allocation/free...) and the user memory itself (part of which will be
>> given
>> + * at each ishmpool_alloc()).
>> + * The element size provided at pool creation time determines whether
>> + * to pool will of type buddy or slab.
>> + * For buddy, all allocations are rounded to the nearest power of 2.
>> + *
>> + * The implementation of the buddy allocator is very traditional: it
>> + * maintains N lists of free buffers.
>> + * The control part actually contains these N queue heads, (N-M are
>> actually
>> + * used), the free buffers themselves being used for chaining (the
>> chaining info
>> + * is in the buffers: as they are "free" they should not be touched by the
>> + * user). The control part also contains a array of bytes for remembering
>> + * the size (actually the order) of the allocated buffers:
>> + * There are 2^(N-M) such bytes, this number being the maximum number of
>> + * allocated buffers (when all allocation are <= 2^M bytes)
>> + * Buddy allocators handle fragmentation by splitting or merging blocks
>> by 2.
>> + * They guarantee a minimum efficiency of 50%, at worse case

Re: [lng-odp] [API-NEXT PATCHv7 1/6] linux-gen: _ishm: adding buddy and slab allocation

[Yi He]

For this patch series:

Reviewed-and-tested-by: Yi He 



On 13 January 2017 at 15:55, Christophe Milard  wrote:

> _ishm now provides functions to create/destroy pools for buddy/slab
> memory allocation, as well as functions to allocated/release memory
> from the created pools.
>
> Signed-off-by: Christophe Milard 
> ---
>  platform/linux-generic/Makefile.am |   2 +
>  platform/linux-generic/_ishm.c |  14 +-
>  platform/linux-generic/_ishmpool.c | 811
> +
>  .../linux-generic/include/_ishmpool_internal.h |  56 ++
>  4 files changed, 882 insertions(+), 1 deletion(-)
>  create mode 100644 platform/linux-generic/_ishmpool.c
>  create mode 100644 platform/linux-generic/include/_ishmpool_internal.h
>
> diff --git a/platform/linux-generic/Makefile.am b/platform/linux-generic/
> Makefile.am
> index 6bbe775..d615088 100644
> --- a/platform/linux-generic/Makefile.am
> +++ b/platform/linux-generic/Makefile.am
> @@ -127,6 +127,7 @@ noinst_HEADERS = \
>   ${srcdir}/include/_fdserver_internal.h \
>   ${srcdir}/include/_ishm_internal.h \
>   ${srcdir}/include/_ishmphy_internal.h \
> + ${srcdir}/include/_ishmpool_internal.h \
>   ${srcdir}/include/odp_align_internal.h \
>   ${srcdir}/include/odp_atomic_internal.h \
>   ${srcdir}/include/odp_buffer_inlines.h \
> @@ -172,6 +173,7 @@ __LIB__libodp_linux_la_SOURCES = \
>_fdserver.c \
>_ishm.c \
>_ishmphy.c \
> +  _ishmpool.c \
>odp_atomic.c \
>odp_barrier.c \
>odp_bitmap.c \
> diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_
> ishm.c
> index 23f620d..4c2578b 100644
> --- a/platform/linux-generic/_ishm.c
> +++ b/platform/linux-generic/_ishm.c
> @@ -59,6 +59,7 @@
>  #include <_fdserver_internal.h>
>  #include <_ishm_internal.h>
>  #include <_ishmphy_internal.h>
> +#include <_ishmpool_internal.h>
>  #include 
>  #include 
>  #include 
> @@ -1441,8 +1442,19 @@ int _odp_ishm_init_global(void)
>  * is performed for the main thread... Many init_global() functions
>  * indeed assume the availability of odp_shm_reserve()...:
>  */
> -   return do_odp_ishm_init_local();
> +   if (do_odp_ishm_init_local()) {
> +   ODP_ERR("unable to init the main thread\n.");
> +   goto init_glob_err4;
> +   }
> +
> +   /* get ready to create pools: */
> +   _odp_ishm_pool_init();
>
> +   return 0;
> +
> +init_glob_err4:
> +   if (_odp_ishmphy_unbook_va())
> +   ODP_ERR("unable to unbook virtual space\n.");
>  init_glob_err3:
> if (munmap(ishm_ftbl, sizeof(ishm_ftable_t)) < 0)
> ODP_ERR("unable to munmap main fragment table\n.");
> diff --git a/platform/linux-generic/_ishmpool.c b/platform/linux-generic/_
> ishmpool.c
> new file mode 100644
> index 000..df6e49e
> --- /dev/null
> +++ b/platform/linux-generic/_ishmpool.c
> @@ -0,0 +1,811 @@
> +/* Copyright (c) 2017, Linaro Limited
> + * All rights reserved.
> + *
> + * SPDX-License-Identifier: BSD-3-Clause
> + */
> +
> +/* This file gathers the buddy and slab allocation functionality provided
> + * by _ishm.
> + * _odp_ishmpool_create() can be used to create a pool for buddy/slab
> + * allocation. _odp_ishmpool_create() will allocate a memory area using
> + * ishm_reserve() for both the control part (needed for tracking
> + * allocation/free...) and the user memory itself (part of which will be
> given
> + * at each ishmpool_alloc()).
> + * The element size provided at pool creation time determines whether
> + * to pool will of type buddy or slab.
> + * For buddy, all allocations are rounded to the nearest power of 2.
> + *
> + * The implementation of the buddy allocator is very traditional: it
> + * maintains N lists of free buffers.
> + * The control part actually contains these N queue heads, (N-M are
> actually
> + * used), the free buffers themselves being used for chaining (the
> chaining info
> + * is in the buffers: as they are "free" they should not be touched by the
> + * user). The control part also contains a array of bytes for remembering
> + * the size (actually the order) of the allocated buffers:
> + * There are 2^(N-M) such bytes, this number being the maximum number of
> + * allocated buffers (when all allocation are <= 2^M bytes)
> + * Buddy allocators handle fragmentation by splitting or merging blocks
> by 2.
> + * They guarantee a minimum efficiency of 50%, at worse case
> fragmentation.
> + *
> + * Slab implementation is even simpler, all free elements being queued in
> + * one single queue at init, taken from this queue when allocated and