On 23 December 2016 at 09:15, Savolainen, Petri (Nokia - FI/Espoo)
<petri.savolai...@nokia-bell-labs.com> wrote:
>
>
>> -----Original Message-----
>> From: Christophe Milard [mailto:christophe.mil...@linaro.org]
>> Sent: Thursday, December 22, 2016 3:13 PM
>> To: Savolainen, Petri (Nokia - FI/Espoo) <petri.savolainen@nokia-bell-
>> labs.com>
>> Cc: Mike Holmes <mike.hol...@linaro.org>; Bill Fischofer
>> <bill.fischo...@linaro.org>; LNG ODP Mailman List <lng-
>> o...@lists.linaro.org>; Yi He <yi...@linaro.org>
>> Subject: Re: api for small buffer allocations
>>
>> On 22 December 2016 at 13:49, Savolainen, Petri (Nokia - FI/Espoo)
>> <petri.savolai...@nokia-bell-labs.com> wrote:
>> >
>> > HTML mail ... otherwise looks reasonable. See some comments under (for
>> the future API spec for the same feature).
>>
>> Sorry for that, Obviously google feels it should revert to HTML from
>> time to time.
>>
>> >
>> >
>> >
>> > From: Christophe Milard [mailto:christophe.mil...@linaro.org]
>> > Sent: Thursday, December 22, 2016 11:24 AM
>> > To: Savolainen, Petri (Nokia - FI/Espoo) <petri.savolainen@nokia-bell-
>> labs.com>; Mike Holmes <mike.hol...@linaro.org>; Bill Fischofer
>> <bill.fischo...@linaro.org>; LNG ODP Mailman List <lng-
>> o...@lists.linaro.org>; Yi He <yi...@linaro.org>
>> > Subject: api for small buffer allocations
>> >
>> > Hi,
>> >
>> > I am trying to sum up what was said at the arch call yesterday,
>> regarding memory allocation to figure out how to rewrite my "buddy
>> allocator for drv interface" patch series.
>> >
>> > - Some voices seemed to say that a single allocation would be best,
>> regardless of the size being allocated (mostly at the beginning of the
>> call). I objected that the current shm_reserve() interface has an overhead
>> of about 64 bytes per allocated block, and probably, the availability of
>> the shm_lookup function is not of much interest for most minimal
>> allocations. Moreover, the handle-to-address indirection is not of much
>> use if the only thing that matters is the address...
>> >
>> > -Towards the end of the call, It seemed the usage of new allocation
>> functions was less a problem, but that Petri wanted each memory pool users
>> to be able to create their own pools rather than using a predefined
>> (common) pool. Also the need for supporting fix element size allocation
>> (slab rather than buddy) was required.
>> >
>> > So here comes a alternative approach. I would like to get Petri's
>> (obviously needed) blessing on this drv API, before its final
>> implementation.
>> >
>> > /*create a pool: elmnt_sz may be zero, meaning unknown size.
>> > * This will create a pool by internally performing a _ishm_reserve()
>> > * of requested pool_size (or more, e.g. nearest power of 2) + needed
>> > * control data.
>> > * for linux-gen a size zero element would create a pool with buddy
>> > * allocation and a fixed (>0) element size would give a slab pool*/
>> > odpdrv_shm_pool_t odpdrv_shm_mempool_create(const char *name, uint64_t
>> pool_sz, uint32_t elmnt_sz);
>> >
>> >
>> >>>
>> >>> elmnt_sz is actually maximum size for the alloc(). So I'd prefer that
>> name (max_alloc_size) and documentation.
>>
>> Not sure I agree. But not worth hours of discussion. Will call it
>> max_alloc_size.
>
> @param max_size Maximum size in bytes for alloc call
>
>
>
>>
>> >>> I think both sizes should be given. Pool_size would be a sum of all
>> (simultaneous) alloc(size).
>> >>>
>>
>> Your comments applies for fixed size (slab) alloc. a size of 0 would
>> imply random sizes (buddy). Not sure what you mean by "I think both
>> sizes should be given. Pool_size would be a sum of all (simultaneous)
>> alloc(size)". both sizes ARE already given in the proposed
>> prototype.
>> "Pool_size would be a sum of all (simultaneous) alloc(size).": ??? for
>> buddy allocation, the pools size may need to be up to 50% larger the
>> sum of all allocs. For slag, Pool_size is the sum of all
>> N*element_max_size where N is the number of current allocation.
>> It that what you meant?
>
>
> // Create a pool of total 520 bytes for my allocs,
> // max alloc size is 302.
> mempool_create("foo", 530, 302);
So your "max_size", or "max_alloc_size" is 302, right? (and 530 the pool size)
What does it mean? From the arch call I understood that both buddy
allocator and fixed size (slab) allocator are wished: So what should I
do on this code:
mempool_create("foo", 530, 302);
A buddy allocator or a fixed size allocator??
My proposal goes like that:
p1 = mempool_create("foo", 530, 30); =>fixed size (slab): (rounded to
540 bytes, i.e. 18*30) all allocations are 30 bytes. 30 is not the
max size, 30 is THE (unique) size. The fact that the user can allocate
30 bytes and only use 3 of those is not of interrest for the API.
p2 = mempool_create("foo", 530, -1); =>buddy (rounded to 1K).
In my case, the choice between buddy/slab is based on the fact that
the element size is given (i.e. >0) or not.
In your proposal, I do not understand what I would base my buddy/slab
choice on. If all allocations that follows require that
max_alloc_size, slab is best. If the allocations are anything between
0 and max_alloc_size, then buddy is best (and the max_alloc_size of no
interrest)
How would I know?
Back to my proposal: (reminder: p1 is slab, p2 is buddy)
alloc(p1, -1) => OK: return a pointer to 30 bytes
alloc(p1, 2) => OK: return a pointer to 30 bytes
alloc(p1, 30) => OK: return a pointer to 30 bytes
alloc(p1, 31) => NOK: return NULL
alloc(p2, 5) => OK returns a pointer to 8 bytes
alloc(p2, 31) => OK returns a pointer to 32 bytes
Anything exceeding the pool capacity returns NULL (we seem to agree on that)
Christophe.
>
> // FAIL. Alloc size is too large. ptr0 == NULL.
> ptr0 = alloc(303);
>
> // All these succeed
> ptr0 = alloc(8);
> ptr1 = alloc(18);
> ptr2 = alloc(202);
> ptr3 = alloc(302);
> free(ptr0);
> free(ptr1);
> ptr0 = alloc(26);
>
> // FAIL. Out of memory. ptr1 == NULL.
> ptr1 = alloc(1);
>
>
>
>>
>> >
>> >
>> > /* destroy a pool */
>> > int odpdrv_shm_pool_t odpdrv_shm_mempool_destroy(odpdrv_shm_pool_t
>> pool);
>> >
>> > /* search for an existing pool
>> > * return the pool or invalid on failure */
>> > odpdrv_shm_pool_t odpdrv_shm_mempool_lookup(const char *name)
>> >
>> > /* allocate memory from a pool:
>> > * if the pool was created with an element size (for linux-gen:slab),
>> > * then the size provided here is checked to be less or equal to
>> > * the element size provided at pool creation time,
>> > * otherwise, an element of at least size bytes is allocateded (buddy)
>> > * returns NULL on error
>> > */
>> > void *odpdrv_shm_pool_alloc(odpdrv_shm_pool_t pool, uint32_t size);
>> >
>> >>>
>> >>> size <= max_alloc_size above, otherwise results are undefined.
>> Only if max_alloc_size was given, i.e. slag.
>>
>> >>>
>> >>> returns NULL if sum of all alloc() sizes exceed pool_sz above == pool
>> is empty.
>>
>> For slag (i.e. if your max_alloc_sz was > 0), returns NULL if
>> N*max_alloc_sz is larger than pool_sz, i.e: pool is FULL.
>> Is that what you meant?
>>
>
>
> See above. Alloc returns NULL when pool is out of memory, or alloc size is
> too large.
>
> -Petri
>
