To start with: happy new year :-)
Are you saying, Francois, that you want this to have support for Numa directely?
In my opinion, having this as it is now enables building lists of
things, which, in turn, enable proceeding on the driver work
(enumerators...). do you think we should delay this to support NUMA
first?
I am not convinced it is the best to do: sometimes it is shorter (in
time) to redo things than not to progress:
Having said that, you are right that redefining the interfaces
(partly, at least for NUMA) costs a bit....
Are you joining the Arch call tomorrow, Francois? Maybe a topic to take then...
Christophe.
On 2 January 2017 at 17:48, Francois Ozog <francois.o...@linaro.org> wrote:
> Sorry for joining the discussion late...
>
> here are a few points:
>
> The more we defer taking into account NUMA node in memory related
> APIs, the more we'll suffer when we'll be forced to integrate it. I
> would strongly suggest we address this topic now.
>
> Petri mentioned that we could do allocations on HW managed buffers, so
> the memory backend of the pool may have to be selected: either
> shm_reserve or something else. This in turn means that there may be a
> need for shm_ops operations and special considerations to free
> allocated "units". DPDK went through a significant rework of pool to
> accommodate external pools.
>
> Performance is a key factor for implementation. If I recall properly
> comments about NGINX bad performance, memory allocations accounted for
> 30% of time to handle a packet.
>
> FF
>
> On 27 December 2016 at 11:51, Savolainen, Petri (Nokia - FI/Espoo)
> <petri.savolai...@nokia-bell-labs.com> wrote:
>>> >> >
>>> >> > typedef struct {
>>> >> > // sum of all (simultaneous) allocs
>>> >> > uint64_t pool_size;
>>> >> >
>>> >> > // Minimum alloc size application will request from pool
>>> >> > uint32_t min_alloc;
>>> >> >
>>> >> > // Maximum alloc size application will request from pool
>>> >> > uint32_t max_alloc;
>>> >> >
>>> >> > } odp_shm_pool_param_t;
>>> >>
>>> >> That makes more sense.
>>> >> Not sure we really need a struct for 3 values, but it could well be so
>>> >> on the north interface if you want.
>>> >
>>> > With param struct it's easier to add new params in backwards compatible
>>> manner (if needed). Also, we use 'param' with all other create calls.
>>> >
>>>
>>> I think the min_alloc will mostly be either be 1 (pool for many
>>> different things unknown at pool cration time) or equal to max_alloc
>>> (Pool of one single thing). So things look a bit overkill to me.
>>>
>>> But I will do as you want. :-)
>>
>>
>> It depends how well application knows in advance about usage. For example,
>> if it uses a pool for building a data base of struct foo's (80 bytes) and
>> struct bar's (100 bytes), then it can expect best performance for min=80,
>> max=100, rather than min=1, max=100.
>>
>>
>>> >> OK. As buddy allocation garantees a minimum of 50% efficiency, we can
>>> >> do like that:
>>> >> if (min_alloc == max_alloc)
>>> >> => slab (size = pool size rounded up to ineteger nb of min_alloc)
>>> >> else
>>> >> => buddy (next_power_of_2(2*given_pool_sz))
>>> >>
>>> >> OK?
>>> >
>>> > Seems OK. Also, the algorithm is easy to tune afterward. E.g. you could
>>> use slab for bit wider range - it should be faster of the two, since it's
>>> simpler.
>>> >
>>> > if (min_alloc == max_alloc || min_alloc > 0.8*max_alloc)
>>>
>>> I guess the tipping point is 2 instead of 0.8: if max_alloc >
>>> 2*min_alloc then a buddy allocator will perform better than a slab. If
>>> not, a slab is better, because the buddy allocator will have to cope
>>> for the worse case.
>>
>>
>> The selection will be based on CPU cycle vs. memory consumption figures. Up
>> to some point, the user is willing to sacrifice higher memory usage for
>> lower CPU cycle consumption. The point will be based on cpu/memory
>> performance difference of the two implementations, system memory size, cache
>> size, etc. It may be min = 0.8*max, min = 0.5*max, or something else.
>>
>> -Petri
>>
>>
>
>
>
> --
> François-Frédéric Ozog | Director Linaro Networking Group
> T: +33.67221.6485
> francois.o...@linaro.org | Skype: ffozog
