Re: [lng-odp] [API-NEXT PATCH v3] api: pool: additional packet length configuration
On Thu, Jun 15, 2017 at 9:21 AM, Petri Savolainen
wrote:
> Added packet pool parameters for more fine grained pool
> configuration. The basic usage of the parameters is not changed,
> except that implementation may now round up 'num' by default.
> Application can limit the round up with new 'max_num' parameter.
> Another new parameter (opt) allows application give hints and
> requirements about e.g. memory to be used for a pool (or parts
> of the pool).
>
> Additionally, pool configuration may be extended with a table of
> num/len/opt values. This gives application more flexibility to
> specify requirements for various packet sizes.
>
> For example, the new specification allows to create this kind of
> pool:
>
> ^
> |
> max_num | - - - - - - - -
> |
> | *
> NUM | * *
> | * * *
> | o * * * *
> | o * * * * *
> +--->
> LEN
>
> o == small number of short packets into fast memory
> * == all other packets into DDR
>
> Signed-off-by: Petri Savolainen
> ---
> include/odp/api/spec/pool.h | 132
> +++-
> 1 file changed, 118 insertions(+), 14 deletions(-)
>
> diff --git a/include/odp/api/spec/pool.h b/include/odp/api/spec/pool.h
> index 6fc5b6b4..6b4e7495 100644
> --- a/include/odp/api/spec/pool.h
> +++ b/include/odp/api/spec/pool.h
> @@ -41,6 +41,9 @@ extern "C" {
> * Maximum pool name length in chars including null char
> */
>
> +/** Maximum number of packet pool subparameters */
> +#define ODP_POOL_PKT_SUB_MAX 7
Should this be defined in the spec or left to the implementation?
Other various MAX things are left to implementations and exposed via
odp_pool_capability(). That would seem to be more flexible.
> +
> /**
> * Pool capabilities
> */
> @@ -156,6 +159,30 @@ typedef struct odp_pool_capability_t {
> int odp_pool_capability(odp_pool_capability_t *capa);
>
> /**
> + * Pool memory type
> + */
> +typedef enum odp_pool_mem_type_t {
> + /** Default memory type */
> + ODP_POOL_MEM_DEFAULT = 0,
> +
> + /** Fast memory. Some implementations may have a restricted amount of
> +* (e.g. chip internal) pool memory that is faster access than the
> +* default memory type. */
> + ODP_POOL_MEM_FAST
Can we find a better name than "FAST" here? Why wouldn't every
application want to be fast? The idea here is that there is a memory
hierarchy that may be multi-level and present various tradeoffs in
terms of capacity and access speed. To the extent that such a
hierarchy is present on a given platform and visible to ODP
applications it may be desirable to allow the application to express
preferences in this regard. But the question is how can this be done
in a reasonably system-independent manner? Especially when we consider
operating in an NFV environment where there are hypervisors and/or
containers added into the mix.
So before we can say what needs to be input to odp_pool_param_t, don't
we need to say what is output from odp_pool_capability() to allow the
application to make informed decisions in this area? Information like:
Number of memory levels available, relative speeds and/or capacities,
etc.
Also, I continue to believe it may be premature to try to optimize in
this direction without addressing the long-deferred overall NUMA
questions and perhaps how this may also relate to header splitting. It
seems to me that lumping packets together simply by size is a very
coarse tradeoff, as if there are truly limited resources doing this on
a flow basis may make a lot more sense (e.g., high priority alerts
should get best service, etc.). Alternately, for general traffic,
being able to say that I want fast access to packet headers and are
less concerned with payload represents a large use case that's not
addressed here.
This whole area really falls into the QoS topic and so questions of
how this ties into RED and other QoS controls also comes into play. So
it may be best to omit mem_type at this stage and just focus on the
subpools. We can add mem_type info back later as part of a more
comprehensive set of strategies for advanced QoS management.
> +
> +} odp_pool_mem_type_t;
> +
> +/**
> + * Additional options for packet pool creation
> + */
> +typedef struct odp_pool_pkt_opt_t {
> + /** Pool memory type. Hint which memory type should be used for the
> +* pool or a subset of the pool. */
> + odp_pool_mem_type_t mem_type;
> +
> +} odp_pool_pkt_opt_t;
odp_pool_pkt_opt_t might be a good place to add info about header
splits, etc. Again it may be best to defer these sort of tuning
controls from this stage of the enhancement.
> +
> +/**
> * Pool parameters
> * Used to communicate pool creation options.
> * @note A single thread may not be able to allocate all 'num' elements
> @@ -185,35 +212,96 @@ typedef struct odp_pool_param_t {
>
> /** Parameters for packet pools */
> struct {
>
Re: [lng-odp] [API-NEXT PATCH v3] api: pool: additional packet length configuration
This concept for sub-pools within a pool is adding unnecessary complexity to
pool configurations. What is the purpose of this? We don't support having
multiple size buffers in same pool. In case application wants to attach
multiple pools to same pktio, we can introduce an API to attach pool to pktio.
In case of sub-pools there will be ambiguity that which sub-pool is used for
odp_packet_alloc.
Regards
Nikhil
-Original Message-
From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of Petri
Savolainen
Sent: Thursday, June 15, 2017 7:51 PM
To: lng-odp@lists.linaro.org
Subject: [lng-odp] [API-NEXT PATCH v3] api: pool: additional packet length
configuration
Added packet pool parameters for more fine grained pool configuration. The
basic usage of the parameters is not changed, except that implementation may
now round up 'num' by default.
Application can limit the round up with new 'max_num' parameter.
Another new parameter (opt) allows application give hints and requirements
about e.g. memory to be used for a pool (or parts of the pool).
Additionally, pool configuration may be extended with a table of num/len/opt
values. This gives application more flexibility to specify requirements for
various packet sizes.
For example, the new specification allows to create this kind of
pool:
^
|
max_num | - - - - - - - -
|
| *
NUM | * *
| * * *
| o * * * *
| o * * * * *
+--->
LEN
o == small number of short packets into fast memory
* == all other packets into DDR
Signed-off-by: Petri Savolainen
---
include/odp/api/spec/pool.h | 132 +++-
1 file changed, 118 insertions(+), 14 deletions(-)
diff --git a/include/odp/api/spec/pool.h b/include/odp/api/spec/pool.h index
6fc5b6b4..6b4e7495 100644
--- a/include/odp/api/spec/pool.h
+++ b/include/odp/api/spec/pool.h
@@ -41,6 +41,9 @@ extern "C" {
* Maximum pool name length in chars including null char
*/
+/** Maximum number of packet pool subparameters */ #define
+ODP_POOL_PKT_SUB_MAX 7
+
/**
* Pool capabilities
*/
@@ -156,6 +159,30 @@ typedef struct odp_pool_capability_t { int
odp_pool_capability(odp_pool_capability_t *capa);
/**
+ * Pool memory type
+ */
+typedef enum odp_pool_mem_type_t {
+ /** Default memory type */
+ ODP_POOL_MEM_DEFAULT = 0,
+
+ /** Fast memory. Some implementations may have a restricted amount of
+* (e.g. chip internal) pool memory that is faster access than the
+* default memory type. */
+ ODP_POOL_MEM_FAST
+
+} odp_pool_mem_type_t;
+
+/**
+ * Additional options for packet pool creation */ typedef struct
+odp_pool_pkt_opt_t {
+ /** Pool memory type. Hint which memory type should be used for the
+* pool or a subset of the pool. */
+ odp_pool_mem_type_t mem_type;
+
+} odp_pool_pkt_opt_t;
+
+/**
* Pool parameters
* Used to communicate pool creation options.
* @note A single thread may not be able to allocate all 'num' elements @@
-185,35 +212,96 @@ typedef struct odp_pool_param_t {
/** Parameters for packet pools */
struct {
- /** The number of packets that the pool must provide
- that are packet length 'len' bytes or smaller.
- The maximum value is defined by pool capability
- pkt.max_num. */
+ /** The minimum number of packets that are packet length
+* 'len' bytes or smaller. The maximum value is defined
+* by pool capability pkt.max_num. An implementation
+* may round up the value, as long as the 'max_num'
+* parameter below is not violated.
+*/
uint32_t num;
- /** Minimum packet length that the pool must provide
- 'num' packets. The number of packets may be less
- than 'num' when packets are larger than 'len'.
- The maximum value is defined by pool capability
- pkt.max_len. Use 0 for default. */
+ /** The minimum packet length that at least 'num'
+* packets are required. The maximum value is defined
+* by pool capability pkt.max_len. Use 0 for default.
+*/
uint32_t len;
+ /** Packet pool options
+*
+* Options contain additional hints and requirements,
+* which quide implementation e.g. to select correct
+* memory type for the pool.
+*/
+ odp_pool_pkt_opt_t opt;
+
+
Re: [lng-odp] [API-NEXT PATCH v3] api: pool: additional packet length configuration
This patch does not define sub-pools. It gives application a way to specify more fine grained how many packets of each size is needed. Implementation may or may not be based on sub-pools. Pool create still returns one pool (odp_pool_t) handle. E.g. instead of today's: num = 4k len = 1.5k max_len = 64k application may give: num[0] = 4k len[0] = 1.5k num[1] = 256 len[1] = 9k num[2] = 32 len[2] = 64k max_len = 64k max_num = 8k which tells more accurately what is needed. Implementation may calculate optimal segment size(s) and round up (up to max_num). Today: max_num == num, and application cannot tell how many e.g. 64k packets it needs. In the example above, one 64k packet would be enough by the spec. -Petri > -Original Message- > From: Nikhil Agarwal [mailto:nikhil.agar...@nxp.com] > Sent: Friday, June 23, 2017 7:58 AM > To: Petri Savolainen ; lng- > o...@lists.linaro.org > Subject: RE: [lng-odp] [API-NEXT PATCH v3] api: pool: additional packet > length configuration > > This concept for sub-pools within a pool is adding unnecessary complexity > to pool configurations. What is the purpose of this? We don't support > having multiple size buffers in same pool. In case application wants to > attach multiple pools to same pktio, we can introduce an API to attach > pool to pktio. In case of sub-pools there will be ambiguity that which > sub-pool is used for odp_packet_alloc. > > Regards > Nikhil > > -Original Message- > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of Petri > Savolainen > Sent: Thursday, June 15, 2017 7:51 PM > To: lng-odp@lists.linaro.org > Subject: [lng-odp] [API-NEXT PATCH v3] api: pool: additional packet length > configuration > > Added packet pool parameters for more fine grained pool configuration. The > basic usage of the parameters is not changed, except that implementation > may now round up 'num' by default. > Application can limit the round up with new 'max_num' parameter. > Another new parameter (opt) allows application give hints and requirements > about e.g. memory to be used for a pool (or parts of the pool). > > Additionally, pool configuration may be extended with a table of > num/len/opt values. This gives application more flexibility to specify > requirements for various packet sizes. > > For example, the new specification allows to create this kind of > pool: > > ^ > | > max_num | - - - - - - - - > | > | * > NUM | * * > | * * * > | o * * * * > | o * * * * * > +---> > LEN > > o == small number of short packets into fast memory > * == all other packets into DDR
Re: [lng-odp] [API-NEXT PATCH v3] api: pool: additional packet length configuration
Regards,
Bala
On 15 June 2017 at 19:51, Petri Savolainen
wrote:
> Added packet pool parameters for more fine grained pool
> configuration. The basic usage of the parameters is not changed,
> except that implementation may now round up 'num' by default.
> Application can limit the round up with new 'max_num' parameter.
> Another new parameter (opt) allows application give hints and
> requirements about e.g. memory to be used for a pool (or parts
> of the pool).
>
> Additionally, pool configuration may be extended with a table of
> num/len/opt values. This gives application more flexibility to
> specify requirements for various packet sizes.
>
> For example, the new specification allows to create this kind of
> pool:
>
> ^
> |
> max_num | - - - - - - - -
> |
> | *
> NUM | * *
> | * * *
> | o * * * *
> | o * * * * *
> +--->
> LEN
>
> o == small number of short packets into fast memory
> * == all other packets into DDR
>
> Signed-off-by: Petri Savolainen
> ---
> include/odp/api/spec/pool.h | 132 ++
> +-
> 1 file changed, 118 insertions(+), 14 deletions(-)
>
> diff --git a/include/odp/api/spec/pool.h b/include/odp/api/spec/pool.h
> index 6fc5b6b4..6b4e7495 100644
> --- a/include/odp/api/spec/pool.h
> +++ b/include/odp/api/spec/pool.h
> @@ -41,6 +41,9 @@ extern "C" {
> * Maximum pool name length in chars including null char
> */
>
> +/** Maximum number of packet pool subparameters */
> +#define ODP_POOL_PKT_SUB_MAX 7
> +
> /**
> * Pool capabilities
> */
> @@ -156,6 +159,30 @@ typedef struct odp_pool_capability_t {
> int odp_pool_capability(odp_pool_capability_t *capa);
>
> /**
> + * Pool memory type
> + */
> +typedef enum odp_pool_mem_type_t {
> + /** Default memory type */
> + ODP_POOL_MEM_DEFAULT = 0,
> +
> + /** Fast memory. Some implementations may have a restricted amount
> of
> +* (e.g. chip internal) pool memory that is faster access than the
> +* default memory type. */
> + ODP_POOL_MEM_FAST
> +
> +} odp_pool_mem_type_t;
>
We need to add this to capability for implementations to expose which is
supported.
> +
> +/**
> + * Additional options for packet pool creation
> + */
> +typedef struct odp_pool_pkt_opt_t {
> + /** Pool memory type. Hint which memory type should be used for the
> +* pool or a subset of the pool. */
> + odp_pool_mem_type_t mem_type;
> +
> +} odp_pool_pkt_opt_t;
> +
> +/**
> * Pool parameters
> * Used to communicate pool creation options.
> * @note A single thread may not be able to allocate all 'num' elements
> @@ -185,35 +212,96 @@ typedef struct odp_pool_param_t {
>
> /** Parameters for packet pools */
> struct {
> - /** The number of packets that the pool must
> provide
> - that are packet length 'len' bytes or smaller.
> - The maximum value is defined by pool capability
> - pkt.max_num. */
> + /** The minimum number of packets that are packet
> length
> +* 'len' bytes or smaller. The maximum value is
> defined
> +* by pool capability pkt.max_num. An
> implementation
> +* may round up the value, as long as the
> 'max_num'
> +* parameter below is not violated.
> +*/
> uint32_t num;
>
> - /** Minimum packet length that the pool must
> provide
> - 'num' packets. The number of packets may be
> less
> - than 'num' when packets are larger than 'len'.
> - The maximum value is defined by pool capability
> - pkt.max_len. Use 0 for default. */
> + /** The minimum packet length that at least 'num'
> +* packets are required. The maximum value is
> defined
> +* by pool capability pkt.max_len. Use 0 for
> default.
> +*/
> uint32_t len;
>
> + /** Packet pool options
> +*
> +* Options contain additional hints and
> requirements,
> +* which quide implementation e.g. to select
> correct
> +* memory type for the pool.
> +*/
> + odp_pool_pkt_opt_t opt;
>
If lets say the implementation does not support MEM_FAST then it could be
documented saying it will revert to DEFAULT memory type.
+
> + /** Number of subparameters
> +*
> +* The number of subparameter table (pkt.sub[])
> entries
> +
