Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Bill Fischofer]

I'd like to discuss this during today's ODP call as there are lots of good
ideas being discussed here.

Bill

On Tue, Apr 7, 2015 at 7:28 AM, Jerin Jacob 
wrote:

> On Tue, Apr 07, 2015 at 01:58:51PM +0300, Taras Kondratiuk wrote:
> > On 04/07/2015 12:40 PM, Jerin Jacob wrote:
> > >On Tue, Apr 07, 2015 at 12:01:30PM +0300, Taras Kondratiuk wrote:
> > >>On 04/06/2015 07:41 PM, Bill Fischofer wrote:
> > >>>I would call these "pool groups" for symmetry with "queue groups" and
> so
> > >>>the API would be odp_pool_create_group(), odp_pool_destroy_group(),
> etc.
> > >>
> > >>If it is called "pool group", then it sounds like a separate
> > >>abstraction. Which in turn needs a separate type and new API functions
> > >>to destroy or attach to somewhere (pktio, CoS, etc.).
> > >
> > >We have introduced the new classification term ODP_PMR_LEN to
> > >address "Segmentation optimization" use case by attaching
> > >different pools based on packet len.
> > >
> > >Are we introducing the new composite pool schematics because
> > >ODP_PMR_LEN cannot implemented in hardware ?
> >
> > Hi Jerin,
> >
> > I've described in this thread why ODP_PMR_LEN does not address
> > this use-case correctly. It sets too strict classification rules,
> > which are not needed in this use-case.
>
> Hi Taras,
>
> Got the use-case description from the thread. If application really
> _demands_ for such
> fine grained memory optimization then composite pool OR additional hint in
> the pool creation
> is the way to go
>
>
>
>
>
>
>
>
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Jerin Jacob]

On Tue, Apr 07, 2015 at 01:58:51PM +0300, Taras Kondratiuk wrote:
> On 04/07/2015 12:40 PM, Jerin Jacob wrote:
> >On Tue, Apr 07, 2015 at 12:01:30PM +0300, Taras Kondratiuk wrote:
> >>On 04/06/2015 07:41 PM, Bill Fischofer wrote:
> >>>I would call these "pool groups" for symmetry with "queue groups" and so
> >>>the API would be odp_pool_create_group(), odp_pool_destroy_group(), etc.
> >>
> >>If it is called "pool group", then it sounds like a separate
> >>abstraction. Which in turn needs a separate type and new API functions
> >>to destroy or attach to somewhere (pktio, CoS, etc.).
> >
> >We have introduced the new classification term ODP_PMR_LEN to
> >address "Segmentation optimization" use case by attaching
> >different pools based on packet len.
> >
> >Are we introducing the new composite pool schematics because
> >ODP_PMR_LEN cannot implemented in hardware ?
> 
> Hi Jerin,
> 
> I've described in this thread why ODP_PMR_LEN does not address
> this use-case correctly. It sets too strict classification rules,
> which are not needed in this use-case.

Hi Taras,

Got the use-case description from the thread. If application really _demands_ 
for such
fine grained memory optimization then composite pool OR additional hint in the 
pool creation
is the way to go









___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Taras Kondratiuk]

On 04/07/2015 12:40 PM, Jerin Jacob wrote:

On Tue, Apr 07, 2015 at 12:01:30PM +0300, Taras Kondratiuk wrote:

On 04/06/2015 07:41 PM, Bill Fischofer wrote:

I would call these "pool groups" for symmetry with "queue groups" and so
the API would be odp_pool_create_group(), odp_pool_destroy_group(), etc.


If it is called "pool group", then it sounds like a separate
abstraction. Which in turn needs a separate type and new API functions
to destroy or attach to somewhere (pktio, CoS, etc.).


We have introduced the new classification term ODP_PMR_LEN to
address "Segmentation optimization" use case by attaching
different pools based on packet len.

Are we introducing the new composite pool schematics because
ODP_PMR_LEN cannot implemented in hardware ?


Hi Jerin,

I've described in this thread why ODP_PMR_LEN does not address
this use-case correctly. It sets too strict classification rules,
which are not needed in this use-case.
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Jerin Jacob]

On Tue, Apr 07, 2015 at 12:01:30PM +0300, Taras Kondratiuk wrote:
> On 04/06/2015 07:41 PM, Bill Fischofer wrote:
> >I would call these "pool groups" for symmetry with "queue groups" and so
> >the API would be odp_pool_create_group(), odp_pool_destroy_group(), etc.
> 
> If it is called "pool group", then it sounds like a separate
> abstraction. Which in turn needs a separate type and new API functions
> to destroy or attach to somewhere (pktio, CoS, etc.).

We have introduced the new classification term ODP_PMR_LEN to 
address "Segmentation optimization" use case by attaching
different pools based on packet len.

Are we introducing the new composite pool schematics because
ODP_PMR_LEN cannot implemented in hardware ?


> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Taras Kondratiuk]

On 04/06/2015 07:41 PM, Bill Fischofer wrote:

I would call these "pool groups" for symmetry with "queue groups" and so
the API would be odp_pool_create_group(), odp_pool_destroy_group(), etc.


If it is called "pool group", then it sounds like a separate
abstraction. Which in turn needs a separate type and new API functions
to destroy or attach to somewhere (pktio, CoS, etc.).
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Zhujianhua]

Yes, I want to split traffic between two applications via classifier. The use 
case look like this:

> eth0 +++> Parser +++> Classifier +++> Separate Queues/Pools for App 1 
---> Scheduler ---> Worker threads(Packets Processing) ---> Egress Queues for 
App 1 ++> Tx Scheduler ++> Shaper++>eth0
   +
+
   +
+
   > Separate Queues/Pools for App 2 ---> 
Scheduler ---> Worker threads (Packets Processing) > Egress Queues for App 
2  

Parser/Classifier/Tx Scheduler/Traffic Shaper will be shared by two data plane 
applications while the other ODP items (Pool/Queue/Scheduler) are private.  

BR / Jianhua 

-Original Message-
From: Taras Kondratiuk [mailto:taras.kondrat...@linaro.org] 
Sent: 2015年4月4日 0:31
To: Zhujianhua; Jerin Jacob
Cc: lng-odp@lists.linaro.org
Subject: Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

On 03/30/2015 03:34 PM, Zhujianhua wrote:
> @ Jerin Jacob:
> What will happen if odp_cos_set_pool(odp_cos_t cos_id, odp_buffer_pool_t 
> pool_id) was called twice?
> Will the new pool_id replace the old one or the CoS have two pools?
>
> @Taras:
> Assume: set several pools per pktio interface.
> What will happen if two data plane applications share one physical Ethernet 
> port to receive packets?
> Since the pool is per pktio interface, will these two applications share the 
> same buffer pool?
> If there is memory leak in one application, the other application will be 
> disturbed.
> Correct me if my understanding were wrong.

That's a nice question. I'm afraid this use-case was not considered before.
Do you want to split traffic between two applications via classifier?

>
> Maybe to let each CoS have more than one pool and limit the max number of 
> Pool to for example 4 (Let the application designer decide how many pools are 
> needed for each CoS) could be one option.

That is a possible solution.
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Bala Manoharan]

I would like to get application use-case for different scenarios where
this would be useful before finalizing "pool groups" and their API
signature.
Also in the above proposal it is not possible to combine multiple
pools to form a pool group, I like this idea as it gives freedom for
implementation to allocate and optimize the individual pools.

Regards,
Bala

On 6 April 2015 at 22:11, Bill Fischofer  wrote:
> I would call these "pool groups" for symmetry with "queue groups" and so the
> API would be odp_pool_create_group(), odp_pool_destroy_group(), etc.
>
> On Mon, Apr 6, 2015 at 11:35 AM, Taras Kondratiuk
>  wrote:
>>
>> On 04/06/2015 03:31 PM, Bill Fischofer wrote:
>> > The /expression/ may be linear, but that doesn't imply that is how any
>> > given implementation needs to realize the expression.  Since PMRs are
>> > reasonably static, I'd expect them to be "compiled" into whatever native
>> > classification capabilities are present in the platform.  Real
>> > classification is typically done in parallel by the HW as the packet is
>> > coming "off the wire".  This is necessary because one of the outputs of
>> > classification is pool selection, so all of this happens while the
>> > packet is in the HW's receive FIFO before the DMA engines are told where
>> > to store it.
>>
>> Following our discussion on a call.
>>
>> Choosing a pool on ingress solves two separate issues:
>> 1. Isolation - packets that belong to different CoS'es may need to use
>>separate packet pools. Pool choice is strictly deterministic.
>>This case is covered by current API.
>> 2. Segmentation optimization - application may want to sort packets of
>>different size into pools with different segment size. In this case
>>pool choice is not very strict. For example a small packet can be
>>allocated from a pool with big segments if small-segment pool is
>>empty. This case can't be expressed with a current API.
>>
>> Assigning several pools to CoS and allowing implementation to pick an
>> optimal pool would be one option to solve #2.
>> During a meeting Maxim proposed to use a 'composite' pool instead, and
>> I like this idea more. I imagine something like this:
>>
>> /**
>>  * Create a composite pool
>>  *
>>  * This routine is used to create a pool which logically consists of
>>  * a set of sub-pools. Each sub-pool has its own configuration
>>  * parameters. Only pool of ODP_POOL_PACKET type can be created.
>>  * On allocation an implementation will choose an optimal sub-pool to
>>  * allocate a packet from.
>>  * A composite pool can be used to optimize memory usage and minimize
>>  * packet segmentation.
>>  *
>>  * @param name  Name of the pool, max ODP_POOL_NAME_LEN-1 chars.
>>  *  May be specified as NULL for anonymous pools.
>>  *
>>  * @param shm   The shared memory object in which to create the pool.
>>  *  Use ODP_SHM_NULL to reserve default memory type
>>  *  for the pool type.
>>  *
>>  * @param num_pools Number of sub-pools in a composite pool
>>  *
>>  * @param paramsArray of sub-pools' parameters.
>>  *
>>  * @return Handle of the created pool
>>  * @retval ODP_POOL_INVALID  Pool could not be created
>>  */
>>
>> odp_pool_t odp_pool_create_composite(const char *name,
>>odp_shm_t shm,
>>uint32_t num_pools,
>>odp_pool_param_t *params[]);
>>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Bill Fischofer]

I would call these "pool groups" for symmetry with "queue groups" and so
the API would be odp_pool_create_group(), odp_pool_destroy_group(), etc.

On Mon, Apr 6, 2015 at 11:35 AM, Taras Kondratiuk <
taras.kondrat...@linaro.org> wrote:

> On 04/06/2015 03:31 PM, Bill Fischofer wrote:
> > The /expression/ may be linear, but that doesn't imply that is how any
> > given implementation needs to realize the expression.  Since PMRs are
> > reasonably static, I'd expect them to be "compiled" into whatever native
> > classification capabilities are present in the platform.  Real
> > classification is typically done in parallel by the HW as the packet is
> > coming "off the wire".  This is necessary because one of the outputs of
> > classification is pool selection, so all of this happens while the
> > packet is in the HW's receive FIFO before the DMA engines are told where
> > to store it.
>
> Following our discussion on a call.
>
> Choosing a pool on ingress solves two separate issues:
> 1. Isolation - packets that belong to different CoS'es may need to use
>separate packet pools. Pool choice is strictly deterministic.
>This case is covered by current API.
> 2. Segmentation optimization - application may want to sort packets of
>different size into pools with different segment size. In this case
>pool choice is not very strict. For example a small packet can be
>allocated from a pool with big segments if small-segment pool is
>empty. This case can't be expressed with a current API.
>
> Assigning several pools to CoS and allowing implementation to pick an
> optimal pool would be one option to solve #2.
> During a meeting Maxim proposed to use a 'composite' pool instead, and
> I like this idea more. I imagine something like this:
>
> /**
>  * Create a composite pool
>  *
>  * This routine is used to create a pool which logically consists of
>  * a set of sub-pools. Each sub-pool has its own configuration
>  * parameters. Only pool of ODP_POOL_PACKET type can be created.
>  * On allocation an implementation will choose an optimal sub-pool to
>  * allocate a packet from.
>  * A composite pool can be used to optimize memory usage and minimize
>  * packet segmentation.
>  *
>  * @param name  Name of the pool, max ODP_POOL_NAME_LEN-1 chars.
>  *  May be specified as NULL for anonymous pools.
>  *
>  * @param shm   The shared memory object in which to create the pool.
>  *  Use ODP_SHM_NULL to reserve default memory type
>  *  for the pool type.
>  *
>  * @param num_pools Number of sub-pools in a composite pool
>  *
>  * @param paramsArray of sub-pools' parameters.
>  *
>  * @return Handle of the created pool
>  * @retval ODP_POOL_INVALID  Pool could not be created
>  */
>
> odp_pool_t odp_pool_create_composite(const char *name,
>odp_shm_t shm,
>uint32_t num_pools,
>odp_pool_param_t *params[]);
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Taras Kondratiuk]

On 04/06/2015 03:31 PM, Bill Fischofer wrote:
> The /expression/ may be linear, but that doesn't imply that is how any 
> given implementation needs to realize the expression.  Since PMRs are 
> reasonably static, I'd expect them to be "compiled" into whatever native 
> classification capabilities are present in the platform.  Real 
> classification is typically done in parallel by the HW as the packet is 
> coming "off the wire".  This is necessary because one of the outputs of 
> classification is pool selection, so all of this happens while the 
> packet is in the HW's receive FIFO before the DMA engines are told where 
> to store it.

Following our discussion on a call.

Choosing a pool on ingress solves two separate issues:
1. Isolation - packets that belong to different CoS'es may need to use
   separate packet pools. Pool choice is strictly deterministic.
   This case is covered by current API.
2. Segmentation optimization - application may want to sort packets of
   different size into pools with different segment size. In this case
   pool choice is not very strict. For example a small packet can be
   allocated from a pool with big segments if small-segment pool is
   empty. This case can't be expressed with a current API.

Assigning several pools to CoS and allowing implementation to pick an
optimal pool would be one option to solve #2.
During a meeting Maxim proposed to use a 'composite' pool instead, and
I like this idea more. I imagine something like this:

/**
 * Create a composite pool
 *
 * This routine is used to create a pool which logically consists of
 * a set of sub-pools. Each sub-pool has its own configuration
 * parameters. Only pool of ODP_POOL_PACKET type can be created.
 * On allocation an implementation will choose an optimal sub-pool to
 * allocate a packet from.
 * A composite pool can be used to optimize memory usage and minimize
 * packet segmentation.
 *
 * @param name  Name of the pool, max ODP_POOL_NAME_LEN-1 chars.
 *  May be specified as NULL for anonymous pools.
 *
 * @param shm   The shared memory object in which to create the pool.
 *  Use ODP_SHM_NULL to reserve default memory type
 *  for the pool type.
 *
 * @param num_pools Number of sub-pools in a composite pool
 *
 * @param paramsArray of sub-pools' parameters.
 *
 * @return Handle of the created pool
 * @retval ODP_POOL_INVALID  Pool could not be created
 */

odp_pool_t odp_pool_create_composite(const char *name,
   odp_shm_t shm,
   uint32_t num_pools,
   odp_pool_param_t *params[]);

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Bill Fischofer]

The *expression* may be linear, but that doesn't imply that is how any
given implementation needs to realize the expression.  Since PMRs are
reasonably static, I'd expect them to be "compiled" into whatever native
classification capabilities are present in the platform.  Real
classification is typically done in parallel by the HW as the packet is
coming "off the wire".  This is necessary because one of the outputs of
classification is pool selection, so all of this happens while the packet
is in the HW's receive FIFO before the DMA engines are told where to store
it.

If you have an alternative API proposal let's discuss that.

On Mon, Apr 6, 2015 at 7:16 AM, Taras Kondratiuk <
taras.kondrat...@linaro.org> wrote:

> On 04/06/2015 02:04 PM, Bill Fischofer wrote:
>
>> Isn't that just another term in the PMR?  If I have a flow with four
>> different sized packets that I want to put in different pools then I
>> simply have a PMR to identify the flow as input to other PMRs that then
>> segregate by pkt_len into four final CoSes which map the flow to the
>> same queue but different pools.
>>
>
> Currently yes, it is a term in PMR and you can model it in that way.
> But in fact you are linearizing two orthogonal parameters. As a result
> you will have overcomplicated model with not necessary redundancy.
> 5 CoS with the same destination queue but different pools. Plus 4
> additional PRMs. Instead of just one CoS with 4 pools and one PMR.
>
> Another issue with a current approach: it can't be implemented on KS2.
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Taras Kondratiuk]

On 04/06/2015 02:04 PM, Bill Fischofer wrote:

Isn't that just another term in the PMR?  If I have a flow with four
different sized packets that I want to put in different pools then I
simply have a PMR to identify the flow as input to other PMRs that then
segregate by pkt_len into four final CoSes which map the flow to the
same queue but different pools.


Currently yes, it is a term in PMR and you can model it in that way.
But in fact you are linearizing two orthogonal parameters. As a result
you will have overcomplicated model with not necessary redundancy.
5 CoS with the same destination queue but different pools. Plus 4
additional PRMs. Instead of just one CoS with 4 pools and one PMR.

Another issue with a current approach: it can't be implemented on KS2.
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Bala Manoharan]

On 6 April 2015 at 16:34, Bill Fischofer  wrote:
> Isn't that just another term in the PMR?  If I have a flow with four
> different sized packets that I want to put in different pools then I simply
> have a PMR to identify the flow as input to other PMRs that then segregate
> by pkt_len into four final CoSes which map the flow to the same queue but
> different pools.

I have the same opinion as Bill and I believe this approach gives
freedom to application to configure which packet_len should go into
which pool.

Regards,
Bala
>
> On Mon, Apr 6, 2015 at 5:28 AM, Taras Kondratiuk
>  wrote:
>>
>> On 04/03/2015 07:53 PM, Bala Manoharan wrote:
>>>
>>> On 3 April 2015 at 22:00, Taras Kondratiuk 
>>> wrote:

 On 03/30/2015 03:34 PM, Zhujianhua wrote:
>
>
> @ Jerin Jacob:
> What will happen if odp_cos_set_pool(odp_cos_t cos_id,
> odp_buffer_pool_t
> pool_id) was called twice?
> Will the new pool_id replace the old one or the CoS have two pools?
>
> @Taras:
> Assume: set several pools per pktio interface.
> What will happen if two data plane applications share one physical
> Ethernet port to receive packets?
> Since the pool is per pktio interface, will these two applications
> share
> the same buffer pool?
> If there is memory leak in one application, the other application will
> be
> disturbed.
> Correct me if my understanding were wrong.



 That's a nice question. I'm afraid this use-case was not considered
 before.
 Do you want to split traffic between two applications via classifier?

>
> Maybe to let each CoS have more than one pool and limit the max number
> of
> Pool to for example 4 (Let the application designer decide how many
> pools
> are needed for each CoS) could be one option.
>>>
>>>
>>> Hi,
>>>
>>> If we are attaching multiple pools per CoS what will be the
>>> distribution algorithm for packets to each of the associated pools?
>>> will it be a simple round-robin in that case wouldn't it be better to
>>> attach a single pool of bigger size to the specific CoS?
>>
>>
>> Packets will be distributed deterministically according to size.
>>
>> In fact distributing packets to different pools by their sizes is
>> orthogonal to the rest of classification which is done by analyzing
>> packet (header) content. IMO it was a mistake to combine them. They
>> normally have different purposes. Classification by size is used to get
>> efficient memory usage and decrease packet segmentation. While content
>> classification is used to separate different 'types' of traffic.
>>
>> In KS2 platform these are two separate steps:
>> 1. Packet is classified by a header content and directed to a Flow
>>(ODP CoS analog).
>> 2. Flow has a destination queue and up to 4 pools assigned. Exact pool
>>is chosen based on a packet size.
>>
>> Actually, the initial use-case in this thread clearly demonstrates
>> orthogonality of packet content and size classification: number of CoS'es
>> explodes as *.
>>
>> ___
>> lng-odp mailing list
>> lng-odp@lists.linaro.org
>> https://lists.linaro.org/mailman/listinfo/lng-odp
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Bill Fischofer]

Isn't that just another term in the PMR?  If I have a flow with four
different sized packets that I want to put in different pools then I simply
have a PMR to identify the flow as input to other PMRs that then segregate
by pkt_len into four final CoSes which map the flow to the same queue but
different pools.

On Mon, Apr 6, 2015 at 5:28 AM, Taras Kondratiuk <
taras.kondrat...@linaro.org> wrote:

> On 04/03/2015 07:53 PM, Bala Manoharan wrote:
>
>> On 3 April 2015 at 22:00, Taras Kondratiuk 
>> wrote:
>>
>>> On 03/30/2015 03:34 PM, Zhujianhua wrote:
>>>

 @ Jerin Jacob:
 What will happen if odp_cos_set_pool(odp_cos_t cos_id, odp_buffer_pool_t
 pool_id) was called twice?
 Will the new pool_id replace the old one or the CoS have two pools?

 @Taras:
 Assume: set several pools per pktio interface.
 What will happen if two data plane applications share one physical
 Ethernet port to receive packets?
 Since the pool is per pktio interface, will these two applications share
 the same buffer pool?
 If there is memory leak in one application, the other application will
 be
 disturbed.
 Correct me if my understanding were wrong.

>>>
>>>
>>> That's a nice question. I'm afraid this use-case was not considered
>>> before.
>>> Do you want to split traffic between two applications via classifier?
>>>
>>>
 Maybe to let each CoS have more than one pool and limit the max number
 of
 Pool to for example 4 (Let the application designer decide how many
 pools
 are needed for each CoS) could be one option.

>>>
>> Hi,
>>
>> If we are attaching multiple pools per CoS what will be the
>> distribution algorithm for packets to each of the associated pools?
>> will it be a simple round-robin in that case wouldn't it be better to
>> attach a single pool of bigger size to the specific CoS?
>>
>
> Packets will be distributed deterministically according to size.
>
> In fact distributing packets to different pools by their sizes is
> orthogonal to the rest of classification which is done by analyzing
> packet (header) content. IMO it was a mistake to combine them. They
> normally have different purposes. Classification by size is used to get
> efficient memory usage and decrease packet segmentation. While content
> classification is used to separate different 'types' of traffic.
>
> In KS2 platform these are two separate steps:
> 1. Packet is classified by a header content and directed to a Flow
>(ODP CoS analog).
> 2. Flow has a destination queue and up to 4 pools assigned. Exact pool
>is chosen based on a packet size.
>
> Actually, the initial use-case in this thread clearly demonstrates
> orthogonality of packet content and size classification: number of CoS'es
> explodes as *.
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Taras Kondratiuk]

On 04/03/2015 07:53 PM, Bala Manoharan wrote:

On 3 April 2015 at 22:00, Taras Kondratiuk  wrote:

On 03/30/2015 03:34 PM, Zhujianhua wrote:


@ Jerin Jacob:
What will happen if odp_cos_set_pool(odp_cos_t cos_id, odp_buffer_pool_t
pool_id) was called twice?
Will the new pool_id replace the old one or the CoS have two pools?

@Taras:
Assume: set several pools per pktio interface.
What will happen if two data plane applications share one physical
Ethernet port to receive packets?
Since the pool is per pktio interface, will these two applications share
the same buffer pool?
If there is memory leak in one application, the other application will be
disturbed.
Correct me if my understanding were wrong.



That's a nice question. I'm afraid this use-case was not considered before.
Do you want to split traffic between two applications via classifier?



Maybe to let each CoS have more than one pool and limit the max number of
Pool to for example 4 (Let the application designer decide how many pools
are needed for each CoS) could be one option.


Hi,

If we are attaching multiple pools per CoS what will be the
distribution algorithm for packets to each of the associated pools?
will it be a simple round-robin in that case wouldn't it be better to
attach a single pool of bigger size to the specific CoS?


Packets will be distributed deterministically according to size.

In fact distributing packets to different pools by their sizes is
orthogonal to the rest of classification which is done by analyzing
packet (header) content. IMO it was a mistake to combine them. They
normally have different purposes. Classification by size is used to get
efficient memory usage and decrease packet segmentation. While content
classification is used to separate different 'types' of traffic.

In KS2 platform these are two separate steps:
1. Packet is classified by a header content and directed to a Flow
   (ODP CoS analog).
2. Flow has a destination queue and up to 4 pools assigned. Exact pool
   is chosen based on a packet size.

Actually, the initial use-case in this thread clearly demonstrates
orthogonality of packet content and size classification: number of 
CoS'es explodes as *.

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Bill Fischofer]

As currently defined, there is a many-to-one mapping between CoS and pools,
but a single CoS can only be associated with a single pool.  PMRs are used
to determine which CoS an incoming packet will be assigned to, and hence to
the pool it will reside in.  It is an application responsibility to ensure
that PMRs are unambiguously defined.  If this restriction is not observed,
it is undefined which PMR will apply, so behavior may vary cross different
ODP implementations.

For the shared interface question, this is a question of virtualization of
the interface.  From the perspective of multiple ODP applications each
interface (virtual or real) is dedicated to it, so the different pool
assignments for the various applications are all orthogonal.  Whether a
given interface can support such virtualization/multiplexing/sharing is up
to the given platform.  ODP itself does not specify whether or how such
restrictions may apply in any given instance.

On Fri, Apr 3, 2015 at 11:53 AM, Bala Manoharan 
wrote:

> On 3 April 2015 at 22:00, Taras Kondratiuk 
> wrote:
> > On 03/30/2015 03:34 PM, Zhujianhua wrote:
> >>
> >> @ Jerin Jacob:
> >> What will happen if odp_cos_set_pool(odp_cos_t cos_id, odp_buffer_pool_t
> >> pool_id) was called twice?
> >> Will the new pool_id replace the old one or the CoS have two pools?
> >>
> >> @Taras:
> >> Assume: set several pools per pktio interface.
> >> What will happen if two data plane applications share one physical
> >> Ethernet port to receive packets?
> >> Since the pool is per pktio interface, will these two applications share
> >> the same buffer pool?
> >> If there is memory leak in one application, the other application will
> be
> >> disturbed.
> >> Correct me if my understanding were wrong.
> >
> >
> > That's a nice question. I'm afraid this use-case was not considered
> before.
> > Do you want to split traffic between two applications via classifier?
> >
> >>
> >> Maybe to let each CoS have more than one pool and limit the max number
> of
> >> Pool to for example 4 (Let the application designer decide how many
> pools
> >> are needed for each CoS) could be one option.
>
> Hi,
>
> If we are attaching multiple pools per CoS what will be the
> distribution algorithm for packets to each of the associated pools?
> will it be a simple round-robin in that case wouldn't it be better to
> attach a single pool of bigger size to the specific CoS?
>
> Since we are attaching pools per CoS object the application can
> configure the PMR rule in such a way that packets which come from the
> same interface and belong to different service can be configured to be
> allocated from multiple pools by attaching to multiple CoS objects.
> Pls let me know if my understanding is wrong.
>
> Regards,
> Bala
> >
> >
> > That is a possible solution.
> > ___
> > lng-odp mailing list
> > lng-odp@lists.linaro.org
> > https://lists.linaro.org/mailman/listinfo/lng-odp
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Bala Manoharan]

On 3 April 2015 at 22:00, Taras Kondratiuk  wrote:
> On 03/30/2015 03:34 PM, Zhujianhua wrote:
>>
>> @ Jerin Jacob:
>> What will happen if odp_cos_set_pool(odp_cos_t cos_id, odp_buffer_pool_t
>> pool_id) was called twice?
>> Will the new pool_id replace the old one or the CoS have two pools?
>>
>> @Taras:
>> Assume: set several pools per pktio interface.
>> What will happen if two data plane applications share one physical
>> Ethernet port to receive packets?
>> Since the pool is per pktio interface, will these two applications share
>> the same buffer pool?
>> If there is memory leak in one application, the other application will be
>> disturbed.
>> Correct me if my understanding were wrong.
>
>
> That's a nice question. I'm afraid this use-case was not considered before.
> Do you want to split traffic between two applications via classifier?
>
>>
>> Maybe to let each CoS have more than one pool and limit the max number of
>> Pool to for example 4 (Let the application designer decide how many pools
>> are needed for each CoS) could be one option.

Hi,

If we are attaching multiple pools per CoS what will be the
distribution algorithm for packets to each of the associated pools?
will it be a simple round-robin in that case wouldn't it be better to
attach a single pool of bigger size to the specific CoS?

Since we are attaching pools per CoS object the application can
configure the PMR rule in such a way that packets which come from the
same interface and belong to different service can be configured to be
allocated from multiple pools by attaching to multiple CoS objects.
Pls let me know if my understanding is wrong.

Regards,
Bala
>
>
> That is a possible solution.
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Taras Kondratiuk]

On 03/30/2015 03:34 PM, Zhujianhua wrote:

@ Jerin Jacob:
What will happen if odp_cos_set_pool(odp_cos_t cos_id, odp_buffer_pool_t 
pool_id) was called twice?
Will the new pool_id replace the old one or the CoS have two pools?

@Taras:
Assume: set several pools per pktio interface.
What will happen if two data plane applications share one physical Ethernet 
port to receive packets?
Since the pool is per pktio interface, will these two applications share the 
same buffer pool?
If there is memory leak in one application, the other application will be 
disturbed.
Correct me if my understanding were wrong.


That's a nice question. I'm afraid this use-case was not considered before.
Do you want to split traffic between two applications via classifier?



Maybe to let each CoS have more than one pool and limit the max number of Pool 
to for example 4 (Let the application designer decide how many pools are needed 
for each CoS) could be one option.


That is a possible solution.
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Jacob, Jerin]

Initial ODP classification specification had odp_cos_set_pool API and it got 
removed in v1.0.

may its time to bring the API back to specification.


int odp_cos_set_pool(odp_cos_t cos_id, odp_buffer_pool_t pool_id);


/Jerin?


From: lng-odp-boun...@lists.linaro.org  on 
behalf of Zhujianhua 
Sent: Saturday, March 28, 2015 2:16 PM
To: Mike Holmes; Bill Fischofer
Cc: lng-odp@lists.linaro.org
Subject: Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

Hi,

First thanks for Mike, Bill's information about the classification APIs:).

But I encountered another problem about the combination of classification, 
queue and buffer pool APIs. Appreciate your help in advance.

Consider following classical use case:

Available resource:

Queue A : expected to store packets with dst IP = a.a.a.a,
Queue B : expected to store packets with dst IP = b.b.b.b
Buffer Pool 1 : Buffer Size = 1k, expected to store pakcets with length little 
than 1k,
Buffer Pool 2 : Buffer Size = 2k, expected to store pakcets with length little 
than 2k but greater than 1k
One pktio.

Expected results:

When pakcet with dst iP a.a.a.a and length <1k is received , buffer for storing 
this packet is allocated from Pool 1 and this packet will be stored in Queue A.

When pakcets with dst ip a.a.a.a with 1k mailto:mike.hol...@linaro.org]
Sent: 2015?3?26? 19:34
To: Zhujianhua
Cc: lng-odp@lists.linaro.org
Subject: Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

Hi Jianhua

Egress API's are in progress currently, we expect to roll out initial proposals 
into the api-next branch over the next few weeks, for classification the 
existing API covers these features 
http://docs.opendataplane.org/linux-generic-doxygen-html/group__odp__classification.html

If you have use cases that the API could be extended to cover patches or RFCs 
are welcome on the list.
Also note that we have a public call once a week 
http://www.opendataplane.org/meetings/ if you would like to discuss your ideas 
there.

As an employee of a member company you can also become more deeply attached to 
the daily discussion if that is valuable to you.

Mike


On 26 March 2015 at 07:13, Zhujianhua 
mailto:zhujian...@huawei.com>> wrote:
Hi,

ODP APIs are attractive and it seems very useful for data communication 
application.

But unfortunately I cannot find any API 
(http://docs.opendataplane.org/html/files.html) for:

Packet classification and Packet Shaping.

Are these APIs not needed from ODP's point of view or will be defined later?

Thanks for any response.

Best Regards
Jianhua Zhu


___
lng-odp mailing list
lng-odp@lists.linaro.org<mailto:lng-odp@lists.linaro.org>
http://lists.linaro.org/mailman/listinfo/lng-odp



--
Mike Holmes
Technical Manager - Linaro Networking Group
Linaro.org<http://www.linaro.org/> ? Open source software for ARM SoCs

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Zhujianhua]

Hi,

First thanks for Mike, Bill’s information about the classification APIs☺.

But I encountered another problem about the combination of classification, 
queue and buffer pool APIs. Appreciate your help in advance.

Consider following classical use case:

Available resource:

Queue A : expected to store packets with dst IP = a.a.a.a,
Queue B : expected to store packets with dst IP = b.b.b.b
Buffer Pool 1 : Buffer Size = 1k, expected to store pakcets with length little 
than 1k,
Buffer Pool 2 : Buffer Size = 2k, expected to store pakcets with length little 
than 2k but greater than 1k
One pktio.

Expected results:

When pakcet with dst iP a.a.a.a and length <1k is received , buffer for storing 
this packet is allocated from Pool 1 and this packet will be stored in Queue A.

When pakcets with dst ip a.a.a.a with 1k mailto:mike.hol...@linaro.org]
Sent: 2015年3月26日 19:34
To: Zhujianhua
Cc: lng-odp@lists.linaro.org
Subject: Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

Hi Jianhua

Egress API's are in progress currently, we expect to roll out initial proposals 
into the api-next branch over the next few weeks, for classification the 
existing API covers these features 
http://docs.opendataplane.org/linux-generic-doxygen-html/group__odp__classification.html

If you have use cases that the API could be extended to cover patches or RFCs 
are welcome on the list.
Also note that we have a public call once a week 
http://www.opendataplane.org/meetings/ if you would like to discuss your ideas 
there.

As an employee of a member company you can also become more deeply attached to 
the daily discussion if that is valuable to you.

Mike


On 26 March 2015 at 07:13, Zhujianhua 
mailto:zhujian...@huawei.com>> wrote:
Hi,

ODP APIs are attractive and it seems very useful for data communication 
application.

But unfortunately I cannot find any API 
(http://docs.opendataplane.org/html/files.html) for:

Packet classification and Packet Shaping.

Are these APIs not needed from ODP’s point of view or will be defined later?

Thanks for any response.

Best Regards
Jianhua Zhu


___
lng-odp mailing list
lng-odp@lists.linaro.org<mailto:lng-odp@lists.linaro.org>
http://lists.linaro.org/mailman/listinfo/lng-odp



--
Mike Holmes
Technical Manager - Linaro Networking Group
Linaro.org<http://www.linaro.org/> │ Open source software for ARM SoCs

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Mike Holmes]

Hi Jianhua

Egress API's are in progress currently, we expect to roll out initial
proposals into the api-next branch over the next few weeks, for
classification the existing API covers these features http://docs.
opendataplane.org/linux-generic-doxygen-html/group__odp
__classification.html

If you have use cases that the API could be extended to cover patches or
RFCs are welcome on the list.
Also note that we have a public call once a week
http://www.opendataplane.org/meetings/
if you would like to discuss your ideas there.

As an employee of a member company you can also become more deeply attached
to the daily discussion if that is valuable to you.

Mike


On 26 March 2015 at 07:13, Zhujianhua  wrote:

>  Hi,
>
>
>
> ODP APIs are attractive and it seems very useful for data communication
> application.
>
>
>
> But unfortunately I cannot find any API (
> http://docs.opendataplane.org/html/files.html) for:
>
>
>
> Packet classification and Packet Shaping.
>
>
>
> Are these APIs not needed from ODP’s point of view or will be defined
> later?
>
>
>
> Thanks for any response.
>
>
>
> Best Regards
>
> Jianhua Zhu
>
>
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp
>
>


-- 
Mike Holmes
Technical Manager - Linaro Networking Group
Linaro.org  *│ *Open source software for ARM SoCs
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] NO ODP API for Packet classification and Packet Shaping

[Bill Fischofer]

ODP v1.0 has an initial set of APIs to support packet classification on
RX.  See
http://docs.opendataplane.org/linux-generic-doxygen-html/group__odp__classification.html
for details.

We are currently working on corresponding TX capabilities for packet
shaping.  These are currently being developed internal to LNG and will be
surfaced to the open mailing list once they firm up a bit.  Since Huawei is
an LNG member company you can get more directly involved in this definition
process if you wish.  Please contact me directly if you are interested.

Thanks.

On Thu, Mar 26, 2015 at 6:13 AM, Zhujianhua  wrote:

>  Hi,
>
>
>
> ODP APIs are attractive and it seems very useful for data communication
> application.
>
>
>
> But unfortunately I cannot find any API (
> http://docs.opendataplane.org/html/files.html) for:
>
>
>
> Packet classification and Packet Shaping.
>
>
>
> Are these APIs not needed from ODP’s point of view or will be defined
> later?
>
>
>
> Thanks for any response.
>
>
>
> Best Regards
>
> Jianhua Zhu
>
>
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] NO ODP API for Packet classification and Packet Shaping

[Zhujianhua]

Hi,

ODP APIs are attractive and it seems very useful for data communication 
application.

But unfortunately I cannot find any API 
(http://docs.opendataplane.org/html/files.html) for:

Packet classification and Packet Shaping.

Are these APIs not needed from ODP's point of view or will be defined later?

Thanks for any response.

Best Regards
Jianhua Zhu

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp