Re: [lng-odp] D01 hw accelerator investigate

[Weilong Chen]

Sorry, there's no doc in English.

SPPE is a D01's Network Processor subsystem. It has 2 Atoms(750MHz), every
Atom has 8 NPU, and every NPU has 8 thread.
It can deal with IPSEC packets by using crypto accelerator, can copy
packets by its internal DMA engin, can fragment
the packets if more than MTU. It even can routing the packets by rules user
sets. It also has some special feature,
such as sending packets by the same ID.

​​


On 20 October 2014 22:36, Maxim Uvarov  wrote:

> On 10/20/2014 05:29 PM, Weilong Chen wrote:
>
>> Hi all,
>>
>> I'm investigating the D01's HW accelerators, to find out if ODP's API can
>> cover all features.
>> There three cases:
>>   1. D01 has the accelerator and ODP has the API, such as the crypto.
>>   2. ODP has but the d01 doesn't support yet.
>>   3. The HW accelerator is a complex, it is compounded by several simple
>> function and special purpose for business.
>>  For example, the SPPE accelerator, it serve end to end packets
>> process, including parsing, forwarding, multicast
>>  counting etc.
>>
>> I think when a new HW is introduced to ODP, he will meet the same problem:
>> The ODP's API does not cover all its HW accelerators.
>>
>> We may allow users to put their own APIs into ODP, like the kernel's
>> arch/..., but that will
>> make problems, the ODP APP can't use the same API, hard to manage the
>> code.
>>
>> Any other idea?
>>
>> Thanks,
>> Weilong
>>
>
> Is there any open documentation / specification for d01? I think it's hard
> to answer which exactly HW accelerators ODP does not cover. I think that
> common procedure that you need came with idea/requirements to include some
> block to ODP. Then if that blocks exist in other platforms and it's
> reasonable to include we do arch doc and implementation for linux-generic.
> Then you implement it for your platform. What is SPPE?
>
>
> Maxim.
>
>
>
>>
>> ___
>> lng-odp mailing list
>> lng-odp@lists.linaro.org
>> http://lists.linaro.org/mailman/listinfo/lng-odp
>>
>
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [ODP PATCH] Doxygen error correction

[Balasubramanian Manoharan]

Signed-off-by: Balasubramanian Manoharan 
---
This patch corrects the errors in classification header file
pointed out by doxygen tool

 platform/linux-generic/include/api/odp_classification.h | 14 +-
 platform/linux-generic/odp_classification.c |  3 ++-
 2 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/platform/linux-generic/include/api/odp_classification.h 
b/platform/linux-generic/include/api/odp_classification.h
index 99b94e9..0c24cd5 100644
--- a/platform/linux-generic/include/api/odp_classification.h
+++ b/platform/linux-generic/include/api/odp_classification.h
@@ -195,7 +195,7 @@ int odp_pktio_set_skip(odp_pktio_t pktio_in, size_t offset);
  *
  * @note Optional.
  */
-int odp_pktio_set_headroom(odp_pktio_t port_id, size_t headroom);
+int odp_pktio_set_headroom(odp_pktio_t pktio_in, size_t headroom);
 
 /**
  * Specify per-cos buffer headroom
@@ -210,7 +210,7 @@ int odp_pktio_set_headroom(odp_pktio_t port_id, size_t 
headroom);
  *
  * @note Optional.
  */
-int odp_cos_set_headroom(odp_cos_t cos_id, size_t req_room);
+int odp_cos_set_headroom(odp_cos_t cos_id, size_t headroom);
 
 /**
  * Request to override per-port class of service
@@ -315,10 +315,14 @@ odp_cos_port_flow_signature(odp_pktio_t pktio_in,
  * PMR - Packet Matching Rule
  * Up to 32 bit of ternary matching of one of the available header fields
  */
-#defineODP_PMR_INVAL ((odp_pmr_t)NULL)
 typedef uint32_t odp_pmr_t;
 
 /**
+ * Macro for Invalid PMR.
+ */
+#defineODP_PMR_INVAL ((odp_pmr_t)NULL)
+
+/**
  * Packet Matching Rule field enumeration
  * for fields that may be used to calculate
  * the PMR, if present in a packet.
@@ -462,7 +466,7 @@ typedef enum odp_pmr_match_type {
  * appropriate fallback action.
  */
 typedef struct odp_pmr_match_t {
-   odp_pmr_match_type_e match_type;
+   odp_pmr_match_type_e match_type; /**< Packet Match Type*/
union {
struct {
odp_pmr_term_e  term;
@@ -501,7 +505,7 @@ typedef uint32_t odp_pmr_set_t;
  * or -1 for error.
  */
 int odp_pmr_match_set_create(int num_terms, odp_pmr_match_t *terms,
-odp_pmr_set_t *pmr_set_id);
+odp_cos_t dst_cos, odp_pmr_set_t *pmr_set_id);
 
 /**
  * Function to delete a composite packet match rule set
diff --git a/platform/linux-generic/odp_classification.c 
b/platform/linux-generic/odp_classification.c
index 901087c..dbc74e2 100644
--- a/platform/linux-generic/odp_classification.c
+++ b/platform/linux-generic/odp_classification.c
@@ -209,11 +209,12 @@ unsigned odp_pmr_terms_avail(void)
 }
 
 int odp_pmr_match_set_create(int num_terms, odp_pmr_match_t *terms,
-odp_pmr_set_t *pmr_set_id)
+odp_cos_t dst_cos, odp_pmr_set_t *pmr_set_id)
 {
(void)num_terms;
(void)terms;
(void)pmr_set_id;
+   (void)dst_cos;
ODP_UNIMPLEMENTED();
return 0;
 }
-- 
2.0.1.472.g6f92e5f


___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [ODP ARCH] Classification API Minor Format changes

[Balasubramanian Manoharan]

Signed-off-by: Balasubramanian Manoharan 
---
Removes struct definition from platform specific opaque handles
Corrects variable definition in odp_pktio_pmr_match_set_cos and 
odp_pmr_match_set_create function
 classification_design.dox | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/classification_design.dox b/classification_design.dox
index 58d55a1..827fe58 100644
--- a/classification_design.dox
+++ b/classification_design.dox
@@ -143,7 +143,7 @@ The names are part of the ODP API and MUST be present in 
any conforming implemen
 /**
  * Class of service instance type
  */
-typedef uint32_t odp_cos_t;
+typedef  odp_cos_t;
 
 
 /**
@@ -518,7 +518,7 @@ The following data structures SHOULD be implemented to 
support the definition of
 
 
 #defineODP_PMR_INVAL ((odp_pmr_t)NULL)
-typedef struct odp_pmr_s *odp_pmr_t;
+typedef  odp_pmr_t;
 @endverbatim
 
 @subsection terms Terms
@@ -765,7 +765,7 @@ typedef struct odp_pmr_match_t {
 
 
 /** An opaque handle to a composite packet match rule-set */
-typedef struct odp_pmr_set_s *odp_pmr_set_t;
+typedef  odp_pmr_set_t;
 @endverbatim
 
 The above structure is used with the following APIs to implement table-based 
PMRs:
@@ -790,7 +790,7 @@ The above structure is used with the following APIs to 
implement table-based PMR
  */
 
 int odp_pmr_match_set_create(int num_terms, odp_pmr_match_t *terms,
- odp_pmr_set_t *pmr_set_id);
+ odp_cos_t dst_cos, odp_pmr_set_t *pmr_set_id);
 @endverbatim
 
 This routine is used to create a PMR match set.
@@ -830,7 +830,7 @@ This routine destroys a PMR match set previously created by 
odp_pmr_match_set_cr
  * @return Success or ODP_PARAMETER_ERROR
  */
 
-int odp_pktio_pmr_match_set_cos(odp_pmr_t pmr_id, odp_pktio_t src_pktio,
+int odp_pktio_pmr_match_set_cos(odp_pmr_set_t pmr_set_id, odp_pktio_t 
src_pktio,
 odp_cos_t dst_cos);
 @endverbatim
 
-- 
2.0.1.472.g6f92e5f


___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] Agenda for Tuesday ODP call

[Bill Fischofer]

The agenda for this week's ODP call is as follows:

Agenda:

   -

   Revised Packet API Design Doc
   

   and Discussion
   
   -

  Do we need SCTP?
  -

  inflag/outflag use
  -

  odp_packet_parse() need
  -

  headroom/tailroom semantics
  -

   Ordered Queues discussion
   
   -

   Revised Atomics discussion
   
   - Follow-up discussions on PktIO Design Doc
   



Please join us at 15:00 UTC at uberconference.com/opendataplane

Thank you.

Bill
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [ARCH DESIGN] Queues and Synchronization/Scheduling models

[Bill Fischofer]

See inline comments.

On Mon, Oct 20, 2014 at 1:09 PM, Ola Liljedahl 
wrote:

> On 20 October 2014 19:09, Bill Fischofer 
> wrote:
>
>> Ok, thanks for the clarification.
>>
>> I agree it makes sense for atomic queues to be implicitly released by
>> subsequent queueing operations on buffers taken from them.  But that sort
>> of reinforces the notion that one of the pieces of buffer meta data needed
>> is the last_queue that the buffer was on so that this release can take
>> place.
>>
> However it is implemented, I think this should be kept hidden from the
> application. Buffers may remember which queue they were scheduled from or
> there is some state in the scheduler which remembers which CPU (thread) is
> associated with with queue etc. But to keep all options open, all calls
> that release atomic or ordered queue should take the buffer as a parameter.
>
>
That makes sense.  There are system meta data items that we explicitly
expose to applications via APIs and others that it is assume an
implementation will make use of.  This would fall into the latter
category.  The only reason for exposing a last_queue meta data item to the
application would be to enable the odp_schedule() call to not take an
output queue parameter since that information would be retrievable from the
returned buffer if the application needed to know that.


>
>> Regarding odp_buffer_enq_multi(), the question is whether the queue types
>> (parallel/atomic/ordered) imply the use of a scheduler or if they are
>> intrinsic to the queue itself.  While we could say that these are only
>> applicable to scheduled queues, the fact that many of these operations are
>> triggered by queue APIs suggest that this is not necessary.  For example, a
>> polled queue could easily be parallel, or atomic, or ordered at essentially
>> no additional cost.  We might want to rename odp_schedule_release_atomic()
>> to something like odp_buffer_release_atomic() to make that clear,
>> especially since odp_schedule_release_atomic() is currently defined to take
>> a void argument when it would seem to need an odp_buffer_t argument for
>> completeness.
>>
> If the semantics of a "polled" queue is only that that a thread specifies
> the queue directly when dequeuing a packet, I think parallel and ordered
> scheduling makes sense. Atomic scheduling does not make much sense because
> if that queue already has a packet outstanding, we cannot dequeue another
> packet from the same queue (this would the atomic guarantee). Of course the
> dequeue call can then return ODP_INVALID_BUFFER and force the application
> to do something else (e.g. dequeue from some other queue) but now we have
> just pushed scheduling into the application.
>

That's exactly how I'd expect a polled atomic queue to behave, otherwise
you'd have potential deadlock.


>
>
>> Having these be applicable only to scheduled queues for ODP v1.0 is
>> probably simpler at this point, but something we may need to revisit post
>> v1.0, especially as we move away from the notion of a monolithic scheduler
>> to something more parameterized.  If we have multiple schedulers acting on
>> queues then it becomes clear that these operations are inherent to the
>> queues themselves and not just the scheduler.
>>
>> Regarding atomic queues being ordered then by that definition we
>> currently do not implement atomic queues.  If atomic queues are ordered
>> then if Threads T1 and T2 each dequeue a buffer from an atomic queue A
>>
> This is exactly what cannot happen with atomic queues. Only one
> outstanding buffer at a time. Atomic is a poor work me think, "exclusive"
> might have been more descriptive.
>
> I was merely saying that the atomic guarantee implicitly ensures ordering
> as well. Except perhaps when the application is explicitly releasing the
> scheduler before enqueuing the packet on a queue which would allow another
> (really quick) thread to schedule another packet from the same atomic queue
> and enqueue it first. What happens then?
>

But the definition of ordered queues explicitly covers precisely this
case.  Ordering is only interesting if you can have more than one buffer
from an ordered queue "in flight" at the same time because it's only then
that system-mediated order restoration guarantees come into play.
Otherwise you just have serial processing of an atomic queue and that
requires no extra effort on the part of the system to ensure order
preservation.  The can of worms is opened by odp_schedule_release_atomic()
here.  That causes no problems *unless *we say that atomic implies
ordered.  To me the two seem independent and while we may wish to have
ATOMIC_ORDERED queues as an additional category, atomic queues that do not
have an ordering guarantee would also seem to be useful.


>
>
> in the order A1 and A2 and then T1 issues an odp_schedule_release_atomic()
>> call that unlocks A (allowing T2 to obtain A2 before T1 has disposed of A1)
>> then a downstream queue would need to ensure that A1 appears

Re: [lng-odp] [ARCH DESIGN] Queues and Synchronization/Scheduling models

[Ola Liljedahl]

On 20 October 2014 19:09, Bill Fischofer  wrote:

> Ok, thanks for the clarification.
>
> I agree it makes sense for atomic queues to be implicitly released by
> subsequent queueing operations on buffers taken from them.  But that sort
> of reinforces the notion that one of the pieces of buffer meta data needed
> is the last_queue that the buffer was on so that this release can take
> place.
>
However it is implemented, I think this should be kept hidden from the
application. Buffers may remember which queue they were scheduled from or
there is some state in the scheduler which remembers which CPU (thread) is
associated with with queue etc. But to keep all options open, all calls
that release atomic or ordered queue should take the buffer as a parameter.


> Regarding odp_buffer_enq_multi(), the question is whether the queue types
> (parallel/atomic/ordered) imply the use of a scheduler or if they are
> intrinsic to the queue itself.  While we could say that these are only
> applicable to scheduled queues, the fact that many of these operations are
> triggered by queue APIs suggest that this is not necessary.  For example, a
> polled queue could easily be parallel, or atomic, or ordered at essentially
> no additional cost.  We might want to rename odp_schedule_release_atomic()
> to something like odp_buffer_release_atomic() to make that clear,
> especially since odp_schedule_release_atomic() is currently defined to take
> a void argument when it would seem to need an odp_buffer_t argument for
> completeness.
>
If the semantics of a "polled" queue is only that that a thread specifies
the queue directly when dequeuing a packet, I think parallel and ordered
scheduling makes sense. Atomic scheduling does not make much sense because
if that queue already has a packet outstanding, we cannot dequeue another
packet from the same queue (this would the atomic guarantee). Of course the
dequeue call can then return ODP_INVALID_BUFFER and force the application
to do something else (e.g. dequeue from some other queue) but now we have
just pushed scheduling into the application.


> Having these be applicable only to scheduled queues for ODP v1.0 is
> probably simpler at this point, but something we may need to revisit post
> v1.0, especially as we move away from the notion of a monolithic scheduler
> to something more parameterized.  If we have multiple schedulers acting on
> queues then it becomes clear that these operations are inherent to the
> queues themselves and not just the scheduler.
>
> Regarding atomic queues being ordered then by that definition we currently
> do not implement atomic queues.  If atomic queues are ordered then if
> Threads T1 and T2 each dequeue a buffer from an atomic queue A
>
This is exactly what cannot happen with atomic queues. Only one outstanding
buffer at a time. Atomic is a poor work me think, "exclusive" might have
been more descriptive.

I was merely saying that the atomic guarantee implicitly ensures ordering
as well. Except perhaps when the application is explicitly releasing the
scheduler before enqueuing the packet on a queue which would allow another
(really quick) thread to schedule another packet from the same atomic queue
and enqueue it first. What happens then?


in the order A1 and A2 and then T1 issues an odp_schedule_release_atomic()
> call that unlocks A (allowing T2 to obtain A2 before T1 has disposed of A1)
> then a downstream queue would need to ensure that A1 appears first even if
> T2 enqueues A2 before T1 enqueues A1.  We do not currently ensure this.  If
> Atomic does not imply Ordered then all Atomic is really doing is protecting
> the queue context from parallel access. This in itself seems useful and
> need not be combined with ordering semantics.
>
Agree that this is a separate function and the one we are primarily after
with atomic scheduling.


> In an earlier thread on this topic it was suggested that perhaps we'd want
> to have an ORDERED_ATOMIC (or ATOMIC_ORDERED) queue scheduling type in
> addition to ATOMIC for when such semantics are needed.  I could see that
> making sense.
>
> The question of order preservation through multiple queueing levels seems
> highly germane and necessary for predictable behavior.  If we take the
> above proposed definition then it is quite precise.  Buffer order
> propagates from one ordered queue to the next, ignoring non-ordered
> intermediates.  This relaxation should facilitate implementations where
> ordering must be handled in SW since it means that HW offloads that may not
> in themselves handle ordering as ODP defines it could be accommodated.  The
> only ambiguity is how are gaps handled?  We have that question independent
> of intermediates.  Perhaps an odp_queue_release_order(buf) might be a good
> counterpart to odp_queue_release_atomic(buf) ?  The former call would say
> that the sequence owned by the referenced buffer should be deemed filled so
> that it no longer blocks subsequent buffers originating from th

[lng-odp] [PATCH] odp.dox: fix bug url

[Mike Holmes]

Signed-off-by: Mike Holmes 
---
 doc/odp.dox | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/odp.dox b/doc/odp.dox
index 73d1f37..b586421 100644
--- a/doc/odp.dox
+++ b/doc/odp.dox
@@ -66,6 +66,6 @@
  * @section contact Contact Details
  * - The main web site is http://www.opendataplane.org/
  * - The git repo is https://git.linaro.org/lng/odp.git
- * - Bug tracking https://launchpad.net/linaro-odp
+ * - Bug tracking is 
https://bugs.linaro.org/buglist.cgi?component=General&list_id=2474&product=OpenDataPlane
  *
  */
-- 
1.9.1


___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [ODP/PATCH v3] Look ma, no barriers! C11 memory model

[Ola Liljedahl]

On 20 October 2014 17:35, Savolainen, Petri (NSN - FI/Espoo) <
petri.savolai...@nsn.com> wrote:

> Hi,
>
> This patch should be split into many. It's modifying atomic API,
> introducing a new API (counters) as well as fixing various implementation
> issues (e.g. barrier).
>
> I'd prefer to define counters after v1.0 due to the fact that there are so
> many other APIs to be "finalize" before v1.0. Also other than "relaxed"
> atomics are mainly needed by lock/synchronization algorithm
> implementations, which do not have the highest priority to abstract (low
> number of (especially application) C lines in lock implementations).
>
And I was under the belief that Nokia was disappointed with the amount of
code (LOC's) in ODP?

Relaxed atomics cannot be used for synchronization and lock
implementations, you need acquire and release operations for that.
Only the atomic counters use relaxed memory order just because the counters
are not used for synchronization and we want to optimize the counters for
low overhead and good scalability.



> You are removing many odp_sync_stores - are you sure all of those are
> redundant? Those are mainly needed to synchronize  stores on user data, not
> on the lock implementation itself.
>
The release operations will include necessary (one-sided) barriers, these
barriers cover all loads and stores issued by the current processor. Watch
the referenced Herb Sutter presentations, they are very good.



>
>
> > void odp_ticketlock_init(odp_ticketlock_t *ticketlock)
> >  {
> > - ticketlock->next_ticket = 0;
> > - ticketlock->cur_ticket  = 0;
> > - odp_sync_stores();
>
> This ensures that previous writes (e.g. memset) to the data don't overlap
> with writes to data when user is holding the lock the first time.
>
For other threads to be able to safely access this ticketlock, they must
either have been created after this lock was initialized or notified in
some way about the existence of this lock. And such thread creation,
notification or synchronization (e.g. odp_barrier_sync) would have included
the necessary barriers. If such synchronization has not occurred before
some other thread accesses the lock, the program has a race condition
(nothing that stopped that thread from accessing the lock *before* it was
initialized) and the behavior is undefined. odp_sync_stores() just makes
sure all processors have seen the preceding stores, it does not synchronize
with the application.

The thread that initialized the lock can also immediately acquire it, don't
need to wait for other threads to have seen the lock initialization (as
they can not access the lock until they have been notified). The executing
thread will see its own writes (e.g. lock initialization, data memset) in
program order, sync doesn't change anything here.



>
> > + odp_counter32_init(&ticketlock->next_ticket, 0);
> > + odp_atomic32_init(&ticketlock->cur_ticket, 0);
> >  }
>
>
> > @@ -22,30 +22,15 @@ void odp_ticketlock_lock(odp_ticketlock_t
> *ticketlock)
> >  {
> >   uint32_t ticket;
> >
> > - ticket = odp_atomic_fetch_inc_u32(&ticketlock->next_ticket);
> > + ticket = odp_counter32_read_inc(&ticketlock->next_ticket);
> >
> > - while (ticket != ticketlock->cur_ticket)
> > + while (ticket != odp_atomic32_load(&ticketlock->cur_ticket,
> > +ODP_MEMORDER_ACQ))
> >   odp_spin();
> > -
> > - odp_mem_barrier();
> >  }
>
> Why next_ticket is counter, but cur_ticket is atomic ?
>
next_ticket is just there for handing out tickets, it is not used for
ownership of the lock.
cur_ticket is the actual lock variable that you acquire and release and
ownership of the lock implies exclusive access to the associated user data
structure. Acquiring the lock variable is done with acquire memory order so
implies a one-sided acquire barrier (i.e. later accesses from this
processor cannot be ordered before the acquire load).


>
> >  void odp_ticketlock_unlock(odp_ticketlock_t *ticketlock)
> >  {
> > - odp_sync_stores();
>
> Sync stores to user data before releasing the lock.
>
The acquire/release design updates the cur_ticket lock variable with the
release memory order. This forces all earlier memory accesses (from this
processor) to complete before the lock is written. Accesses after the
release are still allowed to move before the releasing store (e.g.
speculative load in order to decrease load-to-use latency).
void odp_ticketlock_unlock(odp_ticketlock_t *ticketlock)
{
odp_atomic32_inc(&ticketlock->cur_ticket, ODP_MEMORDER_RLS);
}




>
> > -
> > - ticketlock->cur_ticket++;
> > -
> > -#if defined __OCTEON__
> > - odp_sync_stores();
>
> On Octeon, push out ticket update for better performance.
>
The odp_atomic32_inc() operation with the release memory order will include
HW barriers (before and after as seems to be required),
in case of OCTEON this is "syncw" which also has the side effect of
flushing the write buffer. In any case, J

Re: [lng-odp] [ARCH DESIGN] Queues and Synchronization/Scheduling models

[Bill Fischofer]

Ok, thanks for the clarification.

I agree it makes sense for atomic queues to be implicitly released by
subsequent queueing operations on buffers taken from them.  But that sort
of reinforces the notion that one of the pieces of buffer meta data needed
is the last_queue that the buffer was on so that this release can take
place.

Regarding odp_buffer_enq_multi(), the question is whether the queue types
(parallel/atomic/ordered) imply the use of a scheduler or if they are
intrinsic to the queue itself.  While we could say that these are only
applicable to scheduled queues, the fact that many of these operations are
triggered by queue APIs suggest that this is not necessary.  For example, a
polled queue could easily be parallel, or atomic, or ordered at essentially
no additional cost.  We might want to rename odp_schedule_release_atomic()
to something like odp_buffer_release_atomic() to make that clear,
especially since odp_schedule_release_atomic() is currently defined to take
a void argument when it would seem to need an odp_buffer_t argument for
completeness.

Having these be applicable only to scheduled queues for ODP v1.0 is
probably simpler at this point, but something we may need to revisit post
v1.0, especially as we move away from the notion of a monolithic scheduler
to something more parameterized.  If we have multiple schedulers acting on
queues then it becomes clear that these operations are inherent to the
queues themselves and not just the scheduler.

Regarding atomic queues being ordered then by that definition we currently
do not implement atomic queues.  If atomic queues are ordered then if
Threads T1 and T2 each dequeue a buffer from an atomic queue A in the order
A1 and A2 and then T1 issues an odp_schedule_release_atomic() call that
unlocks A (allowing T2 to obtain A2 before T1 has disposed of A1) then a
downstream queue would need to ensure that A1 appears first even if T2
enqueues A2 before T1 enqueues A1.  We do not currently ensure this.  If
Atomic does not imply Ordered then all Atomic is really doing is protecting
the queue context from parallel access. This in itself seems useful and
need not be combined with ordering semantics.

In an earlier thread on this topic it was suggested that perhaps we'd want
to have an ORDERED_ATOMIC (or ATOMIC_ORDERED) queue scheduling type in
addition to ATOMIC for when such semantics are needed.  I could see that
making sense.

The question of order preservation through multiple queueing levels seems
highly germane and necessary for predictable behavior.  If we take the
above proposed definition then it is quite precise.  Buffer order
propagates from one ordered queue to the next, ignoring non-ordered
intermediates.  This relaxation should facilitate implementations where
ordering must be handled in SW since it means that HW offloads that may not
in themselves handle ordering as ODP defines it could be accommodated.  The
only ambiguity is how are gaps handled?  We have that question independent
of intermediates.  Perhaps an odp_queue_release_order(buf) might be a good
counterpart to odp_queue_release_atomic(buf) ?  The former call would say
that the sequence owned by the referenced buffer should be deemed filled so
that it no longer blocks subsequent buffers originating from the same
ordered queue.

Regarding the point of queues being proxies for a flow, even in that model
a flow will be proxied by multiple queues unless you require that
processing occur in a single stage.  For applications structured in a more
pipelined fashion (which can occur even with single-stages that make use of
HW offload that involves reschedules) you have multiple queues representing
the same flow at different stages of processing (and possibly having unique
per-stage per-flow contexts associated with them).  So maintaining
end-to-end order on a per-flow basis is still required.

On the final point, we're not talking about ordering on the wire (that's
clearly unpredictable and in any case beyond the scope of ODP).  But within
a single ODP application order preservation from ingress to egress, while
allowing parallelism in between, would seem to be one of the main design
points for ODP.

Bill

On Mon, Oct 20, 2014 at 10:02 AM, Ola Liljedahl 
wrote:

> Bill, some spelling errors... I was listing the current calls that release
> the scheduling lock for a queue.
>
> * odp_queue_end() was supposed to be odp_queue_enq()  (q is a d upside
> down, maybe I am getting dyslectic?)
> * odp_queue_end_multi() was of course odp_queue_enq_multi().  Not sure how
> this call would work with atomic or ordered queues (all buffers must come
> from same queue), I guess it can degenerate into only returning one buffer
> at a time.
> * odp_buffer_free(odp_buffer_t buf)
> * odp_schedule_release(odp_buffer_t buf)  <--- new call
> (or odp_schedule_release_atomic() morphed)
>
> -- Ola
>
>
>
>
> On 20 October 2014 16:50, Bill Fischofer 
> wrote:
>
>> Thanks, Ola.  I need to think about this and respon

Re: [lng-odp] [ODP/PATCH v3] Look ma, no barriers! C11 memory model

[Savolainen, Petri (NSN - FI/Espoo)]

Hi,

This patch should be split into many. It's modifying atomic API, introducing a 
new API (counters) as well as fixing various implementation issues (e.g. 
barrier).

I'd prefer to define counters after v1.0 due to the fact that there are so many 
other APIs to be "finalize" before v1.0. Also other than "relaxed" atomics are 
mainly needed by lock/synchronization algorithm implementations, which do not 
have the highest priority to abstract (low number of (especially application) C 
lines in lock implementations).

You are removing many odp_sync_stores - are you sure all of those are 
redundant? Those are mainly needed to synchronize  stores on user data, not on 
the lock implementation itself.


> void odp_ticketlock_init(odp_ticketlock_t *ticketlock)
>  {
> - ticketlock->next_ticket = 0;
> - ticketlock->cur_ticket  = 0;
> - odp_sync_stores();

This ensures that previous writes (e.g. memset) to the data don't overlap with 
writes to data when user is holding the lock the first time.

> + odp_counter32_init(&ticketlock->next_ticket, 0);
> + odp_atomic32_init(&ticketlock->cur_ticket, 0);
>  }
 
 
> @@ -22,30 +22,15 @@ void odp_ticketlock_lock(odp_ticketlock_t *ticketlock)
>  {
>   uint32_t ticket;
>  
> - ticket = odp_atomic_fetch_inc_u32(&ticketlock->next_ticket);
> + ticket = odp_counter32_read_inc(&ticketlock->next_ticket);
>
> - while (ticket != ticketlock->cur_ticket)
> + while (ticket != odp_atomic32_load(&ticketlock->cur_ticket,
> +ODP_MEMORDER_ACQ))
>   odp_spin();
> -
> - odp_mem_barrier();
>  }

Why next_ticket is counter, but cur_ticket is atomic ?
 
 
>  void odp_ticketlock_unlock(odp_ticketlock_t *ticketlock)
>  {
> - odp_sync_stores();

Sync stores to user data before releasing the lock.

> -
> - ticketlock->cur_ticket++;
> -
> -#if defined __OCTEON__
> - odp_sync_stores();

On Octeon, push out ticket update for better performance.

> -#else
> - odp_mem_barrier();
> -#endif
> -}
> -
> -
> -int odp_ticketlock_is_locked(odp_ticketlock_t *ticketlock)
> -{
> - return ticketlock->cur_ticket != ticketlock->next_ticket;
> + odp_atomic32_inc(&ticketlock->cur_ticket, ODP_MEMORDER_RLS);
>  }


-Petri


 

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [ARCH DESIGN] Queues and Synchronization/Scheduling models

[Ola Liljedahl]

Bill, some spelling errors... I was listing the current calls that release
the scheduling lock for a queue.

* odp_queue_end() was supposed to be odp_queue_enq()  (q is a d upside
down, maybe I am getting dyslectic?)
* odp_queue_end_multi() was of course odp_queue_enq_multi().  Not sure how
this call would work with atomic or ordered queues (all buffers must come
from same queue), I guess it can degenerate into only returning one buffer
at a time.
* odp_buffer_free(odp_buffer_t buf)
* odp_schedule_release(odp_buffer_t buf)  <--- new call
(or odp_schedule_release_atomic() morphed)

-- Ola




On 20 October 2014 16:50, Bill Fischofer  wrote:

> Thanks, Ola.  I need to think about this and respond more carefully, but
> in the meantime could you propose the syntax/semantics of odp_queue_end(),
>  odp_queue_end_multi(), and odp_schedule_release() in a bit more detail?
>
> These seem to be new APIs and we need to be clear about their proposed
> semantics and intended use.
>
> Thanks.
>
> Bill
>
> On Mon, Oct 20, 2014 at 9:40 AM, Ola Liljedahl 
> wrote:
>
>> On 17 October 2014 10:01, Alexandru Badicioiu <
>> alexandru.badici...@linaro.org> wrote:
>>
>>> Hi Bill, check my thoughts inline.
>>> Thanks,
>>> Alex
>>>
>>> On 17 October 2014 03:31, Bill Fischofer 
>>> wrote:
>>>
 Based on discussions we had yesterday and today, I'd like to outline
 the open issues regarding queues and synchronization/scheduling models.
 We'd like to get consensus on this in time for next week's Tuesday call.

 ODP identifies three different synchronization/scheduling models for
 queues: Parallel, Atomic, and Ordered.  Here are my current understandings
 of what these mean:

- Parallel: Buffers on a parallel queue can be dequeued by the
scheduler for any caller without restriction.  This permits maximum
scale-out and concurrency for events that are truly independent.


- Atomic: Buffers on an atomic queue can be dequeued by the
scheduler for any caller. However, only one buffer from an atomic queue 
 may
be in process at any given time. When the scheduler dequeues a buffer 
 from
an atomic queue, the queue is locked and cannot dequeue further buffers
until it is released.  Releasing an atomic queue can occur in two ways:


- The dequeued buffer is enqueued to another queue via an
   odp_queue_enq() call. This action implicitly unlocks the atomic 
 queue the
   buffer was sourced from.  Note that this is the most common way in 
 which
   atomic queues are unlocked.


- A call is made to odp_schedule_release_atomic() for the locked
   queue.  This tells the scheduler that the queue's atomicity 
 guarantee is
   deemed satisfied by the application and the queue is free to dequeue 
 items
   to other scheduler callers. This method MUST be used if the caller 
 consumes
   the buffer (e.g., frees it instead of enqueues it to another queue) 
 and MAY
   be used as a performance optimization if the caller is done with any
   references to data that was serialized by the queue (e.g., the 
 queue's
   context).  It is an application programming error to release a queue
   prematurely as references subsequent to the release will not be
   synchronized.

 Third way - odp_buffer_free() called on a buffer which was dequeued
>>> from an atomic queue.
>>>
>> odp_queue_end(): implicit release of atomic queue
>> odp_queue_end_multi(): ? I assume multi-calls cannot be used with atomic
>> scheduling
>> odp_buffer_free(): (buffer is consumed) implicit release of atomic queue
>> odp_schedule_release_atomic(): explicit release of atomic queue
>>
>>
>>
>>>
- Ordered: Buffers on an ordered queue can be dequeued by the
scheduler for any caller, however buffers on an ordered queue retain
knowledge of their sequence on their source queue and this sequence 
 will be
restored whenever they are enqueued to a subsequent ordered queue.  That
is, if ordered queue A contains buffers A1, A2, and A3, and these are
dequeued for processing by three different threads, then when they are
subsequently enqueued to another ordered queue B by these threads, they
will appear on B as B1, B2, and B3 regardless of the order in which 
 their
processing threads issued odp_queue_enq() calls to place them on B.

 Why ordering has to be restored only if the destination is an ordered
>>> queue? I think it should be restored regardless of the destination queue
>>> type. Also it should be restored if there are multiple destination queues,
>>> if the implementation supports it.
>>>
>> Atomic queues are also ordered queues.
>>
>> If you enqueue a packet from an atomic or ordered queue onto a parallel
>> queue, ordering would normally be l

Re: [lng-odp] [ARCH DESIGN] Queues and Synchronization/Scheduling models

[Bill Fischofer]

Thanks, Ola.  I need to think about this and respond more carefully, but in
the meantime could you propose the syntax/semantics of odp_queue_end(),
 odp_queue_end_multi(), and odp_schedule_release() in a bit more detail?

These seem to be new APIs and we need to be clear about their proposed
semantics and intended use.

Thanks.

Bill

On Mon, Oct 20, 2014 at 9:40 AM, Ola Liljedahl 
wrote:

> On 17 October 2014 10:01, Alexandru Badicioiu <
> alexandru.badici...@linaro.org> wrote:
>
>> Hi Bill, check my thoughts inline.
>> Thanks,
>> Alex
>>
>> On 17 October 2014 03:31, Bill Fischofer 
>> wrote:
>>
>>> Based on discussions we had yesterday and today, I'd like to outline the
>>> open issues regarding queues and synchronization/scheduling models.  We'd
>>> like to get consensus on this in time for next week's Tuesday call.
>>>
>>> ODP identifies three different synchronization/scheduling models for
>>> queues: Parallel, Atomic, and Ordered.  Here are my current understandings
>>> of what these mean:
>>>
>>>- Parallel: Buffers on a parallel queue can be dequeued by the
>>>scheduler for any caller without restriction.  This permits maximum
>>>scale-out and concurrency for events that are truly independent.
>>>
>>>
>>>- Atomic: Buffers on an atomic queue can be dequeued by the
>>>scheduler for any caller. However, only one buffer from an atomic queue 
>>> may
>>>be in process at any given time. When the scheduler dequeues a buffer 
>>> from
>>>an atomic queue, the queue is locked and cannot dequeue further buffers
>>>until it is released.  Releasing an atomic queue can occur in two ways:
>>>
>>>
>>>- The dequeued buffer is enqueued to another queue via an
>>>   odp_queue_enq() call. This action implicitly unlocks the atomic queue 
>>> the
>>>   buffer was sourced from.  Note that this is the most common way in 
>>> which
>>>   atomic queues are unlocked.
>>>
>>>
>>>- A call is made to odp_schedule_release_atomic() for the locked
>>>   queue.  This tells the scheduler that the queue's atomicity guarantee 
>>> is
>>>   deemed satisfied by the application and the queue is free to dequeue 
>>> items
>>>   to other scheduler callers. This method MUST be used if the caller 
>>> consumes
>>>   the buffer (e.g., frees it instead of enqueues it to another queue) 
>>> and MAY
>>>   be used as a performance optimization if the caller is done with any
>>>   references to data that was serialized by the queue (e.g., the queue's
>>>   context).  It is an application programming error to release a queue
>>>   prematurely as references subsequent to the release will not be
>>>   synchronized.
>>>
>>> Third way - odp_buffer_free() called on a buffer which was dequeued from
>> an atomic queue.
>>
> odp_queue_end(): implicit release of atomic queue
> odp_queue_end_multi(): ? I assume multi-calls cannot be used with atomic
> scheduling
> odp_buffer_free(): (buffer is consumed) implicit release of atomic queue
> odp_schedule_release_atomic(): explicit release of atomic queue
>
>
>
>>
>>>- Ordered: Buffers on an ordered queue can be dequeued by the
>>>scheduler for any caller, however buffers on an ordered queue retain
>>>knowledge of their sequence on their source queue and this sequence will 
>>> be
>>>restored whenever they are enqueued to a subsequent ordered queue.  That
>>>is, if ordered queue A contains buffers A1, A2, and A3, and these are
>>>dequeued for processing by three different threads, then when they are
>>>subsequently enqueued to another ordered queue B by these threads, they
>>>will appear on B as B1, B2, and B3 regardless of the order in which their
>>>processing threads issued odp_queue_enq() calls to place them on B.
>>>
>>> Why ordering has to be restored only if the destination is an ordered
>> queue? I think it should be restored regardless of the destination queue
>> type. Also it should be restored if there are multiple destination queues,
>> if the implementation supports it.
>>
> Atomic queues are also ordered queues.
>
> If you enqueue a packet from an atomic or ordered queue onto a parallel
> queue, ordering would normally be lost. If there was ever only one thread
> scheduling packets from this parallel queue (how do you guarantee that?),
> then you could argue that ordering is still maintained. But this seems like
> a fragile situation.
>
> Nothing would stop an implementation from always restoring order for
> packets scheduled from ordered queues when enqueuing them.
>
>
> Implicit in these definitions is the fact that all queues associated with
>>> odp_pktio_t objects are ordered queues.
>>>
>> Why is this implicit? The order restoration happens when buffers are
>> enqueued to the destination queue(s). Aren't the queues associated with
>> odp_pktio_t the first queues seen by a packet? If an pktio queue is
>> parallel, there is no requirement at all to ensure any ordering at next

Re: [lng-odp] [ARCH DESIGN] Queues and Synchronization/Scheduling models

[Ola Liljedahl]

On 17 October 2014 10:01, Alexandru Badicioiu <
alexandru.badici...@linaro.org> wrote:

> Hi Bill, check my thoughts inline.
> Thanks,
> Alex
>
> On 17 October 2014 03:31, Bill Fischofer 
> wrote:
>
>> Based on discussions we had yesterday and today, I'd like to outline the
>> open issues regarding queues and synchronization/scheduling models.  We'd
>> like to get consensus on this in time for next week's Tuesday call.
>>
>> ODP identifies three different synchronization/scheduling models for
>> queues: Parallel, Atomic, and Ordered.  Here are my current understandings
>> of what these mean:
>>
>>- Parallel: Buffers on a parallel queue can be dequeued by the
>>scheduler for any caller without restriction.  This permits maximum
>>scale-out and concurrency for events that are truly independent.
>>
>>
>>- Atomic: Buffers on an atomic queue can be dequeued by the scheduler
>>for any caller. However, only one buffer from an atomic queue may be in
>>process at any given time. When the scheduler dequeues a buffer from an
>>atomic queue, the queue is locked and cannot dequeue further buffers until
>>it is released.  Releasing an atomic queue can occur in two ways:
>>
>>
>>- The dequeued buffer is enqueued to another queue via an
>>   odp_queue_enq() call. This action implicitly unlocks the atomic queue 
>> the
>>   buffer was sourced from.  Note that this is the most common way in 
>> which
>>   atomic queues are unlocked.
>>
>>
>>- A call is made to odp_schedule_release_atomic() for the locked
>>   queue.  This tells the scheduler that the queue's atomicity guarantee 
>> is
>>   deemed satisfied by the application and the queue is free to dequeue 
>> items
>>   to other scheduler callers. This method MUST be used if the caller 
>> consumes
>>   the buffer (e.g., frees it instead of enqueues it to another queue) 
>> and MAY
>>   be used as a performance optimization if the caller is done with any
>>   references to data that was serialized by the queue (e.g., the queue's
>>   context).  It is an application programming error to release a queue
>>   prematurely as references subsequent to the release will not be
>>   synchronized.
>>
>> Third way - odp_buffer_free() called on a buffer which was dequeued from
> an atomic queue.
>
odp_queue_end(): implicit release of atomic queue
odp_queue_end_multi(): ? I assume multi-calls cannot be used with atomic
scheduling
odp_buffer_free(): (buffer is consumed) implicit release of atomic queue
odp_schedule_release_atomic(): explicit release of atomic queue



>
>>- Ordered: Buffers on an ordered queue can be dequeued by the
>>scheduler for any caller, however buffers on an ordered queue retain
>>knowledge of their sequence on their source queue and this sequence will 
>> be
>>restored whenever they are enqueued to a subsequent ordered queue.  That
>>is, if ordered queue A contains buffers A1, A2, and A3, and these are
>>dequeued for processing by three different threads, then when they are
>>subsequently enqueued to another ordered queue B by these threads, they
>>will appear on B as B1, B2, and B3 regardless of the order in which their
>>processing threads issued odp_queue_enq() calls to place them on B.
>>
>> Why ordering has to be restored only if the destination is an ordered
> queue? I think it should be restored regardless of the destination queue
> type. Also it should be restored if there are multiple destination queues,
> if the implementation supports it.
>
Atomic queues are also ordered queues.

If you enqueue a packet from an atomic or ordered queue onto a parallel
queue, ordering would normally be lost. If there was ever only one thread
scheduling packets from this parallel queue (how do you guarantee that?),
then you could argue that ordering is still maintained. But this seems like
a fragile situation.

Nothing would stop an implementation from always restoring order for
packets scheduled from ordered queues when enqueuing them.


Implicit in these definitions is the fact that all queues associated with
>> odp_pktio_t objects are ordered queues.
>>
> Why is this implicit? The order restoration happens when buffers are
> enqueued to the destination queue(s). Aren't the queues associated with
> odp_pktio_t the first queues seen by a packet? If an pktio queue is
> parallel, there is no requirement at all to ensure any ordering at next
> enqueue.
>
Packet I/O ingress queues may be atomic and thus also ordered in some sense.
I think the application should decide what ordering requirements there are
for packet I/O ingress and egress queues.



>> First question: Are these definitions accurate and complete?  If not,
>> then what are the correct definitions for these types we wish to define?
>> Assuming these are correct, then there are several areas in ODP API that
>> seem to need refinement:
>>
>>- It seems that ODP buffers need at least two 

Re: [lng-odp] D01 hw accelerator investigate

[Maxim Uvarov]

On 10/20/2014 05:29 PM, Weilong Chen wrote:

Hi all,

I'm investigating the D01's HW accelerators, to find out if ODP's API 
can cover all features.

There three cases:
  1. D01 has the accelerator and ODP has the API, such as the crypto.
  2. ODP has but the d01 doesn't support yet.
  3. The HW accelerator is a complex, it is compounded by several 
simple function and special purpose for business.
 For example, the SPPE accelerator, it serve end to end packets 
process, including parsing, forwarding, multicast

 counting etc.

I think when a new HW is introduced to ODP, he will meet the 
same problem:

The ODP's API does not cover all its HW accelerators.

We may allow users to put their own APIs into ODP, like the kernel's 
arch/..., but that will
make problems, the ODP APP can't use the same API, hard to manage the 
code.


Any other idea?

Thanks,
Weilong


Is there any open documentation / specification for d01? I think it's 
hard to answer which exactly HW accelerators ODP does not cover. I think 
that common procedure that you need came with idea/requirements to 
include some block to ODP. Then if that blocks exist in other platforms 
and it's reasonable to include we do arch doc and implementation for 
linux-generic. Then you implement it for your platform. What is SPPE?



Maxim.




___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp



___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] D01 hw accelerator investigate

[Bill Fischofer]

We've discussed the notion of greater packet offload processing, permitting
HW to do more direct processing of packets and protocols and having SW
"fill in" as needed.  I think this is an area of ODP evolution that will
become increasingly important as we move forward.

While our immediate focus is getting ODP v1.0 out this year, beginning in
January as we work to refine the content for the 2015 release of ODP, I
think we also need to look at this larger question.  Specific proposals in
this area should be brought forward for discussion as I'm sure there will
be great interest in this.

Thanks.

Bill

On Mon, Oct 20, 2014 at 8:29 AM, Weilong Chen 
wrote:

> Hi all,
>
> I'm investigating the D01's HW accelerators, to find out if ODP's API can
> cover all features.
> There three cases:
>   1. D01 has the accelerator and ODP has the API, such as the crypto.
>   2. ODP has but the d01 doesn't support yet.
>   3. The HW accelerator is a complex, it is compounded by several simple
> function and special purpose for business.
>  For example, the SPPE accelerator, it serve end to end packets
> process, including parsing, forwarding, multicast
>  counting etc.
>
> I think when a new HW is introduced to ODP, he will meet the same problem:
> The ODP's API does not cover all its HW accelerators.
>
> We may allow users to put their own APIs into ODP, like the kernel's
> arch/..., but that will
> make problems, the ODP APP can't use the same API, hard to manage the code.
>
> Any other idea?
>
> Thanks,
> Weilong
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] D01 hw accelerator investigate

[Weilong Chen]

Hi all,

I'm investigating the D01's HW accelerators, to find out if ODP's API can
cover all features.
There three cases:
  1. D01 has the accelerator and ODP has the API, such as the crypto.
  2. ODP has but the d01 doesn't support yet.
  3. The HW accelerator is a complex, it is compounded by several simple
function and special purpose for business.
 For example, the SPPE accelerator, it serve end to end packets
process, including parsing, forwarding, multicast
 counting etc.

I think when a new HW is introduced to ODP, he will meet the same problem:
The ODP's API does not cover all its HW accelerators.

We may allow users to put their own APIs into ODP, like the kernel's
arch/..., but that will
make problems, the ODP APP can't use the same API, hard to manage the code.

Any other idea?

Thanks,
Weilong
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv2 NETMAP] linux-netmap: Include README in User's Guide

[Ciprian Barbu]

Signed-off-by: Ciprian Barbu 
---
v2:
- Added linux-generic back to users guide
- Added information about default platform in linux-netmap README

 doc/doxygen.cfg  |  2 +-
 doc/users-guide/guide.dox|  6 ++
 platform/linux-netmap/README | 14 +-
 3 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/doc/doxygen.cfg b/doc/doxygen.cfg
index a77ae1e..fc56317 100644
--- a/doc/doxygen.cfg
+++ b/doc/doxygen.cfg
@@ -1,4 +1,4 @@
-PROJECT_NAME = "API Reference Manual"
+PROJECT_NAME = "ODP Netmap API Reference Manual"
 PROJECT_LOGO = $(SRCDIR)/doc/images/ODP-Logo-HQ.png
 QUIET = YES
 OUTPUT_DIRECTORY = $(DOCDIR)
diff --git a/doc/users-guide/guide.dox b/doc/users-guide/guide.dox
index 314d295..11ec395 100644
--- a/doc/users-guide/guide.dox
+++ b/doc/users-guide/guide.dox
@@ -10,9 +10,7 @@
  *
  * @section sec_gene Linux Generic
  * @verbinclude linux-generic/README
- * @section sec_dpdk Linux DPDK
- * @verbinclude linux-dpdk/README
- * @section sec_keys Linux Keystone2
- * @verbinclude linux-keystone2/README
+ * @section sec_netm Linux Netmap
+ * @verbinclude linux-netmap/README
  *
  */
diff --git a/platform/linux-netmap/README b/platform/linux-netmap/README
index 6021445..795a6fc 100644
--- a/platform/linux-netmap/README
+++ b/platform/linux-netmap/README
@@ -66,8 +66,20 @@ Now compile netmap:
 2.2 Building ODP
 
 
+This default platform for this repository is linux-netmap, if not otherwise
+specified using the --with-platform configure variable.
+The optional --with-sdk-install-path variable is used to point to the netmap
+sources, if it's not specified the build system will try to find netmap header
+files in the standard include directories.
+
 ./bootstrap
-./configure --with-platform=linux-netmap --with-sdk-install-path=
+
+To configure ODP for linux-netmap:
+./configure --with-sdk-install-path=
+
+To configure ODP for linux-generic:
+./configure --with-platform=linux-generic
+
 make
 
 3. Running the example application
-- 
1.8.3.2


___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [ODP/PATCH v3] Look ma, no barriers! C11 memory model

[Ola Liljedahl]

Signed-off-by: Ola Liljedahl 
---
Added header file odp_counter.h with support for 32- and 64-bit atomic counters
using relaxed memory order. 6 operations (init/read/write/add/read_inc/inc) on
32-bit and 64-bit counters respectively.
Renamed odp_atomic_test to odp_counter_test and changed to use odp_counter.h

Implementation of C11-based memory model for atomic operations. 10 operations
(init/load/store/cmp_xchg_weak/fetch_add/add/fetch_inc/inc/fetch_dec/dec) in
odp_atomic.h. The required memory ordering is now a parameter to each call just
like in C11.

Optimized support for ARMv6/v7, x86_64, OCTEON. Other architectures will
fall back to GCC __sync builtins which often include unnecessarily heavy
barrier/sync operations (always sequentially consistent).

Attempt to remove all explicit memory barriers (odp_sync_stores) from code that
implements multithreaded synchronization primitives (e.g. locks, barriers).
Rewrote such primitives to use the new atomic operations.

Fixed race conditions in odp_barrier_sync() (non-atomic wrap of counter),
odp_ticketlock_lock() (missing acquire barrier) and odp_ring enqueue/dequeue
(missing release barrier, had only compiler barrier).

 .gitignore |   2 +-
 example/generator/odp_generator.c  |  43 +-
 example/ipsec/odp_ipsec.c  |   2 +-
 example/odp_example/odp_example.c  |   2 +-
 example/timer/odp_timer_test.c |   2 +-
 helper/include/odph_ring.h |   8 +-
 platform/linux-generic/include/api/odp.h   |   1 +
 platform/linux-generic/include/api/odp_atomic.h| 838 +++--
 platform/linux-generic/include/api/odp_barrier.h   |  10 +-
 platform/linux-generic/include/api/odp_counter.h   | 363 +
 platform/linux-generic/include/api/odp_rwlock.h|  20 +-
 .../linux-generic/include/api/odp_ticketlock.h |   5 +-
 .../linux-generic/include/odp_buffer_internal.h|   2 +-
 platform/linux-generic/include/odp_spin_internal.h |   9 -
 platform/linux-generic/odp_barrier.c   |  49 +-
 platform/linux-generic/odp_buffer.c|   3 +-
 platform/linux-generic/odp_crypto.c|   7 +-
 platform/linux-generic/odp_queue.c |   7 +-
 platform/linux-generic/odp_ring.c  |  94 +--
 platform/linux-generic/odp_rwlock.c|  62 +-
 platform/linux-generic/odp_thread.c|   9 +-
 platform/linux-generic/odp_ticketlock.c|  29 +-
 platform/linux-generic/odp_timer.c |  22 +-
 test/api_test/Makefile.am  |   6 +-
 test/api_test/odp_atomic_test.c| 362 -
 test/api_test/odp_atomic_test.h|  60 --
 test/api_test/odp_common.c |   1 -
 test/api_test/odp_counter_test.c   | 361 +
 28 files changed, 1365 insertions(+), 1014 deletions(-)
 create mode 100644 platform/linux-generic/include/api/odp_counter.h
 delete mode 100644 test/api_test/odp_atomic_test.c
 delete mode 100644 test/api_test/odp_atomic_test.h
 create mode 100644 test/api_test/odp_counter_test.c

diff --git a/.gitignore b/.gitignore
index 6342e34..77db4d6 100644
--- a/.gitignore
+++ b/.gitignore
@@ -35,7 +35,7 @@ build/
 odp_example
 odp_packet
 odp_packet_netmap
-odp_atomic
+odp_counter
 odp_shm
 odp_ring
 odp_timer_ping
diff --git a/example/generator/odp_generator.c 
b/example/generator/odp_generator.c
index eb8b340..252157d 100644
--- a/example/generator/odp_generator.c
+++ b/example/generator/odp_generator.c
@@ -62,10 +62,10 @@ typedef struct {
  * counters
 */
 static struct {
-   odp_atomic_u64_t seq;   /**< ip seq to be send */
-   odp_atomic_u64_t ip;/**< ip packets */
-   odp_atomic_u64_t udp;   /**< udp packets */
-   odp_atomic_u64_t icmp;  /**< icmp packets */
+   odp_counter64_t seq;/**< ip seq to be send */
+   odp_counter64_t ip; /**< ip packets */
+   odp_counter64_t udp;/**< udp packets */
+   odp_counter64_t icmp;   /**< icmp packets */
 } counters;
 
 /** * Thread specific arguments
@@ -201,7 +201,7 @@ static void pack_udp_pkt(odp_buffer_t obuf)
ip->tot_len = odp_cpu_to_be_16(args->appl.payload + ODPH_UDPHDR_LEN +
   ODPH_IPV4HDR_LEN);
ip->proto = ODPH_IPPROTO_UDP;
-   seq = odp_atomic_fetch_add_u64(&counters.seq, 1) % 0x;
+   seq = odp_counter64_read_inc(&counters.seq) % 0x;
ip->id = odp_cpu_to_be_16(seq);
ip->chksum = 0;
odph_ipv4_csum_update(pkt);
@@ -258,7 +258,7 @@ static void pack_icmp_pkt(odp_buffer_t obuf)
ip->tot_len = odp_cpu_to_be_16(args->appl.payload + ODPH_ICMPHDR_LEN +
   ODPH_IPV4HDR_LEN);
ip->proto = ODPH_IPPROTO_ICMP;
-   seq = odp_atomic_fetch_add_u64(&counters.seq, 1) % 0x;
+   seq = odp_counter64_read_inc

[lng-odp] [ODP/PATCH v2] Look ma, no barriers! C11 memory model

[Ola Liljedahl]

Signed-off-by: Ola Liljedahl 
---
Added header file odp_counter.h with support for 32- and 64-bit atomic counters
using relaxed memory order. 6 operations (init/read/write/add/read_inc/inc) on
32-bit and 64-bit counters respectively.A
Renamed odp_atomic_test to odp_counter_test and changed to use odp_counter.h

Implementation of C11-based memory model for atomic operations. 10 operations
(init/load/store/cmp_xchg_weak/fetch_add/add/fetch_inc/inc/fetch_dec/dec) in
odp_atomic.h. The required memory ordering is now a parameter to each call just
like in C11.

Optimized support for ARMv6/v7, x86_64, OCTEON. Other architectures will
fall back to GCC __sync builtins which often include unnecessarily heavy
barrier/sync operations (always sequentially consistent).

Attempt to remove all explicit memory barriers (odp_sync_stores) from code that
implements multithreaded synchronization primitives (e.g. locks, barriers).
Rewrote such primitives to use the new atomic operations.

Fixed race conditions in odp_barrier_sync() (non-atomic wrap of counter),
odp_ticketlock_lock() (missing acquire barrier) and odp_ring enqueue/dequeue
(missing release barrier, had only compiler barrier).

 .gitignore |   2 +-
 example/generator/odp_generator.c  |  43 +-
 example/ipsec/odp_ipsec.c  |   2 +-
 example/odp_example/odp_example.c  |   2 +-
 example/timer/odp_timer_test.c |   2 +-
 helper/include/odph_ring.h |   8 +-
 platform/linux-generic/include/api/odp.h   |   1 +
 platform/linux-generic/include/api/odp_atomic.h| 838 +++--
 platform/linux-generic/include/api/odp_barrier.h   |  10 +-
 platform/linux-generic/include/api/odp_counter.h   | 363 +
 platform/linux-generic/include/api/odp_rwlock.h|  20 +-
 .../linux-generic/include/api/odp_ticketlock.h |   5 +-
 .../linux-generic/include/odp_buffer_internal.h|   2 +-
 platform/linux-generic/include/odp_spin_internal.h |   9 -
 platform/linux-generic/odp_barrier.c   |  49 +-
 platform/linux-generic/odp_buffer.c|   3 +-
 platform/linux-generic/odp_crypto.c|   7 +-
 platform/linux-generic/odp_queue.c |   7 +-
 platform/linux-generic/odp_ring.c  |  94 +--
 platform/linux-generic/odp_rwlock.c|  62 +-
 platform/linux-generic/odp_thread.c|   9 +-
 platform/linux-generic/odp_ticketlock.c|  29 +-
 platform/linux-generic/odp_timer.c |  22 +-
 test/api_test/Makefile.am  |   6 +-
 test/api_test/odp_atomic_test.c| 362 -
 test/api_test/odp_atomic_test.h|  60 --
 test/api_test/odp_common.c |   1 -
 test/api_test/odp_counter_test.c   | 361 +
 28 files changed, 1365 insertions(+), 1014 deletions(-)
 create mode 100644 platform/linux-generic/include/api/odp_counter.h
 delete mode 100644 test/api_test/odp_atomic_test.c
 delete mode 100644 test/api_test/odp_atomic_test.h
 create mode 100644 test/api_test/odp_counter_test.c

diff --git a/.gitignore b/.gitignore
index 6342e34..77db4d6 100644
--- a/.gitignore
+++ b/.gitignore
@@ -35,7 +35,7 @@ build/
 odp_example
 odp_packet
 odp_packet_netmap
-odp_atomic
+odp_counter
 odp_shm
 odp_ring
 odp_timer_ping
diff --git a/example/generator/odp_generator.c 
b/example/generator/odp_generator.c
index eb8b340..252157d 100644
--- a/example/generator/odp_generator.c
+++ b/example/generator/odp_generator.c
@@ -62,10 +62,10 @@ typedef struct {
  * counters
 */
 static struct {
-   odp_atomic_u64_t seq;   /**< ip seq to be send */
-   odp_atomic_u64_t ip;/**< ip packets */
-   odp_atomic_u64_t udp;   /**< udp packets */
-   odp_atomic_u64_t icmp;  /**< icmp packets */
+   odp_counter64_t seq;/**< ip seq to be send */
+   odp_counter64_t ip; /**< ip packets */
+   odp_counter64_t udp;/**< udp packets */
+   odp_counter64_t icmp;   /**< icmp packets */
 } counters;
 
 /** * Thread specific arguments
@@ -201,7 +201,7 @@ static void pack_udp_pkt(odp_buffer_t obuf)
ip->tot_len = odp_cpu_to_be_16(args->appl.payload + ODPH_UDPHDR_LEN +
   ODPH_IPV4HDR_LEN);
ip->proto = ODPH_IPPROTO_UDP;
-   seq = odp_atomic_fetch_add_u64(&counters.seq, 1) % 0x;
+   seq = odp_counter64_read_inc(&counters.seq) % 0x;
ip->id = odp_cpu_to_be_16(seq);
ip->chksum = 0;
odph_ipv4_csum_update(pkt);
@@ -258,7 +258,7 @@ static void pack_icmp_pkt(odp_buffer_t obuf)
ip->tot_len = odp_cpu_to_be_16(args->appl.payload + ODPH_ICMPHDR_LEN +
   ODPH_IPV4HDR_LEN);
ip->proto = ODPH_IPPROTO_ICMP;
-   seq = odp_atomic_fetch_add_u64(&counters.seq, 1) % 0x;
+   seq = odp_counter64_read_in

Re: [lng-odp] [PATCH NETMAP] linux-netmap: set default platform to linux-netmap

[Ciprian Barbu]

On Fri, Oct 17, 2014 at 4:34 PM, Mike Holmes  wrote:
>
>
> On 17 October 2014 09:21, Ciprian Barbu  wrote:
>>
>> Signed-off-by: Ciprian Barbu 
>
>
> Reviewed-by: Mike Holmes 
>
> This makes the default be the build of the repo just pulled rather than a
> confusing build of linux-generic which is also included mostly to steal its
> headers.

Applied, thank you! I added the extra description lines for better tracking.

>
>
>>
>> ---
>>  configure.ac | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/configure.ac b/configure.ac
>> index bba280f..a1c68ba 100644
>> --- a/configure.ac
>> +++ b/configure.ac
>> @@ -45,9 +45,9 @@ AC_TYPE_UINT64_T
>>
>> ##
>>  AC_ARG_WITH([platform],
>>  [AS_HELP_STRING([--with-platform=prefix],
>> -[Select platform to be used, default linux-generic])],
>> +[Select platform to be used, default linux-netmap])],
>>  [],
>> -[with_platform=linux-generic])
>> +[with_platform=linux-netmap])
>>
>>  AC_SUBST([with_platform])
>>
>> --
>> 1.8.3.2
>>
>>
>> ___
>> lng-odp mailing list
>> lng-odp@lists.linaro.org
>> http://lists.linaro.org/mailman/listinfo/lng-odp
>
>
>
>
> --
> Mike Holmes
> Linaro  Sr Technical Manager
> LNG - ODP

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH NETMAP] linux-netmap: update Copyright

[Ciprian Barbu]

On Mon, Oct 20, 2014 at 3:17 PM, Ciprian Barbu  wrote:
> On Fri, Oct 17, 2014 at 4:28 PM, Ciprian Barbu  
> wrote:
>> Signed-off-by: Ciprian Barbu 
>> ---
>>  platform/linux-netmap/include/api/odp_pktio_netmap.h   | 2 +-
>>  platform/linux-netmap/include/api/odp_pktio_types.h| 2 +-
>>  platform/linux-netmap/include/odp_packet_io_internal.h | 2 +-
>>  platform/linux-netmap/odp_packet_io.c  | 2 +-
>>  platform/linux-netmap/odp_packet_netmap.c  | 4 ++--
>>  platform/linux-netmap/odp_packet_netmap.h  | 2 +-
>>  6 files changed, 7 insertions(+), 7 deletions(-)
>>
>> diff --git a/platform/linux-netmap/include/api/odp_pktio_netmap.h 
>> b/platform/linux-netmap/include/api/odp_pktio_netmap.h
>> index 51b5c8d..8c7412e 100644
>> --- a/platform/linux-netmap/include/api/odp_pktio_netmap.h
>> +++ b/platform/linux-netmap/include/api/odp_pktio_netmap.h
>> @@ -1,5 +1,5 @@
>>
>> -/* Copyright (c) 2013, Linaro Limited
>> +/* Copyright (c) 2014, Linaro Limited
>
> Ok, I'm going to updated to 2013-2014 like Maxim suggested.
>
>>   * All rights reserved.
>>   *
>>   * SPDX-License-Identifier: BSD-3-Clause
>> diff --git a/platform/linux-netmap/include/api/odp_pktio_types.h 
>> b/platform/linux-netmap/include/api/odp_pktio_types.h
>> index b7c6341..e302434 100644
>> --- a/platform/linux-netmap/include/api/odp_pktio_types.h
>> +++ b/platform/linux-netmap/include/api/odp_pktio_types.h
>> @@ -1,4 +1,4 @@
>> -/* Copyright (c) 2013, Linaro Limited
>> +/* Copyright (c) 2014, Linaro Limited
>>   * All rights reserved.
>>   *
>>   * SPDX-License-Identifier: BSD-3-Clause
>> diff --git a/platform/linux-netmap/include/odp_packet_io_internal.h 
>> b/platform/linux-netmap/include/odp_packet_io_internal.h
>> index a11ca31..1cdef19 100644
>> --- a/platform/linux-netmap/include/odp_packet_io_internal.h
>> +++ b/platform/linux-netmap/include/odp_packet_io_internal.h
>> @@ -1,4 +1,4 @@
>> -/* Copyright (c) 2013, Linaro Limited
>> +/* Copyright (c) 2014, Linaro Limited
>>   * All rights reserved.
>>   *
>>   * SPDX-License-Identifier: BSD-3-Clause
>> diff --git a/platform/linux-netmap/odp_packet_io.c 
>> b/platform/linux-netmap/odp_packet_io.c
>> index 8d6e7d6..2cbd831 100644
>> --- a/platform/linux-netmap/odp_packet_io.c
>> +++ b/platform/linux-netmap/odp_packet_io.c
>> @@ -1,4 +1,4 @@
>> -/* Copyright (c) 2013, Linaro Limited
>> +/* Copyright (c) 2014, Linaro Limited
>>   * All rights reserved.
>>   *
>>   * SPDX-License-Identifier: BSD-3-Clause
>> diff --git a/platform/linux-netmap/odp_packet_netmap.c 
>> b/platform/linux-netmap/odp_packet_netmap.c
>> index 6b9b120..841c5bd 100644
>> --- a/platform/linux-netmap/odp_packet_netmap.c
>> +++ b/platform/linux-netmap/odp_packet_netmap.c
>> @@ -1,5 +1,5 @@
>> -/* Copyright (c) 2013, Linaro Limited
>> - * Copyright (c) 2013, Nokia Solutions and Networks
>> +/* Copyright (c) 2014, Linaro Limited
>> + * Copyright (c) 2014, Nokia Solutions and Networks
>
> I think I made a mistake to put copyright to Nokia here, the logic at
> that point was that I used a few things from odp_packet_socket.c, but
> the content is very different. Should I updated the copyright to
> 2013-2014 Nokia Solutions and Networks, or is it ok to drop it? I
> don't know to get in any sort of legal troubles because of copyright.

Petri, can you help me with this? I think I can still keep Nokia in
the copyright list, but since NSN renamed to Nokia, should I set this
to 2013-2014 Copyright Nokia ?

>
>>   * All rights reserved.
>>   *
>>   * SPDX-License-Identifier: BSD-3-Clause
>> diff --git a/platform/linux-netmap/odp_packet_netmap.h 
>> b/platform/linux-netmap/odp_packet_netmap.h
>> index 4b66d3d..beff803 100644
>> --- a/platform/linux-netmap/odp_packet_netmap.h
>> +++ b/platform/linux-netmap/odp_packet_netmap.h
>> @@ -1,4 +1,4 @@
>> -/* Copyright (c) 2013, Linaro Limited
>> +/* Copyright (c) 2014, Linaro Limited
>>   * All rights reserved.
>>   *
>>   * SPDX-License-Identifier: BSD-3-Clause
>> --
>> 1.8.3.2
>>

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCHv2 NETMAP] linux-netmap: Include README in User's Guide

[Ciprian Barbu]

On Fri, Oct 17, 2014 at 4:56 PM, Mike Holmes  wrote:
>
>
> On 17 October 2014 09:19, Ciprian Barbu  wrote:
>>
>> Signed-off-by: Ciprian Barbu 
>> ---
>>  doc/doxygen.cfg   | 2 +-
>>  doc/users-guide/guide.dox | 8 ++--
>>  2 files changed, 3 insertions(+), 7 deletions(-)
>>
>> diff --git a/doc/doxygen.cfg b/doc/doxygen.cfg
>> index a77ae1e..fc56317 100644
>> --- a/doc/doxygen.cfg
>> +++ b/doc/doxygen.cfg
>> @@ -1,4 +1,4 @@
>> -PROJECT_NAME = "API Reference Manual"
>> +PROJECT_NAME = "ODP Netmap API Reference Manual"
>>  PROJECT_LOGO = $(SRCDIR)/doc/images/ODP-Logo-HQ.png
>>  QUIET = YES
>>  OUTPUT_DIRECTORY = $(DOCDIR)
>> diff --git a/doc/users-guide/guide.dox b/doc/users-guide/guide.dox
>> index 314d295..16fecb4 100644
>> --- a/doc/users-guide/guide.dox
>> +++ b/doc/users-guide/guide.dox
>> @@ -8,11 +8,7 @@
>>   *
>>   * @page users_guide Users Guide
>>   *
>> - * @section sec_gene Linux Generic
>> - * @verbinclude linux-generic/README
>> - * @section sec_dpdk Linux DPDK
>> - * @verbinclude linux-dpdk/README
>> - * @section sec_keys Linux Keystone2
>> - * @verbinclude linux-keystone2/README
>> + * @section sec_netm Linux Netmap
>> + * @verbinclude linux-netmap/README
>
>
> This newly included file included build instructions that may need an update
> if the default build patch is applied
>
> On such change is
>
> ./configure --with-platform=linux-netmap --with-sdk-install-path=
>
> to
>
> ./configure --with-sdk-install-path=
>
>
> And you may now need to add a linux-generic instruction specifying how to
> build that flavor instead.

Ok, I will work on this and get a new version out.

>>
>>   *
>>   */
>> --
>> 1.8.3.2
>>
>>
>> ___
>> lng-odp mailing list
>> lng-odp@lists.linaro.org
>> http://lists.linaro.org/mailman/listinfo/lng-odp
>
>
>
>
> --
> Mike Holmes
> Linaro  Sr Technical Manager
> LNG - ODP

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH NETMAP] linux-netmap: update Copyright

[Ciprian Barbu]

On Fri, Oct 17, 2014 at 4:28 PM, Ciprian Barbu  wrote:
> Signed-off-by: Ciprian Barbu 
> ---
>  platform/linux-netmap/include/api/odp_pktio_netmap.h   | 2 +-
>  platform/linux-netmap/include/api/odp_pktio_types.h| 2 +-
>  platform/linux-netmap/include/odp_packet_io_internal.h | 2 +-
>  platform/linux-netmap/odp_packet_io.c  | 2 +-
>  platform/linux-netmap/odp_packet_netmap.c  | 4 ++--
>  platform/linux-netmap/odp_packet_netmap.h  | 2 +-
>  6 files changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/platform/linux-netmap/include/api/odp_pktio_netmap.h 
> b/platform/linux-netmap/include/api/odp_pktio_netmap.h
> index 51b5c8d..8c7412e 100644
> --- a/platform/linux-netmap/include/api/odp_pktio_netmap.h
> +++ b/platform/linux-netmap/include/api/odp_pktio_netmap.h
> @@ -1,5 +1,5 @@
>
> -/* Copyright (c) 2013, Linaro Limited
> +/* Copyright (c) 2014, Linaro Limited

Ok, I'm going to updated to 2013-2014 like Maxim suggested.

>   * All rights reserved.
>   *
>   * SPDX-License-Identifier: BSD-3-Clause
> diff --git a/platform/linux-netmap/include/api/odp_pktio_types.h 
> b/platform/linux-netmap/include/api/odp_pktio_types.h
> index b7c6341..e302434 100644
> --- a/platform/linux-netmap/include/api/odp_pktio_types.h
> +++ b/platform/linux-netmap/include/api/odp_pktio_types.h
> @@ -1,4 +1,4 @@
> -/* Copyright (c) 2013, Linaro Limited
> +/* Copyright (c) 2014, Linaro Limited
>   * All rights reserved.
>   *
>   * SPDX-License-Identifier: BSD-3-Clause
> diff --git a/platform/linux-netmap/include/odp_packet_io_internal.h 
> b/platform/linux-netmap/include/odp_packet_io_internal.h
> index a11ca31..1cdef19 100644
> --- a/platform/linux-netmap/include/odp_packet_io_internal.h
> +++ b/platform/linux-netmap/include/odp_packet_io_internal.h
> @@ -1,4 +1,4 @@
> -/* Copyright (c) 2013, Linaro Limited
> +/* Copyright (c) 2014, Linaro Limited
>   * All rights reserved.
>   *
>   * SPDX-License-Identifier: BSD-3-Clause
> diff --git a/platform/linux-netmap/odp_packet_io.c 
> b/platform/linux-netmap/odp_packet_io.c
> index 8d6e7d6..2cbd831 100644
> --- a/platform/linux-netmap/odp_packet_io.c
> +++ b/platform/linux-netmap/odp_packet_io.c
> @@ -1,4 +1,4 @@
> -/* Copyright (c) 2013, Linaro Limited
> +/* Copyright (c) 2014, Linaro Limited
>   * All rights reserved.
>   *
>   * SPDX-License-Identifier: BSD-3-Clause
> diff --git a/platform/linux-netmap/odp_packet_netmap.c 
> b/platform/linux-netmap/odp_packet_netmap.c
> index 6b9b120..841c5bd 100644
> --- a/platform/linux-netmap/odp_packet_netmap.c
> +++ b/platform/linux-netmap/odp_packet_netmap.c
> @@ -1,5 +1,5 @@
> -/* Copyright (c) 2013, Linaro Limited
> - * Copyright (c) 2013, Nokia Solutions and Networks
> +/* Copyright (c) 2014, Linaro Limited
> + * Copyright (c) 2014, Nokia Solutions and Networks

I think I made a mistake to put copyright to Nokia here, the logic at
that point was that I used a few things from odp_packet_socket.c, but
the content is very different. Should I updated the copyright to
2013-2014 Nokia Solutions and Networks, or is it ok to drop it? I
don't know to get in any sort of legal troubles because of copyright.

>   * All rights reserved.
>   *
>   * SPDX-License-Identifier: BSD-3-Clause
> diff --git a/platform/linux-netmap/odp_packet_netmap.h 
> b/platform/linux-netmap/odp_packet_netmap.h
> index 4b66d3d..beff803 100644
> --- a/platform/linux-netmap/odp_packet_netmap.h
> +++ b/platform/linux-netmap/odp_packet_netmap.h
> @@ -1,4 +1,4 @@
> -/* Copyright (c) 2013, Linaro Limited
> +/* Copyright (c) 2014, Linaro Limited
>   * All rights reserved.
>   *
>   * SPDX-License-Identifier: BSD-3-Clause
> --
> 1.8.3.2
>

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH] Rename ODPH_PACKED macro

[Maxim Uvarov]

Yes, that is reasonable because ODP_PACKET is in global include.

Maxim.

On 10/20/2014 03:49 PM, Jerin Jacob wrote:

- Definition of ODPH_PACKED is in include/api/odp_align.h so changing to 
ODP_PACKED

Signed-off-by: Jerin Jacob 
---
  example/ipsec/odp_ipsec_stream.c   | 2 +-
  helper/include/odph_eth.h  | 6 +++---
  helper/include/odph_icmp.h | 2 +-
  helper/include/odph_ip.h   | 4 ++--
  helper/include/odph_ipsec.h| 6 +++---
  helper/include/odph_udp.h  | 2 +-
  platform/linux-generic/include/api/odp_align.h | 2 +-
  7 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/example/ipsec/odp_ipsec_stream.c b/example/ipsec/odp_ipsec_stream.c
index fba425c..a9a52ec 100644
--- a/example/ipsec/odp_ipsec_stream.c
+++ b/example/ipsec/odp_ipsec_stream.c
@@ -39,7 +39,7 @@
  /**
   * Stream packet header
   */
-typedef struct ODPH_PACKED stream_pkt_hdr_s {
+typedef struct ODP_PACKED stream_pkt_hdr_s {
uint64be_t magic;/**< Stream magic value for verification */
uint8_tdata[0];  /**< Incrementing data stream */
  } stream_pkt_hdr_t;
diff --git a/helper/include/odph_eth.h b/helper/include/odph_eth.h
index 55a2b1e..065a94b 100644
--- a/helper/include/odph_eth.h
+++ b/helper/include/odph_eth.h
@@ -34,7 +34,7 @@ extern "C" {
  /**
   * Ethernet MAC address
   */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint8_t addr[ODPH_ETHADDR_LEN]; /**< @private Address */
  } odph_ethaddr_t;
  
@@ -44,7 +44,7 @@ ODP_STATIC_ASSERT(sizeof(odph_ethaddr_t) == ODPH_ETHADDR_LEN, "ODPH_ETHADDR_T__S

  /**
   * Ethernet header
   */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
odph_ethaddr_t dst; /**< Destination address */
odph_ethaddr_t src; /**< Source address */
uint16be_t type;   /**< Type */
@@ -58,7 +58,7 @@ ODP_STATIC_ASSERT(sizeof(odph_ethhdr_t) == ODPH_ETHHDR_LEN, 
"ODPH_ETHHDR_T__SIZE
   *
   * @todo Check usage of tpid vs ethertype. Check outer VLAN TPID.
   */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint16be_t tpid;   /**< Tag protocol ID (located after ethhdr.src) */
uint16be_t tci;/**< Priority / CFI / VLAN ID */
  } odph_vlanhdr_t;
diff --git a/helper/include/odph_icmp.h b/helper/include/odph_icmp.h
index 8414d7e..8533fb5 100644
--- a/helper/include/odph_icmp.h
+++ b/helper/include/odph_icmp.h
@@ -26,7 +26,7 @@ extern "C" {
  #define ODPH_ICMPHDR_LEN 8
  
  /** ICMP header */

-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint8_t type;   /**< message type */
uint8_t code;   /**< type sub-code */
uint16sum_t chksum; /**< checksum of icmp header */
diff --git a/helper/include/odph_ip.h b/helper/include/odph_ip.h
index ca71c44..2c83c0f 100644
--- a/helper/include/odph_ip.h
+++ b/helper/include/odph_ip.h
@@ -48,7 +48,7 @@ extern "C" {
  #define ODPH_IPV4HDR_IS_FRAGMENT(frag_offset) ((frag_offset) & 0x3fff)
  
  /** IPv4 header */

-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint8_tver_ihl; /**< Version / Header length */
uint8_ttos; /**< Type of service */
uint16be_t tot_len; /**< Total length */
@@ -125,7 +125,7 @@ static inline uint16sum_t 
odph_ipv4_csum_update(odp_packet_t pkt)
  /**
   * IPv6 header
   */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint32be_t ver_tc_flow;  /**< Version / Traffic class / Flow label */
uint16be_t payload_len;  /**< Payload length */
uint8_tnext_hdr; /**< Next header */
diff --git a/helper/include/odph_ipsec.h b/helper/include/odph_ipsec.h
index f547b90..c58a1c8 100644
--- a/helper/include/odph_ipsec.h
+++ b/helper/include/odph_ipsec.h
@@ -30,7 +30,7 @@ extern "C" {
  /**
   * IPSec ESP header
   */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint32be_t spi;  /**< Security Parameter Index */
uint32be_t seq_no;   /**< Sequence Number */
uint8_tiv[0];/**< Initialization vector */
@@ -42,7 +42,7 @@ ODP_STATIC_ASSERT(sizeof(odph_esphdr_t) == ODPH_ESPHDR_LEN, 
"ODPH_ESPHDR_T__SIZE
  /**
   * IPSec ESP trailer
   */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint8_t pad_len;  /**< Padding length (0-255) */
uint8_t next_header;  /**< Next header protocol */
uint8_t icv[0];   /**< Integrity Check Value (optional) */
@@ -54,7 +54,7 @@ ODP_STATIC_ASSERT(sizeof(odph_esptrl_t) == ODPH_ESPTRL_LEN, 
"ODPH_ESPTRL_T__SIZE
  /**
   * IPSec AH header
   */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint8_tnext_header;  /**< Next header protocol */
uint8_tah_len;   /**< AH header length */
uint16be_t pad;  /**< Padding (must be 0) */
diff --git a/helper/include/odph_udp.h b/helper/include/odph_udp.h
index 3970f00..b2eaf03 100644
--- a/helper/incl

[lng-odp] [PATCH] Rename ODPH_PACKED macro

[Jerin Jacob]

- Definition of ODPH_PACKED is in include/api/odp_align.h so changing to 
ODP_PACKED

Signed-off-by: Jerin Jacob 
---
 example/ipsec/odp_ipsec_stream.c   | 2 +-
 helper/include/odph_eth.h  | 6 +++---
 helper/include/odph_icmp.h | 2 +-
 helper/include/odph_ip.h   | 4 ++--
 helper/include/odph_ipsec.h| 6 +++---
 helper/include/odph_udp.h  | 2 +-
 platform/linux-generic/include/api/odp_align.h | 2 +-
 7 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/example/ipsec/odp_ipsec_stream.c b/example/ipsec/odp_ipsec_stream.c
index fba425c..a9a52ec 100644
--- a/example/ipsec/odp_ipsec_stream.c
+++ b/example/ipsec/odp_ipsec_stream.c
@@ -39,7 +39,7 @@
 /**
  * Stream packet header
  */
-typedef struct ODPH_PACKED stream_pkt_hdr_s {
+typedef struct ODP_PACKED stream_pkt_hdr_s {
uint64be_t magic;/**< Stream magic value for verification */
uint8_tdata[0];  /**< Incrementing data stream */
 } stream_pkt_hdr_t;
diff --git a/helper/include/odph_eth.h b/helper/include/odph_eth.h
index 55a2b1e..065a94b 100644
--- a/helper/include/odph_eth.h
+++ b/helper/include/odph_eth.h
@@ -34,7 +34,7 @@ extern "C" {
 /**
  * Ethernet MAC address
  */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint8_t addr[ODPH_ETHADDR_LEN]; /**< @private Address */
 } odph_ethaddr_t;
 
@@ -44,7 +44,7 @@ ODP_STATIC_ASSERT(sizeof(odph_ethaddr_t) == ODPH_ETHADDR_LEN, 
"ODPH_ETHADDR_T__S
 /**
  * Ethernet header
  */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
odph_ethaddr_t dst; /**< Destination address */
odph_ethaddr_t src; /**< Source address */
uint16be_t type;   /**< Type */
@@ -58,7 +58,7 @@ ODP_STATIC_ASSERT(sizeof(odph_ethhdr_t) == ODPH_ETHHDR_LEN, 
"ODPH_ETHHDR_T__SIZE
  *
  * @todo Check usage of tpid vs ethertype. Check outer VLAN TPID.
  */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint16be_t tpid;   /**< Tag protocol ID (located after ethhdr.src) */
uint16be_t tci;/**< Priority / CFI / VLAN ID */
 } odph_vlanhdr_t;
diff --git a/helper/include/odph_icmp.h b/helper/include/odph_icmp.h
index 8414d7e..8533fb5 100644
--- a/helper/include/odph_icmp.h
+++ b/helper/include/odph_icmp.h
@@ -26,7 +26,7 @@ extern "C" {
 #define ODPH_ICMPHDR_LEN 8
 
 /** ICMP header */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint8_t type;   /**< message type */
uint8_t code;   /**< type sub-code */
uint16sum_t chksum; /**< checksum of icmp header */
diff --git a/helper/include/odph_ip.h b/helper/include/odph_ip.h
index ca71c44..2c83c0f 100644
--- a/helper/include/odph_ip.h
+++ b/helper/include/odph_ip.h
@@ -48,7 +48,7 @@ extern "C" {
 #define ODPH_IPV4HDR_IS_FRAGMENT(frag_offset) ((frag_offset) & 0x3fff)
 
 /** IPv4 header */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint8_tver_ihl; /**< Version / Header length */
uint8_ttos; /**< Type of service */
uint16be_t tot_len; /**< Total length */
@@ -125,7 +125,7 @@ static inline uint16sum_t 
odph_ipv4_csum_update(odp_packet_t pkt)
 /**
  * IPv6 header
  */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint32be_t ver_tc_flow;  /**< Version / Traffic class / Flow label */
uint16be_t payload_len;  /**< Payload length */
uint8_tnext_hdr; /**< Next header */
diff --git a/helper/include/odph_ipsec.h b/helper/include/odph_ipsec.h
index f547b90..c58a1c8 100644
--- a/helper/include/odph_ipsec.h
+++ b/helper/include/odph_ipsec.h
@@ -30,7 +30,7 @@ extern "C" {
 /**
  * IPSec ESP header
  */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint32be_t spi;  /**< Security Parameter Index */
uint32be_t seq_no;   /**< Sequence Number */
uint8_tiv[0];/**< Initialization vector */
@@ -42,7 +42,7 @@ ODP_STATIC_ASSERT(sizeof(odph_esphdr_t) == ODPH_ESPHDR_LEN, 
"ODPH_ESPHDR_T__SIZE
 /**
  * IPSec ESP trailer
  */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint8_t pad_len;  /**< Padding length (0-255) */
uint8_t next_header;  /**< Next header protocol */
uint8_t icv[0];   /**< Integrity Check Value (optional) */
@@ -54,7 +54,7 @@ ODP_STATIC_ASSERT(sizeof(odph_esptrl_t) == ODPH_ESPTRL_LEN, 
"ODPH_ESPTRL_T__SIZE
 /**
  * IPSec AH header
  */
-typedef struct ODPH_PACKED {
+typedef struct ODP_PACKED {
uint8_tnext_header;  /**< Next header protocol */
uint8_tah_len;   /**< AH header length */
uint16be_t pad;  /**< Padding (must be 0) */
diff --git a/helper/include/odph_udp.h b/helper/include/odph_udp.h
index 3970f00..b2eaf03 100644
--- a/helper/include/odph_udp.h
+++ b/helper/include/odph_udp.h
@@ -26,7 +26,7 @@ extern "C" {
 #define ODPH_UDPHDR_LEN 8
 
 /** UDP header */
-typedef struct ODPH_PACKED {
+typede

Re: [lng-odp] [ARCH DESIGN] Queues and Synchronization/Scheduling models

[Ciprian Barbu]

On Fri, Oct 17, 2014 at 11:01 AM, Alexandru Badicioiu
 wrote:
> Hi Bill, check my thoughts inline.
> Thanks,
> Alex
>
> On 17 October 2014 03:31, Bill Fischofer  wrote:
>>
>> Based on discussions we had yesterday and today, I'd like to outline the
>> open issues regarding queues and synchronization/scheduling models.  We'd
>> like to get consensus on this in time for next week's Tuesday call.
>>
>> ODP identifies three different synchronization/scheduling models for
>> queues: Parallel, Atomic, and Ordered.  Here are my current understandings
>> of what these mean:
>>
>> Parallel: Buffers on a parallel queue can be dequeued by the scheduler for
>> any caller without restriction.  This permits maximum scale-out and
>> concurrency for events that are truly independent.
>>
>> Atomic: Buffers on an atomic queue can be dequeued by the scheduler for
>> any caller. However, only one buffer from an atomic queue may be in process
>> at any given time. When the scheduler dequeues a buffer from an atomic
>> queue, the queue is locked and cannot dequeue further buffers until it is
>> released.  Releasing an atomic queue can occur in two ways:
>>
>> The dequeued buffer is enqueued to another queue via an odp_queue_enq()
>> call. This action implicitly unlocks the atomic queue the buffer was sourced
>> from.  Note that this is the most common way in which atomic queues are
>> unlocked.
>>
>> A call is made to odp_schedule_release_atomic() for the locked queue.
>> This tells the scheduler that the queue's atomicity guarantee is deemed
>> satisfied by the application and the queue is free to dequeue items to other
>> scheduler callers. This method MUST be used if the caller consumes the
>> buffer (e.g., frees it instead of enqueues it to another queue) and MAY be
>> used as a performance optimization if the caller is done with any references
>> to data that was serialized by the queue (e.g., the queue's context).  It is
>> an application programming error to release a queue prematurely as
>> references subsequent to the release will not be synchronized.
>
> Third way - odp_buffer_free() called on a buffer which was dequeued from an
> atomic queue.
>>
>> Ordered: Buffers on an ordered queue can be dequeued by the scheduler for
>> any caller, however buffers on an ordered queue retain knowledge of their
>> sequence on their source queue and this sequence will be restored whenever
>> they are enqueued to a subsequent ordered queue.  That is, if ordered queue
>> A contains buffers A1, A2, and A3, and these are dequeued for processing by
>> three different threads, then when they are subsequently enqueued to another
>> ordered queue B by these threads, they will appear on B as B1, B2, and B3
>> regardless of the order in which their processing threads issued
>> odp_queue_enq() calls to place them on B.
>
> Why ordering has to be restored only if the destination is an ordered queue?
> I think it should be restored regardless of the destination queue type. Also
> it should be restored if there are multiple destination queues, if the
> implementation supports it.

Well then how can you inform the ODP implementation that it's ok to
stop maintaining the order the packets came in? What if the flow (I
understand flows make most sense to  be ordered) contains a further
encapsulation known by the application which can then process the
packets in parallel in any given order?

>>
>> Implicit in these definitions is the fact that all queues associated with
>> odp_pktio_t objects are ordered queues.
>
> Why is this implicit? The order restoration happens when buffers are
> enqueued to the destination queue(s). Aren't the queues associated with
> odp_pktio_t the first queues seen by a packet? If an pktio queue is
> parallel, there is no requirement at all to ensure any ordering at next
> enqueue.

Especially in the case of Ethernet, that is the main target for ODP
v1.0, where packet order is not guaranteed and rather performance
(processing close to line rate) is desired, it doesn't make sense to
make the input queues ordered by default. It makes sense to make the
output queues from the classifier ordered by default, I guess.

>>
>>
>> First question: Are these definitions accurate and complete?  If not, then
>> what are the correct definitions for these types we wish to define?
>> Assuming these are correct, then there are several areas in ODP API that
>> seem to need refinement:
>>
>> It seems that ODP buffers need at least two additional pieces of system
>> meta data that are missing:
>>
>> Buffers need to have a last_queue that is the odp_queue_t of the last
>> queue they were dequeued from.  Needed so that odp_queue_enq() can unlock a
>> previous atomic queue.
>>
>> Buffers need to retain sequence knowledge from the first ordered queue
>> they were sourced from (i.e., their ingress odp_pktio_t) so this can be used
>> for order restoration as they are enqueued to downstream ordered queues
>>
>> odp_schedule_release_atomic() currently ta