Re: [lng-odp] [RFC] Addition to odp_init interface

[Shmulik Ladkani]

Hi,

> I don't have a big picture of what you are trying to achieve, but ODP
> is not a framework for thread-local initialization of 3rd party
> libraries.

I completely agree.

However my usecase is an exception, as I was trying to create a C++
binding for ODP - hence it isn't just a "different" library which uses
ODP, but rather "wraps" ODP.

In that sense, if for example the C++ binding was decided to be integral
part of ODP, than 'odp_init_local' would simply call 'stl_init_local'
*directly*.

Anyway, I would be glad to elaborate on the motivation in today's call.

> odph_linux_pthread_create() is a helper routine, so it is not a part of
> normative ODP API.

OK. So may I suggest the following.

Currently, 'odph_linux_pthread_create' helper *implicitly* calls
'odp_init_local' upon thread start, prior invoking user's start_routine.

Obviously, this removes user's burden of calling 'odp_init_local'
directly early on her start_routine - at least for users using
odph_linux_pthread_create.

However, *if* a different platform implemenets its own helper (yes, not
a part of formal ODP API), it might/might-not follow the same "automatic
convenience" - depending on platform implementor's will/preference.

Thus, the user (which writes platform agnostic code) implementing a
thread routine needs to be aware of what the helper does (does it
automatically call odp_init_local? or, need I call it on my own?).

Therefore, I suggest removing the implicit call to 'odp_init_local' from
the existing odph_linux_pthread_create helper - forcing the user to
*always* call 'odp_init_local' early in her thread routine - similarly
to what she does early on 'main()'.

Regards,
Shmulik

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [Q] Memory allocation in ODP applications

[Gilad Ben Yossef]

I think it's fair to note though that applications that uses any API outside 
ODP may not be portable across other ODP platforms, or even exhibit different 
performance characteristic from ODP APIs on the same platform.

Therefore I expect that *all other things being equal*, there is an advantage 
for the application writer to use ODP APIs where applicable APIs exists.

If you're faced with an already existing hunk of code, that probably doesn't 
matter at all. If you're writing a new application and you are targeting ODP 
anyway, I would find it weird if you'd use a none ODP API for something ODP has 
an API for unless the ODP API semantic is somehow missing something.

Gilad






Gilad Ben-Yossef
Software Architect
EZchip Technologies Ltd.
37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel
Tel: +972-4-959- ext. 576, Fax: +972-8-681-1483
Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388
Email: gil...@ezchip.com, Web: 
http://www.ezchip.com

"Ethernet always wins."
— Andy Bechtolsheim

From: lng-odp-boun...@lists.linaro.org 
[mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of Bill Fischofer
Sent: Monday, November 03, 2014 8:11 PM
To: Shmulik Ladkani
Cc: lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [Q] Memory allocation in ODP applications

ODP APIs are designed to be used a la carte by applications, as ODP is a 
framework, not a platform.  So feel free to mix malloc() or your own memory 
management or other API calls in as needed.

What ODP requires is the types specified in its APIs, so for example the only 
way to get an odp_buffer_t is via the odp_buffer_alloc() call.  
odp_buffer_alloc() in turn requires an odp_buffer_pool_t and that in turn 
requires an odp_buffer_pool_create() call.

ODP_BUFFER_TYPE_RAW simply exposes the basic block manager functions of the ODP 
buffer APIs.  Again, you're free to use them for whatever purpose the 
application wants.  Obviously one reason for doing so is to gain portability 
across potentially different memory management implementations.

On Mon, Nov 3, 2014 at 11:02 AM, Shmulik Ladkani 
mailto:shmulik.ladk...@gmail.com>> wrote:
Hi,

As demonstrated by odp/examples, libc memory allocation routines (malloc
and friends) aren't been used by ODP apps (unless for a very tight
temporary scope, usually in the control thread).

Application's data structures are allocated either using odp_buffer_pool
interface (e.g. odp_ipsec.c allocating its private per packet context
structure using a ODP_BUFFER_TYPE_RAW pool), or directly using
odp_shm_reserve and managing it internally (e.g. odp_ipsec_fwd_db.c,
odp_ipsec_sp_db.c etc).

Questions:

Must the application use ODP memory allocation interfaces at all times?
Or only for data structures accessed at data-path routines?
Or only when allocating them from within data-path routines?

Any preference for when to use a ODP_BUFFER_TYPE_RAW pool vs. allocating
a shared memory and managing data structures internally?

Thanks,
Shmulik

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [RFC] Addition to odp_init interface

[Gilad Ben Yossef]

You are aware that there is no assumption in ODP that the different ODP tasks 
are threads, right?
Gilad

Gilad Ben-Yossef
Software Architect
EZchip Technologies Ltd.
37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel
Tel: +972-4-959- ext. 576, Fax: +972-8-681-1483 
Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388
Email: gil...@ezchip.com, Web: http://www.ezchip.com

"Ethernet always wins."
— Andy Bechtolsheim

> -Original Message-
> From: lng-odp-boun...@lists.linaro.org [mailto:lng-odp-
> boun...@lists.linaro.org] On Behalf Of Shmulik Ladkani
> Sent: Monday, November 03, 2014 11:50 AM
> To: lng-odp@lists.linaro.org
> Subject: [lng-odp] [RFC] Addition to odp_init interface
> 
> Hi,
> 
> I'd like to suggest the following addition to odp_init.h:
> 
> diff --git a/platform/linux-generic/include/api/odp_init.h
> b/platform/linux-generic/include/api/odp_init.h
> index 8e018cbdba..9b75609fc7 100644
> --- a/platform/linux-generic/include/api/odp_init.h
> +++ b/platform/linux-generic/include/api/odp_init.h
> @@ -108,6 +108,15 @@ int odp_term_global(void);
>   */
>  int odp_init_local(void);
> 
> +/**
> + * Register a function to be called upon thread local initialization
> + *
> + * XXX: Should the 'init_local_fn' be invoked for currently existing
> threads?
> + *  Probably yes.
> + *  (Alternative: user must call 'odp_init_local_register' before
> the
> + *  first 'odp_init_local' call)
> + */
> +int odp_init_local_register(int (*init_local_fn)(void));
> 
> Motivation is to allow a custom library built on top of ODP to
> initialize its per-thread local data structures.
> 
> Comments?
> 
> Shmulik
> 
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] Today's ODP Call

[Bill Fischofer]

Apologies for the late notice, but I realized that although we've stated
that this call is scheduled for 15:00 UTC, it was mistakenly put into the
Google calendar in a US timezone that's affected by daylight savings time.
That's now been corrected and the call is indeed now scheduled for 15:00
UTC.

With the US return to standard time this past weekend this means that for
most of the US the call will now occur one hour earlier (10am Eastern, 9am
Central, 7am Pacific).  The time is unchanged for Arizonans (8am) since
they don't participate in daylight savings time.  The call location is
unchanged: uberconference.com/opendataplane

Proposed agenda for today's call is:

Agenda:

   -

   Petri’s Ordered Queue Patch
    and
   discussion
   -

  Step-by-Step queue order definition
  -

  New function odp_schedule_skip_order()
  -

   Follow-up discussions on PktIO Design Doc
   

   -

   Shmulik’s proposal for init extensions
   



Thank you.


Bill
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [RFC] Addition to odp_init interface

[Bill Fischofer]

I think this is an important discussion so I've added it to the agenda for
today's ODP call.  Please see my earlier note regarding the time of the
call.  Its remains at 15:00 UTC.

Gilad's point is correct.  While ODP uses the term 'thread' (at least for
v1.0), 'task' is probably a better term since the ODP APIs are designed to
be independent of whether the application is organized into one process or
multiple processes (address spaces).  It's one of the reasons that ODP
buffer pools (in Linux) are created in shared memory so that applications
organized as processes can share buffers.

Bill

On Tue, Nov 4, 2014 at 3:26 AM, Gilad Ben Yossef  wrote:

>
> You are aware that there is no assumption in ODP that the different ODP
> tasks are threads, right?
> Gilad
>
> Gilad Ben-Yossef
> Software Architect
> EZchip Technologies Ltd.
> 37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel
> Tel: +972-4-959- ext. 576, Fax: +972-8-681-1483
> Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388
> Email: gil...@ezchip.com, Web: http://www.ezchip.com
>
> "Ethernet always wins."
> — Andy Bechtolsheim
>
> > -Original Message-
> > From: lng-odp-boun...@lists.linaro.org [mailto:lng-odp-
> > boun...@lists.linaro.org] On Behalf Of Shmulik Ladkani
> > Sent: Monday, November 03, 2014 11:50 AM
> > To: lng-odp@lists.linaro.org
> > Subject: [lng-odp] [RFC] Addition to odp_init interface
> >
> > Hi,
> >
> > I'd like to suggest the following addition to odp_init.h:
> >
> > diff --git a/platform/linux-generic/include/api/odp_init.h
> > b/platform/linux-generic/include/api/odp_init.h
> > index 8e018cbdba..9b75609fc7 100644
> > --- a/platform/linux-generic/include/api/odp_init.h
> > +++ b/platform/linux-generic/include/api/odp_init.h
> > @@ -108,6 +108,15 @@ int odp_term_global(void);
> >   */
> >  int odp_init_local(void);
> >
> > +/**
> > + * Register a function to be called upon thread local initialization
> > + *
> > + * XXX: Should the 'init_local_fn' be invoked for currently existing
> > threads?
> > + *  Probably yes.
> > + *  (Alternative: user must call 'odp_init_local_register' before
> > the
> > + *  first 'odp_init_local' call)
> > + */
> > +int odp_init_local_register(int (*init_local_fn)(void));
> >
> > Motivation is to allow a custom library built on top of ODP to
> > initialize its per-thread local data structures.
> >
> > Comments?
> >
> > Shmulik
> >
> > ___
> > lng-odp mailing list
> > lng-odp@lists.linaro.org
> > http://lists.linaro.org/mailman/listinfo/lng-odp
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [Q] Memory allocation in ODP applications

[Bill Fischofer]

I'd add to this if the reason for using a non-ODP API is that the
equivalent ODP API is missing something we'd like to know about that so
that may be corrected in a future revision of the API.  Over time I'd
expect the ODP APIs to gain richness both as the framework matures and as
it tracks the increasing capabilities of the platforms that support it.

On Tue, Nov 4, 2014 at 3:24 AM, Gilad Ben Yossef  wrote:

>
>
> I think it's fair to note though that applications that uses any API
> outside ODP may not be portable across other ODP platforms, or even exhibit
> different performance characteristic from ODP APIs on the same platform.
>
>
>
> Therefore I expect that **all other things being equal**, there is an
> advantage for the application writer to use ODP APIs where applicable APIs
> exists.
>
>
>
> If you're faced with an already existing hunk of code, that probably
> doesn't matter at all. If you're writing a new application and you are
> targeting ODP anyway, I would find it weird if you'd use a none ODP API for
> something ODP has an API for unless the ODP API semantic is somehow missing
> something.
>
>
>
> Gilad
>
>
>
>
>
>
>
>
>
>
>
>
>
> *Gilad Ben-Yossef*
>
> Software Architect
>
> EZchip Technologies Ltd.
>
> 37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel
>
> Tel: +972-4-959- ext. 576, Fax: +972-8-681-1483
> Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388
>
> Email: gil...@ezchip.com, Web: http://www.ezchip.com
>
>
>
> *"Ethernet always wins."*
>
> — Andy Bechtolsheim
>
>
>
> *From:* lng-odp-boun...@lists.linaro.org [mailto:
> lng-odp-boun...@lists.linaro.org] *On Behalf Of *Bill Fischofer
> *Sent:* Monday, November 03, 2014 8:11 PM
> *To:* Shmulik Ladkani
> *Cc:* lng-odp@lists.linaro.org
> *Subject:* Re: [lng-odp] [Q] Memory allocation in ODP applications
>
>
>
> ODP APIs are designed to be used *a la carte* by applications, as ODP is
> a framework, not a platform.  So feel free to mix malloc() or your own
> memory management or other API calls in as needed.
>
>
>
> What ODP requires is the types specified in its APIs, so for example the
> only way to get an odp_buffer_t is via the odp_buffer_alloc() call.
>  odp_buffer_alloc() in turn requires an odp_buffer_pool_t and that in turn
> requires an odp_buffer_pool_create() call.
>
>
>
> ODP_BUFFER_TYPE_RAW simply exposes the basic block manager functions of
> the ODP buffer APIs.  Again, you're free to use them for whatever purpose
> the application wants.  Obviously one reason for doing so is to gain
> portability across potentially different memory management implementations.
>
>
>
> On Mon, Nov 3, 2014 at 11:02 AM, Shmulik Ladkani <
> shmulik.ladk...@gmail.com> wrote:
>
> Hi,
>
> As demonstrated by odp/examples, libc memory allocation routines (malloc
> and friends) aren't been used by ODP apps (unless for a very tight
> temporary scope, usually in the control thread).
>
> Application's data structures are allocated either using odp_buffer_pool
> interface (e.g. odp_ipsec.c allocating its private per packet context
> structure using a ODP_BUFFER_TYPE_RAW pool), or directly using
> odp_shm_reserve and managing it internally (e.g. odp_ipsec_fwd_db.c,
> odp_ipsec_sp_db.c etc).
>
> Questions:
>
> Must the application use ODP memory allocation interfaces at all times?
> Or only for data structures accessed at data-path routines?
> Or only when allocating them from within data-path routines?
>
> Any preference for when to use a ODP_BUFFER_TYPE_RAW pool vs. allocating
> a shared memory and managing data structures internally?
>
> Thanks,
> Shmulik
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp
>
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [RFC] Addition to odp_init interface

[Taras Kondratiuk]

On 11/04/2014 10:48 AM, Shmulik Ladkani wrote:

Hi,


I don't have a big picture of what you are trying to achieve, but ODP
is not a framework for thread-local initialization of 3rd party
libraries.


I completely agree.

However my usecase is an exception, as I was trying to create a C++
binding for ODP - hence it isn't just a "different" library which uses
ODP, but rather "wraps" ODP.

In that sense, if for example the C++ binding was decided to be integral
part of ODP, than 'odp_init_local' would simply call 'stl_init_local'
*directly*.

Anyway, I would be glad to elaborate on the motivation in today's call.


Ok, let's discuss this on a call.

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH v3] add tests for queue

[yan.songm...@linaro.org]

Mike,

In fact i never use  checkpatch  before.   I will use it from now on.

This two internal function is not include ,because i just test the base queue 
type.

As you said, i think this two function will be include in the odp_packet_io 
tests  or the deep test in queue later.



yan.songm...@linaro.org
 
From: Mike Holmes
Date: 2014-11-04 01:42
To: Yan Songming
CC: lng-odp
Subject: Re: [lng-odp] [PATCH v3] add tests for queue
http://people.linaro.org/~mike.holmes/linux-generic-gcov-html/linux-generic/index.html

Yan it looks like there  odp_buffer_t queue_sched_buf(odp_queue_t handle) was  
not called in the test suite.

Also the internal void queue_lock(queue_entry_t *queue) was not called but 
maybe odp_packet_io tests may cover that, I believe it is used there.
 
Mike


On 3 November 2014 11:45, Mike Holmes  wrote:
mike@fedora1:~/git/odp$ git am 
~/incoming/lng-odp_PATCH_v3_add_tests_for_queue.mbox
Applying: add tests for queue
/home/mike/git/odp/.git/rebase-apply/patch:81: trailing whitespace.

/home/mike/git/odp/.git/rebase-apply/patch:85: trailing whitespace.

/home/mike/git/odp/.git/rebase-apply/patch:101: trailing whitespace.
CU_ASSERT_EQUAL(queue_creat_id, queue_id); 
/home/mike/git/odp/.git/rebase-apply/patch:152: trailing whitespace.

/home/mike/git/odp/.git/rebase-apply/patch:155: trailing whitespace.

warning: squelched 1 whitespace error
warning: 6 lines add whitespace errors.

If scripts/checkpatch  did not see these problems you need to make a branch or 
new repo and apply the patch so that git am can show you the issues.

On 3 November 2014 11:31, Yan Songming  wrote:
From: "yan.songming" 

Add the cunit test for none sync queue. Test the base queue function.

Signed-off-by: yan.songming 

---
Fix spelling problem and commend problem. Add term_local and term_global.
Move cunit queue test to test_odp_queue_base for none syn queue.
---
 test/cunit/Makefile.am  |   4 +-
 test/cunit/odp_queue_test.c | 168 
 2 files changed, 171 insertions(+), 1 deletion(-)
 create mode 100644 test/cunit/odp_queue_test.c

diff --git a/test/cunit/Makefile.am b/test/cunit/Makefile.am
index 927a5a5..46f46c9 100644
--- a/test/cunit/Makefile.am
+++ b/test/cunit/Makefile.am
@@ -6,8 +6,10 @@ AM_LDFLAGS += -L$(CUNIT_PATH)/lib
 if ODP_CUNIT_ENABLED
 TESTS = ${bin_PROGRAMS}
 check_PROGRAMS = ${bin_PROGRAMS}
-bin_PROGRAMS = odp_init
+bin_PROGRAMS = odp_init odp_queue
 odp_init_LDFLAGS = $(AM_LDFLAGS) -static -lcunit
+odp_queue_LDFLAGS = $(AM_LDFLAGS) -static -lcunit
 endif

+dist_odp_queue_SOURCES = odp_queue_test.c
 dist_odp_init_SOURCES = odp_init_test.c
diff --git a/test/cunit/odp_queue_test.c b/test/cunit/odp_queue_test.c
new file mode 100644
index 000..04342a6
--- /dev/null
+++ b/test/cunit/odp_queue_test.c
@@ -0,0 +1,168 @@
+/* Copyright (c) 2014, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include "odp.h"
+#include "CUnit/Basic.h"
+
+#define MAX_BUFFER_QUEUE(8) /**< Max enqueue buf num */
+#define MSG_POOL_SIZE   (4*1024*1024)   /**< Message pool size */
+
+static int Queue_Contest = 0xff;
+
+static int test_odp_buffer_pool_init(void)
+{
+odp_buffer_pool_t pool;
+void *pool_base;
+odp_shm_t shm;
+
+shm = odp_shm_reserve("msg_pool",
+  MSG_POOL_SIZE, ODP_CACHE_LINE_SIZE, 0);
+
+pool_base = odp_shm_addr(shm);
+
+if (pool_base == NULL) {
+printf("Shared memory reserve failed.\n");
+return -1;
+}
+
+pool = odp_buffer_pool_create("msg_pool", pool_base, MSG_POOL_SIZE,
+  0,
+  ODP_CACHE_LINE_SIZE, ODP_BUFFER_TYPE_RAW);
+
+if (pool == ODP_BUFFER_POOL_INVALID) {
+printf("Pool create failed.\n");
+return -1;
+}
+return 0;
+}
+
+static void test_odp_queue_base(void)
+{
+odp_queue_t   queue_creat_id;
+odp_queue_t   queue_id;
+odp_buffer_t  Enbuf[MAX_BUFFER_QUEUE];
+odp_buffer_t  Debuf[MAX_BUFFER_QUEUE];
+odp_buffer_pool_t msg_pool;
+odp_queue_param_t param;
+
+int  i;
+odp_buffer_t buf;
+void *pRtn = NULL;
+
+/* test odp_queue_create */
+memset(¶m, 0, sizeof(param));
+param.sched.sync  = ODP_SCHED_SYNC_NONE;
+
+queue_creat_id = odp_queue_create("test_queue", ODP_QUEUE_TYPE_POLL, 
¶m);
+CU_ASSERT(ODP_QUEUE_INVALID != queue_creat_id);
+
+/* test odp_queue_type */
+CU_ASSERT_EQUAL(ODP_QUEUE_TYPE_POLL, odp_queue_type(queue_creat_id));
+
+/* test odp_queue_type */
+CU_ASSERT_EQUAL(ODP_SCHED_SYNC_NONE, odp_queue_sched_type(queue_creat_id));
+
+/* test odp_queue_lookup */
+queue_id = odp_queue_lookup("test_queue");
+CU_ASSERT_EQUAL(queue_creat_id, queue_id);
+
+/* test odp_queue_set_context */
+CU_ASSERT(0 == odp_queue_set_context(queue_id, &Queue_Contest));
+
+/* test  odp_queue_get_contex

Re: [lng-odp] [PATCH v3] add tests for queue

[yan.songm...@linaro.org]

Hi jacob,
A) return number of buffers
 The odp_queue_deq_multi do return the number of buffer but the 
odp_queue_enq_multi is not.
  I think odp_queue_enq_multi  is need to return the number of buffers too, 
because there is a limit of eight. 
  When the user  enqueue ten, in fact  just eight buffers will be success , but 
the user don't know that.
B) add multi queue operation 
  I just add the base test of the queue this time , I believe that both multi 
queue operation and multi thread queue operation should be test.
  But i think it should be done  next time later. 



yan.songm...@linaro.org
 
From: Jerin Jacob
Date: 2014-11-04 14:16
To: Yan Songming
CC: lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [PATCH v3] add tests for queue
On Tue, Nov 04, 2014 at 12:31:03AM +0800, Yan Songming wrote:
> From: "yan.songming" 
> 
> Add the cunit test for none sync queue. Test the base queue function.
> 
> Signed-off-by: yan.songming 
> 
> ---
> Fix spelling problem and commend problem. Add term_local and term_global.
> Move cunit queue test to test_odp_queue_base for none syn queue.
> ---
>  test/cunit/Makefile.am  |   4 +-
>  test/cunit/odp_queue_test.c | 168 
> 
>  2 files changed, 171 insertions(+), 1 deletion(-)
>  create mode 100644 test/cunit/odp_queue_test.c
> 
> diff --git a/test/cunit/Makefile.am b/test/cunit/Makefile.am
> index 927a5a5..46f46c9 100644
> --- a/test/cunit/Makefile.am
> +++ b/test/cunit/Makefile.am
> @@ -6,8 +6,10 @@ AM_LDFLAGS += -L$(CUNIT_PATH)/lib
>  if ODP_CUNIT_ENABLED
>  TESTS = ${bin_PROGRAMS}
>  check_PROGRAMS = ${bin_PROGRAMS}
> -bin_PROGRAMS = odp_init
> +bin_PROGRAMS = odp_init odp_queue
>  odp_init_LDFLAGS = $(AM_LDFLAGS) -static -lcunit
> +odp_queue_LDFLAGS = $(AM_LDFLAGS) -static -lcunit
>  endif
>  
> +dist_odp_queue_SOURCES = odp_queue_test.c
>  dist_odp_init_SOURCES = odp_init_test.c
> diff --git a/test/cunit/odp_queue_test.c b/test/cunit/odp_queue_test.c
> new file mode 100644
> index 000..04342a6
> --- /dev/null
> +++ b/test/cunit/odp_queue_test.c
> @@ -0,0 +1,168 @@
> +/* Copyright (c) 2014, Linaro Limited
> + * All rights reserved.
> + *
> + * SPDX-License-Identifier: BSD-3-Clause
> + */
> +
> +#include "odp.h"
> +#include "CUnit/Basic.h"
> +
> +#define MAX_BUFFER_QUEUE(8) /**< Max enqueue buf num */
> +#define MSG_POOL_SIZE   (4*1024*1024)   /**< Message pool size */
> +
> +static int Queue_Contest = 0xff;
> +
> +static int test_odp_buffer_pool_init(void)
> +{
> +odp_buffer_pool_t pool;
> +void *pool_base;
> +odp_shm_t shm;
> +
> +shm = odp_shm_reserve("msg_pool",
> +  MSG_POOL_SIZE, ODP_CACHE_LINE_SIZE, 0);
> +
> +pool_base = odp_shm_addr(shm);
> +
> +if (pool_base == NULL) {
> +printf("Shared memory reserve failed.\n");
> +return -1;
> +}
> +
> +pool = odp_buffer_pool_create("msg_pool", pool_base, MSG_POOL_SIZE,
> +  0,
> +  ODP_CACHE_LINE_SIZE, ODP_BUFFER_TYPE_RAW);
> +
> +if (pool == ODP_BUFFER_POOL_INVALID) {
> +printf("Pool create failed.\n");
> +return -1;
> +}
> +return 0;
> +}
> +
> +static void test_odp_queue_base(void)
> +{
> +odp_queue_t   queue_creat_id;
> +odp_queue_t   queue_id;
> +odp_buffer_t  Enbuf[MAX_BUFFER_QUEUE];
> +odp_buffer_t  Debuf[MAX_BUFFER_QUEUE];
> +odp_buffer_pool_t msg_pool;
> +odp_queue_param_t param;
> +
> +int  i;
> +odp_buffer_t buf;
> +void *pRtn = NULL;
> +
> +/* test odp_queue_create */
> +memset(¶m, 0, sizeof(param));
> +param.sched.sync  = ODP_SCHED_SYNC_NONE;
> +
> +queue_creat_id = odp_queue_create("test_queue", ODP_QUEUE_TYPE_POLL, 
> ¶m);
> +CU_ASSERT(ODP_QUEUE_INVALID != queue_creat_id);
> +
> +/* test odp_queue_type */
> +CU_ASSERT_EQUAL(ODP_QUEUE_TYPE_POLL, odp_queue_type(queue_creat_id));
> +
> +/* test odp_queue_type */
> +CU_ASSERT_EQUAL(ODP_SCHED_SYNC_NONE, 
> odp_queue_sched_type(queue_creat_id));
> +
> +/* test odp_queue_lookup */
> +queue_id = odp_queue_lookup("test_queue");
> +CU_ASSERT_EQUAL(queue_creat_id, queue_id); 
> +
> +/* test odp_queue_set_context */
> +CU_ASSERT(0 == odp_queue_set_context(queue_id, &Queue_Contest));
> +
> +/* test  odp_queue_get_context*/
> +pRtn = odp_queue_get_context(queue_id);
> +CU_ASSERT(&Queue_Contest == (int *)pRtn);
> +
> +/* apply for buffer */
> +msg_pool = odp_buffer_pool_lookup("msg_pool");
> +buf = odp_buffer_alloc(msg_pool);
> +
> +/* test  odp_queue_enq and odp_queue_deq */
> +odp_queue_enq(queue_id, buf);
> +CU_ASSERT_EQUAL(buf, odp_queue_deq(queue_id));
> +odp_buffer_free(buf);
> +
> +/* apply for mutili buffer */
> +for(i=0; i +Enbuf[i] = odp_buffer_alloc(msg_pool);
> +}
> +
> +/* test odp_queue_enq_multi  and odp_queue_enq_multi */

Re: [lng-odp] [PATCH v3] add tests for queue

[Jerin Jacob]

On Tue, Nov 04, 2014 at 07:02:47PM +0800, yan.songm...@linaro.org wrote:
> Hi jacob,
> A) return number of buffers
>  The odp_queue_deq_multi do return the number of buffer but the 
> odp_queue_enq_multi is not.

Correct. But test case should honor the 
return value(number of buffers dequeued) from odp_queue_deq_multi.

>   I think odp_queue_enq_multi  is need to return the number of buffers too, 
> because there is a limit of eight. 
>   When the user  enqueue ten, in fact  just eight buffers will be success , 
> but the user don't know that.

Not just that, a platfrom may have restriction on number buffers can be enqued 
at given point of time 
based on the resource avilablity.


> B) add multi queue operation 
>   I just add the base test of the queue this time , I believe that both multi 
> queue operation and multi thread queue operation should be test.
>   But i think it should be done  next time later. 

Agree. We should include odp_queue_enq_multi in testcase when linux generic has 
support for
multi enqueue operataion which returns the number of enqueue buffers.

> 
> 
> 
> yan.songm...@linaro.org
>  
> From: Jerin Jacob
> Date: 2014-11-04 14:16
> To: Yan Songming
> CC: lng-odp@lists.linaro.org
> Subject: Re: [lng-odp] [PATCH v3] add tests for queue
> On Tue, Nov 04, 2014 at 12:31:03AM +0800, Yan Songming wrote:
> > From: "yan.songming" 
> > 
> > Add the cunit test for none sync queue. Test the base queue function.
> > 
> > Signed-off-by: yan.songming 
> > 
> > ---
> > Fix spelling problem and commend problem. Add term_local and term_global.
> > Move cunit queue test to test_odp_queue_base for none syn queue.
> > ---
> >  test/cunit/Makefile.am  |   4 +-
> >  test/cunit/odp_queue_test.c | 168 
> > 
> >  2 files changed, 171 insertions(+), 1 deletion(-)
> >  create mode 100644 test/cunit/odp_queue_test.c
> > 
> > diff --git a/test/cunit/Makefile.am b/test/cunit/Makefile.am
> > index 927a5a5..46f46c9 100644
> > --- a/test/cunit/Makefile.am
> > +++ b/test/cunit/Makefile.am
> > @@ -6,8 +6,10 @@ AM_LDFLAGS += -L$(CUNIT_PATH)/lib
> >  if ODP_CUNIT_ENABLED
> >  TESTS = ${bin_PROGRAMS}
> >  check_PROGRAMS = ${bin_PROGRAMS}
> > -bin_PROGRAMS = odp_init
> > +bin_PROGRAMS = odp_init odp_queue
> >  odp_init_LDFLAGS = $(AM_LDFLAGS) -static -lcunit
> > +odp_queue_LDFLAGS = $(AM_LDFLAGS) -static -lcunit
> >  endif
> >  
> > +dist_odp_queue_SOURCES = odp_queue_test.c
> >  dist_odp_init_SOURCES = odp_init_test.c
> > diff --git a/test/cunit/odp_queue_test.c b/test/cunit/odp_queue_test.c
> > new file mode 100644
> > index 000..04342a6
> > --- /dev/null
> > +++ b/test/cunit/odp_queue_test.c
> > @@ -0,0 +1,168 @@
> > +/* Copyright (c) 2014, Linaro Limited
> > + * All rights reserved.
> > + *
> > + * SPDX-License-Identifier: BSD-3-Clause
> > + */
> > +
> > +#include "odp.h"
> > +#include "CUnit/Basic.h"
> > +
> > +#define MAX_BUFFER_QUEUE(8) /**< Max enqueue buf num */
> > +#define MSG_POOL_SIZE   (4*1024*1024)   /**< Message pool size */
> > +
> > +static int Queue_Contest = 0xff;
> > +
> > +static int test_odp_buffer_pool_init(void)
> > +{
> > +odp_buffer_pool_t pool;
> > +void *pool_base;
> > +odp_shm_t shm;
> > +
> > +shm = odp_shm_reserve("msg_pool",
> > +  MSG_POOL_SIZE, ODP_CACHE_LINE_SIZE, 0);
> > +
> > +pool_base = odp_shm_addr(shm);
> > +
> > +if (pool_base == NULL) {
> > +printf("Shared memory reserve failed.\n");
> > +return -1;
> > +}
> > +
> > +pool = odp_buffer_pool_create("msg_pool", pool_base, MSG_POOL_SIZE,
> > +  0,
> > +  ODP_CACHE_LINE_SIZE, 
> > ODP_BUFFER_TYPE_RAW);
> > +
> > +if (pool == ODP_BUFFER_POOL_INVALID) {
> > +printf("Pool create failed.\n");
> > +return -1;
> > +}
> > +return 0;
> > +}
> > +
> > +static void test_odp_queue_base(void)
> > +{
> > +odp_queue_t   queue_creat_id;
> > +odp_queue_t   queue_id;
> > +odp_buffer_t  Enbuf[MAX_BUFFER_QUEUE];
> > +odp_buffer_t  Debuf[MAX_BUFFER_QUEUE];
> > +odp_buffer_pool_t msg_pool;
> > +odp_queue_param_t param;
> > +
> > +int  i;
> > +odp_buffer_t buf;
> > +void *pRtn = NULL;
> > +
> > +/* test odp_queue_create */
> > +memset(¶m, 0, sizeof(param));
> > +param.sched.sync  = ODP_SCHED_SYNC_NONE;
> > +
> > +queue_creat_id = odp_queue_create("test_queue", ODP_QUEUE_TYPE_POLL, 
> > ¶m);
> > +CU_ASSERT(ODP_QUEUE_INVALID != queue_creat_id);
> > +
> > +/* test odp_queue_type */
> > +CU_ASSERT_EQUAL(ODP_QUEUE_TYPE_POLL, odp_queue_type(queue_creat_id));
> > +
> > +/* test odp_queue_type */
> > +CU_ASSERT_EQUAL(ODP_SCHED_SYNC_NONE, 
> > odp_queue_sched_type(queue_creat_id));
> > +
> > +/* test odp_queue_lookup */
> > +queue_id = odp_queue_lookup("test_queue");
> > +CU_ASSERT_EQUAL(queue_crea

Re: [lng-odp] [Q] Memory allocation in ODP applications

[Savolainen, Petri (NSN - FI/Espoo)]

Hi,

As noted many times before, “optional” is next to non-existent for application 
programmers – especially if it’s important to achieve  application portability. 
I’d suggest that API v1.0 would not have anything labeled as optional. It’s 
simplifies the API for everybody (implementers/testers/users). Features labeled 
as optional should be either turned into mandatory or dropped out.

-Petri


From: lng-odp-boun...@lists.linaro.org 
[mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of ext Bill Fischofer
Sent: Monday, November 03, 2014 10:35 PM
To: Shmulik Ladkani
Cc: lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [Q] Memory allocation in ODP applications

ODP APIs are divided into two classes: Mandatory, which all conforming ODP 
implementations are expected to provide, and Optional.  The convention is that 
Optional APIs must be present but calling them may simply result in an 
ODP_FUNCTION_NOT_AVAILABLE return code.  These are documented in the associated 
header files (currently under revision) and are part of the doxygen.  
Basically, unless an API calls itself out as OPTIONAL you should assume that 
all ODP implementations will provide functional implementations of it, though 
performance of individual APIs may vary between platforms and within platforms 
over time.

It's expected that each platform will provide implementations of the ODP APIs 
that are optimized for that particular platform.  ODP itself does not specify 
how implementations do that or how they might improve performance over time 
with subsequent releases of the implementation.  This was the reason for 
decoupling the APIs from the implementations and supporting multiple 
repositories.  It's expected that, over time, implementations will evolve and 
improve their performance as the implementations become better tuned to 
refinements in their underlying platform.

Hope that helps.

Bill


On Mon, Nov 3, 2014 at 2:18 PM, Shmulik Ladkani 
mailto:shmulik.ladk...@gmail.com>> wrote:
On Mon, 3 Nov 2014 12:11:26 -0600 Bill Fischofer 
mailto:bill.fischo...@linaro.org>> wrote:
> ODP APIs are designed to be used *a la carte* by applications, as ODP is a
> framework, not a platform.  So feel free to mix malloc() or your own memory
> management or other API calls in as needed.
>
> What ODP requires is the types specified in its APIs, so for example the
> only way to get an odp_buffer_t is via the odp_buffer_alloc() call.
>  odp_buffer_alloc() in turn requires an odp_buffer_pool_t and that in turn
> requires an odp_buffer_pool_create() call.
>
> ODP_BUFFER_TYPE_RAW simply exposes the basic block manager functions of the
> ODP buffer APIs.  Again, you're free to use them for whatever purpose the
> application wants.  Obviously one reason for doing so is to gain
> portability across potentially different memory management implementations.

Thanks Bill.

This leads me to few additional questions:

Are all memory-related ODP APIs mandatory (must be implemented by the
platform)?

Are they required to provide any other benefit over standard (or custom)
allocation routines besides the portability guarantee?

Regards,
Shmulik

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH v2 00/18] Split out platform-specific values from header files

[Ciprian Barbu]

On Mon, Nov 3, 2014 at 5:47 PM, Taras Kondratiuk
 wrote:
> A need to split out a clean API headers is being discussed again, so
> pulling this thread up to remind what the previous attempt was.
>
> On 07/08/2014 05:55 PM, Mike Holmes wrote:
>> We discussed this on a hangout, here are the results as a way forward on
>> this 1 month old topic.
>>
>>
>> Attendees:
>> Bala
>> Taras
>> Maxim
>> Santosh
>> Marshall
>> Mike
>> Bill
>>
>> Actions required:
>>
>> Step one:
>> Move odp/include to odp/platform/linux-generic/include, update the make
>> files to suit.
>>
>> Step two:
>> Pull the doxygen documentation back up to odp/doc/odp_xxx.dox out of the
>> header files.
>>
>> General Rational:
>>
>>  1. Anecdotal evidence from previous projects suggests we will end up
>> with per platform includes anyway

Is this because of the need to inline and to have per-platform defines
/ typedefs ?

>>  2. We are trying to accommodate splitting out the platform specifics on
>> this thread, that is what has driven this issue so No.1 looks correct
>>  3. Linux-generic is the reference implementation and by default it is
>> what is built now so there is no change in observed bechavior. This
>> includes the fact that the platform specifics show though, all that
>> changes is that is it now clear that they are specific to that
>> platform and no pretense that the default doc is is platform
>> independent.
>>  4. We reduce the directory structure complexity, no need to include
>> odp/include for each platforms make file.
>>  5. Other platforms already include Linux-generic to reuse its code so
>> we are not adding any new paths to find the same headers for those
>> cases, there is no impact if the Linux-generic is not needed.
>>  6. The API documentation is still common, it can be stored once in
>> odp/doc/odpx.dox, doxygen will tie this together with the
>> headers found per platform.
>>  7. Platforms are still free to add pages to augment the documentation
>> with platform specifics.
>>
>> Negatives:
>>
>>  1. Documentation is no longer all kept right next to the definitions
>> which is one reason to use doxygen in the first place.

>From my point of view, Doxygen is primarily helpful for having a nice
rendered view of the documentation in html/pdf format. Having the
documentation in the same place as the headers (whether it contains
Doxygen tags or not) makes for a more comfortable view for the
programmer who may not be always looking at the browser (those
hardcore Linux guys :) So I think we should have separate dox file.

>>  2. There is less enforcement of the API across platforms, hopefully
>> mitigated by the ODP-validation test suite.
>>
>>
>>
>> On 3 July 2014 13:14, Mike Holmes > > wrote:
>>
>> Let me take a look, I will ping you offline to make sure I
>> understand and replicate the issue correctly
>>
>>
>> On 3 July 2014 06:26, Taras Kondratiuk > > wrote:
>>
>> On 06/26/2014 06:24 PM, Taras Kondratiuk wrote:
>>
>> As we have discussed during a call, I've tried to implement
>> option #2
>> for several files, but looks like Doxygen is not happy if
>> documented
>> variable can't be found in its input files. I couldn't find
>> a Doxygen
>> option that controls this behavior.
>> Mike, do you have some ideas how to workaround it?
>>
>>
>> Mike, do you have some hints?
>>
>>
>>
>>
>> --
>> *Mike Holmes*
>> Linaro Technical Manager / Lead
>> LNG - ODP
>>
>>
>>
>>
>> --
>> *Mike Holmes*
>> Linaro Technical Manager / Lead
>> LNG - ODP
>
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [Q] Memory allocation in ODP applications

[Bill Fischofer]

I think the criteria I outlined above for considering an API optional are
reasonable.  Recall that the reason certain APIs are designated as optional
is because we received feedback that they were not feasible on specific
platforms.  It's RECOMMENDED that platforms provide all of these APIs,
however I'm not eager to say platform X can't have a conforming ODP
implementation if it's just not feasible to implement every ODP API or
feature on that platform.

Applications will be written to whatever APIs and features they need.  Not
every application needs every API and feature and if an application needs
an optional API then when it chooses which platforms it wants to deploy on
it will take whether that platform provides that API into account in making
its decision.

These are fundamentally product marketing issues, not technical issues.  If
we take a one-size-fits-all approach that will unnecessarily limit the
ecosystem.

On Tue, Nov 4, 2014 at 6:43 AM, Savolainen, Petri (NSN - FI/Espoo) <
petri.savolai...@nsn.com> wrote:

>  Hi,
>
>
>
> As noted many times before, “optional” is next to non-existent for
> application programmers – especially if it’s important to achieve
>  application portability. I’d suggest that API v1.0 would not have anything
> labeled as optional. It’s simplifies the API for everybody
> (implementers/testers/users). Features labeled as optional should be either
> turned into mandatory or dropped out.
>
>
>
> -Petri
>
>
>
>
>
> *From:* lng-odp-boun...@lists.linaro.org [mailto:
> lng-odp-boun...@lists.linaro.org] *On Behalf Of *ext Bill Fischofer
> *Sent:* Monday, November 03, 2014 10:35 PM
> *To:* Shmulik Ladkani
> *Cc:* lng-odp@lists.linaro.org
> *Subject:* Re: [lng-odp] [Q] Memory allocation in ODP applications
>
>
>
> ODP APIs are divided into two classes: Mandatory, which all conforming ODP
> implementations are expected to provide, and Optional.  The convention is
> that Optional APIs must be present but calling them may simply result in an
> ODP_FUNCTION_NOT_AVAILABLE return code.  These are documented in the
> associated header files (currently under revision) and are part of the
> doxygen.  Basically, unless an API calls itself out as OPTIONAL you should
> assume that all ODP implementations will provide functional implementations
> of it, though performance of individual APIs may vary between platforms and
> within platforms over time.
>
>
>
> It's expected that each platform will provide implementations of the ODP
> APIs that are optimized for that particular platform.  ODP itself does not
> specify how implementations do that or how they might improve performance
> over time with subsequent releases of the implementation.  This was the
> reason for decoupling the APIs from the implementations and supporting
> multiple repositories.  It's expected that, over time, implementations will
> evolve and improve their performance as the implementations become better
> tuned to refinements in their underlying platform.
>
>
>
> Hope that helps.
>
>
>
> Bill
>
>
>
>
>
> On Mon, Nov 3, 2014 at 2:18 PM, Shmulik Ladkani 
> wrote:
>
> On Mon, 3 Nov 2014 12:11:26 -0600 Bill Fischofer <
> bill.fischo...@linaro.org> wrote:
> > ODP APIs are designed to be used *a la carte* by applications, as ODP is
> a
> > framework, not a platform.  So feel free to mix malloc() or your own
> memory
> > management or other API calls in as needed.
> >
> > What ODP requires is the types specified in its APIs, so for example the
> > only way to get an odp_buffer_t is via the odp_buffer_alloc() call.
> >  odp_buffer_alloc() in turn requires an odp_buffer_pool_t and that in
> turn
> > requires an odp_buffer_pool_create() call.
> >
> > ODP_BUFFER_TYPE_RAW simply exposes the basic block manager functions of
> the
> > ODP buffer APIs.  Again, you're free to use them for whatever purpose the
> > application wants.  Obviously one reason for doing so is to gain
> > portability across potentially different memory management
> implementations.
>
> Thanks Bill.
>
> This leads me to few additional questions:
>
> Are all memory-related ODP APIs mandatory (must be implemented by the
> platform)?
>
> Are they required to provide any other benefit over standard (or custom)
> allocation routines besides the portability guarantee?
>
> Regards,
> Shmulik
>
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [Q] Memory allocation in ODP applications

[Savolainen, Petri (NSN - FI/Espoo)]

I’m concerned that the API gets too fragmented already in v1.0 and the 
ecosystem will be limited by that. For example, it’s hard to integrate 3rd 
party (or open source) SW into a system, if my app uses optional feature X, the 
3rd party lib uses optional feature Y and the implementation provides only one 
of those, or neither.

-Petri


From: ext Bill Fischofer [mailto:bill.fischo...@linaro.org]
Sent: Tuesday, November 04, 2014 3:16 PM
To: Savolainen, Petri (NSN - FI/Espoo)
Cc: Shmulik Ladkani; lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [Q] Memory allocation in ODP applications

I think the criteria I outlined above for considering an API optional are 
reasonable.  Recall that the reason certain APIs are designated as optional is 
because we received feedback that they were not feasible on specific platforms. 
 It's RECOMMENDED that platforms provide all of these APIs, however I'm not 
eager to say platform X can't have a conforming ODP implementation if it's just 
not feasible to implement every ODP API or feature on that platform.

Applications will be written to whatever APIs and features they need.  Not 
every application needs every API and feature and if an application needs an 
optional API then when it chooses which platforms it wants to deploy on it will 
take whether that platform provides that API into account in making its 
decision.

These are fundamentally product marketing issues, not technical issues.  If we 
take a one-size-fits-all approach that will unnecessarily limit the ecosystem.

On Tue, Nov 4, 2014 at 6:43 AM, Savolainen, Petri (NSN - FI/Espoo) 
mailto:petri.savolai...@nsn.com>> wrote:
Hi,

As noted many times before, “optional” is next to non-existent for application 
programmers – especially if it’s important to achieve  application portability. 
I’d suggest that API v1.0 would not have anything labeled as optional. It’s 
simplifies the API for everybody (implementers/testers/users). Features labeled 
as optional should be either turned into mandatory or dropped out.

-Petri


From: lng-odp-boun...@lists.linaro.org 
[mailto:lng-odp-boun...@lists.linaro.org]
 On Behalf Of ext Bill Fischofer
Sent: Monday, November 03, 2014 10:35 PM
To: Shmulik Ladkani
Cc: lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [Q] Memory allocation in ODP applications

ODP APIs are divided into two classes: Mandatory, which all conforming ODP 
implementations are expected to provide, and Optional.  The convention is that 
Optional APIs must be present but calling them may simply result in an 
ODP_FUNCTION_NOT_AVAILABLE return code.  These are documented in the associated 
header files (currently under revision) and are part of the doxygen.  
Basically, unless an API calls itself out as OPTIONAL you should assume that 
all ODP implementations will provide functional implementations of it, though 
performance of individual APIs may vary between platforms and within platforms 
over time.

It's expected that each platform will provide implementations of the ODP APIs 
that are optimized for that particular platform.  ODP itself does not specify 
how implementations do that or how they might improve performance over time 
with subsequent releases of the implementation.  This was the reason for 
decoupling the APIs from the implementations and supporting multiple 
repositories.  It's expected that, over time, implementations will evolve and 
improve their performance as the implementations become better tuned to 
refinements in their underlying platform.

Hope that helps.

Bill


On Mon, Nov 3, 2014 at 2:18 PM, Shmulik Ladkani 
mailto:shmulik.ladk...@gmail.com>> wrote:
On Mon, 3 Nov 2014 12:11:26 -0600 Bill Fischofer 
mailto:bill.fischo...@linaro.org>> wrote:
> ODP APIs are designed to be used *a la carte* by applications, as ODP is a
> framework, not a platform.  So feel free to mix malloc() or your own memory
> management or other API calls in as needed.
>
> What ODP requires is the types specified in its APIs, so for example the
> only way to get an odp_buffer_t is via the odp_buffer_alloc() call.
>  odp_buffer_alloc() in turn requires an odp_buffer_pool_t and that in turn
> requires an odp_buffer_pool_create() call.
>
> ODP_BUFFER_TYPE_RAW simply exposes the basic block manager functions of the
> ODP buffer APIs.  Again, you're free to use them for whatever purpose the
> application wants.  Obviously one reason for doing so is to gain
> portability across potentially different memory management implementations.

Thanks Bill.

This leads me to few additional questions:

Are all memory-related ODP APIs mandatory (must be implemented by the
platform)?

Are they required to provide any other benefit over standard (or custom)
allocation routines besides the portability guarantee?

Regards,
Shmulik


___
lng-odp mailing list
lng-odp@lists.linaro.org
ht

Re: [lng-odp] [Q] Memory allocation in ODP applications

[Bill Fischofer]

That's precisely why we've said that all implementations must provide at
least stubs for all of the ODP APIs (even those marked optional).  Not
every ODP application will be able to run on every ODP implementation, but
that's OK.  You don't need your home office router to have enterprise-level
features.  Requiring every ODP implementation to provide identical
functionality will either force us to adopt a least-common-denominator
approach to what are acceptable APIs or else we'll severely limit the range
of platforms that can participate in the ecosystem.

Do you have a specific list of APIs that are currently marked optional that
you'd like to see promoted to mandatory?


On Tue, Nov 4, 2014 at 7:26 AM, Savolainen, Petri (NSN - FI/Espoo) <
petri.savolai...@nsn.com> wrote:

>  I’m concerned that the API gets too fragmented already in v1.0 and the
> ecosystem will be limited by that. For example, it’s hard to integrate 3rd
> party (or open source) SW into a system, if my app uses optional feature X,
> the 3rd party lib uses optional feature Y and the implementation provides
> only one of those, or neither.
>
>
>
> -Petri
>
>
>
>
>
> *From:* ext Bill Fischofer [mailto:bill.fischo...@linaro.org]
> *Sent:* Tuesday, November 04, 2014 3:16 PM
> *To:* Savolainen, Petri (NSN - FI/Espoo)
> *Cc:* Shmulik Ladkani; lng-odp@lists.linaro.org
>
> *Subject:* Re: [lng-odp] [Q] Memory allocation in ODP applications
>
>
>
> I think the criteria I outlined above for considering an API optional are
> reasonable.  Recall that the reason certain APIs are designated as optional
> is because we received feedback that they were not feasible on specific
> platforms.  It's RECOMMENDED that platforms provide all of these APIs,
> however I'm not eager to say platform X can't have a conforming ODP
> implementation if it's just not feasible to implement every ODP API or
> feature on that platform.
>
>
>
> Applications will be written to whatever APIs and features they need.  Not
> every application needs every API and feature and if an application needs
> an optional API then when it chooses which platforms it wants to deploy on
> it will take whether that platform provides that API into account in making
> its decision.
>
>
>
> These are fundamentally product marketing issues, not technical issues.
> If we take a one-size-fits-all approach that will unnecessarily limit the
> ecosystem.
>
>
>
> On Tue, Nov 4, 2014 at 6:43 AM, Savolainen, Petri (NSN - FI/Espoo) <
> petri.savolai...@nsn.com> wrote:
>
> Hi,
>
>
>
> As noted many times before, “optional” is next to non-existent for
> application programmers – especially if it’s important to achieve
>  application portability. I’d suggest that API v1.0 would not have anything
> labeled as optional. It’s simplifies the API for everybody
> (implementers/testers/users). Features labeled as optional should be either
> turned into mandatory or dropped out.
>
>
>
> -Petri
>
>
>
>
>
> *From:* lng-odp-boun...@lists.linaro.org [mailto:
> lng-odp-boun...@lists.linaro.org] *On Behalf Of *ext Bill Fischofer
> *Sent:* Monday, November 03, 2014 10:35 PM
> *To:* Shmulik Ladkani
> *Cc:* lng-odp@lists.linaro.org
> *Subject:* Re: [lng-odp] [Q] Memory allocation in ODP applications
>
>
>
> ODP APIs are divided into two classes: Mandatory, which all conforming ODP
> implementations are expected to provide, and Optional.  The convention is
> that Optional APIs must be present but calling them may simply result in an
> ODP_FUNCTION_NOT_AVAILABLE return code.  These are documented in the
> associated header files (currently under revision) and are part of the
> doxygen.  Basically, unless an API calls itself out as OPTIONAL you should
> assume that all ODP implementations will provide functional implementations
> of it, though performance of individual APIs may vary between platforms and
> within platforms over time.
>
>
>
> It's expected that each platform will provide implementations of the ODP
> APIs that are optimized for that particular platform.  ODP itself does not
> specify how implementations do that or how they might improve performance
> over time with subsequent releases of the implementation.  This was the
> reason for decoupling the APIs from the implementations and supporting
> multiple repositories.  It's expected that, over time, implementations will
> evolve and improve their performance as the implementations become better
> tuned to refinements in their underlying platform.
>
>
>
> Hope that helps.
>
>
>
> Bill
>
>
>
>
>
> On Mon, Nov 3, 2014 at 2:18 PM, Shmulik Ladkani 
> wrote:
>
> On Mon, 3 Nov 2014 12:11:26 -0600 Bill Fischofer <
> bill.fischo...@linaro.org> wrote:
> > ODP APIs are designed to be used *a la carte* by applications, as ODP is
> a
> > framework, not a platform.  So feel free to mix malloc() or your own
> memory
> > management or other API calls in as needed.
> >
> > What ODP requires is the types specified in its APIs, so for example the
> > only way to get an odp_buffer_t is vi

Re: [lng-odp] [PATCH v2 00/18] Split out platform-specific values from header files

[Bill Fischofer]

Please take a look at the RFC patches I circulated for odp_buffer.h,
odp_buffer_pool.h, and odp_packet.h.  These now serve as both executables
and doxygen source so the information is in one place.

The split discussion refers to where the platform-specific typedefs are
defined + the question of how inlined implementations of select APIs are
expressed.

On Tue, Nov 4, 2014 at 7:15 AM, Ciprian Barbu 
wrote:

> On Mon, Nov 3, 2014 at 5:47 PM, Taras Kondratiuk
>  wrote:
> > A need to split out a clean API headers is being discussed again, so
> > pulling this thread up to remind what the previous attempt was.
> >
> > On 07/08/2014 05:55 PM, Mike Holmes wrote:
> >> We discussed this on a hangout, here are the results as a way forward on
> >> this 1 month old topic.
> >>
> >>
> >> Attendees:
> >> Bala
> >> Taras
> >> Maxim
> >> Santosh
> >> Marshall
> >> Mike
> >> Bill
> >>
> >> Actions required:
> >>
> >> Step one:
> >> Move odp/include to odp/platform/linux-generic/include, update the make
> >> files to suit.
> >>
> >> Step two:
> >> Pull the doxygen documentation back up to odp/doc/odp_xxx.dox out of the
> >> header files.
> >>
> >> General Rational:
> >>
> >>  1. Anecdotal evidence from previous projects suggests we will end up
> >> with per platform includes anyway
>
> Is this because of the need to inline and to have per-platform defines
> / typedefs ?
>
> >>  2. We are trying to accommodate splitting out the platform specifics on
> >> this thread, that is what has driven this issue so No.1 looks
> correct
> >>  3. Linux-generic is the reference implementation and by default it is
> >> what is built now so there is no change in observed bechavior. This
> >> includes the fact that the platform specifics show though, all that
> >> changes is that is it now clear that they are specific to that
> >> platform and no pretense that the default doc is is platform
> >> independent.
> >>  4. We reduce the directory structure complexity, no need to include
> >> odp/include for each platforms make file.
> >>  5. Other platforms already include Linux-generic to reuse its code so
> >> we are not adding any new paths to find the same headers for those
> >> cases, there is no impact if the Linux-generic is not needed.
> >>  6. The API documentation is still common, it can be stored once in
> >> odp/doc/odpx.dox, doxygen will tie this together with the
> >> headers found per platform.
> >>  7. Platforms are still free to add pages to augment the documentation
> >> with platform specifics.
> >>
> >> Negatives:
> >>
> >>  1. Documentation is no longer all kept right next to the definitions
> >> which is one reason to use doxygen in the first place.
>
> From my point of view, Doxygen is primarily helpful for having a nice
> rendered view of the documentation in html/pdf format. Having the
> documentation in the same place as the headers (whether it contains
> Doxygen tags or not) makes for a more comfortable view for the
> programmer who may not be always looking at the browser (those
> hardcore Linux guys :) So I think we should have separate dox file.
>
> >>  2. There is less enforcement of the API across platforms, hopefully
> >> mitigated by the ODP-validation test suite.
> >>
> >>
> >>
> >> On 3 July 2014 13:14, Mike Holmes  >> > wrote:
> >>
> >> Let me take a look, I will ping you offline to make sure I
> >> understand and replicate the issue correctly
> >>
> >>
> >> On 3 July 2014 06:26, Taras Kondratiuk  >> > wrote:
> >>
> >> On 06/26/2014 06:24 PM, Taras Kondratiuk wrote:
> >>
> >> As we have discussed during a call, I've tried to implement
> >> option #2
> >> for several files, but looks like Doxygen is not happy if
> >> documented
> >> variable can't be found in its input files. I couldn't find
> >> a Doxygen
> >> option that controls this behavior.
> >> Mike, do you have some ideas how to workaround it?
> >>
> >>
> >> Mike, do you have some hints?
> >>
> >>
> >>
> >>
> >> --
> >> *Mike Holmes*
> >> Linaro Technical Manager / Lead
> >> LNG - ODP
> >>
> >>
> >>
> >>
> >> --
> >> *Mike Holmes*
> >> Linaro Technical Manager / Lead
> >> LNG - ODP
> >
> >
> > ___
> > lng-odp mailing list
> > lng-odp@lists.linaro.org
> > http://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions

[Alexandru Badicioiu]

In addition to skip order function I think we need to handle the case when
a packet gets fragmented (e.g. IP fragmentation). In this case the
fragments should share the original sequence number and the order
restoration logic should be aware when the last fragment occurs.

Alex

On 3 November 2014 13:45, Bill Fischofer  wrote:

> OK, thanks.  That certainly matches the step-by-step semantics Petri
> proposed.  We'll have to discuss what, if anything, more we need regarding
> ORDERED queues post-v1.0 next year.
>
> Bill
>
> On Mon, Nov 3, 2014 at 3:03 AM, Gilad Ben Yossef 
> wrote:
>
>>
>>
>>
>>
>> What NPS does in HW is that each buffer you dequeue actually stays on the
>> queue (but is marked as consumed by the application) and you are given a
>> "tag" for it.
>>
>>
>>
>> When I want to queue the buffer to the next, I give the HW the tag I got,
>> not the buffer handle and what the HW does is only than fully dequeue the
>> buffer from the original queue before queuing it into the new queue – but
>> only after all preceding (older) buffers on the original queue have been
>> fully dequeued,
>>
>>
>>
>> Free of a buffer causes a dequeue from the original queue as well.
>>
>>
>>
>> And I can also give release a buffer from the original queue (give the HW
>> the tag back and say "you don't have to wait for this buffer any longer).
>>
>>
>>
>> I hope this helps,
>>
>> Gilad
>>
>>
>>
>> *Gilad Ben-Yossef*
>>
>> Software Architect
>>
>> EZchip Technologies Ltd.
>>
>> 37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel
>>
>> Tel: +972-4-959- ext. 576, Fax: +972-8-681-1483
>> Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388
>>
>> Email: gil...@ezchip.com, Web: http://www.ezchip.com
>>
>>
>>
>> *"Ethernet always wins."*
>>
>> — Andy Bechtolsheim
>>
>>
>>
>> *From:* Bill Fischofer [mailto:bill.fischo...@linaro.org]
>> *Sent:* Sunday, November 02, 2014 5:44 PM
>> *To:* Gilad Ben Yossef
>> *Cc:* Savolainen, Petri (NSN - FI/Espoo); lng-odp@lists.linaro.org
>>
>> *Subject:* Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions
>>
>>
>>
>> Thanks Gilad.  Can you elaborate on that a bit more?  I can understand
>> how step-by-step would be potentially easier to implement, but does it also
>> capture the majority of expected application use cases?
>>
>>
>>
>> A good confirmation that this is the correct approach for v1.0 though.
>>
>>
>>
>> On Sun, Nov 2, 2014 at 9:26 AM, Gilad Ben Yossef 
>> wrote:
>>
>>
>>
>> For what it's worth, as a SoC vendor rep. that has ordered queue in HW,
>> Petri's definition is actually preferred for us even going forward J
>>
>>
>>
>>
>>
>> Thanks,
>>
>> Gilad
>>
>>
>>
>> *Gilad Ben-Yossef*
>>
>> Software Architect
>>
>> EZchip Technologies Ltd.
>>
>> 37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel
>>
>> Tel: +972-4-959- ext. 576, Fax: +972-8-681-1483
>> Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388
>>
>> Email: gil...@ezchip.com, Web: http://www.ezchip.com
>>
>>
>>
>> *"Ethernet always wins."*
>>
>> — Andy Bechtolsheim
>>
>>
>>
>> *From:* lng-odp-boun...@lists.linaro.org [mailto:
>> lng-odp-boun...@lists.linaro.org] *On Behalf Of *Bill Fischofer
>> *Sent:* Friday, October 31, 2014 2:59 PM
>> *To:* Savolainen, Petri (NSN - FI/Espoo)
>> *Cc:* lng-odp@lists.linaro.org
>>
>>
>> *Subject:* Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions
>>
>>
>>
>> This may well be a reasonable restriction for ODP v1.0 but I believe it's
>> something we need to put on the list for "production grade" improvements
>> for 2015.
>>
>>
>>
>> Bill
>>
>>
>>
>> On Fri, Oct 31, 2014 at 7:57 AM, Savolainen, Petri (NSN - FI/Espoo) <
>> petri.savolai...@nsn.com> wrote:
>>
>> Yes, it’s step-by-step and I think it’s the level of ordering we need for
>> v1.0. Most SoCs can implement it, even when  the HW scheduler would not
>> have order support but only atomic/parallel. This way defined atomic
>> scheduling can be used to implement functionality correct ordered queues,
>> the throughput is not improved but it functions correctly.
>>
>>
>>
>> -Petri
>>
>>
>>
>>
>>
>> *From:* ext Bill Fischofer [mailto:bill.fischo...@linaro.org]
>> *Sent:* Friday, October 31, 2014 2:48 PM
>> *To:* Alexandru Badicioiu
>> *Cc:* Petri Savolainen; lng-odp@lists.linaro.org
>> *Subject:* Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions
>>
>>
>>
>> I can well imagine the step-by-step order preservation to be simpler to
>> implement (in SW) but it would also seem to limit performance since the
>> only way to ensure end-to-end order preservation would be if each
>> intermediate queue from ingress to egress were an ordered queue.  If there
>> is a parallel queue anywhere in the chain that would break things.
>>
>>
>>
>> The question is: Is this restriction needed and/or sufficient for ODP
>> v1.0?
>>
>>
>>
>> On Fri, Oct 31, 2014 at 7:42 AM, Alexandru Badicioiu <
>> alexandru.badici...@linaro.org> wrote:
>>
>> "+ * The original enqueue order of the source queue is maintained when
>> buffers a

[lng-odp] [RFC PATCH] Platform specific definitions and inlines

[Ciprian Barbu]

Here is a first attempt to remove inlines from API files. I also moved some
defines and typedefs out of some of the API headers. There is still room for
improvement I guess, the inline implementation and defines could be split into
their own files, but I just tried to show the general idea.

I also tried to make the documentation look good, the platform inlines reference
the documentation in the corresponding API file.

I removed almost all of the individual API includes from C files, keeping only
odp.h, which I remember was desirable. This can still be discussed.

---
 example/ipsec/odp_ipsec.c  |   2 -
 example/ipsec/odp_ipsec_cache.c|   2 -
 example/ipsec/odp_ipsec_fwd_db.c   |   1 -
 example/ipsec/odp_ipsec_loop_db.c  |   2 -
 example/ipsec/odp_ipsec_sa_db.c|   2 -
 example/ipsec/odp_ipsec_sp_db.c|   2 -
 example/ipsec/odp_ipsec_stream.c   |   3 -
 helper/include/odph_eth.h  |   4 -
 helper/include/odph_icmp.h |   3 -
 helper/include/odph_ip.h   |   3 -
 helper/include/odph_ipsec.h|   4 -
 helper/include/odph_udp.h  |   3 -
 platform/linux-generic/include/api/odp.h   |   6 +
 platform/linux-generic/include/api/odp_atomic.h| 282 ---
 platform/linux-generic/include/api/odp_byteorder.h | 187 ++--
 platform/linux-generic/include/api/odp_coremask.h  |  47 +-
 .../include/api/odp_platform_defines.h | 152 ++
 .../include/api/odp_platform_inlines.h | 527 +
 platform/linux-generic/include/api/odp_version.h   |  44 +-
 platform/linux-generic/odp_barrier.c   |   3 +-
 platform/linux-generic/odp_buffer.c|   2 +-
 platform/linux-generic/odp_buffer_pool.c   |   8 +-
 platform/linux-generic/odp_coremask.c  |   3 +-
 platform/linux-generic/odp_crypto.c|   9 +-
 platform/linux-generic/odp_packet.c|   4 +-
 platform/linux-generic/odp_packet_flags.c  |   2 +-
 platform/linux-generic/odp_packet_io.c |  11 +-
 platform/linux-generic/odp_packet_socket.c |   2 +-
 platform/linux-generic/odp_queue.c |  10 +-
 platform/linux-generic/odp_ring.c  |   6 +-
 platform/linux-generic/odp_rwlock.c|   3 +-
 platform/linux-generic/odp_schedule.c  |  11 +-
 platform/linux-generic/odp_thread.c|   7 +-
 platform/linux-generic/odp_ticketlock.c|   4 +-
 platform/linux-generic/odp_timer.c |   7 +-
 35 files changed, 849 insertions(+), 519 deletions(-)
 create mode 100644 platform/linux-generic/include/api/odp_platform_defines.h
 create mode 100644 platform/linux-generic/include/api/odp_platform_inlines.h

diff --git a/example/ipsec/odp_ipsec.c b/example/ipsec/odp_ipsec.c
index ec6c87a..fe91446 100644
--- a/example/ipsec/odp_ipsec.c
+++ b/example/ipsec/odp_ipsec.c
@@ -16,8 +16,6 @@
 #include 
 
 #include 
-#include 
-#include 
 #include 
 #include 
 #include 
diff --git a/example/ipsec/odp_ipsec_cache.c b/example/ipsec/odp_ipsec_cache.c
index 01650d0..d508377 100644
--- a/example/ipsec/odp_ipsec_cache.c
+++ b/example/ipsec/odp_ipsec_cache.c
@@ -8,8 +8,6 @@
 #include 
 
 #include 
-#include 
-#include 
 #include 
 
 #include 
diff --git a/example/ipsec/odp_ipsec_fwd_db.c b/example/ipsec/odp_ipsec_fwd_db.c
index 9ed5d00..d4e3583 100644
--- a/example/ipsec/odp_ipsec_fwd_db.c
+++ b/example/ipsec/odp_ipsec_fwd_db.c
@@ -8,7 +8,6 @@
 #include 
 
 #include 
-#include 
 #include 
 
 #include 
diff --git a/example/ipsec/odp_ipsec_loop_db.c 
b/example/ipsec/odp_ipsec_loop_db.c
index f773352..af4590a 100644
--- a/example/ipsec/odp_ipsec_loop_db.c
+++ b/example/ipsec/odp_ipsec_loop_db.c
@@ -8,8 +8,6 @@
 #include 
 
 #include 
-#include 
-#include 
 
 #include 
 
diff --git a/example/ipsec/odp_ipsec_sa_db.c b/example/ipsec/odp_ipsec_sa_db.c
index f6da623..e8679db 100644
--- a/example/ipsec/odp_ipsec_sa_db.c
+++ b/example/ipsec/odp_ipsec_sa_db.c
@@ -8,8 +8,6 @@
 #include 
 
 #include 
-#include 
-#include 
 
 #include 
 
diff --git a/example/ipsec/odp_ipsec_sp_db.c b/example/ipsec/odp_ipsec_sp_db.c
index f288dfe..8edc0fa 100644
--- a/example/ipsec/odp_ipsec_sp_db.c
+++ b/example/ipsec/odp_ipsec_sp_db.c
@@ -8,8 +8,6 @@
 #include 
 
 #include 
-#include 
-#include 
 
 #include 
 
diff --git a/example/ipsec/odp_ipsec_stream.c b/example/ipsec/odp_ipsec_stream.c
index fba425c..3260b53 100644
--- a/example/ipsec/odp_ipsec_stream.c
+++ b/example/ipsec/odp_ipsec_stream.c
@@ -13,9 +13,6 @@
 #include 
 
 #include 
-#include 
-#include 
-#include 
 #include 
 #include 
 #include 
diff --git a/helper/include/odph_eth.h b/helper/include/odph_eth.h
index 55a2b1e..12a7af5 100644
--- a/helper/include/odph_eth.h
+++ b/helper/include/odph_eth.h
@@ -1

[lng-odp] linux-dpdk: need odp_init_global parameter

[Ciprian Barbu]

Hello Venky,

We've been looking at some of the OVS cards we have and the subject of
testing OVS on top of ODP on top of DPDK came up. Last time I worked
on this I modified odp_init_dpdk with hardcoded values but we need to
solve this problem in the near future. Mike pushed a patch that allows
application to pass platform specific init parameters to
odp_init_global_init and I think that might be enough. Do you have any
thoughts on this?

/Ciprian

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH v2] odp_example.c: fix dead code path

[Ciprian Barbu]

On Mon, Nov 3, 2014 at 8:23 PM, Mike Holmes  wrote:
> ping
>
> On 30 October 2014 18:17, Mike Holmes  wrote:
>>
>> tot is = i which cannot leave the for loop without being positive
>> unless it exits entirely and never reaches the test statement.
>> Or QUEUE_ROUNDS is #defined to 0 rather than (512*1024)
>> Thus tot is always true and the else cannot execute

Why not use QUEUE_ROUNDS instead and get rid of tot altogether ?

>>
>> Signed-off-by: Mike Holmes 
>> ---
>>
>> Previous patch missed a case.
>> Fixed indent
>>
>>  example/odp_example/odp_example.c | 9 ++---
>>  1 file changed, 2 insertions(+), 7 deletions(-)
>>
>> diff --git a/example/odp_example/odp_example.c
>> b/example/odp_example/odp_example.c
>> index 5f25f89..1ed4a0b 100644
>> --- a/example/odp_example/odp_example.c
>> +++ b/example/odp_example/odp_example.c
>> @@ -355,13 +355,8 @@ static int test_schedule_one_single(const char *str,
>> int thr,
>> odp_barrier_sync(barrier);
>> clear_sched_queues();
>>
>> -   if (tot) {
>> -   cycles = cycles/tot;
>> -   ns = ns/tot;
>> -   } else {
>> -   cycles = 0;
>> -   ns = 0;
>> -   }
>> +   cycles = cycles/tot;
>> +   ns = ns/tot;
>>
>> printf("  [%i] %s enq+deq %"PRIu64" cycles, %"PRIu64" ns\n",
>>thr, str, cycles, ns);
>> --
>> 2.1.0
>>
>
>
>
> --
> Mike Holmes
> Linaro  Sr Technical Manager
> LNG - ODP
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp
>

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH v2] fix including odp_std_types.h

[Ciprian Barbu]

Do we still worry about Petri's comment? Right now
platform/linux-generic/include/api/odp_byteorder.h includes endian.h.
Anyway that should be the subject of a different patch so I think this
can go in.

On Mon, Nov 3, 2014 at 10:53 PM, Mike Holmes  wrote:
> API headers should not include C std lib headers directly
>
> Signed-off-by: Mike Holmes 

Reviewed-and-tested-by: Ciprian Barbu 

> ---
>
> v2:
> remove endian.h it should be cleaned up during configuration / build
>
>  platform/linux-generic/include/api/odp_debug.h | 3 +--
>  platform/linux-generic/include/api/odp_std_types.h | 2 ++
>  2 files changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/platform/linux-generic/include/api/odp_debug.h 
> b/platform/linux-generic/include/api/odp_debug.h
> index c9b2edd..99f4574 100644
> --- a/platform/linux-generic/include/api/odp_debug.h
> +++ b/platform/linux-generic/include/api/odp_debug.h
> @@ -12,8 +12,7 @@
>  #ifndef ODP_DEBUG_H_
>  #define ODP_DEBUG_H_
>
> -#include 
> -#include 
> +#include 
>
>  #ifdef __cplusplus
>  extern "C" {
> diff --git a/platform/linux-generic/include/api/odp_std_types.h 
> b/platform/linux-generic/include/api/odp_std_types.h
> index b12a2f3..e3aa5ab 100644
> --- a/platform/linux-generic/include/api/odp_std_types.h
> +++ b/platform/linux-generic/include/api/odp_std_types.h
> @@ -26,6 +26,8 @@ extern "C" {
>  #include 
>  #include 
>  #include 
> +#include 
> +#include 
>
>
>
> --
> 2.1.0
>
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions

[Gilad Ben Yossef]

I don't think it can use the first fragment place in the order. It can use the 
last fragment to arrive place in the queue.
Otherwise you have to stop forwarding until all fragments arrive to keep the 
order… no?

Gilad Ben-Yossef
Software Architect
EZchip Technologies Ltd.
37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel
Tel: +972-4-959- ext. 576, Fax: +972-8-681-1483
Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388
Email: gil...@ezchip.com, Web: 
http://www.ezchip.com

"Ethernet always wins."
— Andy Bechtolsheim

From: Alexandru Badicioiu [mailto:alexandru.badici...@linaro.org]
Sent: Tuesday, November 04, 2014 4:13 PM
To: Bill Fischofer
Cc: Gilad Ben Yossef; Savolainen, Petri (NSN - FI/Espoo); 
lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions

In addition to skip order function I think we need to handle the case when a 
packet gets fragmented (e.g. IP fragmentation). In this case the fragments 
should share the original sequence number and the order restoration logic 
should be aware when the last fragment occurs.

Alex

On 3 November 2014 13:45, Bill Fischofer 
mailto:bill.fischo...@linaro.org>> wrote:
OK, thanks.  That certainly matches the step-by-step semantics Petri proposed.  
We'll have to discuss what, if anything, more we need regarding ORDERED queues 
post-v1.0 next year.

Bill

On Mon, Nov 3, 2014 at 3:03 AM, Gilad Ben Yossef 
mailto:gil...@ezchip.com>> wrote:


What NPS does in HW is that each buffer you dequeue actually stays on the queue 
(but is marked as consumed by the application) and you are given a "tag" for it.

When I want to queue the buffer to the next, I give the HW the tag I got, not 
the buffer handle and what the HW does is only than fully dequeue the buffer 
from the original queue before queuing it into the new queue – but only after 
all preceding (older) buffers on the original queue have been fully dequeued,

Free of a buffer causes a dequeue from the original queue as well.

And I can also give release a buffer from the original queue (give the HW the 
tag back and say "you don't have to wait for this buffer any longer).

I hope this helps,
Gilad

Gilad Ben-Yossef
Software Architect
EZchip Technologies Ltd.
37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel
Tel: +972-4-959- ext. 576, Fax: 
+972-8-681-1483
Mobile: +972-52-826-0388, US Mobile: 
+1-973-826-0388
Email: gil...@ezchip.com, Web: 
http://www.ezchip.com

"Ethernet always wins."
— Andy Bechtolsheim

From: Bill Fischofer 
[mailto:bill.fischo...@linaro.org]
Sent: Sunday, November 02, 2014 5:44 PM
To: Gilad Ben Yossef
Cc: Savolainen, Petri (NSN - FI/Espoo); 
lng-odp@lists.linaro.org

Subject: Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions

Thanks Gilad.  Can you elaborate on that a bit more?  I can understand how 
step-by-step would be potentially easier to implement, but does it also capture 
the majority of expected application use cases?

A good confirmation that this is the correct approach for v1.0 though.

On Sun, Nov 2, 2014 at 9:26 AM, Gilad Ben Yossef 
mailto:gil...@ezchip.com>> wrote:

For what it's worth, as a SoC vendor rep. that has ordered queue in HW, Petri's 
definition is actually preferred for us even going forward ☺


Thanks,
Gilad

Gilad Ben-Yossef
Software Architect
EZchip Technologies Ltd.
37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel
Tel: +972-4-959- ext. 576, Fax: 
+972-8-681-1483
Mobile: +972-52-826-0388, US Mobile: 
+1-973-826-0388
Email: gil...@ezchip.com, Web: 
http://www.ezchip.com

"Ethernet always wins."
— Andy Bechtolsheim

From: lng-odp-boun...@lists.linaro.org 
[mailto:lng-odp-boun...@lists.linaro.org]
 On Behalf Of Bill Fischofer
Sent: Friday, October 31, 2014 2:59 PM
To: Savolainen, Petri (NSN - FI/Espoo)
Cc: lng-odp@lists.linaro.org

Subject: Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions

This may well be a reasonable restriction for ODP v1.0 but I believe it's 
something we need to put on the list for "production grade" improvements for 
2015.

Bill

On Fri, Oct 31, 2014 at 7:57 AM, Savolainen, Petri (NSN - FI/Espoo) 
mailto:petri.savolai...@nsn.com>> wrote:
Yes, it’s step-by-step and I think it’s the level of ordering we need for v1.0. 
Most SoCs can implement it, even when  the HW scheduler would not have order 
support but only atomic/parallel. This way defined atomic scheduling can be 
used to implement functionality correct ordered queues, the throughput is not 
improved but it functions correctly.

-Petri


From: ext Bill Fischofer 
[mailto:bill.fischo...@linaro.org]
Sent: Friday, October 31, 2014 2:

Re: [lng-odp] [Q] Memory allocation in ODP applications

[Victor Kamensky]

I did express the same in the past and agree with Petri
'“optional” is next to non-existent'. It creates fragmentation
that we don't need. When I need to figure out what optional
pieces implemented by specific ODP implementation and
how to match one that not implemented from that point
it is not very far for me just to create my own wrappers/shims
on top of specific SDKs.

For optional APIs being present and return error in run-time
even worse in my opinion. Build time link error would be much
better. Because as usual run-time errors much harder to get
compared to build time errors. Build errors I discover during
single build, run-time errors I may need to create bunch of
specific scenarios to get complete coverage for pieces that
my app uses.

Thanks,
Victor


On 4 November 2014 05:38, Bill Fischofer  wrote:
> That's precisely why we've said that all implementations must provide at
> least stubs for all of the ODP APIs (even those marked optional).  Not every
> ODP application will be able to run on every ODP implementation, but that's
> OK.  You don't need your home office router to have enterprise-level
> features.  Requiring every ODP implementation to provide identical
> functionality will either force us to adopt a least-common-denominator
> approach to what are acceptable APIs or else we'll severely limit the range
> of platforms that can participate in the ecosystem.
>
> Do you have a specific list of APIs that are currently marked optional that
> you'd like to see promoted to mandatory?
>
>
> On Tue, Nov 4, 2014 at 7:26 AM, Savolainen, Petri (NSN - FI/Espoo)
>  wrote:
>>
>> I’m concerned that the API gets too fragmented already in v1.0 and the
>> ecosystem will be limited by that. For example, it’s hard to integrate 3rd
>> party (or open source) SW into a system, if my app uses optional feature X,
>> the 3rd party lib uses optional feature Y and the implementation provides
>> only one of those, or neither.
>>
>>
>>
>> -Petri
>>
>>
>>
>>
>>
>> From: ext Bill Fischofer [mailto:bill.fischo...@linaro.org]
>> Sent: Tuesday, November 04, 2014 3:16 PM
>> To: Savolainen, Petri (NSN - FI/Espoo)
>> Cc: Shmulik Ladkani; lng-odp@lists.linaro.org
>>
>>
>> Subject: Re: [lng-odp] [Q] Memory allocation in ODP applications
>>
>>
>>
>> I think the criteria I outlined above for considering an API optional are
>> reasonable.  Recall that the reason certain APIs are designated as optional
>> is because we received feedback that they were not feasible on specific
>> platforms.  It's RECOMMENDED that platforms provide all of these APIs,
>> however I'm not eager to say platform X can't have a conforming ODP
>> implementation if it's just not feasible to implement every ODP API or
>> feature on that platform.
>>
>>
>>
>> Applications will be written to whatever APIs and features they need.  Not
>> every application needs every API and feature and if an application needs an
>> optional API then when it chooses which platforms it wants to deploy on it
>> will take whether that platform provides that API into account in making its
>> decision.
>>
>>
>>
>> These are fundamentally product marketing issues, not technical issues.
>> If we take a one-size-fits-all approach that will unnecessarily limit the
>> ecosystem.
>>
>>
>>
>> On Tue, Nov 4, 2014 at 6:43 AM, Savolainen, Petri (NSN - FI/Espoo)
>>  wrote:
>>
>> Hi,
>>
>>
>>
>> As noted many times before, “optional” is next to non-existent for
>> application programmers – especially if it’s important to achieve
>> application portability. I’d suggest that API v1.0 would not have anything
>> labeled as optional. It’s simplifies the API for everybody
>> (implementers/testers/users). Features labeled as optional should be either
>> turned into mandatory or dropped out.
>>
>>
>>
>> -Petri
>>
>>
>>
>>
>>
>> From: lng-odp-boun...@lists.linaro.org
>> [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of ext Bill Fischofer
>> Sent: Monday, November 03, 2014 10:35 PM
>> To: Shmulik Ladkani
>> Cc: lng-odp@lists.linaro.org
>> Subject: Re: [lng-odp] [Q] Memory allocation in ODP applications
>>
>>
>>
>> ODP APIs are divided into two classes: Mandatory, which all conforming ODP
>> implementations are expected to provide, and Optional.  The convention is
>> that Optional APIs must be present but calling them may simply result in an
>> ODP_FUNCTION_NOT_AVAILABLE return code.  These are documented in the
>> associated header files (currently under revision) and are part of the
>> doxygen.  Basically, unless an API calls itself out as OPTIONAL you should
>> assume that all ODP implementations will provide functional implementations
>> of it, though performance of individual APIs may vary between platforms and
>> within platforms over time.
>>
>>
>>
>> It's expected that each platform will provide implementations of the ODP
>> APIs that are optimized for that particular platform.  ODP itself does not
>> specify how implementations do that or how they might improve performance
>> over time with s

Re: [lng-odp] [Q] Memory allocation in ODP applications

[Gilad Ben Yossef]

True, but maybe there is a middle way -
What if instead of a mad matrix of optional features we have 2 or 3 levels of 
features?
Say:
ODP Basic support - all mandatory features are supported.
ODP Full support - mandatory + all what we now call optional features are 
supported.
And maybe, if it makes sense -
ODP Intermediate support - mandatory + a certain pre-defined set of now called 
optional features supported.
This let application writer to say: My application targets Basic, Intermediate 
or Full support and the application writer knows exactly what to expect. 
The platform vendor can say: I support ODP Basic, Intermediate or Full. So you 
know what you get.

Gilad

Gilad Ben-Yossef
Software Architect
EZchip Technologies Ltd.
37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel
Tel: +972-4-959- ext. 576, Fax: +972-8-681-1483 
Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388
Email: gil...@ezchip.com, Web: http://www.ezchip.com

"Ethernet always wins."
— Andy Bechtolsheim

> -Original Message-
> From: lng-odp-boun...@lists.linaro.org [mailto:lng-odp-
> boun...@lists.linaro.org] On Behalf Of Victor Kamensky
> Sent: Tuesday, November 04, 2014 5:33 PM
> To: Bill Fischofer
> Cc: Savolainen, Petri (NSN - FI/Espoo); lng-odp@lists.linaro.org
> Subject: Re: [lng-odp] [Q] Memory allocation in ODP applications
> 
> I did express the same in the past and agree with Petri
> '“optional” is next to non-existent'. It creates fragmentation
> that we don't need. When I need to figure out what optional
> pieces implemented by specific ODP implementation and
> how to match one that not implemented from that point
> it is not very far for me just to create my own wrappers/shims
> on top of specific SDKs.
> 
> For optional APIs being present and return error in run-time
> even worse in my opinion. Build time link error would be much
> better. Because as usual run-time errors much harder to get
> compared to build time errors. Build errors I discover during
> single build, run-time errors I may need to create bunch of
> specific scenarios to get complete coverage for pieces that
> my app uses.
> 
> Thanks,
> Victor
> 
> 
> On 4 November 2014 05:38, Bill Fischofer 
> wrote:
> > That's precisely why we've said that all implementations must provide
> at
> > least stubs for all of the ODP APIs (even those marked optional).  Not
> every
> > ODP application will be able to run on every ODP implementation, but
> that's
> > OK.  You don't need your home office router to have enterprise-level
> > features.  Requiring every ODP implementation to provide identical
> > functionality will either force us to adopt a least-common-denominator
> > approach to what are acceptable APIs or else we'll severely limit the
> range
> > of platforms that can participate in the ecosystem.
> >
> > Do you have a specific list of APIs that are currently marked optional
> that
> > you'd like to see promoted to mandatory?
> >
> >
> > On Tue, Nov 4, 2014 at 7:26 AM, Savolainen, Petri (NSN - FI/Espoo)
> >  wrote:
> >>
> >> I’m concerned that the API gets too fragmented already in v1.0 and
> the
> >> ecosystem will be limited by that. For example, it’s hard to
> integrate 3rd
> >> party (or open source) SW into a system, if my app uses optional
> feature X,
> >> the 3rd party lib uses optional feature Y and the implementation
> provides
> >> only one of those, or neither.
> >>
> >>
> >>
> >> -Petri
> >>
> >>
> >>
> >>
> >>
> >> From: ext Bill Fischofer [mailto:bill.fischo...@linaro.org]
> >> Sent: Tuesday, November 04, 2014 3:16 PM
> >> To: Savolainen, Petri (NSN - FI/Espoo)
> >> Cc: Shmulik Ladkani; lng-odp@lists.linaro.org
> >>
> >>
> >> Subject: Re: [lng-odp] [Q] Memory allocation in ODP applications
> >>
> >>
> >>
> >> I think the criteria I outlined above for considering an API optional
> are
> >> reasonable.  Recall that the reason certain APIs are designated as
> optional
> >> is because we received feedback that they were not feasible on
> specific
> >> platforms.  It's RECOMMENDED that platforms provide all of these
> APIs,
> >> however I'm not eager to say platform X can't have a conforming ODP
> >> implementation if it's just not feasible to implement every ODP API
> or
> >> feature on that platform.
> >>
> >>
> >>
> >> Applications will be written to whatever APIs and features they need.
> Not
> >> every application needs every API and feature and if an application
> needs an
> >> optional API then when it chooses which platforms it wants to deploy
> on it
> >> will take whether that platform provides that API into account in
> making its
> >> decision.
> >>
> >>
> >>
> >> These are fundamentally product marketing issues, not technical
> issues.
> >> If we take a one-size-fits-all approach that will unnecessarily limit
> the
> >> ecosystem.
> >>
> >>
> >>
> >> On Tue, Nov 4, 2014 at 6:43 AM, Savolainen, Petri (NSN - FI/Espoo)
> >>  wrote:
> >>
> >> Hi,
> >>
> >>
> >>
> >> As noted many times before, “optional” is next to non-exis

Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions

[Jacob, Jerin]

Since odp_schedule_skip_order has dest queue, So how its differ from 
odp_queue_enque() as

en-queue is going to release it implicitly.


Jerin.




From: lng-odp-boun...@lists.linaro.org  on 
behalf of Alexandru Badicioiu 
Sent: Friday, October 31, 2014 6:40 PM
To: Bill Fischofer
Cc: Savolainen, Petri (NSN - FI/Espoo); lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions

In this case the "dest" argument in the following function:
+void odp_schedule_skip_order(odp_queue_t dest, odp_buffer_t buf);
looses it's meaning.

Alex



On 31 October 2014 14:59, Bill Fischofer 
mailto:bill.fischo...@linaro.org>> wrote:
This may well be a reasonable restriction for ODP v1.0 but I believe it's 
something we need to put on the list for "production grade" improvements for 
2015.

Bill

On Fri, Oct 31, 2014 at 7:57 AM, Savolainen, Petri (NSN - FI/Espoo) 
mailto:petri.savolai...@nsn.com>> wrote:
Yes, it’s step-by-step and I think it’s the level of ordering we need for v1.0. 
Most SoCs can implement it, even when  the HW scheduler would not have order 
support but only atomic/parallel. This way defined atomic scheduling can be 
used to implement functionality correct ordered queues, the throughput is not 
improved but it functions correctly.

-Petri


From: ext Bill Fischofer 
[mailto:bill.fischo...@linaro.org]
Sent: Friday, October 31, 2014 2:48 PM
To: Alexandru Badicioiu
Cc: Petri Savolainen; lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [PATCH] Scheduler atomic and ordered definitions

I can well imagine the step-by-step order preservation to be simpler to 
implement (in SW) but it would also seem to limit performance since the only 
way to ensure end-to-end order preservation would be if each intermediate queue 
from ingress to egress were an ordered queue.  If there is a parallel queue 
anywhere in the chain that would break things.

The question is: Is this restriction needed and/or sufficient for ODP v1.0?

On Fri, Oct 31, 2014 at 7:42 AM, Alexandru Badicioiu 
mailto:alexandru.badici...@linaro.org>> wrote:
"+ * The original enqueue order of the source queue is maintained when buffers 
are
+ * enqueued to their destination queue(s) before another schedule call"

Is this assuming that the order will be restored always at the next enqueue? I 
think there should be an option to explicitly indicate if the next enqueue is 
supposed to restore the order or not, especially when packets move from queue 
to queue. Ordered queues are costly compared with the ordinary ones.

Alex

On 31 October 2014 14:25, Petri Savolainen 
mailto:petri.savolai...@linaro.org>> wrote:
Improved atomic and ordered synchronisation definitions. Added
order skip function prototype.

Signed-off-by: Petri Savolainen 
mailto:petri.savolai...@linaro.org>>

---
This is the ordered queue definition (in patch format) promised
in the call yesterday.
---
 platform/linux-generic/include/api/odp_queue.h| 31 +++-
 platform/linux-generic/include/api/odp_schedule.h | 45 ++-
 2 files changed, 64 insertions(+), 12 deletions(-)

diff --git a/platform/linux-generic/include/api/odp_queue.h 
b/platform/linux-generic/include/api/odp_queue.h
index b8ac4bb..c0c3969 100644
--- a/platform/linux-generic/include/api/odp_queue.h
+++ b/platform/linux-generic/include/api/odp_queue.h
@@ -78,8 +78,35 @@ typedef int odp_schedule_prio_t;
 typedef int odp_schedule_sync_t;

 #define ODP_SCHED_SYNC_NONE 0  /**< Queue not synchronised */
-#define ODP_SCHED_SYNC_ATOMIC   1  /**< Atomic queue */
-#define ODP_SCHED_SYNC_ORDERED  2  /**< Ordered queue */
+
+/**
+ * Atomic queue synchronisation
+ *
+ * The scheduler gives buffers from a queue to a single core at a time. This
+ * serialises processing of the buffers from the source queue and helps user to
+ * avoid SW locking. Another schedule call will implicitely release the atomic
+ * synchronisation of the source queue and free the scheduler to give buffers
+ * from the queue to other cores.
+ *
+ * User can hint the scheduler to release the atomic synchronisation early with
+ * odp_schedule_release_atomic().
+ */
+#define ODP_SCHED_SYNC_ATOMIC   1
+
+/**
+ * Ordered queue synchronisation
+ *
+ * The scheduler may give out buffers to multiple cores for parallel 
processing.
+ * The original enqueue order of the source queue is maintained when buffers 
are
+ * enqueued to their destination queue(s) before another schedule call. Buffers
+ * from the same ordered (source) queue appear in their original order when
+ * dequeued from a destination queue. The destination queue type (POLL/SCHED) 
or
+ * synchronisation (NONE/ATOMIC/ORDERED) is not limited.
+ *
+ * User can command the scheduler to skip ordering of a buffer with
+ * odp_schedule_skip_order().
+ */
+#define ODP_SCHED_SYNC_ORDERED  2

 /** Default queue synchronisation */
 #define ODP_SCHED_SYNC_DEFAUL

Re: [lng-odp] [Q] Memory allocation in ODP applications

[Bill Fischofer]

We we want to call everything we've documented mandatory in v1.0 I'm OK
with that.  What I don't want is for us to say that and then as we approach
end of year someone says "but I really can't do (formerly optional) feature
X".

Quick poll: Are there any APIs currently marked optional that anyone does
not plan to implement in their v1.0 release?

On Tue, Nov 4, 2014 at 9:58 AM, Gilad Ben Yossef  wrote:

>
> True, but maybe there is a middle way -
> What if instead of a mad matrix of optional features we have 2 or 3 levels
> of features?
> Say:
> ODP Basic support - all mandatory features are supported.
> ODP Full support - mandatory + all what we now call optional features are
> supported.
> And maybe, if it makes sense -
> ODP Intermediate support - mandatory + a certain pre-defined set of now
> called optional features supported.
> This let application writer to say: My application targets Basic,
> Intermediate or Full support and the application writer knows exactly what
> to expect.
> The platform vendor can say: I support ODP Basic, Intermediate or Full. So
> you know what you get.
>
> Gilad
>
> Gilad Ben-Yossef
> Software Architect
> EZchip Technologies Ltd.
> 37 Israel Pollak Ave, Kiryat Gat 82025 ,Israel
> Tel: +972-4-959- ext. 576, Fax: +972-8-681-1483
> Mobile: +972-52-826-0388, US Mobile: +1-973-826-0388
> Email: gil...@ezchip.com, Web: http://www.ezchip.com
>
> "Ethernet always wins."
> — Andy Bechtolsheim
>
> > -Original Message-
> > From: lng-odp-boun...@lists.linaro.org [mailto:lng-odp-
> > boun...@lists.linaro.org] On Behalf Of Victor Kamensky
> > Sent: Tuesday, November 04, 2014 5:33 PM
> > To: Bill Fischofer
> > Cc: Savolainen, Petri (NSN - FI/Espoo); lng-odp@lists.linaro.org
> > Subject: Re: [lng-odp] [Q] Memory allocation in ODP applications
> >
> > I did express the same in the past and agree with Petri
> > '“optional” is next to non-existent'. It creates fragmentation
> > that we don't need. When I need to figure out what optional
> > pieces implemented by specific ODP implementation and
> > how to match one that not implemented from that point
> > it is not very far for me just to create my own wrappers/shims
> > on top of specific SDKs.
> >
> > For optional APIs being present and return error in run-time
> > even worse in my opinion. Build time link error would be much
> > better. Because as usual run-time errors much harder to get
> > compared to build time errors. Build errors I discover during
> > single build, run-time errors I may need to create bunch of
> > specific scenarios to get complete coverage for pieces that
> > my app uses.
> >
> > Thanks,
> > Victor
> >
> >
> > On 4 November 2014 05:38, Bill Fischofer 
> > wrote:
> > > That's precisely why we've said that all implementations must provide
> > at
> > > least stubs for all of the ODP APIs (even those marked optional).  Not
> > every
> > > ODP application will be able to run on every ODP implementation, but
> > that's
> > > OK.  You don't need your home office router to have enterprise-level
> > > features.  Requiring every ODP implementation to provide identical
> > > functionality will either force us to adopt a least-common-denominator
> > > approach to what are acceptable APIs or else we'll severely limit the
> > range
> > > of platforms that can participate in the ecosystem.
> > >
> > > Do you have a specific list of APIs that are currently marked optional
> > that
> > > you'd like to see promoted to mandatory?
> > >
> > >
> > > On Tue, Nov 4, 2014 at 7:26 AM, Savolainen, Petri (NSN - FI/Espoo)
> > >  wrote:
> > >>
> > >> I’m concerned that the API gets too fragmented already in v1.0 and
> > the
> > >> ecosystem will be limited by that. For example, it’s hard to
> > integrate 3rd
> > >> party (or open source) SW into a system, if my app uses optional
> > feature X,
> > >> the 3rd party lib uses optional feature Y and the implementation
> > provides
> > >> only one of those, or neither.
> > >>
> > >>
> > >>
> > >> -Petri
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> From: ext Bill Fischofer [mailto:bill.fischo...@linaro.org]
> > >> Sent: Tuesday, November 04, 2014 3:16 PM
> > >> To: Savolainen, Petri (NSN - FI/Espoo)
> > >> Cc: Shmulik Ladkani; lng-odp@lists.linaro.org
> > >>
> > >>
> > >> Subject: Re: [lng-odp] [Q] Memory allocation in ODP applications
> > >>
> > >>
> > >>
> > >> I think the criteria I outlined above for considering an API optional
> > are
> > >> reasonable.  Recall that the reason certain APIs are designated as
> > optional
> > >> is because we received feedback that they were not feasible on
> > specific
> > >> platforms.  It's RECOMMENDED that platforms provide all of these
> > APIs,
> > >> however I'm not eager to say platform X can't have a conforming ODP
> > >> implementation if it's just not feasible to implement every ODP API
> > or
> > >> feature on that platform.
> > >>
> > >>
> > >>
> > >> Applications will be written to whatever APIs and features they need.
> > Not
> > >>

[lng-odp] Fwd: Weekly ODP Design Call - Call Summary

[Bill Fischofer]

Summary of today's ODP call.  Again, I apologize for the time zone
confusion.  This call occurs at 15:00 UTC on Tuesdays. If you reside in a
time zone that observes summer time then this call will move relative to
local time as you move between summer and winter times.  The US moved off
summer time last weekend, hence it now occurs at 10am Eastern, 9am Central,
8am Mountain, and 7am Pacific time.  Please update your calendars if they
do not already reflect this change.

Agenda for todays call was:


   -

   Petri’s Ordered Queue Patch
    and
   discussion
   -

  Step-by-Step queue order definition
  -

  New function odp_schedule_skip_order()
  -

  How to handle IP fragments
  -

   Follow-up discussions on PktIO Design Doc
   

   - Shmulik’s proposal for init extensions
   


*Discussion/Decisions:*

   - Step-by-step queue order restoration is the model for ODP v1.0.  If
   caller requires end-to-end ordering, application must ensure that packets
   traverse only ordered or atomic queues.


   - Alex raised issue of IP fragmentation where an application may receive
   one packet and then fragment it while other threads may receive packets
   that are not fragmented.  How is the order of the individual fragments
   observed with respect to other packets from the ordered queue?  Petri to
   propose v1.0 solution for this, which may be incomplete.


   - Shmulik's proposal was discussed.  Consensus is that if an application
   uses other frameworks besides ODP then it is the application's
   responsibility to initialize each of those frameworks on a per-thread basis
   as required by that framework.  ODP does not have knowledge of other
   frameworks nor can it be responsible for cascading initialization calls
   between them.


   - ODP uses C bindings. Adding C++ bindings may be an interesting future
   case but more use-case analysis would be needed to advance this.


   - Discussed interaction between pktio and classification.  Stuart should
   have revised PktIO doc out later this week and will propose solutions as
   part of that.

Thank you.

Bill

-- Forwarded message --
From: UberConference 
Date: Tue, Nov 4, 2014 at 10:10 AM
Subject: Weekly ODP Design Call - Call Summary
To: bill.fischo...@linaro.org





 Weekly ODP Design Call
 November 4, 7:56AM - 9:08AM MST
 72 minutes
 Shared Files   Chat Transcript
Recording
#1

43.9 MB
--
Participants
In order of appearance
 Bill Fischofer
 7:56AM - 9:08AM
 27 min
   
 Alexandru Badicioiu
 7:58AM - 8:20AM
 4 min
   
 Bala Manoharan
 8:00AM - 9:08AM
 0 min
   
 Shmulik Ladkani
 8:01AM - 9:08AM

8 min
   
 Taras Kondratiuk
 8:03AM - 9:08AM

3 min
   
 447468717073
 8:03AM - 9:08AM

0 min
   Jerin Jacob
 8:04AM - 9:08AM
 0 min
   

 Marshall Guillory
 8:04AM - 9:07AM
 0 min
   

 Mike Holmes
 8:04AM - 9:08AM
 0 min
   
 Job
 8:06AM - 9:08AM
 0 min
   Stuart Haslam
 8:07AM - 9:08AM
 2 min
   Zoltan Kiss
 8:08AM - 8:09AM
 0 min
   
 Tx Instruments
 8:10AM - 9:08AM

 0 min
   Ola Liljedahl
 8:11AM - 8:43AM
 2 min
   Petri Savolainen
 8:13AM - 9:08AM
 18 min
   
 Ciprian Barbu
 8:16AM - 9:08AM

 0 min
   
 Anant Agarwal
 9:01AM - 9:02AM

 0 min
   Keith Wiles
 9:02AM - 9:08AM
 0 min
   
 Leonard Bush
 9:02AM - 9:08AM

0 min
   Barry Spinney
 9:04AM - 9:08AM

0 min
  Tip: Mobile appsCarry UberConference everywhere you go with
the iPhone and Android apps.
Learn More


UberConference
 If you'd like to stop getting emails from UberConference, click here

[lng-odp] Event update: Weekly ODP Design Discussion Call

[Wiles, Roger Keith]

Wiles, Roger Keith has updated the event: Weekly ODP Design Discussion Call, 
scheduled for July 15, 2014 at 10:00 AM (US/Central). To acknowledge this 
invitation, click the link below.



BEGIN:VCALENDAR
CALSCALE:GREGORIAN
VERSION:2.0
METHOD:REQUEST
PRODID:-//Apple Inc.//Mac OS X 10.10//EN
BEGIN:VTIMEZONE
TZID:US/Central
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
DTSTART:20070311T02
TZNAME:CDT
TZOFFSETTO:-0500
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
DTSTART:20071104T02
TZNAME:CST
TZOFFSETTO:-0600
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
ATTENDEE;CN="Bill Fischofer";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-ACTION;ROL
 E=REQ-PARTICIPANT:mailto:bill.fischo...@linaro.org
ATTENDEE;CN="Wiles, Roger Keith";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED:mai
 lto:keith.wi...@windriver.com
ATTENDEE;CN="lng-odp@lists.linaro.org";CUTYPE=INDIVIDUAL;PARTSTAT=NEEDS-
 ACTION;ROLE=REQ-PARTICIPANT:mailto:lng-odp@lists.linaro.org
DTEND;TZID=US/Central:20140715T11
TRANSP:OPAQUE
ORGANIZER;CN="Wiles, Roger Keith":mailto:keith.wi...@windriver.com
UID:agpzfnViZXJjb25mciALEhNTY2hlZHVsZWRDb25mZXJlbmNlGICAwNCOxNsLDA
EXDATE;TZID=US/Central:20141014T10
DTSTAMP:20141104T160057Z
LOCATION:866-230-0007
DESCRIPTION:Your UberConference information is below.\n\n\n\nView or joi
 n the call: https://www.uberconference.com/opendataplane\n\nDial in numb
 er: 866-230-0007\n\nAlternate Number: 512-887-5486\n\nNo PIN needed.\n\n
 International Access Numbers: https://www.uberconference.com/internation
 al\n\nThe following information couldn’t be added to Exchange:\n\nurl:ht
 tps://www.uberconference.com/opendataplane\n\n
SEQUENCE:1
SUMMARY:Weekly ODP Design Discussion Call
DTSTART;TZID=US/Central:20140715T10
CREATED:20141017T175641Z
RRULE:FREQ=WEEKLY;INTERVAL=1;UNTIL=20141103T235959Z;BYDAY=TU
END:VEVENT
END:VCALENDAR
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH v2 00/18] Split out platform-specific values from header files

[Ciprian Barbu]

On Tue, Nov 4, 2014 at 3:45 PM, Bill Fischofer
 wrote:
> Please take a look at the RFC patches I circulated for odp_buffer.h,
> odp_buffer_pool.h, and odp_packet.h.  These now serve as both executables
> and doxygen source so the information is in one place.

Ok, I saw the two patches, but after sending the previous email.
Whether to keep the doxygen annotation in the API or not is one of the
aspects, it has its advantages and disadvantages. One of the lows is
that the API headers that differ between platforms (odp_buffer.h and
odp_buffer_pool.h are likely to do so) will be hard to compare from
platform to platform. Also the documentation can diverge between
platforms without being the case. Could be only formatting, not
necessarily differences in design (it should not be if a platform is
supposed to suport API vx.x.x). But anyway it lacks clarity from my
point of view.

Another aspect that was discussed is how to handle the need to inline
APIs, Taras has done some work for that and he encountered some
problems. I spent a couple of hours today too on this, I was
intrigued, and I sent out a patch. Here is a link to in case it was
missed:

http://lists.linaro.org/pipermail/lng-odp/2014-November/004478.html

/Ciprian

>
> The split discussion refers to where the platform-specific typedefs are
> defined + the question of how inlined implementations of select APIs are
> expressed.
>
> On Tue, Nov 4, 2014 at 7:15 AM, Ciprian Barbu 
> wrote:
>>
>> On Mon, Nov 3, 2014 at 5:47 PM, Taras Kondratiuk
>>  wrote:
>> > A need to split out a clean API headers is being discussed again, so
>> > pulling this thread up to remind what the previous attempt was.
>> >
>> > On 07/08/2014 05:55 PM, Mike Holmes wrote:
>> >> We discussed this on a hangout, here are the results as a way forward
>> >> on
>> >> this 1 month old topic.
>> >>
>> >>
>> >> Attendees:
>> >> Bala
>> >> Taras
>> >> Maxim
>> >> Santosh
>> >> Marshall
>> >> Mike
>> >> Bill
>> >>
>> >> Actions required:
>> >>
>> >> Step one:
>> >> Move odp/include to odp/platform/linux-generic/include, update the make
>> >> files to suit.
>> >>
>> >> Step two:
>> >> Pull the doxygen documentation back up to odp/doc/odp_xxx.dox out of
>> >> the
>> >> header files.
>> >>
>> >> General Rational:
>> >>
>> >>  1. Anecdotal evidence from previous projects suggests we will end up
>> >> with per platform includes anyway
>>
>> Is this because of the need to inline and to have per-platform defines
>> / typedefs ?
>>
>> >>  2. We are trying to accommodate splitting out the platform specifics
>> >> on
>> >> this thread, that is what has driven this issue so No.1 looks
>> >> correct
>> >>  3. Linux-generic is the reference implementation and by default it is
>> >> what is built now so there is no change in observed bechavior. This
>> >> includes the fact that the platform specifics show though, all that
>> >> changes is that is it now clear that they are specific to that
>> >> platform and no pretense that the default doc is is platform
>> >> independent.
>> >>  4. We reduce the directory structure complexity, no need to include
>> >> odp/include for each platforms make file.
>> >>  5. Other platforms already include Linux-generic to reuse its code so
>> >> we are not adding any new paths to find the same headers for those
>> >> cases, there is no impact if the Linux-generic is not needed.
>> >>  6. The API documentation is still common, it can be stored once in
>> >> odp/doc/odpx.dox, doxygen will tie this together with the
>> >> headers found per platform.
>> >>  7. Platforms are still free to add pages to augment the documentation
>> >> with platform specifics.
>> >>
>> >> Negatives:
>> >>
>> >>  1. Documentation is no longer all kept right next to the definitions
>> >> which is one reason to use doxygen in the first place.
>>
>> From my point of view, Doxygen is primarily helpful for having a nice
>> rendered view of the documentation in html/pdf format. Having the
>> documentation in the same place as the headers (whether it contains
>> Doxygen tags or not) makes for a more comfortable view for the
>> programmer who may not be always looking at the browser (those
>> hardcore Linux guys :) So I think we should have separate dox file.
>>
>> >>  2. There is less enforcement of the API across platforms, hopefully
>> >> mitigated by the ODP-validation test suite.
>> >>
>> >>
>> >>
>> >> On 3 July 2014 13:14, Mike Holmes > >> > wrote:
>> >>
>> >> Let me take a look, I will ping you offline to make sure I
>> >> understand and replicate the issue correctly
>> >>
>> >>
>> >> On 3 July 2014 06:26, Taras Kondratiuk > >> > wrote:
>> >>
>> >> On 06/26/2014 06:24 PM, Taras Kondratiuk wrote:
>> >>
>> >> As we have discussed during a call, I've tried to implement
>> >> option #2
>> >> for several files, but looks li

Re: [lng-odp] Event update: Weekly ODP Design Discussion Call

[Wiles, Roger Keith]

Sorry, please ignore. The calendar in Mac Mail did not give me an option to not 
send this email when I moved it :-(

> On Nov 4, 2014, at 11:02 AM, Wiles, Roger Keith  
> wrote:
> 
> Wiles, Roger Keith has updated the event: Weekly ODP Design Discussion Call, 
> scheduled for July 15, 2014 at 10:00 AM (US/Central). To acknowledge this 
> invitation, click the link below.
> 
> 
> 
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/lng-odp

Keith Wiles, Principal Technologist with CTO office, Wind River mobile 
972-213-5533


___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH v2 00/18] Split out platform-specific values from header files

[Bill Fischofer]

These files simply describe the APIs, their parameters, and expected
behaviors.  As such they are implementation-independent.  The only
implementation-specific things in them are the typedef definitions, which
is what we were discussing moving.  The sequence of #includes should be:

ApplicationImplementation Common ODP API

#include odp.h
   #include platform/.../odp_xxx.h
   (defines typedefs and inlines)
  #include
common/odp_xxx.h
  (references typedefs.
If typedefsmissing
or inlines
   conflict, will cause
compile
   error, indicating a
problem in
   platform
understanding of API)

Does this structure make sense?



On Tue, Nov 4, 2014 at 11:14 AM, Ciprian Barbu 
wrote:

> On Tue, Nov 4, 2014 at 3:45 PM, Bill Fischofer
>  wrote:
> > Please take a look at the RFC patches I circulated for odp_buffer.h,
> > odp_buffer_pool.h, and odp_packet.h.  These now serve as both executables
> > and doxygen source so the information is in one place.
>
> Ok, I saw the two patches, but after sending the previous email.
> Whether to keep the doxygen annotation in the API or not is one of the
> aspects, it has its advantages and disadvantages. One of the lows is
> that the API headers that differ between platforms (odp_buffer.h and
> odp_buffer_pool.h are likely to do so) will be hard to compare from
> platform to platform. Also the documentation can diverge between
> platforms without being the case. Could be only formatting, not
> necessarily differences in design (it should not be if a platform is
> supposed to suport API vx.x.x). But anyway it lacks clarity from my
> point of view.
>
> Another aspect that was discussed is how to handle the need to inline
> APIs, Taras has done some work for that and he encountered some
> problems. I spent a couple of hours today too on this, I was
> intrigued, and I sent out a patch. Here is a link to in case it was
> missed:
>
> http://lists.linaro.org/pipermail/lng-odp/2014-November/004478.html
>
> /Ciprian
>
> >
> > The split discussion refers to where the platform-specific typedefs are
> > defined + the question of how inlined implementations of select APIs are
> > expressed.
> >
> > On Tue, Nov 4, 2014 at 7:15 AM, Ciprian Barbu 
> > wrote:
> >>
> >> On Mon, Nov 3, 2014 at 5:47 PM, Taras Kondratiuk
> >>  wrote:
> >> > A need to split out a clean API headers is being discussed again, so
> >> > pulling this thread up to remind what the previous attempt was.
> >> >
> >> > On 07/08/2014 05:55 PM, Mike Holmes wrote:
> >> >> We discussed this on a hangout, here are the results as a way forward
> >> >> on
> >> >> this 1 month old topic.
> >> >>
> >> >>
> >> >> Attendees:
> >> >> Bala
> >> >> Taras
> >> >> Maxim
> >> >> Santosh
> >> >> Marshall
> >> >> Mike
> >> >> Bill
> >> >>
> >> >> Actions required:
> >> >>
> >> >> Step one:
> >> >> Move odp/include to odp/platform/linux-generic/include, update the
> make
> >> >> files to suit.
> >> >>
> >> >> Step two:
> >> >> Pull the doxygen documentation back up to odp/doc/odp_xxx.dox out of
> >> >> the
> >> >> header files.
> >> >>
> >> >> General Rational:
> >> >>
> >> >>  1. Anecdotal evidence from previous projects suggests we will end up
> >> >> with per platform includes anyway
> >>
> >> Is this because of the need to inline and to have per-platform defines
> >> / typedefs ?
> >>
> >> >>  2. We are trying to accommodate splitting out the platform specifics
> >> >> on
> >> >> this thread, that is what has driven this issue so No.1 looks
> >> >> correct
> >> >>  3. Linux-generic is the reference implementation and by default it
> is
> >> >> what is built now so there is no change in observed bechavior.
> This
> >> >> includes the fact that the platform specifics show though, all
> that
> >> >> changes is that is it now clear that they are specific to that
> >> >> platform and no pretense that the default doc is is platform
> >> >> independent.
> >> >>  4. We reduce the directory structure complexity, no need to include
> >> >> odp/include for each platforms make file.
> >> >>  5. Other platforms already include Linux-generic to reuse its code
> so
> >> >> we are not adding any new paths to find the same headers for
> those
> >> >> cases, there is no impact if the Linux-generic is not needed.
> >> >>  6. The API documentation is still common, it can be stored once in
> >> >> odp/doc/odpx.dox, doxygen will tie this together with the
> >> >> headers found per platform.
> >> >>  7. Platforms are still free to add pages to augment the
> documentation
> >> >> with platform specifics.
> >> >>
> >> >> Negatives:
> >> >>
> >> >

Re: [lng-odp] [RFC PATCH] Platform specific definitions and inlines

[Taras Kondratiuk]

On 11/04/2014 04:29 PM, Ciprian Barbu wrote:

Here is a first attempt to remove inlines from API files. I also moved some
defines and typedefs out of some of the API headers. There is still room for
improvement I guess, the inline implementation and defines could be split into
their own files, but I just tried to show the general idea.

I also tried to make the documentation look good, the platform inlines reference
the documentation in the corresponding API file.

I removed almost all of the individual API includes from C files, keeping only
odp.h, which I remember was desirable. This can still be discussed.

---
  example/ipsec/odp_ipsec.c  |   2 -
  example/ipsec/odp_ipsec_cache.c|   2 -
  example/ipsec/odp_ipsec_fwd_db.c   |   1 -
  example/ipsec/odp_ipsec_loop_db.c  |   2 -
  example/ipsec/odp_ipsec_sa_db.c|   2 -
  example/ipsec/odp_ipsec_sp_db.c|   2 -
  example/ipsec/odp_ipsec_stream.c   |   3 -
  helper/include/odph_eth.h  |   4 -
  helper/include/odph_icmp.h |   3 -
  helper/include/odph_ip.h   |   3 -
  helper/include/odph_ipsec.h|   4 -
  helper/include/odph_udp.h  |   3 -
  platform/linux-generic/include/api/odp.h   |   6 +
  platform/linux-generic/include/api/odp_atomic.h| 282 ---
  platform/linux-generic/include/api/odp_byteorder.h | 187 ++--
  platform/linux-generic/include/api/odp_coremask.h  |  47 +-
  .../include/api/odp_platform_defines.h | 152 ++
  .../include/api/odp_platform_inlines.h | 527 +
  platform/linux-generic/include/api/odp_version.h   |  44 +-
  platform/linux-generic/odp_barrier.c   |   3 +-
  platform/linux-generic/odp_buffer.c|   2 +-
  platform/linux-generic/odp_buffer_pool.c   |   8 +-
  platform/linux-generic/odp_coremask.c  |   3 +-
  platform/linux-generic/odp_crypto.c|   9 +-
  platform/linux-generic/odp_packet.c|   4 +-
  platform/linux-generic/odp_packet_flags.c  |   2 +-
  platform/linux-generic/odp_packet_io.c |  11 +-
  platform/linux-generic/odp_packet_socket.c |   2 +-
  platform/linux-generic/odp_queue.c |  10 +-
  platform/linux-generic/odp_ring.c  |   6 +-
  platform/linux-generic/odp_rwlock.c|   3 +-
  platform/linux-generic/odp_schedule.c  |  11 +-
  platform/linux-generic/odp_thread.c|   7 +-
  platform/linux-generic/odp_ticketlock.c|   4 +-
  platform/linux-generic/odp_timer.c |   7 +-
  35 files changed, 849 insertions(+), 519 deletions(-)
  create mode 100644 platform/linux-generic/include/api/odp_platform_defines.h
  create mode 100644 platform/linux-generic/include/api/odp_platform_inlines.h


Having all 'inlines' in one file isn't a good idea. I assume most of
fast path APIs will be there as wrappers to vendor's SDK.
This approach will also face the same issues I've faced in my previous
try.

Current ODP headers contains following parts in each file:

1. Platform-specific
1.a. Handle typedefs: e.g. odp_queue_t
1.b. Macros, enums:   e.g. ODP_SCHED_PRIO_NORMAL

2. Platform-independent
2.a. Data structures, typedefs: e.g. odp_queue_param_t
2.b. Function prototypes:   e.g. odp_queue_create()

Splitting them into platform-specific and platform-independent
parts looks straight forward until one wants to implement some API as
inline but that API needs a platform-independent typedef (2.a).
According to current C standards 'static inline' implementation must be
ahead of its prototype if prototype doesn't have 'static inline'. This
is our case.
If (2.a) and (2.b) bundled together, then there is no way for
implementation to get only typedef (2.a), implement function and then
include API prototype (2.b) to pull-in documentation and verification
of the function prototype.

For now I see two ways to solve it:
1. ODP provides (2.a) and (2.b) as two separate include files, so
   implementation can use them as it wish. E.g. odp_queue_struct.h and
   odp_queue_func.h
2. (2.a) and (2.b) are in the same file, but there is a platform include
   hook between them. Similarly to plat/odp_*.h approach in my last
   series on this topic.

Unfortunately both approaches has its own pros and cons.

___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [RFC PATCH] Platform specific definitions and inlines

[Bill Fischofer]

It's unfortunate that C (and GCC) have these inlining limitations.  This,
BTW, is exactly the sort of thing that LLVM was designed to handle--and
without requiring the programmer to decide in advance which functions
should and should not be inlined.

We could wrapper the independent APIs in macros that the platforms could
override to determine which prototypes should be marked 'static inline'.
Would that work?

On Tue, Nov 4, 2014 at 11:27 AM, Taras Kondratiuk <
taras.kondrat...@linaro.org> wrote:

> On 11/04/2014 04:29 PM, Ciprian Barbu wrote:
>
>> Here is a first attempt to remove inlines from API files. I also moved
>> some
>> defines and typedefs out of some of the API headers. There is still room
>> for
>> improvement I guess, the inline implementation and defines could be split
>> into
>> their own files, but I just tried to show the general idea.
>>
>> I also tried to make the documentation look good, the platform inlines
>> reference
>> the documentation in the corresponding API file.
>>
>> I removed almost all of the individual API includes from C files, keeping
>> only
>> odp.h, which I remember was desirable. This can still be discussed.
>>
>> ---
>>   example/ipsec/odp_ipsec.c  |   2 -
>>   example/ipsec/odp_ipsec_cache.c|   2 -
>>   example/ipsec/odp_ipsec_fwd_db.c   |   1 -
>>   example/ipsec/odp_ipsec_loop_db.c  |   2 -
>>   example/ipsec/odp_ipsec_sa_db.c|   2 -
>>   example/ipsec/odp_ipsec_sp_db.c|   2 -
>>   example/ipsec/odp_ipsec_stream.c   |   3 -
>>   helper/include/odph_eth.h  |   4 -
>>   helper/include/odph_icmp.h |   3 -
>>   helper/include/odph_ip.h   |   3 -
>>   helper/include/odph_ipsec.h|   4 -
>>   helper/include/odph_udp.h  |   3 -
>>   platform/linux-generic/include/api/odp.h   |   6 +
>>   platform/linux-generic/include/api/odp_atomic.h| 282 ---
>>   platform/linux-generic/include/api/odp_byteorder.h | 187 ++--
>>   platform/linux-generic/include/api/odp_coremask.h  |  47 +-
>>   .../include/api/odp_platform_defines.h | 152 ++
>>   .../include/api/odp_platform_inlines.h | 527
>> +
>>   platform/linux-generic/include/api/odp_version.h   |  44 +-
>>   platform/linux-generic/odp_barrier.c   |   3 +-
>>   platform/linux-generic/odp_buffer.c|   2 +-
>>   platform/linux-generic/odp_buffer_pool.c   |   8 +-
>>   platform/linux-generic/odp_coremask.c  |   3 +-
>>   platform/linux-generic/odp_crypto.c|   9 +-
>>   platform/linux-generic/odp_packet.c|   4 +-
>>   platform/linux-generic/odp_packet_flags.c  |   2 +-
>>   platform/linux-generic/odp_packet_io.c |  11 +-
>>   platform/linux-generic/odp_packet_socket.c |   2 +-
>>   platform/linux-generic/odp_queue.c |  10 +-
>>   platform/linux-generic/odp_ring.c  |   6 +-
>>   platform/linux-generic/odp_rwlock.c|   3 +-
>>   platform/linux-generic/odp_schedule.c  |  11 +-
>>   platform/linux-generic/odp_thread.c|   7 +-
>>   platform/linux-generic/odp_ticketlock.c|   4 +-
>>   platform/linux-generic/odp_timer.c |   7 +-
>>   35 files changed, 849 insertions(+), 519 deletions(-)
>>   create mode 100644 platform/linux-generic/include/api/odp_platform_
>> defines.h
>>   create mode 100644 platform/linux-generic/include/api/odp_platform_
>> inlines.h
>>
>
> Having all 'inlines' in one file isn't a good idea. I assume most of
> fast path APIs will be there as wrappers to vendor's SDK.
> This approach will also face the same issues I've faced in my previous
> try.
>
> Current ODP headers contains following parts in each file:
>
> 1. Platform-specific
> 1.a. Handle typedefs: e.g. odp_queue_t
> 1.b. Macros, enums:   e.g. ODP_SCHED_PRIO_NORMAL
>
> 2. Platform-independent
> 2.a. Data structures, typedefs: e.g. odp_queue_param_t
> 2.b. Function prototypes:   e.g. odp_queue_create()
>
> Splitting them into platform-specific and platform-independent
> parts looks straight forward until one wants to implement some API as
> inline but that API needs a platform-independent typedef (2.a).
> According to current C standards 'static inline' implementation must be
> ahead of its prototype if prototype doesn't have 'static inline'. This
> is our case.
> If (2.a) and (2.b) bundled together, then there is no way for
> implementation to get only typedef (2.a), implement function and then
> include API prototype (2.b) to pull-in documentation and verification
> of the function prototype.
>
> For now I see two ways to solve it:
> 1. ODP provides (2.a) and (2.b) as two separate include files, so
>implementation can use them as i

Re: [lng-odp] [PATCH v2 00/18] Split out platform-specific values from header files

[Taras Kondratiuk]

On 11/04/2014 07:24 PM, Bill Fischofer wrote:

These files simply describe the APIs, their parameters, and expected
behaviors.  As such they are implementation-independent.  The only
implementation-specific things in them are the typedef definitions,
which is what we were discussing moving.  The sequence of #includes
should be:

ApplicationImplementation Common ODP API

#include odp.h
#include platform/.../odp_xxx.h
(defines typedefs and inlines)
   #include
common/odp_xxx.h
   (references
typedefs. If typedefs
  missing or inlines
conflict, will
cause compile
error,
indicating a problem in
platform
understanding of API)

Does this structure make sense?


In a reply to Ciprian's RFC I've tried to explain a corner case where 
this structure fails to work.


___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [RFC PATCH] Platform specific definitions and inlines

[Bill Fischofer]

If the ODP prototypes simply all said 'static inline' then would that
permit the implementation to choose later which were actually static inline?

I.e., if the common odp_xxx.h says

static inline void *odp_foo(...);

Then can the implementation can say;

static inline void *odp_foo(...) {  };

to make the function inline and just say

void *odp_foo(...) {  };

to make it an actual call?

On Tue, Nov 4, 2014 at 11:32 AM, Bill Fischofer 
wrote:

> It's unfortunate that C (and GCC) have these inlining limitations.  This,
> BTW, is exactly the sort of thing that LLVM was designed to handle--and
> without requiring the programmer to decide in advance which functions
> should and should not be inlined.
>
> We could wrapper the independent APIs in macros that the platforms could
> override to determine which prototypes should be marked 'static inline'.
> Would that work?
>
> On Tue, Nov 4, 2014 at 11:27 AM, Taras Kondratiuk <
> taras.kondrat...@linaro.org> wrote:
>
>> On 11/04/2014 04:29 PM, Ciprian Barbu wrote:
>>
>>> Here is a first attempt to remove inlines from API files. I also moved
>>> some
>>> defines and typedefs out of some of the API headers. There is still room
>>> for
>>> improvement I guess, the inline implementation and defines could be
>>> split into
>>> their own files, but I just tried to show the general idea.
>>>
>>> I also tried to make the documentation look good, the platform inlines
>>> reference
>>> the documentation in the corresponding API file.
>>>
>>> I removed almost all of the individual API includes from C files,
>>> keeping only
>>> odp.h, which I remember was desirable. This can still be discussed.
>>>
>>> ---
>>>   example/ipsec/odp_ipsec.c  |   2 -
>>>   example/ipsec/odp_ipsec_cache.c|   2 -
>>>   example/ipsec/odp_ipsec_fwd_db.c   |   1 -
>>>   example/ipsec/odp_ipsec_loop_db.c  |   2 -
>>>   example/ipsec/odp_ipsec_sa_db.c|   2 -
>>>   example/ipsec/odp_ipsec_sp_db.c|   2 -
>>>   example/ipsec/odp_ipsec_stream.c   |   3 -
>>>   helper/include/odph_eth.h  |   4 -
>>>   helper/include/odph_icmp.h |   3 -
>>>   helper/include/odph_ip.h   |   3 -
>>>   helper/include/odph_ipsec.h|   4 -
>>>   helper/include/odph_udp.h  |   3 -
>>>   platform/linux-generic/include/api/odp.h   |   6 +
>>>   platform/linux-generic/include/api/odp_atomic.h| 282 ---
>>>   platform/linux-generic/include/api/odp_byteorder.h | 187 ++--
>>>   platform/linux-generic/include/api/odp_coremask.h  |  47 +-
>>>   .../include/api/odp_platform_defines.h | 152 ++
>>>   .../include/api/odp_platform_inlines.h | 527
>>> +
>>>   platform/linux-generic/include/api/odp_version.h   |  44 +-
>>>   platform/linux-generic/odp_barrier.c   |   3 +-
>>>   platform/linux-generic/odp_buffer.c|   2 +-
>>>   platform/linux-generic/odp_buffer_pool.c   |   8 +-
>>>   platform/linux-generic/odp_coremask.c  |   3 +-
>>>   platform/linux-generic/odp_crypto.c|   9 +-
>>>   platform/linux-generic/odp_packet.c|   4 +-
>>>   platform/linux-generic/odp_packet_flags.c  |   2 +-
>>>   platform/linux-generic/odp_packet_io.c |  11 +-
>>>   platform/linux-generic/odp_packet_socket.c |   2 +-
>>>   platform/linux-generic/odp_queue.c |  10 +-
>>>   platform/linux-generic/odp_ring.c  |   6 +-
>>>   platform/linux-generic/odp_rwlock.c|   3 +-
>>>   platform/linux-generic/odp_schedule.c  |  11 +-
>>>   platform/linux-generic/odp_thread.c|   7 +-
>>>   platform/linux-generic/odp_ticketlock.c|   4 +-
>>>   platform/linux-generic/odp_timer.c |   7 +-
>>>   35 files changed, 849 insertions(+), 519 deletions(-)
>>>   create mode 100644 platform/linux-generic/include/api/odp_platform_
>>> defines.h
>>>   create mode 100644 platform/linux-generic/include/api/odp_platform_
>>> inlines.h
>>>
>>
>> Having all 'inlines' in one file isn't a good idea. I assume most of
>> fast path APIs will be there as wrappers to vendor's SDK.
>> This approach will also face the same issues I've faced in my previous
>> try.
>>
>> Current ODP headers contains following parts in each file:
>>
>> 1. Platform-specific
>> 1.a. Handle typedefs: e.g. odp_queue_t
>> 1.b. Macros, enums:   e.g. ODP_SCHED_PRIO_NORMAL
>>
>> 2. Platform-independent
>> 2.a. Data structures, typedefs: e.g. odp_queue_param_t
>> 2.b. Function prototypes:   e.g. odp_queue_create()
>>
>> Splitting them into platform-specific and platform-independent
>> parts looks straight forward until one wants to implement some API as
>> inline but that API needs a platform-independent typedef 

[lng-odp] [PATCH linux-dpdk] Adjust ODP DPDK platform to latest DPDK API (EXPERIMENTAL)

[Zoltan Kiss]

The following commits changed the master branch of DPDK:

http://dpdk.org/browse/dpdk/commit/?id=7869536f
http://dpdk.org/browse/dpdk/commit/?id=ca04aaea
http://dpdk.org/browse/dpdk/commit/?id=08b563ff
http://dpdk.org/browse/dpdk/commit/?id=ea672a8b
http://dpdk.org/browse/dpdk/commit/?id=9aaccf1a
http://dpdk.org/browse/dpdk/commit/?id=e5ffdd14
http://dpdk.org/browse/dpdk/commit/?id=591a9d79

This patch adjust the code to these changes.

Signed-off-by: Zoltan Kiss 
diff --git a/platform/linux-dpdk/odp_buffer.c b/platform/linux-dpdk/odp_buffer.c
index 4914ca2..b586fa2 100644
--- a/platform/linux-dpdk/odp_buffer.c
+++ b/platform/linux-dpdk/odp_buffer.c
@@ -69,8 +69,6 @@ int odp_buffer_snprint(char *str, size_t n, odp_buffer_t buf)
len += snprintf(&str[len], n-len,
"  ref_count%i\n",hdr->mb.refcnt);
len += snprintf(&str[len], n-len,
-   "  dpdk type%i\n",hdr->mb.type);
-   len += snprintf(&str[len], n-len,
"  odp type %i\n",hdr->type);
 
return len;
diff --git a/platform/linux-dpdk/odp_buffer_pool.c 
b/platform/linux-dpdk/odp_buffer_pool.c
index 8325b9e..e2183d6 100644
--- a/platform/linux-dpdk/odp_buffer_pool.c
+++ b/platform/linux-dpdk/odp_buffer_pool.c
@@ -169,18 +169,18 @@ odp_dpdk_mbuf_ctor(struct rte_mempool *mp,
/* keep some headroom between start of buffer and data */
if (mb_ctor_arg->buf_type == ODP_BUFFER_TYPE_PACKET ||
mb_ctor_arg->buf_type == ODP_BUFFER_TYPE_ANY)
-   mb->pkt.data = (char *)mb->buf_addr + RTE_PKTMBUF_HEADROOM;
+   mb->data_off = RTE_PKTMBUF_HEADROOM;
else
-   mb->pkt.data = mb->buf_addr;
+   mb->data_off = 0;
 
/* init some constant fields */
-   mb->type = RTE_MBUF_PKT;
mb->pool = mp;
-   mb->pkt.nb_segs  = 1;
-   mb->pkt.in_port  = 0xff;
+   mb->nb_segs  = 1;
+   mb->port  = 0xff;
mb->ol_flags = 0;
-   mb->pkt.vlan_macip.data = 0;
-   mb->pkt.hash.rss = 0;
+   mb->l2_l3_len = 0;
+   mb->vlan_tci = 0;
+   mb->hash.rss = 0;
 
/* Save index, might be useful for debugging purposes */
buf_hdr = (struct odp_buffer_hdr_t *)raw_mbuf;
@@ -282,5 +282,5 @@ void odp_buffer_free(odp_buffer_t buf)
 
 void odp_buffer_pool_print(odp_buffer_pool_t pool_id)
 {
-   rte_mempool_dump((const struct rte_mempool *)pool_id);
+   rte_mempool_dump(stdout, (const struct rte_mempool *)pool_id);
 }
diff --git a/platform/linux-dpdk/odp_init.c b/platform/linux-dpdk/odp_init.c
index ecc2066..97f8b21 100644
--- a/platform/linux-dpdk/odp_init.c
+++ b/platform/linux-dpdk/odp_init.c
@@ -37,11 +37,6 @@ int odp_init_dpdk(void)
return -1;
}
 
-   if (rte_pmd_init_all() < 0) {
-   ODP_ERR("Cannot init pmd\n");
-   return -1;
-   }
-
if (rte_eal_pci_probe() < 0) {
ODP_ERR("Cannot probe PCI\n");
return -1;
diff --git a/platform/linux-dpdk/odp_packet.c b/platform/linux-dpdk/odp_packet.c
index 1cd190d..0586538 100644
--- a/platform/linux-dpdk/odp_packet.c
+++ b/platform/linux-dpdk/odp_packet.c
@@ -51,14 +51,12 @@ static int odp_packet_set_offset_len(odp_packet_t pkt, 
size_t frame_offset,
 size_t len)
 {
struct rte_mbuf *mb = &(odp_packet_hdr(pkt)->buf_hdr.mb);
-   uint16_t offset;
+   uint16_t offset = mb->data_off;
uint16_t data_len;
 
/* The pkt buf may have been pulled back into the headroom
 * so we cannot rely on finding the data right after the
 * ODP header and HEADROOM */
-   offset = (uint16_t)((unsigned long)mb->pkt.data -
-   (unsigned long)mb->buf_addr);
ODP_ASSERT(mb->buf_len >= offset, "Corrupted mbuf");
data_len = mb->buf_len - offset;
 
@@ -66,14 +64,14 @@ static int odp_packet_set_offset_len(odp_packet_t pkt, 
size_t frame_offset,
ODP_ERR("Frame offset too big");
return -1;
}
-   mb->pkt.data = (void *)((char *)mb->pkt.data + frame_offset);
+   mb->data_off += frame_offset;
data_len -= frame_offset;
 
if (data_len < len) {
ODP_ERR("Packet len too big");
return -1;
}
-   mb->pkt.pkt_len = len;
+   mb->pkt_len = len;
 
return 0;
 }
@@ -86,7 +84,7 @@ void odp_packet_set_len(odp_packet_t pkt, size_t len)
 size_t odp_packet_get_len(odp_packet_t pkt)
 {
struct rte_mbuf *mb = &(odp_packet_hdr(pkt)->buf_hdr.mb);
-   return mb->pkt.pkt_len;
+   return mb->pkt_len;
 }
 
 uint8_t *odp_packet_buf_addr(odp_packet_t pkt)
@@ -97,7 +95,7 @@ uint8_t *odp_packet_buf_addr(odp_packet_t pkt)
 uint8_t *odp_packet_start(odp_packet_t pkt)
 {
struct rte_mbuf *mb = &(odp_packet_hdr(pkt)->buf_hdr.mb);
-   return mb->pkt.data;
+   return ((uint8_t 

Re: [lng-odp] [RFC PATCH] Platform specific definitions and inlines

[Ciprian Barbu]

On Tue, Nov 4, 2014 at 7:27 PM, Taras Kondratiuk
 wrote:
> On 11/04/2014 04:29 PM, Ciprian Barbu wrote:
>>
>> Here is a first attempt to remove inlines from API files. I also moved
>> some
>> defines and typedefs out of some of the API headers. There is still room
>> for
>> improvement I guess, the inline implementation and defines could be split
>> into
>> their own files, but I just tried to show the general idea.
>>
>> I also tried to make the documentation look good, the platform inlines
>> reference
>> the documentation in the corresponding API file.
>>
>> I removed almost all of the individual API includes from C files, keeping
>> only
>> odp.h, which I remember was desirable. This can still be discussed.
>>
>> ---
>>   example/ipsec/odp_ipsec.c  |   2 -
>>   example/ipsec/odp_ipsec_cache.c|   2 -
>>   example/ipsec/odp_ipsec_fwd_db.c   |   1 -
>>   example/ipsec/odp_ipsec_loop_db.c  |   2 -
>>   example/ipsec/odp_ipsec_sa_db.c|   2 -
>>   example/ipsec/odp_ipsec_sp_db.c|   2 -
>>   example/ipsec/odp_ipsec_stream.c   |   3 -
>>   helper/include/odph_eth.h  |   4 -
>>   helper/include/odph_icmp.h |   3 -
>>   helper/include/odph_ip.h   |   3 -
>>   helper/include/odph_ipsec.h|   4 -
>>   helper/include/odph_udp.h  |   3 -
>>   platform/linux-generic/include/api/odp.h   |   6 +
>>   platform/linux-generic/include/api/odp_atomic.h| 282 ---
>>   platform/linux-generic/include/api/odp_byteorder.h | 187 ++--
>>   platform/linux-generic/include/api/odp_coremask.h  |  47 +-
>>   .../include/api/odp_platform_defines.h | 152 ++
>>   .../include/api/odp_platform_inlines.h | 527
>> +
>>   platform/linux-generic/include/api/odp_version.h   |  44 +-
>>   platform/linux-generic/odp_barrier.c   |   3 +-
>>   platform/linux-generic/odp_buffer.c|   2 +-
>>   platform/linux-generic/odp_buffer_pool.c   |   8 +-
>>   platform/linux-generic/odp_coremask.c  |   3 +-
>>   platform/linux-generic/odp_crypto.c|   9 +-
>>   platform/linux-generic/odp_packet.c|   4 +-
>>   platform/linux-generic/odp_packet_flags.c  |   2 +-
>>   platform/linux-generic/odp_packet_io.c |  11 +-
>>   platform/linux-generic/odp_packet_socket.c |   2 +-
>>   platform/linux-generic/odp_queue.c |  10 +-
>>   platform/linux-generic/odp_ring.c  |   6 +-
>>   platform/linux-generic/odp_rwlock.c|   3 +-
>>   platform/linux-generic/odp_schedule.c  |  11 +-
>>   platform/linux-generic/odp_thread.c|   7 +-
>>   platform/linux-generic/odp_ticketlock.c|   4 +-
>>   platform/linux-generic/odp_timer.c |   7 +-
>>   35 files changed, 849 insertions(+), 519 deletions(-)
>>   create mode 100644
>> platform/linux-generic/include/api/odp_platform_defines.h
>>   create mode 100644
>> platform/linux-generic/include/api/odp_platform_inlines.h
>
>
> Having all 'inlines' in one file isn't a good idea. I assume most of
> fast path APIs will be there as wrappers to vendor's SDK.

I already said in the comment that having all inlines in one file is
not the idea, I didn't want to spend a lot of time in making the
perfect solution, only to show the general idea.

> This approach will also face the same issues I've faced in my previous
> try.

Have you tried compiling it? There are no intermediate typedefs and
defines and yet you can compile everything in the form presented. Also
make doxygen-html and have a look at it.

>
> Current ODP headers contains following parts in each file:
>
> 1. Platform-specific
> 1.a. Handle typedefs: e.g. odp_queue_t
> 1.b. Macros, enums:   e.g. ODP_SCHED_PRIO_NORMAL
>
> 2. Platform-independent
> 2.a. Data structures, typedefs: e.g. odp_queue_param_t
> 2.b. Function prototypes:   e.g. odp_queue_create()
>
> Splitting them into platform-specific and platform-independent
> parts looks straight forward until one wants to implement some API as
> inline but that API needs a platform-independent typedef (2.a).
> According to current C standards 'static inline' implementation must be
> ahead of its prototype if prototype doesn't have 'static inline'. This
> is our case.

This is solved in this patch by including odp_platform_defines.h and
odp_platform_inlines.h in odp.h first before anything else. All the
source files include only odp.h so the order is always preserved.

> If (2.a) and (2.b) bundled together, then there is no way for
> implementation to get only typedef (2.a), implement function and then
> include API prototype (2.b) to pull-in documentation and verification
> of the function prototype.
>
> For now I see two 

Re: [lng-odp] [PATCH v2 00/18] Split out platform-specific values from header files

[Bill Fischofer]

Yes, I saw that and agree. Suggest we leave things as is for ODP v1.0 since
it's not a production target and take this as a follow-on work item for
next year to figure out an optimal solution for this problem.

Given the typedef presence this means that other platforms will copy the
linux-generic API files into their own API directory and replace the
typedefs with their own.  Perhaps highlighting the replacement area with a
row of asterisks or something would make that an easier process to eyeball.

Bill

On Tue, Nov 4, 2014 at 11:31 AM, Taras Kondratiuk <
taras.kondrat...@linaro.org> wrote:

> On 11/04/2014 07:24 PM, Bill Fischofer wrote:
>
>> These files simply describe the APIs, their parameters, and expected
>> behaviors.  As such they are implementation-independent.  The only
>> implementation-specific things in them are the typedef definitions,
>> which is what we were discussing moving.  The sequence of #includes
>> should be:
>>
>> ApplicationImplementation Common ODP API
>>
>> #include odp.h
>> #include platform/.../odp_xxx.h
>> (defines typedefs and inlines)
>>#include
>> common/odp_xxx.h
>>(references
>> typedefs. If typedefs
>>   missing or inlines
>> conflict, will
>> cause compile
>> error,
>> indicating a problem in
>> platform
>> understanding of API)
>>
>> Does this structure make sense?
>>
>
> In a reply to Ciprian's RFC I've tried to explain a corner case where this
> structure fails to work.
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/lng-odp