[lng-odp] shmem in process mode.

[Christophe Milard]

Hi,

This is partly an answer to Gary's mail included below. it is about shmem
allocation in process mode. I don't think I really agree with you, Gary: As
Ola, I believe it is important (for performance) that shmem areas are seen
at the same address in their virtual space on different ODP threads, even
if the latter happens to be implemented as linux processes.

I do see a solution for doing this but it implies redefining ODP threads as
we (Gary and myself)  originially wanted them to be:

"in linux, an odp thread is any thread or process *descendant of the ODP
initiation process* which has called odp_init_local()" (def 1)

This definition was rejected (mostly by Petri) who wanted to define an ODP
thread as: "in linux, an odp thread is any thread or process which has
called odp_init_local()" (def 2). This was accepted as the definition, but
I still believe we should go for def 1 instead.

Indeed, if the main ODP instantiation process performs a huge virtual space
reservation, e.g something like:

shmem_base = mmap(NULL, SHM_TOT_SZ, PROT_NONE, MAP_ANONYMOUS,
FLAG_NO_RESERVE);
madvise(shmem_base, SHM_TOT_SZ, MADV_DONTNEED)

where SHM_TOT is the grand max of the sum of all possible shmem areas, then
this area will be mapped in all processes and threads descendant from the
odp instantiation process, ie all odpthread if def 1 is accepted.
Note than the only goal of these two system calls is to reserve virtual
space, not memory. I have an open question with Barry to see if there are
better way to do so.
But my point is that, if we accept definition 1 instead of def 2, we can
pre-reserve a huge virtual address space in the instantiation process and
know for sure that this will be inherited by all odp threads (threads as
processes).

shmem_base to shmem_base + SHM_TOT_SZ can now be handled as the address
range where any odp shared memory will be mmapped: we need a algorythm to
retrieve addresses withing this range, e.g. two ODP internal functions:
void* _odp_shm_space_alloc(int size)
_odp_shm_space_free(void *ptr)
which retrieve address within this range and handle defragmentation.

Then, when a odp_shm_reserve(area1_sz) is performed, the real memory can be
allocaded and mmap with the MAP_FIXED flag to the address returned by
 _odp_shm_space_alloc(area1_sz).
Any other odp thread performng a odp_shm_lookup() and odp_shm_get_addr() on
an existing handle would then mmap it on the same existing address using
the MAP_FIXED flag.
Note that we MAP_FIXED is used, the kernel replaces any existing mapping
with the new one, which is exactely the behaviour we want here.

An odp_free would of course call _odp_shm_space_free(), returning the
address range to the "pool" and remapping it with the NO_RESERVE flag or
whatever better way to tell the kernel that this is just a virtual space
reservation.

As far as I can see, there is a huge performance gain to be able to
guarantee that a given shm area maps at the same address in any odp thread.
Accepting def 1 gives this possibility. If not, I cannot see how 2
unrelated processes could be guarantedd to have the same address range
available at any time within their respectice address space...

Christophe.


The mail received from Gary:

I had intended to try some patches on the ODP Linux-generic shmem
implementation in order to allow child processes to access shared memory as
reliably as well as pthreads do now.  Essentially the problem is that while
pthreads share a single memory space with all threads within their thread
group(except for thread-local storage, that is) - forked processes have
unique memory spaces and shared memory or memory-mapped files CANNOT BE
GUARANTEED to be mapped at the same address from one process to the next.
Hence instances of both of these memory areas must be referenced between
processes via the use of shared memory object names known to all processes
in question.  Each process then binds that shared memory object to an
address unique to its local address space - and elements within the shared
memory / file space are referenced via known offsets from the local base
address.  I do not believe there is any reasonable means for the kernel to
guarantee matching virtual addresses across processes for memory-mapped
system-scoped objects... it would necessitate dynamic remapping of the rest
of a given process's virtual address space when a shared object was mapped
in.  What a nightmare that could be!

In the case of ODP there seems to be a "master catalog" of shared memory
created by the parent ODP instance - and the addresses of shared memory
objects reserved by ODP threads belonging to that instance are stored in
this 'catalog' and then later referenced by the children.  Neither the
address of the 'master catalog' itself nor the addresses of the reserved
areas listed within the 'catalog' may be used reliably from forked child
processes.  Forked processes will have to reference all of these shared
memory objects via filenames  - so the 'master catalog' itself

[lng-odp] [PATCH] validation: lock: tuning the iteration number

[Christophe Milard]

fixing: https://bugs.linaro.org/show_bug.cgi?id=2108

The no_lock_functional_test does not really tests the ODP functionality:
Instead, it actually checks that race conditions can be created between
concurrent running threads (by making these threads writing shared
variables without lock, and later noting the value written was changed by
some of the concurrent threads).
This test therefore validates other tests: if this test passes -i.e. if
we do have race condition- then if the following tests suppress these
race conditions by using some synchronization mechanism, these
synchronization mechanisms can be said to be efficient.
If, on the other hand, the no_lock_functional_test "fails", it says
that the following tests are really inconclusive as the effect of the
tested synchronization mechanism is not proven.

When running with valgrind, no_lock_functional_test failed, probably
because the extra execution time introduced by valgrind itself made
the chance to run the critical section of the different threads "at
the same time" much less probable.

The simple solution is to increase the critical section running time
(by largely increasing the number of iterations performed).
The solution taken here is actually to tune the critical section running
time (currentely to ITER_MPLY_FACTOR=3 times the time needed to note
the first race condition).
This means that the test will take longer to run with valgrind,
but will remain short without valgrind.

Signed-off-by: Christophe Milard 
---
 test/validation/lock/lock.c | 71 -
 1 file changed, 64 insertions(+), 7 deletions(-)

diff --git a/test/validation/lock/lock.c b/test/validation/lock/lock.c
index f1f6d69..515bc77 100644
--- a/test/validation/lock/lock.c
+++ b/test/validation/lock/lock.c
@@ -12,7 +12,10 @@
 #include "lock.h"
 
 #define VERBOSE0
-#define MAX_ITERATIONS 1000
+
+#define MIN_ITERATIONS 1000
+#define MAX_ITERATIONS 3
+#define ITER_MPLY_FACTOR   3
 
 #define SLOW_BARRIER_DELAY 400
 #define BASE_DELAY 6
@@ -325,6 +328,12 @@ static void *rwlock_recursive_api_tests(void *arg UNUSED)
return NULL;
 }
 
+/*
+ * Tests that we do have contention between threads when running.
+ * Also adjust the number of iterations to be done (by other tests)
+ * so we have a fair chance to see that the tested synchronizer
+ * does avoid the race condition.
+ */
 static void *no_lock_functional_test(void *arg UNUSED)
 {
global_shared_mem_t *global_mem;
@@ -335,17 +344,36 @@ static void *no_lock_functional_test(void *arg UNUSED)
thread_num = odp_cpu_id() + 1;
per_thread_mem = thread_init();
global_mem = per_thread_mem->global_mem;
-   iterations = global_mem->g_iterations;
+   iterations = 0;
 
odp_barrier_wait(&global_mem->global_barrier);
 
sync_failures = 0;
current_errs = 0;
rs_idx = 0;
-   resync_cnt = iterations / NUM_RESYNC_BARRIERS;
+   resync_cnt = MAX_ITERATIONS / NUM_RESYNC_BARRIERS;
lock_owner_delay = BASE_DELAY;
 
-   for (cnt = 1; cnt <= iterations; cnt++) {
+   /*
+   * Tunning the iteration number:
+   * Here, we search for an iteration number that guarantees to show
+   * race conditions between the odp threads.
+   * Iterations is set to ITER_MPLY_FACTOR * cnt where cnt is when
+   * the threads start to see "errors" (i.e. effect of other threads
+   * running concurrentely without any synchronisation mechanism).
+   * In other words, "iterations" is set to ITER_MPLY_FACTOR times the
+   * minimum loop count necessary to see a need for synchronisation
+   * mechanism.
+   * If, later, these "errors" disappear when running other tests up to
+   * "iterations" with synchro, the effect of the tested synchro mechanism
+   * is likely proven.
+   * If we reach "MAX_ITERATIONS", and "iteration" remains zero,
+   * it means that we cannot see any race condition between the different
+   * running theads (e.g. the OS is not preemptive) and all other tests
+   * being passed won't tell much about the functionality of the
+   * tested synchro mechanism.
+   */
+   for (cnt = 1; cnt <=  MAX_ITERATIONS; cnt++) {
global_mem->global_lock_owner = thread_num;
odp_mb_full();
thread_delay(per_thread_mem, lock_owner_delay);
@@ -353,6 +381,8 @@ static void *no_lock_functional_test(void *arg UNUSED)
if (global_mem->global_lock_owner != thread_num) {
current_errs++;
sync_failures++;
+   if (!iterations)
+   iterations = cnt;
}
 
global_mem->global_lock_owner = 0;
@@ -362,6 +392,

[lng-odp] [GIT PULL ODPv2] running things in process mode

[Christophe Milard]

Since v1:
-variable declaration gathered in function 's head (Petri)
-linux prefix removed from helper's types and function names (Mike, Christophe)

Hi,

This patch series adds the ability to run tests/ exemples / perf-test
in "process mode" (i.e letting OPD threads being linux processes)
It it hence tackling ODP-171.

This is achieved in 2 main steps:

A]
The 2 pairs of helper functions:
odph_linux_pthread_create(), odph_linux_pthread_join()
and
odph_linux_process_fork_n(), odph_linux_process_wait_n()
are replaced by:
odph_linux_odpthreads_create() and odph_linux_odpthreads_join()
The latter's callers are unaware of the actual implementation of the ODP
thread (making test truly platform agnostic).
The helper functions decide at run time whether an odp thread is to be
a linux process or pthread based on the command line argument.

B] each test/example now calls a new helper function,
odph_linux_parse_options(), so that the helper can get its own arguments
out of the command line.
Currentely supported args are: --odph_proc, --odph_thread.
Defaults assumes thread. specifying both options runs in mixed mode.

The changed are first done on the shmem tests, and thereafter propagated
to other helper users.
Note that this patch series enable the option but does not affect
make check at this time: make check still calls the tests with no options
which default to thread mode.

This patch series nicely splits it two groups (you can split your review
there):
1) up to "validation: pktio: adding command line argument parsing", the new
helper functions are introduced, and used in the validation tests.
2) from "helper: adding a function to merge getopt parameters" the ability
to parse command line arguments in subset in added and applied to
the example and performance tests.

Hope this makes sence for you too!




The following changes since commit 69d0a401c92807852b503418612812242ba9f8f2:

  configure: default ODP_DEBUG to false (2016-04-20 23:00:59 +0300)

are available in the git repository at:

  https://git.linaro.org/people/christophe.milard/odp.git 
test_in_process_mode_v2

for you to fetch changes up to 53a81edc1c77dc7aac08d97cd28e8cc5f7d9d842:

  helper: removing dead code (2016-04-21 17:26:26 +0200)

--------
Christophe Milard (56):
  helpers: adding command line argument parsing
  validation: common: adding command line argument parsing
  validation: shmem: adding command line argument parsing
  helpers: linux: creating common entry for process and thread
  helpers: linux: creating functions to handle odpthreads
  helper: test: adding odpthread functions tests
  validation: using implementation agnostic function for ODP threads
  validation: traffic_mngr: adding command line argument parsing
  validation: timer: adding command line argument parsing
  validation: time: adding command line argument parsing
  validation: thread: adding command line argument parsing
  validation: system: adding command line argument parsing
  validation: std_clib: adding command line argument parsing
  validation: scheduler: adding command line argument parsing
  validation: random: adding command line argument parsing
  validation: queue: adding command line argument parsing
  validation: pool: adding command line argument parsing
  validation: packet: adding command line argument parsing
  validation: lock: adding command line argument parsing
  validation: init: adding command line argument parsing
  validation: hash: adding command line argument parsing
  validation: errno: adding command line argument parsing
  validation: crypto: adding command line argument parsing
  validation: cpumask: adding command line argument parsing
  validation: config: adding command line argument parsing
  validation: classification: adding command line argument parsing
  validation: buffer: adding command line argument parsing
  validation: barrier: adding command line argument parsing
  validation: atomic: adding command line argument parsing
  validation: pktio: adding command line argument parsing
  helper: adding a function to merge getopt parameters
  helper: parsing the complete set of options
  performance: odp_scheduling: proc mode done by helper
  performance: odp_pktio_perf: using agnostic function for ODP threads
  performance: odp_pktio_perf: adding helper cmd line parsing
  performance: odp_l2fwd: using agnostic function for ODP threads
  performance: odp_l2fwd: adding helper cmd line parsing
  performance: crypto: using agnostic function for ODP threads
  performance: crypto: adding helper cmd line parsing
  example: classifier: using agnostic function for ODP threads
  example: classifier: adding helper cmd line parsing
  example: generator: using agnostic func

Re: [lng-odp] shmem in process mode.

[Christophe Milard]

On 21 April 2016 at 18:30, Gary S. Robertson 
wrote:

>
>
> On 04/21/2016 02:03 AM, Christophe Milard wrote:
>
> Hi,
> Indeed, if the main ODP instantiation process performs a huge virtual
> space reservation, e.g something like:
>
> shmem_base = mmap(NULL, SHM_TOT_SZ, PROT_NONE, MAP_ANONYMOUS,
> FLAG_NO_RESERVE);
> madvise(shmem_base, SHM_TOT_SZ, MADV_DONTNEED)
>
> where SHM_TOT is the grand max of the sum of all possible shmem areas,
> then this area will be mapped in all processes and threads descendant from
> the odp instantiation process, ie all odpthread if def 1 is accepted.
>
> This only holds true until the forked child process replaces its memory
> space with a different executable image than that of its parent - eg. via
> exec*().  At that time the memory space of the child is remapped to the
> replacement image and any shared memory mapping inherited from the parent
> is subject to being remapped.  I am not aware of any guarantees that shared
> memory mapped by the parent with a 'MAP_FIXED' attribute will be remapped
> at the same address in the new image.  I am not certain that such a
> guarantee is even possible given the unknown and unconstrained memory
> footprint of the replacement image.
>
> I believe that to be safe and portable we cannot rely upon shared memory
> or memory mapped files to be consistently addressable at the same location
> across processes - descendant or otherwise.  Without a guarantee of such
> consistency in the POSIX specifications, even if it works today it may be
> broken by future code changes in the kernel or the loader.
>
>
Of course, you are right here: But is an ODP thread performing an exec*()
still an ODP thread?
Once again, it depends on the definition of what an ODP thread is! I
assumed that calling exec*() would definitively disqualify the process for
remaining an ODP thread. After exec(), most of what odp_*_init() could have
done will be wiped out (apart from opened file descriptors)
As far as I understood, the intention of supporting process mode was to
perform concurrent processing without enforcing the need of common shared
global memory between the different ODP threads. Hence systems with NUMA
could use local memory for different processes and just use some limited
amount of shared memory for the stuff that really needs to be shared.

So maybe, the definition of an ODP process should be:
"any process descendant from the ODP main instantiation process, which has
called odp_local_init(), and not called any if the exec*() system call".


Regarding the performance aspect, other than additional overhead in initial
> setup and teardown/cleanup I don't see a performance difference between
> base addresses obtained from a "global" shared data structure and a
> process-local one.  Since the process-local addess table avoids the risky
> behaviour of relying on consistent mapping across processes it seems to me
> the obvious way to go.
>
> If both the "shared global" object reference/naming table and
> process-local object mapping tables are "indexed" by a platform-agnostic
> handle for each shared memory object then I believe that performance of
> access to the shared memory objects will be a "wash".  This would be the
> most portable and "best practice" implementation - which I believe is also
> an important consideration for a reference platform.
>
> I is clear for me that an application being able to use the same set of
pointers over its different threads/process will gain in performance:
Assuming the contents of the shared memory contains some complex structure,
it is likely it will be referenced by pointers within some code, and
possibly these pointers will need to be shared as well. It seems reasonable
to believe that sharing the same set of pointers (and being able to pass
then between ODP threads) will be performing better.

And last but not least: thanks for your comments, Gary!

Christophe

Gary R.
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCH] validation: crypto: verify odp crypto capability

[Christophe Milard]

On 22 April 2016 at 16:14, Balakrishna Garapati <
balakrishna.garap...@linaro.org> wrote:

> test update to verify the crypto capability fucntionality
>
> Signed-off-by: Balakrishna Garapati 
> ---
>  test/validation/crypto/odp_crypto_test_inp.c | 28
> 
>  1 file changed, 28 insertions(+)
>
> diff --git a/test/validation/crypto/odp_crypto_test_inp.c
> b/test/validation/crypto/odp_crypto_test_inp.c
> index fc125f7..b608a76 100644
> --- a/test/validation/crypto/odp_crypto_test_inp.c
> +++ b/test/validation/crypto/odp_crypto_test_inp.c
> @@ -46,6 +46,7 @@ static void alg_test(odp_crypto_op_t op,
>  )
>  {
> odp_crypto_session_t session;
> +   odp_crypto_capability_t capability;
> int rc;
> odp_crypto_ses_create_err_t status;
> odp_bool_t posted;
> @@ -53,6 +54,33 @@ static void alg_test(odp_crypto_op_t op,
> odp_crypto_compl_t compl_event;
> odp_crypto_op_result_t result;
>
> +   /* Initialize crypto capability structure */
> +   memset(&capability, 0, sizeof(odp_crypto_capability_t));
> +
> +   rc = odp_crypto_capability(&capability);
> +   CU_ASSERT(!rc);
> +
> +   if (cipher_alg == ODP_CIPHER_ALG_3DES_CBC &&
> +   !(capability.ciphers.bit.trides_cbc))
> +   rc = -1;
> +   if (cipher_alg == ODP_CIPHER_ALG_AES128_CBC &&
> +   !(capability.ciphers.bit.trides_cbc))
> +   rc = -1;
> +   if (cipher_alg == ODP_CIPHER_ALG_AES128_GCM &&
> +   !(capability.ciphers.bit.trides_cbc))
> +   rc = -1;
>

You are testing 3 times the same bit here... copy/paster error?

Christophe


> +
> +   CU_ASSERT(!rc);
> +
> +   if (auth_alg == ODP_AUTH_ALG_AES128_GCM &&
> +   !(capability.auths.bit.aes128_gcm))
> +   rc = -1;
> +   if (auth_alg == ODP_AUTH_ALG_NULL &&
> +   !(capability.auths.bit.null))
> +   rc = -1;
> +
> +   CU_ASSERT(!rc);
> +
> /* Create a crypto session */
> odp_crypto_session_params_t ses_params;
> memset(&ses_params, 0, sizeof(ses_params));
> --
> 1.9.1
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [GIT PULL ODPv3] running things in process mode

[Christophe Milard]

Since v2:
-serious rebase following clash with dba05a28 (api: init: add instance handle)
-squashing the validation changes. I am not very keen on squashing
 small isolated modifs to larger patches. don't really see the gain.
 Squashing more will put even more unrelated things together.

Since v1:
-variable declaration gathered in function 's head (Petri)
-linux prefix removed from helper's types and function names (Mike, Christophe)

Hi,

This patch series adds the ability to run tests/ exemples / perf-test
in "process mode" (i.e letting OPD threads being linux processes)
It it hence tackling ODP-171.

This is achieved in 2 main steps:

A]
The 2 pairs of helper functions:
odph_linux_pthread_create(), odph_linux_pthread_join()
and
odph_linux_process_fork_n(), odph_linux_process_wait_n()
are replaced by:
odph_linux_odpthreads_create() and odph_linux_odpthreads_join()
The latter's callers are unaware of the actual implementation of the ODP
thread (making test truly platform agnostic).
The helper functions decide at run time whether an odp thread is to be
a linux process or pthread based on the command line argument.

B] each test/example now calls a new helper function,
odph_linux_parse_options(), so that the helper can get its own arguments
out of the command line.
Currentely supported args are: --odph_proc, --odph_thread.
Defaults assumes thread. specifying both options runs in mixed mode.

The changed are first done on the shmem tests, and thereafter propagated
to other helper users.
Note that this patch series enable the option but does not affect
make check at this time: make check still calls the tests with no options
which default to thread mode.

This patch series nicely splits it two groups (you can split your review
there):
1) up to "validation: pktio: adding command line argument parsing", the new
helper functions are introduced, and used in the validation tests.
2) from "helper: adding a function to merge getopt parameters" the ability
to parse command line arguments in subset in added and applied to
the example and performance tests.

Hope this makes sence for you too!



The following changes since commit 577eb779447b889e6d34e7732576e474f47e7c70:

  example: packet make check tests (2016-04-26 15:47:24 +0300)

are available in the git repository at:

  https://git.linaro.org/people/christophe.milard/odp.git 
test_in_process_mode_v3

for you to fetch changes up to 71e38c2d49b4e4b7649ddebcb69fb680f46139cc:

  helper: removing dead code (2016-04-27 14:11:58 +0200)

--------
Christophe Milard (36):
  helpers: adding command line argument parsing
  validation: common: adding command line argument parsing
  validation: shmem: adding command line argument parsing
  helpers: linux: creating common entry for process and thread
  helpers: linux: creating functions to handle odpthreads
  helper: test: adding odpthread functions tests
  validation: using implementation agnostic function for ODP threads
  validation: most tests: adding command line argument parsing
  validation: init: adding command line argument parsing
  validation: pktio: adding command line argument parsing
  helper: adding a function to merge getopt parameters
  helper: parsing the complete set of options
  performance: odp_scheduling: proc mode done by helper
  performance: odp_pktio_perf: using agnostic function for ODP threads
  performance: odp_pktio_perf: adding helper cmd line parsing
  performance: odp_l2fwd: using agnostic function for ODP threads
  performance: odp_l2fwd: adding helper cmd line parsing
  performance: crypto: using agnostic function for ODP threads
  performance: crypto: adding helper cmd line parsing
  example: classifier: using agnostic function for ODP threads
  example: classifier: adding helper cmd line parsing
  example: generator: using agnostic function for ODP threads
  example: generator: adding helper cmd line parsing
  example: ipsec: using agnostic function for ODP threads
  example: ipsec: adding helper cmd line parsing
  example: l2fwd_simple: using agnostic function for ODP threads
  example: l2fwd_simple: adding helper cmd line parsing
  example: pktio: using agnostic function for ODP threads
  example: pktio: adding helper cmd line parsing
  example: time: using agnostic function for ODP threads
  example: time: adding helper cmd line parsing
  example: timer: using agnostic function for ODP threads
  example: timer: adding helper cmd line parsing
  example: switch: using agnostic function for ODP threads
  example: switch: adding helper cmd line parsing
  helper: removing dead code

 example/classifier/odp_classifier.c|  39 +-
 example/generator/odp_generator.c  |  45 ++-
 example/ipsec/odp_ipsec.c  

Re: [lng-odp] [PATCH v2] DEPENDENCIES: add codespell for checkpatch

[Christophe Milard]

How come your v2 just shows the diff compared to v1? and not the complete
patch again?

Has  the first patch been merged already? (looks like your v2 has to be
applied in v1)

other than that, for the patch contents, you can apply my reviewed by on
the fix (Reviewed-by: Christophe Milard  ):

Christophe.

On 27 April 2016 at 22:20, Mike Holmes  wrote:

> Signed-off-by: Mike Holmes 
> ---
> v2
>codespell does not check the spelling of checkpatch :)
>
>
>  DEPENDENCIES | 7 +++
>  1 file changed, 7 insertions(+)
>
> diff --git a/DEPENDENCIES b/DEPENDENCIES
> index c83fc36..f8e18e2 100644
> --- a/DEPENDENCIES
> +++ b/DEPENDENCIES
> @@ -253,3 +253,10 @@ The tested version of doxygen is 1.8.8
>  5.2.1 HTML
> # Debian/Ubuntu
> $ apt-get install asciidoc source-highlight librsvg2-bin
> +
> +6.0 Submitting patches
> +
> +   When submitting patches they should be checked with ./scripts/
> checkpatch.pl
> +   To have this tool also check spelling you need codespell.
> +   # Debian/Ubuntu
> +   #sudo apt install codespell
> --
> 2.7.4
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCH 1/2] doc: support ascidoctor

[Christophe Milard]

why are these patches submitted against api-next? they are not related to
any api changes as I can see?
(you are the one that know these stuffs: I guess you have a good reason
that I am just missing...:-) )

Christophe

On 27 April 2016 at 23:59, Mike Holmes  wrote:

> ascidoctor is a python asciidoc interpreter it has greater capabilities
> than asciidoc which is a perl based interpreter
>
> The resulting style sheet improvements result in more professional
> looking docs that can be further enhanced with our own css at some
> point.
>
> This also supports including code snippets in the documentation from the
> doxygen specification  files.
>
> Signed-off-by: Mike Holmes 
> ---
>  DEPENDENCIES|  2 +-
>  doc/Makefile.inc|  4 ++--
>  doc/users-guide/users-guide-tm.adoc | 18 +-
>  3 files changed, 12 insertions(+), 12 deletions(-)
>
> diff --git a/DEPENDENCIES b/DEPENDENCIES
> index c83fc36..678b62a 100644
> --- a/DEPENDENCIES
> +++ b/DEPENDENCIES
> @@ -252,4 +252,4 @@ The tested version of doxygen is 1.8.8
>
>  5.2.1 HTML
> # Debian/Ubuntu
> -   $ apt-get install asciidoc source-highlight librsvg2-bin
> +   $ apt-get install asciidoctor source-highlight librsvg2-bin
> diff --git a/doc/Makefile.inc b/doc/Makefile.inc
> index c0b641e..643b1d4 100644
> --- a/doc/Makefile.inc
> +++ b/doc/Makefile.inc
> @@ -7,6 +7,6 @@ VPATH=$(top_builddir)/doc/images
> dot -T svg $^ -o $@
>
>  .adoc.html:
> -   asciidoc $(ASCIIDOC_FLAGS) --out-file=$@ $<
> +   asciidoctor $(ASCIIDOC_FLAGS) --out-file=$@ $<
>
> -ASCIIDOC_FLAGS =-a data-uri -b html5  -a icons -a toc2  -a max-width=55em
> +ASCIIDOC_FLAGS =-a data-uri -b html5 -a icons=font -a toc2
> diff --git a/doc/users-guide/users-guide-tm.adoc
> b/doc/users-guide/users-guide-tm.adoc
> index 132fdc1..5dc2190 100644
> --- a/doc/users-guide/users-guide-tm.adoc
> +++ b/doc/users-guide/users-guide-tm.adoc
> @@ -269,7 +269,7 @@ result in faster operation and/or less memory used.
>  [source,c]
>  
>
> -   odp_tm_params_init(&tm_params); /* <1> */
> +   odp_tm_params_init(&tm_params); // <1>
> tm_params.pktio = egress_pktio;
> tm = odp_tm_create(“Example TM”, &tm_params);
>
> @@ -285,11 +285,11 @@ result in faster operation and/or less memory used.
> tmq_B2 = odp_tm_queue_create(tm, &queue_params);
> tmq_C2 = odp_tm_queue_create(tm, &queue_params);
>
> -   odp_tm_node_params_init(&node_params); /* <2> */
> +   odp_tm_node_params_init(&node_params); // <2>
> node_params.level = 1;
> tm_node_1 = odp_tm_node_create(tm, “TmNode1”, &node_params);
>
> -   odp_tm_queue_connect(tmq_A1, tm_node_1); /* <3> */
> +   odp_tm_queue_connect(tmq_A1, tm_node_1); // <3>
> odp_tm_queue_connect(tmq_B1, tm_node_1);
> odp_tm_queue_connect(tmq_A2, tm_node_1);
> odp_tm_queue_connect(tmq_B2, tm_node_1);
> @@ -302,7 +302,7 @@ code does is create a scheduler PROFILE, which is
> effectively a registered set
>  of common scheduler parameters.  NOTE that this uses some pseudocode below
>  instead of real C code so as to be more concise. */
>
> -   odp_tm_sched_params_init(&sched_params); /* <4> */
> +   odp_tm_sched_params_init(&sched_params); // <4>
> sched_params.sched_modes = { ODP_TM_FRAME_BASED_WEIGHTS, … };
> sched_params.sched_weights = { 8, 8, 8,  … };
> sched_profile_RR = odp_tm_sched_create(“SchedProfileRR”,
> &sched_params);
> @@ -311,24 +311,24 @@ instead of real C code so as to be more concise. */
> sched_params.sched_weights = { 8, 8, 8, … };
> sched_profile_FQ = odp_tm_sched_create(“SchedProfileFQ”,
> &sched_params);
>
> -   odp_tm_queue_sched_config(tm_node_1, tmq_A1, sched_profile_RR); /* <5>
> */
> +   odp_tm_queue_sched_config(tm_node_1, tmq_A1, sched_profile_RR); // <5>
> odp_tm_queue_sched_config(tm_node_1, tmq_B1, sched_profile_RR);
> odp_tm_queue_sched_config(tm_node_1, tmq_A2, sched_profile_FQ);
> odp_tm_queue_sched_config(tm_node_1, tmq_B2, sched_profile_FQ);
> odp_tm_queue_sched_config(tm_node_1, tmq_C2, sched_profile_FQ);
>
> -   odp_tm_node_params_init(&node_params); /* <6> */
> +   odp_tm_node_params_init(&node_params); // <6>
> node_params.level = 2;
> tm_node_2 = odp_tm_node_create(tm, “TmNode2”, &node_params);
>
> -   odp_tm_node_connect(tm_node_1, tm_node_2); /* <7> */
> +   odp_tm_node_connect(tm_node_1, tm_node_2); // <7>
>
> -   odp_tm_sched_params_init(&sched_params); /* <8> */
> +   odp_tm_sched_params_init(&sched_params); // <8>
> sched_params.sched_modes = { ODP_TM_BYTE_BASED_WEIGHTS, … };
> sched_params.sched_weights = { 8, 16, 24,  … };
> sched_profile_WFQ = odp_tm_sched_create(“SchedProfileWFQ”,
> &sched_params);
>
> -   odp_tm_node_sched_config(tm_node_2, tm_node_1, sched_profile_WFQ); /*
> <9> */
> +   odp_tm_node_sched_config(tm_node_2, tm_node_1, sched_profile_WFQ); //
> <9>
>  
>
>  <1> Create a tm system, since that is a precursor to creating tm_queues.
> --
> 2.7.4
>
> ___

Re: [lng-odp] [PATCH v2] DEPENDENCIES: add codespell for checkpatch

[Christophe Milard]

Got cheated by google mail! this is fine!
Reviewed-by: Christophe Milard 

On 28 April 2016 at 07:52, Christophe Milard 
wrote:

> How come your v2 just shows the diff compared to v1? and not the complete
> patch again?
>
> Has  the first patch been merged already? (looks like your v2 has to be
> applied in v1)
>
> other than that, for the patch contents, you can apply my reviewed by on
> the fix (Reviewed-by: Christophe Milard  ):
>
> Christophe.
>
> On 27 April 2016 at 22:20, Mike Holmes  wrote:
>
>> Signed-off-by: Mike Holmes 
>> ---
>> v2
>>codespell does not check the spelling of checkpatch :)
>>
>>
>>  DEPENDENCIES | 7 +++
>>  1 file changed, 7 insertions(+)
>>
>> diff --git a/DEPENDENCIES b/DEPENDENCIES
>> index c83fc36..f8e18e2 100644
>> --- a/DEPENDENCIES
>> +++ b/DEPENDENCIES
>> @@ -253,3 +253,10 @@ The tested version of doxygen is 1.8.8
>>  5.2.1 HTML
>> # Debian/Ubuntu
>> $ apt-get install asciidoc source-highlight librsvg2-bin
>> +
>> +6.0 Submitting patches
>> +
>> +   When submitting patches they should be checked with ./scripts/
>> checkpatch.pl
>> +   To have this tool also check spelling you need codespell.
>> +   # Debian/Ubuntu
>> +   #sudo apt install codespell
>> --
>> 2.7.4
>>
>> ___
>> lng-odp mailing list
>> lng-odp@lists.linaro.org
>> https://lists.linaro.org/mailman/listinfo/lng-odp
>>
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [GIT PULL ODPv4] running things in process mode

[Christophe Milard]

Since v3:
-fixed rebase error (Christophe)
-rebased

Since v2:
-serious rebase following clash with dba05a28 (api: init: add instance handle)
-squashing the validation changes. I am not very keen on squashing
 small isolated modifs to larger patches. don't really see the gain.
 Squashing more will put even more unrelated things together.

Since v1:
-variable declaration gathered in function 's head (Petri)
-linux prefix removed from helper's types and function names (Mike, Christophe)

Hi,

This patch series adds the ability to run tests/ exemples / perf-test
in "process mode" (i.e letting OPD threads being linux processes)
It it hence tackling ODP-171.

This is achieved in 2 main steps:

A]
The 2 pairs of helper functions:
odph_linux_pthread_create(), odph_linux_pthread_join()
and
odph_linux_process_fork_n(), odph_linux_process_wait_n()
are replaced by:
odph_linux_odpthreads_create() and odph_linux_odpthreads_join()
The latter's callers are unaware of the actual implementation of the ODP
thread (making test truly platform agnostic).
The helper functions decide at run time whether an odp thread is to be
a linux process or pthread based on the command line argument.

B] each test/example now calls a new helper function,
odph_linux_parse_options(), so that the helper can get its own arguments
out of the command line.
Currentely supported args are: --odph_proc, --odph_thread.
Defaults assumes thread. specifying both options runs in mixed mode.

The changed are first done on the shmem tests, and thereafter propagated
to other helper users.
Note that this patch series enable the option but does not affect
make check at this time: make check still calls the tests with no options
which default to thread mode.

This patch series nicely splits it two groups (you can split your review
there):
1) up to "validation: pktio: adding command line argument parsing", the new
helper functions are introduced, and used in the validation tests.
2) from "helper: adding a function to merge getopt parameters" the ability
to parse command line arguments in subset in added and applied to
the example and performance tests.

Hope this makes sence for you too!




The following changes since commit 4834b40e7840fd23b8481b04dfff50d8fd0e1699:

  configure: fix broken conditional example execution (2016-04-27 16:51:38 
+0300)

are available in the git repository at:

  https://git.linaro.org/people/christophe.milard/odp.git 
test_in_process_mode_v4

for you to fetch changes up to d833f72880758a12885775847a6bda558253:

  helper: removing dead code (2016-04-28 17:04:14 +0200)

--------
Christophe Milard (36):
  helpers: adding command line argument parsing
  validation: common: adding command line argument parsing
  validation: shmem: adding command line argument parsing
  helpers: linux: creating common entry for process and thread
  helpers: linux: creating functions to handle odpthreads
  helper: test: adding odpthread functions tests
  validation: using implementation agnostic function for ODP threads
  validation: most tests: adding command line argument parsing
  validation: init: adding command line argument parsing
  validation: pktio: adding command line argument parsing
  helper: adding a function to merge getopt parameters
  helper: parsing the complete set of options
  performance: odp_scheduling: proc mode done by helper
  performance: odp_pktio_perf: using agnostic function for ODP threads
  performance: odp_pktio_perf: adding helper cmd line parsing
  performance: odp_l2fwd: using agnostic function for ODP threads
  performance: odp_l2fwd: adding helper cmd line parsing
  performance: crypto: using agnostic function for ODP threads
  performance: crypto: adding helper cmd line parsing
  example: classifier: using agnostic function for ODP threads
  example: classifier: adding helper cmd line parsing
  example: generator: using agnostic function for ODP threads
  example: generator: adding helper cmd line parsing
  example: ipsec: using agnostic function for ODP threads
  example: ipsec: adding helper cmd line parsing
  example: l2fwd_simple: using agnostic function for ODP threads
  example: l2fwd_simple: adding helper cmd line parsing
  example: pktio: using agnostic function for ODP threads
  example: pktio: adding helper cmd line parsing
  example: time: using agnostic function for ODP threads
  example: time: adding helper cmd line parsing
  example: timer: using agnostic function for ODP threads
  example: timer: adding helper cmd line parsing
  example: switch: using agnostic function for ODP threads
  example: switch: adding helper cmd line parsing
  helper: removing dead code

 example/classifier/odp_classifier.c|  39 +-
 example/generator/odp_gen

Re: [lng-odp] [PATCH] validation: lock: tuning the iteration number

[Christophe Milard]

ping

On 21 April 2016 at 15:32, Christophe Milard 
wrote:

> fixing: https://bugs.linaro.org/show_bug.cgi?id=2108
>
> The no_lock_functional_test does not really tests the ODP functionality:
> Instead, it actually checks that race conditions can be created between
> concurrent running threads (by making these threads writing shared
> variables without lock, and later noting the value written was changed by
> some of the concurrent threads).
> This test therefore validates other tests: if this test passes -i.e. if
> we do have race condition- then if the following tests suppress these
> race conditions by using some synchronization mechanism, these
> synchronization mechanisms can be said to be efficient.
> If, on the other hand, the no_lock_functional_test "fails", it says
> that the following tests are really inconclusive as the effect of the
> tested synchronization mechanism is not proven.
>
> When running with valgrind, no_lock_functional_test failed, probably
> because the extra execution time introduced by valgrind itself made
> the chance to run the critical section of the different threads "at
> the same time" much less probable.
>
> The simple solution is to increase the critical section running time
> (by largely increasing the number of iterations performed).
> The solution taken here is actually to tune the critical section running
> time (currentely to ITER_MPLY_FACTOR=3 times the time needed to note
> the first race condition).
> This means that the test will take longer to run with valgrind,
> but will remain short without valgrind.
>
> Signed-off-by: Christophe Milard 
> ---
>  test/validation/lock/lock.c | 71
> -
>  1 file changed, 64 insertions(+), 7 deletions(-)
>
> diff --git a/test/validation/lock/lock.c b/test/validation/lock/lock.c
> index f1f6d69..515bc77 100644
> --- a/test/validation/lock/lock.c
> +++ b/test/validation/lock/lock.c
> @@ -12,7 +12,10 @@
>  #include "lock.h"
>
>  #define VERBOSE0
> -#define MAX_ITERATIONS 1000
> +
> +#define MIN_ITERATIONS 1000
> +#define MAX_ITERATIONS 3
> +#define ITER_MPLY_FACTOR   3
>
>  #define SLOW_BARRIER_DELAY 400
>  #define BASE_DELAY 6
> @@ -325,6 +328,12 @@ static void *rwlock_recursive_api_tests(void *arg
> UNUSED)
> return NULL;
>  }
>
> +/*
> + * Tests that we do have contention between threads when running.
> + * Also adjust the number of iterations to be done (by other tests)
> + * so we have a fair chance to see that the tested synchronizer
> + * does avoid the race condition.
> + */
>  static void *no_lock_functional_test(void *arg UNUSED)
>  {
> global_shared_mem_t *global_mem;
> @@ -335,17 +344,36 @@ static void *no_lock_functional_test(void *arg
> UNUSED)
> thread_num = odp_cpu_id() + 1;
> per_thread_mem = thread_init();
> global_mem = per_thread_mem->global_mem;
> -   iterations = global_mem->g_iterations;
> +   iterations = 0;
>
> odp_barrier_wait(&global_mem->global_barrier);
>
> sync_failures = 0;
> current_errs = 0;
> rs_idx = 0;
> -   resync_cnt = iterations / NUM_RESYNC_BARRIERS;
> +   resync_cnt = MAX_ITERATIONS / NUM_RESYNC_BARRIERS;
> lock_owner_delay = BASE_DELAY;
>
> -   for (cnt = 1; cnt <= iterations; cnt++) {
> +   /*
> +   * Tunning the iteration number:
> +   * Here, we search for an iteration number that guarantees to show
> +   * race conditions between the odp threads.
> +   * Iterations is set to ITER_MPLY_FACTOR * cnt where cnt is when
> +   * the threads start to see "errors" (i.e. effect of other threads
> +   * running concurrentely without any synchronisation mechanism).
> +   * In other words, "iterations" is set to ITER_MPLY_FACTOR times the
> +   * minimum loop count necessary to see a need for synchronisation
> +   * mechanism.
> +   * If, later, these "errors" disappear when running other tests up
> to
> +   * "iterations" with synchro, the effect of the tested synchro
> mechanism
> +   * is likely proven.
> +   * If we reach "MAX_ITERATIONS", and "iteration" remains zero,
> +   * it means that we cannot see any race condition between the
> different
> +   * running theads (e.g. the OS is not preemptive) and all other
> tests
> +   * being passed won't tell much about the functionality of the
> +   * tested synchro mechanism.
> +   */
> +   for (cnt = 1; cnt <=  MAX_ITERATION

Re: [lng-odp] RFC: inter-process dynamic shared memory support.

[Christophe Milard]

Thanks, Barry,

I definitively think that is a good candidate for the arch call.
Even if I am not 100% sure I fully understand the whole picture, I this
text raise a few questions:

1) is this plain user space code, or are there related kernel modules
involved? I am guessing user space, but please confirm.

2) Do the arenas have to be known at init time? (you wrote "Typically an
initial process will call tmc_shmem_create() to create the file using a
fixed, known file name. Other processes then call tmc_shmem_open() to gain
access to the arena.")

3) Are there any constraint on the processes involved (such as processes
should be descendent of a common ancestor?)

4)Are processes that attempt to map the arena guaranteed a get the same
address AND guaranteed that the mapping will succeed? (I get worried when
reading "If the user specifies ADDR as zero, the system will choose an
address, and all subsequent users of the arena will automatically* try to
load* it at that address": This seems to imply that the mapping will
succees only if the virtual address in the process performing the mapping
happens to be free...)
This is exactly the problem I am trying to tackle by reserving the virtual
area address space from the beginning.

Thanks anyway Barry! Even if there are still questions, I do appreciate
your input! I guess we cann take more discussion on the ARCH call!

Christophe

On 28 April 2016 at 17:50, Barry Spinney  wrote:

>
> One of our Linux developers, Chris Metcalf, several years ago wrote a
> library module for the TileGx chip
> called tmc/shmem.[hc].  While targeted for the TileGx, much of the
> concepts and code are applicable
> to most Linux based platforms.  Instead of sending the entire module
> header and source code (for
> which I probably need some managerial approval), I have instead excerpted
> some of the main
> concepts and API's below.
>
> This module does have the property that "reserving" shared virtual address
> space has no cost -
> i.e. no Linux virtual memory page tables will be added or change, nor will
> there be any cost/effect
> on the file system.  However once reserved virtual addresses are made
> "active" (e.g. via a call to
> tmc_shmem_alloc) then of course page tables can get added, physical memory
> could get used,
> file system use of the associated backing file can start occurring.
>
> IF this approach is chosen to be used by ODP (or is at least a strong
> contender, then I can
> (after getting the likely manager approval) send in a proper ODP header
> file and maybe an strawman
> implementation as a RFC proposal.
>
>
>
> 
>
> Inter-process dynamic shared memory support.
>
> This API provides a convenient method for multiple processes to share
> memory using a persistent filesystem-based arena that is automatically
> mapped at the same, fixed address in all processes sharing the arena. The
> application chooses an address for the arena to be located at and a maximum
> size to which it can grow, and the system manages coordinating access to
> memory mapped from the file among the processes. Since the address is
> fixed, absolute pointer values, etc., may be safely stored into the arena.
>
> As is always true when you use shared memory, you should employ
> appropriate memory fencing to ensure that any modifications are actually
> fully visible before they are used by any other process.
>
> Typically an initial process will call tmc_shmem_create() to create the
> file using a fixed, known file name. Other processes then call
> tmc_shmem_open() to gain access to the arena. The creator should first
> initialize a tmc_alloc_t object to indicate any special attributes of the
> desired memory, such as huge page size, variant cache attributes, etc. If
> huge pages are requested, the tmc_shmem code will automatically open an
> additional file in the appropriate hugetlb file system. The files are
> opened such that they are automatically closed if the process calls exec()
> to start a new executable.
>
> The APIs create a file with permission 0600 (owner-only read/write), but
> the application may invoke fchmod() or fchown() on the underlying file
> descriptors if desired to reset the ownership and permissions.
>
> If the application wishes to create a temporary file name to hold the
> arena, it can use tmc_shmem_create_temp() and pass in a template filename,
> just as is done for mkstemp(). In this case it is typically then necessary
> to communicate the chosen filename out-of-band to the other processes that
> wish to share the arena.
>
> To grow the arena, any process that has the tmc_shmem arena open can call
> tmc_shmem_grow(); this is implemented by extending the underlying file and
> returning a pointer to the newly-allocated chunk of memory at the end of
> the file. Similarly, tmc_shmem_shrink() will truncate the underlying file
> to a shorter length, invalidating any pointers into the truncat

Re: [lng-odp] [GIT PULL ODPv4] running things in process mode

[Christophe Milard]

On 29 April 2016 at 00:48, Brian Brooks  wrote:

> On 04/28 17:18:36, Christophe Milard wrote:
> > Since v3:
> > -fixed rebase error (Christophe)
> > -rebased
>
> Thanks for the rebase. test_in_process_mode_v4 merged cleanly into
> origin/master and build and tests PASS.
>
> I've given this a look, and it appears we're headed in the right direction.
>
> > diff --git a/example/classifier/odp_classifier.c
> b/example/classifier/odp_classifier.c
> > index a477e23..4057457 100644
> > --- a/example/classifier/odp_classifier.c
> > +++ b/example/classifier/odp_classifier.c
> > @@ -815,10 +802,16 @@ static void parse_args(int argc, char *argv[],
> appl_args_t *appl_args)
> >   {NULL, 0, NULL, 0}
> >   };
> >
> > + static const char *shortopts = "+c:t:i:p:m:t:h";
> > +
> > + /* let helper collect its own arguments (e.g. --odph_proc) */
> > + odph_parse_options(argc, argv, shortopts, longopts);
> > +
> > + opterr = 0; /* do not issue errors on helper options */
>
> Please use the default behavior of opterr _or_ add the case statement for
> '?'
> when appropriate.
>

I do think we need opterr=0, sadly: when the test or example (such as the
classifier here) parses it options, the command line options being parsed
still contains all options, i.e. both the option meant for the classifier
itself, and the option meant to the "other layers", (here the helper). If
opt_err is left to 1, the classifier's call to get_opt() will issue errors
when hitting the helper options, even if we have a "?" in the switch, as
far as I understand.
So the mechanism taken here is: the caller (e.g. classifier) passes its
list of option to the helper which builds the complete list of options by
merging it own (helper) option list to the caller options. Then the helper
parse this complete list (with default opterr=1), i.e. reacting to unknown
options (not being in the merged list) and also picking up its own option
semantic, of course. After this the caller (here classifier) parse the
options again, picking up its own options only. but should not react on the
helper's otpions.
I Actually looked at removing the options from argv in the helpers, but
this turned up to be quite tricky as well.
parse_args() seem to be better at spitting command line option in different
owner, but is is not POSIX. not sure we want to insert that kind of
dependency for so little.

>
> >   while (1) {
> > - opt = getopt_long(argc, argv, "+c:t:i:p:m:t:h",
> > - longopts, &long_index);
> > + opt = getopt_long(argc, argv, shortopts,
> > +   longopts, &long_index);
> >
> >   if (opt == -1)
> >   break;  /* No more options */
> > diff --git a/example/l2fwd_simple/odp_l2fwd_simple.c
> b/example/l2fwd_simple/odp_l2fwd_simple.c
> > index 45bb9b1..7b67705 100644
> > --- a/example/l2fwd_simple/odp_l2fwd_simple.c
> > +++ b/example/l2fwd_simple/odp_l2fwd_simple.c
> > @@ -116,13 +117,33 @@ int main(int argc, char **argv)
> >   odp_pool_t pool;
> >   odp_pool_param_t params;
> >   odp_cpumask_t cpumask;
> > - odph_linux_pthread_t thd;
> > + odph_odpthread_t thd;
> >   odp_instance_t instance;
> > - odph_linux_thr_params_t thr_params;
> > + odph_odpthread_params_t thr_params;
> > + int rc = 0;
> > + int opt;
> > + int long_index;
> > +
> > + static const struct option longopts[] = { {NULL, 0, NULL, 0} };
> > + static const char *shortopts = "";
> > +
> > + /* let helper collect its own arguments (e.g. --odph_proc) */
> > + odph_parse_options(argc, argv, shortopts, longopts);
> > +
> > + /*
> > +  * parse own options: currentely none, but this will move optind
> > +  * to the first non-option argument. (in case there where helprt
> args)
> > +  */
>
> I suppose this is OK given the pre-existing arg handling.
>
> > + opterr = 0; /* do not issue errors on helper options */
> > + while (!rc) {
>
> Please use a simpler loop, e.g. for (;;) or while (1)
>

Of course, there is not much left for rc there! will be addressed in v5


>
> > + opt = getopt_long(argc, argv, shortopts, longopts,
> &long_index);
> > + if (-1 == opt)
> > + break;  /* No more options */
> > + }
> >
> > - if (argc != 5 ||
> > - odph_eth_addr_parse(&global.dst, argv[3]) != 0 ||
> > - odph_e

[lng-odp] [PATCH] configure.ac: remove duplicated line

[Christophe Milard]

Removed an instance AM_CONDITIONAL for test_example, as this line was
twice in the file.

Signed-off-by: Christophe Milard 
---
 configure.ac | 1 -
 1 file changed, 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index f88c36b..f138031 100644
--- a/configure.ac
+++ b/configure.ac
@@ -134,7 +134,6 @@ AM_CONDITIONAL([netmap_support], [test x$netmap_support = 
xyes ])
 AM_CONDITIONAL([PKTIO_DPDK], [test x$pktio_dpdk_support = xyes ])
 AM_CONDITIONAL([HAVE_PCAP], [test $have_pcap = yes])
 AM_CONDITIONAL([SDK_INSTALL_PATH_], [test "x${SDK_INSTALL_PATH_}" = "x1"])
-AM_CONDITIONAL([test_example], [test x$test_example = xyes ])
 AM_CONDITIONAL([test_installdir], [test "$testdir" != ""])
 AM_CONDITIONAL([cunit_support], [test x$cunit_support = xyes ])
 AM_CONDITIONAL([test_vald], [test x$test_vald = xyes ])
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [GIT PULL ODPv4] running things in process mode

[Christophe Milard]

Hi Brian

I have commented below, but it looks we need to talk.  would you have time
after the linaro sync call today, for a HO?

On 29 April 2016 at 23:49, Brian Brooks  wrote:

> On 04/29 13:29:03, Christophe Milard wrote:
> > On 29 April 2016 at 00:48, Brian Brooks  wrote:
> >
> > > On 04/28 17:18:36, Christophe Milard wrote:
> > > > Since v3:
> > > > -fixed rebase error (Christophe)
> > > > -rebased
> > >
> > > Thanks for the rebase. test_in_process_mode_v4 merged cleanly into
> > > origin/master and build and tests PASS.
> > >
> > > I've given this a look, and it appears we're headed in the right
> direction.
> > >
> > > > diff --git a/example/classifier/odp_classifier.c
> > > b/example/classifier/odp_classifier.c
> > > > index a477e23..4057457 100644
> > > > --- a/example/classifier/odp_classifier.c
> > > > +++ b/example/classifier/odp_classifier.c
> > > > @@ -815,10 +802,16 @@ static void parse_args(int argc, char *argv[],
> > > appl_args_t *appl_args)
> > > >   {NULL, 0, NULL, 0}
> > > >   };
> > > >
> > > > + static const char *shortopts = "+c:t:i:p:m:t:h";
> > > > +
> > > > + /* let helper collect its own arguments (e.g. --odph_proc) */
> > > > + odph_parse_options(argc, argv, shortopts, longopts);
> > > > +
> > > > + opterr = 0; /* do not issue errors on helper options */
> > >
> > > Please use the default behavior of opterr _or_ add the case statement
> for
> > > '?'
> > > when appropriate.
> > >
> >
> > I do think we need opterr=0, sadly: when the test or example (such as the
> > classifier here) parses it options, the command line options being parsed
> > still contains all options, i.e. both the option meant for the classifier
> > itself, and the option meant to the "other layers", (here the helper). If
> > opt_err is left to 1, the classifier's call to get_opt() will issue
> errors
> > when hitting the helper options, even if we have a "?" in the switch, as
> > far as I understand.
> > So the mechanism taken here is: the caller (e.g. classifier) passes its
> > list of option to the helper which builds the complete list of options by
> > merging it own (helper) option list to the caller options. Then the
> helper
> > parse this complete list (with default opterr=1), i.e. reacting to
> unknown
> > options (not being in the merged list) and also picking up its own option
> > semantic, of course. After this the caller (here classifier) parse the
> > options again, picking up its own options only. but should not react on
> the
> > helper's otpions.
> > I Actually looked at removing the options from argv in the helpers, but
> > this turned up to be quite tricky as well.
> > parse_args() seem to be better at spitting command line option in
> different
> > owner, but is is not POSIX. not sure we want to insert that kind of
> > dependency for so little.
>
> OK, this should not block a contribution.
>
> > > > diff --git a/helper/include/odp/helper/linux.h
> > > b/helper/include/odp/helper/linux.h
> > > > index 7a6504f..a9ec90a 100644
> > > > --- a/helper/include/odp/helper/linux.h
> > > > +++ b/helper/include/odp/helper/linux.h
> > > > @@ -25,111 +25,146 @@ extern "C" {
> > > >  #include 
> > > >
> > > >  #include 
> > > > +#include 
> > >
> > > Please consider migrating CLI parsing via C library to a more
> appropriate
> > > location.
> > >
> >
> > Not sure I understand this: The goal is to have the helpers parsing their
> > own options here. Maybe my comment above makes it clearer. otherwise
> please
> > explain what you meant.
>
> I see. I'll attempt to explain further.
>
> Helper linux.[ch] files contain code for both threading and arg parsing.
> First,
> because threading and arg parsing are separate functional pieces of code
> they
> _should_ belong in separate files. Second, the linux.[ch] filename is
> misleading. Two simple examples:
>
>   lib/thread.h
>   lib/thread.c
>   lib/thread_posix.c
>   lib/thread_linux.c
>
>   include/thread.h
>   lib/thread.c
>   lib/posix/thread.c
>
> Many ways, but the point is that common platform-independent code contained
> in one place, and abstraction occurs 'outward' or 'deeper' in the filenam

[lng-odp] [GIT PULL ODPv5] running things in process mode

[Christophe Milard]

[GIT PULL ODPv5] running things in process mode

Since v4:
-updates following Brian's comments

Since v3:
-fixed rebase error (Christophe)
-rebased

Since v2:
-serious rebase following clash with dba05a28 (api: init: add instance handle)
-squashing the validation changes. I am not very keen on squashing
 small isolated modifs to larger patches. don't really see the gain.
 Squashing more will put even more unrelated things together.

Since v1:
-variable declaration gathered in function 's head (Petri)
-linux prefix removed from helper's types and function names (Mike, Christophe)

Hi,

This patch series adds the ability to run tests/ exemples / perf-test
in "process mode" (i.e letting OPD threads being linux processes)
It it hence tackling ODP-171.

This is achieved in 2 main steps:

A]
The 2 pairs of helper functions:
odph_linux_pthread_create(), odph_linux_pthread_join()
and
odph_linux_process_fork_n(), odph_linux_process_wait_n()
are replaced by:
odph_linux_odpthreads_create() and odph_linux_odpthreads_join()
The latter's callers are unaware of the actual implementation of the ODP
thread (making test truly platform agnostic).
The helper functions decide at run time whether an odp thread is to be
a linux process or pthread based on the command line argument.

B] each test/example now calls a new helper function,
odph_linux_parse_options(), so that the helper can get its own arguments
out of the command line.
Currentely supported args are: --odph_proc, --odph_thread.
Defaults assumes thread. specifying both options runs in mixed mode.

The changed are first done on the shmem tests, and thereafter propagated
to other helper users.
Note that this patch series enable the option but does not affect
make check at this time: make check still calls the tests with no options
which default to thread mode.

This patch series nicely splits it two groups (you can split your review
there):
1) up to "validation: pktio: adding command line argument parsing", the new
helper functions are introduced, and used in the validation tests.
2) from "helper: adding a function to merge getopt parameters" the ability
to parse command line arguments in subset in added and applied to
the example and performance tests.

Hope this makes sence for you too!



The following changes since commit 04149e64e897d79229cf06539cda4d5f29bc7a90:

  configure.ac: remove duplicated test example configure (2016-04-29 18:38:14 
+0300)

are available in the git repository at:

  https://git.linaro.org/people/christophe.milard/odp.git 
test_in_process_mode_v5

for you to fetch changes up to f66d32f2bba5ceca85a894588a3262f75a1e5783:

  helper: removing dead code (2016-05-03 11:39:44 +0200)

--------
Christophe Milard (36):
  helpers: adding command line argument parsing
  validation: common: adding command line argument parsing
  validation: shmem: adding command line argument parsing
  helpers: linux: creating common entry for process and thread
  helpers: linux: creating functions to handle odpthreads
  helper: test: adding odpthread functions tests
  validation: using implementation agnostic function for ODP threads
  validation: most tests: adding command line argument parsing
  validation: init: adding command line argument parsing
  validation: pktio: adding command line argument parsing
  helper: adding a function to merge getopt parameters
  helper: parsing the complete set of options
  performance: odp_scheduling: proc mode done by helper
  performance: odp_pktio_perf: using agnostic function for ODP threads
  performance: odp_pktio_perf: adding helper cmd line parsing
  performance: odp_l2fwd: using agnostic function for ODP threads
  performance: odp_l2fwd: adding helper cmd line parsing
  performance: crypto: using agnostic function for ODP threads
  performance: crypto: adding helper cmd line parsing
  example: classifier: using agnostic function for ODP threads
  example: classifier: adding helper cmd line parsing
  example: generator: using agnostic function for ODP threads
  example: generator: adding helper cmd line parsing
  example: ipsec: using agnostic function for ODP threads
  example: ipsec: adding helper cmd line parsing
  example: l2fwd_simple: using agnostic function for ODP threads
  example: l2fwd_simple: adding helper cmd line parsing
  example: pktio: using agnostic function for ODP threads
  example: pktio: adding helper cmd line parsing
  example: time: using agnostic function for ODP threads
  example: time: adding helper cmd line parsing
  example: timer: using agnostic function for ODP threads
  example: timer: adding helper cmd line parsing
  example: switch: using agnostic function for ODP threads
  example: switch: adding helper cmd line parsing
  helper: r

Re: [lng-odp] [PATCH] helper: remove unused odph_linux_process_fork API

[Christophe Milard]

Please don't do that: It will just create an extra rebase on my "running
things in process mode " patch series which replaces all this (and
therefore delete these also)

Christophe.

On 2 May 2016 at 14:01, Krishna Garapati 
wrote:

>
>
> On 2 May 2016 at 13:47, Mike Holmes  wrote:
>
>> odph_linux_process_fork is not used by any ODP example or test, it is also
>> untested by the helper test suite.
>>
> Just for the note, we use this api currently in our nginx_ofp app.
>
>>
>> odph_linux_process_fork is a wrapper for odph_linux_process_fork_n so
>> just delete this, the impact if there are any users is very small.
>>
>  Agree, need a minor update in the app.
>
> /Krishna
>
>>
>> Signed-off-by: Mike Holmes 
>> ---
>>  helper/include/odp/helper/linux.h | 18 --
>>  helper/linux.c| 10 --
>>  2 files changed, 28 deletions(-)
>>
>> diff --git a/helper/include/odp/helper/linux.h
>> b/helper/include/odp/helper/linux.h
>> index 7a6504f..a68f269 100644
>> --- a/helper/include/odp/helper/linux.h
>> +++ b/helper/include/odp/helper/linux.h
>> @@ -80,24 +80,6 @@ int odph_linux_pthread_create(odph_linux_pthread_t
>> *pthread_tbl,
>>   */
>>  void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num);
>>
>> -
>> -/**
>> - * Fork a process
>> - *
>> - * Forks and sets CPU affinity for the child process. Ignores 'start'
>> and 'arg'
>> - * thread parameters.
>> - *
>> - * @param[out] procPointer to process state info (for output)
>> - * @param  cpu Destination CPU for the child process
>> - * @param  thr_params  Linux helper thread parameters
>> - *
>> - * @return On success: 1 for the parent, 0 for the child
>> - * On failure: -1 for the parent, -2 for the child
>> - */
>> -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu,
>> -   const odph_linux_thr_params_t *thr_params);
>> -
>> -
>>  /**
>>   * Fork a number of processes
>>   *
>> diff --git a/helper/linux.c b/helper/linux.c
>> index 24e243b..a181322 100644
>> --- a/helper/linux.c
>> +++ b/helper/linux.c
>> @@ -183,16 +183,6 @@ int odph_linux_process_fork_n(odph_linux_process_t
>> *proc_tbl,
>> return 1;
>>  }
>>
>> -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu,
>> -   const odph_linux_thr_params_t *thr_params)
>> -{
>> -   odp_cpumask_t mask;
>> -
>> -   odp_cpumask_zero(&mask);
>> -   odp_cpumask_set(&mask, cpu);
>> -   return odph_linux_process_fork_n(proc, &mask, thr_params);
>> -}
>> -
>>  int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num)
>>  {
>> pid_t pid;
>> --
>> 2.7.4
>>
>> ___
>> lng-odp mailing list
>> lng-odp@lists.linaro.org
>> https://lists.linaro.org/mailman/listinfo/lng-odp
>>
>
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH] helper: remove unused odph_linux_process_fork API

[Christophe Milard]

There is a test for these new functions in the patch series. Hope this is
what you meant by "it".

Christophe

On 3 May 2016 at 13:35, Mike Holmes  wrote:

>
>
> On 3 May 2016 at 07:32, Christophe Milard 
> wrote:
>
>> Please don't do that: It will just create an extra rebase on my "running
>> things in process mode " patch series which replaces all this (and
>> therefore delete these also)
>>
>>
> We need a test in helper for it then :) we dont want the full Monarch API
> to go out with an untested API from the test suite perspective, Krishna,
> Christoph any chance you can add one since you both care about the API?
>
> Mike
>
>
>
>> Christophe.
>>
>> On 2 May 2016 at 14:01, Krishna Garapati > > wrote:
>>
>>>
>>>
>>> On 2 May 2016 at 13:47, Mike Holmes  wrote:
>>>
>>>> odph_linux_process_fork is not used by any ODP example or test, it is
>>>> also
>>>> untested by the helper test suite.
>>>>
>>> Just for the note, we use this api currently in our nginx_ofp app.
>>>
>>>>
>>>> odph_linux_process_fork is a wrapper for odph_linux_process_fork_n so
>>>> just delete this, the impact if there are any users is very small.
>>>>
>>>  Agree, need a minor update in the app.
>>>
>>> /Krishna
>>>
>>>>
>>>> Signed-off-by: Mike Holmes 
>>>> ---
>>>>  helper/include/odp/helper/linux.h | 18 --
>>>>  helper/linux.c| 10 --
>>>>  2 files changed, 28 deletions(-)
>>>>
>>>> diff --git a/helper/include/odp/helper/linux.h
>>>> b/helper/include/odp/helper/linux.h
>>>> index 7a6504f..a68f269 100644
>>>> --- a/helper/include/odp/helper/linux.h
>>>> +++ b/helper/include/odp/helper/linux.h
>>>> @@ -80,24 +80,6 @@ int odph_linux_pthread_create(odph_linux_pthread_t
>>>> *pthread_tbl,
>>>>   */
>>>>  void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int
>>>> num);
>>>>
>>>> -
>>>> -/**
>>>> - * Fork a process
>>>> - *
>>>> - * Forks and sets CPU affinity for the child process. Ignores 'start'
>>>> and 'arg'
>>>> - * thread parameters.
>>>> - *
>>>> - * @param[out] procPointer to process state info (for output)
>>>> - * @param  cpu Destination CPU for the child process
>>>> - * @param  thr_params  Linux helper thread parameters
>>>> - *
>>>> - * @return On success: 1 for the parent, 0 for the child
>>>> - * On failure: -1 for the parent, -2 for the child
>>>> - */
>>>> -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu,
>>>> -   const odph_linux_thr_params_t *thr_params);
>>>> -
>>>> -
>>>>  /**
>>>>   * Fork a number of processes
>>>>   *
>>>> diff --git a/helper/linux.c b/helper/linux.c
>>>> index 24e243b..a181322 100644
>>>> --- a/helper/linux.c
>>>> +++ b/helper/linux.c
>>>> @@ -183,16 +183,6 @@ int odph_linux_process_fork_n(odph_linux_process_t
>>>> *proc_tbl,
>>>> return 1;
>>>>  }
>>>>
>>>> -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu,
>>>> -   const odph_linux_thr_params_t *thr_params)
>>>> -{
>>>> -   odp_cpumask_t mask;
>>>> -
>>>> -   odp_cpumask_zero(&mask);
>>>> -   odp_cpumask_set(&mask, cpu);
>>>> -   return odph_linux_process_fork_n(proc, &mask, thr_params);
>>>> -}
>>>> -
>>>>  int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num)
>>>>  {
>>>> pid_t pid;
>>>> --
>>>> 2.7.4
>>>>
>>>> ___
>>>> lng-odp mailing list
>>>> lng-odp@lists.linaro.org
>>>> https://lists.linaro.org/mailman/listinfo/lng-odp
>>>>
>>>
>>>
>>> ___
>>> lng-odp mailing list
>>> lng-odp@lists.linaro.org
>>> https://lists.linaro.org/mailman/listinfo/lng-odp
>>>
>>>
>>
>
>
> --
> Mike Holmes
> Technical Manager - Linaro Networking Group
> Linaro.org <http://www.linaro.org/> *│ *Open source software for ARM SoCs
> "Work should be fun and collaborative, the rest follows"
>
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [GIT PULL ODPv5] running things in process mode

[Christophe Milard]

Thanks for the time you took for it, Brian!

On 3 May 2016 at 15:56, Brian Brooks  wrote:

> v5 LGTM (looks good to me). Thanks Christophe!
>
> Reviewed-by: Brian Brooks 
>
> On 05/03 11:44:47, Christophe Milard wrote:
> > [GIT PULL ODPv5] running things in process mode
> >
> > Since v4:
> > -updates following Brian's comments
> >
> > Since v3:
> > -fixed rebase error (Christophe)
> > -rebased
> >
> > Since v2:
> > -serious rebase following clash with dba05a28 (api: init: add instance
> handle)
> > -squashing the validation changes. I am not very keen on squashing
> >  small isolated modifs to larger patches. don't really see the gain.
> >  Squashing more will put even more unrelated things together.
> >
> > Since v1:
> > -variable declaration gathered in function 's head (Petri)
> > -linux prefix removed from helper's types and function names (Mike,
> Christophe)
> >
> > Hi,
> >
> > This patch series adds the ability to run tests/ exemples / perf-test
> > in "process mode" (i.e letting OPD threads being linux processes)
> > It it hence tackling ODP-171.
> >
> > This is achieved in 2 main steps:
> >
> > A]
> > The 2 pairs of helper functions:
> > odph_linux_pthread_create(), odph_linux_pthread_join()
> > and
> > odph_linux_process_fork_n(), odph_linux_process_wait_n()
> > are replaced by:
> > odph_linux_odpthreads_create() and odph_linux_odpthreads_join()
> > The latter's callers are unaware of the actual implementation of the ODP
> > thread (making test truly platform agnostic).
> > The helper functions decide at run time whether an odp thread is to be
> > a linux process or pthread based on the command line argument.
> >
> > B] each test/example now calls a new helper function,
> > odph_linux_parse_options(), so that the helper can get its own arguments
> > out of the command line.
> > Currentely supported args are: --odph_proc, --odph_thread.
> > Defaults assumes thread. specifying both options runs in mixed mode.
> >
> > The changed are first done on the shmem tests, and thereafter propagated
> > to other helper users.
> > Note that this patch series enable the option but does not affect
> > make check at this time: make check still calls the tests with no options
> > which default to thread mode.
> >
> > This patch series nicely splits it two groups (you can split your review
> > there):
> > 1) up to "validation: pktio: adding command line argument parsing", the
> new
> > helper functions are introduced, and used in the validation tests.
> > 2) from "helper: adding a function to merge getopt parameters" the
> ability
> > to parse command line arguments in subset in added and applied to
> > the example and performance tests.
> >
> > Hope this makes sence for you too!
> >
> > 
> >
> > The following changes since commit
> 04149e64e897d79229cf06539cda4d5f29bc7a90:
> >
> >   configure.ac: remove duplicated test example configure (2016-04-29
> 18:38:14 +0300)
> >
> > are available in the git repository at:
> >
> >   https://git.linaro.org/people/christophe.milard/odp.git
> test_in_process_mode_v5
> >
> > for you to fetch changes up to f66d32f2bba5ceca85a894588a3262f75a1e5783:
> >
> >   helper: removing dead code (2016-05-03 11:39:44 +0200)
> >
> > 
> > Christophe Milard (36):
> >   helpers: adding command line argument parsing
> >   validation: common: adding command line argument parsing
> >   validation: shmem: adding command line argument parsing
> >   helpers: linux: creating common entry for process and thread
> >   helpers: linux: creating functions to handle odpthreads
> >   helper: test: adding odpthread functions tests
> >   validation: using implementation agnostic function for ODP threads
> >   validation: most tests: adding command line argument parsing
> >   validation: init: adding command line argument parsing
> >   validation: pktio: adding command line argument parsing
> >   helper: adding a function to merge getopt parameters
> >   helper: parsing the complete set of options
> >   performance: odp_scheduling: proc mode done by helper
> >   performance: odp_pktio_perf: using agnostic function for ODP
> threads
> >   performance: odp_pktio_perf: adding helper cmd line parsing
> >   performance: odp_l2fwd: using agnos

Re: [lng-odp] [PATCH] helper: remove unused odph_linux_process_fork API

[Christophe Milard]

I am not sure I really agree with that, Petri:
as of today, none of the written validation tests, performance tests nor
example needed to do the difference: of course, that does not mean it will
never be needed, but I don't think we should be adding (nor keeping) unused
code promoting a behaviour we do not wish: If we want to be odpthread type
agnostic as much as possible should we really provide means to go the other
way in helpers?.
And if one piece of code does need to do the difference in the future,
shouldn't that specific code call pthread/fork directely then? should the
helper provide a wrapper for this specific case, really?

Christophe

On 6 May 2016 at 11:09, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia.com> wrote:

> Direct pthreads and process helpers should not be removed. Although our
> validation tests and examples need to be thread type agnostic, there are a
> bunch of other applications that will want to use either Linux pthread or
> process (not opaque threads).
>
> Tests should be added for direct pthread/process helpers, instead of
> removing those.
>
> -Petri
>
> > -Original Message-
> > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> Mike
> > Holmes
> > Sent: Monday, May 02, 2016 2:47 PM
> > To: lng-odp@lists.linaro.org
> > Subject: [lng-odp] [PATCH] helper: remove unused odph_linux_process_fork
> > API
> >
> > odph_linux_process_fork is not used by any ODP example or test, it is
> also
> > untested by the helper test suite.
> >
> > odph_linux_process_fork is a wrapper for odph_linux_process_fork_n so
> > just delete this, the impact if there are any users is very small.
> >
> > Signed-off-by: Mike Holmes 
> > ---
> >  helper/include/odp/helper/linux.h | 18 --
> >  helper/linux.c| 10 --
> >  2 files changed, 28 deletions(-)
> >
> > diff --git a/helper/include/odp/helper/linux.h
> > b/helper/include/odp/helper/linux.h
> > index 7a6504f..a68f269 100644
> > --- a/helper/include/odp/helper/linux.h
> > +++ b/helper/include/odp/helper/linux.h
> > @@ -80,24 +80,6 @@ int odph_linux_pthread_create(odph_linux_pthread_t
> > *pthread_tbl,
> >   */
> >  void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num);
> >
> > -
> > -/**
> > - * Fork a process
> > - *
> > - * Forks and sets CPU affinity for the child process. Ignores 'start'
> and
> > 'arg'
> > - * thread parameters.
> > - *
> > - * @param[out] procPointer to process state info (for output)
> > - * @param  cpu Destination CPU for the child process
> > - * @param  thr_params  Linux helper thread parameters
> > - *
> > - * @return On success: 1 for the parent, 0 for the child
> > - * On failure: -1 for the parent, -2 for the child
> > - */
> > -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu,
> > - const odph_linux_thr_params_t *thr_params);
> > -
> > -
> >  /**
> >   * Fork a number of processes
> >   *
> > diff --git a/helper/linux.c b/helper/linux.c
> > index 24e243b..a181322 100644
> > --- a/helper/linux.c
> > +++ b/helper/linux.c
> > @@ -183,16 +183,6 @@ int odph_linux_process_fork_n(odph_linux_process_t
> > *proc_tbl,
> >   return 1;
> >  }
> >
> > -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu,
> > - const odph_linux_thr_params_t *thr_params)
> > -{
> > - odp_cpumask_t mask;
> > -
> > - odp_cpumask_zero(&mask);
> > - odp_cpumask_set(&mask, cpu);
> > - return odph_linux_process_fork_n(proc, &mask, thr_params);
> > -}
> > -
> >  int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num)
> >  {
> >   pid_t pid;
> > --
> > 2.7.4
> >
> > ___
> > lng-odp mailing list
> > lng-odp@lists.linaro.org
> > https://lists.linaro.org/mailman/listinfo/lng-odp
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCH] helper: remove unused odph_linux_process_fork API

[Christophe Milard]

The test exists already in the "running things in process mode" patch
series (reviewed by Brian). Hopefully I do test what I wrote there. However
his patch series does not add any test for the fork*() helper  functions as
these functions are removed in the last patch of the series.
I still think these should be removed (see separate answer to Petri).

Christophe.

On 3 May 2016 at 14:27, Mike Holmes  wrote:

>
>
> On 3 May 2016 at 07:39, Christophe Milard 
> wrote:
>
>> There is a test for these new functions in the patch series. Hope this is
>> what you meant by "it".
>>
>
> OK - if you will be adding a test that is perfect :), I am just going on
> the current state
> http://docs.opendataplane.org/snapshots/master/linux-generic-helper-lcov-html/helper/linux.c.func-sort-c.html
>
>
>>
>> Christophe
>>
>> On 3 May 2016 at 13:35, Mike Holmes  wrote:
>>
>>>
>>>
>>> On 3 May 2016 at 07:32, Christophe Milard 
>>> wrote:
>>>
>>>> Please don't do that: It will just create an extra rebase on my
>>>> "running things in process mode " patch series which replaces all this (and
>>>> therefore delete these also)
>>>>
>>>>
>>> We need a test in helper for it then :) we dont want the full Monarch
>>> API to go out with an untested API from the test suite perspective,
>>> Krishna, Christoph any chance you can add one since you both care about the
>>> API?
>>>
>>> Mike
>>>
>>>
>>>
>>>> Christophe.
>>>>
>>>> On 2 May 2016 at 14:01, Krishna Garapati <
>>>> balakrishna.garap...@linaro.org> wrote:
>>>>
>>>>>
>>>>>
>>>>> On 2 May 2016 at 13:47, Mike Holmes  wrote:
>>>>>
>>>>>> odph_linux_process_fork is not used by any ODP example or test, it is
>>>>>> also
>>>>>> untested by the helper test suite.
>>>>>>
>>>>> Just for the note, we use this api currently in our nginx_ofp app.
>>>>>
>>>>>>
>>>>>> odph_linux_process_fork is a wrapper for odph_linux_process_fork_n so
>>>>>> just delete this, the impact if there are any users is very small.
>>>>>>
>>>>>  Agree, need a minor update in the app.
>>>>>
>>>>> /Krishna
>>>>>
>>>>>>
>>>>>> Signed-off-by: Mike Holmes 
>>>>>> ---
>>>>>>  helper/include/odp/helper/linux.h | 18 --
>>>>>>  helper/linux.c| 10 --
>>>>>>  2 files changed, 28 deletions(-)
>>>>>>
>>>>>> diff --git a/helper/include/odp/helper/linux.h
>>>>>> b/helper/include/odp/helper/linux.h
>>>>>> index 7a6504f..a68f269 100644
>>>>>> --- a/helper/include/odp/helper/linux.h
>>>>>> +++ b/helper/include/odp/helper/linux.h
>>>>>> @@ -80,24 +80,6 @@ int odph_linux_pthread_create(odph_linux_pthread_t
>>>>>> *pthread_tbl,
>>>>>>   */
>>>>>>  void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int
>>>>>> num);
>>>>>>
>>>>>> -
>>>>>> -/**
>>>>>> - * Fork a process
>>>>>> - *
>>>>>> - * Forks and sets CPU affinity for the child process. Ignores
>>>>>> 'start' and 'arg'
>>>>>> - * thread parameters.
>>>>>> - *
>>>>>> - * @param[out] procPointer to process state info (for output)
>>>>>> - * @param  cpu Destination CPU for the child process
>>>>>> - * @param  thr_params  Linux helper thread parameters
>>>>>> - *
>>>>>> - * @return On success: 1 for the parent, 0 for the child
>>>>>> - * On failure: -1 for the parent, -2 for the child
>>>>>> - */
>>>>>> -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu,
>>>>>> -   const odph_linux_thr_params_t
>>>>>> *thr_params);
>>>>>> -
>>>>>> -
>>>>>>  /**
>>>>>>   * Fork a number of processes
>>>>>>   *
>>>>>> diff --git a/helper/linux.c b/helper/linux.c
>>&

Re: [lng-odp] [PATCHv2 1/2] doc: userguide: add timer/timeout state diagram

[Christophe Milard]

I am a bit confused by this diagram: It feels to me that timers and timeout
have separate lives (even , if of course some dependency exist).
>From this diagram, it feels like these 2 separate objects (timers and time
out events) share states...? do they?
In my head, and from the understanding I had, it feels that the to 4 top
states are for timers, whereas time-out events have only 2 states:
Unallocated or allocated.
It feels you are doing 1 FSM out of two. Maybe , it should be two FSM
(keeping the same transition names to show the dependancy.)

And I guess there is a typo at "odp_timeout_freee().

Christophe

On 7 May 2016 at 18:15, Bill Fischofer  wrote:

> Signed-off-by: Bill Fischofer 
> ---
>  doc/images/.gitignore   |  1 +
>  doc/images/timer_fsm.gv | 38 ++
>  doc/users-guide/Makefile.am |  1 +
>  3 files changed, 40 insertions(+)
>  create mode 100644 doc/images/timer_fsm.gv
>
> diff --git a/doc/images/.gitignore b/doc/images/.gitignore
> index a19aa75..72cf7ec 100644
> --- a/doc/images/.gitignore
> +++ b/doc/images/.gitignore
> @@ -1,2 +1,3 @@
>  resource_management.svg
>  pktio_fsm.svg
> +timer_fsm.svg
> diff --git a/doc/images/timer_fsm.gv b/doc/images/timer_fsm.gv
> new file mode 100644
> index 000..f8cb21a
> --- /dev/null
> +++ b/doc/images/timer_fsm.gv
> @@ -0,0 +1,38 @@
> +digraph timer_state_machine {
> +   rankdir=LR;
> +   size="12,20";
> +   node [fontsize=28];
> +   edge [fontsize=28];
> +   node [shape=doublecircle]; Timer_Unalloc
> +  Timeout_Unalloc
> +  Timeout_Delivered;
> +node [shape=rectangle]; Timeout_Queued;
> +   node [shape=circle];
> +   Timer_Unalloc -> Timer_Alloc [label="odp_timer_alloc()"];
> +   Timer_Alloc -> Timer_Unalloc [label="odp_timer_free()"];
> +   Timer_Alloc -> Timer_Set [label="odp_timer_set_abs()"];
> +   Timer_Alloc -> Timer_Set [label="odp_timer_set_rel()"];
> +   Timer_Set -> Timer_Alloc [label="odp_timer_cancel()"];
> +   Timer_Set -> Timeout_Alloc
> +   [label="odp_timer_cancel()" constraint=false];
> +   Timer_Set -> Timeout_Queued [label="=>odp_queue_enq()"];
> +   Timeout_Queued -> Timeout_Delivered [label="odp_schedule()"];
> +   Timeout_Unalloc -> Timeout_Alloc
> +[label="odp_timeout_alloc()" constraint=false];
> +   Timeout_Alloc -> Timeout_Unalloc
> +[label="odp_timeout_free()" constraint=false];
> +   Timeout_Alloc -> Timer_Set
> +[label="odp_timer_set_abs()" constraint=false];
> +   Timeout_Alloc -> Timer_Set
> +[label="odp_timer_set_rel()"];
> +   Timeout_Delivered -> Timer_Unalloc [label="odp_timer_free()"];
> +   Timeout_Delivered -> Timer_Set [label="odp_timer_set_abs()"];
> +   Timeout_Delivered -> Timer_Set [label="odp_timer_set_rel()"];
> +   Timeout_Delivered -> Timeout_Delivered
> + [label="odp_timeout_from_event()"];
> +   Timeout_Delivered -> Timeout_Delivered
> + [label="odp_timeout_timer()"];
> +   Timeout_Delivered -> Timeout_Unalloc
> + [label="odp_event_free() / odp_timeout_freee()"
> + constraint=false];
> +}
> diff --git a/doc/users-guide/Makefile.am b/doc/users-guide/Makefile.am
> index 74caa96..6bb0131 100644
> --- a/doc/users-guide/Makefile.am
> +++ b/doc/users-guide/Makefile.am
> @@ -30,6 +30,7 @@ IMAGES = $(top_srcdir)/doc/images/overview.svg \
>  $(top_srcdir)/doc/images/release_git.svg \
>  $(top_srcdir)/doc/images/segment.svg \
>  $(top_srcdir)/doc/images/simple_release_git.svg \
> +$(top_srcdir)/doc/images/timer_fsm.svg \
>  $(top_srcdir)/doc/images/tm_hierarchy.svg \
>  $(top_srcdir)/doc/images/tm_node.svg
>
> --
> 2.5.0
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCHv2 1/2] doc: userguide: add timer/timeout state diagram

[Christophe Milard]

On 9 May 2016 at 16:28, Bill Fischofer  wrote:

>
>
> On Mon, May 9, 2016 at 7:58 AM, Christophe Milard <
> christophe.mil...@linaro.org> wrote:
>
>> I am a bit confused by this diagram: It feels to me that timers and
>> timeout have separate lives (even , if of course some dependency exist).
>> From this diagram, it feels like these 2 separate objects (timers and
>> time out events) share states...? do they?
>>
>
> Actually they do, which is what this diagram is trying to express. When a
> timer is set one of the arguments is the timeout event that should be
> associated with it, so it is an error to attempt to free that event while
> it is associated with a set timer (results are undefined if you do).
> Timers are somewhat unique in this respect.
>

Sorry, Bill I still don't get it: Aren't you saying here that these are 2
separate FSMs, but that these 2 separate FSMs are actually
actionned/"triggered" by (partly) the same events? There is nothing wrong
with that... Then they should be represented as 2 separated FSM with the
same event names on the transitions triggered by the same events...
 Or I am very confused...

Christophe.

>
>
>> In my head, and from the understanding I had, it feels that the to 4 top
>> states are for timers, whereas time-out events have only 2 states:
>> Unallocated or allocated.
>> It feels you are doing 1 FSM out of two. Maybe , it should be two FSM
>> (keeping the same transition names to show the dependancy.)
>>
>> And I guess there is a typo at "odp_timeout_freee().
>>
>
> Thanks.  I'll fix that in the next version.
>
>
>>
>> Christophe
>>
>> On 7 May 2016 at 18:15, Bill Fischofer  wrote:
>>
>>> Signed-off-by: Bill Fischofer 
>>> ---
>>>  doc/images/.gitignore   |  1 +
>>>  doc/images/timer_fsm.gv | 38 ++
>>>  doc/users-guide/Makefile.am |  1 +
>>>  3 files changed, 40 insertions(+)
>>>  create mode 100644 doc/images/timer_fsm.gv
>>>
>>> diff --git a/doc/images/.gitignore b/doc/images/.gitignore
>>> index a19aa75..72cf7ec 100644
>>> --- a/doc/images/.gitignore
>>> +++ b/doc/images/.gitignore
>>> @@ -1,2 +1,3 @@
>>>  resource_management.svg
>>>  pktio_fsm.svg
>>> +timer_fsm.svg
>>> diff --git a/doc/images/timer_fsm.gv b/doc/images/timer_fsm.gv
>>> new file mode 100644
>>> index 000..f8cb21a
>>> --- /dev/null
>>> +++ b/doc/images/timer_fsm.gv
>>> @@ -0,0 +1,38 @@
>>> +digraph timer_state_machine {
>>> +   rankdir=LR;
>>> +   size="12,20";
>>> +   node [fontsize=28];
>>> +   edge [fontsize=28];
>>> +   node [shape=doublecircle]; Timer_Unalloc
>>> +  Timeout_Unalloc
>>> +  Timeout_Delivered;
>>> +node [shape=rectangle]; Timeout_Queued;
>>> +   node [shape=circle];
>>> +   Timer_Unalloc -> Timer_Alloc [label="odp_timer_alloc()"];
>>> +   Timer_Alloc -> Timer_Unalloc [label="odp_timer_free()"];
>>> +   Timer_Alloc -> Timer_Set [label="odp_timer_set_abs()"];
>>> +   Timer_Alloc -> Timer_Set [label="odp_timer_set_rel()"];
>>> +   Timer_Set -> Timer_Alloc [label="odp_timer_cancel()"];
>>> +   Timer_Set -> Timeout_Alloc
>>> +   [label="odp_timer_cancel()" constraint=false];
>>> +   Timer_Set -> Timeout_Queued [label="=>odp_queue_enq()"];
>>> +   Timeout_Queued -> Timeout_Delivered [label="odp_schedule()"];
>>> +   Timeout_Unalloc -> Timeout_Alloc
>>> +[label="odp_timeout_alloc()" constraint=false];
>>> +   Timeout_Alloc -> Timeout_Unalloc
>>> +[label="odp_timeout_free()" constraint=false];
>>> +   Timeout_Alloc -> Timer_Set
>>> +[label="odp_timer_set_abs()" constraint=false];
>>> +   Timeout_Alloc -> Timer_Set
>>> +[label="odp_timer_set_rel()"];
>>> +   Timeout_Delivered -> Timer_Unalloc [label="odp_timer_free()"];
>>> +   Timeout_Delivered -> Timer_Set [label="odp_timer_set_abs()"];
>>> +   Timeout_Delivered -> Timer_Set [label="odp_timer_set_rel()"];
>>> +   Timeout_Delivered -> Timeout_Del

Re: [lng-odp] [PATCHv2 1/2] doc: userguide: add timer/timeout state diagram

[Christophe Milard]

Hi Bill,

Just sent you an RFC with 2 FSMs.
Had to fight a bit with graphviz. Not finished yet. I will continue working
on it if you think that makes sense...
Christophe.

On 9 May 2016 at 17:15, Bill Fischofer  wrote:

> As we discussed in our call, we can both play around with graphviz and see
> if a prettier diagram can be constructed that is perhaps clearer, possibly
> with two separate FSMs.
>
> On Mon, May 9, 2016 at 9:39 AM, Christophe Milard <
> christophe.mil...@linaro.org> wrote:
>
>>
>>
>> On 9 May 2016 at 16:28, Bill Fischofer  wrote:
>>
>>>
>>>
>>> On Mon, May 9, 2016 at 7:58 AM, Christophe Milard <
>>> christophe.mil...@linaro.org> wrote:
>>>
>>>> I am a bit confused by this diagram: It feels to me that timers and
>>>> timeout have separate lives (even , if of course some dependency exist).
>>>> From this diagram, it feels like these 2 separate objects (timers and
>>>> time out events) share states...? do they?
>>>>
>>>
>>> Actually they do, which is what this diagram is trying to express. When
>>> a timer is set one of the arguments is the timeout event that should be
>>> associated with it, so it is an error to attempt to free that event while
>>> it is associated with a set timer (results are undefined if you do).
>>> Timers are somewhat unique in this respect.
>>>
>>
>> Sorry, Bill I still don't get it: Aren't you saying here that these are 2
>> separate FSMs, but that these 2 separate FSMs are actually
>> actionned/"triggered" by (partly) the same events? There is nothing wrong
>> with that... Then they should be represented as 2 separated FSM with the
>> same event names on the transitions triggered by the same events...
>>  Or I am very confused...
>>
>> Christophe.
>>
>>>
>>>
>>>> In my head, and from the understanding I had, it feels that the to 4
>>>> top states are for timers, whereas time-out events have only 2 states:
>>>> Unallocated or allocated.
>>>> It feels you are doing 1 FSM out of two. Maybe , it should be two FSM
>>>> (keeping the same transition names to show the dependancy.)
>>>>
>>>> And I guess there is a typo at "odp_timeout_freee().
>>>>
>>>
>>> Thanks.  I'll fix that in the next version.
>>>
>>>
>>>>
>>>> Christophe
>>>>
>>>> On 7 May 2016 at 18:15, Bill Fischofer 
>>>> wrote:
>>>>
>>>>> Signed-off-by: Bill Fischofer 
>>>>> ---
>>>>>  doc/images/.gitignore   |  1 +
>>>>>  doc/images/timer_fsm.gv | 38
>>>>> ++
>>>>>  doc/users-guide/Makefile.am |  1 +
>>>>>  3 files changed, 40 insertions(+)
>>>>>  create mode 100644 doc/images/timer_fsm.gv
>>>>>
>>>>> diff --git a/doc/images/.gitignore b/doc/images/.gitignore
>>>>> index a19aa75..72cf7ec 100644
>>>>> --- a/doc/images/.gitignore
>>>>> +++ b/doc/images/.gitignore
>>>>> @@ -1,2 +1,3 @@
>>>>>  resource_management.svg
>>>>>  pktio_fsm.svg
>>>>> +timer_fsm.svg
>>>>> diff --git a/doc/images/timer_fsm.gv b/doc/images/timer_fsm.gv
>>>>> new file mode 100644
>>>>> index 000..f8cb21a
>>>>> --- /dev/null
>>>>> +++ b/doc/images/timer_fsm.gv
>>>>> @@ -0,0 +1,38 @@
>>>>> +digraph timer_state_machine {
>>>>> +   rankdir=LR;
>>>>> +   size="12,20";
>>>>> +   node [fontsize=28];
>>>>> +   edge [fontsize=28];
>>>>> +   node [shape=doublecircle]; Timer_Unalloc
>>>>> +  Timeout_Unalloc
>>>>> +  Timeout_Delivered;
>>>>> +node [shape=rectangle]; Timeout_Queued;
>>>>> +   node [shape=circle];
>>>>> +   Timer_Unalloc -> Timer_Alloc [label="odp_timer_alloc()"];
>>>>> +   Timer_Alloc -> Timer_Unalloc [label="odp_timer_free()"];
>>>>> +   Timer_Alloc -> Timer_Set [label="odp_timer_set_abs()"];
>>>>> +   Timer_Alloc -> Timer_Set [label="odp_timer_set_rel()"];
>>>>> +   Timer_Set -> Timer_Alloc [label=&quo

Re: [lng-odp] ODP Addressing Model

[Christophe Milard]

On 9 May 2016 23:54, "Bill Fischofer"  wrote:
>
> The purpose of this thread is to summarize the discussions we've had over
the past week on this topic and to foster discussion. The goal here is to
reach consensus on a complete specification for ODP in this area that can
be added to the User Guide for Monarch, as well as to form the basis for
any expansion work needed in this area for Tiger Moth.
>
> Background
> =
>
> An ODP instance is bounded by everything between an odp_init_global()
through a matching odp_term_global() call. Applications are expected to
make use of a single ODP instance, though systems may support multiple
concurrent applications, each of which have their own independent ODP
instances. Communication between different applications is via independent
of whether either of them are making use of ODP and is expected to be via
standard OS or I/O mechanisms. The ODP Pktio IPC mechanism is experimental
and not enabled by default in Monarch.
>
> Threads and Processes
> ==
>
> ODP itself does not define a threading model or provide APIs for creating
or destroying threads. The odp_thread_t type is intended to be an
abstraction of whatever underlying threading model is available to the
application via OS or other means outside the scope of ODP. ODP provides a
limited number of thread APIs for basic identification purposes. These
include:
> odp_thread_id() - to obtain a unique thread ID for the current thread
> odp_thread_count() - to determine the number of threads that are running
in the current ODP instance
> odp_thread_count_max() - to determine the maximum number of threads that
can be supported by the current ODP instance
> odp_thread_type - to determine whether the current thread is a worker or
control thread
> In particular, ODP does not specify whether threads are implemented
within a single address space or separate address spaces.
>
> To facilitate operation in Linux environments, ODP provides a series of
"helper" APIs that layer on top of Linux pthreads. These are found in
helper/include/odp/helper/linux.h and include:
> odph_linux_pthread_create() - to create a pthread
> odph_linux_pthread_join() - to wait for pthreads to exit
> odph_linux_process_fork() - to fork a linux process
> odph_linux_process_fork_n() - to fork a number of linux processes in a
single call
> odph_linux_ process_wait_n() - to wait for a number of linux processes to
exit
> These helper functions provide a means of setting pthread attributes and
process affinity flags as part of their operation, but again being helpers
and not ODP APIs they do not determine ODP API semantics.
>
> Issues
> =
>
> Several issues have been identified as a result of the above. The main
issue is what, if anything, does ODP have to say about whether addresses
derived from handles returned by ODP APIs may or may not be shared among
threads?  The consensus seems to be that ODP itself is silent on this
subject. While it is recognized that it is convenient for applications to
share addresses between threads, whether or not they can do so safely is a
function of both application design and the underlying ODP implementation.
>
> For example, if an application consists of multiple threads prior to the
odp_global_init() call, then it is undefined whether handles, let alone
addresses, derived from that ODP instance are sharable with other
application threads. If the other application threads are pthreads sharing
the same address space, then perhaps yes, but if they are separate
processes then from the perspective of the ODP instance they are
effectively separate applications and hence things like ODP handles would
have no meaning within them.
>
> It seems that ODP should define the scope of an ODP instance (i.e., which
threads ODP handles derived from that instance are valid) to be the thread
that calls odp_init_global() and its direct descendants.
>
> Second, regarding addresses derived from ODP handles (e.g., those
returned by odp_shm_addr(), odp_packet_data(), etc.) again it seems that
these can be shared between threads only under two conditions:
> The threads are within the same ODP instance
> The threads occupy a single shared address space or are forked from the
the same root thread that created the handle from which the returned
address is derived.
> Point 2 is again a function of both application design and the underlying
implementation. If the application creates all of its resources (pools,
shms, etc. during initialization prior to launching threads then these two
criteria should be satisfied).  However, if the application starts creating
threads (particularly those that are not sharing the same address space)
and then those child threads are the ones that create pools, shms, etc.,
then in general it does not seem safe to assume that addresses derived from
these handles have meaning to any other thread that is not a direct
descendent of the creating thread.
>
> From a portability and best practices standpoin

[lng-odp] [PATCHv6 00/38] running things in process mode

[Christophe Milard]

Pull can be performed from the URL at the end of this cover letter
All patches sent to the list following Maxim's request.

Since v5
-rebased
-patch removing old unused helper code dropped. (Petri)
-Doc update to discourage usage of the old functions. (Petri, Mike)

Since v4:
-updates following Brian's comments

Since v3:
-fixed rebase error (Christophe)
-rebased

Since v2:
-serious rebase following clash with dba05a28 (api: init: add instance handle)
-squashing the validation changes. I am not very keen on squashing
 small isolated modifs to larger patches. don't really see the gain.
 Squashing more will put even more unrelated things together.

Since v1:
-variable declaration gathered in function 's head (Petri)
-linux prefix removed from helper's types and function names (Mike, Christophe)

Hi,

This patch series adds the ability to run tests/ exemples / perf-test
in "process mode" (i.e letting OPD threads being linux processes)
It it hence tackling ODP-171.

This is achieved in 2 main steps:

A]
The 2 pairs of helper functions:
odph_linux_pthread_create(), odph_linux_pthread_join()
and
odph_linux_process_fork_n(), odph_linux_process_wait_n()
are replaced by:
odph_linux_odpthreads_create() and odph_linux_odpthreads_join()
The latter's callers are unaware of the actual implementation of the ODP
thread (making test truly platform agnostic).
The helper functions decide at run time whether an odp thread is to be
a linux process or pthread based on the command line argument.

B] each test/example now calls a new helper function,
odph_linux_parse_options(), so that the helper can get its own arguments
out of the command line.
Currentely supported args are: --odph_proc, --odph_thread.
Defaults assumes thread. specifying both options runs in mixed mode.

The changed are first done on the shmem tests, and thereafter propagated
to other helper users.
Note that this patch series enable the option but does not affect
make check at this time: make check still calls the tests with no options
which default to thread mode.

This patch series nicely splits it two groups (you can split your review
there):
1) up to "validation: pktio: adding command line argument parsing", the new
helper functions are introduced, and used in the validation tests.
2) from "helper: adding a function to merge getopt parameters" the ability
to parse command line arguments in subset in added and applied to
the example and performance tests.

Hope this makes sence for you too!


[GIT PULL ODPv6] running things in process mode:

The following changes since commit 24fdbb5060e8702a808a8e2e674521df50e8e830:

  doc: userguide: trim images for better fit (2016-05-10 06:34:34 -0400)

are available in the git repository at:

  https://git.linaro.org/people/christophe.milard/odp.git 
test_in_process_mode_v6

for you to fetch changes up to f4f4e9e4a1d4304902b50b0d08719c4805ac0a3e:

  doc: implementers-guide: adding helper recommendations (2016-05-11 18:10:54 
+0200)

--------
Christophe Milard (38):
  helpers: adding command line argument parsing
  validation: common: adding command line argument parsing
  validation: shmem: adding command line argument parsing
  helpers: linux: creating common entry for process and thread
  helpers: linux: creating functions to handle odpthreads
  helper: test: adding odpthread functions tests
  validation: using implementation agnostic function for ODP threads
  validation: most tests: adding command line argument parsing
  validation: init: adding command line argument parsing
  validation: pktio: adding command line argument parsing
  helper: adding a function to merge getopt parameters
  helper: parsing the complete set of options
  performance: odp_scheduling: proc mode done by helper
  performance: odp_pktio_perf: using agnostic function for ODP threads
  performance: odp_pktio_perf: adding helper cmd line parsing
  performance: odp_l2fwd: using agnostic function for ODP threads
  performance: odp_l2fwd: adding helper cmd line parsing
  performance: crypto: using agnostic function for ODP threads
  performance: crypto: adding helper cmd line parsing
  example: classifier: using agnostic function for ODP threads
  example: classifier: adding helper cmd line parsing
  example: generator: using agnostic function for ODP threads
  example: generator: adding helper cmd line parsing
  example: ipsec: using agnostic function for ODP threads
  example: ipsec: adding helper cmd line parsing
  example: l2fwd_simple: using agnostic function for ODP threads
  example: l2fwd_simple: adding helper cmd line parsing
  example: pktio: using agnostic function for ODP threads
  example: pktio: adding helper cmd line parsing
  example: time: using agnostic function for ODP threads
  example: time

[lng-odp] [PATCHv6 01/38] helpers: adding command line argument parsing

[Christophe Milard]

A function to parse options meant for the helpers out of a the command
line is added. This parse function picks up options altering the behaviour
of the helpers out of the command line. Other options are reject.
No helper options are definied (yet).

Signed-off-by: Christophe Milard 
---
 helper/include/odp/helper/linux.h | 13 +
 helper/linux.c| 30 ++
 2 files changed, 43 insertions(+)

diff --git a/helper/include/odp/helper/linux.h 
b/helper/include/odp/helper/linux.h
index 7a6504f..c6736a0 100644
--- a/helper/include/odp/helper/linux.h
+++ b/helper/include/odp/helper/linux.h
@@ -130,6 +130,19 @@ int odph_linux_process_fork_n(odph_linux_process_t 
*proc_tbl,
 int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num);
 
 
+/**
+ * Parse linux helper options
+ *
+ * Parse the command line options. Pick up options meant for the helper itself.
+ *
+ * @param argc argument count
+ * @param argv argument values
+ *
+ * @return On success: 0
+ *On failure: -1 (failure occurs only if a value passed for a helper
+ *option is invalid. callers cannot have own options)
+ */
+int odph_parse_options(int argc, char *argv[]);
 #ifdef __cplusplus
 }
 #endif
diff --git a/helper/linux.c b/helper/linux.c
index 24e243b..5dbc2da 100644
--- a/helper/linux.c
+++ b/helper/linux.c
@@ -16,6 +16,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -236,3 +237,32 @@ int odph_linux_process_wait_n(odph_linux_process_t 
*proc_tbl, int num)
 
return 0;
 }
+
+/*
+ * Parse command line options to extract options affecting helpers.
+ */
+int odph_parse_options(int argc, char *argv[])
+{
+   int c;
+
+   static struct option long_options[] = {
+   /* These options set a flag. */
+   {0, 0, 0, 0}
+   };
+
+   while (1) {
+   /* getopt_long stores the option index here. */
+   int option_index = 0;
+
+   c = getopt_long (argc, argv, "",
+long_options, &option_index);
+
+   /* Detect the end of the options. */
+   if (c == -1)
+   break;
+   }
+
+   optind = 0; /* caller expects this to be zero if it parses too*/
+
+   return 0;
+}
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 03/38] validation: shmem: adding command line argument parsing

[Christophe Milard]

As the test itself does not have specific args, it just calls
the cunit_common parsing function to pick up cunit_common and helpers
arguments.

Signed-off-by: Christophe Milard 
---
 test/validation/shmem/shmem.c  | 10 --
 test/validation/shmem/shmem.h  |  2 +-
 test/validation/shmem/shmem_main.c |  4 ++--
 3 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/test/validation/shmem/shmem.c b/test/validation/shmem/shmem.c
index 0bf38d2..b4c6847 100644
--- a/test/validation/shmem/shmem.c
+++ b/test/validation/shmem/shmem.c
@@ -91,9 +91,15 @@ odp_suiteinfo_t shmem_suites[] = {
ODP_SUITE_INFO_NULL,
 };
 
-int shmem_main(void)
+int shmem_main(int argc, char *argv[])
 {
-   int ret = odp_cunit_register(shmem_suites);
+   int ret;
+
+   /* parse common options: */
+   if (odp_cunit_parse_options(argc, argv))
+   return -1;
+
+   ret = odp_cunit_register(shmem_suites);
 
if (ret == 0)
ret = odp_cunit_run();
diff --git a/test/validation/shmem/shmem.h b/test/validation/shmem/shmem.h
index d60cf64..a5893d9 100644
--- a/test/validation/shmem/shmem.h
+++ b/test/validation/shmem/shmem.h
@@ -19,6 +19,6 @@ extern odp_testinfo_t shmem_suite[];
 extern odp_suiteinfo_t shmem_suites[];
 
 /* main test program: */
-int shmem_main(void);
+int shmem_main(int argc, char *argv[]);
 
 #endif
diff --git a/test/validation/shmem/shmem_main.c 
b/test/validation/shmem/shmem_main.c
index 445fbf1..4c69130 100644
--- a/test/validation/shmem/shmem_main.c
+++ b/test/validation/shmem/shmem_main.c
@@ -6,7 +6,7 @@
 
 #include "shmem.h"
 
-int main(void)
+int main(int argc, char *argv[])
 {
-   return shmem_main();
+   return shmem_main(argc, argv);
 }
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 02/38] validation: common: adding command line argument parsing

[Christophe Milard]

A function to parse the command line args is added. This function is meant
to parse only arguments altering the behaviour of cunit_common (this
includes the helpers args as cunit_common uses the helpers)
As at this stage only helper args fall into that category, this function
simply calls the helper parsing function.
(parsing pure cunit_commons args might be added later here if needed)

Signed-off-by: Christophe Milard 
---
 test/validation/common/odp_cunit_common.c | 10 ++
 test/validation/common/odp_cunit_common.h |  2 ++
 2 files changed, 12 insertions(+)

diff --git a/test/validation/common/odp_cunit_common.c 
b/test/validation/common/odp_cunit_common.c
index 2712abe..45bdeff 100644
--- a/test/validation/common/odp_cunit_common.c
+++ b/test/validation/common/odp_cunit_common.c
@@ -342,3 +342,13 @@ int odp_cunit_register(odp_suiteinfo_t testsuites[])
 
return 0;
 }
+
+/*
+ * Parse command line options to extract options affectiong cunit_common.
+ * (hence also helpers options as cunit_common uses the helpers)
+ * Options private to the test calling cunit_common are not parsed here.
+ */
+int odp_cunit_parse_options(int argc, char *argv[])
+{
+   return odph_parse_options(argc, argv);
+}
diff --git a/test/validation/common/odp_cunit_common.h 
b/test/validation/common/odp_cunit_common.h
index 22c2a19..3812b0f 100644
--- a/test/validation/common/odp_cunit_common.h
+++ b/test/validation/common/odp_cunit_common.h
@@ -73,6 +73,8 @@ typedef struct {
int numthrds; /**< no of pthreads to create */
 } pthrd_arg;
 
+/* parse parameters that affect the behaviour of odp_cunit_common */
+int odp_cunit_parse_options(int argc, char *argv[]);
 /* register suites to be run via odp_cunit_run() */
 int odp_cunit_register(odp_suiteinfo_t testsuites[]);
 /* update tests previously registered via odp_cunit_register() */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 04/38] helpers: linux: creating common entry for process and thread

[Christophe Milard]

odph_linux_pthread_t and odph_linux_process_t are joined, creating
odph_odpthread_t. Tests should be using the latter only so that
the actual implementation of the ODP thread don't leak in the test itself.
(odph_odpthread_t is opaque to the test, and is common to both
 thread and process omplementation of ODP threads)
odph_linux_pthread_t and odph_linux_process_t are kept for compatibility,
but should be removed in the future.

Signed-off-by: Christophe Milard 
---
 helper/include/odp/helper/linux.h | 39 ++-
 1 file changed, 38 insertions(+), 1 deletion(-)

diff --git a/helper/include/odp/helper/linux.h 
b/helper/include/odp/helper/linux.h
index c6736a0..f99b88a 100644
--- a/helper/include/odp/helper/linux.h
+++ b/helper/include/odp/helper/linux.h
@@ -44,7 +44,6 @@ typedef struct {
odph_linux_thr_params_t thr_params;
 } odph_linux_pthread_t;
 
-
 /** Linux process state information */
 typedef struct {
pid_t pid;  /**< Process ID */
@@ -52,6 +51,44 @@ typedef struct {
int   status;   /**< Process state change status */
 } odph_linux_process_t;
 
+/** odpthread linux type: whether an ODP thread is a linux thread or process */
+typedef enum odph_odpthread_linuxtype_e {
+   ODPTHREAD_NOT_STARTED = 0,
+   ODPTHREAD_PROCESS,
+   ODPTHREAD_PTHREAD
+} odph_odpthread_linuxtype_t;
+
+/** odpthread parameters for odp threads (pthreads and processes) */
+typedef struct {
+   int (*start)(void *);   /**< Thread entry point function */
+   void *arg;  /**< Argument for the function */
+   odp_thread_type_t thr_type; /**< ODP thread type */
+   odp_instance_t instance;/**< ODP instance handle */
+} odph_odpthread_params_t;
+
+/** The odpthread starting arguments, used both in process or thread mode */
+typedef struct {
+   odph_odpthread_linuxtype_t linuxtype;
+   odph_odpthread_params_t thr_params; /*copy of thread start parameter*/
+} odph_odpthread_start_args_t;
+
+/** Linux odpthread state information, used both in process or thread mode */
+typedef struct {
+   odph_odpthread_start_args_t start_args;
+   int cpu;/**< CPU ID */
+   int last;   /**< true if last table entry */
+   union {
+   struct { /* for thread implementation */
+   pthread_t   thread_id; /**< Pthread ID */
+   pthread_attr_t  attr;   /**< Pthread attributes */
+   } thread;
+   struct { /* for process implementation */
+   pid_t   pid;/**< Process ID */
+   int status; /**< Process state chge status*/
+   } proc;
+   };
+} odph_odpthread_t;
+
 /**
  * Creates and launches pthreads
  *
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 05/38] helpers: linux: creating functions to handle odpthreads

[Christophe Milard]

Two functions, odph_odpthreads_create and odph_odpthreads_join
are created to create and termindate odp threads.
These function will create the odp threads either as linux processes or
as linux threads, depending on the command line arguments (flag odph_proc
or odph_thread). If both flags are given, every second odp thread will be
process, (others are threads). default is thead only.
Former functions (odph_linux_pthread_create, odph_linux_pthread_join,
odph_linux_process_fork, odph_linux_process_fork_n and
odph_linux_process_wait_n) are left for a compatibility, but are aimed
to be removed.

Signed-off-by: Christophe Milard 
---
 helper/include/odp/helper/linux.h |  28 
 helper/linux.c| 267 ++
 2 files changed, 295 insertions(+)

diff --git a/helper/include/odp/helper/linux.h 
b/helper/include/odp/helper/linux.h
index f99b88a..f67aa30 100644
--- a/helper/include/odp/helper/linux.h
+++ b/helper/include/odp/helper/linux.h
@@ -166,6 +166,34 @@ int odph_linux_process_fork_n(odph_linux_process_t 
*proc_tbl,
  */
 int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num);
 
+/**
+ * Creates and launches odpthreads (as linux threads or processes)
+ *
+ * Creates, pins and launches threads to separate CPU's based on the cpumask.
+ *
+ * @param thread_tblThread table
+ * @param mask  CPU mask
+ * @param thr_paramsODP thread parameters
+ *
+ * @return Number of threads created
+ */
+int odph_odpthreads_create(odph_odpthread_t *thread_tbl,
+  const odp_cpumask_t *mask,
+  const odph_odpthread_params_t *thr_params);
+
+/**
+ * Waits odpthreads (as linux threads or processes) to exit.
+ *
+ * Returns when all odpthreads have terminated.
+ *
+ * @param thread_tblThread table
+ * @return The number of joined threads or -1 on error.
+ * (error occurs if any of the start_routine return non-zero or if
+ *  the thread join/process wait itself failed -e.g. as the result of a kill)
+ *
+ */
+int odph_odpthreads_join(odph_odpthread_t *thread_tbl);
+
 
 /**
  * Parse linux helper options
diff --git a/helper/linux.c b/helper/linux.c
index 5dbc2da..b8d4f49 100644
--- a/helper/linux.c
+++ b/helper/linux.c
@@ -22,6 +22,11 @@
 #include 
 #include "odph_debug.h"
 
+static struct {
+   int proc; /* true when process mode is required */
+   int thrd; /* true when thread mode is required */
+} helper_options;
+
 static void *odp_run_start_routine(void *arg)
 {
odph_linux_thr_params_t *thr_params = arg;
@@ -239,6 +244,262 @@ int odph_linux_process_wait_n(odph_linux_process_t 
*proc_tbl, int num)
 }
 
 /*
+ * wrapper for odpthreads, either implemented as linux threads or processes.
+ * (in process mode, if start_routine returns NULL, the process return 
FAILURE).
+ */
+static void *odpthread_run_start_routine(void *arg)
+{
+   int status;
+   int ret;
+   odph_odpthread_params_t *thr_params;
+
+   odph_odpthread_start_args_t *start_args = arg;
+
+   thr_params = &start_args->thr_params;
+
+   /* ODP thread local init */
+   if (odp_init_local(thr_params->instance, thr_params->thr_type)) {
+   ODPH_ERR("Local init failed\n");
+   if (start_args->linuxtype == ODPTHREAD_PROCESS)
+   _exit(EXIT_FAILURE);
+   return NULL;
+   }
+
+   ODPH_DBG("helper: ODP %s thread started as linux %s. (pid=%d)\n",
+thr_params->thr_type == ODP_THREAD_WORKER ?
+"worker" : "control",
+(start_args->linuxtype == ODPTHREAD_PTHREAD) ?
+"pthread" : "process",
+(int)getpid());
+
+   status = thr_params->start(thr_params->arg);
+   ret = odp_term_local();
+
+   if (ret < 0)
+   ODPH_ERR("Local term failed\n");
+   else if (ret == 0 && odp_term_global(thr_params->instance))
+   ODPH_ERR("Global term failed\n");
+
+   /* for process implementation of odp threads, just return status... */
+   if (start_args->linuxtype == ODPTHREAD_PROCESS)
+   _exit(status);
+
+   /* threads implementation return void* pointers: cast status to that. */
+   return (void *)(long)status;
+}
+
+/*
+ * Create a single ODPthread as a linux process
+ */
+static int odph_linux_process_create(odph_odpthread_t *thread_tbl,
+int cpu,
+const odph_odpthread_params_t *thr_params)
+{
+   int ret;
+   cpu_set_t cpu_set;
+   pid_t pid;
+
+   CPU_ZERO(&cpu_set);
+   CPU_SET(cpu, &cpu_set);
+
+   thread_tbl->start_args.thr_params= *thr_params; /* copy */
+   thread_tbl->start_args.linuxtype = ODPTHREAD_PROCESS;
+   thread_tbl->cpu = cpu;
+
+

[lng-odp] [PATCHv6 06/38] helper: test: adding odpthread functions tests

[Christophe Milard]

Simple tests testing odph_odpthreads_create() and
odph_odpthreads_join() are added.
A single test binary, odpthreads, is added. This program creates
odp threads as pthreads or linux processes, dependng on its calling args.
Three calling scripts are added to create ODP threads as either processes,
threads or both (mixed).

Signed-off-by: Christophe Milard 
---
 helper/test/.gitignore  |   1 +
 helper/test/Makefile.am |   8 ++-
 helper/test/odpthreads.c| 100 
 helper/test/odpthreads_as_mixed |  15 ++
 helper/test/odpthreads_as_processes |  15 ++
 helper/test/odpthreads_as_pthreads  |  15 ++
 6 files changed, 152 insertions(+), 2 deletions(-)
 create mode 100644 helper/test/odpthreads.c
 create mode 100755 helper/test/odpthreads_as_mixed
 create mode 100755 helper/test/odpthreads_as_processes
 create mode 100755 helper/test/odpthreads_as_pthreads

diff --git a/helper/test/.gitignore b/helper/test/.gitignore
index e232753..5ce3c3b 100644
--- a/helper/test/.gitignore
+++ b/helper/test/.gitignore
@@ -1,6 +1,7 @@
 *.trs
 *.log
 chksum
+odpthreads
 parse
 process
 table
diff --git a/helper/test/Makefile.am b/helper/test/Makefile.am
index 7f0b67d..4ec561b 100644
--- a/helper/test/Makefile.am
+++ b/helper/test/Makefile.am
@@ -11,9 +11,11 @@ EXECUTABLES = chksum$(EXEEXT) \
   process$(EXEEXT)\
   table$(EXEEXT)
 
-COMPILE_ONLY =
+COMPILE_ONLY = odpthreads
 
-TESTSCRIPTS =
+TESTSCRIPTS = odpthreads_as_mixed \
+ odpthreads_as_processes \
+ odpthreads_as_pthreads
 
 if test_helper
 TESTS = $(EXECUTABLES) $(TESTSCRIPTS)
@@ -25,6 +27,8 @@ bin_PROGRAMS = $(EXECUTABLES) $(COMPILE_ONLY)
 
 
 dist_chksum_SOURCES = chksum.c
+dist_odpthreads_SOURCES = odpthreads.c
+odpthreads_LDADD = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la
 dist_thread_SOURCES = thread.c
 thread_LDADD = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la
 dist_process_SOURCES = process.c
diff --git a/helper/test/odpthreads.c b/helper/test/odpthreads.c
new file mode 100644
index 000..369da62
--- /dev/null
+++ b/helper/test/odpthreads.c
@@ -0,0 +1,100 @@
+/* Copyright (c) 2016, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+/*
+ * This program tests the ability of the linux helper to create ODP threads,
+ * either implemented as linux pthreads or as linux processes, depending on
+ * the option passed to the program (--odph_proc, --odph_thread or both)
+ */
+
+#include 
+#include 
+#include 
+
+#define NUMBER_WORKERS 16
+static int worker_fn(void *arg TEST_UNUSED)
+{
+   /* depend on the odp helper to call odp_init_local */
+
+   printf("Worker thread on CPU %d\n", odp_cpu_id());
+
+   /* depend on the odp helper to call odp_term_local */
+
+   return 0;
+}
+
+/* Create additional dataplane opdthreads */
+int main(int argc, char *argv[])
+{
+   odp_instance_t instance;
+   odph_odpthread_params_t thr_params;
+   odph_odpthread_t thread_tbl[NUMBER_WORKERS];
+   odp_cpumask_t cpu_mask;
+   int num_workers;
+   int cpu;
+   int ret;
+   char cpumaskstr[ODP_CPUMASK_STR_SIZE];
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv);
+
+   if (odp_init_global(&instance, NULL, NULL)) {
+   LOG_ERR("Error: ODP global init failed.\n");
+   exit(EXIT_FAILURE);
+   }
+
+   if (odp_init_local(instance, ODP_THREAD_CONTROL)) {
+   LOG_ERR("Error: ODP local init failed.\n");
+   exit(EXIT_FAILURE);
+   }
+
+   /* discover how many opdthreads this system can support */
+   num_workers = odp_cpumask_default_worker(&cpu_mask, NUMBER_WORKERS);
+   if (num_workers < NUMBER_WORKERS) {
+   printf("System can only support %d threads and not the %d 
requested\n",
+  num_workers, NUMBER_WORKERS);
+   }
+
+   /* generate a summary for the user */
+   (void)odp_cpumask_to_str(&cpu_mask, cpumaskstr, sizeof(cpumaskstr));
+   printf("default cpu mask:   %s\n", cpumaskstr);
+   printf("default num worker threads: %i\n", num_workers);
+
+   cpu = odp_cpumask_first(&cpu_mask);
+   printf("the first CPU:  %i\n", cpu);
+
+   /* reserve cpu 0 for the control plane so remove it from
+* the default mask */
+   odp_cpumask_clr(&cpu_mask, 0);
+   num_workers = odp_cpumask_count(&cpu_mask);
+   (void)odp_cpumask_to_str(&cpu_mask, cpumaskstr, sizeof(cpumaskstr));
+   printf("new cpu mask:   %s\n", cpumaskstr);
+   printf("new num worker threads: %i\n\n", num_workers);
+
+   memset(&thr_params, 0, sizeof(thr_params));
+   thr_params.start= worker_fn

[lng-odp] [PATCHv6 07/38] validation: using implementation agnostic function for ODP threads

[Christophe Milard]

cunit_common is changed to use the implementation agnostic ODP thread
create and join functions, from helpers. Tests are no longer aware if
an odp thread is a linux process or linux thread under the hood.
The helper decides.
The function pointer which is passed when creating the odp threads now
points to a function returning an int instead of a ptr.
This is changed so that when odp threads are processes, the int returned
become the process exit code.
The return code is nevertheless just used to detect erros (as before).
Note that it is now important that the ODP threads return a correct
status, as for processes, odp threads runs on a copy of the memory, so
that c_unit assertions will not be reflected on the main process summary.
Failing to return a proper status means that error will be lost when
running the test in process mode.
odp threads returning an error status code will be delected as an error
on the main process at join time...

Signed-off-by: Christophe Milard 
---
 test/validation/atomic/atomic.c   | 34 
 test/validation/barrier/barrier.c |  8 +++---
 test/validation/common/odp_cunit_common.c | 14 ++
 test/validation/common/odp_cunit_common.h |  4 +--
 test/validation/lock/lock.c   | 44 +++
 test/validation/scheduler/scheduler.c |  8 +++---
 test/validation/shmem/shmem.c |  6 ++---
 test/validation/thread/thread.c   |  4 +--
 test/validation/timer/timer.c |  4 +--
 9 files changed, 65 insertions(+), 61 deletions(-)

diff --git a/test/validation/atomic/atomic.c b/test/validation/atomic/atomic.c
index 5eec467..0dfd651 100644
--- a/test/validation/atomic/atomic.c
+++ b/test/validation/atomic/atomic.c
@@ -584,7 +584,7 @@ int atomic_init(odp_instance_t *inst)
 }
 
 /* Atomic tests */
-static void *test_atomic_inc_dec_thread(void *arg UNUSED)
+static int test_atomic_inc_dec_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -594,10 +594,10 @@ static void *test_atomic_inc_dec_thread(void *arg UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_add_sub_thread(void *arg UNUSED)
+static int test_atomic_add_sub_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -607,10 +607,10 @@ static void *test_atomic_add_sub_thread(void *arg UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_fetch_inc_dec_thread(void *arg UNUSED)
+static int test_atomic_fetch_inc_dec_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -620,10 +620,10 @@ static void *test_atomic_fetch_inc_dec_thread(void *arg 
UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_fetch_add_sub_thread(void *arg UNUSED)
+static int test_atomic_fetch_add_sub_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -633,10 +633,10 @@ static void *test_atomic_fetch_add_sub_thread(void *arg 
UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_max_min_thread(void *arg UNUSED)
+static int test_atomic_max_min_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -646,10 +646,10 @@ static void *test_atomic_max_min_thread(void *arg UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_cas_inc_dec_thread(void *arg UNUSED)
+static int test_atomic_cas_inc_dec_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -659,10 +659,10 @@ static void *test_atomic_cas_inc_dec_thread(void *arg 
UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_xchg_thread(void *arg UNUSED)
+static int test_atomic_xchg_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -672,10 +672,10 @@ static void *test_atomic_xchg_thread(void *arg UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_non_relaxed_thread(void *arg UNUSED)
+static int test_atomic_non_relaxed_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -685,10 +685,10 @@ static void *test_atomic_non_relaxed_thread(void *arg 
UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void test_atomic_functional(void *func_ptr(void *), int check)
+static void test_atomic_functional(int func_ptr(void *), int check)
 {
pthrd_arg arg;
 
diff --git a/test/validation/barrier/barrier.c 
b/test/validation

[lng-odp] [PATCHv6 08/38] validation: most tests: adding command line argument parsing

[Christophe Milard]

As the tests themselves do not have specific args, they just call
the cunit_common parsing function to pick up cunit_common and helpers
arguments.
This was originally many patches, now squashed on maintener's request.
init and pktio tests are still separated as they have specificities

Signed-off-by: Christophe Milard 
---
 test/validation/atomic/atomic.c  |  6 +-
 test/validation/atomic/atomic.h  |  2 +-
 test/validation/atomic/atomic_main.c |  4 ++--
 test/validation/barrier/barrier.c|  6 +-
 test/validation/barrier/barrier.h|  2 +-
 test/validation/barrier/barrier_main.c   |  4 ++--
 test/validation/buffer/buffer.c  | 10 --
 test/validation/buffer/buffer.h  |  2 +-
 test/validation/buffer/buffer_main.c |  4 ++--
 test/validation/classification/classification.c  | 10 --
 test/validation/classification/classification.h  |  2 +-
 test/validation/classification/classification_main.c |  4 ++--
 test/validation/cpumask/cpumask.c| 10 --
 test/validation/cpumask/cpumask.h|  2 +-
 test/validation/cpumask/cpumask_main.c   |  4 ++--
 test/validation/crypto/crypto.c  |  6 +-
 test/validation/crypto/crypto.h  |  2 +-
 test/validation/crypto/crypto_main.c |  4 ++--
 test/validation/errno/errno.c| 10 --
 test/validation/errno/errno.h|  2 +-
 test/validation/errno/errno_main.c   |  4 ++--
 test/validation/hash/hash.c  | 10 --
 test/validation/hash/hash.h  |  2 +-
 test/validation/hash/hash_main.c |  4 ++--
 test/validation/init/init_main_ok.c  |  4 ++--
 test/validation/lock/lock.c  |  6 +-
 test/validation/lock/lock.h  |  2 +-
 test/validation/lock/lock_main.c |  4 ++--
 test/validation/packet/packet.c  | 10 --
 test/validation/packet/packet.h  |  2 +-
 test/validation/packet/packet_main.c |  4 ++--
 test/validation/pool/pool.c  | 10 --
 test/validation/pool/pool.h  |  2 +-
 test/validation/pool/pool_main.c |  4 ++--
 test/validation/queue/queue.c| 10 --
 test/validation/queue/queue.h|  2 +-
 test/validation/queue/queue_main.c   |  4 ++--
 test/validation/random/random.c  | 10 --
 test/validation/random/random.h  |  2 +-
 test/validation/random/random_main.c |  4 ++--
 test/validation/scheduler/scheduler.c| 10 --
 test/validation/scheduler/scheduler.h|  2 +-
 test/validation/scheduler/scheduler_main.c   |  4 ++--
 test/validation/std_clib/std_clib.c  | 10 --
 test/validation/std_clib/std_clib.h  |  2 +-
 test/validation/std_clib/std_clib_main.c |  4 ++--
 test/validation/system/system.c  | 10 --
 test/validation/system/system.h  |  2 +-
 test/validation/system/system_main.c |  4 ++--
 test/validation/thread/thread.c  | 10 --
 test/validation/thread/thread.h  |  2 +-
 test/validation/thread/thread_main.c |  4 ++--
 test/validation/time/time.c  | 10 --
 test/validation/time/time.h  |  2 +-
 test/validation/time/time_main.c |  4 ++--
 test/validation/timer/timer.c|  6 +-
 test/validation/timer/timer.h|  2 +-
 test/validation/timer/timer_main.c   |  4 ++--
 test/validation/traffic_mngr/traffic_mngr.c  |  6 +-
 test/validation/traffic_mngr/traffic_mngr.h  |  2 +-
 test/validation/traffic_mngr/traffic_mngr_main.c |  4 ++--
 61 files changed, 204 insertions(+), 96 deletions(-)

diff --git a/test/validation/atomic/atomic.c b/test/validation/atomic/atomic.c
index 0dfd651..c4e9345 100644
--- a/test/validation/atomic/atomic.c
+++ b/test/validation/atomic/atomic.c
@@ -866,10 +866,14 @@ odp_suiteinfo_t atomic_suites[] = {
ODP_SUITE_INFO_NULL
 };
 
-int atomic_main(void)
+int atomic_main(int argc, char *argv[])
 {
int ret;
 
+   /* parse common options: */
+   if (odp_cunit_parse_options(argc, argv))
+   return -1;
+
odp_cunit_register_global_init(atomic_init);
 
ret = odp_cunit_register(atomic_suites);
diff --git a/test/validation/atomic/atomic.h b/test/validation/a

[lng-odp] [PATCHv6 09/38] validation: init: adding command line argument parsing

[Christophe Milard]

As the tests themselves do not have specific args, they just call
the cunit_common parsing function to pick up cunit_common and helpers
arguments.

Signed-off-by: Christophe Milard 
---
 test/validation/init/init.c| 18 +++---
 test/validation/init/init.h|  6 +++---
 test/validation/init/init_main_abort.c |  4 ++--
 test/validation/init/init_main_log.c   |  4 ++--
 4 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/test/validation/init/init.c b/test/validation/init/init.c
index a802d41..61055fa 100644
--- a/test/validation/init/init.c
+++ b/test/validation/init/init.c
@@ -52,10 +52,14 @@ static void odp_init_abort(void)
abort();
 }
 
-int init_main_abort(void)
+int init_main_abort(int argc, char *argv[])
 {
int ret;
 
+   /* parse common options: */
+   if (odp_cunit_parse_options(argc, argv))
+   return -1;
+
/* prevent default ODP init: */
odp_cunit_register_global_init(NULL);
odp_cunit_register_global_term(NULL);
@@ -116,10 +120,14 @@ static int odp_init_log(odp_log_level_t level 
__attribute__((unused)),
return r;
 }
 
-int init_main_log(void)
+int init_main_log(int argc, char *argv[])
 {
int ret;
 
+   /* parse common options: */
+   if (odp_cunit_parse_options(argc, argv))
+   return -1;
+
/* prevent default ODP init: */
odp_cunit_register_global_init(NULL);
odp_cunit_register_global_term(NULL);
@@ -157,10 +165,14 @@ odp_suiteinfo_t init_suites_ok[] = {
ODP_SUITE_INFO_NULL,
 };
 
-int init_main_ok(void)
+int init_main_ok(int argc, char *argv[])
 {
int ret;
 
+   /* parse common options: */
+   if (odp_cunit_parse_options(argc, argv))
+   return -1;
+
/* prevent default ODP init: */
odp_cunit_register_global_init(NULL);
odp_cunit_register_global_term(NULL);
diff --git a/test/validation/init/init.h b/test/validation/init/init.h
index 272d426..cad9cf9 100644
--- a/test/validation/init/init.h
+++ b/test/validation/init/init.h
@@ -25,8 +25,8 @@ extern odp_suiteinfo_t init_suites_log[];
 extern odp_suiteinfo_t init_suites_ok[];
 
 /* main test program: */
-int init_main_abort(void);
-int init_main_log(void);
-int init_main_ok(void);
+int init_main_abort(int argc, char *argv[]);
+int init_main_log(int argc, char *argv[]);
+int init_main_ok(int argc, char *argv[]);
 
 #endif
diff --git a/test/validation/init/init_main_abort.c 
b/test/validation/init/init_main_abort.c
index c7bdd9d..2e0faaf 100644
--- a/test/validation/init/init_main_abort.c
+++ b/test/validation/init/init_main_abort.c
@@ -5,7 +5,7 @@
  */
 #include "init.h"
 
-int main(void)
+int main(int argc, char *argv[])
 {
-   return init_main_abort();
+   return init_main_abort(argc, argv);
 }
diff --git a/test/validation/init/init_main_log.c 
b/test/validation/init/init_main_log.c
index f35ac38..41dd00d 100644
--- a/test/validation/init/init_main_log.c
+++ b/test/validation/init/init_main_log.c
@@ -5,7 +5,7 @@
  */
 #include "init.h"
 
-int main(void)
+int main(int argc, char *argv[])
 {
-   return init_main_log();
+   return init_main_log(argc, argv);
 }
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 10/38] validation: pktio: adding command line argument parsing

[Christophe Milard]

As the test itself does not have specific args, it just calls
the cunit_common parsing function to pick up cunit_common and helpers
arguments.

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/test/pktio/pktio_run| 21 ++---
 platform/linux-generic/test/pktio/pktio_run_dpdk   | 17 -
 platform/linux-generic/test/pktio/pktio_run_netmap |  5 -
 platform/linux-generic/test/pktio/pktio_run_pcap   |  5 -
 platform/linux-generic/test/pktio/pktio_run_tap|  6 +-
 test/validation/pktio/pktio.c  | 10 --
 test/validation/pktio/pktio.h  |  2 +-
 test/validation/pktio/pktio_main.c |  4 ++--
 8 files changed, 58 insertions(+), 12 deletions(-)

diff --git a/platform/linux-generic/test/pktio/pktio_run 
b/platform/linux-generic/test/pktio/pktio_run
index 7029ab2..4086fc2 100755
--- a/platform/linux-generic/test/pktio/pktio_run
+++ b/platform/linux-generic/test/pktio/pktio_run
@@ -6,6 +6,15 @@
 # SPDX-License-Identifier: BSD-3-Clause
 #
 
+# Proceed the pktio tests. This script expects at least one argument:
+#  setup)   setup the pktio test environment
+#  cleanup) cleanup the pktio test environment
+#  run) run the pktio tests (setup, run, cleanup)
+# extra arguments are passed unchanged to the test itself (pktio_main)
+# Without arguments, "run" is assumed and no extra argument is passed to the
+# test (legacy mode).
+#
+
 # directories where pktio_main binary can be found:
 # -in the validation dir when running make check (intree or out of tree)
 # -in the script directory, when running after 'make install', or
@@ -59,7 +68,7 @@ run_test()
if [ "$disabletype" != "SKIP" ]; then
export ODP_PKTIO_DISABLE_SOCKET_${distype}=y
fi
-   pktio_main${EXEEXT}
+   pktio_main${EXEEXT} $*
if [ $? -ne 0 ]; then
ret=1
fi
@@ -75,7 +84,7 @@ run_test()
 run()
 {
echo "pktio: using 'loop' device"
-   pktio_main${EXEEXT}
+   pktio_main${EXEEXT} $*
loop_ret=$?
 
# need to be root to run tests with real interfaces
@@ -103,8 +112,14 @@ run()
exit $ret
 }
 
-case "$1" in
+if [ $#  > 0]; then
+   action=$1
+   shift
+fi
+
+case "$action" in
setup)   setup_pktio_env   ;;
cleanup) cleanup_pktio_env ;;
+   run) run ;;
*)   run ;;
 esac
diff --git a/platform/linux-generic/test/pktio/pktio_run_dpdk 
b/platform/linux-generic/test/pktio/pktio_run_dpdk
index 50e910a..1a1efac 100755
--- a/platform/linux-generic/test/pktio/pktio_run_dpdk
+++ b/platform/linux-generic/test/pktio/pktio_run_dpdk
@@ -6,6 +6,15 @@
 # SPDX-License-Identifier: BSD-3-Clause
 #
 
+# Proceed the pktio tests. This script expects at least one argument:
+#  setup)   setup the pktio test environment
+#  cleanup) cleanup the pktio test environment
+#  run) run the pktio tests (setup, run, cleanup)
+# extra arguments are passed unchanged to the test itself (pktio_main)
+# Without arguments, "run" is assumed and no extra argument is passed to the
+# test (legacy mode).
+#
+
 # directories where pktio_main binary can be found:
 # -in the validation dir when running make check (intree or out of tree)
 # -in the script directory, when running after 'make install', or
@@ -46,7 +55,7 @@ run_test()
 {
local ret=0
 
-   pktio_main${EXEEXT}
+   pktio_main${EXEEXT} $*
ret=$?
if [ $ret -ne 0 ]; then
echo "!!! FAILED !!!"
@@ -73,8 +82,14 @@ run()
run_test
 }
 
+if [ $#  > 0]; then
+   action=$1
+   shift
+fi
+
 case "$1" in
setup)   setup_pktio_env   ;;
cleanup) cleanup_pktio_env ;;
+   run) run ;;
*)   run ;;
 esac
diff --git a/platform/linux-generic/test/pktio/pktio_run_netmap 
b/platform/linux-generic/test/pktio/pktio_run_netmap
index b7f2575..b651ea3 100755
--- a/platform/linux-generic/test/pktio/pktio_run_netmap
+++ b/platform/linux-generic/test/pktio/pktio_run_netmap
@@ -6,6 +6,9 @@
 # SPDX-License-Identifier: BSD-3-Clause
 #
 
+# any parameter passed as arguments to this script is passed unchanged to
+# the test itself (pktio_main)
+
 # directories where pktio_main binary can be found:
 # -in the validation dir when running make check (intree or out of tree)
 # -in the script directory, when running after 'make install', or
@@ -45,7 +48,7 @@ run_test()
 {
local ret=0
 
-   pktio_main${EXEEXT}
+   pktio_main${EXEEXT} $*
ret=$?
 
if [ $ret -ne 0 ]; then
diff --git a/platform/linux-generic/test/pktio/pktio_run_pcap 
b/platform/linux-generic/test/pktio/pktio_run_pcap
index c130417..51716fb 100755
--- a/platform/linux-generic/test/pktio/pktio_

[lng-odp] [PATCHv6 11/38] helper: adding a function to merge getopt parameters

[Christophe Milard]

A function merging two sets to getopt parmameters (command line argument
description) into one single set is added

Signed-off-by: Christophe Milard 
---
 helper/include/odp/helper/linux.h | 42 ++
 helper/linux.c| 74 +++
 2 files changed, 116 insertions(+)

diff --git a/helper/include/odp/helper/linux.h 
b/helper/include/odp/helper/linux.h
index f67aa30..9767af4 100644
--- a/helper/include/odp/helper/linux.h
+++ b/helper/include/odp/helper/linux.h
@@ -194,6 +194,48 @@ int odph_odpthreads_create(odph_odpthread_t *thread_tbl,
  */
 int odph_odpthreads_join(odph_odpthread_t *thread_tbl);
 
+/**
+ * Merge getopt options
+ *
+ * Given two sets of getopt options (each containing possibly both short
+ * options -a string- and long options -a option array-) this function
+ * return a single set (i.e. a string for short and an array for long)
+ * being the concatenation of the two given sets.
+ * Due to the fact that the size of these arrays is unknown at compilation
+ * time, this function actually mallocs the the resulting arrays.
+ * The fourth and fith parameters are actually pointers where these malloc'ed
+ * areas are returned.
+ * This means that the caller of this function has to free the two returned
+ * areas!
+ *
+ * @param shortopts1 first set of short options (a string)
+ * @param shortopts2 second set of short options (a string)
+ * @param longopts1  first set of long options (a getopt option array)
+ * @param longopts2  second set of long options (a getopt option array)
+ * @param shortopts  a pointer where the address of the short options list
+ *  (a string) is returned. It contains the concatenation of
+ *  the two given short option strings.
+ * @param longopts   a pointer where the address of the long options list
+ *  (a getopt option array) is returned.
+ *  It contains the concatenation of the two given long
+ *  option arrays.
+ * if any of shortopts1, shortopts2, longopts1, longopts2 is NULL, the
+ * corresponding list as assumed to be empty.
+ * if any of shortopts, longopts is NULL, the corresponding malloc is not
+ * performed.
+ *
+ * @return On success: 0 : both shortopts and longopts are returned (assuming
+ *the given pointer where not null), possibly
+ *pointing to an empty string or an empty option array.
+ *On success, the caller is due to free these areas.
+ *On failure: -1: Nothing is malloc'ed.
+ */
+int odph_merge_getopt_options(const char *shortopts1,
+ const char *shortopts2,
+ const struct option *longopts1,
+ const struct option *longopts2,
+ char **shortopts,
+ struct option **longopts);
 
 /**
  * Parse linux helper options
diff --git a/helper/linux.c b/helper/linux.c
index b8d4f49..d1b7825 100644
--- a/helper/linux.c
+++ b/helper/linux.c
@@ -500,6 +500,80 @@ int odph_odpthreads_join(odph_odpthread_t *thread_tbl)
 }
 
 /*
+ * return the number of elements in an array of getopt options, excluding the
+ * terminating {0,0,0,0}
+ */
+static int get_getopt_options_length(const struct option *longopts)
+{
+   int l = 0;
+
+   if (!longopts)
+   return 0;
+
+   while (longopts[l].name)
+   l++;
+
+   return l;
+}
+
+/* Merge getopt options */
+int odph_merge_getopt_options(const char *shortopts1,
+ const char *shortopts2,
+ const struct option *longopts1,
+ const struct option *longopts2,
+ char **shortopts,
+ struct option **longopts)
+{
+   int shortopts1_len;
+   int shortopts2_len;
+   int longopts1_len;
+   int longopts2_len;
+   int index;
+   int res_index = 0;
+   struct option termination = {0, 0, 0, 0};
+
+   /* merge short options: */
+   if (shortopts) {
+   shortopts1_len = (shortopts1) ? strlen(shortopts1) : 0;
+   shortopts2_len = (shortopts2) ? strlen(shortopts2) : 0;
+   *shortopts = malloc(shortopts1_len + shortopts2_len + 1);
+   if (!*shortopts)
+   return -1;
+
+   (*shortopts)[0] = 0;
+
+   if (shortopts1)
+   strcpy((*shortopts), shortopts1);
+   if (shortopts2)
+   strcat((*shortopts), shortopts2);
+   }
+
+   /* merge long options */
+   if (!longopts)
+   return 0;
+
+   longopts1_len = get_getopt_options_length(longopts1);
+   longopts2_len = get_getopt_options_length(longopts2);
+   *longopts = malloc(sizeof(struct option) *
+   (lon

[lng-odp] [PATCHv6 13/38] performance: odp_scheduling: proc mode done by helper

[Christophe Milard]

Remove the --proc option from odp_scheduling.c and use the helper
functions (odph_odpthreads_create, odph_odpthreads_join)
to handle odp threads. Let helper parse its command line args, hence
recognising the --odph_proc option doing what --proc did.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_scheduling.c | 91 +--
 1 file changed, 31 insertions(+), 60 deletions(-)

diff --git a/test/performance/odp_scheduling.c 
b/test/performance/odp_scheduling.c
index 6592277..3902765 100644
--- a/test/performance/odp_scheduling.c
+++ b/test/performance/odp_scheduling.c
@@ -49,7 +49,6 @@ typedef struct {
 /** Test arguments */
 typedef struct {
int cpu_count;  /**< CPU count */
-   int proc_mode;  /**< Process mode */
 } test_args_t;
 
 
@@ -591,9 +590,9 @@ static int test_schedule_multi(const char *str, int thr,
  *
  * @param arg  Arguments
  *
- * @return NULL on failure
+ * @return non zero on failure
  */
-static void *run_thread(void *arg)
+static int run_thread(void *arg ODP_UNUSED)
 {
int thr;
odp_pool_t msg_pool;
@@ -610,7 +609,7 @@ static void *run_thread(void *arg)
 
if (globals == NULL) {
LOG_ERR("Shared mem lookup failed\n");
-   return NULL;
+   return -1;
}
 
barrier = &globals->barrier;
@@ -630,23 +629,23 @@ static void *run_thread(void *arg)
 
if (msg_pool == ODP_POOL_INVALID) {
LOG_ERR("  [%i] msg_pool not found\n", thr);
-   return NULL;
+   return -1;
}
 
odp_barrier_wait(barrier);
 
if (test_alloc_single(thr, msg_pool))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_alloc_multi(thr, msg_pool))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_plain_queue(thr, msg_pool))
-   return NULL;
+   return -1;
 
/* Low prio */
 
@@ -654,19 +653,19 @@ static void *run_thread(void *arg)
 
if (test_schedule_single("sched_s_lo", thr, msg_pool,
 ODP_SCHED_PRIO_LOWEST, barrier))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_schedule_many("sched_m_lo", thr, msg_pool,
   ODP_SCHED_PRIO_LOWEST, barrier))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_schedule_multi("sched_multi_lo", thr, msg_pool,
ODP_SCHED_PRIO_LOWEST, barrier))
-   return NULL;
+   return -1;
 
/* High prio */
 
@@ -674,24 +673,24 @@ static void *run_thread(void *arg)
 
if (test_schedule_single("sched_s_hi", thr, msg_pool,
 ODP_SCHED_PRIO_HIGHEST, barrier))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_schedule_many("sched_m_hi", thr, msg_pool,
   ODP_SCHED_PRIO_HIGHEST, barrier))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_schedule_multi("sched_multi_hi", thr, msg_pool,
ODP_SCHED_PRIO_HIGHEST, barrier))
-   return NULL;
+   return -1;
 
 
printf("Thread %i exits\n", thr);
fflush(NULL);
-   return arg;
+   return 0;
 }
 
 /**
@@ -762,24 +761,25 @@ static void parse_args(int argc, char *argv[], 
test_args_t *args)
int opt;
int long_index;
 
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
{"help", no_argument, NULL, 'h'},
-   {"proc", no_argument, NULL, 0},
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+c:h";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
+   opterr = 0; /* do not issue errors on helper options */
while (1) {
-   opt = getopt_long(argc, argv, "+c:h", longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
 
if (opt == -1)
break;  /* No more options */
 
switch (opt) {
-   case 0:
-   args->proc_mode = 1;
-   break;
-
case 'c':
args->cpu_count = atoi(optarg);
break;
@@ -801,7 +801,7 @@ static void p

[lng-odp] [PATCHv6 12/38] helper: parsing the complete set of options

[Christophe Milard]

The odph_parse_options() function is given the ability to receive
getopt command line description parameter from it caller, hence allowing
the caller to have some command line parameter(s).
The caller shall first call odph_parse_options() with its own parameter
description as parameter.
odph_parse_options() is then checking the complete set of options,
issuing error message for unknown options (those being neither a caller's
valid command line option or a helper valid command line option),
and collecting the sementic of helper options.
Then the caller shall parse the sementic of its own options,
with the opterr variable set to zero (hence ignoring helper options).

Signed-off-by: Christophe Milard 
---
 helper/include/odp/helper/linux.h | 16 +++-
 helper/linux.c| 24 +++-
 helper/test/odpthreads.c  |  2 +-
 test/validation/common/odp_cunit_common.c |  2 +-
 4 files changed, 36 insertions(+), 8 deletions(-)

diff --git a/helper/include/odp/helper/linux.h 
b/helper/include/odp/helper/linux.h
index 9767af4..71c8027 100644
--- a/helper/include/odp/helper/linux.h
+++ b/helper/include/odp/helper/linux.h
@@ -25,6 +25,7 @@ extern "C" {
 #include 
 
 #include 
+#include 
 #include 
 
 /** Thread parameter for Linux pthreads and processes */
@@ -241,15 +242,28 @@ int odph_merge_getopt_options(const char *shortopts1,
  * Parse linux helper options
  *
  * Parse the command line options. Pick up options meant for the helper itself.
+ * If the caller is also having a set of option to parse, it should include
+ * their description here (shortopts desribes the short options and longopts
+ * describes the long options, as for getopt_long()).
+ * This function will issue errors on unknown arguments, so callers failing
+ * to pass their own command line options description here will see their
+ * options rejected.
+ * (the caller wants to set opterr to zero when parsing its own stuff
+ * with getopts to avoid reacting on helper's options).
  *
  * @param argc argument count
  * @param argv argument values
+ * @param caller_shortopts caller's set of short options (string). or NULL.
+ * @param caller_longopts  caller's set of long options (getopt option array).
+ *or NULL.
  *
  * @return On success: 0
  *On failure: -1 (failure occurs only if a value passed for a helper
  *option is invalid. callers cannot have own options)
  */
-int odph_parse_options(int argc, char *argv[]);
+int odph_parse_options(int argc, char *argv[],
+  const char *caller_shortopts,
+  const struct option *caller_longopts);
 #ifdef __cplusplus
 }
 #endif
diff --git a/helper/linux.c b/helper/linux.c
index d1b7825..5fc09a1 100644
--- a/helper/linux.c
+++ b/helper/linux.c
@@ -16,7 +16,6 @@
 #include 
 #include 
 #include 
-#include 
 
 #include 
 #include 
@@ -576,27 +575,39 @@ int odph_merge_getopt_options(const char *shortopts1,
 /*
  * Parse command line options to extract options affecting helpers.
  */
-int odph_parse_options(int argc, char *argv[])
+int odph_parse_options(int argc, char *argv[],
+  const char *caller_shortopts,
+  const struct option *caller_longopts)
 {
int c;
+   char *shortopts;
+   struct option *longopts;
 
-   static struct option long_options[] = {
+   static struct option helper_long_options[] = {
/* These options set a flag. */
{"odph_proc",   no_argument, &helper_options.proc, 1},
{"odph_thread", no_argument, &helper_options.thrd, 1},
{0, 0, 0, 0}
};
 
+   static char *helper_short_options = "";
+
/* defaults: */
helper_options.proc = false;
helper_options.thrd = false;
 
+   /* merge caller's command line options descriptions with helper's: */
+   if (odph_merge_getopt_options(caller_shortopts, helper_short_options,
+ caller_longopts, helper_long_options,
+ &shortopts, &longopts) < 0)
+   return -1;
+
while (1) {
/* getopt_long stores the option index here. */
int option_index = 0;
 
-   c = getopt_long (argc, argv, "",
-long_options, &option_index);
+   c = getopt_long (argc, argv,
+shortopts, longopts, &option_index);
 
/* Detect the end of the options. */
if (c == -1)
@@ -605,5 +616,8 @@ int odph_parse_options(int argc, char *argv[])
 
optind = 0; /* caller expects this to be zero if it parses too*/
 
+   free(shortopts);
+   free(longopts);
+
return 0;
 }
diff --git a/helper/test/odpthreads.

[lng-odp] [PATCHv6 14/38] performance: odp_pktio_perf: using agnostic function for ODP threads

[Christophe Milard]

odp_pktio_perf is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_pktio_perf is no longer aware on how the odpthread is implemented any
longer.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_pktio_perf.c | 23 +++
 1 file changed, 11 insertions(+), 12 deletions(-)

diff --git a/test/performance/odp_pktio_perf.c 
b/test/performance/odp_pktio_perf.c
index 57635d3..e1efc4a 100644
--- a/test/performance/odp_pktio_perf.c
+++ b/test/performance/odp_pktio_perf.c
@@ -303,7 +303,7 @@ static int send_packets(odp_pktout_queue_t pktout,
  * Main packet transmit routine. Transmit packets at a fixed rate for
  * specified length of time.
  */
-static void *run_thread_tx(void *arg)
+static int run_thread_tx(void *arg)
 {
test_globals_t *globals;
int thr_id;
@@ -381,7 +381,7 @@ static void *run_thread_tx(void *arg)
   odp_time_to_ns(stats->s.idle_ticks) /
   (uint64_t)ODP_TIME_MSEC_IN_NS);
 
-   return NULL;
+   return 0;
 }
 
 static int receive_packets(odp_queue_t queue,
@@ -411,7 +411,7 @@ static int receive_packets(odp_queue_t queue,
return n_ev;
 }
 
-static void *run_thread_rx(void *arg)
+static int run_thread_rx(void *arg)
 {
test_globals_t *globals;
int thr_id, batch_len;
@@ -456,7 +456,7 @@ static void *run_thread_rx(void *arg)
break;
}
 
-   return NULL;
+   return 0;
 }
 
 /*
@@ -601,11 +601,11 @@ static int run_test_single(odp_cpumask_t *thd_mask_tx,
   odp_cpumask_t *thd_mask_rx,
   test_status_t *status)
 {
-   odph_linux_pthread_t thd_tbl[MAX_WORKERS];
+   odph_odpthread_t thd_tbl[MAX_WORKERS];
thread_args_t args_tx, args_rx;
uint64_t expected_tx_cnt;
int num_tx_workers, num_rx_workers;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
memset(&thr_params, 0, sizeof(thr_params));
thr_params.thr_type = ODP_THREAD_WORKER;
@@ -623,7 +623,7 @@ static int run_test_single(odp_cpumask_t *thd_mask_tx,
thr_params.start  = run_thread_rx;
thr_params.arg= &args_rx;
args_rx.batch_len = gbl_args->args.rx_batch_len;
-   odph_linux_pthread_create(&thd_tbl[0], thd_mask_rx, &thr_params);
+   odph_odpthreads_create(&thd_tbl[0], thd_mask_rx, &thr_params);
odp_barrier_wait(&gbl_args->rx_barrier);
num_rx_workers = odp_cpumask_count(thd_mask_rx);
 
@@ -634,13 +634,12 @@ static int run_test_single(odp_cpumask_t *thd_mask_tx,
args_tx.pps   = status->pps_curr / num_tx_workers;
args_tx.duration  = gbl_args->args.duration;
args_tx.batch_len = gbl_args->args.tx_batch_len;
-   odph_linux_pthread_create(&thd_tbl[num_rx_workers], thd_mask_tx,
- &thr_params);
+   odph_odpthreads_create(&thd_tbl[num_rx_workers], thd_mask_tx,
+  &thr_params);
odp_barrier_wait(&gbl_args->tx_barrier);
 
/* wait for transmitter threads to terminate */
-   odph_linux_pthread_join(&thd_tbl[num_rx_workers],
-   num_tx_workers);
+   odph_odpthreads_join(&thd_tbl[num_rx_workers]);
 
/* delay to allow transmitted packets to reach the receivers */
odp_time_wait_ns(SHUTDOWN_DELAY_NS);
@@ -649,7 +648,7 @@ static int run_test_single(odp_cpumask_t *thd_mask_tx,
odp_atomic_store_u32(&shutdown, 1);
 
/* wait for receivers */
-   odph_linux_pthread_join(&thd_tbl[0], num_rx_workers);
+   odph_odpthreads_join(&thd_tbl[0]);
 
if (!status->warmup)
return process_results(expected_tx_cnt, status);
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 15/38] performance: odp_pktio_perf: adding helper cmd line parsing

[Christophe Milard]

odp_pktio_perf now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_pktio_perf.c | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/test/performance/odp_pktio_perf.c 
b/test/performance/odp_pktio_perf.c
index e1efc4a..98ec681 100644
--- a/test/performance/odp_pktio_perf.c
+++ b/test/performance/odp_pktio_perf.c
@@ -916,7 +916,7 @@ static void parse_args(int argc, char *argv[], test_args_t 
*args)
int opt;
int long_index;
 
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
{"txcount",   required_argument, NULL, 't'},
{"txbatch",   required_argument, NULL, 'b'},
@@ -931,6 +931,11 @@ static void parse_args(int argc, char *argv[], test_args_t 
*args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+c:t:b:pR:l:r:i:d:vh";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
args->cpu_count  = 0; /* all CPUs */
args->num_tx_workers = 0; /* defaults to cpu_count+1/2 */
args->tx_batch_len   = BATCH_LEN_MAX;
@@ -941,8 +946,10 @@ static void parse_args(int argc, char *argv[], test_args_t 
*args)
args->schedule   = 1;
args->verbose= 0;
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (1) {
-   opt = getopt_long(argc, argv, "+c:t:b:pR:l:r:i:d:vh",
+   opt = getopt_long(argc, argv, shortopts,
  longopts, &long_index);
 
if (opt == -1)
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 16/38] performance: odp_l2fwd: using agnostic function for ODP threads

[Christophe Milard]

odp_l2fwd is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_l2fwd is hence no longer aware on how the odpthread is implemented.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_l2fwd.c | 25 +
 1 file changed, 13 insertions(+), 12 deletions(-)

diff --git a/test/performance/odp_l2fwd.c b/test/performance/odp_l2fwd.c
index 8b83475..9a8d578 100644
--- a/test/performance/odp_l2fwd.c
+++ b/test/performance/odp_l2fwd.c
@@ -296,7 +296,7 @@ static inline int event_queue_send(odp_queue_t queue, 
odp_packet_t *pkt_tbl,
  *
  * @param arg  thread arguments of type 'thread_args_t *'
  */
-static void *run_worker_sched_mode(void *arg)
+static int run_worker_sched_mode(void *arg)
 {
odp_event_t  ev_tbl[MAX_PKT_BURST];
odp_packet_t pkt_tbl[MAX_PKT_BURST];
@@ -398,7 +398,7 @@ static void *run_worker_sched_mode(void *arg)
/* Make sure that latest stat writes are visible to other threads */
odp_mb_full();
 
-   return NULL;
+   return 0;
 }
 
 /**
@@ -406,7 +406,7 @@ static void *run_worker_sched_mode(void *arg)
  *
  * @param arg  thread arguments of type 'thread_args_t *'
  */
-static void *run_worker_plain_queue_mode(void *arg)
+static int run_worker_plain_queue_mode(void *arg)
 {
int thr;
int pkts;
@@ -497,7 +497,7 @@ static void *run_worker_plain_queue_mode(void *arg)
/* Make sure that latest stat writes are visible to other threads */
odp_mb_full();
 
-   return NULL;
+   return 0;
 }
 
 /**
@@ -505,7 +505,7 @@ static void *run_worker_plain_queue_mode(void *arg)
  *
  * @param arg  thread arguments of type 'thread_args_t *'
  */
-static void *run_worker_direct_mode(void *arg)
+static int run_worker_direct_mode(void *arg)
 {
int thr;
int pkts;
@@ -591,7 +591,7 @@ static void *run_worker_direct_mode(void *arg)
/* Make sure that latest stat writes are visible to other threads */
odp_mb_full();
 
-   return NULL;
+   return 0;
 }
 
 /**
@@ -1287,7 +1287,7 @@ static void gbl_args_init(args_t *args)
  */
 int main(int argc, char *argv[])
 {
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
odp_pool_t pool;
int i;
int cpu;
@@ -1300,7 +1300,7 @@ int main(int argc, char *argv[])
int ret;
stats_t *stats;
int if_count;
-   void *(*thr_run_func)(void *);
+   int (*thr_run_func)(void *);
odp_instance_t instance;
 
/* Init ODP before calling anything else */
@@ -1435,7 +1435,7 @@ int main(int argc, char *argv[])
cpu = odp_cpumask_first(&cpumask);
for (i = 0; i < num_workers; ++i) {
odp_cpumask_t thd_mask;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
memset(&thr_params, 0, sizeof(thr_params));
thr_params.start= thr_run_func;
@@ -1447,8 +1447,8 @@ int main(int argc, char *argv[])
 
odp_cpumask_zero(&thd_mask);
odp_cpumask_set(&thd_mask, cpu);
-   odph_linux_pthread_create(&thread_tbl[i], &thd_mask,
- &thr_params);
+   odph_odpthreads_create(&thread_tbl[i], &thd_mask,
+  &thr_params);
cpu = odp_cpumask_next(&cpumask, cpu);
}
 
@@ -1470,7 +1470,8 @@ int main(int argc, char *argv[])
exit_threads = 1;
 
/* Master thread waits for other threads to exit */
-   odph_linux_pthread_join(thread_tbl, num_workers);
+   for (i = 0; i < num_workers; ++i)
+   odph_odpthreads_join(&thread_tbl[i]);
 
free(gbl_args->appl.if_names);
free(gbl_args->appl.if_str);
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 17/38] performance: odp_l2fwd: adding helper cmd line parsing

[Christophe Milard]

odp_l2fwd now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_l2fwd.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/test/performance/odp_l2fwd.c b/test/performance/odp_l2fwd.c
index 9a8d578..e38e6f8 100644
--- a/test/performance/odp_l2fwd.c
+++ b/test/performance/odp_l2fwd.c
@@ -1051,7 +1051,7 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
char *addr_str;
size_t len;
int i;
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
{"time", required_argument, NULL, 't'},
{"accuracy", required_argument, NULL, 'a'},
@@ -1066,15 +1066,21 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts =  "+c:+t:+a:i:m:o:r:d:s:e:h";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
appl_args->time = 0; /* loop forever if time to run is 0 */
appl_args->accuracy = 1; /* get and print pps stats second */
appl_args->dst_change = 1; /* change eth dst address by default */
appl_args->src_change = 1; /* change eth src address by default */
appl_args->error_check = 0; /* don't check packet errors by default */
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (1) {
-   opt = getopt_long(argc, argv, "+c:+t:+a:i:m:o:r:d:s:e:h",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
 
if (opt == -1)
break;  /* No more options */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 18/38] performance: crypto: using agnostic function for ODP threads

[Christophe Milard]

odp_crypto is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_crypto is hence no longer aware on how the odpthread is implemented.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_crypto.c | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/test/performance/odp_crypto.c b/test/performance/odp_crypto.c
index 595a266..893a5a3 100644
--- a/test/performance/odp_crypto.c
+++ b/test/performance/odp_crypto.c
@@ -704,13 +704,13 @@ typedef struct thr_arg {
crypto_alg_config_t *crypto_alg_config;
 } thr_arg_t;
 
-static void *run_thr_func(void *arg)
+static int run_thr_func(void *arg)
 {
thr_arg_t *thr_args = (thr_arg_t *)arg;
 
run_measure_one_config(&thr_args->crypto_args,
   thr_args->crypto_alg_config);
-   return NULL;
+   return 0;
 }
 
 int main(int argc, char *argv[])
@@ -724,7 +724,7 @@ int main(int argc, char *argv[])
odp_cpumask_t cpumask;
char cpumaskstr[ODP_CPUMASK_STR_SIZE];
int num_workers = 1;
-   odph_linux_pthread_t thr[num_workers];
+   odph_odpthread_t thr[num_workers];
odp_instance_t instance;
 
memset(&cargs, 0, sizeof(cargs));
@@ -797,7 +797,7 @@ int main(int argc, char *argv[])
memset(thr, 0, sizeof(thr));
 
if (cargs.alg_config) {
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
memset(&thr_params, 0, sizeof(thr_params));
thr_params.start= run_thr_func;
@@ -806,8 +806,8 @@ int main(int argc, char *argv[])
thr_params.instance = instance;
 
if (cargs.schedule) {
-   odph_linux_pthread_create(&thr[0], &cpumask, 
&thr_params);
-   odph_linux_pthread_join(&thr[0], num_workers);
+   odph_odpthreads_create(&thr[0], &cpumask, &thr_params);
+   odph_odpthreads_join(&thr[0]);
} else {
run_measure_one_config(&cargs, cargs.alg_config);
}
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 19/38] performance: crypto: adding helper cmd line parsing

[Christophe Milard]

odp_crypto now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_crypto.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/test/performance/odp_crypto.c b/test/performance/odp_crypto.c
index 893a5a3..404984d 100644
--- a/test/performance/odp_crypto.c
+++ b/test/performance/odp_crypto.c
@@ -828,7 +828,7 @@ static void parse_args(int argc, char *argv[], 
crypto_args_t *cargs)
 {
int opt;
int long_index;
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"algorithm", optional_argument, NULL, 'a'},
{"debug",  no_argument, NULL, 'd'},
{"flight", optional_argument, NULL, 'f'},
@@ -843,6 +843,11 @@ static void parse_args(int argc, char *argv[], 
crypto_args_t *cargs)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+a:c:df:hi:m:nl:spr";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
cargs->in_place = 0;
cargs->in_flight = 1;
cargs->debug_packets = 0;
@@ -852,9 +857,10 @@ static void parse_args(int argc, char *argv[], 
crypto_args_t *cargs)
cargs->reuse_packet = 0;
cargs->schedule = 0;
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (1) {
-   opt = getopt_long(argc, argv, "+a:c:df:hi:m:nl:spr",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
 
if (opt == -1)
break;  /* No more options */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 20/38] example: classifier: using agnostic function for ODP threads

[Christophe Milard]

odp_classifier is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_classifier is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/classifier/odp_classifier.c | 27 +++
 1 file changed, 7 insertions(+), 20 deletions(-)

diff --git a/example/classifier/odp_classifier.c 
b/example/classifier/odp_classifier.c
index 071788d..5982446 100644
--- a/example/classifier/odp_classifier.c
+++ b/example/classifier/odp_classifier.c
@@ -254,7 +254,7 @@ static odp_pktio_t create_pktio(const char *dev, odp_pool_t 
pool)
  * Worker threads to receive the packet
  *
  */
-static void *pktio_receive_thread(void *arg)
+static int pktio_receive_thread(void *arg)
 {
int thr;
odp_pktout_queue_t pktout;
@@ -298,7 +298,7 @@ static void *pktio_receive_thread(void *arg)
 
if (odp_pktout_queue(pktio_tmp, &pktout, 1) != 1) {
EXAMPLE_ERR("  [%02i] Error: no output queue\n", thr);
-   return NULL;
+   return -1;
}
 
pool = odp_packet_pool(pkt);
@@ -324,7 +324,7 @@ static void *pktio_receive_thread(void *arg)
}
}
 
-   return NULL;
+   return 0;
 }
 
 static odp_cos_t configure_default_cos(odp_pktio_t pktio, appl_args_t *args)
@@ -469,11 +469,10 @@ static void configure_cos(odp_cos_t default_cos, 
appl_args_t *args)
  */
 int main(int argc, char *argv[])
 {
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
odp_pool_t pool;
int num_workers;
int i;
-   int cpu;
odp_cpumask_t cpumask;
char cpumaskstr[ODP_CPUMASK_STR_SIZE];
odp_pool_param_t params;
@@ -483,7 +482,7 @@ int main(int argc, char *argv[])
odp_shm_t shm;
int ret;
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
/* Init ODP before calling anything else */
if (odp_init_global(&instance, NULL, NULL)) {
@@ -571,25 +570,13 @@ int main(int argc, char *argv[])
thr_params.arg  = args;
thr_params.thr_type = ODP_THREAD_WORKER;
thr_params.instance = instance;
-
-   cpu = odp_cpumask_first(&cpumask);
-   for (i = 0; i < num_workers; ++i) {
-   odp_cpumask_t thd_mask;
-   /*
-* Calls odp_thread_create(cpu) for each thread
-*/
-   odp_cpumask_zero(&thd_mask);
-   odp_cpumask_set(&thd_mask, cpu);
-   odph_linux_pthread_create(&thread_tbl[i], &thd_mask,
- &thr_params);
-   cpu = odp_cpumask_next(&cpumask, cpu);
-   }
+   odph_odpthreads_create(thread_tbl, &cpumask, &thr_params);
 
print_cls_statistics(args);
 
odp_pktio_stop(pktio);
shutdown = 1;
-   odph_linux_pthread_join(thread_tbl, num_workers);
+   odph_odpthreads_join(thread_tbl);
 
for (i = 0; i < args->policy_count; i++) {
if ((i !=  args->policy_count - 1) &&
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 21/38] example: classifier: adding helper cmd line parsing

[Christophe Milard]

odp_classifier now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/classifier/odp_classifier.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/example/classifier/odp_classifier.c 
b/example/classifier/odp_classifier.c
index 5982446..20e64ec 100644
--- a/example/classifier/odp_classifier.c
+++ b/example/classifier/odp_classifier.c
@@ -792,7 +792,7 @@ static void parse_args(int argc, char *argv[], appl_args_t 
*appl_args)
int interface = 0;
int policy = 0;
 
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
{"interface", required_argument, NULL, 'i'},/* return 'i' */
{"policy", required_argument, NULL, 'p'},   /* return 'p' */
@@ -802,10 +802,16 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+c:t:i:p:m:t:h";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
+   opterr = 0; /* do not issue errors on helper options */
 
while (1) {
-   opt = getopt_long(argc, argv, "+c:t:i:p:m:t:h",
-   longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts,
+ longopts, &long_index);
 
if (opt == -1)
break;  /* No more options */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 22/38] example: generator: using agnostic function for ODP threads

[Christophe Milard]

odp_generator is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_generator is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/generator/odp_generator.c | 33 -
 1 file changed, 16 insertions(+), 17 deletions(-)

diff --git a/example/generator/odp_generator.c 
b/example/generator/odp_generator.c
index fd1141a..dd3e023 100644
--- a/example/generator/odp_generator.c
+++ b/example/generator/odp_generator.c
@@ -352,7 +352,7 @@ static odp_pktio_t create_pktio(const char *dev, odp_pool_t 
pool)
  * @param arg  thread arguments of type 'thread_args_t *'
  */
 
-static void *gen_send_thread(void *arg)
+static int gen_send_thread(void *arg)
 {
int thr;
int ret;
@@ -368,12 +368,12 @@ static void *gen_send_thread(void *arg)
if (pktio == ODP_PKTIO_INVALID) {
EXAMPLE_ERR("  [%02i] Error: lookup of pktio %s failed\n",
thr, thr_args->pktio_dev);
-   return NULL;
+   return -1;
}
 
if (odp_pktout_queue(pktio, &pktout, 1) != 1) {
EXAMPLE_ERR("  [%02i] Error: no output queue\n", thr);
-   return NULL;
+   return -1;
}
 
printf("  [%02i] created mode: SEND\n", thr);
@@ -441,7 +441,7 @@ static void *gen_send_thread(void *arg)
}
}
 
-   return arg;
+   return 0;
 }
 
 /**
@@ -515,7 +515,7 @@ static void print_pkts(int thr, odp_packet_t pkt_tbl[], 
unsigned len)
  *
  * @param arg  thread arguments of type 'thread_args_t *'
  */
-static void *gen_recv_thread(void *arg)
+static int gen_recv_thread(void *arg)
 {
int thr;
odp_pktio_t pktio;
@@ -530,7 +530,7 @@ static void *gen_recv_thread(void *arg)
if (pktio == ODP_PKTIO_INVALID) {
EXAMPLE_ERR("  [%02i] Error: lookup of pktio %s failed\n",
thr, thr_args->pktio_dev);
-   return NULL;
+   return -1;
}
 
printf("  [%02i] created mode: RECEIVE\n", thr);
@@ -558,7 +558,7 @@ static void *gen_recv_thread(void *arg)
odp_packet_free(pkt);
}
 
-   return arg;
+   return 0;
 }
 
 /**
@@ -621,7 +621,7 @@ static void print_global_stats(int num_workers)
  */
 int main(int argc, char *argv[])
 {
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
odp_pool_t pool;
int num_workers;
int i;
@@ -636,7 +636,7 @@ int main(int argc, char *argv[])
odp_event_t ev;
odp_pktio_t *pktio;
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
/* Init ODP before calling anything else */
if (odp_init_global(&instance, NULL, NULL)) {
@@ -786,8 +786,7 @@ int main(int argc, char *argv[])
thr_params.thr_type = ODP_THREAD_WORKER;
thr_params.instance = instance;
 
-   odph_linux_pthread_create(&thread_tbl[1], &cpu_mask,
- &thr_params);
+   odph_odpthreads_create(&thread_tbl[1], &cpu_mask, &thr_params);
 
tq = odp_queue_create("", NULL);
if (tq == ODP_QUEUE_INVALID)
@@ -810,14 +809,13 @@ int main(int argc, char *argv[])
thr_params.start = gen_send_thread;
thr_params.arg   = &args->thread[0];
 
-   odph_linux_pthread_create(&thread_tbl[0], &cpu_mask,
- &thr_params);
+   odph_odpthreads_create(&thread_tbl[0], &cpu_mask, &thr_params);
 
} else {
int cpu = odp_cpumask_first(&cpumask);
for (i = 0; i < num_workers; ++i) {
odp_cpumask_t thd_mask;
-   void *(*thr_run_func) (void *);
+   int (*thr_run_func)(void *);
int if_idx;
 
if_idx = i % args->appl.if_count;
@@ -856,8 +854,8 @@ int main(int argc, char *argv[])
thr_params.start = thr_run_func;
thr_params.arg   = &args->thread[i];
 
-   odph_linux_pthread_create(&thread_tbl[i],
- &thd_mask, &thr_params);
+   odph_odpthreads_create(&thread_tbl[i],
+  &thd_mask, &thr_params);
cpu = odp_cpumask_next(&cpumask, cpu);
 
}
@@ -866,7 +864,8 @@ int main(int argc, char *argv[])
print_global_stats(num_workers);
 
/* Master thread waits for other th

[lng-odp] [PATCHv6 23/38] example: generator: adding helper cmd line parsing

[Christophe Milard]

odp_generator now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/generator/odp_generator.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/example/generator/odp_generator.c 
b/example/generator/odp_generator.c
index dd3e023..ccae907 100644
--- a/example/generator/odp_generator.c
+++ b/example/generator/odp_generator.c
@@ -919,7 +919,7 @@ static void parse_args(int argc, char *argv[], appl_args_t 
*appl_args)
size_t len;
odp_cpumask_t cpumask, cpumask_args, cpumask_and;
int i, num_workers;
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"interface", required_argument, NULL, 'I'},
{"workers", required_argument, NULL, 'w'},
{"cpumask", required_argument, NULL, 'c'},
@@ -936,14 +936,20 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+I:a:b:s:d:p:i:m:n:t:w:c:h";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
appl_args->mode = -1; /* Invalid, must be changed by parsing */
appl_args->number = -1;
appl_args->payload = 56;
appl_args->timeout = -1;
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (1) {
-   opt = getopt_long(argc, argv, "+I:a:b:s:d:p:i:m:n:t:w:c:h",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
if (opt == -1)
break;  /* No more options */
 
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 25/38] example: ipsec: adding helper cmd line parsing

[Christophe Milard]

odp_ipsec now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/ipsec/odp_ipsec.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/example/ipsec/odp_ipsec.c b/example/ipsec/odp_ipsec.c
index 4d08772..fb4385f 100644
--- a/example/ipsec/odp_ipsec.c
+++ b/example/ipsec/odp_ipsec.c
@@ -1381,7 +1381,7 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
int rc = 0;
int i;
 
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
{"interface", required_argument, NULL, 'i'},/* return 'i' */
{"mode", required_argument, NULL, 'm'}, /* return 'm' */
@@ -1395,13 +1395,19 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+c:i:m:h:r:p:a:e:t:s:";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
printf("\nParsing command line options\n");
 
appl_args->mode = 0;  /* turn off async crypto API by default */
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (!rc) {
-   opt = getopt_long(argc, argv, "+c:i:m:h:r:p:a:e:t:s:",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
 
if (-1 == opt)
break;  /* No more options */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 24/38] example: ipsec: using agnostic function for ODP threads

[Christophe Milard]

odp_ipsec is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_ipsec is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/ipsec/odp_ipsec.c | 15 +++
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/example/ipsec/odp_ipsec.c b/example/ipsec/odp_ipsec.c
index e75db61..4d08772 100644
--- a/example/ipsec/odp_ipsec.c
+++ b/example/ipsec/odp_ipsec.c
@@ -1047,12 +1047,12 @@ pkt_disposition_e do_ipsec_out_finish(odp_packet_t pkt,
  *  - Sequence number assignment queue
  *  - Per packet crypto API completion queue
  *
- * @param arg  Required by "odph_linux_pthread_create", unused
+ * @param arg  Required by "odph_odpthreads_create", unused
  *
  * @return NULL (should never return)
  */
 static
-void *pktio_thread(void *arg EXAMPLE_UNUSED)
+int pktio_thread(void *arg EXAMPLE_UNUSED)
 {
int thr;
odp_packet_t pkt;
@@ -1203,7 +1203,7 @@ void *pktio_thread(void *arg EXAMPLE_UNUSED)
}
 
/* unreachable */
-   return NULL;
+   return 0;
 }
 
 /**
@@ -1212,7 +1212,7 @@ void *pktio_thread(void *arg EXAMPLE_UNUSED)
 int
 main(int argc, char *argv[])
 {
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
int num_workers;
int i;
int stream_count;
@@ -1221,7 +1221,7 @@ main(int argc, char *argv[])
char cpumaskstr[ODP_CPUMASK_STR_SIZE];
odp_pool_param_t params;
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
/* create by default scheduled queues */
queue_create = odp_queue_create;
@@ -1341,8 +1341,7 @@ main(int argc, char *argv[])
thr_params.arg  = NULL;
thr_params.thr_type = ODP_THREAD_WORKER;
thr_params.instance = instance;
-
-   odph_linux_pthread_create(thread_tbl, &cpumask, &thr_params);
+   odph_odpthreads_create(thread_tbl, &cpumask, &thr_params);
 
/*
 * If there are streams attempt to verify them else
@@ -1356,7 +1355,7 @@ main(int argc, char *argv[])
} while (!done);
printf("All received\n");
} else {
-   odph_linux_pthread_join(thread_tbl, num_workers);
+   odph_odpthreads_join(thread_tbl);
}
 
free(args->appl.if_names);
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 26/38] example: l2fwd_simple: using agnostic function for ODP threads

[Christophe Milard]

l2fwd_simple is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
l2fwd_simple is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/l2fwd_simple/odp_l2fwd_simple.c | 15 ---
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/example/l2fwd_simple/odp_l2fwd_simple.c 
b/example/l2fwd_simple/odp_l2fwd_simple.c
index fb096ab..c075744 100644
--- a/example/l2fwd_simple/odp_l2fwd_simple.c
+++ b/example/l2fwd_simple/odp_l2fwd_simple.c
@@ -68,7 +68,7 @@ static odp_pktio_t create_pktio(const char *name, odp_pool_t 
pool,
return pktio;
 }
 
-static void *run_worker(void *arg ODP_UNUSED)
+static int run_worker(void *arg ODP_UNUSED)
 {
odp_packet_t pkt_tbl[MAX_PKT_BURST];
int pkts, sent, tx_drops, i;
@@ -97,7 +97,7 @@ static void *run_worker(void *arg ODP_UNUSED)
 
if (odp_unlikely(!odp_packet_has_eth(pkt))) {
printf("warning: packet has no eth header\n");
-   return NULL;
+   return 0;
}
eth = (odph_ethhdr_t *)odp_packet_l2_ptr(pkt, NULL);
eth->src = global.src;
@@ -110,7 +110,7 @@ static void *run_worker(void *arg ODP_UNUSED)
if (odp_unlikely(tx_drops))
odp_packet_free_multi(&pkt_tbl[sent], tx_drops);
}
-   return NULL;
+   return 0;
 }
 
 int main(int argc, char **argv)
@@ -118,9 +118,9 @@ int main(int argc, char **argv)
odp_pool_t pool;
odp_pool_param_t params;
odp_cpumask_t cpumask;
-   odph_linux_pthread_t thd;
+   odph_odpthread_t thd;
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
if (argc != 5 ||
odph_eth_addr_parse(&global.dst, argv[3]) != 0 ||
@@ -169,7 +169,8 @@ int main(int argc, char **argv)
thr_params.thr_type = ODP_THREAD_WORKER;
thr_params.instance = instance;
 
-   odph_linux_pthread_create(&thd, &cpumask, &thr_params);
-   odph_linux_pthread_join(&thd, 1);
+   odph_odpthreads_create(&thd, &cpumask, &thr_params);
+   odph_odpthreads_join(&thd);
+
return 0;
 }
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 27/38] example: l2fwd_simple: adding helper cmd line parsing

[Christophe Milard]

l2fwd_simple now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/l2fwd_simple/odp_l2fwd_simple.c | 32 +++-
 1 file changed, 27 insertions(+), 5 deletions(-)

diff --git a/example/l2fwd_simple/odp_l2fwd_simple.c 
b/example/l2fwd_simple/odp_l2fwd_simple.c
index c075744..daae038 100644
--- a/example/l2fwd_simple/odp_l2fwd_simple.c
+++ b/example/l2fwd_simple/odp_l2fwd_simple.c
@@ -6,6 +6,7 @@
 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -121,10 +122,29 @@ int main(int argc, char **argv)
odph_odpthread_t thd;
odp_instance_t instance;
odph_odpthread_params_t thr_params;
+   int opt;
+   int long_index;
+
+   static const struct option longopts[] = { {NULL, 0, NULL, 0} };
+   static const char *shortopts = "";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
+   /*
+* parse own options: currentely none, but this will move optind
+* to the first non-option argument. (in case there where helprt args)
+*/
+   opterr = 0; /* do not issue errors on helper options */
+   while (1) {
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
+   if (-1 == opt)
+   break;  /* No more options */
+   }
 
-   if (argc != 5 ||
-   odph_eth_addr_parse(&global.dst, argv[3]) != 0 ||
-   odph_eth_addr_parse(&global.src, argv[4]) != 0) {
+   if (argc != optind + 4 ||
+   odph_eth_addr_parse(&global.dst, argv[optind + 2]) != 0 ||
+   odph_eth_addr_parse(&global.src, argv[optind + 3]) != 0) {
printf("Usage: odp_l2fwd_simple eth0 eth1 01:02:03:04:05:06"
   " 07:08:09:0a:0b:0c\n");
printf("Where eth0 and eth1 are the used interfaces"
@@ -158,8 +178,10 @@ int main(int argc, char **argv)
exit(1);
}
 
-   global.if0 = create_pktio(argv[1], pool, &global.if0in, &global.if0out);
-   global.if1 = create_pktio(argv[2], pool, &global.if1in, &global.if1out);
+   global.if0 = create_pktio(argv[optind], pool, &global.if0in,
+   &global.if0out);
+   global.if1 = create_pktio(argv[optind + 1], pool, &global.if1in,
+   &global.if1out);
 
odp_cpumask_default_worker(&cpumask, 1);
 
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 28/38] example: pktio: using agnostic function for ODP threads

[Christophe Milard]

pktio is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
pktio is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/packet/odp_pktio.c | 32 
 1 file changed, 16 insertions(+), 16 deletions(-)

diff --git a/example/packet/odp_pktio.c b/example/packet/odp_pktio.c
index dce9447..f500cf0 100644
--- a/example/packet/odp_pktio.c
+++ b/example/packet/odp_pktio.c
@@ -169,7 +169,7 @@ static odp_pktio_t create_pktio(const char *dev, odp_pool_t 
pool, int mode)
  *
  * @param arg  thread arguments of type 'thread_args_t *'
  */
-static void *pktio_queue_thread(void *arg)
+static int pktio_queue_thread(void *arg)
 {
int thr;
odp_pktio_t pktio;
@@ -189,7 +189,7 @@ static void *pktio_queue_thread(void *arg)
if (pktio == ODP_PKTIO_INVALID) {
EXAMPLE_ERR("  [%02i] Error: lookup of pktio %s failed\n",
thr, thr_args->pktio_dev);
-   return NULL;
+   return -1;
}
 
printf("  [%02i] looked up pktio:%02" PRIu64
@@ -203,7 +203,7 @@ static void *pktio_queue_thread(void *arg)
(odp_pktin_event_queue(pktio, &inq, 1) != 1)) {
EXAMPLE_ERR("  [%02i] Error: no input queue for %s\n",
thr, thr_args->pktio_dev);
-   return NULL;
+   return -1;
}
 
/* Loop packets */
@@ -232,7 +232,7 @@ static void *pktio_queue_thread(void *arg)
 
if (odp_pktout_queue(pktio_tmp, &pktout, 1) != 1) {
EXAMPLE_ERR("  [%02i] Error: no pktout queue\n", thr);
-   return NULL;
+   return -1;
}
 
/* Swap Eth MACs and possibly IP-addrs before sending back */
@@ -252,7 +252,7 @@ static void *pktio_queue_thread(void *arg)
}
}
 
-   return NULL;
+   return 0;
 }
 
 /**
@@ -260,7 +260,7 @@ static void *pktio_queue_thread(void *arg)
  *
  * @param arg  thread arguments of type 'thread_args_t *'
  */
-static void *pktio_ifburst_thread(void *arg)
+static int pktio_ifburst_thread(void *arg)
 {
int thr;
odp_pktio_t pktio;
@@ -280,7 +280,7 @@ static void *pktio_ifburst_thread(void *arg)
if (pktio == ODP_PKTIO_INVALID) {
EXAMPLE_ERR("  [%02i] Error: lookup of pktio %s failed\n",
thr, thr_args->pktio_dev);
-   return NULL;
+   return -1;
}
 
printf("  [%02i] looked up pktio:%02" PRIu64 ", burst mode\n",
@@ -288,12 +288,12 @@ static void *pktio_ifburst_thread(void *arg)
 
if (odp_pktin_queue(pktio, &pktin, 1) != 1) {
EXAMPLE_ERR("  [%02i] Error: no pktin queue\n", thr);
-   return NULL;
+   return -1;
}
 
if (odp_pktout_queue(pktio, &pktout, 1) != 1) {
EXAMPLE_ERR("  [%02i] Error: no pktout queue\n", thr);
-   return NULL;
+   return -1;
}
 
/* Loop packets */
@@ -334,7 +334,7 @@ static void *pktio_ifburst_thread(void *arg)
}
}
 
-   return NULL;
+   return 0;
 }
 
 /**
@@ -342,7 +342,7 @@ static void *pktio_ifburst_thread(void *arg)
  */
 int main(int argc, char *argv[])
 {
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
odp_pool_t pool;
int num_workers;
int i;
@@ -351,7 +351,7 @@ int main(int argc, char *argv[])
char cpumaskstr[ODP_CPUMASK_STR_SIZE];
odp_pool_param_t params;
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
args = calloc(1, sizeof(args_t));
if (args == NULL) {
@@ -419,7 +419,7 @@ int main(int argc, char *argv[])
cpu = odp_cpumask_first(&cpumask);
for (i = 0; i < num_workers; ++i) {
odp_cpumask_t thd_mask;
-   void *(*thr_run_func) (void *);
+   int (*thr_run_func)(void *);
int if_idx;
 
if_idx = i % args->appl.if_count;
@@ -442,8 +442,7 @@ int main(int argc, char *argv[])
thr_params.start = thr_run_func;
thr_params.arg   = &args->thread[i];
 
-   odph_linux_pthread_create(&thread_tbl[i], &thd_mask,
- &thr_params);
+   odph_odpthreads_create(&thread_tbl[i], &thd_mask, &thr_params);
cpu = odp_cpumask_next(&cpumask, cpu);
}
 
@@ -462,7 +461,8 @@ int main(int argc, char *argv[])
}
 
/* Master thread waits for other threads to exit */
-   odph_linux_

[lng-odp] [PATCHv6 29/38] example: pktio: adding helper cmd line parsing

[Christophe Milard]

pktio now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/packet/odp_pktio.c | 13 +
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/example/packet/odp_pktio.c b/example/packet/odp_pktio.c
index f500cf0..9028ab2 100644
--- a/example/packet/odp_pktio.c
+++ b/example/packet/odp_pktio.c
@@ -559,8 +559,7 @@ static void parse_args(int argc, char *argv[], appl_args_t 
*appl_args)
char *token;
size_t len;
int i;
-
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
{"time", required_argument, NULL, 't'},
{"interface", required_argument, NULL, 'i'},/* return 'i' */
@@ -569,12 +568,18 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+c:i:+m:t:h";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
appl_args->mode = APPL_MODE_PKT_SCHED;
appl_args->time = 0; /**< loop forever */
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (1) {
-   opt = getopt_long(argc, argv, "+c:i:+m:t:h",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
 
if (opt == -1)
break;  /* No more options */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 30/38] example: time: using agnostic function for ODP threads

[Christophe Milard]

time is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
time is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/time/time_global_test.c | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/example/time/time_global_test.c b/example/time/time_global_test.c
index 8e3de5c..df4da4e 100644
--- a/example/time/time_global_test.c
+++ b/example/time/time_global_test.c
@@ -163,7 +163,7 @@ static void test_global_timestamps(test_globals_t *gbls,
  *
  * @return Pointer to exit status
  */
-static void *run_thread(void *ptr)
+static int run_thread(void *ptr)
 {
int thr;
uint32_t id;
@@ -238,7 +238,7 @@ static void *run_thread(void *ptr)
 
printf("Thread %i exits\n", thr);
fflush(NULL);
-   return NULL;
+   return 0;
 }
 
 int main(void)
@@ -252,9 +252,9 @@ int main(void)
odp_shm_t shm_glbls = ODP_SHM_INVALID;
odp_shm_t shm_log = ODP_SHM_INVALID;
int log_size, log_enries_num;
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
printf("\nODP global time test starts\n");
 
@@ -323,10 +323,10 @@ int main(void)
thr_params.instance = instance;
 
/* Create and launch worker threads */
-   odph_linux_pthread_create(thread_tbl, &cpumask, &thr_params);
+   odph_odpthreads_create(thread_tbl, &cpumask, &thr_params);
 
/* Wait for worker threads to exit */
-   odph_linux_pthread_join(thread_tbl, num_workers);
+   odph_odpthreads_join(thread_tbl);
 
print_log(gbls);
 
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 31/38] example: time: adding helper cmd line parsing

[Christophe Milard]

time now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/time/time_global_test.c | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/example/time/time_global_test.c b/example/time/time_global_test.c
index df4da4e..372d96b 100644
--- a/example/time/time_global_test.c
+++ b/example/time/time_global_test.c
@@ -241,7 +241,7 @@ static int run_thread(void *ptr)
return 0;
 }
 
-int main(void)
+int main(int argc, char *argv[])
 {
int err = 0;
odp_pool_t pool = ODP_POOL_INVALID;
@@ -256,6 +256,9 @@ int main(void)
odp_instance_t instance;
odph_odpthread_params_t thr_params;
 
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, NULL, NULL);
+
printf("\nODP global time test starts\n");
 
if (odp_init_global(&instance, NULL, NULL)) {
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 33/38] example: timer: adding helper cmd line parsing

[Christophe Milard]

timer_test now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/timer/odp_timer_test.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/example/timer/odp_timer_test.c b/example/timer/odp_timer_test.c
index 87690ab..5c86555 100644
--- a/example/timer/odp_timer_test.c
+++ b/example/timer/odp_timer_test.c
@@ -257,7 +257,7 @@ static void parse_args(int argc, char *argv[], test_args_t 
*args)
int opt;
int long_index;
 
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count",  required_argument, NULL, 'c'},
{"resolution", required_argument, NULL, 'r'},
{"min",required_argument, NULL, 'm'},
@@ -268,6 +268,11 @@ static void parse_args(int argc, char *argv[], test_args_t 
*args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+c:r:m:x:p:t:h";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
/* defaults */
args->cpu_count = 0; /* all CPU's */
args->resolution_us = 1;
@@ -276,9 +281,10 @@ static void parse_args(int argc, char *argv[], test_args_t 
*args)
args->period_us = 100;
args->tmo_count = 30;
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (1) {
-   opt = getopt_long(argc, argv, "+c:r:m:x:p:t:h",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
 
if (opt == -1)
break;  /* No more options */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 35/38] example: switch: adding helper cmd line parsing

[Christophe Milard]

odp_switch now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/switch/odp_switch.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/example/switch/odp_switch.c b/example/switch/odp_switch.c
index 9358c2c..0c9b257 100644
--- a/example/switch/odp_switch.c
+++ b/example/switch/odp_switch.c
@@ -748,7 +748,7 @@ static void parse_args(int argc, char *argv[], appl_args_t 
*appl_args)
char *token;
size_t len;
unsigned i;
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
{"time", required_argument, NULL, 't'},
{"accuracy", required_argument, NULL, 'a'},
@@ -757,12 +757,18 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+c:+t:+a:i:h";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
appl_args->time = 0; /* loop forever if time to run is 0 */
appl_args->accuracy = 10; /* get and print pps stats second */
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (1) {
-   opt = getopt_long(argc, argv, "+c:+t:+a:i:h",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
 
if (opt == -1)
break;  /* No more options */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 34/38] example: switch: using agnostic function for ODP threads

[Christophe Milard]

odp_switch is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_switch is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/switch/odp_switch.c | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/example/switch/odp_switch.c b/example/switch/odp_switch.c
index 96229ff..9358c2c 100644
--- a/example/switch/odp_switch.c
+++ b/example/switch/odp_switch.c
@@ -574,7 +574,7 @@ static void bind_workers(void)
  *
  * @param arg  Thread arguments of type 'thread_args_t *'
  */
-static void *run_worker(void *arg)
+static int run_worker(void *arg)
 {
thread_args_t *thr_args = arg;
odp_packet_t pkt_tbl[MAX_PKT_BURST];
@@ -652,7 +652,7 @@ static void *run_worker(void *arg)
/* Make sure that latest stat writes are visible to other threads */
odp_mb_full();
 
-   return NULL;
+   return 0;
 }
 
 /*
@@ -878,7 +878,7 @@ static void gbl_args_init(args_t *args)
 
 int main(int argc, char **argv)
 {
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
int i, j;
int cpu;
int num_workers;
@@ -890,7 +890,7 @@ int main(int argc, char **argv)
stats_t (*stats)[MAX_PKTIOS];
int if_count;
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
/* Init ODP before calling anything else */
if (odp_init_global(&instance, NULL, NULL)) {
@@ -1003,8 +1003,7 @@ int main(int argc, char **argv)
 
odp_cpumask_zero(&thd_mask);
odp_cpumask_set(&thd_mask, cpu);
-   odph_linux_pthread_create(&thread_tbl[i], &thd_mask,
- &thr_params);
+   odph_odpthreads_create(&thread_tbl[i], &thd_mask, &thr_params);
cpu = odp_cpumask_next(&cpumask, cpu);
}
 
@@ -1026,7 +1025,8 @@ int main(int argc, char **argv)
exit_threads = 1;
 
/* Master thread waits for other threads to exit */
-   odph_linux_pthread_join(thread_tbl, num_workers);
+   for (i = 0; i < num_workers; ++i)
+   odph_odpthreads_join(&thread_tbl[i]);
 
free(gbl_args->appl.if_names);
free(gbl_args->appl.if_str);
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 36/38] linux-generic: test: ringtest: using agnostic function for ODP threads

[Christophe Milard]

ringtest.c is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
ringtest is hence no longer aware on how the odpthread is implemented.

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/test/ring/ringtest.c | 19 +--
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/platform/linux-generic/test/ring/ringtest.c 
b/platform/linux-generic/test/ring/ringtest.c
index ac0aa61..a613fe7 100644
--- a/platform/linux-generic/test/ring/ringtest.c
+++ b/platform/linux-generic/test/ring/ringtest.c
@@ -351,7 +351,7 @@ static void test_ring_stress(stress_type_t type)
}
 }
 
-static void *test_ring(void *arg)
+static int test_ring(void *arg)
 {
ring_arg_t *parg = (ring_arg_t *)arg;
int thr;
@@ -414,18 +414,17 @@ static void *test_ring(void *arg)
 
fflush(stdout);
 
-   return parg;
+   return 0;
 }
 
 int main(int argc TEST_UNUSED, char *argv[] TEST_UNUSED)
 {
ring_arg_t rarg;
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
odp_cpumask_t cpu_mask;
-   int num_workers;
char ring_name[_RING_NAMESIZE];
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
if (odp_init_global(&instance, NULL, NULL)) {
LOG_ERR("Error: ODP global init failed.\n");
@@ -439,7 +438,7 @@ int main(int argc TEST_UNUSED, char *argv[] TEST_UNUSED)
 
_ring_tailq_init();
 
-   num_workers = odp_cpumask_default_worker(&cpu_mask, MAX_WORKERS);
+   odp_cpumask_default_worker(&cpu_mask, MAX_WORKERS);
rarg.thrdarg.numthrds = rarg.thrdarg.numthrds;
 
rarg.thrdarg.testcase = ODP_RING_TEST_BASIC;
@@ -451,8 +450,8 @@ int main(int argc TEST_UNUSED, char *argv[] TEST_UNUSED)
thr_params.instance = instance;
 
printf("starting stess test type : %d..\n", rarg.stress_type);
-   odph_linux_pthread_create(&thread_tbl[0], &cpu_mask, &thr_params);
-   odph_linux_pthread_join(thread_tbl, num_workers);
+   odph_odpthreads_create(&thread_tbl[0], &cpu_mask, &thr_params);
+   odph_odpthreads_join(thread_tbl);
 
rarg.thrdarg.testcase = ODP_RING_TEST_STRESS;
rarg.stress_type = one_enq_one_deq;
@@ -475,8 +474,8 @@ int main(int argc TEST_UNUSED, char *argv[] TEST_UNUSED)
thr_params.start = test_ring;
thr_params.arg   = &rarg;
 
-   odph_linux_pthread_create(&thread_tbl[0], &cpu_mask, &thr_params);
-   odph_linux_pthread_join(thread_tbl, num_workers);
+   odph_odpthreads_create(&thread_tbl[0], &cpu_mask, &thr_params);
+   odph_odpthreads_join(thread_tbl);
 
 fail:
if (odp_term_local()) {
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 38/38] doc: implementers-guide: adding helper recommendations

[Christophe Milard]

The implementation guide is updated with recommendations regarding
the usage of helpers and some of its functions.

Signed-off-by: Christophe Milard 
---
 doc/implementers-guide/implementers-guide.adoc | 32 ++
 1 file changed, 32 insertions(+)

diff --git a/doc/implementers-guide/implementers-guide.adoc 
b/doc/implementers-guide/implementers-guide.adoc
index 8863004..abd49c0 100644
--- a/doc/implementers-guide/implementers-guide.adoc
+++ b/doc/implementers-guide/implementers-guide.adoc
@@ -506,5 +506,37 @@ from ODP_TEST_INFO_INACTIVE to ODP_TEST_INFO_CONDITIONAL:
 --
 =
 
+ helper usage 
+
+The tests (both platform agnostic and platform dependent tests) make use of
+a set of functions defined in a helper library. The helper library tries to
+abstract and regroup common actions that applications may perform but
+which are not part of the ODP API (i.e. mostly OS system calls).
+Using these functions is recommended, as running the tests on a different OS
+could (hopefully) be as simple as changing the OS related helper lib.
+
+In the linux helper, two functions are given to create and join ODP threads:
+
+`odph_odpthreads_create()`
+
+`odph_odpthreads_join()`
+
+These two functions abstract what an ODP thread really is and their usage
+is recommended as they would be implemented in other OS`s helper lib.
+
+Five older functions exist to tie and ODP thread to a specific implementation:
+
+`odph_linux_pthread_create()`
+
+`odph_linux_pthread_join()`
+
+`odph_linux_process_fork_n()`
+
+`odph_linux_process_fork()`
+
+`odph_linux_process_wait_n()`
+
+The usage of these functions should not occur within ODP examples nor tests.
+The usage of these functions in other application is not recommended.
 
 include::../glossary.adoc[]
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 37/38] linux-generic: test: ringtest: adding helper cmd line parsing

[Christophe Milard]

ringtests.c now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/test/ring/ringtest.c | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/platform/linux-generic/test/ring/ringtest.c 
b/platform/linux-generic/test/ring/ringtest.c
index a613fe7..2ebef8a 100644
--- a/platform/linux-generic/test/ring/ringtest.c
+++ b/platform/linux-generic/test/ring/ringtest.c
@@ -417,7 +417,7 @@ static int test_ring(void *arg)
return 0;
 }
 
-int main(int argc TEST_UNUSED, char *argv[] TEST_UNUSED)
+int main(int argc, char *argv[])
 {
ring_arg_t rarg;
odph_odpthread_t thread_tbl[MAX_WORKERS];
@@ -436,6 +436,9 @@ int main(int argc TEST_UNUSED, char *argv[] TEST_UNUSED)
exit(EXIT_FAILURE);
}
 
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, NULL, NULL);
+
_ring_tailq_init();
 
odp_cpumask_default_worker(&cpu_mask, MAX_WORKERS);
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv6 32/38] example: timer: using agnostic function for ODP threads

[Christophe Milard]

timer_test is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
timer_test is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/timer/odp_timer_test.c | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/example/timer/odp_timer_test.c b/example/timer/odp_timer_test.c
index f1d3be4..87690ab 100644
--- a/example/timer/odp_timer_test.c
+++ b/example/timer/odp_timer_test.c
@@ -195,7 +195,7 @@ static void test_abs_timeouts(int thr, test_globals_t *gbls)
  *
  * @return Pointer to exit status
  */
-static void *run_thread(void *ptr)
+static int run_thread(void *ptr)
 {
int thr;
odp_pool_t msg_pool;
@@ -213,7 +213,7 @@ static void *run_thread(void *ptr)
 
if (msg_pool == ODP_POOL_INVALID) {
EXAMPLE_ERR("  [%i] msg_pool not found\n", thr);
-   return NULL;
+   return -1;
}
 
odp_barrier_wait(&gbls->test_barrier);
@@ -223,7 +223,7 @@ static void *run_thread(void *ptr)
 
printf("Thread %i exits\n", thr);
fflush(NULL);
-   return NULL;
+   return 0;
 }
 
 
@@ -322,7 +322,7 @@ static void parse_args(int argc, char *argv[], test_args_t 
*args)
  */
 int main(int argc, char *argv[])
 {
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
int num_workers;
odp_queue_t queue;
uint64_t tick, ns;
@@ -333,7 +333,7 @@ int main(int argc, char *argv[])
odp_cpumask_t cpumask;
char cpumaskstr[ODP_CPUMASK_STR_SIZE];
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
odp_shm_t shm = ODP_SHM_INVALID;
test_globals_t *gbls = NULL;
int err = 0;
@@ -497,10 +497,10 @@ int main(int argc, char *argv[])
thr_params.thr_type = ODP_THREAD_WORKER;
thr_params.instance = instance;
 
-   odph_linux_pthread_create(thread_tbl, &cpumask, &thr_params);
+   odph_odpthreads_create(thread_tbl, &cpumask, &thr_params);
 
/* Wait for worker threads to exit */
-   odph_linux_pthread_join(thread_tbl, num_workers);
+   odph_odpthreads_join(thread_tbl);
 
/* free resources */
if (odp_queue_destroy(queue))
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCHv2 1/2] doc: userguide: add timer/timeout state diagram

[Christophe Milard]

Hi,

I am still confused with this shared FSM for the 2 distinct objects: The
red seems to be actions associated with the transitions (and the timer
expiration event is no longer shown on the transition showing the actions).
I am not sure what the distinction between the green and blue events are.
I propose the following: I will send a complete proposal to you, including
both the FSMs and the text. If we still disagree on which one is clearer,
we'll let other reviewer decide (you/I would sent a v4 with my stuff in).
If no one does react, I am ok to mark your proposal as reviewed because
some doc is better than none.

So you are sure to  win :-)

Christophe.

On 11 May 2016 at 01:47, Bill Fischofer  wrote:

> I looked at them and I liked the added color so reworked my diagram adding
> that and posted a v3 for it. I'm not sure trying to separate into two
> diagrams really helps but perhaps this reworked diagram is easier to follow?
>
> On Tue, May 10, 2016 at 3:45 AM, Christophe Milard <
> christophe.mil...@linaro.org> wrote:
>
>> Hi Bill,
>>
>> Just sent you an RFC with 2 FSMs.
>> Had to fight a bit with graphviz. Not finished yet. I will continue
>> working on it if you think that makes sense...
>> Christophe.
>>
>> On 9 May 2016 at 17:15, Bill Fischofer  wrote:
>>
>>> As we discussed in our call, we can both play around with graphviz and
>>> see if a prettier diagram can be constructed that is perhaps clearer,
>>> possibly with two separate FSMs.
>>>
>>> On Mon, May 9, 2016 at 9:39 AM, Christophe Milard <
>>> christophe.mil...@linaro.org> wrote:
>>>
>>>>
>>>>
>>>> On 9 May 2016 at 16:28, Bill Fischofer 
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Mon, May 9, 2016 at 7:58 AM, Christophe Milard <
>>>>> christophe.mil...@linaro.org> wrote:
>>>>>
>>>>>> I am a bit confused by this diagram: It feels to me that timers and
>>>>>> timeout have separate lives (even , if of course some dependency exist).
>>>>>> From this diagram, it feels like these 2 separate objects (timers and
>>>>>> time out events) share states...? do they?
>>>>>>
>>>>>
>>>>> Actually they do, which is what this diagram is trying to express.
>>>>> When a timer is set one of the arguments is the timeout event that should
>>>>> be associated with it, so it is an error to attempt to free that event
>>>>> while it is associated with a set timer (results are undefined if you do).
>>>>> Timers are somewhat unique in this respect.
>>>>>
>>>>
>>>> Sorry, Bill I still don't get it: Aren't you saying here that these are
>>>> 2 separate FSMs, but that these 2 separate FSMs are actually
>>>> actionned/"triggered" by (partly) the same events? There is nothing wrong
>>>> with that... Then they should be represented as 2 separated FSM with the
>>>> same event names on the transitions triggered by the same events...
>>>>  Or I am very confused...
>>>>
>>>> Christophe.
>>>>
>>>>>
>>>>>
>>>>>> In my head, and from the understanding I had, it feels that the to 4
>>>>>> top states are for timers, whereas time-out events have only 2 states:
>>>>>> Unallocated or allocated.
>>>>>> It feels you are doing 1 FSM out of two. Maybe , it should be two FSM
>>>>>> (keeping the same transition names to show the dependancy.)
>>>>>>
>>>>>> And I guess there is a typo at "odp_timeout_freee().
>>>>>>
>>>>>
>>>>> Thanks.  I'll fix that in the next version.
>>>>>
>>>>>
>>>>>>
>>>>>> Christophe
>>>>>>
>>>>>> On 7 May 2016 at 18:15, Bill Fischofer 
>>>>>> wrote:
>>>>>>
>>>>>>> Signed-off-by: Bill Fischofer 
>>>>>>> ---
>>>>>>>  doc/images/.gitignore   |  1 +
>>>>>>>  doc/images/timer_fsm.gv | 38
>>>>>>> ++
>>>>>>>  doc/users-guide/Makefile.am |  1 +
>>>>>>>  3 files changed, 40 insertions(+)
>>>>>>>  create mode 100644 doc/images/timer_fsm.gv
>>>>>>>
>>>>>>&g

Re: [lng-odp] [PATCHv6 05/38] helpers: linux: creating functions to handle odpthreads

[Christophe Milard]

On 12 May 2016 at 10:17, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia.com> wrote:

>
>
> > -Original Message-
> > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> > Christophe Milard
> > Sent: Wednesday, May 11, 2016 7:42 PM
> > To: brian.bro...@linaro.org; mike.hol...@linaro.org; lng-
> > o...@lists.linaro.org
> > Subject: [lng-odp] [PATCHv6 05/38] helpers: linux: creating functions to
> > handle odpthreads
> >
> > Two functions, odph_odpthreads_create and odph_odpthreads_join
> > are created to create and termindate odp threads.
> > These function will create the odp threads either as linux processes or
> > as linux threads, depending on the command line arguments (flag odph_proc
> > or odph_thread). If both flags are given, every second odp thread will be
> > process, (others are threads). default is thead only.
> > Former functions (odph_linux_pthread_create, odph_linux_pthread_join,
> > odph_linux_process_fork, odph_linux_process_fork_n and
> > odph_linux_process_wait_n) are left for a compatibility, but are aimed
> > to be removed.
> >
> > Signed-off-by: Christophe Milard 
> > ---
> >  helper/include/odp/helper/linux.h |  28 
> >  helper/linux.c| 267
> > ++
> >  2 files changed, 295 insertions(+)
> >
> > diff --git a/helper/include/odp/helper/linux.h
> > b/helper/include/odp/helper/linux.h
> > index f99b88a..f67aa30 100644
> > --- a/helper/include/odp/helper/linux.h
> > +++ b/helper/include/odp/helper/linux.h
> > @@ -166,6 +166,34 @@ int odph_linux_process_fork_n(odph_linux_process_t
> > *proc_tbl,
> >   */
> >  int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num);
> >
> > +/**
> > + * Creates and launches odpthreads (as linux threads or processes)
> > + *
> > + * Creates, pins and launches threads to separate CPU's based on the
> > cpumask.
> > + *
> > + * @param thread_tblThread table
> > + * @param mask  CPU mask
> > + * @param thr_paramsODP thread parameters
> > + *
> > + * @return Number of threads created
> > + */
> > +int odph_odpthreads_create(odph_odpthread_t *thread_tbl,
> > +const odp_cpumask_t *mask,
> > +const odph_odpthread_params_t *thr_params);
> > +
> > +/**
> > + * Waits odpthreads (as linux threads or processes) to exit.
> > + *
> > + * Returns when all odpthreads have terminated.
> > + *
> > + * @param thread_tblThread table
> > + * @return The number of joined threads or -1 on error.
> > + * (error occurs if any of the start_routine return non-zero or if
> > + *  the thread join/process wait itself failed -e.g. as the result of a
> > kill)
> > + *
> > + */
> > +int odph_odpthreads_join(odph_odpthread_t *thread_tbl);
> > +
> >
> >  /**
> >   * Parse linux helper options
> > diff --git a/helper/linux.c b/helper/linux.c
> > index 5dbc2da..b8d4f49 100644
> > --- a/helper/linux.c
> > +++ b/helper/linux.c
> > @@ -22,6 +22,11 @@
> >  #include 
> >  #include "odph_debug.h"
> >
> > +static struct {
> > + int proc; /* true when process mode is required */
> > + int thrd; /* true when thread mode is required */
> > +} helper_options;
> > +
> >  static void *odp_run_start_routine(void *arg)
> >  {
> >   odph_linux_thr_params_t *thr_params = arg;
> > @@ -239,6 +244,262 @@ int odph_linux_process_wait_n(odph_linux_process_t
> > *proc_tbl, int num)
> >  }
> >
> >  /*
> > + * wrapper for odpthreads, either implemented as linux threads or
> > processes.
> > + * (in process mode, if start_routine returns NULL, the process return
> > FAILURE).
> > + */
> > +static void *odpthread_run_start_routine(void *arg)
> > +{
> > + int status;
> > + int ret;
> > + odph_odpthread_params_t *thr_params;
> > +
> > + odph_odpthread_start_args_t *start_args = arg;
> > +
> > + thr_params = &start_args->thr_params;
> > +
> > + /* ODP thread local init */
> > + if (odp_init_local(thr_params->instance, thr_params->thr_type)) {
> > + ODPH_ERR("Local init failed\n");
> > + if (start_args->linuxtype == ODPTHREAD_PROCESS)
> > + _exit(EXIT_FAILURE);
> > + return NULL;
> > + }
> > +
> > + ODPH_DBG("

Re: [lng-odp] [PATCHv6 12/38] helper: parsing the complete set of options

[Christophe Milard]

Not sure which patches you want to squash here: "helper: adding a function
to merge getopt parameters" and "parsing the complete set of options" ?
Aren't these 2 different steps? They were at least tested separately...
Or did I misunderstood?

Christophe.

On 12 May 2016 at 10:21, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia.com> wrote:

> It would be really good to squash these patches introducing (and then
> modifying the previously introduced) new helper function prototypes into a
> single patch. This way it's hard to see the complete picture what functions
> were actually added into helper/linux.h
>
>
> -Petri
>
>
> > -Original Message-
> > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> > Christophe Milard
> > Sent: Wednesday, May 11, 2016 7:42 PM
> > To: brian.bro...@linaro.org; mike.hol...@linaro.org; lng-
> > o...@lists.linaro.org
> > Subject: [lng-odp] [PATCHv6 12/38] helper: parsing the complete set of
> > options
> >
> > The odph_parse_options() function is given the ability to receive
> > getopt command line description parameter from it caller, hence allowing
> > the caller to have some command line parameter(s).
> > The caller shall first call odph_parse_options() with its own parameter
> > description as parameter.
> > odph_parse_options() is then checking the complete set of options,
> > issuing error message for unknown options (those being neither a caller's
> > valid command line option or a helper valid command line option),
> > and collecting the sementic of helper options.
> > Then the caller shall parse the sementic of its own options,
> > with the opterr variable set to zero (hence ignoring helper options).
> >
> > Signed-off-by: Christophe Milard 
> > ---
> >  helper/include/odp/helper/linux.h | 16 +++-
> >  helper/linux.c| 24 +++-
> >  helper/test/odpthreads.c  |  2 +-
> >  test/validation/common/odp_cunit_common.c |  2 +-
> >  4 files changed, 36 insertions(+), 8 deletions(-)
> >
> > diff --git a/helper/include/odp/helper/linux.h
> > b/helper/include/odp/helper/linux.h
> > index 9767af4..71c8027 100644
> > --- a/helper/include/odp/helper/linux.h
> > +++ b/helper/include/odp/helper/linux.h
> > @@ -25,6 +25,7 @@ extern "C" {
> >  #include 
> >
> >  #include 
> > +#include 
> >  #include 
> >
> >  /** Thread parameter for Linux pthreads and processes */
> > @@ -241,15 +242,28 @@ int odph_merge_getopt_options(const char
> > *shortopts1,
> >   * Parse linux helper options
> >   *
> >   * Parse the command line options. Pick up options meant for the helper
> > itself.
> > + * If the caller is also having a set of option to parse, it should
> > include
> > + * their description here (shortopts desribes the short options and
> > longopts
> > + * describes the long options, as for getopt_long()).
> > + * This function will issue errors on unknown arguments, so callers
> > failing
> > + * to pass their own command line options description here will see
> their
> > + * options rejected.
> > + * (the caller wants to set opterr to zero when parsing its own stuff
> > + * with getopts to avoid reacting on helper's options).
> >   *
> >   * @param argc argument count
> >   * @param argv argument values
> > + * @param caller_shortopts caller's set of short options (string). or
> > NULL.
> > + * @param caller_longopts  caller's set of long options (getopt option
> > array).
> > + *  or NULL.
> >   *
> >   * @return On success: 0
> >   *  On failure: -1 (failure occurs only if a value passed for a
> > helper
> >   *  option is invalid. callers cannot have own
> options)
> >   */
> > -int odph_parse_options(int argc, char *argv[]);
> > +int odph_parse_options(int argc, char *argv[],
> > +const char *caller_shortopts,
> > +const struct option *caller_longopts);
> >  #ifdef __cplusplus
> >  }
> >  #endif
> > diff --git a/helper/linux.c b/helper/linux.c
> > index d1b7825..5fc09a1 100644
> > --- a/helper/linux.c
> > +++ b/helper/linux.c
> > @@ -16,7 +16,6 @@
> >  #include 
> >  #include 
> >  #include 
> > -#include 
> >
> >  #include 
> >  #include 
> > @@ -576,27 +575,39 @@ int odph_merge_getopt_options(const char
> > *shortopts1,

Re: [lng-odp] [PATCHv6 05/38] helpers: linux: creating functions to handle odpthreads

[Christophe Milard]

On 12 May 2016 at 12:28, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia.com> wrote:

>
> > +int odph_odpthreads_create(odph_odpthread_t *thread_tbl,
> > +const odp_cpumask_t *mask,
> > +const odph_odpthread_params_t *thr_params)
> > +{
> > + int i;
> > + int num;
> > + int cpu_count;
> > + int cpu;
> > +
> > + num = odp_cpumask_count(mask);
> > +
> > + memset(thread_tbl, 0, num * sizeof(odph_odpthread_t));
> > +
> > + cpu_count = odp_cpu_count();
> > +
> > + if (num < 1 || num > cpu_count) {
> > + ODPH_ERR("Invalid number of odpthreads:%d"
> > +  " (%d cores available)\n",
> > +  num, cpu_count);
> > + return -1;
> > + }
> > +
> > + cpu = odp_cpumask_first(mask);
> > + for (i = 0; i < num; i++) {
> > + /*
> > +  * Thread mode by default, or if both thread and proc mode
> > +  * are required each second odpthread is a linux thread.
> > +  */
>
> This is a weird logic. All threads should be one type (when you have
> boolean cmd line params for thread type). Pthread should be the default, so
> all threads are pthreads if user don't give any param. It's an error if
> user gives both.
>
>
>
> This follows a requirement from you that the linux ODP implementation
> should support a mix of both ODP threads implemented as pthread and process
> at the same time. I felt that a "--odph_mixed" option is not more clear
> than specifying both the things we want, i.e. processes and thread.
>
> Any better suggestion?
>
>
>
> You’d need more detailed information (than Boolean) for that advanced use
> case. At this point, all pthreads or all processes is sufficient for
> odp-linux. I have not required implementation of thread/process mix, but
> required that ODP API does not prevent different threading models.
>
>
>
> Create all threads by default, all processes if helper_options.proc is
> true and report error is both are true.
>
>
>
> if (helper_options.proc && helper_options.thrd)
>
>   return -1;
>
>
>
> if (helper_options.proc) {
>
>   odph_linux_process_create();
>
> } else {
>
>   odph_linux_thread_create();
>
> }
>
>
>
>
>

Now, I am getting very confused... I thought we agreed that support for
both process mode and thread mode at the same time was needed. I actually
remember asking that specific question at some arch call and  the answer
being positive: And it made sense!: If ODP threads are OS objects created
by the application directly towards the OS, how would you expect ODP to
prevent an app to mix pthreads and fork?. If we cannot prevent it, don't we
have to support it?

Shall I really remove support for the mixed mode from the patch series?


>
>
>
>
> > + if ((!helper_options.proc) ||
> > + (helper_options.proc && helper_options.thrd && (i &
> 1))) {
> > + if (odph_linux_thread_create(&thread_tbl[i],
> > +  cpu,
> > +  thr_params))
> > + break;
> > +  } else {
> > + if (odph_linux_process_create(&thread_tbl[i],
> > +   cpu,
> > +   thr_params))
> > + break;
> > + }
> > +
> > + cpu = odp_cpumask_next(mask, cpu);
> > + }
> > + thread_tbl[num - 1].last = 1;
> > +
> > + return i;
> > +}
> > +
> > +/*
> > + * wait for the odpthreads termination (linux processes and threads)
> > + */
> > +int odph_odpthreads_join(odph_odpthread_t *thread_tbl)
> > +{
> > + pid_t pid;
> > + int i = 0;
> > + int terminated = 0;
> > + int status = 0; /* child process return code (!=0 is error)
> > */
> > + void *thread_ret;   /* "child" thread return code (NULL is
> error) */
> > + int ret;
> > + int retval = 0;
> > +
> > + /* joins linux threads or wait for processes */
> > + do {
> > + /* pthreads: */
> > + switch (thread_tbl[i].start_args.linuxtype) {
> > + case ODPTHREAD_PTHREAD:
> > + /* Wait thread to exit */
> > + ret = pthread_join(thread_tbl[i].thread.thread_id,
> > +&thread_ret);
> > + if (ret != 0) {
> > + ODPH_ERR("Failed to join thread from cpu
> #%d\n",
> > +  thread_tbl[i].cpu);
> > + retval = -1;
> > + } else {
> > + terminated++;
> > + if (thread_ret != NULL)
> > + retval = -1;
> > + }
> > + 

Re: [lng-odp] [PATCHv6 05/38] helpers: linux: creating functions to handle odpthreads

[Christophe Milard]

On 12 May 2016 at 13:33, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia.com> wrote:

>
>
>
>
> *From:* lng-odp [mailto:lng-odp-boun...@lists.linaro.org] *On Behalf Of 
> *Christophe
> Milard
> *Sent:* Thursday, May 12, 2016 2:14 PM
> *To:* Savolainen, Petri (Nokia - FI/Espoo) 
> *Cc:* lng-odp@lists.linaro.org
> *Subject:* Re: [lng-odp] [PATCHv6 05/38] helpers: linux: creating
> functions to handle odpthreads
>
>
>
>
>
>
>
> On 12 May 2016 at 12:28, Savolainen, Petri (Nokia - FI/Espoo) <
> petri.savolai...@nokia.com> wrote:
>
>
> > +int odph_odpthreads_create(odph_odpthread_t *thread_tbl,
> > +const odp_cpumask_t *mask,
> > +const odph_odpthread_params_t *thr_params)
> > +{
> > + int i;
> > + int num;
> > + int cpu_count;
> > + int cpu;
> > +
> > + num = odp_cpumask_count(mask);
> > +
> > + memset(thread_tbl, 0, num * sizeof(odph_odpthread_t));
> > +
> > + cpu_count = odp_cpu_count();
> > +
> > + if (num < 1 || num > cpu_count) {
> > + ODPH_ERR("Invalid number of odpthreads:%d"
> > +  " (%d cores available)\n",
> > +  num, cpu_count);
> > + return -1;
> > + }
> > +
> > + cpu = odp_cpumask_first(mask);
> > + for (i = 0; i < num; i++) {
> > + /*
> > +  * Thread mode by default, or if both thread and proc mode
> > +  * are required each second odpthread is a linux thread.
> > +  */
>
> This is a weird logic. All threads should be one type (when you have
> boolean cmd line params for thread type). Pthread should be the default, so
> all threads are pthreads if user don't give any param. It's an error if
> user gives both.
>
>
>
> This follows a requirement from you that the linux ODP implementation
> should support a mix of both ODP threads implemented as pthread and process
> at the same time. I felt that a "--odph_mixed" option is not more clear
> than specifying both the things we want, i.e. processes and thread.
>
> Any better suggestion?
>
>
>
> You’d need more detailed information (than Boolean) for that advanced use
> case. At this point, all pthreads or all processes is sufficient for
> odp-linux. I have not required implementation of thread/process mix, but
> required that ODP API does not prevent different threading models.
>
>
>
> Create all threads by default, all processes if helper_options.proc is
> true and report error is both are true.
>
>
>
> if (helper_options.proc && helper_options.thrd)
>
>   return -1;
>
>
>
> if (helper_options.proc) {
>
>   odph_linux_process_create();
>
> } else {
>
>   odph_linux_thread_create();
>
> }
>
>
>
>
>
>
>
> Now, I am getting very confused... I thought we agreed that support for
> both process mode and thread mode at the same time was needed. I actually
> remember asking that specific question at some arch call and  the answer
> being positive: And it made sense!: If ODP threads are OS objects created
> by the application directly towards the OS, how would you expect ODP to
> prevent an app to mix pthreads and fork?. If we cannot prevent it, don't we
> have to support it?
>
>
>
> Shall I really remove support for the mixed mode from the patch series?
>
>
>
>
>
> There is no need for _*helper*_ to setup every second thread as process
> and every second as pthread. If an application needs mixed mode, it can set
> it up without a helper. We just need basic stuff working first (all
> processes). More flexible setup would need more params: “create 1 process
> with 3 additional pthread, and a another 2 processes with 2 additional
> pthreads each, and …”
>

But wasn't the point of this patch series to be able to run
test/examples/perf test ... in all suported modes?


>
>
>
>
>
>
>
>
>
> > + if ((!helper_options.proc) ||
> > + (helper_options.proc && helper_options.thrd && (i &
> 1))) {
> > + if (odph_linux_thread_create(&thread_tbl[i],
> > +  cpu,
> > +  thr_params))
> > + break;
> > +  } else {
> > + if (odph_linux_process_create(&thread_tbl[i],
> > +   

Re: [lng-odp] [PATCHv2 1/2] doc: userguide: add timer/timeout state diagram

[Christophe Milard]

Hi Bill,
Just sent you a v4 containing my proposal regarding the FSMs and the text
around it.
I still have a few comments:

First, your FSM mixed events and actions on the transition arrow. I could
not get graphviz to have 2 colors on the same transition (one for events,
one for actions), so I have restricted them to events. The only action
worth mentioning is the event queuing action which is clearly described by
your text (and a reminder in mine). The square node could be a way to
represent it, but I don't like mixing event and action undistinguished. As
long as it is clear and consistent (it seems I have a need for that these
days...), I won't bother you.

Then a few questions:

1)My Time-out FSM does not show what happens to a time-out event when an
odp_timer_free() is called while the timeout is in delivered state. I
assume the time-out remains in delivered state until a
odp_event_free/timeout_free() is called manually, but I am not sure (having
2 different FSM actally clarify this point). You are welcome to update my
FSMs if we should keep them.

2)Your text says "Following start, applications may allocate, set, cancel,
and free, timers...". I guess the comma after "timer" is a typo.

3)The last sentence in your text is: "However, upon receiving a timeout
event the application can use the odp_timeout_fresh() API to inquire
whether the timeout event is fresh or stale". What is the definition of a
stale timer?

4) And I kept the best for the end... :-)
What about the validity of the pointer returned by odp_timeout_user_ptr()
when the timer was set by odp thread A on a queue belonging to ODP thread B
(B != A). Interresting question isn't it? :-).

Thanks for reading...

Christophe.

On 12 May 2016 at 14:08, Bill Fischofer  wrote:

> The green is for Timer related transitions while the blue is for Timeout
> related transitions. This format also seemed to get rid of some of the
> annoying line crossings. The red arrows are the transitions that relate to
> timer expiration and resulting timeout event scheduling, neither of which
> involve Timer APIs. So the colors were supposed to make things easier to
> read.
>
> I'm all for improving clarity, so if you have a better way of diagraming
> this that's great. I believe the diagrams and text go together in either
> case and are intended to be taken together to improve understanding.  The
> goal of each of these sections of the User Guide should be to give readers
> confidence that they understand how to use these APIs in writing their own
> ODP applications, or for ODP implementers to understand the semantics they
> need to provide in their own implementations of them.
>
> On Thu, May 12, 2016 at 3:13 AM, Christophe Milard <
> christophe.mil...@linaro.org> wrote:
>
>> Hi,
>>
>> I am still confused with this shared FSM for the 2 distinct objects: The
>> red seems to be actions associated with the transitions (and the timer
>> expiration event is no longer shown on the transition showing the actions).
>> I am not sure what the distinction between the green and blue events are.
>> I propose the following: I will send a complete proposal to you,
>> including both the FSMs and the text. If we still disagree on which one is
>> clearer, we'll let other reviewer decide (you/I would sent a v4 with my
>> stuff in). If no one does react, I am ok to mark your proposal as reviewed
>> because some doc is better than none.
>>
>> So you are sure to  win :-)
>>
>> Christophe.
>>
>> On 11 May 2016 at 01:47, Bill Fischofer 
>> wrote:
>>
>>> I looked at them and I liked the added color so reworked my diagram
>>> adding that and posted a v3 for it. I'm not sure trying to separate into
>>> two diagrams really helps but perhaps this reworked diagram is easier to
>>> follow?
>>>
>>> On Tue, May 10, 2016 at 3:45 AM, Christophe Milard <
>>> christophe.mil...@linaro.org> wrote:
>>>
>>>> Hi Bill,
>>>>
>>>> Just sent you an RFC with 2 FSMs.
>>>> Had to fight a bit with graphviz. Not finished yet. I will continue
>>>> working on it if you think that makes sense...
>>>> Christophe.
>>>>
>>>> On 9 May 2016 at 17:15, Bill Fischofer 
>>>> wrote:
>>>>
>>>>> As we discussed in our call, we can both play around with graphviz and
>>>>> see if a prettier diagram can be constructed that is perhaps clearer,
>>>>> possibly with two separate FSMs.
>>>>>
>>>>> On Mon, May 9, 2016 at 9:39 AM, Christophe Milard <
>>>>> christophe.mil...@linaro.org> wrote:
>>>>>
>

Re: [lng-odp] [PATCHv6 12/38] helper: parsing the complete set of options

[Christophe Milard]

On 12 May 2016 at 12:54, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia.com> wrote:

> Those that add new definitions into helper/linux.h and depend on each
> other, so that the entire proposed spec is easy to see today and later on
> from git history. Since those are new calls, it should be easy to introduce
> those in their final form. Patches (e.g. 1 and 12) should not expose
> intermediate, unnecessary steps done during development of a new feature,
> but the new feature in its final form.
>
>
>
> At least 1 and 12, since those introduce and modify the same function.
> Maybe 11 if odph_merge_getopt_options() is always needed with 
> odph_parse_options().
> Is merge function defined for the user or is it internal to the helper?
>

Does not make sense to me: as explained in the cover letter, patch 1 to 10
is introducing the ability to parse options in helpers. This is applied to
the tests (patch 6 to 10) as none of the tests have their own options (as
of today)

Patch  11... enables the caller helper to have its own options. This is
then applied to exemple/perf... which already had their own option, and
hence could not be handled as the tests were.

>
>
> Maybe  4 and 5, because the first introduces types that the other depends
> on.
>

That could be done if you wish

>
>
> Maybe 5 and 12 (at least 12 should be before 5), if odph_parse_options()
> must be always called before odph_odpthreads_create().
>

same problem as with patch 1 and 12: Patch 1 to 10 is one  group. 11 to 37
another. I hoped the cover-letter made that clear.

Christophe.

>
>
>
>
>
>
> -Petri
>
>
>
>
>
> *From:* lng-odp [mailto:lng-odp-boun...@lists.linaro.org] *On Behalf Of 
> *Christophe
> Milard
> *Sent:* Thursday, May 12, 2016 12:05 PM
> *To:* Savolainen, Petri (Nokia - FI/Espoo) 
> *Cc:* lng-odp@lists.linaro.org
> *Subject:* Re: [lng-odp] [PATCHv6 12/38] helper: parsing the complete set
> of options
>
>
>
> Not sure which patches you want to squash here: "helper: adding a function
> to merge getopt parameters" and "parsing the complete set of options" ?
>
> Aren't these 2 different steps? They were at least tested separately...
>
> Or did I misunderstood?
>
>
>
> Christophe.
>
>
>
> On 12 May 2016 at 10:21, Savolainen, Petri (Nokia - FI/Espoo) <
> petri.savolai...@nokia.com> wrote:
>
> It would be really good to squash these patches introducing (and then
> modifying the previously introduced) new helper function prototypes into a
> single patch. This way it's hard to see the complete picture what functions
> were actually added into helper/linux.h
>
>
> -Petri
>
>
>
> > -Original Message-
> > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> > Christophe Milard
> > Sent: Wednesday, May 11, 2016 7:42 PM
> > To: brian.bro...@linaro.org; mike.hol...@linaro.org; lng-
> > o...@lists.linaro.org
> > Subject: [lng-odp] [PATCHv6 12/38] helper: parsing the complete set of
> > options
> >
> > The odph_parse_options() function is given the ability to receive
> > getopt command line description parameter from it caller, hence allowing
> > the caller to have some command line parameter(s).
> > The caller shall first call odph_parse_options() with its own parameter
> > description as parameter.
> > odph_parse_options() is then checking the complete set of options,
> > issuing error message for unknown options (those being neither a caller's
> > valid command line option or a helper valid command line option),
> > and collecting the sementic of helper options.
> > Then the caller shall parse the sementic of its own options,
> > with the opterr variable set to zero (hence ignoring helper options).
> >
> > Signed-off-by: Christophe Milard 
> > ---
> >  helper/include/odp/helper/linux.h | 16 +++-
> >  helper/linux.c| 24 +++-
> >  helper/test/odpthreads.c  |  2 +-
> >  test/validation/common/odp_cunit_common.c |  2 +-
> >  4 files changed, 36 insertions(+), 8 deletions(-)
> >
> > diff --git a/helper/include/odp/helper/linux.h
> > b/helper/include/odp/helper/linux.h
> > index 9767af4..71c8027 100644
> > --- a/helper/include/odp/helper/linux.h
> > +++ b/helper/include/odp/helper/linux.h
> > @@ -25,6 +25,7 @@ extern "C" {
> >  #include 
> >
> >  #include 
> > +#include 
> >  #include 
> >
> >  /** Thread parameter for Linux pthreads and processes */
> > @@ -241,15 +242,28 @@ int odph_merge_ge

[lng-odp] process thread and mixed mode

[Christophe Milard]

Hi,

Let me start in a bit unusual way :-). I do not like my "running things in
process mode" patch series  that much...
But I do think it should be merged. With the mixed mode.

The main goal of the patch series is to push the work regarding a proper
and consistent definition of some ODP low level concepts and objects. The
patch series makes things fail, hence pushing for defining and coding
things better, I hope.  Removing "provocative" things like the mixed mode
from it is just defeating the purpose of this series: I'll be happy to
remove it (and plenty of other things) when we know which well documented
definition/rule makes this code not relevant.
As long as we keep saying that odp threads could be indifferently  linux
processes or pthreads, I'll be thinking that this series should work,
inclusive any kind of mixed mode.

However, processes and threads are different things in linux, and I believe
this distinction should be kept within ODP: Trying to make processes behave
as pthreads for some ODP objects (such as addresses for shmem objects) does
not sound like a good idea to me. Yes, this can maybe be done with linux
-and even possibly without preallocating the memory-, but still, linux
creates 2 distinct virtual address spaces on fork() so why would ODP put
any exception to that?.
...And If we do want to create such exceptions (which I don't like),
shouln't ODP define its own concept of process and thread as it obviously
wouldn't match 100% the definition of the OS?

I do nevertheless definitively agree with Petri when saying that that ODP
applications should be given a way to "share pointers" between odpthreads.
Taking the "process only" behavior forcing every concurrent execution tasks
to use tricky offset calculation to find their data is too nasty and
inefficient.

I think this will boil down to 2 choices:
1) Take these object definitions from the OS and accept the OS rules.
2) Define our own ODP objects (and possibly defines our own rules)

1)Taking the OS definitions:
Choice 1) is consistent as long as we accept the OS rules. Taking that
choice would mean that any address remain valid within a process only:
shmem pointers will not be able to be shared between different processes: I
do not see that as a big problem as the applications can use pthreads if it
needs to share the same VA, or can fork() at a conveniant time. Moreover,
existing applications which should be ported to ODP are probably OK as they
would already be using pthreads/processes in the correct way.

A more tricky aspect of choice 1) is the development of ODP itself:
Operations involving many "ODPthreads" will have to cope with both
processes and threads possibly without a clear view of what is done by the
application. ( I guess in linux if getpid()==gettid() when calling
odp_init_local() one can assume safely that the app forked...?).
I guess it is doable: assuming all odpthreads are descendant of the
instanciation process, ODP can initially reserve shared memory in
odp_init_global() for its own purpose. It has a price: within ODP (and
anything external to the app itself, such as drivers) a common VA cannot be
assumed. ODP/drivers will have be prepared for an app fork at any time and
hence do its internal stuff as if pointers could not be shared (except
within the initially allocated memory). This will cost ODP performance.

Taking choice 1 probably also mean that shmen should no longer be part of
ODP: Allocating memory is as much an OS thing as forking a process... why
would shmem objects be ODP objects while ODP thread would not be?

2) Defining new ODP objects
I guess in this case, we'd have to create both ODP threads and ODP
processes objects and related usual methods. The advantage if this approach
is that ODP has more control of what is being done. It also allows for a
ODP definition of things (such as the above mentioned exception), if we
really want to...
It makes applications more portable (if ODP abstracts some of the common OS
functionality, the apps using these ODP api becomes OS agnostic).
As we already have done for shmem, ODP methods for odp threads and
processes could be provided to retrieve underlying OS stuff (such as PID...)

The problem of having to assume non common VA within ODP remains.

... and of course where should we stop wrapping the OS...?

3) There is a third alternative as well: stating that ODP is thread only
:-). No process, no problem?

Thank you for reading that far :-). waiting for your comments...

Christophe.
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [PATCHv2 1/2] doc: userguide: add timer/timeout state diagram

[Christophe Milard]

On 12 May 2016 at 19:44, Bill Fischofer  wrote:

>
>
> On Thu, May 12, 2016 at 8:24 AM, Christophe Milard <
> christophe.mil...@linaro.org> wrote:
>
>> Hi Bill,
>> Just sent you a v4 containing my proposal regarding the FSMs and the text
>> around it.
>> I still have a few comments:
>>
>> First, your FSM mixed events and actions on the transition arrow. I could
>> not get graphviz to have 2 colors on the same transition (one for events,
>> one for actions), so I have restricted them to events. The only action
>> worth mentioning is the event queuing action which is clearly described by
>> your text (and a reminder in mine). The square node could be a way to
>> represent it, but I don't like mixing event and action undistinguished. As
>> long as it is clear and consistent (it seems I have a need for that these
>> days...), I won't bother you.
>>
>
> I think it's fine this way.  The expiration/schedule call adds detail
> because a timeout isn't actually delivered until the scheduler returns it
> to a thread, but I think this is clear enough here.
>
>

On V5:
Maybe the error originated from me (?), but I see that the TO_Delivered
node is double ringed: Double ringed node would show the start state, so I
think it shouldn't be double ringed. If the intention was to show the final
state (though there isn't really one here: a TO event can be reused I
assume), then the Timer_Expired state should be double ringed as well. But
as mentionned, as I cannot clearly see any final state here, I would simply
remove the double ring from the TO_delivered state.

>
>> Then a few questions:
>>
>> 1)My Time-out FSM does not show what happens to a time-out event when an
>> odp_timer_free() is called while the timeout is in delivered state. I
>> assume the time-out remains in delivered state until a
>> odp_event_free/timeout_free() is called manually, but I am not sure (having
>> 2 different FSM actally clarify this point). You are welcome to update my
>> FSMs if we should keep them.
>>
>
> Nothing happens to the event since the association between timer and
> timeout ends as soon as the timer expires. At that point the timer is in
> the "Timer_Expired" state and the Timeout moves to the "Timeout_Delivered"
> state (Timeout_Pending in my previous diagram). So they're no longer
> coupled and operations on one no longer affect the other.
>

In that case there should be a transition TO_delivered->TO_delivered
labeled odp_timer_free()


>
>
>>
>> 2)Your text says "Following start, applications may allocate, set,
>> cancel, and free, timers...". I guess the comma after "timer" is a typo.
>>
>
> Yes, thanks for the correction.
>
>
>>
>> 3)The last sentence in your text is: "However, upon receiving a timeout
>> event the application can use the odp_timeout_fresh() API to inquire
>> whether the timeout event is fresh or stale". What is the definition of a
>> stale timer?
>>
>
> This was a back reference to the earlier sentence discussing
> odp_timer_cancel(): "An attempted cancel will fail if the timer is not set
> or if it has already
> expired."  However, I recall this was an area that Ola and Petri debated
> extensively when these were first defined. There is an implicit imprecision
> in any efficient timer system since independent threads can be setting,
> resetting, and cancelling timers while they are "in flight". Generally when
> a timeout event is received it's the responsibility of the application to
> determine whether that event is meaningful or not based on other
> information it has (e.g., it's a TCP delayed ACK timer, but we've just
> received a packet on that connection so I should ignore this event, etc.).
>  odp_timer_fresh() is a hook that allows the implementation to communicate
> staleness information if it knows something that the application might not
> otherwise know, but I expect most applications will just ignore this as a
> reported "fresh" timer may still not be meaningful from the application's
> perspective for reasons it knows.
>

Not sure I understand that. I'll need explanations, and I guess the doc as
well.

>
>
>>
>> 4) And I kept the best for the end... :-)
>> What about the validity of the pointer returned by odp_timeout_user_ptr()
>> when the timer was set by odp thread A on a queue belonging to ODP thread B
>> (B != A). Interresting question isn't it? :-).
>>
>>
> The user ptr should really be revised for Tiger Moth to include a length
> (just as we did for queues) so tha

Re: [lng-odp] [PATCHv6 1/2] doc: userguide: add timer/timeout state diagrams

[Christophe Milard]

For the series:
Reviewed-by: Christophe Milard 

Christophe.


On 13 May 2016 at 17:01, Bill Fischofer  wrote:

> Signed-off-by: Bill Fischofer 
> Signed-off-by: Christophe Milard 

---
>  doc/images/.gitignore   |  2 ++
>  doc/images/timeout_fsm.gv   | 28 
>  doc/images/timer_fsm.gv | 20 
>  doc/users-guide/Makefile.am |  2 ++
>  4 files changed, 52 insertions(+)
>  create mode 100644 doc/images/timeout_fsm.gv
>  create mode 100644 doc/images/timer_fsm.gv
>
> diff --git a/doc/images/.gitignore b/doc/images/.gitignore
> index a19aa75..003dbe6 100644
> --- a/doc/images/.gitignore
> +++ b/doc/images/.gitignore
> @@ -1,2 +1,4 @@
>  resource_management.svg
>  pktio_fsm.svg
> +timer_fsm.svg
> +timeout_fsm.svg
> diff --git a/doc/images/timeout_fsm.gv b/doc/images/timeout_fsm.gv
> new file mode 100644
> index 000..21ecb59
> --- /dev/null
> +++ b/doc/images/timeout_fsm.gv
> @@ -0,0 +1,28 @@
> +digraph timer_state_machine {
> +   rankdir=LR;
> +   size="12,20";
> +   node [fontsize=28];
> +   edge [fontsize=28];
> +   node [shape=doublecircle]; TO_Unalloc;
> +   node [shape=circle]; TO_Alloc TO_Pending TO_Delivered;
> +   node [shape=rect]; TO_Enqueued;
> +   TO_Unalloc -> TO_Alloc [label="odp_timeout_alloc()"];
> +   TO_Alloc -> TO_Unalloc [label="odp_timeout_free()"];
> +   TO_Alloc -> TO_Pending [fontcolor=green,
> +  label="odp_timer_set_abs()"];
> +   TO_Alloc -> TO_Pending [fontcolor=green,
> +  label="odp_timer_set_rel()"];
> +   TO_Pending -> TO_Alloc [fontcolor=green,
> +  label="odp_timer_cancel()"];
> +   TO_Pending -> TO_Enqueued [fontcolor=green, label="timer expires"];
> +   TO_Enqueued -> TO_Delivered [label="odp_schedule()"];
> +   TO_Delivered -> TO_Pending [fontcolor=green,
> +  label="odp_timer_set_abs()"];
> +   TO_Delivered -> TO_Pending [fontcolor=green,
> +  label="odp_timer_set_rel()"];
> +   TO_Delivered -> TO_Delivered [label="odp_timeout_from_event()"];
> +   TO_Delivered -> TO_Delivered [label="odp_timeout_timer()"];
> +   TO_Delivered -> TO_Unalloc
> +   [label="odp_timeout_free() / odp_event_free()"];
> +
> +}
> diff --git a/doc/images/timer_fsm.gv b/doc/images/timer_fsm.gv
> new file mode 100644
> index 000..1798d31
> --- /dev/null
> +++ b/doc/images/timer_fsm.gv
> @@ -0,0 +1,20 @@
> +digraph timer_state_machine {
> +   rankdir=LR;
> +   size="12,20";
> +   node [fontsize=28];
> +   edge [fontsize=28];
> +   node [shape=doublecircle]; Timer_Unalloc;
> +   node [shape=circle]; Timer_Alloc Timer_Set Timer_Expired
> +   Timer_Unalloc -> Timer_Alloc [label="odp_timer_alloc()"];
> +   Timer_Alloc -> Timer_Unalloc [label="odp_timer_free()"];
> +   Timer_Alloc -> Timer_Set
> [fontcolor=green,label="odp_timer_set_abs()"];
> +   Timer_Alloc -> Timer_Set
> [fontcolor=green,label="odp_timer_set_rel()"];
> +   Timer_Set -> Timer_Alloc
> [fontcolor=green,label="odp_timer_cancel()"];
> +   Timer_Set -> Timer_Expired [fontcolor=green,label="timer expires"];
> +   Timer_Expired -> Timer_Unalloc [label="odp_timer_free()"];
> +   Timer_Expired -> Timer_Set [fontcolor=green,
> +  label="odp_timer_set_abs()"];
> +   Timer_Expired -> Timer_Set [fontcolor=green,
> +  label="odp_timer_set_rel()"];
> +
> +}
> diff --git a/doc/users-guide/Makefile.am b/doc/users-guide/Makefile.am
> index 74caa96..2e1195e 100644
> --- a/doc/users-guide/Makefile.am
> +++ b/doc/users-guide/Makefile.am
> @@ -30,6 +30,8 @@ IMAGES = $(top_srcdir)/doc/images/overview.svg \
>  $(top_srcdir)/doc/images/release_git.svg \
>  $(top_srcdir)/doc/images/segment.svg \
>  $(top_srcdir)/doc/images/simple_release_git.svg \
> +$(top_srcdir)/doc/images/timeout_fsm.svg \
> +$(top_srcdir)/doc/images/timer_fsm.svg \
>  $(top_srcdir)/doc/images/tm_hierarchy.svg \
>  $(top_srcdir)/doc/images/tm_node.svg
>
> --
> 2.7.4
>
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] process thread and mixed mode

[Christophe Milard]

On 17 May 2016 at 02:09, Bill Fischofer  wrote:

>
>
> On Fri, May 13, 2016 at 6:32 AM, Savolainen, Petri (Nokia - FI/Espoo) <
> petri.savolai...@nokia.com> wrote:
>
>>
>>
>>
>>
>> *From:* Christophe Milard [mailto:christophe.mil...@linaro.org]
>> *Sent:* Friday, May 13, 2016 11:16 AM
>> *To:* Petri Savolainen ; Brian Brooks <
>> brian.bro...@linaro.org>; Bill Fischofer ;
>> Mike Holmes ; LNG ODP Mailman List <
>> lng-odp@lists.linaro.org>; Anders Roxell 
>> *Cc:* Tobias Lindquist 
>> *Subject:* process thread and mixed mode
>>
>>
>>
>> Hi,
>>
>>
>>
>> Let me start in a bit unusual way :-). I do not like my "running things
>> in process mode" patch series  that much...
>>
>> But I do think it should be merged. With the mixed mode.
>>
>>
>>
>> I think mixed mode is good addition, but would need more options: e.g.
>> --odph-num-proc  --odph-num-pthread-per-proc  and thus can be
>> added later. Pure proc mode is enough to get things started.
>>
>>
>>
>> The main goal of the patch series is to push the work regarding a proper
>> and consistent definition of some ODP low level concepts and objects. The
>> patch series makes things fail, hence pushing for defining and coding
>> things better, I hope.  Removing "provocative" things like the mixed mode
>> from it is just defeating the purpose of this series: I'll be happy to
>> remove it (and plenty of other things) when we know which well documented
>> definition/rule makes this code not relevant.
>>
>> As long as we keep saying that odp threads could be indifferently  linux
>> processes or pthreads, I'll be thinking that this series should work,
>> inclusive any kind of mixed mode.
>>
>>
>>
>> However, processes and threads are different things in linux, and I
>> believe this distinction should be kept within ODP: Trying to make
>> processes behave as pthreads for some ODP objects (such as addresses for
>> shmem objects) does not sound like a good idea to me. Yes, this can maybe
>> be done with linux -and even possibly without preallocating the memory-,
>> but still, linux creates 2 distinct virtual address spaces on fork() so why
>> would ODP put any exception to that?.
>> ...And If we do want to create such exceptions (which I don't like),
>> shouln't ODP define its own concept of process and thread as it obviously
>> wouldn't match 100% the definition of the OS?
>>
>>
>>
>>
>>
>> A lot of application internal state data may be directly managed with OS.
>> But I think we cannot depend 100% on OS shared data (== remove shm API all
>> together), since HW/implementation could have multiple ways optimize
>> where/how the memory is allocated/mapped (chip internal vs. external
>> memory, huge page mapped vs. normal page mapped, IO-MMU mapped, etc) … or
>> when passing pointers to ODP API, the API spec may require that the memory
>> is allocated from SHM (application would not know if the API function is
>> implemented as HW or SW).
>>
>>
>>
>>
>>
>> I do nevertheless definitively agree with Petri when saying that that ODP
>> applications should be given a way to "share pointers" between odpthreads.
>> Taking the "process only" behavior forcing every concurrent execution tasks
>> to use tricky offset calculation to find their data is too nasty and
>> inefficient.
>>
>>
>>
>> I think this will boil down to 2 choices:
>>
>> 1) Take these object definitions from the OS and accept the OS rules.
>>
>> 2) Define our own ODP objects (and possibly defines our own rules)
>>
>>
>>
>> 1)Taking the OS definitions:
>>
>> Choice 1) is consistent as long as we accept the OS rules. Taking that
>> choice would mean that any address remain valid within a process only:
>> shmem pointers will not be able to be shared between different processes: I
>> do not see that as a big problem as the applications can use pthreads if it
>> needs to share the same VA, or can fork() at a conveniant time. Moreover,
>> existing applications which should be ported to ODP are probably OK as they
>> would already be using pthreads/processes in the correct way.
>>
>>
>>
>> A more tricky aspect of choice 1) is the development of ODP itself:
>> Operations involving many "ODPthreads" will have to cope with both
>> processes and threads possibly without a clear view of what is done b

[lng-odp] [PATCHv7 00/35] running things in process mode

[Christophe Milard]

Pull can be performed from the URL at the end of this cover letter
All patches sent to the list following Maxim's request.

Since v6
-rebased
-regrouping history, squashing patch 1 and 12; 4, 5 and 12 (Petri)
-mixed mode test removed (but still claimed to be supported in ODP). (Petri)

Since v5
-rebased
-patch removing old unused helper code dropped. (Petri)
-Doc update to discourage usage of the old functions. (Petri, Mike)

Since v4:
-updates following Brian's comments

Since v3:
-fixed rebase error (Christophe)
-rebased

Since v2:
-serious rebase following clash with dba05a28 (api: init: add instance handle)
-squashing the validation changes.

Since v1:
-variable declaration gathered in function 's head (Petri)
-linux prefix removed from helper's types and function names (Mike, Christophe)

Hi,

This patch series adds the ability to run tests/ exemples / perf-test
in "process mode" (i.e letting OPD threads being linux processes)
It it hence tackling ODP-171.

This is achieved in 2 main steps:

A]
The 2 pairs of helper functions:
odph_linux_pthread_create(), odph_linux_pthread_join()
and
odph_linux_process_fork_n(), odph_linux_process_wait_n()
are complemented by:
odph_linux_odpthreads_create() and odph_linux_odpthreads_join()
The latter's callers are unaware of the actual implementation of the ODP
thread (making test truly platform agnostic).
The helper functions decide at run time whether an odp thread is to be
a linux process or pthread based on the command line argument.

B] each test/example now calls a new helper function,
odph_linux_parse_options(), so that the helper can get its own arguments
out of the command line.
Currentely supported args is: --odph_proc.
Defaults assumes thread.

The changed are first done on the shmem tests, and thereafter propagated
to other helper users.
Note that this patch series enable the option but does not affect
make check at this time: make check still calls the tests with no options
which default to thread mode.


[GIT PULL ODPv7] running things in process mode:

The following changes since commit 9359d1b2fedb88c57ee4ef96e4a7282b4e07ffc6:

  linux-generic: scheduler: avoid starvation (2016-05-13 17:20:55 +0300)

are available in the git repository at:

  https://git.linaro.org/people/christophe.milard/odp.git 
test_in_process_mode_v7

for you to fetch changes up to 1ee35450efcd4efe513cf2bd959501b5521b6cbb:

  doc: implementers-guide: adding helper recommendations (2016-05-17 14:04:41 
+0200)

--------


Christophe Milard (35):
  helpers: adding command line argument parsing
  validation: common: adding command line argument parsing
  validation: shmem: adding command line argument parsing
  helpers: linux: creating functions to handle odpthreads
  helper: test: adding odpthread functions tests
  validation: using implementation agnostic function for ODP threads
  validation: most tests: adding command line argument parsing
  validation: init: adding command line argument parsing
  validation: pktio: adding command line argument parsing
  performance: odp_scheduling: proc mode done by helper
  performance: odp_pktio_perf: using agnostic function for ODP threads
  performance: odp_pktio_perf: adding helper cmd line parsing
  performance: odp_l2fwd: using agnostic function for ODP threads
  performance: odp_l2fwd: adding helper cmd line parsing
  performance: crypto: using agnostic function for ODP threads
  performance: crypto: adding helper cmd line parsing
  example: classifier: using agnostic function for ODP threads
  example: classifier: adding helper cmd line parsing
  example: generator: using agnostic function for ODP threads
  example: generator: adding helper cmd line parsing
  example: ipsec: using agnostic function for ODP threads
  example: ipsec: adding helper cmd line parsing
  example: l2fwd_simple: using agnostic function for ODP threads
  example: l2fwd_simple: adding helper cmd line parsing
  example: pktio: using agnostic function for ODP threads
  example: pktio: adding helper cmd line parsing
  example: time: using agnostic function for ODP threads
  example: time: adding helper cmd line parsing
  example: timer: using agnostic function for ODP threads
  example: timer: adding helper cmd line parsing
  example: switch: using agnostic function for ODP threads
  example: switch: adding helper cmd line parsing
  linux-generic: test: ringtest: using agnostic function for ODP threads
  linux-generic: test: ringtest: adding helper cmd line parsing
  doc: implementers-guide: adding helper recommendations

 doc/implementers-guide/implementers-guide.adoc |  32 ++
 example/classifier/odp_classifier.c|  39 +--
 example/generator/odp_generator.c  |  45 +--
 example/ipsec/odp_ipsec.c  |  27 +-
 example/l2fwd_simple/odp_l2fwd_simple.c|  47 ++-
 example/packet/odp_pktio.c   

[lng-odp] [PATCHv7 01/35] helpers: adding command line argument parsing

[Christophe Milard]

A function (odph_merge_getopt_options()) to merge command line arguments
is added. This function can be called by intermediate clients
(e.g c_unit_common) before calling odph_parse_options().
The new function, odph_parse_options() is also added
to parse options meant for the helpers out of a the command line. This
parse function picks up options altering the behaviour of the helpers out
of the command line, and also checks for unknown options (i.e. options not
belonging to the caller set of options or to the helpers.)
No helper options are definied (yet).

Signed-off-by: Christophe Milard 
---
 helper/include/odp/helper/linux.h |  71 ++
 helper/linux.c| 123 ++
 2 files changed, 194 insertions(+)

diff --git a/helper/include/odp/helper/linux.h 
b/helper/include/odp/helper/linux.h
index e2dca35..d47abef 100644
--- a/helper/include/odp/helper/linux.h
+++ b/helper/include/odp/helper/linux.h
@@ -25,6 +25,7 @@ extern "C" {
 #include 
 
 #include 
+#include 
 #include 
 
 /** @addtogroup odph_linux ODPH LINUX
@@ -134,6 +135,76 @@ int odph_linux_process_fork_n(odph_linux_process_t 
*proc_tbl,
 int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num);
 
 /**
+ * Merge getopt options
+ *
+ * Given two sets of getopt options (each containing possibly both short
+ * options -a string- and long options -a option array-) this function
+ * return a single set (i.e. a string for short and an array for long)
+ * being the concatenation of the two given sets.
+ * Due to the fact that the size of these arrays is unknown at compilation
+ * time, this function actually mallocs the the resulting arrays.
+ * The fourth and fith parameters are actually pointers where these malloc'ed
+ * areas are returned.
+ * This means that the caller of this function has to free the two returned
+ * areas!
+ *
+ * @param shortopts1 first set of short options (a string)
+ * @param shortopts2 second set of short options (a string)
+ * @param longopts1  first set of long options (a getopt option array)
+ * @param longopts2  second set of long options (a getopt option array)
+ * @param shortopts  a pointer where the address of the short options list
+ *  (a string) is returned. It contains the concatenation of
+ *  the two given short option strings.
+ * @param longopts   a pointer where the address of the long options list
+ *  (a getopt option array) is returned.
+ *  It contains the concatenation of the two given long
+ *  option arrays.
+ * if any of shortopts1, shortopts2, longopts1, longopts2 is NULL, the
+ * corresponding list as assumed to be empty.
+ * if any of shortopts, longopts is NULL, the corresponding malloc is not
+ * performed.
+ *
+ * @return On success: 0 : both shortopts and longopts are returned (assuming
+ *the given pointer where not null), possibly
+ *pointing to an empty string or an empty option array.
+ *On success, the caller is due to free these areas.
+ *On failure: -1: Nothing is malloc'ed.
+ */
+int odph_merge_getopt_options(const char *shortopts1,
+ const char *shortopts2,
+ const struct option *longopts1,
+ const struct option *longopts2,
+ char **shortopts,
+ struct option **longopts);
+
+/**
+ * Parse linux helper options
+ *
+ * Parse the command line options. Pick up options meant for the helper itself.
+ * If the caller is also having a set of option to parse, it should include
+ * their description here (shortopts desribes the short options and longopts
+ * describes the long options, as for getopt_long()).
+ * This function will issue errors on unknown arguments, so callers failing
+ * to pass their own command line options description here will see their
+ * options rejected.
+ * (the caller wants to set opterr to zero when parsing its own stuff
+ * with getopts to avoid reacting on helper's options).
+ *
+ * @param argc argument count
+ * @param argv argument values
+ * @param caller_shortopts caller's set of short options (string). or NULL.
+ * @param caller_longopts  caller's set of long options (getopt option array).
+ *or NULL.
+ *
+ * @return On success: 0
+ *On failure: -1 failure occurs if a value passed for a helper
+ *   option is invalid, or on meeting unknown options.
+ */
+int odph_parse_options(int argc, char *argv[],
+  const char *caller_shortopts,
+  const struct option *caller_longopts);
+
+/**
  * @}
  */
 
diff --git a/helper/linux.c b/helper/linux.c
index 24e243b..ef435bd 100644
--- a/helper/linux.c
+++ b/helper/linux.c
@@ -236,3 +236,126 @@ int odph_linux_process_wait_n(odph_

[lng-odp] [PATCHv7 02/35] validation: common: adding command line argument parsing

[Christophe Milard]

A function to parse the command line args is added. This function is meant
to parse only arguments altering the behaviour of cunit_common (this
includes the helpers args as cunit_common uses the helpers)
As at this stage only helper args fall into that category, this function
simply calls the helper parsing function.
(parsing pure cunit_commons args might be added later here if needed)

Signed-off-by: Christophe Milard 
---
 test/validation/common/odp_cunit_common.c | 10 ++
 test/validation/common/odp_cunit_common.h |  2 ++
 2 files changed, 12 insertions(+)

diff --git a/test/validation/common/odp_cunit_common.c 
b/test/validation/common/odp_cunit_common.c
index 2712abe..2a5864f 100644
--- a/test/validation/common/odp_cunit_common.c
+++ b/test/validation/common/odp_cunit_common.c
@@ -342,3 +342,13 @@ int odp_cunit_register(odp_suiteinfo_t testsuites[])
 
return 0;
 }
+
+/*
+ * Parse command line options to extract options affectiong cunit_common.
+ * (hence also helpers options as cunit_common uses the helpers)
+ * Options private to the test calling cunit_common are not parsed here.
+ */
+int odp_cunit_parse_options(int argc, char *argv[])
+{
+   return odph_parse_options(argc, argv, NULL, NULL);
+}
diff --git a/test/validation/common/odp_cunit_common.h 
b/test/validation/common/odp_cunit_common.h
index 22c2a19..3812b0f 100644
--- a/test/validation/common/odp_cunit_common.h
+++ b/test/validation/common/odp_cunit_common.h
@@ -73,6 +73,8 @@ typedef struct {
int numthrds; /**< no of pthreads to create */
 } pthrd_arg;
 
+/* parse parameters that affect the behaviour of odp_cunit_common */
+int odp_cunit_parse_options(int argc, char *argv[]);
 /* register suites to be run via odp_cunit_run() */
 int odp_cunit_register(odp_suiteinfo_t testsuites[]);
 /* update tests previously registered via odp_cunit_register() */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 04/35] helpers: linux: creating functions to handle odpthreads

[Christophe Milard]

odph_linux_pthread_t and odph_linux_process_t are joined, creating
odph_odpthread_t. Tests should be using the latter only so that
the actual implementation of the ODP thread don't leak in the test itself.
(odph_odpthread_t is opaque to the test, and is common to both
 thread and process omplementation of ODP threads)
Two functions, odph_odpthreads_create and odph_odpthreads_join
are then created to create and termindate odp threads.
These functions create the odp threads either as linux processes or
as linux threads, depending on the command line arguments (flag odph_proc
for processes).

Signed-off-by: Christophe Milard 
---
 helper/include/odp/helper/linux.h |  67 +-
 helper/linux.c| 259 ++
 2 files changed, 325 insertions(+), 1 deletion(-)

diff --git a/helper/include/odp/helper/linux.h 
b/helper/include/odp/helper/linux.h
index d47abef..01c348d 100644
--- a/helper/include/odp/helper/linux.h
+++ b/helper/include/odp/helper/linux.h
@@ -49,7 +49,6 @@ typedef struct {
odph_linux_thr_params_t thr_params;
 } odph_linux_pthread_t;
 
-
 /** Linux process state information */
 typedef struct {
pid_t pid;  /**< Process ID */
@@ -57,6 +56,44 @@ typedef struct {
int   status;   /**< Process state change status */
 } odph_linux_process_t;
 
+/** odpthread linux type: whether an ODP thread is a linux thread or process */
+typedef enum odph_odpthread_linuxtype_e {
+   ODPTHREAD_NOT_STARTED = 0,
+   ODPTHREAD_PROCESS,
+   ODPTHREAD_PTHREAD
+} odph_odpthread_linuxtype_t;
+
+/** odpthread parameters for odp threads (pthreads and processes) */
+typedef struct {
+   int (*start)(void *);   /**< Thread entry point function */
+   void *arg;  /**< Argument for the function */
+   odp_thread_type_t thr_type; /**< ODP thread type */
+   odp_instance_t instance;/**< ODP instance handle */
+} odph_odpthread_params_t;
+
+/** The odpthread starting arguments, used both in process or thread mode */
+typedef struct {
+   odph_odpthread_linuxtype_t linuxtype;
+   odph_odpthread_params_t thr_params; /*copy of thread start parameter*/
+} odph_odpthread_start_args_t;
+
+/** Linux odpthread state information, used both in process or thread mode */
+typedef struct {
+   odph_odpthread_start_args_t start_args;
+   int cpu;/**< CPU ID */
+   int last;   /**< true if last table entry */
+   union {
+   struct { /* for thread implementation */
+   pthread_t   thread_id; /**< Pthread ID */
+   pthread_attr_t  attr;   /**< Pthread attributes */
+   } thread;
+   struct { /* for process implementation */
+   pid_t   pid;/**< Process ID */
+   int status; /**< Process state chge status*/
+   } proc;
+   };
+} odph_odpthread_t;
+
 /**
  * Creates and launches pthreads
  *
@@ -135,6 +172,34 @@ int odph_linux_process_fork_n(odph_linux_process_t 
*proc_tbl,
 int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num);
 
 /**
+ * Creates and launches odpthreads (as linux threads or processes)
+ *
+ * Creates, pins and launches threads to separate CPU's based on the cpumask.
+ *
+ * @param thread_tblThread table
+ * @param mask  CPU mask
+ * @param thr_paramsODP thread parameters
+ *
+ * @return Number of threads created
+ */
+int odph_odpthreads_create(odph_odpthread_t *thread_tbl,
+  const odp_cpumask_t *mask,
+  const odph_odpthread_params_t *thr_params);
+
+/**
+ * Waits odpthreads (as linux threads or processes) to exit.
+ *
+ * Returns when all odpthreads have terminated.
+ *
+ * @param thread_tblThread table
+ * @return The number of joined threads or -1 on error.
+ * (error occurs if any of the start_routine return non-zero or if
+ *  the thread join/process wait itself failed -e.g. as the result of a kill)
+ *
+ */
+int odph_odpthreads_join(odph_odpthread_t *thread_tbl);
+
+/**
  * Merge getopt options
  *
  * Given two sets of getopt options (each containing possibly both short
diff --git a/helper/linux.c b/helper/linux.c
index ef435bd..6366694 100644
--- a/helper/linux.c
+++ b/helper/linux.c
@@ -21,6 +21,10 @@
 #include 
 #include "odph_debug.h"
 
+static struct {
+   int proc; /* true when process mode is required, false otherwise */
+} helper_options;
+
 static void *odp_run_start_routine(void *arg)
 {
odph_linux_thr_params_t *thr_params = arg;
@@ -238,6 +242,257 @@ int odph_linux_process_wait_n(odph_linux_process_t 
*proc_tbl, int num)
 }
 
 /*
+ * wrapper for odpthreads, either implemented as linux threads or processes.
+ * (in process mode, if start_routine returns NULL, the process return 
FAILURE).

[lng-odp] [PATCHv7 03/35] validation: shmem: adding command line argument parsing

[Christophe Milard]

As the test itself does not have specific args, it just calls
the cunit_common parsing function to pick up cunit_common and helpers
arguments.

Signed-off-by: Christophe Milard 
---
 test/validation/shmem/shmem.c  | 10 --
 test/validation/shmem/shmem.h  |  2 +-
 test/validation/shmem/shmem_main.c |  4 ++--
 3 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/test/validation/shmem/shmem.c b/test/validation/shmem/shmem.c
index 0bf38d2..b4c6847 100644
--- a/test/validation/shmem/shmem.c
+++ b/test/validation/shmem/shmem.c
@@ -91,9 +91,15 @@ odp_suiteinfo_t shmem_suites[] = {
ODP_SUITE_INFO_NULL,
 };
 
-int shmem_main(void)
+int shmem_main(int argc, char *argv[])
 {
-   int ret = odp_cunit_register(shmem_suites);
+   int ret;
+
+   /* parse common options: */
+   if (odp_cunit_parse_options(argc, argv))
+   return -1;
+
+   ret = odp_cunit_register(shmem_suites);
 
if (ret == 0)
ret = odp_cunit_run();
diff --git a/test/validation/shmem/shmem.h b/test/validation/shmem/shmem.h
index d60cf64..a5893d9 100644
--- a/test/validation/shmem/shmem.h
+++ b/test/validation/shmem/shmem.h
@@ -19,6 +19,6 @@ extern odp_testinfo_t shmem_suite[];
 extern odp_suiteinfo_t shmem_suites[];
 
 /* main test program: */
-int shmem_main(void);
+int shmem_main(int argc, char *argv[]);
 
 #endif
diff --git a/test/validation/shmem/shmem_main.c 
b/test/validation/shmem/shmem_main.c
index 445fbf1..4c69130 100644
--- a/test/validation/shmem/shmem_main.c
+++ b/test/validation/shmem/shmem_main.c
@@ -6,7 +6,7 @@
 
 #include "shmem.h"
 
-int main(void)
+int main(int argc, char *argv[])
 {
-   return shmem_main();
+   return shmem_main(argc, argv);
 }
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 06/35] validation: using implementation agnostic function for ODP threads

[Christophe Milard]

cunit_common is changed to use the implementation agnostic ODP thread
create and join functions, from helpers. Tests are no longer aware if
an odp thread is a linux process or linux thread under the hood.
The helper decides.
The function pointer which is passed when creating the odp threads now
points to a function returning an int instead of a ptr.
This is changed so that when odp threads are processes, the int returned
become the process exit code.
The return code is nevertheless just used to detect erros (as before).
Note that it is now important that the ODP threads return a correct
status, as for processes, odp threads runs on a copy of the memory, so
that c_unit assertions will not be reflected on the main process summary.
Failing to return a proper status means that error will be lost when
running the test in process mode.
odp threads returning an error status code will be detected as an error
on the main process at join time...

Signed-off-by: Christophe Milard 
---
 test/validation/atomic/atomic.c   | 34 
 test/validation/barrier/barrier.c |  8 +++---
 test/validation/common/odp_cunit_common.c | 14 ++
 test/validation/common/odp_cunit_common.h |  4 +--
 test/validation/lock/lock.c   | 44 +++
 test/validation/scheduler/scheduler.c |  8 +++---
 test/validation/shmem/shmem.c |  6 ++---
 test/validation/thread/thread.c   |  4 +--
 test/validation/timer/timer.c |  4 +--
 9 files changed, 65 insertions(+), 61 deletions(-)

diff --git a/test/validation/atomic/atomic.c b/test/validation/atomic/atomic.c
index 5eec467..0dfd651 100644
--- a/test/validation/atomic/atomic.c
+++ b/test/validation/atomic/atomic.c
@@ -584,7 +584,7 @@ int atomic_init(odp_instance_t *inst)
 }
 
 /* Atomic tests */
-static void *test_atomic_inc_dec_thread(void *arg UNUSED)
+static int test_atomic_inc_dec_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -594,10 +594,10 @@ static void *test_atomic_inc_dec_thread(void *arg UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_add_sub_thread(void *arg UNUSED)
+static int test_atomic_add_sub_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -607,10 +607,10 @@ static void *test_atomic_add_sub_thread(void *arg UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_fetch_inc_dec_thread(void *arg UNUSED)
+static int test_atomic_fetch_inc_dec_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -620,10 +620,10 @@ static void *test_atomic_fetch_inc_dec_thread(void *arg 
UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_fetch_add_sub_thread(void *arg UNUSED)
+static int test_atomic_fetch_add_sub_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -633,10 +633,10 @@ static void *test_atomic_fetch_add_sub_thread(void *arg 
UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_max_min_thread(void *arg UNUSED)
+static int test_atomic_max_min_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -646,10 +646,10 @@ static void *test_atomic_max_min_thread(void *arg UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_cas_inc_dec_thread(void *arg UNUSED)
+static int test_atomic_cas_inc_dec_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -659,10 +659,10 @@ static void *test_atomic_cas_inc_dec_thread(void *arg 
UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_xchg_thread(void *arg UNUSED)
+static int test_atomic_xchg_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -672,10 +672,10 @@ static void *test_atomic_xchg_thread(void *arg UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void *test_atomic_non_relaxed_thread(void *arg UNUSED)
+static int test_atomic_non_relaxed_thread(void *arg UNUSED)
 {
per_thread_mem_t *per_thread_mem;
 
@@ -685,10 +685,10 @@ static void *test_atomic_non_relaxed_thread(void *arg 
UNUSED)
 
thread_finalize(per_thread_mem);
 
-   return NULL;
+   return CU_get_number_of_failures();
 }
 
-static void test_atomic_functional(void *func_ptr(void *), int check)
+static void test_atomic_functional(int func_ptr(void *), int check)
 {
pthrd_arg arg;
 
diff --git a/test/validation/barrier/barrier.c 
b/test/validation

[lng-odp] [PATCHv7 05/35] helper: test: adding odpthread functions tests

[Christophe Milard]

Simple tests testing odph_odpthreads_create() and
odph_odpthreads_join() are added.
A single test binary, odpthreads, is added. This program creates
odp threads as pthreads or linux processes, dependng on its calling args.
Two calling scripts are added to create ODP threads as either processes,
or threads.

Signed-off-by: Christophe Milard 
---
 helper/test/.gitignore  |   1 +
 helper/test/Makefile.am |   7 ++-
 helper/test/odpthreads.c| 100 
 helper/test/odpthreads_as_processes |  14 +
 helper/test/odpthreads_as_pthreads  |  14 +
 5 files changed, 134 insertions(+), 2 deletions(-)
 create mode 100644 helper/test/odpthreads.c
 create mode 100755 helper/test/odpthreads_as_processes
 create mode 100755 helper/test/odpthreads_as_pthreads

diff --git a/helper/test/.gitignore b/helper/test/.gitignore
index e232753..5ce3c3b 100644
--- a/helper/test/.gitignore
+++ b/helper/test/.gitignore
@@ -1,6 +1,7 @@
 *.trs
 *.log
 chksum
+odpthreads
 parse
 process
 table
diff --git a/helper/test/Makefile.am b/helper/test/Makefile.am
index 7f0b67d..8e65948 100644
--- a/helper/test/Makefile.am
+++ b/helper/test/Makefile.am
@@ -11,9 +11,10 @@ EXECUTABLES = chksum$(EXEEXT) \
   process$(EXEEXT)\
   table$(EXEEXT)
 
-COMPILE_ONLY =
+COMPILE_ONLY = odpthreads
 
-TESTSCRIPTS =
+TESTSCRIPTS = odpthreads_as_processes \
+ odpthreads_as_pthreads
 
 if test_helper
 TESTS = $(EXECUTABLES) $(TESTSCRIPTS)
@@ -25,6 +26,8 @@ bin_PROGRAMS = $(EXECUTABLES) $(COMPILE_ONLY)
 
 
 dist_chksum_SOURCES = chksum.c
+dist_odpthreads_SOURCES = odpthreads.c
+odpthreads_LDADD = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la
 dist_thread_SOURCES = thread.c
 thread_LDADD = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la
 dist_process_SOURCES = process.c
diff --git a/helper/test/odpthreads.c b/helper/test/odpthreads.c
new file mode 100644
index 000..bba4fa5
--- /dev/null
+++ b/helper/test/odpthreads.c
@@ -0,0 +1,100 @@
+/* Copyright (c) 2016, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+/*
+ * This program tests the ability of the linux helper to create ODP threads,
+ * either implemented as linux pthreads or as linux processes, depending on
+ * the option passed to the program (--odph_proc, --odph_thread or both)
+ */
+
+#include 
+#include 
+#include 
+
+#define NUMBER_WORKERS 16
+static int worker_fn(void *arg TEST_UNUSED)
+{
+   /* depend on the odp helper to call odp_init_local */
+
+   printf("Worker thread on CPU %d\n", odp_cpu_id());
+
+   /* depend on the odp helper to call odp_term_local */
+
+   return 0;
+}
+
+/* Create additional dataplane opdthreads */
+int main(int argc, char *argv[])
+{
+   odp_instance_t instance;
+   odph_odpthread_params_t thr_params;
+   odph_odpthread_t thread_tbl[NUMBER_WORKERS];
+   odp_cpumask_t cpu_mask;
+   int num_workers;
+   int cpu;
+   int ret;
+   char cpumaskstr[ODP_CPUMASK_STR_SIZE];
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, NULL, NULL);
+
+   if (odp_init_global(&instance, NULL, NULL)) {
+   LOG_ERR("Error: ODP global init failed.\n");
+   exit(EXIT_FAILURE);
+   }
+
+   if (odp_init_local(instance, ODP_THREAD_CONTROL)) {
+   LOG_ERR("Error: ODP local init failed.\n");
+   exit(EXIT_FAILURE);
+   }
+
+   /* discover how many opdthreads this system can support */
+   num_workers = odp_cpumask_default_worker(&cpu_mask, NUMBER_WORKERS);
+   if (num_workers < NUMBER_WORKERS) {
+   printf("System can only support %d threads and not the %d 
requested\n",
+  num_workers, NUMBER_WORKERS);
+   }
+
+   /* generate a summary for the user */
+   (void)odp_cpumask_to_str(&cpu_mask, cpumaskstr, sizeof(cpumaskstr));
+   printf("default cpu mask:   %s\n", cpumaskstr);
+   printf("default num worker threads: %i\n", num_workers);
+
+   cpu = odp_cpumask_first(&cpu_mask);
+   printf("the first CPU:  %i\n", cpu);
+
+   /* reserve cpu 0 for the control plane so remove it from
+* the default mask */
+   odp_cpumask_clr(&cpu_mask, 0);
+   num_workers = odp_cpumask_count(&cpu_mask);
+   (void)odp_cpumask_to_str(&cpu_mask, cpumaskstr, sizeof(cpumaskstr));
+   printf("new cpu mask:   %s\n", cpumaskstr);
+   printf("new num worker threads: %i\n\n", num_workers);
+
+   memset(&thr_params, 0, sizeof(thr_params));
+   thr_params.start= worker_fn;
+   thr_params.arg  = NULL;
+   thr_params.thr_type = ODP_THREAD_WORKER;
+   thr_params.instance = instance;
+
+   odph_odp

[lng-odp] [PATCHv7 08/35] validation: init: adding command line argument parsing

[Christophe Milard]

As the tests themselves do not have specific args, they just call
the cunit_common parsing function to pick up cunit_common and helpers
arguments.

Signed-off-by: Christophe Milard 
---
 test/validation/init/init.c| 18 +++---
 test/validation/init/init.h|  6 +++---
 test/validation/init/init_main_abort.c |  4 ++--
 test/validation/init/init_main_log.c   |  4 ++--
 4 files changed, 22 insertions(+), 10 deletions(-)

diff --git a/test/validation/init/init.c b/test/validation/init/init.c
index a802d41..61055fa 100644
--- a/test/validation/init/init.c
+++ b/test/validation/init/init.c
@@ -52,10 +52,14 @@ static void odp_init_abort(void)
abort();
 }
 
-int init_main_abort(void)
+int init_main_abort(int argc, char *argv[])
 {
int ret;
 
+   /* parse common options: */
+   if (odp_cunit_parse_options(argc, argv))
+   return -1;
+
/* prevent default ODP init: */
odp_cunit_register_global_init(NULL);
odp_cunit_register_global_term(NULL);
@@ -116,10 +120,14 @@ static int odp_init_log(odp_log_level_t level 
__attribute__((unused)),
return r;
 }
 
-int init_main_log(void)
+int init_main_log(int argc, char *argv[])
 {
int ret;
 
+   /* parse common options: */
+   if (odp_cunit_parse_options(argc, argv))
+   return -1;
+
/* prevent default ODP init: */
odp_cunit_register_global_init(NULL);
odp_cunit_register_global_term(NULL);
@@ -157,10 +165,14 @@ odp_suiteinfo_t init_suites_ok[] = {
ODP_SUITE_INFO_NULL,
 };
 
-int init_main_ok(void)
+int init_main_ok(int argc, char *argv[])
 {
int ret;
 
+   /* parse common options: */
+   if (odp_cunit_parse_options(argc, argv))
+   return -1;
+
/* prevent default ODP init: */
odp_cunit_register_global_init(NULL);
odp_cunit_register_global_term(NULL);
diff --git a/test/validation/init/init.h b/test/validation/init/init.h
index 272d426..cad9cf9 100644
--- a/test/validation/init/init.h
+++ b/test/validation/init/init.h
@@ -25,8 +25,8 @@ extern odp_suiteinfo_t init_suites_log[];
 extern odp_suiteinfo_t init_suites_ok[];
 
 /* main test program: */
-int init_main_abort(void);
-int init_main_log(void);
-int init_main_ok(void);
+int init_main_abort(int argc, char *argv[]);
+int init_main_log(int argc, char *argv[]);
+int init_main_ok(int argc, char *argv[]);
 
 #endif
diff --git a/test/validation/init/init_main_abort.c 
b/test/validation/init/init_main_abort.c
index c7bdd9d..2e0faaf 100644
--- a/test/validation/init/init_main_abort.c
+++ b/test/validation/init/init_main_abort.c
@@ -5,7 +5,7 @@
  */
 #include "init.h"
 
-int main(void)
+int main(int argc, char *argv[])
 {
-   return init_main_abort();
+   return init_main_abort(argc, argv);
 }
diff --git a/test/validation/init/init_main_log.c 
b/test/validation/init/init_main_log.c
index f35ac38..41dd00d 100644
--- a/test/validation/init/init_main_log.c
+++ b/test/validation/init/init_main_log.c
@@ -5,7 +5,7 @@
  */
 #include "init.h"
 
-int main(void)
+int main(int argc, char *argv[])
 {
-   return init_main_log();
+   return init_main_log(argc, argv);
 }
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 07/35] validation: most tests: adding command line argument parsing

[Christophe Milard]

As the tests themselves do not have specific args, they just call
the cunit_common parsing function to pick up cunit_common and helpers
arguments.
This was originally many patches, now squashed on maintener's request.
init and pktio tests are still separated as they have specificities

Signed-off-by: Christophe Milard 
---
 test/validation/atomic/atomic.c  |  6 +-
 test/validation/atomic/atomic.h  |  2 +-
 test/validation/atomic/atomic_main.c |  4 ++--
 test/validation/barrier/barrier.c|  6 +-
 test/validation/barrier/barrier.h|  2 +-
 test/validation/barrier/barrier_main.c   |  4 ++--
 test/validation/buffer/buffer.c  | 10 --
 test/validation/buffer/buffer.h  |  2 +-
 test/validation/buffer/buffer_main.c |  4 ++--
 test/validation/classification/classification.c  | 10 --
 test/validation/classification/classification.h  |  2 +-
 test/validation/classification/classification_main.c |  4 ++--
 test/validation/cpumask/cpumask.c| 10 --
 test/validation/cpumask/cpumask.h|  2 +-
 test/validation/cpumask/cpumask_main.c   |  4 ++--
 test/validation/crypto/crypto.c  |  6 +-
 test/validation/crypto/crypto.h  |  2 +-
 test/validation/crypto/crypto_main.c |  4 ++--
 test/validation/errno/errno.c| 10 --
 test/validation/errno/errno.h|  2 +-
 test/validation/errno/errno_main.c   |  4 ++--
 test/validation/hash/hash.c  | 10 --
 test/validation/hash/hash.h  |  2 +-
 test/validation/hash/hash_main.c |  4 ++--
 test/validation/init/init_main_ok.c  |  4 ++--
 test/validation/lock/lock.c  |  6 +-
 test/validation/lock/lock.h  |  2 +-
 test/validation/lock/lock_main.c |  4 ++--
 test/validation/packet/packet.c  | 10 --
 test/validation/packet/packet.h  |  2 +-
 test/validation/packet/packet_main.c |  4 ++--
 test/validation/pool/pool.c  | 10 --
 test/validation/pool/pool.h  |  2 +-
 test/validation/pool/pool_main.c |  4 ++--
 test/validation/queue/queue.c| 10 --
 test/validation/queue/queue.h|  2 +-
 test/validation/queue/queue_main.c   |  4 ++--
 test/validation/random/random.c  | 10 --
 test/validation/random/random.h  |  2 +-
 test/validation/random/random_main.c |  4 ++--
 test/validation/scheduler/scheduler.c| 10 --
 test/validation/scheduler/scheduler.h|  2 +-
 test/validation/scheduler/scheduler_main.c   |  4 ++--
 test/validation/std_clib/std_clib.c  | 10 --
 test/validation/std_clib/std_clib.h  |  2 +-
 test/validation/std_clib/std_clib_main.c |  4 ++--
 test/validation/system/system.c  | 10 --
 test/validation/system/system.h  |  2 +-
 test/validation/system/system_main.c |  4 ++--
 test/validation/thread/thread.c  | 10 --
 test/validation/thread/thread.h  |  2 +-
 test/validation/thread/thread_main.c |  4 ++--
 test/validation/time/time.c  | 10 --
 test/validation/time/time.h  |  2 +-
 test/validation/time/time_main.c |  4 ++--
 test/validation/timer/timer.c|  6 +-
 test/validation/timer/timer.h|  2 +-
 test/validation/timer/timer_main.c   |  4 ++--
 test/validation/traffic_mngr/traffic_mngr.c  |  6 +-
 test/validation/traffic_mngr/traffic_mngr.h  |  2 +-
 test/validation/traffic_mngr/traffic_mngr_main.c |  4 ++--
 61 files changed, 204 insertions(+), 96 deletions(-)

diff --git a/test/validation/atomic/atomic.c b/test/validation/atomic/atomic.c
index 0dfd651..c4e9345 100644
--- a/test/validation/atomic/atomic.c
+++ b/test/validation/atomic/atomic.c
@@ -866,10 +866,14 @@ odp_suiteinfo_t atomic_suites[] = {
ODP_SUITE_INFO_NULL
 };
 
-int atomic_main(void)
+int atomic_main(int argc, char *argv[])
 {
int ret;
 
+   /* parse common options: */
+   if (odp_cunit_parse_options(argc, argv))
+   return -1;
+
odp_cunit_register_global_init(atomic_init);
 
ret = odp_cunit_register(atomic_suites);
diff --git a/test/validation/atomic/atomic.h b/test/validation/a

[lng-odp] [PATCHv7 09/35] validation: pktio: adding command line argument parsing

[Christophe Milard]

As the test itself does not have specific args, it just calls
the cunit_common parsing function to pick up cunit_common and helpers
arguments.

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/test/pktio/pktio_run| 21 ++---
 platform/linux-generic/test/pktio/pktio_run_dpdk   | 17 -
 platform/linux-generic/test/pktio/pktio_run_netmap |  5 -
 platform/linux-generic/test/pktio/pktio_run_pcap   |  5 -
 platform/linux-generic/test/pktio/pktio_run_tap|  6 +-
 test/validation/pktio/pktio.c  | 10 --
 test/validation/pktio/pktio.h  |  2 +-
 test/validation/pktio/pktio_main.c |  4 ++--
 8 files changed, 58 insertions(+), 12 deletions(-)

diff --git a/platform/linux-generic/test/pktio/pktio_run 
b/platform/linux-generic/test/pktio/pktio_run
index 7029ab2..4086fc2 100755
--- a/platform/linux-generic/test/pktio/pktio_run
+++ b/platform/linux-generic/test/pktio/pktio_run
@@ -6,6 +6,15 @@
 # SPDX-License-Identifier: BSD-3-Clause
 #
 
+# Proceed the pktio tests. This script expects at least one argument:
+#  setup)   setup the pktio test environment
+#  cleanup) cleanup the pktio test environment
+#  run) run the pktio tests (setup, run, cleanup)
+# extra arguments are passed unchanged to the test itself (pktio_main)
+# Without arguments, "run" is assumed and no extra argument is passed to the
+# test (legacy mode).
+#
+
 # directories where pktio_main binary can be found:
 # -in the validation dir when running make check (intree or out of tree)
 # -in the script directory, when running after 'make install', or
@@ -59,7 +68,7 @@ run_test()
if [ "$disabletype" != "SKIP" ]; then
export ODP_PKTIO_DISABLE_SOCKET_${distype}=y
fi
-   pktio_main${EXEEXT}
+   pktio_main${EXEEXT} $*
if [ $? -ne 0 ]; then
ret=1
fi
@@ -75,7 +84,7 @@ run_test()
 run()
 {
echo "pktio: using 'loop' device"
-   pktio_main${EXEEXT}
+   pktio_main${EXEEXT} $*
loop_ret=$?
 
# need to be root to run tests with real interfaces
@@ -103,8 +112,14 @@ run()
exit $ret
 }
 
-case "$1" in
+if [ $#  > 0]; then
+   action=$1
+   shift
+fi
+
+case "$action" in
setup)   setup_pktio_env   ;;
cleanup) cleanup_pktio_env ;;
+   run) run ;;
*)   run ;;
 esac
diff --git a/platform/linux-generic/test/pktio/pktio_run_dpdk 
b/platform/linux-generic/test/pktio/pktio_run_dpdk
index 50e910a..1a1efac 100755
--- a/platform/linux-generic/test/pktio/pktio_run_dpdk
+++ b/platform/linux-generic/test/pktio/pktio_run_dpdk
@@ -6,6 +6,15 @@
 # SPDX-License-Identifier: BSD-3-Clause
 #
 
+# Proceed the pktio tests. This script expects at least one argument:
+#  setup)   setup the pktio test environment
+#  cleanup) cleanup the pktio test environment
+#  run) run the pktio tests (setup, run, cleanup)
+# extra arguments are passed unchanged to the test itself (pktio_main)
+# Without arguments, "run" is assumed and no extra argument is passed to the
+# test (legacy mode).
+#
+
 # directories where pktio_main binary can be found:
 # -in the validation dir when running make check (intree or out of tree)
 # -in the script directory, when running after 'make install', or
@@ -46,7 +55,7 @@ run_test()
 {
local ret=0
 
-   pktio_main${EXEEXT}
+   pktio_main${EXEEXT} $*
ret=$?
if [ $ret -ne 0 ]; then
echo "!!! FAILED !!!"
@@ -73,8 +82,14 @@ run()
run_test
 }
 
+if [ $#  > 0]; then
+   action=$1
+   shift
+fi
+
 case "$1" in
setup)   setup_pktio_env   ;;
cleanup) cleanup_pktio_env ;;
+   run) run ;;
*)   run ;;
 esac
diff --git a/platform/linux-generic/test/pktio/pktio_run_netmap 
b/platform/linux-generic/test/pktio/pktio_run_netmap
index b7f2575..b651ea3 100755
--- a/platform/linux-generic/test/pktio/pktio_run_netmap
+++ b/platform/linux-generic/test/pktio/pktio_run_netmap
@@ -6,6 +6,9 @@
 # SPDX-License-Identifier: BSD-3-Clause
 #
 
+# any parameter passed as arguments to this script is passed unchanged to
+# the test itself (pktio_main)
+
 # directories where pktio_main binary can be found:
 # -in the validation dir when running make check (intree or out of tree)
 # -in the script directory, when running after 'make install', or
@@ -45,7 +48,7 @@ run_test()
 {
local ret=0
 
-   pktio_main${EXEEXT}
+   pktio_main${EXEEXT} $*
ret=$?
 
if [ $ret -ne 0 ]; then
diff --git a/platform/linux-generic/test/pktio/pktio_run_pcap 
b/platform/linux-generic/test/pktio/pktio_run_pcap
index c130417..51716fb 100755
--- a/platform/linux-generic/test/pktio/pktio_

[lng-odp] [PATCHv7 12/35] performance: odp_pktio_perf: adding helper cmd line parsing

[Christophe Milard]

odp_pktio_perf now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_pktio_perf.c | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/test/performance/odp_pktio_perf.c 
b/test/performance/odp_pktio_perf.c
index e1efc4a..98ec681 100644
--- a/test/performance/odp_pktio_perf.c
+++ b/test/performance/odp_pktio_perf.c
@@ -916,7 +916,7 @@ static void parse_args(int argc, char *argv[], test_args_t 
*args)
int opt;
int long_index;
 
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
{"txcount",   required_argument, NULL, 't'},
{"txbatch",   required_argument, NULL, 'b'},
@@ -931,6 +931,11 @@ static void parse_args(int argc, char *argv[], test_args_t 
*args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+c:t:b:pR:l:r:i:d:vh";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
args->cpu_count  = 0; /* all CPUs */
args->num_tx_workers = 0; /* defaults to cpu_count+1/2 */
args->tx_batch_len   = BATCH_LEN_MAX;
@@ -941,8 +946,10 @@ static void parse_args(int argc, char *argv[], test_args_t 
*args)
args->schedule   = 1;
args->verbose= 0;
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (1) {
-   opt = getopt_long(argc, argv, "+c:t:b:pR:l:r:i:d:vh",
+   opt = getopt_long(argc, argv, shortopts,
  longopts, &long_index);
 
if (opt == -1)
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 11/35] performance: odp_pktio_perf: using agnostic function for ODP threads

[Christophe Milard]

odp_pktio_perf is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_pktio_perf is no longer aware on how the odpthread is implemented any
longer.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_pktio_perf.c | 23 +++
 1 file changed, 11 insertions(+), 12 deletions(-)

diff --git a/test/performance/odp_pktio_perf.c 
b/test/performance/odp_pktio_perf.c
index 57635d3..e1efc4a 100644
--- a/test/performance/odp_pktio_perf.c
+++ b/test/performance/odp_pktio_perf.c
@@ -303,7 +303,7 @@ static int send_packets(odp_pktout_queue_t pktout,
  * Main packet transmit routine. Transmit packets at a fixed rate for
  * specified length of time.
  */
-static void *run_thread_tx(void *arg)
+static int run_thread_tx(void *arg)
 {
test_globals_t *globals;
int thr_id;
@@ -381,7 +381,7 @@ static void *run_thread_tx(void *arg)
   odp_time_to_ns(stats->s.idle_ticks) /
   (uint64_t)ODP_TIME_MSEC_IN_NS);
 
-   return NULL;
+   return 0;
 }
 
 static int receive_packets(odp_queue_t queue,
@@ -411,7 +411,7 @@ static int receive_packets(odp_queue_t queue,
return n_ev;
 }
 
-static void *run_thread_rx(void *arg)
+static int run_thread_rx(void *arg)
 {
test_globals_t *globals;
int thr_id, batch_len;
@@ -456,7 +456,7 @@ static void *run_thread_rx(void *arg)
break;
}
 
-   return NULL;
+   return 0;
 }
 
 /*
@@ -601,11 +601,11 @@ static int run_test_single(odp_cpumask_t *thd_mask_tx,
   odp_cpumask_t *thd_mask_rx,
   test_status_t *status)
 {
-   odph_linux_pthread_t thd_tbl[MAX_WORKERS];
+   odph_odpthread_t thd_tbl[MAX_WORKERS];
thread_args_t args_tx, args_rx;
uint64_t expected_tx_cnt;
int num_tx_workers, num_rx_workers;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
memset(&thr_params, 0, sizeof(thr_params));
thr_params.thr_type = ODP_THREAD_WORKER;
@@ -623,7 +623,7 @@ static int run_test_single(odp_cpumask_t *thd_mask_tx,
thr_params.start  = run_thread_rx;
thr_params.arg= &args_rx;
args_rx.batch_len = gbl_args->args.rx_batch_len;
-   odph_linux_pthread_create(&thd_tbl[0], thd_mask_rx, &thr_params);
+   odph_odpthreads_create(&thd_tbl[0], thd_mask_rx, &thr_params);
odp_barrier_wait(&gbl_args->rx_barrier);
num_rx_workers = odp_cpumask_count(thd_mask_rx);
 
@@ -634,13 +634,12 @@ static int run_test_single(odp_cpumask_t *thd_mask_tx,
args_tx.pps   = status->pps_curr / num_tx_workers;
args_tx.duration  = gbl_args->args.duration;
args_tx.batch_len = gbl_args->args.tx_batch_len;
-   odph_linux_pthread_create(&thd_tbl[num_rx_workers], thd_mask_tx,
- &thr_params);
+   odph_odpthreads_create(&thd_tbl[num_rx_workers], thd_mask_tx,
+  &thr_params);
odp_barrier_wait(&gbl_args->tx_barrier);
 
/* wait for transmitter threads to terminate */
-   odph_linux_pthread_join(&thd_tbl[num_rx_workers],
-   num_tx_workers);
+   odph_odpthreads_join(&thd_tbl[num_rx_workers]);
 
/* delay to allow transmitted packets to reach the receivers */
odp_time_wait_ns(SHUTDOWN_DELAY_NS);
@@ -649,7 +648,7 @@ static int run_test_single(odp_cpumask_t *thd_mask_tx,
odp_atomic_store_u32(&shutdown, 1);
 
/* wait for receivers */
-   odph_linux_pthread_join(&thd_tbl[0], num_rx_workers);
+   odph_odpthreads_join(&thd_tbl[0]);
 
if (!status->warmup)
return process_results(expected_tx_cnt, status);
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 14/35] performance: odp_l2fwd: adding helper cmd line parsing

[Christophe Milard]

odp_l2fwd now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_l2fwd.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/test/performance/odp_l2fwd.c b/test/performance/odp_l2fwd.c
index 9a8d578..e38e6f8 100644
--- a/test/performance/odp_l2fwd.c
+++ b/test/performance/odp_l2fwd.c
@@ -1051,7 +1051,7 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
char *addr_str;
size_t len;
int i;
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
{"time", required_argument, NULL, 't'},
{"accuracy", required_argument, NULL, 'a'},
@@ -1066,15 +1066,21 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts =  "+c:+t:+a:i:m:o:r:d:s:e:h";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
appl_args->time = 0; /* loop forever if time to run is 0 */
appl_args->accuracy = 1; /* get and print pps stats second */
appl_args->dst_change = 1; /* change eth dst address by default */
appl_args->src_change = 1; /* change eth src address by default */
appl_args->error_check = 0; /* don't check packet errors by default */
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (1) {
-   opt = getopt_long(argc, argv, "+c:+t:+a:i:m:o:r:d:s:e:h",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
 
if (opt == -1)
break;  /* No more options */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 13/35] performance: odp_l2fwd: using agnostic function for ODP threads

[Christophe Milard]

odp_l2fwd is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_l2fwd is hence no longer aware on how the odpthread is implemented.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_l2fwd.c | 25 +
 1 file changed, 13 insertions(+), 12 deletions(-)

diff --git a/test/performance/odp_l2fwd.c b/test/performance/odp_l2fwd.c
index 8b83475..9a8d578 100644
--- a/test/performance/odp_l2fwd.c
+++ b/test/performance/odp_l2fwd.c
@@ -296,7 +296,7 @@ static inline int event_queue_send(odp_queue_t queue, 
odp_packet_t *pkt_tbl,
  *
  * @param arg  thread arguments of type 'thread_args_t *'
  */
-static void *run_worker_sched_mode(void *arg)
+static int run_worker_sched_mode(void *arg)
 {
odp_event_t  ev_tbl[MAX_PKT_BURST];
odp_packet_t pkt_tbl[MAX_PKT_BURST];
@@ -398,7 +398,7 @@ static void *run_worker_sched_mode(void *arg)
/* Make sure that latest stat writes are visible to other threads */
odp_mb_full();
 
-   return NULL;
+   return 0;
 }
 
 /**
@@ -406,7 +406,7 @@ static void *run_worker_sched_mode(void *arg)
  *
  * @param arg  thread arguments of type 'thread_args_t *'
  */
-static void *run_worker_plain_queue_mode(void *arg)
+static int run_worker_plain_queue_mode(void *arg)
 {
int thr;
int pkts;
@@ -497,7 +497,7 @@ static void *run_worker_plain_queue_mode(void *arg)
/* Make sure that latest stat writes are visible to other threads */
odp_mb_full();
 
-   return NULL;
+   return 0;
 }
 
 /**
@@ -505,7 +505,7 @@ static void *run_worker_plain_queue_mode(void *arg)
  *
  * @param arg  thread arguments of type 'thread_args_t *'
  */
-static void *run_worker_direct_mode(void *arg)
+static int run_worker_direct_mode(void *arg)
 {
int thr;
int pkts;
@@ -591,7 +591,7 @@ static void *run_worker_direct_mode(void *arg)
/* Make sure that latest stat writes are visible to other threads */
odp_mb_full();
 
-   return NULL;
+   return 0;
 }
 
 /**
@@ -1287,7 +1287,7 @@ static void gbl_args_init(args_t *args)
  */
 int main(int argc, char *argv[])
 {
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
odp_pool_t pool;
int i;
int cpu;
@@ -1300,7 +1300,7 @@ int main(int argc, char *argv[])
int ret;
stats_t *stats;
int if_count;
-   void *(*thr_run_func)(void *);
+   int (*thr_run_func)(void *);
odp_instance_t instance;
 
/* Init ODP before calling anything else */
@@ -1435,7 +1435,7 @@ int main(int argc, char *argv[])
cpu = odp_cpumask_first(&cpumask);
for (i = 0; i < num_workers; ++i) {
odp_cpumask_t thd_mask;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
memset(&thr_params, 0, sizeof(thr_params));
thr_params.start= thr_run_func;
@@ -1447,8 +1447,8 @@ int main(int argc, char *argv[])
 
odp_cpumask_zero(&thd_mask);
odp_cpumask_set(&thd_mask, cpu);
-   odph_linux_pthread_create(&thread_tbl[i], &thd_mask,
- &thr_params);
+   odph_odpthreads_create(&thread_tbl[i], &thd_mask,
+  &thr_params);
cpu = odp_cpumask_next(&cpumask, cpu);
}
 
@@ -1470,7 +1470,8 @@ int main(int argc, char *argv[])
exit_threads = 1;
 
/* Master thread waits for other threads to exit */
-   odph_linux_pthread_join(thread_tbl, num_workers);
+   for (i = 0; i < num_workers; ++i)
+   odph_odpthreads_join(&thread_tbl[i]);
 
free(gbl_args->appl.if_names);
free(gbl_args->appl.if_str);
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 16/35] performance: crypto: adding helper cmd line parsing

[Christophe Milard]

odp_crypto now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_crypto.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/test/performance/odp_crypto.c b/test/performance/odp_crypto.c
index 893a5a3..404984d 100644
--- a/test/performance/odp_crypto.c
+++ b/test/performance/odp_crypto.c
@@ -828,7 +828,7 @@ static void parse_args(int argc, char *argv[], 
crypto_args_t *cargs)
 {
int opt;
int long_index;
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"algorithm", optional_argument, NULL, 'a'},
{"debug",  no_argument, NULL, 'd'},
{"flight", optional_argument, NULL, 'f'},
@@ -843,6 +843,11 @@ static void parse_args(int argc, char *argv[], 
crypto_args_t *cargs)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+a:c:df:hi:m:nl:spr";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
cargs->in_place = 0;
cargs->in_flight = 1;
cargs->debug_packets = 0;
@@ -852,9 +857,10 @@ static void parse_args(int argc, char *argv[], 
crypto_args_t *cargs)
cargs->reuse_packet = 0;
cargs->schedule = 0;
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (1) {
-   opt = getopt_long(argc, argv, "+a:c:df:hi:m:nl:spr",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
 
if (opt == -1)
break;  /* No more options */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 15/35] performance: crypto: using agnostic function for ODP threads

[Christophe Milard]

odp_crypto is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_crypto is hence no longer aware on how the odpthread is implemented.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_crypto.c | 12 ++--
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/test/performance/odp_crypto.c b/test/performance/odp_crypto.c
index 595a266..893a5a3 100644
--- a/test/performance/odp_crypto.c
+++ b/test/performance/odp_crypto.c
@@ -704,13 +704,13 @@ typedef struct thr_arg {
crypto_alg_config_t *crypto_alg_config;
 } thr_arg_t;
 
-static void *run_thr_func(void *arg)
+static int run_thr_func(void *arg)
 {
thr_arg_t *thr_args = (thr_arg_t *)arg;
 
run_measure_one_config(&thr_args->crypto_args,
   thr_args->crypto_alg_config);
-   return NULL;
+   return 0;
 }
 
 int main(int argc, char *argv[])
@@ -724,7 +724,7 @@ int main(int argc, char *argv[])
odp_cpumask_t cpumask;
char cpumaskstr[ODP_CPUMASK_STR_SIZE];
int num_workers = 1;
-   odph_linux_pthread_t thr[num_workers];
+   odph_odpthread_t thr[num_workers];
odp_instance_t instance;
 
memset(&cargs, 0, sizeof(cargs));
@@ -797,7 +797,7 @@ int main(int argc, char *argv[])
memset(thr, 0, sizeof(thr));
 
if (cargs.alg_config) {
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
memset(&thr_params, 0, sizeof(thr_params));
thr_params.start= run_thr_func;
@@ -806,8 +806,8 @@ int main(int argc, char *argv[])
thr_params.instance = instance;
 
if (cargs.schedule) {
-   odph_linux_pthread_create(&thr[0], &cpumask, 
&thr_params);
-   odph_linux_pthread_join(&thr[0], num_workers);
+   odph_odpthreads_create(&thr[0], &cpumask, &thr_params);
+   odph_odpthreads_join(&thr[0]);
} else {
run_measure_one_config(&cargs, cargs.alg_config);
}
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 19/35] example: generator: using agnostic function for ODP threads

[Christophe Milard]

odp_generator is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_generator is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/generator/odp_generator.c | 33 -
 1 file changed, 16 insertions(+), 17 deletions(-)

diff --git a/example/generator/odp_generator.c 
b/example/generator/odp_generator.c
index fd1141a..dd3e023 100644
--- a/example/generator/odp_generator.c
+++ b/example/generator/odp_generator.c
@@ -352,7 +352,7 @@ static odp_pktio_t create_pktio(const char *dev, odp_pool_t 
pool)
  * @param arg  thread arguments of type 'thread_args_t *'
  */
 
-static void *gen_send_thread(void *arg)
+static int gen_send_thread(void *arg)
 {
int thr;
int ret;
@@ -368,12 +368,12 @@ static void *gen_send_thread(void *arg)
if (pktio == ODP_PKTIO_INVALID) {
EXAMPLE_ERR("  [%02i] Error: lookup of pktio %s failed\n",
thr, thr_args->pktio_dev);
-   return NULL;
+   return -1;
}
 
if (odp_pktout_queue(pktio, &pktout, 1) != 1) {
EXAMPLE_ERR("  [%02i] Error: no output queue\n", thr);
-   return NULL;
+   return -1;
}
 
printf("  [%02i] created mode: SEND\n", thr);
@@ -441,7 +441,7 @@ static void *gen_send_thread(void *arg)
}
}
 
-   return arg;
+   return 0;
 }
 
 /**
@@ -515,7 +515,7 @@ static void print_pkts(int thr, odp_packet_t pkt_tbl[], 
unsigned len)
  *
  * @param arg  thread arguments of type 'thread_args_t *'
  */
-static void *gen_recv_thread(void *arg)
+static int gen_recv_thread(void *arg)
 {
int thr;
odp_pktio_t pktio;
@@ -530,7 +530,7 @@ static void *gen_recv_thread(void *arg)
if (pktio == ODP_PKTIO_INVALID) {
EXAMPLE_ERR("  [%02i] Error: lookup of pktio %s failed\n",
thr, thr_args->pktio_dev);
-   return NULL;
+   return -1;
}
 
printf("  [%02i] created mode: RECEIVE\n", thr);
@@ -558,7 +558,7 @@ static void *gen_recv_thread(void *arg)
odp_packet_free(pkt);
}
 
-   return arg;
+   return 0;
 }
 
 /**
@@ -621,7 +621,7 @@ static void print_global_stats(int num_workers)
  */
 int main(int argc, char *argv[])
 {
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
odp_pool_t pool;
int num_workers;
int i;
@@ -636,7 +636,7 @@ int main(int argc, char *argv[])
odp_event_t ev;
odp_pktio_t *pktio;
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
/* Init ODP before calling anything else */
if (odp_init_global(&instance, NULL, NULL)) {
@@ -786,8 +786,7 @@ int main(int argc, char *argv[])
thr_params.thr_type = ODP_THREAD_WORKER;
thr_params.instance = instance;
 
-   odph_linux_pthread_create(&thread_tbl[1], &cpu_mask,
- &thr_params);
+   odph_odpthreads_create(&thread_tbl[1], &cpu_mask, &thr_params);
 
tq = odp_queue_create("", NULL);
if (tq == ODP_QUEUE_INVALID)
@@ -810,14 +809,13 @@ int main(int argc, char *argv[])
thr_params.start = gen_send_thread;
thr_params.arg   = &args->thread[0];
 
-   odph_linux_pthread_create(&thread_tbl[0], &cpu_mask,
- &thr_params);
+   odph_odpthreads_create(&thread_tbl[0], &cpu_mask, &thr_params);
 
} else {
int cpu = odp_cpumask_first(&cpumask);
for (i = 0; i < num_workers; ++i) {
odp_cpumask_t thd_mask;
-   void *(*thr_run_func) (void *);
+   int (*thr_run_func)(void *);
int if_idx;
 
if_idx = i % args->appl.if_count;
@@ -856,8 +854,8 @@ int main(int argc, char *argv[])
thr_params.start = thr_run_func;
thr_params.arg   = &args->thread[i];
 
-   odph_linux_pthread_create(&thread_tbl[i],
- &thd_mask, &thr_params);
+   odph_odpthreads_create(&thread_tbl[i],
+  &thd_mask, &thr_params);
cpu = odp_cpumask_next(&cpumask, cpu);
 
}
@@ -866,7 +864,8 @@ int main(int argc, char *argv[])
print_global_stats(num_workers);
 
/* Master thread waits for other th

[lng-odp] [PATCHv7 21/35] example: ipsec: using agnostic function for ODP threads

[Christophe Milard]

odp_ipsec is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_ipsec is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/ipsec/odp_ipsec.c | 15 +++
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/example/ipsec/odp_ipsec.c b/example/ipsec/odp_ipsec.c
index e75db61..4d08772 100644
--- a/example/ipsec/odp_ipsec.c
+++ b/example/ipsec/odp_ipsec.c
@@ -1047,12 +1047,12 @@ pkt_disposition_e do_ipsec_out_finish(odp_packet_t pkt,
  *  - Sequence number assignment queue
  *  - Per packet crypto API completion queue
  *
- * @param arg  Required by "odph_linux_pthread_create", unused
+ * @param arg  Required by "odph_odpthreads_create", unused
  *
  * @return NULL (should never return)
  */
 static
-void *pktio_thread(void *arg EXAMPLE_UNUSED)
+int pktio_thread(void *arg EXAMPLE_UNUSED)
 {
int thr;
odp_packet_t pkt;
@@ -1203,7 +1203,7 @@ void *pktio_thread(void *arg EXAMPLE_UNUSED)
}
 
/* unreachable */
-   return NULL;
+   return 0;
 }
 
 /**
@@ -1212,7 +1212,7 @@ void *pktio_thread(void *arg EXAMPLE_UNUSED)
 int
 main(int argc, char *argv[])
 {
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
int num_workers;
int i;
int stream_count;
@@ -1221,7 +1221,7 @@ main(int argc, char *argv[])
char cpumaskstr[ODP_CPUMASK_STR_SIZE];
odp_pool_param_t params;
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
/* create by default scheduled queues */
queue_create = odp_queue_create;
@@ -1341,8 +1341,7 @@ main(int argc, char *argv[])
thr_params.arg  = NULL;
thr_params.thr_type = ODP_THREAD_WORKER;
thr_params.instance = instance;
-
-   odph_linux_pthread_create(thread_tbl, &cpumask, &thr_params);
+   odph_odpthreads_create(thread_tbl, &cpumask, &thr_params);
 
/*
 * If there are streams attempt to verify them else
@@ -1356,7 +1355,7 @@ main(int argc, char *argv[])
} while (!done);
printf("All received\n");
} else {
-   odph_linux_pthread_join(thread_tbl, num_workers);
+   odph_odpthreads_join(thread_tbl);
}
 
free(args->appl.if_names);
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 18/35] example: classifier: adding helper cmd line parsing

[Christophe Milard]

odp_classifier now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/classifier/odp_classifier.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/example/classifier/odp_classifier.c 
b/example/classifier/odp_classifier.c
index 5982446..20e64ec 100644
--- a/example/classifier/odp_classifier.c
+++ b/example/classifier/odp_classifier.c
@@ -792,7 +792,7 @@ static void parse_args(int argc, char *argv[], appl_args_t 
*appl_args)
int interface = 0;
int policy = 0;
 
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
{"interface", required_argument, NULL, 'i'},/* return 'i' */
{"policy", required_argument, NULL, 'p'},   /* return 'p' */
@@ -802,10 +802,16 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+c:t:i:p:m:t:h";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
+   opterr = 0; /* do not issue errors on helper options */
 
while (1) {
-   opt = getopt_long(argc, argv, "+c:t:i:p:m:t:h",
-   longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts,
+ longopts, &long_index);
 
if (opt == -1)
break;  /* No more options */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 17/35] example: classifier: using agnostic function for ODP threads

[Christophe Milard]

odp_classifier is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
odp_classifier is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/classifier/odp_classifier.c | 27 +++
 1 file changed, 7 insertions(+), 20 deletions(-)

diff --git a/example/classifier/odp_classifier.c 
b/example/classifier/odp_classifier.c
index 071788d..5982446 100644
--- a/example/classifier/odp_classifier.c
+++ b/example/classifier/odp_classifier.c
@@ -254,7 +254,7 @@ static odp_pktio_t create_pktio(const char *dev, odp_pool_t 
pool)
  * Worker threads to receive the packet
  *
  */
-static void *pktio_receive_thread(void *arg)
+static int pktio_receive_thread(void *arg)
 {
int thr;
odp_pktout_queue_t pktout;
@@ -298,7 +298,7 @@ static void *pktio_receive_thread(void *arg)
 
if (odp_pktout_queue(pktio_tmp, &pktout, 1) != 1) {
EXAMPLE_ERR("  [%02i] Error: no output queue\n", thr);
-   return NULL;
+   return -1;
}
 
pool = odp_packet_pool(pkt);
@@ -324,7 +324,7 @@ static void *pktio_receive_thread(void *arg)
}
}
 
-   return NULL;
+   return 0;
 }
 
 static odp_cos_t configure_default_cos(odp_pktio_t pktio, appl_args_t *args)
@@ -469,11 +469,10 @@ static void configure_cos(odp_cos_t default_cos, 
appl_args_t *args)
  */
 int main(int argc, char *argv[])
 {
-   odph_linux_pthread_t thread_tbl[MAX_WORKERS];
+   odph_odpthread_t thread_tbl[MAX_WORKERS];
odp_pool_t pool;
int num_workers;
int i;
-   int cpu;
odp_cpumask_t cpumask;
char cpumaskstr[ODP_CPUMASK_STR_SIZE];
odp_pool_param_t params;
@@ -483,7 +482,7 @@ int main(int argc, char *argv[])
odp_shm_t shm;
int ret;
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
/* Init ODP before calling anything else */
if (odp_init_global(&instance, NULL, NULL)) {
@@ -571,25 +570,13 @@ int main(int argc, char *argv[])
thr_params.arg  = args;
thr_params.thr_type = ODP_THREAD_WORKER;
thr_params.instance = instance;
-
-   cpu = odp_cpumask_first(&cpumask);
-   for (i = 0; i < num_workers; ++i) {
-   odp_cpumask_t thd_mask;
-   /*
-* Calls odp_thread_create(cpu) for each thread
-*/
-   odp_cpumask_zero(&thd_mask);
-   odp_cpumask_set(&thd_mask, cpu);
-   odph_linux_pthread_create(&thread_tbl[i], &thd_mask,
- &thr_params);
-   cpu = odp_cpumask_next(&cpumask, cpu);
-   }
+   odph_odpthreads_create(thread_tbl, &cpumask, &thr_params);
 
print_cls_statistics(args);
 
odp_pktio_stop(pktio);
shutdown = 1;
-   odph_linux_pthread_join(thread_tbl, num_workers);
+   odph_odpthreads_join(thread_tbl);
 
for (i = 0; i < args->policy_count; i++) {
if ((i !=  args->policy_count - 1) &&
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 22/35] example: ipsec: adding helper cmd line parsing

[Christophe Milard]

odp_ipsec now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/ipsec/odp_ipsec.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/example/ipsec/odp_ipsec.c b/example/ipsec/odp_ipsec.c
index 4d08772..fb4385f 100644
--- a/example/ipsec/odp_ipsec.c
+++ b/example/ipsec/odp_ipsec.c
@@ -1381,7 +1381,7 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
int rc = 0;
int i;
 
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
{"interface", required_argument, NULL, 'i'},/* return 'i' */
{"mode", required_argument, NULL, 'm'}, /* return 'm' */
@@ -1395,13 +1395,19 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+c:i:m:h:r:p:a:e:t:s:";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
printf("\nParsing command line options\n");
 
appl_args->mode = 0;  /* turn off async crypto API by default */
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (!rc) {
-   opt = getopt_long(argc, argv, "+c:i:m:h:r:p:a:e:t:s:",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
 
if (-1 == opt)
break;  /* No more options */
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 10/35] performance: odp_scheduling: proc mode done by helper

[Christophe Milard]

Remove the --proc option from odp_scheduling.c and use the helper
functions (odph_odpthreads_create, odph_odpthreads_join)
to handle odp threads. Let helper parse its command line args, hence
recognising the --odph_proc option doing what --proc did.

Signed-off-by: Christophe Milard 
---
 test/performance/odp_scheduling.c | 91 +--
 1 file changed, 30 insertions(+), 61 deletions(-)

diff --git a/test/performance/odp_scheduling.c 
b/test/performance/odp_scheduling.c
index c7a3049..1d3bfd1 100644
--- a/test/performance/odp_scheduling.c
+++ b/test/performance/odp_scheduling.c
@@ -51,7 +51,6 @@ typedef struct {
 /** Test arguments */
 typedef struct {
int cpu_count;  /**< CPU count */
-   int proc_mode;  /**< Process mode */
int fairness;   /**< Check fairness */
 } test_args_t;
 
@@ -562,9 +561,9 @@ static int test_schedule_multi(const char *str, int thr,
  *
  * @param arg  Arguments
  *
- * @return NULL on failure
+ * @return non zero on failure
  */
-static void *run_thread(void *arg)
+static int run_thread(void *arg ODP_UNUSED)
 {
int thr;
odp_shm_t shm;
@@ -580,7 +579,7 @@ static void *run_thread(void *arg)
 
if (globals == NULL) {
LOG_ERR("Shared mem lookup failed\n");
-   return NULL;
+   return -1;
}
 
barrier = &globals->barrier;
@@ -605,17 +604,17 @@ static void *run_thread(void *arg)
odp_barrier_wait(barrier);
 
if (test_alloc_single(thr, globals))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_alloc_multi(thr, globals))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_plain_queue(thr, globals))
-   return NULL;
+   return -1;
 
/* Low prio */
 
@@ -623,19 +622,19 @@ static void *run_thread(void *arg)
 
if (test_schedule_single("sched_s_lo", thr,
 ODP_SCHED_PRIO_LOWEST, globals))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_schedule_many("sched_m_lo", thr,
   ODP_SCHED_PRIO_LOWEST, globals))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_schedule_multi("sched_multi_lo", thr,
ODP_SCHED_PRIO_LOWEST, globals))
-   return NULL;
+   return -1;
 
/* High prio */
 
@@ -643,24 +642,24 @@ static void *run_thread(void *arg)
 
if (test_schedule_single("sched_s_hi", thr,
 ODP_SCHED_PRIO_HIGHEST, globals))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_schedule_many("sched_m_hi", thr,
   ODP_SCHED_PRIO_HIGHEST, globals))
-   return NULL;
+   return -1;
 
odp_barrier_wait(barrier);
 
if (test_schedule_multi("sched_multi_hi", thr,
ODP_SCHED_PRIO_HIGHEST, globals))
-   return NULL;
+   return -1;
 
 
printf("Thread %i exits\n", thr);
fflush(NULL);
-   return arg;
+   return 0;
 }
 
 /**
@@ -715,7 +714,6 @@ static void print_usage(void)
printf("Options:\n");
printf("  -c, --count CPU count\n");
printf("  -h, --help  this help\n");
-   printf("  -p, --proc  process mode\n");
printf("  -f, --fair  collect fairness statistics\n");
printf("\n\n");
 }
@@ -732,26 +730,26 @@ static void parse_args(int argc, char *argv[], 
test_args_t *args)
int opt;
int long_index;
 
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"count", required_argument, NULL, 'c'},
-   {"proc", no_argument, NULL, 'p'},
{"fair", no_argument, NULL, 'f'},
{"help", no_argument, NULL, 'h'},
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+c:fh";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
+   opterr = 0; /* do not issue errors on helper options */
while (1) {
-   opt = getopt_long(argc, argv, "+c:pfh",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
 
if (opt == -1)
break;  /* No

[lng-odp] [PATCHv7 20/35] example: generator: adding helper cmd line parsing

[Christophe Milard]

odp_generator now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/generator/odp_generator.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/example/generator/odp_generator.c 
b/example/generator/odp_generator.c
index dd3e023..ccae907 100644
--- a/example/generator/odp_generator.c
+++ b/example/generator/odp_generator.c
@@ -919,7 +919,7 @@ static void parse_args(int argc, char *argv[], appl_args_t 
*appl_args)
size_t len;
odp_cpumask_t cpumask, cpumask_args, cpumask_and;
int i, num_workers;
-   static struct option longopts[] = {
+   static const struct option longopts[] = {
{"interface", required_argument, NULL, 'I'},
{"workers", required_argument, NULL, 'w'},
{"cpumask", required_argument, NULL, 'c'},
@@ -936,14 +936,20 @@ static void parse_args(int argc, char *argv[], 
appl_args_t *appl_args)
{NULL, 0, NULL, 0}
};
 
+   static const char *shortopts = "+I:a:b:s:d:p:i:m:n:t:w:c:h";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
appl_args->mode = -1; /* Invalid, must be changed by parsing */
appl_args->number = -1;
appl_args->payload = 56;
appl_args->timeout = -1;
 
+   opterr = 0; /* do not issue errors on helper options */
+
while (1) {
-   opt = getopt_long(argc, argv, "+I:a:b:s:d:p:i:m:n:t:w:c:h",
- longopts, &long_index);
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
if (opt == -1)
break;  /* No more options */
 
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 23/35] example: l2fwd_simple: using agnostic function for ODP threads

[Christophe Milard]

l2fwd_simple is changed to use the implementation agnostic ODP thread
create and join functions, from helpers.
l2fwd_simple is hence no longer aware on how the odpthread is
implemented.

Signed-off-by: Christophe Milard 
---
 example/l2fwd_simple/odp_l2fwd_simple.c | 15 ---
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/example/l2fwd_simple/odp_l2fwd_simple.c 
b/example/l2fwd_simple/odp_l2fwd_simple.c
index fb096ab..c075744 100644
--- a/example/l2fwd_simple/odp_l2fwd_simple.c
+++ b/example/l2fwd_simple/odp_l2fwd_simple.c
@@ -68,7 +68,7 @@ static odp_pktio_t create_pktio(const char *name, odp_pool_t 
pool,
return pktio;
 }
 
-static void *run_worker(void *arg ODP_UNUSED)
+static int run_worker(void *arg ODP_UNUSED)
 {
odp_packet_t pkt_tbl[MAX_PKT_BURST];
int pkts, sent, tx_drops, i;
@@ -97,7 +97,7 @@ static void *run_worker(void *arg ODP_UNUSED)
 
if (odp_unlikely(!odp_packet_has_eth(pkt))) {
printf("warning: packet has no eth header\n");
-   return NULL;
+   return 0;
}
eth = (odph_ethhdr_t *)odp_packet_l2_ptr(pkt, NULL);
eth->src = global.src;
@@ -110,7 +110,7 @@ static void *run_worker(void *arg ODP_UNUSED)
if (odp_unlikely(tx_drops))
odp_packet_free_multi(&pkt_tbl[sent], tx_drops);
}
-   return NULL;
+   return 0;
 }
 
 int main(int argc, char **argv)
@@ -118,9 +118,9 @@ int main(int argc, char **argv)
odp_pool_t pool;
odp_pool_param_t params;
odp_cpumask_t cpumask;
-   odph_linux_pthread_t thd;
+   odph_odpthread_t thd;
odp_instance_t instance;
-   odph_linux_thr_params_t thr_params;
+   odph_odpthread_params_t thr_params;
 
if (argc != 5 ||
odph_eth_addr_parse(&global.dst, argv[3]) != 0 ||
@@ -169,7 +169,8 @@ int main(int argc, char **argv)
thr_params.thr_type = ODP_THREAD_WORKER;
thr_params.instance = instance;
 
-   odph_linux_pthread_create(&thd, &cpumask, &thr_params);
-   odph_linux_pthread_join(&thd, 1);
+   odph_odpthreads_create(&thd, &cpumask, &thr_params);
+   odph_odpthreads_join(&thd);
+
return 0;
 }
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] [PATCHv7 24/35] example: l2fwd_simple: adding helper cmd line parsing

[Christophe Milard]

l2fwd_simple now calls the helper command line parsing so that helper
can collect its options. Hence enabling process mode run.

Signed-off-by: Christophe Milard 
---
 example/l2fwd_simple/odp_l2fwd_simple.c | 32 +++-
 1 file changed, 27 insertions(+), 5 deletions(-)

diff --git a/example/l2fwd_simple/odp_l2fwd_simple.c 
b/example/l2fwd_simple/odp_l2fwd_simple.c
index c075744..daae038 100644
--- a/example/l2fwd_simple/odp_l2fwd_simple.c
+++ b/example/l2fwd_simple/odp_l2fwd_simple.c
@@ -6,6 +6,7 @@
 
 #include 
 #include 
+#include 
 
 #include 
 #include 
@@ -121,10 +122,29 @@ int main(int argc, char **argv)
odph_odpthread_t thd;
odp_instance_t instance;
odph_odpthread_params_t thr_params;
+   int opt;
+   int long_index;
+
+   static const struct option longopts[] = { {NULL, 0, NULL, 0} };
+   static const char *shortopts = "";
+
+   /* let helper collect its own arguments (e.g. --odph_proc) */
+   odph_parse_options(argc, argv, shortopts, longopts);
+
+   /*
+* parse own options: currentely none, but this will move optind
+* to the first non-option argument. (in case there where helprt args)
+*/
+   opterr = 0; /* do not issue errors on helper options */
+   while (1) {
+   opt = getopt_long(argc, argv, shortopts, longopts, &long_index);
+   if (-1 == opt)
+   break;  /* No more options */
+   }
 
-   if (argc != 5 ||
-   odph_eth_addr_parse(&global.dst, argv[3]) != 0 ||
-   odph_eth_addr_parse(&global.src, argv[4]) != 0) {
+   if (argc != optind + 4 ||
+   odph_eth_addr_parse(&global.dst, argv[optind + 2]) != 0 ||
+   odph_eth_addr_parse(&global.src, argv[optind + 3]) != 0) {
printf("Usage: odp_l2fwd_simple eth0 eth1 01:02:03:04:05:06"
   " 07:08:09:0a:0b:0c\n");
printf("Where eth0 and eth1 are the used interfaces"
@@ -158,8 +178,10 @@ int main(int argc, char **argv)
exit(1);
}
 
-   global.if0 = create_pktio(argv[1], pool, &global.if0in, &global.if0out);
-   global.if1 = create_pktio(argv[2], pool, &global.if1in, &global.if1out);
+   global.if0 = create_pktio(argv[optind], pool, &global.if0in,
+   &global.if0out);
+   global.if1 = create_pktio(argv[optind + 1], pool, &global.if1in,
+   &global.if1out);
 
odp_cpumask_default_worker(&cpumask, 1);
 
-- 
2.5.0

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp