Re: [lng-odp] [PATCH] scripts/git_hash: match more digits

[Ricardo Salveti]

On Mon, May 2, 2016 at 6:07 PM, Anders Roxell  wrote:
> Make it possible to match more digits than groups of one.
>
> Signed-off-by: Anders Roxell 
> ---
>  scripts/git_hash.sh | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/git_hash.sh b/scripts/git_hash.sh
> index 6ba265e..336eb01 100755
> --- a/scripts/git_hash.sh
> +++ b/scripts/git_hash.sh
> @@ -7,7 +7,8 @@ fi
>  ROOTDIR=${1}
>
>  if [ -d ${ROOTDIR}/.git ]; then
> -   hash=$(git --git-dir=${ROOTDIR}/.git describe  --match 'v?\.?\.?\.?' 
> | tr -d "\n")
> +   hash=$(git --git-dir=${ROOTDIR}/.git describe --match 
> 'v[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*'\
> +  | tr -d "\n")
> if [[ $(git --git-dir=${ROOTDIR}/.git diff --shortstat 2> /dev/null \
> | tail -n1) != "" ]]; then
> dirty=.dirty
> --
> 2.1.4

Reviewed-by: Ricardo Salveti 

-- 
Ricardo Salveti
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] RFC: inter-process dynamic shared memory support.

[Brian Brooks]

Perhaps a fixed address somewhere below the fixed address used by the
kernel when "the system chooses an address" can help solve this problem.


On Fri, Apr 29, 2016 at 1:32 AM, Christophe Milard <
christophe.mil...@linaro.org> wrote:

> Thanks, Barry,
>
> I definitively think that is a good candidate for the arch call.
> Even if I am not 100% sure I fully understand the whole picture, I this
> text raise a few questions:
>
> 1) is this plain user space code, or are there related kernel modules
> involved? I am guessing user space, but please confirm.
>
> 2) Do the arenas have to be known at init time? (you wrote "Typically an
> initial process will call tmc_shmem_create() to create the file using a
> fixed, known file name. Other processes then call tmc_shmem_open() to gain
> access to the arena.")
>
> 3) Are there any constraint on the processes involved (such as processes
> should be descendent of a common ancestor?)
>
> 4)Are processes that attempt to map the arena guaranteed a get the same
> address AND guaranteed that the mapping will succeed? (I get worried when
> reading "If the user specifies ADDR as zero, the system will choose an
> address, and all subsequent users of the arena will automatically* try to
> load* it at that address": This seems to imply that the mapping will
> succees only if the virtual address in the process performing the mapping
> happens to be free...)
> This is exactly the problem I am trying to tackle by reserving the virtual
> area address space from the beginning.
>
> Thanks anyway Barry! Even if there are still questions, I do appreciate
> your input! I guess we cann take more discussion on the ARCH call!
>
> Christophe
>
> On 28 April 2016 at 17:50, Barry Spinney  wrote:
>
>>
>> One of our Linux developers, Chris Metcalf, several years ago wrote a
>> library module for the TileGx chip
>> called tmc/shmem.[hc].  While targeted for the TileGx, much of the
>> concepts and code are applicable
>> to most Linux based platforms.  Instead of sending the entire module
>> header and source code (for
>> which I probably need some managerial approval), I have instead excerpted
>> some of the main
>> concepts and API's below.
>>
>> This module does have the property that "reserving" shared virtual
>> address space has no cost -
>> i.e. no Linux virtual memory page tables will be added or change, nor
>> will there be any cost/effect
>> on the file system.  However once reserved virtual addresses are made
>> "active" (e.g. via a call to
>> tmc_shmem_alloc) then of course page tables can get added, physical
>> memory could get used,
>> file system use of the associated backing file can start occurring.
>>
>> IF this approach is chosen to be used by ODP (or is at least a strong
>> contender, then I can
>> (after getting the likely manager approval) send in a proper ODP header
>> file and maybe an strawman
>> implementation as a RFC proposal.
>>
>>
>>
>> 
>>
>> Inter-process dynamic shared memory support.
>>
>> This API provides a convenient method for multiple processes to share
>> memory using a persistent filesystem-based arena that is automatically
>> mapped at the same, fixed address in all processes sharing the arena. The
>> application chooses an address for the arena to be located at and a maximum
>> size to which it can grow, and the system manages coordinating access to
>> memory mapped from the file among the processes. Since the address is
>> fixed, absolute pointer values, etc., may be safely stored into the arena.
>>
>> As is always true when you use shared memory, you should employ
>> appropriate memory fencing to ensure that any modifications are actually
>> fully visible before they are used by any other process.
>>
>> Typically an initial process will call tmc_shmem_create() to create the
>> file using a fixed, known file name. Other processes then call
>> tmc_shmem_open() to gain access to the arena. The creator should first
>> initialize a tmc_alloc_t object to indicate any special attributes of the
>> desired memory, such as huge page size, variant cache attributes, etc. If
>> huge pages are requested, the tmc_shmem code will automatically open an
>> additional file in the appropriate hugetlb file system. The files are
>> opened such that they are automatically closed if the process calls exec()
>> to start a new executable.
>>
>> The APIs create a file with permission 0600 (owner-only read/write), but
>> the application may invoke fchmod() or fchown() on the underlying file
>> descriptors if desired to reset the ownership and permissions.
>>
>> If the application wishes to create a temporary file name to hold the
>> arena, it can use tmc_shmem_create_temp() and pass in a template filename,
>> just as is done for mkstemp(). In this case it is typically then necessary
>> to communicate the chosen filename out-of-band to the other processes that
>> wish to share 

[lng-odp] [PATCH] scripts/git_hash: match more digits

[Anders Roxell]

Make it possible to match more digits than groups of one.

Signed-off-by: Anders Roxell 
---
 scripts/git_hash.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/git_hash.sh b/scripts/git_hash.sh
index 6ba265e..336eb01 100755
--- a/scripts/git_hash.sh
+++ b/scripts/git_hash.sh
@@ -7,7 +7,8 @@ fi
 ROOTDIR=${1}
 
 if [ -d ${ROOTDIR}/.git ]; then
-   hash=$(git --git-dir=${ROOTDIR}/.git describe  --match 'v?\.?\.?\.?' | 
tr -d "\n")
+   hash=$(git --git-dir=${ROOTDIR}/.git describe --match 
'v[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*'\
+  | tr -d "\n")
if [[ $(git --git-dir=${ROOTDIR}/.git diff --shortstat 2> /dev/null \
| tail -n1) != "" ]]; then
dirty=.dirty
-- 
2.1.4

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [GIT PULL ODPv4] running things in process mode

[Brian Brooks]

On 05/02 11:18:19, Christophe Milard wrote:
> Hi Brian
> 
> I have commented below, but it looks we need to talk.  would you have time
> after the linaro sync call today, for a HO?
> 

Hi Christophe,

Remaining comments from 'helper_abstraction_odp_thread' following our
discussion.

Brian

> diff --git a/helper/linux.c b/helper/linux.c
> index 24e243b..2714e51 100644
> --- a/helper/linux.c
> +++ b/helper/linux.c
> +/*
> + * wrapper for odpthreads, either implemented as linux threads or processes.
> + * (in process mode, if start_routine returns NULL, the process return 1.
> + */
> +static void *odpthread_run_start_routine(void *arg)
>  {
> - odph_linux_thr_params_t *thr_params = arg;
> + int status;
> + int ret;
> + odph_odpthread_params_t *thr_params;
> +
> + _odph_odpthread_start_args_t *start_args = arg;
> +
> + thr_params = _args->thr_params;
>  
>   /* ODP thread local init */
>   if (odp_init_local(thr_params->instance, thr_params->thr_type)) {
>   ODPH_ERR("Local init failed\n");
> + if (start_args->linuxtype == PROCESS)
> + exit(-1);

We discussed -1 return value. However, _exit() vs exit() is relevant but beyond
the scope of this patch series.

>   return NULL;
>   }
>  
> - void *ret_ptr = thr_params->start(thr_params->arg);
> - int ret = odp_term_local();
> + ODPH_DBG("helper: ODP %s thread started as linux %s. (pid=%d)\n",
> +  thr_params->thr_type == ODP_THREAD_WORKER ?
> +  "worker" : "control",
> +  (start_args->linuxtype == PTHREAD) ?
> +  "pthread" : "process",
> +  (int)getpid());

for extra sanity, perhaps:

  (start_args->linuxtype == PTHREAD) ? (int)pthread_self() : (int)getpid()

A potential RFC to logging facilities (beyond scope of this series) would
include pid and thread_id in log format alongside timestamp.

> + status = thr_params->start(thr_params->arg);
> + ret = odp_term_local();
>  
>   if (ret < 0)
>   ODPH_ERR("Local term failed\n");
>   else if (ret == 0 && odp_term_global(thr_params->instance))
>   ODPH_ERR("Global term failed\n");
>  
> - return ret_ptr;
> + /* for process implementation of odp threads, just return status... */
> + if (start_args->linuxtype == PROCESS)
> + exit(status);
> +
> + /* threads implementation return void* pointers: cast status to that. */
> + return (void *)(long)status;

Seems the cast to long is unnecessary?

>  }
>  
> -int odph_linux_pthread_create(odph_linux_pthread_t *pthread_tbl,
> -   const odp_cpumask_t *mask,
> -   const odph_linux_thr_params_t *thr_params)
> +/*
> + * Create a single ODPthread as a linux process
> + */
> +static int odph_linux_process_create(_odph_odpthread_t *thread_tbl,
> +  int cpu,
> +  const odph_odpthread_params_t *thr_params)
>  {
> - int i;
> - int num;
> - int cpu_count;
> - int cpu;
>   int ret;
> + cpu_set_t cpu_set;
> + pid_t pid;
>  
> - num = odp_cpumask_count(mask);
> + CPU_ZERO(_set);
> + CPU_SET(cpu, _set);
>  
> - memset(pthread_tbl, 0, num * sizeof(odph_linux_pthread_t));
> + thread_tbl->start_args.thr_params= *thr_params; /* copy */
> + thread_tbl->start_args.linuxtype = PROCESS;
> + thread_tbl->cpu = cpu;
>  
> - cpu_count = odp_cpu_count();
> + pid = fork();
> + if (pid < 0) {
> + ODPH_ERR("fork() failed\n");
> + thread_tbl->start_args.linuxtype = NOT_STARTED;
> + return -1;
> + }
>  
> - if (num < 1 || num > cpu_count) {
> - ODPH_ERR("Invalid number of threads:%d (%d cores available)\n",
> -  num, cpu_count);
> + /* Parent continues to fork */
> + if (pid > 0) {
> + thread_tbl->proc.pid  = pid;
>   return 0;
>   }
>  
> - cpu = odp_cpumask_first(mask);
> - for (i = 0; i < num; i++) {
> - cpu_set_t cpu_set;
> + /* Child process */
>  
> - CPU_ZERO(_set);
> - CPU_SET(cpu, _set);
> + /* Request SIGTERM if parent dies */
> + prctl(PR_SET_PDEATHSIG, SIGTERM);

OK. Seems only TM example doing basic sig handling.

> + /* Parent died already? */
> + if (getppid() == 1)
> + kill(getpid(), SIGTERM);

OK.

> - pthread_attr_init(_tbl[i].attr);
> + if (sched_setaffinity(0, sizeof(cpu_set_t), _set)) {
> + ODPH_ERR("sched_setaffinity() failed\n");
> + return -2;

OK.

> + }
>  
> - pthread_tbl[i].cpu = cpu;
> + odpthread_run_start_routine(_tbl->start_args);
>  
> - pthread_attr_setaffinity_np(_tbl[i].attr,
> - sizeof(cpu_set_t), _set);
> + return 0; /* never reached */
> +}
>  
> -

Re: [lng-odp] [API-NEXT PATCH 1/2] doc: support ascidoctor

[Bill Fischofer]

For Part 1:

Reviewed-by: Bill Fischofer 

On Wed, Apr 27, 2016 at 4:59 PM, Mike Holmes  wrote:

> ascidoctor is a python asciidoc interpreter it has greater capabilities
> than asciidoc which is a perl based interpreter
>
> The resulting style sheet improvements result in more professional
> looking docs that can be further enhanced with our own css at some
> point.
>
> This also supports including code snippets in the documentation from the
> doxygen specification  files.
>
> Signed-off-by: Mike Holmes 
> ---
>  DEPENDENCIES|  2 +-
>  doc/Makefile.inc|  4 ++--
>  doc/users-guide/users-guide-tm.adoc | 18 +-
>  3 files changed, 12 insertions(+), 12 deletions(-)
>
> diff --git a/DEPENDENCIES b/DEPENDENCIES
> index c83fc36..678b62a 100644
> --- a/DEPENDENCIES
> +++ b/DEPENDENCIES
> @@ -252,4 +252,4 @@ The tested version of doxygen is 1.8.8
>
>  5.2.1 HTML
> # Debian/Ubuntu
> -   $ apt-get install asciidoc source-highlight librsvg2-bin
> +   $ apt-get install asciidoctor source-highlight librsvg2-bin
> diff --git a/doc/Makefile.inc b/doc/Makefile.inc
> index c0b641e..643b1d4 100644
> --- a/doc/Makefile.inc
> +++ b/doc/Makefile.inc
> @@ -7,6 +7,6 @@ VPATH=$(top_builddir)/doc/images
> dot -T svg $^ -o $@
>
>  .adoc.html:
> -   asciidoc $(ASCIIDOC_FLAGS) --out-file=$@ $<
> +   asciidoctor $(ASCIIDOC_FLAGS) --out-file=$@ $<
>
> -ASCIIDOC_FLAGS =-a data-uri -b html5  -a icons -a toc2  -a max-width=55em
> +ASCIIDOC_FLAGS =-a data-uri -b html5 -a icons=font -a toc2
> diff --git a/doc/users-guide/users-guide-tm.adoc
> b/doc/users-guide/users-guide-tm.adoc
> index 132fdc1..5dc2190 100644
> --- a/doc/users-guide/users-guide-tm.adoc
> +++ b/doc/users-guide/users-guide-tm.adoc
> @@ -269,7 +269,7 @@ result in faster operation and/or less memory used.
>  [source,c]
>  
>
> -   odp_tm_params_init(_params); /* <1> */
> +   odp_tm_params_init(_params); // <1>
> tm_params.pktio = egress_pktio;
> tm = odp_tm_create(“Example TM”, _params);
>
> @@ -285,11 +285,11 @@ result in faster operation and/or less memory used.
> tmq_B2 = odp_tm_queue_create(tm, _params);
> tmq_C2 = odp_tm_queue_create(tm, _params);
>
> -   odp_tm_node_params_init(_params); /* <2> */
> +   odp_tm_node_params_init(_params); // <2>
> node_params.level = 1;
> tm_node_1 = odp_tm_node_create(tm, “TmNode1”, _params);
>
> -   odp_tm_queue_connect(tmq_A1, tm_node_1); /* <3> */
> +   odp_tm_queue_connect(tmq_A1, tm_node_1); // <3>
> odp_tm_queue_connect(tmq_B1, tm_node_1);
> odp_tm_queue_connect(tmq_A2, tm_node_1);
> odp_tm_queue_connect(tmq_B2, tm_node_1);
> @@ -302,7 +302,7 @@ code does is create a scheduler PROFILE, which is
> effectively a registered set
>  of common scheduler parameters.  NOTE that this uses some pseudocode below
>  instead of real C code so as to be more concise. */
>
> -   odp_tm_sched_params_init(_params); /* <4> */
> +   odp_tm_sched_params_init(_params); // <4>
> sched_params.sched_modes = { ODP_TM_FRAME_BASED_WEIGHTS, … };
> sched_params.sched_weights = { 8, 8, 8,  … };
> sched_profile_RR = odp_tm_sched_create(“SchedProfileRR”,
> _params);
> @@ -311,24 +311,24 @@ instead of real C code so as to be more concise. */
> sched_params.sched_weights = { 8, 8, 8, … };
> sched_profile_FQ = odp_tm_sched_create(“SchedProfileFQ”,
> _params);
>
> -   odp_tm_queue_sched_config(tm_node_1, tmq_A1, sched_profile_RR); /* <5>
> */
> +   odp_tm_queue_sched_config(tm_node_1, tmq_A1, sched_profile_RR); // <5>
> odp_tm_queue_sched_config(tm_node_1, tmq_B1, sched_profile_RR);
> odp_tm_queue_sched_config(tm_node_1, tmq_A2, sched_profile_FQ);
> odp_tm_queue_sched_config(tm_node_1, tmq_B2, sched_profile_FQ);
> odp_tm_queue_sched_config(tm_node_1, tmq_C2, sched_profile_FQ);
>
> -   odp_tm_node_params_init(_params); /* <6> */
> +   odp_tm_node_params_init(_params); // <6>
> node_params.level = 2;
> tm_node_2 = odp_tm_node_create(tm, “TmNode2”, _params);
>
> -   odp_tm_node_connect(tm_node_1, tm_node_2); /* <7> */
> +   odp_tm_node_connect(tm_node_1, tm_node_2); // <7>
>
> -   odp_tm_sched_params_init(_params); /* <8> */
> +   odp_tm_sched_params_init(_params); // <8>
> sched_params.sched_modes = { ODP_TM_BYTE_BASED_WEIGHTS, … };
> sched_params.sched_weights = { 8, 16, 24,  … };
> sched_profile_WFQ = odp_tm_sched_create(“SchedProfileWFQ”,
> _params);
>
> -   odp_tm_node_sched_config(tm_node_2, tm_node_1, sched_profile_WFQ); /*
> <9> */
> +   odp_tm_node_sched_config(tm_node_2, tm_node_1, sched_profile_WFQ); //
> <9>
>  
>
>  <1> Create a tm system, since that is a precursor to creating tm_queues.
> --
> 2.7.4
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp 

Re: [lng-odp] odp_api.h missing in ODP 1.9 package

[Anders Roxell]

On 2 May 2016 at 12:33, Oriol Arcas  wrote:
> Hello,
>
> I noticed that odp_api.h is missing in the new packages for ODP 1.9.
>
> In ODP 1.8:
>
> $> dpkg-deb -c
> libodp108-dev_1.8.0.0.git28.g4fc79fc-1.linarojessie.1_amd64.deb | grep
> odp_api.h
> -rw-r--r-- root/root  1458 2016-03-28 21:04 ./usr/include/odp_api.h
>
> While in ODP 1.9:
>
> $> dpkg-deb -c
> libodp-linux-dev_1.9.0.0.git11.ga555042-1linarojessie1_amd64.deb | grep
> odp_api.h
>
> Our application fails to build with the new package.

Thank you for reporting this issue.

> Am I including it
> wrong,

No.

> or the package is missing this file?

Yes

> Maybe after migrating the
> package building scripts outside the source tree something was broken?

It's been missing since a few snapshots back yes.
It should be fixed with [1] on amd64 that you are building on.

Cheers,
Anders
[1] 
http://deb.opendataplane.org/pool/main/o/opendataplane/libodp-linux-dev_1.9.0.0.git12.g04149e6-1linarojessie1_amd64.deb

>
> Best regards,
>
> --
> Oriol Arcas
> Software Engineer
> Starflow Networks
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCHv2] CHANGELOG: Update for v1.10.0.0

[Mike Holmes]

Are there any more updates anyone needs to get into the release notes ?

On 30 April 2016 at 19:13, Bill Fischofer  wrote:

> From: Mike Holmes 
>
> Signed-off-by: Mike Holmes 
> Signed-off-by: Bill Fischofer 
> ---
>  CHANGELOG | 102
> +++---
>  1 file changed, 98 insertions(+), 4 deletions(-)
>
> diff --git a/CHANGELOG b/CHANGELOG
> index d402047..d9c66ed 100644
> --- a/CHANGELOG
> +++ b/CHANGELOG
> @@ -1,10 +1,103 @@
> +== OpenDataPlane (1.10.0.0)
>
> -   Since ODP 1.9.0.0 change logs will be maintained separately of
> main git tree
> -   and available at this url:
> -   https://git.linaro.org/lng/odp-release-notes.git
> +=== New Features
> + General
> +* Move to capabilite structures: The classification, crypto, pktio, pool,
> +  queue, shared memory, and traffic manager areas all add new APIs to
> query
> +  implementation-specific capability limits for enhanced program
> portability.
>
> -opendataplane (1.9.0.0)
> + Classification
> +* Addtion of a structure that defines system level classification
> capability
> +* Addition of range PMRs to complement the existing value PMRs to permit
> +  application to specifiy classification rules over a range of values.
>
> + Cryptography
> +* Provides a way to get the available cipher and authentication
> algorithms.
> +
> + Debug
> +* Adds the  ODP_STATIC_ASSERT() API to permit compile-time assertions.
> +
> + Packet
> +* Rename and add new packet copy and move functionality.
> +* Add packet prefetch API for performance optimization.
> +* New APIs to permit packet headroom and tailroom to be extended to
> additional
> +  segments or truncated as needed.
> +* Align an area of packet data (e.g. IP/UDP headers) so that it is
> contiguous in
> +  memory and starts in a specified minimum alignment.
> +* Add packet time stamp metadata support.
> +* Add packet input index metadata support to allow application context to
> +  be anchored from the associated interface that received the packet.
> +* Deprecation and removal of segment-level buffer APIs that are now
> +  superseded by additional packet-level manipulation APIs discussed above.
> +
> + Packet io
> +* Replaced config definition for maximum pktio entries with maximum
> packet IO
> +  index call
> +* Added the classify_enable bit to the odp_pktin_queue_param_t that allows
> +  applications to explicity control which pktin queues are subject to full
> +  classifier support.
> +* Addition of pktin configuration options to control packet timestamping
> +  and checksum validation processing.
> +* Addition of pktout configuration options to control packet checksum
> +  offload processing.
> +* Add the ability to query (via capabilties) whether a pktio interface
> +  supports operating in loopback mode and if so to enable/disable this
> +  mode of operation.
> +* Round out the polling APIs be adding the ability to receive packets in
> +  poll mode from one or more pktin queues with timeout.
> +* Add the ability to set the pktio interface index for ease of setting up
> +  application context areas associated with each pktio.
> +* Add the ability to inqure the timestamp resolution associated with
> +  pktio interfaces that support packet timestamping.
> +
> + Queue
> +* Add a queue capability API to query system-wide queue capabilities and
> limits.
> +* Extend queue context with an explicit data length parameter that serves
> +  as a performance hint for the amount of context to prefetch when the
> +  scheduler selects an event from a scheduled queue.
> +
> + Scheduler
> +* Add the ability to query information about a defined scheduler group
> +
> + Shared Memory
> +* Add a capability inquiry API to obtain implementation shared memory
> limits.
> +
> + Traffic Manager
> +* Add a capability inquiry API to obtain implementation traffic mngr
> limits.
> +* Split TOS marking into two separate calls.
> +* Add new APIs to support VLAN, ECN, and drop precedence marking.
> +* Add destroy APIs for shapers, scheduler profiles, threshholds, WRED
> profiles,
> +  TM queues, and TM nodes for symmetry and completeness to allow
> applictions
> +  to terminate gracefully without resource leakage.
> +* Add the ability to disconnect TM queues from their fanouts.
> +* Add TM node contexts to permit applications to anchor user context areas
> +  to TM nodes.
> +* Add the ability to query info about TM node fanins and TM queues.
> +* Deprecate and remove the odp_tm_periodic_update() API
> +
> + Utilities/Helpers
> +* More complete and correct checksum implementation
> +
> +=== Known Issues
> +
> +== OpenDataPlane (1.9.0.0)
> +
> +=== New Features
> +
> +* A focus has been made on improving the packaging support of ODP for
> +  distributions such as Debian.
> +* Improved creation of source tar ball and packaging 

Re: [lng-odp] [PATCH] helper: remove unused odph_linux_process_fork API

[Krishna Garapati]

On 2 May 2016 at 13:47, Mike Holmes  wrote:

> odph_linux_process_fork is not used by any ODP example or test, it is also
> untested by the helper test suite.
>
Just for the note, we use this api currently in our nginx_ofp app.

>
> odph_linux_process_fork is a wrapper for odph_linux_process_fork_n so
> just delete this, the impact if there are any users is very small.
>
 Agree, need a minor update in the app.

/Krishna

>
> Signed-off-by: Mike Holmes 
> ---
>  helper/include/odp/helper/linux.h | 18 --
>  helper/linux.c| 10 --
>  2 files changed, 28 deletions(-)
>
> diff --git a/helper/include/odp/helper/linux.h
> b/helper/include/odp/helper/linux.h
> index 7a6504f..a68f269 100644
> --- a/helper/include/odp/helper/linux.h
> +++ b/helper/include/odp/helper/linux.h
> @@ -80,24 +80,6 @@ int odph_linux_pthread_create(odph_linux_pthread_t
> *pthread_tbl,
>   */
>  void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num);
>
> -
> -/**
> - * Fork a process
> - *
> - * Forks and sets CPU affinity for the child process. Ignores 'start' and
> 'arg'
> - * thread parameters.
> - *
> - * @param[out] procPointer to process state info (for output)
> - * @param  cpu Destination CPU for the child process
> - * @param  thr_params  Linux helper thread parameters
> - *
> - * @return On success: 1 for the parent, 0 for the child
> - * On failure: -1 for the parent, -2 for the child
> - */
> -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu,
> -   const odph_linux_thr_params_t *thr_params);
> -
> -
>  /**
>   * Fork a number of processes
>   *
> diff --git a/helper/linux.c b/helper/linux.c
> index 24e243b..a181322 100644
> --- a/helper/linux.c
> +++ b/helper/linux.c
> @@ -183,16 +183,6 @@ int odph_linux_process_fork_n(odph_linux_process_t
> *proc_tbl,
> return 1;
>  }
>
> -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu,
> -   const odph_linux_thr_params_t *thr_params)
> -{
> -   odp_cpumask_t mask;
> -
> -   odp_cpumask_zero();
> -   odp_cpumask_set(, cpu);
> -   return odph_linux_process_fork_n(proc, , thr_params);
> -}
> -
>  int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num)
>  {
> pid_t pid;
> --
> 2.7.4
>
> ___
> lng-odp mailing list
> lng-odp@lists.linaro.org
> https://lists.linaro.org/mailman/listinfo/lng-odp
>
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [API-NEXT PATCH v2] doc/users-quide: correct timer API section

[Bill Fischofer]

ping. Are we expecting a v2 of this patch?

On Sun, Apr 17, 2016 at 2:58 PM, Bill Fischofer 
wrote:

> General comment: This explanation could benefit considerably from the
> inclusion of a state diagram like that used for the pktio documentation.
> Here's the suggested timer_fsm.gv file:
>
> digraph timer_state_machine {
> rankdir=LR;
> size="10,12";
> node [fontsize=28];
> edge [fontsize=28];
> node [shape=doublecircle]; Timer_Unalloc
>   Timeout_Unalloc
>   Timeout_Delivered;
> node [shape=circle];
> Timer_Unalloc -> Timer_Alloc [label="odp_timer_alloc()"];
> Timer_Alloc -> Timer_Unalloc [label="odp_timer_free()"];
> Timer_Alloc -> Timer_Set [label="odp_timer_set_abs()"];
> Timer_Alloc -> Timer_Set [label="odp_timer_set_rel()"];
> Timer_Set -> Timer_Alloc [label="odp_timer_cancel()"];
> Timer_Set -> Timeout_Alloc
> [label="odp_timer_cancel()" constraint=false];
> Timer_Set -> Timer_Fired [label="(timer expires)"];
> Timer_Fired -> Timeout_Delivered [label="odp_schedule()"];
> Timeout_Unalloc -> Timeout_Alloc
> [label="odp_timeout_alloc()" constraint=false];
> Timeout_Alloc -> Timeout_Unalloc
>   [label="odp_timeout_free()" constraint=false];
> Timeout_Alloc -> Timer_Set
>   [label="odp_timer_set_abs()" constraint=false];
> Timeout_Alloc -> Timer_Set
>   [label="odp_timer_set_rel()"];
> Timeout_Delivered -> Timer_Unalloc [label="odp_timer_free()"];
> Timeout_Delivered -> Timer_Set [label="odp_timer_set_abs()"];
> Timeout_Delivered -> Timer_Set [label="odp_timer_set_rel()"];
> Timeout_Delivered -> Timeout_Delivered
>  [label="odp_timeout_from_event()"];
> Timeout_Delivered -> Timeout_Delivered
>  [label="odp_timeout_timer()"];
> Timeout_Delivered -> Timeout_Unalloc
>  [label="odp_event_free() / odp_timeout_freee()"
>  constraint=false];
> }
>
> This should be added to doc/images and the .svg file referenced from there
> and should help clarify the text in explaining the relationship between
> timers and timeouts.
>
> Additional suggestions/corrections inline:
>
> On Thu, Apr 14, 2016 at 8:31 AM, Ivan Khoronzhuk <
> ivan.khoronz...@linaro.org> wrote:
>
>> Signed-off-by: Ivan Khoronzhuk 
>> ---
>> v2..v1:
>> - just rebased with several not important corrections
>>
>>  doc/users-guide/users-guide.adoc | 50
>> 
>>  1 file changed, 45 insertions(+), 5 deletions(-)
>>
>> diff --git a/doc/users-guide/users-guide.adoc
>> b/doc/users-guide/users-guide.adoc
>> index a2e5058..69b1930 100644
>> --- a/doc/users-guide/users-guide.adoc
>> +++ b/doc/users-guide/users-guide.adoc
>> @@ -335,11 +335,51 @@ The +odp_time_t+ opaque type represents local or
>> global timestamps.
>>
>>  === Timer
>>  Timers are how ODP applications measure and respond to the passage of
>> time.
>> -Timers are drawn from specialized pools called timer pools that have
>> their
>> -own abstract type (+odp_timer_pool_t+). Applications may have many timers
>> -active at the same time and can set them to use either relative or
>> absolute
>> -time. When timers expire they create events of type +odp_timeout_t+,
>> which
>> -serve as notifications of timer expiration.
>> +The timer API is supposed to be used when time synchronization with
>> events is
>>
>
> delete 'supposed to be' (extraneous)
>
>
>> +needed and has different nature than time API has. Usually, timers
>> require a
>>
>
> rephrase (simplify): ...and is different than the time API.
> Delete next sentence ("Usually, timers require...") as that's not relevant
> to the API or its usage.
>
>
>> +separate h/w module to be used for generating timeouts. Timers are drawn
>> from
>> +specialized pools called timer pools that have their own abstract type
>> +(+odp_timer_pool_t+). Applications may have many timers active at the
>> same
>> +time and can set them to use either relative or absolute time. When
>> timers
>> +expire they create events of type +odp_timeout_t+, which serve as
>> notifications
>> +of timer expiration. The event is placed on a queue pointed while timer
>>
>
> rephrase (clarify): "...placed on a queue specified at timer allocation."
>
>
>> +allocation.
>> +
>> +Each timer pool can be set with it's own resolution in ns and number of
>>
>
> grammar: its, not it's
>
>
>> +supported timers. So, timer pool can be considered as a time source with
>>
>
> grammar: So, a timer pool can be considered a time source...
>
>
>> +it's own resolution and defined number of timers. All timers in timer
>> pool
>> +are handled with same time source with same resolution. If user needs
>> two types
>> +of timers with different requirements for resolution then better to
>> create
>> +two pools with it's own resolution, it can decrease load on hardware.
>>
>
> grammar and rephrase: All timers in a timer pool have the same time source
> and resolution.
> If two types of timers with different resolutions are needed, then
> applications should create two
> timer pools, each with its own 

[lng-odp] [PATCH] helper: remove unused odph_linux_process_fork API

[Mike Holmes]

odph_linux_process_fork is not used by any ODP example or test, it is also
untested by the helper test suite.

odph_linux_process_fork is a wrapper for odph_linux_process_fork_n so
just delete this, the impact if there are any users is very small.

Signed-off-by: Mike Holmes 
---
 helper/include/odp/helper/linux.h | 18 --
 helper/linux.c| 10 --
 2 files changed, 28 deletions(-)

diff --git a/helper/include/odp/helper/linux.h 
b/helper/include/odp/helper/linux.h
index 7a6504f..a68f269 100644
--- a/helper/include/odp/helper/linux.h
+++ b/helper/include/odp/helper/linux.h
@@ -80,24 +80,6 @@ int odph_linux_pthread_create(odph_linux_pthread_t 
*pthread_tbl,
  */
 void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num);
 
-
-/**
- * Fork a process
- *
- * Forks and sets CPU affinity for the child process. Ignores 'start' and 'arg'
- * thread parameters.
- *
- * @param[out] procPointer to process state info (for output)
- * @param  cpu Destination CPU for the child process
- * @param  thr_params  Linux helper thread parameters
- *
- * @return On success: 1 for the parent, 0 for the child
- * On failure: -1 for the parent, -2 for the child
- */
-int odph_linux_process_fork(odph_linux_process_t *proc, int cpu,
-   const odph_linux_thr_params_t *thr_params);
-
-
 /**
  * Fork a number of processes
  *
diff --git a/helper/linux.c b/helper/linux.c
index 24e243b..a181322 100644
--- a/helper/linux.c
+++ b/helper/linux.c
@@ -183,16 +183,6 @@ int odph_linux_process_fork_n(odph_linux_process_t 
*proc_tbl,
return 1;
 }
 
-int odph_linux_process_fork(odph_linux_process_t *proc, int cpu,
-   const odph_linux_thr_params_t *thr_params)
-{
-   odp_cpumask_t mask;
-
-   odp_cpumask_zero();
-   odp_cpumask_set(, cpu);
-   return odph_linux_process_fork_n(proc, , thr_params);
-}
-
 int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num)
 {
pid_t pid;
-- 
2.7.4

___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

[lng-odp] odp_api.h missing in ODP 1.9 package

[Oriol Arcas]

Hello,

I noticed that odp_api.h is missing in the new packages for ODP 1.9.

In ODP 1.8:

$> dpkg-deb -c
libodp108-dev_1.8.0.0.git28.g4fc79fc-1.linarojessie.1_amd64.deb | grep
odp_api.h
-rw-r--r-- root/root  1458 2016-03-28 21:04 ./usr/include/odp_api.h

While in ODP 1.9:

$> dpkg-deb -c
libodp-linux-dev_1.9.0.0.git11.ga555042-1linarojessie1_amd64.deb | grep
odp_api.h

Our application fails to build with the new package. Am I including it
wrong, or the package is missing this file? Maybe after migrating the
package building scripts outside the source tree something was broken?

Best regards,

--
Oriol Arcas
Software Engineer
Starflow Networks
___
lng-odp mailing list
lng-odp@lists.linaro.org
https://lists.linaro.org/mailman/listinfo/lng-odp

Re: [lng-odp] [GIT PULL ODPv4] running things in process mode

[Christophe Milard]

Hi Brian

I have commented below, but it looks we need to talk.  would you have time
after the linaro sync call today, for a HO?

On 29 April 2016 at 23:49, Brian Brooks  wrote:

> On 04/29 13:29:03, Christophe Milard wrote:
> > On 29 April 2016 at 00:48, Brian Brooks  wrote:
> >
> > > On 04/28 17:18:36, Christophe Milard wrote:
> > > > Since v3:
> > > > -fixed rebase error (Christophe)
> > > > -rebased
> > >
> > > Thanks for the rebase. test_in_process_mode_v4 merged cleanly into
> > > origin/master and build and tests PASS.
> > >
> > > I've given this a look, and it appears we're headed in the right
> direction.
> > >
> > > > diff --git a/example/classifier/odp_classifier.c
> > > b/example/classifier/odp_classifier.c
> > > > index a477e23..4057457 100644
> > > > --- a/example/classifier/odp_classifier.c
> > > > +++ b/example/classifier/odp_classifier.c
> > > > @@ -815,10 +802,16 @@ static void parse_args(int argc, char *argv[],
> > > appl_args_t *appl_args)
> > > >   {NULL, 0, NULL, 0}
> > > >   };
> > > >
> > > > + static const char *shortopts = "+c:t:i:p:m:t:h";
> > > > +
> > > > + /* let helper collect its own arguments (e.g. --odph_proc) */
> > > > + odph_parse_options(argc, argv, shortopts, longopts);
> > > > +
> > > > + opterr = 0; /* do not issue errors on helper options */
> > >
> > > Please use the default behavior of opterr _or_ add the case statement
> for
> > > '?'
> > > when appropriate.
> > >
> >
> > I do think we need opterr=0, sadly: when the test or example (such as the
> > classifier here) parses it options, the command line options being parsed
> > still contains all options, i.e. both the option meant for the classifier
> > itself, and the option meant to the "other layers", (here the helper). If
> > opt_err is left to 1, the classifier's call to get_opt() will issue
> errors
> > when hitting the helper options, even if we have a "?" in the switch, as
> > far as I understand.
> > So the mechanism taken here is: the caller (e.g. classifier) passes its
> > list of option to the helper which builds the complete list of options by
> > merging it own (helper) option list to the caller options. Then the
> helper
> > parse this complete list (with default opterr=1), i.e. reacting to
> unknown
> > options (not being in the merged list) and also picking up its own option
> > semantic, of course. After this the caller (here classifier) parse the
> > options again, picking up its own options only. but should not react on
> the
> > helper's otpions.
> > I Actually looked at removing the options from argv in the helpers, but
> > this turned up to be quite tricky as well.
> > parse_args() seem to be better at spitting command line option in
> different
> > owner, but is is not POSIX. not sure we want to insert that kind of
> > dependency for so little.
>
> OK, this should not block a contribution.
>
> > > > diff --git a/helper/include/odp/helper/linux.h
> > > b/helper/include/odp/helper/linux.h
> > > > index 7a6504f..a9ec90a 100644
> > > > --- a/helper/include/odp/helper/linux.h
> > > > +++ b/helper/include/odp/helper/linux.h
> > > > @@ -25,111 +25,146 @@ extern "C" {
> > > >  #include 
> > > >
> > > >  #include 
> > > > +#include 
> > >
> > > Please consider migrating CLI parsing via C library to a more
> appropriate
> > > location.
> > >
> >
> > Not sure I understand this: The goal is to have the helpers parsing their
> > own options here. Maybe my comment above makes it clearer. otherwise
> please
> > explain what you meant.
>
> I see. I'll attempt to explain further.
>
> Helper linux.[ch] files contain code for both threading and arg parsing.
> First,
> because threading and arg parsing are separate functional pieces of code
> they
> _should_ belong in separate files. Second, the linux.[ch] filename is
> misleading. Two simple examples:
>
>   lib/thread.h
>   lib/thread.c
>   lib/thread_posix.c
>   lib/thread_linux.c
>
>   include/thread.h
>   lib/thread.c
>   lib/posix/thread.c
>
> Many ways, but the point is that common platform-independent code contained
> in one place, and abstraction occurs 'outward' or 'deeper' in the filename
> or directory structure.
>
> The filename and directory path corresponds to the code inside.
>
> helper/include/odp/helper/linux.h <- CLI code goes here. Too subjective?
>

hmm,
To start with, I will keep using the term "ODP thread", even if I don't
like it more than you, because that is the accepted term here. It is
important we talk the same language, all of us. You are welcome to alias it
to "ODPtask" on your head, or argue for a name change (which I alreay
 tried !), but we should be using the same accepted term everywhere, and
currently, it is "ODPthread".

I think our disagreement mostly comes from the fact that "helpers" is not a
very well defined thing. And interestingly, your comments match the
feelings I had when I started... There must be some good in them :-)
ODP is a