[lng-odp] My last day at linaro/LNG

[Christophe Milard]

Hi all,
As most of you know, my assignment for linaro/LNG is ending today.
I was really willing to join connect in Budapest as this would have
given me an opportunity to see you a last time face to face but this
did not happen, sadly.
I wish to thank all of you that helped me growing in this open source
world. It has been a very interesting time.
I do regret that the DDF work did not fully land before my departure.
Hopefully my work helped on the way...

I'd be happy to see ODP growing... I'll keep an eye on it

I wish you all the best,

Regards,

Christophe.

[lng-odp] [API-NEXT PATCHv2 23/23] test: drv: test for setting and retrieving driver's data

[Christophe Milard]

trivial tests for function odpdrv_device_set_data() and
odpdrv_device_get_data().

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 test/common_plat/validation/drv/drvdriver/drvdriver_driver.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c 
b/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
index d90aa56..edcb0f4 100644
--- a/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
+++ b/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
@@ -380,6 +380,8 @@ static int driver1_probe(odpdrv_device_t dev, 
odpdrv_devio_t devio, int idx)
if (dev == E1_devs[i]) {
driver1_probed_index |= (1 << i);
dev_found = 1;
+   /* just set dev index as driver data */
+   odpdrv_device_set_data(dev, (void *)(uintptr_t)i);
}
}
CU_ASSERT(dev_found);
@@ -430,6 +432,7 @@ static int driver1_unbind(odpdrv_device_t dev,
  void (*callback)(odpdrv_device_t dev),
  uint32_t flags)
 {
+   CU_ASSERT(E1_devs[(uintptr_t)odpdrv_device_get_data(dev)] == dev);
CU_ASSERT(dev != ODPDRV_DEVICE_INVALID);
CU_ASSERT(flags == ODPDRV_DRV_UNBIND_IMMEDIATE);
callback(dev);
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 21/23] drv: driver: adding functions to attach driver's data to the device

[Christophe Milard]

Driver will need to attach their data to devices when bound.
The patch introduce a data setter and a data getter function to do so.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 21 +
 1 file changed, 21 insertions(+)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index ad2b8db..224baf4 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -450,6 +450,27 @@ odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t 
*param);
 odpdrv_driver_t odpdrv_driver_register(odpdrv_driver_param_t *param);
 
 /**
+* Sets the device driver data, i.e. the driver data which should be attached to
+* the device.
+* After a driver is bound to a device, this driver will need to keep
+* data attached to this device. This data is, of course, driver dependent.
+*
+* @param dev: the device to which data should be attached.
+* @param data Pointer to whatever thre driver want to keep attached to the
+* device
+*/
+void odpdrv_device_set_data(odpdrv_device_t dev, void *data);
+
+/**
+* Gets the device driver data, i.e. the driver data which should be attached to
+* the device.
+* Retrieve the pointer which was set with odpdrv_device_set_data()
+* @param dev: the device from which the driver data should be retrieved.
+* @return the driver data pointer (as set by odpdrv_device_set_data()) or NULL.
+*/
+void *odpdrv_device_get_data(odpdrv_device_t dev);
+
+/**
 * Print (ODP_DBG) the driver interface status (debug).
 *
 * @return 0 on success, less than zero on error (inconsistency detected)
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 22/23] linux-gen: adding functions to attach driver's data to the device

[Christophe Milard]

Implementation of the functions to set/retrieve driver's data to/from
a bound device.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/drv_driver.c | 18 ++
 1 file changed, 18 insertions(+)

diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index ea92457..70bf318 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -70,6 +70,7 @@ struct _odpdrv_device_s {
odpdrv_device_param_t param;
_odpdrv_driver_t *driver; /* driver for the device (if bound), or NULL*/
_odpdrv_devio_t *devio;   /* devio used for device (if bound), or NULL*/
+   void *driver_data;/* anything that the driver need to attach. */
void (*enumr_destroy_callback)(void *enum_dev);/*dev destroy callback */
struct _odpdrv_device_s *next;
 } _odpdrv_device_s;
@@ -395,6 +396,7 @@ odpdrv_device_t odpdrv_device_create(odpdrv_device_param_t 
*param)
dev->enumr_destroy_callback = NULL;
dev->driver = NULL;
dev->devio = NULL;
+   dev->driver_data = NULL;
dev_list_write_lock();
dev->next = device_lst.head;
device_lst.head = dev;
@@ -852,6 +854,22 @@ void _odpdrv_driver_probe_drv_items(void)
probe_all();
 }
 
+void odpdrv_device_set_data(odpdrv_device_t dev, void *data)
+{
+   _odpdrv_device_t *_dev;
+
+   _dev = get_device(dev);
+   _dev->driver_data = data;
+}
+
+void *odpdrv_device_get_data(odpdrv_device_t dev)
+{
+   _odpdrv_device_t *_dev;
+
+   _dev = get_device(dev);
+   return _dev->driver_data;
+}
+
 int odpdrv_print_all(void)
 {
_odpdrv_enumr_class_t *enumr_c;
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 14/23] drv: driver: adding a probe and remove callback for devio

[Christophe Milard]

Needed to delete the resources needed for the devio. That is possibly
the memory allocated for its "ops" part if it was allocated. May be NULL
if nothing needs to be done at devio deletion time.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 18 ++
 1 file changed, 18 insertions(+)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index e861396..f89c282 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -269,6 +269,24 @@ struct odpdrv_devio_param_t {
char enumr_api_name[ODPDRV_NAME_SIZE];
uint32_t enumr_api_version; /**<< required enumerator API version */
 
+   /** Probe function:
+* Tell whether this devio can handle the given device.
+* The devio is hence given a chance to reject a given device for
+* any reason. No binding occurs here. binding occurs when the
+* driver is probed.
+* returns 0 if this devio can handle the given device, or a negative
+* value if not.
+* If left to NULL, a 0 returned value is assumed
+*/
+   int (*probe)(odpdrv_device_t dev);
+
+   /** Remove function:
+* Should destroy the memory allocated for ops and anything else
+* under it, or any resource for this devio.
+* Returns 0 on success or a negative value on error.
+*/
+   int (*remove)(void);
+
/** Ops
 * Pointer to a devio ops structure (specific to each devio)
 */
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 13/23] test: drv: device creation and destruction

[Christophe Milard]

Testing that devices can be created and removed from ODP.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 .../validation/drv/drvdriver/.gitignore|   1 +
 .../validation/drv/drvdriver/Makefile.am   |  11 ++
 .../validation/drv/drvdriver/drvdriver_device.c| 218 +
 .../validation/drv/drvdriver/drvdriver_device.h|  24 +++
 .../drv/drvdriver/drvdriver_device_main.c  |  12 ++
 test/linux-generic/Makefile.am |   1 +
 6 files changed, 267 insertions(+)
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_device.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_device.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_device_main.c

diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore 
b/test/common_plat/validation/drv/drvdriver/.gitignore
index a842448..97b4312 100644
--- a/test/common_plat/validation/drv/drvdriver/.gitignore
+++ b/test/common_plat/validation/drv/drvdriver/.gitignore
@@ -1,2 +1,3 @@
 drvdriver_enumr_class_main
 drvdriver_enumr_main
+drvdriver_device_main
diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am 
b/test/common_plat/validation/drv/drvdriver/Makefile.am
index 3476c50..544586c 100644
--- a/test/common_plat/validation/drv/drvdriver/Makefile.am
+++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
@@ -25,3 +25,14 @@ drvdriver_enumr_main_LDADD = libtestdrvdriverenumr.la \
   $(LIBCUNIT_COMMON) $(LIBODP)
 
 EXTRA_DIST += drvdriver_enumr.h
+
+#tests for device creation:
+noinst_LTLIBRARIES += libtestdrvdriverdevice.la
+libtestdrvdriverdevice_la_SOURCES = drvdriver_device.c
+
+test_PROGRAMS += drvdriver_device_main$(EXEEXT)
+dist_drvdriver_device_main_SOURCES = drvdriver_device_main.c
+drvdriver_device_main_LDADD = libtestdrvdriverdevice.la \
+  $(LIBCUNIT_COMMON) $(LIBODP)
+
+EXTRA_DIST += drvdriver_device.h
diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_device.c 
b/test/common_plat/validation/drv/drvdriver/drvdriver_device.c
new file mode 100644
index 000..d010026
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/drvdriver_device.c
@@ -0,0 +1,218 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include "drvdriver_device.h"
+#include 
+
+static odp_instance_t odp_instance;
+static odpdrv_enumr_class_t enumr_class1;
+static odpdrv_enumr_t enumr1;
+
+typedef struct dev_enumr_data_t { /* enumerator data for registered devices */
+   odpdrv_shm_tshm_handle;
+   int device_number;
+} dev_enumr_data_t;
+
+#define NB_DEVICES 5
+
+/* forward declaration */
+static int enumr1_probe(void);
+static int enumr1_remove(void);
+static int enumr_class1_probe(void);
+static int enumr_class1_remove(void);
+
+/* because many things to be checked are performed during ODP initialisation,
+ * the initialisation functions have to be a part of the test
+ */
+static int tests_global_init(void)
+{
+   if (0 != odp_init_global(_instance, NULL, NULL)) {
+   fprintf(stderr, "error: odp_init_global() failed.\n");
+   return -1;
+   }
+   if (0 != odp_init_local(odp_instance, ODP_THREAD_CONTROL)) {
+   fprintf(stderr, "error: odp_init_local() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+static int tests_global_term(void)
+{
+   if (0 != odp_term_local()) {
+   fprintf(stderr, "error: odp_term_local() failed.\n");
+   return -1;
+   }
+
+   if (0 != odp_term_global(odp_instance)) {
+   fprintf(stderr, "error: odp_term_global() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+/*enumerator register functions */
+static odpdrv_enumr_t enumr1_register(void)
+{
+   odpdrv_enumr_param_t param = {
+   .enumr_class = enumr_class1,
+   .api_name = "Enumerator_interface_1",
+   .api_version = 1,
+   .probe = enumr1_probe,
+   .remove = enumr1_remove,
+   .register_notifier = NULL
+   };
+
+   enumr1 = odpdrv_enumr_register();
+   return enumr1;
+}
+
+/*enumerator probe functions, just making sure they have been run: */
+static int enumr1_probe(void)
+{
+   int dev;
+   odpdrv_shm_t shm;
+   dev_enumr_data_t *dev_data;
+
+   odpdrv_device_param_t param = {
+   .enumerator = enumr1,
+   .address = "00:00:0X",
+   .enum_dev = NULL
+   };
+
+   /* create 5 devices: */
+   for (dev = 0; dev < NB_DEVICES; dev++) {
+   shm = odpdrv_shm_reserve(NULL, sizeof(dev_enumr_data_t),
+  

[lng-odp] [API-NEXT PATCHv2 20/23] test: drv: driver registration and probing

[Christophe Milard]

Register driver, devios, enumerators, create devices, and check
that probing occurs correctely.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 .../validation/drv/drvdriver/.gitignore|   1 +
 .../validation/drv/drvdriver/Makefile.am   |  11 +
 .../validation/drv/drvdriver/drvdriver_driver.c| 515 +
 .../validation/drv/drvdriver/drvdriver_driver.h|  24 +
 .../drv/drvdriver/drvdriver_driver_main.c  |  12 +
 test/linux-generic/Makefile.am |   1 +
 6 files changed, 564 insertions(+)
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_driver.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_driver_main.c

diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore 
b/test/common_plat/validation/drv/drvdriver/.gitignore
index 829c8b4..76bb6ba 100644
--- a/test/common_plat/validation/drv/drvdriver/.gitignore
+++ b/test/common_plat/validation/drv/drvdriver/.gitignore
@@ -2,3 +2,4 @@ drvdriver_enumr_class_main
 drvdriver_enumr_main
 drvdriver_device_main
 drvdriver_devio_main
+drvdriver_driver_main
diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am 
b/test/common_plat/validation/drv/drvdriver/Makefile.am
index 8e695ba..88bd828 100644
--- a/test/common_plat/validation/drv/drvdriver/Makefile.am
+++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
@@ -47,3 +47,14 @@ drvdriver_devio_main_LDADD = libtestdrvdriverdevio.la \
   $(LIBCUNIT_COMMON) $(LIBODP)
 
 EXTRA_DIST += drvdriver_devio.h
+
+#tests for driver registration and probing:
+noinst_LTLIBRARIES += libtestdrvdriverdriver.la
+libtestdrvdriverdriver_la_SOURCES = drvdriver_driver.c
+
+test_PROGRAMS += drvdriver_driver_main$(EXEEXT)
+dist_drvdriver_driver_main_SOURCES = drvdriver_driver_main.c
+drvdriver_driver_main_LDADD = libtestdrvdriverdriver.la \
+  $(LIBCUNIT_COMMON) $(LIBODP)
+
+EXTRA_DIST += drvdriver_driver.h
diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c 
b/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
new file mode 100644
index 000..d90aa56
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
@@ -0,0 +1,515 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+/* This file is a bit long as it tries to simulate the presence of 2
+ * enumerator classes, 2 enumerators, and 3 drivers in one go to
+ * see how things are handled by the driver framework.
+ * The following is done:
+ * - create 2 enumerator classes,
+ * - each with its own enumerator providing interfaces: E1 and E2.
+ * - E1 and E2 create 4 devices each.
+ * - the following devio are created:
+ *   devio1 enabling device handling from DRVIF-1 to E1
+ *   devio2 enabling device handling from DRVIF-2 to E2
+ *   devio3 enabling device handling from DRVIF-3 to E2
+ *   devio4 enabling device handling from DRVIF-3 to E3 (does not exist)
+ *   devio5 enabling device handling from DRVIF-4 to E3 (does not exist)
+ *
+ * -then the following driver are created:
+ *  driver1, requiring devio DRVIF-1
+ *  driver2, requiring devio DRVIF-2 (preferred) and DRVIF-3
+ *  driver3, requiring devio DRVIF-4
+ *
+ * The test amkes sure that:
+ * driver 1 is probed (and accepts) the 4 devices of E1 with devio1
+ * driver 2 is probed (and rejects) the 4 devices of E2 with devio2 and devio4
+ * driver3 is never probed.
+ */
+
+#include 
+#include 
+#include 
+#include "drvdriver_driver.h"
+#include 
+
+static odp_instance_t odp_instance;
+static odpdrv_enumr_class_t enumr_class1;
+static odpdrv_enumr_class_t enumr_class2;
+static odpdrv_enumr_t enumr1;
+static odpdrv_enumr_t enumr2;
+#define NB_DEVICES 4
+static odpdrv_device_t E1_devs[NB_DEVICES];
+static odpdrv_device_t E2_devs[NB_DEVICES];
+static odpdrv_devio_t devio1;
+static odpdrv_devio_t devio2;
+static odpdrv_devio_t devio3;
+static odpdrv_devio_t devio4;
+static odpdrv_devio_t devio5;
+static odpdrv_driver_t driver1;
+static odpdrv_driver_t driver2;
+static odpdrv_driver_t driver3;
+
+static int driver1_probed_index;
+static int driver2_probed_index;
+
+/* forward declaration */
+static int enumr1_probe(void);
+static int enumr2_probe(void);
+static int enumr1_remove(void);
+static int enumr2_remove(void);
+static int enumr_class1_probe(void);
+static int enumr_class2_probe(void);
+static int driver1_probe(odpdrv_device_t dev, odpdrv_devio_t devio, int idx);
+static int driver2_probe(odpdrv_device_t dev, odpdrv_devio_t devio, int idx);
+static int driver3_probe(odpdrv_device_t dev, odpdrv_devio_t devio, int idx);
+static int driver1_unbind(odpdrv_device_t dev,
+ void (*callback)(odpdrv_device_t dev),
+ uint32_t flags);
+static in

[lng-odp] [API-NEXT PATCHv2 19/23] linux-gen: driver registration and probing

[Christophe Milard]

Driver registration and probing is implemented for linux-gen ODP.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/drv_driver.c | 349 ++--
 1 file changed, 336 insertions(+), 13 deletions(-)

diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index 873ec3c..ea92457 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -12,6 +12,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -29,6 +30,11 @@ typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
 typedef struct _odpdrv_enumr_s _odpdrv_enumr_t;
 typedef struct _odpdrv_device_s _odpdrv_device_t;
 typedef struct _odpdrv_devio_s _odpdrv_devio_t;
+typedef struct _odpdrv_driver_s _odpdrv_driver_t;
+
+static int unbind_device_driver(_odpdrv_device_t *dev,
+   void (*callback)(odpdrv_device_t odpdrv_dev),
+   int immediate);
 
 /* an enumerator class (list element) */
 struct _odpdrv_enumr_class_s {
@@ -62,6 +68,8 @@ static struct _odpdrv_enumr_lst_t enumr_lst;
 /* a device (list element) */
 struct _odpdrv_device_s {
odpdrv_device_param_t param;
+   _odpdrv_driver_t *driver; /* driver for the device (if bound), or NULL*/
+   _odpdrv_devio_t *devio;   /* devio used for device (if bound), or NULL*/
void (*enumr_destroy_callback)(void *enum_dev);/*dev destroy callback */
struct _odpdrv_device_s *next;
 } _odpdrv_device_s;
@@ -87,6 +95,21 @@ typedef struct _odpdrv_devio_lst_t {
 } _odpdrv_devio_lst_t;
 static struct _odpdrv_devio_lst_t devio_lst;
 
+/* a driver (list element) */
+struct _odpdrv_driver_s {
+   odpdrv_driver_param_t param;
+   _odp_ishm_pool_t *pool;
+   odp_ticketlock_t probelock; /* to avoid concurrent probe on same drv*/
+   struct _odpdrv_driver_s *next;
+};
+
+/* the driver list: */
+typedef struct _odpdrv_driver_lst_t {
+   odp_rwlock_recursive_t lock;
+   _odpdrv_driver_t *head;
+} _odpdrv_driver_lst_t;
+static struct _odpdrv_driver_lst_t driver_lst;
+
 /* some driver elements (such as enumeraor classes, drivers, devio) may
  * register before init_global and init_local complete. Mutex will fail
  * in this cases but should be used later on.
@@ -188,6 +211,30 @@ static void devio_list_write_unlock(void)
odp_rwlock_recursive_write_unlock(_lst.lock);
 }
 
+static void driver_list_read_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_lock(_lst.lock);
+}
+
+static void driver_list_read_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_unlock(_lst.lock);
+}
+
+static void driver_list_write_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_lock(_lst.lock);
+}
+
+static void driver_list_write_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_unlock(_lst.lock);
+}
+
 /* some functions to get internal pointers from handles... */
 static inline _odpdrv_enumr_class_t *get_enumr_class(odpdrv_enumr_class_t 
class)
 {
@@ -346,6 +393,8 @@ odpdrv_device_t odpdrv_device_create(odpdrv_device_param_t 
*param)
/* save and set dev init parameters and insert new device in list */
dev->param = *param;
dev->enumr_destroy_callback = NULL;
+   dev->driver = NULL;
+   dev->devio = NULL;
dev_list_write_lock();
dev->next = device_lst.head;
device_lst.head = dev;
@@ -399,19 +448,17 @@ int odpdrv_device_destroy(odpdrv_device_t dev,
 */
target->enumr_destroy_callback = callback;
 
-   /* TODO: if a driver is bound to the device, unbind it!
-* passing the flag andf device_destroy_terminate() as a callback */
-
-   /* no driver is handling this device, or no callback was
-* provided: continue removing the device: */
-   device_destroy_terminate(dev);
+   /* unbind the driver from the device (if bound).
+* The callback is always called. */
+   unbind_device_driver(target,
+device_destroy_terminate,
+(flags & ODPDRV_DEV_DESTROY_IMMEDIATE));
 
return 0;
 }
 
 /* This function is called as a callback from the driver, when unbindind
- * a device, or directely from odpdrv_device_destroy() if no driver
- * was bound to the device.
+ * a device drom odpdrv_device_destroy()
  * just call the enumerator callback to cleanup the enumerator part
  * and free device memory */
 static void device_destroy_terminate(odpdrv_device_t drv_device)
@@ -532,10 +579,239 @@ odpdrv_devio_t 
odpdrv_devio_register(odpdrv_devio_param_t *param)
 
 odpdrv_driver_t odpdrv_driver_register(odpdrv_driver_param_t *param)
 {
-   ODP_ERR("NOT Supported yet! Driver %s Registration!

[lng-odp] [API-NEXT PATCHv2 18/23] drv: complement parameters to the driver probe() function

[Christophe Milard]

Of course, when probing a driver, the latter should be given the devio
handle to be used. This is what this patch adds. It also adds the index
of the devio, hence telling the driver which of the possible
ODPDRV_MAX_DEVIOS devios was selected, as this is going to be the first
thing the driver needs to know.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index 394aa92..ad2b8db 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -314,9 +314,12 @@ struct odpdrv_driver_param_t {
 
/** Probe function:
 * Called by ODP to see if the driver can drive a given device
-*
+* -dev the device to be probed
+* -devio is the devio to be used.
+* -devio_idx actually tells which devio was selected: it is the
+* index in the devios array above.
 */
-   int (*probe)(odpdrv_device_t *dev);
+   int (*probe)(odpdrv_device_t dev, odpdrv_devio_t devio, int devio_idx);
 
/** unbind function:
 * Only called with devices whose probe() returned true
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 17/23] drv: adding driver remove function

[Christophe Milard]

The remove function, as for other driver items (such as enumerators...) is
called before the driver is to be removed, i.e. after all devices have been
been unbound from the driver. remove() should release any resource held
by the driver.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 8 
 1 file changed, 8 insertions(+)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index f89c282..394aa92 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -336,6 +336,14 @@ struct odpdrv_driver_param_t {
int (*unbind)(odpdrv_device_t dev,
  void (*callback)(odpdrv_device_t dev),
  uint32_t flags);
+
+   /** remove function:
+* remove any resource taken by the driver. Called when the driver
+* itself is to be removed, i.e. after all devices are unbound
+* Can be set to NULL if the driver has nothing to release.
+*
+*/
+   int (*remove)(void);
 };
 
 /** The callback function must be called mmediately by the current ODP thread 
*/
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 16/23] test: drv: devio creation and destruction

[Christophe Milard]

Testing that devios can be registered and removed in/from ODP.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 .../validation/drv/drvdriver/.gitignore|   1 +
 .../validation/drv/drvdriver/Makefile.am   |  11 ++
 .../validation/drv/drvdriver/drvdriver_devio.c | 209 +
 .../validation/drv/drvdriver/drvdriver_devio.h |  24 +++
 .../drv/drvdriver/drvdriver_devio_main.c   |  12 ++
 test/linux-generic/Makefile.am |   1 +
 6 files changed, 258 insertions(+)
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_devio.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_devio.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_devio_main.c

diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore 
b/test/common_plat/validation/drv/drvdriver/.gitignore
index 97b4312..829c8b4 100644
--- a/test/common_plat/validation/drv/drvdriver/.gitignore
+++ b/test/common_plat/validation/drv/drvdriver/.gitignore
@@ -1,3 +1,4 @@
 drvdriver_enumr_class_main
 drvdriver_enumr_main
 drvdriver_device_main
+drvdriver_devio_main
diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am 
b/test/common_plat/validation/drv/drvdriver/Makefile.am
index 544586c..8e695ba 100644
--- a/test/common_plat/validation/drv/drvdriver/Makefile.am
+++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
@@ -36,3 +36,14 @@ drvdriver_device_main_LDADD = libtestdrvdriverdevice.la \
   $(LIBCUNIT_COMMON) $(LIBODP)
 
 EXTRA_DIST += drvdriver_device.h
+
+#tests for devio creation:
+noinst_LTLIBRARIES += libtestdrvdriverdevio.la
+libtestdrvdriverdevio_la_SOURCES = drvdriver_devio.c
+
+test_PROGRAMS += drvdriver_devio_main$(EXEEXT)
+dist_drvdriver_devio_main_SOURCES = drvdriver_devio_main.c
+drvdriver_devio_main_LDADD = libtestdrvdriverdevio.la \
+  $(LIBCUNIT_COMMON) $(LIBODP)
+
+EXTRA_DIST += drvdriver_devio.h
diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_devio.c 
b/test/common_plat/validation/drv/drvdriver/drvdriver_devio.c
new file mode 100644
index 000..cf7c7c9
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/drvdriver_devio.c
@@ -0,0 +1,209 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include "drvdriver_devio.h"
+#include 
+
+static odp_instance_t odp_instance;
+
+static int devio1_removed;
+static int devio2_removed;
+static int devio3_removed;
+static int devio4_removed;
+
+static int devio1_remove(void);
+static int devio2_remove(void);
+static int devio3_remove(void);
+
+/* because many things to be checked are performed during ODP initialisation,
+ * the initialisation functions have to be a part of the test
+ */
+static int tests_global_init(void)
+{
+   if (0 != odp_init_global(_instance, NULL, NULL)) {
+   fprintf(stderr, "error: odp_init_global() failed.\n");
+   return -1;
+   }
+   if (0 != odp_init_local(odp_instance, ODP_THREAD_CONTROL)) {
+   fprintf(stderr, "error: odp_init_local() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+static int tests_global_term(void)
+{
+   if (0 != odp_term_local()) {
+   fprintf(stderr, "error: odp_term_local() failed.\n");
+   return -1;
+   }
+
+   if (0 != odp_term_global(odp_instance)) {
+   fprintf(stderr, "error: odp_term_global() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+/*devio register functions, all "statically linked"
+ *(i.e. directely run at start), due to the fact that platorm independent
+ * shared lib loading in autotools is a mess */
+static void ODPDRV_CONSTRUCTOR devio1_register(void)
+{
+   odpdrv_devio_param_t param = {
+   .api_name = "devio_api_1",
+   .api_version = 1,
+   .enumr_api_name = "Enumerator_interface_1",
+   .enumr_api_version = 1,
+   .remove = devio1_remove,
+   .ops = NULL,
+   };
+
+   odpdrv_devio_register();
+}
+
+static void ODPDRV_CONSTRUCTOR devio2_register(void)
+{
+   odpdrv_devio_param_t param = {
+   .api_name = "devio_api_2",
+   .api_version = 1,
+   .enumr_api_name = "Enumerator_interface_1",
+   .enumr_api_version = 1,
+   .probe = NULL,
+   .remove = devio2_remove,
+   .ops = NULL,
+   };
+
+   odpdrv_devio_register();
+}
+
+static odpdrv_devio_t devio2_register_retry(void)
+{
+   odpdrv_devio_param_t param = {
+   .api_name = "devio_api_2",
+   .api_version = 1,
+

[lng-odp] [API-NEXT PATCHv2 15/23] linux-gen: drv: devio registration

[Christophe Milard]

devios (dev IO) provide a interface for drivers to access a device:
Devices enumerated by enumerators may be accessed in by different
mechanisms (depending on iommu presence or other factors). This extra
abstraction is provided by devios, which provide a sets of methods to
access the devices of a given type (i.e. registred enumerator(s)
enumerating devices of the same kind (e.g. PCI)).
This patch just implements the devio registration method provided by the
driver API.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/drv_driver.c | 134 +++-
 1 file changed, 131 insertions(+), 3 deletions(-)

diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index 3e4780b..873ec3c 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -28,6 +28,7 @@ static _odp_ishm_pool_t *list_elt_pool;
 typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
 typedef struct _odpdrv_enumr_s _odpdrv_enumr_t;
 typedef struct _odpdrv_device_s _odpdrv_device_t;
+typedef struct _odpdrv_devio_s _odpdrv_devio_t;
 
 /* an enumerator class (list element) */
 struct _odpdrv_enumr_class_s {
@@ -72,6 +73,20 @@ typedef struct _odpdrv_device_lst_t {
 } _odpdrv_device_lst_t;
 static struct _odpdrv_device_lst_t device_lst;
 
+/* a devio (list element) */
+struct _odpdrv_devio_s {
+   odpdrv_devio_param_t param;
+   _odp_ishm_pool_t *pool;
+   struct _odpdrv_devio_s *next;
+} _odpdrv_devio_s;
+
+/* the devio list: */
+typedef struct _odpdrv_devio_lst_t {
+   odp_rwlock_recursive_t lock;
+   _odpdrv_devio_t *head;
+} _odpdrv_devio_lst_t;
+static struct _odpdrv_devio_lst_t devio_lst;
+
 /* some driver elements (such as enumeraor classes, drivers, devio) may
  * register before init_global and init_local complete. Mutex will fail
  * in this cases but should be used later on.
@@ -149,6 +164,30 @@ static void dev_list_write_unlock(void)
odp_rwlock_recursive_write_unlock(_lst.lock);
 }
 
+static void devio_list_read_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_lock(_lst.lock);
+}
+
+static void devio_list_read_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_unlock(_lst.lock);
+}
+
+static void devio_list_write_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_lock(_lst.lock);
+}
+
+static void devio_list_write_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_unlock(_lst.lock);
+}
+
 /* some functions to get internal pointers from handles... */
 static inline _odpdrv_enumr_class_t *get_enumr_class(odpdrv_enumr_class_t 
class)
 {
@@ -430,10 +469,65 @@ odpdrv_device_t *odpdrv_device_query(odpdrv_enumr_t 
enumr, const char *address)
 
 odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t *param)
 {
-   ODP_ERR("NOT Supported yet! Driver %s Registration!\n.",
-   param->api_name);
+   _odpdrv_devio_t *devio;
+
+   /* parse the list of already registered devios to make
+* sure no devio providing the same interface using th esame enumerator
+* already exists:
+*/
+   devio_list_read_lock();
+   devio = devio_lst.head;
+   while (devio) {
+   if ((strncmp(param->api_name, devio->param.api_name,
+ODPDRV_NAME_SIZE) == 0) &&
+   (strncmp(param->enumr_api_name, devio->param.enumr_api_name,
+ODPDRV_NAME_SIZE) == 0)) {
+   ODP_ERR("a devio providing interface '%s' for devices "
+   "of type '%s' is already registered\n!",
+   param->api_name, param->enumr_api_name);
+   devio_list_read_unlock();
+   return ODPDRV_DEVIO_INVALID;
+   }
+   devio = devio->next;
+   }
+   devio_list_read_unlock();
 
-   return ODPDRV_DEVIO_INVALID;
+   /* allocate memory for the new devio:
+* If init_global has not been done yet, then, we cannot allocate
+* from any _ishm pool (ishm has not even been initialised at this
+* stage...this happens when statically linked devios
+* register: their __constructor__ function is run before main()
+* is called). But any malloc performed here(before init_global)
+* will be inherited by any odpthreads (process or pthreads) as we
+* are still running in the ODP instantiation processes and all
+* other processes are guaranteed to be descendent of this one...
+* If init_global has been done, then we allocate from the _ishm pool
+* to guarantee visibility from any ODP thread.
+*/
+
+   if (init_global_statu

[lng-odp] [API-NEXT PATCHv2 12/23] linux-gen: drv: driver: adding device querry function

[Christophe Milard]

Implementation of the device query function for the linux-gen ODP.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/drv_driver.c | 37 +
 1 file changed, 37 insertions(+)

diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index 60edde1..3e4780b 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -391,6 +391,43 @@ static void device_destroy_terminate(odpdrv_device_t 
drv_device)
_odp_ishm_pool_free(list_elt_pool, device);
 }
 
+odpdrv_device_t *odpdrv_device_query(odpdrv_enumr_t enumr, const char *address)
+{
+   _odpdrv_device_t *dev;
+   odpdrv_device_t *res;
+   int index = 0;
+
+   int size = sizeof(odpdrv_device_t); /* for the ODPDRV_DEVICE_INVALID */
+
+   /* parse the list of device a first time to determine the size of
+* the memory to be allocated:
+*/
+   dev_list_read_lock();
+   dev = device_lst.head;
+   while (dev) {
+   if ((dev->param.enumerator == enumr) &&
+   ((address == NULL) ||
+(strcmp(dev->param.address, address) == 0)))
+   size += sizeof(odpdrv_device_t);
+   dev = dev->next;
+   }
+
+   /* then fill the list: */
+   res = (odpdrv_device_t *)malloc(size);
+   dev = device_lst.head;
+   while (dev) {
+   if ((dev->param.enumerator == enumr) &&
+   ((address == NULL) ||
+(strcmp(dev->param.address, address) == 0)))
+   res[index++] = (odpdrv_device_t)dev;
+   dev = dev->next;
+   }
+   dev_list_read_unlock();
+   res[index++] = ODPDRV_DEVICE_INVALID;
+
+   return res; /* must be freed by caller! */
+}
+
 odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t *param)
 {
ODP_ERR("NOT Supported yet! Driver %s Registration!\n.",
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 10/23] linux-gen: drv: device creation and deletion

[Christophe Milard]

Functions to create and remove devices are populated to do
more proper things.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/drv_driver.c | 181 ++--
 1 file changed, 173 insertions(+), 8 deletions(-)

diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index 4ade3c3..60edde1 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -19,12 +19,15 @@
 
 static enum {UNDONE, IN_PROGRESS, DONE} init_global_status;
 
+static void device_destroy_terminate(odpdrv_device_t device);
+
 /* pool from which different list elements are alocated: */
 #define ELT_POOL_SIZE (1 << 20)  /* 1Mb */
 static _odp_ishm_pool_t *list_elt_pool;
 
 typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
 typedef struct _odpdrv_enumr_s _odpdrv_enumr_t;
+typedef struct _odpdrv_device_s _odpdrv_device_t;
 
 /* an enumerator class (list element) */
 struct _odpdrv_enumr_class_s {
@@ -55,6 +58,20 @@ typedef struct _odpdrv_enumr_lst_t {
 } _odpdrv_enumr_lst_t;
 static struct _odpdrv_enumr_lst_t enumr_lst;
 
+/* a device (list element) */
+struct _odpdrv_device_s {
+   odpdrv_device_param_t param;
+   void (*enumr_destroy_callback)(void *enum_dev);/*dev destroy callback */
+   struct _odpdrv_device_s *next;
+} _odpdrv_device_s;
+
+/* the device list (all devices, from all enumerators): */
+typedef struct _odpdrv_device_lst_t {
+   odp_rwlock_recursive_t lock;
+   _odpdrv_device_t *head;
+} _odpdrv_device_lst_t;
+static struct _odpdrv_device_lst_t device_lst;
+
 /* some driver elements (such as enumeraor classes, drivers, devio) may
  * register before init_global and init_local complete. Mutex will fail
  * in this cases but should be used later on.
@@ -108,12 +125,46 @@ static void enumr_list_write_unlock(void)
odp_rwlock_recursive_write_unlock(_lst.lock);
 }
 
+static void dev_list_read_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_lock(_lst.lock);
+}
+
+static void dev_list_read_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_unlock(_lst.lock);
+}
+
+static void dev_list_write_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_lock(_lst.lock);
+}
+
+static void dev_list_write_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_unlock(_lst.lock);
+}
+
 /* some functions to get internal pointers from handles... */
 static inline _odpdrv_enumr_class_t *get_enumr_class(odpdrv_enumr_class_t 
class)
 {
return (_odpdrv_enumr_class_t *)(void *)class;
 }
 
+static inline _odpdrv_enumr_t *get_enumr(odpdrv_enumr_t enumr)
+{
+   return (_odpdrv_enumr_t *)(void *)enumr;
+}
+
+static inline _odpdrv_device_t *get_device(odpdrv_device_t dev)
+{
+   return (_odpdrv_device_t *)(void *)dev;
+}
+
 odpdrv_enumr_class_t odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
 *param)
 {
@@ -227,24 +278,119 @@ odpdrv_enumr_t 
odpdrv_enumr_register(odpdrv_enumr_param_t *param)
 
 odpdrv_device_t odpdrv_device_create(odpdrv_device_param_t *param)
 {
-   ODP_ERR("odpdrv_device_create not Supported yet! devaddress: %s\n.",
-   param->address);
-   return ODPDRV_DEVICE_INVALID;
+   _odpdrv_device_t *dev;
+
+   /* If init_global has not been done yet, we have a big issue. */
+   if (init_global_status == UNDONE)
+   return ODPDRV_DEVICE_INVALID;
+
+   /* make sure that the provided device address does not already exist: */
+   dev_list_read_lock();
+   dev = device_lst.head;
+   while (dev) {
+   if (strcmp(param->address, dev->param.address) == 0) {
+   ODP_ERR("device already exists!\n");
+   dev_list_read_unlock();
+   return ODPDRV_DEVICE_INVALID;
+   }
+   dev = dev->next;
+   }
+   dev_list_read_unlock();
+
+   dev = _odp_ishm_pool_alloc(list_elt_pool,
+  sizeof(_odpdrv_device_t));
+   if (!dev) {
+   ODP_ERR("_odp_ishm_pool_alloc failed!\n");
+   return ODPDRV_DEVICE_INVALID;
+   }
+
+   /* save and set dev init parameters and insert new device in list */
+   dev->param = *param;
+   dev->enumr_destroy_callback = NULL;
+   dev_list_write_lock();
+   dev->next = device_lst.head;
+   device_lst.head = dev;
+   dev_list_write_unlock();
+
+   /* todo: probe for drivers */
+
+   return (odpdrv_device_t)dev;
 }
 
 int odpdrv_device_destroy(odpdrv_device_t dev,
  void (*callback)(void *enum_dev), uint32_t flags)
 {
-   if (dev == ODPDRV_DEVICE_INVALID)
+   _odpdrv_dev

[lng-odp] [API-NEXT PATCHv2 11/23] drv: driver: adding device query function

[Christophe Milard]

Adding a function for querying a list of devices: this function may be
used by enumerators to query for the list of their registered devices
or for a subset of them.
Note that this function returns a malloc'd list of devices which is to
be freed by the caller.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 12 
 1 file changed, 12 insertions(+)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index 9643268..e861396 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -387,6 +387,18 @@ int odpdrv_device_destroy(odpdrv_device_t dev,
 /** The callback function must be called by the current ODP thread */
 #define ODPDRV_DEV_DESTROY_IMMEDIATE   0x0001
 
+/** query for a list of devices
+ * Enumerators are responsable for device creation and destruction.
+ * Upon request, ODP can build a list of devices belonging to a given 
enumerator
+ * and possibly having a specific address.
+ * This function builds this list.
+ * @param enumr The enumerator which created the device
+ * @param address The device address (or NULL if don't care)
+ * @return A malloc'd ODPDRV_DEVICE_INVALID terminated array of odpdrv_device_t
+ * This array MUST BE FREED by the caller!
+ */
+odpdrv_device_t *odpdrv_device_query(odpdrv_enumr_t enumr, const char 
*address);
+
 /**
 * Register an devio.
 * Each devio calls this function at init time.
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 08/23] drv: driver: change drv unbind function name and pass correct parameter

[Christophe Milard]

The driver removal function expects a device, of course...
Also unbind seems a better name to disconnect from a device
since remove has been used for removing the object itself for
enumerators.
Some extra parameters to allow for graceful unbinding are also added.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 21 ++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index b08d7fb..8ff856c 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -300,14 +300,29 @@ struct odpdrv_driver_param_t {
 */
int (*probe)(odpdrv_device_t *dev);
 
-   /** Remove function:
+   /** unbind function:
 * Only called with devices whose probe() returned true
 *
+* dev: the device to unbind
+* callback: if flag ODPDRV_DRV_UNBIND_IMMEDIATE is not specified,
+*  unbind should be attempted gracefully, meaning that some IO may need
+*  to terminate before the driver is really unbound from the device:
+*  In this case (when the flag is not set), the driver is due to call
+*  the callback function when the driver is unbound from the device.
+*  This callback may occurs within the unbind() call if the driver
+*  does unbind immediately.
+*  If the ODPDRV_DRV_UNBIND_IMMEDIATE is specified, the driver is due
+*  to release the device immediately (poosibly less gracefully).
+*  The callback must be called immediately in this case.
 */
-   int (*remove)(odpdrv_device_param_t *dev);
-
+   int (*unbind)(odpdrv_device_t dev,
+ void (*callback)(odpdrv_device_t dev),
+ uint32_t flags);
 };
 
+/** The callback function must be called mmediately by the current ODP thread 
*/
+#define ODPDRV_DRV_UNBIND_IMMEDIATE0x0001
+
 /**
 * Register an enumerator class.
 * Each enumerator class calls this function at init time.
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 09/23] drv: driver: add callback function for device destruction

[Christophe Milard]

When a device is destroyed by an enumerator, odpdrv_device_destroy() is
called.
However, the complete device destruction may require waiting for IO to be
completed: the device destruction is therefore divided in 2 steps:
odpdrv_device_destroy() starts the device destruction, and the provided
callback function is called when the device can be fully removed, i.e.
when it no longer has any driver bound to it.
An extra flag is also added to select the destruction type:
The default is a graceful destruction, letting the time for any attached
driver to terminate. This may imply that the callback function is called
from another ODP thread, later on.
ODPDRV_DEV_DESTROY_IMMEDIATE forces an immediate device destruction,
possibly terminating things abrubtly, but it guarantees that the
callback is performed by the same ODP thread. This is to be used at ODP
terminaison time.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h   | 31 +++
 platform/linux-generic/drv_driver.c |  9 -
 2 files changed, 35 insertions(+), 5 deletions(-)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index 8ff856c..9643268 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -357,12 +357,35 @@ odpdrv_device_t 
odpdrv_device_create(odpdrv_device_param_t *param);
 
 /**
 * Destroy a device
-* Called by each enumerator at probe time, or anytime later, for each
-* destroyed created device
+* Called by each enumerator after probe time, for each device to be
+* destroyed.
+* Destroying a device may require tearing down a driver and waiting for some IO
+* to terminate: The device destruction is therefore done in 2 steps:
+* Calling this function starts the device destruction: when the device has
+* no driver attached any longer, ODP calls the provided callback()
+* function  which should free the enumerator-allocated resources for
+* this device.
+* If the flag ODPDRV_DEV_DESTROY_IMMEDIATE is given, the device destruction
+* is immediate, i.e. the callback function is guaranteed to be called by the
+* same ODP thread: This might however not let the time for the bound driver
+* (if any) to terminate gracefully. This would typically be used at ODP
+* terminaison. By default, the callback may be called later, when the driver
+* has gracefully terminated, hence possibly from another ODP thread.
 * @param dev A odpdrv device handle as returned by odpdrv_device_create.
-* @return 0 on success or a negative value on error.
+* @param callback a pointer to a function to be called when the device is
+*freed (no more driver). The parameter to the callback function is
+*   the pointer to the enumerator specific part of the device as provided
+*   at device creation time (void *enum_dev). The callback function
+*   should release these resources.
+* @param flags 0 or ODPDRV_DEV_DESTROY_IMMEDIATE for immediate shut down
+* @return 0 on success or a negative value on error. On error, the callback
+* function is not called.
 */
-void odpdrv_device_destroy(odpdrv_device_t dev);
+int odpdrv_device_destroy(odpdrv_device_t dev,
+ void (*callback)(void *enum_dev), uint32_t flags);
+
+/** The callback function must be called by the current ODP thread */
+#define ODPDRV_DEV_DESTROY_IMMEDIATE   0x0001
 
 /**
 * Register an devio.
diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index 35473bd..4ade3c3 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -232,10 +232,17 @@ odpdrv_device_t 
odpdrv_device_create(odpdrv_device_param_t *param)
return ODPDRV_DEVICE_INVALID;
 }
 
-void odpdrv_device_destroy(odpdrv_device_t dev)
+int odpdrv_device_destroy(odpdrv_device_t dev,
+ void (*callback)(void *enum_dev), uint32_t flags)
 {
if (dev == ODPDRV_DEVICE_INVALID)
ODP_ERR("Invalid device\n");
+   if (callback != NULL)
+   ODP_ERR("Callback not supported yet\n");
+   if (flags != 0)
+   ODP_ERR("flags not supported yet\n");
+
+   return 0;
 }
 
 odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t *param)
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 07/23] test: drv: enumerator registration tests

[Christophe Milard]

making sure that enumerators are probed.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 .../validation/drv/drvdriver/.gitignore|   1 +
 .../validation/drv/drvdriver/Makefile.am   |  11 +
 .../validation/drv/drvdriver/drvdriver_enumr.c | 303 +
 .../validation/drv/drvdriver/drvdriver_enumr.h |  24 ++
 .../drv/drvdriver/drvdriver_enumr_main.c   |  12 +
 test/linux-generic/Makefile.am |   1 +
 6 files changed, 352 insertions(+)
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_enumr.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_enumr_main.c

diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore 
b/test/common_plat/validation/drv/drvdriver/.gitignore
index 9268315..a842448 100644
--- a/test/common_plat/validation/drv/drvdriver/.gitignore
+++ b/test/common_plat/validation/drv/drvdriver/.gitignore
@@ -1 +1,2 @@
 drvdriver_enumr_class_main
+drvdriver_enumr_main
diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am 
b/test/common_plat/validation/drv/drvdriver/Makefile.am
index 9e941ee..3476c50 100644
--- a/test/common_plat/validation/drv/drvdriver/Makefile.am
+++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
@@ -14,3 +14,14 @@ drvdriver_enumr_class_main_LDADD = 
libtestdrvdriverenumrclass.la \
   $(LIBCUNIT_COMMON) $(LIBODP)
 
 EXTRA_DIST = drvdriver_enumr_class.h
+
+#tests for enumerator registration:
+noinst_LTLIBRARIES += libtestdrvdriverenumr.la
+libtestdrvdriverenumr_la_SOURCES = drvdriver_enumr.c
+
+test_PROGRAMS += drvdriver_enumr_main$(EXEEXT)
+dist_drvdriver_enumr_main_SOURCES = drvdriver_enumr_main.c
+drvdriver_enumr_main_LDADD = libtestdrvdriverenumr.la \
+  $(LIBCUNIT_COMMON) $(LIBODP)
+
+EXTRA_DIST += drvdriver_enumr.h
diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c 
b/test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c
new file mode 100644
index 000..38aac3d
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c
@@ -0,0 +1,303 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include "drvdriver_enumr.h"
+#include 
+
+static odp_instance_t odp_instance;
+static odpdrv_enumr_class_t enumr_class1, enumr_class2;
+
+/* markers showing that different stages have been run */
+static int enumr1_probed;
+static int enumr2_probed;
+static int enumr3_probed;
+static int enumr4_probed;
+
+/* forward declaration */
+static int enumr1_probe(void);
+static int enumr2_probe(void);
+static int enumr3_probe(void);
+static int enumr4_probe(void);
+
+static int enumr1_remove(void);
+static int enumr2_remove(void);
+static int enumr3_remove(void);
+static int enumr4_remove(void);
+
+static int enumr_class1_probe(void);
+static int enumr_class2_probe(void);
+
+static int enumr_class1_remove(void);
+static int enumr_class2_remove(void);
+
+/* because many things to be checked are performed during ODP initialisation,
+ * the initialisation functions have to be a part of the test
+ */
+static int tests_global_init(void)
+{
+   if (0 != odp_init_global(_instance, NULL, NULL)) {
+   fprintf(stderr, "error: odp_init_global() failed.\n");
+   return -1;
+   }
+   if (0 != odp_init_local(odp_instance, ODP_THREAD_CONTROL)) {
+   fprintf(stderr, "error: odp_init_local() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+static int tests_global_term(void)
+{
+   if (0 != odp_term_local()) {
+   fprintf(stderr, "error: odp_term_local() failed.\n");
+   return -1;
+   }
+
+   if (0 != odp_term_global(odp_instance)) {
+   fprintf(stderr, "error: odp_term_global() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+/*enumerator register functions */
+static odpdrv_enumr_t enumr1_register(void)
+{
+   odpdrv_enumr_param_t param = {
+   .enumr_class = enumr_class1,
+   .api_name = "Enumerator_interface_1",
+   .api_version = 1,
+   .probe = enumr1_probe,
+   .remove = enumr1_remove,
+   .register_notifier = NULL
+   };
+
+   return odpdrv_enumr_register();
+}
+
+static odpdrv_enumr_t enumr2_register(void)
+{
+   odpdrv_enumr_param_t param = {
+   .enumr_class = enumr_class1,
+   .api_name = "Enumerator_interface_2",
+   .api_version = 1,
+   .probe = enumr2_probe,
+   .remove = enumr2_remove,
+   .register_notifier = NULL
+   };
+
+   retu

[lng-odp] [API-NEXT PATCHv2 06/23] linux-gen: drv: enumerator registration

[Christophe Milard]

The enumerator registration functions for the linux-gen ODP implementation.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/drv_driver.c | 139 +++-
 1 file changed, 136 insertions(+), 3 deletions(-)

diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index 50956a7..35473bd 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -24,6 +24,7 @@ static enum {UNDONE, IN_PROGRESS, DONE} init_global_status;
 static _odp_ishm_pool_t *list_elt_pool;
 
 typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
+typedef struct _odpdrv_enumr_s _odpdrv_enumr_t;
 
 /* an enumerator class (list element) */
 struct _odpdrv_enumr_class_s {
@@ -40,6 +41,20 @@ typedef struct _odpdrv_enumr_class_lst_t {
 } _odpdrv_enumr_class_lst_t;
 static struct _odpdrv_enumr_class_lst_t enumr_class_lst;
 
+/* an enumerator (list element) */
+struct _odpdrv_enumr_s {
+   odpdrv_enumr_param_t param;
+   int probed;
+   struct _odpdrv_enumr_s *next;
+};
+
+/* the enumerator list: */
+typedef struct _odpdrv_enumr_lst_t {
+   odp_rwlock_recursive_t lock;
+   _odpdrv_enumr_t *head;
+} _odpdrv_enumr_lst_t;
+static struct _odpdrv_enumr_lst_t enumr_lst;
+
 /* some driver elements (such as enumeraor classes, drivers, devio) may
  * register before init_global and init_local complete. Mutex will fail
  * in this cases but should be used later on.
@@ -69,6 +84,35 @@ static void enumr_class_list_write_unlock(void)
odp_rwlock_recursive_write_unlock(_class_lst.lock);
 }
 
+static void enumr_list_read_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_lock(_lst.lock);
+}
+
+static void enumr_list_read_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_unlock(_lst.lock);
+}
+
+static void enumr_list_write_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_lock(_lst.lock);
+}
+
+static void enumr_list_write_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_unlock(_lst.lock);
+}
+
+/* some functions to get internal pointers from handles... */
+static inline _odpdrv_enumr_class_t *get_enumr_class(odpdrv_enumr_class_t 
class)
+{
+   return (_odpdrv_enumr_class_t *)(void *)class;
+}
 
 odpdrv_enumr_class_t odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
 *param)
@@ -133,10 +177,52 @@ odpdrv_enumr_class_t 
odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
 
 odpdrv_enumr_t odpdrv_enumr_register(odpdrv_enumr_param_t *param)
 {
-   ODP_ERR("NOT Supported yet! Enumerator API %s Registration!\n.",
-   param->api_name);
+   _odpdrv_enumr_t *enumr;
+   _odpdrv_enumr_class_t *enumr_c;
+   int found_class = 0;
 
-   return ODPDRV_ENUMR_INVALID;
+   /* make sure that the provided enumerator_class does indeed exist: */
+   enumr_class_list_read_lock();
+   enumr_c = enumr_class_lst.head;
+   while (enumr_c) {
+   if (get_enumr_class(param->enumr_class) == enumr_c) {
+   found_class = 1;
+   break;
+   }
+   enumr_c = enumr_c->next;
+   }
+   enumr_class_list_read_unlock();
+   if (!found_class) {
+   ODP_ERR("invalid enumerator class provided!\n");
+   return ODPDRV_ENUMR_INVALID;
+   }
+
+   /* allocate memory for the new enumerator:
+* If init_global has not been done yet, we have a big issue,
+* as none of the enumerator classes should be probed before that!
+* We cannot even issue an error as ODP_* functions have not been
+* initialised yet, but this is no good...
+*/
+
+   if (init_global_status == UNDONE)
+   return ODPDRV_ENUMR_INVALID;
+
+   enumr = _odp_ishm_pool_alloc(list_elt_pool,
+sizeof(_odpdrv_enumr_t));
+   if (!enumr) {
+   ODP_ERR("_odp_ishm_pool_alloc failed!\n");
+   return ODPDRV_ENUMR_INVALID;
+   }
+
+   /* save init parameters and insert enumerator in list */
+   enumr->param = *param;
+   enumr->probed = 0;
+   enumr_list_write_lock();
+   enumr->next = enumr_lst.head;
+   enumr_lst.head = enumr;
+   enumr_list_write_unlock();
+
+   return (odpdrv_enumr_t)enumr;
 }
 
 odpdrv_device_t odpdrv_device_create(odpdrv_device_param_t *param)
@@ -174,6 +260,7 @@ odpdrv_driver_t 
odpdrv_driver_register(odpdrv_driver_param_t *param)
 void _odpdrv_driver_probe_drv_items(void)
 {
_odpdrv_enumr_class_t *enumr_c;
+   _odpdrv_enumr_t *enumr;
 
/* probe unprobed enumerators: */
enumr_class_list_read_lock();
@@ -186,

[lng-odp] [API-NEXT PATCHv2 05/23] test: drv: enumerator_class registration tests

[Christophe Milard]

Testing that enumerators classes can register properly.
Saddly restricted to statically linked enumerators classes, as testing with
modules in autotools seems to be an issue so far.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 test/common_plat/m4/configure.m4   |   1 +
 test/common_plat/validation/drv/Makefile.am|   1 +
 .../validation/drv/drvdriver/.gitignore|   1 +
 .../validation/drv/drvdriver/Makefile.am   |  16 ++
 .../drv/drvdriver/drvdriver_enumr_class.c  | 174 +
 .../drv/drvdriver/drvdriver_enumr_class.h  |  24 +++
 .../drv/drvdriver/drvdriver_enumr_class_main.c |  12 ++
 test/linux-generic/Makefile.am |   1 +
 8 files changed, 230 insertions(+)
 create mode 100644 test/common_plat/validation/drv/drvdriver/.gitignore
 create mode 100644 test/common_plat/validation/drv/drvdriver/Makefile.am
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.c
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class_main.c

diff --git a/test/common_plat/m4/configure.m4 b/test/common_plat/m4/configure.m4
index 13a13bd..400750c 100644
--- a/test/common_plat/m4/configure.m4
+++ b/test/common_plat/m4/configure.m4
@@ -34,4 +34,5 @@ AC_CONFIG_FILES([test/common_plat/Makefile
 test/common_plat/validation/api/traffic_mngr/Makefile
 test/common_plat/validation/drv/Makefile
 test/common_plat/validation/drv/drvatomic/Makefile
+test/common_plat/validation/drv/drvdriver/Makefile
 test/common_plat/validation/drv/drvshmem/Makefile])
diff --git a/test/common_plat/validation/drv/Makefile.am 
b/test/common_plat/validation/drv/Makefile.am
index bcdb92e..7329a89 100644
--- a/test/common_plat/validation/drv/Makefile.am
+++ b/test/common_plat/validation/drv/Makefile.am
@@ -1,4 +1,5 @@
 ODPDRV_MODULES = drvatomic \
+drvdriver \
 drvshmem
 
 SUBDIRS = $(ODPDRV_MODULES)
diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore 
b/test/common_plat/validation/drv/drvdriver/.gitignore
new file mode 100644
index 000..9268315
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/.gitignore
@@ -0,0 +1 @@
+drvdriver_enumr_class_main
diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am 
b/test/common_plat/validation/drv/drvdriver/Makefile.am
new file mode 100644
index 000..9e941ee
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
@@ -0,0 +1,16 @@
+include ../Makefile.inc
+
+# because most of driver activity occurs at init time, and due to the
+# fact that many sequential ODP runs are not allowed from the same process,
+# we need different binaries for each things being tested (as API init)
+
+#tests for enumerator class registration:
+noinst_LTLIBRARIES = libtestdrvdriverenumrclass.la
+libtestdrvdriverenumrclass_la_SOURCES = drvdriver_enumr_class.c
+
+test_PROGRAMS = drvdriver_enumr_class_main$(EXEEXT)
+dist_drvdriver_enumr_class_main_SOURCES = drvdriver_enumr_class_main.c
+drvdriver_enumr_class_main_LDADD = libtestdrvdriverenumrclass.la \
+  $(LIBCUNIT_COMMON) $(LIBODP)
+
+EXTRA_DIST = drvdriver_enumr_class.h
diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.c 
b/test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.c
new file mode 100644
index 000..f7dd42c
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.c
@@ -0,0 +1,174 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include "drvdriver_enumr_class.h"
+#include 
+
+static odp_instance_t odp_instance;
+
+static int enumr_class1_probed;
+static int enumr_class2_probed;
+
+/* forward declaration */
+static int enumr_class1_probe(void);
+static int enumr_class2_probe(void);
+
+static int enumr_class1_remove(void);
+static int enumr_class2_remove(void);
+
+/* because many things to be checked are performed during ODP initialisation,
+ * the initialisation functions have to be a part of the test
+ */
+static int tests_global_init(void)
+{
+   if (0 != odp_init_global(_instance, NULL, NULL)) {
+   fprintf(stderr, "error: odp_init_global() failed.\n");
+   return -1;
+   }
+   if (0 != odp_init_local(odp_instance, ODP_THREAD_CONTROL)) {
+   fprintf(stderr, "error: odp_init_local() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+static int tests_global_term(void)
+{
+   if (0 != odp_term_local()) {
+   fprintf(stderr, "error: odp_term_local() failed.\n");
+   return -1;
+   }
+

[lng-odp] [API-NEXT PATCHv2 04/23] linux-gen: drv: enumerator_class registration

[Christophe Milard]

The functions to register and probe enumerator classes are added.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/Makefile.am |   1 +
 platform/linux-generic/_modules.c  |   4 +
 platform/linux-generic/drv_driver.c| 212 -
 .../linux-generic/include/drv_driver_internal.h|  22 +++
 platform/linux-generic/include/odp_internal.h  |   5 +
 platform/linux-generic/odp_init.c  |  21 +-
 6 files changed, 260 insertions(+), 5 deletions(-)
 create mode 100644 platform/linux-generic/include/drv_driver_internal.h

diff --git a/platform/linux-generic/Makefile.am 
b/platform/linux-generic/Makefile.am
index deab284..02f13fe 100644
--- a/platform/linux-generic/Makefile.am
+++ b/platform/linux-generic/Makefile.am
@@ -133,6 +133,7 @@ noinst_HEADERS = \
  ${srcdir}/include/_ishm_internal.h \
  ${srcdir}/include/_ishmphy_internal.h \
  ${srcdir}/include/_ishmpool_internal.h \
+ ${srcdir}/include/drv_driver_internal.h\
  ${srcdir}/include/odp_align_internal.h \
  ${srcdir}/include/odp_atomic_internal.h \
  ${srcdir}/include/odp_buffer_inlines.h \
diff --git a/platform/linux-generic/_modules.c 
b/platform/linux-generic/_modules.c
index 6bb854e..b23c81f 100644
--- a/platform/linux-generic/_modules.c
+++ b/platform/linux-generic/_modules.c
@@ -9,6 +9,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -40,6 +41,9 @@ static int load_modules(void)
ODP_DBG("module %s loaded.\n", module_name);
}
 
+   /* give a chance top the driver interface to probe for new things: */
+   _odpdrv_driver_probe_drv_items();
+
return 0;
 }
 
diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index 529da48..50956a7 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -4,20 +4,131 @@
  * SPDX-License-Identifier: BSD-3-Clause
  */
 
+#include 
+
 #include 
+#include <_ishmpool_internal.h>
 
 #include 
 #include 
+#include 
 #include 
+#include 
 #include 
+#include 
+
+static enum {UNDONE, IN_PROGRESS, DONE} init_global_status;
+
+/* pool from which different list elements are alocated: */
+#define ELT_POOL_SIZE (1 << 20)  /* 1Mb */
+static _odp_ishm_pool_t *list_elt_pool;
+
+typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
+
+/* an enumerator class (list element) */
+struct _odpdrv_enumr_class_s {
+   odpdrv_enumr_class_param_t param;
+   int probed;
+   _odp_ishm_pool_t *pool;
+   struct _odpdrv_enumr_class_s *next;
+};
+
+/* the enumerator class list: */
+typedef struct _odpdrv_enumr_class_lst_t {
+   odp_rwlock_recursive_t lock;
+   _odpdrv_enumr_class_t *head;
+} _odpdrv_enumr_class_lst_t;
+static struct _odpdrv_enumr_class_lst_t enumr_class_lst;
+
+/* some driver elements (such as enumeraor classes, drivers, devio) may
+ * register before init_global and init_local complete. Mutex will fail
+ * in this cases but should be used later on.
+ * These functions disable the usage of Mutex while it is global init i.e.
+ * while single threaded*/
+static void enumr_class_list_read_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_lock(_class_lst.lock);
+}
+
+static void enumr_class_list_read_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_unlock(_class_lst.lock);
+}
+
+static void enumr_class_list_write_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_lock(_class_lst.lock);
+}
+
+static void enumr_class_list_write_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_unlock(_class_lst.lock);
+}
+
 
 odpdrv_enumr_class_t odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
 *param)
 {
-   ODP_ERR("NOT Supported yet! Enumerator Class %s Registration!\n.",
-   param->name);
+   _odpdrv_enumr_class_t *enumr_c;
 
-   return ODPDRV_ENUMR_CLASS_INVALID;
+   /* parse the list of already registered enumerator class to make
+* sure no enumerator with identical name already exists:
+*/
+   enumr_class_list_read_lock();
+   enumr_c = enumr_class_lst.head;
+   while (enumr_c) {
+   if (strncmp(param->name, enumr_c->param.name,
+   ODPDRV_NAME_SIZE) == 0) {
+   ODP_ERR("enumerator class %s already exists!\n",
+   param->name);
+   enumr_class_list_read_unlock();
+   return ODPDRV_ENUMR_CLASS_INVALID;
+   }
+   enumr_c = enumr_c->next;
+   }
+

[lng-odp] [API-NEXT PATCHv2 03/23] drv: making parameter strings dynamically computable

[Christophe Milard]

Declaring strings as const in the driver API prevents dynamic calculation
of these strings, which is a drawback. For instance,
the device addresses (string) are typically calculated by enumerators,
and should therefore not be const... Other strings may also be the result
of a computation. This change is made to allow this.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index d83e907..b08d7fb 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -170,7 +170,7 @@ struct odpdrv_enumr_class_param_t {
/** Enumerator name: mostly used for debug purpose.
 * Name must be unique (e.g. "PCI-DPAA2")
 */
-   const char name[ODPDRV_NAME_SIZE];
+   char name[ODPDRV_NAME_SIZE];
 
/** Probe function:
 * Called by ODP to get the enumerator class instances registered
@@ -198,7 +198,7 @@ struct odpdrv_enumr_param_t {
 * The format of the enum_dev part for the odpdrv_device_param_t
 * structure is identified by the api-name and version below
 */
-   const char api_name[ODPDRV_NAME_SIZE];
+   char api_name[ODPDRV_NAME_SIZE];
uint32_t api_version; /**<< the version of the provided API */
 
/** Probe function:
@@ -240,7 +240,7 @@ struct odpdrv_device_param_t {
 * e.g. ".23.12.1" for PCI domain 0, bus 23, device 12, function 1.
 * This string identifies the device uniquely.
 */
-   const char  address[ODPDRV_NAME_ADDR_SZ];
+   char address[ODPDRV_NAME_ADDR_SZ];
 
/** Enumerator dependent part
 * This part is allocated by the enumerator and is enumerator dependent
@@ -260,13 +260,13 @@ struct odpdrv_devio_param_t {
 * with same provided interface should refer to a common enumerator
 * class)
 */
-   const char api_name[ODPDRV_NAME_SIZE];
+   char api_name[ODPDRV_NAME_SIZE];
uint32_t api_version; /**<< the version of the provided API */
 
/** Enumerator interface name and version
 * The enumerator interface this devio needs.
 */
-   const char enumr_api_name[ODPDRV_NAME_SIZE];
+   char enumr_api_name[ODPDRV_NAME_SIZE];
uint32_t enumr_api_version; /**<< required enumerator API version */
 
/** Ops
@@ -283,14 +283,14 @@ struct odpdrv_driver_param_t {
 * The driver name (the pair {driver-name, enum-api-name} must
 * be unique)
 */
-   const char name[ODPDRV_NAME_SIZE];
+   char name[ODPDRV_NAME_SIZE];
 
/** Supported devios:
 * The list of supported devio: one of the following devio
 * (with correct version) must be available for the driver to work:
 */
struct {
-   const char api_name[ODPDRV_NAME_SIZE]; /**<< devio API name */
+   char api_name[ODPDRV_NAME_SIZE]; /**<< devio API name */
uint32_t   api_version; /**<< devio API version */
} devios[ODPDRV_MAX_DEVIOS];
 
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 02/23] linux-gen: adding compiler hints in the driver interface

[Christophe Milard]

Just including the spec file from the linux-generic side, as usual.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp_drv.h  |  1 +
 platform/linux-generic/include/odp/drv/hints.h | 34 ++
 2 files changed, 35 insertions(+)
 create mode 100644 platform/linux-generic/include/odp/drv/hints.h

diff --git a/include/odp_drv.h b/include/odp_drv.h
index 96d81ba..cdba226 100644
--- a/include/odp_drv.h
+++ b/include/odp_drv.h
@@ -24,6 +24,7 @@ extern C {
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
diff --git a/platform/linux-generic/include/odp/drv/hints.h 
b/platform/linux-generic/include/odp/drv/hints.h
new file mode 100644
index 000..808c22c
--- /dev/null
+++ b/platform/linux-generic/include/odp/drv/hints.h
@@ -0,0 +1,34 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+/**
+ * @file
+ *
+ * ODPDRV compiler hints
+ */
+
+#ifndef ODPDRV_PLAT_HINTS_H_
+#define ODPDRV_PLAT_HINTS_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/** @ingroup odpdrv_compiler_optim
+ *  @{
+ */
+
+/**
+ * @}
+ */
+
+#include 
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 00/23] driver items registration and probing

[Christophe Milard]

This patch series can be pulled from:
https://git.linaro.org/people/christophe.milard/odp.git/log/?h=drv_framework_v2

Since V1: Fixes following Bill's comments.

Note: I am not really sure this is still in phase with what was discussed
at connect, since I couldn't attend. But, at least I did the changes following
the comments I recieved. Hope that still makes sence.
Also, I am aware that patch 1 generates a warning: I copied this part from the
north API so I assume this is an agreed decision.

This patch series implements the driver interface, i.e.
enumerator class, enumerator, devio and drivers registration and probing.
This interface is depicted in:
https://docs.google.com/document/d/1eCKPJF6uSlOllXi_sKDvRwUD2BXm-ZzxZoKT0nVEsl4/edit
The associated tests are testing these mechanisms. Note that these tests
are testing staticaly linked modules only (hence avoiding the
module/platform/test debate). Also note that these tests are gathering
all the elements (enumerators, enumerator classes, devio, drivers) making
up the driver interface so as their interactions can be checked.
Real elements (pci enumerators, drivers...) will likely be written in a much
more stand-alone way.

Christophe Milard (23):
  drv: adding compiler hints in the driver interface
  linux-gen: adding compiler hints in the driver interface
  drv: making parameter strings dynamically computable
  linux-gen: drv: enumerator_class registration
  test: drv: enumerator_class registration tests
  linux-gen: drv: enumerator registration
  test: drv: enumerator registration tests
  drv: driver: change drv unbind function name and pass correct
parameter
  drv: driver: add callback function for device destruction
  linux-gen: drv: device creation and deletion
  drv: driver: adding device query function
  linux-gen: drv: driver: adding device querry function
  test: drv: device creation and destruction
  drv: driver: adding a probe and remove callback for devio
  linux-gen: drv: devio registration
  test: drv: devio creation and destruction
  drv: adding driver remove function
  drv: complement parameters to the driver probe() function
  linux-gen: driver registration and probing
  test: drv: driver registration and probing
  drv: driver: adding functions to attach driver's data to the device
  linux-gen: adding functions to attach driver's data to the device
  test: drv: test for setting and retrieving driver's data

 include/odp/drv/spec/driver.h  |  132 ++-
 include/odp/drv/spec/hints.h   |  119 +++
 include/odp_drv.h  |1 +
 platform/Makefile.inc  |1 +
 platform/linux-generic/Makefile.am |1 +
 platform/linux-generic/_modules.c  |4 +
 platform/linux-generic/drv_driver.c| 1051 +++-
 .../linux-generic/include/drv_driver_internal.h|   22 +
 platform/linux-generic/include/odp/drv/hints.h |   34 +
 platform/linux-generic/include/odp_internal.h  |5 +
 platform/linux-generic/odp_init.c  |   21 +-
 test/common_plat/m4/configure.m4   |1 +
 test/common_plat/validation/drv/Makefile.am|1 +
 .../validation/drv/drvdriver/.gitignore|5 +
 .../validation/drv/drvdriver/Makefile.am   |   60 ++
 .../validation/drv/drvdriver/drvdriver_device.c|  218 
 .../validation/drv/drvdriver/drvdriver_device.h|   24 +
 .../drv/drvdriver/drvdriver_device_main.c  |   12 +
 .../validation/drv/drvdriver/drvdriver_devio.c |  209 
 .../validation/drv/drvdriver/drvdriver_devio.h |   24 +
 .../drv/drvdriver/drvdriver_devio_main.c   |   12 +
 .../validation/drv/drvdriver/drvdriver_driver.c|  518 ++
 .../validation/drv/drvdriver/drvdriver_driver.h|   24 +
 .../drv/drvdriver/drvdriver_driver_main.c  |   12 +
 .../validation/drv/drvdriver/drvdriver_enumr.c |  303 ++
 .../validation/drv/drvdriver/drvdriver_enumr.h |   24 +
 .../drv/drvdriver/drvdriver_enumr_class.c  |  174 
 .../drv/drvdriver/drvdriver_enumr_class.h  |   24 +
 .../drv/drvdriver/drvdriver_enumr_class_main.c |   12 +
 .../drv/drvdriver/drvdriver_enumr_main.c   |   12 +
 test/linux-generic/Makefile.am |5 +
 31 files changed, 3030 insertions(+), 35 deletions(-)
 create mode 100644 include/odp/drv/spec/hints.h
 create mode 100644 platform/linux-generic/include/drv_driver_internal.h
 create mode 100644 platform/linux-generic/include/odp/drv/hints.h
 create mode 100644 test/common_plat/validation/drv/drvdriver/.gitignore
 create mode 100644 test/common_plat/validation/drv/drvdriver/Makefile.am
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_device.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_device.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_device_main.c

[lng-odp] [API-NEXT PATCHv2 01/23] drv: adding compiler hints in the driver interface

[Christophe Milard]

Largely inspired from its north api counterpart, the drv/spec/hint.h
file is added. Also includes the __constructor__ attributes that many
driver interface items will need.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/hints.h | 119 +++
 platform/Makefile.inc|   1 +
 2 files changed, 120 insertions(+)
 create mode 100644 include/odp/drv/spec/hints.h

diff --git a/include/odp/drv/spec/hints.h b/include/odp/drv/spec/hints.h
new file mode 100644
index 000..ddae22e
--- /dev/null
+++ b/include/odp/drv/spec/hints.h
@@ -0,0 +1,119 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+/**
+ * @file
+ *
+ * ODPDRV compiler hints
+ */
+
+#ifndef ODPDRV_API_HINTS_H_
+#define ODPDRV_API_HINTS_H_
+#include 
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/** @addtogroup odpdrv_compiler_optim
+ *  Macros that will give hints to the compiler.
+ *  @{
+ */
+
+#ifdef __GNUC__
+
+/** Define a function that should be run at early init (constructor)
+ */
+#define ODPDRV_CONSTRUCTOR __attribute__((__constructor__))
+
+/** Define a function that does not return
+ */
+#define ODPDRV_NORETURN __attribute__((__noreturn__))
+
+/** Define a weak symbol
+ * This is primarily useful in defining library functions that can be
+ * overridden in user code.
+ */
+#define ODPDRV_WEAK_SYMBOL __attribute__((__weak__))
+
+/**
+ * Hot code section
+ */
+#define ODPDRV_HOT_CODE__attribute__((__hot__))
+
+/**
+ * Cold code section
+ */
+#define ODPDRV_COLD_CODE   __attribute__((__cold__))
+
+/**
+ * Printf format attribute
+ */
+#define ODPDRV_PRINTF_FORMAT(x, y) __attribute__((format(printf, (x), (y
+
+/**
+ * Indicate deprecated variables, functions or types
+ */
+#define ODPDRV_DEPRECATED __attribute__((__deprecated__))
+
+/**
+ * Intentionally unused variables of functions
+ */
+#define ODPDRV_UNUSED __attribute__((__unused__))
+
+/**
+ * Branch likely taken
+ */
+#define odpdrv_likely(x)   __builtin_expect((x), 1)
+
+/**
+ * Branch unlikely taken
+ */
+#define odpdrv_unlikely(x) __builtin_expect((x), 0)
+
+/*
+ * __builtin_prefetch (const void *addr, rw, locality)
+ *
+ * rw 0..1   (0: read, 1: write)
+ * locality 0..3 (0: don't leave to cache, 3: leave on all cache levels)
+ */
+
+/**
+ * Cache prefetch address
+ */
+#define odpdrv_prefetch(x) __builtin_prefetch((x), 0, 3)
+
+/**
+ * Cache prefetch address for storing
+ */
+#define odpdrv_prefetch_store(x)   __builtin_prefetch((x), 1, 3)
+
+#else
+
+#define ODPDRV_CONSTRUCTOR
+#define ODPDRV_NORETURN
+#define ODPDRV_WEAK_SYMBOL
+#define ODPDRV_HOT_CODE
+#define ODPDRV_COLD_CODE
+#define ODPDRV_DEPRECATED
+#define ODPDRV_UNUSED
+#define odpdrv_likely(x)
+#define odpdrv_unlikely(x)
+#define odpdrv_prefetch(x)
+#define odpdrv_prefetch_store(x)
+
+#endif
+
+/**
+ * @}
+ */
+
+#ifdef __cplusplus
+}
+#endif
+
+#include 
+#endif
diff --git a/platform/Makefile.inc b/platform/Makefile.inc
index 874cf88..e439e3c 100644
--- a/platform/Makefile.inc
+++ b/platform/Makefile.inc
@@ -69,6 +69,7 @@ odpdrvspecinclude_HEADERS = \
  $(top_srcdir)/include/odp/drv/spec/byteorder.h \
  $(top_srcdir)/include/odp/drv/spec/compiler.h \
  $(top_srcdir)/include/odp/drv/spec/driver.h \
+ $(top_srcdir)/include/odp/drv/spec/hints.h \
  $(top_srcdir)/include/odp/drv/spec/shm.h \
  $(top_srcdir)/include/odp/drv/spec/spinlock.h \
  $(top_srcdir)/include/odp/drv/spec/std_types.h \
-- 
2.7.4

Re: [lng-odp] [API-NEXT PATCH 16/21] drv: complement parameters to the driver probe() function

[Christophe Milard]

On 23 February 2017 at 00:00, Bill Fischofer <bill.fischo...@linaro.org> wrote:
>
>
> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>>
>> Of course, when probing a driver, the latter should be given the devio
>> handle to be used. This is what this patch adds. It also adds the index
>> of the devio, hence telling the driver which of the possible
>> ODPDRV_MAX_DEVIOS devios was selected, as this is going to be the first
>> thing the driver needs to know.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  include/odp/drv/spec/driver.h | 7 +--
>>  1 file changed, 5 insertions(+), 2 deletions(-)
>>
>> diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
>> index 221a6ce..b3c9b76 100644
>> --- a/include/odp/drv/spec/driver.h
>> +++ b/include/odp/drv/spec/driver.h
>> @@ -314,9 +314,12 @@ struct odpdrv_driver_param_t {
>>
>> /** Probe function:
>>  * Called by ODP to see if the driver can drive a given device
>> -*
>> +* -dev is obviously the device we try to handle.
>
>
> I'd delete "obviously" here. Sounds a bit condescending in a spec. Perhaps
> just "-dev the device to be probed"

OK: => V2

Christophe

>
>>
>> +* -devio is the devio to be used.
>> +* -devio_idx actually tells which devio was selected: it is the
>> +* index in the devios array above.
>>  */
>> -   int (*probe)(odpdrv_device_t *dev);
>> +   int (*probe)(odpdrv_device_t dev, odpdrv_devio_t devio, int
>> devio_idx);
>>
>> /** unbind function:
>>  * Only called with devices whose probe() returned true
>> --
>> 2.7.4
>>
>

Re: [lng-odp] [API-NEXT PATCH 15/21] drv: adding driver remove function

[Christophe Milard]

On 22 February 2017 at 23:56, Bill Fischofer <bill.fischo...@linaro.org> wrote:
>
>
> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>>
>> The remove function, as for other driver items (such as enumeratos...) is
>
>
> Typo: enumerators

=> V2

>
>>
>> called before the driver is to be removed, i.e. after all devices have
>> been
>> been unboud from the driver. remove() should release any resource held
>
>
> unbound

=> V2

Christophe

>
>>
>> by the driver.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  include/odp/drv/spec/driver.h | 8 
>>  1 file changed, 8 insertions(+)
>>
>> diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
>> index 0b62c1b..221a6ce 100644
>> --- a/include/odp/drv/spec/driver.h
>> +++ b/include/odp/drv/spec/driver.h
>> @@ -336,6 +336,14 @@ struct odpdrv_driver_param_t {
>> int (*unbind)(odpdrv_device_t dev,
>>   void (*callback)(odpdrv_device_t dev),
>>   uint32_t flags);
>> +
>> +   /** remove function:
>> +* remove any resource taken by the driver. Called when the driver
>> +* itself is to be removed, i.e. after all devices are unbound
>> +* Can be set to NULL if the driver has nothing to release.
>> +*
>> +*/
>> +   int (*remove)(void);
>>  };
>>
>>  /** The callback function must be called mmediately by the current ODP
>> thread */
>> --
>> 2.7.4
>>
>

Re: [lng-odp] [API-NEXT PATCH 13/21] linux-gen: drv: devio registration

[Christophe Milard]

On 22 February 2017 at 23:54, Bill Fischofer <bill.fischo...@linaro.org> wrote:
>
>
> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>>
>> devios (dev IO) provide a interface for drivers to access a device:
>> Devices enumerated by enumerators may be accessed in by different
>> mechanisms (depending on iommu presence or other factors). This extra
>> abstraction is provided by devios, which provide a sets of methods to
>> access the devices of a given type (i.e. registred enumerator(s)
>> enumerating devices of the same kind (e.g. PCI)).
>> This patch just implements the devio registration method provided by the
>> driver API.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  platform/linux-generic/drv_driver.c | 134
>> +++-
>>  1 file changed, 131 insertions(+), 3 deletions(-)
>>
>> diff --git a/platform/linux-generic/drv_driver.c
>> b/platform/linux-generic/drv_driver.c
>> index 517a3c6..eb0dc48 100644
>> --- a/platform/linux-generic/drv_driver.c
>> +++ b/platform/linux-generic/drv_driver.c
>> @@ -28,6 +28,7 @@ static _odp_ishm_pool_t *list_elt_pool;
>>  typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
>>  typedef struct _odpdrv_enumr_s _odpdrv_enumr_t;
>>  typedef struct _odpdrv_device_s _odpdrv_device_t;
>> +typedef struct _odpdrv_devio_s _odpdrv_devio_t;
>>
>>  /* an enumerator class (list element) */
>>  struct _odpdrv_enumr_class_s {
>> @@ -72,6 +73,20 @@ typedef struct _odpdrv_device_lst_t {
>>  } _odpdrv_device_lst_t;
>>  static struct _odpdrv_device_lst_t device_lst;
>>
>> +/* a devio (list element) */
>> +struct _odpdrv_devio_s {
>> +   odpdrv_devio_param_t param;
>> +   _odp_ishm_pool_t *pool;
>> +   struct _odpdrv_devio_s *next;
>> +} _odpdrv_devio_s;
>> +
>> +/* the devio list: */
>> +typedef struct _odpdrv_devio_lst_t {
>> +   odp_rwlock_recursive_t lock;
>> +   _odpdrv_devio_t *head;
>> +} _odpdrv_devio_lst_t;
>> +static struct _odpdrv_devio_lst_t devio_lst;
>> +
>>  /* some driver elements (such as enumeraor classes, drivers, devio) may
>>   * register before init_global and init_local complete. Mutex will fail
>>   * in this cases but should be used later on.
>> @@ -149,6 +164,30 @@ static void dev_list_write_unlock(void)
>> odp_rwlock_recursive_write_unlock(_lst.lock);
>>  }
>>
>> +static void devio_list_read_lock(void)
>> +{
>> +   if (init_global_status == DONE)
>
>
> Same comment on the need for these guards.

same answer :-)

>
>>
>> +   odp_rwlock_recursive_read_lock(_lst.lock);
>> +}
>> +
>> +static void devio_list_read_unlock(void)
>> +{
>> +   if (init_global_status == DONE)
>> +   odp_rwlock_recursive_read_unlock(_lst.lock);
>> +}
>> +
>> +static void devio_list_write_lock(void)
>> +{
>> +   if (init_global_status == DONE)
>> +   odp_rwlock_recursive_write_lock(_lst.lock);
>> +}
>> +
>> +static void devio_list_write_unlock(void)
>> +{
>> +   if (init_global_status == DONE)
>> +   odp_rwlock_recursive_write_unlock(_lst.lock);
>> +}
>> +
>>  odpdrv_enumr_class_t
>> odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
>>  *param)
>>  {
>> @@ -415,10 +454,65 @@ odpdrv_device_t *odpdrv_device_query(odpdrv_enumr_t
>> enumr, const char *address)
>>
>>  odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t *param)
>>  {
>> -   ODP_ERR("NOT Supported yet! Driver %s Registration!\n.",
>> -   param->api_name);
>> +   _odpdrv_devio_t *devio;
>> +
>> +   /* parse the list of already registered devios to make
>> +* sure no devio providing the same interface using th esame
>> enumerator
>> +* already exists:
>> +*/
>> +   devio_list_read_lock();
>> +   devio = devio_lst.head;
>
>
> Same comment on needing to initialize the head field (should be done at init
> time along with the lock).

same answer :-)

>
>>
>> +   while (devio) {
>> +   if ((strncmp(param->api_name, devio->param.api_name,
>> +ODPDRV_NAME_SIZE) == 0) &&
>> +   (strncmp(param->enumr_api_name,
>> devio->param.enumr_api_name,
>> 

Re: [lng-odp] [API-NEXT PATCH 11/21] test: drv: device creation and destruction

[Christophe Milard]

On 22 February 2017 at 23:40, Bill Fischofer <bill.fischo...@linaro.org> wrote:
>
>
> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>>
>> Testing that devices can be created and removed from ODP.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  .../validation/drv/drvdriver/.gitignore|   1 +
>>  .../validation/drv/drvdriver/Makefile.am   |  11 ++
>>  .../validation/drv/drvdriver/drvdriver_device.c| 218
>> +
>>  .../validation/drv/drvdriver/drvdriver_device.h|  24 +++
>>  .../drv/drvdriver/drvdriver_device_main.c  |  12 ++
>>  test/linux-generic/Makefile.am |   1 +
>>  6 files changed, 267 insertions(+)
>>  create mode 100644
>> test/common_plat/validation/drv/drvdriver/drvdriver_device.c
>>  create mode 100644
>> test/common_plat/validation/drv/drvdriver/drvdriver_device.h
>>  create mode 100644
>> test/common_plat/validation/drv/drvdriver/drvdriver_device_main.c
>>
>> diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore
>> b/test/common_plat/validation/drv/drvdriver/.gitignore
>> index a842448..97b4312 100644
>> --- a/test/common_plat/validation/drv/drvdriver/.gitignore
>> +++ b/test/common_plat/validation/drv/drvdriver/.gitignore
>> @@ -1,2 +1,3 @@
>>  drvdriver_enumr_class_main
>>  drvdriver_enumr_main
>> +drvdriver_device_main
>> diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am
>> b/test/common_plat/validation/drv/drvdriver/Makefile.am
>> index 3476c50..544586c 100644
>> --- a/test/common_plat/validation/drv/drvdriver/Makefile.am
>> +++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
>> @@ -25,3 +25,14 @@ drvdriver_enumr_main_LDADD = libtestdrvdriverenumr.la \
>>$(LIBCUNIT_COMMON) $(LIBODP)
>>
>>  EXTRA_DIST += drvdriver_enumr.h
>> +
>> +#tests for device creation:
>> +noinst_LTLIBRARIES += libtestdrvdriverdevice.la
>> +libtestdrvdriverdevice_la_SOURCES = drvdriver_device.c
>> +
>> +test_PROGRAMS += drvdriver_device_main$(EXEEXT)
>> +dist_drvdriver_device_main_SOURCES = drvdriver_device_main.c
>> +drvdriver_device_main_LDADD = libtestdrvdriverdevice.la \
>> +  $(LIBCUNIT_COMMON) $(LIBODP)
>> +
>> +EXTRA_DIST += drvdriver_device.h
>> diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_device.c
>> b/test/common_plat/validation/drv/drvdriver/drvdriver_device.c
>> new file mode 100644
>> index 000..9254e17
>> --- /dev/null
>> +++ b/test/common_plat/validation/drv/drvdriver/drvdriver_device.c
>> @@ -0,0 +1,218 @@
>> +/* Copyright (c) 2017, Linaro Limited
>> + * All rights reserved.
>> + *
>> + * SPDX-License-Identifier: BSD-3-Clause
>> + */
>> +
>> +#include 
>> +#include 
>> +#include 
>> +#include "drvdriver_device.h"
>> +#include 
>> +
>> +static odp_instance_t odp_instance;
>> +static odpdrv_enumr_class_t enumr_class1;
>> +static odpdrv_enumr_t enumr1;
>> +
>> +typedef struct dev_enumr_data_t { /* enumerator data for registered
>> devices */
>> +   odpdrv_shm_tshm_handle;
>> +   int device_number;
>> +} dev_enumr_data_t;
>> +
>> +#define NB_DEVICES 5
>> +
>> +/* forward declaration */
>> +static int enumr1_probe(void);
>> +static int enumr1_remove(void);
>> +static int enumr_class1_probe(void);
>> +static int enumr_class1_remove(void);
>> +
>> +/* because many things to be checked are performed during ODP
>> initialisation,
>> + * the initialisation functions have to be a part of the test
>> + */
>> +static int tests_global_init(void)
>> +{
>> +   if (0 != odp_init_global(_instance, NULL, NULL)) {
>> +   fprintf(stderr, "error: odp_init_global() failed.\n");
>> +   return -1;
>> +   }
>> +   if (0 != odp_init_local(odp_instance, ODP_THREAD_CONTROL)) {
>> +   fprintf(stderr, "error: odp_init_local() failed.\n");
>> +   return -1;
>> +   }
>> +
>> +   return 0;
>> +}
>> +
>> +static int tests_global_term(void)
>> +{
>> +   if (0 != odp_term_local()) {
>> +   fprintf(stderr, "error: odp_term_local() failed.\n");
>> +   return -1;
>> +   }
>> +
>> +   if (0 != odp_term_globa

Re: [lng-odp] [API-NEXT PATCH 10/21] linux-gen: drv: driver: adding device querry function

[Christophe Milard]

hmmm,
Sure it is applicable here, but while most function using this
strategy would be happy to have some part of the job done, this one
won't:
for instance: receiving N packets always makes sense, even if M>N
packet are waiting to be received.
In this case, getting half of an anwer does not really make sense...
But I see your point regarding allocation.
Didn't we say that libC was always supported (hence malloc).
There is no need for speed here...

Christophe

On 22 February 2017 at 23:38, Bill Fischofer <bill.fischo...@linaro.org> wrote:
>
>
> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>>
>> Implementation of the device query function for the linux-gen ODP.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  platform/linux-generic/drv_driver.c | 37
>> +
>>  1 file changed, 37 insertions(+)
>>
>> diff --git a/platform/linux-generic/drv_driver.c
>> b/platform/linux-generic/drv_driver.c
>> index 48a90a2..517a3c6 100644
>> --- a/platform/linux-generic/drv_driver.c
>> +++ b/platform/linux-generic/drv_driver.c
>> @@ -376,6 +376,43 @@ static void device_destroy_terminate(odpdrv_device_t
>> drv_device)
>> _odp_ishm_pool_free(list_elt_pool, device);
>>  }
>>
>> +odpdrv_device_t *odpdrv_device_query(odpdrv_enumr_t enumr, const char
>> *address)
>> +{
>> +   _odpdrv_device_t *dev;
>> +   odpdrv_device_t *res;
>> +   int index = 0;
>> +
>> +   int size = sizeof(odpdrv_device_t); /* for the
>> ODPDRV_DEVICE_INVALID */
>> +
>> +   /* parse the list of device a first time to determine the size of
>> +* the memory to be allocated:
>> +*/
>> +   dev_list_read_lock();
>> +   dev = device_lst.head;
>> +   while (dev) {
>> +   if ((dev->param.enumerator == enumr) &&
>> +   ((address == NULL) ||
>> +(strcmp(dev->param.address, address) == 0)))
>> +   size += sizeof(odpdrv_device_t);
>> +   dev = dev->next;
>> +   }
>> +
>> +   /* then fill the list: */
>> +   res = (odpdrv_device_t *)malloc(size);
>> +   dev = device_lst.head;
>> +   while (dev) {
>> +   if ((dev->param.enumerator == enumr) &&
>> +   ((address == NULL) ||
>> +(strcmp(dev->param.address, address) == 0)))
>> +   res[index++] = (odpdrv_device_t)dev;
>> +   dev = dev->next;
>> +   }
>> +   dev_list_read_unlock();
>> +   res[index++] = ODPDRV_DEVICE_INVALID;
>> +
>> +   return res; /* must be freed by caller! */
>
>
> Most other ODP APIs that return a variable-number of return parameters have
> the caller supply a return array and size and the routine fills that in and
> returns the number of elements returned. Why is that model not suitable
> here? The concern about doing malloc() calls within ODP code is that
> constrains the caller as to the memory model being used. If the caller
> supplies the return array then it can obtain that memory from wherever it
> wishes.
>
>>
>> +}
>> +
>>  odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t *param)
>>  {
>> ODP_ERR("NOT Supported yet! Driver %s Registration!\n.",
>> --
>> 2.7.4
>>
>

Re: [lng-odp] [API-NEXT PATCH 08/21] linux-gen: drv: device creation and deletion

[Christophe Milard]

On 22 February 2017 at 23:28, Bill Fischofer <bill.fischo...@linaro.org> wrote:
>
>
> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>>
>> Functions to create and remove devices are populated to do
>> more proper things.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  platform/linux-generic/drv_driver.c | 172
>> ++--
>>  1 file changed, 164 insertions(+), 8 deletions(-)
>>
>> diff --git a/platform/linux-generic/drv_driver.c
>> b/platform/linux-generic/drv_driver.c
>> index f8844f5..48a90a2 100644
>> --- a/platform/linux-generic/drv_driver.c
>> +++ b/platform/linux-generic/drv_driver.c
>> @@ -19,12 +19,15 @@
>>
>>  static enum {UNDONE, IN_PROGRESS, DONE} init_global_status;
>>
>> +static void device_destroy_terminate(odpdrv_device_t device);
>> +
>>  /* pool from which different list elements are alocated: */
>>  #define ELT_POOL_SIZE (1 << 20)  /* 1Mb */
>>  static _odp_ishm_pool_t *list_elt_pool;
>>
>>  typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
>>  typedef struct _odpdrv_enumr_s _odpdrv_enumr_t;
>> +typedef struct _odpdrv_device_s _odpdrv_device_t;
>>
>>  /* an enumerator class (list element) */
>>  struct _odpdrv_enumr_class_s {
>> @@ -55,6 +58,20 @@ typedef struct _odpdrv_enumr_lst_t {
>>  } _odpdrv_enumr_lst_t;
>>  static struct _odpdrv_enumr_lst_t enumr_lst;
>>
>> +/* a device (list element) */
>> +struct _odpdrv_device_s {
>> +   odpdrv_device_param_t param;
>> +   void (*enumr_destroy_callback)(void *enum_dev);/*dev destroy
>> callback */
>> +   struct _odpdrv_device_s *next;
>> +} _odpdrv_device_s;
>> +
>> +/* the device list (all devices, from all enumerators): */
>> +typedef struct _odpdrv_device_lst_t {
>> +   odp_rwlock_recursive_t lock;
>> +   _odpdrv_device_t *head;
>> +} _odpdrv_device_lst_t;
>> +static struct _odpdrv_device_lst_t device_lst;
>> +
>>  /* some driver elements (such as enumeraor classes, drivers, devio) may
>>   * register before init_global and init_local complete. Mutex will fail
>>   * in this cases but should be used later on.
>> @@ -108,6 +125,30 @@ static void enumr_list_write_unlock(void)
>> odp_rwlock_recursive_write_unlock(_lst.lock);
>>  }
>>
>> +static void dev_list_read_lock(void)
>> +{
>> +   if (init_global_status == DONE)
>
>
> Same comment as for earlier locking routines regarding the necessity of
> these guards.

Hope my previous answer made that clear :-)

>
>>
>> +   odp_rwlock_recursive_read_lock(_lst.lock);
>> +}
>> +
>> +static void dev_list_read_unlock(void)
>> +{
>> +   if (init_global_status == DONE)
>> +   odp_rwlock_recursive_read_unlock(_lst.lock);
>> +}
>> +
>> +static void dev_list_write_lock(void)
>> +{
>> +   if (init_global_status == DONE)
>> +   odp_rwlock_recursive_write_lock(_lst.lock);
>> +}
>> +
>> +static void dev_list_write_unlock(void)
>> +{
>> +   if (init_global_status == DONE)
>> +   odp_rwlock_recursive_write_unlock(_lst.lock);
>> +}
>> +
>>  odpdrv_enumr_class_t
>> odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
>>  *param)
>>  {
>> @@ -222,24 +263,119 @@ odpdrv_enumr_t
>> odpdrv_enumr_register(odpdrv_enumr_param_t *param)
>>
>>  odpdrv_device_t odpdrv_device_create(odpdrv_device_param_t *param)
>>  {
>> -   ODP_ERR("odpdrv_device_create not Supported yet! devaddress:
>> %s\n.",
>> -   param->address);
>> -   return ODPDRV_DEVICE_INVALID;
>> +   _odpdrv_device_t *dev;
>> +
>> +   /* If init_global has not been done yet, we have a big issue. */
>> +   if (init_global_status == UNDONE)
>> +   return ODPDRV_DEVICE_INVALID;
>> +
>> +   /* make sure that the provided device address does not already
>> exist: */
>> +   dev_list_read_lock();
>> +   dev = device_lst.head;
>
>
> Same question as for other lists. Where is the head field initialized? This
> should be done in the init routine along with lock initialization.

hope my previous answer made that clear

Christophe

>
>>
>> +   while (dev) {
>> +   if (strcmp(param->address, dev->param.address) == 0) {
>> + 

Re: [lng-odp] [API-NEXT PATCH 07/21] drv: driver: add callback function for device destruction

[Christophe Milard]

On 22 February 2017 at 23:20, Bill Fischofer <bill.fischo...@linaro.org> wrote:
>
>
> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>>
>> When a device is destroyed by an enumerator, odpdrv_device_destroy() is
>> called.
>> However, the complete device destruction may require waiting for IO to be
>> completed: the device destruction is therefore divided in 2 steps:
>> odpdrv_device_destroy() starts the device destruction, and the provided
>> callback function is called when the device can be fully removed, i.e.
>> when it no longer has any driver bound to it.
>> An extra flag is also added to select the destruction type:
>> The default is a graceful destruction, letting the time for any attached
>> driver to terminate. This may imply that the callback function is called
>> from another ODP thread, later on.
>> ODPDRV_DEV_DESTROY_IMMEDIATE forces an immediate device destruction,
>> possibly terminating things abrubtly, but it guarantees that the
>> callback is performed by the same ODP thread. This is to be used at ODP
>> terminaison time.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  include/odp/drv/spec/driver.h   | 31 +++
>>  platform/linux-generic/drv_driver.c |  9 -
>>  2 files changed, 35 insertions(+), 5 deletions(-)
>>
>> diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
>> index a978d12..21a5fe1 100644
>> --- a/include/odp/drv/spec/driver.h
>> +++ b/include/odp/drv/spec/driver.h
>> @@ -357,12 +357,35 @@ odpdrv_device_t
>> odpdrv_device_create(odpdrv_device_param_t *param);
>>
>>  /**
>>  * Destroy a device
>> -* Called by each enumerator at probe time, or anytime later, for each
>> -* destroyed created device
>> +* Called by each enumerator after probe time, for each device to be
>> +* destroyed.
>> +* Destroying a device may require tearing down a driver and waiting for
>> some IO
>> +* to terminate: The device destruction is therefore done in 2 steps:
>> +* Calling this function starts the device destruction: when the device
>> has
>> +* no driver attached any longer, ODP calls the provided callback()
>> +* function  which should free the enumerator-allocated resources for
>> +* this device.
>> +* If the flag ODPDRV_DEV_DESTROY_IMMEDIATE is given, the device
>> destruction
>> +* is immediate, i.e. the callback function is guaranteed to be called by
>> the
>> +* same ODP thread: This might however not let the time for the bound
>> driver
>> +* (if any) to terminate gracefully. This would typically be used at ODP
>> +* terminaison. By default, the callback may be called later, when the
>> driver
>
>
> Typo: termination

=> V2

Christophe

>
>>
>> +* has gracefully terminated, hence possibly from another ODP thread.
>>  * @param dev A odpdrv device handle as returned by odpdrv_device_create.
>> -* @return 0 on success or a negative value on error.
>> +* @param callback a pointer to a function to be called when the device is
>> +*freed (no more driver). The parameter to the callback function
>> is
>> +*   the pointer to the enumerator specific part of the device as
>> provided
>> +*   at device creation time (void *enum_dev). The callback function
>> +*   should release these resources.
>> +* @param flags 0 or ODPDRV_DEV_DESTROY_IMMEDIATE for immediate shut down
>> +* @return 0 on success or a negative value on error. On error, the
>> callback
>> +* function is not called.
>>  */
>> -void odpdrv_device_destroy(odpdrv_device_t dev);
>> +int odpdrv_device_destroy(odpdrv_device_t dev,
>> + void (*callback)(void *enum_dev), uint32_t
>> flags);
>> +
>> +/** The callback function must be called by the current ODP thread */
>> +#define ODPDRV_DEV_DESTROY_IMMEDIATE   0x0001
>>
>>  /**
>>  * Register an devio.
>> diff --git a/platform/linux-generic/drv_driver.c
>> b/platform/linux-generic/drv_driver.c
>> index ee0a75c..f8844f5 100644
>> --- a/platform/linux-generic/drv_driver.c
>> +++ b/platform/linux-generic/drv_driver.c
>> @@ -227,10 +227,17 @@ odpdrv_device_t
>> odpdrv_device_create(odpdrv_device_param_t *param)
>> return ODPDRV_DEVICE_INVALID;
>>  }
>>
>> -void odpdrv_device_destroy(odpdrv_device_t dev)
>> +int odpdrv_device_destroy(odpdrv_device_t dev,
>> + void (*callback)(void *enum_dev), uint32_t
>> flags)
>>  {
>> if (dev == ODPDRV_DEVICE_INVALID)
>> ODP_ERR("Invalid device\n");
>> +   if (callback != NULL)
>> +   ODP_ERR("Callback not supported yet\n");
>> +   if (flags != 0)
>> +   ODP_ERR("flags not supported yet\n");
>> +
>> +   return 0;
>>  }
>>
>>  odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t *param)
>> --
>> 2.7.4
>>
>

Re: [lng-odp] [API-NEXT PATCH 06/21] drv: driver: change drv unbind function name and pass correct parameter

[Christophe Milard]

On 22 February 2017 at 23:14, Bill Fischofer <bill.fischo...@linaro.org> wrote:
>
>
> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>>
>> The driver removal function expects a device, of course...
>> Also unbind seems a better name to disconnect from a device
>> since remove has been used for removing the object itself for
>> enumerators.
>> Some extra parameters to allow for graceful unbinding are also added.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  include/odp/drv/spec/driver.h | 21 ++---
>>  1 file changed, 18 insertions(+), 3 deletions(-)
>>
>> diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
>> index b08d7fb..a978d12 100644
>> --- a/include/odp/drv/spec/driver.h
>> +++ b/include/odp/drv/spec/driver.h
>> @@ -300,14 +300,29 @@ struct odpdrv_driver_param_t {
>>  */
>> int (*probe)(odpdrv_device_t *dev);
>>
>> -   /** Remove function:
>> +   /** unbind function:
>>  * Only called with devices whose probe() returned true
>>  *
>> +* dev: the device to unbind
>> +* callback: if flag ODPDRV_DRV_UNBIND_IMMEDIATE is not specified,
>> +*  unbind should be attempted gracefuly, meaning that some IO may
>> need
>> +*  to terminate before the driver is really unbound from the
>> device:
>> +*  In this case (when the flag is not set), the driver is due to
>> call
>> +*  the callback function when the driver is unbound from the
>> device.
>> +*  This callback may occurs within the unbind() call if the
>> driver
>> +*  does unbind immediately.
>> +*  If the ODPDRV_DRV_UNBIND_IMMEDIATE is specified, the driver is
>> due
>> +*  to release the device immediately (poosibly less gracefuly).
>
>
> Typo: possibly less gracefully

Wonder why check-odp did not catch that...
=> V2

Christophe

>
>>
>> +*  The callback must be called immediately in this case.
>>  */
>> -   int (*remove)(odpdrv_device_param_t *dev);
>> -
>> +   int (*unbind)(odpdrv_device_t dev,
>> + void (*callback)(odpdrv_device_t dev),
>> + uint32_t flags);
>>  };
>>
>> +/** The callback function must be called mmediately by the current ODP
>> thread */
>> +#define ODPDRV_DRV_UNBIND_IMMEDIATE0x0001
>> +
>>  /**
>>  * Register an enumerator class.
>>  * Each enumerator class calls this function at init time.
>> --
>> 2.7.4
>>
>

Re: [lng-odp] [API-NEXT PATCH 05/21] test: drv: enumerator registration tests

[Christophe Milard]

will fix all these in V2,

Christophe.

On 22 February 2017 at 22:52, Bill Fischofer <bill.fischo...@linaro.org> wrote:
>
>
> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>>
>> making sure that enumerators are probed.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  .../validation/drv/drvdriver/.gitignore|   1 +
>>  .../validation/drv/drvdriver/Makefile.am   |  11 +
>>  .../validation/drv/drvdriver/drvdriver_enumr.c | 303
>> +
>>  .../validation/drv/drvdriver/drvdriver_enumr.h |  24 ++
>>  .../drv/drvdriver/drvdriver_enumr_main.c   |  12 +
>>  test/linux-generic/Makefile.am |   1 +
>>  6 files changed, 352 insertions(+)
>>  create mode 100644
>> test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c
>>  create mode 100644
>> test/common_plat/validation/drv/drvdriver/drvdriver_enumr.h
>>  create mode 100644
>> test/common_plat/validation/drv/drvdriver/drvdriver_enumr_main.c
>>
>> diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore
>> b/test/common_plat/validation/drv/drvdriver/.gitignore
>> index 9268315..a842448 100644
>> --- a/test/common_plat/validation/drv/drvdriver/.gitignore
>> +++ b/test/common_plat/validation/drv/drvdriver/.gitignore
>> @@ -1 +1,2 @@
>>  drvdriver_enumr_class_main
>> +drvdriver_enumr_main
>> diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am
>> b/test/common_plat/validation/drv/drvdriver/Makefile.am
>> index 9e941ee..3476c50 100644
>> --- a/test/common_plat/validation/drv/drvdriver/Makefile.am
>> +++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
>> @@ -14,3 +14,14 @@ drvdriver_enumr_class_main_LDADD =
>> libtestdrvdriverenumrclass.la \
>>$(LIBCUNIT_COMMON) $(LIBODP)
>>
>>  EXTRA_DIST = drvdriver_enumr_class.h
>> +
>> +#tests for enumerator registration:
>> +noinst_LTLIBRARIES += libtestdrvdriverenumr.la
>> +libtestdrvdriverenumr_la_SOURCES = drvdriver_enumr.c
>> +
>> +test_PROGRAMS += drvdriver_enumr_main$(EXEEXT)
>> +dist_drvdriver_enumr_main_SOURCES = drvdriver_enumr_main.c
>> +drvdriver_enumr_main_LDADD = libtestdrvdriverenumr.la \
>> +  $(LIBCUNIT_COMMON) $(LIBODP)
>> +
>> +EXTRA_DIST += drvdriver_enumr.h
>> diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c
>> b/test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c
>> new file mode 100644
>> index 000..cf844cf
>> --- /dev/null
>> +++ b/test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c
>> @@ -0,0 +1,303 @@
>> +/* Copyright (c) 2017, Linaro Limited
>> + * All rights reserved.
>> + *
>> + * SPDX-License-Identifier: BSD-3-Clause
>> + */
>> +
>> +#include 
>> +#include 
>> +#include 
>> +#include "drvdriver_enumr.h"
>> +#include 
>> +
>> +static odp_instance_t odp_instance;
>> +static odpdrv_enumr_class_t enumr_class1, enumr_class2;
>> +
>> +/* markers showing that different stages have been run */
>> +static int enumr1_probed;
>> +static int enumr2_probed;
>> +static int enumr3_probed;
>> +static int enumr4_probed;
>> +
>> +/* forward declaration */
>> +static int enumr1_probe(void);
>> +static int enumr2_probe(void);
>> +static int enumr3_probe(void);
>> +static int enumr4_probe(void);
>> +
>> +static int enumr1_remove(void);
>> +static int enumr2_remove(void);
>> +static int enumr3_remove(void);
>> +static int enumr4_remove(void);
>> +
>> +static int enumr_class1_probe(void);
>> +static int enumr_class2_probe(void);
>> +
>> +static int enumr_class1_remove(void);
>> +static int enumr_class2_remove(void);
>> +
>> +/* because many things to be checked are performed during ODP
>> initialisation,
>> + * the initialisation functions have to be a part of the test
>> + */
>> +static int tests_global_init(void)
>> +{
>> +   if (0 != odp_init_global(_instance, NULL, NULL)) {
>> +   fprintf(stderr, "error: odp_init_global() failed.\n");
>> +   return -1;
>> +   }
>> +   if (0 != odp_init_local(odp_instance, ODP_THREAD_CONTROL)) {
>> +   fprintf(stderr, "error: odp_init_local() failed.\n");
>> +   return -1;
>> +   }
>> +
>> +   return 0;
>> +}
>> 

Re: [lng-odp] [API-NEXT PATCH 04/21] linux-gen: drv: enumerator registration

[Christophe Milard]

On 22 February 2017 at 22:46, Bill Fischofer <bill.fischo...@linaro.org> wrote:
>
>
> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>>
>> The enumerator registration functions for the linux-gen ODP
>> implementation.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  platform/linux-generic/drv_driver.c | 135
>> +++-
>>  1 file changed, 132 insertions(+), 3 deletions(-)
>>
>> diff --git a/platform/linux-generic/drv_driver.c
>> b/platform/linux-generic/drv_driver.c
>> index 50956a7..ee0a75c 100644
>> --- a/platform/linux-generic/drv_driver.c
>> +++ b/platform/linux-generic/drv_driver.c
>> @@ -24,6 +24,7 @@ static enum {UNDONE, IN_PROGRESS, DONE}
>> init_global_status;
>>  static _odp_ishm_pool_t *list_elt_pool;
>>
>>  typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
>> +typedef struct _odpdrv_enumr_s _odpdrv_enumr_t;
>>
>>  /* an enumerator class (list element) */
>>  struct _odpdrv_enumr_class_s {
>> @@ -40,6 +41,20 @@ typedef struct _odpdrv_enumr_class_lst_t {
>>  } _odpdrv_enumr_class_lst_t;
>>  static struct _odpdrv_enumr_class_lst_t enumr_class_lst;
>>
>> +/* an enumerator (list element) */
>> +struct _odpdrv_enumr_s {
>> +   odpdrv_enumr_param_t param;
>> +   int probed;
>> +   struct _odpdrv_enumr_s *next;
>> +};
>> +
>> +/* the enumerator list: */
>> +typedef struct _odpdrv_enumr_lst_t {
>> +   odp_rwlock_recursive_t lock;
>> +   _odpdrv_enumr_t *head;
>> +} _odpdrv_enumr_lst_t;
>> +static struct _odpdrv_enumr_lst_t enumr_lst;
>> +
>>  /* some driver elements (such as enumeraor classes, drivers, devio) may
>>   * register before init_global and init_local complete. Mutex will fail
>>   * in this cases but should be used later on.
>> @@ -69,6 +84,29 @@ static void enumr_class_list_write_unlock(void)
>> odp_rwlock_recursive_write_unlock(_class_lst.lock);
>>  }
>>
>> +static void enumr_list_read_lock(void)
>> +{
>> +   if (init_global_status == DONE)
>> +   odp_rwlock_recursive_read_lock(_lst.lock);
>
>
> Same comments here as before with respect to the need for
> init_global_status. This seems unnecessarily complicated.

The problem is the folowing: there are two possible points (in time)
when a driver items (enumerator classes, drivers, devios...) may
register:
1) when ODP is running: this typically happens when a shared module is loaded
2) at early init, before main() is run, i.e. long before
odp_init_global(): this happens when the driver elements are
statically linked with ODP: their constructors are run before main().
This is a gcc construct. nothing we can do about it.
In case 1) we want to protect the lists again concurrent accesses.
in case 2) the ODP mutexes are not even initalised and won't work, but
we are monothreaded at thos time so we can ignore the mutex.

Hence this code

>
>>
>> +}
>> +
>> +static void enumr_list_read_unlock(void)
>> +{
>> +   if (init_global_status == DONE)
>> +   odp_rwlock_recursive_read_unlock(_lst.lock);
>> +}
>> +
>> +static void enumr_list_write_lock(void)
>> +{
>> +   if (init_global_status == DONE)
>> +   odp_rwlock_recursive_write_lock(_lst.lock);
>> +}
>> +
>> +static void enumr_list_write_unlock(void)
>> +{
>> +   if (init_global_status == DONE)
>> +   odp_rwlock_recursive_write_unlock(_lst.lock);
>> +}
>>
>>  odpdrv_enumr_class_t
>> odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
>>  *param)
>> @@ -133,10 +171,53 @@ odpdrv_enumr_class_t
>> odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
>>
>>  odpdrv_enumr_t odpdrv_enumr_register(odpdrv_enumr_param_t *param)
>>  {
>> -   ODP_ERR("NOT Supported yet! Enumerator API %s Registration!\n.",
>> -   param->api_name);
>> +   _odpdrv_enumr_t *enumr;
>> +   _odpdrv_enumr_class_t *enumr_c;
>> +   int found_class = 0;
>> +
>> +   /* make sure that the provided enumerator_class does indeed exist:
>> */
>> +   enumr_class_list_read_lock();
>> +   enumr_c = enumr_class_lst.head;
>
>
> Same question here regarding where is enumr_class_lst.head initialized? It
> seems we're referencing an uninitialized / stale value here. My guess is
> coverity would flag this.

The list heads are stat

Re: [lng-odp] [API-NEXT PATCH 03/21] test: drv: enumerator_class registration tests

[Christophe Milard]

On 22 February 2017 at 22:13, Bill Fischofer <bill.fischo...@linaro.org> wrote:
>
>
> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>>
>> Testing that enumerators classes can register properly.
>> Saddly restricted to statically linked enumerators classes, as testing
>> with
>> modules in autotools seems to be an issue so far.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  test/common_plat/m4/configure.m4   |   1 +
>>  test/common_plat/validation/drv/Makefile.am|   1 +
>>  .../validation/drv/drvdriver/.gitignore|   1 +
>>  .../validation/drv/drvdriver/Makefile.am   |  16 ++
>>  .../drv/drvdriver/drvdriver_enumr_class.c  | 174
>> +
>>  .../drv/drvdriver/drvdriver_enumr_class.h  |  24 +++
>>  .../drv/drvdriver/drvdriver_enumr_class_main.c |  12 ++
>>  test/linux-generic/Makefile.am |   1 +
>>  8 files changed, 230 insertions(+)
>>  create mode 100644 test/common_plat/validation/drv/drvdriver/.gitignore
>>  create mode 100644 test/common_plat/validation/drv/drvdriver/Makefile.am
>>  create mode 100644
>> test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.c
>>  create mode 100644
>> test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.h
>>  create mode 100644
>> test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class_main.c
>>
>> diff --git a/test/common_plat/m4/configure.m4
>> b/test/common_plat/m4/configure.m4
>> index 13a13bd..400750c 100644
>> --- a/test/common_plat/m4/configure.m4
>> +++ b/test/common_plat/m4/configure.m4
>> @@ -34,4 +34,5 @@ AC_CONFIG_FILES([test/common_plat/Makefile
>>  test/common_plat/validation/api/traffic_mngr/Makefile
>>  test/common_plat/validation/drv/Makefile
>>  test/common_plat/validation/drv/drvatomic/Makefile
>> +test/common_plat/validation/drv/drvdriver/Makefile
>>  test/common_plat/validation/drv/drvshmem/Makefile])
>> diff --git a/test/common_plat/validation/drv/Makefile.am
>> b/test/common_plat/validation/drv/Makefile.am
>> index bcdb92e..7329a89 100644
>> --- a/test/common_plat/validation/drv/Makefile.am
>> +++ b/test/common_plat/validation/drv/Makefile.am
>> @@ -1,4 +1,5 @@
>>  ODPDRV_MODULES = drvatomic \
>> +drvdriver \
>>  drvshmem
>>
>>  SUBDIRS = $(ODPDRV_MODULES)
>> diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore
>> b/test/common_plat/validation/drv/drvdriver/.gitignore
>> new file mode 100644
>> index 000..9268315
>> --- /dev/null
>> +++ b/test/common_plat/validation/drv/drvdriver/.gitignore
>> @@ -0,0 +1 @@
>> +drvdriver_enumr_class_main
>> diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am
>> b/test/common_plat/validation/drv/drvdriver/Makefile.am
>> new file mode 100644
>> index 000..9e941ee
>> --- /dev/null
>> +++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
>> @@ -0,0 +1,16 @@
>> +include ../Makefile.inc
>> +
>> +# because most of driver activity occurs at init time, and due to the
>> +# fact that many sequential ODP runs are not allowed from the same
>> process,
>> +# we need different binaries for each things being tested (as API init)
>
>
> I'm not sure I understand this comment. While this may be a good idea from a
> test modularity standpoint, the intent of ODP is that after
> odp_term_global() is called it's perfectly fine to call odp_init_global()
> again to start up another ODP instance. What we haven't (yet) worked out is
> the full ramifications of supporting multiple ODP instances on the same
> platform simultaneously, something we need to do as part of the Cloud
> profile for NFV.
>

I did call odp_init_global() after odp_term_global() when I worked on
the init test suite a while ago: most ODP implementation failed on
that. We eventually agreed that once a given process has call
odp_term_global() is is not allowed to "restart a new odp instance",
i.e. to call odp_init_global() again.
This is why the init test are 3 different binaries rather than one
simgle test suite.

>>
>> +
>> +#tests for enumerator class registration:
>> +noinst_LTLIBRARIES = libtestdrvdriverenumrclass.la
>> +libtestdrvdriverenumrclass_la_SOURCES = drvdriver_enumr_class.c
>> +
>> +test_PROGRAMS = drvdriver_enumr_class_main$(EXEEXT)
>> +dist_drvdriver_enumr_class_main_

Re: [lng-odp] [API-NEXT PATCH 00/21] driver items registration and probing

[Christophe Milard]

Hi all,

Maxim, are you saying you prefer a pull request?
Last time I did one, you required the patch to be sent on the list, so
I am not sure what to do next time.

Thx,

Christophe.

On 27 February 2017 at 10:15, Maxim Uvarov <maxim.uva...@linaro.org> wrote:
> for big serries of patches >20  it's good to provide some git link to
> download all of that. That Might be Linaro's people git or github.
>
> Maxim.
>
> On 25 February 2017 at 01:03, Mike Holmes <mike.hol...@linaro.org> wrote:
>
>> I did make a ticket for IT to look into this
>>
>> On 24 February 2017 at 16:55, Bill Fischofer <bill.fischo...@linaro.org>
>> wrote:
>>
>> > It looks like Christophe sent the original series to Mike, Forrest,
>> > Sachin, Yi, Me, and the odp mailing list. I suspect that's why Mike
>> > and I (and presumably the others on the direct mail list) have a
>> > complete copy of the series. A number of the parts didn't make it
>> > either to patchworks or the ODP mailing list archives, so I suspect
>> > some sort of IT glitch.
>> >
>> > On Fri, Feb 24, 2017 at 6:15 AM, Mike Holmes <mike.hol...@linaro.org>
>> > wrote:
>> > > Christophe will be back next week and we can check into where they
>> went.
>> > I
>> > > will ping Philip if I can prove I got them on CC and the list lost
>> them.
>> > >
>> > > On Feb 24, 2017 3:31 AM, "Josep Puigdemont" <
>> josep.puigdem...@linaro.org
>> > >
>> > > wrote:
>> > >>
>> > >> On Thu, Feb 23, 2017 at 10:51:27AM -0600, Bill Fischofer wrote:
>> > >> > I see all the patches in this series on the mailing list, but it
>> > appears
>> > >> > that the patchwork series[1] is missing parts 14 and 16.
>> > >>
>> > >> and part 20.
>> > >>
>> > >> I don't have parts 14, 16, and 20 on my inbox either, and it doesn't
>> > look
>> > >> like they ever reached the list, since they are missing from the
>> > >> archives too. Part 16/21 in the archives is a reply from you, Bill, so
>> > >> you definitely got the email (maybe you were cc'd?).
>> > >>
>> > >> >
>> > >> > ---
>> > >> > [1] http://patches.opendataplane.org/project/lng-odp/list/?
>> series=65
>> > >> >
>> > >> > On Thu, Feb 23, 2017 at 4:16 AM, Yi He <yi...@linaro.org> wrote:
>> > >> >
>> > >> > > In patchwork for this series I saw 14, 16, 21 are missing, do you
>> > see
>> > >> > > the
>> > >> > > same problem?
>> > >> > >
>> > >> > > Best Regards, Yi
>> > >> > >
>> > >> > > On 23 February 2017 at 07:32, Bill Fischofer
>> > >> > > <bill.fischo...@linaro.org>
>> > >> > > wrote:
>> > >> > >
>> > >> > >> This series compiles and runs/tests fine using gcc and clang on
>> > both
>> > >> > >> 64
>> > >> > >> and 32 bit systems. Many comments, mostly cosmetic, however some
>> > >> > >> missing
>> > >> > >> field initializations are noted as well as some suggestions for
>> > >> > >> handling
>> > >> > >> type conversions.
>> > >> > >>
>> > >> > >> On Wed, Feb 22, 2017 at 6:55 AM, Christophe Milard <
>> > >> > >> christophe.mil...@linaro.org> wrote:
>> > >> > >>
>> > >> > >>> This patch series implements the driver interface, i.e.
>> > >> > >>> enumerator class, enumerator, devio and drivers registration and
>> > >> > >>> probing.
>> > >> > >>> This interface is depicted in:
>> > >> > >>> https://docs.google.com/document/d/1eCKPJF6uSlOllXi_sKDvRwUD
>> > >> > >>> 2BXm-ZzxZoKT0nVEsl4/edit
>> > >> > >>> The associated tests are testing these mechanisms. Note that
>> these
>> > >> > >>> tests
>> > >> > >>> are testing staticaly linked modules only (hence avoiding the
>> > >> > >>> module/platform/test debate). Also note that these tests are
>> > >> > >>> gathering
>> > >

[lng-odp] [API-NEXT PATCH 21/21] test: drv: test for setting and retrieving driver's data

[Christophe Milard]

trivial tests for function odpdrv_device_set_data() and
odpdrv_device_get_data().

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 test/common_plat/validation/drv/drvdriver/drvdriver_driver.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c 
b/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
index c293b06..4143acd 100644
--- a/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
+++ b/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
@@ -380,6 +380,8 @@ static int driver1_probe(odpdrv_device_t dev, 
odpdrv_devio_t devio, int idx)
if (dev == E1_devs[i]) {
driver1_probed_index |= (1 << i);
dev_found = 1;
+   /* just set dev index as driver data */
+   odpdrv_device_set_data(dev, (void *)(uintptr_t)i);
}
}
CU_ASSERT(dev_found);
@@ -430,6 +432,7 @@ static int driver1_unbind(odpdrv_device_t dev,
  void (*callback)(odpdrv_device_t dev),
  uint32_t flags)
 {
+   CU_ASSERT(E1_devs[(uintptr_t)odpdrv_device_get_data(dev)] == dev);
CU_ASSERT(dev != ODPDRV_DEVICE_INVALID);
CU_ASSERT(flags == ODPDRV_DRV_UNBIND_IMMEDIATE);
callback(dev);
-- 
2.7.4

[lng-odp] [API-NEXT PATCH 19/21] drv: driver: adding functions to attach driver's data to the device

[Christophe Milard]

Driver will need to attach their data to devices when bound.
The patch introduce a data setter and a data getter function to do so.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 21 +
 1 file changed, 21 insertions(+)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index b3c9b76..50ed3a0 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -450,6 +450,27 @@ odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t 
*param);
 odpdrv_driver_t odpdrv_driver_register(odpdrv_driver_param_t *param);
 
 /**
+* Sets the device driver data, i.e. the driver data which should be attached to
+* the device.
+* After a driver is bound to a device, this driver will need to keep
+* data attached to this device. This data is, of course, driver dependent.
+*
+* @param dev: the device to which data should be attached.
+* @param data Pointer to whatever thre driver want to keep attached to the
+* device
+*/
+void odpdrv_device_set_data(odpdrv_device_t dev, void *data);
+
+/**
+* Gets the device driver data, i.e. the driver data which should be attached to
+* the device.
+* Retrieve the pointer which was set with odpdrv_device_set_data()
+* @param dev: the device from which the driver data should be retrieved.
+* @return the driver data pointer (as set by odpdrv_device_set_data()) or NULL.
+*/
+void *odpdrv_device_get_data(odpdrv_device_t dev);
+
+/**
 * Print (ODP_DBG) the driver interface status (debug).
 *
 * @return 0 on success, less than zero on error (inconsistency detected)
-- 
2.7.4

[lng-odp] [API-NEXT PATCH 18/21] test: drv: driver registration and probing

[Christophe Milard]

Register driver, devios, enumerators, create devices, and check
that probing occurs correctely.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 .../validation/drv/drvdriver/.gitignore|   1 +
 .../validation/drv/drvdriver/Makefile.am   |  11 +
 .../validation/drv/drvdriver/drvdriver_driver.c| 515 +
 .../validation/drv/drvdriver/drvdriver_driver.h|  24 +
 .../drv/drvdriver/drvdriver_driver_main.c  |  12 +
 test/linux-generic/Makefile.am |   1 +
 6 files changed, 564 insertions(+)
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_driver.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_driver_main.c

diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore 
b/test/common_plat/validation/drv/drvdriver/.gitignore
index 829c8b4..76bb6ba 100644
--- a/test/common_plat/validation/drv/drvdriver/.gitignore
+++ b/test/common_plat/validation/drv/drvdriver/.gitignore
@@ -2,3 +2,4 @@ drvdriver_enumr_class_main
 drvdriver_enumr_main
 drvdriver_device_main
 drvdriver_devio_main
+drvdriver_driver_main
diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am 
b/test/common_plat/validation/drv/drvdriver/Makefile.am
index 8e695ba..88bd828 100644
--- a/test/common_plat/validation/drv/drvdriver/Makefile.am
+++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
@@ -47,3 +47,14 @@ drvdriver_devio_main_LDADD = libtestdrvdriverdevio.la \
   $(LIBCUNIT_COMMON) $(LIBODP)
 
 EXTRA_DIST += drvdriver_devio.h
+
+#tests for driver registration and probing:
+noinst_LTLIBRARIES += libtestdrvdriverdriver.la
+libtestdrvdriverdriver_la_SOURCES = drvdriver_driver.c
+
+test_PROGRAMS += drvdriver_driver_main$(EXEEXT)
+dist_drvdriver_driver_main_SOURCES = drvdriver_driver_main.c
+drvdriver_driver_main_LDADD = libtestdrvdriverdriver.la \
+  $(LIBCUNIT_COMMON) $(LIBODP)
+
+EXTRA_DIST += drvdriver_driver.h
diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c 
b/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
new file mode 100644
index 000..c293b06
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
@@ -0,0 +1,515 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+/* This file is a bit long as it tries to simulate the presence of 2
+ * enumerator classes, 2 enumerators, and 3 drivers in one go to
+ * see how things are handled by the driver framework.
+ * The following is done:
+ * - create 2 enumerator classes,
+ * - each with its own enumerator providing interfaces: E1 and E2.
+ * - E1 and E2 create 4 devices each.
+ * - the following devio are created:
+ *   devio1 enabling device handling from DRVIF-1 to E1
+ *   devio2 enabling device handling from DRVIF-2 to E2
+ *   devio3 enabling device handling from DRVIF-3 to E2
+ *   devio4 enabling device handling from DRVIF-3 to E3 (does not exist)
+ *   devio5 enabling device handling from DRVIF-4 to E3 (does not exist)
+ *
+ * -then the following driver are created:
+ *  driver1, requiring devio DRVIF-1
+ *  driver2, requiring devio DRVIF-2 (preferred) and DRVIF-3
+ *  driver3, requiring devio DRVIF-4
+ *
+ * The test amkes sure that:
+ * driver 1 is probed (and accepts) the 4 devices of E1 with devio1
+ * driver 2 is probed (and rejects) the 4 devices of E2 with devio2 and devio4
+ * driver3 is never probed.
+ */
+
+#include 
+#include 
+#include 
+#include "drvdriver_driver.h"
+#include 
+
+static odp_instance_t odp_instance;
+static odpdrv_enumr_class_t enumr_class1;
+static odpdrv_enumr_class_t enumr_class2;
+static odpdrv_enumr_t enumr1;
+static odpdrv_enumr_t enumr2;
+#define NB_DEVICES 4
+static odpdrv_device_t E1_devs[NB_DEVICES];
+static odpdrv_device_t E2_devs[NB_DEVICES];
+static odpdrv_devio_t devio1;
+static odpdrv_devio_t devio2;
+static odpdrv_devio_t devio3;
+static odpdrv_devio_t devio4;
+static odpdrv_devio_t devio5;
+static odpdrv_driver_t driver1;
+static odpdrv_driver_t driver2;
+static odpdrv_driver_t driver3;
+
+static int driver1_probed_index;
+static int driver2_probed_index;
+
+/* forward declaration */
+static int enumr1_probe(void);
+static int enumr2_probe(void);
+static int enumr1_remove(void);
+static int enumr2_remove(void);
+static int enumr_class1_probe(void);
+static int enumr_class2_probe(void);
+static int driver1_probe(odpdrv_device_t dev, odpdrv_devio_t devio, int idx);
+static int driver2_probe(odpdrv_device_t dev, odpdrv_devio_t devio, int idx);
+static int driver3_probe(odpdrv_device_t dev, odpdrv_devio_t devio, int idx);
+static int driver1_unbind(odpdrv_device_t dev,
+ void (*callback)(odpdrv_device_t dev),
+ uint32_t flags);
+static in

[lng-odp] [API-NEXT PATCH 17/21] linux-gen: driver registration and probing

[Christophe Milard]

Driver registration and probing is implemented for linux-gen ODP.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/drv_driver.c | 348 ++--
 1 file changed, 335 insertions(+), 13 deletions(-)

diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index eb0dc48..0148fc3 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -12,6 +12,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -29,6 +30,11 @@ typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
 typedef struct _odpdrv_enumr_s _odpdrv_enumr_t;
 typedef struct _odpdrv_device_s _odpdrv_device_t;
 typedef struct _odpdrv_devio_s _odpdrv_devio_t;
+typedef struct _odpdrv_driver_s _odpdrv_driver_t;
+
+static int unbind_device_driver(_odpdrv_device_t *dev,
+   void (*callback)(odpdrv_device_t odpdrv_dev),
+   int immediate);
 
 /* an enumerator class (list element) */
 struct _odpdrv_enumr_class_s {
@@ -62,6 +68,8 @@ static struct _odpdrv_enumr_lst_t enumr_lst;
 /* a device (list element) */
 struct _odpdrv_device_s {
odpdrv_device_param_t param;
+   _odpdrv_driver_t *driver; /* driver for the device (if bound), or NULL*/
+   _odpdrv_devio_t *devio;   /* devio used for device (if bound), or NULL*/
void (*enumr_destroy_callback)(void *enum_dev);/*dev destroy callback */
struct _odpdrv_device_s *next;
 } _odpdrv_device_s;
@@ -87,6 +95,21 @@ typedef struct _odpdrv_devio_lst_t {
 } _odpdrv_devio_lst_t;
 static struct _odpdrv_devio_lst_t devio_lst;
 
+/* a driver (list element) */
+struct _odpdrv_driver_s {
+   odpdrv_driver_param_t param;
+   _odp_ishm_pool_t *pool;
+   odp_spinlock_t probelock; /* to avoid concurrent probe on the same drv*/
+   struct _odpdrv_driver_s *next;
+};
+
+/* the driver list: */
+typedef struct _odpdrv_driver_lst_t {
+   odp_rwlock_recursive_t lock;
+   _odpdrv_driver_t *head;
+} _odpdrv_driver_lst_t;
+static struct _odpdrv_driver_lst_t driver_lst;
+
 /* some driver elements (such as enumeraor classes, drivers, devio) may
  * register before init_global and init_local complete. Mutex will fail
  * in this cases but should be used later on.
@@ -188,6 +211,30 @@ static void devio_list_write_unlock(void)
odp_rwlock_recursive_write_unlock(_lst.lock);
 }
 
+static void driver_list_read_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_lock(_lst.lock);
+}
+
+static void driver_list_read_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_unlock(_lst.lock);
+}
+
+static void driver_list_write_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_lock(_lst.lock);
+}
+
+static void driver_list_write_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_unlock(_lst.lock);
+}
+
 odpdrv_enumr_class_t odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
 *param)
 {
@@ -331,6 +378,8 @@ odpdrv_device_t odpdrv_device_create(odpdrv_device_param_t 
*param)
/* save and set dev init parameters and insert new device in list */
dev->param = *param;
dev->enumr_destroy_callback = NULL;
+   dev->driver = NULL;
+   dev->devio = NULL;
dev_list_write_lock();
dev->next = device_lst.head;
device_lst.head = dev;
@@ -384,19 +433,17 @@ int odpdrv_device_destroy(odpdrv_device_t dev,
 */
target->enumr_destroy_callback = callback;
 
-   /* TODO: if a driver is bound to the device, unbind it!
-* passing the flag andf device_destroy_terminate() as a callback */
-
-   /* no driver is handling this device, or no callback was
-* provided: continue removing the device: */
-   device_destroy_terminate(dev);
+   /* unbind the driver from the device (if bound).
+* The callback is always called. */
+   unbind_device_driver(target,
+device_destroy_terminate,
+(flags & ODPDRV_DEV_DESTROY_IMMEDIATE));
 
return 0;
 }
 
 /* This function is called as a callback from the driver, when unbindind
- * a device, or directely from odpdrv_device_destroy() if no driver
- * was bound to the device.
+ * a device drom odpdrv_device_destroy()
  * just call the enumerator callback to cleanup the enumerator part
  * and free device memory */
 static void device_destroy_terminate(odpdrv_device_t drv_device)
@@ -517,10 +564,238 @@ odpdrv_devio_t 
odpdrv_devio_register(odpdrv_devio_param_t *param)
 
 odpdrv_driver_t odpdrv_driver_register(odpdrv_driver_param_t *param)
 {
-   ODP_ERR("NOT Supported yet! Driver %s Registration!\n.",
-   

[lng-odp] [API-NEXT PATCH 13/21] linux-gen: drv: devio registration

[Christophe Milard]

devios (dev IO) provide a interface for drivers to access a device:
Devices enumerated by enumerators may be accessed in by different
mechanisms (depending on iommu presence or other factors). This extra
abstraction is provided by devios, which provide a sets of methods to
access the devices of a given type (i.e. registred enumerator(s)
enumerating devices of the same kind (e.g. PCI)).
This patch just implements the devio registration method provided by the
driver API.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/drv_driver.c | 134 +++-
 1 file changed, 131 insertions(+), 3 deletions(-)

diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index 517a3c6..eb0dc48 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -28,6 +28,7 @@ static _odp_ishm_pool_t *list_elt_pool;
 typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
 typedef struct _odpdrv_enumr_s _odpdrv_enumr_t;
 typedef struct _odpdrv_device_s _odpdrv_device_t;
+typedef struct _odpdrv_devio_s _odpdrv_devio_t;
 
 /* an enumerator class (list element) */
 struct _odpdrv_enumr_class_s {
@@ -72,6 +73,20 @@ typedef struct _odpdrv_device_lst_t {
 } _odpdrv_device_lst_t;
 static struct _odpdrv_device_lst_t device_lst;
 
+/* a devio (list element) */
+struct _odpdrv_devio_s {
+   odpdrv_devio_param_t param;
+   _odp_ishm_pool_t *pool;
+   struct _odpdrv_devio_s *next;
+} _odpdrv_devio_s;
+
+/* the devio list: */
+typedef struct _odpdrv_devio_lst_t {
+   odp_rwlock_recursive_t lock;
+   _odpdrv_devio_t *head;
+} _odpdrv_devio_lst_t;
+static struct _odpdrv_devio_lst_t devio_lst;
+
 /* some driver elements (such as enumeraor classes, drivers, devio) may
  * register before init_global and init_local complete. Mutex will fail
  * in this cases but should be used later on.
@@ -149,6 +164,30 @@ static void dev_list_write_unlock(void)
odp_rwlock_recursive_write_unlock(_lst.lock);
 }
 
+static void devio_list_read_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_lock(_lst.lock);
+}
+
+static void devio_list_read_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_unlock(_lst.lock);
+}
+
+static void devio_list_write_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_lock(_lst.lock);
+}
+
+static void devio_list_write_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_unlock(_lst.lock);
+}
+
 odpdrv_enumr_class_t odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
 *param)
 {
@@ -415,10 +454,65 @@ odpdrv_device_t *odpdrv_device_query(odpdrv_enumr_t 
enumr, const char *address)
 
 odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t *param)
 {
-   ODP_ERR("NOT Supported yet! Driver %s Registration!\n.",
-   param->api_name);
+   _odpdrv_devio_t *devio;
+
+   /* parse the list of already registered devios to make
+* sure no devio providing the same interface using th esame enumerator
+* already exists:
+*/
+   devio_list_read_lock();
+   devio = devio_lst.head;
+   while (devio) {
+   if ((strncmp(param->api_name, devio->param.api_name,
+ODPDRV_NAME_SIZE) == 0) &&
+   (strncmp(param->enumr_api_name, devio->param.enumr_api_name,
+ODPDRV_NAME_SIZE) == 0)) {
+   ODP_ERR("a devio providing interface '%s' for devices "
+   "of type '%s' is already registered\n!",
+   param->api_name, param->enumr_api_name);
+   devio_list_read_unlock();
+   return ODPDRV_DEVIO_INVALID;
+   }
+   devio = devio->next;
+   }
+   devio_list_read_unlock();
 
-   return ODPDRV_DEVIO_INVALID;
+   /* allocate memory for the new devio:
+* If init_global has not been done yet, then, we cannot allocate
+* from any _ishm pool (ishm has not even been initialised at this
+* stage...this happens when statically linked devios
+* register: their __constructor__ function is run before main()
+* is called). But any malloc performed here(before init_global)
+* will be inherited by any odpthreads (process or pthreads) as we
+* are still running in the ODP instantiation processes and all
+* other processes are guaranteed to be descendent of this one...
+* If init_global has been done, then we allocate from the _ishm pool
+* to guarantee visibility from any ODP thread.
+*/
+
+   if (init_global_statu

[lng-odp] [API-NEXT PATCH 15/21] drv: adding driver remove function

[Christophe Milard]

The remove function, as for other driver items (such as enumeratos...) is
called before the driver is to be removed, i.e. after all devices have been
been unboud from the driver. remove() should release any resource held
by the driver.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 8 
 1 file changed, 8 insertions(+)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index 0b62c1b..221a6ce 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -336,6 +336,14 @@ struct odpdrv_driver_param_t {
int (*unbind)(odpdrv_device_t dev,
  void (*callback)(odpdrv_device_t dev),
  uint32_t flags);
+
+   /** remove function:
+* remove any resource taken by the driver. Called when the driver
+* itself is to be removed, i.e. after all devices are unbound
+* Can be set to NULL if the driver has nothing to release.
+*
+*/
+   int (*remove)(void);
 };
 
 /** The callback function must be called mmediately by the current ODP thread 
*/
-- 
2.7.4

[lng-odp] [API-NEXT PATCH 12/21] drv: driver: adding a probe and remove callback for devio

[Christophe Milard]

Needed to delete the resources needed for the devio. That is possibly
the memory allocated for its "ops" part if it was allocated. May be NULL
if nothing needs to be done at devio deletion time.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 18 ++
 1 file changed, 18 insertions(+)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index abac138..0b62c1b 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -269,6 +269,24 @@ struct odpdrv_devio_param_t {
char enumr_api_name[ODPDRV_NAME_SIZE];
uint32_t enumr_api_version; /**<< required enumerator API version */
 
+   /** Probe function:
+* Tell whether this devio can handle the given device.
+* The devio is hence given a chance to reject a given device for
+* any reason. No binding occurs here. binding occurs when the
+* driver is probed.
+* returns 0 if this devio can handle the given device, or a negative
+* value if not.
+* If left to NULL, a 0 returned value is assumed
+*/
+   int (*probe)(odpdrv_device_t dev);
+
+   /** Remove function:
+* Should destroy the memory allocated for ops and anything else
+* under it, or any resource for this devio.
+* Returns 0 on success or a negative value on error.
+*/
+   int (*remove)(void);
+
/** Ops
 * Pointer to a devio ops structure (specific to each devio)
 */
-- 
2.7.4

[lng-odp] [API-NEXT PATCH 11/21] test: drv: device creation and destruction

[Christophe Milard]

Testing that devices can be created and removed from ODP.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 .../validation/drv/drvdriver/.gitignore|   1 +
 .../validation/drv/drvdriver/Makefile.am   |  11 ++
 .../validation/drv/drvdriver/drvdriver_device.c| 218 +
 .../validation/drv/drvdriver/drvdriver_device.h|  24 +++
 .../drv/drvdriver/drvdriver_device_main.c  |  12 ++
 test/linux-generic/Makefile.am |   1 +
 6 files changed, 267 insertions(+)
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_device.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_device.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_device_main.c

diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore 
b/test/common_plat/validation/drv/drvdriver/.gitignore
index a842448..97b4312 100644
--- a/test/common_plat/validation/drv/drvdriver/.gitignore
+++ b/test/common_plat/validation/drv/drvdriver/.gitignore
@@ -1,2 +1,3 @@
 drvdriver_enumr_class_main
 drvdriver_enumr_main
+drvdriver_device_main
diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am 
b/test/common_plat/validation/drv/drvdriver/Makefile.am
index 3476c50..544586c 100644
--- a/test/common_plat/validation/drv/drvdriver/Makefile.am
+++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
@@ -25,3 +25,14 @@ drvdriver_enumr_main_LDADD = libtestdrvdriverenumr.la \
   $(LIBCUNIT_COMMON) $(LIBODP)
 
 EXTRA_DIST += drvdriver_enumr.h
+
+#tests for device creation:
+noinst_LTLIBRARIES += libtestdrvdriverdevice.la
+libtestdrvdriverdevice_la_SOURCES = drvdriver_device.c
+
+test_PROGRAMS += drvdriver_device_main$(EXEEXT)
+dist_drvdriver_device_main_SOURCES = drvdriver_device_main.c
+drvdriver_device_main_LDADD = libtestdrvdriverdevice.la \
+  $(LIBCUNIT_COMMON) $(LIBODP)
+
+EXTRA_DIST += drvdriver_device.h
diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_device.c 
b/test/common_plat/validation/drv/drvdriver/drvdriver_device.c
new file mode 100644
index 000..9254e17
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/drvdriver_device.c
@@ -0,0 +1,218 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include "drvdriver_device.h"
+#include 
+
+static odp_instance_t odp_instance;
+static odpdrv_enumr_class_t enumr_class1;
+static odpdrv_enumr_t enumr1;
+
+typedef struct dev_enumr_data_t { /* enumerator data for registered devices */
+   odpdrv_shm_tshm_handle;
+   int device_number;
+} dev_enumr_data_t;
+
+#define NB_DEVICES 5
+
+/* forward declaration */
+static int enumr1_probe(void);
+static int enumr1_remove(void);
+static int enumr_class1_probe(void);
+static int enumr_class1_remove(void);
+
+/* because many things to be checked are performed during ODP initialisation,
+ * the initialisation functions have to be a part of the test
+ */
+static int tests_global_init(void)
+{
+   if (0 != odp_init_global(_instance, NULL, NULL)) {
+   fprintf(stderr, "error: odp_init_global() failed.\n");
+   return -1;
+   }
+   if (0 != odp_init_local(odp_instance, ODP_THREAD_CONTROL)) {
+   fprintf(stderr, "error: odp_init_local() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+static int tests_global_term(void)
+{
+   if (0 != odp_term_local()) {
+   fprintf(stderr, "error: odp_term_local() failed.\n");
+   return -1;
+   }
+
+   if (0 != odp_term_global(odp_instance)) {
+   fprintf(stderr, "error: odp_term_global() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+/*enumerator register functions */
+static odpdrv_enumr_t enumr1_register(void)
+{
+   odpdrv_enumr_param_t param = {
+   .enumr_class = enumr_class1,
+   .api_name = "Enumerator_interface_1",
+   .api_version = 1,
+   .probe = enumr1_probe,
+   .remove = enumr1_remove,
+   .register_notifier = NULL
+   };
+
+   enumr1 = odpdrv_enumr_register();
+   return enumr1;
+}
+
+/*enumerator probe functions, just making sure they have been ran: */
+static int enumr1_probe(void)
+{
+   int dev;
+   odpdrv_shm_t shm;
+   dev_enumr_data_t *dev_data;
+
+   odpdrv_device_param_t param = {
+   .enumerator = enumr1,
+   .address = "00:00:0X",
+   .enum_dev = NULL
+   };
+
+   /* create 5 devices: */
+   for (dev = 0; dev < NB_DEVICES; dev++) {
+   shm = odpdrv_shm_reserve(NULL, sizeof(dev_enumr_data_t),
+  

[lng-odp] [API-NEXT PATCH 10/21] linux-gen: drv: driver: adding device querry function

[Christophe Milard]

Implementation of the device query function for the linux-gen ODP.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/drv_driver.c | 37 +
 1 file changed, 37 insertions(+)

diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index 48a90a2..517a3c6 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -376,6 +376,43 @@ static void device_destroy_terminate(odpdrv_device_t 
drv_device)
_odp_ishm_pool_free(list_elt_pool, device);
 }
 
+odpdrv_device_t *odpdrv_device_query(odpdrv_enumr_t enumr, const char *address)
+{
+   _odpdrv_device_t *dev;
+   odpdrv_device_t *res;
+   int index = 0;
+
+   int size = sizeof(odpdrv_device_t); /* for the ODPDRV_DEVICE_INVALID */
+
+   /* parse the list of device a first time to determine the size of
+* the memory to be allocated:
+*/
+   dev_list_read_lock();
+   dev = device_lst.head;
+   while (dev) {
+   if ((dev->param.enumerator == enumr) &&
+   ((address == NULL) ||
+(strcmp(dev->param.address, address) == 0)))
+   size += sizeof(odpdrv_device_t);
+   dev = dev->next;
+   }
+
+   /* then fill the list: */
+   res = (odpdrv_device_t *)malloc(size);
+   dev = device_lst.head;
+   while (dev) {
+   if ((dev->param.enumerator == enumr) &&
+   ((address == NULL) ||
+(strcmp(dev->param.address, address) == 0)))
+   res[index++] = (odpdrv_device_t)dev;
+   dev = dev->next;
+   }
+   dev_list_read_unlock();
+   res[index++] = ODPDRV_DEVICE_INVALID;
+
+   return res; /* must be freed by caller! */
+}
+
 odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t *param)
 {
ODP_ERR("NOT Supported yet! Driver %s Registration!\n.",
-- 
2.7.4

[lng-odp] [API-NEXT PATCH 08/21] linux-gen: drv: device creation and deletion

[Christophe Milard]

Functions to create and remove devices are populated to do
more proper things.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/drv_driver.c | 172 ++--
 1 file changed, 164 insertions(+), 8 deletions(-)

diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index f8844f5..48a90a2 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -19,12 +19,15 @@
 
 static enum {UNDONE, IN_PROGRESS, DONE} init_global_status;
 
+static void device_destroy_terminate(odpdrv_device_t device);
+
 /* pool from which different list elements are alocated: */
 #define ELT_POOL_SIZE (1 << 20)  /* 1Mb */
 static _odp_ishm_pool_t *list_elt_pool;
 
 typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
 typedef struct _odpdrv_enumr_s _odpdrv_enumr_t;
+typedef struct _odpdrv_device_s _odpdrv_device_t;
 
 /* an enumerator class (list element) */
 struct _odpdrv_enumr_class_s {
@@ -55,6 +58,20 @@ typedef struct _odpdrv_enumr_lst_t {
 } _odpdrv_enumr_lst_t;
 static struct _odpdrv_enumr_lst_t enumr_lst;
 
+/* a device (list element) */
+struct _odpdrv_device_s {
+   odpdrv_device_param_t param;
+   void (*enumr_destroy_callback)(void *enum_dev);/*dev destroy callback */
+   struct _odpdrv_device_s *next;
+} _odpdrv_device_s;
+
+/* the device list (all devices, from all enumerators): */
+typedef struct _odpdrv_device_lst_t {
+   odp_rwlock_recursive_t lock;
+   _odpdrv_device_t *head;
+} _odpdrv_device_lst_t;
+static struct _odpdrv_device_lst_t device_lst;
+
 /* some driver elements (such as enumeraor classes, drivers, devio) may
  * register before init_global and init_local complete. Mutex will fail
  * in this cases but should be used later on.
@@ -108,6 +125,30 @@ static void enumr_list_write_unlock(void)
odp_rwlock_recursive_write_unlock(_lst.lock);
 }
 
+static void dev_list_read_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_lock(_lst.lock);
+}
+
+static void dev_list_read_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_unlock(_lst.lock);
+}
+
+static void dev_list_write_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_lock(_lst.lock);
+}
+
+static void dev_list_write_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_unlock(_lst.lock);
+}
+
 odpdrv_enumr_class_t odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
 *param)
 {
@@ -222,24 +263,119 @@ odpdrv_enumr_t 
odpdrv_enumr_register(odpdrv_enumr_param_t *param)
 
 odpdrv_device_t odpdrv_device_create(odpdrv_device_param_t *param)
 {
-   ODP_ERR("odpdrv_device_create not Supported yet! devaddress: %s\n.",
-   param->address);
-   return ODPDRV_DEVICE_INVALID;
+   _odpdrv_device_t *dev;
+
+   /* If init_global has not been done yet, we have a big issue. */
+   if (init_global_status == UNDONE)
+   return ODPDRV_DEVICE_INVALID;
+
+   /* make sure that the provided device address does not already exist: */
+   dev_list_read_lock();
+   dev = device_lst.head;
+   while (dev) {
+   if (strcmp(param->address, dev->param.address) == 0) {
+   ODP_ERR("device already exists!\n");
+   dev_list_read_unlock();
+   return ODPDRV_DEVICE_INVALID;
+   }
+   dev = dev->next;
+   }
+   dev_list_read_unlock();
+
+   dev = _odp_ishm_pool_alloc(list_elt_pool,
+  sizeof(_odpdrv_device_t));
+   if (!dev) {
+   ODP_ERR("_odp_ishm_pool_alloc failed!\n");
+   return ODPDRV_DEVICE_INVALID;
+   }
+
+   /* save and set dev init parameters and insert new device in list */
+   dev->param = *param;
+   dev->enumr_destroy_callback = NULL;
+   dev_list_write_lock();
+   dev->next = device_lst.head;
+   device_lst.head = dev;
+   dev_list_write_unlock();
+
+   /* todo: probe for drivers */
+
+   return (odpdrv_device_t)dev;
 }
 
 int odpdrv_device_destroy(odpdrv_device_t dev,
  void (*callback)(void *enum_dev), uint32_t flags)
 {
-   if (dev == ODPDRV_DEVICE_INVALID)
+   _odpdrv_device_t *device = (_odpdrv_device_t *)(void *)dev;
+   _odpdrv_device_t *_dev;
+   _odpdrv_device_t *target = NULL;
+
+   if (dev == ODPDRV_DEVICE_INVALID) {
ODP_ERR("Invalid device\n");
-   if (callback != NULL)
-   ODP_ERR("Callback not supported yet\n");
-   if (flags != 0)
-   ODP_ERR("flags not supported yet\n");
+   ret

[lng-odp] [API-NEXT PATCH 09/21] drv: driver: adding device query function

[Christophe Milard]

Adding a function for querying a list of devices: this function may be
used by enumerators to query for the list of their registered devices
or for a subset of them.
Note that this function returns a malloc'd list of devices which is to
be freed by the caller.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 12 
 1 file changed, 12 insertions(+)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index 21a5fe1..abac138 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -387,6 +387,18 @@ int odpdrv_device_destroy(odpdrv_device_t dev,
 /** The callback function must be called by the current ODP thread */
 #define ODPDRV_DEV_DESTROY_IMMEDIATE   0x0001
 
+/** query for a list of devices
+ * Enumerators are responsable for device creation and destruction.
+ * Upon request, ODP can build a list of devices belonging to a given 
enumerator
+ * and possibly having a specific address.
+ * This function builds this list.
+ * @param enumr The enumerator which created the device
+ * @param address The device address (or NULL if don't care)
+ * @return A malloc'd ODPDRV_DEVICE_INVALID terminated array of odpdrv_device_t
+ * This array MUST BE FREED by the caller!
+ */
+odpdrv_device_t *odpdrv_device_query(odpdrv_enumr_t enumr, const char 
*address);
+
 /**
 * Register an devio.
 * Each devio calls this function at init time.
-- 
2.7.4

[lng-odp] [API-NEXT PATCH 05/21] test: drv: enumerator registration tests

[Christophe Milard]

making sure that enumerators are probed.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 .../validation/drv/drvdriver/.gitignore|   1 +
 .../validation/drv/drvdriver/Makefile.am   |  11 +
 .../validation/drv/drvdriver/drvdriver_enumr.c | 303 +
 .../validation/drv/drvdriver/drvdriver_enumr.h |  24 ++
 .../drv/drvdriver/drvdriver_enumr_main.c   |  12 +
 test/linux-generic/Makefile.am |   1 +
 6 files changed, 352 insertions(+)
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_enumr.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_enumr_main.c

diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore 
b/test/common_plat/validation/drv/drvdriver/.gitignore
index 9268315..a842448 100644
--- a/test/common_plat/validation/drv/drvdriver/.gitignore
+++ b/test/common_plat/validation/drv/drvdriver/.gitignore
@@ -1 +1,2 @@
 drvdriver_enumr_class_main
+drvdriver_enumr_main
diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am 
b/test/common_plat/validation/drv/drvdriver/Makefile.am
index 9e941ee..3476c50 100644
--- a/test/common_plat/validation/drv/drvdriver/Makefile.am
+++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
@@ -14,3 +14,14 @@ drvdriver_enumr_class_main_LDADD = 
libtestdrvdriverenumrclass.la \
   $(LIBCUNIT_COMMON) $(LIBODP)
 
 EXTRA_DIST = drvdriver_enumr_class.h
+
+#tests for enumerator registration:
+noinst_LTLIBRARIES += libtestdrvdriverenumr.la
+libtestdrvdriverenumr_la_SOURCES = drvdriver_enumr.c
+
+test_PROGRAMS += drvdriver_enumr_main$(EXEEXT)
+dist_drvdriver_enumr_main_SOURCES = drvdriver_enumr_main.c
+drvdriver_enumr_main_LDADD = libtestdrvdriverenumr.la \
+  $(LIBCUNIT_COMMON) $(LIBODP)
+
+EXTRA_DIST += drvdriver_enumr.h
diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c 
b/test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c
new file mode 100644
index 000..cf844cf
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c
@@ -0,0 +1,303 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include "drvdriver_enumr.h"
+#include 
+
+static odp_instance_t odp_instance;
+static odpdrv_enumr_class_t enumr_class1, enumr_class2;
+
+/* markers showing that different stages have been run */
+static int enumr1_probed;
+static int enumr2_probed;
+static int enumr3_probed;
+static int enumr4_probed;
+
+/* forward declaration */
+static int enumr1_probe(void);
+static int enumr2_probe(void);
+static int enumr3_probe(void);
+static int enumr4_probe(void);
+
+static int enumr1_remove(void);
+static int enumr2_remove(void);
+static int enumr3_remove(void);
+static int enumr4_remove(void);
+
+static int enumr_class1_probe(void);
+static int enumr_class2_probe(void);
+
+static int enumr_class1_remove(void);
+static int enumr_class2_remove(void);
+
+/* because many things to be checked are performed during ODP initialisation,
+ * the initialisation functions have to be a part of the test
+ */
+static int tests_global_init(void)
+{
+   if (0 != odp_init_global(_instance, NULL, NULL)) {
+   fprintf(stderr, "error: odp_init_global() failed.\n");
+   return -1;
+   }
+   if (0 != odp_init_local(odp_instance, ODP_THREAD_CONTROL)) {
+   fprintf(stderr, "error: odp_init_local() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+static int tests_global_term(void)
+{
+   if (0 != odp_term_local()) {
+   fprintf(stderr, "error: odp_term_local() failed.\n");
+   return -1;
+   }
+
+   if (0 != odp_term_global(odp_instance)) {
+   fprintf(stderr, "error: odp_term_global() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+/*enumerator register functions */
+static odpdrv_enumr_t enumr1_register(void)
+{
+   odpdrv_enumr_param_t param = {
+   .enumr_class = enumr_class1,
+   .api_name = "Enumerator_interface_1",
+   .api_version = 1,
+   .probe = enumr1_probe,
+   .remove = enumr1_remove,
+   .register_notifier = NULL
+   };
+
+   return odpdrv_enumr_register();
+}
+
+static odpdrv_enumr_t enumr2_register(void)
+{
+   odpdrv_enumr_param_t param = {
+   .enumr_class = enumr_class1,
+   .api_name = "Enumerator_interface_2",
+   .api_version = 1,
+   .probe = enumr2_probe,
+   .remove = enumr2_remove,
+   .register_notifier = NULL
+   };
+
+   retu

[lng-odp] [API-NEXT PATCH 07/21] drv: driver: add callback function for device destruction

[Christophe Milard]

When a device is destroyed by an enumerator, odpdrv_device_destroy() is
called.
However, the complete device destruction may require waiting for IO to be
completed: the device destruction is therefore divided in 2 steps:
odpdrv_device_destroy() starts the device destruction, and the provided
callback function is called when the device can be fully removed, i.e.
when it no longer has any driver bound to it.
An extra flag is also added to select the destruction type:
The default is a graceful destruction, letting the time for any attached
driver to terminate. This may imply that the callback function is called
from another ODP thread, later on.
ODPDRV_DEV_DESTROY_IMMEDIATE forces an immediate device destruction,
possibly terminating things abrubtly, but it guarantees that the
callback is performed by the same ODP thread. This is to be used at ODP
terminaison time.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h   | 31 +++
 platform/linux-generic/drv_driver.c |  9 -
 2 files changed, 35 insertions(+), 5 deletions(-)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index a978d12..21a5fe1 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -357,12 +357,35 @@ odpdrv_device_t 
odpdrv_device_create(odpdrv_device_param_t *param);
 
 /**
 * Destroy a device
-* Called by each enumerator at probe time, or anytime later, for each
-* destroyed created device
+* Called by each enumerator after probe time, for each device to be
+* destroyed.
+* Destroying a device may require tearing down a driver and waiting for some IO
+* to terminate: The device destruction is therefore done in 2 steps:
+* Calling this function starts the device destruction: when the device has
+* no driver attached any longer, ODP calls the provided callback()
+* function  which should free the enumerator-allocated resources for
+* this device.
+* If the flag ODPDRV_DEV_DESTROY_IMMEDIATE is given, the device destruction
+* is immediate, i.e. the callback function is guaranteed to be called by the
+* same ODP thread: This might however not let the time for the bound driver
+* (if any) to terminate gracefully. This would typically be used at ODP
+* terminaison. By default, the callback may be called later, when the driver
+* has gracefully terminated, hence possibly from another ODP thread.
 * @param dev A odpdrv device handle as returned by odpdrv_device_create.
-* @return 0 on success or a negative value on error.
+* @param callback a pointer to a function to be called when the device is
+*freed (no more driver). The parameter to the callback function is
+*   the pointer to the enumerator specific part of the device as provided
+*   at device creation time (void *enum_dev). The callback function
+*   should release these resources.
+* @param flags 0 or ODPDRV_DEV_DESTROY_IMMEDIATE for immediate shut down
+* @return 0 on success or a negative value on error. On error, the callback
+* function is not called.
 */
-void odpdrv_device_destroy(odpdrv_device_t dev);
+int odpdrv_device_destroy(odpdrv_device_t dev,
+ void (*callback)(void *enum_dev), uint32_t flags);
+
+/** The callback function must be called by the current ODP thread */
+#define ODPDRV_DEV_DESTROY_IMMEDIATE   0x0001
 
 /**
 * Register an devio.
diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index ee0a75c..f8844f5 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -227,10 +227,17 @@ odpdrv_device_t 
odpdrv_device_create(odpdrv_device_param_t *param)
return ODPDRV_DEVICE_INVALID;
 }
 
-void odpdrv_device_destroy(odpdrv_device_t dev)
+int odpdrv_device_destroy(odpdrv_device_t dev,
+ void (*callback)(void *enum_dev), uint32_t flags)
 {
if (dev == ODPDRV_DEVICE_INVALID)
ODP_ERR("Invalid device\n");
+   if (callback != NULL)
+   ODP_ERR("Callback not supported yet\n");
+   if (flags != 0)
+   ODP_ERR("flags not supported yet\n");
+
+   return 0;
 }
 
 odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t *param)
-- 
2.7.4

[lng-odp] [API-NEXT PATCH 06/21] drv: driver: change drv unbind function name and pass correct parameter

[Christophe Milard]

The driver removal function expects a device, of course...
Also unbind seems a better name to disconnect from a device
since remove has been used for removing the object itself for
enumerators.
Some extra parameters to allow for graceful unbinding are also added.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 21 ++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index b08d7fb..a978d12 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -300,14 +300,29 @@ struct odpdrv_driver_param_t {
 */
int (*probe)(odpdrv_device_t *dev);
 
-   /** Remove function:
+   /** unbind function:
 * Only called with devices whose probe() returned true
 *
+* dev: the device to unbind
+* callback: if flag ODPDRV_DRV_UNBIND_IMMEDIATE is not specified,
+*  unbind should be attempted gracefuly, meaning that some IO may need
+*  to terminate before the driver is really unbound from the device:
+*  In this case (when the flag is not set), the driver is due to call
+*  the callback function when the driver is unbound from the device.
+*  This callback may occurs within the unbind() call if the driver
+*  does unbind immediately.
+*  If the ODPDRV_DRV_UNBIND_IMMEDIATE is specified, the driver is due
+*  to release the device immediately (poosibly less gracefuly).
+*  The callback must be called immediately in this case.
 */
-   int (*remove)(odpdrv_device_param_t *dev);
-
+   int (*unbind)(odpdrv_device_t dev,
+ void (*callback)(odpdrv_device_t dev),
+ uint32_t flags);
 };
 
+/** The callback function must be called mmediately by the current ODP thread 
*/
+#define ODPDRV_DRV_UNBIND_IMMEDIATE0x0001
+
 /**
 * Register an enumerator class.
 * Each enumerator class calls this function at init time.
-- 
2.7.4

[lng-odp] [API-NEXT PATCH 04/21] linux-gen: drv: enumerator registration

[Christophe Milard]

The enumerator registration functions for the linux-gen ODP implementation.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/drv_driver.c | 135 +++-
 1 file changed, 132 insertions(+), 3 deletions(-)

diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index 50956a7..ee0a75c 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -24,6 +24,7 @@ static enum {UNDONE, IN_PROGRESS, DONE} init_global_status;
 static _odp_ishm_pool_t *list_elt_pool;
 
 typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
+typedef struct _odpdrv_enumr_s _odpdrv_enumr_t;
 
 /* an enumerator class (list element) */
 struct _odpdrv_enumr_class_s {
@@ -40,6 +41,20 @@ typedef struct _odpdrv_enumr_class_lst_t {
 } _odpdrv_enumr_class_lst_t;
 static struct _odpdrv_enumr_class_lst_t enumr_class_lst;
 
+/* an enumerator (list element) */
+struct _odpdrv_enumr_s {
+   odpdrv_enumr_param_t param;
+   int probed;
+   struct _odpdrv_enumr_s *next;
+};
+
+/* the enumerator list: */
+typedef struct _odpdrv_enumr_lst_t {
+   odp_rwlock_recursive_t lock;
+   _odpdrv_enumr_t *head;
+} _odpdrv_enumr_lst_t;
+static struct _odpdrv_enumr_lst_t enumr_lst;
+
 /* some driver elements (such as enumeraor classes, drivers, devio) may
  * register before init_global and init_local complete. Mutex will fail
  * in this cases but should be used later on.
@@ -69,6 +84,29 @@ static void enumr_class_list_write_unlock(void)
odp_rwlock_recursive_write_unlock(_class_lst.lock);
 }
 
+static void enumr_list_read_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_lock(_lst.lock);
+}
+
+static void enumr_list_read_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_unlock(_lst.lock);
+}
+
+static void enumr_list_write_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_lock(_lst.lock);
+}
+
+static void enumr_list_write_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_unlock(_lst.lock);
+}
 
 odpdrv_enumr_class_t odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
 *param)
@@ -133,10 +171,53 @@ odpdrv_enumr_class_t 
odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
 
 odpdrv_enumr_t odpdrv_enumr_register(odpdrv_enumr_param_t *param)
 {
-   ODP_ERR("NOT Supported yet! Enumerator API %s Registration!\n.",
-   param->api_name);
+   _odpdrv_enumr_t *enumr;
+   _odpdrv_enumr_class_t *enumr_c;
+   int found_class = 0;
+
+   /* make sure that the provided enumerator_class does indeed exist: */
+   enumr_class_list_read_lock();
+   enumr_c = enumr_class_lst.head;
+   while (enumr_c) {
+   if ((_odpdrv_enumr_class_t *)(void *)param->enumr_class ==
+enumr_c) {
+   found_class = 1;
+   break;
+   }
+   enumr_c = enumr_c->next;
+   }
+   enumr_class_list_read_unlock();
+   if (!found_class) {
+   ODP_ERR("invalid enumerator class provided!\n");
+   return ODPDRV_ENUMR_INVALID;
+   }
+
+   /* allocate memory for the new enumerator:
+* If init_global has not been done yet, we have a big issue,
+* as none of the enumerator classes should be porbed before that!
+* We cannot even issue an error as ODP_* functions have not been
+* initialised yet, but this is no good...
+*/
+
+   if (init_global_status == UNDONE)
+   return ODPDRV_ENUMR_INVALID;
+
+   enumr = _odp_ishm_pool_alloc(list_elt_pool,
+sizeof(_odpdrv_enumr_t));
+   if (!enumr) {
+   ODP_ERR("_odp_ishm_pool_alloc failed!\n");
+   return ODPDRV_ENUMR_INVALID;
+   }
+
+   /* save init parameters and insert enumerator in list */
+   enumr->param = *param;
+   enumr->probed = 0;
+   enumr_list_write_lock();
+   enumr->next = enumr_lst.head;
+   enumr_lst.head = enumr;
+   enumr_list_write_unlock();
 
-   return ODPDRV_ENUMR_INVALID;
+   return (odpdrv_enumr_t)enumr;
 }
 
 odpdrv_device_t odpdrv_device_create(odpdrv_device_param_t *param)
@@ -174,6 +255,7 @@ odpdrv_driver_t 
odpdrv_driver_register(odpdrv_driver_param_t *param)
 void _odpdrv_driver_probe_drv_items(void)
 {
_odpdrv_enumr_class_t *enumr_c;
+   _odpdrv_enumr_t *enumr;
 
/* probe unprobed enumerators: */
enumr_class_list_read_lock();
@@ -186,11 +268,26 @@ void _odpdrv_driver_probe_drv_items(void)
enumr_c = enumr_c->next;
}
enumr_class_list_read_unlock();
+
+   /* go through the list o

[lng-odp] [API-NEXT PATCH 03/21] test: drv: enumerator_class registration tests

[Christophe Milard]

Testing that enumerators classes can register properly.
Saddly restricted to statically linked enumerators classes, as testing with
modules in autotools seems to be an issue so far.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 test/common_plat/m4/configure.m4   |   1 +
 test/common_plat/validation/drv/Makefile.am|   1 +
 .../validation/drv/drvdriver/.gitignore|   1 +
 .../validation/drv/drvdriver/Makefile.am   |  16 ++
 .../drv/drvdriver/drvdriver_enumr_class.c  | 174 +
 .../drv/drvdriver/drvdriver_enumr_class.h  |  24 +++
 .../drv/drvdriver/drvdriver_enumr_class_main.c |  12 ++
 test/linux-generic/Makefile.am |   1 +
 8 files changed, 230 insertions(+)
 create mode 100644 test/common_plat/validation/drv/drvdriver/.gitignore
 create mode 100644 test/common_plat/validation/drv/drvdriver/Makefile.am
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.c
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class_main.c

diff --git a/test/common_plat/m4/configure.m4 b/test/common_plat/m4/configure.m4
index 13a13bd..400750c 100644
--- a/test/common_plat/m4/configure.m4
+++ b/test/common_plat/m4/configure.m4
@@ -34,4 +34,5 @@ AC_CONFIG_FILES([test/common_plat/Makefile
 test/common_plat/validation/api/traffic_mngr/Makefile
 test/common_plat/validation/drv/Makefile
 test/common_plat/validation/drv/drvatomic/Makefile
+test/common_plat/validation/drv/drvdriver/Makefile
 test/common_plat/validation/drv/drvshmem/Makefile])
diff --git a/test/common_plat/validation/drv/Makefile.am 
b/test/common_plat/validation/drv/Makefile.am
index bcdb92e..7329a89 100644
--- a/test/common_plat/validation/drv/Makefile.am
+++ b/test/common_plat/validation/drv/Makefile.am
@@ -1,4 +1,5 @@
 ODPDRV_MODULES = drvatomic \
+drvdriver \
 drvshmem
 
 SUBDIRS = $(ODPDRV_MODULES)
diff --git a/test/common_plat/validation/drv/drvdriver/.gitignore 
b/test/common_plat/validation/drv/drvdriver/.gitignore
new file mode 100644
index 000..9268315
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/.gitignore
@@ -0,0 +1 @@
+drvdriver_enumr_class_main
diff --git a/test/common_plat/validation/drv/drvdriver/Makefile.am 
b/test/common_plat/validation/drv/drvdriver/Makefile.am
new file mode 100644
index 000..9e941ee
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/Makefile.am
@@ -0,0 +1,16 @@
+include ../Makefile.inc
+
+# because most of driver activity occurs at init time, and due to the
+# fact that many sequential ODP runs are not allowed from the same process,
+# we need different binaries for each things being tested (as API init)
+
+#tests for enumerator class registration:
+noinst_LTLIBRARIES = libtestdrvdriverenumrclass.la
+libtestdrvdriverenumrclass_la_SOURCES = drvdriver_enumr_class.c
+
+test_PROGRAMS = drvdriver_enumr_class_main$(EXEEXT)
+dist_drvdriver_enumr_class_main_SOURCES = drvdriver_enumr_class_main.c
+drvdriver_enumr_class_main_LDADD = libtestdrvdriverenumrclass.la \
+  $(LIBCUNIT_COMMON) $(LIBODP)
+
+EXTRA_DIST = drvdriver_enumr_class.h
diff --git a/test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.c 
b/test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.c
new file mode 100644
index 000..62b99ea
--- /dev/null
+++ b/test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.c
@@ -0,0 +1,174 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+#include 
+#include 
+#include "drvdriver_enumr_class.h"
+#include 
+
+static odp_instance_t odp_instance;
+
+static int enumr_class1_probed;
+static int enumr_class2_probed;
+
+/* forward declaration */
+static int enumr_class1_probe(void);
+static int enumr_class2_probe(void);
+
+static int enumr_class1_remove(void);
+static int enumr_class2_remove(void);
+
+/* because many things to be checked are performed during ODP initialisation,
+ * the initialisation functions have to be a part of the test
+ */
+static int tests_global_init(void)
+{
+   if (0 != odp_init_global(_instance, NULL, NULL)) {
+   fprintf(stderr, "error: odp_init_global() failed.\n");
+   return -1;
+   }
+   if (0 != odp_init_local(odp_instance, ODP_THREAD_CONTROL)) {
+   fprintf(stderr, "error: odp_init_local() failed.\n");
+   return -1;
+   }
+
+   return 0;
+}
+
+static int tests_global_term(void)
+{
+   if (0 != odp_term_local()) {
+   fprintf(stderr, "error: odp_term_local() failed.\n");
+   return -1;
+   }
+

[lng-odp] [API-NEXT PATCH 02/21] linux-gen: drv: enumerator_class registration

[Christophe Milard]

The functions to register and probe enumerator classes are added.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/Makefile.am |   1 +
 platform/linux-generic/_modules.c  |   4 +
 platform/linux-generic/drv_driver.c| 212 -
 .../linux-generic/include/drv_driver_internal.h|  22 +++
 platform/linux-generic/include/odp_internal.h  |   5 +
 platform/linux-generic/odp_init.c  |  21 +-
 6 files changed, 260 insertions(+), 5 deletions(-)
 create mode 100644 platform/linux-generic/include/drv_driver_internal.h

diff --git a/platform/linux-generic/Makefile.am 
b/platform/linux-generic/Makefile.am
index 66ff53d..b7d1b1a 100644
--- a/platform/linux-generic/Makefile.am
+++ b/platform/linux-generic/Makefile.am
@@ -133,6 +133,7 @@ noinst_HEADERS = \
  ${srcdir}/include/_ishm_internal.h \
  ${srcdir}/include/_ishmphy_internal.h \
  ${srcdir}/include/_ishmpool_internal.h \
+ ${srcdir}/include/drv_driver_internal.h\
  ${srcdir}/include/odp_align_internal.h \
  ${srcdir}/include/odp_atomic_internal.h \
  ${srcdir}/include/odp_buffer_inlines.h \
diff --git a/platform/linux-generic/_modules.c 
b/platform/linux-generic/_modules.c
index 6bb854e..b23c81f 100644
--- a/platform/linux-generic/_modules.c
+++ b/platform/linux-generic/_modules.c
@@ -9,6 +9,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 
@@ -40,6 +41,9 @@ static int load_modules(void)
ODP_DBG("module %s loaded.\n", module_name);
}
 
+   /* give a chance top the driver interface to probe for new things: */
+   _odpdrv_driver_probe_drv_items();
+
return 0;
 }
 
diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
index 529da48..50956a7 100644
--- a/platform/linux-generic/drv_driver.c
+++ b/platform/linux-generic/drv_driver.c
@@ -4,20 +4,131 @@
  * SPDX-License-Identifier: BSD-3-Clause
  */
 
+#include 
+
 #include 
+#include <_ishmpool_internal.h>
 
 #include 
 #include 
+#include 
 #include 
+#include 
 #include 
+#include 
+
+static enum {UNDONE, IN_PROGRESS, DONE} init_global_status;
+
+/* pool from which different list elements are alocated: */
+#define ELT_POOL_SIZE (1 << 20)  /* 1Mb */
+static _odp_ishm_pool_t *list_elt_pool;
+
+typedef struct _odpdrv_enumr_class_s _odpdrv_enumr_class_t;
+
+/* an enumerator class (list element) */
+struct _odpdrv_enumr_class_s {
+   odpdrv_enumr_class_param_t param;
+   int probed;
+   _odp_ishm_pool_t *pool;
+   struct _odpdrv_enumr_class_s *next;
+};
+
+/* the enumerator class list: */
+typedef struct _odpdrv_enumr_class_lst_t {
+   odp_rwlock_recursive_t lock;
+   _odpdrv_enumr_class_t *head;
+} _odpdrv_enumr_class_lst_t;
+static struct _odpdrv_enumr_class_lst_t enumr_class_lst;
+
+/* some driver elements (such as enumeraor classes, drivers, devio) may
+ * register before init_global and init_local complete. Mutex will fail
+ * in this cases but should be used later on.
+ * These functions disable the usage of Mutex while it is global init i.e.
+ * while single threaded*/
+static void enumr_class_list_read_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_lock(_class_lst.lock);
+}
+
+static void enumr_class_list_read_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_read_unlock(_class_lst.lock);
+}
+
+static void enumr_class_list_write_lock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_lock(_class_lst.lock);
+}
+
+static void enumr_class_list_write_unlock(void)
+{
+   if (init_global_status == DONE)
+   odp_rwlock_recursive_write_unlock(_class_lst.lock);
+}
+
 
 odpdrv_enumr_class_t odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
 *param)
 {
-   ODP_ERR("NOT Supported yet! Enumerator Class %s Registration!\n.",
-   param->name);
+   _odpdrv_enumr_class_t *enumr_c;
 
-   return ODPDRV_ENUMR_CLASS_INVALID;
+   /* parse the list of already registered enumerator class to make
+* sure no enumerator with identical name already exists:
+*/
+   enumr_class_list_read_lock();
+   enumr_c = enumr_class_lst.head;
+   while (enumr_c) {
+   if (strncmp(param->name, enumr_c->param.name,
+   ODPDRV_NAME_SIZE) == 0) {
+   ODP_ERR("enumerator class %s already exists!\n",
+   param->name);
+   enumr_class_list_read_unlock();
+   return ODPDRV_ENUMR_CLASS_INVALID;
+   }
+   enumr_c = enumr_c->next;
+   }
+

[lng-odp] [API-NEXT PATCH 01/21] drv: making parameter strings dynamically computable

[Christophe Milard]

Declaring strings as const in the driver API prevents dynamic calculation
of these strings, which is a drawback. For instance,
the device addresses (string) are typically calculated by enumerators,
and should therefore not be const... Other strings may also be the result
of a computation. This change is made to allow this.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
index d83e907..b08d7fb 100644
--- a/include/odp/drv/spec/driver.h
+++ b/include/odp/drv/spec/driver.h
@@ -170,7 +170,7 @@ struct odpdrv_enumr_class_param_t {
/** Enumerator name: mostly used for debug purpose.
 * Name must be unique (e.g. "PCI-DPAA2")
 */
-   const char name[ODPDRV_NAME_SIZE];
+   char name[ODPDRV_NAME_SIZE];
 
/** Probe function:
 * Called by ODP to get the enumerator class instances registered
@@ -198,7 +198,7 @@ struct odpdrv_enumr_param_t {
 * The format of the enum_dev part for the odpdrv_device_param_t
 * structure is identified by the api-name and version below
 */
-   const char api_name[ODPDRV_NAME_SIZE];
+   char api_name[ODPDRV_NAME_SIZE];
uint32_t api_version; /**<< the version of the provided API */
 
/** Probe function:
@@ -240,7 +240,7 @@ struct odpdrv_device_param_t {
 * e.g. ".23.12.1" for PCI domain 0, bus 23, device 12, function 1.
 * This string identifies the device uniquely.
 */
-   const char  address[ODPDRV_NAME_ADDR_SZ];
+   char address[ODPDRV_NAME_ADDR_SZ];
 
/** Enumerator dependent part
 * This part is allocated by the enumerator and is enumerator dependent
@@ -260,13 +260,13 @@ struct odpdrv_devio_param_t {
 * with same provided interface should refer to a common enumerator
 * class)
 */
-   const char api_name[ODPDRV_NAME_SIZE];
+   char api_name[ODPDRV_NAME_SIZE];
uint32_t api_version; /**<< the version of the provided API */
 
/** Enumerator interface name and version
 * The enumerator interface this devio needs.
 */
-   const char enumr_api_name[ODPDRV_NAME_SIZE];
+   char enumr_api_name[ODPDRV_NAME_SIZE];
uint32_t enumr_api_version; /**<< required enumerator API version */
 
/** Ops
@@ -283,14 +283,14 @@ struct odpdrv_driver_param_t {
 * The driver name (the pair {driver-name, enum-api-name} must
 * be unique)
 */
-   const char name[ODPDRV_NAME_SIZE];
+   char name[ODPDRV_NAME_SIZE];
 
/** Supported devios:
 * The list of supported devio: one of the following devio
 * (with correct version) must be available for the driver to work:
 */
struct {
-   const char api_name[ODPDRV_NAME_SIZE]; /**<< devio API name */
+   char api_name[ODPDRV_NAME_SIZE]; /**<< devio API name */
uint32_t   api_version; /**<< devio API version */
} devios[ODPDRV_MAX_DEVIOS];
 
-- 
2.7.4

[lng-odp] [API-NEXT PATCH 00/21] driver items registration and probing

[Christophe Milard]

This patch series implements the driver interface, i.e.
enumerator class, enumerator, devio and drivers registration and probing.
This interface is depicted in:
https://docs.google.com/document/d/1eCKPJF6uSlOllXi_sKDvRwUD2BXm-ZzxZoKT0nVEsl4/edit
The associated tests are testing these mechanisms. Note that these tests
are testing staticaly linked modules only (hence avoiding the
module/platform/test debate). Also note that these tests are gathering
all the elements (enumerators, enumerator classes, devio, drivers) making
up the driver interface so as their interactions can be checked.
Real elements (pci enumerators, drivers...) will likely be written in a much
more stand-alone way.

Christophe Milard (21):
  drv: making parameter strings dynamically computable
  linux-gen: drv: enumerator_class registration
  test: drv: enumerator_class registration tests
  linux-gen: drv: enumerator registration
  test: drv: enumerator registration tests
  drv: driver: change drv unbind function name and pass correct
parameter
  drv: driver: add callback function for device destruction
  linux-gen: drv: device creation and deletion
  drv: driver: adding device query function
  linux-gen: drv: driver: adding device querry function
  test: drv: device creation and destruction
  drv: driver: adding a probe and remove callback for devio
  linux-gen: drv: devio registration
  test: drv: devio creation and destruction
  drv: adding driver remove function
  drv: complement parameters to the driver probe() function
  linux-gen: driver registration and probing
  test: drv: driver registration and probing
  drv: driver: adding functions to attach driver's data to the device
  linux-gen: adding functions to attach driver's data to the device
  test: drv: test for setting and retrieving driver's data

 include/odp/drv/spec/driver.h  |  132 ++-
 platform/linux-generic/Makefile.am |1 +
 platform/linux-generic/_modules.c  |4 +
 platform/linux-generic/drv_driver.c| 1037 +++-
 .../linux-generic/include/drv_driver_internal.h|   22 +
 platform/linux-generic/include/odp_internal.h  |5 +
 platform/linux-generic/odp_init.c  |   21 +-
 test/common_plat/m4/configure.m4   |1 +
 test/common_plat/validation/drv/Makefile.am|1 +
 .../validation/drv/drvdriver/.gitignore|5 +
 .../validation/drv/drvdriver/Makefile.am   |   60 ++
 .../validation/drv/drvdriver/drvdriver_device.c|  218 
 .../validation/drv/drvdriver/drvdriver_device.h|   24 +
 .../drv/drvdriver/drvdriver_device_main.c  |   12 +
 .../validation/drv/drvdriver/drvdriver_devio.c |  209 
 .../validation/drv/drvdriver/drvdriver_devio.h |   24 +
 .../drv/drvdriver/drvdriver_devio_main.c   |   12 +
 .../validation/drv/drvdriver/drvdriver_driver.c|  518 ++
 .../validation/drv/drvdriver/drvdriver_driver.h|   24 +
 .../drv/drvdriver/drvdriver_driver_main.c  |   12 +
 .../validation/drv/drvdriver/drvdriver_enumr.c |  303 ++
 .../validation/drv/drvdriver/drvdriver_enumr.h |   24 +
 .../drv/drvdriver/drvdriver_enumr_class.c  |  174 
 .../drv/drvdriver/drvdriver_enumr_class.h  |   24 +
 .../drv/drvdriver/drvdriver_enumr_class_main.c |   12 +
 .../drv/drvdriver/drvdriver_enumr_main.c   |   12 +
 test/linux-generic/Makefile.am |5 +
 27 files changed, 2861 insertions(+), 35 deletions(-)
 create mode 100644 platform/linux-generic/include/drv_driver_internal.h
 create mode 100644 test/common_plat/validation/drv/drvdriver/.gitignore
 create mode 100644 test/common_plat/validation/drv/drvdriver/Makefile.am
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_device.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_device.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_device_main.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_devio.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_devio.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_devio_main.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_driver.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_driver.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_driver_main.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_enumr.c
 create mode 100644 test/common_plat/validation/drv/drvdriver/drvdriver_enumr.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.c
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class.h
 create mode 100644 
test/common_plat/validation/drv/drvdriver/drvdriver_enumr_class_main.c
 create mode 100644 
test/common_plat/validation/drv

Re: [lng-odp] [PATCHv2] configure: libatomic check

[Christophe Milard]

Hi,

If these flags are required for compiling the examples, maybe, they
should not be moved to the platform m4.
But the problem is maybe more: Why do the examples (which are supposed
to be compilable by all) requires this?

So maybe this change is good, and will trigger the example question...

This seems to fix an issue, anyway,  so I am positive to apply this patch

Christophe

On 16 February 2017 at 21:26, Maxim Uvarov <maxim.uva...@linaro.org> wrote:
> upcoming patch ip fragmentation example fails to with
> gcc 4.8.4 on linking __atomic_compare_exchange_16 functions
> on x86. For some version of gcc both -latomic and -mcx16
> needs to be provided in that case. For clang only -mcx16. That
> options are set internally for platform but do not set for examples.
> This patch unhides setting this options, make them common and printed
> on configure log.

Reviewed-by: Christophe Milard <christophe.mil...@linaro.org>

>
> Signed-off-by: Maxim Uvarov <maxim.uva...@linaro.org>
> ---
>  v2: unhide options and make frag code compile.
>  CI now happy:
>  https://travis-ci.org/muvarov/odp/builds/202380977
>
>
>  configure.ac   | 16 --
>  platform/linux-generic/m4/configure.m4 | 57 
> ++
>  2 files changed, 57 insertions(+), 16 deletions(-)
>
> diff --git a/configure.ac b/configure.ac
> index 6153efd2..a514f6be 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -314,22 +314,6 @@ ODP_CFLAGS="$ODP_CFLAGS -std=c99"
>  # Extra flags for example to suppress certain warning types
>  ODP_CFLAGS="$ODP_CFLAGS $ODP_CFLAGS_EXTRA"
>
> -#
> -# Check if compiler supports cmpxchng16
> -##
> -if test "${CC}" != "gcc" -o ${CC_VERSION_MAJOR} -ge 5; then
> -   my_save_cflags="$CFLAGS"
> -
> -   CFLAGS=-mcx16
> -   AC_MSG_CHECKING([whether CC supports -mcx16])
> -   AC_COMPILE_IFELSE([AC_LANG_PROGRAM([])],
> -   [AC_MSG_RESULT([yes])]
> -   [ODP_CFLAGS="$ODP_CFLAGS $CFLAGS"],
> -   [AC_MSG_RESULT([no])]
> -   )
> -   CFLAGS="$my_save_cflags"
> -fi
> -
>  ##
>  # Default include setup
>  ##
> diff --git a/platform/linux-generic/m4/configure.m4 
> b/platform/linux-generic/m4/configure.m4
> index d3e5528c..92172d4b 100644
> --- a/platform/linux-generic/m4/configure.m4
> +++ b/platform/linux-generic/m4/configure.m4
> @@ -28,6 +28,63 @@ AC_LINK_IFELSE(
>  echo "Use newer version. For gcc > 4.7.0"
>  exit -1)
>
> +#
> +# Check if compiler supports cmpxchng16
> +##
> +my_save_cflags="$CFLAGS"
> +
> +CFLAGS=-mcx16
> +AC_MSG_CHECKING([whether CC supports -mcx16])
> +AC_COMPILE_IFELSE([AC_LANG_PROGRAM([])],
> +[AC_MSG_RESULT([yes])]
> +[MCX16_CFLAGS="-mcx16"],
> +[AC_MSG_RESULT([no])]
> +)
> +CFLAGS="$my_save_cflags"
> +
> +#
> +# Check if compiler supports __atomic_compare_exchange
> +##
> +my_save_cflags="$CFLAGS"
> +LD_LIBATOMIC=""
> +
> +AC_MSG_CHECKING(if libatomic required)
> +AC_LINK_IFELSE(
> +[AC_LANG_SOURCE(
> +  [[int main() {
> +unsigned __int128 x = 0, y = 0;
> +y = __atomic_load_n(, 0);
> +__atomic_store_n(, y, 0);
> +__atomic_compare_exchange_n(, , x, 0, 0, 0);
> +return 0;
> +}
> +]])],
> +AC_MSG_RESULT(yes)
> +LD_LIBATOMIC="-latomic"
> +echo "adding -lbatomic",
> +AC_MSG_RESULT(no))
> +
> +CFLAGS="$my_save_cflags $LD_LIBATOMIC $MCX16_CFLAGS"
> +AC_MSG_CHECKING([CFLAGS=$CFLAGS])
> +AC_LINK_IFELSE(
> +[AC_LANG_SOURCE(
> +  [[int main() {
> +unsigned __int128 x = 0, y = 0;
> +y = __atomic_load_n(, 0);
> +__atomic_store_n(, y, 0);
> +__atomic_compare_exchange_n(, , x, 0, 0, 0);
> +return 0;
> +}
> +]])],
> +AC_MSG_RESULT(yes),
> +AC_MSG_RESULT(no)
> +echo "atomic operations compilation fail."
> +exit -1)
> +
> +# Restore LDFLAGS
> +CFLAGS="$my_save_cflags $MCX16_CFLAGS"
> +LDFLAGS="$LDFLAGS $LD_LIBATOMIC"
> +
>  m4_include([platform/linux-generic/m4/odp_pthread.m4])
>  m4_include([platform/linux-generic/m4/odp_openssl.m4])
>  m4_include([platform/linux-generic/m4/odp_pcap.m4])
> --
> 2.11.0.295.gd7dffce
>

Re: [lng-odp] [PATCH 1/2] helper: linux: renamed threads_extn to linux helpers

[Christophe Milard]

On 15 February 2017 at 09:34, Savolainen, Petri (Nokia - FI/Espoo)
<petri.savolai...@nokia-bell-labs.com> wrote:
>
>
>> -Original Message-
>> From: Christophe Milard [mailto:christophe.mil...@linaro.org]
>> Sent: Monday, February 13, 2017 5:41 PM
>> To: Savolainen, Petri (Nokia - FI/Espoo) <petri.savolainen@nokia-bell-
>> labs.com>
>> Cc: Mike Holmes <mike.hol...@linaro.org>; lng-odp > o...@lists.linaro.org>
>> Subject: Re: [lng-odp] [PATCH 1/2] helper: linux: renamed threads_extn to
>> linux helpers
>>
>> This is getting very confusing now: why helper/?  do we have
>> odp/?: shouldn't  these be symetric?
>
>
> Application (like OFP today) which wants to use e.g. Linux pthread helpers 
> include:
>
> #include 
> #include 
>
> odph_linux_pthread_create()
> ...
> odph_linux_pthread_join()
>
>
> It explicitly uses Linux pthreads through a helper function. There's no need 
> to provide these helpers and functions on a non-Linux implementation (because 
> there is no Linux).
>
> Linux helpers are totally different thing than ODP implementation internal 
> directory structure.

Different: Yes. But related:
We are back, sadly, to my first question: "what is a platform".
To that, you, Petri, answered: "platform == implementation". And I agree.

Now, can we agree that the above platform definition implies:
different OS => different platform? Or are you saying that a OSE ODP
and linux ODP are the same implementation???.

If we can agree that, for ODP implementation, different OSes means
different platforms, do you feel comfortable calling these variation
differently for the helpers?

What if some helper function requires to be different for a given OS?
Say for instance, some company wants to implement ODP threads using
libdill or libmill? (on linux). Or some helper function would depend
on memory available on a given HW?

Yes, ODP implementation and helpers are 2 differents things, but they
are facing the same issue: They try to implement a given interface
(the ODP API  for ODP and the helper API for the helpers) on different
HW, OS, ... I'd say on different platform. I cannot see why helpers
variants would only be limited to OSes.

If we can agree on the above, we should agree that a common solution
to tackle the same problem would be nice.
On the ODP implementation side, we agreed that a platform was just a
way to diverge for the main delivery.
I still think it makes sense to have the same approach for the helpers:
If we deliver ODP implementation "linux-gen" and "ODP-dpdk" we should
deliver 2 variant of helper for these as well (even if they happen to
be the same).

A constructor willing to have his own ODP (whatever the reason) would
have the framework to diverge: a divergence is a new platform. Both
for helpers and ODP
The interesting question is how do we write the code so that these
divergence would be easy to do and so that maximum code can be reuse.
That is where the next challenge is.  But first we need to agree on
what a platform is and how we handle helpers.

Christophe.

>
>
>>
>> My view is getting clearer and clearer:
>> On the test side, was have made tests for {OS, HW}= {linux/PC} and
>> given the possibility to diverge from these (for any  reason, e.g.
>> difference of HW or OS) with the use of "platform":
>> If a test defined on the common validation test is inappropriate for a
>> ODP developer (whatever the reason), that developer can overwrite this
>> test (or part of it, such as its setup) with an appropriate one on the
>> platform side.
>> Kallray, AFAIK, if using the platform trick for both OS and HW divergence.
>
> If the system under test is not a Linux system. You cannot run linux helper 
> code there and thus you just skip the linux helper tests.
>
>
>>
>> Why not the same approach on the helper side? We give helpers for
>> {linux/PC} and provide a way to redefine these function for those who
>> need to,  on the platform side (e.g. using a weak symbol on the common
>> helpers and letting those who need define the same function as
>> "strong" on the helper platform side?... or any other better trick).
>
> Why a vendor which do not support Linux would rewrite e.g. 
> odph_linux_pthread_create() to emulate Linux pthreads? If the system is not 
> Linux, there is no point to run Linux system calls there and application 
> would not call odph_linux_pthread_create() there.
>
> Remember, helper code is application code. Application would not call Linux 
> system calls in an non-Linux system. So, no need for a non-Linux based ODP 
> implementation to emulate any Linux system calls.
>
>
>>
>> Then, any implementer 

Re: [lng-odp] Ask a few questions about odp

[Christophe Milard]

On 15 February 2017 at 04:32, ice <odp_d...@163.com> wrote:
> HI：
>
> I am from a network security company in China,  We intend to use odp to
> complete a bottom forwarding engine and data channel.
> What I am concerned about before is that in the case of multithreading, the
> crash of the thread will hang all threads of the engine.
>
> I have a few questions want to ask you：
>
> 1. I see that there are several functions in the helper/linux.c, such as:
> odph_linux_process_fork() and odph_linux_process_fork_n(),
> But as you said. the process support is not really being pushed and is
> faultly. And all the code used in the example is a multithreaded
> architecture.
> So odp support for multi-process architecture is not perfect, and is not
> fully tested?

ODP is first an open  API definition. Anyone can do an ODP
implementation on its the hardware of his choice. Today, companies
like cavium, Kallray, among others, develop their own product
according to the ODP specification. The creation of a "concurrent
execution environment" (we call them -sadly- "ODP threads", even if
I'd personally prefer the choice of "ODP tasks" as they would be
processes or threads in most cases) is not, as of today, part of ODP:
It is the OS business. But a given ODP implementation may say it needs
"ODP threads" to be of a specific nature (i.e.: they have to be
pthreads, or they have to be processes, or anything else).
Most implementation I have heard of wants ODP threads to be linux pthreads.

Linaro also provides two specific ODP API implementations for linux,
which we call "odp-linux-generic" and "odp-dpdk". Both work works with
ODPthreads being pthreads. Besides the "linux-generic" implementation
claims to support ODP threads being linux processes (and even a mix of
pthreads and processes). I am not sure about what odp-dpdk claims to
support. But I know for a fact that some functionality both will break
if linux processes are used. I have fixed the issue for some part, but
some other problems remains. There has been relatively low interrest
for these issues, som far.
If you need process support, make your voice heard! write to the list,
join the public calls, contribute, or join Linaro!

The "odph_linux_process_fork*" familly of functions are just wrappers
around linux fork. nothing else. The equivalent set of functions
exists for pthreads.There is no requirements to use them, even when
using ODP.
In my humble opinion, the presence of these function in the helpers is
confusing... and is being discussed. If you feel more confident using
pthread_create() and fork() directely, you should keep doing so.
The odph_odpthreads_create() set of function, however, may be of more
interest: They abstract the ODP thread concept: they will create
whatever the ODP implementation wants as an ODPthread. For
odp-linux-generic, either pthreads or processes, depending on a flag
(as linux-generic claims to support both): This may be of interest for
those who wish to develop applications which would work regardless of
the choices which are made for an ODP thread implementation: Using
these functions will help writing a more mortable ODP app (which would
work on any ODP implementations even  if the latter made different
choices regarding the nature of an "ODP thread".)

> 2. If we use a multithreaded architecture, if one of the threads crashes,
> how does the stability of other threads guarantee?

pthreads are pthreads. No difference with ODP. They are plain linux
things: they share memory, file descriptors... So statbility is what
you would expect with them regardless of ODP.

> 3. If I want to transmit buffer from appA to appB and appC, Can I use odp's
> schedule module to implement it? Or I can only use shm ipc to achieve it?
> If I can use odp schedule to achieve, then how the performance?

If appA, appB, appC are ODPthreads belonging to the same ODP instance,
yes, you can transfer buffers between them. Support for multi
instances (many ODP in the same machine) varies from ODP
implementation to ODP implementation, and communication between
different ODP instances is more limited today (but is being worked
on).
Performance will depends on the chosen ODP implementation and HW, of course.

> 4. If I use odp to transfer data between a variety of app, the performance
> will be how?

not sure how this question differs from the previous one (3)

>
> Do you know the open event machine? What do you think of it and odp
> contrast?

I don't know much about the open event machine: My understanding was
that is was targetting embeded systems only (mostly bare-metal). If
this assumption is true, ODP is more general as it targets both
embedded systems and the VM world. But my knowledge of the open event
machine is too weak to comment more :-)
Maybe some other on the list have comments, ther

Re: [lng-odp] Why do not odp use a multi-process model?

[Christophe Milard]

Hi,

I am not sure about who you are, what you are trying to do, and how
you are trying to do it. I am not even sure I understand fully your
concerns.

ODP itself does not specify what concurrency model is used. Some ODP
implementations may use pthreads, some others processes, or whatever
they want.
Having said that, I feel most implementation use pthread, as you said.
The linux-generic implementation provided by linaro actually claims to
be supporting poth pthreads and processes.  Saddly the process support
is not really being pushed and is faultly (despite my effort to push
it for a while).

But as It claims to support it, you are welcome to file a bug, or even
better contribute to fix that.

Hope that helps,

Christophe

On 14 February 2017 at 07:26, ice  wrote:
>
>
> I am trying to apply odp to our project,
>
>
> But I found odp is a multi-threaded model .
>
>
> So I began to worry about the stability of the work-thread,
>
>
> If a thread crashes, what happens to other threads?
>
>
> Why do not odp use a multi-process model?
>
>
> Who can answer my question?
>
>
> thanks

Re: [lng-odp] [PATCH 1/2] helper: linux: renamed threads_extn to linux helpers

[Christophe Milard]

This is getting very confusing now: why helper/?  do we have
odp/?: shouldn't  these be symetric?

My view is getting clearer and clearer:
On the test side, was have made tests for {OS, HW}= {linux/PC} and
given the possibility to diverge from these (for any  reason, e.g.
difference of HW or OS) with the use of "platform":
If a test defined on the common validation test is inappropriate for a
ODP developer (whatever the reason), that developer can overwrite this
test (or part of it, such as its setup) with an appropriate one on the
platform side.
Kallray, AFAIK, if using the platform trick for both OS and HW divergence.

Why not the same approach on the helper side? We give helpers for
{linux/PC} and provide a way to redefine these function for those who
need to,  on the platform side (e.g. using a weak symbol on the common
helpers and letting those who need define the same function as
"strong" on the helper platform side?... or any other better trick).

Then, any implementer knows where to place their hacks: on the
"platform side". Maybe not best, but at least consistent and clear.

That would just give helpers, with different platform (for any reason)
implementations.

The linux-specific helper would of course not fit on the common helper
part as they are linux specific and cannot be implementation on
different OS (platform). Once again, I don't understand why we needed
to do wrappers around well known Linux system calls: IMHO, helpers
should contains functions that we expect to be delivered on any
implementation (OS/HW) helpers, so these linux wrappers don't fit
there.
If we really want to keep that (I'd vote against), let's create
another directory name for what it is: "linux-wrappers"... and
separate them from function we'd expect to see everywhere (the rest of
the helpers, even if different platform dependent implementations for
those might exist)

Christophe

On 13 February 2017 at 16:09, Savolainen, Petri (Nokia - FI/Espoo)
 wrote:
>
>
> From: Mike Holmes [mailto:mike.hol...@linaro.org]
> Sent: Monday, February 13, 2017 5:02 PM
> To: Savolainen, Petri (Nokia - FI/Espoo) 
> 
> Cc: lng-odp 
> Subject: Re: [lng-odp] [PATCH 1/2] helper: linux: renamed threads_extn to 
> linux helpers
>
>
>
> On 13 February 2017 at 09:41, Savolainen, Petri (Nokia - FI/Espoo) 
>  wrote:
>
>
> From: Mike Holmes [mailto:mailto:mike.hol...@linaro.org]
> Sent: Friday, February 10, 2017 5:02 PM
> To: Petri Savolainen 
> Cc: lng-odp 
> Subject: Re: [lng-odp] [PATCH 1/2] helper: linux: renamed threads_extn to 
> linux helpers
>
>
>
> On 3 February 2017 at 06:23, Petri Savolainen 
>  wrote:
> There's no platform specific helpers. Helpers may depend on
> Linux and make it easier to do common series of Linux system
> calls. These kind of helpers are grouped into helper/linux
> directory.
>
> Use --enable-helper-linux configuration option to enable
> support for Linux helpers.
>
> Signed-off-by: Petri Savolainen 
>
> Reviewed-by: Mike Holmes 
>
> required 3 way apply
>
>
>
> This should be merged. It moves linux dependent helper code under linux 
> folder. If there would be helper code dependent on some other OS, that would 
> go to their own helper/OS_foo/ folder. All Linux based ODP implementations 
> can run this helper code - there is no need to have N different linux helpers 
> for N different Linux based implementations.
>
>
> Also, the build is currently broken in master for --enable-helper-extn. E.g. 
> OFP cannot be built against it.
>
> This is not broken IMHO, it is by design, OFP depends on the helpers, we need 
> helpers to either be support for apps that we maintain and don't use (current 
> case) or we enforce  OFP and others to use the abstract API that all our own 
> executable use. Maintaining code we don't use is dangerous, you should have 
> to choose to do it, but I like the rename, that is fine and is progress, 
> hence the review.
>
> I think splitting out all of all the executables  will make this much cleaner.
> All ODP upstream executables will use the abstract API and be portable to any 
> platform and OS. Meanwhile the odp-Linux repo can have a Linux specific apis 
> such as the legacy one OFP is using and that is just that implementations 
> choice, as we have said there is no requirement to use helpers, or linux, and 
> OFP might want to attach itself to that,  but the tests and examples must 
> remain agnostic until we state we are dropping that goal.
>
>
> Please try:
> ./configure --enable-helper-extn
> make
>
> It does not compile, due to:
>
> platform/linux-generic/thread.c:282:12: error: 
> \u2018odph_linux_thread_create\u2019 defined but not used 
> 

[lng-odp] what is a platform?

[Christophe Milard]

Hi,

I am afraid I got very confused after Petri's last helper patch
series, and I think we do need to agree on definition of things. If we
cannot agree via this thread, I think the topic should be raised at
BUD17.

1) Platform = ODP implementation, i.e. a couple {OS, HW}?

This definition would involve that "platform agnostics" things would
be things that are both OS and HW independent. Any changes of either
OS or HW would define a new platform. Assuming we don't put any
restriction on the OS at all (including bare metal, Windows,
file-system free OSes...), this would meant that nearly anything
ends-up in the platform side.
This is not really what we do: The latest reaction on Mike's proposal
to reorganise the helpers to show different "platform" for different
OSes shows it. Petri was clear here: Many OS support does not mean
many "platforms".
So obviously some people do not agree with 1).
My understanding is that Kallray would agree to this definition, as
they want to use the platform concept for both OS abstraction (they
have 2 OSes) and HW (they define their tests specificities to match
memory siszes...)

2)Platform = HW only...
Hmmm. what do we do about different OS support then? We would have
different platforms for, e.g. different memory size, or CPU speed,
enabling the tests to be "sized" accordinaly. This is partly what we
to today: The "platform side" of the pktio tests creates the
interfaces matching the HW. fine.
But interresting enough, the tests testing the ODP_SHM_PROC flag of
shm is also in the platform side, though it really is testing an OS
dependant thing only.
And we talk of bare metal as a platform. So we don't really do that either.

My feeling is that:
a) the ODP api is OS, HW or anything agnostic: yes, but
b) The test environment (tests and helpers) actually assume linux (or
unix like), and anything strange (linux variation, HW) ended up as
"platform". So what we do is closer to 1), in the sense that a
platform is today something like a couple {linux variant, HW}. So here
my feeling was joining Mikes: different OSes do define different
platforms. different OS support would imply different platform.

I am happy for any kind of definition , as long as we can answer the
questions like:
- where should an test assuming an OS function be placed (as loading a
*.so file)
-How to we tune tests to Memory size, CPU speed, etc.
-how to we make the provided tests framework usable for all without
any assumption on OS, HW,...?
-How to we handle OS variant for a single HW, or HW variant for a single OS ?

At this stage, I do not know what a platform is


Christophe.

[lng-odp] driver probe time...

[Christophe Milard]

Hi,

I am facing a issue: When should driver elements (enumerators-classes,
enumerators, drivers) be probed at startup:
My initial idea was to perform probing at init global time, so that a
starting app would have a chance to see the present interfaces at
startup:
This turns out to be a problem as many things are not yet usable at
global init time: thread, some locks... And I do not like the idea to
have the probing function of each driver being in the need to
distinguish between global init time probing and on the run
probing...( a driver would very likely need synchronisation mechanisms
and the latter would only be usable after the initial probing... not
nice)
The other alternative is to do all this at local init time: but that
would require a guarantee that the ODP instantiation process
init_local() function is called before any other thread init_local().
I could not find such requirement in the API description.Otherwise
application may have finished starting up before the drivers are
loaded...

Is this an indication that the current 2 step booting process is too weak?
I actually faced a similar problem with _ishm where there is now a
"hack" to run what should have been part of the instantiation process
local init at the global init time instead: Because other blocks are
assuming that _ishm is fully functional at global_init_time...

Other facing similar issues and having ideas?

Christophe

Re: [lng-odp] [API-NEXT PATCH 15/15] linux-gen: support older libconfig version

[Christophe Milard]

On 27 January 2017 at 15:29, Maxim Uvarov <maxim.uva...@linaro.org> wrote:
> Travis CI uses old libconfig8 amd64 1.3.2-2ubuntu2,
> which does not have defines for errors. Move error print
> under ifdef to fix compilation.
>
> Signed-off-by: Maxim Uvarov <maxim.uva...@linaro.org>

Reviewed-by: Christophe Milard <christophe.mil...@linaro.org>

> ---
>  platform/linux-generic/odp_init.c | 7 ++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/platform/linux-generic/odp_init.c 
> b/platform/linux-generic/odp_init.c
> index 2cd8fe3f..685e02fa 100644
> --- a/platform/linux-generic/odp_init.c
> +++ b/platform/linux-generic/odp_init.c
> @@ -130,12 +130,17 @@ static int read_configfile(void)
> if (config_filename) {
> ODP_DBG("Reading configuration file: %s\n", config_filename);
> if (!config_read_file(cf, config_filename)) {
> +#if defined(LIBCONFIG_VER_MAJOR) && LIBCONFIG_VER_MAJOR >= 1 && \
> +   LIBCONFIG_VER_MINOR >= 4
> ODP_ERR("%s:%d - %s\n",
> config_error_file(cf),
> config_error_line(cf),
> config_error_text(cf));
> +#else
> +   ODP_ERR("config_read_file\n");
> +#endif
> config_destroy(cf);
> -   return(-1);
> +   return -1;
> }
> }
>
> --
> 2.11.0.295.gd7dffce
>

Re: [lng-odp] [API-NEXT PATCH 15/15] linux-gen: support older libconfig version

[Christophe Milard]

Could apply it on current API next, today. I am fine with this patch (15/15)


On 1 February 2017 at 18:46, Christophe Milard
<christophe.mil...@linaro.org> wrote:
> needs rebase, I think:
> erachmi@erachmi-ericsson:~/linaro/ODP/odp$ git am
> ~/incoming/lng-odp_API-NEXT_PATCH_*
> Applying: merge fix: platform/linux-generic/Makefile.am
> error: patch failed: platform/linux-generic/Makefile.am:20
> error: platform/linux-generic/Makefile.am: patch does not apply
> Patch failed at 0001 merge fix: platform/linux-generic/Makefile.am
> The copy of the patch that failed is found in: .git/rebase-apply/patch
> When you have resolved this problem, run "git am --continue".
> If you prefer to skip this patch, run "git am --skip" instead.
> To restore the original branch and stop patching, run "git am --abort".
>
> Tried both with patch 15 alone and the hole series...
>
> sorry...
>
> Christophe.
>
> On 27 January 2017 at 15:29, Maxim Uvarov <maxim.uva...@linaro.org> wrote:
>> Travis CI uses old libconfig8 amd64 1.3.2-2ubuntu2,
>> which does not have defines for errors. Move error print
>> under ifdef to fix compilation.
>>
>> Signed-off-by: Maxim Uvarov <maxim.uva...@linaro.org>
>> ---
>>  platform/linux-generic/odp_init.c | 7 ++-
>>  1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/platform/linux-generic/odp_init.c 
>> b/platform/linux-generic/odp_init.c
>> index 2cd8fe3f..685e02fa 100644
>> --- a/platform/linux-generic/odp_init.c
>> +++ b/platform/linux-generic/odp_init.c
>> @@ -130,12 +130,17 @@ static int read_configfile(void)
>> if (config_filename) {
>> ODP_DBG("Reading configuration file: %s\n", config_filename);
>> if (!config_read_file(cf, config_filename)) {
>> +#if defined(LIBCONFIG_VER_MAJOR) && LIBCONFIG_VER_MAJOR >= 1 && \
>> +   LIBCONFIG_VER_MINOR >= 4
>> ODP_ERR("%s:%d - %s\n",
>> config_error_file(cf),
>> config_error_line(cf),
>> config_error_text(cf));
>> +#else
>> +   ODP_ERR("config_read_file\n");
>> +#endif
>> config_destroy(cf);
>> -   return(-1);
>> +   return -1;
>> }
>> }
>>
>> --
>> 2.11.0.295.gd7dffce
>>

[lng-odp] drv API issue

[Christophe Milard]

Hi,

I am trying to implement to API we agreed on, and I am facing a small
issue which is preceeding larger problems, I am afraid:
The small issue is:
We defined:
odpdrv_enumr_t odpdrv_enumr_register(odpdrv_enumr_param_t *param);
to register a enumerator and :
odpdrv_device_t odpdrv_device_create(odpdrv_device_param_t *param);
to register a device where param has to contains the enumerator
enumerating the device (odpdrv_enumr_t).
Seen for the enumerator, this means that the registration of the
enumerator must have returned before the enumerator probe() function
can be called. How can we guarantee this
My simple approach in
https://git.linaro.org/people/christophe.milard/odp.git/log/?h=drv_enumerator_registration_v0.3
was to call probe at registration time, which does not work...
One solution could be to pass the odp_enumr_t at probe time. Do we
want to do that?
The larger problem: I am not sure we will be able to make "single
threaded" drivers at all times: I feels probable that at some point,
some driver will want to create another ODP thread (for supervision of
link status for instance).
But it is said that ODP thread creation is not part of ODP, but
remains an OS stuff (we just have shortcuts in helpers). If a driver
need to create an ODP thread, and the driver should remain OS
independent, aren't we stuck?

Christophe.

Re: [lng-odp] [PATCH 1/2] helper: linux: renamed threads_extn to linux helpers

[Christophe Milard]

On 3 February 2017 at 14:11, Savolainen, Petri (Nokia - FI/Espoo)
<petri.savolai...@nokia-bell-labs.com> wrote:
>
>
>> -Original Message-
>> From: Christophe Milard [mailto:christophe.mil...@linaro.org]
>> Sent: Friday, February 03, 2017 2:29 PM
>> To: Petri Savolainen <petri.savolai...@linaro.org>
>> Cc: LNG ODP Mailman List <lng-odp@lists.linaro.org>
>> Subject: Re: [lng-odp] [PATCH 1/2] helper: linux: renamed threads_extn to
>> linux helpers
>>
>> On 3 February 2017 at 12:23, Petri Savolainen
>> <petri.savolai...@linaro.org> wrote:
>> > There's no platform specific helpers. Helpers may depend on
>> > Linux and make it easier to do common series of Linux system
>> > calls. These kind of helpers are grouped into helper/linux
>> > directory.
>>
>> This is getting really confusing to me! Haven't we defined a
>> "platform" as being a couple {OS, HW}? That is any change in this
>> couple makes a new platform...
>> If we hold to this definition, then linux-helpers becomes platform
>> helpers...
>> Now, my humble opinion is that no-one is using these linux-only
>> helpers and no one ever will unless to enforce their usage by example
>> that people could copy/paste: why would a programmer want to replace a
>> well known linux system call by an ODP helper call? to save a for
>> loop?.
>> We are hitting the same usual problem of lack of proper definition for
>> things, but in this special case, my proposal is called "deletion".
>> :-)
>>
>> Christophe
>
> This is needed by OFP, for example.
>
> The current ODP master (after v1.13) moved things backwards as linux helper 
> (which create e.g. pthreads and call odp_local_init / _term() for those) was 
> renamed as "linux-generic helper". That would need OFP to expose 
> "helper/linux-generic/..." in its Makefiles, which is both ugly and wrong (in 
> the sense that helper code is part application code, not implementation code).

I am really getting confused here...
There are two sorts of things today in the helper (regarding ODPthread
creation):
1)odph_odpthreads_create() familly of functions abstracts the ODP
thread creation, i.e. will create OPD threads. Whether that is a
pthread or linux process or anything else under the hood is hidden: It
just creates a set of ODPthreads. (for those implementation which
(should) supports different implemenation, a helper flag can be used
to select what should be done)

2)linux specific functions creating pthreads or processes. If the
application uses those, the application is definitively binding itself
to linux and is then using these functions instead of calling
pthread_create() or fork() directely.

>From Mikes comment, I understand that OFP is using functions from the
second set. Do they have any reason for doing that? Possibly, if they
share data allocated after the thread creatiion, not using
odp_shm_reserve().
If an application really binds itseft to linux, I don't see anything
wrong if it is reflected in its Makefile ("helper/linux-generic/...").
If OFP does not require any specific type of ODP thread (they just
want concurrency but don't care how this is achieved) , then they
should be using function from the first set.

The Second set of function is not of much value in my eyes: They do
not provide any kind of abstraction, but just some kind of shortcut
avoiding a few things round fork() or pthread_create().
When I see the confusion that they generate, compared to the little
gain they provide (avoiding to write about 100lines of code), I still
feel those function should be part of the application. Most programmer
knows much more about pthread_create() and fork(), and the price
rewritting the little piece of code will probably been seen as lower
than learning the helper API.

The role of the helpers becomes then clearer: Abstracting thing that
can be done differentely on different platform.

Christophe

>
> -Petri
>
>
>
>
>

Re: [lng-odp] [PATCH 1/2] helper: linux: renamed threads_extn to linux helpers

[Christophe Milard]

On 3 February 2017 at 12:23, Petri Savolainen
 wrote:
> There's no platform specific helpers. Helpers may depend on
> Linux and make it easier to do common series of Linux system
> calls. These kind of helpers are grouped into helper/linux
> directory.

This is getting really confusing to me! Haven't we defined a
"platform" as being a couple {OS, HW}? That is any change in this
couple makes a new platform...
If we hold to this definition, then linux-helpers becomes platform helpers...
Now, my humble opinion is that no-one is using these linux-only
helpers and no one ever will unless to enforce their usage by example
that people could copy/paste: why would a programmer want to replace a
well known linux system call by an ODP helper call? to save a for
loop?.
We are hitting the same usual problem of lack of proper definition for
things, but in this special case, my proposal is called "deletion".
:-)

Christophe

>
> Use --enable-helper-linux configuration option to enable
> support for Linux helpers.
>
> Signed-off-by: Petri Savolainen 
> ---
>  configure.ac   |  17 +-
>  example/Makefile.inc   |   2 +-
>  helper/Makefile.am |  18 +-
>  helper/include/odp/helper/linux/process.h  |  84 ++
>  helper/include/odp/helper/linux/pthread.h  |  66 +
>  .../helper/platform/linux-generic/threads_extn.h   | 112 
>  helper/linux/thread.c  | 239 
>  helper/m4/configure.m4 |   8 +-
>  helper/platform/linux-generic/thread.c | 313 
> -
>  helper/test/.gitignore |   1 +
>  helper/test/Makefile.am|   9 +-
>  helper/test/linux-generic/Makefile.am  |   5 -
>  helper/test/linux-generic/process.c|  92 --
>  helper/test/linux-generic/thread.c |  87 --
>  helper/test/linux/Makefile.am  |   5 +
>  helper/test/linux/process.c|  93 ++
>  helper/test/linux/pthread.c|  87 ++
>  test/Makefile.inc  |   2 +-
>  test/common_plat/validation/api/Makefile.inc   |   2 +-
>  test/linux-generic/Makefile.inc|   2 +-
>  20 files changed, 599 insertions(+), 645 deletions(-)
>  create mode 100644 helper/include/odp/helper/linux/process.h
>  create mode 100644 helper/include/odp/helper/linux/pthread.h
>  delete mode 100644 
> helper/include/odp/helper/platform/linux-generic/threads_extn.h
>  create mode 100644 helper/linux/thread.c
>  delete mode 100644 helper/platform/linux-generic/thread.c
>  delete mode 100644 helper/test/linux-generic/Makefile.am
>  delete mode 100644 helper/test/linux-generic/process.c
>  delete mode 100644 helper/test/linux-generic/thread.c
>  create mode 100644 helper/test/linux/Makefile.am
>  create mode 100644 helper/test/linux/process.c
>  create mode 100644 helper/test/linux/pthread.c
>
> diff --git a/configure.ac b/configure.ac
> index daa9b31..b672a1a 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -138,18 +138,6 @@ AC_SUBST([with_platform])
>  AC_SUBST([platform_with_platform], ["platform/${with_platform}"])
>
>  ##
> -# Determine which helper platform to build for
> -##
> -AC_ARG_WITH([helper_platform],
> -[AS_HELP_STRING([--with-helper_platform=platform],
> -   [select helper platform to be used, default linux-generic])],
> -[],
> -[with_helper_platform=${with_platform}
> -])
> -
> -AC_SUBST([with_helper_platform])
> -
> -##
>  # Run platform specific checks and settings
>  ##
>  IMPLEMENTATION_NAME=""
> @@ -214,7 +202,7 @@ AM_CONDITIONAL([test_example], [test x$test_example = 
> xyes ])
>  AM_CONDITIONAL([HAVE_DOXYGEN], [test "x${DOXYGEN}" = "xdoxygen"])
>  AM_CONDITIONAL([user_guide], [test "x${user_guides}" = "xyes" ])
>  AM_CONDITIONAL([HAVE_MSCGEN], [test "x${MSCGEN}" = "xmscgen"])
> -AM_CONDITIONAL([helper_extn], [test x$helper_extn = xyes ])
> +AM_CONDITIONAL([helper_linux], [test x$helper_linux = xyes ])
>
>  ##
>  # Setup doxygen documentation
> @@ -345,8 +333,7 @@ AC_MSG_RESULT([
> implementation_name:${IMPLEMENTATION_NAME}
> ARCH_DIR${ARCH_DIR}
> with_platform:  ${with_platform}
> -   with_helper_platform:   ${with_helper_platform}
> -   helper_extn:${helper_extn}
> +   helper_linux:   ${helper_linux}
> prefix:  

Re: [lng-odp] [PATCH 1/2] linux-gen: dpdk: improve pmd driver linking

[Christophe Milard]

hmmm... that sound promising. thanks for the update. may I ask which
libtool version you tried with (latest)?

Thanks anyway!

Christophe.

On 2 February 2017 at 08:22, Elo, Matias (Nokia - FI/Espoo)
<matias@nokia-bell-labs.com> wrote:
>
>> On 1 Feb 2017, at 16:01, Christophe Milard <christophe.mil...@linaro.org> 
>> wrote:
>>
>> No, saddly. I got stuck on this.
>> I summed up the situation here:
>> https://lists.linaro.org/pipermail/lng-odp/2016-October/026120.html
>> ...
>> But if you get it to go, it is a good new: Just make sure that works
>> on the latest libtool/autotools: Going forward is OK. If it does not
>> work on latest, then, it is problematic...
>> (Cannot remember what version I got it to fail, to be honest...)
>>
>> Just let me know what you get to. I'd be glad if I am wrong (or things
>> have changed)
>> I think Krishna may want to know as well.
>>
>> Christophe.
>
> Good description of the problem. I checked my libtool/automake and I’m using 
> the latest version of both of them. Hopefully this also helps on your driver 
> issue.
>
> -Matias
>

Re: [lng-odp] [API-NEXT PATCH 15/15] linux-gen: support older libconfig version

[Christophe Milard]

needs rebase, I think:
erachmi@erachmi-ericsson:~/linaro/ODP/odp$ git am
~/incoming/lng-odp_API-NEXT_PATCH_*
Applying: merge fix: platform/linux-generic/Makefile.am
error: patch failed: platform/linux-generic/Makefile.am:20
error: platform/linux-generic/Makefile.am: patch does not apply
Patch failed at 0001 merge fix: platform/linux-generic/Makefile.am
The copy of the patch that failed is found in: .git/rebase-apply/patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

Tried both with patch 15 alone and the hole series...

sorry...

Christophe.

On 27 January 2017 at 15:29, Maxim Uvarov  wrote:
> Travis CI uses old libconfig8 amd64 1.3.2-2ubuntu2,
> which does not have defines for errors. Move error print
> under ifdef to fix compilation.
>
> Signed-off-by: Maxim Uvarov 
> ---
>  platform/linux-generic/odp_init.c | 7 ++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/platform/linux-generic/odp_init.c 
> b/platform/linux-generic/odp_init.c
> index 2cd8fe3f..685e02fa 100644
> --- a/platform/linux-generic/odp_init.c
> +++ b/platform/linux-generic/odp_init.c
> @@ -130,12 +130,17 @@ static int read_configfile(void)
> if (config_filename) {
> ODP_DBG("Reading configuration file: %s\n", config_filename);
> if (!config_read_file(cf, config_filename)) {
> +#if defined(LIBCONFIG_VER_MAJOR) && LIBCONFIG_VER_MAJOR >= 1 && \
> +   LIBCONFIG_VER_MINOR >= 4
> ODP_ERR("%s:%d - %s\n",
> config_error_file(cf),
> config_error_line(cf),
> config_error_text(cf));
> +#else
> +   ODP_ERR("config_read_file\n");
> +#endif
> config_destroy(cf);
> -   return(-1);
> +   return -1;
> }
> }
>
> --
> 2.11.0.295.gd7dffce
>

Re: [lng-odp] [PATCH 1/2] linux-gen: dpdk: improve pmd driver linking

[Christophe Milard]

No, saddly. I got stuck on this.
I summed up the situation here:
https://lists.linaro.org/pipermail/lng-odp/2016-October/026120.html
...
But if you get it to go, it is a good new: Just make sure that works
on the latest libtool/autotools: Going forward is OK. If it does not
work on latest, then, it is problematic...
(Cannot remember what version I got it to fail, to be honest...)

Just let me know what you get to. I'd be glad if I am wrong (or things
have changed)
I think Krishna may want to know as well.

Christophe.

On 1 February 2017 at 14:23, Elo, Matias (Nokia - FI/Espoo)
 wrote:
>
>>
>> As far as I remember, some autotools version won't allow that: it will
>> reject it with a message saying you are not allowed to set LDD libs
>> withing flags (LDFLAGS)...
>> What autotools version are you using?
>> Hopefully, new ones will no longer do that, but I remember trying
>> different versions...
>>
>> Christophe
>
>
> Interesting, I’m using autoconf v2.69 (automake v1.15). Any ideas where to 
> put this if AM_LDFLAGS is still causing problems? I tried LIBS but that 
> didn’t work.
>
> -Matias
>

Re: [lng-odp] [PATCH 1/2] linux-gen: dpdk: improve pmd driver linking

[Christophe Milard]

On 1 February 2017 at 13:47, Matias Elo  wrote:
> Previously each dpdk pmd driver had to be individually referred in the odp
> code to ensure proper gcc constructor linking. Using the -—whole-archive
> option when linking the drivers removes this need. After this patch new
> dpdk pmd drivers are automatically linked.
>
> Signed-off-by: Matias Elo 
> ---
>
> Putting the whole dpdk lib inside --whole-archive flags doesn't work as it
> breaks dpdk initialisation when building a shared odp library (dpdk gcc
> constructors get called multiple times).
>
>  platform/linux-generic/m4/odp_dpdk.m4 |  27 +++---
>  platform/linux-generic/pktio/dpdk.c   | 152 
> +-
>  2 files changed, 16 insertions(+), 163 deletions(-)
>
> diff --git a/platform/linux-generic/m4/odp_dpdk.m4 
> b/platform/linux-generic/m4/odp_dpdk.m4
> index 30347dc..c3ff39d 100644
> --- a/platform/linux-generic/m4/odp_dpdk.m4
> +++ b/platform/linux-generic/m4/odp_dpdk.m4
> @@ -2,22 +2,10 @@
>  # Enable DPDK support
>  ##
>  pktio_dpdk_support=no
> -AC_ARG_ENABLE([dpdk_support],
> -[  --enable-dpdk-support  include dpdk IO support],
> -[if test x$enableval = xyes; then
> -pktio_dpdk_support=yes
> -fi])
> -
> -##
> -# Set optional DPDK path
> -##
>  AC_ARG_WITH([dpdk-path],
> -AC_HELP_STRING([--with-dpdk-path=DIR   path to dpdk build directory],
> -   [(or in the default path if not specified).]),
> +AC_HELP_STRING([--with-dpdk-path=DIR   path to dpdk build directory]),
>  [DPDK_PATH=$withval
>  AM_CPPFLAGS="$AM_CPPFLAGS -msse4.2 -isystem $DPDK_PATH/include"
> -AM_LDFLAGS="$AM_LDFLAGS -L$DPDK_PATH/lib"
> -LIBS="$LIBS -ldpdk -ldl -lpcap"
>  pktio_dpdk_support=yes],[])
>
>  ##
> @@ -28,12 +16,25 @@ CPPFLAGS="$AM_CPPFLAGS $CPPFLAGS"
>
>  ##
>  # Check for DPDK availability
> +#
> +# DPDK pmd drivers are not linked unless the --whole-archive option is
> +# used. No spaces are allowed between the --whole-arhive flags.
>  ##
>  if test x$pktio_dpdk_support = xyes
>  then
>  AC_CHECK_HEADERS([rte_config.h], [],
>  [AC_MSG_FAILURE(["can't find DPDK header"])])
> +
> +DPDK_PMD=--whole-archive,
> +for filename in $with_dpdk_path/lib/*.a; do
> +DPDK_PMD+=`echo $(basename "$filename" .a) | \
> +sed -n 's/^\(librte_pmd_\)/-lrte_pmd_/p' | sed -n 's/$/,/p'`
> +done
> +DPDK_PMD+=--no-whole-archive
> +
>  ODP_CFLAGS="$ODP_CFLAGS -DODP_PKTIO_DPDK"
> +AM_LDFLAGS="$AM_LDFLAGS -L$DPDK_PATH/lib -Wl,$DPDK_PMD"

As far as I remember, some autotools version won't allow that: it will
reject it with a message saying you are not allowed to set LDD libs
withing flags (LDFLAGS)...
What autotools version are you using?
Hopefully, new ones will no longer do that, but I remember trying
different versions...

Christophe

> +LIBS="$LIBS -ldpdk -ldl -lpcap"
>  else
>  pktio_dpdk_support=no
>  fi
> diff --git a/platform/linux-generic/pktio/dpdk.c 
> b/platform/linux-generic/pktio/dpdk.c
> index 0eb025a..a8cd3c2 100644
> --- a/platform/linux-generic/pktio/dpdk.c
> +++ b/platform/linux-generic/pktio/dpdk.c
> @@ -31,49 +31,6 @@ static int disable_pktio; /** !0 this pktio disabled, 0 
> enabled */
>  /* Has dpdk_pktio_init() been called */
>  static odp_bool_t dpdk_initialized;
>
> -#define PMD_EXT(drv) \
> -extern void devinitfn_##drv(void)
> -
> -PMD_EXT(aesni_gcm_pmd_drv);
> -PMD_EXT(cryptodev_aesni_mb_pmd_drv);
> -PMD_EXT(cryptodev_kasumi_pmd_drv);
> -PMD_EXT(cryptodev_null_pmd_drv);
> -PMD_EXT(cryptodev_snow3g_pmd_drv);
> -PMD_EXT(pmd_qat_drv);
> -PMD_EXT(pmd_af_packet_drv);
> -PMD_EXT(rte_bnx2x_driver);
> -PMD_EXT(rte_bnx2xvf_driver);
> -PMD_EXT(bnxt_pmd_drv);
> -PMD_EXT(bond_drv);
> -PMD_EXT(rte_cxgbe_driver);
> -PMD_EXT(em_pmd_drv);
> -PMD_EXT(pmd_igb_drv);
> -PMD_EXT(pmd_igbvf_drv);
> -PMD_EXT(ena_pmd_drv);
> -PMD_EXT(rte_enic_driver);
> -PMD_EXT(rte_fm10k_driver);
> -PMD_EXT(rte_i40e_driver);
> -PMD_EXT(rte_i40evf_driver);
> -PMD_EXT(rte_ixgbe_driver);
> -PMD_EXT(rte_ixgbevf_driver);
> -PMD_EXT(rte_mlx4_driver);
> -PMD_EXT(rte_mlx5_driver);
> -PMD_EXT(pmd_mpipe_xgbe_drv);
> -PMD_EXT(pmd_mpipe_gbe_drv);
> -PMD_EXT(rte_nfp_net_driver);
> -PMD_EXT(pmd_null_drv);
> -PMD_EXT(pmd_pcap_drv);
> -PMD_EXT(rte_qede_driver);
> -PMD_EXT(rte_qedevf_driver);
> -PMD_EXT(pmd_ring_drv);
> -PMD_EXT(pmd_szedata2_drv);
> -PMD_EXT(rte_nicvf_driver);
> -PMD_EXT(pmd_vhost_drv);
> -PMD_EXT(rte_virtio_driver);
> -PMD_EXT(virtio_user_driver);
> -PMD_EXT(rte_vmxnet3_driver);
> 

Re: [lng-odp] [PATCH 0/4] introduce odph_api.h and clean up public helper API

[Christophe Milard]

For the series:

Reviewed-by: Christophe Milard <christophe.mil...@linaro.org>

On 23 January 2017 at 20:46, Mike Holmes <mike.hol...@linaro.org> wrote:
> Greatly reduce the proliferation of helper includes that every app needs
> Make the public helper API very obvious
> Fix recent inclusion of table APIs that were not in the helper include dir and
> were not exported during install.
>
> Mike Holmes (4):
>   helper: add odph_api.h for existing exported headers
>   helper: use odph_api.h for test include for unexported files
>   test: use odph_api.h
>   examples: use odph_api.h
>
>  example/classifier/odp_classifier.c|  4 +--
>  example/generator/odp_generator.c  |  6 +---
>  example/ipsec/odp_ipsec.c  |  6 +---
>  example/ipsec/odp_ipsec_fwd_db.h   |  1 -
>  example/ipsec/odp_ipsec_loop_db.h  |  1 -
>  example/ipsec/odp_ipsec_misc.h |  4 +--
>  example/ipsec/odp_ipsec_stream.c   |  4 +--
>  example/l2fwd_simple/odp_l2fwd_simple.c|  4 +--
>  example/l3fwd/odp_l3fwd.c  |  6 +---
>  example/l3fwd/odp_l3fwd_db.h   |  2 +-
>  example/packet/odp_pktio.c |  4 +--
>  example/switch/odp_switch.c|  4 +--
>  example/time/time_global_test.c|  2 +-
>  example/timer/odp_timer_test.c |  2 +-
>  helper/Makefile.am | 11 +++---
>  helper/cuckootable.c   |  2 +-
>  helper/hashtable.c |  2 +-
>  helper/include/odp/helper/odph_api.h   | 39 
> ++
>  helper/{ => include/odp/helper}/odph_cuckootable.h |  0
>  helper/{ => include/odp/helper}/odph_hashtable.h   |  0
>  .../{ => include/odp/helper}/odph_iplookuptable.h  |  0
>  helper/{ => include/odp/helper}/odph_lineartable.h |  0
>  helper/iplookuptable.c |  2 +-
>  helper/lineartable.c   |  2 +-
>  helper/test/chksum.c   |  4 +--
>  helper/test/cuckootable.c  |  2 +-
>  helper/test/iplookuptable.c|  2 +-
>  helper/test/odpthreads.c   |  2 +-
>  helper/test/parse.c|  3 +-
>  helper/test/table.c|  3 +-
>  test/common_plat/common/odp_cunit_common.c |  2 +-
>  test/common_plat/performance/odp_crypto.c  |  2 +-
>  test/common_plat/performance/odp_l2fwd.c   |  4 +--
>  test/common_plat/performance/odp_pktio_perf.c  |  5 +--
>  test/common_plat/performance/odp_sched_latency.c   |  2 +-
>  test/common_plat/performance/odp_scheduling.c  |  2 +-
>  .../api/classification/odp_classification_common.c |  4 ---
>  .../classification/odp_classification_test_pmr.c   |  4 ---
>  .../api/classification/odp_classification_tests.c  |  4 ---
>  .../classification/odp_classification_testsuites.h |  1 +
>  test/common_plat/validation/api/pktio/pktio.c  |  4 +--
>  test/common_plat/validation/api/timer/timer.c  |  2 +-
>  .../validation/api/traffic_mngr/traffic_mngr.c |  6 +---
>  test/linux-generic/mmap_vlan_ins/mmap_vlan_ins.c   |  4 +--
>  test/linux-generic/pktio_ipc/ipc_common.h  |  5 +--
>  test/linux-generic/ring/ring_stress.c  |  2 +-
>  46 files changed, 80 insertions(+), 97 deletions(-)
>  create mode 100644 helper/include/odp/helper/odph_api.h
>  rename helper/{ => include/odp/helper}/odph_cuckootable.h (100%)
>  rename helper/{ => include/odp/helper}/odph_hashtable.h (100%)
>  rename helper/{ => include/odp/helper}/odph_iplookuptable.h (100%)
>  rename helper/{ => include/odp/helper}/odph_lineartable.h (100%)
>
> --
> 2.9.3

Re: [lng-odp] [PATCH 2/4] helper: use odph_api.h for test include for unexported files

[Christophe Milard]

On 25 January 2017 at 15:17, Mike Holmes <mike.hol...@linaro.org> wrote:
> Yes, it should be < > I think
>
> From stackoverflow
> For #include "filename" the preprocessor searches in the same
> directory as the file containing the directive. This method is
> normally used to include programmer-defined header files.
>
> For #include  the preprocessor searches in an implementation
> dependent manner, normally in search directories pre-designated by the
> compiler/IDE. This method is normally used to include standard library
> header files.

I understand. using <> will confuse CPP if we have filename matching
the standart linux names. I remember having issues with errno.h in the
past. I cannot remember how I fixed it :-). Anyway not really part of
this patchset.

>
> On 25 January 2017 at 09:09, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>> On 23 January 2017 at 20:46, Mike Holmes <mike.hol...@linaro.org> wrote:
>>> Adding the previously missing table functions to the public helper api
>>> allows the tests to use just odph_api.h
>>>
>>> Signed-off-by: Mike Holmes <mike.hol...@linaro.org>
>>> ---
>>>  helper/Makefile.am   | 10 +-
>>>  helper/cuckootable.c |  2 +-
>>>  helper/hashtable.c   |  2 +-
>>>  helper/include/odp/helper/odph_api.h |  4 
>>>  helper/{ => include/odp/helper}/odph_cuckootable.h   |  0
>>>  helper/{ => include/odp/helper}/odph_hashtable.h |  0
>>>  helper/{ => include/odp/helper}/odph_iplookuptable.h |  0
>>>  helper/{ => include/odp/helper}/odph_lineartable.h   |  0
>>>  helper/iplookuptable.c   |  2 +-
>>>  helper/lineartable.c |  2 +-
>>>  helper/test/cuckootable.c|  2 +-
>>>  helper/test/iplookuptable.c  |  2 +-
>>>  helper/test/table.c  |  3 +--
>>>  13 files changed, 16 insertions(+), 13 deletions(-)
>>>  rename helper/{ => include/odp/helper}/odph_cuckootable.h (100%)
>>>  rename helper/{ => include/odp/helper}/odph_hashtable.h (100%)
>>>  rename helper/{ => include/odp/helper}/odph_iplookuptable.h (100%)
>>>  rename helper/{ => include/odp/helper}/odph_lineartable.h (100%)
>>>
>>> diff --git a/helper/Makefile.am b/helper/Makefile.am
>>> index 76cad1b..4d24a69 100644
>>> --- a/helper/Makefile.am
>>> +++ b/helper/Makefile.am
>>> @@ -20,6 +20,10 @@ helperinclude_HEADERS = \
>>>   $(srcdir)/include/odp/helper/ip.h\
>>>   $(srcdir)/include/odp/helper/ipsec.h\
>>>   $(srcdir)/include/odp/helper/odph_api.h\
>>> + $(srcdir)/include/odp/helper/odph_cuckootable.h\
>>> + $(srcdir)/include/odp/helper/odph_hashtable.h\
>>> + $(srcdir)/include/odp/helper/odph_iplookuptable.h\
>>> + $(srcdir)/include/odp/helper/odph_lineartable.h\
>>>   $(srcdir)/include/odp/helper/strong_types.h\
>>>   $(srcdir)/include/odp/helper/tcp.h\
>>>   $(srcdir)/include/odp/helper/table.h\
>>> @@ -33,11 +37,7 @@ endif
>>>
>>>  noinst_HEADERS = \
>>>  $(srcdir)/odph_debug.h \
>>> -$(srcdir)/odph_hashtable.h \
>>> -$(srcdir)/odph_lineartable.h \
>>> -$(srcdir)/odph_cuckootable.h \
>>> -$(srcdir)/odph_list_internal.h \
>>> -$(srcdir)/odph_iplookuptable.h
>>> +$(srcdir)/odph_list_internal.h
>>>
>>>  __LIB__libodphelper_@with_platform@_la_SOURCES = \
>>> eth.c \
>>> diff --git a/helper/cuckootable.c b/helper/cuckootable.c
>>> index b4fce6c..83647ec 100644
>>> --- a/helper/cuckootable.c
>>> +++ b/helper/cuckootable.c
>>> @@ -42,7 +42,7 @@
>>>  #include 
>>>  #include 
>>>
>>> -#include "odph_cuckootable.h"
>>> +#include "odp/helper/odph_cuckootable.h"
>>>  #include "odph_debug.h"
>>>  #include 
>>>
>>> diff --git a/helper/hashtable.c b/helper/hashtable.c
>>> index 8bb1ae5..983b3da 100644
>>> --- a/helper/hashtable.c
>>> +++ b/helper/hashtable.c
>>> @@ -7,7 +7,7 @@
>>&

Re: [lng-odp] [PATCH 0/4] introduce odph_api.h and clean up public helper API

[Christophe Milard]

On 24 January 2017 at 16:51, Mike Holmes  wrote:
> On 24 January 2017 at 03:23, Savolainen, Petri (Nokia - FI/Espoo)
>  wrote:
>>
>>
>>> -Original Message-
>>> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of Mike
>>> Holmes
>>> Sent: Monday, January 23, 2017 9:46 PM
>>> To: lng-odp@lists.linaro.org
>>> Subject: [lng-odp] [PATCH 0/4] introduce odph_api.h and clean up public
>>> helper API
>>>
>>> Greatly reduce the proliferation of helper includes that every app needs
>>> Make the public helper API very obvious
>>> Fix recent inclusion of table APIs that were not in the helper include dir
>>> and
>>> were not exported during install.
>>
>> I think we should not do this (combine all helpers into an "API").
>>
>> First, "odph_api.h" gives the impression that helper definitions are part of 
>> ODP API.
>
> I dont think it is confusing, but we can use odp_helper_ap.h rather
> than odph_api and we do need a stable helper API so that apps are
> portable
>
> Those are not, since e.g. Ethernet header struct definition is not
> needed for HW acceleration
>
> Agree that is why they are helpers
>
>  - it's needed (reused) for writing tests and examples. We could
> decide to create and maintain an "ODP protocol suite" which would
> follow RFC's closely and provide those struct definitions for tens of
> protocols, all modes, options, etc included. Anyway, helper is not
> that today.
>
> We can in future make better helper distinctions, but to date it has
> all been in one, and it has been in a confusing state for a long time,
> just look at the latest table stuff that was incorrectly accepted.
> I can see OS_helpers, protocol_helpers, algorithm_helpers (tables) etc
> all being their own mini lib that is ABI  portable and installed once.

That is what I like "odph_api": i.e. _. If ODP
helpers is to be splitted into different parts, this kind of naming
structure make sense. like future odph_ipheaders, odph_os ...
Not sure about the "api" part, but that can be changed when the
different blocks get clear.

>
>>
>> Secondly, when application does this ...
>>
>> #include 
>> #include 
>> #include 
>>
>> ... instead of this.
>>
>> #include 
>>
>> It is very easy to see what is happening, and what (extra) dependencies 
>> application has.
>
> The same for ODP, we need to pick one mechanism or another, I am happy
> to agree if ODP also includes everything in parts, but having it in
> one file makes the portable API for helper very concrete, it is need
> to make the helpers be ABI compatible which is needed for the cloud.

Agree with the consistency: if we accept a single odp_api.h why not
the same for helpers?


I am rather positive to these changes: they gather the different
helper interfaces in one spot, making clearer what it the helper api
and what is not.
The different API "subblocks" are not clear here, that could be improved later.

Christophe.
>
>>
>>
>> -Petri
>>
>>
>>
>>
>
>
>
> --
> Mike Holmes
> Program Manager - Linaro Networking Group
> Linaro.org │ Open source software for ARM SoCs
> "Work should be fun and collaborative, the rest follows"

Re: [lng-odp] [PATCH 2/4] helper: use odph_api.h for test include for unexported files

[Christophe Milard]

On 23 January 2017 at 20:46, Mike Holmes  wrote:
> Adding the previously missing table functions to the public helper api
> allows the tests to use just odph_api.h
>
> Signed-off-by: Mike Holmes 
> ---
>  helper/Makefile.am   | 10 +-
>  helper/cuckootable.c |  2 +-
>  helper/hashtable.c   |  2 +-
>  helper/include/odp/helper/odph_api.h |  4 
>  helper/{ => include/odp/helper}/odph_cuckootable.h   |  0
>  helper/{ => include/odp/helper}/odph_hashtable.h |  0
>  helper/{ => include/odp/helper}/odph_iplookuptable.h |  0
>  helper/{ => include/odp/helper}/odph_lineartable.h   |  0
>  helper/iplookuptable.c   |  2 +-
>  helper/lineartable.c |  2 +-
>  helper/test/cuckootable.c|  2 +-
>  helper/test/iplookuptable.c  |  2 +-
>  helper/test/table.c  |  3 +--
>  13 files changed, 16 insertions(+), 13 deletions(-)
>  rename helper/{ => include/odp/helper}/odph_cuckootable.h (100%)
>  rename helper/{ => include/odp/helper}/odph_hashtable.h (100%)
>  rename helper/{ => include/odp/helper}/odph_iplookuptable.h (100%)
>  rename helper/{ => include/odp/helper}/odph_lineartable.h (100%)
>
> diff --git a/helper/Makefile.am b/helper/Makefile.am
> index 76cad1b..4d24a69 100644
> --- a/helper/Makefile.am
> +++ b/helper/Makefile.am
> @@ -20,6 +20,10 @@ helperinclude_HEADERS = \
>   $(srcdir)/include/odp/helper/ip.h\
>   $(srcdir)/include/odp/helper/ipsec.h\
>   $(srcdir)/include/odp/helper/odph_api.h\
> + $(srcdir)/include/odp/helper/odph_cuckootable.h\
> + $(srcdir)/include/odp/helper/odph_hashtable.h\
> + $(srcdir)/include/odp/helper/odph_iplookuptable.h\
> + $(srcdir)/include/odp/helper/odph_lineartable.h\
>   $(srcdir)/include/odp/helper/strong_types.h\
>   $(srcdir)/include/odp/helper/tcp.h\
>   $(srcdir)/include/odp/helper/table.h\
> @@ -33,11 +37,7 @@ endif
>
>  noinst_HEADERS = \
>  $(srcdir)/odph_debug.h \
> -$(srcdir)/odph_hashtable.h \
> -$(srcdir)/odph_lineartable.h \
> -$(srcdir)/odph_cuckootable.h \
> -$(srcdir)/odph_list_internal.h \
> -$(srcdir)/odph_iplookuptable.h
> +$(srcdir)/odph_list_internal.h
>
>  __LIB__libodphelper_@with_platform@_la_SOURCES = \
> eth.c \
> diff --git a/helper/cuckootable.c b/helper/cuckootable.c
> index b4fce6c..83647ec 100644
> --- a/helper/cuckootable.c
> +++ b/helper/cuckootable.c
> @@ -42,7 +42,7 @@
>  #include 
>  #include 
>
> -#include "odph_cuckootable.h"
> +#include "odp/helper/odph_cuckootable.h"
>  #include "odph_debug.h"
>  #include 
>
> diff --git a/helper/hashtable.c b/helper/hashtable.c
> index 8bb1ae5..983b3da 100644
> --- a/helper/hashtable.c
> +++ b/helper/hashtable.c
> @@ -7,7 +7,7 @@
>  #include 
>  #include 
>
> -#include "odph_hashtable.h"
> +#include "odp/helper/odph_hashtable.h"
>  #include "odph_list_internal.h"
>  #include "odph_debug.h"
>  #include 
> diff --git a/helper/include/odp/helper/odph_api.h 
> b/helper/include/odp/helper/odph_api.h
> index ae6e77b..7ed0e77 100644
> --- a/helper/include/odp/helper/odph_api.h
> +++ b/helper/include/odp/helper/odph_api.h
> @@ -19,10 +19,14 @@ extern "C" {
>  #endif
>
>  #include 
> +#include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> +#include 
> +#include 
>  #include 
>  #include 
>  #include 
> diff --git a/helper/odph_cuckootable.h 
> b/helper/include/odp/helper/odph_cuckootable.h
> similarity index 100%
> rename from helper/odph_cuckootable.h
> rename to helper/include/odp/helper/odph_cuckootable.h
> diff --git a/helper/odph_hashtable.h 
> b/helper/include/odp/helper/odph_hashtable.h
> similarity index 100%
> rename from helper/odph_hashtable.h
> rename to helper/include/odp/helper/odph_hashtable.h
> diff --git a/helper/odph_iplookuptable.h 
> b/helper/include/odp/helper/odph_iplookuptable.h
> similarity index 100%
> rename from helper/odph_iplookuptable.h
> rename to helper/include/odp/helper/odph_iplookuptable.h
> diff --git a/helper/odph_lineartable.h 
> b/helper/include/odp/helper/odph_lineartable.h
> similarity index 100%
> rename from helper/odph_lineartable.h
> rename to helper/include/odp/helper/odph_lineartable.h
> diff --git a/helper/iplookuptable.c b/helper/iplookuptable.c
> index 5f80743..f6d6f88 100644
> --- a/helper/iplookuptable.c
> +++ b/helper/iplookuptable.c
> @@ -9,7 +9,7 @@
>  #include 
>  #include 
>
> -#include "odph_iplookuptable.h"
> +#include 

Do we have any rule regarding the usage of #include"file" vs #include ?

>  #include 

[lng-odp] Christophe OOO 26-31 january

[Christophe Milard]

I'll miss you all :-)
Christophe.

[lng-odp] [API-NEXT PATCHv11 4/5] test: preventing odp.conf loading for tests

[Christophe Milard]

The tests should not be affected by any system or user ODP configuration
file. The ODP_SYSCONFIG_FILE environment variables is therefore set
to "none" in TESTS_ENVIRONMENT.
Tests which need specific a configuration file will have to overwrite
this setting.
Note that tests ran manually (not using make check) may be affected
by configuration files. Setting ODP_SYSCONFIG_FILE to an appropriate value
(possibly "none") may be required.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 test/Makefile.inc | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/test/Makefile.inc b/test/Makefile.inc
index 1ebc047..896f863 100644
--- a/test/Makefile.inc
+++ b/test/Makefile.inc
@@ -22,4 +22,6 @@ AM_LDFLAGS += -L$(LIB)
 @VALGRIND_CHECK_RULES@
 valgrind_tools = memcheck
 
-TESTS_ENVIRONMENT= ODP_PLATFORM=${with_platform} EXEEXT=${EXEEXT}
+TESTS_ENVIRONMENT = ODP_PLATFORM=${with_platform} \
+   EXEEXT=${EXEEXT} \
+   ODP_SYSCONFIG_FILE=none
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv11 5/5] linux-gen: modules: adding initial file to load modules

[Christophe Milard]

The shared objects listed in the ODP configuration files are
loaded at init time. The odp configuration file lists the
shared objects to be loaded as shown in the following example:
module = {
modules = ["enumerator1.so", "driver1.so"];
};

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 configure.ac  |  4 +-
 platform/linux-generic/Makefile.am|  1 +
 platform/linux-generic/_modules.c | 53 +++
 platform/linux-generic/include/odp_internal.h |  3 ++
 platform/linux-generic/m4/configure.m4|  1 +
 platform/linux-generic/m4/odp_modules.m4  | 11 ++
 platform/linux-generic/odp_init.c |  7 
 7 files changed, 78 insertions(+), 2 deletions(-)
 create mode 100644 platform/linux-generic/_modules.c
 create mode 100644 platform/linux-generic/m4/odp_modules.m4

diff --git a/configure.ac b/configure.ac
index 3a20959..e2b4f9d 100644
--- a/configure.ac
+++ b/configure.ac
@@ -55,7 +55,7 @@ AC_PROG_MAKE_SET
 
 AM_PROG_AR
 #Use libtool
-LT_INIT([])
+LT_INIT([dlopen])
 AC_SUBST([LIBTOOL_DEPS])
 AM_PROG_LIBTOOL
 
@@ -66,7 +66,7 @@ AC_CHECK_FUNCS([bzero clock_gettime gethostbyname getpagesize 
gettimeofday memse
 
 # Checks for header files.
 AC_HEADER_RESOLV
-AC_CHECK_HEADERS([arpa/inet.h fcntl.h inttypes.h limits.h netdb.h netinet/in.h 
stddef.h stdint.h stdlib.h string.h sys/ioctl.h sys/socket.h sys/time.h 
unistd.h])
+AC_CHECK_HEADERS([arpa/inet.h fcntl.h inttypes.h limits.h netdb.h netinet/in.h 
stddef.h stdint.h stdlib.h string.h sys/ioctl.h sys/socket.h sys/time.h 
unistd.h dlfcn.h])
 
 # Checks for typedefs, structures, and compiler characteristics.
 AC_HEADER_STDBOOL
diff --git a/platform/linux-generic/Makefile.am 
b/platform/linux-generic/Makefile.am
index 579fc3c..f4ef497 100644
--- a/platform/linux-generic/Makefile.am
+++ b/platform/linux-generic/Makefile.am
@@ -177,6 +177,7 @@ __LIB__libodp_linux_la_SOURCES = \
   _ishm.c \
   _ishmphy.c \
   _ishmpool.c \
+  _modules.c \
   odp_atomic.c \
   odp_barrier.c \
   odp_bitmap.c \
diff --git a/platform/linux-generic/_modules.c 
b/platform/linux-generic/_modules.c
new file mode 100644
index 000..6bb854e
--- /dev/null
+++ b/platform/linux-generic/_modules.c
@@ -0,0 +1,53 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+static int load_modules(void)
+{
+   config_t *cf;
+   const config_setting_t *modules_section;
+   int module_count;
+   int i;
+   const char *module_name;
+
+   cf = _global_data.configuration;
+   modules_section = config_lookup(cf, "module.modules");
+   if (!modules_section)
+   return 0;
+
+   module_count = config_setting_length(modules_section);
+   if (!module_count)
+   return 0;
+
+   for (i = 0; i < module_count; i++) {
+   module_name = config_setting_get_string_elem(modules_section,
+i);
+   if (dlopen(module_name, RTLD_NOW) == NULL) {
+   ODP_ERR("dlopen failed for %s: %s\n",
+   module_name, dlerror());
+   return -1;
+   }
+   ODP_DBG("module %s loaded.\n", module_name);
+   }
+
+   return 0;
+}
+
+int _odp_modules_init_global(void)
+{
+   /* load modules (enumerator and drivers...) */
+   if (load_modules())
+   return -1;
+
+   return 0;
+}
diff --git a/platform/linux-generic/include/odp_internal.h 
b/platform/linux-generic/include/odp_internal.h
index 9d1fc58..05c8a42 100644
--- a/platform/linux-generic/include/odp_internal.h
+++ b/platform/linux-generic/include/odp_internal.h
@@ -71,6 +71,7 @@ enum init_stage {
CLASSIFICATION_INIT,
TRAFFIC_MNGR_INIT,
NAME_TABLE_INIT,
+   MODULES_INIT,
ALL_INIT  /* All init stages completed */
 };
 
@@ -129,6 +130,8 @@ int _odp_ishm_init_local(void);
 int _odp_ishm_term_global(void);
 int _odp_ishm_term_local(void);
 
+int _odp_modules_init_global(void);
+
 int cpuinfo_parser(FILE *file, system_info_t *sysinfo);
 uint64_t odp_cpu_hz_current(int id);
 
diff --git a/platform/linux-generic/m4/configure.m4 
b/platform/linux-generic/m4/configure.m4
index 5fab0cc..f5f076a 100644
--- a/platform/linux-generic/m4/configure.m4
+++ b/platform/linux-generic/m4/configure.m4
@@ -42,6 +42,7 @@ fi
 m4_include([platform/linux-generic/m4/odp_pthread.m4])
 m4_include([platform/linux-generic/m4/odp_openssl.m4])
 m4_include([platform/linux-generic/m4/odp_pcap.m4])
+m4_include([platform/linux-generic

[lng-odp] [API-NEXT PATCHv11 3/5] linux-gen: init: adding configuration file parsing

[Christophe Milard]

The parsing of the odp.conf configuration file is added.
The file is searched first in filename specified in the environment
variable $ODP_SYSCONFIG_FILE (where the special string "none" prevent
any file loading)
The file is then searched in the user home directory (~) and
then is the $prefix/etc directory.
This requires libconfig (sudo apt-get install libconfig-dev)

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 .travis.yml   |  2 +-
 DEPENDENCIES  |  8 +--
 platform/linux-generic/Makefile.am|  1 +
 platform/linux-generic/include/odp_internal.h |  2 +
 platform/linux-generic/m4/configure.m4| 11 
 platform/linux-generic/odp_init.c | 80 +++
 6 files changed, 99 insertions(+), 5 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 03e61b1..a6f770c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -19,7 +19,7 @@ sudo: required
 
 before_install:
 - sudo apt-get -qq update
-- sudo apt-get install automake autoconf libtool libssl-dev graphviz 
mscgen doxygen
+- sudo apt-get install automake autoconf libtool libssl-dev graphviz 
mscgen doxygen libconfig-dev
 - sudo apt-get install libpcap-dev linux-headers-`uname -r`
 - gem install asciidoctor
 
diff --git a/DEPENDENCIES b/DEPENDENCIES
index f1f0edd..f285783 100644
--- a/DEPENDENCIES
+++ b/DEPENDENCIES
@@ -18,18 +18,18 @@ Prerequisites for building the OpenDataPlane (ODP) API
 
 3. Required libraries
 
-   Libraries currently required to link: openssl
+   Libraries currently required to link: openssl, libconfig
 
-3.1 OpenSSL native compile
+3.1 OpenSSL and libconf native compile
 
For native compilation, simply load the necessary libraries using the 
appropriate
tool set.
 
On Debian/Ubuntu systems:
-   $ sudo apt-get install libssl-dev
+   $ sudo apt-get install libssl-dev libconfig-dev
 
On CentOS/RedHat/Fedora systems:
-   $ sudo yum install openssl-devel
+   $ sudo yum install openssl-devel libconfig-devel
 
 3.2 OpenSSL cross compilation
 
diff --git a/platform/linux-generic/Makefile.am 
b/platform/linux-generic/Makefile.am
index 2d97414..579fc3c 100644
--- a/platform/linux-generic/Makefile.am
+++ b/platform/linux-generic/Makefile.am
@@ -8,6 +8,7 @@ AM_CFLAGS +=  -I$(srcdir)/include
 AM_CFLAGS +=  -I$(top_srcdir)/include
 AM_CFLAGS +=  -I$(top_builddir)/include
 AM_CFLAGS +=  -Iinclude
+AM_CFLAGS +=  -DSYSCONFDIR=\"@sysconfdir@\"
 
 include_HEADERS = \
  $(top_srcdir)/include/odp.h \
diff --git a/platform/linux-generic/include/odp_internal.h 
b/platform/linux-generic/include/odp_internal.h
index b313b1f..9d1fc58 100644
--- a/platform/linux-generic/include/odp_internal.h
+++ b/platform/linux-generic/include/odp_internal.h
@@ -22,6 +22,7 @@ extern "C" {
 #include 
 #include 
 #include 
+#include 
 
 extern __thread int __odp_errno;
 
@@ -50,6 +51,7 @@ struct odp_global_data_s {
odp_cpumask_t control_cpus;
odp_cpumask_t worker_cpus;
int num_cpus_installed;
+   config_t configuration;
 };
 
 enum init_stage {
diff --git a/platform/linux-generic/m4/configure.m4 
b/platform/linux-generic/m4/configure.m4
index d3e5528..5fab0cc 100644
--- a/platform/linux-generic/m4/configure.m4
+++ b/platform/linux-generic/m4/configure.m4
@@ -28,6 +28,17 @@ AC_LINK_IFELSE(
 echo "Use newer version. For gcc > 4.7.0"
 exit -1)
 
+# Check for libconfig (required)
+AC_CHECK_HEADERS([libconfig.h], HEADER_LIBCONFIG="yes")
+PKG_CHECK_MODULES([PKGCONFIG], [libconfig >= 1.3.2], LIBRARY_LIBCONFIG="yes")
+if test "x$LIBRARY_LIBCONFIG" != "x" && test "x$HEADER_LIBCONFIG" != "x" ; then
+CFLAGS="$CFLAGS $PKGCONFIG_CFLAGS"
+LIBS="$LIBS $PKGCONFIG_LIBS"
+AM_CPPFLAGS="$AM_CPPFLAGS `pkg-config --cflags-only-I libconfig`"
+else
+AC_MSG_FAILURE([libconfig not found (required)])
+fi
+
 m4_include([platform/linux-generic/m4/odp_pthread.m4])
 m4_include([platform/linux-generic/m4/odp_openssl.m4])
 m4_include([platform/linux-generic/m4/odp_pcap.m4])
diff --git a/platform/linux-generic/odp_init.c 
b/platform/linux-generic/odp_init.c
index 06c6143..0e9167c 100644
--- a/platform/linux-generic/odp_init.c
+++ b/platform/linux-generic/odp_init.c
@@ -10,6 +10,8 @@
 #include 
 #include 
 #include 
+#include 
+#include 
 #include 
 #include 
 #include 
@@ -21,6 +23,15 @@
 #define _ODP_FILES_FMT "odp-%d-"
 #define _ODP_TMPDIR"/tmp"
 
+/* the name of the ODP configuration file: */
+#define CONFIGURATION_FILE_ENV_NONE "none"
+#define CONFIGURATION_FILE "odp.conf"
+#define CONFIGURATION_FILE_USR ("." CONFIGURATION_FILE)
+#define CONFIGURATION_FILE_SYS (SYSCONFDIR "/" CONFIGURATION_FILE)
+
+/* the ODP configuration file name can also be 

[lng-odp] [API-NEXT PATCHv11 2/5] linux-gen: adding enum, devio and driver registration interface (stub)

[Christophe Milard]

The linux implementation for the enumerator class registration function,
enumerator instance registration function,
devio and driver registration functions (stub)

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp_drv.h  |  1 +
 platform/linux-generic/Makefile.am |  3 +
 platform/linux-generic/drv_driver.c| 64 ++
 platform/linux-generic/include/odp/drv/driver.h| 37 +
 .../include/odp/drv/plat/driver_types.h| 52 ++
 5 files changed, 157 insertions(+)
 create mode 100644 platform/linux-generic/drv_driver.c
 create mode 100644 platform/linux-generic/include/odp/drv/driver.h
 create mode 100644 platform/linux-generic/include/odp/drv/plat/driver_types.h

diff --git a/include/odp_drv.h b/include/odp_drv.h
index 0959879..96d81ba 100644
--- a/include/odp_drv.h
+++ b/include/odp_drv.h
@@ -23,6 +23,7 @@ extern C {
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
diff --git a/platform/linux-generic/Makefile.am 
b/platform/linux-generic/Makefile.am
index 74e606d..2d97414 100644
--- a/platform/linux-generic/Makefile.am
+++ b/platform/linux-generic/Makefile.am
@@ -109,6 +109,7 @@ odpdrvinclude_HEADERS = \
  $(srcdir)/include/odp/drv/barrier.h \
  $(srcdir)/include/odp/drv/byteorder.h \
  $(srcdir)/include/odp/drv/compiler.h \
+ $(srcdir)/include/odp/drv/driver.h \
  $(srcdir)/include/odp/drv/shm.h \
  $(srcdir)/include/odp/drv/spinlock.h \
  $(srcdir)/include/odp/drv/std_types.h \
@@ -119,6 +120,7 @@ odpdrvplatinclude_HEADERS = \
  $(srcdir)/include/odp/drv/plat/atomic_types.h \
  $(srcdir)/include/odp/drv/plat/barrier_types.h \
  $(srcdir)/include/odp/drv/plat/byteorder_types.h \
+ $(srcdir)/include/odp/drv/plat/driver_types.h \
  $(srcdir)/include/odp/drv/plat/shm_types.h \
  $(srcdir)/include/odp/drv/plat/spinlock_types.h \
  $(srcdir)/include/odp/drv/plat/strong_types.h
@@ -233,6 +235,7 @@ __LIB__libodp_linux_la_SOURCES = \
   odp_weak.c \
   drv_atomic.c \
   drv_barrier.c \
+  drv_driver.c \
   drv_shm.c \
   drv_spinlock.c \
   arch/@ARCH_DIR@/odp_cpu_arch.c \
diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
new file mode 100644
index 000..529da48
--- /dev/null
+++ b/platform/linux-generic/drv_driver.c
@@ -0,0 +1,64 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+
+#include 
+#include 
+#include 
+#include 
+
+odpdrv_enumr_class_t odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
+*param)
+{
+   ODP_ERR("NOT Supported yet! Enumerator Class %s Registration!\n.",
+   param->name);
+
+   return ODPDRV_ENUMR_CLASS_INVALID;
+}
+
+odpdrv_enumr_t odpdrv_enumr_register(odpdrv_enumr_param_t *param)
+{
+   ODP_ERR("NOT Supported yet! Enumerator API %s Registration!\n.",
+   param->api_name);
+
+   return ODPDRV_ENUMR_INVALID;
+}
+
+odpdrv_device_t odpdrv_device_create(odpdrv_device_param_t *param)
+{
+   ODP_ERR("odpdrv_device_create not Supported yet! devaddress: %s\n.",
+   param->address);
+   return ODPDRV_DEVICE_INVALID;
+}
+
+void odpdrv_device_destroy(odpdrv_device_t dev)
+{
+   if (dev == ODPDRV_DEVICE_INVALID)
+   ODP_ERR("Invalid device\n");
+}
+
+odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t *param)
+{
+   ODP_ERR("NOT Supported yet! Driver %s Registration!\n.",
+   param->api_name);
+
+   return ODPDRV_DEVIO_INVALID;
+}
+
+odpdrv_driver_t odpdrv_driver_register(odpdrv_driver_param_t *param)
+{
+   ODP_ERR("NOT Supported yet! Driver %s Registration!\n.",
+   param->name);
+
+   return ODPDRV_DRIVER_INVALID;
+}
+
+int odpdrv_print_all(void)
+{
+   ODP_ERR("odpdrv_print_all not Supported yet!\n.");
+   return 0;
+}
diff --git a/platform/linux-generic/include/odp/drv/driver.h 
b/platform/linux-generic/include/odp/drv/driver.h
new file mode 100644
index 000..b12c83d
--- /dev/null
+++ b/platform/linux-generic/include/odp/drv/driver.h
@@ -0,0 +1,37 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+/**
+ * @file
+ *
+ * ODPDRV drivers
+ */
+
+#ifndef ODPDRV_PLAT_DRIVER_H_
+#define ODPDRV_PLAT_DRIVER_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#inc

[lng-odp] [API-NEXT PATCHv11 1/5] drv: adding driver registration interface (stub)

[Christophe Milard]

The enumerator class, enumerator instance, devio and driver registration
functions prototypes (and a draft of their parameters) are
defined, the goal being to define the registration framework only.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 389 ++
 platform/Makefile.inc |   1 +
 2 files changed, 390 insertions(+)
 create mode 100644 include/odp/drv/spec/driver.h

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
new file mode 100644
index 000..d83e907
--- /dev/null
+++ b/include/odp/drv/spec/driver.h
@@ -0,0 +1,389 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+/**
+ * @file
+ *
+ * ODPDRV driver
+ */
+
+#ifndef ODPDRV_DRIVER_H_
+#define ODPDRV_DRIVER_H_
+#include 
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+* @addtogroup odpdrv_driver
+* @details
+* enumerator and driver interface to ODP
+*
+*  1) ODP loads the different modules (i.e. it loads shared libraries, *.so).
+* In the context of drivers, shared libraries may contain enumerators,
+* drivers and devios. These register in step 2.
+*
+*  2)
+* @code
+*  odpdrv_enumr_class_register(int (probe*)()...)
+*  --->
+*  odpdrv_driver_register(int (probe*)()...)
+*  --->
+*  odpdrv_devio_register()
+*  --->
+* @endcode
+*  A number of device_enumerator_classes are registered at the ODP startup.
+*  Many classes are expected: static, ACPI, PCI, switchdev, virtual, DPAA2...
+*  A number of drivers also register to ODP (passing their own probe function).
+*  A number of device IO may also register to ODP (defining available devices
+*  interfaces).
+*
+*  3)  ODP calls the probe function of each enumerator class 
+* @code
+*  enumerator class probe()
+*  <---
+*  odpdrv_emum_register(int (probe*)()...)
+*  --->
+*  --->
+*  --->
+*  odpdrv_devio_register(...)
+*  --->
+*  --->
+*  --->
+* @endcode
+*  ODP calls the probe function of each registered enumerator_class.
+*  This result in the enumerator_class registering some
+*  enumerators (instances of the class) by calling
+*  odpdrv_emumerator_register() for each instance.
+*  A given enumerator_class may create many enumerators based on its platform:
+*  For instance Linux defines a number of PCI domains that can be viewed as
+*  multiple PCI enumerators. In addition, it could be considered that each PCI
+*  root of each processor socket in a NUMA environment has its own PCI
+*  enumerator.
+*  For enumerator class PCI, there could be one instance for each PCI
+*  domain.
+*  The devios delivered with their enumerator may also register at this stage.
+*
+* 4)
+* @code
+*  enumerator probe()
+*  <---
+*  odpdrv_device_create()
+*  --->
+*  odpdrv_device_create()
+*  --->
+*  odpdrv_device_create()
+*  --->
+* @endcode
+*  For each enumerator instance, odp calls the probe function.
+*  This will trigger devices creation (Enumerators calls odpdrv
+*  odpdrv_device_create() for each new device). Enumerators are allowed
+*  to call odpdrv_device_create() at any time once they have been probed
+*  (hotplug). They also may call odpdrv_device_destroy() if needed.
+*
+*  5) The driver framework calls the drivers probe(D,I) functions of the
+*  drivers, with device D and devio I as parameter, assuming that:
+*  -devio I was on the driver supported list of devio (and version matches)
+*  -the devio I is registered and found its enumerator interface(E) api
+*   (name and version)
+*  -device D was enumerated by an enumerator providing interface E.
+*  The return value of the driver probe function tells whether the driver
+*  can handle the device or not.
+*
+* @{
+*/
+
+/* Forward declarations for a top down description of structures */
+/** Parameters for enumerator class registration */
+typedef struct odpdrv_enumr_class_param_t odpdrv_enumr_class_param_t;
+/** Parameters for enumerator registration */
+typedef struct odpdrv_enumr_param_t odpdrv_enumr_pa

[lng-odp] [API-NEXT PATCHv11 0/5] driver initialisation framework

[Christophe Milard]

Since V10:
 -fixed distcheck (Maxim)

Since V9:
 -driver interface reworked to be more consistent with the mem allocator
   which is now merged. (more handle based)
 -rebased

Since V8:
 -fixed corrupt V8
 -copyright updates: 2017 (BTW: Happy new year)

Since V7:
 -changed include order in drv_driver.c (Anders)
 -correction of package name for Fedora install in DEPENDENCIES (Anders)
 -user config file changed from ./odp.conf to ~/.odp.conf: then
   the user config file and the usage of the environment variable are
   clearly distinct (Anders)
 -fixed typo (Christophe)
 -module loading separated from driver file as modules are more general
  (i.e. can be used for other plugins such as schedulers...) (Anders)
 -Fix variable name to remove the "drv" prefix when handling modules

Since V6:
 -more inforamtion added in the DEPENDENCIES file for libconf installation
  (Maxim)

Since V5:
 -name and comment changes as suggested by Maxim
  in https://lists.linaro.org/pipermail/lng-odp/2016-December/027400.html
 -update .travis.yml

Since V4:
 -typo fix (Thanks Yi!)
 -rebased.

Since V3:
 -minor interface simplification and name change (Christophe)
 -Fix for clang (Bill)
 -Google doc describing the driver and device frameworks structure:
https://docs.google.com/document/d/1eCKPJF6uSlOllXi_sKDvRwUD2BXm-ZzxZoKT0nVEsl4/edit#heading=h.osxoshqj1bj

Since V2:
 -function odp_load_driver removed. replaced by config file. (Petri, FF)
 -configuration file "odp.conf" added. Configuration file is:
1) as specified in env variable ODP_SYSCONFIG_FILE (which can be "none").
2) ./odp.conf
3) $(prefix)/etc/odp.conf
 -test removed: will be sent in a separate patch as many questions remains.
 -All libdl tests removed: libdl is assumed to always be on linux (Maxim)

Since V1:
 -enum names prefixed by ODPDRV (Yi)
 -better commit message for last patch (Christophe)
 -typo fix (Christophe)

This patch series puts the driver initialisation framework in place:
Loadable modules (*.so) are given in the odp.conf file added here.
Once loaded, the drivers module init function (declared as __constructor__)
calls the ODP odp_*_register() intialialisation function which,
at this stage does nothing (just print an error message).
odp_*_register() is of course part of the driver interface (south).

Christophe Milard (5):
  drv: adding driver registration interface (stub)
  linux-gen: adding enum, devio and driver registration interface (stub)
  linux-gen: init: adding configuration file parsing
  test: preventing odp.conf loading for tests
  linux-gen: modules: adding initial file to load modules

 .travis.yml|   2 +-
 DEPENDENCIES   |   8 +-
 configure.ac   |   4 +-
 include/odp/drv/spec/driver.h  | 389 +
 include/odp_drv.h  |   1 +
 platform/Makefile.inc  |   1 +
 platform/linux-generic/Makefile.am |   5 +
 platform/linux-generic/_modules.c  |  53 +++
 platform/linux-generic/drv_driver.c|  64 
 platform/linux-generic/include/odp/drv/driver.h|  37 ++
 .../include/odp/drv/plat/driver_types.h|  52 +++
 platform/linux-generic/include/odp_internal.h  |   5 +
 platform/linux-generic/m4/configure.m4 |  12 +
 platform/linux-generic/m4/odp_modules.m4   |  11 +
 platform/linux-generic/odp_init.c  |  87 +
 test/Makefile.inc  |   4 +-
 16 files changed, 727 insertions(+), 8 deletions(-)
 create mode 100644 include/odp/drv/spec/driver.h
 create mode 100644 platform/linux-generic/_modules.c
 create mode 100644 platform/linux-generic/drv_driver.c
 create mode 100644 platform/linux-generic/include/odp/drv/driver.h
 create mode 100644 platform/linux-generic/include/odp/drv/plat/driver_types.h
 create mode 100644 platform/linux-generic/m4/odp_modules.m4

-- 
2.7.4

Re: [lng-odp] [PATCHv2] linux-gen: _ishm: fix normal page fallback

[Christophe Milard]

ter:
pid=1070 key=12
_fdserver.c:515:handle_request():drop {ctx=1, key=12}->fd=17
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=11
_fdserver.c:515:handle_request():drop {ctx=1, key=11}->fd=16
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=8
_fdserver.c:515:handle_request():drop {ctx=1, key=8}->fd=13
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=9
_fdserver.c:515:handle_request():drop {ctx=1, key=9}->fd=14
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=10
_fdserver.c:515:handle_request():drop {ctx=1, key=10}->fd=15
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=6
_fdserver.c:515:handle_request():drop {ctx=1, key=6}->fd=11
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=7
_fdserver.c:515:handle_request():drop {ctx=1, key=7}->fd=12
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=5
_fdserver.c:515:handle_request():drop {ctx=1, key=5}->fd=10
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=4
_fdserver.c:515:handle_request():drop {ctx=1, key=4}->fd=9
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=3
_fdserver.c:515:handle_request():drop {ctx=1, key=3}->fd=8
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=2
_fdserver.c:515:handle_request():drop {ctx=1, key=2}->fd=7
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=1
_fdserver.c:515:handle_request():drop {ctx=1, key=1}->fd=6
_fdserver.c:323:_odp_fdserver_deregister_fd():FD client deregister:
pid=1070 key=0
_fdserver.c:515:handle_request():drop {ctx=1, key=0}->fd=5
_fdserver.c:409:stop_server():FD sending server stop request
_fdserver.c:531:handle_request():Stoping FD server
root@erachmi-ericsson:~/linaro/ODP/odp#

On 23 January 2017 at 17:28, Maxim Uvarov <maxim.uva...@linaro.org> wrote:
> On 01/23/17 11:47, Christophe Milard wrote:
>> Fixing failure due to lack of huge pages.
>> Fixes: https://bugs.linaro.org/show_bug.cgi?id=2842
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>> ---
>>  platform/linux-generic/_ishm.c | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c
>> index f889834..3797f20 100644
>> --- a/platform/linux-generic/_ishm.c
>> +++ b/platform/linux-generic/_ishm.c
>> @@ -547,7 +547,7 @@ static void *do_map(int block_index, uint64_t len, 
>> uint32_t align,
>>   addr = alloc_fragment(len, block_index, align, );
>>   if (!addr) {
>>   ODP_ERR("alloc_fragment failed.\n");
>> - if (new_block->filename[0]) {
>> + if (!new_block->external_fd) {
>>   close(*fd);
>>   *fd = -1;
>>   delete_file(new_block);
>> @@ -562,7 +562,7 @@ static void *do_map(int block_index, uint64_t len, 
>> uint32_t align,
>>   if (mapped_addr == NULL) {
>>   if (flags & _ODP_ISHM_SINGLE_VA)
>>   free_fragment(fragment);
>> - if (new_block->filename[0]) {
>> + if (!new_block->external_fd) {
>>   close(*fd);
>>   *fd = -1;
>>   delete_file(new_block);
>>
>
> default setting for:
> Ubuntu 16.04.1 LTS
>
> f4df4d24c6 linux-gen: _ishm: fix normal page fallback
> 03aafc3 helper: remove dependence on test dir
> 19a457a helper: move thread implementation under platform
> 7025b1b helper: cleanup Linux rename to thread
> 0f18e31 configure: use helper configure.m4
> 3875d6e update API version number from v1.12.0.0 to v1.13.0.024c6
> linux-gen: _ishm: fix normal page fallback
> 03aafc3 helper: remove dependence on test dir
> 19a457a helper: move thread implementation under platform
> 7025b1b helper: cleanup Linux rename to thread
> 0f18e31 configure: use helper configure.m4
> 3875d6e update API version number from v1.12.0.0 to v1.13.0.0
>
> root@ubuntu:/opt/odp.git# cat
> ./test/common_plat/performance/odp_pktio_perf.log
> _ishmphy.c:128:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> _ishm.c:859:_odp_ishm_reserve():No huge pages, fall back to normal
> pages. check: /proc/sys/vm/nr_hugepages.
> _ishmphy.c:128:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> _ishmphy.c:128:_odp_ishmphy_map():mmap failed:Cannot allocate memory
> _ishmphy.c:128:_odp_ishmphy_map():mmap failed:Cannot 

[lng-odp] [PATCH] linux-gen: _ishmphy: fix possible race with malloc

[Christophe Milard]

_ishm prereserves address space for _ODP_ISHM_SINGLE_VA allocated memory
by reserving unreachable "memory", using PROT_NONE, hence really allocating
virtual space (as the memory cannot be hit).
The problem came when trying to use some of this preallocated space:
strangely, if a new mapping on the preallocated virtual space failed (e.g.
due to lack of huge pages), linux would returned MAP_FAILED (OK) but
also cancel the previous PROT_NONE mapping, hence making the virtual
memory space available for further mmaps. (code Point A)
Before this patch, the code simply re-reserved (PROT_NONE) the area
(point B):
This was NOT OK: yes, _ishm_reserve calls are mutexed, so no other
odpthread 2 could do a reserve between point A and B of opdthread1, but if
thread2 did its own mmap(), malloc(),...,  there was a chance for thread 2
to get virtual space from the preserved area (which should be blocked).
This patch does not allow any A-B window by first doing an mmap (non fixed)
and then performing a second mapping at the correct address (in the
pre-reserved area), which is guaranteed to succeed, and finally removing
the non-fixed mapping.
The non-fix mapping just acts as a failure guard when the proper maping
is done.
Fixes https://bugs.linaro.org/show_bug.cgi?id=2834
(but very hard to trigger i.e. to prove)

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/_ishmphy.c | 42 +--
 1 file changed, 32 insertions(+), 10 deletions(-)

diff --git a/platform/linux-generic/_ishmphy.c 
b/platform/linux-generic/_ishmphy.c
index 2b2d100..d519af6 100644
--- a/platform/linux-generic/_ishmphy.c
+++ b/platform/linux-generic/_ishmphy.c
@@ -94,7 +94,7 @@ int _odp_ishmphy_unbook_va(void)
 void *_odp_ishmphy_map(int fd, void *start, uint64_t size,
   int flags)
 {
-   void *mapped_addr;
+   void *mapped_addr_tmp, *mapped_addr;
int mmap_flags = 0;
 
if (flags & _ODP_ISHM_SINGLE_VA) {
@@ -103,15 +103,37 @@ void *_odp_ishmphy_map(int fd, void *start, uint64_t size,
return NULL;
}
/* maps over fragment of reserved VA: */
-   mapped_addr = mmap(start, size, PROT_READ | PROT_WRITE,
-  MAP_SHARED | MAP_FIXED | mmap_flags, fd, 0);
-   /* if mapping fails, re-block the space we tried to take
-* as it seems a mapping failure still affect what was there??*/
-   if (mapped_addr == MAP_FAILED) {
-   mmap_flags = MAP_SHARED | MAP_FIXED |
-MAP_ANONYMOUS | MAP_NORESERVE;
-   mmap(start, size, PROT_NONE, mmap_flags, -1, 0);
-   mprotect(start, size, PROT_NONE);
+   /* first, try a normal map. If that works, remap it where it
+* should (on the prereverved space), and remove the initial
+* normal mapping:
+* This is because it turned out that if a mapping fails
+* on a the prereserved virtual address space, then
+* the prereserved address space which was tried to be mapped
+* on becomes available to the kernel again! This was not
+* according to expectations: the assumption was that if a
+* mapping fails, the system should remain unchanged, but this
+* is obvioulsy not true (at least for huge pages when
+* exhausted).
+* So the strategy is to first map at a non reserved place
+* (which can then be freed and returned to the kernel on
+* failure) and peform a new map to the prereserved space on
+* success (which is then guaranteed to work).
+* The initial free maping can then be removed.
+*/
+   mapped_addr = MAP_FAILED;
+   mapped_addr_tmp = mmap(NULL, size, PROT_READ | PROT_WRITE,
+  MAP_SHARED | mmap_flags, fd, 0);
+   if (mapped_addr_tmp != MAP_FAILED) {
+   /* If OK, do new map at right fixed location... */
+   mapped_addr = mmap(start,
+  size, PROT_READ | PROT_WRITE,
+  MAP_SHARED | MAP_FIXED | mmap_flags,
+  fd, 0);
+   if (mapped_addr != start)
+   ODP_ERR("new map failed:%s\n", strerror(errno));
+   /* ... and remove initial mapping: */
+   if (munmap(mapped_addr_tmp, size))
+   ODP_ERR("munmap failed:%s\n", strerror(errno));
}
} else {
/* just do a new mapping in the VA space: */
-- 
2.7.4

[lng-odp] [PATCHv2] linux-gen: _ishm: fix normal page fallback

[Christophe Milard]

Fixing failure due to lack of huge pages.
Fixes: https://bugs.linaro.org/show_bug.cgi?id=2842

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/_ishm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c
index f889834..3797f20 100644
--- a/platform/linux-generic/_ishm.c
+++ b/platform/linux-generic/_ishm.c
@@ -547,7 +547,7 @@ static void *do_map(int block_index, uint64_t len, uint32_t 
align,
addr = alloc_fragment(len, block_index, align, );
if (!addr) {
ODP_ERR("alloc_fragment failed.\n");
-   if (new_block->filename[0]) {
+   if (!new_block->external_fd) {
close(*fd);
*fd = -1;
delete_file(new_block);
@@ -562,7 +562,7 @@ static void *do_map(int block_index, uint64_t len, uint32_t 
align,
if (mapped_addr == NULL) {
if (flags & _ODP_ISHM_SINGLE_VA)
free_fragment(fragment);
-   if (new_block->filename[0]) {
+   if (!new_block->external_fd) {
close(*fd);
*fd = -1;
delete_file(new_block);
-- 
2.7.4

[lng-odp] [PATCHv2] linux-gen: _ishm: checking fstat return value.

[Christophe Milard]

Hence fixing CID 174663
(Fixes https://bugs.linaro.org/show_bug.cgi?id=2827)

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---

since V1:
  -print strerror(errno) and sets __odp_errno (Maxim)
  -ref to bugzilla bug ID (Mike, Bill)

 platform/linux-generic/_ishm.c | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c
index f889834..5ca1067 100644
--- a/platform/linux-generic/_ishm.c
+++ b/platform/linux-generic/_ishm.c
@@ -818,7 +818,14 @@ int _odp_ishm_reserve(const char *name, uint64_t size, int 
fd,
 
/* If a file descriptor is provided, get the real size and map: */
if (fd >= 0) {
-   fstat(fd, );
+   if (fstat(fd, ) < 0) {
+   close(fd);
+   odp_spinlock_unlock(_tbl->lock);
+   ODP_ERR("_ishm_reserve failed (fstat failed: %s).\n",
+   strerror(errno));
+   __odp_errno = errno;
+   return -1;
+   }
len = statbuf.st_size;
/* note that the huge page flag is meningless here as huge
 * page is determined by the provided file descriptor: */
-- 
2.7.4

Re: [lng-odp] [PATCH] linux-gen: _ishm: fix normal page fallback

[Christophe Milard]

Shall I open a bug and mark it resolved immediately?

On 21 January 2017 at 23:29, Bill Fischofer <bill.fischo...@linaro.org> wrote:
> This appears to be a bug fix. Bug fix patches should reference the bug
> being resolved in the commit log. I'm not sure I see this in bugzilla,
> so one should be opened for this.
>
> On Fri, Jan 20, 2017 at 3:19 AM, Christophe Milard
> <christophe.mil...@linaro.org> wrote:
>> Fixing failure due to lack of huge pages.
>>
>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>
> Reviewed-and-tested-by: Bill Fischofer <bill.fischo...@linaro.org>
>
>> ---
>>  platform/linux-generic/_ishm.c | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c
>> index f889834..3797f20 100644
>> --- a/platform/linux-generic/_ishm.c
>> +++ b/platform/linux-generic/_ishm.c
>> @@ -547,7 +547,7 @@ static void *do_map(int block_index, uint64_t len, 
>> uint32_t align,
>> addr = alloc_fragment(len, block_index, align, );
>> if (!addr) {
>> ODP_ERR("alloc_fragment failed.\n");
>> -   if (new_block->filename[0]) {
>> +   if (!new_block->external_fd) {
>> close(*fd);
>> *fd = -1;
>> delete_file(new_block);
>> @@ -562,7 +562,7 @@ static void *do_map(int block_index, uint64_t len, 
>> uint32_t align,
>> if (mapped_addr == NULL) {
>> if (flags & _ODP_ISHM_SINGLE_VA)
>> free_fragment(fragment);
>> -   if (new_block->filename[0]) {
>> +   if (!new_block->external_fd) {
>> close(*fd);
>> *fd = -1;
>> delete_file(new_block);
>> --
>> 2.7.4
>>

[lng-odp] [PATCH] linux-gen: _ishm: fix normal page fallback

[Christophe Milard]

Fixing failure due to lack of huge pages.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/_ishm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c
index f889834..3797f20 100644
--- a/platform/linux-generic/_ishm.c
+++ b/platform/linux-generic/_ishm.c
@@ -547,7 +547,7 @@ static void *do_map(int block_index, uint64_t len, uint32_t 
align,
addr = alloc_fragment(len, block_index, align, );
if (!addr) {
ODP_ERR("alloc_fragment failed.\n");
-   if (new_block->filename[0]) {
+   if (!new_block->external_fd) {
close(*fd);
*fd = -1;
delete_file(new_block);
@@ -562,7 +562,7 @@ static void *do_map(int block_index, uint64_t len, uint32_t 
align,
if (mapped_addr == NULL) {
if (flags & _ODP_ISHM_SINGLE_VA)
free_fragment(fragment);
-   if (new_block->filename[0]) {
+   if (!new_block->external_fd) {
close(*fd);
*fd = -1;
delete_file(new_block);
-- 
2.7.4

Re: [lng-odp] memory allocation issues

[Christophe Milard]

Thanks, Steve.

THP is not really what we want. It could create jitter among the ODP
threads which we don't really want.
Your answer is quite clear. I could definitively talk to you as to
grab some of your knowledge, but I actually think your answer both
shows that you understood my problem and could relate it to the kernel
code.

At this point I think our only hope would be to get mremap support for
HP in the kernel.

Christophe

On 19 January 2017 at 17:31, Mike Holmes <mike.hol...@linaro.org> wrote:
> Maybe the LNG Kernel team need to pick this up as a topic ?
>
> On 19 January 2017 at 11:18, Steve Capper <steve.cap...@linaro.org> wrote:
>> On 19 January 2017 at 13:04, Christophe Milard
>> <christophe.mil...@linaro.org> wrote:
>>> Hi Steve,
>>
>> Hey Christophe,
>>
>>>
>>> Maybe you remember me as we have had contact before. Christophe. from
>>> the LNG ODP team (mikes Holmes team).
>>>
>>> I have written the ODP memory allocator and I am having an issue with
>>> it: It has a requirement that linux processes (we call them ODP
>>> threads) have to be able to share memory between each other, as normal
>>> pthreads do. (an "ODP thread" can be either a linux process or a
>>> pthread)
>>> The memory should be shareable (at same virtual address) even if it is
>>> ODP allocated after processes have fork()'d.
>>>
>>> I did that the following way: as all our ODP processes are descendant
>>> of a single root process (we call it the ODP instantiation process), I
>>> actually pre-reserve a large virtual space area in this process). this
>>> is done as follows:
>>>
>>>  pre_reserved_zone = mmap(NULL, len, PROT_NONE,  MAP_SHARED |
>>> MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
>>>
>>> The PROT_NONE makes sure that the physical memory is unaccessible,
>>> hence not used.
>>>
>>> Later, when one of the linux processes does an odp_reserve(), in the
>>> related mmap(), I want to map the real memory on some part of that
>>> preallocated area, using MAP_FIXED:
>>>
>>> mapped_addr = mmap(start, size, PROT_READ | PROT_WRITE, MAP_SHARED |
>>> MAP_FIXED | mmap_flags, fd, 0);
>>>
>>> If "start" is in the pre_reserved_zone, we know it is available in all
>>> processes, as the prereserved zone is inheritaed by all (because they
>>> are all descendent of the instantiation process which did this
>>> pre-reservation)
>>>
>>> However, I noticed that, for huge pages at least, if this call fails
>>> due to a lack of huge pages, the virtual space (from start to
>>> start+size), seems to be returned as available to the kernel! I
>>> expected a failed call to leave the system unchanged, not to do half
>>> of the job...
>>
>> Unfortunately this appears to make sense due to the pluggable logic in
>> the kernel. If one mmaps a location, anything in the way is first
>> munmapp'ed. We need to call munmap as the previous mmap may have been
>> from special driver logic (remember one can supply an mmap handler for
>> a driver). Likewise due to the munmap also being potentially special,
>> we can't roll this back. The only safe thing we can do is leave the
>> space empty if the later mmap logic fails.
>> (Also it took me a while and a very strong coffee to understand this,
>> so it certainly isn't obvious :-)).
>>
>>
>>> This is of course a problem, since I want my pre-reserved area to
>>> remain pre-reserved on failure!
>>> What I did (until now), is that I simply remade the pre-reservation
>>> (with PROT_NONE) on the specific area behind the failed call.
>>> This was OK, I though, as concurrent access (from different thread) to
>>> my odp_reserve() function are mutexed.
>>> What I forgot is that the differrent threads can actually use malloc()
>>> or mmap() directely:
>>> If a thread 1 does a odp_reserve, fails on lack of huge page (point A
>>> in the code) and re-pre-reserve the area (point B), another thread 2
>>> could be unlucky enough to do a mmap(NULL,...) between thread 1's A
>>> and B, and be returned a part of my so-called preallocated address
>>> space :-(.
>>>
>>> So I am working on another strategy: doing a first mapping outside the
>>> preallocated space, and, on success only, move the resulting area
>>> (using mremap) into the proeallocated space.
>>>
>>> The patch (from the old strategy to the new one) looks as flllows:
>>> -   mapped_ad

Re: [lng-odp] [PATCH v3 0/4] remove Linux specifics and dependence on test dir

[Christophe Milard]

For the series:

Reviewed-by: Christophe Milard <christophe.mil...@linaro.org>


On 18 January 2017 at 23:01, Mike Holmes <mike.hol...@linaro.org> wrote:
> Starting with some clean up, rename the Linux specific files and then move 
> them
> to a platform specific directory. A new configure option is introduced
> "with-helper-platform", this defaults to match the existing odp selector
> "with-platform" so that there is no operational change.
>
> With the move made the default is to build only the portable helper API, if 
> the
> legacy and previously unused Linux thread APIs are needed they can be built in
> with --enable-helper-extn
>
> Then remove the dependence on the test directory.
>
> v2:
> sort some lists into alphabetical order (Maxim)
> addtional code simplification removing a .h helper(Christophe)
>
> v3:
> rebase
> whitespace error is there due to matching existing formatting in file
>
> Mike Holmes (4):
>   configure: use helper configure.m4
>   helper: cleanup Linux rename to thread
>   helper: move thread implementation under platform
>   helper: remove dependence on test dir
>
>  configure.ac   |  19 +-
>  example/Makefile.inc   |   2 +-
>  example/classifier/odp_classifier.c|   2 +-
>  example/generator/odp_generator.c  |   2 +-
>  example/ipsec/odp_ipsec.c  |   2 +-
>  example/l2fwd_simple/odp_l2fwd_simple.c|   2 +-
>  example/l3fwd/odp_l3fwd.c  |   2 +-
>  example/packet/odp_pktio.c |   2 +-
>  example/switch/odp_switch.c|   2 +-
>  example/time/time_global_test.c|   5 +-
>  example/timer/odp_timer_test.c |   2 +-
>  helper/Makefile.am |  22 +-
>  .../helper/platform/linux-generic/threads_extn.h   | 112 
>  helper/include/odp/helper/{linux.h => threads.h}   |  78 -
>  helper/m4/configure.m4 |  14 +
>  helper/platform/linux-generic/thread.c | 313 
> +
>  helper/test/Makefile.am|  43 ++-
>  helper/test/chksum.c   |  18 +-
>  helper/test/cuckootable.c  |   4 +-
>  helper/test/iplookuptable.c|   4 +-
>  helper/test/linux-generic/Makefile.am  |   5 +
>  helper/test/{ => linux-generic}/process.c  |  16 +-
>  helper/test/{ => linux-generic}/thread.c   |  16 +-
>  helper/test/odpthreads.c   |  20 +-
>  helper/test/parse.c|  86 +++---
>  helper/test/table.c|  12 +-
>  helper/{linux.c => threads.c}  | 240 +---
>  ...inux.pc.in => libodphelper-linux-generic.pc.in} |   4 +-
>  test/Makefile.inc  |   2 +-
>  test/common_plat/common/odp_cunit_common.c |   2 +-
>  .../common_plat/miscellaneous/odp_api_from_cpp.cpp |   2 +-
>  test/common_plat/performance/odp_crypto.c  |   2 +-
>  test/common_plat/performance/odp_l2fwd.c   |   2 +-
>  test/common_plat/performance/odp_pktio_perf.c  |   2 +-
>  test/common_plat/performance/odp_sched_latency.c   |   2 +-
>  test/common_plat/performance/odp_scheduling.c  |   2 +-
>  test/common_plat/validation/api/Makefile.inc   |   2 +-
>  test/common_plat/validation/api/timer/timer.c  |   2 +-
>  test/linux-generic/Makefile.inc|   2 +-
>  test/linux-generic/mmap_vlan_ins/mmap_vlan_ins.c   |   2 +-
>  test/linux-generic/pktio_ipc/ipc_common.h  |   2 +-
>  test/linux-generic/ring/ring_stress.c  |   2 +-
>  42 files changed, 633 insertions(+), 444 deletions(-)
>  create mode 100644 
> helper/include/odp/helper/platform/linux-generic/threads_extn.h
>  rename helper/include/odp/helper/{linux.h => threads.h} (74%)
>  create mode 100644 helper/platform/linux-generic/thread.c
>  create mode 100644 helper/test/linux-generic/Makefile.am
>  rename helper/test/{ => linux-generic}/process.c (84%)
>  rename helper/test/{ => linux-generic}/thread.c (84%)
>  rename helper/{linux.c => threads.c} (67%)
>  rename pkgconfig/{libodphelper-linux.pc.in => 
> libodphelper-linux-generic.pc.in} (72%)
>
> --
> 2.9.3

[lng-odp] [PATCHv2] travis: better comments to generate token in the travis file

[Christophe Milard]

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 .travis.yml | 12 
 1 file changed, 12 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index 03e61b1..cfa6c3d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -11,6 +11,18 @@ env:
   global:
 # COVERITY_SCAN_TOKEN
 # ** specific to your project **
+# Note:
+# You should have a github account and travis linked travis account.
+# The secure key to be filled below is the 685 character long encrypted
+# token you can find as follow from your coverity dashboard
+# (at https://scan.coverity.com/dashboard):
+# Click on the github project (/odp)
+# Click on "submit build"
+# Click on "Configure Travis CI"
+# Look at the COVERITY_SCAN_TOKEN in the env: global: section
+# of the configuration example.
+# copy the secure: below
+#
 - secure: ""
 
 language: c
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv10 5/5] linux-gen: modules: adding initial file to load modules

[Christophe Milard]

The shared objects listed in the ODP configuration files are
loaded at init time. The odp configuration file lists the
shared objects to be loaded as shown in the following example:
module = {
modules = ["enumerator1.so", "driver1.so"];
};

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 configure.ac  |  4 +-
 platform/linux-generic/Makefile.am|  1 +
 platform/linux-generic/_modules.c | 53 +++
 platform/linux-generic/include/odp_internal.h |  3 ++
 platform/linux-generic/m4/configure.m4|  1 +
 platform/linux-generic/m4/odp_modules.m4  | 11 ++
 platform/linux-generic/odp_init.c |  7 
 7 files changed, 78 insertions(+), 2 deletions(-)
 create mode 100644 platform/linux-generic/_modules.c
 create mode 100644 platform/linux-generic/m4/odp_modules.m4

diff --git a/configure.ac b/configure.ac
index 3a20959..e2b4f9d 100644
--- a/configure.ac
+++ b/configure.ac
@@ -55,7 +55,7 @@ AC_PROG_MAKE_SET
 
 AM_PROG_AR
 #Use libtool
-LT_INIT([])
+LT_INIT([dlopen])
 AC_SUBST([LIBTOOL_DEPS])
 AM_PROG_LIBTOOL
 
@@ -66,7 +66,7 @@ AC_CHECK_FUNCS([bzero clock_gettime gethostbyname getpagesize 
gettimeofday memse
 
 # Checks for header files.
 AC_HEADER_RESOLV
-AC_CHECK_HEADERS([arpa/inet.h fcntl.h inttypes.h limits.h netdb.h netinet/in.h 
stddef.h stdint.h stdlib.h string.h sys/ioctl.h sys/socket.h sys/time.h 
unistd.h])
+AC_CHECK_HEADERS([arpa/inet.h fcntl.h inttypes.h limits.h netdb.h netinet/in.h 
stddef.h stdint.h stdlib.h string.h sys/ioctl.h sys/socket.h sys/time.h 
unistd.h dlfcn.h])
 
 # Checks for typedefs, structures, and compiler characteristics.
 AC_HEADER_STDBOOL
diff --git a/platform/linux-generic/Makefile.am 
b/platform/linux-generic/Makefile.am
index 2d92c97..855f04e 100644
--- a/platform/linux-generic/Makefile.am
+++ b/platform/linux-generic/Makefile.am
@@ -176,6 +176,7 @@ __LIB__libodp_linux_la_SOURCES = \
   _ishm.c \
   _ishmphy.c \
   _ishmpool.c \
+  _modules.c \
   odp_atomic.c \
   odp_barrier.c \
   odp_bitmap.c \
diff --git a/platform/linux-generic/_modules.c 
b/platform/linux-generic/_modules.c
new file mode 100644
index 000..6bb854e
--- /dev/null
+++ b/platform/linux-generic/_modules.c
@@ -0,0 +1,53 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+static int load_modules(void)
+{
+   config_t *cf;
+   const config_setting_t *modules_section;
+   int module_count;
+   int i;
+   const char *module_name;
+
+   cf = _global_data.configuration;
+   modules_section = config_lookup(cf, "module.modules");
+   if (!modules_section)
+   return 0;
+
+   module_count = config_setting_length(modules_section);
+   if (!module_count)
+   return 0;
+
+   for (i = 0; i < module_count; i++) {
+   module_name = config_setting_get_string_elem(modules_section,
+i);
+   if (dlopen(module_name, RTLD_NOW) == NULL) {
+   ODP_ERR("dlopen failed for %s: %s\n",
+   module_name, dlerror());
+   return -1;
+   }
+   ODP_DBG("module %s loaded.\n", module_name);
+   }
+
+   return 0;
+}
+
+int _odp_modules_init_global(void)
+{
+   /* load modules (enumerator and drivers...) */
+   if (load_modules())
+   return -1;
+
+   return 0;
+}
diff --git a/platform/linux-generic/include/odp_internal.h 
b/platform/linux-generic/include/odp_internal.h
index 9d1fc58..05c8a42 100644
--- a/platform/linux-generic/include/odp_internal.h
+++ b/platform/linux-generic/include/odp_internal.h
@@ -71,6 +71,7 @@ enum init_stage {
CLASSIFICATION_INIT,
TRAFFIC_MNGR_INIT,
NAME_TABLE_INIT,
+   MODULES_INIT,
ALL_INIT  /* All init stages completed */
 };
 
@@ -129,6 +130,8 @@ int _odp_ishm_init_local(void);
 int _odp_ishm_term_global(void);
 int _odp_ishm_term_local(void);
 
+int _odp_modules_init_global(void);
+
 int cpuinfo_parser(FILE *file, system_info_t *sysinfo);
 uint64_t odp_cpu_hz_current(int id);
 
diff --git a/platform/linux-generic/m4/configure.m4 
b/platform/linux-generic/m4/configure.m4
index 5fab0cc..f5f076a 100644
--- a/platform/linux-generic/m4/configure.m4
+++ b/platform/linux-generic/m4/configure.m4
@@ -42,6 +42,7 @@ fi
 m4_include([platform/linux-generic/m4/odp_pthread.m4])
 m4_include([platform/linux-generic/m4/odp_openssl.m4])
 m4_include([platform/linux-generic/m4/odp_pcap.m4])
+m4_include([platform/linux-generic

[lng-odp] [API-NEXT PATCHv10 4/5] test: preventing odp.conf loading for tests

[Christophe Milard]

The tests should not be affected by any system or user ODP configuration
file. The ODP_SYSCONFIG_FILE environment variables is therefore set
to "none" in TESTS_ENVIRONMENT.
Tests which need specific a configuration file will have to overwrite
this setting.
Note that tests ran manually (not using make check) may be affected
by configuration files. Setting ODP_SYSCONFIG_FILE to an appropriate value
(possibly "none") may be required.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 test/Makefile.inc | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/test/Makefile.inc b/test/Makefile.inc
index 1ebc047..896f863 100644
--- a/test/Makefile.inc
+++ b/test/Makefile.inc
@@ -22,4 +22,6 @@ AM_LDFLAGS += -L$(LIB)
 @VALGRIND_CHECK_RULES@
 valgrind_tools = memcheck
 
-TESTS_ENVIRONMENT= ODP_PLATFORM=${with_platform} EXEEXT=${EXEEXT}
+TESTS_ENVIRONMENT = ODP_PLATFORM=${with_platform} \
+   EXEEXT=${EXEEXT} \
+   ODP_SYSCONFIG_FILE=none
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv10 3/5] linux-gen: init: adding configuration file parsing

[Christophe Milard]

The parsing of the odp.conf configuration file is added.
The file is searched first in filename specified in the environment
variable $ODP_SYSCONFIG_FILE (where the special string "none" prevent
any file loading)
The file is then searched in the user home directory (~) and
then is the $prefix/etc directory.
This requires libconfig (sudo apt-get install libconfig-dev)

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 .travis.yml   |  2 +-
 DEPENDENCIES  |  8 +--
 platform/linux-generic/Makefile.am|  1 +
 platform/linux-generic/include/odp_internal.h |  2 +
 platform/linux-generic/m4/configure.m4| 11 
 platform/linux-generic/odp_init.c | 80 +++
 6 files changed, 99 insertions(+), 5 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index 03e61b1..a6f770c 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -19,7 +19,7 @@ sudo: required
 
 before_install:
 - sudo apt-get -qq update
-- sudo apt-get install automake autoconf libtool libssl-dev graphviz 
mscgen doxygen
+- sudo apt-get install automake autoconf libtool libssl-dev graphviz 
mscgen doxygen libconfig-dev
 - sudo apt-get install libpcap-dev linux-headers-`uname -r`
 - gem install asciidoctor
 
diff --git a/DEPENDENCIES b/DEPENDENCIES
index f1f0edd..f285783 100644
--- a/DEPENDENCIES
+++ b/DEPENDENCIES
@@ -18,18 +18,18 @@ Prerequisites for building the OpenDataPlane (ODP) API
 
 3. Required libraries
 
-   Libraries currently required to link: openssl
+   Libraries currently required to link: openssl, libconfig
 
-3.1 OpenSSL native compile
+3.1 OpenSSL and libconf native compile
 
For native compilation, simply load the necessary libraries using the 
appropriate
tool set.
 
On Debian/Ubuntu systems:
-   $ sudo apt-get install libssl-dev
+   $ sudo apt-get install libssl-dev libconfig-dev
 
On CentOS/RedHat/Fedora systems:
-   $ sudo yum install openssl-devel
+   $ sudo yum install openssl-devel libconfig-devel
 
 3.2 OpenSSL cross compilation
 
diff --git a/platform/linux-generic/Makefile.am 
b/platform/linux-generic/Makefile.am
index c752645..2d92c97 100644
--- a/platform/linux-generic/Makefile.am
+++ b/platform/linux-generic/Makefile.am
@@ -8,6 +8,7 @@ AM_CFLAGS +=  -I$(srcdir)/include
 AM_CFLAGS +=  -I$(top_srcdir)/include
 AM_CFLAGS +=  -I$(top_builddir)/include
 AM_CFLAGS +=  -Iinclude
+AM_CFLAGS +=  -DSYSCONFDIR=\"@sysconfdir@\"
 
 include_HEADERS = \
  $(top_srcdir)/include/odp.h \
diff --git a/platform/linux-generic/include/odp_internal.h 
b/platform/linux-generic/include/odp_internal.h
index b313b1f..9d1fc58 100644
--- a/platform/linux-generic/include/odp_internal.h
+++ b/platform/linux-generic/include/odp_internal.h
@@ -22,6 +22,7 @@ extern "C" {
 #include 
 #include 
 #include 
+#include 
 
 extern __thread int __odp_errno;
 
@@ -50,6 +51,7 @@ struct odp_global_data_s {
odp_cpumask_t control_cpus;
odp_cpumask_t worker_cpus;
int num_cpus_installed;
+   config_t configuration;
 };
 
 enum init_stage {
diff --git a/platform/linux-generic/m4/configure.m4 
b/platform/linux-generic/m4/configure.m4
index d3e5528..5fab0cc 100644
--- a/platform/linux-generic/m4/configure.m4
+++ b/platform/linux-generic/m4/configure.m4
@@ -28,6 +28,17 @@ AC_LINK_IFELSE(
 echo "Use newer version. For gcc > 4.7.0"
 exit -1)
 
+# Check for libconfig (required)
+AC_CHECK_HEADERS([libconfig.h], HEADER_LIBCONFIG="yes")
+PKG_CHECK_MODULES([PKGCONFIG], [libconfig >= 1.3.2], LIBRARY_LIBCONFIG="yes")
+if test "x$LIBRARY_LIBCONFIG" != "x" && test "x$HEADER_LIBCONFIG" != "x" ; then
+CFLAGS="$CFLAGS $PKGCONFIG_CFLAGS"
+LIBS="$LIBS $PKGCONFIG_LIBS"
+AM_CPPFLAGS="$AM_CPPFLAGS `pkg-config --cflags-only-I libconfig`"
+else
+AC_MSG_FAILURE([libconfig not found (required)])
+fi
+
 m4_include([platform/linux-generic/m4/odp_pthread.m4])
 m4_include([platform/linux-generic/m4/odp_openssl.m4])
 m4_include([platform/linux-generic/m4/odp_pcap.m4])
diff --git a/platform/linux-generic/odp_init.c 
b/platform/linux-generic/odp_init.c
index 06c6143..0e9167c 100644
--- a/platform/linux-generic/odp_init.c
+++ b/platform/linux-generic/odp_init.c
@@ -10,6 +10,8 @@
 #include 
 #include 
 #include 
+#include 
+#include 
 #include 
 #include 
 #include 
@@ -21,6 +23,15 @@
 #define _ODP_FILES_FMT "odp-%d-"
 #define _ODP_TMPDIR"/tmp"
 
+/* the name of the ODP configuration file: */
+#define CONFIGURATION_FILE_ENV_NONE "none"
+#define CONFIGURATION_FILE "odp.conf"
+#define CONFIGURATION_FILE_USR ("." CONFIGURATION_FILE)
+#define CONFIGURATION_FILE_SYS (SYSCONFDIR "/" CONFIGURATION_FILE)
+
+/* the ODP configuration file name can also be 

[lng-odp] [API-NEXT PATCHv10 2/5] linux-gen: adding enum, devio and driver registration interface (stub)

[Christophe Milard]

The linux implementation for the enumerator class registration function,
enumerator instance registration function,
devio and driver registration functions (stub)

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp_drv.h  |  1 +
 platform/linux-generic/Makefile.am |  2 +
 platform/linux-generic/drv_driver.c| 64 ++
 platform/linux-generic/include/odp/drv/driver.h| 37 +
 .../include/odp/drv/plat/driver_types.h| 52 ++
 5 files changed, 156 insertions(+)
 create mode 100644 platform/linux-generic/drv_driver.c
 create mode 100644 platform/linux-generic/include/odp/drv/driver.h
 create mode 100644 platform/linux-generic/include/odp/drv/plat/driver_types.h

diff --git a/include/odp_drv.h b/include/odp_drv.h
index 0959879..96d81ba 100644
--- a/include/odp_drv.h
+++ b/include/odp_drv.h
@@ -23,6 +23,7 @@ extern C {
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
diff --git a/platform/linux-generic/Makefile.am 
b/platform/linux-generic/Makefile.am
index 74e606d..c752645 100644
--- a/platform/linux-generic/Makefile.am
+++ b/platform/linux-generic/Makefile.am
@@ -109,6 +109,7 @@ odpdrvinclude_HEADERS = \
  $(srcdir)/include/odp/drv/barrier.h \
  $(srcdir)/include/odp/drv/byteorder.h \
  $(srcdir)/include/odp/drv/compiler.h \
+ $(srcdir)/include/odp/drv/driver.h \
  $(srcdir)/include/odp/drv/shm.h \
  $(srcdir)/include/odp/drv/spinlock.h \
  $(srcdir)/include/odp/drv/std_types.h \
@@ -233,6 +234,7 @@ __LIB__libodp_linux_la_SOURCES = \
   odp_weak.c \
   drv_atomic.c \
   drv_barrier.c \
+  drv_driver.c \
   drv_shm.c \
   drv_spinlock.c \
   arch/@ARCH_DIR@/odp_cpu_arch.c \
diff --git a/platform/linux-generic/drv_driver.c 
b/platform/linux-generic/drv_driver.c
new file mode 100644
index 000..529da48
--- /dev/null
+++ b/platform/linux-generic/drv_driver.c
@@ -0,0 +1,64 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+#include 
+
+#include 
+#include 
+#include 
+#include 
+
+odpdrv_enumr_class_t odpdrv_enumr_class_register(odpdrv_enumr_class_param_t
+*param)
+{
+   ODP_ERR("NOT Supported yet! Enumerator Class %s Registration!\n.",
+   param->name);
+
+   return ODPDRV_ENUMR_CLASS_INVALID;
+}
+
+odpdrv_enumr_t odpdrv_enumr_register(odpdrv_enumr_param_t *param)
+{
+   ODP_ERR("NOT Supported yet! Enumerator API %s Registration!\n.",
+   param->api_name);
+
+   return ODPDRV_ENUMR_INVALID;
+}
+
+odpdrv_device_t odpdrv_device_create(odpdrv_device_param_t *param)
+{
+   ODP_ERR("odpdrv_device_create not Supported yet! devaddress: %s\n.",
+   param->address);
+   return ODPDRV_DEVICE_INVALID;
+}
+
+void odpdrv_device_destroy(odpdrv_device_t dev)
+{
+   if (dev == ODPDRV_DEVICE_INVALID)
+   ODP_ERR("Invalid device\n");
+}
+
+odpdrv_devio_t odpdrv_devio_register(odpdrv_devio_param_t *param)
+{
+   ODP_ERR("NOT Supported yet! Driver %s Registration!\n.",
+   param->api_name);
+
+   return ODPDRV_DEVIO_INVALID;
+}
+
+odpdrv_driver_t odpdrv_driver_register(odpdrv_driver_param_t *param)
+{
+   ODP_ERR("NOT Supported yet! Driver %s Registration!\n.",
+   param->name);
+
+   return ODPDRV_DRIVER_INVALID;
+}
+
+int odpdrv_print_all(void)
+{
+   ODP_ERR("odpdrv_print_all not Supported yet!\n.");
+   return 0;
+}
diff --git a/platform/linux-generic/include/odp/drv/driver.h 
b/platform/linux-generic/include/odp/drv/driver.h
new file mode 100644
index 000..b12c83d
--- /dev/null
+++ b/platform/linux-generic/include/odp/drv/driver.h
@@ -0,0 +1,37 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+/**
+ * @file
+ *
+ * ODPDRV drivers
+ */
+
+#ifndef ODPDRV_PLAT_DRIVER_H_
+#define ODPDRV_PLAT_DRIVER_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#include 
+
+/** @ingroup odpdrv_driver
+ *  @{
+ */
+
+/**
+ * @}
+ */
+
+#include 
+#include 
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif
diff --git a/platform/linux-generic/include/odp/drv/plat/driver_types.h 
b/platform/linux-generic/include/odp/drv/plat/driver_types.h
new file mode 100644
index 000..6c611bd
--- /dev/null
+++ b/platform/linux-generic/include/odp/drv/plat/driver_types.h
@@ -0,0 +1,52 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-C

[lng-odp] [API-NEXT PATCHv10 1/5] drv: adding driver registration interface (stub)

[Christophe Milard]

The enumerator class, enumerator instance, devio and driver registration
functions prototypes (and a draft of their parameters) are
defined, the goal being to define the registration framework only.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 include/odp/drv/spec/driver.h | 389 ++
 platform/Makefile.inc |   1 +
 2 files changed, 390 insertions(+)
 create mode 100644 include/odp/drv/spec/driver.h

diff --git a/include/odp/drv/spec/driver.h b/include/odp/drv/spec/driver.h
new file mode 100644
index 000..d83e907
--- /dev/null
+++ b/include/odp/drv/spec/driver.h
@@ -0,0 +1,389 @@
+/* Copyright (c) 2017, Linaro Limited
+ * All rights reserved.
+ *
+ * SPDX-License-Identifier: BSD-3-Clause
+ */
+
+/**
+ * @file
+ *
+ * ODPDRV driver
+ */
+
+#ifndef ODPDRV_DRIVER_H_
+#define ODPDRV_DRIVER_H_
+#include 
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/**
+* @addtogroup odpdrv_driver
+* @details
+* enumerator and driver interface to ODP
+*
+*  1) ODP loads the different modules (i.e. it loads shared libraries, *.so).
+* In the context of drivers, shared libraries may contain enumerators,
+* drivers and devios. These register in step 2.
+*
+*  2)
+* @code
+*  odpdrv_enumr_class_register(int (probe*)()...)
+*  --->
+*  odpdrv_driver_register(int (probe*)()...)
+*  --->
+*  odpdrv_devio_register()
+*  --->
+* @endcode
+*  A number of device_enumerator_classes are registered at the ODP startup.
+*  Many classes are expected: static, ACPI, PCI, switchdev, virtual, DPAA2...
+*  A number of drivers also register to ODP (passing their own probe function).
+*  A number of device IO may also register to ODP (defining available devices
+*  interfaces).
+*
+*  3)  ODP calls the probe function of each enumerator class 
+* @code
+*  enumerator class probe()
+*  <---
+*  odpdrv_emum_register(int (probe*)()...)
+*  --->
+*  --->
+*  --->
+*  odpdrv_devio_register(...)
+*  --->
+*  --->
+*  --->
+* @endcode
+*  ODP calls the probe function of each registered enumerator_class.
+*  This result in the enumerator_class registering some
+*  enumerators (instances of the class) by calling
+*  odpdrv_emumerator_register() for each instance.
+*  A given enumerator_class may create many enumerators based on its platform:
+*  For instance Linux defines a number of PCI domains that can be viewed as
+*  multiple PCI enumerators. In addition, it could be considered that each PCI
+*  root of each processor socket in a NUMA environment has its own PCI
+*  enumerator.
+*  For enumerator class PCI, there could be one instance for each PCI
+*  domain.
+*  The devios delivered with their enumerator may also register at this stage.
+*
+* 4)
+* @code
+*  enumerator probe()
+*  <---
+*  odpdrv_device_create()
+*  --->
+*  odpdrv_device_create()
+*  --->
+*  odpdrv_device_create()
+*  --->
+* @endcode
+*  For each enumerator instance, odp calls the probe function.
+*  This will trigger devices creation (Enumerators calls odpdrv
+*  odpdrv_device_create() for each new device). Enumerators are allowed
+*  to call odpdrv_device_create() at any time once they have been probed
+*  (hotplug). They also may call odpdrv_device_destroy() if needed.
+*
+*  5) The driver framework calls the drivers probe(D,I) functions of the
+*  drivers, with device D and devio I as parameter, assuming that:
+*  -devio I was on the driver supported list of devio (and version matches)
+*  -the devio I is registered and found its enumerator interface(E) api
+*   (name and version)
+*  -device D was enumerated by an enumerator providing interface E.
+*  The return value of the driver probe function tells whether the driver
+*  can handle the device or not.
+*
+* @{
+*/
+
+/* Forward declarations for a top down description of structures */
+/** Parameters for enumerator class registration */
+typedef struct odpdrv_enumr_class_param_t odpdrv_enumr_class_param_t;
+/** Parameters for enumerator registration */
+typedef struct odpdrv_enumr_param_t odpdrv_enumr_pa

[lng-odp] [API-NEXT PATCHv10 0/5] driver initialisation framework

[Christophe Milard]

Since V9:
 -driver interface reworked to be more consistent with the mem allocator
   which is now merged. (more handle based)
 -rebased

Since V8:
 -fixed corrupt V8
 -copyright updates: 2017 (BTW: Happy new year)

Since V7:
 -changed include order in drv_driver.c (Anders)
 -correction of package name for Fedora install in DEPENDENCIES (Anders)
 -user config file changed from ./odp.conf to ~/.odp.conf: then
   the user config file and the usage of the environment variable are
   clearly distinct (Anders)
 -fixed typo (Christophe)
 -module loading separated from driver file as modules are more general
  (i.e. can be used for other plugins such as schedulers...) (Anders)
 -Fix variable name to remove the "drv" prefix when handling modules

Since V6:
 -more inforamtion added in the DEPENDENCIES file for libconf installation
  (Maxim)

Since V5:
 -name and comment changes as suggested by Maxim
  in https://lists.linaro.org/pipermail/lng-odp/2016-December/027400.html
 -update .travis.yml

Since V4:
 -typo fix (Thanks Yi!)
 -rebased.

Since V3:
 -minor interface simplification and name change (Christophe)
 -Fix for clang (Bill)
 -Google doc describing the driver and device frameworks structure:
https://docs.google.com/document/d/1eCKPJF6uSlOllXi_sKDvRwUD2BXm-ZzxZoKT0nVEsl4/edit#heading=h.osxoshqj1bj

Since V2:
 -function odp_load_driver removed. replaced by config file. (Petri, FF)
 -configuration file "odp.conf" added. Configuration file is:
1) as specified in env variable ODP_SYSCONFIG_FILE (which can be "none").
2) ./odp.conf
3) $(prefix)/etc/odp.conf
 -test removed: will be sent in a separate patch as many questions remains.
 -All libdl tests removed: libdl is assumed to always be on linux (Maxim)

Since V1:
 -enum names prefixed by ODPDRV (Yi)
 -better commit message for last patch (Christophe)
 -typo fix (Christophe)

This patch series puts the driver initialisation framework in place:
Loadable modules (*.so) are given in the odp.conf file added here.
Once loaded, the drivers module init function (declared as __constructor__)
calls the ODP odp_*_register() intialialisation function which,
at this stage does nothing (just print an error message).
odp_*_register() is of course part of the driver interface (south).

Christophe Milard (5):
  drv: adding driver registration interface (stub)
  linux-gen: adding enum, devio and driver registration interface (stub)
  linux-gen: init: adding configuration file parsing
  test: preventing odp.conf loading for tests
  linux-gen: modules: adding initial file to load modules

 .travis.yml|   2 +-
 DEPENDENCIES   |   8 +-
 configure.ac   |   4 +-
 include/odp/drv/spec/driver.h  | 389 +
 include/odp_drv.h  |   1 +
 platform/Makefile.inc  |   1 +
 platform/linux-generic/Makefile.am |   4 +
 platform/linux-generic/_modules.c  |  53 +++
 platform/linux-generic/drv_driver.c|  64 
 platform/linux-generic/include/odp/drv/driver.h|  37 ++
 .../include/odp/drv/plat/driver_types.h|  52 +++
 platform/linux-generic/include/odp_internal.h  |   5 +
 platform/linux-generic/m4/configure.m4 |  12 +
 platform/linux-generic/m4/odp_modules.m4   |  11 +
 platform/linux-generic/odp_init.c  |  87 +
 test/Makefile.inc  |   4 +-
 16 files changed, 726 insertions(+), 8 deletions(-)
 create mode 100644 include/odp/drv/spec/driver.h
 create mode 100644 platform/linux-generic/_modules.c
 create mode 100644 platform/linux-generic/drv_driver.c
 create mode 100644 platform/linux-generic/include/odp/drv/driver.h
 create mode 100644 platform/linux-generic/include/odp/drv/plat/driver_types.h
 create mode 100644 platform/linux-generic/m4/odp_modules.m4

-- 
2.7.4

Re: [lng-odp] [PATCH] travis: better comments to generate token in the travis file

[Christophe Milard]

Easier way exists. This patch should be dropped.

On 18 January 2017 at 13:34, Christophe Milard
<christophe.mil...@linaro.org> wrote:
> On 18 January 2017 at 13:26, Maxim Uvarov <maxim.uva...@linaro.org> wrote:
>> On 01/18/17 16:14, Christophe Milard wrote:
>>> Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
>>> ---
>>>  .travis.yml | 18 ++
>>>  1 file changed, 18 insertions(+)
>>>
>>> diff --git a/.travis.yml b/.travis.yml
>>> index 03e61b1..909f0a3 100644
>>> --- a/.travis.yml
>>> +++ b/.travis.yml
>>> @@ -6,11 +6,29 @@
>>>  # pushing to github/master will run make check
>>>  # pushing to github/coverity_scan will also launch a static analysis
>>>  # See https://scan.coverity.com/travis_ci
>>> +# Note: "push -f" seems to have problems, do not rely on it to trigger CI 
>>> from
>>> +# github (push new commits, without '-f').
>>>
>>>  env:
>>>global:
>>>  # COVERITY_SCAN_TOKEN
>>>  # ** specific to your project **
>>> +# Note:
>>> +# You should have a github account and travis linked travis account.
>>> +# To generate the proper encrypted token you need to install ruby-dev
>>> +# and gem_install travis as follows:
>>> +# % sudo apt-get install ruby-dev
>>> +# % sudo gem install travis
>>> +# then run:
>>> +# % travis encrypt  -r / encrypt COVERITY_SCAN_TOKEN=
>>> +# where  is your github login
>>> +# where  is your github repo, e.g. "odp"
>>> +# where  is the "Project Token" found on the coverity web page:
>>> +# https://scan.coverity.com/dashboard -> / -> 
>>> Project_settings
>>> +# e.g.:
>>> +# % travis encrypt  -r christophe/odp encrypt  
>>> COVERITY_SCAN_TOKEN="v-HOHTCHPkya4cx2cscrYg"
>>> +# The output string, about 700 characters long, should replace  
>>> below.
>>> +#
>>>  - secure: ""
>>>
>>>  language: c
>>>
>>
>>
>> I did not do all that things. I just copy-pasted token from web page.
>> Also there is button to generate new if needed.
>
> ??? Did you find the 687 char encryptred token anywhere???
> Or did you just paste the normal short unencrypted token (length = 22 or so)?
> And you got coverity to go? not just the build?
>
> I did copy the TOKEN as found in the web page, but travis kept failing, with:
>
> "Coverity Scan API access denied. Check $PROJECT_NAME and 
> $COVERITY_SCAN_TOKEN."
>
> Confused.
>
> Christophe.
>
>>
>> Maxim.

[lng-odp] [API-NEXT PATCH] linux-gen: _ishm: fix normal page fallback

[Christophe Milard]

Fixing failure due to lack of huge pages.

Signed-off-by: Christophe Milard <christophe.mil...@linaro.org>
---
 platform/linux-generic/_ishm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c
index 4c2578b..32c0737 100644
--- a/platform/linux-generic/_ishm.c
+++ b/platform/linux-generic/_ishm.c
@@ -548,7 +548,7 @@ static void *do_map(int block_index, uint64_t len, uint32_t 
align,
addr = alloc_fragment(len, block_index, align, );
if (!addr) {
ODP_ERR("alloc_fragment failed.\n");
-   if (new_block->filename[0]) {
+   if (!new_block->external_fd) {
close(*fd);
*fd = -1;
delete_file(new_block);
@@ -563,7 +563,7 @@ static void *do_map(int block_index, uint64_t len, uint32_t 
align,
if (mapped_addr == NULL) {
if (flags & _ODP_ISHM_SINGLE_VA)
free_fragment(fragment);
-   if (new_block->filename[0]) {
+   if (!new_block->external_fd) {
close(*fd);
*fd = -1;
delete_file(new_block);
-- 
2.7.4