[DRAFT] What the new Status Pages potentially look like
All, We're piloting a new format for the podling status pages. Specifically, the current status page leaves a lot to be desired - it's basically crafted html, there's no structure and its hard to find missing items. The end goal is to have a web form editable in Whimsy, but until we get more input on what the content looks like I don't want to make changes like that. I'm asking at this point for podlings to take a look at the status output, potentially update their own content, and see if they like the output. If you navigate to https://whimsy.apache.org/roster/ppmc/ (authentication is via your ASF committer username/password) and find your podling, you'll see your current roster as well as your current status information. The status pages can be found in SVN at this directory https://svn.apache.org/repos/asf/incubator/public/trunk/content/podlings/ Some more information can be found at https://lists.apache.org/thread.html/9504139b5ee9880fdb278f86757803c711fbc962e1934ef2c01a8ed0@%3Cgeneral.incubator.apache.org%3E If you're not sure what to fill out, feel free to reach out. John
[jira] [Resolved] (UNOMI-101) New feature to add the ability to import profiles
[ https://issues.apache.org/jira/browse/UNOMI-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abdelkader Midani resolved UNOMI-101. - Resolution: Done > New feature to add the ability to import profiles > - > > Key: UNOMI-101 > URL: https://issues.apache.org/jira/browse/UNOMI-101 > Project: Apache Unomi > Issue Type: Improvement > Components: core >Affects Versions: 1.2.0-incubating >Reporter: Abdelkader Midani >Assignee: Abdelkader Midani > Fix For: 1.2.0-incubating > > > Add a new feature to UNOMI to allow configuring recurrent / oneshot profiles > import. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
incubator-unomi git commit: UNOMI-101 : Typo
Repository: incubator-unomi Updated Branches: refs/heads/master c8a070f17 -> 9cfa2a007 UNOMI-101 : Typo Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/9cfa2a00 Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/9cfa2a00 Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/9cfa2a00 Branch: refs/heads/master Commit: 9cfa2a00794f1855727c10289a0e266952fc093d Parents: c8a070f Author: Abdelkader Midani Authored: Mon Jun 12 20:14:24 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 20:14:24 2017 +0200 -- .../main/resources/org.apache.unomi.persistence.elasticsearch.cfg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/9cfa2a00/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg -- diff --git a/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg b/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg index 12980e1..77eb091 100644 --- a/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg +++ b/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg @@ -15,7 +15,7 @@ # limitations under the License. # -cluster.name=contextElasticSearch_amidani +cluster.name=contextElasticSearch # The elasticSearchAddresses may be a comma seperated list of host names and ports such as # hostA:9300,hostB:9300 # Note: the port number must be repeated for each host.
[incubator-unomi] Git Push Summary
Repository: incubator-unomi Updated Branches: refs/heads/feature-DMF-1343 [deleted] 7194962f6
[2/3] incubator-unomi git commit: UNOMI-101 : New feature to add the ability to import profiles
UNOMI-101 : New feature to add the ability to import profiles Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/89d4c8eb Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/89d4c8eb Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/89d4c8eb Branch: refs/heads/master Commit: 89d4c8eb9646705cfd6c4bdb79fdac27ee9a5634 Parents: 0f44140 Author: Abdelkader Midani Authored: Tue May 23 02:30:28 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 20:09:59 2017 +0200 -- extensions/pom.xml | 1 + extensions/router/README.md | 113 ++ extensions/router/pom.xml | 64 ++ extensions/router/router-api/pom.xml| 43 .../unomi/router/api/ImportConfiguration.java | 223 +++ .../unomi/router/api/ProfileToImport.java | 77 +++ .../services/ImportConfigurationService.java| 60 + .../api/services/ProfileImportService.java | 29 +++ extensions/router/router-core/pom.xml | 182 +++ .../core/context/ProfileImportCamelContext.java | 170 ++ .../core/processor/ConfigUpdateProcessor.java | 44 .../ImportConfigByFileNameProcessor.java| 44 .../core/processor/LineSplitProcessor.java | 114 ++ .../core/processor/UnomiStorageProcessor.java | 46 .../ProfileImportConfigUpdateRouteBuilder.java | 62 ++ .../ProfileImportKafkaToUnomiRouteBuilder.java | 77 +++ .../route/ProfileImportOneShotRouteBuilder.java | 99 .../ProfileImportSourceToKafkaRouteBuilder.java | 120 ++ .../resources/OSGI-INF/blueprint/blueprint.xml | 94 .../main/resources/org.apache.unomi.router.cfg | 25 +++ extensions/router/router-karaf-feature/pom.xml | 157 + extensions/router/router-rest/pom.xml | 75 +++ .../ImportConfigurationServiceEndPoint.java | 176 +++ .../resources/OSGI-INF/blueprint/blueprint.xml | 80 +++ extensions/router/router-service/pom.xml| 104 + .../ImportConfigurationServiceImpl.java | 114 ++ .../services/ProfileImportServiceImpl.java | 121 ++ .../resources/OSGI-INF/blueprint/blueprint.xml | 39 itests/pom.xml | 8 +- package/pom.xml | 11 + performance-tests/pom.xml | 4 +- ...g.apache.unomi.persistence.elasticsearch.cfg | 2 +- persistence-elasticsearch/plugins/pom.xml | 2 +- samples/trainingplugin.zip | Bin 0 -> 48508 bytes samples/trainingplugin/pom.xml | 55 + .../training/TrainedNotificationAction.java | 61 + .../cxs/actions/trainingNotifAction.json| 13 ++ .../META-INF/cxs/rules/trainedNotification.json | 20 ++ .../resources/OSGI-INF/blueprint/blueprint.xml | 34 +++ 39 files changed, 2755 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/89d4c8eb/extensions/pom.xml -- diff --git a/extensions/pom.xml b/extensions/pom.xml index 7126b61..dea4bc2 100644 --- a/extensions/pom.xml +++ b/extensions/pom.xml @@ -34,6 +34,7 @@ lists-extension privacy-extension geonames +router http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/89d4c8eb/extensions/router/README.md -- diff --git a/extensions/router/README.md b/extensions/router/README.md new file mode 100644 index 000..0d382d4 --- /dev/null +++ b/extensions/router/README.md @@ -0,0 +1,113 @@ + + +Unomi Router +== + +## Getting started +Unomi Router Extension a Karaf Feature that provide an Enterprise Application Integration tool. +It is optional so you must configure it and install it in Karaf, and can be used for Machine - Machine or Human - Machine integration with Unomi. +Mainly Unomi Router Extension aim to make it easy to import third party applications/platforms profiles into Unomi. +This extension is implemented using Apache Camel routes and is using Apache Kafka to buffer import process and make it failsafe. + +## Getting started +1. Configure your Unomi Router: +In the `etc/org.apache.unomi.router.cfg` file, you might want to update the following settings: +Kafka settings +>`#Kafka settings` + +>`kafka.host=localhost` + +>`kafka.port=9092` + +>`kafka.import.topic=camel-deposit` + +>`kafka.import.groupId=unomi-import-group` + +Kafka host and port with the topic name and the groupId t
[3/3] incubator-unomi git commit: UNOMI-101 : Update POM version
UNOMI-101 : Update POM version Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/c8a070f1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/c8a070f1 Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/c8a070f1 Branch: refs/heads/master Commit: c8a070f17b5aaf64ecd28daaaca96a6cff13dcb0 Parents: 89d4c8e Author: Abdelkader Midani Authored: Mon Jun 12 20:03:40 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 20:11:37 2017 +0200 -- extensions/router/pom.xml | 2 +- extensions/router/router-api/pom.xml | 2 +- extensions/router/router-core/pom.xml | 2 +- extensions/router/router-karaf-feature/pom.xml | 2 +- extensions/router/router-rest/pom.xml | 2 +- extensions/router/router-service/pom.xml | 2 +- itests/pom.xml | 2 +- package/pom.xml| 10 ++ performance-tests/pom.xml | 2 +- persistence-elasticsearch/plugins/pom.xml | 2 +- 10 files changed, 19 insertions(+), 9 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/c8a070f1/extensions/router/pom.xml -- diff --git a/extensions/router/pom.xml b/extensions/router/pom.xml index c69320c..225f3d1 100644 --- a/extensions/router/pom.xml +++ b/extensions/router/pom.xml @@ -22,7 +22,7 @@ org.apache.unomi unomi-extensions -1.2.0-incubating-dmf_1343-SNAPSHOT +1.2.0-incubating-SNAPSHOT unomi-router http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/c8a070f1/extensions/router/router-api/pom.xml -- diff --git a/extensions/router/router-api/pom.xml b/extensions/router/router-api/pom.xml index 06207b3..9085a30 100644 --- a/extensions/router/router-api/pom.xml +++ b/extensions/router/router-api/pom.xml @@ -21,7 +21,7 @@ unomi-router org.apache.unomi -1.2.0-incubating-dmf_1343-SNAPSHOT +1.2.0-incubating-SNAPSHOT 4.0.0 http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/c8a070f1/extensions/router/router-core/pom.xml -- diff --git a/extensions/router/router-core/pom.xml b/extensions/router/router-core/pom.xml index 53780e7..c6fbf5d 100644 --- a/extensions/router/router-core/pom.xml +++ b/extensions/router/router-core/pom.xml @@ -21,7 +21,7 @@ unomi-router org.apache.unomi -1.2.0-incubating-dmf_1343-SNAPSHOT +1.2.0-incubating-SNAPSHOT 4.0.0 http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/c8a070f1/extensions/router/router-karaf-feature/pom.xml -- diff --git a/extensions/router/router-karaf-feature/pom.xml b/extensions/router/router-karaf-feature/pom.xml index 8a8e62c..06c16da 100644 --- a/extensions/router/router-karaf-feature/pom.xml +++ b/extensions/router/router-karaf-feature/pom.xml @@ -21,7 +21,7 @@ unomi-router org.apache.unomi -1.2.0-incubating-dmf_1343-SNAPSHOT +1.2.0-incubating-SNAPSHOT 4.0.0 http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/c8a070f1/extensions/router/router-rest/pom.xml -- diff --git a/extensions/router/router-rest/pom.xml b/extensions/router/router-rest/pom.xml index e39e919..fc1065f 100644 --- a/extensions/router/router-rest/pom.xml +++ b/extensions/router/router-rest/pom.xml @@ -21,7 +21,7 @@ unomi-router org.apache.unomi -1.2.0-incubating-dmf_1343-SNAPSHOT +1.2.0-incubating-SNAPSHOT 4.0.0 http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/c8a070f1/extensions/router/router-service/pom.xml -- diff --git a/extensions/router/router-service/pom.xml b/extensions/router/router-service/pom.xml index 0aed70e..750cd1e 100644 --- a/extensions/router/router-service/pom.xml +++ b/extensions/router/router-service/pom.xml @@ -21,7 +21,7 @@ unomi-router org.apache.unomi -1.2.0-incubating-dmf_1343-SNAPSHOT +1.2.0-incubating-SNAPSHOT 4.0.0 http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/c8a070f1/itests/pom.xml -- diff --git a/itests/pom.xml b/itests/pom.xml index 55fe567..036cb1d 100644 --- a/itests/pom.xml +++ b/itests/pom.xml @@ -21,7 +21,7 @@ org.ap
[1/3] incubator-unomi git commit: UNOMI-101 : New feature to add the ability to import profiles
Repository: incubator-unomi Updated Branches: refs/heads/master 0f4414039 -> c8a070f17 http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/89d4c8eb/extensions/router/router-karaf-feature/pom.xml -- diff --git a/extensions/router/router-karaf-feature/pom.xml b/extensions/router/router-karaf-feature/pom.xml new file mode 100644 index 000..8a8e62c --- /dev/null +++ b/extensions/router/router-karaf-feature/pom.xml @@ -0,0 +1,157 @@ + + +http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> + +unomi-router +org.apache.unomi +1.2.0-incubating-dmf_1343-SNAPSHOT + +4.0.0 + +unomi-router-karaf-feature +Apache Unomi :: Extensions :: Router :: Apache Karaf Feature +Apache Karaf feature for the Apache Unomi Context Server extension +feature + + +org.apache.servicemix.bundles +org.apache.servicemix.bundles.jsch +0.1.54_1 + + +commons-net +commons-net +3.5 + + +org.apache.kafka +kafka-clients +0.10.1.0 + + +org.apache.camel +camel-core +${camel.version} + + +org.apache.camel +camel-blueprint +${camel.version} + + +org.apache.camel +camel-jackson +${camel.version} + + +org.apache.camel +camel-http-common +${camel.version} + + +org.apache.camel +camel-servlet +${camel.version} + + +org.apache.camel +camel-ftp +${camel.version} + + +org.apache.camel +camel-stream +${camel.version} + + +org.apache.camel +camel-kafka +${camel.version} + + + +org.apache.cxf +cxf-rt-rs-security-cors +${cxf.version} + + + + +org.apache.unomi +unomi-persistence-spi +${project.version} +provided + + +org.apache.unomi +unomi-router-api +${project.version} + + +org.apache.unomi +unomi-router-core +${project.version} + + +org.apache.unomi +unomi-router-service +${project.version} + + +org.apache.unomi +unomi-router-rest +${project.version} + + +org.apache.httpcomponents +httpclient-osgi +bundle + + +org.apache.httpcomponents +httpcore-osgi + + + + + + + +org.apache.karaf.tooling +karaf-maven-plugin +true + + false + + + + + + + +org.apache.karaf.tooling +karaf-maven-plugin + +90 + + + + + \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/89d4c8eb/extensions/router/router-rest/pom.xml -- diff --git a/extensions/router/router-rest/pom.xml b/extensions/router/router-rest/pom.xml new file mode 100644 index 000..e39e919 --- /dev/null +++ b/extensions/router/router-rest/pom.xml @@ -0,0 +1,75 @@ + + +http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> + +unomi-router +org.apache.unomi +1.2.0-incubating-dmf_1343-SNAPSHOT + +4.0.0 + +unomi-router-rest +Apache Unomi :: Extensions :: Router :: REST API +Router REST API +bundle + + + +org.apache.unomi +unomi-router-api +${project.version} +provided + + +org.apache.unomi +unomi-api +${project.version} +provided + + +javax.ws.rs +javax.ws.rs-api +2.0.1 +provided + + +org.apache.cxf +cxf-rt-rs-security-cors +provided + + +org.apache.ht
incubator-unomi git commit: UNOMI-101 : Update POM version
Repository: incubator-unomi Updated Branches: refs/heads/feature-DMF-1343 7c36249b7 -> 7194962f6 UNOMI-101 : Update POM version Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/7194962f Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/7194962f Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/7194962f Branch: refs/heads/feature-DMF-1343 Commit: 7194962f67c88cbccdbf47a62c5df9f720563d47 Parents: 7c36249 Author: Abdelkader Midani Authored: Mon Jun 12 20:03:40 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 20:03:40 2017 +0200 -- api/pom.xml| 2 +- extensions/geonames/pom.xml| 2 +- extensions/geonames/rest/pom.xml | 8 +++--- extensions/geonames/services/pom.xml | 8 +++--- extensions/lists-extension/actions/pom.xml | 8 +++--- extensions/lists-extension/pom.xml | 4 +-- extensions/lists-extension/rest/pom.xml| 8 +++--- extensions/lists-extension/services/pom.xml| 8 +++--- extensions/pom.xml | 2 +- extensions/privacy-extension/pom.xml | 4 +-- extensions/privacy-extension/rest/pom.xml | 8 +++--- extensions/privacy-extension/services/pom.xml | 8 +++--- extensions/router/pom.xml | 2 +- extensions/router/router-api/pom.xml | 2 +- extensions/router/router-core/pom.xml | 2 +- extensions/router/router-karaf-feature/pom.xml | 2 +- extensions/router/router-rest/pom.xml | 2 +- extensions/router/router-service/pom.xml | 2 +- itests/pom.xml | 2 +- kar/pom.xml| 32 ++--- lifecycle-watcher/pom.xml | 2 +- package/pom.xml| 14 +++-- performance-tests/pom.xml | 2 +- persistence-elasticsearch/core/pom.xml | 6 ++-- persistence-elasticsearch/plugins/pom.xml | 2 +- persistence-elasticsearch/pom.xml | 2 +- persistence-spi/pom.xml| 4 +-- plugins/baseplugin/pom.xml | 6 ++-- plugins/hover-event/pom.xml| 4 +-- plugins/mail/pom.xml | 4 +-- plugins/optimization-test/pom.xml | 2 +- plugins/past-event/pom.xml | 6 ++-- plugins/pom.xml| 4 +-- plugins/request/pom.xml| 2 +- plugins/tracked-event/pom.xml | 6 ++-- pom.xml| 2 +- rest/pom.xml | 6 ++-- samples/pom.xml| 2 +- samples/tweet-button-plugin/pom.xml| 4 +-- services/pom.xml | 6 ++-- wab/pom.xml| 6 ++-- 41 files changed, 109 insertions(+), 99 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/7194962f/api/pom.xml -- diff --git a/api/pom.xml b/api/pom.xml index b69b98e..6662b6f 100644 --- a/api/pom.xml +++ b/api/pom.xml @@ -22,7 +22,7 @@ org.apache.unomi unomi-root -1.2.0-incubating-dmf_1343-SNAPSHOT +1.2.0-incubating-SNAPSHOT unomi-api http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/7194962f/extensions/geonames/pom.xml -- diff --git a/extensions/geonames/pom.xml b/extensions/geonames/pom.xml index 7559e08..f601a7c 100644 --- a/extensions/geonames/pom.xml +++ b/extensions/geonames/pom.xml @@ -27,7 +27,7 @@ org.apache.unomi unomi-extensions -1.2.0-incubating-dmf_1343-SNAPSHOT +1.2.0-incubating-SNAPSHOT cxs-geonames http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/7194962f/extensions/geonames/rest/pom.xml -- diff --git a/extensions/geonames/rest/pom.xml b/extensions/geonames/rest/pom.xml index 9df2e43..cec5546 100644 --- a/extensions/geonames/rest/pom.xml +++ b/extensions/geonames/rest/pom.xml @@ -20,7 +20,7 @@ org.apache.unomi cxs-geonames -1.2.0-incubating-dmf_1343-SNAPSHOT +1.2.0-incubating-SNAPSHOT 4.0.0 @@ -33,14 +33,14 @@ org.apache.unomi unomi-api -1.2.0-incubating-dmf_1343-SNAPSHOT +1.2.0-incubating-SNAPSHOT provided org.apache.unomi cxs-geonames-ser
[2/2] incubator-unomi git commit: UNOMI-101 : New feature to add the ability to import profiles
UNOMI-101 : New feature to add the ability to import profiles Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/7c36249b Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/7c36249b Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/7c36249b Branch: refs/heads/feature-DMF-1343 Commit: 7c36249b774064196b31f787834c51cfdb43426e Parents: b3c8953 Author: Abdelkader Midani Authored: Tue May 23 02:30:28 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 19:34:10 2017 +0200 -- extensions/pom.xml | 1 + extensions/router/README.md | 113 ++ extensions/router/pom.xml | 64 ++ extensions/router/router-api/pom.xml| 43 .../unomi/router/api/ImportConfiguration.java | 223 +++ .../unomi/router/api/ProfileToImport.java | 77 +++ .../services/ImportConfigurationService.java| 60 + .../api/services/ProfileImportService.java | 29 +++ extensions/router/router-core/pom.xml | 182 +++ .../core/context/ProfileImportCamelContext.java | 170 ++ .../core/processor/ConfigUpdateProcessor.java | 44 .../ImportConfigByFileNameProcessor.java| 44 .../core/processor/LineSplitProcessor.java | 114 ++ .../core/processor/UnomiStorageProcessor.java | 46 .../ProfileImportConfigUpdateRouteBuilder.java | 62 ++ .../ProfileImportKafkaToUnomiRouteBuilder.java | 77 +++ .../route/ProfileImportOneShotRouteBuilder.java | 99 .../ProfileImportSourceToKafkaRouteBuilder.java | 120 ++ .../resources/OSGI-INF/blueprint/blueprint.xml | 94 .../main/resources/org.apache.unomi.router.cfg | 25 +++ extensions/router/router-karaf-feature/pom.xml | 157 + extensions/router/router-rest/pom.xml | 75 +++ .../ImportConfigurationServiceEndPoint.java | 176 +++ .../resources/OSGI-INF/blueprint/blueprint.xml | 80 +++ extensions/router/router-service/pom.xml| 104 + .../ImportConfigurationServiceImpl.java | 114 ++ .../services/ProfileImportServiceImpl.java | 121 ++ .../resources/OSGI-INF/blueprint/blueprint.xml | 39 itests/pom.xml | 8 +- package/pom.xml | 11 + performance-tests/pom.xml | 4 +- ...g.apache.unomi.persistence.elasticsearch.cfg | 2 +- persistence-elasticsearch/plugins/pom.xml | 2 +- samples/trainingplugin.zip | Bin 0 -> 48508 bytes samples/trainingplugin/pom.xml | 55 + .../training/TrainedNotificationAction.java | 61 + .../cxs/actions/trainingNotifAction.json| 13 ++ .../META-INF/cxs/rules/trainedNotification.json | 20 ++ .../resources/OSGI-INF/blueprint/blueprint.xml | 34 +++ 39 files changed, 2755 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/7c36249b/extensions/pom.xml -- diff --git a/extensions/pom.xml b/extensions/pom.xml index 6aa2fd8..8a874a6 100644 --- a/extensions/pom.xml +++ b/extensions/pom.xml @@ -34,6 +34,7 @@ lists-extension privacy-extension geonames +router http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/7c36249b/extensions/router/README.md -- diff --git a/extensions/router/README.md b/extensions/router/README.md new file mode 100644 index 000..0d382d4 --- /dev/null +++ b/extensions/router/README.md @@ -0,0 +1,113 @@ + + +Unomi Router +== + +## Getting started +Unomi Router Extension a Karaf Feature that provide an Enterprise Application Integration tool. +It is optional so you must configure it and install it in Karaf, and can be used for Machine - Machine or Human - Machine integration with Unomi. +Mainly Unomi Router Extension aim to make it easy to import third party applications/platforms profiles into Unomi. +This extension is implemented using Apache Camel routes and is using Apache Kafka to buffer import process and make it failsafe. + +## Getting started +1. Configure your Unomi Router: +In the `etc/org.apache.unomi.router.cfg` file, you might want to update the following settings: +Kafka settings +>`#Kafka settings` + +>`kafka.host=localhost` + +>`kafka.port=9092` + +>`kafka.import.topic=camel-deposit` + +>`kafka.import.groupId=unomi-import-group` + +Kafka host and port with the topic name and the
[1/2] incubator-unomi git commit: UNOMI-101 : New feature to add the ability to import profiles [Forced Update!]
Repository: incubator-unomi Updated Branches: refs/heads/feature-DMF-1343 63debd5fc -> 7c36249b7 (forced update) http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/7c36249b/extensions/router/router-karaf-feature/pom.xml -- diff --git a/extensions/router/router-karaf-feature/pom.xml b/extensions/router/router-karaf-feature/pom.xml new file mode 100644 index 000..8a8e62c --- /dev/null +++ b/extensions/router/router-karaf-feature/pom.xml @@ -0,0 +1,157 @@ + + +http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> + +unomi-router +org.apache.unomi +1.2.0-incubating-dmf_1343-SNAPSHOT + +4.0.0 + +unomi-router-karaf-feature +Apache Unomi :: Extensions :: Router :: Apache Karaf Feature +Apache Karaf feature for the Apache Unomi Context Server extension +feature + + +org.apache.servicemix.bundles +org.apache.servicemix.bundles.jsch +0.1.54_1 + + +commons-net +commons-net +3.5 + + +org.apache.kafka +kafka-clients +0.10.1.0 + + +org.apache.camel +camel-core +${camel.version} + + +org.apache.camel +camel-blueprint +${camel.version} + + +org.apache.camel +camel-jackson +${camel.version} + + +org.apache.camel +camel-http-common +${camel.version} + + +org.apache.camel +camel-servlet +${camel.version} + + +org.apache.camel +camel-ftp +${camel.version} + + +org.apache.camel +camel-stream +${camel.version} + + +org.apache.camel +camel-kafka +${camel.version} + + + +org.apache.cxf +cxf-rt-rs-security-cors +${cxf.version} + + + + +org.apache.unomi +unomi-persistence-spi +${project.version} +provided + + +org.apache.unomi +unomi-router-api +${project.version} + + +org.apache.unomi +unomi-router-core +${project.version} + + +org.apache.unomi +unomi-router-service +${project.version} + + +org.apache.unomi +unomi-router-rest +${project.version} + + +org.apache.httpcomponents +httpclient-osgi +bundle + + +org.apache.httpcomponents +httpcore-osgi + + + + + + + +org.apache.karaf.tooling +karaf-maven-plugin +true + + false + + + + + + + +org.apache.karaf.tooling +karaf-maven-plugin + +90 + + + + + \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/7c36249b/extensions/router/router-rest/pom.xml -- diff --git a/extensions/router/router-rest/pom.xml b/extensions/router/router-rest/pom.xml new file mode 100644 index 000..e39e919 --- /dev/null +++ b/extensions/router/router-rest/pom.xml @@ -0,0 +1,75 @@ + + +http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> + +unomi-router +org.apache.unomi +1.2.0-incubating-dmf_1343-SNAPSHOT + +4.0.0 + +unomi-router-rest +Apache Unomi :: Extensions :: Router :: REST API +Router REST API +bundle + + + +org.apache.unomi +unomi-router-api +${project.version} +provided + + +org.apache.unomi +unomi-api +${project.version} +provided + + +javax.ws.rs +javax.ws.rs-api +2.0.1 +provided + + +org.apache.cxf +cxf-rt-rs-security-cors +provided + +
[jira] [Created] (UNOMI-101) New feature to add the ability to import profiles
Abdelkader Midani created UNOMI-101: --- Summary: New feature to add the ability to import profiles Key: UNOMI-101 URL: https://issues.apache.org/jira/browse/UNOMI-101 Project: Apache Unomi Issue Type: Improvement Components: core Affects Versions: 1.2.0-incubating Reporter: Abdelkader Midani Assignee: Abdelkader Midani Fix For: 1.2.0-incubating Add a new feature to UNOMI to allow configuring recurrent / oneshot profiles import. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[12/14] incubator-unomi git commit: DMF-1566 : Flat files import - Ability to define column and line separator
DMF-1566 : Flat files import - Ability to define column and line separator Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/d95595fd Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/d95595fd Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/d95595fd Branch: refs/heads/feature-DMF-1343 Commit: d95595fdc27dee4af1ca8741431e188c7ff875bc Parents: e78c014 Author: Abdelkader Midani Authored: Tue Jun 6 17:35:17 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 19:24:29 2017 +0200 -- .../unomi/router/api/ImportConfiguration.java | 39 +++- .../core/processor/LineSplitProcessor.java | 12 +- .../route/ProfileImportOneShotRouteBuilder.java | 2 +- .../ProfileImportSourceToKafkaRouteBuilder.java | 3 +- .../resources/OSGI-INF/blueprint/blueprint.xml | 4 +- .../services/ProfileImportServiceImpl.java | 2 +- 6 files changed, 55 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/d95595fd/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/ImportConfiguration.java -- diff --git a/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/ImportConfiguration.java b/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/ImportConfiguration.java index 770a7b5..29900f9 100644 --- a/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/ImportConfiguration.java +++ b/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/ImportConfiguration.java @@ -17,7 +17,6 @@ package org.apache.unomi.router.api; import org.apache.unomi.api.Item; -import org.apache.unomi.api.MetadataItem; import java.util.HashMap; import java.util.List; @@ -41,6 +40,9 @@ public class ImportConfiguration extends Item { private String mergingProperty; private boolean overwriteExistingProfiles = false; private List propertiesToOverwrite; + +private String columnSeparator = ","; +private String lineSeparator = "\n"; private boolean active = false; /** @@ -182,4 +184,39 @@ public class ImportConfiguration extends Item { this.propertiesToOverwrite = propertiesToOverwrite; } +/** + * gets the column separator. + */ +public String getColumnSeparator() { +return this.columnSeparator; +} + +/** + * Sets the column separator. + * @param columnSeparator property used to specify a line separator. Defaults to ',' + */ +public void setColumnSeparator(String columnSeparator) { +if(this.columnSeparator !=null) { +this.columnSeparator = columnSeparator; +} +} + +/** + * gets the line separator. + */ +public String getLineSeparator() { +return this.lineSeparator; +} + +/** + * Sets the line separator. + * @param lineSeparator property used to specify a line separator. Defaults to '\n' + */ +public void setLineSeparator(String lineSeparator) { +if(lineSeparator != null) { +this.lineSeparator = lineSeparator; +} +} + + } http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/d95595fd/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/processor/LineSplitProcessor.java -- diff --git a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/processor/LineSplitProcessor.java b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/processor/LineSplitProcessor.java index 30f79bf..150ef6d 100644 --- a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/processor/LineSplitProcessor.java +++ b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/processor/LineSplitProcessor.java @@ -37,6 +37,7 @@ public class LineSplitProcessor implements Processor { private List propertiesToOverwrite; private String mergingProperty; private boolean overwriteExistingProfiles; +private String columnSeparator; @Override public void process(Exchange exchange) throws Exception { @@ -47,8 +48,9 @@ public class LineSplitProcessor implements Processor { propertiesToOverwrite = importConfigOneShot.getPropertiesToOverwrite(); mergingProperty = importConfigOneShot.getMergingProperty(); overwriteExistingProfiles = importConfigOneShot.isOverwriteExistingProfiles(); +columnSeparator = importConfigOneShot.getColumnSeparator(); } -String[] profileData = ((String)exchange.getIn().getBody()).split(","); +String[] profi
[07/14] incubator-unomi git commit: [jgitflow-maven-plugin]updating poms for dmf_1343 version
[jgitflow-maven-plugin]updating poms for dmf_1343 version Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/b3c8953e Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/b3c8953e Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/b3c8953e Branch: refs/heads/feature-DMF-1343 Commit: b3c8953ebae847d2e13f34e88c2585b12ae3494d Parents: 0f44140 Author: Abdelkader Midani Authored: Tue May 23 02:28:32 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 19:24:29 2017 +0200 -- api/pom.xml | 2 +- extensions/geonames/pom.xml | 2 +- extensions/geonames/rest/pom.xml | 8 +++--- extensions/geonames/services/pom.xml | 8 +++--- extensions/lists-extension/actions/pom.xml| 8 +++--- extensions/lists-extension/pom.xml| 4 +-- extensions/lists-extension/rest/pom.xml | 8 +++--- extensions/lists-extension/services/pom.xml | 8 +++--- extensions/pom.xml| 2 +- extensions/privacy-extension/pom.xml | 4 +-- extensions/privacy-extension/rest/pom.xml | 8 +++--- extensions/privacy-extension/services/pom.xml | 8 +++--- kar/pom.xml | 32 +++--- lifecycle-watcher/pom.xml | 6 ++-- package/pom.xml | 4 +-- persistence-elasticsearch/core/pom.xml| 6 ++-- persistence-elasticsearch/pom.xml | 2 +- persistence-spi/pom.xml | 4 +-- plugins/baseplugin/pom.xml| 6 ++-- plugins/hover-event/pom.xml | 4 +-- plugins/mail/pom.xml | 4 +-- plugins/optimization-test/pom.xml | 2 +- plugins/past-event/pom.xml| 6 ++-- plugins/pom.xml | 4 +-- plugins/request/pom.xml | 2 +- plugins/tracked-event/pom.xml | 6 ++-- pom.xml | 2 +- rest/pom.xml | 6 ++-- samples/pom.xml | 2 +- samples/tweet-button-plugin/pom.xml | 4 +-- services/pom.xml | 6 ++-- wab/pom.xml | 6 ++-- 32 files changed, 91 insertions(+), 93 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/b3c8953e/api/pom.xml -- diff --git a/api/pom.xml b/api/pom.xml index 6662b6f..b69b98e 100644 --- a/api/pom.xml +++ b/api/pom.xml @@ -22,7 +22,7 @@ org.apache.unomi unomi-root -1.2.0-incubating-SNAPSHOT +1.2.0-incubating-dmf_1343-SNAPSHOT unomi-api http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/b3c8953e/extensions/geonames/pom.xml -- diff --git a/extensions/geonames/pom.xml b/extensions/geonames/pom.xml index f601a7c..7559e08 100644 --- a/extensions/geonames/pom.xml +++ b/extensions/geonames/pom.xml @@ -27,7 +27,7 @@ org.apache.unomi unomi-extensions -1.2.0-incubating-SNAPSHOT +1.2.0-incubating-dmf_1343-SNAPSHOT cxs-geonames http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/b3c8953e/extensions/geonames/rest/pom.xml -- diff --git a/extensions/geonames/rest/pom.xml b/extensions/geonames/rest/pom.xml index cec5546..9df2e43 100644 --- a/extensions/geonames/rest/pom.xml +++ b/extensions/geonames/rest/pom.xml @@ -20,7 +20,7 @@ org.apache.unomi cxs-geonames -1.2.0-incubating-SNAPSHOT +1.2.0-incubating-dmf_1343-SNAPSHOT 4.0.0 @@ -33,14 +33,14 @@ org.apache.unomi unomi-api -1.2.0-incubating-SNAPSHOT +1.2.0-incubating-dmf_1343-SNAPSHOT provided org.apache.unomi cxs-geonames-services -1.2.0-incubating-SNAPSHOT +1.2.0-incubating-dmf_1343-SNAPSHOT provided @@ -76,7 +76,7 @@ org.apache.unomi unomi-persistence-spi -1.2.0-incubating-SNAPSHOT +1.2.0-incubating-dmf_1343-SNAPSHOT provided http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/b3c8953e/extensions/geonames/services/pom.xml -- diff --git a/extensions/geonames/services/pom.xml b/extensions/geonames/services/pom.xml in
[14/14] incubator-unomi git commit: DMF-1343 : Typo
DMF-1343 : Typo Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/4f3b17f0 Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/4f3b17f0 Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/4f3b17f0 Branch: refs/heads/feature-DMF-1343 Commit: 4f3b17f0a64e0141be70b59c568e2a8e71b80808 Parents: e193b51 Author: Abdelkader Midani Authored: Mon May 29 16:58:12 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 19:24:29 2017 +0200 -- extensions/router/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/4f3b17f0/extensions/router/README.md -- diff --git a/extensions/router/README.md b/extensions/router/README.md index 8fa1994..0d382d4 100644 --- a/extensions/router/README.md +++ b/extensions/router/README.md @@ -26,7 +26,7 @@ This extension is implemented using Apache Camel routes and is using Apache Kafk ## Getting started 1. Configure your Unomi Router: -In the `etc/org.apache.unomi.sfdc.cfg` file, you might want to update the following settings: +In the `etc/org.apache.unomi.router.cfg` file, you might want to update the following settings: Kafka settings >`#Kafka settings`
[08/14] incubator-unomi git commit: DMF-1343 Import profiles from CSV through API
http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/b68909b1/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java -- diff --git a/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java b/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java new file mode 100644 index 000..e81930a --- /dev/null +++ b/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.unomi.router.rest; + +import com.fasterxml.jackson.databind.ObjectMapper; +import org.apache.cxf.jaxrs.ext.multipart.Multipart; +import org.apache.cxf.jaxrs.ext.multipart.Attachment; +import org.apache.cxf.rs.security.cors.CrossOriginResourceSharing; +import org.apache.http.HttpResponse; +import org.apache.http.client.methods.HttpPut; +import org.apache.http.entity.StringEntity; +import org.apache.http.impl.client.CloseableHttpClient; +import org.apache.http.impl.client.HttpClients; +import org.apache.unomi.router.api.ImportConfiguration; +import org.apache.unomi.router.api.services.ImportConfigurationService; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.jws.WebMethod; +import javax.jws.WebService; +import javax.ws.rs.*; +import javax.ws.rs.core.MediaType; +import javax.ws.rs.core.Response; +import java.io.IOException; +import java.io.InputStream; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.List; + +/** + * A JAX-RS endpoint to manage {@link org.apache.unomi.router.api.ImportConfiguration}s. + */ +@WebService +@CrossOriginResourceSharing( +allowAllOrigins = true, +allowCredentials = true +) +public class ImportConfigurationServiceEndPoint { + +private static final Logger logger = LoggerFactory.getLogger(ImportConfigurationServiceEndPoint.class.getName()); + +private ImportConfigurationService importConfigurationService; +private String uploadDir; + +public ImportConfigurationServiceEndPoint () { +logger.info("Initializing import configuration service endpoint..."); +} + +@WebMethod(exclude = true) +public void setImportConfigurationService(ImportConfigurationService importConfigurationService) { +this.importConfigurationService = importConfigurationService; +} + +@WebMethod(exclude = true) +public void setUploadDir(String uploadDir) { +this.uploadDir = uploadDir; +} + +/** + * Retrieves all the import configurations. + * + * @return all the import configurations. + */ +@GET +@Path("/") +@Produces(MediaType.APPLICATION_JSON) +@Consumes(MediaType.APPLICATION_JSON) +public List getImportConfigurations() { +return importConfigurationService.getImportConfigurations(); +} + +/** + * Retrieves an import configuration by id. + * + * @return the import configuration that matches the given id. + */ +@GET +@Path("/{configId}") +@Produces(MediaType.APPLICATION_JSON) +@Consumes(MediaType.APPLICATION_JSON) +public ImportConfiguration getImportConfiguration(@PathParam("configId") String configId) { +return importConfigurationService.load(configId); +} + +/** + * Delete an import configuration by id. + * + * @return the deleted import configuration. + */ +@DELETE +@Path("/{configId}") +@Produces(MediaType.APPLICATION_JSON) +@Consumes(MediaType.APPLICATION_JSON) +public void deleteImportConfiguration(@PathParam("configId") String configId) { +importConfigurationService.delete(configId); +} + + + +/** + * Save the given import configuration. + * + * @return the import configuration saved. + */ +@POST +@Path("/") +@Produces(MediaType.APPLICATION_JSON) +@Consumes(MediaType.APPLICATION_JSON) +public ImportConfiguration saveImportConfiguration(ImportConfiguration importConfiguration) { +ImportConfiguration importConfigS
[06/14] incubator-unomi git commit: DMF-1343 : Handle transforming an import config oneshot <--> recurrent
DMF-1343 : Handle transforming an import config oneshot <--> recurrent Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/e193b511 Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/e193b511 Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/e193b511 Branch: refs/heads/feature-DMF-1343 Commit: e193b5112835b4f2a4a15abee4cb7f0d227b143c Parents: 77fc9e4 Author: Abdelkader Midani Authored: Mon May 29 12:14:20 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 19:24:29 2017 +0200 -- .../core/context/ProfileImportCamelContext.java | 15 ++- 1 file changed, 10 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/e193b511/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/ProfileImportCamelContext.java -- diff --git a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/ProfileImportCamelContext.java b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/ProfileImportCamelContext.java index 2f3eaad..df734d3 100644 --- a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/ProfileImportCamelContext.java +++ b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/ProfileImportCamelContext.java @@ -54,6 +54,8 @@ public class ProfileImportCamelContext implements SynchronousBundleListener { private String uploadDir; private Map kafkaProps; +private final String IMPORT_CONFIG_TYPE_RECURRENT = "recurrent"; + private BundleContext bundleContext; public void setBundleContext(BundleContext bundleContext) { @@ -109,11 +111,14 @@ public class ProfileImportCamelContext implements SynchronousBundleListener { if(route!=null && stopRoute(importConfiguration.getItemId())) { camelContext.removeRoute(importConfiguration.getItemId()); } -ProfileImportSourceToKafkaRouteBuilder builder = new ProfileImportSourceToKafkaRouteBuilder(kafkaProps); -builder.setImportConfigurationList(Arrays.asList(importConfiguration)); -builder.setJacksonDataFormat(jacksonDataFormat); -builder.setContext(camelContext); -camelContext.addRoutes(builder); +//Handle transforming an import config oneshot <--> recurrent + if(IMPORT_CONFIG_TYPE_RECURRENT.equals(importConfiguration.getConfigType())){ +ProfileImportSourceToKafkaRouteBuilder builder = new ProfileImportSourceToKafkaRouteBuilder(kafkaProps); + builder.setImportConfigurationList(Arrays.asList(importConfiguration)); +builder.setJacksonDataFormat(jacksonDataFormat); +builder.setContext(camelContext); +camelContext.addRoutes(builder); +} } public CamelContext getCamelContext() {
[09/14] incubator-unomi git commit: DMF-1343 Import profiles from CSV through API
DMF-1343 Import profiles from CSV through API Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/b68909b1 Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/b68909b1 Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/b68909b1 Branch: refs/heads/feature-DMF-1343 Commit: b68909b1ef190485430e3f794f6793343d68b34c Parents: b3c8953 Author: Abdelkader Midani Authored: Tue May 23 02:30:28 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 19:24:29 2017 +0200 -- pom.xml | 1 + router/pom.xml | 64 +++ router/router-api/pom.xml | 43 + .../unomi/router/api/ImportConfiguration.java | 138 ++ .../unomi/router/api/ProfileToImport.java | 77 .../services/ImportConfigurationService.java| 60 ++ .../api/services/ProfileImportService.java | 29 +++ router/router-core/pom.xml | 182 +++ .../core/context/ProfileImportCamelContext.java | 165 + .../core/processor/ConfigUpdateProcessor.java | 44 + .../ImportConfigByFileNameProcessor.java| 44 + .../core/processor/LineSplitProcessor.java | 104 +++ .../core/processor/UnomiStorageProcessor.java | 46 + .../ProfileImportConfigUpdateRouteBuilder.java | 67 +++ .../ProfileImportKafkaToUnomiRouteBuilder.java | 77 .../route/ProfileImportOneShotRouteBuilder.java | 99 ++ .../ProfileImportSourceToKafkaRouteBuilder.java | 119 .../resources/OSGI-INF/blueprint/blueprint.xml | 94 ++ .../main/resources/org.apache.unomi.router.cfg | 23 +++ router/router-karaf-feature/pom.xml | 164 + router/router-rest/pom.xml | 75 .../ImportConfigurationServiceEndPoint.java | 167 + .../resources/OSGI-INF/blueprint/blueprint.xml | 72 router/router-service/pom.xml | 104 +++ .../ImportConfigurationServiceImpl.java | 114 .../services/ProfileImportServiceImpl.java | 122 + .../resources/OSGI-INF/blueprint/blueprint.xml | 39 samples/trainingplugin.zip | Bin 0 -> 48508 bytes samples/trainingplugin/pom.xml | 55 ++ .../training/TrainedNotificationAction.java | 61 +++ .../cxs/actions/trainingNotifAction.json| 13 ++ .../META-INF/cxs/rules/trainedNotification.json | 20 ++ .../resources/OSGI-INF/blueprint/blueprint.xml | 34 33 files changed, 2516 insertions(+) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/b68909b1/pom.xml -- diff --git a/pom.xml b/pom.xml index 3881b47..cdce2e6 100644 --- a/pom.xml +++ b/pom.xml @@ -844,6 +844,7 @@ extensions kar samples +router package http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/b68909b1/router/pom.xml -- diff --git a/router/pom.xml b/router/pom.xml new file mode 100644 index 000..73e9953 --- /dev/null +++ b/router/pom.xml @@ -0,0 +1,64 @@ + +http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +unomi-root +org.apache.unomi +1.2.0-incubating-SNAPSHOT + + +unomi-router +Apache Unomi :: Extensions :: Router +Apache Camel Router for the Apache Unomi Context server +pom + + +2.18.3 + + + + + +org.apache.felix +maven-bundle-plugin +true + + + *;scope=compile|runtime + +sun.misc;resolution:=optional, +* + + + + + + + + +router-api +router-service +router-core +router-rest +router-karaf-feature + + + http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/b68909b1/router/router-api/pom.xml -- diff --git a/router/router-api/pom.xml b/router/router-api/pom.xml new file mode 100644 index 000..9085a30 --- /dev/null +++ b/router/router-api/pom.xml @@ -0,0 +1,43 @@ + + +http://maven.apache.org/POM/4.0.0"; + xmlns:xsi="h
[02/14] incubator-unomi git commit: DMF-1343 : Update POM versions
DMF-1343 : Update POM versions Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/e78c0145 Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/e78c0145 Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/e78c0145 Branch: refs/heads/feature-DMF-1343 Commit: e78c014564739f0b50aa382336d3240f97eb5cd7 Parents: 4f3b17f Author: Abdelkader Midani Authored: Fri Jun 2 16:39:32 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 19:24:29 2017 +0200 -- extensions/router/router-core/pom.xml | 2 +- itests/pom.xml| 8 performance-tests/pom.xml | 4 ++-- persistence-elasticsearch/plugins/pom.xml | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/e78c0145/extensions/router/router-core/pom.xml -- diff --git a/extensions/router/router-core/pom.xml b/extensions/router/router-core/pom.xml index b0b97c4..53780e7 100644 --- a/extensions/router/router-core/pom.xml +++ b/extensions/router/router-core/pom.xml @@ -56,7 +56,7 @@ org.apache.unomi unomi-router-api -1.2.0-incubating-SNAPSHOT +${project.version} org.apache.camel http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/e78c0145/itests/pom.xml -- diff --git a/itests/pom.xml b/itests/pom.xml index 6f3724c..55fe567 100644 --- a/itests/pom.xml +++ b/itests/pom.xml @@ -21,7 +21,7 @@ org.apache.unomi unomi-root -1.2.0-incubating-SNAPSHOT +1.2.0-incubating-dmf_1343-SNAPSHOT unomi-itests Apache Unomi :: Integration Tests @@ -32,19 +32,19 @@ org.apache.unomi unomi-kar features -1.2.0-incubating-SNAPSHOT +${project.version} xml org.apache.unomi unomi-persistence-spi -1.2.0-incubating-SNAPSHOT +${project.version} provided org.apache.unomi unomi-wab -1.2.0-incubating-SNAPSHOT +${project.version} test http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/e78c0145/performance-tests/pom.xml -- diff --git a/performance-tests/pom.xml b/performance-tests/pom.xml index 72c7abd..44117fc 100644 --- a/performance-tests/pom.xml +++ b/performance-tests/pom.xml @@ -20,7 +20,7 @@ org.apache.unomi unomi-root -1.2.0-incubating-SNAPSHOT +1.2.0-incubating-dmf_1343-SNAPSHOT unomi-performance-tests Apache Unomi :: Performance Tests @@ -30,7 +30,7 @@ org.apache.unomi unomi-wab -1.2.0-incubating-SNAPSHOT +${project.version} test http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/e78c0145/persistence-elasticsearch/plugins/pom.xml -- diff --git a/persistence-elasticsearch/plugins/pom.xml b/persistence-elasticsearch/plugins/pom.xml index 9114ece..57eec00 100644 --- a/persistence-elasticsearch/plugins/pom.xml +++ b/persistence-elasticsearch/plugins/pom.xml @@ -22,7 +22,7 @@ org.apache.unomi unomi-persistence-elasticsearch -1.2.0-incubating-SNAPSHOT +1.2.0-incubating-dmf_1343-SNAPSHOT unomi-persistence-elasticsearch-plugins
[03/14] incubator-unomi git commit: DMF-1343 : Move unomi-router under extensions
http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/9cffa13e/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileImportSourceToKafkaRouteBuilder.java -- diff --git a/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileImportSourceToKafkaRouteBuilder.java b/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileImportSourceToKafkaRouteBuilder.java deleted file mode 100644 index 382b69d..000 --- a/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileImportSourceToKafkaRouteBuilder.java +++ /dev/null @@ -1,119 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.unomi.router.core.route; - -import org.apache.camel.builder.RouteBuilder; -import org.apache.camel.component.jackson.JacksonDataFormat; -import org.apache.camel.component.kafka.KafkaComponent; -import org.apache.camel.component.kafka.KafkaConfiguration; -import org.apache.camel.component.kafka.KafkaEndpoint; -import org.apache.commons.lang3.StringUtils; -import org.apache.unomi.router.api.ImportConfiguration; -import org.apache.unomi.router.core.processor.LineSplitProcessor; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.util.List; -import java.util.Map; - -/** - * Created by amidani on 26/04/2017. - */ - -public class ProfileImportSourceToKafkaRouteBuilder extends RouteBuilder { - -private static final Logger logger = LoggerFactory.getLogger(ProfileImportSourceToKafkaRouteBuilder.class.getName()); - -private List importConfigurationList; -private JacksonDataFormat jacksonDataFormat; -private String kafkaHost; -private String kafkaPort; -private String kafkaImportTopic; -private String kafkaImportGroupId; - -public ProfileImportSourceToKafkaRouteBuilder(Map kafkaProps) { -kafkaHost = kafkaProps.get("kafkaHost"); -kafkaPort = kafkaProps.get("kafkaPort"); -kafkaImportTopic = kafkaProps.get("kafkaImportTopic"); -kafkaImportGroupId = kafkaProps.get("kafkaImportGroupId"); -} - -@Override -public void configure() throws Exception { -//Prepare Kafka Deposit -StringBuilder kafkaUri = new StringBuilder("kafka:"); - kafkaUri.append(kafkaHost).append(":").append(kafkaPort).append("?topic=").append(kafkaImportTopic); -if(StringUtils.isNotBlank(kafkaImportGroupId)) { -kafkaUri.append("&groupId="+ kafkaImportGroupId); -} - -KafkaConfiguration kafkaConfiguration = new KafkaConfiguration(); -kafkaConfiguration.setBrokers(kafkaHost+":"+kafkaPort); -kafkaConfiguration.setTopic(kafkaImportTopic); -kafkaConfiguration.setGroupId(kafkaImportGroupId); -KafkaEndpoint endpoint = new KafkaEndpoint(kafkaUri.toString(), new KafkaComponent(this.getContext())); -endpoint.setConfiguration(kafkaConfiguration); - -//Loop on multiple import configuration -for(ImportConfiguration importConfiguration : importConfigurationList) { -if(importConfiguration.getProperties().size() > 0 && -StringUtils.isNotEmpty((String) importConfiguration.getProperties().get("source"))) { -//Prepare Split Processor -LineSplitProcessor lineSplitProcessor = new LineSplitProcessor(); -lineSplitProcessor.setFieldsMapping((Map) importConfiguration.getProperties().get("mapping")); - lineSplitProcessor.setOverwriteExistingProfiles(importConfiguration.isOverwriteExistingProfiles()); - lineSplitProcessor.setPropertiesToOverwrite(importConfiguration.getPropertiesToOverwrite()); - lineSplitProcessor.setMergingProperty(importConfiguration.getMergingProperty()); - -from((String) importConfiguration.getProperties().get("source")) -.routeId(importConfiguration.getItemId())// This allow identification of the route for manual start/stop -.autoStartup(importConfiguration.isActive())// Auto-start if the import configuration is set active -.spl
[05/14] incubator-unomi git commit: DMF-1343 : Move unomi-router under extensions
DMF-1343 : Move unomi-router under extensions Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/9cffa13e Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/9cffa13e Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/9cffa13e Branch: refs/heads/feature-DMF-1343 Commit: 9cffa13e80375b3bfbee14bdbfd0be291f75d0fa Parents: f65173c Author: Abdelkader Midani Authored: Wed May 24 15:19:55 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 19:24:29 2017 +0200 -- extensions/pom.xml | 1 + extensions/router/pom.xml | 64 +++ extensions/router/router-api/pom.xml| 43 + .../unomi/router/api/ImportConfiguration.java | 185 +++ .../unomi/router/api/ProfileToImport.java | 77 .../services/ImportConfigurationService.java| 60 ++ .../api/services/ProfileImportService.java | 29 +++ extensions/router/router-core/pom.xml | 182 ++ .../core/context/ProfileImportCamelContext.java | 165 + .../core/processor/ConfigUpdateProcessor.java | 44 + .../ImportConfigByFileNameProcessor.java| 44 + .../core/processor/LineSplitProcessor.java | 104 +++ .../core/processor/UnomiStorageProcessor.java | 46 + .../ProfileImportConfigUpdateRouteBuilder.java | 62 +++ .../ProfileImportKafkaToUnomiRouteBuilder.java | 77 .../route/ProfileImportOneShotRouteBuilder.java | 99 ++ .../ProfileImportSourceToKafkaRouteBuilder.java | 119 .../resources/OSGI-INF/blueprint/blueprint.xml | 94 ++ .../main/resources/org.apache.unomi.router.cfg | 23 +++ extensions/router/router-karaf-feature/pom.xml | 157 extensions/router/router-rest/pom.xml | 75 .../ImportConfigurationServiceEndPoint.java | 167 + .../resources/OSGI-INF/blueprint/blueprint.xml | 72 extensions/router/router-service/pom.xml| 104 +++ .../ImportConfigurationServiceImpl.java | 114 .../services/ProfileImportServiceImpl.java | 122 .../resources/OSGI-INF/blueprint/blueprint.xml | 39 pom.xml | 1 - router/pom.xml | 64 --- router/router-api/pom.xml | 43 - .../unomi/router/api/ImportConfiguration.java | 185 --- .../unomi/router/api/ProfileToImport.java | 77 .../services/ImportConfigurationService.java| 60 -- .../api/services/ProfileImportService.java | 29 --- router/router-core/pom.xml | 182 -- .../core/context/ProfileImportCamelContext.java | 165 - .../core/processor/ConfigUpdateProcessor.java | 44 - .../ImportConfigByFileNameProcessor.java| 44 - .../core/processor/LineSplitProcessor.java | 104 --- .../core/processor/UnomiStorageProcessor.java | 46 - .../ProfileImportConfigUpdateRouteBuilder.java | 62 --- .../ProfileImportKafkaToUnomiRouteBuilder.java | 77 .../route/ProfileImportOneShotRouteBuilder.java | 99 -- .../ProfileImportSourceToKafkaRouteBuilder.java | 119 .../resources/OSGI-INF/blueprint/blueprint.xml | 94 -- .../main/resources/org.apache.unomi.router.cfg | 23 --- router/router-karaf-feature/pom.xml | 157 router/router-rest/pom.xml | 75 .../ImportConfigurationServiceEndPoint.java | 167 - .../resources/OSGI-INF/blueprint/blueprint.xml | 72 router/router-service/pom.xml | 104 --- .../ImportConfigurationServiceImpl.java | 114 .../services/ProfileImportServiceImpl.java | 122 .../resources/OSGI-INF/blueprint/blueprint.xml | 39 54 files changed, 2368 insertions(+), 2368 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/9cffa13e/extensions/pom.xml -- diff --git a/extensions/pom.xml b/extensions/pom.xml index 6aa2fd8..8a874a6 100644 --- a/extensions/pom.xml +++ b/extensions/pom.xml @@ -34,6 +34,7 @@ lists-extension privacy-extension geonames +router http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/9cffa13e/extensions/router/pom.xml -- diff --git a/extensions/router/pom.xml b/extensions/router/pom.xml new file mode 100644 index 000..c69320c ---
[04/14] incubator-unomi git commit: DMF-1343 : Move unomi-router under extensions
http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/9cffa13e/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java -- diff --git a/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java b/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java new file mode 100644 index 000..e81930a --- /dev/null +++ b/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.unomi.router.rest; + +import com.fasterxml.jackson.databind.ObjectMapper; +import org.apache.cxf.jaxrs.ext.multipart.Multipart; +import org.apache.cxf.jaxrs.ext.multipart.Attachment; +import org.apache.cxf.rs.security.cors.CrossOriginResourceSharing; +import org.apache.http.HttpResponse; +import org.apache.http.client.methods.HttpPut; +import org.apache.http.entity.StringEntity; +import org.apache.http.impl.client.CloseableHttpClient; +import org.apache.http.impl.client.HttpClients; +import org.apache.unomi.router.api.ImportConfiguration; +import org.apache.unomi.router.api.services.ImportConfigurationService; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import javax.jws.WebMethod; +import javax.jws.WebService; +import javax.ws.rs.*; +import javax.ws.rs.core.MediaType; +import javax.ws.rs.core.Response; +import java.io.IOException; +import java.io.InputStream; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.List; + +/** + * A JAX-RS endpoint to manage {@link org.apache.unomi.router.api.ImportConfiguration}s. + */ +@WebService +@CrossOriginResourceSharing( +allowAllOrigins = true, +allowCredentials = true +) +public class ImportConfigurationServiceEndPoint { + +private static final Logger logger = LoggerFactory.getLogger(ImportConfigurationServiceEndPoint.class.getName()); + +private ImportConfigurationService importConfigurationService; +private String uploadDir; + +public ImportConfigurationServiceEndPoint () { +logger.info("Initializing import configuration service endpoint..."); +} + +@WebMethod(exclude = true) +public void setImportConfigurationService(ImportConfigurationService importConfigurationService) { +this.importConfigurationService = importConfigurationService; +} + +@WebMethod(exclude = true) +public void setUploadDir(String uploadDir) { +this.uploadDir = uploadDir; +} + +/** + * Retrieves all the import configurations. + * + * @return all the import configurations. + */ +@GET +@Path("/") +@Produces(MediaType.APPLICATION_JSON) +@Consumes(MediaType.APPLICATION_JSON) +public List getImportConfigurations() { +return importConfigurationService.getImportConfigurations(); +} + +/** + * Retrieves an import configuration by id. + * + * @return the import configuration that matches the given id. + */ +@GET +@Path("/{configId}") +@Produces(MediaType.APPLICATION_JSON) +@Consumes(MediaType.APPLICATION_JSON) +public ImportConfiguration getImportConfiguration(@PathParam("configId") String configId) { +return importConfigurationService.load(configId); +} + +/** + * Delete an import configuration by id. + * + * @return the deleted import configuration. + */ +@DELETE +@Path("/{configId}") +@Produces(MediaType.APPLICATION_JSON) +@Consumes(MediaType.APPLICATION_JSON) +public void deleteImportConfiguration(@PathParam("configId") String configId) { +importConfigurationService.delete(configId); +} + + + +/** + * Save the given import configuration. + * + * @return the import configuration saved. + */ +@POST +@Path("/") +@Produces(MediaType.APPLICATION_JSON) +@Consumes(MediaType.APPLICATION_JSON) +public ImportConfiguration saveImportConfiguration(ImportConfiguration importConfiguration)
[10/14] incubator-unomi git commit: DMF-1343 : Improvement and documentation
DMF-1343 : Improvement and documentation Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/77fc9e44 Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/77fc9e44 Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/77fc9e44 Branch: refs/heads/feature-DMF-1343 Commit: 77fc9e44deb09f470cfad2ac791facd85d50cf50 Parents: 9cffa13 Author: Abdelkader Midani Authored: Wed May 24 17:26:14 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 19:24:29 2017 +0200 -- extensions/router/README.md | 113 +++ .../ProfileImportConfigUpdateRouteBuilder.java | 2 +- .../resources/OSGI-INF/blueprint/blueprint.xml | 6 +- .../main/resources/org.apache.unomi.router.cfg | 4 +- .../ImportConfigurationServiceEndPoint.java | 37 +++--- .../resources/OSGI-INF/blueprint/blueprint.xml | 14 ++- 6 files changed, 154 insertions(+), 22 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/77fc9e44/extensions/router/README.md -- diff --git a/extensions/router/README.md b/extensions/router/README.md new file mode 100644 index 000..8fa1994 --- /dev/null +++ b/extensions/router/README.md @@ -0,0 +1,113 @@ + + +Unomi Router +== + +## Getting started +Unomi Router Extension a Karaf Feature that provide an Enterprise Application Integration tool. +It is optional so you must configure it and install it in Karaf, and can be used for Machine - Machine or Human - Machine integration with Unomi. +Mainly Unomi Router Extension aim to make it easy to import third party applications/platforms profiles into Unomi. +This extension is implemented using Apache Camel routes and is using Apache Kafka to buffer import process and make it failsafe. + +## Getting started +1. Configure your Unomi Router: +In the `etc/org.apache.unomi.sfdc.cfg` file, you might want to update the following settings: +Kafka settings +>`#Kafka settings` + +>`kafka.host=localhost` + +>`kafka.port=9092` + +>`kafka.import.topic=camel-deposit` + +>`kafka.import.groupId=unomi-import-group` + +Kafka host and port with the topic name and the groupId ti which the topic is assigned + +>`#Import One Shot upload directory` + +>`import.oneshot.uploadDir=/tmp/unomi_oneshot_import_configs/` + +Path to the folder where unomi should stock file imported for a oneshot processing + + +2. Deploy into Apache Unomi using the following commands from the Apache Karaf shell: +```sh +$ feature:repo-add mvn:org.apache.unomi/unomi-router-karaf-feature/${version}/xml/features +$ feature:install unomi-router-karaf-feature +``` + +3. Send your import configuration: + +An import configuration is nothing else than a simple JSON to describe how you want to import your data (Profiles). +To create/update an import configuration + +`POST /cxs/importConfiguration` +```json + { + "itemId": "f57f1f86-97bf-4ba0-b4e4-7d5e77e7c0bd", + "itemType": "importConfig", + "scope": "integration", + "name": "Test Recurrent", + "description": "Just test recurrent import", + "configType": "recurrent", + "properties": { + "source": "{file/ftp}://{path}?fileName={file-name}.csv&move=.done&consumer.delay=2", + "mapping": { + "firstName": 0, + "lastName": 1, + ... + } + }, + "mergingProperty": "email", + "overwriteExistingProfiles": true, + "propertiesToOverwrite": ["firstName", "lastName"], + "active": true + } +``` + +Omit the `itemId` when creating new entry, `configType` can be '**recurrent**' for file/ftp/network path polling or '**oneshot**' for one time import. + +The `properties.source` attribute is an Apache Camel endpoint uri (See http://camel.apache.org/uris.html for more details). Unomi Router is designed to use **File** and **FTP** Camel components. + +The attribute `properties.mapping` is a Map of: +* Key: Profile property id in Unomi +* Value: Index of the column in the imported file to copy the in the previous property. + +The attribute `mergingProperty` is the profile property id in Unomi to use to check for duplication. + +The attribute `propertiesToOverwrite` is a list of profile properties ids to overwrite, if **null** all properties +will be overwritten. + +The attribute `active` is the flag to activate or deactivate the import configuration. + +Concerning oneshot import configuration using the previously
[01/14] incubator-unomi git commit: UNOMI-100 fix issue with isDay and isNotDay date wasn't converted as ISO, add unit test [Forced Update!]
Repository: incubator-unomi Updated Branches: refs/heads/feature-DMF-1343 1637a9122 -> 63debd5fc (forced update) UNOMI-100 fix issue with isDay and isNotDay date wasn't converted as ISO, add unit test Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/0f441403 Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/0f441403 Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/0f441403 Branch: refs/heads/feature-DMF-1343 Commit: 0f4414039274750a3fe2b62fdcbecbfda73eaed4 Parents: 46be002 Author: dgaillard Authored: Wed Jun 7 18:51:09 2017 +0200 Committer: dgaillard Committed: Wed Jun 7 18:51:09 2017 +0200 -- .../test/java/org/apache/unomi/itests/ConditionBuilder.java | 8 .../java/org/apache/unomi/itests/ConditionEvaluatorIT.java | 3 +++ .../conditions/PropertyConditionESQueryBuilder.java | 2 +- 3 files changed, 12 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/0f441403/itests/src/test/java/org/apache/unomi/itests/ConditionBuilder.java -- diff --git a/itests/src/test/java/org/apache/unomi/itests/ConditionBuilder.java b/itests/src/test/java/org/apache/unomi/itests/ConditionBuilder.java index 0fbffcd..38356b5 100644 --- a/itests/src/test/java/org/apache/unomi/itests/ConditionBuilder.java +++ b/itests/src/test/java/org/apache/unomi/itests/ConditionBuilder.java @@ -129,6 +129,14 @@ public class ConditionBuilder { return op("in").dateValues(values); } +public ComparisonCondition isDay(Date value) { +return op("isDay").dateValue(value); +} + +public ComparisonCondition isNotDay(Date value) { +return op("isNotDay").dateValue(value); +} + public ComparisonCondition in(Integer... values) { return op("in").integerValues(values); } http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/0f441403/itests/src/test/java/org/apache/unomi/itests/ConditionEvaluatorIT.java -- diff --git a/itests/src/test/java/org/apache/unomi/itests/ConditionEvaluatorIT.java b/itests/src/test/java/org/apache/unomi/itests/ConditionEvaluatorIT.java index b75a8f5..5fb6b6f 100644 --- a/itests/src/test/java/org/apache/unomi/itests/ConditionEvaluatorIT.java +++ b/itests/src/test/java/org/apache/unomi/itests/ConditionEvaluatorIT.java @@ -112,6 +112,9 @@ public class ConditionEvaluatorIT extends BaseIT { assertTrue(eval(builder.profileProperty("properties.lastVisit").all(lastVisit).build())); assertFalse(eval(builder.profileProperty("properties.lastVisit") .all(new Date(lastVisit.getTime() + 1), lastVisit).build())); + + assertTrue(eval(builder.profileProperty("properties.lastVisit").isDay(lastVisit).build())); + assertTrue(eval(builder.profileProperty("properties.lastVisit").isNotDay(new Date(lastVisit.getTime() + 1)).build())); } @Test http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/0f441403/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/conditions/PropertyConditionESQueryBuilder.java -- diff --git a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/conditions/PropertyConditionESQueryBuilder.java b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/conditions/PropertyConditionESQueryBuilder.java index 26e480a..b06c94e 100644 --- a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/conditions/PropertyConditionESQueryBuilder.java +++ b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/conditions/PropertyConditionESQueryBuilder.java @@ -166,7 +166,7 @@ public class PropertyConditionESQueryBuilder implements ConditionESQueryBuilder DateTime date = new DateTime(value); DateTime dayStart = date.withTimeAtStartOfDay(); DateTime dayAfterStart = date.plusDays(1).withTimeAtStartOfDay(); -return QueryBuilders.rangeQuery(name).gte(dayStart.toDate()).lte(dayAfterStart.toDate()); +return QueryBuilders.rangeQuery(name).gte(convertDateToISO(dayStart.toDate())).lte(convertDateToISO(dayAfterStart.toDate())); } private Object convertDateToISO(Object dateValue) {
[11/14] incubator-unomi git commit: DMF-1343 Import profiles from CSV through API
DMF-1343 Import profiles from CSV through API Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/f65173c9 Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/f65173c9 Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/f65173c9 Branch: refs/heads/feature-DMF-1343 Commit: f65173c96cb671fcd7dcbeeea35f148dec7f249e Parents: b68909b Author: Abdelkader Midani Authored: Wed May 24 08:36:19 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 19:24:29 2017 +0200 -- package/pom.xml | 11 + ...g.apache.unomi.persistence.elasticsearch.cfg | 2 +- router/pom.xml | 2 +- router/router-api/pom.xml | 2 +- .../unomi/router/api/ImportConfiguration.java | 47 router/router-core/pom.xml | 2 +- .../core/context/ProfileImportCamelContext.java | 4 +- .../ProfileImportConfigUpdateRouteBuilder.java | 5 --- router/router-karaf-feature/pom.xml | 9 +--- router/router-rest/pom.xml | 2 +- router/router-service/pom.xml | 2 +- 11 files changed, 67 insertions(+), 21 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/f65173c9/package/pom.xml -- diff --git a/package/pom.xml b/package/pom.xml index ae7c17e..2e7e479 100644 --- a/package/pom.xml +++ b/package/pom.xml @@ -269,6 +269,17 @@ org.apache.unomi.plugins.mail.cfg + +org.apache.unomi + unomi-router-core + ${project.version} +routercfg +cfg + + ${project.build.directory}/assembly/etc + + org.apache.unomi.router.cfg + org.apache.aries.blueprint http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/f65173c9/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg -- diff --git a/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg b/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg index 77eb091..12980e1 100644 --- a/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg +++ b/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg @@ -15,7 +15,7 @@ # limitations under the License. # -cluster.name=contextElasticSearch +cluster.name=contextElasticSearch_amidani # The elasticSearchAddresses may be a comma seperated list of host names and ports such as # hostA:9300,hostB:9300 # Note: the port number must be repeated for each host. http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/f65173c9/router/pom.xml -- diff --git a/router/pom.xml b/router/pom.xml index 73e9953..5bf43ea 100644 --- a/router/pom.xml +++ b/router/pom.xml @@ -22,7 +22,7 @@ unomi-root org.apache.unomi -1.2.0-incubating-SNAPSHOT +1.2.0-incubating-dmf_1343-SNAPSHOT unomi-router http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/f65173c9/router/router-api/pom.xml -- diff --git a/router/router-api/pom.xml b/router/router-api/pom.xml index 9085a30..06207b3 100644 --- a/router/router-api/pom.xml +++ b/router/router-api/pom.xml @@ -21,7 +21,7 @@ unomi-router org.apache.unomi -1.2.0-incubating-SNAPSHOT +1.2.0-incubating-dmf_1343-SNAPSHOT 4.0.0 http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/f65173c9/router/router-api/src/main/java/org/apache/unomi/router/api/ImportConfiguration.java -- diff --git a/router/router-api/src/main/java/org/apache/unomi/router/api/ImportConfiguration.java b/router/router-api/src/main/jav
[13/14] incubator-unomi git commit: DMF-1556 : Csv without First name, and last name will show null, null in log file
DMF-1556 : Csv without First name, and last name will show null, null in log file Project: http://git-wip-us.apache.org/repos/asf/incubator-unomi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-unomi/commit/63debd5f Tree: http://git-wip-us.apache.org/repos/asf/incubator-unomi/tree/63debd5f Diff: http://git-wip-us.apache.org/repos/asf/incubator-unomi/diff/63debd5f Branch: refs/heads/feature-DMF-1343 Commit: 63debd5fcda21fda77aa20dc353267800bdcab74 Parents: d95595f Author: Abdelkader Midani Authored: Tue Jun 6 17:58:31 2017 +0200 Committer: Abdelkader Midani Committed: Mon Jun 12 19:24:29 2017 +0200 -- .../router/services/ProfileImportServiceImpl.java| 15 +++ 1 file changed, 7 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-unomi/blob/63debd5f/extensions/router/router-service/src/main/java/org/apache/unomi/router/services/ProfileImportServiceImpl.java -- diff --git a/extensions/router/router-service/src/main/java/org/apache/unomi/router/services/ProfileImportServiceImpl.java b/extensions/router/router-service/src/main/java/org/apache/unomi/router/services/ProfileImportServiceImpl.java index 6dcc152..cb1d706 100644 --- a/extensions/router/router-service/src/main/java/org/apache/unomi/router/services/ProfileImportServiceImpl.java +++ b/extensions/router/router-service/src/main/java/org/apache/unomi/router/services/ProfileImportServiceImpl.java @@ -80,38 +80,37 @@ public class ProfileImportServiceImpl implements ProfileImportService, Synchrono public boolean saveMergeDeleteImportedProfile(ProfileToImport profileToImport) throws InvocationTargetException, IllegalAccessException { -logger.info("Importing profile: {}, {}", profileToImport.getProperties().get("firstName"), profileToImport.getProperties().get("lastName")); +logger.debug("Importing profile with ID : {}", profileToImport.getItemId()); Profile existingProfile = new Profile(); List existingProfiles = persistenceService.query("properties."+profileToImport.getMergingProperty(), (String)profileToImport.getProperties().get(profileToImport.getMergingProperty()), null, Profile.class); -logger.info("Query existing profile with mergingProperty: {}", profileToImport.getMergingProperty()); -logger.info("Found: {}", existingProfiles.size()); +logger.debug("Query existing profile with mergingProperty: {}. Found: {}", profileToImport.getMergingProperty(), existingProfiles.size()); //Profile already exist, and import config allow to overwrite profiles if(existingProfiles.size() == 1) { existingProfile = existingProfiles.get(0); if(profileToImport.isProfileToDelete()) { -logger.info("Profile is to delete!"); +logger.debug("Profile is to delete!"); persistenceService.remove(existingProfile.getItemId(), Profile.class); return true; } List propertiesToOverwrite = profileToImport.getPropertiesToOverwrite(); if(profileToImport.isOverwriteExistingProfiles() && propertiesToOverwrite!=null && propertiesToOverwrite.size() > 0) { // We overwrite only properties marked to overwrite -logger.info("Properties to overwrite: {}", propertiesToOverwrite); +logger.debug("Properties to overwrite: {}", propertiesToOverwrite); for(String propName : propertiesToOverwrite) { existingProfile.getProperties().put(propName, profileToImport.getProperties().get(propName)); } } else { //If no property is marked to overwrite we replace the whole properties map -logger.info("Overwrite all properties"); +logger.debug("Overwrite all properties"); existingProfile.setProperties(profileToImport.getProperties()); } } else if(existingProfiles.size() == 0) { -logger.info("New profile to add..."); +logger.debug("New profile to add..."); BeanUtils.copyProperties(existingProfile, profileToImport); } else { logger.warn("{} occurences found for profile with {} = {}. Profile import is skipped", existingProfiles.size(), profileToImport.getMergingProperty(), profileToImport.getProperties().get(profileToImport.getMergingProperty())); } -logger.info("-"); +logger.debug("-"); return persistenceService.save(existingProfile); }