Well described Srini. A clarification - the Kafka broker is installed as part of ONAP correct?
Since DMaaP is a wrapper over Kafka, I am more inclined towards the datalake connectivity enablement being part of DMaaP project. Thanks, Ramki From: onap-tsc@lists.onap.org <onap-tsc@lists.onap.org> On Behalf Of Srini Sent: Saturday, January 19, 2019 1:11 PM To: 'onap-tsc@lists.onap.org' <onap-tsc@lists.onap.org> Subject: Re: [onap-tsc] Data Lake Just to add some more information: * Apache Kafka added Authorization and ACLs functionality (http://kafka.apache.org/documentation/#securoty_authz<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fkafka.apache.org%2Fdocumentation%2F%23securoty_authz&data=02%7C01%7Cramkik%40vmware.com%7C0be423483c4f46a4e33208d67e52b18b%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636835291004260562&sdata=CcCqJEHrx8G1OfRG%2Bz6Fskos0POvRF3RZ4KaTU%2FTD2c%3D&reserved=0>) which allows providing configurable access control to external entities that need access to Kafka topics.. * Kafka Operator (https://github.com/strimzi/strimzi-kafka-operator<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fstrimzi%2Fstrimzi-kafka-operator&data=02%7C01%7Cramkik%40vmware.com%7C0be423483c4f46a4e33208d67e52b18b%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636835291004260562&sdata=asGgFuZAnpqltDsjTuATgnGvrRkAbdJMXj59FSs50vQ%3D&reserved=0>) made it simple to create ACLs using Kubernetes resources. Both of above technologies/features can be added. >From the workflow perspective, there are two operations by the ONAP user: * Register external Kafka entity (In this case, external data like is a kafka client) * Unregister the entity. It is similar to the way cloud-regions are registered via ESR. Records can be added to A&AI DB. One can provide GUI/CLI to make it simple for ONAP-users to add these in A&AI. As part of registration, one can add users or certificate names and topic they are allowed to subscribe to. When external entity registered in A&AI, it can trigger the kafka operator to add the ACL records. With this, each data lake can be brought up outside of ONAP and register that with A&AI by ONAP-User. Once that happens, data lake can get hold of any messages produced on the topic and can store in its databases and provide any analytics. Thanks Srini From: Addepalli, Srinivasa R Sent: Thursday, January 17, 2019 2:12 PM To: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> Subject: RE: Data Lake My view is that data lakes (this one or any other data lake) can be considered as external entities (can be brought up ONAP like any other VNFs or via some other mechanisms). That includes feeder and UI. >From ONAP perspective, we need to ensure that Kafka bus is accessible by third >party systems, that is, authenticated registration of third party systems and >RBAC on topics etc... Since this datalake is specific to ONAP monitoring, I guess it is required that all modules send all its telemetry data and transaction data to Kafka. In my view, ideal thing is to do this integration from ISTIO side car. Thanks Srini From: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> [mailto:onap-tsc@lists.onap.org] On Behalf Of Stephen Terrill Sent: Thursday, January 17, 2019 1:19 PM To: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> Subject: [onap-tsc] Data Lake Hi All, During the TSC discussion today on the data lake; the discussion was leading towards considering the data lake feeder as part of DCAE. Upon further reflection, I was wondering whether we should also consider including it as part of DMAAP as an alternative option. Just for consideration. BR, Steve [http://www.ericsson.com]<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ericsson.com%2F&data=02%7C01%7Cramkik%40vmware.com%7C0be423483c4f46a4e33208d67e52b18b%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636835291004260562&sdata=wepsxRxsQ31CVOf1Nozkd4PG1yiVn1A%2B9ycnFYty1WA%3D&reserved=0> Stephen Terrill Senior Expert, Automation and Management TECHNOLOGY SPECIALIST BDGS RDP Architecture & Technology Phone: +34913393005 Mobile: +34609168515 stephen.terr...@ericsson.com<mailto:stephen.terr...@ericsson.com> Ericsson C/ Via de los Poblados 13. B 28033,Madrid, Madrid Spain ericsson.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ericsson.com%2F&data=02%7C01%7Cramkik%40vmware.com%7C0be423483c4f46a4e33208d67e52b18b%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636835291004260562&sdata=wepsxRxsQ31CVOf1Nozkd4PG1yiVn1A%2B9ycnFYty1WA%3D&reserved=0> [http://www.ericsson.com/current_campaign]<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ericsson.com%2Fcurrent_campaign&data=02%7C01%7Cramkik%40vmware.com%7C0be423483c4f46a4e33208d67e52b18b%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636835291004260562&sdata=ZUNHI1JMl39HqSTwHVswyS6mdTexNhs3T6UKG%2B%2BUkaU%3D&reserved=0> Our commitment to Technology for Good<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ericsson.com%2Fthecompany%2Fsustainability-corporateresponsibility&data=02%7C01%7Cramkik%40vmware.com%7C0be423483c4f46a4e33208d67e52b18b%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636835291004260562&sdata=%2Balc2gTkEfpmxMogi8S0flEmlFyfPCuuS4LL4B838UI%3D&reserved=0> and Diversity and Inclusion<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ericsson.com%2Fthecompany%2Fdiversity-inclusion&data=02%7C01%7Cramkik%40vmware.com%7C0be423483c4f46a4e33208d67e52b18b%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636835291004260562&sdata=m1PMx8T%2FE%2BqB5JFx9QWUGyCc4zbGhTpOr81rHvea1m0%3D&reserved=0> contributes to positive change. Follow us on: Facebook<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.facebook.com%2Fericsson&data=02%7C01%7Cramkik%40vmware.com%7C0be423483c4f46a4e33208d67e52b18b%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636835291004260562&sdata=59Ygda3ATQ3CJOI9BOUJCeNDoERf%2BVFjiV05scLlHwc%3D&reserved=0> LinkedIn<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fericsson&data=02%7C01%7Cramkik%40vmware.com%7C0be423483c4f46a4e33208d67e52b18b%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636835291004260562&sdata=viiU4ApSDrLjOfYPIJB4ghm%2FRK9fEoj3sjHdGSgGeJg%3D&reserved=0> Twitter<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2FEricsson&data=02%7C01%7Cramkik%40vmware.com%7C0be423483c4f46a4e33208d67e52b18b%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636835291004260562&sdata=pdtc%2FGQHTW%2BsDV5AN%2BtOF6gDqxhTfNDfZiF9K3bXb%2Bw%3D&reserved=0> Legal entity:ERICSSON AB registration number 556056-6258, registered office in Stockholm. This communication is confidential. Our email terms: www.ericsson.com/en/legal/privacy/email-disclaimer<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ericsson.com%2Fen%2Flegal%2Fprivacy%2Femail-disclaimer&data=02%7C01%7Cramkik%40vmware.com%7C0be423483c4f46a4e33208d67e52b18b%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C636835291004260562&sdata=ELfd%2Bee%2B2LCb7Cu%2F1qdGgAUQcom8XlihwzBTROycY3k%3D&reserved=0> -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#4493): https://lists.onap.org/g/onap-tsc/message/4493 Mute This Topic: https://lists.onap.org/mt/29189968/21656 Group Owner: onap-tsc+ow...@lists.onap.org Unsubscribe: https://lists.onap.org/g/onap-tsc/leave/2743226/1412191262/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
