On Thu, Aug 25, 2016 at 11:33 AM, Malintha Amarasinghe <malint...@wso2.com> wrote:
> Hi, > > On Wed, Aug 24, 2016 at 11:02 AM, Mohanadarshan Vivekanandalingam < > mo...@wso2.com> wrote: > >> HI Malintha/Nuwan, >> >> I have few queries and like to provide few clarifications on this.. >> >> Actually most of the points that you guys raised are discussed before and >> changed based on some experiences.. >> >> - Defining stream from client side : We had this in old datapublisher >> component and removed due to various complications.. As Srinath mentioned >> this was done after many rounds of discussions.. >> >> - Having single configuration file to add DAS endpoint : In past, we had >> two level of configs (one for adapter which deals with transport and one >> for event formatting which deals with stream). That means we can use one >> event adapter for multiple streams.. But, there were some complaints saying >> it is not user friendly and should have one config to publish events.. >> >> And also, event publisher is also another deployable artifact like other >> synapse files.. In APIM, we are configuring key manager url in multiple api >> configs (revoke api and etc.. ), event publisher is also similar to that.. >> >> But, I do understand that your points are valid and think we have two >> options.. >> >> - Have a global property in output-adapter-config.xml and use that in >> Thrift event publisher (if that property exists).. >> > @All, I have incorporated above fix by introducing a global default property in master branch Thanks, Mohan > > - Allow to keep the DAS endpoint as a registry resource and use that.. >> >> Thanks for the response. It would be great to have above in order to > mitigate the duplicated configuration issues. > > Thanks! > Malintha > >> Thanks, >> Mohan >> >> >> On Wed, Aug 24, 2016 at 10:05 AM, Nuwan Dias <nuw...@wso2.com> wrote: >> >>> Hi, >>> >>> I spoke with Srinath regarding this too. It would be ideal to have a >>> global config which can override the per-stream configs so that we won't >>> have to maintain the DAS configs in multiple places. In all cases we've >>> come across so far we've only seen people use a single DAS (or cluster) and >>> publish all streams to that DAS cluster. So at least as of now I don't see >>> much value in having per stream config files. >>> >>> It would also be good if we can omit the necessity of having to define >>> the streams on the client side (because they're already defined on the >>> server). I see two main problems of having to define them on the clients. >>> >>> 1. When we have multiple event generators (a gateway cluster), the >>> streams will have to be defined on all nodes. Do we use dep-sync to sync up >>> the configs on the clients? Even if we do, I think its better to not depend >>> on dep-sync for the new stuff we're developing because we're moving away >>> from dep-sync soon. >>> >>> 2. Problems in upgrading to newer product versions. When upgrading the >>> product version, if a schema change has occurred, currently we only have to >>> worry about changing the schema on the server side. With the above >>> limitation, we now have to worry about changing the clients too. >>> >>> Thanks, >>> NuwanD. >>> >>> On Tue, Aug 23, 2016 at 10:16 PM, Malintha Amarasinghe < >>> malint...@wso2.com> wrote: >>> >>>> Hi All, >>>> >>>> Currently APIM is using an internal APIM specific configuration resides >>>> in api-manager.xml which include DAS server URL, username, password etc. >>>> Those configuration are used to instantiate an org.wso2.carbon.databridge. >>>> agent.DataPublisher object, which is then used to publish events >>>> directly to streams in DAS. >>>> >>>> As it is an APIM specific configuration, it is not reusable by other >>>> non-APIM features. Also org.wso2.carbon.databridge.agent.DataPublisher is >>>> intended to be used in non-carbon environments, for carbon environments, >>>> the recommended way is to use the Event Stream OSGi Service. >>>> >>>> So we were trying to re-structure APIM data publishing code to use new >>>> Event Stream OSGi service explained in [1]. >>>> >>>> 1. Create all the streams defined in DAS in APIM. >>>> 2. Create an *event publisher per each stream *which takes data from >>>> the stream and publish to DAS. >>>> 3. In each event publisher we need to configure DAS specific >>>> configuration. >>>> >>>> Ex: >>>> <?xml version="1.0" encoding="UTF-8"?> >>>> <eventPublisher name="test" statistics="disable" trace="disable" xmlns=" >>>> http://wso2.org/carbon/eventpublisher";> >>>> <from streamName="streamTest" version="1.0.0"/> >>>> <mapping customMapping="disable" type="wso2event"/> >>>> <to eventAdapterType="wso2event"> >>>> <property name="username">admin</property> >>>> <property name="protocol">thrift</property> >>>> <property name="publishingMode">non-blocking</property> >>>> <property name="publishTimeout">0</property> >>>> <property name="receiverURL">tcp://localhost:7611</property> >>>> <property encrypted="true" name="password">XXXXXXXX</property> >>>> </to> >>>> </eventPublisher> >>>> >>>> But we are facing a small problem here; in APIM there are many event >>>> streams being used. If there's a change happen to DAS configuration, we >>>> need to change it in all the event publishers which makes it difficult to >>>> maintain. As per the offline chat with DAS team, this is a current >>>> limitation. >>>> >>>> Are we going to move forward with the existing implementation? >>>> >>>> Thanks, >>>> Malintha >>>> >>>> [1] [Dev] Common configuration for publishing events from carbon >>>> servers to DAS/CEP >>>> -- >>>> Malintha Amarasinghe >>>> Software Engineer >>>> *WSO2, Inc. - lean | enterprise | middleware* >>>> http://wso2.com/ >>>> >>>> Mobile : +94 712383306 >>>> >>> >>> >>> >>> -- >>> Nuwan Dias >>> >>> Software Architect - WSO2, Inc. http://wso2.com >>> email : nuw...@wso2.com >>> Phone : +94 777 775 729 >>> >>> _______________________________________________ >>> Dev mailing list >>> Dev@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >>> >> >> >> -- >> *V. Mohanadarshan* >> *Associate Tech Lead,* >> *Data Technologies Team,* >> *WSO2, Inc. http://wso2.com <http://wso2.com> * >> *lean.enterprise.middleware.* >> >> email: mo...@wso2.com >> phone:(+94) 771117673 >> > > > > -- > Malintha Amarasinghe > Software Engineer > *WSO2, Inc. - lean | enterprise | middleware* > http://wso2.com/ > > Mobile : +94 712383306 > -- *V. Mohanadarshan* *Associate Tech Lead,* *Data Technologies Team,* *WSO2, Inc. http://wso2.com <http://wso2.com> * *lean.enterprise.middleware.* email: mo...@wso2.com phone:(+94) 771117673
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
