Hi, I spoke with Srinath regarding this too. It would be ideal to have a global config which can override the per-stream configs so that we won't have to maintain the DAS configs in multiple places. In all cases we've come across so far we've only seen people use a single DAS (or cluster) and publish all streams to that DAS cluster. So at least as of now I don't see much value in having per stream config files.
It would also be good if we can omit the necessity of having to define the streams on the client side (because they're already defined on the server). I see two main problems of having to define them on the clients. 1. When we have multiple event generators (a gateway cluster), the streams will have to be defined on all nodes. Do we use dep-sync to sync up the configs on the clients? Even if we do, I think its better to not depend on dep-sync for the new stuff we're developing because we're moving away from dep-sync soon. 2. Problems in upgrading to newer product versions. When upgrading the product version, if a schema change has occurred, currently we only have to worry about changing the schema on the server side. With the above limitation, we now have to worry about changing the clients too. Thanks, NuwanD. On Tue, Aug 23, 2016 at 10:16 PM, Malintha Amarasinghe <malint...@wso2.com> wrote: > Hi All, > > Currently APIM is using an internal APIM specific configuration resides in > api-manager.xml which include DAS server URL, username, password etc. Those > configuration are used to instantiate an org.wso2.carbon.databridge.agent. > DataPublisher object, which is then used to publish events directly to > streams in DAS. > > As it is an APIM specific configuration, it is not reusable by other > non-APIM features. Also org.wso2.carbon.databridge.agent.DataPublisher is > intended to be used in non-carbon environments, for carbon environments, > the recommended way is to use the Event Stream OSGi Service. > > So we were trying to re-structure APIM data publishing code to use new > Event Stream OSGi service explained in [1]. > > 1. Create all the streams defined in DAS in APIM. > 2. Create an *event publisher per each stream *which takes data from the > stream and publish to DAS. > 3. In each event publisher we need to configure DAS specific configuration. > > Ex: > <?xml version="1.0" encoding="UTF-8"?> > <eventPublisher name="test" statistics="disable" trace="disable" xmlns=" > http://wso2.org/carbon/eventpublisher";> > <from streamName="streamTest" version="1.0.0"/> > <mapping customMapping="disable" type="wso2event"/> > <to eventAdapterType="wso2event"> > <property name="username">admin</property> > <property name="protocol">thrift</property> > <property name="publishingMode">non-blocking</property> > <property name="publishTimeout">0</property> > <property name="receiverURL">tcp://localhost:7611</property> > <property encrypted="true" name="password">XXXXXXXX</property> > </to> > </eventPublisher> > > But we are facing a small problem here; in APIM there are many event > streams being used. If there's a change happen to DAS configuration, we > need to change it in all the event publishers which makes it difficult to > maintain. As per the offline chat with DAS team, this is a current > limitation. > > Are we going to move forward with the existing implementation? > > Thanks, > Malintha > > [1] [Dev] Common configuration for publishing events from carbon servers > to DAS/CEP > -- > Malintha Amarasinghe > Software Engineer > *WSO2, Inc. - lean | enterprise | middleware* > http://wso2.com/ > > Mobile : +94 712383306 > -- Nuwan Dias Software Architect - WSO2, Inc. http://wso2.com email : nuw...@wso2.com Phone : +94 777 775 729
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
