On Wed, Aug 14, 2013 at 9:04 PM, Malith Dhanushka <[email protected]> wrote:
> > > > On Wed, Aug 14, 2013 at 1:07 PM, Maninda Edirisooriya <[email protected]>wrote: > >> At the moment in BAM, all the data received from data bridge are directly >> stored to the Cassandra. When a new stream creation message is received, >> that stream is automatically created and starts to listen to that stream >> automatically. >> >> As a new feature we are going to re-use the InputTransportAdapter and >> InputMessageBuilder (currently used by CEP) to receive messages to BAM. The >> rationale is to unify the message receiving features in both CEP and BAM >> which will enable BAM to receive message from other transports such as JMS. >> >> One problem with this new feature is enabling the automatic stream >> creation with InputTransportAdapter and InputMessageBuilder. The user has >> to pre-configure the streams it is intended to be listened so that new >> message builders are created for each stream. >> >> The second problem is filtering the streams come to CEP and BAM. Some >> streams may be only required to be used by CEP but not BAM. The reason is >> keeping all the messages comes to CEP in Cassandra will waste the disk >> space. But there is no stream filtering mechanism either in CEP or BAM at >> the moment. >> > > Yes. This is a limitation and there should be a proper mechanism to route > streams based on stream content. This can be implemented as a generic > feature. > > This can be done by CEP it self, and its one of CEP's use-case, in that case we can use the CEP's Cassandra writing capabilities Suho > >> The solution we have come up is as follows. >> >> 1. BAM is listening to both event builders and data bridge. >> 2. Event builders should be pre configured if the streams have to be >> received via event builders. >> 3. But still all the streams come to data bridge will be received by the >> data bridge listener. >> 4. When a new stream creation message is received either to event builder >> or to data bridge, they will notify it to our new lisetner. >> 5. There is a separate configuration stating what are the streams >> accepted by the BAM. If that configuration is "*", all the streams should >> be accepted. (should go to Cassandra) If there is a comma seperated list of >> streams, only those streams should go to Cassandra. If a set of streams >> stated after a "!" symbol, all the streams except those streams should go >> to Cassandra. >> 6. When a stream creation message is notified to a listener, that >> listener should create a stream in the Cassandra only if that stream is >> matching with the stream set stated in the config file. >> 7. Now the problem is receiving the same message from both event builder >> and the data bridge. Therefore when a stream creation is notified, the >> notifier (either event builder or the data bridge) is remembered by the >> listener. Only that notifier will be allowed to store events to the >> Cassandra using that stream. In other words, either event builder or the >> data bridge can store events using a unique stream. >> > > +1 for the approach. > > >> >> >> * >> Maninda Edirisooriya* >> Software Engineer >> *WSO2, Inc. >> *lean.enterprise.middleware. >> >> *Blog* : http://maninda.blogspot.com/ >> *Phone* : +94 777603226 >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Malith Dhanushka > > Engineer - Data Technologies > *WSO2, Inc. : wso2.com* > > *Mobile* : +94 716 506 693 > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *S. Suhothayan * Associate Technical Lead, *WSO2 Inc. *http://wso2.com * <http://wso2.com/>* lean . enterprise . middleware *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan* * *
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
