Currently we are having a single stream definition for service data agent for including both service statistics and activity information. However for activity mediator (BAM mediator) agent a different stream definition is used since it's different in that no statistics information is present. However this makes things bit difficult and less performant to correlate a message flow between an App server and ESB since it would require join to get data from two data column families. Analysis would become easier if both activity service and mediation are to be published in to a single stream.
A possible solution might be to separate out publishing of statistics data and activity data in service agent in to two streams (this means two events need to be published as well instead of one) and also having a separate stream definition for baseline attributes (another stream def would be required in the case of publishing custom attributes) of BAM mediator publisher. This would require changes to both service and mediation publisher to accommodate publishing to separate streams and make things bit complex IMO since there would be couple of additional stream definitions in certain cases. We might also have to think about the performance cost of publishing two events instead of one. Another option might be to use the current stream definition of activity service agent to activity mediator agent so that all the data for activity service, service statistics and activity mediation would be published to a single column family. In this case we can just use Hive to have different views of the same column family for different types of analytics (e.g: analytics required for service stats, activity etc). Downside of this approach is since all the data is that a single column family is now loaded more. We were thinking on going along with the first approach after some discussion. Kasun please add anything else I have missed. Please shout if there are any concerns and suggestions. Regards Buddhika
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
