Hi, Thank you for the feed backs. So our implementation approach will be BAM pre-calculates a historical view of the system using hive queries and CEP calculates the most recent view of the system using siddhi queries. The service layer which will be a CEP Event table (RDBMS table) responsible for satisfying the queries given by both BAM and CEP.
On Thu, Nov 27, 2014 at 6:49 PM, Rajeev Sampath <raje...@wso2.com> wrote: > Hi, > > On Thu, Nov 27, 2014 at 6:36 PM, Inosh Goonewardena <in...@wso2.com> > wrote: > >> Hi, >> >> On Thu, Nov 27, 2014 at 4:34 PM, Maninda Edirisooriya <mani...@wso2.com> >> wrote: >> >>> This seems nice. But I am not sure how the precomputation works and how >>> the intermediate indexes are stored in Service Layer. >>> >> >> Basically the idea should be to store outputs from both batch and speed >> layers in the serving layer so that service layer can respond to ad-hoc >> queries using already processed data. According to the given architecture, >> outputs from both BAM and CEP is going to be saved in CEP Event table which >> can be a RDBMS table or in memory grid. >> >> > > For this use case RDBMS tables seem to be a better option over the > in-memory tables. > > In addition to the RDBMS, Cassandra event tables can also be used through > an existing extension. (however for the current version, this involves > applying a patch to the siddhi core). > > > Thanks > Rajeev > > > >> And also does the user have to write both hive scripts and Siddhi queries >>> for the same scenario? >>> >>> Thanks. >>> >>> >>> *Maninda Edirisooriya* >>> Senior Software Engineer >>> >>> *WSO2, Inc.*lean.enterprise.middleware. >>> >>> *Blog* : http://maninda.blogspot.com/ >>> *E-mail* : mani...@wso2.com >>> *Skype* : @manindae >>> *Twitter* : @maninda >>> >>> On Thu, Nov 27, 2014 at 2:25 PM, Dinithi De Silva <dinit...@wso2.com> >>> wrote: >>> >>>> Hi all, >>>> >>>> Myself with Zamly (za...@wso2.com) and Inshaf (ins...@wso2.com) are >>>> going to develop $subject as described below. >>>> >>>> *Introduction to Lambda architecture* >>>> The lambda architecture solves the problem of computing arbitrary >>>> functions on arbitrary data in real time by decomposing the problem into >>>> three layers: the batch layer, the serving layer, and the speed layer.[1] >>>> >>>> *Batch layer* >>>> The batch layer stores the master copy of the dataset and precomputes >>>> batch views on that master dataset. >>>> >>>> *Speed layer* >>>> The speed layer is needed to compensate for the high latency of the >>>> batch layer and it does this by computing real time views in Storm. >>>> >>>> *Serving layer* >>>> The serving layer is responsible for indexing and exposing the batch >>>> view and real time view, so that they can be queried for updated results. >>>> >>>> *Approach* >>>> We are planing to implement Lambda architecture using CEP and BAM. >>>> Please find the diagram for the architecture below. >>>> >>>> ​Feed backs and suggestions for improvements would be appreciated. >>>> >>>> [1] - >>>> http://jameskinley.tumblr.com/post/37398560534/the-lambda-architecture-principles-for >>>> >>>> -- >>>> *Dinithi De Silva* >>>> Associate Software Engineer, WSO2 Inc. >>>> m:+94716667655 | e:dinit...@wso2.com | w: www.wso2.com >>>> | a: #20, Palm Grove, Colombo 03 >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Regards, >> >> Inosh Goonewardena >> Associate Technical Lead- WSO2 Inc. >> Mobile: +94779966317 >> >> _______________________________________________ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > Rajeev Sampath > Senior Software Engineer > WSO2, Inc.; http://www.wso2.com. > > Mobile: > * +94716265766 <%2B94716265766>* > > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Inshaf Mahath Associate Software Engineer Mobile: +94775907181 WSO2 Inc. Lean . Enterprise . Middleware
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
