Thanks for the pull request there are some comments on the code please fix them
Thanks Suho On Thu, Sep 17, 2015 at 4:56 PM, Miyuru Dayarathna <miyu...@wso2.com> wrote: > Hi Suho, > > Today's afternoon I merged the out-of-order handler (OOH) code to Siddhi > and created a pull request. It comprises of a simple test case [2 > <https://github.com/miyurud/siddhi/blob/master/modules/siddhi-extensions/ooh/src/test/java/org/wso2/siddhi/extension/ooh/KSlackExtensionTestCase.java>] > which is described in [3 > <https://drive.google.com/a/wso2.com/file/d/0B8UIlB9H3rv_THpoRi0xTFFERVE/view?usp=sharing>]. > I tested the code with DEBS 2015 Grand Challenge's taxi trips data set [1 > <http://www.debs2015.org/call-grand-challenge.html>] as well. I used the > small 2 million events data set in the tests. Since the events in the > original data set were in sorted order, I introduced a delay to certain > events in the event stream before sending them to the OOH (Shown in green > color highlighting in [4 > <https://docs.google.com/a/wso2.com/spreadsheets/d/1xYGJoNfCSQTv1KsAuPNYsSSFqOEpn8gqCZy35L8GDmo/edit?usp=sharing>]). > The results are listed in [4 > <https://docs.google.com/a/wso2.com/spreadsheets/d/1xYGJoNfCSQTv1KsAuPNYsSSFqOEpn8gqCZy35L8GDmo/edit?usp=sharing>]. > The OOH reorders event streams in both example scenarios except for few > glitches. For 100 events case there was one unordered event output by OOH > (Highlighted in yellow color in [4 > <https://docs.google.com/a/wso2.com/spreadsheets/d/1xYGJoNfCSQTv1KsAuPNYsSSFqOEpn8gqCZy35L8GDmo/edit?usp=sharing>]). > I am investigating solutions for such unordered events output by OOH. > However, a similar situation has been described in Figure 2 of [3 > <https://drive.google.com/a/wso2.com/file/d/0B8UIlB9H3rv_THpoRi0xTFFERVE/view?usp=sharing>] > as well. Another notable observation is that considerable amount of events > were buffered in the OOH while the extension was running. In certain cases > (as shown in [4 > <https://docs.google.com/a/wso2.com/spreadsheets/d/1xYGJoNfCSQTv1KsAuPNYsSSFqOEpn8gqCZy35L8GDmo/edit?usp=sharing>]) > there were about 41 events buffered in OOH. Hence, both the test cases > ended up outputting lesser number of events than what is input to OOH > because the remaining set of events were buffered inside the OOH when the > test case terminated. This is expected behavior of a buffer based > out-of-order handler. Next, I will continue experiments with full data sets > of DEBS 2013 and 2015 data sets and observe the characteristics of OOH > buffer for different out-of-order scenarios. > > [1] http://www.debs2015.org/call-grand-challenge.html > > [2] > https://github.com/miyurud/siddhi/blob/master/modules/siddhi-extensions/ooh/src/test/java/org/wso2/siddhi/extension/ooh/KSlackExtensionTestCase.java > > [3] > https://drive.google.com/a/wso2.com/file/d/0B8UIlB9H3rv_THpoRi0xTFFERVE/view?usp=sharing > > [4] > https://docs.google.com/a/wso2.com/spreadsheets/d/1xYGJoNfCSQTv1KsAuPNYsSSFqOEpn8gqCZy35L8GDmo/edit?usp=sharing > > -- > Thanks, > Miyuru Dayarathna > Senior Technical Lead > Mobile: +94713527783 > Blog: http://miyurublog.blogspot.com > > On Wed, Sep 16, 2015 at 11:10 PM, Sriskandarajah Suhothayan <s...@wso2.com > > wrote: > >> Hi Miyuru >> >> Any update on this ? >> >> Suho >> >> On Tue, Sep 8, 2015 at 7:58 PM, Miyuru Dayarathna <miyu...@wso2.com> >> wrote: >> >>> Hi Suho, >>> >>> Ok, I will work on that. >>> >>> -- >>> Thanks, >>> Miyuru Dayarathna >>> Senior Technical Lead >>> Mobile: +94713527783 >>> Blog: http://miyurublog.blogspot.com >>> >>> On Tue, Sep 8, 2015 at 5:38 PM, Sriskandarajah Suhothayan <s...@wso2.com >>> > wrote: >>> >>>> Can you merge this code to siddhi and send a pull we will be able to >>>> add this to the release >>>> >>>> Suho >>>> >>>> On Tue, Sep 8, 2015 at 5:17 PM, Miyuru Dayarathna <miyu...@wso2.com> >>>> wrote: >>>> >>>>> Hi Suho, >>>>> >>>>> I have completed developing the K-Slack extension for Siddhi and I >>>>> have hosted the code in [1 <https://github.com/miyurud/Siddhi_Dev>]. >>>>> I have tested the KSlackExtension class [2 >>>>> <https://github.com/miyurud/Siddhi_Dev/blob/master/modules/siddhi-extensions/ooh/src/main/java/org/wso2/siddhi/extension/ooh/KSlackExtension.java>] >>>>> using a test scenario described in [3 >>>>> <https://drive.google.com/a/wso2.com/file/d/0B8UIlB9H3rv_THpoRi0xTFFERVE/view?usp=sharing>] >>>>> and I have verified the correctness of the implementation. The test case >>>>> is >>>>> accessible from [4 >>>>> <https://github.com/miyurud/Siddhi_Dev/blob/master/modules/siddhi-extensions/ooh/src/test/java/org/wso2/siddhi/extension/ooh/KSlackExtensionTestCase.java>]. >>>>> We need to discuss whether we should keep the following Siddhi syntax or >>>>> change it to some different version by removing the use of "window." >>>>> construct. This code assumes that the very first field present in the >>>>> input >>>>> event as the timestamp used for reordering the events. >>>>> >>>>> @info(name = 'query1') from inputStream#window.ooh:kslack() select >>>>> eventtt , price, volume insert into outputStream; >>>>> >>>>> I have assigned myself the Redmine issue [5 >>>>> <https://redmine.wso2.com/issues/4032>]. Next, I will move onto >>>>> testing the performance of K-slack approach using the DEBS 2013 Grand >>>>> Challenge data set. Also I will try out other approaches for Out-of-Order >>>>> event processing which I have mentioned in the slides. >>>>> >>>>> [1] https://github.com/miyurud/Siddhi_Dev >>>>> >>>>> [2] >>>>> https://github.com/miyurud/Siddhi_Dev/blob/master/modules/siddhi-extensions/ooh/src/main/java/org/wso2/siddhi/extension/ooh/KSlackExtension.java >>>>> >>>>> [3] >>>>> https://drive.google.com/a/wso2.com/file/d/0B8UIlB9H3rv_THpoRi0xTFFERVE/view?usp=sharing >>>>> >>>>> [4] >>>>> https://github.com/miyurud/Siddhi_Dev/blob/master/modules/siddhi-extensions/ooh/src/test/java/org/wso2/siddhi/extension/ooh/KSlackExtensionTestCase.java >>>>> >>>>> [5] https://redmine.wso2.com/issues/4032 >>>>> >>>>> <https://drive.google.com/a/wso2.com/file/d/0B8UIlB9H3rv_THpoRi0xTFFERVE/view?usp=sharing> >>>>> -- >>>>> Thanks, >>>>> Miyuru Dayarathna >>>>> Senior Technical Lead >>>>> Mobile: +94713527783 >>>>> Blog: http://miyurublog.blogspot.com >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> *S. Suhothayan* >>>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>>> *WSO2 Inc. *http://wso2.com >>>> * <http://wso2.com/>* >>>> lean . enterprise . middleware >>>> >>>> >>>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>>> http://lk.linkedin.com/in/suhothayan >>>> <http://lk.linkedin.com/in/suhothayan>* >>>> >>> >>> >>> >>> -- >>> Thanks, >>> Miyuru Dayarathna >>> Senior Technical Lead >>> Mobile: +94713527783 >>> Blog: http://miyurublog.blogspot.com >>> >> >> >> >> -- >> >> *S. Suhothayan* >> Technical Lead & Team Lead of WSO2 Complex Event Processor >> *WSO2 Inc. *http://wso2.com >> * <http://wso2.com/>* >> lean . enterprise . middleware >> >> >> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >> http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* >> > > > > -- > Thanks, > Miyuru Dayarathna > Senior Technical Lead > Mobile: +94713527783 > Blog: http://miyurublog.blogspot.com > -- *S. Suhothayan* Technical Lead & Team Lead of WSO2 Complex Event Processor *WSO2 Inc. *http://wso2.com * <http://wso2.com/>* lean . enterprise . middleware *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>*
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
