I guess what I am suggesting is to allow the developer to explicitly state that something is shared, in the same way that they can read/write to a data table.
Paul On 10 June 2013 10:33, Sriskandarajah Suhothayan <s...@wso2.com> wrote: > > > > On Mon, Jun 10, 2013 at 2:26 PM, Paul Fremantle <p...@wso2.com> wrote: > >> Suho >> >> Is there a way to use Hazelcast to just communicate just one variable >> (e.g. how many messages in the last hour) vs sharing the whole CEP context? >> > Yes its possible for the batch window cases, here the Hazelcast will > provide good performance. but for the general case with the meta data we > also need to store the events that are in that window, in the cache, since > they are necessary to expire the events when the time expires. > > Suho > >> >> Paul >> >> >> On 10 June 2013 09:43, Sriskandarajah Suhothayan <s...@wso2.com> wrote: >> >>> >>> Here are some of my findings, >>> 1.Using thrift to send the information to centralised CEP will be much >>> efficient then using Hazelcast cluster, >>> 2.In Siddhi since its open for users to write their own quires, if they >>> write any with sliding time window the memory consumption will be high as >>> CEP need to remember all the events that occurred during that time window, >>> but if the query uses only batch windows (like every min/hour/day you are >>> allowed x ) then it wont be a problem >>> >>> Suho >>> >>> >>> On Mon, Jun 10, 2013 at 1:51 PM, Paul Fremantle <p...@wso2.com> wrote: >>> >>>> Suho >>>> >>>> In this case, where the rules are pretty simple, would the CEP overhead >>>> be large? >>>> >>>> Paul >>>> >>>> >>>> On 10 June 2013 09:06, Sriskandarajah Suhothayan <s...@wso2.com> wrote: >>>> >>>>> In addition to what Paul suggested if we want to offload the CEP's >>>>> memory conception and processing overhead from the clustered APIM gateway, >>>>> we can send all the events to CEP via thrift (same as sending to BAM ) and >>>>> the CEP will send the throttling events back when the throttling threshold >>>>> is reached >>>>> >>>>> Suho >>>>> >>>>> >>>>> On Mon, Jun 10, 2013 at 1:18 PM, Paul Fremantle <p...@wso2.com> wrote: >>>>> >>>>>> Isabelle >>>>>> >>>>>> We will need to have Siddhi talk to either a database or to >>>>>> Hazelcast. For many cases, we will need some persistent store because >>>>>> there >>>>>> will be rules such as up to 10k / month and we need to persist that. >>>>>> Siddhi >>>>>> already has some DB access, as well as some sharing across a cluster with >>>>>> Hazelcast. >>>>>> >>>>>> Paul >>>>>> >>>>>> >>>>>> On 10 June 2013 07:08, Isabelle Mauny <isabe...@wso2.com> wrote: >>>>>> >>>>>>> How will this work in a cluster ? >>>>>>> Throttling decisions are typically taken using cluster-wide data ( >>>>>>> such as 50 reqs for this API, no matter how many gateways are running ) >>>>>>> . >>>>>>> >>>>>>> Isabelle. >>>>>>> >>>>>>> ------ >>>>>>> Isabelle Mauny >>>>>>> Director, Product Management; WSO2, Inc.; http://wso2.com/ >>>>>>> email: isabe...@wso2.com <isabe...@wso2.com> - mobile: +34 616050684 >>>>>>> >>>>>>> >>>>>>> On Thu, Jun 6, 2013 at 7:04 PM, Paul Fremantle <p...@wso2.com>wrote: >>>>>>> >>>>>>>> Nuwan >>>>>>>> >>>>>>>> Siddhi is just a Java library, so this can be embedded. >>>>>>>> >>>>>>>> Paul >>>>>>>> >>>>>>>> >>>>>>>> On 6 June 2013 17:52, Nuwan Dias <nuw...@wso2.com> wrote: >>>>>>>> >>>>>>>>> On Thu, Jun 6, 2013 at 3:28 PM, Sriskandarajah Suhothayan < >>>>>>>>> s...@wso2.com> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Jun 6, 2013 at 3:10 PM, Vijayaratha Vijayasingam < >>>>>>>>>> rat...@wso2.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi; >>>>>>>>>>> APIM team has different kind of throttling+rate limiting >>>>>>>>>>> requirements (eg:ip/timer/geo based)..As Srinath pointed we believe >>>>>>>>>>> that >>>>>>>>>>> CEP would be the right solution, since we cannot have all these >>>>>>>>>>> different >>>>>>>>>>> scenarios/requirements to be implemented in our throttling module.. >>>>>>>>>>> Is there any possibility integrate CEP engine with our >>>>>>>>>>> throttling module ? >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes this is possible >>>>>>>>>> Currently you can use Siddhi directly to achieve this. In this >>>>>>>>>> case you have to pass all the events the trigger throttling directly >>>>>>>>>> to >>>>>>>>>> Siddhi and, when a matching throttling condition reached CEP will >>>>>>>>>> fire an >>>>>>>>>> output back to APIM so that it can take action based on the CEP's >>>>>>>>>> response. >>>>>>>>>> >>>>>>>>> >>>>>>>>> How feasible is it to run Siddhi as an embedded engine within >>>>>>>>> APIM? I am thinking along the lines of avoiding an additional hop >>>>>>>>> between >>>>>>>>> the API Gateway and CEP. >>>>>>>>> >>>>>>>>> Is it also possible to persist the throttling information to a >>>>>>>>> database? Probably not on every request but batch by batch >>>>>>>>> >>>>>>>>>> >>>>>>>>>> With CEP 3.0.0 we are writing an Event Processor component( as of >>>>>>>>>> mail thread "CEP 3.0 towards a Complete eventing solution to the >>>>>>>>>> platform"), when that is done you will also be able to provide a >>>>>>>>>> proper UI >>>>>>>>>> for the users to edit Siddhi throttling queries >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> So, all products can still use throttling module, which can >>>>>>>>>>> handle all the complex rate/thorrtling policies.. >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> +1 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Suho >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 5 June 2013 15:13, Sriskandarajah Suhothayan >>>>>>>>>>> <s...@wso2.com>wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Jun 5, 2013 at 1:27 AM, Srinath Perera < >>>>>>>>>>>> srin...@wso2.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi All, >>>>>>>>>>>>> >>>>>>>>>>>>> Currently we have a java based throttling solution. But we >>>>>>>>>>>>> need that to extended (e.g. support throughput based throttling), >>>>>>>>>>>>> and >>>>>>>>>>>>> support more complicated condition that currently parameterized. >>>>>>>>>>>>> >>>>>>>>>>>>> IMO, best way to do this is to support this by integrating CEP >>>>>>>>>>>>> (Siddhi) engine directly at java level. It is very light weight . >>>>>>>>>>>>> We can >>>>>>>>>>>>> let users provide CEP queries which will control throttling. >>>>>>>>>>>>> Basically, >>>>>>>>>>>>> there will be inbuilt event stream definitions, and Siddhi >>>>>>>>>>>>> listener that >>>>>>>>>>>>> monitors a given event stream and adjust event acceptance. Users >>>>>>>>>>>>> provide >>>>>>>>>>>>> CEP queries. >>>>>>>>>>>>> >>>>>>>>>>>>> I think it is too heavy publish events via thrift API if we >>>>>>>>>>>>> try to send it via the network. >>>>>>>>>>>>> >>>>>>>>>>>>> WDYT? >>>>>>>>>>>>> >>>>>>>>>>>> +1 >>>>>>>>>>>> CEP team can provide the necessary support >>>>>>>>>>>> if any of the product teams (eg: ELB, BPS, AF or AM) is willing >>>>>>>>>>>> to replace their current or have an alternate throttling module >>>>>>>>>>>> >>>>>>>>>>>> Suho >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> --Srinath >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> ============================ >>>>>>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>>>>>> Director, Research, WSO2 Inc. >>>>>>>>>>>>> Visiting Faculty, University of Moratuwa >>>>>>>>>>>>> Member, Apache Software Foundation >>>>>>>>>>>>> Research Scientist, Lanka Software Foundation >>>>>>>>>>>>> Blog: http://srinathsview.blogspot.com/ >>>>>>>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>>>>>>> Phone: 0772360902 >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *S. Suhothayan >>>>>>>>>>>> * >>>>>>>>>>>> Software Engineer, >>>>>>>>>>>> Management Committee Member, Data Technologies Team, >>>>>>>>>>>> *WSO2 Inc. *http://wso2.com * >>>>>>>>>>>> <http://wso2.com/>* >>>>>>>>>>>> lean . enterprise . middleware >>>>>>>>>>>> >>>>>>>>>>>> *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>>> twitter: http://twitter.com/suhothayan | linked-in: >>>>>>>>>>>> http://lk.linkedin.com/in/suhothayan* >>>>>>>>>>>> * >>>>>>>>>>>> * >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>> Architecture@wso2.org >>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> -Ratha >>>>>>>>>>> mobile: (+94)755906608 >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Architecture mailing list >>>>>>>>>>> Architecture@wso2.org >>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *S. Suhothayan >>>>>>>>>> * >>>>>>>>>> Associate Technical Lead, >>>>>>>>>> >>>>>>>>>> Management Committee Member, Data Technologies Team, >>>>>>>>>> *WSO2 Inc. *http://wso2.com * >>>>>>>>>> <http://wso2.com/>* >>>>>>>>>> lean . enterprise . middleware >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ >>>>>>>>>> twitter: http://twitter.com/suhothayan | linked-in: >>>>>>>>>> http://lk.linkedin.com/in/suhothayan* >>>>>>>>>> * >>>>>>>>>> * >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Architecture mailing list >>>>>>>>>> Architecture@wso2.org >>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Nuwan Dias >>>>>>>>> >>>>>>>>> Senior Software Engineer - WSO2, Inc. http://wso2.com >>>>>>>>> email : nuw...@wso2.com >>>>>>>>> Phone : +94 777 775 729 >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Architecture mailing list >>>>>>>>> Architecture@wso2.org >>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Paul Fremantle >>>>>>>> CTO and Co-Founder, WSO2 >>>>>>>> OASIS WS-RX TC Co-chair, VP, Apache Synapse >>>>>>>> >>>>>>>> UK: +44 207 096 0336 >>>>>>>> US: +1 646 595 7614 >>>>>>>> >>>>>>>> blog: http://pzf.fremantle.org >>>>>>>> twitter.com/pzfreo >>>>>>>> p...@wso2.com >>>>>>>> >>>>>>>> wso2.com Lean Enterprise Middleware >>>>>>>> >>>>>>>> Disclaimer: This communication may contain privileged or other >>>>>>>> confidential information and is intended exclusively for the >>>>>>>> addressee/s. >>>>>>>> If you are not the intended recipient/s, or believe that you may have >>>>>>>> received this communication in error, please reply to the sender >>>>>>>> indicating >>>>>>>> that fact and delete the copy you received and in addition, you should >>>>>>>> not >>>>>>>> print, copy, retransmit, disseminate, or otherwise use the information >>>>>>>> contained in this communication. Internet communications cannot be >>>>>>>> guaranteed to be timely, secure, error or virus-free. The sender does >>>>>>>> not >>>>>>>> accept liability for any errors or omissions. >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> Architecture@wso2.org >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Architecture mailing list >>>>>>> Architecture@wso2.org >>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Paul Fremantle >>>>>> CTO and Co-Founder, WSO2 >>>>>> OASIS WS-RX TC Co-chair, VP, Apache Synapse >>>>>> >>>>>> UK: +44 207 096 0336 >>>>>> US: +1 646 595 7614 >>>>>> >>>>>> blog: http://pzf.fremantle.org >>>>>> twitter.com/pzfreo >>>>>> p...@wso2.com >>>>>> >>>>>> wso2.com Lean Enterprise Middleware >>>>>> >>>>>> Disclaimer: This communication may contain privileged or other >>>>>> confidential information and is intended exclusively for the addressee/s. >>>>>> If you are not the intended recipient/s, or believe that you may have >>>>>> received this communication in error, please reply to the sender >>>>>> indicating >>>>>> that fact and delete the copy you received and in addition, you should >>>>>> not >>>>>> print, copy, retransmit, disseminate, or otherwise use the information >>>>>> contained in this communication. Internet communications cannot be >>>>>> guaranteed to be timely, secure, error or virus-free. The sender does not >>>>>> accept liability for any errors or omissions. >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> Architecture@wso2.org >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *S. Suhothayan >>>>> * >>>>> Associate Technical Lead, >>>>> Management Committee Member, Data Technologies Team, >>>>> *WSO2 Inc. *http://wso2.com * >>>>> <http://wso2.com/>* >>>>> lean . enterprise . middleware >>>>> >>>>> *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ >>>>> twitter: http://twitter.com/suhothayan | linked-in: >>>>> http://lk.linkedin.com/in/suhothayan* >>>>> * >>>>> * >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> Architecture@wso2.org >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> Paul Fremantle >>>> CTO and Co-Founder, WSO2 >>>> OASIS WS-RX TC Co-chair, VP, Apache Synapse >>>> >>>> UK: +44 207 096 0336 >>>> US: +1 646 595 7614 >>>> >>>> blog: http://pzf.fremantle.org >>>> twitter.com/pzfreo >>>> p...@wso2.com >>>> >>>> wso2.com Lean Enterprise Middleware >>>> >>>> Disclaimer: This communication may contain privileged or other >>>> confidential information and is intended exclusively for the addressee/s. >>>> If you are not the intended recipient/s, or believe that you may have >>>> received this communication in error, please reply to the sender indicating >>>> that fact and delete the copy you received and in addition, you should not >>>> print, copy, retransmit, disseminate, or otherwise use the information >>>> contained in this communication. Internet communications cannot be >>>> guaranteed to be timely, secure, error or virus-free. The sender does not >>>> accept liability for any errors or omissions. >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> *S. Suhothayan >>> * >>> Associate Technical Lead, >>> Management Committee Member, Data Technologies Team, >>> *WSO2 Inc. *http://wso2.com * >>> <http://wso2.com/>* >>> lean . enterprise . middleware >>> >>> *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ >>> twitter: http://twitter.com/suhothayan | linked-in: >>> http://lk.linkedin.com/in/suhothayan* >>> * >>> * >>> >>> _______________________________________________ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> Paul Fremantle >> CTO and Co-Founder, WSO2 >> OASIS WS-RX TC Co-chair, VP, Apache Synapse >> >> UK: +44 207 096 0336 >> US: +1 646 595 7614 >> >> blog: http://pzf.fremantle.org >> twitter.com/pzfreo >> p...@wso2.com >> >> wso2.com Lean Enterprise Middleware >> >> Disclaimer: This communication may contain privileged or other >> confidential information and is intended exclusively for the addressee/s. >> If you are not the intended recipient/s, or believe that you may have >> received this communication in error, please reply to the sender indicating >> that fact and delete the copy you received and in addition, you should not >> print, copy, retransmit, disseminate, or otherwise use the information >> contained in this communication. Internet communications cannot be >> guaranteed to be timely, secure, error or virus-free. The sender does not >> accept liability for any errors or omissions. >> >> _______________________________________________ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *S. Suhothayan > * > Associate Technical Lead, > Management Committee Member, Data Technologies Team, > *WSO2 Inc. *http://wso2.com * > <http://wso2.com/>* > lean . enterprise . middleware > > *cell: (+94) 779 756 757 | blog: http://suhothayan.blogspot.com/ > twitter: http://twitter.com/suhothayan | linked-in: > http://lk.linkedin.com/in/suhothayan* > * > * > > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Paul Fremantle CTO and Co-Founder, WSO2 OASIS WS-RX TC Co-chair, VP, Apache Synapse UK: +44 207 096 0336 US: +1 646 595 7614 blog: http://pzf.fremantle.org twitter.com/pzfreo p...@wso2.com wso2.com Lean Enterprise Middleware Disclaimer: This communication may contain privileged or other confidential information and is intended exclusively for the addressee/s. If you are not the intended recipient/s, or believe that you may have received this communication in error, please reply to the sender indicating that fact and delete the copy you received and in addition, you should not print, copy, retransmit, disseminate, or otherwise use the information contained in this communication. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions.
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
