On 18 March 2014 11:33, Maninda Edirisooriya <mani...@wso2.com> wrote:
> Hi Lasantha, > > Actually, Data Publisher too has some asynchronous features like queues. > The difference between AsyncDataPublisher and DataPublisher is that > AsyncDataPublisher has asynchronous connections where DataPublisher crashes > when connections fail AFAIK. The reality is that the DataPublisher was > designed first but in order to overcome its issues, AsyncDataPublisher was > later created, wrapping the existing DataPublisher. So my point is > DataPublisher is not suitable for blocking calls as it has some other > issues like connection issue I mentioned. > > In future we may want to reconsider all the data publishers including > LoadBalancingDataPublisher and come up with a simplified data publisher. > But for now we can provide the flexibility to the user with this > configuration. > OK... Thanks for clarifying. > > Thanks. > > > > * Maninda Edirisooriya* > Software Engineer > > *WSO2, Inc. *lean.enterprise.middleware. > > *Blog* : http://maninda.blogspot.com/ > *Phone* : +94 777603226 > > > On Tue, Mar 18, 2014 at 11:01 AM, Lasantha Fernando <lasan...@wso2.com>wrote: > >> Hi Maninda, >> >> Thanks for the clarification. >> >> +1 for making it configurable. My point is, since DataPublisher is >> designed to be synchronous and AsyncDataPublisher (which wraps around >> DataPublisher) is designed to be asynchronous, can't we get the >> blocking/non-blocking behaviour by simply switching between the publishers >> when it is configured so in thfitt-agent-config.xml ? >> >> Thanks, >> Lasantha >> >> >> On 14 March 2014 15:36, Maninda Edirisooriya <mani...@wso2.com> wrote: >> >>> Hi Lasantha, >>> >>> >>> Before the given fix was done, when data input rate is greater than >>> output rate, >>> >>> 1. Data Publisher queue started to fill >>> 2. After Data Publisher queue was filled Async Data Publisher was filling >>> 3. After the Async Data Publisher was filled the thread was blocked >>> >>> After the patch was given, >>> >>> 1. Above 3rd step will not happen. Instead next coming messages were >>> dropping >>> 2. So this patch will not block threads. But this will drop messages >>> will reduces the reliability. >>> >>> In the above mail, I have asked whether the *functionality before the >>> patch* and *functionality after the patch* should be configurable. >>> >>> >>> *Maninda Edirisooriya* >>> Software Engineer >>> >>> *WSO2, Inc. *lean.enterprise.middleware. >>> >>> *Blog* : http://maninda.blogspot.com/ >>> *Phone* : +94 777603226 >>> >>> >>> On Fri, Mar 14, 2014 at 2:54 PM, Lasantha Fernando <lasan...@wso2.com>wrote: >>> >>>> Hi Maninda, >>>> >>>> Isn't the default behaviour of the DataPublisher to block if the event >>>> queue is full? I thought it was only for AsyncDataPublisher that we drop >>>> events when the queue is full. Please correct me if I am wrong. >>>> >>>> Or did you mean that currently we do not have a mechanism of switching >>>> between DataPublisher and AsynDataPublisher and this will introduce a >>>> configuration parameter to switch between Async and Synchronous publishing? >>>> >>>> +1 to implementing this if this functionality is already not there. >>>> >>>> Thanks, >>>> Lasantha >>>> >>>> Thanks, >>>> Lasantha >>>> >>>> >>>> On 13 March 2014 15:15, Maninda Edirisooriya <mani...@wso2.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> In JIRA [1], in order to prevent Thread Death caused by blocked >>>>> threads in Data Publisher, we have introduced a new method to try >>>>> publish. >>>>> That will drop messages when both Data Publisher queue and Async Data >>>>> Publisher are filled. This will prevent the thread from blocking. >>>>> >>>>> But for some users the message logging may be a must and messages >>>>> cannot be dropped. In such situations we are going to provide a >>>>> configuration in *thrift-agent-config.xml* to set. >>>>> The default value (as given below) will continue with the current >>>>> non-blocking functionality. >>>>> >>>>> *<BlockThreadsOfPublishers>false</BlockThreadsOfPublishers>* >>>>> >>>>> Please suggest if there are any changes required. >>>>> >>>>> >>>>> [1] https://wso2.org/jira/browse/BAMDATAAGENTS-9 >>>>> >>>>> *Maninda Edirisooriya* >>>>> Software Engineer >>>>> >>>>> *WSO2, Inc. *lean.enterprise.middleware. >>>>> >>>>> *Blog* : http://maninda.blogspot.com/ >>>>> *Phone* : +94 777603226 >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> Architecture@wso2.org >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Lasantha Fernando* >>>> Software Engineer - Data Technologies Team >>>> WSO2 Inc. http://wso2.com >>>> >>>> email: lasan...@wso2.com >>>> mobile: (+94) 71 5247551 >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> _______________________________________________ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *Lasantha Fernando* >> Software Engineer - Data Technologies Team >> WSO2 Inc. http://wso2.com >> >> email: lasan...@wso2.com >> mobile: (+94) 71 5247551 >> >> _______________________________________________ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- *Lasantha Fernando* Software Engineer - Data Technologies Team WSO2 Inc. http://wso2.com email: lasan...@wso2.com mobile: (+94) 71 5247551
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
