Hi Lasantha, Actually, Data Publisher too has some asynchronous features like queues. The difference between AsyncDataPublisher and DataPublisher is that AsyncDataPublisher has asynchronous connections where DataPublisher crashes when connections fail AFAIK. The reality is that the DataPublisher was designed first but in order to overcome its issues, AsyncDataPublisher was later created, wrapping the existing DataPublisher. So my point is DataPublisher is not suitable for blocking calls as it has some other issues like connection issue I mentioned.
In future we may want to reconsider all the data publishers including LoadBalancingDataPublisher and come up with a simplified data publisher. But for now we can provide the flexibility to the user with this configuration. Thanks. *Maninda Edirisooriya* Software Engineer *WSO2, Inc.*lean.enterprise.middleware. *Blog* : http://maninda.blogspot.com/ *Phone* : +94 777603226 On Tue, Mar 18, 2014 at 11:01 AM, Lasantha Fernando <lasan...@wso2.com>wrote: > Hi Maninda, > > Thanks for the clarification. > > +1 for making it configurable. My point is, since DataPublisher is > designed to be synchronous and AsyncDataPublisher (which wraps around > DataPublisher) is designed to be asynchronous, can't we get the > blocking/non-blocking behaviour by simply switching between the publishers > when it is configured so in thfitt-agent-config.xml ? > > Thanks, > Lasantha > > > On 14 March 2014 15:36, Maninda Edirisooriya <mani...@wso2.com> wrote: > >> Hi Lasantha, >> >> >> Before the given fix was done, when data input rate is greater than >> output rate, >> >> 1. Data Publisher queue started to fill >> 2. After Data Publisher queue was filled Async Data Publisher was filling >> 3. After the Async Data Publisher was filled the thread was blocked >> >> After the patch was given, >> >> 1. Above 3rd step will not happen. Instead next coming messages were >> dropping >> 2. So this patch will not block threads. But this will drop messages will >> reduces the reliability. >> >> In the above mail, I have asked whether the *functionality before the >> patch* and *functionality after the patch* should be configurable. >> >> >> *Maninda Edirisooriya* >> Software Engineer >> >> *WSO2, Inc. *lean.enterprise.middleware. >> >> *Blog* : http://maninda.blogspot.com/ >> *Phone* : +94 777603226 >> >> >> On Fri, Mar 14, 2014 at 2:54 PM, Lasantha Fernando <lasan...@wso2.com>wrote: >> >>> Hi Maninda, >>> >>> Isn't the default behaviour of the DataPublisher to block if the event >>> queue is full? I thought it was only for AsyncDataPublisher that we drop >>> events when the queue is full. Please correct me if I am wrong. >>> >>> Or did you mean that currently we do not have a mechanism of switching >>> between DataPublisher and AsynDataPublisher and this will introduce a >>> configuration parameter to switch between Async and Synchronous publishing? >>> >>> +1 to implementing this if this functionality is already not there. >>> >>> Thanks, >>> Lasantha >>> >>> Thanks, >>> Lasantha >>> >>> >>> On 13 March 2014 15:15, Maninda Edirisooriya <mani...@wso2.com> wrote: >>> >>>> Hi, >>>> >>>> In JIRA [1], in order to prevent Thread Death caused by blocked threads >>>> in Data Publisher, we have introduced a new method to try publish. That >>>> will drop messages when both Data Publisher queue and Async Data Publisher >>>> are filled. This will prevent the thread from blocking. >>>> >>>> But for some users the message logging may be a must and messages >>>> cannot be dropped. In such situations we are going to provide a >>>> configuration in *thrift-agent-config.xml* to set. >>>> The default value (as given below) will continue with the current >>>> non-blocking functionality. >>>> >>>> *<BlockThreadsOfPublishers>false</BlockThreadsOfPublishers>* >>>> >>>> Please suggest if there are any changes required. >>>> >>>> >>>> [1] https://wso2.org/jira/browse/BAMDATAAGENTS-9 >>>> >>>> *Maninda Edirisooriya* >>>> Software Engineer >>>> >>>> *WSO2, Inc. *lean.enterprise.middleware. >>>> >>>> *Blog* : http://maninda.blogspot.com/ >>>> *Phone* : +94 777603226 >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> Architecture@wso2.org >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> *Lasantha Fernando* >>> Software Engineer - Data Technologies Team >>> WSO2 Inc. http://wso2.com >>> >>> email: lasan...@wso2.com >>> mobile: (+94) 71 5247551 >>> >>> _______________________________________________ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> _______________________________________________ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *Lasantha Fernando* > Software Engineer - Data Technologies Team > WSO2 Inc. http://wso2.com > > email: lasan...@wso2.com > mobile: (+94) 71 5247551 > > _______________________________________________ > Architecture mailing list > Architecture@wso2.org > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > >
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
