Ok got it. Thanks. On Tue, Jul 21, 2015 at 12:25 PM, Sasikala Kottegoda <[email protected]> wrote:
> Hi Asitha, > > The problem here is that we cannot convert the bytes value into the chunk > count. > > For example when the bytes value is configured to 1MB, > > - If the user publishes messages of size 10K, 100 chunks will be > allowed > - If the user publishes messages of size 1K, 1000 chunks will be > allowed > > The smaller the message size gets, the larger the number of allowed > chunks. Therefore, it's unpredictable at the time of configuration. > > Thank you > > On Tue, Jul 21, 2015 at 4:43 AM, Asitha Nanayakkara <[email protected]> > wrote: > >> Hi all, >> >> Can't we use the bytes value in configuration (broker.xml) and convert >> into chunk count when reading the configuration value and use it. That way >> internally nothing is changed plus the user has a better idea on sizes as >> well. >> A win win situation IMO. Anyway if the global chunk count is greater >> than 4/5 (or some reasonable fraction) of the Disruptor ring size we can >> throw an exception and stop the server at startup. >> >> Am I missing something here in that approach? >> >> Regards, >> Asitha >> >> On Thu, Jul 16, 2015 at 6:12 PM, Sasikala Kottegoda <[email protected]> >> wrote: >> >>> Hi all, >>> >>> Currently Message Broker enables flow controlling based on the content >>> chunk count. The configuration as shown below. >>> >>> <flowControl> >>> <global> >>> <lowLimit>800</lowLimit> >>> <highLimit>8000</highLimit> >>> </global> >>> >>> <bufferBased> >>> <lowLimit>100</lowLimit> >>> <highLimit>1000</highLimit> >>> </bufferBased> >>> </flowControl> >>> >>> Flow controlling is enabled when the chunk count that is read from the >>> buffer is higher than the specified number. >>> >>> Since this is misleading for a user, we decided to set the above limits >>> for buffer sizes in Bytes. >>> >>> With the new implementation, we face many problems since we cannot not >>> relate between the number of messages and the total space occupied by >>> messages at the configuration time (Messages with different sizes could be >>> published). Following are the problems faced due to this: >>> >>> - Flow controlling buffer limits are set for sizes in bytes. >>> - Mina buffer accepts all the messages until we enable flow >>> controlling. (It is enabled when the total size of the messages read >>> by *the >>> broker* is higher than the specified limit.) >>> - For the given limit, the number of messages that could be taken >>> in is high for small messages. >>> - More time is taken to read and process a large number of small >>> messages. >>> - Therefore, by the time the broker enables flow controlling, the >>> Mina buffer would have overflowed. >>> >>> - The ring buffer in the disruptor is initialized with a >>> specified buffer size with a given number of elements. >>> - If the publisher sends messages with smaller sizes, a large >>> number of messages will be taken in before flow controlling is >>> enabled, and >>> the disruptor will overflow. >>> - The user cannot relate between the number of messages that are >>> published before flow controlling is enabled (By having a count specified >>> as in the existing approach, the number of messages that are taken in is >>> constant whatever the message size is). >>> >>> Therefore, we decided to leave the configuration as it is. Any comments >>> are highly appreciated. >>> >>> Thank you >>> -- >>> Sasikala Kottegoda >>> *Software Engineer* >>> WSO2 Inc., http://wso2.com/ >>> lean. enterprise. middleware >>> Mobile: +94 774835928/712792401 >>> >>> _______________________________________________ >>> Architecture mailing list >>> [email protected] >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *Asitha Nanayakkara* >> Software Engineer >> WSO2, Inc. http://wso2.com/ >> Mob: + 94 77 85 30 682 >> >> > > > -- > Sasikala Kottegoda > *Software Engineer* > WSO2 Inc., http://wso2.com/ > lean. enterprise. middleware > Mobile: +94 774835928/712792401 > -- *Asitha Nanayakkara* Software Engineer WSO2, Inc. http://wso2.com/ Mob: + 94 77 85 30 682
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
