Hi Asitha, The problem here is that we cannot convert the bytes value into the chunk count.
For example when the bytes value is configured to 1MB, - If the user publishes messages of size 10K, 100 chunks will be allowed - If the user publishes messages of size 1K, 1000 chunks will be allowed The smaller the message size gets, the larger the number of allowed chunks. Therefore, it's unpredictable at the time of configuration. Thank you On Tue, Jul 21, 2015 at 4:43 AM, Asitha Nanayakkara <[email protected]> wrote: > Hi all, > > Can't we use the bytes value in configuration (broker.xml) and convert > into chunk count when reading the configuration value and use it. That way > internally nothing is changed plus the user has a better idea on sizes as > well. > A win win situation IMO. Anyway if the global chunk count is greater > than 4/5 (or some reasonable fraction) of the Disruptor ring size we can > throw an exception and stop the server at startup. > > Am I missing something here in that approach? > > Regards, > Asitha > > On Thu, Jul 16, 2015 at 6:12 PM, Sasikala Kottegoda <[email protected]> > wrote: > >> Hi all, >> >> Currently Message Broker enables flow controlling based on the content >> chunk count. The configuration as shown below. >> >> <flowControl> >> <global> >> <lowLimit>800</lowLimit> >> <highLimit>8000</highLimit> >> </global> >> >> <bufferBased> >> <lowLimit>100</lowLimit> >> <highLimit>1000</highLimit> >> </bufferBased> >> </flowControl> >> >> Flow controlling is enabled when the chunk count that is read from the >> buffer is higher than the specified number. >> >> Since this is misleading for a user, we decided to set the above limits >> for buffer sizes in Bytes. >> >> With the new implementation, we face many problems since we cannot not >> relate between the number of messages and the total space occupied by >> messages at the configuration time (Messages with different sizes could be >> published). Following are the problems faced due to this: >> >> - Flow controlling buffer limits are set for sizes in bytes. >> - Mina buffer accepts all the messages until we enable flow >> controlling. (It is enabled when the total size of the messages read >> by *the >> broker* is higher than the specified limit.) >> - For the given limit, the number of messages that could be taken >> in is high for small messages. >> - More time is taken to read and process a large number of small >> messages. >> - Therefore, by the time the broker enables flow controlling, the >> Mina buffer would have overflowed. >> >> - The ring buffer in the disruptor is initialized with a specified >> buffer size with a given number of elements. >> - If the publisher sends messages with smaller sizes, a large >> number of messages will be taken in before flow controlling is >> enabled, and >> the disruptor will overflow. >> - The user cannot relate between the number of messages that are >> published before flow controlling is enabled (By having a count specified >> as in the existing approach, the number of messages that are taken in is >> constant whatever the message size is). >> >> Therefore, we decided to leave the configuration as it is. Any comments >> are highly appreciated. >> >> Thank you >> -- >> Sasikala Kottegoda >> *Software Engineer* >> WSO2 Inc., http://wso2.com/ >> lean. enterprise. middleware >> Mobile: +94 774835928/712792401 >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > *Asitha Nanayakkara* > Software Engineer > WSO2, Inc. http://wso2.com/ > Mob: + 94 77 85 30 682 > > -- Sasikala Kottegoda *Software Engineer* WSO2 Inc., http://wso2.com/ lean. enterprise. middleware Mobile: +94 774835928/712792401
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
