Hi, This is the proposed design for implementing this feature.
GlobalQueueWorker(GQW): - Maintain a HashMap of number of messages copied to each node queue. Key of this HashMap is NodeQueue+queueName. - Before Copying messages to nodeQueues GQW checks the message copied count in the Hashmap and copies only if that value is lower than HighWaterMark(HWM) threshold. - If the value is equal to HWM level GQW notifies the related NodeQueue(NQ) that it has reached the HWM level and cease sending messages further to that NQ. After that GQW selects another subscriber NQ from the rest of the nodeQueues randomly. - If QDW received a notification from NQ that it has reached its LowWaterMark (LWM) level, it reset the value related to that NQ in the hashmap. QueueDiliveryWorker(QDW): - When QDW is notified by the GQW that it has reached the HWM level it keeps a count on number of messages to be delivered. When it reaches to the LWM level it notifies the QDW that it has reached to its LWM level. Message notification is done using Hazelcast. Any suggestions will be appreciated. Thank you, Sajini. On Tue, Jul 22, 2014 at 2:00 PM, Sajini De Silva <[email protected]> wrote: > Adding to architecture@. > > > On Mon, Jul 21, 2014 at 3:09 PM, Sajini De Silva <[email protected]> wrote: > >> Hi, >> >> I'm implementing the $subject in MB 3.0.0. Currently in an MB cluster if >> there is only one subscriber for a particular queue GlobalQueueWorker(GQW) >> copy all the messages which come to that particular queue, to node queue in >> the subscriber's MB node. If another subscriber joins from another node >> after GQW copied its all messages to another node queue there is no way the >> new subscriber can get any messages even though most of the messages are >> not delivered to the previous subscriber. >> >> The idea here is to introduce two threshold values known as >> HighWaterMark(HWM) level and LowWaterMark(LWM) level in order to avoid the >> above scenario. So that GQW copies messages to a node queue only up to HWM >> even though there are many more messages. If another subscriber joins to >> the cluster later he will not be starved. When a nodeQueue hits the LWM >> threshold if there are more messages in the global queue, node queue will >> be filled up to HWM by the GQW. >> >> I'll update the thread as I progress. >> >> Thank you, >> Sajini. >> >> -- >> Sajini De SIlva >> Software Engineer; WSO2 Inc.; http://wso2.com , >> Email: [email protected] >> Blog: http://sajinid.blogspot.com/ >> Git hub profile: https://github.com/sajinidesilva >> >> > > > -- > Sajini De SIlva > Software Engineer; WSO2 Inc.; http://wso2.com , > Email: [email protected] > Blog: http://sajinid.blogspot.com/ > Git hub profile: https://github.com/sajinidesilva > > -- Sajini De SIlva Software Engineer; WSO2 Inc.; http://wso2.com , Email: [email protected] Blog: http://sajinid.blogspot.com/ Git hub profile: https://github.com/sajinidesilva
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
