Hi Pamod, Inverted bitmap technique[1] looks good for the purpose. Algorithm is O(n) where n is number of sections in the destination, not subscribers.
Only concern is memory consumption and updating the maps when new subscribers come while messages are flowing through the system at higher rates (This will introduce some pauses to delivery). Since adding and removing subscriptions doesn't happen frequently we can live with that I think. Need to look at the memory consumption to make sure this is feasible. [1] http://zeromq.org/whitepapers:message-matching On Mon, Dec 15, 2014 at 5:52 PM, Pamod Sylvester <pa...@wso2.com> wrote: > > Hi All, > > Since QoS 0 in MQTT is a performance oriented delivery mode, an in-memory > based message store was implemented to bypass the message database when > running MB in stand alone mode, however when running in cluster in-memory > store will not be applicable, since in order to exchange messages across > the cluster, it requires a mechanism to communicate between brokers (Which > will be introduced in future MB versions) > > In Stand-Alone mode the current message flow would be similar to the > following, > > Publisher ---> Message ---> Deliver from In-Memory Store ---> Deliver from > Andes Kernal Store ---> Message ---> Subscription > > In the above flow, the subscriptions with QoS 0 would receive the messages > through the in-memory store and the also currently the published message > would be sent to the Andes kernal as well. Since for the same topic there > could be subscriptions which are > QoS 0. Making a call to Andes Kernal > strore would not make much sense if there're no subscriptions for a > particular topic which is > QoS 0, also it will cause additional operation > overhead. Therefore, we're in the process of implementing a solution which > would be able to identify whether there are subscriptions > QoS 0 for a > given topic to identify the necessity to perform a call to the store. > > If we maintain the state of topics which has subscriptions > QoS 0, for > each message published it would require to iterate through all the > subscriptions O(n), since there will be cases where the published topic > will not 1:1 map with the subscribed topic, (i.e hierarchical topics with > wildcards). In order to minimize the iteration cycles one option would be > to partition subscriptions based on its attributes such as hierarchy, > inclusive of wild card characters etc so that we could omit odds and > iterate only through the possibilities. > > WDYT ? Please do advice on most optimized ways of filtering out matching > subscriptions. > > Thanks, > Pamod > > -- > *Pamod Sylvester * > *Senior Software Engineer * > Integration Technologies Team, WSO2 Inc.; http://wso2.com > email: pa...@wso2.com cell: +94 77 7779495 > -- *Asitha Nanayakkara* Software Engineer WSO2, Inc. http://wso2.com/ Mob: + 94 77 85 30 682
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
