Re: [Dev] [MB] TombstoneOverwhelmingException When Purge Operation is Triggered When Subscription Disconnection/Deletion

2015-03-12 Thread Pamod Sylvester
+1 that's a good point. We could define a function to normalize the slot range through consolidating of all the remaining slots together. On Thu, Mar 12, 2015 at 1:22 PM, Asitha Nanayakkara asi...@wso2.com wrote: Hi Pamod Querying the remaining slots and removing content directly is OK. As an

Re: [Dev] [MB] TombstoneOverwhelmingException When Purge Operation is Triggered When Subscription Disconnection/Deletion

2015-03-12 Thread Hasitha Amal De Silva
Hi Pamod, +1 for the solution. as discussed, we can use a single service call to fetch all slots from the coordinator for the given queue, and trigger single message deletions (referring to OnflightMessageTracker and SlotDeliveryWorker) . This will ensure that tombstones are not read when purging

Re: [Dev] [MB] TombstoneOverwhelmingException When Purge Operation is Triggered When Subscription Disconnection/Deletion

2015-03-12 Thread Asitha Nanayakkara
Hi Pamod Querying the remaining slots and removing content directly is OK. As an improvement you can add a separate method in slot coordinator to get that message range of all the slots, to reduce cluster communication, and then do the necessary processing in the local node it self. Thanks,

Re: [Dev] [MB] TombstoneOverwhelmingException When Purge Operation is Triggered When Subscription Disconnection/Deletion

2015-03-12 Thread Hasitha Hiranya
Hi, Idea was when last subscriber goes purge the messages without getting slots involved because it need to happen lightweight and fast. Thanks On Thu, Mar 12, 2015 at 3:16 AM, Pamod Sylvester pa...@wso2.com wrote: +1 Agreed, also as you and Ramith mentioned deleting the content directly

Re: [Dev] [MB] TombstoneOverwhelmingException When Purge Operation is Triggered When Subscription Disconnection/Deletion

2015-03-12 Thread Pamod Sylvester
Tombstones became the killer :( The plan is to consolidate the unassigned slot ranges together and schedule the content to be removed within that range only so that the query will not grab tombstones. Thanks, Pamod On Thu, Mar 12, 2015 at 2:11 PM, Hasitha Hiranya hasit...@wso2.com wrote: Hi,

[Dev] [MB] TombstoneOverwhelmingException When Purge Operation is Triggered When Subscription Disconnection/Deletion

2015-03-12 Thread Pamod Sylvester
Hi All, During the subscription disconnection/deletion purge operation is being called on the the relevant non durable topic the subscription/s was bound to. During purge operation all existing data (if any) relevant to that topic will be removed. When using hector the removal is done through a