Hi Pamod, +1 for the solution. as discussed, we can use a single service call to fetch all slots from the coordinator for the given queue, and trigger single message deletions (referring to OnflightMessageTracker and SlotDeliveryWorker) . This will ensure that tombstones are not read when purging in a subscription closed scenario.
However, we also need to think about triggering a queue purge from the UI, at which point we may not have active subscriptions or slot references. Thanks On Thu, Mar 12, 2015 at 11:56 AM, Pamod Sylvester <pa...@wso2.com> wrote: > Hi All, > > During the subscription disconnection/deletion purge operation is being > called on the the relevant non durable topic the subscription/s was bound > to. During purge operation all existing data (if any) relevant to that > topic will be removed. > > When using hector the removal is done through a range query following LOC, > > List<Long> currentPage = > HectorDataAccessHelper.getNumericColumnKeysOfRow > (keyspace, > CassandraConstants.META_DATA_COLUMN_FAMILY, storageQueueName, pageSize, > lastProcessedID); > > Initially the relevant message ids are retrieved from MetaData CF and then > removed from the MessageContent CF. Once more than 200,000 messages are > sent since the range query queries the entire row of data belonging to a > relevant topic, it results in the following, > > > > *Caused by: org.apache.cassandra.db.filter.TombstoneOverwhelmingException: > nullat > org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:214) > ~[apache-cassandra-2.1.2.jar:2.1.2]at > org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:107) > ~[apache-cassandra-2.1.2.jar:2.1.2]* > > And causes *me.prettyprint.hector.api.exceptions.HTimedOutException: > TimedOutException() *in the broker end, > > As solution, > > Since the idea of purging during disconnection is to clear out remaining > data which have not being given out to its subscribers, we could query the > slots which are still remaining and directly delete them from the > MessageContentCF instead of querying them from meta data CF. Would there be > any implication to this ? WDYT ? > > Thanks, > Pamod > > -- > *Pamod Sylvester * > *Senior Software Engineer * > Integration Technologies Team, WSO2 Inc.; http://wso2.com > email: pa...@wso2.com cell: +94 77 7779495 > -- Cheers, Hasitha Amal De Silva Software Engineer Mobile : 0772037426 Blog : http://devnutshell.tumblr.com/ WSO2 Inc.: http://wso2.com ( lean.enterprise.middleware. )
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
