+1 that's a good point. We could define a function to normalize the slot range through consolidating of all the remaining slots together.
On Thu, Mar 12, 2015 at 1:22 PM, Asitha Nanayakkara <asi...@wso2.com> wrote: > Hi Pamod > > Querying the remaining slots and removing content directly is OK. As an > improvement you can add a separate method in slot coordinator to get that > message range of all the slots, to reduce cluster communication, and then > do the necessary processing in the local node it self. > > Thanks, > Asitha > > On Thu, Mar 12, 2015 at 11:56 AM, Pamod Sylvester <pa...@wso2.com> wrote: > >> Hi All, >> >> During the subscription disconnection/deletion purge operation is being >> called on the the relevant non durable topic the subscription/s was bound >> to. During purge operation all existing data (if any) relevant to that >> topic will be removed. >> >> When using hector the removal is done through a range query following LOC, >> >> List<Long> currentPage = >> HectorDataAccessHelper.getNumericColumnKeysOfRow >> (keyspace, >> CassandraConstants.META_DATA_COLUMN_FAMILY, storageQueueName, pageSize, >> lastProcessedID); >> >> Initially the relevant message ids are retrieved from MetaData CF and >> then removed from the MessageContent CF. Once more than 200,000 messages >> are sent since the range query queries the entire row of data belonging to >> a relevant topic, it results in the following, >> >> >> >> *Caused by: >> org.apache.cassandra.db.filter.TombstoneOverwhelmingException: nullat >> org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:214) >> ~[apache-cassandra-2.1.2.jar:2.1.2]at >> org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:107) >> ~[apache-cassandra-2.1.2.jar:2.1.2]* >> >> And causes *me.prettyprint.hector.api.exceptions.HTimedOutException: >> TimedOutException() *in the broker end, >> >> As solution, >> >> Since the idea of purging during disconnection is to clear out remaining >> data which have not being given out to its subscribers, we could query the >> slots which are still remaining and directly delete them from the >> MessageContentCF instead of querying them from meta data CF. Would there be >> any implication to this ? WDYT ? >> >> Thanks, >> Pamod >> >> -- >> *Pamod Sylvester * >> *Senior Software Engineer * >> Integration Technologies Team, WSO2 Inc.; http://wso2.com >> email: pa...@wso2.com cell: +94 77 7779495 >> > > > > -- > *Asitha Nanayakkara* > Software Engineer > WSO2, Inc. http://wso2.com/ > Mob: + 94 77 85 30 682 > > -- *Pamod Sylvester * *Senior Software Engineer * Integration Technologies Team, WSO2 Inc.; http://wso2.com email: pa...@wso2.com cell: +94 77 7779495
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
