Hi Gaurav, I recommend you just run a MapReduce job for this computation.
Alternatively, you can look at the code for the C* MapReduce input format: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hadoop/cql3/CqlInputFormat.java That should give you what you need to iterate over independent token ranges. If you want, you can also just divide up the total token range for the partitioner you are using into equal chunks and have each of your threads execute a separate scan. Best regards, Clint On Tue, Feb 24, 2015 at 9:50 AM, Gaurav Bhatnagar <gauravb...@gmail.com> wrote: > Hi, > I have a cassandra cluster of 3 nodes holding around 300 million rows > of items. I have a replication factor of 3 with read/write consistency as > Quorum. I want to scan all rows of database to generate sum of items having > value "available" in column name state and value "batch1" in column name > batch. Row key for item is a 15 digit random number. > I want to do this processing in multiple threads for instance one > thread generating sum for one portion of data and other thread generating > sum for another disjoint portion of data and later I would add up total > from these 2 threads to get final sum. > What can be the possible way to achieve this? Can I use concept of > virtual nodes here. Each node owns set of virtual nodes. > Can I get data owned by a particular node and this way generate sum > on different nodes by iterating over data from virtual nodes and later > generate total sum by doing sum of data from all virtual nodes. > > Regards, > Gaurav >
