Thanks all, Yes it was the issue Inosh mentioned.
I had to support "pagination" in client side using hector. ( by reading records as batches). On Tue, Sep 1, 2015 at 4:12 PM, Inosh Goonewardena <in...@wso2.com> wrote: > Hi Hemika/Madhawa, > > I don't think this is related to tombstones or gc_grace. I suppose this > happens because Cassandra 1.x version doesn't support streaming when > reading larger resultsets using CQL queries. > > Hi Sachith, > > I think you might have to use hector for do this [1]. You can refer > Cassandra Exlporer component and Hive Cassandra Handler [3] > > [1] > http://stackoverflow.com/questions/8418448/cassandra-hector-how-to-retrieve-all-rows-of-a-column-family > [2] > https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/components/cassandra-explorer/org.wso2.carbon.cassandra.explorer.mgt/4.2.2 > [3] > https://svn.wso2.org/repos/wso2/carbon/platform/branches/turing/dependencies/hive/0.8.1-wso2v10/cassandra-handler/src/java/org/apache/hadoop/hive/cassandra/input > > On Tue, Sep 1, 2015 at 11:40 AM, Hemika Kodikara <hem...@wso2.com> wrote: > >> Hi Sachith, >> >> If a heap-dump was created during OOM, you can dissect it using Eclipse >> MAT of JProfiler and see what causing. >> >> Regards, >> Hemika >> >> Hemika Kodikara >> Software Engineer >> WSO2 Inc. >> lean . enterprise . middleware >> http://wso2.com >> >> Mobile : +94777688882 >> >> On Tue, Sep 1, 2015 at 3:07 PM, Sachith Withana <sach...@wso2.com> wrote: >> >>> It actually happens even after running the query immediately after >>> inserting the data. >>> >>> It's just that I need to have a way to get the data by chunks rather >>> than the whole thing at once which causes the Out of memory issue. >>> >>> On Tue, Sep 1, 2015 at 9:31 AM, Hemika Kodikara <hem...@wso2.com> wrote: >>> >>>> Hi Sachith, >>>> >>>> Could you try running the same query in a 2.x version and see what >>>> error is occurring ? As far as I know, it could be a tombstone issue, 1.x >>>> versions doesnt show it. If it gives an error of tombstone overwhelming in >>>> 2.x, then its the case what Madhawa has explained. >>>> >>>> You manually can get rid of tombstones using nodetool by running flush >>>> command and then compaction. >>>> >>>> Regards, >>>> Hemika >>>> >>>> Hemika Kodikara >>>> Software Engineer >>>> WSO2 Inc. >>>> lean . enterprise . middleware >>>> http://wso2.com >>>> >>>> Mobile : +94777688882 >>>> >>>> On Tue, Sep 1, 2015 at 9:26 AM, Madhawa Gunasekara <madha...@wso2.com> >>>> wrote: >>>> >>>>> Hi Sachith, >>>>> >>>>> I'm not sure about this is the correct answer, but you can configure >>>>> gc_grace_seconds. It will help you to skip the OOM exception. >>>>> [1] http://docs.datastax.com/en/cql/3.0/cql/cql_reference/tabProp.html >>>>> >>>>> Thanks, >>>>> Madhawa >>>>> >>>>> On Mon, Aug 31, 2015 at 8:36 AM, Sachith Withana <sach...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> I need to read a whole column family from a previous version of >>>>>> Cassandra (1.x). It works for a small result set ( for a CQL select * >>>>>> query). But for a larger dataset ( around 1 million records) it throws >>>>>> an >>>>>> out of memory exception (GC). >>>>>> >>>>>> Are there any mechanisms like paging for Cassandra 1.x? Or any other >>>>>> workaround for this issue? >>>>>> >>>>>> I observed that paging is supported from Cassandra 2.0 and above but >>>>>> not in the previous versions. >>>>>> >>>>>> >>>>>> Thanks. >>>>>> -- >>>>>> Sachith Withana >>>>>> Software Engineer; WSO2 Inc.; http://wso2.com >>>>>> E-mail: sachith AT wso2.com >>>>>> M: +94715518127 >>>>>> Linked-In: <http://goog_416592669> >>>>>> https://lk.linkedin.com/in/sachithwithana >>>>>> >>>>>> _______________________________________________ >>>>>> Dev mailing list >>>>>> Dev@wso2.org >>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Madhawa Gunasekara* >>>>> Software Engineer >>>>> WSO2 Inc.; http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> mobile: +94 719411002 <+94+719411002> >>>>> blog: *http://madhawa-gunasekara.blogspot.com >>>>> <http://madhawa-gunasekara.blogspot.com>* >>>>> linkedin: *http://lk.linkedin.com/in/mgunasekara >>>>> <http://lk.linkedin.com/in/mgunasekara>* >>>>> >>>>> _______________________________________________ >>>>> Dev mailing list >>>>> Dev@wso2.org >>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>> >>>>> >>>> >>> >>> >>> -- >>> Sachith Withana >>> Software Engineer; WSO2 Inc.; http://wso2.com >>> E-mail: sachith AT wso2.com >>> M: +94715518127 >>> Linked-In: <http://goog_416592669> >>> https://lk.linkedin.com/in/sachithwithana >>> >> >> >> _______________________________________________ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Thanks & Regards, > > Inosh Goonewardena > Associate Technical Lead- WSO2 Inc. > Mobile: +94779966317 > -- Sachith Withana Software Engineer; WSO2 Inc.; http://wso2.com E-mail: sachith AT wso2.com M: +94715518127 Linked-In: <http://goog_416592669>https://lk.linkedin.com/in/sachithwithana
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
