I upgraded my instance from 8GB to a 14GB one. Allocated 8GB to jvm heap in cassandra-env.sh.
And now, it crashes even faster with an OOM.. Earlier, with 4GB heap, I could go upto ~90% replication completion (as reported by nodetool netstats); now, with 8GB heap, I cannot even get there. I've already restarted cassandra service 4 times with 8GB heap. No clue what's going on.. :( Kunal On 10 July 2015 at 17:45, Jack Krupansky <jack.krupan...@gmail.com> wrote: > You, and only you, are responsible for knowing your data and data model. > > If columns per row or rows per partition can be large, then an 8GB system > is probably too small. But the real issue is that you need to keep your > partition size from getting too large. > > Generally, an 8GB system is okay, but only for reasonably-sized > partitions, like under 10MB. > > > -- Jack Krupansky > > On Fri, Jul 10, 2015 at 8:05 AM, Kunal Gangakhedkar < > kgangakhed...@gmail.com> wrote: > >> I'm new to cassandra >> How do I find those out? - mainly, the partition params that you asked >> for. Others, I think I can figure out. >> >> We don't have any large objects/blobs in the column values - it's all >> textual, date-time, numeric and uuid data. >> >> We use cassandra to primarily store segmentation data - with segment type >> as partition key. That is again divided into two separate column families; >> but they have similar structure. >> >> Columns per row can be fairly large - each segment type as the row key >> and associated user ids and timestamp as column value. >> >> Thanks, >> Kunal >> >> On 10 July 2015 at 16:36, Jack Krupansky <jack.krupan...@gmail.com> >> wrote: >> >>> What does your data and data model look like - partition size, rows per >>> partition, number of columns per row, any large values/blobs in column >>> values? >>> >>> You could run fine on an 8GB system, but only if your rows and >>> partitions are reasonably small. Any large partitions could blow you away. >>> >>> -- Jack Krupansky >>> >>> On Fri, Jul 10, 2015 at 4:22 AM, Kunal Gangakhedkar < >>> kgangakhed...@gmail.com> wrote: >>> >>>> Attaching the stack dump captured from the last OOM. >>>> >>>> Kunal >>>> >>>> On 10 July 2015 at 13:32, Kunal Gangakhedkar <kgangakhed...@gmail.com> >>>> wrote: >>>> >>>>> Forgot to mention: the data size is not that big - it's barely 10GB in >>>>> all. >>>>> >>>>> Kunal >>>>> >>>>> On 10 July 2015 at 13:29, Kunal Gangakhedkar <kgangakhed...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I have a 2 node setup on Azure (east us region) running Ubuntu server >>>>>> 14.04LTS. >>>>>> Both nodes have 8GB RAM. >>>>>> >>>>>> One of the nodes (seed node) died with OOM - so, I am trying to add a >>>>>> replacement node with same configuration. >>>>>> >>>>>> The problem is this new node also keeps dying with OOM - I've >>>>>> restarted the cassandra service like 8-10 times hoping that it would >>>>>> finish >>>>>> the replication. But it didn't help. >>>>>> >>>>>> The one node that is still up is happily chugging along. >>>>>> All nodes have similar configuration - with libjna installed. >>>>>> >>>>>> Cassandra is installed from datastax's debian repo - pkg: dsc21 >>>>>> version 2.1.7. >>>>>> I started off with the default configuration - i.e. the default >>>>>> cassandra-env.sh - which calculates the heap size automatically (1/4 * >>>>>> RAM >>>>>> = 2GB) >>>>>> >>>>>> But, that didn't help. So, I then tried to increase the heap to 4GB >>>>>> manually and restarted. It still keeps crashing. >>>>>> >>>>>> Any clue as to why it's happening? >>>>>> >>>>>> Thanks, >>>>>> Kunal >>>>>> >>>>> >>>>> >>>> >>> >> >
