Re: Forcing Cassandra to free up some space

2011-06-15 Thread Ryan King
starts with the easiest/fastest to find and then sees what more it >>> > thinks it needs to do to create enough memory for anticipated near >>> > future needs. >>> > >>> > On Thu, May 26, 2011 at 10:16 PM, Jonathan Ellis >>> >

Re: Forcing Cassandra to free up some space

2011-06-15 Thread Jeffrey Kesselman
rately done >> >> something like setting the -XX:-DisableExplicitGC flag. >> >> >> >> On Thu, May 26, 2011 at 5:58 PM, Konstantin  Naryshkin >> >> wrote: >> >>> So, in summary, there is no way to predictably and efficiently tell >>

Re: Forcing Cassandra to free up some space

2011-06-15 Thread Terje Marthinussen
> >> wrote: > >>> So, in summary, there is no way to predictably and efficiently tell > Cassandra to get rid of all of the extra space it is using on disk? > >>> > >>> - Original Message - > >>> From: "Jeffrey Kesselman"

Re: Forcing Cassandra to free up some space

2011-06-15 Thread Peter Schuller
> Even if the gc call cleaned all files, it is not really acceptable on a > decent sized cluster due to the impact full gc has on performance. > Especially non-needed ones. You can run with -XX:+ExplicitGCInvokesConcurrent to "safely" trigger CMS cycles. However that also means System.gc() semanti

Re: Forcing Cassandra to free up some space

2011-06-15 Thread Ryan King
There's a ticket open to address this: https://issues.apache.org/jira/browse/CASSANDRA-1974 -ryan On Wed, Jun 15, 2011 at 8:49 AM, Terje Marthinussen wrote: > > > On Thu, Jun 16, 2011 at 12:48 AM, Terje Marthinussen > wrote: >> >> Even if the gc call cleaned all files, it is not really accepta

Re: Forcing Cassandra to free up some space

2011-06-15 Thread AJ
tell Cassandra to get rid of all of the extra space it is using on disk? >>> >>> - Original Message - >>> From: "Jeffrey Kesselman" mailto:jef...@gmail.com>> >>> To: user@cassandra.apache.org <mailto:user@cassand

Re: Forcing Cassandra to free up some space

2011-06-15 Thread Terje Marthinussen
On Thu, Jun 16, 2011 at 12:48 AM, Terje Marthinussen < tmarthinus...@gmail.com> wrote: > Even if the gc call cleaned all files, it is not really acceptable on a > decent sized cluster due to the impact full gc has on performance. > Especially non-needed ones. > > Not acceptable as running GC on ev

Re: Forcing Cassandra to free up some space

2011-06-15 Thread Terje Marthinussen
aryshkin > >> wrote: > >>> So, in summary, there is no way to predictably and efficiently tell > Cassandra to get rid of all of the extra space it is using on disk? > >>> > >>> - Original Message - > >>> From: "Jeffrey Kesselman"

Re: Forcing Cassandra to free up some space

2011-06-15 Thread Shotaro Kamio
--- Original Message - >>> From: "Jeffrey Kesselman" >>> To: user@cassandra.apache.org >>> Sent: Thursday, May 26, 2011 8:57:49 PM >>> Subject: Re: Forcing Cassandra to free up some space >>> >>> Which JVM?  Which collector?  Ther

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jeremy Hanna
For the purposes of clearing out disk space, you might also occasionally check to see if you have snapshots that you no longer need. Certain operations create snapshots (point-in-time backups of sstables) in the (default) /var/lib/cassandra/data//snapshots directory. If you are absolutely sure

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jeffrey Kesselman
-- >> From: "Jeffrey Kesselman" >> To: user@cassandra.apache.org >> Sent: Thursday, May 26, 2011 8:57:49 PM >> Subject: Re: Forcing Cassandra to free up some space >> >> Which JVM?  Which collector?  There have been and continue to be many. >> &g

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jeffrey Kesselman
sing on disk? >> >> - Original Message - >> From: "Jeffrey Kesselman" >> To: user@cassandra.apache.org >> Sent: Thursday, May 26, 2011 8:57:49 PM >> Subject: Re: Forcing Cassandra to free up some space >> >> Which JVM?  Which coll

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jonathan Ellis
f the extra space it is using on disk? > > - Original Message - > From: "Jeffrey Kesselman" > To: user@cassandra.apache.org > Sent: Thursday, May 26, 2011 8:57:49 PM > Subject: Re: Forcing Cassandra to free up some space > > Which JVM?  Which collec

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jake Luciani
"Is there a way for me to make (or even gently suggest to) Cassandra that it may be a good time to free up some space?" Disregarding what's been said and until ref-counting is implemented this is a useful tool to gently suggest cleanup: https://github.com/ceocoder/jmxgc On Thu, May 26, 2011 at

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jeffrey Kesselman
to get rid of all of the extra space it is using on disk? > > - Original Message - > From: "Jeffrey Kesselman" > To: user@cassandra.apache.org > Sent: Thursday, May 26, 2011 8:57:49 PM > Subject: Re: Forcing Cassandra to free up some space > > Which JVM

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Konstantin Naryshkin
ssandra to free up some space Which JVM? Which collector? There have been and continue to be many. Hotspot itself supports a number of different collectors with different behaviors. Many of them do not collect every candidate on every gc, but merely the easiest ones to find. This is why de

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jeffrey Kesselman
Which JVM? Which collector? There have been and continue to be many. Hotspot itself supports a number of different collectors with different behaviors. Many of them do not collect every candidate on every gc, but merely the easiest ones to find. This is why depending on finalizers is a *bad*

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jonathan Ellis
I've read the relevant source. While you're pedantically correct re the spec, you're wrong as to what the JVM actually does. On Thu, May 26, 2011 at 3:14 PM, Jeffrey Kesselman wrote: > Some references... > > "An object enters an unreachable state when no more strong references > to it exist. When

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jeffrey Kesselman
Some references... "An object enters an unreachable state when no more strong references to it exist. When an object is unreachable, it is a candidate for collection. Note the wording: Just because an object is a candidate for collection doesn't mean it will be immediately collected. The JVM is fr

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jeffrey Kesselman
Im sorry. This was my business at Sun. You are certainly wrong about the Hotspot VM. See this chapter of my book http://java.sun.com/docs/books/performance/1st_edition/html/JPAppGC.fm.html#998394 On Thu, May 26, 2011 at 3:53 PM, Jonathan Ellis wrote: > It's a common misunderstanding that

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jonathan Ellis
It's a common misunderstanding that system.gc is only a suggestion; on any VM you're likely to run Cassandra on, System.gc will actually invoke a full collection. On Thu, May 26, 2011 at 2:18 PM, Jeffrey Kesselman wrote: > Actually this is no gaurantee.   Its a common misunderstanding that > Syst

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jeffrey Kesselman
Actually this is no gaurantee. Its a common misunderstanding that System.gc "forces" gc. It does not. It is a suggestion only. The vm always has the option as to when and how much it gcs On May 26, 2011 2:51 PM, "Jonathan Ellis" wrote:

Re: Forcing Cassandra to free up some space

2011-05-26 Thread Jonathan Ellis
You'd have to call system.gc via JMX. https://issues.apache.org/jira/browse/CASSANDRA-2521 is open to address this, btw. On Thu, May 26, 2011 at 1:09 PM, Konstantin Naryshkin wrote: > I have a basic understanding of how Cassandra handles the file system > (flushes in Memtables out to SSTables,

Forcing Cassandra to free up some space

2011-05-26 Thread Konstantin Naryshkin
I have a basic understanding of how Cassandra handles the file system (flushes in Memtables out to SSTables, SSTables get compacted) and I understand that old files are only deleted when a node is restarted, when Java does a GC, or when Cassandra feels like it is running out of space. My quest