I think I didn't make myself clear. I don't have a problem with disk space. I have a problem with the data size. I have a simple crud application. Most of the requests are read but there are update/delete and when the time pass the number of deleted rows is big enough in order to free some disk space (a matter of days and not hours). Since not all of the data can fit to RAM (and I have a lot of RAM) the rest is served from disk. Since disk is slow I want to reduce as much as possible the number of requests that goes to the disk. The more requests to the disk, the disk wait time gets longer and it takes more time to return a response.
Bottom line is that I want to reduce the number of requests that goes to disk. Since there is enough data that is no longer valid I can do it by reclaiming the space. The only way to do it is by running Major compaction. I can wait and let Cassandra do it for me but then the data size will get even bigger and the response time will be worst. I can do it manually but I prefer it to happen in the background with less impact on the system Shimi On Tue, Jan 4, 2011 at 2:33 PM, Peter Schuller <peter.schul...@infidyne.com>wrote: > > This is what I thought. I was wishing there might be another way to > reclaim > > the space. > > Be sure you really need this first :) Normally you just let it happen in > the bg. > > > The problem is that the more data you have the more time it will take to > > Cassandra to response. > > Relative to what though? There are definitely important side-effects > of having very large data sets, and part of that involves compactions, > but in a normal steady state type of system you should never be in the > position to "wait" for a major compaction to run. Compactions are > something that is intended to run every now and then in the > background. It will result in variations in disk space within certain > bounds, which is expected. > > Certainly the situation can be improved and the current disk space > utilization situation is not perfect, but the above suggests to me > that you're trying to do something that is not really intended to be > done. > > > Reclaim space of deleted rows in the biggest SSTable requires Major > > compaction. This compaction can be triggered by adding x2 data (or x4 > data > > in the default configuration) to the system or by executing it manually > > using JMX. > > You can indeed choose to trigger major compactions by e.g. cron jobs. > But just be aware that if you're operating under conditions where you > are close to disk space running out, you have other concerns too - > such as periodic repair operations also needing disk space. > > Also; suppose you're overwriting lots of data (or replacing by > deleting and adding other data). It is not necessarily true that you > need 4x the space relative to what you otherwise do just because of > the compaction threshold. > > Keep in mind that compactions already need extra space anyway. If > you're *not* overwriting or adding data, a compaction of a single CF > is expected to need up to twice the amount of space that it occupies. > If you're doing more overwrites and deletions though, as you point out > you will have more "dead" data at any given point in time. But on the > other hand, the peak disk space usage during compactions is lower. So > the actual peak disk space usage (which is what matters since you must > have this much disk space) is actually helped by the > deletions/overwrites too. > > Further, suppose you trigger major compactions more often. That means > each compaction will have a higher relative spike of disk usage > because less data has had time to be overwritten or removed. > > So in a sense, it's like the disk space demands is being moved between > the category of "dead data retained for longer than necessary" and > "peak disk usage during compaction". > > Also keep in mind that the *low* peak of disk space usage is not > subject to any fragmentation concerns. Depending on the size of your > data compared to e.g. column names, that disk space usage might be > significantly lower than what you would get with an in-place updating > database. There are lots of trade-offs :) > > You say you have to "wait" for deletions though which sounds like > you're doing something unusual. Are you doing stuff like deleting lots > of data in bulk from one CF, only to then write data to *another* CF? > Such that you're actually having to wait for disk space to be freed to > make room for data somewhere else? > > > In case of a system that deletes data regularly, which needs to serve > > customers all day and the time it takes should be in ms, this is a > problem. > > Not in general. I am afraid there may be some misunderstanding here. > Unless disk space is a problem for you (i.e., you're running out of > space), there is no need to wait for compactions. And certainly > whether you can serve traffic 24/7 at low-ms latencies is an important > consideration, and does become complex when disk I/O is involved, but > it is not about disk *space*. If you have important performance > requirements, make sure you can service the read load at all given > your data set size. If you're runnning out of disk, I presume your > data is big. See > http://wiki.apache.org/cassandra/LargeDataSetConsiderations > > Perhaps if you can describe your situation in more detail? > > > It appears to me that in order to use Cassandra you must have a process > that > > will trigger major compaction on the nodes once in X amount of time. > > For some cases this will be beneficial, but not always. It's been > further improved for 0.7 too w.r.t. tomb stone handling in non-major > compactions (I don't have the JIRA ticket number handy). It's > certainly not a hard requirement and would only ever be relevant if > you're operating nodes that are significantly full. > > > One case where you would do that is when you don't (or hardly) delete > data. > > Or just in most cases where you don't push disk space concerns. > > > Another one is when your upper limit of time it should take to response > is > > very high so major compaction will not hurt you. > > To be really clear: Compaction is a background operation. It is never > the case that reads or writes somehow "wait" for compaction to > complete. > > -- > / Peter Schuller >
