> How do you set the compaction threshold from storage-conf.xml? is this > possible?
Not in 0.6 I believe, but I believe this recently went into trunk for the upcoming 0.7. > What is the consensus on a basic Key-Value store of setting the > compactionthreshold min/max from > > ./nodetool --host=localhost --port=8181 getcompactionthreshold > Current compaction threshold: Min=4, Max=32 > > to say something like 2/5 ? I make no claims about consensus, but in general if compaction is affecting your node so adversely that it's effectively going offline for the duration, while tweaking those settings may affect e.g. frequency of compactions, it won't really fundamentally mitigate their effects when they do happen. I am curious about and other cases where people have sever problems with compaction. In particular, what kind of row/column sizes are we talking about? I would expect compaction to be a bigger killer for data sets where I/O becomes the bottleneck rather than CPU (which would tend to mean larger rows with fewer columns, meaning less CPU bound compaction). It would be interest to confirm whether this is the case. There is some ongoing work to mitigate the impact of compaction and it would be interesting to gather reports about the nature of the problems people are having with it. Have you observed 'iostat -x 1' or similar during compaction? Is the underlying storage maxed out? What kind of I/O load do you normally see due to regular traffic when compaction is not happening? -- / Peter Schuller
