> My understanding of the compaction process was that since data files keep > continuously merging we should not have data files with very old last > modified timestamps It is perfectly OK to have very old SSTables.
> But performing an upgradesstables did decrease the number of data files and > removed all the data files with the old timestamps. upgradetables re-writes every sstable to have the same contents in the newest format. Cheers ----------------- Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 19/11/2012, at 4:57 PM, Ananth Gundabattula <agundabatt...@gmail.com> wrote: > Hello Aaron, > > Thanks a lot for the reply. > > Looks like the documentation is confusing. Here is the link I am referring > to: http://www.datastax.com/docs/1.1/operations/tuning#tuning-compaction > > > > It does not disable compaction. > As per the above url, " After running a major compaction, automatic minor > compactions are no longer triggered, frequently requiring you to manually run > major compactions on a routine basis." ( Just before the heading Tuning > Column Family compression in the above link) > > With respect to the replies below : > > > > it creates one big file, which will not be compacted until there are (by > > default) 3 other very big files. > This is for the minor compaction and major compaction should theoretically > result in one large file irrespective of the number of data files initially? > > >This is not something you have to worry about. Unless you are seeing 1,000's > >of files using the default compaction. > > Well my worry has been because of the large amount of node movements we have > done in the ring. We started off with 6 nodes and increased the capacity to > 12 with disproportionate increases every time which resulted in a lot of > clean of data folders except system, run repair and then a cleanup with an > aborted attempt in between. > > There were some data.db files older by more than 2 weeks and were not > modified since then. My understanding of the compaction process was that > since data files keep continuously merging we should not have data files with > very old last modified timestamps (assuming there is a good amount of writes > to the table continuously) I did not have a for sure way of telling if > everything is alright with the compaction looking at the last modified > timestamps of all the data.db files. > > >What are the compaction issues you are having ? > Your replies confirm that the timestamps should not be an issue to worry > about. So I guess I should not be calling them as issues any more. But > performing an upgradesstables did decrease the number of data files and > removed all the data files with the old timestamps. > > > > Regards, > Ananth > > > On Mon, Nov 19, 2012 at 6:54 AM, aaron morton <aa...@thelastpickle.com> wrote: >> As per datastax documentation, a manual compaction forces the admin to start >> compaction manually and disables the automated compaction (atleast for major >> compactions but not minor compactions ) > It does not disable compaction. > it creates one big file, which will not be compacted until there are (by > default) 3 other very big files. > > >> 1. Does a nodetool stop compaction also force the admin to manually run >> major compaction ( I.e. disable automated major compactions ? ) > No. > Stop just stops the current compaction. > Nothing is disabled. > >> 2. Can a node restart reset the automated major compaction if a node gets >> into a manual mode compaction for whatever reason ? > Major compaction is not automatic. It is the manual nodetool compact command. > Automatic (minor) compaction is controlled by min_compaction_threshold and > max_compaction_threshold (for the default compaction strategy). > >> 3. What is the ideal number of SSTables for a table in a keyspace ( I mean >> are there any indicators as to whether my compaction is alright or not ? ) > This is not something you have to worry about. > Unless you are seeing 1,000's of files using the default compaction. > >> For example, I have seen SSTables on the disk more than 10 days old wherein >> there were other SSTables belonging to the same table but much younger than >> the older SSTables ( > No problems. > >> 4. Does a upgradesstables fix any compaction issues ? > What are the compaction issues you are having ? > > > Cheers > > ----------------- > Aaron Morton > Freelance Cassandra Developer > New Zealand > > @aaronmorton > http://www.thelastpickle.com > > On 18/11/2012, at 1:18 AM, Ananth Gundabattula <agundabatt...@gmail.com> > wrote: > >> >> We have a cluster running cassandra 1.1.4. On this cluster, >> >> 1. We had to move the nodes around a bit when we were adding new nodes >> (there was quite a good amount of node movement ) >> >> 2. We had to stop compactions during some of the days to save some disk >> space on some of the nodes when they were running very very low on disk >> spaces. (via nodetool stop COMPACTION) >> >> >> As per datastax documentation, a manual compaction forces the admin to start >> compaction manually and disables the automated compaction (atleast for major >> compactions but not minor compactions ) >> >> >> Here are the questions I have regarding compaction: >> >> 1. Does a nodetool stop compaction also force the admin to manually run >> major compaction ( I.e. disable automated major compactions ? ) >> >> 2. Can a node restart reset the automated major compaction if a node gets >> into a manual mode compaction for whatever reason ? >> >> 3. What is the ideal number of SSTables for a table in a keyspace ( I mean >> are there any indicators as to whether my compaction is alright or not ? ) >> . For example, I have seen SSTables on the disk more than 10 days old >> wherein there were other SSTables belonging to the same table but much >> younger than the older SSTables ( The node movement and repair and cleanup >> happened between the older SSTables and the new SSTables being >> touched/modified) >> >> 4. Does a upgradesstables fix any compaction issues ? >> >> Regards, >> Ananth > >
