[jira] [Updated] (CASSANDRA-5911) Commit logs are not removed after nodetool flush or nodetool drain

2014-10-20 Thread Aleksey Yeschenko (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-5911:
-
Fix Version/s: (was: 2.0.5)

> Commit logs are not removed after nodetool flush or nodetool drain
> --
>
> Key: CASSANDRA-5911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5911
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: J.B. Langston
>Assignee: Vijay
>Priority: Minor
> Attachments: 0001-5911-v2.patch, 0001-5911-v3.patch, 
> 0001-CASSANDRA-5911.patch, 6528_140171_knwmuqxe9bjv5re_system.log
>
>
> Commit logs are not removed after nodetool flush or nodetool drain. This can 
> lead to unnecessary commit log replay during startup.  I've reproduced this 
> on Apache Cassandra 1.2.8.  Usually this isn't much of an issue but on a 
> Solr-indexed column family in DSE, each replayed mutation has to be reindexed 
> which can make startup take a long time (on the order of 20-30 min).
> Reproduction follows:
> {code}
> jblangston:bin jblangston$ ./cassandra > /dev/null
> jblangston:bin jblangston$ ../tools/bin/cassandra-stress -n 2000 > 
> /dev/null
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool flush
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool drain
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ pkill java
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ ./cassandra -f | grep Replaying
>  INFO 10:03:42,915 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566772.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566773.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566774.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566775.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566776.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566777.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566778.log
>  INFO 10:03:42,922 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566772.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566773.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566774.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566775.log
>  INFO 10:03:43,912 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566776.log
>  INFO 10:03:43,912 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566777.log
>  INFO 10:03:43,912 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-13770

[jira] [Updated] (CASSANDRA-5911) Commit logs are not removed after nodetool flush or nodetool drain

2013-11-04 Thread Vijay (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vijay updated CASSANDRA-5911:
-

Attachment: 0001-5911-v3.patch

v3 fixes the activateNextArchiveSegment issue in v2. I had to modify the test 
cases to avoid initialization of the commit log before recover method is 
called. Hope thats ok.

> Commit logs are not removed after nodetool flush or nodetool drain
> --
>
> Key: CASSANDRA-5911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5911
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: J.B. Langston
>Assignee: Vijay
>Priority: Minor
> Fix For: 2.0.3
>
> Attachments: 0001-5911-v2.patch, 0001-5911-v3.patch, 
> 0001-CASSANDRA-5911.patch, 6528_140171_knwmuqxe9bjv5re_system.log
>
>
> Commit logs are not removed after nodetool flush or nodetool drain. This can 
> lead to unnecessary commit log replay during startup.  I've reproduced this 
> on Apache Cassandra 1.2.8.  Usually this isn't much of an issue but on a 
> Solr-indexed column family in DSE, each replayed mutation has to be reindexed 
> which can make startup take a long time (on the order of 20-30 min).
> Reproduction follows:
> {code}
> jblangston:bin jblangston$ ./cassandra > /dev/null
> jblangston:bin jblangston$ ../tools/bin/cassandra-stress -n 2000 > 
> /dev/null
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool flush
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool drain
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ pkill java
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ ./cassandra -f | grep Replaying
>  INFO 10:03:42,915 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566772.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566773.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566774.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566775.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566776.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566777.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566778.log
>  INFO 10:03:42,922 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566772.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566773.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566774.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566775.log
>  INFO 10:03:43,912 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566776.log
>  INFO

[jira] [Updated] (CASSANDRA-5911) Commit logs are not removed after nodetool flush or nodetool drain

2013-10-28 Thread Vijay (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vijay updated CASSANDRA-5911:
-

Attachment: 0001-5911-v2.patch

Attached patch has warn message and fix the test to do blocking wait till the 
new segment arrives. Also little more logic to makes sure if we really need to 
switch...

{code}
if (!activeSegment.isUnused() && activeSegment.id == 
context.segment)
{
if (allocator.numSegmentsAvailable() > 0 || 
allocator.createReserveSegments)
activateNextArchiveSegment();
else
logger.warn("no active commitlog to switch, additional 
mutations might be replayed if the node is restarted immediatly. See: 
CASSANDRA-5911");
}
{code}

> Commit logs are not removed after nodetool flush or nodetool drain
> --
>
> Key: CASSANDRA-5911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5911
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: J.B. Langston
>Assignee: Vijay
>Priority: Minor
> Fix For: 2.0.3
>
> Attachments: 0001-5911-v2.patch, 0001-CASSANDRA-5911.patch, 
> 6528_140171_knwmuqxe9bjv5re_system.log
>
>
> Commit logs are not removed after nodetool flush or nodetool drain. This can 
> lead to unnecessary commit log replay during startup.  I've reproduced this 
> on Apache Cassandra 1.2.8.  Usually this isn't much of an issue but on a 
> Solr-indexed column family in DSE, each replayed mutation has to be reindexed 
> which can make startup take a long time (on the order of 20-30 min).
> Reproduction follows:
> {code}
> jblangston:bin jblangston$ ./cassandra > /dev/null
> jblangston:bin jblangston$ ../tools/bin/cassandra-stress -n 2000 > 
> /dev/null
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool flush
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool drain
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ pkill java
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ ./cassandra -f | grep Replaying
>  INFO 10:03:42,915 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566772.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566773.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566774.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566775.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566776.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566777.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566778.log
>  INFO 10:03:42,922 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.

[jira] [Updated] (CASSANDRA-5911) Commit logs are not removed after nodetool flush or nodetool drain

2013-10-07 Thread Vijay (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vijay updated CASSANDRA-5911:
-

Attachment: 0001-CASSANDRA-5911.patch

Hi Jonathan, Yeah it will replay the active segment currently ( 2. above ).
Please see the attached which provides a alternative, we can just switch to a 
different segment before it is full on flush.

> Commit logs are not removed after nodetool flush or nodetool drain
> --
>
> Key: CASSANDRA-5911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5911
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: J.B. Langston
>Assignee: Vijay
>Priority: Minor
> Fix For: 2.0.2
>
> Attachments: 0001-CASSANDRA-5911.patch, 
> 6528_140171_knwmuqxe9bjv5re_system.log
>
>
> Commit logs are not removed after nodetool flush or nodetool drain. This can 
> lead to unnecessary commit log replay during startup.  I've reproduced this 
> on Apache Cassandra 1.2.8.  Usually this isn't much of an issue but on a 
> Solr-indexed column family in DSE, each replayed mutation has to be reindexed 
> which can make startup take a long time (on the order of 20-30 min).
> Reproduction follows:
> {code}
> jblangston:bin jblangston$ ./cassandra > /dev/null
> jblangston:bin jblangston$ ../tools/bin/cassandra-stress -n 2000 > 
> /dev/null
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool flush
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool drain
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ pkill java
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ ./cassandra -f | grep Replaying
>  INFO 10:03:42,915 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566772.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566773.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566774.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566775.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566776.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566777.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566778.log
>  INFO 10:03:42,922 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566772.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566773.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566774.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566775.log
>  INFO 10:03:43,912 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566776.log
>  INFO 10:03:4

[jira] [Updated] (CASSANDRA-5911) Commit logs are not removed after nodetool flush or nodetool drain

2013-10-07 Thread J.B. Langston (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

J.B. Langston updated CASSANDRA-5911:
-

Attachment: 6528_140171_knwmuqxe9bjv5re_system.log

Attached system.log showing commitlog replay.  This was produced by running 
stress against a single-node cassandra cluster, then running drain and 
restarting.

> Commit logs are not removed after nodetool flush or nodetool drain
> --
>
> Key: CASSANDRA-5911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5911
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: J.B. Langston
>Assignee: Vijay
>Priority: Minor
> Fix For: 2.0.2
>
> Attachments: 6528_140171_knwmuqxe9bjv5re_system.log
>
>
> Commit logs are not removed after nodetool flush or nodetool drain. This can 
> lead to unnecessary commit log replay during startup.  I've reproduced this 
> on Apache Cassandra 1.2.8.  Usually this isn't much of an issue but on a 
> Solr-indexed column family in DSE, each replayed mutation has to be reindexed 
> which can make startup take a long time (on the order of 20-30 min).
> Reproduction follows:
> {code}
> jblangston:bin jblangston$ ./cassandra > /dev/null
> jblangston:bin jblangston$ ../tools/bin/cassandra-stress -n 2000 > 
> /dev/null
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool flush
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool drain
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ pkill java
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ ./cassandra -f | grep Replaying
>  INFO 10:03:42,915 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566772.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566773.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566774.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566775.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566776.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566777.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566778.log
>  INFO 10:03:42,922 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566772.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566773.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566774.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566775.log
>  INFO 10:03:43,912 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566776.log
>  INFO 10:03:43,912 Replaying 
> /opt/apache-cassandra-

[jira] [Updated] (CASSANDRA-5911) Commit logs are not removed after nodetool flush or nodetool drain

2013-08-22 Thread Jonathan Ellis (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5911:
--

 Reviewer: jbellis
Fix Version/s: 2.0.1

> Commit logs are not removed after nodetool flush or nodetool drain
> --
>
> Key: CASSANDRA-5911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5911
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: J.B. Langston
>Assignee: Vijay
>Priority: Minor
> Fix For: 2.0.1
>
>
> Commit logs are not removed after nodetool flush or nodetool drain. This can 
> lead to unnecessary commit log replay during startup.  I've reproduced this 
> on Apache Cassandra 1.2.8.  Usually this isn't much of an issue but on a 
> Solr-indexed column family in DSE, each replayed mutation has to be reindexed 
> which can make startup take a long time (on the order of 20-30 min).
> Reproduction follows:
> {code}
> jblangston:bin jblangston$ ./cassandra > /dev/null
> jblangston:bin jblangston$ ../tools/bin/cassandra-stress -n 2000 > 
> /dev/null
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool flush
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ nodetool drain
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ pkill java
> jblangston:bin jblangston$ du -h ../commitlog
> 576M  ../commitlog
> jblangston:bin jblangston$ ./cassandra -f | grep Replaying
>  INFO 10:03:42,915 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566772.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566773.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566774.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566775.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566776.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566777.log, 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566778.log
>  INFO 10:03:42,922 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566761.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566762.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566763.log
>  INFO 10:03:43,907 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566764.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566765.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566766.log
>  INFO 10:03:43,908 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566767.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566768.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566769.log
>  INFO 10:03:43,909 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566770.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566771.log
>  INFO 10:03:43,910 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566772.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566773.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566774.log
>  INFO 10:03:43,911 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566775.log
>  INFO 10:03:43,912 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566776.log
>  INFO 10:03:43,912 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566777.log
>  INFO 10:03:43,912 Replaying 
> /opt/apache-cassandra-1.2.8/commitlog/CommitLog-2-1377096566778.log
> {code}

--
This message is automatically generated by JIRA.
If you think