[jira] [Updated] (CASSANDRA-11716) cassandra 2.2 fails to start on jdk7u101
[ https://issues.apache.org/jira/browse/CASSANDRA-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tiago Batista updated CASSANDRA-11716: -- Status: Open (was: Patch Available) > cassandra 2.2 fails to start on jdk7u101 > > > Key: CASSANDRA-11716 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11716 > Project: Cassandra > Issue Type: Bug > Environment: ubuntu 12.04 fully updated >Reporter: Tiago Batista > > Today I updated one of my clusters to 2.2.6, and was greeted with the message > complaining about the jdk version: > {code} > $ nodetool status > Cassandra 2.0 and later require Java 7u25 or later. > {code} > After digging into it, on cassandra-env.sh, i found that you are comparing > the patch levels as strings, meaning that "101" is before "25": > {code} > if [ "$JVM_VERSION" \< "1.8" ] && [ "$JVM_PATCH_VERSION" \< "25" ] ; then > echo "Cassandra 2.0 and later require Java 7u25 or later." > exit 1; > fi > {code} > I patched this on my system to > {code} > if [ "$JVM_VERSION" \< "1.8" ] && [ "$JVM_PATCH_VERSION" -lt "25" ] ; then > echo "Cassandra 2.0 and later require Java 7u25 or later." > exit 1; > fi > {code} > this seems to work on bash. I can now start cassandra successfully on that > cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11716) cassandra 2.2 fails to start on jdk7u101
[ https://issues.apache.org/jira/browse/CASSANDRA-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tiago Batista updated CASSANDRA-11716: -- Status: Patch Available (was: Open) > cassandra 2.2 fails to start on jdk7u101 > > > Key: CASSANDRA-11716 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11716 > Project: Cassandra > Issue Type: Bug > Environment: ubuntu 12.04 fully updated >Reporter: Tiago Batista > > Today I updated one of my clusters to 2.2.6, and was greeted with the message > complaining about the jdk version: > {code} > $ nodetool status > Cassandra 2.0 and later require Java 7u25 or later. > {code} > After digging into it, on cassandra-env.sh, i found that you are comparing > the patch levels as strings, meaning that "101" is before "25": > {code} > if [ "$JVM_VERSION" \< "1.8" ] && [ "$JVM_PATCH_VERSION" \< "25" ] ; then > echo "Cassandra 2.0 and later require Java 7u25 or later." > exit 1; > fi > {code} > I patched this on my system to > {code} > if [ "$JVM_VERSION" \< "1.8" ] && [ "$JVM_PATCH_VERSION" -lt "25" ] ; then > echo "Cassandra 2.0 and later require Java 7u25 or later." > exit 1; > fi > {code} > this seems to work on bash. I can now start cassandra successfully on that > cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11716) cassandra 2.2 fails to start on jdk7u101
Tiago Batista created CASSANDRA-11716: - Summary: cassandra 2.2 fails to start on jdk7u101 Key: CASSANDRA-11716 URL: https://issues.apache.org/jira/browse/CASSANDRA-11716 Project: Cassandra Issue Type: Bug Environment: ubuntu 12.04 fully updated Reporter: Tiago Batista Today I updated one of my clusters to 2.2.6, and was greeted with the message complaining about the jdk version: {code} $ nodetool status Cassandra 2.0 and later require Java 7u25 or later. {code} After digging into it, on cassandra-env.sh, i found that you are comparing the patch levels as strings, meaning that "101" is before "25": {code} if [ "$JVM_VERSION" \< "1.8" ] && [ "$JVM_PATCH_VERSION" \< "25" ] ; then echo "Cassandra 2.0 and later require Java 7u25 or later." exit 1; fi {code} I patched this on my system to {code} if [ "$JVM_VERSION" \< "1.8" ] && [ "$JVM_PATCH_VERSION" -lt "25" ] ; then echo "Cassandra 2.0 and later require Java 7u25 or later." exit 1; fi {code} this seems to work on bash. I can now start cassandra successfully on that cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (CASSANDRA-11713) Add ability to log thread dump when NTR pool is blocked
[ https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-11713: --- Comment: was deleted (was: I know this sounds weird, but we've seen clusters that actually LOOK like they're behaving DESPITE the high NTR blocks, and I suspect a lot of unsuspecting users who would have been happy with that type of situation would be hurt severely by this patch because the log output in those clusters would be significant - for example, we've had 2-3 hour long stress runs where individual nodes have 7 digit ntr-blocked counters, can you imagine dumping threads to logs a million times an hour?. At the very least, I'd personally like to see it guarded with a system property or log spam protection. As an operator, I'd strongly suggest we not commit this patch until one of those were available.) > Add ability to log thread dump when NTR pool is blocked > --- > > Key: CASSANDRA-11713 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11713 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Paulo Motta >Assignee: Paulo Motta >Priority: Minor > > Thread dumps are very useful for troubleshooting Native-Transport-Requests > contention issues like CASSANDRA-11363 and CASSANDRA-11529. > While they could be generated externally with {{jstack}}, sometimes the > conditions are transient and it's hard to catch the exact moment when they > happen, so it could be useful to generate and log them upon user request when > certain internal condition happens. > I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} > that when enabled via JMX generates and logs a single thread dump on the > system log when the thread pool queue is full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11713) Add ability to log thread dump when NTR pool is blocked
[ https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271910#comment-15271910 ] Jeff Jirsa commented on CASSANDRA-11713: I know this sounds weird, but we've seen clusters that actually LOOK like they're behaving DESPITE the high NTR blocks, and I suspect a lot of unsuspecting users who would have been happy with that type of situation would be hurt severely by this patch because the log output in those clusters would be significant - for example, we've had 2-3 hour long stress runs where individual nodes have 7 digit ntr-blocked counters, can you imagine dumping threads to logs a million times an hour?. At the very least, I'd personally like to see it guarded with a system property or log spam protection. As an operator, I'd strongly suggest we not commit this patch until one of those were available. > Add ability to log thread dump when NTR pool is blocked > --- > > Key: CASSANDRA-11713 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11713 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Paulo Motta >Assignee: Paulo Motta >Priority: Minor > > Thread dumps are very useful for troubleshooting Native-Transport-Requests > contention issues like CASSANDRA-11363 and CASSANDRA-11529. > While they could be generated externally with {{jstack}}, sometimes the > conditions are transient and it's hard to catch the exact moment when they > happen, so it could be useful to generate and log them upon user request when > certain internal condition happens. > I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} > that when enabled via JMX generates and logs a single thread dump on the > system log when the thread pool queue is full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11715) Make GCInspector's MIN_LOG_DURATION configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-11715: - Description: It's common for people to run C* with the G1 collector on appropriately-sized heaps. Quite often, the target pause time is set to 500ms, but GCI fires on anything over 200ms. We can already control the warn threshold, but these are acceptable GCs for the configuration and create noise at the INFO log level. (was: It's common for people to run C* with the G1 collector on appropriately-sized heaps. Quite often, the target pause time is set to 500ms, but GCI fires on anything over 200ms. We can already control the warn threshold, but these are acceptable GCs for the configuring and create noise at the INFO log level.) > Make GCInspector's MIN_LOG_DURATION configurable > > > Key: CASSANDRA-11715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11715 > Project: Cassandra > Issue Type: Improvement >Reporter: Brandon Williams > Labels: lhf > > It's common for people to run C* with the G1 collector on appropriately-sized > heaps. Quite often, the target pause time is set to 500ms, but GCI fires on > anything over 200ms. We can already control the warn threshold, but these > are acceptable GCs for the configuration and create noise at the INFO log > level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11715) Make GCInspector's MIN_LOG_DURATION configurable
Brandon Williams created CASSANDRA-11715: Summary: Make GCInspector's MIN_LOG_DURATION configurable Key: CASSANDRA-11715 URL: https://issues.apache.org/jira/browse/CASSANDRA-11715 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams It's common for people to run C* with the G1 collector on appropriately-sized heaps. Quite often, the target pause time is set to 500ms, but GCI fires on anything over 200ms. We can already control the warn threshold, but these are acceptable GCs for the configuring and create noise at the INFO log level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11715) Make GCInspector's MIN_LOG_DURATION configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-11715: - Labels: lhf (was: ) > Make GCInspector's MIN_LOG_DURATION configurable > > > Key: CASSANDRA-11715 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11715 > Project: Cassandra > Issue Type: Improvement >Reporter: Brandon Williams > Labels: lhf > > It's common for people to run C* with the G1 collector on appropriately-sized > heaps. Quite often, the target pause time is set to 500ms, but GCI fires on > anything over 200ms. We can already control the warn threshold, but these > are acceptable GCs for the configuring and create noise at the INFO log level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11714) shutdown script should wait more than 30 seconds before force kill
Wei Deng created CASSANDRA-11714: Summary: shutdown script should wait more than 30 seconds before force kill Key: CASSANDRA-11714 URL: https://issues.apache.org/jira/browse/CASSANDRA-11714 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Wei Deng Priority: Minor I'm running with 3.0.6 and it appears that if I let it run for multiple days (ingesting a never-ending streaming data from Kafka), then the next time I'm restarting the JVM by "service cassandra restart", it will take 9 minutes to replay the commit log segments (the commitlog_total_space_in_mb is using the default 8GB and I observed close to 8GB commitlog segment files before the restart). Here is a fragment of the system.log showing how long it takes: {noformat} INFO [main] 2016-05-04 19:06:54,356 CommitLog.java:168 - Replaying /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248187.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248191.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248195.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248198.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248201.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248205.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248209.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248213.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248217.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248221.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248225.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248230.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248234.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248238.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248242.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248246.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248250.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248254.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248258.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248262.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248266.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248270.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248274.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248278.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248282.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248286.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248290.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248294.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248299.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248303.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248307.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248311.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248315.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248319.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248323.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248327.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248331.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248335.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248340.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248344.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248348.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248352.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248356.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248360.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248364.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248368.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248372.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248376.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248380.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248384.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248388.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248392.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248396.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248397.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248401.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248405.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248409.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248413.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248417.log, /mnt/ephemeral/cassandra/commitlog/CommitLog-6-1462146248421.log, /mnt
[jira] [Commented] (CASSANDRA-11279) dtest failure in disk_balance_test.TestDiskBalance.disk_balance_bootstrap_test
[ https://issues.apache.org/jira/browse/CASSANDRA-11279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271726#comment-15271726 ] Paulo Motta commented on CASSANDRA-11279: - Should we maybe keep the asserts while unsetting {{allocate_tokens_for_keyspace}} to test that disks are balanced for the random allocation bootstrap case? We can extend these after CASSANDRA-11643 to include the balanced bootstrap case. > dtest failure in disk_balance_test.TestDiskBalance.disk_balance_bootstrap_test > -- > > Key: CASSANDRA-11279 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11279 > Project: Cassandra > Issue Type: Test >Reporter: Russ Hatch >Assignee: Marcus Eriksson > Labels: dtest > > example failure: > http://cassci.datastax.com/job/trunk_dtest/1011/testReport/disk_balance_test/TestDiskBalance/disk_balance_bootstrap_test > Failed on CassCI build trunk_dtest #1011 > This looks likely to be a test issue, perhaps we need to relax the assertion > here a bit: > {noformat} > values not within 20.00% of the max: (474650, 382235, 513385) (node1) > {noformat} > This is flaky with several flaps in the last few weeks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11626) cqlsh fails and exits on non-ascii chars
[ https://issues.apache.org/jira/browse/CASSANDRA-11626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271612#comment-15271612 ] Paulo Motta commented on CASSANDRA-11626: - LGTM (tested with latin chars), let's hope this is our last juggling with encoding on cqlsh. > cqlsh fails and exits on non-ascii chars > > > Key: CASSANDRA-11626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11626 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Robert Stupp >Assignee: Tyler Hobbs >Priority: Minor > Labels: cqlsh > Fix For: 2.2.x, 3.0.x, 3.x > > > Just seen on cqlsh on current trunk: > To repro, copy {{ä}} (german umlaut) to cqlsh and press return. > cqlsh errors out and immediately exits. > {code} > $ bin/cqlsh > Connected to Test Cluster at 127.0.0.1:9042. > [cqlsh 5.0.1 | Cassandra 2.1.13-SNAPSHOT | CQL spec 3.2.1 | Native protocol > v3] > Use HELP for help. > cqlsh> ä > Invalid syntax at line 1, char 1 > Traceback (most recent call last): > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2636, in > > main(*read_options(sys.argv[1:], os.environ)) > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2625, in main > shell.cmdloop() > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 1114, in > cmdloop > if self.onecmd(self.statement.getvalue()): > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 1139, in onecmd > self.printerr(' %s' % statementline) > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2314, in > printerr > self.writeresult(text, color, newline=newline, out=sys.stderr) > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2303, in > writeresult > out.write(self.applycolor(str(text), color) + ('\n' if newline else '')) > UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position > 2: ordinal not in range(128) > $ > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11626) cqlsh fails and exits on non-ascii chars
[ https://issues.apache.org/jira/browse/CASSANDRA-11626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-11626: Status: Ready to Commit (was: Patch Available) > cqlsh fails and exits on non-ascii chars > > > Key: CASSANDRA-11626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11626 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Robert Stupp >Assignee: Tyler Hobbs >Priority: Minor > Labels: cqlsh > Fix For: 2.2.x, 3.0.x, 3.x > > > Just seen on cqlsh on current trunk: > To repro, copy {{ä}} (german umlaut) to cqlsh and press return. > cqlsh errors out and immediately exits. > {code} > $ bin/cqlsh > Connected to Test Cluster at 127.0.0.1:9042. > [cqlsh 5.0.1 | Cassandra 2.1.13-SNAPSHOT | CQL spec 3.2.1 | Native protocol > v3] > Use HELP for help. > cqlsh> ä > Invalid syntax at line 1, char 1 > Traceback (most recent call last): > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2636, in > > main(*read_options(sys.argv[1:], os.environ)) > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2625, in main > shell.cmdloop() > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 1114, in > cmdloop > if self.onecmd(self.statement.getvalue()): > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 1139, in onecmd > self.printerr(' %s' % statementline) > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2314, in > printerr > self.writeresult(text, color, newline=newline, out=sys.stderr) > File "/Users/snazy/devel/cassandra/trunk/bin/cqlsh.py", line 2303, in > writeresult > out.write(self.applycolor(str(text), color) + ('\n' if newline else '')) > UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position > 2: ordinal not in range(128) > $ > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11713) Add ability to log thread dump when NTR pool is blocked
[ https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-11713: Description: Thread dumps are very useful for troubleshooting Native-Transport-Requests contention issues like CASSANDRA-11363 and CASSANDRA-11529. While they could be generated externally with {{jstack}}, sometimes the conditions are transient and it's hard to catch the exact moment when they happen, so it could be useful to generate and log them upon user request when certain internal condition happens. I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} that when enabled via JMX generates and logs a single thread dump on the system log when the thread pool queue is full. was: Thread dumps are very useful for troubleshooting thread pool contention issues like CASSANDRA-11363 and CASSANDRA-11529. While they could be generated externally with {{jstack}}, sometimes the conditions are transient and it's hard to catch the exact moment when they happen, so it could be useful to generate and log them upon user request when certain internal condition happens. I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} and {{JMXEnabledThreadPoolExecutor}} that when enabled via JMX generates and logs a single thread dump on the system log when the thread pool queue is full. Summary: Add ability to log thread dump when NTR pool is blocked (was: Add ability to log thread dump when thread pool is full/blocked) > Add ability to log thread dump when NTR pool is blocked > --- > > Key: CASSANDRA-11713 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11713 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Paulo Motta >Assignee: Paulo Motta >Priority: Minor > > Thread dumps are very useful for troubleshooting Native-Transport-Requests > contention issues like CASSANDRA-11363 and CASSANDRA-11529. > While they could be generated externally with {{jstack}}, sometimes the > conditions are transient and it's hard to catch the exact moment when they > happen, so it could be useful to generate and log them upon user request when > certain internal condition happens. > I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} > that when enabled via JMX generates and logs a single thread dump on the > system log when the thread pool queue is full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11713) Add ability to log thread dump when thread pool is full/blocked
[ https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271568#comment-15271568 ] Paulo Motta commented on CASSANDRA-11713: - That's right, thanks for pointing this out. I initially added that only to {{SEPExecutor}} with the objective of troubleshooting {{Native-Transport-Requests}}, but then thought it could be useful for other thread pools too without noticing their "unboundedness". :-) Updated patch to only add this ability only to {{SEPExecutor}}. > Add ability to log thread dump when thread pool is full/blocked > --- > > Key: CASSANDRA-11713 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11713 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Paulo Motta >Assignee: Paulo Motta >Priority: Minor > > Thread dumps are very useful for troubleshooting thread pool contention > issues like CASSANDRA-11363 and CASSANDRA-11529. > While they could be generated externally with {{jstack}}, sometimes the > conditions are transient and it's hard to catch the exact moment when they > happen, so it could be useful to generate and log them upon user request when > certain internal condition happens. > I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} > and {{JMXEnabledThreadPoolExecutor}} that when enabled via JMX generates and > logs a single thread dump on the system log when the thread pool queue is > full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11713) Add ability to log thread dump when thread pool is full/blocked
[ https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271545#comment-15271545 ] Chris Lohfink commented on CASSANDRA-11713: --- Probably worth mentioning that since 2.1 or so the only thread pool where its actually really possible to be in a blocked state is the {{Native-Transport-Requests}} one. The queues are basically unbounded (MAX_INT) so they will all OOM before the rejection policy gets kicked > Add ability to log thread dump when thread pool is full/blocked > --- > > Key: CASSANDRA-11713 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11713 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Paulo Motta >Assignee: Paulo Motta >Priority: Minor > > Thread dumps are very useful for troubleshooting thread pool contention > issues like CASSANDRA-11363 and CASSANDRA-11529. > While they could be generated externally with {{jstack}}, sometimes the > conditions are transient and it's hard to catch the exact moment when they > happen, so it could be useful to generate and log them upon user request when > certain internal condition happens. > I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} > and {{JMXEnabledThreadPoolExecutor}} that when enabled via JMX generates and > logs a single thread dump on the system log when the thread pool queue is > full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11363) Blocked NTR When Connecting Causing Excessive Load
[ https://issues.apache.org/jira/browse/CASSANDRA-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-11363: Reproduced In: 3.0.3, 2.1.13, 2.1.12 (was: 2.1.12, 2.1.13, 3.0.3) Priority: Major (was: Critical) > Blocked NTR When Connecting Causing Excessive Load > -- > > Key: CASSANDRA-11363 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11363 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Russell Bradberry >Assignee: Paulo Motta > Attachments: cassandra-102-cms.stack, cassandra-102-g1gc.stack > > > When upgrading from 2.1.9 to 2.1.13, we are witnessing an issue where the > machine load increases to very high levels (> 120 on an 8 core machine) and > native transport requests get blocked in tpstats. > I was able to reproduce this in both CMS and G1GC as well as on JVM 7 and 8. > The issue does not seem to affect the nodes running 2.1.9. > The issue seems to coincide with the number of connections OR the number of > total requests being processed at a given time (as the latter increases with > the former in our system) > Currently there is between 600 and 800 client connections on each machine and > each machine is handling roughly 2000-3000 client requests per second. > Disabling the binary protocol fixes the issue for this node but isn't a > viable option cluster-wide. > Here is the output from tpstats: > {code} > Pool NameActive Pending Completed Blocked All > time blocked > MutationStage 0 88387821 0 > 0 > ReadStage 0 0 355860 0 > 0 > RequestResponseStage 0 72532457 0 > 0 > ReadRepairStage 0 0150 0 > 0 > CounterMutationStage 32 104 897560 0 > 0 > MiscStage 0 0 0 0 > 0 > HintedHandoff 0 0 65 0 > 0 > GossipStage 0 0 2338 0 > 0 > CacheCleanupExecutor 0 0 0 0 > 0 > InternalResponseStage 0 0 0 0 > 0 > CommitLogArchiver 0 0 0 0 > 0 > CompactionExecutor2 190474 0 > 0 > ValidationExecutor0 0 0 0 > 0 > MigrationStage0 0 10 0 > 0 > AntiEntropyStage 0 0 0 0 > 0 > PendingRangeCalculator0 0310 0 > 0 > Sampler 0 0 0 0 > 0 > MemtableFlushWriter 110 94 0 > 0 > MemtablePostFlush 134257 0 > 0 > MemtableReclaimMemory 0 0 94 0 > 0 > Native-Transport-Requests 128 156 38795716 > 278451 > Message type Dropped > READ 0 > RANGE_SLICE 0 > _TRACE 0 > MUTATION 0 > COUNTER_MUTATION 0 > BINARY 0 > REQUEST_RESPONSE 0 > PAGED_RANGE 0 > READ_REPAIR 0 > {code} > Attached is the jstack output for both CMS and G1GC. > Flight recordings are here: > https://s3.amazonaws.com/simple-logs/cassandra-102-cms.jfr > https://s3.amazonaws.com/simple-logs/cassandra-102-g1gc.jfr > It is interesting to note that while the flight recording was taking place, > the load on the machine went back to healthy, and when the flight recording > finished the load went back to > 100. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11363) Blocked NTR When Connecting Causing Excessive Load
[ https://issues.apache.org/jira/browse/CASSANDRA-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271519#comment-15271519 ] Paulo Motta commented on CASSANDRA-11363: - Tried reproducing this in a variety of workloads without success in the following environment: * 3 nodes m3.xlarge (m3.xlarge, 4 vcpu, 15GB RAM, 2 x 40) * 8GB Heap, CMS * 100M keys (24GB per node) * C* 3.0.3 with default settings and a variation with native_transport_max_threads=10 * no-vnodes The workloads were based a variation of https://raw.githubusercontent.com/mesosphere/cassandra-mesos/master/driver-extensions/cluster-loadtest/cqlstress-example.yaml with 100M keys and RF=3 executed from 2 m3.xlarge stress nodes for 1, 2 and 6 hours with 10, 20 and 30 threads without exhausting the cluster CPU/IO capacity. I tried several combinations of the following workloads: * read-only * write-only * range-only * triggering repairs during the execution * unthrottling compaction I recorded and analyzed flight recordings during the tests but didn't find anything suspicious. No blocked native transport threads were verified during tests with above scenarios, so this might indicate that this condition is not a widespread bug like CASSANDRA-11529 but probably some edgy combination of workload, environment and bad scheduling that happens in production but is harder to reproduce with synthetic workloads. A thread dump of when this condition happens would probably help us detect where is the bottleneck or contention, so I created CASSANDRA-11713 to add ability of logging a thread dump when the thread pool queue is full. If someone could install that patch and enable it in production to capture a thread dump when the blockage happens that would probably help us elucidate what's going on here. > Blocked NTR When Connecting Causing Excessive Load > -- > > Key: CASSANDRA-11363 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11363 > Project: Cassandra > Issue Type: Bug > Components: Coordination >Reporter: Russell Bradberry >Assignee: Paulo Motta >Priority: Critical > Attachments: cassandra-102-cms.stack, cassandra-102-g1gc.stack > > > When upgrading from 2.1.9 to 2.1.13, we are witnessing an issue where the > machine load increases to very high levels (> 120 on an 8 core machine) and > native transport requests get blocked in tpstats. > I was able to reproduce this in both CMS and G1GC as well as on JVM 7 and 8. > The issue does not seem to affect the nodes running 2.1.9. > The issue seems to coincide with the number of connections OR the number of > total requests being processed at a given time (as the latter increases with > the former in our system) > Currently there is between 600 and 800 client connections on each machine and > each machine is handling roughly 2000-3000 client requests per second. > Disabling the binary protocol fixes the issue for this node but isn't a > viable option cluster-wide. > Here is the output from tpstats: > {code} > Pool NameActive Pending Completed Blocked All > time blocked > MutationStage 0 88387821 0 > 0 > ReadStage 0 0 355860 0 > 0 > RequestResponseStage 0 72532457 0 > 0 > ReadRepairStage 0 0150 0 > 0 > CounterMutationStage 32 104 897560 0 > 0 > MiscStage 0 0 0 0 > 0 > HintedHandoff 0 0 65 0 > 0 > GossipStage 0 0 2338 0 > 0 > CacheCleanupExecutor 0 0 0 0 > 0 > InternalResponseStage 0 0 0 0 > 0 > CommitLogArchiver 0 0 0 0 > 0 > CompactionExecutor2 190474 0 > 0 > ValidationExecutor0 0 0 0 > 0 > MigrationStage0 0 10 0 > 0 > AntiEntropyStage 0 0 0 0 > 0 > PendingRangeCalculator0 0310 0 > 0 > Sampler 0 0 0 0 > 0 > MemtableFlushWriter 110 94 0
[jira] [Updated] (CASSANDRA-11713) Add ability to log thread dump when thread pool is full/blocked
[ https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-11713: Status: Patch Available (was: Open) > Add ability to log thread dump when thread pool is full/blocked > --- > > Key: CASSANDRA-11713 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11713 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Paulo Motta >Assignee: Paulo Motta >Priority: Minor > > Thread dumps are very useful for troubleshooting thread pool contention > issues like CASSANDRA-11363 and CASSANDRA-11529. > While they could be generated externally with {{jstack}}, sometimes the > conditions are transient and it's hard to catch the exact moment when they > happen, so it could be useful to generate and log them upon user request when > certain internal condition happens. > I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} > and {{JMXEnabledThreadPoolExecutor}} that when enabled via JMX generates and > logs a single thread dump on the system log when the thread pool queue is > full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11713) Add ability to log thread dump when thread pool is full/blocked
[ https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271514#comment-15271514 ] Paulo Motta commented on CASSANDRA-11713: - Initial patch available [here|https://github.com/pauloricardomg/cassandra/tree/thread-dumper] for review. > Add ability to log thread dump when thread pool is full/blocked > --- > > Key: CASSANDRA-11713 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11713 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Paulo Motta >Assignee: Paulo Motta >Priority: Minor > > Thread dumps are very useful for troubleshooting thread pool contention > issues like CASSANDRA-11363 and CASSANDRA-11529. > While they could be generated externally with {{jstack}}, sometimes the > conditions are transient and it's hard to catch the exact moment when they > happen, so it could be useful to generate and log them upon user request when > certain internal condition happens. > I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} > and {{JMXEnabledThreadPoolExecutor}} that when enabled via JMX generates and > logs a single thread dump on the system log when the thread pool queue is > full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11713) Add ability to log thread dump when thread pool is full/blocked
[ https://issues.apache.org/jira/browse/CASSANDRA-11713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-11713: Description: Thread dumps are very useful for troubleshooting thread pool contention issues like CASSANDRA-11363 and CASSANDRA-11529. While they could be generated externally with {{jstack}}, sometimes the conditions are transient and it's hard to catch the exact moment when they happen, so it could be useful to generate and log them upon user request when certain internal condition happens. I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} and {{JMXEnabledThreadPoolExecutor}} that when enabled via JMX generates and logs a single thread dump on the system log when the thread pool queue is full. was: Thread dumps are very useful for troubleshooting thread pool contention issues like CASSANDRA-11363 and CASSANDRA-11529. While they could be generated externally with {{jstack}}, sometimes the conditions are transient and it's hard to catch the exact moment when they happen, so it could be useful to generate and log them upon user request when certain internal condition happens. I propose adding a {{logThreadDumpOnNextContention}} JMX flag to {{SEPExecutor}} and {{JMXEnabledThreadPoolExecutor}} that when enabled generates and logs a single thread dump on the system log when the thread pool queue is full. > Add ability to log thread dump when thread pool is full/blocked > --- > > Key: CASSANDRA-11713 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11713 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Paulo Motta >Assignee: Paulo Motta >Priority: Minor > > Thread dumps are very useful for troubleshooting thread pool contention > issues like CASSANDRA-11363 and CASSANDRA-11529. > While they could be generated externally with {{jstack}}, sometimes the > conditions are transient and it's hard to catch the exact moment when they > happen, so it could be useful to generate and log them upon user request when > certain internal condition happens. > I propose adding a {{logThreadDumpOnNextContention}} flag to {{SEPExecutor}} > and {{JMXEnabledThreadPoolExecutor}} that when enabled via JMX generates and > logs a single thread dump on the system log when the thread pool queue is > full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11713) Add ability to log thread dump when thread pool is full/blocked
Paulo Motta created CASSANDRA-11713: --- Summary: Add ability to log thread dump when thread pool is full/blocked Key: CASSANDRA-11713 URL: https://issues.apache.org/jira/browse/CASSANDRA-11713 Project: Cassandra Issue Type: Improvement Components: Observability Reporter: Paulo Motta Assignee: Paulo Motta Priority: Minor Thread dumps are very useful for troubleshooting thread pool contention issues like CASSANDRA-11363 and CASSANDRA-11529. While they could be generated externally with {{jstack}}, sometimes the conditions are transient and it's hard to catch the exact moment when they happen, so it could be useful to generate and log them upon user request when certain internal condition happens. I propose adding a {{logThreadDumpOnNextContention}} JMX flag to {{SEPExecutor}} and {{JMXEnabledThreadPoolExecutor}} that when enabled generates and logs a single thread dump on the system log when the thread pool queue is full. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9766) Faster Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-9766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-9766: -- Resolution: Fixed Fix Version/s: (was: 3.x) 3.8 Status: Resolved (was: Patch Available) committed {{1e92ce43a5a730f81d3f6cfd72e7f4b126db788a}} > Faster Streaming > > > Key: CASSANDRA-9766 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9766 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging > Environment: Cassandra 2.1.2. more details in the pdf attached >Reporter: Alexei K >Assignee: T Jake Luciani > Labels: performance > Fix For: 3.8 > > Attachments: problem.pdf > > > I have a cluster in Amazon cloud , its described in detail in the attachment. > What I've noticed is that we during bootstrap we never go above 12MB/sec > transmission speeds and also those speeds flat line almost like we're hitting > some sort of a limit ( this remains true for other tests that I've ran) > however during the repair we see much higher,variable sending rates. I've > provided network charts in the attachment as well . Is there an explanation > for this? Is something wrong with my configuration, or is it a possible bug? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Faster Streaming
Repository: cassandra Updated Branches: refs/heads/trunk 594183b03 -> 1e92ce43a Faster Streaming Patch by tjake; reviewed by Stefania Alborghetti for CASSANDRA-9766 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1e92ce43 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1e92ce43 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1e92ce43 Branch: refs/heads/trunk Commit: 1e92ce43a5a730f81d3f6cfd72e7f4b126db788a Parents: 594183b Author: T Jake Luciani Authored: Sat Apr 9 10:03:32 2016 -0400 Committer: T Jake Luciani Committed: Wed May 4 17:04:12 2016 -0400 -- CHANGES.txt | 1 + .../concurrent/NamedThreadFactory.java | 4 +- .../apache/cassandra/concurrent/SEPWorker.java | 3 +- .../org/apache/cassandra/db/ColumnIndex.java| 69 +--- .../columniterator/SSTableReversedIterator.java | 2 +- .../db/commitlog/FileDirectSegment.java | 3 +- .../org/apache/cassandra/db/rows/BTreeRow.java | 30 ++-- .../cassandra/db/rows/ComplexColumnData.java| 10 +- .../cassandra/db/rows/UnfilteredSerializer.java | 40 - .../io/sstable/format/big/BigTableWriter.java | 50 +++--- .../cassandra/io/util/DataOutputBuffer.java | 38 - .../io/util/DataOutputBufferFixed.java | 4 +- .../cassandra/io/util/DataOutputStreamPlus.java | 3 +- .../cassandra/io/util/SafeMemoryWriter.java | 2 +- .../cassandra/streaming/ConnectionHandler.java | 4 +- .../compress/CompressedInputStream.java | 59 +-- .../compress/CompressedStreamReader.java| 11 +- .../cassandra/tools/SSTableMetadataViewer.java | 10 +- .../org/apache/cassandra/utils/BloomFilter.java | 7 +- .../apache/cassandra/utils/FilterFactory.java | 6 +- .../cassandra/utils/StreamingHistogram.java | 84 + .../org/apache/cassandra/utils/btree/BTree.java | 67 .../apache/cassandra/utils/btree/BTreeSet.java | 3 - .../cassandra/utils/btree/TreeBuilder.java | 30 +++- .../concurrent/WrappedSharedCloseable.java | 4 +- .../cassandra/utils/memory/BufferPool.java | 16 +- .../apache/cassandra/utils/vint/VIntCoding.java | 3 +- .../cassandra/streaming/LongStreamingTest.java | 171 +++ .../apache/cassandra/db/RowIndexEntryTest.java | 9 +- .../org/apache/cassandra/utils/BTreeTest.java | 20 ++- .../cassandra/utils/StreamingHistogramTest.java | 48 +- 31 files changed, 597 insertions(+), 214 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e92ce43/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 6fe57b2..c2165db 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.8 + * Faster streaming (CASSANDRA-9766) 3.6 * Enhanced Compaction Logging (CASSANDRA-10805) http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e92ce43/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java -- diff --git a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java index 33c80d5..85edf74 100644 --- a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java +++ b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java @@ -20,6 +20,8 @@ package org.apache.cassandra.concurrent; import java.util.concurrent.ThreadFactory; import java.util.concurrent.atomic.AtomicInteger; +import io.netty.util.concurrent.FastThreadLocalThread; + /** * This class is an implementation of the ThreadFactory interface. This * is useful to give Java threads meaningful names which is useful when using @@ -55,7 +57,7 @@ public class NamedThreadFactory implements ThreadFactory public Thread newThread(Runnable runnable) { String name = id + ":" + n.getAndIncrement(); -Thread thread = new Thread(threadGroup, runnable, name); +Thread thread = new FastThreadLocalThread(threadGroup, runnable, name); thread.setPriority(priority); thread.setDaemon(true); if (contextClassLoader != null) http://git-wip-us.apache.org/repos/asf/cassandra/blob/1e92ce43/src/java/org/apache/cassandra/concurrent/SEPWorker.java -- diff --git a/src/java/org/apache/cassandra/concurrent/SEPWorker.java b/src/java/org/apache/cassandra/concurrent/SEPWorker.java index 3b3e7ad..d7c21bc 100644 --- a/src/java/org/apache/cassandra/concurrent/SEPWorker.java +++ b/src/java/org/apache/cassandra/concurrent/SEPWorker.java @@ -24,6 +24,7 @@ import java.util.concurrent.locks.LockSupport; import org.slf4j.Logger; import
[jira] [Commented] (CASSANDRA-11483) Enhance sstablemetadata
[ https://issues.apache.org/jira/browse/CASSANDRA-11483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271404#comment-15271404 ] Yuki Morishita commented on CASSANDRA-11483: Alright, let's just fix dtest. Looks like adding backward compatibility isn't worth the effort at all. > Enhance sstablemetadata > --- > > Key: CASSANDRA-11483 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11483 > Project: Cassandra > Issue Type: Improvement > Components: Observability >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Minor > Fix For: 3.x > > Attachments: CASSANDRA-11483.txt, CASSANDRA-11483v2.txt, > CASSANDRA-11483v3.txt, CASSANDRA-11483v4.txt, Screen Shot 2016-04-03 at > 11.40.32 PM.png > > > sstablemetadata provides quite a bit of useful information but theres a few > hiccups I would like to see addressed: > * Does not use client mode > * Units are not provided (or anything for that matter). There is data in > micros, millis, seconds as durations and timestamps from epoch. But there is > no way to tell what one is without a non-trival code dive > * in general pretty frustrating to parse -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11503) Need a tool to detect what percentage of SSTables on a node have been repaired when using incremental repairs.
[ https://issues.apache.org/jira/browse/CASSANDRA-11503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271401#comment-15271401 ] Yuki Morishita commented on CASSANDRA-11503: bq. that sound good? Yes. One thing to consider is, how do users access global repaired% other than direct JMX access? Per keyspace or table we can use nodetool tablestat, maybe we can use {{nodetool info}} for global. > Need a tool to detect what percentage of SSTables on a node have been > repaired when using incremental repairs. > -- > > Key: CASSANDRA-11503 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11503 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Sean Usher >Assignee: Chris Lohfink >Priority: Minor > Attachments: CASSANDRA-11503.patch > > > When using incremental repair, we should be able to look at SSTables and > understand how many sstables are in the repaired and unrepaired buckets on > each machine. This can help us track the repair progress and if we are > hitting any issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11657) LongLeveledCompactionStrategyTest broken since CASSANDRA-10099
[ https://issues.apache.org/jira/browse/CASSANDRA-11657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271378#comment-15271378 ] Yuki Morishita commented on CASSANDRA-11657: I still am not sure what's causing that error (and I still cannot reproduce in local). Looking at the code change around {{LongLeveledCompactionStrategyTest}}, background task is disabled and {{CompactionTask}} is created sequentially. > LongLeveledCompactionStrategyTest broken since CASSANDRA-10099 > -- > > Key: CASSANDRA-11657 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11657 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 3.x > > > Seems CASSANDRA-10099 broke LongLeveledCompactionStrategyTest -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[3/3] cassandra git commit: replace deprecated sameThreadExecutor with directExecutor
replace deprecated sameThreadExecutor with directExecutor Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/594183b0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/594183b0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/594183b0 Branch: refs/heads/trunk Commit: 594183b0360aa252624f04e0e29fc12815fa39f4 Parents: f4233eb Author: Jonathan Ellis Authored: Wed May 4 10:21:25 2016 -0500 Committer: Jonathan Ellis Committed: Wed May 4 14:32:09 2016 -0500 -- .../org/apache/cassandra/repair/RepairMessageVerbHandler.java| 2 +- src/java/org/apache/cassandra/service/ActiveRepairService.java | 4 ++-- src/java/org/apache/cassandra/streaming/StreamManager.java | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/594183b0/src/java/org/apache/cassandra/repair/RepairMessageVerbHandler.java -- diff --git a/src/java/org/apache/cassandra/repair/RepairMessageVerbHandler.java b/src/java/org/apache/cassandra/repair/RepairMessageVerbHandler.java index 519fe5f..b06abb2 100644 --- a/src/java/org/apache/cassandra/repair/RepairMessageVerbHandler.java +++ b/src/java/org/apache/cassandra/repair/RepairMessageVerbHandler.java @@ -154,7 +154,7 @@ public class RepairMessageVerbHandler implements IVerbHandler { MessagingService.instance().sendReply(new MessageOut(MessagingService.Verb.INTERNAL_RESPONSE), id, message.from); } -}, MoreExecutors.sameThreadExecutor()); +}, MoreExecutors.directExecutor()); break; case CLEANUP: http://git-wip-us.apache.org/repos/asf/cassandra/blob/594183b0/src/java/org/apache/cassandra/service/ActiveRepairService.java -- diff --git a/src/java/org/apache/cassandra/service/ActiveRepairService.java b/src/java/org/apache/cassandra/service/ActiveRepairService.java index 6c75149..dd196b9 100644 --- a/src/java/org/apache/cassandra/service/ActiveRepairService.java +++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java @@ -151,7 +151,7 @@ public class ActiveRepairService gossiper.unregister(session); sessions.remove(session.getId()); } -}, MoreExecutors.sameThreadExecutor()); +}, MoreExecutors.directExecutor()); session.start(executor); return session; } @@ -401,7 +401,7 @@ public class ActiveRepairService { removeParentRepairSession(parentRepairSession); } -}, MoreExecutors.sameThreadExecutor()); +}, MoreExecutors.directExecutor()); return allAntiCompactionResults; } http://git-wip-us.apache.org/repos/asf/cassandra/blob/594183b0/src/java/org/apache/cassandra/streaming/StreamManager.java -- diff --git a/src/java/org/apache/cassandra/streaming/StreamManager.java b/src/java/org/apache/cassandra/streaming/StreamManager.java index dc8ec19..52652c0 100644 --- a/src/java/org/apache/cassandra/streaming/StreamManager.java +++ b/src/java/org/apache/cassandra/streaming/StreamManager.java @@ -131,7 +131,7 @@ public class StreamManager implements StreamManagerMBean { initiatedStreams.remove(result.planId); } -}, MoreExecutors.sameThreadExecutor()); +}, MoreExecutors.directExecutor()); initiatedStreams.put(result.planId, result); } @@ -146,7 +146,7 @@ public class StreamManager implements StreamManagerMBean { receivingStreams.remove(result.planId); } -}, MoreExecutors.sameThreadExecutor()); +}, MoreExecutors.directExecutor()); receivingStreams.put(result.planId, result); }
[1/3] cassandra git commit: replace deprecated Iterables.emptyIterator with Collections.emptyIterator
Repository: cassandra Updated Branches: refs/heads/trunk 06cd4e824 -> 594183b03 replace deprecated Iterables.emptyIterator with Collections.emptyIterator Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f4233eb7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f4233eb7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f4233eb7 Branch: refs/heads/trunk Commit: f4233eb77c4703c951050b864abec40f4b7b03f2 Parents: e8a0d0a Author: Jonathan Ellis Authored: Wed May 4 10:15:47 2016 -0500 Committer: Jonathan Ellis Committed: Wed May 4 14:32:08 2016 -0500 -- src/java/org/apache/cassandra/db/MutableDeletionInfo.java | 5 +++-- src/java/org/apache/cassandra/db/Slices.java | 2 +- src/java/org/apache/cassandra/utils/IntervalTree.java | 2 +- test/unit/org/apache/cassandra/Util.java | 2 +- 4 files changed, 6 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4233eb7/src/java/org/apache/cassandra/db/MutableDeletionInfo.java -- diff --git a/src/java/org/apache/cassandra/db/MutableDeletionInfo.java b/src/java/org/apache/cassandra/db/MutableDeletionInfo.java index 1ba77bb..4a2c455 100644 --- a/src/java/org/apache/cassandra/db/MutableDeletionInfo.java +++ b/src/java/org/apache/cassandra/db/MutableDeletionInfo.java @@ -17,6 +17,7 @@ */ package org.apache.cassandra.db; +import java.util.Collections; import java.util.Iterator; import com.google.common.base.Objects; @@ -151,12 +152,12 @@ public class MutableDeletionInfo implements DeletionInfo // Use sparingly, not the most efficient thing public Iterator rangeIterator(boolean reversed) { -return ranges == null ? Iterators.emptyIterator() : ranges.iterator(reversed); +return ranges == null ? Collections.emptyIterator() : ranges.iterator(reversed); } public Iterator rangeIterator(Slice slice, boolean reversed) { -return ranges == null ? Iterators.emptyIterator() : ranges.iterator(slice, reversed); +return ranges == null ? Collections.emptyIterator() : ranges.iterator(slice, reversed); } public RangeTombstone rangeCovering(Clustering name) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4233eb7/src/java/org/apache/cassandra/db/Slices.java -- diff --git a/src/java/org/apache/cassandra/db/Slices.java b/src/java/org/apache/cassandra/db/Slices.java index 4fcda59..07da5b2 100644 --- a/src/java/org/apache/cassandra/db/Slices.java +++ b/src/java/org/apache/cassandra/db/Slices.java @@ -811,7 +811,7 @@ public abstract class Slices implements Iterable public Iterator iterator() { -return Iterators.emptyIterator(); +return Collections.emptyIterator(); } @Override http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4233eb7/src/java/org/apache/cassandra/utils/IntervalTree.java -- diff --git a/src/java/org/apache/cassandra/utils/IntervalTree.java b/src/java/org/apache/cassandra/utils/IntervalTree.java index 4cf1222..f761180 100644 --- a/src/java/org/apache/cassandra/utils/IntervalTree.java +++ b/src/java/org/apache/cassandra/utils/IntervalTree.java @@ -114,7 +114,7 @@ public class IntervalTree, D, I extends Interval public Iterator iterator() { if (head == null) -return Iterators.emptyIterator(); +return Collections.emptyIterator(); return new TreeIterator(head); } http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4233eb7/test/unit/org/apache/cassandra/Util.java -- diff --git a/test/unit/org/apache/cassandra/Util.java b/test/unit/org/apache/cassandra/Util.java index 87a07b0..ed4fe10 100644 --- a/test/unit/org/apache/cassandra/Util.java +++ b/test/unit/org/apache/cassandra/Util.java @@ -470,7 +470,7 @@ public class Util // moved & refactored from KeyspaceTest in < 3.0 public static void assertColumns(Row row, String... expectedColumnNames) { -Iterator cells = row == null ? Iterators.emptyIterator() : row.cells().iterator(); +Iterator cells = row == null ? Collections.emptyIterator() : row.cells().iterator(); String[] actual = Iterators.toArray(Iterators.transform(cells, new Function() { public String apply(Cell cell)
[2/3] cassandra git commit: update deprecated Objects.toStringHelper -> MoreObjects.toStringHelper
update deprecated Objects.toStringHelper -> MoreObjects.toStringHelper Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e8a0d0aa Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e8a0d0aa Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e8a0d0aa Branch: refs/heads/trunk Commit: e8a0d0aac5bdcd468e334a46b5aade4d4559ef76 Parents: 06cd4e8 Author: Jonathan Ellis Authored: Wed May 4 10:00:15 2016 -0500 Committer: Jonathan Ellis Committed: Wed May 4 14:32:08 2016 -0500 -- .../apache/cassandra/config/ColumnDefinition.java | 13 +++-- .../apache/cassandra/cql3/ColumnSpecification.java | 9 + .../apache/cassandra/cql3/selection/Selection.java | 15 --- .../cassandra/cql3/statements/SelectStatement.java | 13 +++-- .../apache/cassandra/schema/KeyspaceMetadata.java | 17 + .../apache/cassandra/schema/KeyspaceParams.java| 9 + .../apache/cassandra/schema/TriggerMetadata.java | 9 + 7 files changed, 46 insertions(+), 39 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/e8a0d0aa/src/java/org/apache/cassandra/config/ColumnDefinition.java -- diff --git a/src/java/org/apache/cassandra/config/ColumnDefinition.java b/src/java/org/apache/cassandra/config/ColumnDefinition.java index a18ed3f..a900ce7 100644 --- a/src/java/org/apache/cassandra/config/ColumnDefinition.java +++ b/src/java/org/apache/cassandra/config/ColumnDefinition.java @@ -22,6 +22,7 @@ import java.util.*; import com.google.common.annotations.VisibleForTesting; import com.google.common.base.Function; +import com.google.common.base.MoreObjects; import com.google.common.base.Objects; import com.google.common.collect.Collections2; @@ -282,12 +283,12 @@ public class ColumnDefinition extends ColumnSpecification implements Comparable< @Override public String toString() { -return Objects.toStringHelper(this) - .add("name", name) - .add("type", type) - .add("kind", kind) - .add("position", position) - .toString(); +return MoreObjects.toStringHelper(this) + .add("name", name) + .add("type", type) + .add("kind", kind) + .add("position", position) + .toString(); } public boolean isPrimaryKeyColumn() http://git-wip-us.apache.org/repos/asf/cassandra/blob/e8a0d0aa/src/java/org/apache/cassandra/cql3/ColumnSpecification.java -- diff --git a/src/java/org/apache/cassandra/cql3/ColumnSpecification.java b/src/java/org/apache/cassandra/cql3/ColumnSpecification.java index e64f5f9..8cf869b 100644 --- a/src/java/org/apache/cassandra/cql3/ColumnSpecification.java +++ b/src/java/org/apache/cassandra/cql3/ColumnSpecification.java @@ -17,6 +17,7 @@ */ package org.apache.cassandra.cql3; +import com.google.common.base.MoreObjects; import com.google.common.base.Objects; import org.apache.cassandra.db.marshal.AbstractType; @@ -96,9 +97,9 @@ public class ColumnSpecification @Override public String toString() { -return Objects.toStringHelper(this) - .add("name", name) - .add("type", type) - .toString(); +return MoreObjects.toStringHelper(this) + .add("name", name) + .add("type", type) + .toString(); } } http://git-wip-us.apache.org/repos/asf/cassandra/blob/e8a0d0aa/src/java/org/apache/cassandra/cql3/selection/Selection.java -- diff --git a/src/java/org/apache/cassandra/cql3/selection/Selection.java b/src/java/org/apache/cassandra/cql3/selection/Selection.java index f337c1a..664fd4f 100644 --- a/src/java/org/apache/cassandra/cql3/selection/Selection.java +++ b/src/java/org/apache/cassandra/cql3/selection/Selection.java @@ -20,6 +20,7 @@ package org.apache.cassandra.cql3.selection; import java.nio.ByteBuffer; import java.util.*; +import com.google.common.base.MoreObjects; import com.google.common.base.Objects; import com.google.common.base.Predicate; import com.google.common.collect.Iterables; @@ -248,13 +249,13 @@ public abstract class Selection @Override public String toString() { -return Objects.toStringHelper(this) -.add("columns", columns) -.add("columnMapping", columnMapping) -.add("metadata", metadata) -
[jira] [Commented] (CASSANDRA-7826) support non-frozen, nested collections
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271250#comment-15271250 ] Alex Petrov commented on CASSANDRA-7826: Would the updates and deletes for particular cells be within the scope of the ticket? After talking to [~thobbs] there's also the question if we would like to support arbitrary updates or deletes right away, too. That would require new syntax for selecting the path for update or delete (maybe something like {{mymap['a']['b'][1]}}). Also, {{BufferCell/tombstone}} can most likely handle deletes as they are, without slices, as we'll always know at least the partial path, but if I understand it correctly, it will require additional logic for discarding columns whose partial path matches the path of the tombstone. Also, most likely [7396|https://issues.apache.org/jira/browse/CASSANDRA-7396] will land before this patch, so it we might have to add support for selecting individual parts on arbitrary level (or just on the top level at first and leave the rest for the next iterations). So the question is what we'd like to have on the current step. Right now, arbitrary nesting of maps is fully working (also, dtests and UDFs with nested structures are working). Non-frozen Set and List can be nested arbitrarily deep within the map, although nothing can be nested within Set or List. As was mentioned above. > support non-frozen, nested collections > -- > > Key: CASSANDRA-7826 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Alex Petrov > Labels: ponies > Fix For: 3.x > > > The inability to nest collections is one of the bigger data modelling > limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (CASSANDRA-7826) support non-frozen, nested collections
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-7826: --- Comment: was deleted (was: S) > support non-frozen, nested collections > -- > > Key: CASSANDRA-7826 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Alex Petrov > Labels: ponies > Fix For: 3.x > > > The inability to nest collections is one of the bigger data modelling > limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7826) support non-frozen, nested collections
[ https://issues.apache.org/jira/browse/CASSANDRA-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271221#comment-15271221 ] Alex Petrov commented on CASSANDRA-7826: S > support non-frozen, nested collections > -- > > Key: CASSANDRA-7826 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7826 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Alex Petrov > Labels: ponies > Fix For: 3.x > > > The inability to nest collections is one of the bigger data modelling > limitations we have right now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11540) The JVM should exit if jmx fails to bind
[ https://issues.apache.org/jira/browse/CASSANDRA-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271159#comment-15271159 ] Alex Petrov commented on CASSANDRA-11540: - oh :) thank you [~beobal] :) looks like I got late with my comment about tests ) > The JVM should exit if jmx fails to bind > > > Key: CASSANDRA-11540 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11540 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Brandon Williams >Assignee: Alex Petrov > Labels: lhf > Fix For: 2.2.7, 3.7, 3.0.7 > > > If you are already running a cassandra instance, but for some reason try to > start another one, this happens: > {noformat} > INFO 20:57:09 JNA mlockall successful > WARN 20:57:09 JMX is not enabled to receive remote connections. Please see > cassandra-env.sh for more info. > ERROR 20:57:10 Error starting local jmx server: > java.rmi.server.ExportException: Port already in use: 7199; nested exception > is: > java.net.BindException: Address already in use > at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:340) > ~[na:1.7.0_76] > at > sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:248) > ~[na:1.7.0_76] > at > sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411) > ~[na:1.7.0_76] > at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147) > ~[na:1.7.0_76] > at > sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207) > ~[na:1.7.0_76] > at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:122) > ~[na:1.7.0_76] > at sun.rmi.registry.RegistryImpl.(RegistryImpl.java:98) > ~[na:1.7.0_76] > at > java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239) > ~[na:1.7.0_76] > at > org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:100) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:222) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653) > [main/:na] > Caused by: java.net.BindException: Address already in use > at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.7.0_76] > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376) > ~[na:1.7.0_76] > at java.net.ServerSocket.bind(ServerSocket.java:376) ~[na:1.7.0_76] > at java.net.ServerSocket.(ServerSocket.java:237) ~[na:1.7.0_76] > at > javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231) > ~[na:1.7.0_76] > at > org.apache.cassandra.utils.RMIServerSocketFactoryImpl.createServerSocket(RMIServerSocketFactoryImpl.java:13) > ~[main/:na] > at > sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666) > ~[na:1.7.0_76] > at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:329) > ~[na:1.7.0_76] > ... 11 common frames omitted > {noformat} > However the startup continues, and ends up replaying commitlogs, which is > probably not a good thing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11540) The JVM should exit if jmx fails to bind
[ https://issues.apache.org/jira/browse/CASSANDRA-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271157#comment-15271157 ] Alex Petrov commented on CASSANDRA-11540: - >From dtests, {{restart_node_localhost_test}} is already reported was >introduced before the patch: >[11702|https://issues.apache.org/jira/browse/CASSANDRA-11702] (although it >looked related. also, passes locally) same with {{test_upgrade_index_summary}} >[11127|https://issues.apache.org/jira/browse/CASSANDRA-11127] and >{{upgrade_with_wide_partition_reversed_test}} >[11663|https://issues.apache.org/jira/browse/CASSANDRA-11663]. The rest of >them also look unrelated, although I ran them just in case. >{{testJsonThreadSafety}} is failing for me quite often, so I filed the issue, >too: [11712|https://issues.apache.org/jira/browse/CASSANDRA-11712]. > The JVM should exit if jmx fails to bind > > > Key: CASSANDRA-11540 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11540 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Brandon Williams >Assignee: Alex Petrov > Labels: lhf > Fix For: 2.2.7, 3.7, 3.0.7 > > > If you are already running a cassandra instance, but for some reason try to > start another one, this happens: > {noformat} > INFO 20:57:09 JNA mlockall successful > WARN 20:57:09 JMX is not enabled to receive remote connections. Please see > cassandra-env.sh for more info. > ERROR 20:57:10 Error starting local jmx server: > java.rmi.server.ExportException: Port already in use: 7199; nested exception > is: > java.net.BindException: Address already in use > at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:340) > ~[na:1.7.0_76] > at > sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:248) > ~[na:1.7.0_76] > at > sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411) > ~[na:1.7.0_76] > at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147) > ~[na:1.7.0_76] > at > sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207) > ~[na:1.7.0_76] > at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:122) > ~[na:1.7.0_76] > at sun.rmi.registry.RegistryImpl.(RegistryImpl.java:98) > ~[na:1.7.0_76] > at > java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239) > ~[na:1.7.0_76] > at > org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:100) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:222) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653) > [main/:na] > Caused by: java.net.BindException: Address already in use > at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.7.0_76] > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376) > ~[na:1.7.0_76] > at java.net.ServerSocket.bind(ServerSocket.java:376) ~[na:1.7.0_76] > at java.net.ServerSocket.(ServerSocket.java:237) ~[na:1.7.0_76] > at > javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231) > ~[na:1.7.0_76] > at > org.apache.cassandra.utils.RMIServerSocketFactoryImpl.createServerSocket(RMIServerSocketFactoryImpl.java:13) > ~[main/:na] > at > sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666) > ~[na:1.7.0_76] > at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:329) > ~[na:1.7.0_76] > ... 11 common frames omitted > {noformat} > However the startup continues, and ends up replaying commitlogs, which is > probably not a good thing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10962) Cassandra should not create snapshot at restart for compactions_in_progress
[ https://issues.apache.org/jira/browse/CASSANDRA-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Knighton updated CASSANDRA-10962: -- Status: Ready to Commit (was: Patch Available) > Cassandra should not create snapshot at restart for compactions_in_progress > --- > > Key: CASSANDRA-10962 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10962 > Project: Cassandra > Issue Type: Bug > Environment: Ubuntu 14.04.3 LTS >Reporter: FACORAT >Assignee: Alex Petrov >Priority: Minor > Fix For: 2.2.x > > > If auto_snapshot is set to true in cassandra.yaml, each time you restart > Cassandra, a snapshot is created for system.compactions_in_progress as the > table is truncated at cassandra start. > However as datas in this table are temporary, Cassandra should not create > snapshot for this table (or maybe even for system.* tables). This will be > coherent with the fact that "nodetool listsnapshots" doesn't even list this > table. > Exemple: > {code} > $ nodetool listsnapshots | grep compactions > $ ls -lh > system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/snapshots/ > total 16K > drwxr-xr-x 2 cassandra cassandra 4.0K Nov 30 13:12 > 1448885530280-compactions_in_progress > drwxr-xr-x 2 cassandra cassandra 4.0K Dec 7 15:36 > 1449498977181-compactions_in_progress > drwxr-xr-x 2 cassandra cassandra 4.0K Dec 14 18:20 > 1450113621506-compactions_in_progress > drwxr-xr-x 2 cassandra cassandra 4.0K Jan 4 12:53 > 1451908396364-compactions_in_progress > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10962) Cassandra should not create snapshot at restart for compactions_in_progress
[ https://issues.apache.org/jira/browse/CASSANDRA-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271088#comment-15271088 ] Joel Knighton commented on CASSANDRA-10962: --- +1, LGTM. CI is clean relative to upstream. As [~ifesdjeen] noted, this should only go to 2.2, since the related subsystem was rewritten in 3.0. > Cassandra should not create snapshot at restart for compactions_in_progress > --- > > Key: CASSANDRA-10962 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10962 > Project: Cassandra > Issue Type: Bug > Environment: Ubuntu 14.04.3 LTS >Reporter: FACORAT >Assignee: Alex Petrov >Priority: Minor > Fix For: 2.2.x > > > If auto_snapshot is set to true in cassandra.yaml, each time you restart > Cassandra, a snapshot is created for system.compactions_in_progress as the > table is truncated at cassandra start. > However as datas in this table are temporary, Cassandra should not create > snapshot for this table (or maybe even for system.* tables). This will be > coherent with the fact that "nodetool listsnapshots" doesn't even list this > table. > Exemple: > {code} > $ nodetool listsnapshots | grep compactions > $ ls -lh > system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/snapshots/ > total 16K > drwxr-xr-x 2 cassandra cassandra 4.0K Nov 30 13:12 > 1448885530280-compactions_in_progress > drwxr-xr-x 2 cassandra cassandra 4.0K Dec 7 15:36 > 1449498977181-compactions_in_progress > drwxr-xr-x 2 cassandra cassandra 4.0K Dec 14 18:20 > 1450113621506-compactions_in_progress > drwxr-xr-x 2 cassandra cassandra 4.0K Jan 4 12:53 > 1451908396364-compactions_in_progress > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11540) The JVM should exit if jmx fails to bind
[ https://issues.apache.org/jira/browse/CASSANDRA-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-11540: Resolution: Fixed Fix Version/s: (was: 3.0.x) (was: 2.2.x) (was: 3.x) 3.0.7 3.7 2.2.7 Status: Resolved (was: Patch Available) CI looks good, so committed to 2.2 in {{93c5bc616e21ffa7f31266ad095ca374f2ba73a4}} and merged to 3.0/3.7/trunk. > The JVM should exit if jmx fails to bind > > > Key: CASSANDRA-11540 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11540 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Brandon Williams >Assignee: Alex Petrov > Labels: lhf > Fix For: 2.2.7, 3.7, 3.0.7 > > > If you are already running a cassandra instance, but for some reason try to > start another one, this happens: > {noformat} > INFO 20:57:09 JNA mlockall successful > WARN 20:57:09 JMX is not enabled to receive remote connections. Please see > cassandra-env.sh for more info. > ERROR 20:57:10 Error starting local jmx server: > java.rmi.server.ExportException: Port already in use: 7199; nested exception > is: > java.net.BindException: Address already in use > at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:340) > ~[na:1.7.0_76] > at > sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:248) > ~[na:1.7.0_76] > at > sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411) > ~[na:1.7.0_76] > at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147) > ~[na:1.7.0_76] > at > sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207) > ~[na:1.7.0_76] > at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:122) > ~[na:1.7.0_76] > at sun.rmi.registry.RegistryImpl.(RegistryImpl.java:98) > ~[na:1.7.0_76] > at > java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239) > ~[na:1.7.0_76] > at > org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:100) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:222) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653) > [main/:na] > Caused by: java.net.BindException: Address already in use > at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.7.0_76] > at > java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376) > ~[na:1.7.0_76] > at java.net.ServerSocket.bind(ServerSocket.java:376) ~[na:1.7.0_76] > at java.net.ServerSocket.(ServerSocket.java:237) ~[na:1.7.0_76] > at > javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231) > ~[na:1.7.0_76] > at > org.apache.cassandra.utils.RMIServerSocketFactoryImpl.createServerSocket(RMIServerSocketFactoryImpl.java:13) > ~[main/:na] > at > sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666) > ~[na:1.7.0_76] > at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:329) > ~[na:1.7.0_76] > ... 11 common frames omitted > {noformat} > However the startup continues, and ends up replaying commitlogs, which is > probably not a good thing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[02/10] cassandra git commit: Avoid starting Cassandra on JMX bind failure
Avoid starting Cassandra on JMX bind failure Patch by Alex Petrov; reviewed by Sam Tunnicliffe for CASSANDRA-11540 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/93c5bc61 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/93c5bc61 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/93c5bc61 Branch: refs/heads/cassandra-3.0 Commit: 93c5bc616e21ffa7f31266ad095ca374f2ba73a4 Parents: 3d73282 Author: Alex Petrov Authored: Mon Apr 18 13:47:59 2016 +0200 Committer: Sam Tunnicliffe Committed: Wed May 4 17:55:47 2016 +0100 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/service/CassandraDaemon.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/93c5bc61/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7c7295d..0d9d3e9 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) http://git-wip-us.apache.org/repos/asf/cassandra/blob/93c5bc61/src/java/org/apache/cassandra/service/CassandraDaemon.java -- diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java b/src/java/org/apache/cassandra/service/CassandraDaemon.java index 6f38545..7e33e9c 100644 --- a/src/java/org/apache/cassandra/service/CassandraDaemon.java +++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java @@ -98,7 +98,7 @@ public class CassandraDaemon logger = LoggerFactory.getLogger(CassandraDaemon.class); } -private static void maybeInitJmx() +private void maybeInitJmx() { if (System.getProperty("com.sun.management.jmxremote.port") != null) return; @@ -119,7 +119,7 @@ public class CassandraDaemon } catch (IOException e) { -logger.error("Error starting local jmx server: ", e); +exitOrFail(1, e.getMessage(), e.getCause()); } }
[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5980b336 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5980b336 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5980b336 Branch: refs/heads/trunk Commit: 5980b336f0511987066ba175b9b52f0eafda77fd Parents: 2e202d4 93c5bc6 Author: Sam Tunnicliffe Authored: Wed May 4 17:58:01 2016 +0100 Committer: Sam Tunnicliffe Committed: Wed May 4 17:58:01 2016 +0100 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/service/CassandraDaemon.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/5980b336/CHANGES.txt -- diff --cc CHANGES.txt index 31e46d9,0d9d3e9..860a4c2 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,25 -1,7 +1,26 @@@ -2.2.7 +3.0.6 + * Disallow creating view with a static column (CASSANDRA-11602) + * Reduce the amount of object allocations caused by the getFunctions methods (CASSANDRA-11593) + * Potential error replaying commitlog with smallint/tinyint/date/time types (CASSANDRA-11618) + * Fix queries with filtering on counter columns (CASSANDRA-11629) + * Improve tombstone printing in sstabledump (CASSANDRA-11655) + * Fix paging for range queries where all clustering columns are specified (CASSANDRA-11669) + * Don't require HEAP_NEW_SIZE to be set when using G1 (CASSANDRA-11600) + * Fix sstabledump not showing cells after tombstone marker (CASSANDRA-11654) + * Ignore all LocalStrategy keyspaces for streaming and other related + operations (CASSANDRA-11627) + * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523) + * Only open one sstable scanner per sstable (CASSANDRA-11412) + * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410) + * ArithmeticException in avgFunctionForDecimal (CASSANDRA-11485) + * LogAwareFileLister should only use OLD sstable files in current folder to determine disk consistency (CASSANDRA-11470) + * Notify indexers of expired rows during compaction (CASSANDRA-11329) + * Properly respond with ProtocolError when a v1/v2 native protocol + header is received (CASSANDRA-11464) + * Validate that num_tokens and initial_token are consistent with one another (CASSANDRA-10120) +Merged from 2.2: + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) - * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) http://git-wip-us.apache.org/repos/asf/cassandra/blob/5980b336/src/java/org/apache/cassandra/service/CassandraDaemon.java --
[03/10] cassandra git commit: Avoid starting Cassandra on JMX bind failure
Avoid starting Cassandra on JMX bind failure Patch by Alex Petrov; reviewed by Sam Tunnicliffe for CASSANDRA-11540 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/93c5bc61 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/93c5bc61 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/93c5bc61 Branch: refs/heads/cassandra-3.7 Commit: 93c5bc616e21ffa7f31266ad095ca374f2ba73a4 Parents: 3d73282 Author: Alex Petrov Authored: Mon Apr 18 13:47:59 2016 +0200 Committer: Sam Tunnicliffe Committed: Wed May 4 17:55:47 2016 +0100 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/service/CassandraDaemon.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/93c5bc61/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7c7295d..0d9d3e9 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) http://git-wip-us.apache.org/repos/asf/cassandra/blob/93c5bc61/src/java/org/apache/cassandra/service/CassandraDaemon.java -- diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java b/src/java/org/apache/cassandra/service/CassandraDaemon.java index 6f38545..7e33e9c 100644 --- a/src/java/org/apache/cassandra/service/CassandraDaemon.java +++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java @@ -98,7 +98,7 @@ public class CassandraDaemon logger = LoggerFactory.getLogger(CassandraDaemon.class); } -private static void maybeInitJmx() +private void maybeInitJmx() { if (System.getProperty("com.sun.management.jmxremote.port") != null) return; @@ -119,7 +119,7 @@ public class CassandraDaemon } catch (IOException e) { -logger.error("Error starting local jmx server: ", e); +exitOrFail(1, e.getMessage(), e.getCause()); } }
[04/10] cassandra git commit: Avoid starting Cassandra on JMX bind failure
Avoid starting Cassandra on JMX bind failure Patch by Alex Petrov; reviewed by Sam Tunnicliffe for CASSANDRA-11540 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/93c5bc61 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/93c5bc61 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/93c5bc61 Branch: refs/heads/trunk Commit: 93c5bc616e21ffa7f31266ad095ca374f2ba73a4 Parents: 3d73282 Author: Alex Petrov Authored: Mon Apr 18 13:47:59 2016 +0200 Committer: Sam Tunnicliffe Committed: Wed May 4 17:55:47 2016 +0100 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/service/CassandraDaemon.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/93c5bc61/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7c7295d..0d9d3e9 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) http://git-wip-us.apache.org/repos/asf/cassandra/blob/93c5bc61/src/java/org/apache/cassandra/service/CassandraDaemon.java -- diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java b/src/java/org/apache/cassandra/service/CassandraDaemon.java index 6f38545..7e33e9c 100644 --- a/src/java/org/apache/cassandra/service/CassandraDaemon.java +++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java @@ -98,7 +98,7 @@ public class CassandraDaemon logger = LoggerFactory.getLogger(CassandraDaemon.class); } -private static void maybeInitJmx() +private void maybeInitJmx() { if (System.getProperty("com.sun.management.jmxremote.port") != null) return; @@ -119,7 +119,7 @@ public class CassandraDaemon } catch (IOException e) { -logger.error("Error starting local jmx server: ", e); +exitOrFail(1, e.getMessage(), e.getCause()); } }
[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/68580c7a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/68580c7a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/68580c7a Branch: refs/heads/cassandra-3.7 Commit: 68580c7a81fb20ef76abdfbe5eaefcc07b97 Parents: 415bb56 5980b33 Author: Sam Tunnicliffe Authored: Wed May 4 18:02:40 2016 +0100 Committer: Sam Tunnicliffe Committed: Wed May 4 18:02:40 2016 +0100 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/service/CassandraDaemon.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/68580c7a/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/68580c7a/src/java/org/apache/cassandra/service/CassandraDaemon.java --
[01/10] cassandra git commit: Avoid starting Cassandra on JMX bind failure
Repository: cassandra Updated Branches: refs/heads/cassandra-2.2 3d7328232 -> 93c5bc616 refs/heads/cassandra-3.0 2e202d47d -> 5980b336f refs/heads/cassandra-3.7 415bb569d -> 68580c7a8 refs/heads/trunk d6daf1b66 -> 06cd4e824 Avoid starting Cassandra on JMX bind failure Patch by Alex Petrov; reviewed by Sam Tunnicliffe for CASSANDRA-11540 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/93c5bc61 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/93c5bc61 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/93c5bc61 Branch: refs/heads/cassandra-2.2 Commit: 93c5bc616e21ffa7f31266ad095ca374f2ba73a4 Parents: 3d73282 Author: Alex Petrov Authored: Mon Apr 18 13:47:59 2016 +0200 Committer: Sam Tunnicliffe Committed: Wed May 4 17:55:47 2016 +0100 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/service/CassandraDaemon.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/93c5bc61/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 7c7295d..0d9d3e9 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) http://git-wip-us.apache.org/repos/asf/cassandra/blob/93c5bc61/src/java/org/apache/cassandra/service/CassandraDaemon.java -- diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java b/src/java/org/apache/cassandra/service/CassandraDaemon.java index 6f38545..7e33e9c 100644 --- a/src/java/org/apache/cassandra/service/CassandraDaemon.java +++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java @@ -98,7 +98,7 @@ public class CassandraDaemon logger = LoggerFactory.getLogger(CassandraDaemon.class); } -private static void maybeInitJmx() +private void maybeInitJmx() { if (System.getProperty("com.sun.management.jmxremote.port") != null) return; @@ -119,7 +119,7 @@ public class CassandraDaemon } catch (IOException e) { -logger.error("Error starting local jmx server: ", e); +exitOrFail(1, e.getMessage(), e.getCause()); } }
[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5980b336 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5980b336 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5980b336 Branch: refs/heads/cassandra-3.0 Commit: 5980b336f0511987066ba175b9b52f0eafda77fd Parents: 2e202d4 93c5bc6 Author: Sam Tunnicliffe Authored: Wed May 4 17:58:01 2016 +0100 Committer: Sam Tunnicliffe Committed: Wed May 4 17:58:01 2016 +0100 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/service/CassandraDaemon.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/5980b336/CHANGES.txt -- diff --cc CHANGES.txt index 31e46d9,0d9d3e9..860a4c2 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,25 -1,7 +1,26 @@@ -2.2.7 +3.0.6 + * Disallow creating view with a static column (CASSANDRA-11602) + * Reduce the amount of object allocations caused by the getFunctions methods (CASSANDRA-11593) + * Potential error replaying commitlog with smallint/tinyint/date/time types (CASSANDRA-11618) + * Fix queries with filtering on counter columns (CASSANDRA-11629) + * Improve tombstone printing in sstabledump (CASSANDRA-11655) + * Fix paging for range queries where all clustering columns are specified (CASSANDRA-11669) + * Don't require HEAP_NEW_SIZE to be set when using G1 (CASSANDRA-11600) + * Fix sstabledump not showing cells after tombstone marker (CASSANDRA-11654) + * Ignore all LocalStrategy keyspaces for streaming and other related + operations (CASSANDRA-11627) + * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523) + * Only open one sstable scanner per sstable (CASSANDRA-11412) + * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410) + * ArithmeticException in avgFunctionForDecimal (CASSANDRA-11485) + * LogAwareFileLister should only use OLD sstable files in current folder to determine disk consistency (CASSANDRA-11470) + * Notify indexers of expired rows during compaction (CASSANDRA-11329) + * Properly respond with ProtocolError when a v1/v2 native protocol + header is received (CASSANDRA-11464) + * Validate that num_tokens and initial_token are consistent with one another (CASSANDRA-10120) +Merged from 2.2: + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) - * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) http://git-wip-us.apache.org/repos/asf/cassandra/blob/5980b336/src/java/org/apache/cassandra/service/CassandraDaemon.java --
[10/10] cassandra git commit: Merge branch 'cassandra-3.7' into trunk
Merge branch 'cassandra-3.7' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06cd4e82 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06cd4e82 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06cd4e82 Branch: refs/heads/trunk Commit: 06cd4e824539e266305733e5b3ffee65e42fe602 Parents: d6daf1b 68580c7 Author: Sam Tunnicliffe Authored: Wed May 4 18:02:49 2016 +0100 Committer: Sam Tunnicliffe Committed: Wed May 4 18:02:49 2016 +0100 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/service/CassandraDaemon.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/06cd4e82/CHANGES.txt --
[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5980b336 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5980b336 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5980b336 Branch: refs/heads/cassandra-3.7 Commit: 5980b336f0511987066ba175b9b52f0eafda77fd Parents: 2e202d4 93c5bc6 Author: Sam Tunnicliffe Authored: Wed May 4 17:58:01 2016 +0100 Committer: Sam Tunnicliffe Committed: Wed May 4 17:58:01 2016 +0100 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/service/CassandraDaemon.java | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/5980b336/CHANGES.txt -- diff --cc CHANGES.txt index 31e46d9,0d9d3e9..860a4c2 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,25 -1,7 +1,26 @@@ -2.2.7 +3.0.6 + * Disallow creating view with a static column (CASSANDRA-11602) + * Reduce the amount of object allocations caused by the getFunctions methods (CASSANDRA-11593) + * Potential error replaying commitlog with smallint/tinyint/date/time types (CASSANDRA-11618) + * Fix queries with filtering on counter columns (CASSANDRA-11629) + * Improve tombstone printing in sstabledump (CASSANDRA-11655) + * Fix paging for range queries where all clustering columns are specified (CASSANDRA-11669) + * Don't require HEAP_NEW_SIZE to be set when using G1 (CASSANDRA-11600) + * Fix sstabledump not showing cells after tombstone marker (CASSANDRA-11654) + * Ignore all LocalStrategy keyspaces for streaming and other related + operations (CASSANDRA-11627) + * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523) + * Only open one sstable scanner per sstable (CASSANDRA-11412) + * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410) + * ArithmeticException in avgFunctionForDecimal (CASSANDRA-11485) + * LogAwareFileLister should only use OLD sstable files in current folder to determine disk consistency (CASSANDRA-11470) + * Notify indexers of expired rows during compaction (CASSANDRA-11329) + * Properly respond with ProtocolError when a v1/v2 native protocol + header is received (CASSANDRA-11464) + * Validate that num_tokens and initial_token are consistent with one another (CASSANDRA-10120) +Merged from 2.2: + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) * Produce a heap dump when exiting on OOM (CASSANDRA-9861) - * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) http://git-wip-us.apache.org/repos/asf/cassandra/blob/5980b336/src/java/org/apache/cassandra/service/CassandraDaemon.java --
[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/68580c7a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/68580c7a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/68580c7a Branch: refs/heads/trunk Commit: 68580c7a81fb20ef76abdfbe5eaefcc07b97 Parents: 415bb56 5980b33 Author: Sam Tunnicliffe Authored: Wed May 4 18:02:40 2016 +0100 Committer: Sam Tunnicliffe Committed: Wed May 4 18:02:40 2016 +0100 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/service/CassandraDaemon.java | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/68580c7a/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/68580c7a/src/java/org/apache/cassandra/service/CassandraDaemon.java --
[jira] [Updated] (CASSANDRA-11552) Reduce amount of logging calls from ColumnFamilyStore.selectAndReference
[ https://issues.apache.org/jira/browse/CASSANDRA-11552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-11552: - Fix Version/s: 3.0.7 3.7 2.2.7 2.1.15 > Reduce amount of logging calls from ColumnFamilyStore.selectAndReference > > > Key: CASSANDRA-11552 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11552 > Project: Cassandra > Issue Type: Improvement >Reporter: Robert Stupp >Assignee: Robert Stupp > Fix For: 2.1.15, 2.2.7, 3.7, 3.0.7 > > > {{org.apache.cassandra.db.ColumnFamilyStore#selectAndReference}} logs two > messages at _info_ level "as fast as it can" if it waits for more than 100ms. > The following code is executed in a while-true fashion in this case: > {code} > logger.info("Spinning trying to capture released readers {}", > released); > logger.info("Spinning trying to capture all readers {}", > view.sstables); > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11552) Reduce amount of logging calls from ColumnFamilyStore.selectAndReference
[ https://issues.apache.org/jira/browse/CASSANDRA-11552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-11552: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks! Committed as aacd4526309693233f7f41e966e06ffcc0edf4d0 to 2.1 and merged to 2.2, 3.0, 3.7 and trunk. > Reduce amount of logging calls from ColumnFamilyStore.selectAndReference > > > Key: CASSANDRA-11552 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11552 > Project: Cassandra > Issue Type: Improvement >Reporter: Robert Stupp >Assignee: Robert Stupp > > {{org.apache.cassandra.db.ColumnFamilyStore#selectAndReference}} logs two > messages at _info_ level "as fast as it can" if it waits for more than 100ms. > The following code is executed in a while-true fashion in this case: > {code} > logger.info("Spinning trying to capture released readers {}", > released); > logger.info("Spinning trying to capture all readers {}", > view.sstables); > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[11/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2e202d47 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2e202d47 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2e202d47 Branch: refs/heads/cassandra-3.7 Commit: 2e202d47d34a3a0b91670b7fe7bf0e1e35d20b1b Parents: 7480202 3d73282 Author: Robert Stupp Authored: Wed May 4 18:44:52 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:52 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2e202d47/src/java/org/apache/cassandra/db/ColumnFamilyStore.java --
[02/15] cassandra git commit: Reduce amount of logging calls from ColumnFamilyStore.selectAndReference
Reduce amount of logging calls from ColumnFamilyStore.selectAndReference patch by Robert Stupp; reviewed by Benjamin Lerer for CASSANDRA-11552 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aacd4526 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aacd4526 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aacd4526 Branch: refs/heads/cassandra-2.2 Commit: aacd4526309693233f7f41e966e06ffcc0edf4d0 Parents: 5202147 Author: Robert Stupp Authored: Wed May 4 18:44:08 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:08 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/aacd4526/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index edc3564..b64d5de 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -1941,8 +1941,8 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean for (SSTableReader reader : view.sstables) if (reader.selfRef().globalCount() == 0) released.add(reader); -logger.info("Spinning trying to capture released readers {}", released); -logger.info("Spinning trying to capture all readers {}", view.sstables); +NoSpamLogger.log(logger, NoSpamLogger.Level.WARN, 1, TimeUnit.SECONDS, + "Spinning trying to capture readers {}, released: {}, ", view.sstables, released); failingSince = System.nanoTime(); } }
[10/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2e202d47 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2e202d47 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2e202d47 Branch: refs/heads/trunk Commit: 2e202d47d34a3a0b91670b7fe7bf0e1e35d20b1b Parents: 7480202 3d73282 Author: Robert Stupp Authored: Wed May 4 18:44:52 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:52 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2e202d47/src/java/org/apache/cassandra/db/ColumnFamilyStore.java --
[08/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d732823 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d732823 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d732823 Branch: refs/heads/cassandra-3.0 Commit: 3d7328232552d14daaa9f99d842aa53d92fb51f9 Parents: b189a7f aacd452 Author: Robert Stupp Authored: Wed May 4 18:44:35 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:35 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d732823/src/java/org/apache/cassandra/db/ColumnFamilyStore.java --
[13/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/415bb569 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/415bb569 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/415bb569 Branch: refs/heads/cassandra-3.7 Commit: 415bb569d050c0e15ed8c13bff93602591388f7c Parents: 9e5161b 2e202d4 Author: Robert Stupp Authored: Wed May 4 18:47:04 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:47:04 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/415bb569/src/java/org/apache/cassandra/db/ColumnFamilyStore.java --
[12/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2e202d47 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2e202d47 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2e202d47 Branch: refs/heads/cassandra-3.0 Commit: 2e202d47d34a3a0b91670b7fe7bf0e1e35d20b1b Parents: 7480202 3d73282 Author: Robert Stupp Authored: Wed May 4 18:44:52 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:52 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2e202d47/src/java/org/apache/cassandra/db/ColumnFamilyStore.java --
[05/15] cassandra git commit: Reduce amount of logging calls from ColumnFamilyStore.selectAndReference
Reduce amount of logging calls from ColumnFamilyStore.selectAndReference patch by Robert Stupp; reviewed by Benjamin Lerer for CASSANDRA-11552 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aacd4526 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aacd4526 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aacd4526 Branch: refs/heads/cassandra-3.7 Commit: aacd4526309693233f7f41e966e06ffcc0edf4d0 Parents: 5202147 Author: Robert Stupp Authored: Wed May 4 18:44:08 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:08 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/aacd4526/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index edc3564..b64d5de 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -1941,8 +1941,8 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean for (SSTableReader reader : view.sstables) if (reader.selfRef().globalCount() == 0) released.add(reader); -logger.info("Spinning trying to capture released readers {}", released); -logger.info("Spinning trying to capture all readers {}", view.sstables); +NoSpamLogger.log(logger, NoSpamLogger.Level.WARN, 1, TimeUnit.SECONDS, + "Spinning trying to capture readers {}, released: {}, ", view.sstables, released); failingSince = System.nanoTime(); } }
[03/15] cassandra git commit: Reduce amount of logging calls from ColumnFamilyStore.selectAndReference
Reduce amount of logging calls from ColumnFamilyStore.selectAndReference patch by Robert Stupp; reviewed by Benjamin Lerer for CASSANDRA-11552 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aacd4526 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aacd4526 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aacd4526 Branch: refs/heads/trunk Commit: aacd4526309693233f7f41e966e06ffcc0edf4d0 Parents: 5202147 Author: Robert Stupp Authored: Wed May 4 18:44:08 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:08 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/aacd4526/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index edc3564..b64d5de 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -1941,8 +1941,8 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean for (SSTableReader reader : view.sstables) if (reader.selfRef().globalCount() == 0) released.add(reader); -logger.info("Spinning trying to capture released readers {}", released); -logger.info("Spinning trying to capture all readers {}", view.sstables); +NoSpamLogger.log(logger, NoSpamLogger.Level.WARN, 1, TimeUnit.SECONDS, + "Spinning trying to capture readers {}, released: {}, ", view.sstables, released); failingSince = System.nanoTime(); } }
[04/15] cassandra git commit: Reduce amount of logging calls from ColumnFamilyStore.selectAndReference
Reduce amount of logging calls from ColumnFamilyStore.selectAndReference patch by Robert Stupp; reviewed by Benjamin Lerer for CASSANDRA-11552 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aacd4526 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aacd4526 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aacd4526 Branch: refs/heads/cassandra-3.0 Commit: aacd4526309693233f7f41e966e06ffcc0edf4d0 Parents: 5202147 Author: Robert Stupp Authored: Wed May 4 18:44:08 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:08 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/aacd4526/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index edc3564..b64d5de 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -1941,8 +1941,8 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean for (SSTableReader reader : view.sstables) if (reader.selfRef().globalCount() == 0) released.add(reader); -logger.info("Spinning trying to capture released readers {}", released); -logger.info("Spinning trying to capture all readers {}", view.sstables); +NoSpamLogger.log(logger, NoSpamLogger.Level.WARN, 1, TimeUnit.SECONDS, + "Spinning trying to capture readers {}, released: {}, ", view.sstables, released); failingSince = System.nanoTime(); } }
[14/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.7
Merge branch 'cassandra-3.0' into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/415bb569 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/415bb569 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/415bb569 Branch: refs/heads/trunk Commit: 415bb569d050c0e15ed8c13bff93602591388f7c Parents: 9e5161b 2e202d4 Author: Robert Stupp Authored: Wed May 4 18:47:04 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:47:04 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/415bb569/src/java/org/apache/cassandra/db/ColumnFamilyStore.java --
[09/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d732823 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d732823 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d732823 Branch: refs/heads/cassandra-3.7 Commit: 3d7328232552d14daaa9f99d842aa53d92fb51f9 Parents: b189a7f aacd452 Author: Robert Stupp Authored: Wed May 4 18:44:35 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:35 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d732823/src/java/org/apache/cassandra/db/ColumnFamilyStore.java --
[07/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d732823 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d732823 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d732823 Branch: refs/heads/cassandra-2.2 Commit: 3d7328232552d14daaa9f99d842aa53d92fb51f9 Parents: b189a7f aacd452 Author: Robert Stupp Authored: Wed May 4 18:44:35 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:35 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d732823/src/java/org/apache/cassandra/db/ColumnFamilyStore.java --
[15/15] cassandra git commit: Merge branch 'cassandra-3.7' into trunk
Merge branch 'cassandra-3.7' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d6daf1b6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d6daf1b6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d6daf1b6 Branch: refs/heads/trunk Commit: d6daf1b667b584cf30545cdb5cb3fac726a9cf4b Parents: f5ae572 415bb56 Author: Robert Stupp Authored: Wed May 4 18:47:29 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:47:29 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --
[01/15] cassandra git commit: Reduce amount of logging calls from ColumnFamilyStore.selectAndReference
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 5202147d9 -> aacd45263 refs/heads/cassandra-2.2 b189a7f6c -> 3d7328232 refs/heads/cassandra-3.0 7480202e5 -> 2e202d47d refs/heads/cassandra-3.7 9e5161b67 -> 415bb569d refs/heads/trunk f5ae57292 -> d6daf1b66 Reduce amount of logging calls from ColumnFamilyStore.selectAndReference patch by Robert Stupp; reviewed by Benjamin Lerer for CASSANDRA-11552 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aacd4526 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aacd4526 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aacd4526 Branch: refs/heads/cassandra-2.1 Commit: aacd4526309693233f7f41e966e06ffcc0edf4d0 Parents: 5202147 Author: Robert Stupp Authored: Wed May 4 18:44:08 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:08 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/aacd4526/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index edc3564..b64d5de 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -1941,8 +1941,8 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean for (SSTableReader reader : view.sstables) if (reader.selfRef().globalCount() == 0) released.add(reader); -logger.info("Spinning trying to capture released readers {}", released); -logger.info("Spinning trying to capture all readers {}", view.sstables); +NoSpamLogger.log(logger, NoSpamLogger.Level.WARN, 1, TimeUnit.SECONDS, + "Spinning trying to capture readers {}, released: {}, ", view.sstables, released); failingSince = System.nanoTime(); } }
[06/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d732823 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d732823 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d732823 Branch: refs/heads/trunk Commit: 3d7328232552d14daaa9f99d842aa53d92fb51f9 Parents: b189a7f aacd452 Author: Robert Stupp Authored: Wed May 4 18:44:35 2016 +0200 Committer: Robert Stupp Committed: Wed May 4 18:44:35 2016 +0200 -- src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d732823/src/java/org/apache/cassandra/db/ColumnFamilyStore.java --
[jira] [Updated] (CASSANDRA-10309) Avoid always looking up column type
[ https://issues.apache.org/jira/browse/CASSANDRA-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-10309: --- Assignee: Carl Yeksigian > Avoid always looking up column type > --- > > Key: CASSANDRA-10309 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10309 > Project: Cassandra > Issue Type: Improvement >Reporter: T Jake Luciani >Assignee: Carl Yeksigian >Priority: Minor > Labels: perfomance > Fix For: 3.x > > > Doing some read profiling I noticed we always seem to look up the type of a > column from the schema metadata when we have the type already in the column > class. > This one simple change to SerializationHeader improves read performance > non-trivially. > https://github.com/tjake/cassandra/commit/69b94c389b3f36aa035ac4619fd22d1f62ea80b2 > http://cstar.datastax.com/graph?stats=3fb1ced4-58c7-11e5-9faf-42010af0688f&metric=op_rate&operation=2_read&smoothing=1&show_aggregates=true&xmin=0&xmax=357.94&ymin=0&ymax=157416.6 > I assume we are looking this up to deal with schema changes. But I'm sure > there is a more performant way of doing this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270947#comment-15270947 ] Paulo Motta commented on CASSANDRA-8911: bq. the nice thing about this is that it requires no coordination or scheduling at all - all nodes should run this (slowly) all the time is the idea to have one thread per table running continuously, or a fixed-size pool for all tables? If the second option we may still need to prioritize which table should run MBR next. > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270926#comment-15270926 ] Marcus Eriksson commented on CASSANDRA-8911: thanks [~pauloricardomg] it is not ready for review yet so finish your other things first :) Regarding CASSANDRA-10070 the nice thing about this is that it requires no coordination or scheduling at all - all nodes should run this (slowly) all the time. What we need to figure out is how to automatically size the repair pages and rate limiting/error handling etc. > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9766) Faster Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-9766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270910#comment-15270910 ] T Jake Luciani commented on CASSANDRA-9766: --- I've made those changes, squashed and restarted tests. I'll commit assuming a clean CI. The final streaming improvement is 25% > Faster Streaming > > > Key: CASSANDRA-9766 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9766 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging > Environment: Cassandra 2.1.2. more details in the pdf attached >Reporter: Alexei K >Assignee: T Jake Luciani > Labels: performance > Fix For: 3.x > > Attachments: problem.pdf > > > I have a cluster in Amazon cloud , its described in detail in the attachment. > What I've noticed is that we during bootstrap we never go above 12MB/sec > transmission speeds and also those speeds flat line almost like we're hitting > some sort of a limit ( this remains true for other tests that I've ran) > however during the repair we see much higher,variable sending rates. I've > provided network charts in the attachment as well . Is there an explanation > for this? Is something wrong with my configuration, or is it a possible bug? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10787) OutOfMemoryError after few hours from node restart
[ https://issues.apache.org/jira/browse/CASSANDRA-10787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270908#comment-15270908 ] Piotr Westfalewicz commented on CASSANDRA-10787: [~ifesdjeen], I've deleted those big, potentially problematic sstables and the problem is gone. Since then I've changed compaction strategy to create smaller sstables and the problem never happened again. Hence, it's no longer possible to recreate the issue and it can be closed. > OutOfMemoryError after few hours from node restart > -- > > Key: CASSANDRA-10787 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10787 > Project: Cassandra > Issue Type: Bug > Environment: Amazon DataStax Auto-Clustering AMI 2.6.3-1404-pv > on 2x m1.large instances (2 vCPU, 64-bit, 7.5GB RAM, Raid0 2x420GB Disk) > [cqlsh 5.0.1 | Cassandra 2.2.3 | CQL spec 3.3.1 | Native protocol v4] > RF=3 >Reporter: Piotr Westfalewicz > Fix For: 2.2.x > > Attachments: case2_debuglog_head.txt, case2_debuglog_tail.txt, > case2_systemlog.txt, case3_debuglog_tail.txt, case3_systemlog_tail.txt, > case4_debuglog_tail.txt, case4_systemlog.txt, case5_debuglog.txt, > case5_systemlog.txt > > > Cassandra Cluster was operating flawessly for around 3 months. Lately I've > got a critical problem with it - after few hours of running clients are > disconnected permanently (that may be Datastax C# Driver problem, though), > however few more hours later (with smaller load), on all 2 nodes there is > thrown an exception (details in files): > bq. java.lang.OutOfMemoryError: Java heap space > Cases description: > Case 2 (heavy load): > - 2015-11-26 16:09:40,834 Restarted all nodes in cassandra cluster > - 2015-11-26 17:03:46,774 First client disconnected permanently > - 2015-11-26 22:17:02,327 Node shutdown > Case 3 (unknown load, different node): > - 2015-11-26 02:19:49,585 Node shutdown (visible only in > systemlog, I don't know why not in debug log) > Case 4 (low load): > - 2015-11-27 13:00:24,994 Node restart > - 2015-11-27 22:26:56,131 Node shutdown > Is that a software issue or I am using too weak Amazon instances? If so, how > can the required amount of memory be calculated? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9766) Faster Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-9766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-9766: -- Summary: Faster Streaming (was: Bootstrap outgoing streaming speeds are much slower than during repair) > Faster Streaming > > > Key: CASSANDRA-9766 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9766 > Project: Cassandra > Issue Type: Improvement > Components: Streaming and Messaging > Environment: Cassandra 2.1.2. more details in the pdf attached >Reporter: Alexei K >Assignee: T Jake Luciani > Labels: performance > Fix For: 3.x > > Attachments: problem.pdf > > > I have a cluster in Amazon cloud , its described in detail in the attachment. > What I've noticed is that we during bootstrap we never go above 12MB/sec > transmission speeds and also those speeds flat line almost like we're hitting > some sort of a limit ( this remains true for other tests that I've ran) > however during the repair we see much higher,variable sending rates. I've > provided network charts in the attachment as well . Is there an explanation > for this? Is something wrong with my configuration, or is it a possible bug? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270884#comment-15270884 ] Paulo Motta commented on CASSANDRA-8911: I'd be interested in reviewing this, but since it's quite a big/relevant change another reviewer would probably be welcome (specially in the read-path side, since I'm not totally comfortable with it). Have some priority things on my plate right now, so will have a better look and post comments next week. For bq.* continuously run this we could probably rely on CASSANDRA-10070, which should provide facilities for automatizing repairs independent of implementation. > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11552) Reduce amount of logging calls from ColumnFamilyStore.selectAndReference
[ https://issues.apache.org/jira/browse/CASSANDRA-11552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270874#comment-15270874 ] Benjamin Lerer commented on CASSANDRA-11552: +1 > Reduce amount of logging calls from ColumnFamilyStore.selectAndReference > > > Key: CASSANDRA-11552 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11552 > Project: Cassandra > Issue Type: Improvement >Reporter: Robert Stupp >Assignee: Robert Stupp > > {{org.apache.cassandra.db.ColumnFamilyStore#selectAndReference}} logs two > messages at _info_ level "as fast as it can" if it waits for more than 100ms. > The following code is executed in a while-true fashion in this case: > {code} > logger.info("Spinning trying to capture released readers {}", > released); > logger.info("Spinning trying to capture all readers {}", > view.sstables); > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7056) Add RAMP transactions
[ https://issues.apache.org/jira/browse/CASSANDRA-7056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270846#comment-15270846 ] Cyril Scetbon commented on CASSANDRA-7056: -- [~iamaleksey] Thanks > Add RAMP transactions > - > > Key: CASSANDRA-7056 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7056 > Project: Cassandra > Issue Type: Wish >Reporter: Tupshin Harper >Priority: Minor > Fix For: 3.x > > > We should take a look at > [RAMP|http://www.bailis.org/blog/scalable-atomic-visibility-with-ramp-transactions/] > transactions, and figure out if they can be used to provide more efficient > LWT (or LWT-like) operations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270842#comment-15270842 ] Marcus Eriksson commented on CASSANDRA-8911: No, it still has a few problems - first, we wont be able to eliminate as many sstables during queries based on the min/max clustering values since the all cover a large time span, and second, since "all" sstables will contain very old data, we can't do the "drop this sstable as it only contains expired data"-optimization during compaction since that old data might still be newer than the data in other sstables. > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10070) Automatic repair scheduling
[ https://issues.apache.org/jira/browse/CASSANDRA-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270836#comment-15270836 ] Marcus Olsson commented on CASSANDRA-10070: --- @[~jbellis] bq. How closely does this match the design doc from February? Is it worth posting an updated design for those of us joining late? I'd say there have been enough changes for it to be a good idea to update the document, so I'll work on that! :) > Automatic repair scheduling > --- > > Key: CASSANDRA-10070 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10070 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Olsson >Assignee: Marcus Olsson >Priority: Minor > Fix For: 3.x > > Attachments: Distributed Repair Scheduling.doc > > > Scheduling and running repairs in a Cassandra cluster is most often a > required task, but this can both be hard for new users and it also requires a > bit of manual configuration. There are good tools out there that can be used > to simplify things, but wouldn't this be a good feature to have inside of > Cassandra? To automatically schedule and run repairs, so that when you start > up your cluster it basically maintains itself in terms of normal > anti-entropy, with the possibility for manual configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11678) cassandra crush when enable hints_compression
[ https://issues.apache.org/jira/browse/CASSANDRA-11678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-11678: -- Reviewer: Aleksey Yeschenko > cassandra crush when enable hints_compression > - > > Key: CASSANDRA-11678 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11678 > Project: Cassandra > Issue Type: Bug > Components: Core, Local Write-Read Paths > Environment: Centos 7 >Reporter: Weijian Lin >Assignee: Blake Eggleston >Priority: Critical > > When I enable hints_compression and set the compression class to > LZ4Compressor,the > cassandra (v3.05, V3.5.0) will crush。That is a bug, or any conf is wrong? > *Exception in V 3.5.0 * > {code} > ERROR [HintsDispatcher:2] 2016-04-26 15:02:56,970 > HintsDispatchExecutor.java:225 - Failed to dispatch hints file > abc4dda2-b551-427e-bb0b-e383d4a392e1-1461654138963-1.hints: file is > corrupted ({}) > org.apache.cassandra.io.FSReadError: java.io.EOFException > at > org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:284) > ~[apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:254) > ~[apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:156) > ~[apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:137) > ~[apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:119) > ~[apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:91) > ~[apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.deliver(HintsDispatchExecutor.java:259) > [apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242) > [apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:220) > [apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:199) > [apache-cassandra-3.5.0.jar:3.5.0] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_65] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_65] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_65] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65] > Caused by: java.io.EOFException: null > at > org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:146) > ~[apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.io.util.RebufferingInputStream.readPrimitiveSlowly(RebufferingInputStream.java:108) > ~[apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.io.util.RebufferingInputStream.readInt(RebufferingInputStream.java:188) > ~[apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNextInternal(HintsReader.java:297) > ~[apache-cassandra-3.5.0.jar:3.5.0] > at > org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:280) > ~[apache-cassandra-3.5.0.jar:3.5.0] > ... 15 common frames omitted > {code} > *Exception in V 3.0.5 * > {code} > ERROR [HintsDispatcher:2] 2016-04-26 15:54:46,294 > HintsDispatchExecutor.java:225 - Failed to dispatch hints file > 8603be13-6878-4de3-8bc3-a7a7146b0376-1461657251205-1.hints: file is > corrupted ({}) > org.apache.cassandra.io.FSReadError: java.io.EOFException > at > org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:282) > ~[apache-cassandra-3.0.5.jar:3.0.5] > at > org.apache.cassandra.hints.HintsReader$BuffersIterator.computeNext(HintsReader.java:252) > ~[apache-cassandra-3.0.5.jar:3.0.5] > at > org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) > ~[apache-cassandra-3.0.5.jar:3.0.5] > at > org.apache.cassandra.hints.HintsDispatcher.sendHints(HintsDispatcher.java:156) > ~[apache-cassandra-3.0.5.jar:3.0.5] > at > org.apache.cassandra.hints.HintsDispatcher.sendHintsAndAwait(HintsDispatcher.java:137) > ~[apache-cassandra-3.0.5.jar:3.0.5] > at > org.apache.cassandra.hints.HintsDispatcher.dispatch(HintsDispatcher.java:119) > ~[apache-cassandra-3.0.5.jar
[jira] [Commented] (CASSANDRA-11702) dtest failure in pushed_notifications_test.TestPushedNotifications.restart_node_localhost_test
[ https://issues.apache.org/jira/browse/CASSANDRA-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270835#comment-15270835 ] Joel Knighton commented on CASSANDRA-11702: --- For another datapoint, same failure on a 2.2 branch that didn't touch related functionality [here|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-10962-2.2-dtest/2/testReport/junit/pushed_notifications_test/TestPushedNotifications/restart_node_localhost_test/]. > dtest failure in > pushed_notifications_test.TestPushedNotifications.restart_node_localhost_test > -- > > Key: CASSANDRA-11702 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11702 > Project: Cassandra > Issue Type: Test >Reporter: Russ Hatch >Assignee: DS Test Eng > Labels: dtest > > from offheap test job, failure. looks like it could be a routine timeout, but > I think I saw the exact same timeout for this test on another job. > {noformat} > ('Unable to connect to any servers', {}) > {noformat} > http://cassci.datastax.com/job/trunk_offheap_dtest/178/testReport/pushed_notifications_test/TestPushedNotifications/restart_node_localhost_test > Failed on CassCI build trunk_offheap_dtest #178 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-9861: -- Fix Version/s: (was: 2.2.x) 3.0.7 3.7 2.2.7 > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Bug > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.7, 3.7, 3.0.7 > > Attachments: 9861-2.2-V2.txt, 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-9861: -- Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed into 2.2 at b189a7f6c955908d8b79d2b4104562a38ea62979 and merged into 3.0, 3.7 and trunk > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Bug > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.x > > Attachments: 9861-2.2-V2.txt, 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9861) When forcibly exiting due to OOM, we should produce a heap dump
[ https://issues.apache.org/jira/browse/CASSANDRA-9861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270821#comment-15270821 ] Benjamin Lerer commented on CASSANDRA-9861: --- Thanks for the review. > When forcibly exiting due to OOM, we should produce a heap dump > --- > > Key: CASSANDRA-9861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9861 > Project: Cassandra > Issue Type: Bug > Components: Lifecycle >Reporter: Benedict >Assignee: Benjamin Lerer >Priority: Minor > Labels: lhf > Fix For: 2.2.x > > Attachments: 9861-2.2-V2.txt, 9861-2.2.txt > > > CASSANDRA-7507 introduced earlier termination on encountering an OOM, due to > lack of certainty about system state. However a side effect is that we never > produce heap dumps on OOM. We should ideally try to produce one forcibly > before exiting. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[4/4] cassandra git commit: Merge branch cassandra-3.7 into trunk
Merge branch cassandra-3.7 into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f5ae5729 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f5ae5729 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f5ae5729 Branch: refs/heads/trunk Commit: f5ae572929dd6a44dbfa9217b2b5636b327d693b Parents: 17d30c9 9e5161b Author: Benjamin Lerer Authored: Wed May 4 17:20:02 2016 +0200 Committer: Benjamin Lerer Committed: Wed May 4 17:20:14 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/utils/HeapUtils.java | 205 +++ .../cassandra/utils/JVMStabilityInspector.java | 3 + 3 files changed, 209 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f5ae5729/CHANGES.txt --
[2/4] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0
Merge branch cassandra-2.2 into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7480202e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7480202e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7480202e Branch: refs/heads/trunk Commit: 7480202e540f86ae4ff02c9a67253d90ad8dde29 Parents: c7d216e b189a7f Author: Benjamin Lerer Authored: Wed May 4 17:17:08 2016 +0200 Committer: Benjamin Lerer Committed: Wed May 4 17:17:08 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/utils/HeapUtils.java | 205 +++ .../cassandra/utils/JVMStabilityInspector.java | 5 +- 3 files changed, 209 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7480202e/CHANGES.txt -- diff --cc CHANGES.txt index f947568,7c7295d..31e46d9 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,24 -1,6 +1,25 @@@ -2.2.7 +3.0.6 + * Disallow creating view with a static column (CASSANDRA-11602) + * Reduce the amount of object allocations caused by the getFunctions methods (CASSANDRA-11593) + * Potential error replaying commitlog with smallint/tinyint/date/time types (CASSANDRA-11618) + * Fix queries with filtering on counter columns (CASSANDRA-11629) + * Improve tombstone printing in sstabledump (CASSANDRA-11655) + * Fix paging for range queries where all clustering columns are specified (CASSANDRA-11669) + * Don't require HEAP_NEW_SIZE to be set when using G1 (CASSANDRA-11600) + * Fix sstabledump not showing cells after tombstone marker (CASSANDRA-11654) + * Ignore all LocalStrategy keyspaces for streaming and other related + operations (CASSANDRA-11627) + * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523) + * Only open one sstable scanner per sstable (CASSANDRA-11412) + * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410) + * ArithmeticException in avgFunctionForDecimal (CASSANDRA-11485) + * LogAwareFileLister should only use OLD sstable files in current folder to determine disk consistency (CASSANDRA-11470) + * Notify indexers of expired rows during compaction (CASSANDRA-11329) + * Properly respond with ProtocolError when a v1/v2 native protocol + header is received (CASSANDRA-11464) + * Validate that num_tokens and initial_token are consistent with one another (CASSANDRA-10120) +Merged from 2.2: + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) - * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7480202e/src/java/org/apache/cassandra/utils/JVMStabilityInspector.java --
[1/4] cassandra git commit: Produce a heap dump when exiting on OO a heap dump when exiting on OOM
Repository: cassandra Updated Branches: refs/heads/trunk 17d30c9e5 -> f5ae57292 Produce a heap dump when exiting on OO a heap dump when exiting on OOM patch by Benjamin Lerer; reviewed by Aleksey Yeschenko for CASSANDRA-9861 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b189a7f6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b189a7f6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b189a7f6 Branch: refs/heads/trunk Commit: b189a7f6c955908d8b79d2b4104562a38ea62979 Parents: 8bf453a Author: Benjamin Lerer Authored: Thu Mar 31 15:23:33 2016 +0200 Committer: Benjamin Lerer Committed: Wed May 4 17:10:58 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/utils/HeapUtils.java | 205 +++ .../cassandra/utils/JVMStabilityInspector.java | 5 +- 3 files changed, 209 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b189a7f6/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 19e1afe..7c7295d 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b189a7f6/src/java/org/apache/cassandra/utils/HeapUtils.java -- diff --git a/src/java/org/apache/cassandra/utils/HeapUtils.java b/src/java/org/apache/cassandra/utils/HeapUtils.java new file mode 100644 index 000..bfc8a0b --- /dev/null +++ b/src/java/org/apache/cassandra/utils/HeapUtils.java @@ -0,0 +1,205 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.utils; + +import java.io.*; +import java.lang.management.ManagementFactory; +import java.lang.management.RuntimeMXBean; +import java.nio.file.FileSystems; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; + +import org.apache.commons.lang3.ArrayUtils; +import org.apache.commons.lang3.text.StrBuilder; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Utility to generate heap dumps. + * + */ +public final class HeapUtils +{ +private static final Logger logger = LoggerFactory.getLogger(HeapUtils.class); + +/** + * Generates a HEAP dump in the directory specified by the HeapDumpPath JVM option + * or in the CASSANDRA_HOME directory. + */ +public static void generateHeapDump() +{ +Long processId = getProcessId(); +if (processId == null) +{ +logger.error("The process ID could not be retrieved. Skipping heap dump generation."); +return; +} + +String heapDumpPath = getHeapDumpPathOption(); +if (heapDumpPath == null) +{ +String cassandraHome = System.getenv("CASSANDRA_HOME"); +if (cassandraHome == null) +{ +return; +} + +heapDumpPath = cassandraHome; +} + +Path dumpPath = FileSystems.getDefault().getPath(heapDumpPath); +if (Files.isDirectory(dumpPath)) +{ +dumpPath = dumpPath.resolve("java_pid" + processId + ".hprof"); +} + +String jmapPath = getJmapPath(); + +// The jmap file could not be found. In this case let's default to jmap in the hope that it is in the path. +String jmapCommand = jmapPath == null ? "jmap" : jmapPath; + +String[] dumpCommands = new String[] {jmapCommand, + "-dump:format=b,file=" + dumpPath, + processId.toString()}; + +// Lets also log the Heap histogram +
[3/4] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.7
Merge branch cassandra-3.0 into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9e5161b6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9e5161b6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9e5161b6 Branch: refs/heads/trunk Commit: 9e5161b6752611515e3e349a7bc4d3f76f8d15d4 Parents: 54d7fc4 7480202 Author: Benjamin Lerer Authored: Wed May 4 17:18:18 2016 +0200 Committer: Benjamin Lerer Committed: Wed May 4 17:18:32 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/utils/HeapUtils.java | 205 +++ .../cassandra/utils/JVMStabilityInspector.java | 3 + 3 files changed, 209 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e5161b6/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e5161b6/src/java/org/apache/cassandra/utils/JVMStabilityInspector.java --
[2/3] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0
Merge branch cassandra-2.2 into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7480202e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7480202e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7480202e Branch: refs/heads/cassandra-3.7 Commit: 7480202e540f86ae4ff02c9a67253d90ad8dde29 Parents: c7d216e b189a7f Author: Benjamin Lerer Authored: Wed May 4 17:17:08 2016 +0200 Committer: Benjamin Lerer Committed: Wed May 4 17:17:08 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/utils/HeapUtils.java | 205 +++ .../cassandra/utils/JVMStabilityInspector.java | 5 +- 3 files changed, 209 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7480202e/CHANGES.txt -- diff --cc CHANGES.txt index f947568,7c7295d..31e46d9 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,24 -1,6 +1,25 @@@ -2.2.7 +3.0.6 + * Disallow creating view with a static column (CASSANDRA-11602) + * Reduce the amount of object allocations caused by the getFunctions methods (CASSANDRA-11593) + * Potential error replaying commitlog with smallint/tinyint/date/time types (CASSANDRA-11618) + * Fix queries with filtering on counter columns (CASSANDRA-11629) + * Improve tombstone printing in sstabledump (CASSANDRA-11655) + * Fix paging for range queries where all clustering columns are specified (CASSANDRA-11669) + * Don't require HEAP_NEW_SIZE to be set when using G1 (CASSANDRA-11600) + * Fix sstabledump not showing cells after tombstone marker (CASSANDRA-11654) + * Ignore all LocalStrategy keyspaces for streaming and other related + operations (CASSANDRA-11627) + * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523) + * Only open one sstable scanner per sstable (CASSANDRA-11412) + * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410) + * ArithmeticException in avgFunctionForDecimal (CASSANDRA-11485) + * LogAwareFileLister should only use OLD sstable files in current folder to determine disk consistency (CASSANDRA-11470) + * Notify indexers of expired rows during compaction (CASSANDRA-11329) + * Properly respond with ProtocolError when a v1/v2 native protocol + header is received (CASSANDRA-11464) + * Validate that num_tokens and initial_token are consistent with one another (CASSANDRA-10120) +Merged from 2.2: + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) - * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7480202e/src/java/org/apache/cassandra/utils/JVMStabilityInspector.java --
[1/3] cassandra git commit: Produce a heap dump when exiting on OO a heap dump when exiting on OOM
Repository: cassandra Updated Branches: refs/heads/cassandra-3.7 54d7fc4b0 -> 9e5161b67 Produce a heap dump when exiting on OO a heap dump when exiting on OOM patch by Benjamin Lerer; reviewed by Aleksey Yeschenko for CASSANDRA-9861 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b189a7f6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b189a7f6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b189a7f6 Branch: refs/heads/cassandra-3.7 Commit: b189a7f6c955908d8b79d2b4104562a38ea62979 Parents: 8bf453a Author: Benjamin Lerer Authored: Thu Mar 31 15:23:33 2016 +0200 Committer: Benjamin Lerer Committed: Wed May 4 17:10:58 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/utils/HeapUtils.java | 205 +++ .../cassandra/utils/JVMStabilityInspector.java | 5 +- 3 files changed, 209 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b189a7f6/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 19e1afe..7c7295d 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b189a7f6/src/java/org/apache/cassandra/utils/HeapUtils.java -- diff --git a/src/java/org/apache/cassandra/utils/HeapUtils.java b/src/java/org/apache/cassandra/utils/HeapUtils.java new file mode 100644 index 000..bfc8a0b --- /dev/null +++ b/src/java/org/apache/cassandra/utils/HeapUtils.java @@ -0,0 +1,205 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.utils; + +import java.io.*; +import java.lang.management.ManagementFactory; +import java.lang.management.RuntimeMXBean; +import java.nio.file.FileSystems; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; + +import org.apache.commons.lang3.ArrayUtils; +import org.apache.commons.lang3.text.StrBuilder; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Utility to generate heap dumps. + * + */ +public final class HeapUtils +{ +private static final Logger logger = LoggerFactory.getLogger(HeapUtils.class); + +/** + * Generates a HEAP dump in the directory specified by the HeapDumpPath JVM option + * or in the CASSANDRA_HOME directory. + */ +public static void generateHeapDump() +{ +Long processId = getProcessId(); +if (processId == null) +{ +logger.error("The process ID could not be retrieved. Skipping heap dump generation."); +return; +} + +String heapDumpPath = getHeapDumpPathOption(); +if (heapDumpPath == null) +{ +String cassandraHome = System.getenv("CASSANDRA_HOME"); +if (cassandraHome == null) +{ +return; +} + +heapDumpPath = cassandraHome; +} + +Path dumpPath = FileSystems.getDefault().getPath(heapDumpPath); +if (Files.isDirectory(dumpPath)) +{ +dumpPath = dumpPath.resolve("java_pid" + processId + ".hprof"); +} + +String jmapPath = getJmapPath(); + +// The jmap file could not be found. In this case let's default to jmap in the hope that it is in the path. +String jmapCommand = jmapPath == null ? "jmap" : jmapPath; + +String[] dumpCommands = new String[] {jmapCommand, + "-dump:format=b,file=" + dumpPath, + processId.toString()}; + +// Lets also log the Heap hi
[3/3] cassandra git commit: Merge branch cassandra-3.0 into cassandra-3.7
Merge branch cassandra-3.0 into cassandra-3.7 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9e5161b6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9e5161b6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9e5161b6 Branch: refs/heads/cassandra-3.7 Commit: 9e5161b6752611515e3e349a7bc4d3f76f8d15d4 Parents: 54d7fc4 7480202 Author: Benjamin Lerer Authored: Wed May 4 17:18:18 2016 +0200 Committer: Benjamin Lerer Committed: Wed May 4 17:18:32 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/utils/HeapUtils.java | 205 +++ .../cassandra/utils/JVMStabilityInspector.java | 3 + 3 files changed, 209 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e5161b6/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9e5161b6/src/java/org/apache/cassandra/utils/JVMStabilityInspector.java --
[1/2] cassandra git commit: Produce a heap dump when exiting on OO a heap dump when exiting on OOM
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 c7d216e3f -> 7480202e5 Produce a heap dump when exiting on OO a heap dump when exiting on OOM patch by Benjamin Lerer; reviewed by Aleksey Yeschenko for CASSANDRA-9861 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b189a7f6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b189a7f6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b189a7f6 Branch: refs/heads/cassandra-3.0 Commit: b189a7f6c955908d8b79d2b4104562a38ea62979 Parents: 8bf453a Author: Benjamin Lerer Authored: Thu Mar 31 15:23:33 2016 +0200 Committer: Benjamin Lerer Committed: Wed May 4 17:10:58 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/utils/HeapUtils.java | 205 +++ .../cassandra/utils/JVMStabilityInspector.java | 5 +- 3 files changed, 209 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b189a7f6/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 19e1afe..7c7295d 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b189a7f6/src/java/org/apache/cassandra/utils/HeapUtils.java -- diff --git a/src/java/org/apache/cassandra/utils/HeapUtils.java b/src/java/org/apache/cassandra/utils/HeapUtils.java new file mode 100644 index 000..bfc8a0b --- /dev/null +++ b/src/java/org/apache/cassandra/utils/HeapUtils.java @@ -0,0 +1,205 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.utils; + +import java.io.*; +import java.lang.management.ManagementFactory; +import java.lang.management.RuntimeMXBean; +import java.nio.file.FileSystems; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; + +import org.apache.commons.lang3.ArrayUtils; +import org.apache.commons.lang3.text.StrBuilder; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Utility to generate heap dumps. + * + */ +public final class HeapUtils +{ +private static final Logger logger = LoggerFactory.getLogger(HeapUtils.class); + +/** + * Generates a HEAP dump in the directory specified by the HeapDumpPath JVM option + * or in the CASSANDRA_HOME directory. + */ +public static void generateHeapDump() +{ +Long processId = getProcessId(); +if (processId == null) +{ +logger.error("The process ID could not be retrieved. Skipping heap dump generation."); +return; +} + +String heapDumpPath = getHeapDumpPathOption(); +if (heapDumpPath == null) +{ +String cassandraHome = System.getenv("CASSANDRA_HOME"); +if (cassandraHome == null) +{ +return; +} + +heapDumpPath = cassandraHome; +} + +Path dumpPath = FileSystems.getDefault().getPath(heapDumpPath); +if (Files.isDirectory(dumpPath)) +{ +dumpPath = dumpPath.resolve("java_pid" + processId + ".hprof"); +} + +String jmapPath = getJmapPath(); + +// The jmap file could not be found. In this case let's default to jmap in the hope that it is in the path. +String jmapCommand = jmapPath == null ? "jmap" : jmapPath; + +String[] dumpCommands = new String[] {jmapCommand, + "-dump:format=b,file=" + dumpPath, + processId.toString()}; + +// Lets also log the Heap hi
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270811#comment-15270811 ] T Jake Luciani commented on CASSANDRA-8911: --- I'll volunteer to write some large scale tests for this feature > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11258) Repair scheduling - Resource locking API
[ https://issues.apache.org/jira/browse/CASSANDRA-11258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270807#comment-15270807 ] Marcus Olsson commented on CASSANDRA-11258: --- I've added the check for if the lease was acquired in the case of a WTE and also added the mentioned tests to the same branch as before. I also did some other changes: * Added javadoc to package/classes (based on the ongoing proposal in the dev mailing list regarding mandatory comments). * Changed from insert to update when doing CAS for the lease. The reason for changing from {{insert}} to {{update}} was that with the previous {{insert}} a row marker is added with a TTL of 60(default TTL) and it doesn't get updated when renewing the lease since that uses an {{update}}. So if we would "forget" a lease or are unable to renew it in time we would have a row marker that is blocking us from acquiring the lease until the TTL has expired. Another alternative would be to do a {{SERIAL}} read and check if the host column isn't set and then either delete it and retry or use update with CAS. Always using update seemed cleaner for the implementation, but it might be that this is a small enough issue that the 60 second wait isn't a problem since it's only the first 60 seconds from when the lease was acquired. bq. While it would be interesting to add dtests to make sure leases are working in a distributed environment, we would probably need to expose the LeaseFactory interface over JMX, but we want to keep this strictly internal to avoid external misuse. So it's probably better to move on and test this more extensively on dtests via scheduled repairs, for instance by triggering simultaneous scheduled repair requests and modifying the resource lease tables manually to test leases in scenarios with tampering, network partitions and nodes out of sync, so let's leave this for a future task. I had the same thoughts about dtest and it makes more sense to wait until we have scheduled repairs until we test it like that. --- Since the comments/implementation details regarding update vs insert shouldn't affect the interface as such I'll start implementing the repair scheduling. :) > Repair scheduling - Resource locking API > > > Key: CASSANDRA-11258 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11258 > Project: Cassandra > Issue Type: Sub-task >Reporter: Marcus Olsson >Assignee: Marcus Olsson >Priority: Minor > > Create a resource locking API & implementation that is able to lock a > resource in a specified data center. It should handle priorities to avoid > node starvation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/2] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0
Merge branch cassandra-2.2 into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7480202e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7480202e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7480202e Branch: refs/heads/cassandra-3.0 Commit: 7480202e540f86ae4ff02c9a67253d90ad8dde29 Parents: c7d216e b189a7f Author: Benjamin Lerer Authored: Wed May 4 17:17:08 2016 +0200 Committer: Benjamin Lerer Committed: Wed May 4 17:17:08 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/utils/HeapUtils.java | 205 +++ .../cassandra/utils/JVMStabilityInspector.java | 5 +- 3 files changed, 209 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/7480202e/CHANGES.txt -- diff --cc CHANGES.txt index f947568,7c7295d..31e46d9 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,24 -1,6 +1,25 @@@ -2.2.7 +3.0.6 + * Disallow creating view with a static column (CASSANDRA-11602) + * Reduce the amount of object allocations caused by the getFunctions methods (CASSANDRA-11593) + * Potential error replaying commitlog with smallint/tinyint/date/time types (CASSANDRA-11618) + * Fix queries with filtering on counter columns (CASSANDRA-11629) + * Improve tombstone printing in sstabledump (CASSANDRA-11655) + * Fix paging for range queries where all clustering columns are specified (CASSANDRA-11669) + * Don't require HEAP_NEW_SIZE to be set when using G1 (CASSANDRA-11600) + * Fix sstabledump not showing cells after tombstone marker (CASSANDRA-11654) + * Ignore all LocalStrategy keyspaces for streaming and other related + operations (CASSANDRA-11627) + * Ensure columnfilter covers indexed columns for thrift 2i queries (CASSANDRA-11523) + * Only open one sstable scanner per sstable (CASSANDRA-11412) + * Option to specify ProtocolVersion in cassandra-stress (CASSANDRA-11410) + * ArithmeticException in avgFunctionForDecimal (CASSANDRA-11485) + * LogAwareFileLister should only use OLD sstable files in current folder to determine disk consistency (CASSANDRA-11470) + * Notify indexers of expired rows during compaction (CASSANDRA-11329) + * Properly respond with ProtocolError when a v1/v2 native protocol + header is received (CASSANDRA-11464) + * Validate that num_tokens and initial_token are consistent with one another (CASSANDRA-10120) +Merged from 2.2: + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) - * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) http://git-wip-us.apache.org/repos/asf/cassandra/blob/7480202e/src/java/org/apache/cassandra/utils/JVMStabilityInspector.java --
cassandra git commit: Produce a heap dump when exiting on OO a heap dump when exiting on OOM
Repository: cassandra Updated Branches: refs/heads/cassandra-2.2 8bf453a44 -> b189a7f6c Produce a heap dump when exiting on OO a heap dump when exiting on OOM patch by Benjamin Lerer; reviewed by Aleksey Yeschenko for CASSANDRA-9861 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b189a7f6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b189a7f6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b189a7f6 Branch: refs/heads/cassandra-2.2 Commit: b189a7f6c955908d8b79d2b4104562a38ea62979 Parents: 8bf453a Author: Benjamin Lerer Authored: Thu Mar 31 15:23:33 2016 +0200 Committer: Benjamin Lerer Committed: Wed May 4 17:10:58 2016 +0200 -- CHANGES.txt | 1 + .../org/apache/cassandra/utils/HeapUtils.java | 205 +++ .../cassandra/utils/JVMStabilityInspector.java | 5 +- 3 files changed, 209 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b189a7f6/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 19e1afe..7c7295d 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.2.7 + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) * JSON datetime formatting needs timezone (CASSANDRA-11137) http://git-wip-us.apache.org/repos/asf/cassandra/blob/b189a7f6/src/java/org/apache/cassandra/utils/HeapUtils.java -- diff --git a/src/java/org/apache/cassandra/utils/HeapUtils.java b/src/java/org/apache/cassandra/utils/HeapUtils.java new file mode 100644 index 000..bfc8a0b --- /dev/null +++ b/src/java/org/apache/cassandra/utils/HeapUtils.java @@ -0,0 +1,205 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.utils; + +import java.io.*; +import java.lang.management.ManagementFactory; +import java.lang.management.RuntimeMXBean; +import java.nio.file.FileSystems; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; + +import org.apache.commons.lang3.ArrayUtils; +import org.apache.commons.lang3.text.StrBuilder; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Utility to generate heap dumps. + * + */ +public final class HeapUtils +{ +private static final Logger logger = LoggerFactory.getLogger(HeapUtils.class); + +/** + * Generates a HEAP dump in the directory specified by the HeapDumpPath JVM option + * or in the CASSANDRA_HOME directory. + */ +public static void generateHeapDump() +{ +Long processId = getProcessId(); +if (processId == null) +{ +logger.error("The process ID could not be retrieved. Skipping heap dump generation."); +return; +} + +String heapDumpPath = getHeapDumpPathOption(); +if (heapDumpPath == null) +{ +String cassandraHome = System.getenv("CASSANDRA_HOME"); +if (cassandraHome == null) +{ +return; +} + +heapDumpPath = cassandraHome; +} + +Path dumpPath = FileSystems.getDefault().getPath(heapDumpPath); +if (Files.isDirectory(dumpPath)) +{ +dumpPath = dumpPath.resolve("java_pid" + processId + ".hprof"); +} + +String jmapPath = getJmapPath(); + +// The jmap file could not be found. In this case let's default to jmap in the hope that it is in the path. +String jmapCommand = jmapPath == null ? "jmap" : jmapPath; + +String[] dumpCommands = new String[] {jmapCommand, + "-dump:format=b,file=" + dumpPath, + processId.toString()}; + +// Lets also log the Heap hi
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270790#comment-15270790 ] Jon Haddad commented on CASSANDRA-8911: --- Regarding DTCS, wouldn't that have been solved with CASSANDRA-11056, if it had been merged? > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270766#comment-15270766 ] Marcus Eriksson commented on CASSANDRA-8911: Have pushed a bunch new commits to the [branch|https://github.com/krummas/cassandra/commits/marcuse/8911] Apart from a bunch of fixes, it adds a nodetool command to trigger a repair {{$ nodetool -p7100 mutationbasedrepair stresscql datatest -w 2000 -r 8000}} where -w is the page size and -r is the number of rows per second to repair. Also pushed a bunch of dtests [here|https://github.com/krummas/cassandra-dtest/commits/marcuse/8911] My plan for this is: * Add a separate memtable without commitlog, instead we record the last repaired page when we flush this memtable, so if we lose the memtable, we will start repairing from the point where we flushed the memtable last time. We can probably also skip serving reads from this separate memtable. * Make it impossible to start if you run DTCS * Clean up code, get it reviewed etc. * Release it as an "experimental" feature * Create new tickets: ** make it incremental ** continuously run this ** investigate how to handle gc grace ** make it safe to use with DTCS Reason I prefer to release it incrementally like this is that I don't want us to waste a lot of time if the approach does not really work in real life > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement >Reporter: Tyler Hobbs >Assignee: Marcus Eriksson > Fix For: 3.x > > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11712) testJsonThreadSafety is failing / flapping
Alex Petrov created CASSANDRA-11712: --- Summary: testJsonThreadSafety is failing / flapping Key: CASSANDRA-11712 URL: https://issues.apache.org/jira/browse/CASSANDRA-11712 Project: Cassandra Issue Type: Bug Components: Testing Reporter: Alex Petrov Priority: Minor {{JsonTest::testJsonThreadSafety}} is failing quite often recently: https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-11540-2.2-testall/lastCompletedBuild/testReport/org.apache.cassandra.cql3.validation.entities/JsonTest/testJsonThreadSafety/ Output looks like {code} Stacktrace java.util.concurrent.TimeoutException at java.util.concurrent.FutureTask.get(FutureTask.java:201) at org.apache.cassandra.cql3.validation.entities.JsonTest.testJsonThreadSafety(JsonTest.java:1028) WARN 12:19:23 Small commitlog volume detected at build/test/cassandra/commitlog:30; setting commitlog_total_space_in_mb to 1982. You can override this in cassandra.yaml WARN 12:19:23 Small commitlog volume detected at build/test/cassandra/commitlog:30; setting commitlog_total_space_in_mb to 1982. You can override this in cassandra.yaml WARN 12:19:23 Only 5581 MB free across all data volumes. Consider adding more capacity to your cluster or removing obsolete snapshots WARN 12:19:23 Only 5581 MB free across all data volumes. Consider adding more capacity to your cluster or removing obsolete snapshots WARN 12:19:26 Aggregation query used without partition key WARN 12:19:26 Aggregation query used without partition key WARN 12:19:26 Aggregation query used without partition key WARN 12:19:26 Aggregation query used without partition key Seed 889742091470 {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11711) testJsonThreadSafety is failing / flapping
Alex Petrov created CASSANDRA-11711: --- Summary: testJsonThreadSafety is failing / flapping Key: CASSANDRA-11711 URL: https://issues.apache.org/jira/browse/CASSANDRA-11711 Project: Cassandra Issue Type: Bug Components: Testing Reporter: Alex Petrov Priority: Minor {{JsonTest::testJsonThreadSafety}} is failing quite often recently: https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-11540-2.2-testall/lastCompletedBuild/testReport/org.apache.cassandra.cql3.validation.entities/JsonTest/testJsonThreadSafety/ Output looks like {code} Stacktrace java.util.concurrent.TimeoutException at java.util.concurrent.FutureTask.get(FutureTask.java:201) at org.apache.cassandra.cql3.validation.entities.JsonTest.testJsonThreadSafety(JsonTest.java:1028) WARN 12:19:23 Small commitlog volume detected at build/test/cassandra/commitlog:30; setting commitlog_total_space_in_mb to 1982. You can override this in cassandra.yaml WARN 12:19:23 Small commitlog volume detected at build/test/cassandra/commitlog:30; setting commitlog_total_space_in_mb to 1982. You can override this in cassandra.yaml WARN 12:19:23 Only 5581 MB free across all data volumes. Consider adding more capacity to your cluster or removing obsolete snapshots WARN 12:19:23 Only 5581 MB free across all data volumes. Consider adding more capacity to your cluster or removing obsolete snapshots WARN 12:19:26 Aggregation query used without partition key WARN 12:19:26 Aggregation query used without partition key WARN 12:19:26 Aggregation query used without partition key WARN 12:19:26 Aggregation query used without partition key Seed 889742091470 {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11679) Cassandra Driver returns different number of results depending on fetchsize
[ https://issues.apache.org/jira/browse/CASSANDRA-11679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-11679: --- Fix Version/s: 2.1.x > Cassandra Driver returns different number of results depending on fetchsize > --- > > Key: CASSANDRA-11679 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11679 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Varun Barala >Assignee: Benjamin Lerer > Fix For: 2.1.x > > Attachments: 11679-2.1.txt > > > I'm trying to fetch all distinct keys from a CF using cassandra-driver > (2.1.7.1) and I observed some strange behavior :- > The total distinct rows are 498 so If I perform a query get All distinctKeys > It returns 503 instead of 498(five keys twice). > But If I define the fetch size in select statement more than 498 then it > returns exact 498 rows. > And If I execute same statement on Dev-center it returns 498 rows (because > the default fetch size is 5000). In `cqlsh` it returns 503 rows (because > cqlsh uses fetch size=100). > Some Additional and useful information :- > --- > Cassandra-2.1.13 (C)* version > Consistency level: ONE > local machine(ubuntu 14.04) > Table Schema:- > -- > {code:xml} > CREATE TABLE sample ( > pk1 text, > pk2 text, > row_id uuid, > value blob, > PRIMARY KEY (( pk1, pk2)) > ) WITH bloom_filter_fp_chance = 0.01 > AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}' > AND comment = '' > AND compaction = {'class': > 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'} > AND compression = {'sstable_compression': > 'org.apache.cassandra.io.compress.LZ4Compressor'} > AND dclocal_read_repair_chance = 0.1 > AND default_time_to_live = 0 > AND gc_grace_seconds = 864000 > AND max_index_interval = 2048 > AND memtable_flush_period_in_ms = 0 > AND min_index_interval = 128 > AND read_repair_chance = 0.0 > AND speculative_retry = '99.0PERCENTILE'; > {code} > query :- > > {code:xml} > SELECT DISTINCT pk2, pk1 FROM sample LIMIT 2147483647; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11679) Cassandra Driver returns different number of results depending on fetchsize
[ https://issues.apache.org/jira/browse/CASSANDRA-11679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-11679: --- Status: Patch Available (was: Open) > Cassandra Driver returns different number of results depending on fetchsize > --- > > Key: CASSANDRA-11679 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11679 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Varun Barala >Assignee: Benjamin Lerer > Attachments: 11679-2.1.txt > > > I'm trying to fetch all distinct keys from a CF using cassandra-driver > (2.1.7.1) and I observed some strange behavior :- > The total distinct rows are 498 so If I perform a query get All distinctKeys > It returns 503 instead of 498(five keys twice). > But If I define the fetch size in select statement more than 498 then it > returns exact 498 rows. > And If I execute same statement on Dev-center it returns 498 rows (because > the default fetch size is 5000). In `cqlsh` it returns 503 rows (because > cqlsh uses fetch size=100). > Some Additional and useful information :- > --- > Cassandra-2.1.13 (C)* version > Consistency level: ONE > local machine(ubuntu 14.04) > Table Schema:- > -- > {code:xml} > CREATE TABLE sample ( > pk1 text, > pk2 text, > row_id uuid, > value blob, > PRIMARY KEY (( pk1, pk2)) > ) WITH bloom_filter_fp_chance = 0.01 > AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}' > AND comment = '' > AND compaction = {'class': > 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'} > AND compression = {'sstable_compression': > 'org.apache.cassandra.io.compress.LZ4Compressor'} > AND dclocal_read_repair_chance = 0.1 > AND default_time_to_live = 0 > AND gc_grace_seconds = 864000 > AND max_index_interval = 2048 > AND memtable_flush_period_in_ms = 0 > AND min_index_interval = 128 > AND read_repair_chance = 0.0 > AND speculative_retry = '99.0PERCENTILE'; > {code} > query :- > > {code:xml} > SELECT DISTINCT pk2, pk1 FROM sample LIMIT 2147483647; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11679) Cassandra Driver returns different number of results depending on fetchsize
[ https://issues.apache.org/jira/browse/CASSANDRA-11679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-11679: --- Attachment: 11679-2.1.txt |[unit tests|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-11679-2.1-testall/2/]|[dtests|http://cassci.datastax.com/view/Dev/view/blerer/job/blerer-11679-2.1-dtest/2/]| The {{paging_test.TestPagingData.test_paging_for_range_name_queries}} is failing because the test is invalid for 2.1. I provided a fix for it. [~thobbs] could you review? It is a backport of CASSANDRA-10010. > Cassandra Driver returns different number of results depending on fetchsize > --- > > Key: CASSANDRA-11679 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11679 > Project: Cassandra > Issue Type: Bug > Components: CQL >Reporter: Varun Barala >Assignee: Benjamin Lerer > Attachments: 11679-2.1.txt > > > I'm trying to fetch all distinct keys from a CF using cassandra-driver > (2.1.7.1) and I observed some strange behavior :- > The total distinct rows are 498 so If I perform a query get All distinctKeys > It returns 503 instead of 498(five keys twice). > But If I define the fetch size in select statement more than 498 then it > returns exact 498 rows. > And If I execute same statement on Dev-center it returns 498 rows (because > the default fetch size is 5000). In `cqlsh` it returns 503 rows (because > cqlsh uses fetch size=100). > Some Additional and useful information :- > --- > Cassandra-2.1.13 (C)* version > Consistency level: ONE > local machine(ubuntu 14.04) > Table Schema:- > -- > {code:xml} > CREATE TABLE sample ( > pk1 text, > pk2 text, > row_id uuid, > value blob, > PRIMARY KEY (( pk1, pk2)) > ) WITH bloom_filter_fp_chance = 0.01 > AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}' > AND comment = '' > AND compaction = {'class': > 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'} > AND compression = {'sstable_compression': > 'org.apache.cassandra.io.compress.LZ4Compressor'} > AND dclocal_read_repair_chance = 0.1 > AND default_time_to_live = 0 > AND gc_grace_seconds = 864000 > AND max_index_interval = 2048 > AND memtable_flush_period_in_ms = 0 > AND min_index_interval = 128 > AND read_repair_chance = 0.0 > AND speculative_retry = '99.0PERCENTILE'; > {code} > query :- > > {code:xml} > SELECT DISTINCT pk2, pk1 FROM sample LIMIT 2147483647; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10962) Cassandra should not create snapshot at restart for compactions_in_progress
[ https://issues.apache.org/jira/browse/CASSANDRA-10962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270665#comment-15270665 ] Alex Petrov commented on CASSANDRA-10962: - true, my initial patch was to add {{skipSnapshot}}, although logic there was getting tricky, so I simplified it. Just to give heads up, in 3.0+ this problem does not appear as [~fabrice.facorat] mentioned due to [7066|https://issues.apache.org/jira/browse/CASSANDRA-7066]. I've checked the failing dtests locally, both are passing. With unit tests, it's same. Same with utests... |[2.2|https://github.com/ifesdjeen/cassandra/tree/10962-2.2]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-10962-2.2-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-10962-2.2-dtest/]| > Cassandra should not create snapshot at restart for compactions_in_progress > --- > > Key: CASSANDRA-10962 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10962 > Project: Cassandra > Issue Type: Bug > Environment: Ubuntu 14.04.3 LTS >Reporter: FACORAT >Assignee: Alex Petrov >Priority: Minor > Fix For: 2.2.x > > > If auto_snapshot is set to true in cassandra.yaml, each time you restart > Cassandra, a snapshot is created for system.compactions_in_progress as the > table is truncated at cassandra start. > However as datas in this table are temporary, Cassandra should not create > snapshot for this table (or maybe even for system.* tables). This will be > coherent with the fact that "nodetool listsnapshots" doesn't even list this > table. > Exemple: > {code} > $ nodetool listsnapshots | grep compactions > $ ls -lh > system/compactions_in_progress-55080ab05d9c388690a4acb25fe1f77b/snapshots/ > total 16K > drwxr-xr-x 2 cassandra cassandra 4.0K Nov 30 13:12 > 1448885530280-compactions_in_progress > drwxr-xr-x 2 cassandra cassandra 4.0K Dec 7 15:36 > 1449498977181-compactions_in_progress > drwxr-xr-x 2 cassandra cassandra 4.0K Dec 14 18:20 > 1450113621506-compactions_in_progress > drwxr-xr-x 2 cassandra cassandra 4.0K Jan 4 12:53 > 1451908396364-compactions_in_progress > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)