[jira] [Commented] (CASSANDRA-8404) CQLSSTableLoader can not create SSTable for csv file of 10M rows.
[ https://issues.apache.org/jira/browse/CASSANDRA-8404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14390048#comment-14390048 ] Manish commented on CASSANDRA-8404: --- I will check and let you know. I had faced this issue only on 32 bit Ubuntu OS and not on 64 bit Ubuntu OS. Since our QA and production machines are on 64 bit Ubuntu OS I did not face any issue in dev testing on QA machine. > CQLSSTableLoader can not create SSTable for csv file of 10M rows. > - > > Key: CASSANDRA-8404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8404 > Project: Cassandra > Issue Type: Bug > Environment: I am using Cassandra 2.1.1 on 32 bit Ubuntu 12.04. I am > running the program with -Xmx1000M > manish@manish[~]:> uname -a > Linux manish 3.2.0-72-generic-pae #107-Ubuntu SMP Thu Nov 6 14:44:10 UTC 2014 > i686 i686 i386 GNU/Linux >Reporter: Manish > Fix For: 2.1.4 > > Attachments: Test1.java, cassandra.yaml > > > I am able to create SSTable for one file of 10M rows but not for other file. > The data file which works is subscribers1.gz and data file which does not > work is subscriber2.gz. Both files have same values in first column but > different values for second column. I wonder why CQLSSTableLoader does not > work for different set of data. > Program expected unzipped txt files. So please unzip files before running > program. What I have observed is High GC when program processes around 5.2M > lines of file subscriber2.gz. It is able to process till 5.8M lines with very > frequent Full GC runs. It is not able to process beyond 5.8M rows because of > memory not being available. > I have attached Test1.java and cassandra.yaml I used for creating sstable. In > classpath I am specifying all jars of lib folder of extracted > apache-cassandra-2.1.1-bin.tar.gz > Jira does not allow a file of size greater than 10 MB. So I am sharing data > files in google drive. > link to download subscribers1.gz > https://drive.google.com/file/d/0B6_-ugKWlrfoOTRTa2FCNTFWU2c/view?usp=sharing > link to download subscribers2.gz > https://drive.google.com/file/d/0B6_-ugKWlrfocndycm9yM21rN0E/view?usp=sharing -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7168) Add repair aware consistency levels
[ https://issues.apache.org/jira/browse/CASSANDRA-7168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389957#comment-14389957 ] prmg commented on CASSANDRA-7168: - [~tjake] I'm giving a try on this ticket for learning purposes and so far I calculated maxPartitionRepairTime on the coordinator, sent over via the MessagingService to the replicas on the ReadCommand and skipped sstables with repairedAt <= maxPartitionRepairTime on the CollationController. One part that was not clear to me in your description was: bq. We will also need to include tombstones in the results of the non-repaired column family result since they need to be merged with the repaired result. Is that tombstone inclusion already done by the normal flow of the collation controller or is it necessary to add some post-processing after repaired sstables <= maxPartitionRepairTime are skipped? Would be great if you could clarify that a bit for me. Thanks! > Add repair aware consistency levels > --- > > Key: CASSANDRA-7168 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7168 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: T Jake Luciani > Labels: performance > Fix For: 3.1 > > > With CASSANDRA-5351 and CASSANDRA-2424 I think there is an opportunity to > avoid a lot of extra disk I/O when running queries with higher consistency > levels. > Since repaired data is by definition consistent and we know which sstables > are repaired, we can optimize the read path by having a REPAIRED_QUORUM which > breaks reads into two phases: > > 1) Read from one replica the result from the repaired sstables. > 2) Read from a quorum only the un-repaired data. > For the node performing 1) we can pipeline the call so it's a single hop. > In the long run (assuming data is repaired regularly) we will end up with > much closer to CL.ONE performance while maintaining consistency. > Some things to figure out: > - If repairs fail on some nodes we can have a situation where we don't have > a consistent repaired state across the replicas. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: remove dead code
Repository: cassandra
Updated Branches:
refs/heads/trunk e4b158f6a -> 7f0a8542d
remove dead code
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7f0a8542
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7f0a8542
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7f0a8542
Branch: refs/heads/trunk
Commit: 7f0a8542d684d3bd9b597b5eab332355315c4942
Parents: e4b158f
Author: Dave Brosius
Authored: Tue Mar 31 23:53:31 2015 -0400
Committer: Dave Brosius
Committed: Tue Mar 31 23:53:31 2015 -0400
--
src/java/org/apache/cassandra/thrift/CassandraServer.java | 7 ---
1 file changed, 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f0a8542/src/java/org/apache/cassandra/thrift/CassandraServer.java
--
diff --git a/src/java/org/apache/cassandra/thrift/CassandraServer.java
b/src/java/org/apache/cassandra/thrift/CassandraServer.java
index e92e0c6..c6d2600 100644
--- a/src/java/org/apache/cassandra/thrift/CassandraServer.java
+++ b/src/java/org/apache/cassandra/thrift/CassandraServer.java
@@ -581,13 +581,6 @@ public class CassandraServer implements Cassandra.Iface
}
}
-private static ByteBuffer getName(ColumnOrSuperColumn cosc)
-{
-return cosc.isSetSuper_column() ? cosc.super_column.name :
- (cosc.isSetColumn() ? cosc.column.name :
- (cosc.isSetCounter_column() ? cosc.counter_column.name
: cosc.counter_super_column.name));
-}
-
public Map multiget_count(List keys,
ColumnParent column_parent, SlicePredicate predicate, ConsistencyLevel
consistency_level)
throws InvalidRequestException, UnavailableException, TimedOutException
{
cassandra git commit: simplify
Repository: cassandra
Updated Branches:
refs/heads/trunk 0352a15a3 -> e4b158f6a
simplify
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4b158f6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4b158f6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4b158f6
Branch: refs/heads/trunk
Commit: e4b158f6ad2d9f59a06d722bc97a3d423486e298
Parents: 0352a15
Author: Dave Brosius
Authored: Tue Mar 31 23:32:05 2015 -0400
Committer: Dave Brosius
Committed: Tue Mar 31 23:32:05 2015 -0400
--
src/java/org/apache/cassandra/utils/concurrent/Ref.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4b158f6/src/java/org/apache/cassandra/utils/concurrent/Ref.java
--
diff --git a/src/java/org/apache/cassandra/utils/concurrent/Ref.java
b/src/java/org/apache/cassandra/utils/concurrent/Ref.java
index 4e6cef7..25f8510 100644
--- a/src/java/org/apache/cassandra/utils/concurrent/Ref.java
+++ b/src/java/org/apache/cassandra/utils/concurrent/Ref.java
@@ -214,7 +214,7 @@ public final class Ref implements RefCounted,
AutoCloseable
String print(String thread, StackTraceElement[] trace)
{
StringBuilder sb = new StringBuilder();
-sb.append(thread.toString());
+sb.append(thread);
sb.append("\n");
for (StackTraceElement element : trace)
{
[jira] [Updated] (CASSANDRA-9083) cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
[
https://issues.apache.org/jira/browse/CASSANDRA-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-9083:
---
Assignee: Tyler Hobbs
> cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
> ---
>
> Key: CASSANDRA-9083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9083
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: 2.1.3
>Reporter: Joseph Chu
>Assignee: Tyler Hobbs
>Priority: Minor
> Labels: cqlsh
> Fix For: 2.1.4
>
>
> Executing a COPY command from an external file using the cqlsh -f or the
> SOURCE command results in the error:
> filename.cql:7:descriptor 'lower' requires a 'str' object but received a
> 'unicode'
> Looks like there was a change in the cqlsh code from 2.1.2 to 2.1.3 which
> makes use of codecs.open() instead of open(), which returns a unicode object.
> The offending line of code that returns the error seems to be in cqlsh, line
> 1415:
> copyoptnames = map(str.lower, parsed.get_binding('optnames', ()))
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9083) cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
[
https://issues.apache.org/jira/browse/CASSANDRA-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389887#comment-14389887
]
Philip Thompson commented on CASSANDRA-9083:
What version of python are you using?
> cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
> ---
>
> Key: CASSANDRA-9083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9083
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: 2.1.3
>Reporter: Joseph Chu
>Priority: Minor
> Labels: cqlsh
> Fix For: 2.1.4
>
>
> Executing a COPY command from an external file using the cqlsh -f or the
> SOURCE command results in the error:
> filename.cql:7:descriptor 'lower' requires a 'str' object but received a
> 'unicode'
> Looks like there was a change in the cqlsh code from 2.1.2 to 2.1.3 which
> makes use of codecs.open() instead of open(), which returns a unicode object.
> The offending line of code that returns the error seems to be in cqlsh, line
> 1415:
> copyoptnames = map(str.lower, parsed.get_binding('optnames', ()))
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9083) cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
[
https://issues.apache.org/jira/browse/CASSANDRA-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-9083:
---
Reproduced In: 2.1.3
Fix Version/s: 2.1.4
Labels: cqlsh (was: )
> cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
> ---
>
> Key: CASSANDRA-9083
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9083
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: 2.1.3
>Reporter: Joseph Chu
>Priority: Minor
> Labels: cqlsh
> Fix For: 2.1.4
>
>
> Executing a COPY command from an external file using the cqlsh -f or the
> SOURCE command results in the error:
> filename.cql:7:descriptor 'lower' requires a 'str' object but received a
> 'unicode'
> Looks like there was a change in the cqlsh code from 2.1.2 to 2.1.3 which
> makes use of codecs.open() instead of open(), which returns a unicode object.
> The offending line of code that returns the error seems to be in cqlsh, line
> 1415:
> copyoptnames = map(str.lower, parsed.get_binding('optnames', ()))
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7486) Compare CMS and G1 pause times
[ https://issues.apache.org/jira/browse/CASSANDRA-7486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389850#comment-14389850 ] Albert P Tobey commented on CASSANDRA-7486: --- I managed to get G1 (Java 8) to beat CMS on both latency and throughput on my NUC cluster. Preliminary results: https://gist.github.com/tobert/ea9328e4873441c7fc34 > Compare CMS and G1 pause times > -- > > Key: CASSANDRA-7486 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7486 > Project: Cassandra > Issue Type: Test > Components: Config >Reporter: Jonathan Ellis >Assignee: Shawn Kumar > Fix For: 2.1.4 > > > See > http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-7486gc-migration-to-expectations-and-advanced-tuning > and https://twitter.com/rbranson/status/482113561431265281 > May want to default 2.1 to G1. > 2.1 is a different animal from 2.0 after moving most of memtables off heap. > Suspect this will help G1 even more than CMS. (NB this is off by default but > needs to be part of the test.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7814) enable describe on indices
[ https://issues.apache.org/jira/browse/CASSANDRA-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389802#comment-14389802 ] Stefania commented on CASSANDRA-7814: - Thank you for your review [~blerer] and [~thobbs] Here are the python driver pull request and ticket: https://github.com/datastax/python-driver/pull/263 https://datastax-oss.atlassian.net/browse/PYTHON-241 Here is the --tag-build cmd line option pull request: https://github.com/datastax/python-driver/pull/262 I will keep track of PYTHON-241 to see when we can rebuild the driver and commit. > enable describe on indices > -- > > Key: CASSANDRA-7814 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7814 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: radha >Assignee: Stefania >Priority: Minor > Fix For: 2.1.4 > > > Describe index should be supported, right now, the only way is to export the > schema and find what it really is before updating/dropping the index. > verified in > [cqlsh 3.1.8 | Cassandra 1.2.18.1 | CQL spec 3.0.0 | Thrift protocol 19.36.2] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-9083) cqlsh COPY functionality doesn't work together with SOURCE or with cqlsh -f
Joseph Chu created CASSANDRA-9083:
-
Summary: cqlsh COPY functionality doesn't work together with
SOURCE or with cqlsh -f
Key: CASSANDRA-9083
URL: https://issues.apache.org/jira/browse/CASSANDRA-9083
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: 2.1.3
Reporter: Joseph Chu
Priority: Minor
Executing a COPY command from an external file using the cqlsh -f or the SOURCE
command results in the error:
filename.cql:7:descriptor 'lower' requires a 'str' object but received a
'unicode'
Looks like there was a change in the cqlsh code from 2.1.2 to 2.1.3 which makes
use of codecs.open() instead of open(), which returns a unicode object.
The offending line of code that returns the error seems to be in cqlsh, line
1415:
copyoptnames = map(str.lower, parsed.get_binding('optnames', ()))
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7976) Changes to index_interval table properties revert after subsequent modifications
[
https://issues.apache.org/jira/browse/CASSANDRA-7976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389622#comment-14389622
]
Stefania commented on CASSANDRA-7976:
-
Thanks!
> Changes to index_interval table properties revert after subsequent
> modifications
>
>
> Key: CASSANDRA-7976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7976
> Project: Cassandra
> Issue Type: Bug
> Components: Config
> Environment: cqlsh 4.1.1, Cassandra 2.0.9-SNAPSHOT (built w/ `ccm` on
> Mac OS X 10.9.4 with Java 1.7.0_67 - more detail below)
> $ java -version
> java version "1.7.0_67"
> Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
> $ mvn --version
> Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4;
> 2014-08-11T13:58:10-07:00)
> Maven home: /usr/local/Cellar/maven/3.2.3/libexec
> Java version: 1.7.0_67, vendor: Oracle Corporation
> Java home: /Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.9.4", arch: "x86_64", family: "mac"
>Reporter: Andrew Lenards
>Assignee: Stefania
> Labels: cql3, metadata
> Fix For: 2.0.14
>
> Attachments: 7976-2.0.txt
>
>
> It appears that if you want to increase the sampling in *-Summary.db files,
> you would change the default for {{index_interval}} table property from the
> {{128}} default value to {{256}} on a given CQL {{TABLE}}.
> However, if you {{ALTER TABLE}} after setting the value, {{index_interval}}
> returns to the default, {{128}}. This is unexpected behavior. I would expect
> the value for {{index_interval}} to not be affected by subsequent {{ALTER
> TABLE}} statements.
> As noted in Environment, this was seen with a 2.0.9-SNAPSHOT built w/ `ccm`.
> If I just use a table from one of DataStax documentation tutorials (musicdb
> as mdb):
> {noformat}
> cqlsh:mdb> DESC TABLE songs;
> CREATE TABLE songs (
> id uuid,
> album text,
> artist text,
> data blob,
> reviews list,
> tags set,
> title text,
> venue map,
> PRIMARY KEY ((id))
> ) WITH
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.10 AND
> gc_grace_seconds=864000 AND
> index_interval=128 AND
> read_repair_chance=0.00 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='99.0PERCENTILE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> {noformat}
> We've got {{128}} as expected.
> We alter it:
> {noformat}
> cqlsh:mdb> ALTER TABLE songs WITH index_interval = 256;
> {noformat}
> And the change appears:
> {noformat}
> cqlsh:mdb> DESC TABLE songs;
> CREATE TABLE songs (
> id uuid,
> album text,
> artist text,
> data blob,
> reviews list,
> tags set,
> title text,
> venue map,
> PRIMARY KEY ((id))
> ) WITH
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.10 AND
> gc_grace_seconds=864000 AND
> index_interval=256 AND
> read_repair_chance=0.00 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='99.0PERCENTILE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> {noformat}
> But if do another {{ALTER TABLE}}, say, change the caching or comment, the
> {{index_interval}} will revert back to {{128}}.
> {noformat}
> cqlsh:mdb> ALTER TABLE songs WITH caching = 'none';
> cqlsh:mdb> DESC TABLE songs;
> CREATE TABLE songs (
> id uuid,
> album text,
> artist text,
> data blob,
> reviews list,
> tags set,
> title text,
> venue map,
> PRIMARY KEY ((id))
> ) WITH
> bloom_filter_fp_chance=0.01 AND
> caching='NONE' AND
> comment='' AND
> dclocal_read_repair_chance=0.10 AND
> gc_grace_seconds=864000 AND
> index_interval=128 AND
> read_repair_chance=0.00 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='99.0PERCENTILE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> {noformat}
> It should be {{index_interval=256}}.
> I know that 2.1 will replace {{index_interval}}.
> I have not confirmed any behavior with {{min_index_interval}} no
[jira] [Commented] (CASSANDRA-9082) sstableloader error on trunk due to loading read meter
[
https://issues.apache.org/jira/browse/CASSANDRA-9082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389573#comment-14389573
]
Tyler Hobbs commented on CASSANDRA-9082:
Linking to CASSANDRA-8996, as this is responsible for the failure of the
{{snapshot_test.TestSnapshot.test_basic_snapshot_and_restore}} dtest.
> sstableloader error on trunk due to loading read meter
> --
>
> Key: CASSANDRA-9082
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9082
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Tyler Hobbs
>Assignee: Benedict
> Fix For: 3.0
>
>
> If you try to run sstableloader on trunk, you'll get an error like the
> following:
> {noformat}
> Exception: sstableloader command
> '/tmp/dtest-p5eSr3/test/node1/bin/sstableloader -d 127.0.0.1
> /tmp/tmpzd5CCh/ks/cf' failed; exit status: 1'; stdout: Established connection
> to initial hosts
> Opening sstables and calculating sections to stream
> ; stderr: null
> java.lang.AssertionError
> org.apache.cassandra.exceptions.ConfigurationException
> at
> org.apache.cassandra.locator.AbstractReplicationStrategy.createInternal(AbstractReplicationStrategy.java:249)
> at
> org.apache.cassandra.locator.AbstractReplicationStrategy.createReplicationStrategy(AbstractReplicationStrategy.java:264)
> at
> org.apache.cassandra.db.Keyspace.createReplicationStrategy(Keyspace.java:279)
> at org.apache.cassandra.db.Keyspace.(Keyspace.java:267)
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:115)
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:92)
> at
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.(StatementRestrictions.java:128)
> at
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepareRestrictions(SelectStatement.java:788)
> at
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:741)
> at
> org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:488)
> at
> org.apache.cassandra.cql3.QueryProcessor.parseStatement(QueryProcessor.java:266)
> at
> org.apache.cassandra.cql3.QueryProcessor.prepareInternal(QueryProcessor.java:300)
> at
> org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:308)
> at
> org.apache.cassandra.db.SystemKeyspace.getSSTableReadMeter(SystemKeyspace.java:899)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$GlobalTidy.(SSTableReader.java:1973)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$GlobalTidy.get(SSTableReader.java:2012)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$DescriptorTypeTidy.(SSTableReader.java:1890)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$DescriptorTypeTidy.get(SSTableReader.java:1926)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.setup(SSTableReader.java:1809)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.setup(SSTableReader.java:1754)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:398)
> at
> org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:117)
> at java.io.File.list(File.java:1155)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:78)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:162)
> at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:106)
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.locator.AbstractReplicationStrategy.(AbstractReplicationStrategy.java:66)
> at
> org.apache.cassandra.locator.LocalStrategy.(LocalStrategy.java:36)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.cassandra.locator.AbstractReplicationStrategy.createInternal(AbstractReplicationStrategy.java:244)
> ... 25 more
> {noformat}
> At first glance, it looks like the SSTableReader is trying to load the read
> meter even though it shouldn't (or doesn't need to). Assigning to Benedict
> since this seems to be most related to SSTableReader management.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9082) sstableloader error on trunk due to loading read meter
Tyler Hobbs created CASSANDRA-9082:
--
Summary: sstableloader error on trunk due to loading read meter
Key: CASSANDRA-9082
URL: https://issues.apache.org/jira/browse/CASSANDRA-9082
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Tyler Hobbs
Assignee: Benedict
Fix For: 3.0
If you try to run sstableloader on trunk, you'll get an error like the
following:
{noformat}
Exception: sstableloader command
'/tmp/dtest-p5eSr3/test/node1/bin/sstableloader -d 127.0.0.1
/tmp/tmpzd5CCh/ks/cf' failed; exit status: 1'; stdout: Established connection
to initial hosts
Opening sstables and calculating sections to stream
; stderr: null
java.lang.AssertionError
org.apache.cassandra.exceptions.ConfigurationException
at
org.apache.cassandra.locator.AbstractReplicationStrategy.createInternal(AbstractReplicationStrategy.java:249)
at
org.apache.cassandra.locator.AbstractReplicationStrategy.createReplicationStrategy(AbstractReplicationStrategy.java:264)
at
org.apache.cassandra.db.Keyspace.createReplicationStrategy(Keyspace.java:279)
at org.apache.cassandra.db.Keyspace.(Keyspace.java:267)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:115)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:92)
at
org.apache.cassandra.cql3.restrictions.StatementRestrictions.(StatementRestrictions.java:128)
at
org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepareRestrictions(SelectStatement.java:788)
at
org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:741)
at
org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:488)
at
org.apache.cassandra.cql3.QueryProcessor.parseStatement(QueryProcessor.java:266)
at
org.apache.cassandra.cql3.QueryProcessor.prepareInternal(QueryProcessor.java:300)
at
org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:308)
at
org.apache.cassandra.db.SystemKeyspace.getSSTableReadMeter(SystemKeyspace.java:899)
at
org.apache.cassandra.io.sstable.format.SSTableReader$GlobalTidy.(SSTableReader.java:1973)
at
org.apache.cassandra.io.sstable.format.SSTableReader$GlobalTidy.get(SSTableReader.java:2012)
at
org.apache.cassandra.io.sstable.format.SSTableReader$DescriptorTypeTidy.(SSTableReader.java:1890)
at
org.apache.cassandra.io.sstable.format.SSTableReader$DescriptorTypeTidy.get(SSTableReader.java:1926)
at
org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier.setup(SSTableReader.java:1809)
at
org.apache.cassandra.io.sstable.format.SSTableReader.setup(SSTableReader.java:1754)
at
org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:398)
at
org.apache.cassandra.io.sstable.SSTableLoader$1.accept(SSTableLoader.java:117)
at java.io.File.list(File.java:1155)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:78)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:162)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:106)
Caused by: java.lang.AssertionError
at
org.apache.cassandra.locator.AbstractReplicationStrategy.(AbstractReplicationStrategy.java:66)
at
org.apache.cassandra.locator.LocalStrategy.(LocalStrategy.java:36)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.cassandra.locator.AbstractReplicationStrategy.createInternal(AbstractReplicationStrategy.java:244)
... 25 more
{noformat}
At first glance, it looks like the SSTableReader is trying to load the read
meter even though it shouldn't (or doesn't need to). Assigning to Benedict
since this seems to be most related to SSTableReader management.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8359) Make DTCS consider removing SSTables much more frequently
[ https://issues.apache.org/jira/browse/CASSANDRA-8359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389501#comment-14389501 ] Jonathan Shook commented on CASSANDRA-8359: --- Linking to a 9056, possibly a duplicate. > Make DTCS consider removing SSTables much more frequently > - > > Key: CASSANDRA-8359 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8359 > Project: Cassandra > Issue Type: Improvement >Reporter: Björn Hegerfors >Assignee: Björn Hegerfors >Priority: Minor > Attachments: cassandra-2.0-CASSANDRA-8359.txt > > > When I run DTCS on a table where every value has a TTL (always the same TTL), > SSTables are completely expired, but still stay on disk for much longer than > they need to. I've applied CASSANDRA-8243, but it doesn't make an apparent > difference (probably because the subject SSTables are purged via compaction > anyway, if not by directly dropping them). > Disk size graphs show clearly that tombstones are only removed when the > oldest SSTable participates in compaction. In the long run, size on disk > continually grows bigger. This should not have to happen. It should easily be > able to stay constant, thanks to DTCS separating the expired data from the > rest. > I think checks for whether SSTables can be dropped should happen much more > frequently. This is something that probably only needs to be tweaked for > DTCS, but perhaps there's a more general place to put this. Anyway, my > thinking is that DTCS should, on every call to getNextBackgroundTask, check > which SSTables can be dropped. It would be something like a call to > CompactionController.getFullyExpiredSSTables with all non-compactingSSTables > sent in as "compacting" and all other SSTables sent in as "overlapping". The > returned SSTables, if any, are then added to whichever set of SSTables that > DTCS decides to compact. Then before the compaction happens, Cassandra is > going to make another call to CompactionController.getFullyExpiredSSTables, > where it will see that it can just drop them. > This approach has a bit of redundancy in that it needs to call > CompactionController.getFullyExpiredSSTables twice. To avoid that, the code > path for deciding SSTables to drop would have to be changed. > (Side tracking a little here: I'm also thinking that tombstone compactions > could be considered more often in DTCS. Maybe even some kind of multi-SSTable > tombstone compaction involving the oldest couple of SSTables...) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9079) "respectTombstoneThresholds" should be configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389499#comment-14389499 ] Vinay Chella commented on CASSANDRA-9079: - This discussion leads to another interesting point, maybe setting these properties at column family level would give much more granular control, since cluster could have different column families with different behavior for TTL and tombstone data in terms of the payload and gc_grace_seconds. Disabling “tombstoneThresholds” at cluster level might lead to CASSANDRA-6117 if there is any column family in the cluster, which is having moderate/high payload and large number of tombstones. > "respectTombstoneThresholds" should be configurable > > > Key: CASSANDRA-9079 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9079 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Vinay Chella > Labels: tombstone > Attachments: 9079.txt > > > Currently there is no way to control/ configure the functionality of "drop > queries exceeding a configurable number of tombstones". Making this > functionality configurable through yaml helps use case like: > We have several production clusters with large number of tombstones due to > aggressive TTLs with very small payloads, and low gc_grace_seconds. Most of > the read queries on these column families are ending up in drop operations or > excessive warn messages due to "tombstone_failure_threshold" and > "tombstone_warn_threshold". One cannot deterministically disable these > tombstone thresholds by setting high values. Hence, making > "SliceQueryFilter.respectTombstoneThresholds()" configurable through yaml > would be the better approach. > Column families with small payloads (couple hundred bytes) having large > number of tombstones would not lead to "Avoid death-by-tombstone by default > (CASSANDRA-6117)", which is the reason for observed drop operations. We see > drop operations before compactions catch up, and clean those tombstones with > both STCS and LCS. Configurable "respectTombstoneThresholds()" will help in > reducing tombstone warnings noise. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (CASSANDRA-8979) MerkleTree mismatch for deleted and non-existing rows
[ https://issues.apache.org/jira/browse/CASSANDRA-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita reopened CASSANDRA-8979: --- dtest is failing on trunk. 2.0 and 2.1 are fine. I will investigate. > MerkleTree mismatch for deleted and non-existing rows > - > > Key: CASSANDRA-8979 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8979 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Stefan Podkowinski >Assignee: Stefan Podkowinski > Fix For: 2.1.4, 2.0.14 > > Attachments: 8979-AvoidBufferAllocation-2.0_patch.txt, > cassandra-2.0-8979-lazyrow_patch.txt, cassandra-2.0-8979-validator_patch.txt, > cassandra-2.0-8979-validatortest_patch.txt, > cassandra-2.1-8979-lazyrow_patch.txt, cassandra-2.1-8979-validator_patch.txt > > > Validation compaction will currently create different hashes for rows that > have been deleted compared to nodes that have not seen the rows at all or > have already compacted them away. > In case this sounds familiar to you, see CASSANDRA-4905 which was supposed to > prevent hashing of expired tombstones. This still seems to be in place, but > does not address the issue completely. Or there was a change in 2.0 that > rendered the patch ineffective. > The problem is that rowHash() in the Validator will return a new hash in any > case, whether the PrecompactedRow did actually update the digest or not. This > will lead to the case that a purged, PrecompactedRow will not change the > digest, but we end up with a different tree compared to not having rowHash > called at all (such as in case the row already doesn't exist). > As an implication, repair jobs will constantly detect mismatches between > older sstables containing purgable rows and nodes that have already compacted > these rows. After transfering the reported ranges, the newly created sstables > will immediately get deleted again during the following compaction. This will > happen for each repair run over again until the sstable with the purgable row > finally gets compacted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8979) MerkleTree mismatch for deleted and non-existing rows
[ https://issues.apache.org/jira/browse/CASSANDRA-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389476#comment-14389476 ] Tyler Hobbs commented on CASSANDRA-8979: [~yukim] the new dtest is failing on 2.0, 2.1, and trunk. Do you want to reopen this ticket to investigate? > MerkleTree mismatch for deleted and non-existing rows > - > > Key: CASSANDRA-8979 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8979 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Stefan Podkowinski >Assignee: Stefan Podkowinski > Fix For: 2.1.4, 2.0.14 > > Attachments: 8979-AvoidBufferAllocation-2.0_patch.txt, > cassandra-2.0-8979-lazyrow_patch.txt, cassandra-2.0-8979-validator_patch.txt, > cassandra-2.0-8979-validatortest_patch.txt, > cassandra-2.1-8979-lazyrow_patch.txt, cassandra-2.1-8979-validator_patch.txt > > > Validation compaction will currently create different hashes for rows that > have been deleted compared to nodes that have not seen the rows at all or > have already compacted them away. > In case this sounds familiar to you, see CASSANDRA-4905 which was supposed to > prevent hashing of expired tombstones. This still seems to be in place, but > does not address the issue completely. Or there was a change in 2.0 that > rendered the patch ineffective. > The problem is that rowHash() in the Validator will return a new hash in any > case, whether the PrecompactedRow did actually update the digest or not. This > will lead to the case that a purged, PrecompactedRow will not change the > digest, but we end up with a different tree compared to not having rowHash > called at all (such as in case the row already doesn't exist). > As an implication, repair jobs will constantly detect mismatches between > older sstables containing purgable rows and nodes that have already compacted > these rows. After transfering the reported ranges, the newly created sstables > will immediately get deleted again during the following compaction. This will > happen for each repair run over again until the sstable with the purgable row > finally gets compacted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4003) cqlsh still failing to handle decode errors in some column names
[
https://issues.apache.org/jira/browse/CASSANDRA-4003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389474#comment-14389474
]
Tyler Hobbs commented on CASSANDRA-4003:
[~gadelkareem] if you're seeing this problem with Cassandra 2.0 or 2.1, can you
open a new ticket with more details and a new stacktrace?
> cqlsh still failing to handle decode errors in some column names
>
>
> Key: CASSANDRA-4003
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4003
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Affects Versions: 1.0.8
>Reporter: paul cannon
>Assignee: paul cannon
>Priority: Minor
> Labels: cqlsh
> Fix For: 1.0.9, 1.1.0
>
> Attachments: 4003-2.txt
>
>
> Columns which are expected to be text, but which are not valid utf8, cause
> cqlsh to display an error and not show any output:
> {noformat}
> cqlsh:ks> CREATE COLUMNFAMILY test (a text PRIMARY KEY) WITH comparator =
> timestamp;
> cqlsh:ks> INSERT INTO test (a, '2012-03-05') VALUES ('val1', 'val2');
> cqlsh:ks> ASSUME test NAMES ARE text;
> cqlsh:ks> select * from test;
> 'utf8' codec can't decode byte 0xe1 in position 4: invalid continuation byte
> {noformat}
> the traceback with cqlsh --debug:
> {noformat}
> Traceback (most recent call last):
> File "bin/cqlsh", line 581, in onecmd
> self.handle_statement(st)
> File "bin/cqlsh", line 606, in handle_statement
> return custom_handler(parsed)
> File "bin/cqlsh", line 663, in do_select
> self.perform_statement_as_tokens(parsed.matched, decoder=decoder)
> File "bin/cqlsh", line 666, in perform_statement_as_tokens
> return self.perform_statement(cqlhandling.cql_detokenize(tokens),
> decoder=decoder)
> File "bin/cqlsh", line 693, in perform_statement
> self.print_result(self.cursor)
> File "bin/cqlsh", line 728, in print_result
> self.print_static_result(cursor)
> File "bin/cqlsh", line 742, in print_static_result
> formatted_names = map(self.myformat_colname, colnames)
> File "bin/cqlsh", line 413, in myformat_colname
> wcwidth.wcswidth(name.decode(self.output_codec.name)))
> File "/usr/local/Cellar/python/2.7.2/lib/python2.7/encodings/utf_8.py",
> line 16, in decode
> return codecs.utf_8_decode(input, errors, True)
> UnicodeDecodeError: 'utf8' codec can't decode byte 0xe1 in position 4:
> invalid continuation byte
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9081) Update bundled python driver
[ https://issues.apache.org/jira/browse/CASSANDRA-9081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-9081: --- Reviewer: Tyler Hobbs > Update bundled python driver > > > Key: CASSANDRA-9081 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9081 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Blake Eggleston >Assignee: Blake Eggleston > Fix For: 2.1.4 > > Attachments: 0001-C9081-upgrading-python-driver-to-2.5.0.patch > > > https://datastax-oss.atlassian.net/browse/PYTHON-231, included in 2.5.0, > prevents triggers from being displayed in cqlsh -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8359) Make DTCS consider removing SSTables much more frequently
[ https://issues.apache.org/jira/browse/CASSANDRA-8359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389451#comment-14389451 ] Jim Plush commented on CASSANDRA-8359: -- would love to see this, at high scale PB+ clusters the additional disk space used by completely expired SSTables is considerable. E.G. I have a table with a 1 day TTL and I have SSTables on disk with DTCS from 5+ days ago and the max timestamp from the sstable metadata is 5+ days ago. that should just be an rm -f type operation. I assumed this was how DTCS was supposed to work. > Make DTCS consider removing SSTables much more frequently > - > > Key: CASSANDRA-8359 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8359 > Project: Cassandra > Issue Type: Improvement >Reporter: Björn Hegerfors >Assignee: Björn Hegerfors >Priority: Minor > Attachments: cassandra-2.0-CASSANDRA-8359.txt > > > When I run DTCS on a table where every value has a TTL (always the same TTL), > SSTables are completely expired, but still stay on disk for much longer than > they need to. I've applied CASSANDRA-8243, but it doesn't make an apparent > difference (probably because the subject SSTables are purged via compaction > anyway, if not by directly dropping them). > Disk size graphs show clearly that tombstones are only removed when the > oldest SSTable participates in compaction. In the long run, size on disk > continually grows bigger. This should not have to happen. It should easily be > able to stay constant, thanks to DTCS separating the expired data from the > rest. > I think checks for whether SSTables can be dropped should happen much more > frequently. This is something that probably only needs to be tweaked for > DTCS, but perhaps there's a more general place to put this. Anyway, my > thinking is that DTCS should, on every call to getNextBackgroundTask, check > which SSTables can be dropped. It would be something like a call to > CompactionController.getFullyExpiredSSTables with all non-compactingSSTables > sent in as "compacting" and all other SSTables sent in as "overlapping". The > returned SSTables, if any, are then added to whichever set of SSTables that > DTCS decides to compact. Then before the compaction happens, Cassandra is > going to make another call to CompactionController.getFullyExpiredSSTables, > where it will see that it can just drop them. > This approach has a bit of redundancy in that it needs to call > CompactionController.getFullyExpiredSSTables twice. To avoid that, the code > path for deciding SSTables to drop would have to be changed. > (Side tracking a little here: I'm also thinking that tombstone compactions > could be considered more often in DTCS. Maybe even some kind of multi-SSTable > tombstone compaction involving the oldest couple of SSTables...) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS
[ https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389447#comment-14389447 ] Jim Plush commented on CASSANDRA-8460: -- We also have this use case... for the PB+ size clusters where 90% of the data is cold storage and rarely used it would be nice to have some cheap spinning disks that could hold the data. Read latencies would be less of a concern do to the in-frequency of reads. > Make it possible to move non-compacting sstables to slow/big storage in DTCS > > > Key: CASSANDRA-8460 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8460 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson > > It would be nice if we could configure DTCS to have a set of extra data > directories where we move the sstables once they are older than > max_sstable_age_days. > This would enable users to have a quick, small SSD for hot, new data, and big > spinning disks for data that is rarely read and never compacted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9079) "respectTombstoneThresholds" should be configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389446#comment-14389446 ] Tyler Hobbs commented on CASSANDRA-9079: bq. But isn't making "respectTombstoneThresholds" configurable would be the cleaner approach? That might be simpler, but it adds another option to the (already large) configuration. The fewer config options we have, the easier it is for users to understand them and tune them appropriately. Since this new config option wouldn't do anything that existing options can't do, I don't think it's worth adding. > "respectTombstoneThresholds" should be configurable > > > Key: CASSANDRA-9079 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9079 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Vinay Chella > Labels: tombstone > Attachments: 9079.txt > > > Currently there is no way to control/ configure the functionality of "drop > queries exceeding a configurable number of tombstones". Making this > functionality configurable through yaml helps use case like: > We have several production clusters with large number of tombstones due to > aggressive TTLs with very small payloads, and low gc_grace_seconds. Most of > the read queries on these column families are ending up in drop operations or > excessive warn messages due to "tombstone_failure_threshold" and > "tombstone_warn_threshold". One cannot deterministically disable these > tombstone thresholds by setting high values. Hence, making > "SliceQueryFilter.respectTombstoneThresholds()" configurable through yaml > would be the better approach. > Column families with small payloads (couple hundred bytes) having large > number of tombstones would not lead to "Avoid death-by-tombstone by default > (CASSANDRA-6117)", which is the reason for observed drop operations. We see > drop operations before compactions catch up, and clean those tombstones with > both STCS and LCS. Configurable "respectTombstoneThresholds()" will help in > reducing tombstone warnings noise. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Restore performance of writeUTF; follow up commit to CASSANDRA-8670
Repository: cassandra
Updated Branches:
refs/heads/trunk 230cca580 -> 0352a15a3
Restore performance of writeUTF; follow up commit to CASSANDRA-8670
patch by ariel and benedict for CASSANDRA-8670
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0352a15a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0352a15a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0352a15a
Branch: refs/heads/trunk
Commit: 0352a15a318e8121f8ec977d28379961a9aec387
Parents: 230cca5
Author: Ariel Weisberg
Authored: Tue Mar 31 22:21:43 2015 +0100
Committer: Benedict Elliott Smith
Committed: Tue Mar 31 22:21:43 2015 +0100
--
.../cassandra/io/util/DataOutputStreamPlus.java | 2 +-
.../io/util/UnbufferedDataOutputStreamPlus.java | 99 +++
.../test/microbench/OutputStreamBench.java | 274 +++
.../io/util/BufferedDataOutputStreamTest.java | 26 +-
4 files changed, 342 insertions(+), 59 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0352a15a/src/java/org/apache/cassandra/io/util/DataOutputStreamPlus.java
--
diff --git a/src/java/org/apache/cassandra/io/util/DataOutputStreamPlus.java
b/src/java/org/apache/cassandra/io/util/DataOutputStreamPlus.java
index 6de2879..a846384 100644
--- a/src/java/org/apache/cassandra/io/util/DataOutputStreamPlus.java
+++ b/src/java/org/apache/cassandra/io/util/DataOutputStreamPlus.java
@@ -55,7 +55,7 @@ public abstract class DataOutputStreamPlus extends
OutputStream implements DataO
protected static byte[] retrieveTemporaryBuffer(int minSize)
{
byte[] bytes = tempBuffer.get();
-if (bytes.length < minSize)
+if (bytes.length < Math.min(minSize, MAX_BUFFER_SIZE))
{
// increase in powers of 2, to avoid wasted repeat allocations
bytes = new byte[Math.min(MAX_BUFFER_SIZE, 2 *
Integer.highestOneBit(minSize))];
http://git-wip-us.apache.org/repos/asf/cassandra/blob/0352a15a/src/java/org/apache/cassandra/io/util/UnbufferedDataOutputStreamPlus.java
--
diff --git
a/src/java/org/apache/cassandra/io/util/UnbufferedDataOutputStreamPlus.java
b/src/java/org/apache/cassandra/io/util/UnbufferedDataOutputStreamPlus.java
index 31abfa8..ac3bae5 100644
--- a/src/java/org/apache/cassandra/io/util/UnbufferedDataOutputStreamPlus.java
+++ b/src/java/org/apache/cassandra/io/util/UnbufferedDataOutputStreamPlus.java
@@ -252,81 +252,72 @@ public abstract class UnbufferedDataOutputStreamPlus
extends DataOutputStreamPlu
public static void writeUTF(String str, DataOutput out) throws IOException
{
int length = str.length();
-int utfCount = calculateUTFLength(str, length);
+int utfCount = 0;
+for (int i = 0 ; i < length ; i++)
+{
+int ch = str.charAt(i);
+if ((ch > 0) & (ch <= 127))
+utfCount += 1;
+else if (ch <= 2047)
+utfCount += 2;
+else
+utfCount += 3;
+}
if (utfCount > 65535)
throw new UTFDataFormatException(); //$NON-NLS-1$
byte[] utfBytes = retrieveTemporaryBuffer(utfCount + 2);
-int utfIndex = 2;
-utfBytes[0] = (byte) (utfCount >> 8);
-utfBytes[1] = (byte) utfCount;
int bufferLength = utfBytes.length;
-
-if (utfCount == length && utfCount + utfIndex < bufferLength)
+if (utfCount == length)
{
-for (int charIndex = 0 ; charIndex < length ; charIndex++)
-utfBytes[utfIndex++] = (byte) str.charAt(charIndex);
+utfBytes[0] = (byte) (utfCount >> 8);
+utfBytes[1] = (byte) utfCount;
+int firstIndex = 2;
+for (int offset = 0 ; offset < length ; offset += bufferLength)
+{
+int runLength = Math.min(bufferLength - firstIndex, length -
offset) + firstIndex;
+offset -= firstIndex;
+for (int i = firstIndex ; i < runLength; i++)
+utfBytes[i] = (byte) str.charAt(offset + i);
+out.write(utfBytes, 0, runLength);
+offset += firstIndex;
+firstIndex = 0;
+}
}
else
{
-int charIndex = 0;
-while (charIndex < length)
+int utfIndex = 2;
+utfBytes[0] = (byte) (utfCount >> 8);
+utfBytes[1] = (byte) utfCount;
+for (int charIndex = 0 ; charIndex < length ; charIndex++)
{
-char ch = str.charAt(charIndex);
-int sizeOfChar = sizeOfChar(ch);
-if (utfInd
[jira] [Commented] (CASSANDRA-9079) "respectTombstoneThresholds" should be configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389434#comment-14389434 ] Vinay Chella commented on CASSANDRA-9079: - Yes, that is a good work around and is being done in our clusters. But isn't making "respectTombstoneThresholds" configurable would be the cleaner approach? I am trying to understand the intricacies of exposing "SliceQueryFilter.respectTombstoneThresholds()" to be configurable. > "respectTombstoneThresholds" should be configurable > > > Key: CASSANDRA-9079 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9079 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Vinay Chella > Labels: tombstone > Attachments: 9079.txt > > > Currently there is no way to control/ configure the functionality of "drop > queries exceeding a configurable number of tombstones". Making this > functionality configurable through yaml helps use case like: > We have several production clusters with large number of tombstones due to > aggressive TTLs with very small payloads, and low gc_grace_seconds. Most of > the read queries on these column families are ending up in drop operations or > excessive warn messages due to "tombstone_failure_threshold" and > "tombstone_warn_threshold". One cannot deterministically disable these > tombstone thresholds by setting high values. Hence, making > "SliceQueryFilter.respectTombstoneThresholds()" configurable through yaml > would be the better approach. > Column families with small payloads (couple hundred bytes) having large > number of tombstones would not lead to "Avoid death-by-tombstone by default > (CASSANDRA-6117)", which is the reason for observed drop operations. We see > drop operations before compactions catch up, and clean those tombstones with > both STCS and LCS. Configurable "respectTombstoneThresholds()" will help in > reducing tombstone warnings noise. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8911) Consider Mutation-based Repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389420#comment-14389420 ] Robert Coli commented on CASSANDRA-8911: Mutation-based repair seems to help us in the case that we have to mark a ranges in various sstables unrepaired due to bitrot (CASSANDRA-8703 et al) : we get a CRC error on the read path and can know exactly which single-row-key-wide range needs to be repaired by a mutation. As I understand it, in the status quo, we would have to re-repair all ranges in all sstables which we are marking unrepaired. > Consider Mutation-based Repairs > --- > > Key: CASSANDRA-8911 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8911 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tyler Hobbs > > We should consider a mutation-based repair to replace the existing streaming > repair. While we're at it, we could do away with a lot of the complexity > around merkle trees. > I have not planned this out in detail, but here's roughly what I'm thinking: > * Instead of building an entire merkle tree up front, just send the "leaves" > one-by-one. Instead of dealing with token ranges, make the leaves primary > key ranges. The PK ranges would need to be contiguous, so that the start of > each range would match the end of the previous range. (The first and last > leaves would need to be open-ended on one end of the PK range.) This would be > similar to doing a read with paging. > * Once one page of data is read, compute a hash of it and send it to the > other replicas along with the PK range that it covers and a row count. > * When the replicas receive the hash, the perform a read over the same PK > range (using a LIMIT of the row count + 1) and compare hashes (unless the row > counts don't match, in which case this can be skipped). > * If there is a mismatch, the replica will send a mutation covering that > page's worth of data (ignoring the row count this time) to the source node. > Here are the advantages that I can think of: > * With the current repair behavior of streaming, vnode-enabled clusters may > need to stream hundreds of small SSTables. This results in increased compact > ion load on the receiving node. With the mutation-based approach, memtables > would naturally merge these. > * It's simple to throttle. For example, you could give a number of rows/sec > that should be repaired. > * It's easy to see what PK range has been repaired so far. This could make > it simpler to resume a repair that fails midway. > * Inconsistencies start to be repaired almost right away. > * Less special code \(?\) > * Wide partitions are no longer a problem. > There are a few problems I can think of: > * Counters. I don't know if this can be made safe, or if they need to be > skipped. > * To support incremental repair, we need to be able to read from only > repaired sstables. Probably not too difficult to do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9081) Update bundled python driver
[ https://issues.apache.org/jira/browse/CASSANDRA-9081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-9081: --- Attachment: 0001-C9081-upgrading-python-driver-to-2.5.0.patch > Update bundled python driver > > > Key: CASSANDRA-9081 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9081 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Blake Eggleston > Fix For: 2.1.4 > > Attachments: 0001-C9081-upgrading-python-driver-to-2.5.0.patch > > > https://datastax-oss.atlassian.net/browse/PYTHON-231, included in 2.5.0, > prevents triggers from being displayed in cqlsh -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-9077) Deleting an element from a List which is null throws a NPE
[
https://issues.apache.org/jira/browse/CASSANDRA-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389255#comment-14389255
]
Jeff Jirsa edited comment on CASSANDRA-9077 at 3/31/15 8:36 PM:
option1-v2 addresses all 3 requests:
- Changed wording to be consistent
- Handles NPE in Discarder (removing element from null list)
- Adds unit tests for all 3 cases (Discarder, DiscarderByIndex, SetterByIndex)
Diff was based on trunk, applies cleanly to 2.1 as well.
was (Author: jjirsa):
option1-v2 addresses all 3 requests:
- Changed wording to be consistent
- Handles NPE in Discarder (removing element from null list)
- Adds unit tests for all 3 cases (Discarder, DiscarderByIndex, SetterByIndex)
> Deleting an element from a List which is null throws a NPE
> --
>
> Key: CASSANDRA-9077
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9077
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: dan jatnieks
>Assignee: Jeff Jirsa
>Priority: Minor
> Attachments: 9077-stacktrace.log, cassandra-9077-option1-v2.txt,
> cassandra-9077-option1.txt, cassandra-9077-option2.txt
>
>
> I am seeing an NPE on the latest 2.1 branch with this sequence of deletes
> from a list - first delete the entire list, then attempt to delete one
> element.
> I expected to see {{List index 0 out of bound, list has size 0}} but instead
> got an NPE.
> {noformat}
> ./bin/cqlsh
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 2.1.3-SNAPSHOT | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh> use frozen_collections ;
> cqlsh:frozen_collections> DROP TABLE IF EXISTS t;
> cqlsh:frozen_collections> CREATE TABLE t (id text PRIMARY KEY, l list,
> s set);
> cqlsh:frozen_collections> INSERT INTO t (id, l, s) VALUES ('user', ['1'],
> {'1'});
> cqlsh:frozen_collections>
> cqlsh:frozen_collections> DELETE l FROM t WHERE id ='user';
> cqlsh:frozen_collections> //INSERT INTO t (id, l) VALUES ('user', ['1']);
> cqlsh:frozen_collections> DELETE l[0] FROM t WHERE id = 'user';
> ServerError: message="java.lang.NullPointerException">
> cqlsh:frozen_collections>
> cqlsh:frozen_collections> DELETE s FROM t WHERE id ='user';
> cqlsh:frozen_collections> DELETE s['1'] FROM t WHERE id = 'user';
> {noformat}
> It appears the {{DELETE emails...}} directly followed by {{DELETE
> emails[0]...}} is the offending sequence. Either one alone works fine, as
> does adding an intervening insert/update.
> The same sequence performed on a Set rather than List works (as shown above).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-9081) Update bundled python driver
Blake Eggleston created CASSANDRA-9081: -- Summary: Update bundled python driver Key: CASSANDRA-9081 URL: https://issues.apache.org/jira/browse/CASSANDRA-9081 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Blake Eggleston Fix For: 2.1.4 https://datastax-oss.atlassian.net/browse/PYTHON-231, included in 2.5.0, prevents triggers from being displayed in cqlsh -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9056) Tombstoned SSTables are not removed past max_sstable_age_days when using DTCS
[ https://issues.apache.org/jira/browse/CASSANDRA-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389299#comment-14389299 ] Tyler Hobbs commented on CASSANDRA-9056: Linking to CASSANDRA-8996, as this is one of the few remaining dtest failures on trunk. > Tombstoned SSTables are not removed past max_sstable_age_days when using DTCS > - > > Key: CASSANDRA-9056 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9056 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Shawn Kumar >Assignee: Marcus Eriksson > Labels: compaction, dtcs > > When using DTCS, tombstoned sstables past max_sstable_age_days are not > removed by minor compactions. I was able to reproduce this manually and also > wrote a dtest (currently failing) which reproduces this issue: > [dtcs_deletion_test|https://github.com/riptano/cassandra-dtest/blob/master/compaction_test.py#L115] > in compaction_test.py. I tried applying the patch in CASSANDRA-8359 but > found that the test still fails with the same issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9077) Deleting an element from a List which is null throws a NPE
[
https://issues.apache.org/jira/browse/CASSANDRA-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-9077:
--
Attachment: cassandra-9077-option1-v2.txt
option1-v2 addresses all 3 requests:
- Changed wording to be consistent
- Handles NPE in Discarder (removing element from null list)
- Adds unit tests for all 3 cases (Discarder, DiscarderByIndex, SetterByIndex)
> Deleting an element from a List which is null throws a NPE
> --
>
> Key: CASSANDRA-9077
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9077
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: dan jatnieks
>Assignee: Jeff Jirsa
>Priority: Minor
> Attachments: 9077-stacktrace.log, cassandra-9077-option1-v2.txt,
> cassandra-9077-option1.txt, cassandra-9077-option2.txt
>
>
> I am seeing an NPE on the latest 2.1 branch with this sequence of deletes
> from a list - first delete the entire list, then attempt to delete one
> element.
> I expected to see {{List index 0 out of bound, list has size 0}} but instead
> got an NPE.
> {noformat}
> ./bin/cqlsh
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 2.1.3-SNAPSHOT | CQL spec 3.2.0 | Native protocol v3]
> Use HELP for help.
> cqlsh> use frozen_collections ;
> cqlsh:frozen_collections> DROP TABLE IF EXISTS t;
> cqlsh:frozen_collections> CREATE TABLE t (id text PRIMARY KEY, l list,
> s set);
> cqlsh:frozen_collections> INSERT INTO t (id, l, s) VALUES ('user', ['1'],
> {'1'});
> cqlsh:frozen_collections>
> cqlsh:frozen_collections> DELETE l FROM t WHERE id ='user';
> cqlsh:frozen_collections> //INSERT INTO t (id, l) VALUES ('user', ['1']);
> cqlsh:frozen_collections> DELETE l[0] FROM t WHERE id = 'user';
> ServerError: message="java.lang.NullPointerException">
> cqlsh:frozen_collections>
> cqlsh:frozen_collections> DELETE s FROM t WHERE id ='user';
> cqlsh:frozen_collections> DELETE s['1'] FROM t WHERE id = 'user';
> {noformat}
> It appears the {{DELETE emails...}} directly followed by {{DELETE
> emails[0]...}} is the offending sequence. Either one alone works fine, as
> does adding an intervening insert/update.
> The same sequence performed on a Set rather than List works (as shown above).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8351) Running COPY FROM in cqlsh aborts with errors or segmentation fault
[
https://issues.apache.org/jira/browse/CASSANDRA-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs resolved CASSANDRA-8351.
Resolution: Fixed
I believe this has been (coincidentally) fixed by CASSANDRA-8225. I was able
to reproduce the issue against Cassandra 2.1.3, but the latest cassandra-2.1
does not reproduce. Although I'd like to understand the original reason for
the segfault, debugging threaded python applications in gdb is particularly
painful, so I'm not going to dig in any further.
I did notice one bug from 8225 relating to quoting column names, so I've opened
CASSANDRA-9080 to fix that.
> Running COPY FROM in cqlsh aborts with errors or segmentation fault
> ---
>
> Key: CASSANDRA-8351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8351
> Project: Cassandra
> Issue Type: Bug
>Reporter: Joseph Chu
>Assignee: Tyler Hobbs
>Priority: Minor
> Labels: cqlsh
> Fix For: 2.1.4
>
> Attachments: stress.cql, stress.csv
>
>
> Running Cassandra 2.1.2 binary tarball on a single instance.
> Put together a script to try to reproduce this using data generated by
> cassandra-stress.
> Reproduction steps: Download files and run cqlsh -f stress.cql
> This may need to run a couple of times before errors are encountered. I've
> seen this work best when running after a fresh install.
> Errors seen:
> 1.Segmentation fault (core dumped)
> 2.
> {code}
>stress.cql:24:line contains NULL byte
>stress.cql:24:Aborting import at record #0. Previously-inserted values
> still present.
>71 rows imported in 0.100 seconds.{code}
> 3.
> {code}*** glibc detected *** python: corrupted double-linked list:
> 0x01121ad0 ***
> === Backtrace: =
> /lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7f80fe0cdb96]
> /lib/x86_64-linux-gnu/libc.so.6(+0x7fead)[0x7f80fe0ceead]
> python[0x42615d]
> python[0x501dc8]
> python[0x4ff715]
> python[0x425d02]
> python(PyEval_EvalCodeEx+0x1c4)[0x575db4]
> python[0x577be2]
> python(PyObject_Call+0x36)[0x4d91b6]
> python(PyEval_EvalFrameEx+0x2035)[0x54d8a5]
> python(PyEval_EvalCodeEx+0x1a2)[0x575d92]
> python(PyEval_EvalFrameEx+0x7b8)[0x54c028]
> python(PyEval_EvalCodeEx+0x1a2)[0x575d92]
> python(PyEval_EvalFrameEx+0x7b8)[0x54c028]
> python(PyEval_EvalFrameEx+0xa02)[0x54c272]
> python(PyEval_EvalFrameEx+0xa02)[0x54c272]
> python(PyEval_EvalFrameEx+0xa02)[0x54c272]
> python(PyEval_EvalCodeEx+0x1a2)[0x575d92]
> python(PyEval_EvalFrameEx+0x7b8)[0x54c028]
> python(PyEval_EvalCodeEx+0x1a2)[0x575d92]
> python(PyEval_EvalFrameEx+0x7b8)[0x54c028]
> python(PyEval_EvalCodeEx+0x1a2)[0x575d92]
> python[0x577be2]
> python(PyObject_Call+0x36)[0x4d91b6]
> python(PyEval_EvalFrameEx+0x2035)[0x54d8a5]
> python(PyEval_EvalFrameEx+0xa02)[0x54c272]
> python(PyEval_EvalFrameEx+0xa02)[0x54c272]
> python(PyEval_EvalCodeEx+0x1a2)[0x575d92]
> python[0x577ab0]
> python(PyObject_Call+0x36)[0x4d91b6]
> python[0x4c91fa]
> python(PyObject_Call+0x36)[0x4d91b6]
> python(PyEval_CallObjectWithKeywords+0x36)[0x4d97c6]
> python[0x4f7f58]
> /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7f80ff369e9a]
> /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f80fe1433fd]
> === Memory map:
> 0040-00672000 r-xp 08:01 1447344
> /usr/bin/python2.7
> 00871000-00872000 r--p 00271000 08:01 1447344
> /usr/bin/python2.7
> 00872000-008db000 rw-p 00272000 08:01 1447344
> /usr/bin/python2.7
> 008db000-008ed000 rw-p 00:00 0
> 0090e000-0126 rw-p 00:00 0
> [heap]
> 7f80ec00-7f80ec0aa000 rw-p 00:00 0
> 7f80ec0aa000-7f80f000 ---p 00:00 0
> 7f80f000-7f80f0021000 rw-p 00:00 0
> 7f80f0021000-7f80f400 ---p 00:00 0
> 7f80f400-7f80f4021000 rw-p 00:00 0
> 7f80f4021000-7f80f800 ---p 00:00 0
> 7f80fa713000-7f80fa714000 ---p 00:00 0
> 7f80fa714000-7f80faf14000 rw-p 00:00 0
> [stack:7493]
> 7f80faf14000-7f80faf15000 ---p 00:00 0
> 7f80faf15000-7f80fb715000 rw-p 00:00 0
> [stack:7492]
> 7f80fb715000-7f80fb716000 ---p 00:00 0
> 7f80fb716000-7f80fbf16000 rw-p 00:00 0
> [stack:7491]
> 7f80fbf16000-7f80fbf21000 r-xp 08:01 1456254
> /usr/lib/python2.7/lib-dynload/_json.so
> 7f80fbf21000-7f80fc12 ---p b000 08:01 1456254
> /usr/lib/python2.7/lib-dynload/_json.so
> 7f80fc12-7f80fc121000 r--p a000 08:01 1456254
> /usr/lib/python2.7/lib-dynload/_json.so
> 7f80fc121000-7f80fc122000 rw-p b000 0
[jira] [Commented] (CASSANDRA-8938) Full Row Scan does not count towards Reads
[ https://issues.apache.org/jira/browse/CASSANDRA-8938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389253#comment-14389253 ] Jonathan Ellis commented on CASSANDRA-8938: --- Sounds reasonable. > Full Row Scan does not count towards Reads > -- > > Key: CASSANDRA-8938 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8938 > Project: Cassandra > Issue Type: Bug > Components: API, Core, Tools > Environment: Unix, Cassandra 2.0.3 >Reporter: Amit Singh Chowdhery >Assignee: Marcus Eriksson >Priority: Minor > Labels: none > > When a CQL SELECT statement is executed with WHERE clause, Read Count is > incremented in cfstats of the column family. But, when a full row scan is > done using SELECT statement without WHERE clause, Read Count is not > incremented. > Similarly, when using Size Tiered Compaction, if we do a full row scan using > Hector RangeslicesQuery, Read Count is not incremented in cfstats, Cassandra > still considers all sstables as cold and does not trigger compaction for > them. If we fire MultigetSliceQuery, Read Count is incremented and sstables > becomes hot, triggering compaction of these sstables. > Expected Behavior: > 1. Read Count must be incremented by number of rows read during a full row > scan done using CQL SELECT statement or Hector RangeslicesQuery. > 2. Size Tiered compaction must consider all sstables as Hot after a full row > scan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9080) cqlsh: COPY FROM doesn't quote column names
[ https://issues.apache.org/jira/browse/CASSANDRA-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-9080: --- Attachment: 9080-2.1.txt The attached patch quotes column names in the insert statement. > cqlsh: COPY FROM doesn't quote column names > --- > > Key: CASSANDRA-9080 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9080 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Tyler Hobbs >Assignee: Tyler Hobbs > Labels: cqlsh > Fix For: 2.1.4 > > Attachments: 9080-2.1.txt > > > The changes in CASSANDRA-8225 had one bug: column names aren't quoted in the > insert statement (when needed). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9080) cqlsh: COPY FROM doesn't quote column names
[ https://issues.apache.org/jira/browse/CASSANDRA-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-9080: --- Labels: cqlsh (was: ) > cqlsh: COPY FROM doesn't quote column names > --- > > Key: CASSANDRA-9080 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9080 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Tyler Hobbs >Assignee: Tyler Hobbs > Labels: cqlsh > Fix For: 2.1.4 > > > The changes in CASSANDRA-8225 had one bug: column names aren't quoted in the > insert statement (when needed). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-9080) cqlsh: COPY FROM doesn't quote column names
Tyler Hobbs created CASSANDRA-9080: -- Summary: cqlsh: COPY FROM doesn't quote column names Key: CASSANDRA-9080 URL: https://issues.apache.org/jira/browse/CASSANDRA-9080 Project: Cassandra Issue Type: Bug Components: Tools Reporter: Tyler Hobbs Assignee: Tyler Hobbs Fix For: 2.1.4 The changes in CASSANDRA-8225 had one bug: column names aren't quoted in the insert statement (when needed). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8336) Quarantine nodes after receiving the gossip shutdown message
[ https://issues.apache.org/jira/browse/CASSANDRA-8336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-8336: Attachment: 8336-v4.txt After wrestling with exceptions for a bit, I came up with a simpler solution. Gossiper's stop() can examine the local state itself, and skip shutdown announcement if it doesn't exist. We still need stopSilently (which I renamed in this patch from stopForLeaving) for cases like decom, where we aren't coming back and don't wait to mutate our state on shutdown. > Quarantine nodes after receiving the gossip shutdown message > > > Key: CASSANDRA-8336 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8336 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Brandon Williams >Assignee: Brandon Williams > Fix For: 2.0.14 > > Attachments: 8336-v2.txt, 8336-v3.txt, 8336-v4.txt, 8336.txt > > > In CASSANDRA-3936 we added a gossip shutdown announcement. The problem here > is that this isn't sufficient; you can still get TOEs and have to wait on the > FD to figure things out. This happens due to gossip propagation time and > variance; if node X shuts down and sends the message to Y, but Z has a > greater gossip version than Y for X and has not yet received the message, it > can initiate gossip with Y and thus mark X alive again. I propose > quarantining to solve this, however I feel it should be a -D parameter you > have to specify, so as not to destroy current dev and test practices, since > this will mean a node that shuts down will not be able to restart until the > quarantine expires. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9079) "respectTombstoneThresholds" should be configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389171#comment-14389171
]
Tyler Hobbs commented on CASSANDRA-9079:
Why not just set {{tombstone_failure_threshold}} to {{Integer.MAX_VALUE}} (2^31
- 1)? I don't think we need to add a new config option for this.
> "respectTombstoneThresholds" should be configurable
>
>
> Key: CASSANDRA-9079
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9079
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Vinay Chella
> Labels: tombstone
> Attachments: 9079.txt
>
>
> Currently there is no way to control/ configure the functionality of "drop
> queries exceeding a configurable number of tombstones". Making this
> functionality configurable through yaml helps use case like:
> We have several production clusters with large number of tombstones due to
> aggressive TTLs with very small payloads, and low gc_grace_seconds. Most of
> the read queries on these column families are ending up in drop operations or
> excessive warn messages due to "tombstone_failure_threshold" and
> "tombstone_warn_threshold". One cannot deterministically disable these
> tombstone thresholds by setting high values. Hence, making
> "SliceQueryFilter.respectTombstoneThresholds()" configurable through yaml
> would be the better approach.
> Column families with small payloads (couple hundred bytes) having large
> number of tombstones would not lead to "Avoid death-by-tombstone by default
> (CASSANDRA-6117)", which is the reason for observed drop operations. We see
> drop operations before compactions catch up, and clean those tombstones with
> both STCS and LCS. Configurable "respectTombstoneThresholds()" will help in
> reducing tombstone warnings noise.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8805) runWithCompactionsDisabled only cancels compactions, which is not the only source of markCompacted
[ https://issues.apache.org/jira/browse/CASSANDRA-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs reassigned CASSANDRA-8805: -- Assignee: Tyler Hobbs > runWithCompactionsDisabled only cancels compactions, which is not the only > source of markCompacted > -- > > Key: CASSANDRA-8805 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8805 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Tyler Hobbs > Fix For: 2.1.4 > > > Operations like repair that may operate over all sstables cancel compactions > before beginning, and fail if there are any files marked compacting after > doing so. Redistribution of index summaries is not a compaction, so is not > cancelled by this action, but does mark sstables as compacting, so such an > action will fail to initiate if there is an index summary redistribution in > progress. It seems that IndexSummaryManager needs to register itself as > interruptible along with compactions (AFAICT no other actions that may > markCompacting are not themselves compactions). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9079) "respectTombstoneThresholds" should be configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay Chella updated CASSANDRA-9079: Fix Version/s: (was: 2.0.14) > "respectTombstoneThresholds" should be configurable > > > Key: CASSANDRA-9079 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9079 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Vinay Chella > Labels: tombstone > Attachments: 9079.txt > > > Currently there is no way to control/ configure the functionality of "drop > queries exceeding a configurable number of tombstones". Making this > functionality configurable through yaml helps use case like: > We have several production clusters with large number of tombstones due to > aggressive TTLs with very small payloads, and low gc_grace_seconds. Most of > the read queries on these column families are ending up in drop operations or > excessive warn messages due to "tombstone_failure_threshold" and > "tombstone_warn_threshold". One cannot deterministically disable these > tombstone thresholds by setting high values. Hence, making > "SliceQueryFilter.respectTombstoneThresholds()" configurable through yaml > would be the better approach. > Column families with small payloads (couple hundred bytes) having large > number of tombstones would not lead to "Avoid death-by-tombstone by default > (CASSANDRA-6117)", which is the reason for observed drop operations. We see > drop operations before compactions catch up, and clean those tombstones with > both STCS and LCS. Configurable "respectTombstoneThresholds()" will help in > reducing tombstone warnings noise. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-9079) "respectTombstoneThresholds" should be configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389100#comment-14389100 ] Vinay Chella edited comment on CASSANDRA-9079 at 3/31/15 6:27 PM: -- Adding the Patch was (Author: vinaykumarcse): Patch > "respectTombstoneThresholds" should be configurable > > > Key: CASSANDRA-9079 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9079 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Vinay Chella > Labels: tombstone > Fix For: 2.0.14 > > Attachments: 9079.txt > > > Currently there is no way to control/ configure the functionality of "drop > queries exceeding a configurable number of tombstones". Making this > functionality configurable through yaml helps use case like: > We have several production clusters with large number of tombstones due to > aggressive TTLs with very small payloads, and low gc_grace_seconds. Most of > the read queries on these column families are ending up in drop operations or > excessive warn messages due to "tombstone_failure_threshold" and > "tombstone_warn_threshold". One cannot deterministically disable these > tombstone thresholds by setting high values. Hence, making > "SliceQueryFilter.respectTombstoneThresholds()" configurable through yaml > would be the better approach. > Column families with small payloads (couple hundred bytes) having large > number of tombstones would not lead to "Avoid death-by-tombstone by default > (CASSANDRA-6117)", which is the reason for observed drop operations. We see > drop operations before compactions catch up, and clean those tombstones with > both STCS and LCS. Configurable "respectTombstoneThresholds()" will help in > reducing tombstone warnings noise. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9079) "respectTombstoneThresholds" should be configurable
[ https://issues.apache.org/jira/browse/CASSANDRA-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay Chella updated CASSANDRA-9079: Attachment: 9079.txt Patch > "respectTombstoneThresholds" should be configurable > > > Key: CASSANDRA-9079 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9079 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Vinay Chella > Labels: tombstone > Fix For: 2.0.14 > > Attachments: 9079.txt > > > Currently there is no way to control/ configure the functionality of "drop > queries exceeding a configurable number of tombstones". Making this > functionality configurable through yaml helps use case like: > We have several production clusters with large number of tombstones due to > aggressive TTLs with very small payloads, and low gc_grace_seconds. Most of > the read queries on these column families are ending up in drop operations or > excessive warn messages due to "tombstone_failure_threshold" and > "tombstone_warn_threshold". One cannot deterministically disable these > tombstone thresholds by setting high values. Hence, making > "SliceQueryFilter.respectTombstoneThresholds()" configurable through yaml > would be the better approach. > Column families with small payloads (couple hundred bytes) having large > number of tombstones would not lead to "Avoid death-by-tombstone by default > (CASSANDRA-6117)", which is the reason for observed drop operations. We see > drop operations before compactions catch up, and clean those tombstones with > both STCS and LCS. Configurable "respectTombstoneThresholds()" will help in > reducing tombstone warnings noise. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-9079) "respectTombstoneThresholds" should be configurable
Vinay Chella created CASSANDRA-9079: --- Summary: "respectTombstoneThresholds" should be configurable Key: CASSANDRA-9079 URL: https://issues.apache.org/jira/browse/CASSANDRA-9079 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Vinay Chella Fix For: 2.0.14 Currently there is no way to control/ configure the functionality of "drop queries exceeding a configurable number of tombstones". Making this functionality configurable through yaml helps use case like: We have several production clusters with large number of tombstones due to aggressive TTLs with very small payloads, and low gc_grace_seconds. Most of the read queries on these column families are ending up in drop operations or excessive warn messages due to "tombstone_failure_threshold" and "tombstone_warn_threshold". One cannot deterministically disable these tombstone thresholds by setting high values. Hence, making "SliceQueryFilter.respectTombstoneThresholds()" configurable through yaml would be the better approach. Column families with small payloads (couple hundred bytes) having large number of tombstones would not lead to "Avoid death-by-tombstone by default (CASSANDRA-6117)", which is the reason for observed drop operations. We see drop operations before compactions catch up, and clean those tombstones with both STCS and LCS. Configurable "respectTombstoneThresholds()" will help in reducing tombstone warnings noise. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8672) Ambiguous WriteTimeoutException while completing pending CAS commits
[
https://issues.apache.org/jira/browse/CASSANDRA-8672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389063#comment-14389063
]
Tyler Hobbs commented on CASSANDRA-8672:
I think the best option is to add a new {{WriteType.CAS_PREPARE}} for the v4
protocol (and continue to use {{WriteType.CAS}} for v1 through v3). If we
timeout during {{beginAndRepairPaxos()}}, we use {{WriteType.CAS_PREPARE}} for
the WriteTimeoutException. What do you think, [~slebresne]?
> Ambiguous WriteTimeoutException while completing pending CAS commits
>
>
> Key: CASSANDRA-8672
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8672
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Stefan Podkowinski
>Assignee: Tyler Hobbs
>Priority: Minor
> Labels: CAS
> Fix For: 3.0
>
>
> Any CAS update has a chance to trigger a pending/stalled commit of any
> previously agreed on CAS update. After completing the pending commit, the CAS
> operation will resume to execute the actual update and also possibly create a
> new commit. See StorageProxy.cas()
> Theres two possbile execution paths that might end up throwing a
> WriteTimeoutException:
> cas() -> beginAndRepairPaxos() -> commitPaxos()
> cas() -> commitPaxos()
> Unfortunatelly clients catching a WriteTimeoutException won't be able to tell
> at which stage the commit failed. My guess would be that most developers are
> not aware that the beginAndRepairPaxos() could also trigger a write and
> assume that write timeouts would refer to a timeout while writting the actual
> CAS update. Its therefor not safe to assume that successive CAS or SERIAL
> read operations will cause a (write-)timeouted CAS operation to get
> eventually applied. Although some [best-practices
> advise|http://www.datastax.com/dev/blog/cassandra-error-handling-done-right]
> claims otherwise.
> At this point the safest bet is possibly to retry the complete business
> transaction in case of an WriteTimeoutException. However, as theres a chance
> that the timeout occurred while writing the actual CAS operation, another
> write could potentially complete it and our CAS condition will get a
> different result upon retry.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7814) enable describe on indices
[
https://issues.apache.org/jira/browse/CASSANDRA-7814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388946#comment-14388946
]
Tyler Hobbs commented on CASSANDRA-7814:
Since this is a low-priority improvement for cqlsh, I would prefer to wait for
the python driver changes to be included in an official release before
upgrading the bundled python driver. If the driver changes don't get merged,
we can commit the more expensive version in cqlsh.
[~Stefania] can you go ahead and open a pull request for the python driver?
(Make sure to use a branch other than master for that.)
As for automatically including the hash in the zip filename, I agree that that
would probably not be committed unless it was controlled by an optional flag.
You could add support for a flag similarly to the existing {{--no-extensions}},
{{--no-murmur3}}, and {{--no-libev}} flags. I would make those changes in a
second pull request to make them easier to exclude, if needed.
> enable describe on indices
> --
>
> Key: CASSANDRA-7814
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7814
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: radha
>Assignee: Stefania
>Priority: Minor
> Fix For: 2.1.4
>
>
> Describe index should be supported, right now, the only way is to export the
> schema and find what it really is before updating/dropping the index.
> verified in
> [cqlsh 3.1.8 | Cassandra 1.2.18.1 | CQL spec 3.0.0 | Thrift protocol 19.36.2]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9070) Race in cancelling compactions
[
https://issues.apache.org/jira/browse/CASSANDRA-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388934#comment-14388934
]
Yuki Morishita commented on CASSANDRA-9070:
---
You are right. It's {{BackgroundCompactionTask}} that's queued.
Cancelling minor compaction seems good with the patch.
+1.
> Race in cancelling compactions
> --
>
> Key: CASSANDRA-9070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9070
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.0.14
>
> Attachments:
> 0001-check-if-we-are-paused-before-starting-compaction.patch
>
>
> seems we might have a race situation when cancelling compactions
> currently we do the following to ensure that we don't start any new
> compactions when we try to do markAllCompacting()
> # pause compactions - this makes sure we don't create any new compaction
> tasks from the compaction strategies
> # cancel any ongoing compactions - compactions register themselves with the
> CompactionMetrics and then, when cancelling we get all compactions here, and
> tell them to stop
> Problem is that there is a window between when the CompactionTask is created
> and when it is registered in CompactionMetrics meaning with a bit of bad
> luck, we could have a situation like this:
> # we finish a compaction and create a new CompactionTask from the compaction
> strategy
> # we pause the compaction strategies to not create any new CompactionTasks
> # we cancel all ongoing compactions
> # The CompactionTask created in #1 above registers itself in
> CompactionMetrics and misses that it should be cancelled
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8938) Full Row Scan does not count towards Reads
[ https://issues.apache.org/jira/browse/CASSANDRA-8938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1431#comment-1431 ] Tyler Hobbs commented on CASSANDRA-8938: Historically we have excluded range scans from counting towards sstable hotness on purpose, but perhaps we should reconsider. A good cluster setup may use multiple DCs to split analytical and transactional workloads. In that scenario, we would ideally measure sstable hotness in the analytical DC by range scans. Even if analytical and transactional workloads are both run on the same nodes, the range scans should account for a comparatively small percentage of reads, so they shouldn't impact the hotness measurements by very much. In other words, I think it would be safe to let a range scan count for one read (and adjust the hotness measurement accordingly). > Full Row Scan does not count towards Reads > -- > > Key: CASSANDRA-8938 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8938 > Project: Cassandra > Issue Type: Bug > Components: API, Core, Tools > Environment: Unix, Cassandra 2.0.3 >Reporter: Amit Singh Chowdhery >Assignee: Marcus Eriksson >Priority: Minor > Labels: none > > When a CQL SELECT statement is executed with WHERE clause, Read Count is > incremented in cfstats of the column family. But, when a full row scan is > done using SELECT statement without WHERE clause, Read Count is not > incremented. > Similarly, when using Size Tiered Compaction, if we do a full row scan using > Hector RangeslicesQuery, Read Count is not incremented in cfstats, Cassandra > still considers all sstables as cold and does not trigger compaction for > them. If we fire MultigetSliceQuery, Read Count is incremented and sstables > becomes hot, triggering compaction of these sstables. > Expected Behavior: > 1. Read Count must be incremented by number of rows read during a full row > scan done using CQL SELECT statement or Hector RangeslicesQuery. > 2. Size Tiered compaction must consider all sstables as Hot after a full row > scan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9070) Race in cancelling compactions
[ https://issues.apache.org/jira/browse/CASSANDRA-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388874#comment-14388874 ] Marcus Eriksson commented on CASSANDRA-9070: bq. If CompactionTask is already queued with 'marked' SSTables I think that would still work (unless I misunderstood you comment) - for background compactions we queue up BackgroundCompactionTasks and don't actually ask for the compaction task until the BGTask executes, and any other compactions would start executing and throw CompactionException since we are paused and release any sstables they have > Race in cancelling compactions > -- > > Key: CASSANDRA-9070 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9070 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.0.14 > > Attachments: > 0001-check-if-we-are-paused-before-starting-compaction.patch > > > seems we might have a race situation when cancelling compactions > currently we do the following to ensure that we don't start any new > compactions when we try to do markAllCompacting() > # pause compactions - this makes sure we don't create any new compaction > tasks from the compaction strategies > # cancel any ongoing compactions - compactions register themselves with the > CompactionMetrics and then, when cancelling we get all compactions here, and > tell them to stop > Problem is that there is a window between when the CompactionTask is created > and when it is registered in CompactionMetrics meaning with a bit of bad > luck, we could have a situation like this: > # we finish a compaction and create a new CompactionTask from the compaction > strategy > # we pause the compaction strategies to not create any new CompactionTasks > # we cancel all ongoing compactions > # The CompactionTask created in #1 above registers itself in > CompactionMetrics and misses that it should be cancelled -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8348) allow takeColumnFamilySnapshot to take a list of ColumnFamilies
[ https://issues.apache.org/jira/browse/CASSANDRA-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388850#comment-14388850 ] Nick Bailey commented on CASSANDRA-8348: Also, we'll probably want a second patch against 3.0 since we changed 'column family' to 'table' everywhere. > allow takeColumnFamilySnapshot to take a list of ColumnFamilies > --- > > Key: CASSANDRA-8348 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8348 > Project: Cassandra > Issue Type: Improvement >Reporter: Peter Halliday >Priority: Minor > Fix For: 2.1.4 > > Attachments: Patch-8348.patch > > > Within StorageServiceMBean.java the function takeSnapshot allows for a list > of keyspaces to snapshot. However, the function takeColumnFamilySnapshot > only allows for a single ColumnFamily to snapshot. This should allow for > multiple ColumnFamilies within the same Keyspace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9078) Fail repair when cluster is in a mixed version state
[ https://issues.apache.org/jira/browse/CASSANDRA-9078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-9078: --- Fix Version/s: 3.0 > Fail repair when cluster is in a mixed version state > > > Key: CASSANDRA-9078 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9078 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Darla Baker >Priority: Minor > Fix For: 3.0 > > > Users often have cron jobs or other automation set up to perform regular > repair maintenance in the cluster. During an upgrade when the cluster is in > a mixed state this can be problematic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8348) allow takeColumnFamilySnapshot to take a list of ColumnFamilies
[ https://issues.apache.org/jira/browse/CASSANDRA-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388846#comment-14388846 ] Nick Bailey edited comment on CASSANDRA-8348 at 3/31/15 5:04 PM: - Unfortunately the patch doesn't apply to 2.1 anymore. Sorry for the delay, can you update the patch [~SachinJanani]? The code looks pretty good to me except I think you call getValidKeyspace() on the same keyspace twice. Also it looks like your IDE auto expanded any '\*' imports. I don't remember what the c\* team's policy on that is. was (Author: nickmbailey): Unfortunately the patch doesn't apply to 2.1 anymore. Sorry for the delay, can you update the patch [~SachinJanani]? The code looks pretty good to me except I think you call getValidKeyspace() on the same keyspace twice. Also it looks like your IDE auto expanded any '*' imports. I don't remember what the c* team's policy on that is. > allow takeColumnFamilySnapshot to take a list of ColumnFamilies > --- > > Key: CASSANDRA-8348 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8348 > Project: Cassandra > Issue Type: Improvement >Reporter: Peter Halliday >Priority: Minor > Fix For: 2.1.4 > > Attachments: Patch-8348.patch > > > Within StorageServiceMBean.java the function takeSnapshot allows for a list > of keyspaces to snapshot. However, the function takeColumnFamilySnapshot > only allows for a single ColumnFamily to snapshot. This should allow for > multiple ColumnFamilies within the same Keyspace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8348) allow takeColumnFamilySnapshot to take a list of ColumnFamilies
[ https://issues.apache.org/jira/browse/CASSANDRA-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388846#comment-14388846 ] Nick Bailey commented on CASSANDRA-8348: Unfortunately the patch doesn't apply to 2.1 anymore. Sorry for the delay, can you update the patch [~SachinJanani]? The code looks pretty good to me except I think you call getValidKeyspace() on the same keyspace twice. Also it looks like your IDE auto expanded any '*' imports. I don't remember what the c* team's policy on that is. > allow takeColumnFamilySnapshot to take a list of ColumnFamilies > --- > > Key: CASSANDRA-8348 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8348 > Project: Cassandra > Issue Type: Improvement >Reporter: Peter Halliday >Priority: Minor > Fix For: 2.1.4 > > Attachments: Patch-8348.patch > > > Within StorageServiceMBean.java the function takeSnapshot allows for a list > of keyspaces to snapshot. However, the function takeColumnFamilySnapshot > only allows for a single ColumnFamily to snapshot. This should allow for > multiple ColumnFamilies within the same Keyspace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9045) Deleted columns are resurrected after repair in wide rows
[
https://issues.apache.org/jira/browse/CASSANDRA-9045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388845#comment-14388845
]
Philip Thompson commented on CASSANDRA-9045:
Nope, no branch. I'll turn my scripts into a real test, and add the compaction
step and more deletes to see what I can get to happen.
> Deleted columns are resurrected after repair in wide rows
> -
>
> Key: CASSANDRA-9045
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9045
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Roman Tkachenko
>Assignee: Marcus Eriksson
>Priority: Critical
> Fix For: 2.0.14
>
> Attachments: cqlsh.txt
>
>
> Hey guys,
> After almost a week of researching the issue and trying out multiple things
> with (almost) no luck I was suggested (on the user@cass list) to file a
> report here.
> h5. Setup
> Cassandra 2.0.13 (we had the issue with 2.0.10 as well and upgraded to see if
> it goes away)
> Multi datacenter 12+6 nodes cluster.
> h5. Schema
> {code}
> cqlsh> describe keyspace blackbook;
> CREATE KEYSPACE blackbook WITH replication = {
> 'class': 'NetworkTopologyStrategy',
> 'IAD': '3',
> 'ORD': '3'
> };
> USE blackbook;
> CREATE TABLE bounces (
> domainid text,
> address text,
> message text,
> "timestamp" bigint,
> PRIMARY KEY (domainid, address)
> ) WITH
> bloom_filter_fp_chance=0.10 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.10 AND
> gc_grace_seconds=864000 AND
> index_interval=128 AND
> read_repair_chance=0.00 AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='99.0PERCENTILE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'LeveledCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> {code}
> h5. Use case
> Each row (defined by a domainid) can have many many columns (bounce entries)
> so rows can get pretty wide. In practice, most of the rows are not that big
> but some of them contain hundreds of thousands and even millions of columns.
> Columns are not TTL'ed but can be deleted using the following CQL3 statement:
> {code}
> delete from bounces where domainid = 'domain.com' and address =
> 'al...@example.com';
> {code}
> All queries are performed using LOCAL_QUORUM CL.
> h5. Problem
> We weren't very diligent about running repairs on the cluster initially, but
> shorty after we started doing it we noticed that some of previously deleted
> columns (bounce entries) are there again, as if tombstones have disappeared.
> I have run this test multiple times via cqlsh, on the row of the customer who
> originally reported the issue:
> * delete an entry
> * verify it's not returned even with CL=ALL
> * run repair on nodes that own this row's key
> * the columns reappear and are returned even with CL=ALL
> I tried the same test on another row with much less data and everything was
> correctly deleted and didn't reappear after repair.
> h5. Other steps I've taken so far
> Made sure NTP is running on all servers and clocks are synchronized.
> Increased gc_grace_seconds to 100 days, ran full repair (on the affected
> keyspace) on all nodes, then changed it back to the default 10 days again.
> Didn't help.
> Performed one more test. Updated one of the resurrected columns, then deleted
> it and ran repair again. This time the updated version of the column
> reappeared.
> Finally, I noticed these log entries for the row in question:
> {code}
> INFO [ValidationExecutor:77] 2015-03-25 20:27:43,936
> CompactionController.java (line 192) Compacting large row
> blackbook/bounces:4ed558feba8a483733001d6a (279067683 bytes) incrementally
> {code}
> Figuring it may be related I bumped "in_memory_compaction_limit_in_mb" to
> 512MB so the row fits into it, deleted the entry and ran repair once again.
> The log entry for this row was gone and the columns didn't reappear.
> We have a lot of rows much larger than 512MB so can't increase this
> parameters forever, if that is the issue.
> Please let me know if you need more information on the case or if I can run
> more experiments.
> Thanks!
> Roman
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9045) Deleted columns are resurrected after repair in wide rows
[
https://issues.apache.org/jira/browse/CASSANDRA-9045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388840#comment-14388840
]
Tyler Hobbs commented on CASSANDRA-9045:
Do you have a branch with the test somewhere?
I believe you need to run a compaction after the "delete" step. You should
perhaps also randomly delete a lot of rows (say, 10k) instead of just one (if
that's what you're doing). I don't believe the repair step should be necessary
to repro, if it's failing in the way I think it is.
> Deleted columns are resurrected after repair in wide rows
> -
>
> Key: CASSANDRA-9045
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9045
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Roman Tkachenko
>Assignee: Marcus Eriksson
>Priority: Critical
> Fix For: 2.0.14
>
> Attachments: cqlsh.txt
>
>
> Hey guys,
> After almost a week of researching the issue and trying out multiple things
> with (almost) no luck I was suggested (on the user@cass list) to file a
> report here.
> h5. Setup
> Cassandra 2.0.13 (we had the issue with 2.0.10 as well and upgraded to see if
> it goes away)
> Multi datacenter 12+6 nodes cluster.
> h5. Schema
> {code}
> cqlsh> describe keyspace blackbook;
> CREATE KEYSPACE blackbook WITH replication = {
> 'class': 'NetworkTopologyStrategy',
> 'IAD': '3',
> 'ORD': '3'
> };
> USE blackbook;
> CREATE TABLE bounces (
> domainid text,
> address text,
> message text,
> "timestamp" bigint,
> PRIMARY KEY (domainid, address)
> ) WITH
> bloom_filter_fp_chance=0.10 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.10 AND
> gc_grace_seconds=864000 AND
> index_interval=128 AND
> read_repair_chance=0.00 AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='99.0PERCENTILE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'LeveledCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> {code}
> h5. Use case
> Each row (defined by a domainid) can have many many columns (bounce entries)
> so rows can get pretty wide. In practice, most of the rows are not that big
> but some of them contain hundreds of thousands and even millions of columns.
> Columns are not TTL'ed but can be deleted using the following CQL3 statement:
> {code}
> delete from bounces where domainid = 'domain.com' and address =
> 'al...@example.com';
> {code}
> All queries are performed using LOCAL_QUORUM CL.
> h5. Problem
> We weren't very diligent about running repairs on the cluster initially, but
> shorty after we started doing it we noticed that some of previously deleted
> columns (bounce entries) are there again, as if tombstones have disappeared.
> I have run this test multiple times via cqlsh, on the row of the customer who
> originally reported the issue:
> * delete an entry
> * verify it's not returned even with CL=ALL
> * run repair on nodes that own this row's key
> * the columns reappear and are returned even with CL=ALL
> I tried the same test on another row with much less data and everything was
> correctly deleted and didn't reappear after repair.
> h5. Other steps I've taken so far
> Made sure NTP is running on all servers and clocks are synchronized.
> Increased gc_grace_seconds to 100 days, ran full repair (on the affected
> keyspace) on all nodes, then changed it back to the default 10 days again.
> Didn't help.
> Performed one more test. Updated one of the resurrected columns, then deleted
> it and ran repair again. This time the updated version of the column
> reappeared.
> Finally, I noticed these log entries for the row in question:
> {code}
> INFO [ValidationExecutor:77] 2015-03-25 20:27:43,936
> CompactionController.java (line 192) Compacting large row
> blackbook/bounces:4ed558feba8a483733001d6a (279067683 bytes) incrementally
> {code}
> Figuring it may be related I bumped "in_memory_compaction_limit_in_mb" to
> 512MB so the row fits into it, deleted the entry and ran repair once again.
> The log entry for this row was gone and the columns didn't reappear.
> We have a lot of rows much larger than 512MB so can't increase this
> parameters forever, if that is the issue.
> Please let me know if you need more information on the case or if I can run
> more experiments.
> Thanks!
> Roman
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8979) MerkleTree mismatch for deleted and non-existing rows
[ https://issues.apache.org/jira/browse/CASSANDRA-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388836#comment-14388836 ] Yuki Morishita commented on CASSANDRA-8979: --- Committed, thanks! > MerkleTree mismatch for deleted and non-existing rows > - > > Key: CASSANDRA-8979 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8979 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Stefan Podkowinski >Assignee: Stefan Podkowinski > Fix For: 2.1.4, 2.0.14 > > Attachments: 8979-AvoidBufferAllocation-2.0_patch.txt, > cassandra-2.0-8979-lazyrow_patch.txt, cassandra-2.0-8979-validator_patch.txt, > cassandra-2.0-8979-validatortest_patch.txt, > cassandra-2.1-8979-lazyrow_patch.txt, cassandra-2.1-8979-validator_patch.txt > > > Validation compaction will currently create different hashes for rows that > have been deleted compared to nodes that have not seen the rows at all or > have already compacted them away. > In case this sounds familiar to you, see CASSANDRA-4905 which was supposed to > prevent hashing of expired tombstones. This still seems to be in place, but > does not address the issue completely. Or there was a change in 2.0 that > rendered the patch ineffective. > The problem is that rowHash() in the Validator will return a new hash in any > case, whether the PrecompactedRow did actually update the digest or not. This > will lead to the case that a purged, PrecompactedRow will not change the > digest, but we end up with a different tree compared to not having rowHash > called at all (such as in case the row already doesn't exist). > As an implication, repair jobs will constantly detect mismatches between > older sstables containing purgable rows and nodes that have already compacted > these rows. After transfering the reported ranges, the newly created sstables > will immediately get deleted again during the following compaction. This will > happen for each repair run over again until the sstable with the purgable row > finally gets compacted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[3/6] cassandra git commit: remove unnecessary BB allocatetion; follow up for CASSANDRA-8979
remove unnecessary BB allocatetion; follow up for CASSANDRA-8979
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/74bfa773
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/74bfa773
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/74bfa773
Branch: refs/heads/trunk
Commit: 74bfa773a299f2f0c825096c9c6ec521a5073e69
Parents: ec958f0
Author: Oleg Anastasyev
Authored: Tue Mar 31 11:55:37 2015 -0500
Committer: Yuki Morishita
Committed: Tue Mar 31 11:57:14 2015 -0500
--
.../db/compaction/PrecompactedRow.java | 20 ++--
1 file changed, 10 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/74bfa773/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
b/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
index 4627fa2..f41e073 100644
--- a/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
+++ b/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
@@ -157,20 +157,20 @@ public class PrecompactedRow extends AbstractCompactedRow
if (compactedCf == null)
return;
-DataOutputBuffer buffer = new DataOutputBuffer();
-try
+// do not update digest in case of missing or purged row level
tombstones, see CASSANDRA-8979
+if (compactedCf.deletionInfo().getTopLevelDeletion() !=
DeletionTime.LIVE)
{
-
DeletionTime.serializer.serialize(compactedCf.deletionInfo().getTopLevelDeletion(),
buffer);
-
-// do not update digest in case of missing or purged row level
tombstones, see CASSANDRA-8979
-if (compactedCf.deletionInfo().getTopLevelDeletion() !=
DeletionTime.LIVE)
+DataOutputBuffer buffer = new DataOutputBuffer();
+try
{
+
DeletionTime.serializer.serialize(compactedCf.deletionInfo().getTopLevelDeletion(),
buffer);
+
digest.update(buffer.getData(), 0, buffer.getLength());
}
-}
-catch (IOException e)
-{
-throw new RuntimeException(e);
+catch (IOException e)
+{
+throw new RuntimeException(e);
+}
}
compactedCf.updateDigest(digest);
}
[2/6] cassandra git commit: remove unnecessary BB allocatetion; follow up for CASSANDRA-8979
remove unnecessary BB allocatetion; follow up for CASSANDRA-8979
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/74bfa773
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/74bfa773
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/74bfa773
Branch: refs/heads/cassandra-2.1
Commit: 74bfa773a299f2f0c825096c9c6ec521a5073e69
Parents: ec958f0
Author: Oleg Anastasyev
Authored: Tue Mar 31 11:55:37 2015 -0500
Committer: Yuki Morishita
Committed: Tue Mar 31 11:57:14 2015 -0500
--
.../db/compaction/PrecompactedRow.java | 20 ++--
1 file changed, 10 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/74bfa773/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
b/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
index 4627fa2..f41e073 100644
--- a/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
+++ b/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
@@ -157,20 +157,20 @@ public class PrecompactedRow extends AbstractCompactedRow
if (compactedCf == null)
return;
-DataOutputBuffer buffer = new DataOutputBuffer();
-try
+// do not update digest in case of missing or purged row level
tombstones, see CASSANDRA-8979
+if (compactedCf.deletionInfo().getTopLevelDeletion() !=
DeletionTime.LIVE)
{
-
DeletionTime.serializer.serialize(compactedCf.deletionInfo().getTopLevelDeletion(),
buffer);
-
-// do not update digest in case of missing or purged row level
tombstones, see CASSANDRA-8979
-if (compactedCf.deletionInfo().getTopLevelDeletion() !=
DeletionTime.LIVE)
+DataOutputBuffer buffer = new DataOutputBuffer();
+try
{
+
DeletionTime.serializer.serialize(compactedCf.deletionInfo().getTopLevelDeletion(),
buffer);
+
digest.update(buffer.getData(), 0, buffer.getLength());
}
-}
-catch (IOException e)
-{
-throw new RuntimeException(e);
+catch (IOException e)
+{
+throw new RuntimeException(e);
+}
}
compactedCf.updateDigest(digest);
}
[5/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c9b83e0a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c9b83e0a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c9b83e0a Branch: refs/heads/cassandra-2.1 Commit: c9b83e0a811635428735a0b9e217cb106e499bea Parents: eed27ac 74bfa77 Author: Yuki Morishita Authored: Tue Mar 31 11:57:31 2015 -0500 Committer: Yuki Morishita Committed: Tue Mar 31 11:57:31 2015 -0500 -- --
[6/6] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/230cca58 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/230cca58 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/230cca58 Branch: refs/heads/trunk Commit: 230cca580aa9a2c8704b41b29c0b6b2343478adc Parents: 4a4ea06 c9b83e0 Author: Yuki Morishita Authored: Tue Mar 31 11:58:47 2015 -0500 Committer: Yuki Morishita Committed: Tue Mar 31 11:58:47 2015 -0500 -- --
[1/6] cassandra git commit: remove unnecessary BB allocatetion; follow up for CASSANDRA-8979
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 ec958f05e -> 74bfa773a
refs/heads/cassandra-2.1 eed27ac3e -> c9b83e0a8
refs/heads/trunk 4a4ea06a6 -> 230cca580
remove unnecessary BB allocatetion; follow up for CASSANDRA-8979
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/74bfa773
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/74bfa773
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/74bfa773
Branch: refs/heads/cassandra-2.0
Commit: 74bfa773a299f2f0c825096c9c6ec521a5073e69
Parents: ec958f0
Author: Oleg Anastasyev
Authored: Tue Mar 31 11:55:37 2015 -0500
Committer: Yuki Morishita
Committed: Tue Mar 31 11:57:14 2015 -0500
--
.../db/compaction/PrecompactedRow.java | 20 ++--
1 file changed, 10 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/74bfa773/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
b/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
index 4627fa2..f41e073 100644
--- a/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
+++ b/src/java/org/apache/cassandra/db/compaction/PrecompactedRow.java
@@ -157,20 +157,20 @@ public class PrecompactedRow extends AbstractCompactedRow
if (compactedCf == null)
return;
-DataOutputBuffer buffer = new DataOutputBuffer();
-try
+// do not update digest in case of missing or purged row level
tombstones, see CASSANDRA-8979
+if (compactedCf.deletionInfo().getTopLevelDeletion() !=
DeletionTime.LIVE)
{
-
DeletionTime.serializer.serialize(compactedCf.deletionInfo().getTopLevelDeletion(),
buffer);
-
-// do not update digest in case of missing or purged row level
tombstones, see CASSANDRA-8979
-if (compactedCf.deletionInfo().getTopLevelDeletion() !=
DeletionTime.LIVE)
+DataOutputBuffer buffer = new DataOutputBuffer();
+try
{
+
DeletionTime.serializer.serialize(compactedCf.deletionInfo().getTopLevelDeletion(),
buffer);
+
digest.update(buffer.getData(), 0, buffer.getLength());
}
-}
-catch (IOException e)
-{
-throw new RuntimeException(e);
+catch (IOException e)
+{
+throw new RuntimeException(e);
+}
}
compactedCf.updateDigest(digest);
}
[4/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c9b83e0a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c9b83e0a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c9b83e0a Branch: refs/heads/trunk Commit: c9b83e0a811635428735a0b9e217cb106e499bea Parents: eed27ac 74bfa77 Author: Yuki Morishita Authored: Tue Mar 31 11:57:31 2015 -0500 Committer: Yuki Morishita Committed: Tue Mar 31 11:57:31 2015 -0500 -- --
[jira] [Updated] (CASSANDRA-8238) NPE in SizeTieredCompactionStrategy.filterColdSSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-8238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-8238:
---
Fix Version/s: 2.0.14
I've backported this to 2.0 as {{ec958f05e49}}.
> NPE in SizeTieredCompactionStrategy.filterColdSSTables
> --
>
> Key: CASSANDRA-8238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8238
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Fix For: 2.1.4, 2.0.14
>
> Attachments: 0001-assert-that-readMeter-is-not-null.patch,
> 0001-dont-always-set-client-mode-for-sstable-loader.patch
>
>
> {noformat}
> ERROR [CompactionExecutor:15] 2014-10-31 15:28:32,318
> CassandraDaemon.java:153 - Exception in thread
> Thread[CompactionExecutor:15,1,main]
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.filterColdSSTables(SizeTieredCompactionStrategy.java:181)
> ~[apache-cassandra-2.1.1.jar:2.1.1]
> at
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.getNextBackgroundSSTables(SizeTieredCompactionStrategy.java:83)
> ~[apache-cassandra-2.1.1.jar:2.1.1]
> at
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.getNextBackgroundTask(SizeTieredCompactionStrategy.java:267)
> ~[apache-cassandra-2.1.1.jar:2.1.1]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:226)
> ~[apache-cassandra-2.1.1.jar:2.1.1]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_72]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_72]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_72]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_72]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_72]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eed27ac3 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eed27ac3 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eed27ac3 Branch: refs/heads/trunk Commit: eed27ac3e58491f1e8e852059f3b66d95296881c Parents: aaee407 ec958f0 Author: Tyler Hobbs Authored: Tue Mar 31 11:55:49 2015 -0500 Committer: Tyler Hobbs Committed: Tue Mar 31 11:55:49 2015 -0500 -- --
cassandra git commit: Don't set clientMode when bulk-loading
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 88b2f3832 -> ec958f05e
Don't set clientMode when bulk-loading
Backport of CASSANDRA-8238 to 2.0 by Tyler Hobbs
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ec958f05
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ec958f05
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ec958f05
Branch: refs/heads/cassandra-2.0
Commit: ec958f05e493d6b59ada4e71572466baeb7b1467
Parents: 88b2f38
Author: Tyler Hobbs
Authored: Tue Mar 31 11:54:54 2015 -0500
Committer: Tyler Hobbs
Committed: Tue Mar 31 11:54:54 2015 -0500
--
CHANGES.txt | 2 ++
.../org/apache/cassandra/config/YamlConfigurationLoader.java| 3 ++-
src/java/org/apache/cassandra/io/sstable/SSTableLoader.java | 5 -
src/java/org/apache/cassandra/tools/BulkLoader.java | 1 +
4 files changed, 5 insertions(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cdd2c77..a956eb6 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.0.14:
+ * Don't set clientMode to true when bulk-loading sstables to avoid
+ a NullPointerException (CASSANDRA-8238)
* Fix ignored index_interval change in ALTER TABLE statements (CASSANDRA-7976)
* Do more aggressive compaction in old time windows in DTCS (CASSANDRA-8360)
* java.lang.AssertionError when reading saved cache (CASSANDRA-8740)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
--
diff --git a/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
b/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
index b520d07..fb82639 100644
--- a/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
+++ b/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
@@ -61,7 +61,8 @@ public class YamlConfigurationLoader implements
ConfigurationLoader
ClassLoader loader = DatabaseDescriptor.class.getClassLoader();
url = loader.getResource(configUrl);
if (url == null)
-throw new ConfigurationException("Cannot locate " + configUrl);
+throw new ConfigurationException("Cannot locate " + configUrl
+ ". If you are executing this from an " +
+"external tool, it needs to set
Config.setClientMode(true) to avoid loading configuration.\"");
}
return url;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
index 85dc0e4..e32b3cf 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
@@ -53,11 +53,6 @@ public class SSTableLoader implements StreamEventHandler
private final List sstables = new ArrayList<>();
private final Multimap streamingDetails =
HashMultimap.create();
-static
-{
-Config.setClientMode(true);
-}
-
public SSTableLoader(File directory, Client client, OutputHandler
outputHandler)
{
this.directory = directory;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/src/java/org/apache/cassandra/tools/BulkLoader.java
--
diff --git a/src/java/org/apache/cassandra/tools/BulkLoader.java
b/src/java/org/apache/cassandra/tools/BulkLoader.java
index 8e9cfb3..685ebf7 100644
--- a/src/java/org/apache/cassandra/tools/BulkLoader.java
+++ b/src/java/org/apache/cassandra/tools/BulkLoader.java
@@ -74,6 +74,7 @@ public class BulkLoader
public static void main(String args[])
{
+Config.setClientMode(true);
LoaderOptions options = LoaderOptions.parseArgs(args);
OutputHandler handler = new
OutputHandler.SystemOutput(options.verbose, options.debug);
SSTableLoader loader = new SSTableLoader(options.directory, new
ExternalClient(options.hosts,
[1/3] cassandra git commit: Don't set clientMode when bulk-loading
Repository: cassandra
Updated Branches:
refs/heads/trunk acf427e4b -> 4a4ea06a6
Don't set clientMode when bulk-loading
Backport of CASSANDRA-8238 to 2.0 by Tyler Hobbs
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ec958f05
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ec958f05
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ec958f05
Branch: refs/heads/trunk
Commit: ec958f05e493d6b59ada4e71572466baeb7b1467
Parents: 88b2f38
Author: Tyler Hobbs
Authored: Tue Mar 31 11:54:54 2015 -0500
Committer: Tyler Hobbs
Committed: Tue Mar 31 11:54:54 2015 -0500
--
CHANGES.txt | 2 ++
.../org/apache/cassandra/config/YamlConfigurationLoader.java| 3 ++-
src/java/org/apache/cassandra/io/sstable/SSTableLoader.java | 5 -
src/java/org/apache/cassandra/tools/BulkLoader.java | 1 +
4 files changed, 5 insertions(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cdd2c77..a956eb6 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.0.14:
+ * Don't set clientMode to true when bulk-loading sstables to avoid
+ a NullPointerException (CASSANDRA-8238)
* Fix ignored index_interval change in ALTER TABLE statements (CASSANDRA-7976)
* Do more aggressive compaction in old time windows in DTCS (CASSANDRA-8360)
* java.lang.AssertionError when reading saved cache (CASSANDRA-8740)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
--
diff --git a/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
b/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
index b520d07..fb82639 100644
--- a/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
+++ b/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
@@ -61,7 +61,8 @@ public class YamlConfigurationLoader implements
ConfigurationLoader
ClassLoader loader = DatabaseDescriptor.class.getClassLoader();
url = loader.getResource(configUrl);
if (url == null)
-throw new ConfigurationException("Cannot locate " + configUrl);
+throw new ConfigurationException("Cannot locate " + configUrl
+ ". If you are executing this from an " +
+"external tool, it needs to set
Config.setClientMode(true) to avoid loading configuration.\"");
}
return url;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
index 85dc0e4..e32b3cf 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
@@ -53,11 +53,6 @@ public class SSTableLoader implements StreamEventHandler
private final List sstables = new ArrayList<>();
private final Multimap streamingDetails =
HashMultimap.create();
-static
-{
-Config.setClientMode(true);
-}
-
public SSTableLoader(File directory, Client client, OutputHandler
outputHandler)
{
this.directory = directory;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/src/java/org/apache/cassandra/tools/BulkLoader.java
--
diff --git a/src/java/org/apache/cassandra/tools/BulkLoader.java
b/src/java/org/apache/cassandra/tools/BulkLoader.java
index 8e9cfb3..685ebf7 100644
--- a/src/java/org/apache/cassandra/tools/BulkLoader.java
+++ b/src/java/org/apache/cassandra/tools/BulkLoader.java
@@ -74,6 +74,7 @@ public class BulkLoader
public static void main(String args[])
{
+Config.setClientMode(true);
LoaderOptions options = LoaderOptions.parseArgs(args);
OutputHandler handler = new
OutputHandler.SystemOutput(options.verbose, options.debug);
SSTableLoader loader = new SSTableLoader(options.directory, new
ExternalClient(options.hosts,
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4a4ea06a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4a4ea06a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4a4ea06a Branch: refs/heads/trunk Commit: 4a4ea06a6375bf1b6387476449480b0ecff147af Parents: acf427e eed27ac Author: Tyler Hobbs Authored: Tue Mar 31 11:56:13 2015 -0500 Committer: Tyler Hobbs Committed: Tue Mar 31 11:56:13 2015 -0500 -- --
[1/2] cassandra git commit: Don't set clientMode when bulk-loading
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 aaee407dd -> eed27ac3e
Don't set clientMode when bulk-loading
Backport of CASSANDRA-8238 to 2.0 by Tyler Hobbs
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ec958f05
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ec958f05
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ec958f05
Branch: refs/heads/cassandra-2.1
Commit: ec958f05e493d6b59ada4e71572466baeb7b1467
Parents: 88b2f38
Author: Tyler Hobbs
Authored: Tue Mar 31 11:54:54 2015 -0500
Committer: Tyler Hobbs
Committed: Tue Mar 31 11:54:54 2015 -0500
--
CHANGES.txt | 2 ++
.../org/apache/cassandra/config/YamlConfigurationLoader.java| 3 ++-
src/java/org/apache/cassandra/io/sstable/SSTableLoader.java | 5 -
src/java/org/apache/cassandra/tools/BulkLoader.java | 1 +
4 files changed, 5 insertions(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index cdd2c77..a956eb6 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,6 @@
2.0.14:
+ * Don't set clientMode to true when bulk-loading sstables to avoid
+ a NullPointerException (CASSANDRA-8238)
* Fix ignored index_interval change in ALTER TABLE statements (CASSANDRA-7976)
* Do more aggressive compaction in old time windows in DTCS (CASSANDRA-8360)
* java.lang.AssertionError when reading saved cache (CASSANDRA-8740)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
--
diff --git a/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
b/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
index b520d07..fb82639 100644
--- a/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
+++ b/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java
@@ -61,7 +61,8 @@ public class YamlConfigurationLoader implements
ConfigurationLoader
ClassLoader loader = DatabaseDescriptor.class.getClassLoader();
url = loader.getResource(configUrl);
if (url == null)
-throw new ConfigurationException("Cannot locate " + configUrl);
+throw new ConfigurationException("Cannot locate " + configUrl
+ ". If you are executing this from an " +
+"external tool, it needs to set
Config.setClientMode(true) to avoid loading configuration.\"");
}
return url;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
index 85dc0e4..e32b3cf 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
@@ -53,11 +53,6 @@ public class SSTableLoader implements StreamEventHandler
private final List sstables = new ArrayList<>();
private final Multimap streamingDetails =
HashMultimap.create();
-static
-{
-Config.setClientMode(true);
-}
-
public SSTableLoader(File directory, Client client, OutputHandler
outputHandler)
{
this.directory = directory;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ec958f05/src/java/org/apache/cassandra/tools/BulkLoader.java
--
diff --git a/src/java/org/apache/cassandra/tools/BulkLoader.java
b/src/java/org/apache/cassandra/tools/BulkLoader.java
index 8e9cfb3..685ebf7 100644
--- a/src/java/org/apache/cassandra/tools/BulkLoader.java
+++ b/src/java/org/apache/cassandra/tools/BulkLoader.java
@@ -74,6 +74,7 @@ public class BulkLoader
public static void main(String args[])
{
+Config.setClientMode(true);
LoaderOptions options = LoaderOptions.parseArgs(args);
OutputHandler handler = new
OutputHandler.SystemOutput(options.verbose, options.debug);
SSTableLoader loader = new SSTableLoader(options.directory, new
ExternalClient(options.hosts,
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/eed27ac3 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/eed27ac3 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/eed27ac3 Branch: refs/heads/cassandra-2.1 Commit: eed27ac3e58491f1e8e852059f3b66d95296881c Parents: aaee407 ec958f0 Author: Tyler Hobbs Authored: Tue Mar 31 11:55:49 2015 -0500 Committer: Tyler Hobbs Committed: Tue Mar 31 11:55:49 2015 -0500 -- --
[jira] [Commented] (CASSANDRA-6363) CAS not applied on rows containing an expired ttl column
[
https://issues.apache.org/jira/browse/CASSANDRA-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388814#comment-14388814
]
Tyler Hobbs commented on CASSANDRA-6363:
Thanks for writing the test -- I've committed that to dtest master.
> CAS not applied on rows containing an expired ttl column
>
>
> Key: CASSANDRA-6363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6363
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Linux/x64 2.0.2 4-node cluster
>Reporter: Michał Ziemski
>Assignee: Stefania
>
> CREATE TABLE session (
> id text,
> usr text,
> valid int,
> PRIMARY KEY (id)
> );
> insert into session (id, usr) values ('abc', 'abc');
> update session using ttl 1 set valid = 1 where id = 'abc';
> (wait 1 sec)
> And
> delete from session where id = 'DSYUCTCLSOEKVLAQWNWYLVQMEQGGXD' if usr
> ='demo';
> Yields:
> [applied] | usr
> ---+-
> False | abc
> Rather than applying the delete.
> Executing:
> update session set valid = null where id = 'abc';
> and again
> delete from session where id = 'DSYUCTCLSOEKVLAQWNWYLVQMEQGGXD' if usr
> ='demo';
> Positively deletes the row.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7976) Changes to index_interval table properties revert after subsequent modifications
[
https://issues.apache.org/jira/browse/CASSANDRA-7976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-7976:
---
Attachment: 7976-2.0.txt
> Changes to index_interval table properties revert after subsequent
> modifications
>
>
> Key: CASSANDRA-7976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7976
> Project: Cassandra
> Issue Type: Bug
> Components: Config
> Environment: cqlsh 4.1.1, Cassandra 2.0.9-SNAPSHOT (built w/ `ccm` on
> Mac OS X 10.9.4 with Java 1.7.0_67 - more detail below)
> $ java -version
> java version "1.7.0_67"
> Java(TM) SE Runtime Environment (build 1.7.0_67-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
> $ mvn --version
> Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4;
> 2014-08-11T13:58:10-07:00)
> Maven home: /usr/local/Cellar/maven/3.2.3/libexec
> Java version: 1.7.0_67, vendor: Oracle Corporation
> Java home: /Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.9.4", arch: "x86_64", family: "mac"
>Reporter: Andrew Lenards
>Assignee: Stefania
> Labels: cql3, metadata
> Fix For: 2.0.14
>
> Attachments: 7976-2.0.txt
>
>
> It appears that if you want to increase the sampling in *-Summary.db files,
> you would change the default for {{index_interval}} table property from the
> {{128}} default value to {{256}} on a given CQL {{TABLE}}.
> However, if you {{ALTER TABLE}} after setting the value, {{index_interval}}
> returns to the default, {{128}}. This is unexpected behavior. I would expect
> the value for {{index_interval}} to not be affected by subsequent {{ALTER
> TABLE}} statements.
> As noted in Environment, this was seen with a 2.0.9-SNAPSHOT built w/ `ccm`.
> If I just use a table from one of DataStax documentation tutorials (musicdb
> as mdb):
> {noformat}
> cqlsh:mdb> DESC TABLE songs;
> CREATE TABLE songs (
> id uuid,
> album text,
> artist text,
> data blob,
> reviews list,
> tags set,
> title text,
> venue map,
> PRIMARY KEY ((id))
> ) WITH
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.10 AND
> gc_grace_seconds=864000 AND
> index_interval=128 AND
> read_repair_chance=0.00 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='99.0PERCENTILE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> {noformat}
> We've got {{128}} as expected.
> We alter it:
> {noformat}
> cqlsh:mdb> ALTER TABLE songs WITH index_interval = 256;
> {noformat}
> And the change appears:
> {noformat}
> cqlsh:mdb> DESC TABLE songs;
> CREATE TABLE songs (
> id uuid,
> album text,
> artist text,
> data blob,
> reviews list,
> tags set,
> title text,
> venue map,
> PRIMARY KEY ((id))
> ) WITH
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.10 AND
> gc_grace_seconds=864000 AND
> index_interval=256 AND
> read_repair_chance=0.00 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='99.0PERCENTILE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> {noformat}
> But if do another {{ALTER TABLE}}, say, change the caching or comment, the
> {{index_interval}} will revert back to {{128}}.
> {noformat}
> cqlsh:mdb> ALTER TABLE songs WITH caching = 'none';
> cqlsh:mdb> DESC TABLE songs;
> CREATE TABLE songs (
> id uuid,
> album text,
> artist text,
> data blob,
> reviews list,
> tags set,
> title text,
> venue map,
> PRIMARY KEY ((id))
> ) WITH
> bloom_filter_fp_chance=0.01 AND
> caching='NONE' AND
> comment='' AND
> dclocal_read_repair_chance=0.10 AND
> gc_grace_seconds=864000 AND
> index_interval=128 AND
> read_repair_chance=0.00 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> default_time_to_live=0 AND
> speculative_retry='99.0PERCENTILE' AND
> memtable_flush_period_in_ms=0 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> {noformat}
> It should be {{index_interval=256}}.
> I know that 2.1 will replace {{index_interval}}.
> I have not confirmed any behavior with {{min_index_interval}} nor
> {{max_index_interval}} (wh
[1/3] cassandra git commit: Update indexInterval in CFMetadata.apply()
Repository: cassandra
Updated Branches:
refs/heads/trunk 8b3221a8f -> acf427e4b
Update indexInterval in CFMetadata.apply()
Patch by Stefania Alborghetti; reviewed by Tyler Hobbs for
CASSANDRA-7976
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/88b2f383
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/88b2f383
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/88b2f383
Branch: refs/heads/trunk
Commit: 88b2f383268467cb1756edad0ec50c5bead802a4
Parents: f7116c9
Author: Stefania Alborghetti
Authored: Tue Mar 31 11:17:15 2015 -0500
Committer: Tyler Hobbs
Committed: Tue Mar 31 11:17:15 2015 -0500
--
CHANGES.txt | 1 +
.../org/apache/cassandra/config/CFMetaData.java | 1 +
.../apache/cassandra/cql3/AlterTableTest.java | 75
3 files changed, 77 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/88b2f383/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0bcc5cb..cdd2c77 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.14:
+ * Fix ignored index_interval change in ALTER TABLE statements (CASSANDRA-7976)
* Do more aggressive compaction in old time windows in DTCS (CASSANDRA-8360)
* java.lang.AssertionError when reading saved cache (CASSANDRA-8740)
* "disk full" when running cleanup (CASSANDRA-9036)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/88b2f383/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 04a5b01..056b305 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -,6 +,7 @@ public final class CFMetaData
memtableFlushPeriod = cfm.memtableFlushPeriod;
caching = cfm.caching;
defaultTimeToLive = cfm.defaultTimeToLive;
+indexInterval = cfm.indexInterval;
speculativeRetry = cfm.speculativeRetry;
populateIoCacheOnFlush = cfm.populateIoCacheOnFlush;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/88b2f383/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
b/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
new file mode 100644
index 000..099da54
--- /dev/null
+++ b/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.cql3;
+
+import org.apache.cassandra.SchemaLoader;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.ConsistencyLevel;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.gms.Gossiper;
+import org.apache.cassandra.service.ClientState;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import static org.apache.cassandra.cql3.QueryProcessor.process;
+import static org.junit.Assert.assertEquals;
+
+public class AlterTableTest
+{
+private static final String KEYSPACE = "alter_table_test";
+static ClientState clientState;
+
+@BeforeClass
+public static void setUpClass() throws Throwable
+{
+SchemaLoader.loadSchema();
+executeSchemaChange("CREATE KEYSPACE IF NOT EXISTS %s WITH replication
= {'class': 'SimpleStrategy', 'replication_factor': '1'}");
+clientState = ClientState.forInternalCalls();
+}
+
+@AfterClass
+public static void stopGossiper()
+{
+Gossiper.instance.stop();
+}
+
+private static void executeSchemaChange(String query) throws Throwable
+{
+try
+{
+process(String.format(query, KEYSPACE), ConsistencyLevel.ONE);
+} catc
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk
Conflicts:
test/unit/org/apache/cassandra/cql3/CQLTester.java
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/acf427e4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/acf427e4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/acf427e4
Branch: refs/heads/trunk
Commit: acf427e4b140d4322917d782e951bcab8b35576d
Parents: 8b3221a aaee407
Author: Tyler Hobbs
Authored: Tue Mar 31 11:36:29 2015 -0500
Committer: Tyler Hobbs
Committed: Tue Mar 31 11:36:29 2015 -0500
--
CHANGES.txt | 1 +
.../apache/cassandra/cql3/AlterTableTest.java | 20
.../org/apache/cassandra/cql3/CQLTester.java| 3 ++-
3 files changed, 23 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/acf427e4/CHANGES.txt
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/acf427e4/test/unit/org/apache/cassandra/cql3/CQLTester.java
--
diff --cc test/unit/org/apache/cassandra/cql3/CQLTester.java
index 0fe323e,e994f01..98a696c
--- a/test/unit/org/apache/cassandra/cql3/CQLTester.java
+++ b/test/unit/org/apache/cassandra/cql3/CQLTester.java
@@@ -339,43 -194,14 +339,44 @@@ public abstract class CQLTeste
return typeName;
}
+protected String createFunction(String keyspace, String argTypes, String
query) throws Throwable
+{
+String functionName = keyspace + ".function_" +
seqNumber.getAndIncrement();
+createFunctionOverload(functionName, argTypes, query);
+return functionName;
+}
+
+protected void createFunctionOverload(String functionName, String
argTypes, String query) throws Throwable
+{
+String fullQuery = String.format(query, functionName);
+functions.add(functionName + '(' + argTypes + ')');
+logger.info(fullQuery);
+schemaChange(fullQuery);
+}
+
+protected String createAggregate(String keyspace, String argTypes, String
query) throws Throwable
+{
+String aggregateName = keyspace + "." + "aggregate_" +
seqNumber.getAndIncrement();
+createAggregateOverload(aggregateName, argTypes, query);
+return aggregateName;
+}
+
+protected void createAggregateOverload(String aggregateName, String
argTypes, String query) throws Throwable
+{
+String fullQuery = String.format(query, aggregateName);
+aggregates.add(aggregateName + '(' + argTypes + ')');
+logger.info(fullQuery);
+schemaChange(fullQuery);
+}
+
- protected void createTable(String query)
+ protected String createTable(String query)
{
String currentTable = "table_" + seqNumber.getAndIncrement();
tables.add(currentTable);
-String fullQuery = String.format(query, KEYSPACE + "." +
currentTable);
+String fullQuery = formatQuery(query);
logger.info(fullQuery);
schemaChange(fullQuery);
+ return currentTable;
}
protected void createTableMayThrow(String query) throws Throwable
[2/3] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt test/unit/org/apache/cassandra/cql3/AlterTableTest.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aaee407d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aaee407d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aaee407d Branch: refs/heads/trunk Commit: aaee407ddce83cfccb4bd83f6633b7ad2ceb0998 Parents: 963dd54 88b2f38 Author: Tyler Hobbs Authored: Tue Mar 31 11:26:23 2015 -0500 Committer: Tyler Hobbs Committed: Tue Mar 31 11:26:23 2015 -0500 -- CHANGES.txt | 1 + .../apache/cassandra/cql3/AlterTableTest.java | 20 .../org/apache/cassandra/cql3/CQLTester.java| 3 ++- 3 files changed, 23 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/aaee407d/CHANGES.txt -- diff --cc CHANGES.txt index e22c6b3,cdd2c77..7ded07b --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,69 -1,5 +1,70 @@@ -2.0.14: +2.1.4 + * Buffer bloom filter serialization (CASSANDRA-9066) + * Fix anti-compaction target bloom filter size (CASSANDRA-9060) + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047) + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034) + * Avoid overwriting index summaries for sstables with an older format that + does not support downsampling; rebuild summaries on startup when this + is detected (CASSANDRA-8993) + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949) + * Make PasswordAuthenticator number of hashing rounds configurable (CASSANDRA-8085) + * Fix AssertionError when binding nested collections in DELETE (CASSANDRA-8900) + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739) + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914) + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225) + * IndexSummary effectiveIndexInterval is now a guideline, not a rule (CASSANDRA-8993) + * Use correct bounds for page cache eviction of compressed files (CASSANDRA-8746) + * SSTableScanner enforces its bounds (CASSANDRA-8946) + * Cleanup cell equality (CASSANDRA-8947) + * Introduce intra-cluster message coalescing (CASSANDRA-8692) + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839) + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841) + * Don't set clientMode in SSTableLoader (CASSANDRA-8238) + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535) + * Allow invalidating permissions and cache time (CASSANDRA-8722) + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0 + are executed (CASSANDRA-8418) + * Fix cassandra-stress so it respects the CL passed in user mode (CASSANDRA-8948) + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786) + * cassandra-stress reports per-operation statistics, plus misc (CASSANDRA-8769) + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523) + * Use long for key count in cfstats (CASSANDRA-8913) + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832) + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860) + * Make EstimatedHistogram#percentile() use ceil instead of floor (CASSANDRA-8883) + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834) + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067) + * Pick sstables for validation as late as possible inc repairs (CASSANDRA-8366) + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856) + * Fix parallelism adjustment in range and secondary index queries + when the first fetch does not satisfy the limit (CASSANDRA-8856) + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843) + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842) + * Fix CommitLog.forceRecycleAllSegments() memory access error (CASSANDRA-8812) + * Improve assertions in Memory (CASSANDRA-8792) + * Fix SSTableRewriter cleanup (CASSANDRA-8802) + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758) + * 'nodetool info' prints exception against older node (CASSANDRA-8796) + * Ensure SSTableReader.last corresponds exactly with the file end (CASSANDRA-8750) + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747) + * Enforce SSTableReader.first/last (CASSANDRA-8744) + * Cleanup SegmentedFile API (CASSANDRA-8749) + * Avoid overlap with early compaction replacement (CASSANDRA-8683) + * Safer Resource Management++ (CASSANDRA-8707) + * Write partition size estimates into a system table (CASSANDRA-7688) + * cqlsh: Fix keys
cassandra git commit: Optimise max purgeable timestamp calculation in compaction
Repository: cassandra
Updated Branches:
refs/heads/trunk 16499ca9b -> 8b3221a8f
Optimise max purgeable timestamp calculation in compaction
patch by benedict; reviewed by marcus for CASSANDRA-8920
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8b3221a8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8b3221a8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8b3221a8
Branch: refs/heads/trunk
Commit: 8b3221a8fcdcafc78e4cee908768b9f8612df31e
Parents: 16499ca
Author: Benedict Elliott Smith
Authored: Tue Mar 31 17:32:18 2015 +0100
Committer: Benedict Elliott Smith
Committed: Tue Mar 31 17:32:18 2015 +0100
--
CHANGES.txt | 1 +
.../org/apache/cassandra/db/DataTracker.java| 7 +-
.../db/compaction/CompactionController.java | 21 ++--
.../cassandra/utils/AsymmetricOrdering.java | 3 -
.../org/apache/cassandra/utils/Interval.java| 2 +
.../apache/cassandra/utils/OverlapIterator.java | 58 ++
.../cassandra/utils/OverlapIteratorTest.java| 114 +++
7 files changed, 191 insertions(+), 15 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8b3221a8/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 22bdc5e..e51be76 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0
+ * Optimise max purgeable timestamp calculation in compaction (CASSANDRA-8920)
* Constrain internode message buffer sizes, and improve IO class hierarchy
(CASSANDRA-8670)
* New tool added to validate all sstables in a node (CASSANDRA-5791)
* Push notification when tracing completes for an operation (CASSANDRA-7807)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8b3221a8/src/java/org/apache/cassandra/db/DataTracker.java
--
diff --git a/src/java/org/apache/cassandra/db/DataTracker.java
b/src/java/org/apache/cassandra/db/DataTracker.java
index dd1dc5a..6de0b2c 100644
--- a/src/java/org/apache/cassandra/db/DataTracker.java
+++ b/src/java/org/apache/cassandra/db/DataTracker.java
@@ -575,10 +575,15 @@ public class DataTracker
public static SSTableIntervalTree
buildIntervalTree(Iterable sstables)
{
+return new SSTableIntervalTree(buildIntervals(sstables));
+}
+
+public static List>
buildIntervals(Iterable sstables)
+{
List> intervals = new
ArrayList<>(Iterables.size(sstables));
for (SSTableReader sstable : sstables)
intervals.add(Interval.create(sstable.first, sstable.last, sstable));
-return new SSTableIntervalTree(intervals);
+return intervals;
}
public Set getCompacting()
http://git-wip-us.apache.org/repos/asf/cassandra/blob/8b3221a8/src/java/org/apache/cassandra/db/compaction/CompactionController.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
index 148b1b6..a49a3ea 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionController.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionController.java
@@ -17,24 +17,22 @@
*/
package org.apache.cassandra.db.compaction;
-import java.util.ArrayList;
-import java.util.Collections;
-import java.util.HashSet;
-import java.util.Iterator;
-import java.util.List;
-import java.util.Set;
+import java.util.*;
import org.apache.cassandra.io.sstable.format.SSTableReader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.cassandra.db.ColumnFamilyStore;
-import org.apache.cassandra.db.DataTracker;
import org.apache.cassandra.db.DecoratedKey;
+import org.apache.cassandra.db.RowPosition;
import org.apache.cassandra.utils.AlwaysPresentFilter;
+import org.apache.cassandra.utils.OverlapIterator;
import org.apache.cassandra.utils.concurrent.Refs;
+import static org.apache.cassandra.db.DataTracker.buildIntervals;
+
/**
* Manage compaction options.
*/
@@ -43,8 +41,8 @@ public class CompactionController implements AutoCloseable
private static final Logger logger =
LoggerFactory.getLogger(CompactionController.class);
public final ColumnFamilyStore cfs;
-private DataTracker.SSTableIntervalTree overlappingTree;
private Refs overlappingSSTables;
+private OverlapIterator overlapIterator;
private final Iterable compacting;
public final int gcBefore;
@@ -84,7 +82,7 @@ public class CompactionController implements AutoCloseable
overlappingSSTables =
Refs.tryRef(Collections.emptyList());
else
overlappingSS
[3/3] cassandra git commit: Constrain internode message buffer sizes, and improve IO class hierarchy
Constrain internode message buffer sizes, and improve IO class hierarchy patch by ariel and benedict for CASSANDRA-8670 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/16499ca9 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/16499ca9 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/16499ca9 Branch: refs/heads/trunk Commit: 16499ca9b0080ea4d3c4ed3bc55c753bacc3c24e Parents: dbe909e Author: Ariel Weisberg Authored: Tue Mar 31 17:28:15 2015 +0100 Committer: Benedict Elliott Smith Committed: Tue Mar 31 17:28:15 2015 +0100 -- CHANGES.txt | 1 + .../apache/cassandra/cache/AutoSavingCache.java | 4 +- .../org/apache/cassandra/cache/OHCProvider.java | 8 + .../apache/cassandra/db/BatchlogManager.java| 2 +- .../org/apache/cassandra/db/SuperColumns.java | 2 +- .../org/apache/cassandra/db/SystemKeyspace.java | 2 +- .../cassandra/db/commitlog/CommitLog.java | 4 +- .../io/compress/CompressedSequentialWriter.java | 5 +- .../io/sstable/IndexSummaryBuilder.java | 5 +- .../io/sstable/format/SSTableReader.java| 5 +- .../io/sstable/format/big/BigTableWriter.java | 15 +- .../io/sstable/metadata/MetadataSerializer.java | 7 +- .../cassandra/io/util/AbstractDataOutput.java | 329 - .../io/util/BufferedDataOutputStreamPlus.java | 301 + .../cassandra/io/util/ByteBufferDataInput.java | 1 - .../cassandra/io/util/DataOutputBuffer.java | 95 ++- .../cassandra/io/util/DataOutputByteBuffer.java | 59 -- .../cassandra/io/util/DataOutputPlus.java | 14 +- .../io/util/DataOutputStreamAndChannel.java | 55 -- .../cassandra/io/util/DataOutputStreamPlus.java | 111 ++- .../io/util/FastByteArrayOutputStream.java | 266 .../org/apache/cassandra/io/util/Memory.java| 1 + .../cassandra/io/util/NIODataInputStream.java | 312 + .../cassandra/io/util/SafeMemoryWriter.java | 117 +--- .../cassandra/io/util/SequentialWriter.java | 3 +- .../io/util/UnbufferedDataOutputStreamPlus.java | 374 +++ .../io/util/WrappedDataOutputStreamPlus.java| 68 ++ .../cassandra/net/IncomingTcpConnection.java| 9 +- .../cassandra/net/OutboundTcpConnection.java| 10 +- .../apache/cassandra/service/GCInspector.java | 46 +- .../cassandra/service/pager/PagingState.java| 2 +- .../cassandra/streaming/ConnectionHandler.java | 22 +- .../cassandra/streaming/StreamWriter.java | 11 +- .../compress/CompressedStreamWriter.java| 28 +- .../streaming/messages/CompleteMessage.java | 4 +- .../streaming/messages/IncomingFileMessage.java | 4 +- .../streaming/messages/OutgoingFileMessage.java | 7 +- .../streaming/messages/PrepareMessage.java | 4 +- .../streaming/messages/ReceivedMessage.java | 4 +- .../streaming/messages/RetryMessage.java| 4 +- .../messages/SessionFailedMessage.java | 4 +- .../streaming/messages/StreamMessage.java | 6 +- .../cassandra/thrift/CassandraServer.java | 4 + .../org/apache/cassandra/tools/NodeProbe.java | 2 +- .../org/apache/cassandra/tools/NodeTool.java| 4 +- .../org/apache/cassandra/transport/CBUtil.java | 4 +- .../cassandra/utils/memory/MemoryUtil.java | 47 ++ .../utils/vint/EncodedDataOutputStream.java | 4 +- .../cassandra/AbstractSerializationsTester.java | 8 +- .../apache/cassandra/db/SerializationsTest.java | 19 +- .../cassandra/gms/SerializationsTest.java | 6 +- .../cassandra/io/sstable/IndexSummaryTest.java | 4 + .../metadata/MetadataSerializerTest.java| 7 +- .../io/util/BufferedDataOutputStreamTest.java | 391 +++ .../cassandra/io/util/DataOutputTest.java | 50 +- .../io/util/NIODataInputStreamTest.java | 667 +++ .../cassandra/service/SerializationsTest.java | 5 +- .../apache/cassandra/utils/BloomFilterTest.java | 11 +- .../cassandra/utils/SerializationsTest.java | 6 +- 59 files changed, 2605 insertions(+), 965 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/16499ca9/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index beb05ab..22bdc5e 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0 + * Constrain internode message buffer sizes, and improve IO class hierarchy (CASSANDRA-8670) * New tool added to validate all sstables in a node (CASSANDRA-5791) * Push notification when tracing completes for an operation (CASSANDRA-7807) * Delay "node up" and "node added" notifications until native protocol server is started (CASSANDRA-8236) http://git-wip-us.apache.org/repos/asf/cassandra/blob/164
[2/3] cassandra git commit: Constrain internode message buffer sizes, and improve IO class hierarchy
http://git-wip-us.apache.org/repos/asf/cassandra/blob/16499ca9/src/java/org/apache/cassandra/io/util/UnbufferedDataOutputStreamPlus.java
--
diff --git
a/src/java/org/apache/cassandra/io/util/UnbufferedDataOutputStreamPlus.java
b/src/java/org/apache/cassandra/io/util/UnbufferedDataOutputStreamPlus.java
new file mode 100644
index 000..31abfa8
--- /dev/null
+++ b/src/java/org/apache/cassandra/io/util/UnbufferedDataOutputStreamPlus.java
@@ -0,0 +1,374 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.io.util;
+
+import java.io.DataOutput;
+import java.io.IOException;
+import java.io.UTFDataFormatException;
+import java.nio.ByteBuffer;
+import java.nio.channels.WritableByteChannel;
+
+import org.apache.cassandra.config.Config;
+import org.apache.cassandra.utils.memory.MemoryUtil;
+
+import com.google.common.base.Function;
+
+/**
+ * Base class for DataOutput implementations that does not have an optimized
implementations of Plus methods
+ * and does no buffering.
+ *
+ * Unlike BufferedDataOutputStreamPlus this is capable of operating as an
unbuffered output stream.
+ * Currently necessary because SequentialWriter implements its own buffering
along with mark/reset/truncate.
+ */
+public abstract class UnbufferedDataOutputStreamPlus extends
DataOutputStreamPlus
+{
+protected UnbufferedDataOutputStreamPlus()
+{
+super();
+}
+
+protected UnbufferedDataOutputStreamPlus(WritableByteChannel channel)
+{
+super(channel);
+}
+
+/*
+!! DataOutput methods below are copied from the implementation in Apache
Harmony RandomAccessFile.
+*/
+
+/**
+ * Writes the entire contents of the byte array buffer to
+ * this RandomAccessFile starting at the current file pointer.
+ *
+ * @param buffer the buffer to be written.
+ * @throws IOException If an error occurs trying to write to this
RandomAccessFile.
+ */
+public void write(byte[] buffer) throws IOException
+{
+write(buffer, 0, buffer.length);
+}
+
+/**
+ * Writes count bytes from the byte array buffer
+ * starting at offset to this RandomAccessFile starting at
+ * the current file pointer..
+ *
+ * @param buffer the bytes to be written
+ * @param offset offset in buffer to get bytes
+ * @param count number of bytes in buffer to write
+ * @throws IOException If an error occurs attempting to
write to this
+ * RandomAccessFile.
+ * @throws IndexOutOfBoundsException If offset or count are outside of
bounds.
+ */
+public abstract void write(byte[] buffer, int offset, int count) throws
IOException;
+
+/**
+ * Writes the specified byte oneByte to this RandomAccessFile
+ * starting at the current file pointer. Only the low order byte of
+ * oneByte is written.
+ *
+ * @param oneByte the byte to be written
+ * @throws IOException If an error occurs attempting to write to this
+ * RandomAccessFile.
+ */
+public abstract void write(int oneByte) throws IOException;
+
+/**
+ * Writes a boolean to this output stream.
+ *
+ * @param val the boolean value to write to the OutputStream
+ * @throws IOException If an error occurs attempting to write to this
+ * DataOutputStream.
+ */
+public final void writeBoolean(boolean val) throws IOException
+{
+write(val ? 1 : 0);
+}
+
+/**
+ * Writes a 8-bit byte to this output stream.
+ *
+ * @param val the byte value to write to the OutputStream
+ * @throws java.io.IOException If an error occurs attempting to write to
this
+ * DataOutputStream.
+ */
+public final void writeByte(int val) throws IOException
+{
+write(val & 0xFF);
+}
+
+/**
+ * Writes the low order 8-bit bytes from a String to this output stream.
+ *
+ * @param str the String containing the bytes to write to the OutputStream
+ * @throws IOException If an error occurs attempting to write to this
+ *
[1/3] cassandra git commit: Constrain internode message buffer sizes, and improve IO class hierarchy
Repository: cassandra
Updated Branches:
refs/heads/trunk dbe909e06 -> 16499ca9b
http://git-wip-us.apache.org/repos/asf/cassandra/blob/16499ca9/test/unit/org/apache/cassandra/io/util/DataOutputTest.java
--
diff --git a/test/unit/org/apache/cassandra/io/util/DataOutputTest.java
b/test/unit/org/apache/cassandra/io/util/DataOutputTest.java
index 624ca9b..37bff4f 100644
--- a/test/unit/org/apache/cassandra/io/util/DataOutputTest.java
+++ b/test/unit/org/apache/cassandra/io/util/DataOutputTest.java
@@ -36,27 +36,37 @@ import java.util.concurrent.ThreadLocalRandom;
import org.junit.Assert;
import org.junit.Test;
-
import org.apache.cassandra.utils.ByteBufferUtil;
public class DataOutputTest
{
@Test
-public void testDataOutputStreamPlus() throws IOException
+public void testWrappedDataOutputStreamPlus() throws IOException
+{
+ByteArrayOutputStream bos = new ByteArrayOutputStream();
+DataOutputStreamPlus write = new WrappedDataOutputStreamPlus(bos);
+DataInput canon = testWrite(write);
+DataInput test = new DataInputStream(new
ByteArrayInputStream(bos.toByteArray()));
+testRead(test, canon);
+}
+
+@Test
+public void testWrappedDataOutputChannelAndChannel() throws IOException
{
ByteArrayOutputStream bos = new ByteArrayOutputStream();
-DataOutputStreamPlus write = new DataOutputStreamPlus(bos);
+DataOutputStreamPlus write = new WrappedDataOutputStreamPlus(bos);
DataInput canon = testWrite(write);
DataInput test = new DataInputStream(new
ByteArrayInputStream(bos.toByteArray()));
testRead(test, canon);
}
@Test
-public void testDataOutputChannelAndChannel() throws IOException
+public void testBufferedDataOutputStreamPlusAndChannel() throws IOException
{
ByteArrayOutputStream bos = new ByteArrayOutputStream();
-DataOutputStreamPlus write = new
DataOutputStreamAndChannel(Channels.newChannel(bos));
+DataOutputStreamPlus write = new
BufferedDataOutputStreamPlus(Channels.newChannel(bos));
DataInput canon = testWrite(write);
+write.close();
DataInput test = new DataInputStream(new
ByteArrayInputStream(bos.toByteArray()));
testRead(test, canon);
}
@@ -74,7 +84,7 @@ public class DataOutputTest
public void testDataOutputDirectByteBuffer() throws IOException
{
ByteBuffer buf = wrap(new byte[345], true);
-DataOutputByteBuffer write = new DataOutputByteBuffer(buf.duplicate());
+BufferedDataOutputStreamPlus write = new
BufferedDataOutputStreamPlus(null, buf.duplicate());
DataInput canon = testWrite(write);
DataInput test = new DataInputStream(new
ByteArrayInputStream(ByteBufferUtil.getArray(buf)));
testRead(test, canon);
@@ -84,7 +94,7 @@ public class DataOutputTest
public void testDataOutputHeapByteBuffer() throws IOException
{
ByteBuffer buf = wrap(new byte[345], false);
-DataOutputByteBuffer write = new DataOutputByteBuffer(buf.duplicate());
+BufferedDataOutputStreamPlus write = new
BufferedDataOutputStreamPlus(null, buf.duplicate());
DataInput canon = testWrite(write);
DataInput test = new DataInputStream(new
ByteArrayInputStream(ByteBufferUtil.getArray(buf)));
testRead(test, canon);
@@ -102,12 +112,31 @@ public class DataOutputTest
}
@Test
+public void testWrappedFileOutputStream() throws IOException
+{
+File file = FileUtils.createTempFile("dataoutput", "test");
+try
+{
+DataOutputStreamPlus write = new WrappedDataOutputStreamPlus(new
FileOutputStream(file));
+DataInput canon = testWrite(write);
+write.close();
+DataInputStream test = new DataInputStream(new
FileInputStream(file));
+testRead(test, canon);
+test.close();
+}
+finally
+{
+Assert.assertTrue(file.delete());
+}
+}
+
+@Test
public void testFileOutputStream() throws IOException
{
File file = FileUtils.createTempFile("dataoutput", "test");
try
{
-DataOutputStreamAndChannel write = new
DataOutputStreamAndChannel(new FileOutputStream(file));
+DataOutputStreamPlus write = new BufferedDataOutputStreamPlus(new
FileOutputStream(file));
DataInput canon = testWrite(write);
write.close();
DataInputStream test = new DataInputStream(new
FileInputStream(file));
@@ -126,8 +155,9 @@ public class DataOutputTest
File file = FileUtils.createTempFile("dataoutput", "test");
try
{
+@SuppressWarnings("resource")
final RandomAccessFile raf = new RandomAccessFile(file, "rw");
-DataOutputStreamAndChannel write = new
DataOutputStreamAndCha
[1/2] cassandra git commit: Update indexInterval in CFMetadata.apply()
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 963dd54b7 -> aaee407dd
Update indexInterval in CFMetadata.apply()
Patch by Stefania Alborghetti; reviewed by Tyler Hobbs for
CASSANDRA-7976
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/88b2f383
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/88b2f383
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/88b2f383
Branch: refs/heads/cassandra-2.1
Commit: 88b2f383268467cb1756edad0ec50c5bead802a4
Parents: f7116c9
Author: Stefania Alborghetti
Authored: Tue Mar 31 11:17:15 2015 -0500
Committer: Tyler Hobbs
Committed: Tue Mar 31 11:17:15 2015 -0500
--
CHANGES.txt | 1 +
.../org/apache/cassandra/config/CFMetaData.java | 1 +
.../apache/cassandra/cql3/AlterTableTest.java | 75
3 files changed, 77 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/88b2f383/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0bcc5cb..cdd2c77 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.14:
+ * Fix ignored index_interval change in ALTER TABLE statements (CASSANDRA-7976)
* Do more aggressive compaction in old time windows in DTCS (CASSANDRA-8360)
* java.lang.AssertionError when reading saved cache (CASSANDRA-8740)
* "disk full" when running cleanup (CASSANDRA-9036)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/88b2f383/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 04a5b01..056b305 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -,6 +,7 @@ public final class CFMetaData
memtableFlushPeriod = cfm.memtableFlushPeriod;
caching = cfm.caching;
defaultTimeToLive = cfm.defaultTimeToLive;
+indexInterval = cfm.indexInterval;
speculativeRetry = cfm.speculativeRetry;
populateIoCacheOnFlush = cfm.populateIoCacheOnFlush;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/88b2f383/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
b/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
new file mode 100644
index 000..099da54
--- /dev/null
+++ b/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.cql3;
+
+import org.apache.cassandra.SchemaLoader;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.ConsistencyLevel;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.gms.Gossiper;
+import org.apache.cassandra.service.ClientState;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import static org.apache.cassandra.cql3.QueryProcessor.process;
+import static org.junit.Assert.assertEquals;
+
+public class AlterTableTest
+{
+private static final String KEYSPACE = "alter_table_test";
+static ClientState clientState;
+
+@BeforeClass
+public static void setUpClass() throws Throwable
+{
+SchemaLoader.loadSchema();
+executeSchemaChange("CREATE KEYSPACE IF NOT EXISTS %s WITH replication
= {'class': 'SimpleStrategy', 'replication_factor': '1'}");
+clientState = ClientState.forInternalCalls();
+}
+
+@AfterClass
+public static void stopGossiper()
+{
+Gossiper.instance.stop();
+}
+
+private static void executeSchemaChange(String query) throws Throwable
+{
+try
+{
+process(String.format(query, KEYSPACE), ConsistencyLevel.ONE);
[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt test/unit/org/apache/cassandra/cql3/AlterTableTest.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aaee407d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aaee407d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aaee407d Branch: refs/heads/cassandra-2.1 Commit: aaee407ddce83cfccb4bd83f6633b7ad2ceb0998 Parents: 963dd54 88b2f38 Author: Tyler Hobbs Authored: Tue Mar 31 11:26:23 2015 -0500 Committer: Tyler Hobbs Committed: Tue Mar 31 11:26:23 2015 -0500 -- CHANGES.txt | 1 + .../apache/cassandra/cql3/AlterTableTest.java | 20 .../org/apache/cassandra/cql3/CQLTester.java| 3 ++- 3 files changed, 23 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/aaee407d/CHANGES.txt -- diff --cc CHANGES.txt index e22c6b3,cdd2c77..7ded07b --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,69 -1,5 +1,70 @@@ -2.0.14: +2.1.4 + * Buffer bloom filter serialization (CASSANDRA-9066) + * Fix anti-compaction target bloom filter size (CASSANDRA-9060) + * Make FROZEN and TUPLE unreserved keywords in CQL (CASSANDRA-9047) + * Prevent AssertionError from SizeEstimatesRecorder (CASSANDRA-9034) + * Avoid overwriting index summaries for sstables with an older format that + does not support downsampling; rebuild summaries on startup when this + is detected (CASSANDRA-8993) + * Fix potential data loss in CompressedSequentialWriter (CASSANDRA-8949) + * Make PasswordAuthenticator number of hashing rounds configurable (CASSANDRA-8085) + * Fix AssertionError when binding nested collections in DELETE (CASSANDRA-8900) + * Check for overlap with non-early sstables in LCS (CASSANDRA-8739) + * Only calculate max purgable timestamp if we have to (CASSANDRA-8914) + * (cqlsh) Greatly improve performance of COPY FROM (CASSANDRA-8225) + * IndexSummary effectiveIndexInterval is now a guideline, not a rule (CASSANDRA-8993) + * Use correct bounds for page cache eviction of compressed files (CASSANDRA-8746) + * SSTableScanner enforces its bounds (CASSANDRA-8946) + * Cleanup cell equality (CASSANDRA-8947) + * Introduce intra-cluster message coalescing (CASSANDRA-8692) + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839) + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841) + * Don't set clientMode in SSTableLoader (CASSANDRA-8238) + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535) + * Allow invalidating permissions and cache time (CASSANDRA-8722) + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0 + are executed (CASSANDRA-8418) + * Fix cassandra-stress so it respects the CL passed in user mode (CASSANDRA-8948) + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786) + * cassandra-stress reports per-operation statistics, plus misc (CASSANDRA-8769) + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523) + * Use long for key count in cfstats (CASSANDRA-8913) + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832) + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860) + * Make EstimatedHistogram#percentile() use ceil instead of floor (CASSANDRA-8883) + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834) + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067) + * Pick sstables for validation as late as possible inc repairs (CASSANDRA-8366) + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856) + * Fix parallelism adjustment in range and secondary index queries + when the first fetch does not satisfy the limit (CASSANDRA-8856) + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843) + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842) + * Fix CommitLog.forceRecycleAllSegments() memory access error (CASSANDRA-8812) + * Improve assertions in Memory (CASSANDRA-8792) + * Fix SSTableRewriter cleanup (CASSANDRA-8802) + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758) + * 'nodetool info' prints exception against older node (CASSANDRA-8796) + * Ensure SSTableReader.last corresponds exactly with the file end (CASSANDRA-8750) + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747) + * Enforce SSTableReader.first/last (CASSANDRA-8744) + * Cleanup SegmentedFile API (CASSANDRA-8749) + * Avoid overlap with early compaction replacement (CASSANDRA-8683) + * Safer Resource Management++ (CASSANDRA-8707) + * Write partition size estimates into a system table (CASSANDRA-7688) + * cqlsh:
[jira] [Updated] (CASSANDRA-8979) MerkleTree mismatch for deleted and non-existing rows
[ https://issues.apache.org/jira/browse/CASSANDRA-8979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Anastasyev updated CASSANDRA-8979: --- Attachment: 8979-AvoidBufferAllocation-2.0_patch.txt Noticed an unneccessary ByteBuffer allocation in PrecompactedRow in case it does not updates a digest. Attached a patch that avoids it. > MerkleTree mismatch for deleted and non-existing rows > - > > Key: CASSANDRA-8979 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8979 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Stefan Podkowinski >Assignee: Stefan Podkowinski > Fix For: 2.1.4, 2.0.14 > > Attachments: 8979-AvoidBufferAllocation-2.0_patch.txt, > cassandra-2.0-8979-lazyrow_patch.txt, cassandra-2.0-8979-validator_patch.txt, > cassandra-2.0-8979-validatortest_patch.txt, > cassandra-2.1-8979-lazyrow_patch.txt, cassandra-2.1-8979-validator_patch.txt > > > Validation compaction will currently create different hashes for rows that > have been deleted compared to nodes that have not seen the rows at all or > have already compacted them away. > In case this sounds familiar to you, see CASSANDRA-4905 which was supposed to > prevent hashing of expired tombstones. This still seems to be in place, but > does not address the issue completely. Or there was a change in 2.0 that > rendered the patch ineffective. > The problem is that rowHash() in the Validator will return a new hash in any > case, whether the PrecompactedRow did actually update the digest or not. This > will lead to the case that a purged, PrecompactedRow will not change the > digest, but we end up with a different tree compared to not having rowHash > called at all (such as in case the row already doesn't exist). > As an implication, repair jobs will constantly detect mismatches between > older sstables containing purgable rows and nodes that have already compacted > these rows. After transfering the reported ranges, the newly created sstables > will immediately get deleted again during the following compaction. This will > happen for each repair run over again until the sstable with the purgable row > finally gets compacted. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: Update indexInterval in CFMetadata.apply()
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 f7116c91b -> 88b2f3832
Update indexInterval in CFMetadata.apply()
Patch by Stefania Alborghetti; reviewed by Tyler Hobbs for
CASSANDRA-7976
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/88b2f383
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/88b2f383
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/88b2f383
Branch: refs/heads/cassandra-2.0
Commit: 88b2f383268467cb1756edad0ec50c5bead802a4
Parents: f7116c9
Author: Stefania Alborghetti
Authored: Tue Mar 31 11:17:15 2015 -0500
Committer: Tyler Hobbs
Committed: Tue Mar 31 11:17:15 2015 -0500
--
CHANGES.txt | 1 +
.../org/apache/cassandra/config/CFMetaData.java | 1 +
.../apache/cassandra/cql3/AlterTableTest.java | 75
3 files changed, 77 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/88b2f383/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 0bcc5cb..cdd2c77 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.14:
+ * Fix ignored index_interval change in ALTER TABLE statements (CASSANDRA-7976)
* Do more aggressive compaction in old time windows in DTCS (CASSANDRA-8360)
* java.lang.AssertionError when reading saved cache (CASSANDRA-8740)
* "disk full" when running cleanup (CASSANDRA-9036)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/88b2f383/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 04a5b01..056b305 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -,6 +,7 @@ public final class CFMetaData
memtableFlushPeriod = cfm.memtableFlushPeriod;
caching = cfm.caching;
defaultTimeToLive = cfm.defaultTimeToLive;
+indexInterval = cfm.indexInterval;
speculativeRetry = cfm.speculativeRetry;
populateIoCacheOnFlush = cfm.populateIoCacheOnFlush;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/88b2f383/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
b/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
new file mode 100644
index 000..099da54
--- /dev/null
+++ b/test/unit/org/apache/cassandra/cql3/AlterTableTest.java
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.cql3;
+
+import org.apache.cassandra.SchemaLoader;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.ConsistencyLevel;
+import org.apache.cassandra.db.Keyspace;
+import org.apache.cassandra.gms.Gossiper;
+import org.apache.cassandra.service.ClientState;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import static org.apache.cassandra.cql3.QueryProcessor.process;
+import static org.junit.Assert.assertEquals;
+
+public class AlterTableTest
+{
+private static final String KEYSPACE = "alter_table_test";
+static ClientState clientState;
+
+@BeforeClass
+public static void setUpClass() throws Throwable
+{
+SchemaLoader.loadSchema();
+executeSchemaChange("CREATE KEYSPACE IF NOT EXISTS %s WITH replication
= {'class': 'SimpleStrategy', 'replication_factor': '1'}");
+clientState = ClientState.forInternalCalls();
+}
+
+@AfterClass
+public static void stopGossiper()
+{
+Gossiper.instance.stop();
+}
+
+private static void executeSchemaChange(String query) throws Throwable
+{
+try
+{
+process(String.format(query, KEYSPACE), ConsistencyLevel.ONE);
[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage
[
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388757#comment-14388757
]
Ariel Weisberg commented on CASSANDRA-8670:
---
Unit tets pass and I am +1 on your changes.
> Large columns + NIO memory pooling causes excessive direct memory usage
> ---
>
> Key: CASSANDRA-8670
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
> Fix For: 3.0
>
> Attachments: largecolumn_test.py
>
>
> If you provide a large byte array to NIO and ask it to populate the byte
> array from a socket it will allocate a thread local byte buffer that is the
> size of the requested read no matter how large it is. Old IO wraps new IO for
> sockets (but not files) so old IO is effected as well.
> Even If you are using Buffered{Input | Output}Stream you can end up passing a
> large byte array to NIO. The byte array read method will pass the array to
> NIO directly if it is larger than the internal buffer.
> Passing large cells between nodes as part of intra-cluster messaging can
> cause the NIO pooled buffers to quickly reach a high watermark and stay
> there. This ends up costing 2x the largest cell size because there is a
> buffer for input and output since they are different threads. This is further
> multiplied by the number of nodes in the cluster - 1 since each has a
> dedicated thread pair with separate thread locals.
> Anecdotally it appears that the cost is doubled beyond that although it isn't
> clear why. Possibly the control connections or possibly there is some way in
> which multiple
> Need a workload in CI that tests the advertised limits of cells on a cluster.
> It would be reasonable to ratchet down the max direct memory for the test to
> trigger failures if a memory pooling issue is introduced. I don't think we
> need to test concurrently pulling in a lot of them, but it should at least
> work serially.
> The obvious fix to address this issue would be to read in smaller chunks when
> dealing with large values. I think small should still be relatively large (4
> megabytes) so that code that is reading from a disk can amortize the cost of
> a seek. It can be hard to tell what the underlying thing being read from is
> going to be in some of the contexts where we might choose to implement
> switching to reading chunks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8099) Refactor and modernize the storage engine
[
https://issues.apache.org/jira/browse/CASSANDRA-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388753#comment-14388753
]
Sylvain Lebresne commented on CASSANDRA-8099:
-
Thanks for some initial review. I've pushed a commit for most of the remarks. I
answer the rest below:
bq. I wonder if it will not be better to have two sub-classes for
{{PrimaryKeyRestrictionSet}} one for the partition key and one for the
clustering columns rather than having a boolean variable
I don't disagree, but think we should generally clean the handling of partition
keys so it doesn't "pretend" to be clustering columns (which will imply
separating into 2 classes). And so I'd like to do that properly as a followup
since it's not essential to this ticket (and I'm sure we can agree it's big
enough as is).
bq. In {{StatementRestrictions}} I do not understand the use of
{{useFiltering}}. My understanding was that we should return an error message
specifying that {{ALLOW FILTERING}} is required and that this problem should
have been handled by {{checkNeedsFiltering}} in {{SelectStatement}}.
If you have restriction on an indexed column but that restriction is no
"queriable" (not an equality), we actually always reject the query (as in, even
with {{ALLOW FILTERING}}) with an error message that says we can't find a
usable 2ndary index. I'm not saying this is a good thing, it's really just
historical and we should fix it, but this ticket is arguably not the right
place for this (CASSANDRA-4987 would typically be a better place for that).
We also don't even call {{needsFiltering}} if the query is not a range one
(because we don't support {{ALLOW FILTERING}} there yet, which is
CASSANDRA-6377), but we should still reject the queries desribed above
(restriction on an indexed column but not one usable by the index) for single
partition queries.
Another way to put it is that the added validation is just the validation that
is done on trunk in {{SecondaryIndexManager.validateIndexSearchersForQuery}}
(and so was not handled by {{checkNeedsFiltering}}) which I moved in
{{StatementRestrictions}} because that was convenient for the patch. TL;DR, we
should clean all this in follow-ups, but I'd rather keep it simple for this
ticket.
> Refactor and modernize the storage engine
> -
>
> Key: CASSANDRA-8099
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8099
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 3.0
>
> Attachments: 8099-nit
>
>
> The current storage engine (which for this ticket I'll loosely define as "the
> code implementing the read/write path") is suffering from old age. One of the
> main problem is that the only structure it deals with is the cell, which
> completely ignores the more high level CQL structure that groups cell into
> (CQL) rows.
> This leads to many inefficiencies, like the fact that during a reads we have
> to group cells multiple times (to count on replica, then to count on the
> coordinator, then to produce the CQL resultset) because we forget about the
> grouping right away each time (so lots of useless cell names comparisons in
> particular). But outside inefficiencies, having to manually recreate the CQL
> structure every time we need it for something is hindering new features and
> makes the code more complex that it should be.
> Said storage engine also has tons of technical debt. To pick an example, the
> fact that during range queries we update {{SliceQueryFilter.count}} is pretty
> hacky and error prone. Or the overly complex ways {{AbstractQueryPager}} has
> to go into to simply "remove the last query result".
> So I want to bite the bullet and modernize this storage engine. I propose to
> do 2 main things:
> # Make the storage engine more aware of the CQL structure. In practice,
> instead of having partitions be a simple iterable map of cells, it should be
> an iterable list of row (each being itself composed of per-column cells,
> though obviously not exactly the same kind of cell we have today).
> # Make the engine more iterative. What I mean here is that in the read path,
> we end up reading all cells in memory (we put them in a ColumnFamily object),
> but there is really no reason to. If instead we were working with iterators
> all the way through, we could get to a point where we're basically
> transferring data from disk to the network, and we should be able to reduce
> GC substantially.
> Please note that such refactor should provide some performance improvements
> right off the bat but it's not it's primary goal either. It's primary goal is
> to simplify the storage engine and adds abstraction that are better suited to
> further optimizations.
[jira] [Commented] (CASSANDRA-9074) Hadoop Cassandra CqlInputFormat pagination - not reading all input rows
[ https://issues.apache.org/jira/browse/CASSANDRA-9074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388706#comment-14388706 ] Alex Liu commented on CASSANDRA-9074: - yes, please test it on the latest branch. > Hadoop Cassandra CqlInputFormat pagination - not reading all input rows > --- > > Key: CASSANDRA-9074 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9074 > Project: Cassandra > Issue Type: Bug > Components: Hadoop > Environment: Cassandra 2.0.11, Hadoop 1.0.4, Datastax java > cassandra-driver-core 2.1.4 >Reporter: fuggy_yama >Assignee: Alex Liu >Priority: Minor > Fix For: 2.0.14 > > > I have a 7-node Cassandra (v2.0.11) cluster and a table with 10k rows. I run > a hadoop job (datanodes reside on cassandra nodes of course) that reads data > from that table and I see that only 7k rows is read to map phase. > I checked CqlInputFormat source code and noticed that a CQL query is build to > select node-local date and also LIMIT clause is added (1k default). So that > 7k read rows can be explained: > 7 nodes * 1k limit = 7k rows read total > The limit can be changed using CqlConfigHelper: > CqlConfigHelper.setInputCQLPageRowSize(job.getConfiguration(), "1000"); > Please help me with questions below: > Is this a desired behavior? > Why CqlInputFormat does not page through the rest of rows? > Is it a bug or should I just increase the InputCQLPageRowSize value? > What if I want to read all data in table and do not know the row count? > What if the amount of rows I need to read per cassandra node is very large - > in other words how to avoid OOM when setting InputCQLPageRowSize very large > to handle all data? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9070) Race in cancelling compactions
[
https://issues.apache.org/jira/browse/CASSANDRA-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388695#comment-14388695
]
Yuki Morishita commented on CASSANDRA-9070:
---
I think there is still a chance that {{ColumnFamilyStore#markAllCompacting()}}
can throw AE after this patch.
If {{CompactionTask}} is already queued with 'marked' SSTables, those SSTables
are not unmarked until the task actually runs.
If we really want to cancel all the things, we need to cancel all the task in
the queue and release resources associated with.
> Race in cancelling compactions
> --
>
> Key: CASSANDRA-9070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9070
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 2.0.14
>
> Attachments:
> 0001-check-if-we-are-paused-before-starting-compaction.patch
>
>
> seems we might have a race situation when cancelling compactions
> currently we do the following to ensure that we don't start any new
> compactions when we try to do markAllCompacting()
> # pause compactions - this makes sure we don't create any new compaction
> tasks from the compaction strategies
> # cancel any ongoing compactions - compactions register themselves with the
> CompactionMetrics and then, when cancelling we get all compactions here, and
> tell them to stop
> Problem is that there is a window between when the CompactionTask is created
> and when it is registered in CompactionMetrics meaning with a bit of bad
> luck, we could have a situation like this:
> # we finish a compaction and create a new CompactionTask from the compaction
> strategy
> # we pause the compaction strategies to not create any new CompactionTasks
> # we cancel all ongoing compactions
> # The CompactionTask created in #1 above registers itself in
> CompactionMetrics and misses that it should be cancelled
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[1/3] cassandra git commit: make default max_hint_window_in_ms 3 hours
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 994eb2fa7 -> 963dd54b7 refs/heads/trunk 2d7909dc1 -> dbe909e06 make default max_hint_window_in_ms 3 hours Patch by Jeff Jirsa, reviewed by brandonwilliams for CASSANDRA-9076 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/963dd54b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/963dd54b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/963dd54b Branch: refs/heads/cassandra-2.1 Commit: 963dd54b71bdcac29c484d89a826773a3f090496 Parents: 994eb2f Author: Brandon Williams Authored: Tue Mar 31 10:20:39 2015 -0500 Committer: Brandon Williams Committed: Tue Mar 31 10:20:39 2015 -0500 -- src/java/org/apache/cassandra/config/Config.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/963dd54b/src/java/org/apache/cassandra/config/Config.java -- diff --git a/src/java/org/apache/cassandra/config/Config.java b/src/java/org/apache/cassandra/config/Config.java index 378a1ad..201e729 100644 --- a/src/java/org/apache/cassandra/config/Config.java +++ b/src/java/org/apache/cassandra/config/Config.java @@ -59,7 +59,7 @@ public class Config public volatile boolean hinted_handoff_enabled_global = true; public String hinted_handoff_enabled; public Set hinted_handoff_enabled_by_dc = Sets.newConcurrentHashSet(); -public volatile Integer max_hint_window_in_ms = 3600 * 1000; // one hour +public volatile Integer max_hint_window_in_ms = 3 * 3600 * 1000; // three hours public SeedProviderDef seed_provider; public DiskAccessMode disk_access_mode = DiskAccessMode.auto;
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dbe909e0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dbe909e0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dbe909e0 Branch: refs/heads/trunk Commit: dbe909e062e1779b59fb8566b34c6cc821d8862f Parents: 2d7909d 963dd54 Author: Brandon Williams Authored: Tue Mar 31 10:21:18 2015 -0500 Committer: Brandon Williams Committed: Tue Mar 31 10:21:18 2015 -0500 -- src/java/org/apache/cassandra/config/Config.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbe909e0/src/java/org/apache/cassandra/config/Config.java -- diff --cc src/java/org/apache/cassandra/config/Config.java index 1e69b26,201e729..c6131f5 --- a/src/java/org/apache/cassandra/config/Config.java +++ b/src/java/org/apache/cassandra/config/Config.java @@@ -62,9 -59,9 +62,9 @@@ public class Confi public volatile boolean hinted_handoff_enabled_global = true; public String hinted_handoff_enabled; public Set hinted_handoff_enabled_by_dc = Sets.newConcurrentHashSet(); - public volatile Integer max_hint_window_in_ms = 3600 * 1000; // one hour + public volatile Integer max_hint_window_in_ms = 3 * 3600 * 1000; // three hours -public SeedProviderDef seed_provider; +public ParameterizedClass seed_provider; public DiskAccessMode disk_access_mode = DiskAccessMode.auto; public DiskFailurePolicy disk_failure_policy = DiskFailurePolicy.ignore;
[2/3] cassandra git commit: make default max_hint_window_in_ms 3 hours
make default max_hint_window_in_ms 3 hours Patch by Jeff Jirsa, reviewed by brandonwilliams for CASSANDRA-9076 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/963dd54b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/963dd54b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/963dd54b Branch: refs/heads/trunk Commit: 963dd54b71bdcac29c484d89a826773a3f090496 Parents: 994eb2f Author: Brandon Williams Authored: Tue Mar 31 10:20:39 2015 -0500 Committer: Brandon Williams Committed: Tue Mar 31 10:20:39 2015 -0500 -- src/java/org/apache/cassandra/config/Config.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/963dd54b/src/java/org/apache/cassandra/config/Config.java -- diff --git a/src/java/org/apache/cassandra/config/Config.java b/src/java/org/apache/cassandra/config/Config.java index 378a1ad..201e729 100644 --- a/src/java/org/apache/cassandra/config/Config.java +++ b/src/java/org/apache/cassandra/config/Config.java @@ -59,7 +59,7 @@ public class Config public volatile boolean hinted_handoff_enabled_global = true; public String hinted_handoff_enabled; public Set hinted_handoff_enabled_by_dc = Sets.newConcurrentHashSet(); -public volatile Integer max_hint_window_in_ms = 3600 * 1000; // one hour +public volatile Integer max_hint_window_in_ms = 3 * 3600 * 1000; // three hours public SeedProviderDef seed_provider; public DiskAccessMode disk_access_mode = DiskAccessMode.auto;
[jira] [Updated] (CASSANDRA-9076) Consistency: make default max_hint_window_in_ms actually 3 hours
[ https://issues.apache.org/jira/browse/CASSANDRA-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-9076: Reviewer: Brandon Williams (was: Aleksey Yeschenko) > Consistency: make default max_hint_window_in_ms actually 3 hours > > > Key: CASSANDRA-9076 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9076 > Project: Cassandra > Issue Type: Improvement >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Trivial > Fix For: 3.0, 2.1.4 > > Attachments: max_hint_window.diff > > > From IRC/#cassandra: > CASSANDRA-4632 changed max_hint_window_in_ms from 1 hour to 3 hours, however, > it only changed the default in cassandra.yaml - it left the actual default in > Config.java at 1 hour. > This issue is to update Config.java for consistency, in case users comment > out the actual entry in cassandra.yaml and revert to the "real" default of 1 > hour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9076) Consistency: make default max_hint_window_in_ms actually 3 hours
[ https://issues.apache.org/jira/browse/CASSANDRA-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388679#comment-14388679 ] Jeff Jirsa commented on CASSANDRA-9076: --- The change itself is trivial and should be safe. At least 3.0 and 2.1. The issue was mentioned when it bit a 2.0 user. I know 2.0 is fairly late in the game at this point, but this is also a pretty non-invasive change - I'd personally like to see it in 2.0, but I'll live either way, since there's an easy cassandra.yaml workaround for getting 3hrs instead of 1hr. Up to Aleksey. > Consistency: make default max_hint_window_in_ms actually 3 hours > > > Key: CASSANDRA-9076 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9076 > Project: Cassandra > Issue Type: Improvement >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Trivial > Fix For: 3.0, 2.1.4 > > Attachments: max_hint_window.diff > > > From IRC/#cassandra: > CASSANDRA-4632 changed max_hint_window_in_ms from 1 hour to 3 hours, however, > it only changed the default in cassandra.yaml - it left the actual default in > Config.java at 1 hour. > This issue is to update Config.java for consistency, in case users comment > out the actual entry in cassandra.yaml and revert to the "real" default of 1 > hour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8143) Partitioner should not be accessed through StorageService
[ https://issues.apache.org/jira/browse/CASSANDRA-8143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388678#comment-14388678 ] Bryn Cooke commented on CASSANDRA-8143: --- If the partitioner could be defined at the column family level then the location of data can be tuned to the dataset. Combined with custom secondary index implementations this could be a very powerful feature as you could effectively tune data locality and do local reads where appropriate. Support would need to be all the way up to the CQL level though. > Partitioner should not be accessed through StorageService > - > > Key: CASSANDRA-8143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8143 > Project: Cassandra > Issue Type: Bug >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Minor > Fix For: 3.0 > > > The configured partitioner is no longer the only partitioner in use in the > database, as e.g. index tables use LocalPartitioner. > To make sure the correct partitioner is used for each table, accesses of > StorageService.getPartitioner() should be replaced with retrieval of the > CFS-specific partitioner. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9076) Consistency: make default max_hint_window_in_ms actually 3 hours
[ https://issues.apache.org/jira/browse/CASSANDRA-9076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-9076: --- Fix Version/s: 2.1.4 3.0 > Consistency: make default max_hint_window_in_ms actually 3 hours > > > Key: CASSANDRA-9076 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9076 > Project: Cassandra > Issue Type: Improvement >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa >Priority: Trivial > Fix For: 3.0, 2.1.4 > > Attachments: max_hint_window.diff > > > From IRC/#cassandra: > CASSANDRA-4632 changed max_hint_window_in_ms from 1 hour to 3 hours, however, > it only changed the default in cassandra.yaml - it left the actual default in > Config.java at 1 hour. > This issue is to update Config.java for consistency, in case users comment > out the actual entry in cassandra.yaml and revert to the "real" default of 1 > hour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (CASSANDRA-6992) Bootstrap on vnodes clusters can cause stampeding/storm behavior
[ https://issues.apache.org/jira/browse/CASSANDRA-6992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reopened CASSANDRA-6992: --- Assignee: Yuki Morishita Reopening since this is an improvement we should think carefully about. Will mark such instead of bug. > Bootstrap on vnodes clusters can cause stampeding/storm behavior > > > Key: CASSANDRA-6992 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6992 > Project: Cassandra > Issue Type: Bug > Environment: Various vnodes-enabled clusters in EC2, m1.xlarge and > hi1.4xlarge, ~3000-8000 tokens. >Reporter: Rick Branson >Assignee: Yuki Morishita > > Assuming this is an issue with vnodes clusters because > SSTableReader#getPositionsForRanges is more expensive to compute with 256x > the ranges, but could be wrong. On even well-provisioned hosts, this can > cause a severe spike in network throughput & CPU utilization from a storm of > flushes, which impacts long-tail times pretty badly. On weaker hosts (like > m1.xlarge with ~500GB of data), it can result in minutes of churn while the > node gets through StreamOut#createPendingFiles. This *might* be better in > 2.0, but it's probably still reproducible because the bootstrapping node > sends out all of it's streaming requests at once. > I'm thinking that this could be staggered at the bootstrapping node to avoid > the simultaneous spike across the whole cluster. Not sure on how to stagger > it besides something very naive like one-at-a-time with a pause. Maybe this > should also be throttled in StreamOut#createPendingFiles on the out-streaming > host? Any thoughts? > From the stack dump of one of our weaker nodes that was struggling for a few > minutes just starting the StreamOut: > "MiscStage:1" daemon prio=10 tid=0x0292f000 nid=0x688 runnable > [0x7f7b03df6000] >java.lang.Thread.State: RUNNABLE > at > org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:361) > at > org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:371) > at > org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:187) > at > org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:125) > at > org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:889) > at > org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:790) > at > org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:730) > at > org.apache.cassandra.streaming.StreamOut.createPendingFiles(StreamOut.java:172) > at > org.apache.cassandra.streaming.StreamOut.transferSSTables(StreamOut.java:157) > at > org.apache.cassandra.streaming.StreamOut.transferRanges(StreamOut.java:148) > at > org.apache.cassandra.streaming.StreamOut.transferRanges(StreamOut.java:116) > at > org.apache.cassandra.streaming.StreamRequestVerbHandler.doVerb(StreamRequestVerbHandler.java:44) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:662) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6992) Bootstrap on vnodes clusters can cause stampeding/storm behavior
[ https://issues.apache.org/jira/browse/CASSANDRA-6992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6992: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) > Bootstrap on vnodes clusters can cause stampeding/storm behavior > > > Key: CASSANDRA-6992 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6992 > Project: Cassandra > Issue Type: Improvement > Environment: Various vnodes-enabled clusters in EC2, m1.xlarge and > hi1.4xlarge, ~3000-8000 tokens. >Reporter: Rick Branson >Assignee: Yuki Morishita >Priority: Minor > > Assuming this is an issue with vnodes clusters because > SSTableReader#getPositionsForRanges is more expensive to compute with 256x > the ranges, but could be wrong. On even well-provisioned hosts, this can > cause a severe spike in network throughput & CPU utilization from a storm of > flushes, which impacts long-tail times pretty badly. On weaker hosts (like > m1.xlarge with ~500GB of data), it can result in minutes of churn while the > node gets through StreamOut#createPendingFiles. This *might* be better in > 2.0, but it's probably still reproducible because the bootstrapping node > sends out all of it's streaming requests at once. > I'm thinking that this could be staggered at the bootstrapping node to avoid > the simultaneous spike across the whole cluster. Not sure on how to stagger > it besides something very naive like one-at-a-time with a pause. Maybe this > should also be throttled in StreamOut#createPendingFiles on the out-streaming > host? Any thoughts? > From the stack dump of one of our weaker nodes that was struggling for a few > minutes just starting the StreamOut: > "MiscStage:1" daemon prio=10 tid=0x0292f000 nid=0x688 runnable > [0x7f7b03df6000] >java.lang.Thread.State: RUNNABLE > at > org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:361) > at > org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:371) > at > org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:187) > at > org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:125) > at > org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:889) > at > org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:790) > at > org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:730) > at > org.apache.cassandra.streaming.StreamOut.createPendingFiles(StreamOut.java:172) > at > org.apache.cassandra.streaming.StreamOut.transferSSTables(StreamOut.java:157) > at > org.apache.cassandra.streaming.StreamOut.transferRanges(StreamOut.java:148) > at > org.apache.cassandra.streaming.StreamOut.transferRanges(StreamOut.java:116) > at > org.apache.cassandra.streaming.StreamRequestVerbHandler.doVerb(StreamRequestVerbHandler.java:44) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:662) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-5499) Clean out TokenMetadata.getPrimaryRangeFor
[ https://issues.apache.org/jira/browse/CASSANDRA-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-5499. --- Resolution: Incomplete superseded by CASSANDRA-6061 > Clean out TokenMetadata.getPrimaryRangeFor > -- > > Key: CASSANDRA-5499 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5499 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Jonathan Ellis > > TokenMetadata.getPrimaryRangeFor is dangerous because it can return invalid > results under NetworkTopologyStrategy. Callers should generally use > StorageService.getPrimaryRangesForEndpoint instead. See CASSANDRA-5424. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8606) sstablesplit does not remove original sstable
[ https://issues.apache.org/jira/browse/CASSANDRA-8606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-8606: -- Reviewer: T Jake Luciani [~tjake] to review > sstablesplit does not remove original sstable > - > > Key: CASSANDRA-8606 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8606 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Branimir Lambov > Fix For: 2.1.4 > > > sstablesplit leaves the original file on disk, it should not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-6992) Bootstrap on vnodes clusters can cause stampeding/storm behavior
[ https://issues.apache.org/jira/browse/CASSANDRA-6992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson resolved CASSANDRA-6992. Resolution: Cannot Reproduce Closing due to no response from reporter for > 2 months. If this is being reproduced on a 2.0.x or 2.1.x cassandra release, please re-open. > Bootstrap on vnodes clusters can cause stampeding/storm behavior > > > Key: CASSANDRA-6992 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6992 > Project: Cassandra > Issue Type: Bug > Environment: Various vnodes-enabled clusters in EC2, m1.xlarge and > hi1.4xlarge, ~3000-8000 tokens. >Reporter: Rick Branson > > Assuming this is an issue with vnodes clusters because > SSTableReader#getPositionsForRanges is more expensive to compute with 256x > the ranges, but could be wrong. On even well-provisioned hosts, this can > cause a severe spike in network throughput & CPU utilization from a storm of > flushes, which impacts long-tail times pretty badly. On weaker hosts (like > m1.xlarge with ~500GB of data), it can result in minutes of churn while the > node gets through StreamOut#createPendingFiles. This *might* be better in > 2.0, but it's probably still reproducible because the bootstrapping node > sends out all of it's streaming requests at once. > I'm thinking that this could be staggered at the bootstrapping node to avoid > the simultaneous spike across the whole cluster. Not sure on how to stagger > it besides something very naive like one-at-a-time with a pause. Maybe this > should also be throttled in StreamOut#createPendingFiles on the out-streaming > host? Any thoughts? > From the stack dump of one of our weaker nodes that was struggling for a few > minutes just starting the StreamOut: > "MiscStage:1" daemon prio=10 tid=0x0292f000 nid=0x688 runnable > [0x7f7b03df6000] >java.lang.Thread.State: RUNNABLE > at > org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:361) > at > org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:371) > at > org.apache.cassandra.io.sstable.IndexHelper$IndexInfo.deserialize(IndexHelper.java:187) > at > org.apache.cassandra.db.RowIndexEntry$Serializer.deserialize(RowIndexEntry.java:125) > at > org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:889) > at > org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:790) > at > org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:730) > at > org.apache.cassandra.streaming.StreamOut.createPendingFiles(StreamOut.java:172) > at > org.apache.cassandra.streaming.StreamOut.transferSSTables(StreamOut.java:157) > at > org.apache.cassandra.streaming.StreamOut.transferRanges(StreamOut.java:148) > at > org.apache.cassandra.streaming.StreamOut.transferRanges(StreamOut.java:116) > at > org.apache.cassandra.streaming.StreamRequestVerbHandler.doVerb(StreamRequestVerbHandler.java:44) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:662) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8002) Cassandra SSTableReader hang on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388617#comment-14388617
]
Philip Thompson commented on CASSANDRA-8002:
Are you still having this problem? If it was only occurring on one sstable, did
you try running scrub?
> Cassandra SSTableReader hang on startup
> ---
>
> Key: CASSANDRA-8002
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8002
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: centos6.5
> jre-7u60.x86_64
> kernel:2.6.32-431.17.1.el6.x86_64
>Reporter: Tim Isted
>Priority: Minor
> Fix For: 2.0.14
>
> Attachments: htop.png
>
>
> We are experiencing a hang on startup when SSTableReader tries to open
> certain DataIndex-jb files, each time its the same SSTable
> {code}
> #/var/log/system.log
> INFO [SSTableBatchOpen:1] 2014-09-25 07:58:13,660 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25125
> (291271971 bytes)
> INFO [SSTableBatchOpen:2] 2014-09-25 07:58:13,660 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25182
> (97969101 bytes)
> INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,660 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25217
> (8787563 bytes)
> INFO [SSTableBatchOpen:4] 2014-09-25 07:58:13,661 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-23265
> (395830480191 bytes)
> INFO [SSTableBatchOpen:5] 2014-09-25 07:58:13,661 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25219
> (80412 bytes)
> INFO [SSTableBatchOpen:7] 2014-09-25 07:58:13,663 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25130
> (52552441 bytes)
> INFO [SSTableBatchOpen:6] 2014-09-25 07:58:13,663 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25106
> (1160245456 bytes)
> INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,663 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25218
> (5068542 bytes)
> INFO [SSTableBatchOpen:5] 2014-09-25 07:58:13,737 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25021
> (1169149628 bytes)
> INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,745 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25163
> (64078835 bytes)
> INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,746 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25187
> (53540520 bytes)
> INFO [SSTableBatchOpen:7] 2014-09-25 07:58:13,771 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24537
> (18890626389 bytes)
> INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,779 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24936
> (1183874336 bytes)
> INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,785 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25158
> (99217260 bytes)
> INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,813 SSTableReader.java (line
> 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859
> (4761451567 bytes){code}
> {code}
> #/var/log/cassandra.log
> INFO 07:58:13,663 Opening
> /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25106 (1160245456
> bytes)
> INFO 07:58:13,663 Opening
> /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25218 (5068542
> bytes)
> INFO 07:58:13,737 Opening
> /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25021 (1169149628
> bytes)
> INFO 07:58:13,745 Opening
> /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25163 (64078835
> bytes)
> INFO 07:58:13,746 Opening
> /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25187 (53540520
> bytes)
> INFO 07:58:13,771 Opening
> /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24537 (18890626389
> bytes)
> INFO 07:58:13,779 Opening
> /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24936 (1183874336
> bytes)
> INFO 07:58:13,785 Opening
> /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25158 (99217260
> bytes)
> INFO 07:58:13,813 Opening
> /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859 (4761451567
> bytes){code}
> {code}
> -rw-r--r--. 1 cassandra cassandra 1.4M Sep 24 07:23
> /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-CompressionInf
[jira] [Resolved] (CASSANDRA-8387) Schema inconsistency (cached vs schema_columnfamilies)
[
https://issues.apache.org/jira/browse/CASSANDRA-8387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson resolved CASSANDRA-8387.
Resolution: Not a Problem
Reproduced In: 2.1.2, 2.1.1 (was: 2.1.1, 2.1.2)
> Schema inconsistency (cached vs schema_columnfamilies)
> --
>
> Key: CASSANDRA-8387
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8387
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: C* 2.1.1 3-node cluster
>Reporter: Marcus Olsson
>
> While running some tests on a 3-node cluster running C* 2.1.1 we encountered
> a problem creating the same table schema twice(on different nodes). One thing
> to note is that one of the nodes clock was ~4 seconds behind the others, but
> I don't think that's the problem since the exception was reproduced here
> aswell: http://www.mail-archive.com/user@cassandra.apache.org/msg39560.html.
> While running the same create table statement more than once(on different
> clients) the logs outputted this on one of the nodes:
> {noformat}
> (node x.x.x.1):
> 2014-11-25T16:11:44.651+0100 INFO [SharedPool-Worker-2]
> MigrationManager.java:248 Create new ColumnFamily:
> org.apache.cassandra.config.CFMetaData@45c290de[cfId=5e334b40-74b5-11e4-b1b6-017ad0689f5d,ksName=test,cfName=test,cfType=Standard,comparator=org.apache.cassandra.db.marshal.UTF8Type,comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=id,
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING_COLUMN,
> componentIndex=null, indexName=null, indexType=null},
> ColumnDefinition{name=key, type=org.apache.cassandra.db.marshal.UTF8Type,
> kind=PARTITION_KEY, componentIndex=null, indexName=null, indexType=null},
> ColumnDefinition{name=value, type=org.apache.cassandra.db.marshal.BytesType,
> kind=COMPACT_VALUE, componentIndex=null, indexName=null,
> indexType=null}],compactionStrategyClass=class
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL",
>
> "rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=true]
> ...
> 2014-11-25T16:11:44.667+0100 INFO [MigrationStage:1] DefsTables.java:373
> Loading
> org.apache.cassandra.config.CFMetaData@40a1ee90[cfId=5bc7c980-74b5-11e4-9131-d9b94a3d8927,ksName=test,cfName=test,cfType=Standard,comparator=org.apache.cassandra.db.marshal.UTF8Type,comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=id,
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING_COLUMN,
> componentIndex=null, indexName=null, indexType=null},
> ColumnDefinition{name=key, type=org.apache.cassandra.db.marshal.UTF8Type,
> kind=PARTITION_KEY, componentIndex=null, indexName=null, indexType=null},
> ColumnDefinition{name=value, type=org.apache.cassandra.db.marshal.BytesType,
> kind=COMPACT_VALUE, componentIndex=null, indexName=null,
> indexType=null}],compactionStrategyClass=class
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL",
>
> "rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=true]
> ...
> java.lang.RuntimeException:
> org.apache.cassandra.exceptions.ConfigurationException: Column family ID
> mismatch (found 5e334b40-74b5-11e4-b1b6-017ad0689f5d; expected
> 5bc7c980-74b5-11e4-9131-d9b94a3d8927)
> at
> org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1171)
> ~[apache-cassandra-2.1.1.jar:2.1.1]
> at
> org.apache.cassandra.db.DefsTables.updateColumnFamily(DefsTables.java:422)
> ~[apache-cassandra-2.1.1.jar:2.1.1]
> at
> org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:295)
> ~[apache-cassandra-2.1.1.jar:2.1.1]
> at
> org.apache.cassandra.db.DefsTables.mergeSchemaInternal(DefsTables.java:194)
> ~[apache-cassandra-2.1.1.jar:2.1.1]
> at
> org.apache.cassandra.db.
[jira] [Commented] (CASSANDRA-8387) Schema inconsistency (cached vs schema_columnfamilies)
[
https://issues.apache.org/jira/browse/CASSANDRA-8387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388615#comment-14388615
]
Philip Thompson commented on CASSANDRA-8387:
Unfortunately, this is as expected. Multiple clients should not be trying to
create the same table. If you really want some sort of Create Table via LWT,
please open a new ticket marked "Improvement" for that feature request.
> Schema inconsistency (cached vs schema_columnfamilies)
> --
>
> Key: CASSANDRA-8387
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8387
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: C* 2.1.1 3-node cluster
>Reporter: Marcus Olsson
>
> While running some tests on a 3-node cluster running C* 2.1.1 we encountered
> a problem creating the same table schema twice(on different nodes). One thing
> to note is that one of the nodes clock was ~4 seconds behind the others, but
> I don't think that's the problem since the exception was reproduced here
> aswell: http://www.mail-archive.com/user@cassandra.apache.org/msg39560.html.
> While running the same create table statement more than once(on different
> clients) the logs outputted this on one of the nodes:
> {noformat}
> (node x.x.x.1):
> 2014-11-25T16:11:44.651+0100 INFO [SharedPool-Worker-2]
> MigrationManager.java:248 Create new ColumnFamily:
> org.apache.cassandra.config.CFMetaData@45c290de[cfId=5e334b40-74b5-11e4-b1b6-017ad0689f5d,ksName=test,cfName=test,cfType=Standard,comparator=org.apache.cassandra.db.marshal.UTF8Type,comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=id,
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING_COLUMN,
> componentIndex=null, indexName=null, indexType=null},
> ColumnDefinition{name=key, type=org.apache.cassandra.db.marshal.UTF8Type,
> kind=PARTITION_KEY, componentIndex=null, indexName=null, indexType=null},
> ColumnDefinition{name=value, type=org.apache.cassandra.db.marshal.BytesType,
> kind=COMPACT_VALUE, componentIndex=null, indexName=null,
> indexType=null}],compactionStrategyClass=class
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL",
>
> "rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=true]
> ...
> 2014-11-25T16:11:44.667+0100 INFO [MigrationStage:1] DefsTables.java:373
> Loading
> org.apache.cassandra.config.CFMetaData@40a1ee90[cfId=5bc7c980-74b5-11e4-9131-d9b94a3d8927,ksName=test,cfName=test,cfType=Standard,comparator=org.apache.cassandra.db.marshal.UTF8Type,comment=,readRepairChance=0.0,dcLocalReadRepairChance=0.1,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.UTF8Type,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata=[ColumnDefinition{name=id,
> type=org.apache.cassandra.db.marshal.UTF8Type, kind=CLUSTERING_COLUMN,
> componentIndex=null, indexName=null, indexType=null},
> ColumnDefinition{name=key, type=org.apache.cassandra.db.marshal.UTF8Type,
> kind=PARTITION_KEY, componentIndex=null, indexName=null, indexType=null},
> ColumnDefinition{name=value, type=org.apache.cassandra.db.marshal.BytesType,
> kind=COMPACT_VALUE, componentIndex=null, indexName=null,
> indexType=null}],compactionStrategyClass=class
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching={"keys":"ALL",
>
> "rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=99.0PERCENTILE,droppedColumns={},triggers=[],isDense=true]
> ...
> java.lang.RuntimeException:
> org.apache.cassandra.exceptions.ConfigurationException: Column family ID
> mismatch (found 5e334b40-74b5-11e4-b1b6-017ad0689f5d; expected
> 5bc7c980-74b5-11e4-9131-d9b94a3d8927)
> at
> org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1171)
> ~[apache-cassandra-2.1.1.jar:2.1.1]
> at
> org.apache.cassandra.db.DefsTables.updateColumnFamily(DefsTables.java:422)
> ~[apache-cassandra-2.1.1.jar:2.1.1]
> at
> org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:295)
> ~[apache-
[jira] [Updated] (CASSANDRA-8002) Cassandra SSTableReader hang on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson updated CASSANDRA-8002:
---
Description:
We are experiencing a hang on startup when SSTableReader tries to open certain
DataIndex-jb files, each time its the same SSTable
{code}
#/var/log/system.log
INFO [SSTableBatchOpen:1] 2014-09-25 07:58:13,660 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25125
(291271971 bytes)
INFO [SSTableBatchOpen:2] 2014-09-25 07:58:13,660 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25182
(97969101 bytes)
INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,660 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25217
(8787563 bytes)
INFO [SSTableBatchOpen:4] 2014-09-25 07:58:13,661 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-23265
(395830480191 bytes)
INFO [SSTableBatchOpen:5] 2014-09-25 07:58:13,661 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25219
(80412 bytes)
INFO [SSTableBatchOpen:7] 2014-09-25 07:58:13,663 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25130
(52552441 bytes)
INFO [SSTableBatchOpen:6] 2014-09-25 07:58:13,663 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25106
(1160245456 bytes)
INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,663 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25218
(5068542 bytes)
INFO [SSTableBatchOpen:5] 2014-09-25 07:58:13,737 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25021
(1169149628 bytes)
INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,745 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25163
(64078835 bytes)
INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,746 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25187
(53540520 bytes)
INFO [SSTableBatchOpen:7] 2014-09-25 07:58:13,771 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24537
(18890626389 bytes)
INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,779 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24936
(1183874336 bytes)
INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,785 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25158
(99217260 bytes)
INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,813 SSTableReader.java (line
223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859
(4761451567 bytes){code}
{code}
#/var/log/cassandra.log
INFO 07:58:13,663 Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25106 (1160245456
bytes)
INFO 07:58:13,663 Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25218 (5068542 bytes)
INFO 07:58:13,737 Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25021 (1169149628
bytes)
INFO 07:58:13,745 Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25163 (64078835
bytes)
INFO 07:58:13,746 Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25187 (53540520
bytes)
INFO 07:58:13,771 Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24537 (18890626389
bytes)
INFO 07:58:13,779 Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24936 (1183874336
bytes)
INFO 07:58:13,785 Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25158 (99217260
bytes)
INFO 07:58:13,813 Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859 (4761451567
bytes){code}
{code}
-rw-r--r--. 1 cassandra cassandra 1.4M Sep 24 07:23
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-CompressionInfo.db
-rw-r--r--. 1 cassandra cassandra 4.5G Sep 24 07:23
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Data.db
-rw-r--r--. 1 cassandra cassandra 53M Sep 24 07:23
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Filter.db
-rw-r--r--. 1 cassandra cassandra 3.3G Sep 24 07:23
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Index.db
-rw-r--r--. 1 cassandra cassandra 5.9K Sep 24 07:23
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Statistics.db
-rw-r--r--. 1 cassandra cassandra 26M Sep 24 07:23
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Summary.db
-rw-r--r--. 1 cassandra cassandra 79 Sep 24 07:23
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-TOC.txt{code}
#nodetool netstats{code}
Mode: STA
