[jira] [Commented] (CASSANDRA-6739) Exception on nodetool cfstats
[ https://issues.apache.org/jira/browse/CASSANDRA-6739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906738#comment-13906738 ] Mateusz Gajewski commented on CASSANDRA-6739: - LGTM. Thanks. P.S. I think that nodetool implementation needs a little bit of refactoring. Can I contribute in this field? Just to explore Cassandra internals better? Exception on nodetool cfstats - Key: CASSANDRA-6739 URL: https://issues.apache.org/jira/browse/CASSANDRA-6739 Project: Cassandra Issue Type: Bug Reporter: Mateusz Gajewski Assignee: Mikhail Stepura Fix For: 2.1 Attachments: 2014-02-19 14-51-43.png, CASSANDRA-2.1-6739.patch, screenshot-1402196.jpg While running nodetool cfstats exception is thrown: javax.management.InstanceNotFoundException: org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system_traces,scope=sessions,name=PendingTasks VisualVM MBeans list attached PendingTasks is not actually exposed. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6739) Exception on nodetool cfstats
[ https://issues.apache.org/jira/browse/CASSANDRA-6739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-6739: Fix Version/s: (was: 2.1) 2.1 beta2 Exception on nodetool cfstats - Key: CASSANDRA-6739 URL: https://issues.apache.org/jira/browse/CASSANDRA-6739 Project: Cassandra Issue Type: Bug Reporter: Mateusz Gajewski Assignee: Mikhail Stepura Fix For: 2.1 beta2 Attachments: 2014-02-19 14-51-43.png, CASSANDRA-2.1-6739.patch, screenshot-1402196.jpg While running nodetool cfstats exception is thrown: javax.management.InstanceNotFoundException: org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system_traces,scope=sessions,name=PendingTasks VisualVM MBeans list attached PendingTasks is not actually exposed. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6741) cqlsh DESCRIBE KEYSPACE returns 'NoneType' object has no attribute 'get_usertypes_names'
[
https://issues.apache.org/jira/browse/CASSANDRA-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-6741:
Fix Version/s: (was: 2.1)
2.1 beta2
cqlsh DESCRIBE KEYSPACE returns 'NoneType' object has no attribute
'get_usertypes_names'
--
Key: CASSANDRA-6741
URL: https://issues.apache.org/jira/browse/CASSANDRA-6741
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Mikhail Stepura
Fix For: 2.1 beta2
Attachments: CASSANDRA-2.1-6741.patch
Start a fresh cluster on trunk and try to describe any keyspace :
{code}
ccm create -v git:trunk test
ccm populate -n 1
ccm start
ccm node1 cqlsh
cqlsh DESCRIBE KEYSPACE system;
CREATE KEYSPACE system WITH replication = {
'class': 'LocalStrategy'
};
'NoneType' object has no attribute 'get_usertypes_names'
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Assigned] (CASSANDRA-6733) Upgrade of 1.2.11 to 2.0.5 make IllegalArgumentException in Buffer.limit on read of a super column family
[
https://issues.apache.org/jira/browse/CASSANDRA-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne reassigned CASSANDRA-6733:
---
Assignee: Sylvain Lebresne
Upgrade of 1.2.11 to 2.0.5 make IllegalArgumentException in Buffer.limit on
read of a super column family
-
Key: CASSANDRA-6733
URL: https://issues.apache.org/jira/browse/CASSANDRA-6733
Project: Cassandra
Issue Type: Bug
Reporter: Nicolas Lalevée
Assignee: Sylvain Lebresne
We have a super column family which was first created with a 1.0.x. Then
upgraded to 1.1.x, then to 1.2.11, and now to 2.0.5.
{noformat}
cqlsh:QaUser desc table user_view;
CREATE TABLE user_view (
key bigint,
column1 varint,
column2 text,
value counter,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=1.00 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='99.0PERCENTILE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
{noformat}
With cqlsh, the following query was doing a timeout:
{noformat}
select * from user_view where key = 3 and column1 = 1 and column2 =
'20130218';
{noformat}
In the log of cassandra, we could read:
{noformat}
ERROR [ReadStage:1385] 2014-02-19 14:45:19,549 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:1385,5,main]
java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:267)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:82)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35)
at
org.apache.cassandra.db.marshal.AbstractType$1.compare(AbstractType.java:63)
at
org.apache.cassandra.db.marshal.AbstractType$1.compare(AbstractType.java:60)
at java.util.Collections.indexedBinarySearch(Collections.java:377)
at java.util.Collections.binarySearch(Collections.java:365)
at
org.apache.cassandra.io.sstable.IndexHelper.indexFor(IndexHelper.java:144)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.setNextSlice(IndexedSliceReader.java:262)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.init(IndexedSliceReader.java:255)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader.init(IndexedSliceReader.java:91)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:42)
at
org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:167)
at
org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:250)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1560)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1379)
at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
at
org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
{noformat}
I tried launching repair on our 2 nodes, nothing improved.
I tried launching a major compaction on this column family, the query doesn't
fail anymore and return expected results;
This happens on our cluster which is used for integration and test purpose,
not much activity on it. There are
[jira] [Commented] (CASSANDRA-6733) Upgrade of 1.2.11 to 2.0.5 make IllegalArgumentException in Buffer.limit on read of a super column family
[
https://issues.apache.org/jira/browse/CASSANDRA-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906754#comment-13906754
]
Sylvain Lebresne commented on CASSANDRA-6733:
-
bq. I have a quite small (2 x ~500K) snapshot done before the upgrade of the
cluster I could share, if needed.
If it's no problem for you (and feel free to send it to me privately if you
prefer), that would definitively simplify checking what's wrong (if you can
share the original thrift table definition for completeness sake, that would be
perfect).
Upgrade of 1.2.11 to 2.0.5 make IllegalArgumentException in Buffer.limit on
read of a super column family
-
Key: CASSANDRA-6733
URL: https://issues.apache.org/jira/browse/CASSANDRA-6733
Project: Cassandra
Issue Type: Bug
Reporter: Nicolas Lalevée
Assignee: Sylvain Lebresne
We have a super column family which was first created with a 1.0.x. Then
upgraded to 1.1.x, then to 1.2.11, and now to 2.0.5.
{noformat}
cqlsh:QaUser desc table user_view;
CREATE TABLE user_view (
key bigint,
column1 varint,
column2 text,
value counter,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=1.00 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='99.0PERCENTILE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
{noformat}
With cqlsh, the following query was doing a timeout:
{noformat}
select * from user_view where key = 3 and column1 = 1 and column2 =
'20130218';
{noformat}
In the log of cassandra, we could read:
{noformat}
ERROR [ReadStage:1385] 2014-02-19 14:45:19,549 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:1385,5,main]
java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:267)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:82)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35)
at
org.apache.cassandra.db.marshal.AbstractType$1.compare(AbstractType.java:63)
at
org.apache.cassandra.db.marshal.AbstractType$1.compare(AbstractType.java:60)
at java.util.Collections.indexedBinarySearch(Collections.java:377)
at java.util.Collections.binarySearch(Collections.java:365)
at
org.apache.cassandra.io.sstable.IndexHelper.indexFor(IndexHelper.java:144)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.setNextSlice(IndexedSliceReader.java:262)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.init(IndexedSliceReader.java:255)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader.init(IndexedSliceReader.java:91)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:42)
at
org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:167)
at
org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:250)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1560)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1379)
at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
at
org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:65)
at
org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
[jira] [Updated] (CASSANDRA-6742) ArrayIndexOutOfBoundsException on outbound connection attempt by coordinator on table creation
[
https://issues.apache.org/jira/browse/CASSANDRA-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-6742:
Fix Version/s: (was: 2.1 beta1)
2.1 beta2
ArrayIndexOutOfBoundsException on outbound connection attempt by coordinator
on table creation
--
Key: CASSANDRA-6742
URL: https://issues.apache.org/jira/browse/CASSANDRA-6742
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Aleksey Yeschenko
Priority: Critical
Fix For: 2.1 beta2
Attachments: bdplab0.alternate.log, bdplab0.log,
bdplab0_cassandra.yaml
This is a physical four node cluster. Configuration is attached.
Create a keyspace and table from the first node:
{code}
CREATE KEYSPACE Keyspace1 WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '1'
};
USE Keyspace1;
CREATE TABLE Counter1 (
key blob,
column1 ascii,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={};
{code}
And the find the following in the logs:
{code}
INFO [Thrift:1] 2014-02-19 14:04:35,828 MigrationManager.java:210 - Create
new ColumnFamily:
org.apache.cassandra.config.CFMetaData@d824292[cfId=d1bc0c30-99b1-11e3-a5f9-c187ff8103e2,ksName=Keyspace1,cfName=Counter1,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.CounterColumnType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.CounterColumnType, kind=COMPACT_VALUE,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.AsciiType, kind=CLUSTERING_COLUMN,
componentIndex=null, indexName=null,
indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching=KEYS_ONLY,defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=NONE,populateIoCacheOnFlush=false,droppedColumns={},triggers={},rowsPerPartitionToCache=100]
ERROR [WRITE-/172.16.1.211] 2014-02-19 14:04:35,838
OutboundTcpConnection.java:256 - error writing to /172.16.1.211
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.cassandra.db.ArrayBackedSortedColumns.internalAppendOrReconcile(ArrayBackedSortedColumns.java:231)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:143)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:103)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.getColumnCount(ArrayBackedSortedColumns.java:313)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.contentSerializedSize(ColumnFamilySerializer.java:117)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.serializedSize(ColumnFamilySerializer.java:132)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:337)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:397)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:371)
~[main/:na]
at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:116)
~[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeInternal(OutboundTcpConnection.java:273)
[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:225)
[jira] [Assigned] (CASSANDRA-6738) java.lang.ClassCastException: org.apache.cassandra.db.composites.CompoundComposite cannot be cast to org.apache.cassandra.db.composites.CellName
[ https://issues.apache.org/jira/browse/CASSANDRA-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne reassigned CASSANDRA-6738: --- Assignee: Sylvain Lebresne java.lang.ClassCastException: org.apache.cassandra.db.composites.CompoundComposite cannot be cast to org.apache.cassandra.db.composites.CellName Key: CASSANDRA-6738 URL: https://issues.apache.org/jira/browse/CASSANDRA-6738 Project: Cassandra Issue Type: Bug Reporter: Mateusz Gajewski Assignee: Sylvain Lebresne Fix For: 2.1 beta2 When using nodetool upgradesstables (2.0.4 - 2.1-beta) class cast exception occurs: ERROR [CompactionExecutor:7] 2014-02-19 21:34:16,839 CassandraDaemon.java:165 - Exception in thread Thread[CompactionExecutor:7,1,main] java.lang.ClassCastException: org.apache.cassandra.db.composites.CompoundComposite cannot be cast to org.apache.cassandra.db.composites.CellName at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:86) ~[main/:na] at org.apache.cassandra.db.Cell$1.computeNext(Cell.java:75) ~[main/:na] at org.apache.cassandra.db.Cell$1.computeNext(Cell.java:64) ~[main/:na] at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) ~[guava-16.0.jar:na] at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) ~[guava-16.0.jar:na] at org.apache.cassandra.io.sstable.SSTableIdentityIterator.hasNext(SSTableIdentityIterator.java:129) ~[main/:na] at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200) ~[main/:na] at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) ~[guava-16.0.jar:na] at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) ~[guava-16.0.jar:na] at com.google.common.collect.Iterators$7.computeNext(Iterators.java:645) ~[guava-16.0.jar:na] at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) ~[guava-16.0.jar:na] at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) ~[guava-16.0.jar:na] at org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(ColumnIndex.java:165) ~[main/:na] at org.apache.cassandra.db.compaction.LazilyCompactedRow.write(LazilyCompactedRow.java:110) ~[main/:na] at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:178) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:172) ~[main/:na] at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) ~[main/:na] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:67) ~[main/:na] at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:64) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionManager$4.perform(CompactionManager.java:262) ~[main/:na] at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:227) ~[main/:na] at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906818#comment-13906818
]
Andreas Schnitzerling commented on CASSANDRA-6283:
--
So the repair issue (with snapshots) is another problem compared to the
compaction issue, where my finalizer patch works perfectly in normal operation
and shows, that handles are not closed from C* ? Relying to compaction issue,
if C* generates a tmp folder, there will be thousands of files (I had for
instance more than 64K files in one KS w/o finalizer patch). Is that a good
idea for the operating system, when C* is running for weeks?
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: leakdetect.patch, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6692) AtomicBTreeColumns Improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-6692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6692: Attachment: 6692.fix Hmm. This actually exposed a bug we already had: reuse of the ThreadLocal cached instance of the BTree.Builder, through nested BTree updates. This basically completely breaks everything with SecondaryIndex updates, as we apply the SI update whilst performing the BTree.update(). I've converted the ThreadLocal to a queue instead of an instance. Investigating this I also exposed another bug in the same area, with CASSANDRA-6655, where we would lose topLevel deletionInfo. I've included the fix for that in the attached patch. AtomicBTreeColumns Improvements --- Key: CASSANDRA-6692 URL: https://issues.apache.org/jira/browse/CASSANDRA-6692 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Priority: Minor Labels: easyfix, performance Fix For: 2.1 beta2 Attachments: 6692.fix There are two improvements to make to the BTree code that should help: 1) It turns out Stack Allocation is more rubbish than we had hoped, and so the fast route actually allocates garbage. It's unlikely this reduces throughput, but the increased young-gen pressure is probably unwelcome. I propose to remove the fast route for now. 2) It is not uncommon to race to perform an update, so that the new values are actually out-of-date when we come to modify the tree. In this case the update should recognise that the original (portion of) the tree has not been modified, and simply return it, without allocating a new one. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6662) Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is called
[ https://issues.apache.org/jira/browse/CASSANDRA-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6662: Attachment: patch Attaching patch with a few possible changes: - We can avoid making size volatile (when dealing with CASSANDRA-6742) if we just keep the size modifications in a local variable in sortCells(). This is probably better anyway, and whilst it probably won't have much impact on x86, there's a good chance it will on other platforms - I've added a unitFactory which creates an ABSC with only one item, for use in SI.insert()/delete() . We may be able to switch to it in other places as well. - I've added an optimised addAll(), with fast paths for inserting with the current set is empty. It might be worth optimising when non-empty as well, but this seems like the biggest win and easy to apply. This helps in some SI search methods, but in general adding to a new ABSC is a pretty common pattern. Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is called Key: CASSANDRA-6662 URL: https://issues.apache.org/jira/browse/CASSANDRA-6662 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 2.1 beta1 Attachments: patch To avoid poor performance with huge numbers of cells added out of order (which should be rare, but *can* happen with certain batch scenarios) we should make ABSC only sort/reconcile its cells when an actual accessor is actually called, delaying sorting until the very end. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6737) A batch statements on a single partition should not create a new CF object for each update
[ https://issues.apache.org/jira/browse/CASSANDRA-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6737: Attachment: 6737.2.patch Mostly LGTM. One actual problem, and a couple of suggestions: - Looks like in MS.addUpdatesForKey() rows should be being passed as the last parameter to the constructor of params. - It would be nice if addUpdateForKey and addUpdatesForKey were next to each other in MS - Maybe fast path in unzipMutations should use values().iterator instead of lookup based on keyspace hash. Pretty ambivalent about that though. - Nit: unused import in BS Attached new patch with my suggestions. A batch statements on a single partition should not create a new CF object for each update -- Key: CASSANDRA-6737 URL: https://issues.apache.org/jira/browse/CASSANDRA-6737 Project: Cassandra Issue Type: Improvement Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 2.0.6 Attachments: 6737.2.patch, 6737.txt BatchStatement creates a new ColumnFamily object (as well as a new RowMutation object) for every update in the batch, even if all those update are actually on the same partition. This is particularly inefficient when bulkloading data into a single partition (which is not all that uncommon). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6739) Exception on nodetool cfstats
[ https://issues.apache.org/jira/browse/CASSANDRA-6739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906913#comment-13906913 ] Jeremy Hanna commented on CASSANDRA-6739: - I'm sure any contribution would be welcome, although trunk has CASSANDRA-6381 which did some cleanup of nodetool. You might look at [the unresolved lhf (low hanging fruit) labeled tickets|https://issues.apache.org/jira/issues/?jql=project%20%3D%2012310865%20AND%20labels%20%3D%20lhf%20AND%20status%20!%3D%20resolved]. Exception on nodetool cfstats - Key: CASSANDRA-6739 URL: https://issues.apache.org/jira/browse/CASSANDRA-6739 Project: Cassandra Issue Type: Bug Reporter: Mateusz Gajewski Assignee: Mikhail Stepura Fix For: 2.1 beta2 Attachments: 2014-02-19 14-51-43.png, CASSANDRA-2.1-6739.patch, screenshot-1402196.jpg While running nodetool cfstats exception is thrown: javax.management.InstanceNotFoundException: org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system_traces,scope=sessions,name=PendingTasks VisualVM MBeans list attached PendingTasks is not actually exposed. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6739) Exception on nodetool cfstats
[ https://issues.apache.org/jira/browse/CASSANDRA-6739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906918#comment-13906918 ] Mateusz Gajewski commented on CASSANDRA-6739: - Sure, thanks for advice. Do I need to sign some kind of agreement before contributing? Exception on nodetool cfstats - Key: CASSANDRA-6739 URL: https://issues.apache.org/jira/browse/CASSANDRA-6739 Project: Cassandra Issue Type: Bug Reporter: Mateusz Gajewski Assignee: Mikhail Stepura Fix For: 2.1 beta2 Attachments: 2014-02-19 14-51-43.png, CASSANDRA-2.1-6739.patch, screenshot-1402196.jpg While running nodetool cfstats exception is thrown: javax.management.InstanceNotFoundException: org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system_traces,scope=sessions,name=PendingTasks VisualVM MBeans list attached PendingTasks is not actually exposed. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6739) Exception on nodetool cfstats
[ https://issues.apache.org/jira/browse/CASSANDRA-6739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906942#comment-13906942 ] Jeremy Hanna commented on CASSANDRA-6739: - I would take a look at http://wiki.apache.org/cassandra/HowToContribute but really the only thing you need to do is when you submit a patch, you are essentially giving rights to the patch to the apache software foundation. Exception on nodetool cfstats - Key: CASSANDRA-6739 URL: https://issues.apache.org/jira/browse/CASSANDRA-6739 Project: Cassandra Issue Type: Bug Reporter: Mateusz Gajewski Assignee: Mikhail Stepura Fix For: 2.1 beta2 Attachments: 2014-02-19 14-51-43.png, CASSANDRA-2.1-6739.patch, screenshot-1402196.jpg While running nodetool cfstats exception is thrown: javax.management.InstanceNotFoundException: org.apache.cassandra.metrics:type=ColumnFamily,keyspace=system_traces,scope=sessions,name=PendingTasks VisualVM MBeans list attached PendingTasks is not actually exposed. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-3732) Update POM generation after migration to git
[ https://issues.apache.org/jira/browse/CASSANDRA-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907085#comment-13907085 ] ASF GitHub Bot commented on CASSANDRA-3732: --- Github user asfgit closed the pull request at: https://github.com/apache/cassandra/pull/9 Update POM generation after migration to git Key: CASSANDRA-3732 URL: https://issues.apache.org/jira/browse/CASSANDRA-3732 Project: Cassandra Issue Type: Bug Components: Packaging Reporter: Sylvain Lebresne Assignee: Stephen Connolly Priority: Minor Fix For: 1.0.10, 1.1.0 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Assigned] (CASSANDRA-6740) Non-empty flush directory causes NPE on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reassigned CASSANDRA-6740:
-
Assignee: Jonathan Ellis
Non-empty flush directory causes NPE on startup
---
Key: CASSANDRA-6740
URL: https://issues.apache.org/jira/browse/CASSANDRA-6740
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Priority: Minor
With the changes in CASSANDRA-6357, there is now the case where someone may
want to start a fresh cluster on a machine that previously hosted one, clean
out the directories they are familiar with (eg, data_file_directories,
commitlog_directories) but fail to clean out the flush_directory. If they
don't clean that out, they will see this in the logs:
{code}
INFO [main] 2014-02-19 12:20:13,530 ColumnFamilyStore.java:281 -
Initializing system.IndexInfo
INFO [main] 2014-02-19 12:20:13,534 ColumnFamilyStore.java:281 -
Initializing system.peers
INFO [main] 2014-02-19 12:20:13,539 ColumnFamilyStore.java:281 -
Initializing system.local
ERROR [main] 2014-02-19 12:20:13,703 CassandraDaemon.java:458 - Exception
encountered during startup
java.lang.NullPointerException: null
at
org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:167)
~[main/:na]
at
org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:39)
~[main/:na]
at
org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:26)
~[main/:na]
at
org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:66)
~[main/:na]
at
org.apache.cassandra.cql3.UntypedResultSet$Row.getString(UntypedResultSet.java:150)
~[main/:na]
at
org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1761)
~[main/:na]
at
org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1883)
~[main/:na]
at
org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:320)
~[main/:na]
at
org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:301)
~[main/:na]
at
org.apache.cassandra.db.DefsTables.loadFromKeyspace(DefsTables.java:131)
~[main/:na]
at
org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:539)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:230)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:441)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:530)
[main/:na]
{code}
I suggest a better warning message telling them it's due to the unclean flush
dir, instead of an NPE.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6742) Make it safe to concurrently access ABSC after its construction
[
https://issues.apache.org/jira/browse/CASSANDRA-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-6742:
-
Summary: Make it safe to concurrently access ABSC after its construction
(was: ArrayIndexOutOfBoundsException on outbound connection attempt by
coordinator on table creation)
Make it safe to concurrently access ABSC after its construction
---
Key: CASSANDRA-6742
URL: https://issues.apache.org/jira/browse/CASSANDRA-6742
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Aleksey Yeschenko
Priority: Critical
Fix For: 2.1 beta2
Attachments: bdplab0.alternate.log, bdplab0.log,
bdplab0_cassandra.yaml
This is a physical four node cluster. Configuration is attached.
Create a keyspace and table from the first node:
{code}
CREATE KEYSPACE Keyspace1 WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '1'
};
USE Keyspace1;
CREATE TABLE Counter1 (
key blob,
column1 ascii,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={};
{code}
And the find the following in the logs:
{code}
INFO [Thrift:1] 2014-02-19 14:04:35,828 MigrationManager.java:210 - Create
new ColumnFamily:
org.apache.cassandra.config.CFMetaData@d824292[cfId=d1bc0c30-99b1-11e3-a5f9-c187ff8103e2,ksName=Keyspace1,cfName=Counter1,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.CounterColumnType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.CounterColumnType, kind=COMPACT_VALUE,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.AsciiType, kind=CLUSTERING_COLUMN,
componentIndex=null, indexName=null,
indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching=KEYS_ONLY,defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=NONE,populateIoCacheOnFlush=false,droppedColumns={},triggers={},rowsPerPartitionToCache=100]
ERROR [WRITE-/172.16.1.211] 2014-02-19 14:04:35,838
OutboundTcpConnection.java:256 - error writing to /172.16.1.211
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.cassandra.db.ArrayBackedSortedColumns.internalAppendOrReconcile(ArrayBackedSortedColumns.java:231)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:143)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:103)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.getColumnCount(ArrayBackedSortedColumns.java:313)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.contentSerializedSize(ColumnFamilySerializer.java:117)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.serializedSize(ColumnFamilySerializer.java:132)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:337)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:397)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:371)
~[main/:na]
at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:116)
~[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeInternal(OutboundTcpConnection.java:273)
[main/:na]
at
[jira] [Updated] (CASSANDRA-6742) Make it safe to concurrently access ABSC after its construction
[
https://issues.apache.org/jira/browse/CASSANDRA-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-6742:
-
Reproduced In: 2.1 beta1 (was: 2.1)
Make it safe to concurrently access ABSC after its construction
---
Key: CASSANDRA-6742
URL: https://issues.apache.org/jira/browse/CASSANDRA-6742
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Aleksey Yeschenko
Priority: Critical
Fix For: 2.1 beta2
Attachments: bdplab0.alternate.log, bdplab0.log,
bdplab0_cassandra.yaml
This is a physical four node cluster. Configuration is attached.
Create a keyspace and table from the first node:
{code}
CREATE KEYSPACE Keyspace1 WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '1'
};
USE Keyspace1;
CREATE TABLE Counter1 (
key blob,
column1 ascii,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={};
{code}
And the find the following in the logs:
{code}
INFO [Thrift:1] 2014-02-19 14:04:35,828 MigrationManager.java:210 - Create
new ColumnFamily:
org.apache.cassandra.config.CFMetaData@d824292[cfId=d1bc0c30-99b1-11e3-a5f9-c187ff8103e2,ksName=Keyspace1,cfName=Counter1,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.CounterColumnType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.CounterColumnType, kind=COMPACT_VALUE,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.AsciiType, kind=CLUSTERING_COLUMN,
componentIndex=null, indexName=null,
indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching=KEYS_ONLY,defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=NONE,populateIoCacheOnFlush=false,droppedColumns={},triggers={},rowsPerPartitionToCache=100]
ERROR [WRITE-/172.16.1.211] 2014-02-19 14:04:35,838
OutboundTcpConnection.java:256 - error writing to /172.16.1.211
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.cassandra.db.ArrayBackedSortedColumns.internalAppendOrReconcile(ArrayBackedSortedColumns.java:231)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:143)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:103)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.getColumnCount(ArrayBackedSortedColumns.java:313)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.contentSerializedSize(ColumnFamilySerializer.java:117)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.serializedSize(ColumnFamilySerializer.java:132)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:337)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:397)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:371)
~[main/:na]
at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:116)
~[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeInternal(OutboundTcpConnection.java:273)
[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:225)
[main/:na]
at
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907134#comment-13907134
]
Joshua McKenzie commented on CASSANDRA-6283:
It's possible the two are the same underlying problem. We use hardlinks both
during the snapshot process and during flushing of memtables - I'll have to
think for a bit on the best way to test that theory out since your stack shows
us the LEAK being cleaned up but doesn't (and can't) give us any indication of
who it is that might still hold the reference to the handle.
As for the tmp folder and file growth - the hard links won't be kept open in
the FD table for java so we shouldn't run the risk of an FD limit, however the
drive space would still be used until the process was shut down. I was
thinking the addition of a periodic delete tmp/moved_hard_links/* type pass
should help mitigate that, though it smells hacky to me.
Another option would be copying files instead of hard linking them during these
operations on Windows only, but I think the performance ramifications of that
make that option unworkable.
Windows 7 data files keept open / can't be deleted after compaction.
Key: CASSANDRA-6283
URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Windows 7 (32) / Java 1.7.0.45
Reporter: Andreas Schnitzerling
Assignee: Joshua McKenzie
Labels: compaction
Fix For: 2.0.6
Attachments: leakdetect.patch, screenshot-1.jpg, system.log
Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
is: Opened file handles seem to be lost and not closed properly. Win 7
blames, that another process is still using the file (but its obviously
cassandra). Only restart of the server makes the files deleted. But after
heavy using (changes) of tables, there are about 24K files in the data folder
(instead of 35 after every restart) and Cassandra crashes. I experiminted and
I found out, that a finalizer fixes the problem. So after GC the files will
be deleted (not optimal, but working fine). It runs now 2 days continously
without problem. Possible fix/test:
I wrote the following finalizer at the end of class
org.apache.cassandra.io.util.RandomAccessReader:
{code:title=RandomAccessReader.java|borderStyle=solid}
@Override
protected void finalize() throws Throwable {
deallocate();
super.finalize();
}
{code}
Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6742) Make it safe to concurrently access ABSC after its construction
[
https://issues.apache.org/jira/browse/CASSANDRA-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-6742:
-
Attachment: 6742-v2.txt
Make it safe to concurrently access ABSC after its construction
---
Key: CASSANDRA-6742
URL: https://issues.apache.org/jira/browse/CASSANDRA-6742
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Aleksey Yeschenko
Priority: Critical
Fix For: 2.1 beta2
Attachments: 6742-v2.txt, bdplab0.alternate.log, bdplab0.log,
bdplab0_cassandra.yaml
This is a physical four node cluster. Configuration is attached.
Create a keyspace and table from the first node:
{code}
CREATE KEYSPACE Keyspace1 WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '1'
};
USE Keyspace1;
CREATE TABLE Counter1 (
key blob,
column1 ascii,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={};
{code}
And the find the following in the logs:
{code}
INFO [Thrift:1] 2014-02-19 14:04:35,828 MigrationManager.java:210 - Create
new ColumnFamily:
org.apache.cassandra.config.CFMetaData@d824292[cfId=d1bc0c30-99b1-11e3-a5f9-c187ff8103e2,ksName=Keyspace1,cfName=Counter1,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.CounterColumnType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.CounterColumnType, kind=COMPACT_VALUE,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.AsciiType, kind=CLUSTERING_COLUMN,
componentIndex=null, indexName=null,
indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching=KEYS_ONLY,defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=NONE,populateIoCacheOnFlush=false,droppedColumns={},triggers={},rowsPerPartitionToCache=100]
ERROR [WRITE-/172.16.1.211] 2014-02-19 14:04:35,838
OutboundTcpConnection.java:256 - error writing to /172.16.1.211
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.cassandra.db.ArrayBackedSortedColumns.internalAppendOrReconcile(ArrayBackedSortedColumns.java:231)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:143)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:103)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.getColumnCount(ArrayBackedSortedColumns.java:313)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.contentSerializedSize(ColumnFamilySerializer.java:117)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.serializedSize(ColumnFamilySerializer.java:132)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:337)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:397)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:371)
~[main/:na]
at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:116)
~[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeInternal(OutboundTcpConnection.java:273)
[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:225)
[main/:na]
at
[jira] [Commented] (CASSANDRA-6742) Make it safe to concurrently access ABSC after its construction
[
https://issues.apache.org/jira/browse/CASSANDRA-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907143#comment-13907143
]
Aleksey Yeschenko commented on CASSANDRA-6742:
--
[~benedict] Moving CASSANDRA-6662 discussion here. Attaching a cleaned
up/rewritten version of the suggested patch, with
- changed Factory interface that accepts initialCapacity now, making it more
generic than 'unitFactory'. Should use it from now on when we know the final
size.
- altered sort() to not shadow *size*, and appendOrReconcile() to not mutate
the passed variable as well (and got rid of rightStart/rightEnd vars)
- moved fast add all path into a separate method, and got rid of all the
redundant *this*
Actually, I think we can do better than this, avoiding volatile altogether.
Will attach a v3 soon.
Make it safe to concurrently access ABSC after its construction
---
Key: CASSANDRA-6742
URL: https://issues.apache.org/jira/browse/CASSANDRA-6742
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Aleksey Yeschenko
Priority: Critical
Fix For: 2.1 beta2
Attachments: 6742-v2.txt, bdplab0.alternate.log, bdplab0.log,
bdplab0_cassandra.yaml
This is a physical four node cluster. Configuration is attached.
Create a keyspace and table from the first node:
{code}
CREATE KEYSPACE Keyspace1 WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '1'
};
USE Keyspace1;
CREATE TABLE Counter1 (
key blob,
column1 ascii,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={};
{code}
And the find the following in the logs:
{code}
INFO [Thrift:1] 2014-02-19 14:04:35,828 MigrationManager.java:210 - Create
new ColumnFamily:
org.apache.cassandra.config.CFMetaData@d824292[cfId=d1bc0c30-99b1-11e3-a5f9-c187ff8103e2,ksName=Keyspace1,cfName=Counter1,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.CounterColumnType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.CounterColumnType, kind=COMPACT_VALUE,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.AsciiType, kind=CLUSTERING_COLUMN,
componentIndex=null, indexName=null,
indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching=KEYS_ONLY,defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=NONE,populateIoCacheOnFlush=false,droppedColumns={},triggers={},rowsPerPartitionToCache=100]
ERROR [WRITE-/172.16.1.211] 2014-02-19 14:04:35,838
OutboundTcpConnection.java:256 - error writing to /172.16.1.211
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.cassandra.db.ArrayBackedSortedColumns.internalAppendOrReconcile(ArrayBackedSortedColumns.java:231)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:143)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:103)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.getColumnCount(ArrayBackedSortedColumns.java:313)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.contentSerializedSize(ColumnFamilySerializer.java:117)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.serializedSize(ColumnFamilySerializer.java:132)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:337)
~[main/:na]
at
[jira] [Updated] (CASSANDRA-6742) Make it safe to concurrently access ABSC after its construction
[
https://issues.apache.org/jira/browse/CASSANDRA-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-6742:
-
Attachment: 6742-v2.txt
Make it safe to concurrently access ABSC after its construction
---
Key: CASSANDRA-6742
URL: https://issues.apache.org/jira/browse/CASSANDRA-6742
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Aleksey Yeschenko
Priority: Critical
Fix For: 2.1 beta2
Attachments: 6742-v2.txt, 6742-v2.txt, bdplab0.alternate.log,
bdplab0.log, bdplab0_cassandra.yaml
This is a physical four node cluster. Configuration is attached.
Create a keyspace and table from the first node:
{code}
CREATE KEYSPACE Keyspace1 WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '1'
};
USE Keyspace1;
CREATE TABLE Counter1 (
key blob,
column1 ascii,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={};
{code}
And the find the following in the logs:
{code}
INFO [Thrift:1] 2014-02-19 14:04:35,828 MigrationManager.java:210 - Create
new ColumnFamily:
org.apache.cassandra.config.CFMetaData@d824292[cfId=d1bc0c30-99b1-11e3-a5f9-c187ff8103e2,ksName=Keyspace1,cfName=Counter1,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.CounterColumnType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.CounterColumnType, kind=COMPACT_VALUE,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.AsciiType, kind=CLUSTERING_COLUMN,
componentIndex=null, indexName=null,
indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching=KEYS_ONLY,defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=NONE,populateIoCacheOnFlush=false,droppedColumns={},triggers={},rowsPerPartitionToCache=100]
ERROR [WRITE-/172.16.1.211] 2014-02-19 14:04:35,838
OutboundTcpConnection.java:256 - error writing to /172.16.1.211
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.cassandra.db.ArrayBackedSortedColumns.internalAppendOrReconcile(ArrayBackedSortedColumns.java:231)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:143)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:103)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.getColumnCount(ArrayBackedSortedColumns.java:313)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.contentSerializedSize(ColumnFamilySerializer.java:117)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.serializedSize(ColumnFamilySerializer.java:132)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:337)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:397)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:371)
~[main/:na]
at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:116)
~[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeInternal(OutboundTcpConnection.java:273)
[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:225)
[main/:na]
at
[jira] [Reopened] (CASSANDRA-6742) Make it safe to concurrently access ABSC after its construction
[
https://issues.apache.org/jira/browse/CASSANDRA-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko reopened CASSANDRA-6742:
--
Reopening, b/c while the issue is fixed, we can do better.
Make it safe to concurrently access ABSC after its construction
---
Key: CASSANDRA-6742
URL: https://issues.apache.org/jira/browse/CASSANDRA-6742
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Aleksey Yeschenko
Priority: Critical
Fix For: 2.1 beta2
Attachments: 6742-v2.txt, bdplab0.alternate.log, bdplab0.log,
bdplab0_cassandra.yaml
This is a physical four node cluster. Configuration is attached.
Create a keyspace and table from the first node:
{code}
CREATE KEYSPACE Keyspace1 WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '1'
};
USE Keyspace1;
CREATE TABLE Counter1 (
key blob,
column1 ascii,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={};
{code}
And the find the following in the logs:
{code}
INFO [Thrift:1] 2014-02-19 14:04:35,828 MigrationManager.java:210 - Create
new ColumnFamily:
org.apache.cassandra.config.CFMetaData@d824292[cfId=d1bc0c30-99b1-11e3-a5f9-c187ff8103e2,ksName=Keyspace1,cfName=Counter1,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.CounterColumnType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.CounterColumnType, kind=COMPACT_VALUE,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.AsciiType, kind=CLUSTERING_COLUMN,
componentIndex=null, indexName=null,
indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching=KEYS_ONLY,defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=NONE,populateIoCacheOnFlush=false,droppedColumns={},triggers={},rowsPerPartitionToCache=100]
ERROR [WRITE-/172.16.1.211] 2014-02-19 14:04:35,838
OutboundTcpConnection.java:256 - error writing to /172.16.1.211
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.cassandra.db.ArrayBackedSortedColumns.internalAppendOrReconcile(ArrayBackedSortedColumns.java:231)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:143)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:103)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.getColumnCount(ArrayBackedSortedColumns.java:313)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.contentSerializedSize(ColumnFamilySerializer.java:117)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.serializedSize(ColumnFamilySerializer.java:132)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:337)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:397)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:371)
~[main/:na]
at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:116)
~[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeInternal(OutboundTcpConnection.java:273)
[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:225)
[main/:na]
at
[jira] [Commented] (CASSANDRA-6742) Make it safe to concurrently access ABSC after its construction
[
https://issues.apache.org/jira/browse/CASSANDRA-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907152#comment-13907152
]
Benedict commented on CASSANDRA-6742:
-
LGTM so far, though I might have appendOrReconcile encapsulate applying -1 when
necessary, to keep the merge consistent in sortCells().
Make it safe to concurrently access ABSC after its construction
---
Key: CASSANDRA-6742
URL: https://issues.apache.org/jira/browse/CASSANDRA-6742
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Aleksey Yeschenko
Priority: Critical
Fix For: 2.1 beta2
Attachments: 6742-v2.txt, 6742-v2.txt, bdplab0.alternate.log,
bdplab0.log, bdplab0_cassandra.yaml
This is a physical four node cluster. Configuration is attached.
Create a keyspace and table from the first node:
{code}
CREATE KEYSPACE Keyspace1 WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '1'
};
USE Keyspace1;
CREATE TABLE Counter1 (
key blob,
column1 ascii,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={};
{code}
And the find the following in the logs:
{code}
INFO [Thrift:1] 2014-02-19 14:04:35,828 MigrationManager.java:210 - Create
new ColumnFamily:
org.apache.cassandra.config.CFMetaData@d824292[cfId=d1bc0c30-99b1-11e3-a5f9-c187ff8103e2,ksName=Keyspace1,cfName=Counter1,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.CounterColumnType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.CounterColumnType, kind=COMPACT_VALUE,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.AsciiType, kind=CLUSTERING_COLUMN,
componentIndex=null, indexName=null,
indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching=KEYS_ONLY,defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=NONE,populateIoCacheOnFlush=false,droppedColumns={},triggers={},rowsPerPartitionToCache=100]
ERROR [WRITE-/172.16.1.211] 2014-02-19 14:04:35,838
OutboundTcpConnection.java:256 - error writing to /172.16.1.211
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.cassandra.db.ArrayBackedSortedColumns.internalAppendOrReconcile(ArrayBackedSortedColumns.java:231)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:143)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:103)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.getColumnCount(ArrayBackedSortedColumns.java:313)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.contentSerializedSize(ColumnFamilySerializer.java:117)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.serializedSize(ColumnFamilySerializer.java:132)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:337)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:397)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:371)
~[main/:na]
at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:116)
~[main/:na]
at
org.apache.cassandra.net.OutboundTcpConnection.writeInternal(OutboundTcpConnection.java:273)
[main/:na]
at
[jira] [Updated] (CASSANDRA-6381) Refactor nodetool
[ https://issues.apache.org/jira/browse/CASSANDRA-6381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna updated CASSANDRA-6381: Fix Version/s: 2.1 beta1 Refactor nodetool - Key: CASSANDRA-6381 URL: https://issues.apache.org/jira/browse/CASSANDRA-6381 Project: Cassandra Issue Type: Improvement Reporter: Yuki Morishita Assignee: Clément Lardeur Priority: Minor Labels: lhf, nodetool Fix For: 2.1 beta1 Attachments: trunk-6381-fix-windows-nodetool-script.patch, trunk-6381-v2.patch, trunk-6381-v3.patch, trunk-6381-v4.patch, trunk-6381.patch We have way too many nodetool commands(more than 40) packed in one NodeCmd class. And we are trying to add more commands. https://github.com/airlift/airline could be a good fit to take out each command into sub command class. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[3/6] Merge branch 'cassandra-2.0' into cassandra-2.1
http://git-wip-us.apache.org/repos/asf/cassandra/blob/63b1ef4e/src/java/org/apache/cassandra/db/composites/CompoundSparseCellNameType.java
--
diff --cc
src/java/org/apache/cassandra/db/composites/CompoundSparseCellNameType.java
index d0e66f8,000..e0cbc0f
mode 100644,00..100644
---
a/src/java/org/apache/cassandra/db/composites/CompoundSparseCellNameType.java
+++
b/src/java/org/apache/cassandra/db/composites/CompoundSparseCellNameType.java
@@@ -1,245 -1,0 +1,301 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.db.composites;
+
+import java.nio.ByteBuffer;
+import java.util.*;
+
++import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.cql3.CQL3Row;
+import org.apache.cassandra.cql3.ColumnIdentifier;
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.marshal.CollectionType;
+import org.apache.cassandra.db.marshal.ColumnToCollectionType;
+import org.apache.cassandra.db.marshal.UTF8Type;
+import org.apache.cassandra.utils.ByteBufferUtil;
++import org.apache.cassandra.utils.memory.AbstractAllocator;
++import org.apache.cassandra.utils.memory.PoolAllocator;
+
+public class CompoundSparseCellNameType extends AbstractCompoundCellNameType
+{
+private static final ColumnIdentifier rowMarkerId = new
ColumnIdentifier(ByteBufferUtil.EMPTY_BYTE_BUFFER, UTF8Type.instance);
- private static final CellName rowMarkerNoPrefix = new
CompoundSparseCellName(rowMarkerId);
++private static final CellName rowMarkerNoPrefix = new
CompoundSparseCellName(rowMarkerId, false);
+
+// For CQL3 columns, this is always UTF8Type. However, for compatibility
with super columns, we need to allow it to be non-UTF8.
+private final AbstractType? columnNameType;
+protected final MapByteBuffer, ColumnIdentifier internedIds;
+
++private final Composite staticPrefix;
++
+public CompoundSparseCellNameType(ListAbstractType? types)
+{
+this(types, UTF8Type.instance);
+}
+
+public CompoundSparseCellNameType(ListAbstractType? types,
AbstractType? columnNameType)
+{
+this(new CompoundCType(types), columnNameType);
+}
+
+private CompoundSparseCellNameType(CompoundCType clusteringType,
AbstractType? columnNameType)
+{
+this(clusteringType, columnNameType, makeCType(clusteringType,
columnNameType, null), new HashMapByteBuffer, ColumnIdentifier());
+}
+
+private CompoundSparseCellNameType(CompoundCType clusteringType,
AbstractType? columnNameType, CompoundCType fullType, MapByteBuffer,
ColumnIdentifier internedIds)
+{
+super(clusteringType, fullType);
+this.columnNameType = columnNameType;
+this.internedIds = internedIds;
++this.staticPrefix = makeStaticPrefix(clusteringType.size());
++}
++
++private static Composite makeStaticPrefix(int size)
++{
++ByteBuffer[] elements = new ByteBuffer[size];
++for (int i = 0; i size; i++)
++elements[i] = ByteBufferUtil.EMPTY_BYTE_BUFFER;
++
++return new CompoundComposite(elements, size, true)
++{
++@Override
++public boolean isStatic()
++{
++return true;
++}
++
++@Override
++public long unsharedHeapSize()
++{
++// We'll share this for a given type.
++return 0;
++}
++
++@Override
++public Composite copy(AbstractAllocator allocator)
++{
++return this;
++}
++
++@Override
++public void free(PoolAllocator? allocator)
++{
++}
++};
+}
+
+protected static CompoundCType makeCType(CompoundCType clusteringType,
AbstractType? columnNameType, ColumnToCollectionType collectionType)
+{
+ListAbstractType? allSubtypes = new
ArrayListAbstractType?(clusteringType.size() + (collectionType == null ? 1
: 2));
+for (int i = 0; i clusteringType.size(); i++)
+
[7/7] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4d75ed20 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4d75ed20 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4d75ed20 Branch: refs/heads/trunk Commit: 4d75ed206c4df5f9cb880b8a1133f4b5ccdd6ecf Parents: 1debf25 63b1ef4 Author: Sylvain Lebresne sylv...@datastax.com Authored: Thu Feb 20 17:51:44 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Thu Feb 20 17:51:44 2014 +0100 -- CHANGES.txt | 1 + doc/cql3/CQL.textile| 29 +- .../org/apache/cassandra/config/CFMetaData.java | 30 +- .../cassandra/config/ColumnDefinition.java | 16 + src/java/org/apache/cassandra/cql3/CQL3Row.java | 7 +- .../apache/cassandra/cql3/ColumnCondition.java | 183 ++ .../org/apache/cassandra/cql3/Constants.java| 8 +- src/java/org/apache/cassandra/cql3/Cql.g| 29 +- src/java/org/apache/cassandra/cql3/Lists.java | 12 +- src/java/org/apache/cassandra/cql3/Maps.java| 14 +- src/java/org/apache/cassandra/cql3/Sets.java| 12 +- .../cql3/statements/AlterTableStatement.java| 30 +- .../cql3/statements/BatchStatement.java | 162 +++-- .../cql3/statements/CQL3CasConditions.java | 164 + .../cql3/statements/CreateIndexStatement.java | 9 + .../cql3/statements/CreateTableStatement.java | 41 ++- .../cql3/statements/DeleteStatement.java| 23 +- .../cql3/statements/ModificationStatement.java | 330 ++- .../cql3/statements/SelectStatement.java| 238 +++-- .../cassandra/cql3/statements/Selection.java| 29 +- .../cql3/statements/UpdateStatement.java| 20 +- .../org/apache/cassandra/db/CFRowAdder.java | 10 +- .../cassandra/db/composites/AbstractCType.java | 7 +- .../db/composites/AbstractCellNameType.java | 188 +++ .../db/composites/AbstractComposite.java| 17 +- .../AbstractCompoundCellNameType.java | 37 ++- .../db/composites/BoundedComposite.java | 5 + .../cassandra/db/composites/CellNameType.java | 21 +- .../cassandra/db/composites/Composite.java | 2 + .../cassandra/db/composites/Composites.java | 5 + .../cassandra/db/composites/CompoundCType.java | 14 +- .../db/composites/CompoundComposite.java| 16 +- .../db/composites/CompoundDenseCellName.java| 2 +- .../composites/CompoundDenseCellNameType.java | 14 +- .../db/composites/CompoundSparseCellName.java | 34 +- .../composites/CompoundSparseCellNameType.java | 98 -- .../db/composites/SimpleDenseCellNameType.java | 5 +- .../db/composites/SimpleSparseCellNameType.java | 7 +- .../cassandra/db/filter/ExtendedFilter.java | 10 +- .../CompositesIndexOnCollectionKey.java | 2 +- .../CompositesIndexOnCollectionValue.java | 2 +- .../composites/CompositesIndexOnRegular.java| 2 +- .../db/index/composites/CompositesSearcher.java | 25 +- .../db/marshal/AbstractCompositeType.java | 47 +-- .../cassandra/db/marshal/CompositeType.java | 72 +++- .../db/marshal/DynamicCompositeType.java| 6 + .../hadoop/pig/AbstractCassandraStorage.java| 3 +- .../apache/cassandra/service/CASConditions.java | 3 +- 48 files changed, 1552 insertions(+), 489 deletions(-) --
[3/7] Merge branch 'cassandra-2.0' into cassandra-2.1
http://git-wip-us.apache.org/repos/asf/cassandra/blob/63b1ef4e/src/java/org/apache/cassandra/db/composites/CompoundSparseCellNameType.java
--
diff --cc
src/java/org/apache/cassandra/db/composites/CompoundSparseCellNameType.java
index d0e66f8,000..e0cbc0f
mode 100644,00..100644
---
a/src/java/org/apache/cassandra/db/composites/CompoundSparseCellNameType.java
+++
b/src/java/org/apache/cassandra/db/composites/CompoundSparseCellNameType.java
@@@ -1,245 -1,0 +1,301 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.db.composites;
+
+import java.nio.ByteBuffer;
+import java.util.*;
+
++import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.cql3.CQL3Row;
+import org.apache.cassandra.cql3.ColumnIdentifier;
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.marshal.CollectionType;
+import org.apache.cassandra.db.marshal.ColumnToCollectionType;
+import org.apache.cassandra.db.marshal.UTF8Type;
+import org.apache.cassandra.utils.ByteBufferUtil;
++import org.apache.cassandra.utils.memory.AbstractAllocator;
++import org.apache.cassandra.utils.memory.PoolAllocator;
+
+public class CompoundSparseCellNameType extends AbstractCompoundCellNameType
+{
+private static final ColumnIdentifier rowMarkerId = new
ColumnIdentifier(ByteBufferUtil.EMPTY_BYTE_BUFFER, UTF8Type.instance);
- private static final CellName rowMarkerNoPrefix = new
CompoundSparseCellName(rowMarkerId);
++private static final CellName rowMarkerNoPrefix = new
CompoundSparseCellName(rowMarkerId, false);
+
+// For CQL3 columns, this is always UTF8Type. However, for compatibility
with super columns, we need to allow it to be non-UTF8.
+private final AbstractType? columnNameType;
+protected final MapByteBuffer, ColumnIdentifier internedIds;
+
++private final Composite staticPrefix;
++
+public CompoundSparseCellNameType(ListAbstractType? types)
+{
+this(types, UTF8Type.instance);
+}
+
+public CompoundSparseCellNameType(ListAbstractType? types,
AbstractType? columnNameType)
+{
+this(new CompoundCType(types), columnNameType);
+}
+
+private CompoundSparseCellNameType(CompoundCType clusteringType,
AbstractType? columnNameType)
+{
+this(clusteringType, columnNameType, makeCType(clusteringType,
columnNameType, null), new HashMapByteBuffer, ColumnIdentifier());
+}
+
+private CompoundSparseCellNameType(CompoundCType clusteringType,
AbstractType? columnNameType, CompoundCType fullType, MapByteBuffer,
ColumnIdentifier internedIds)
+{
+super(clusteringType, fullType);
+this.columnNameType = columnNameType;
+this.internedIds = internedIds;
++this.staticPrefix = makeStaticPrefix(clusteringType.size());
++}
++
++private static Composite makeStaticPrefix(int size)
++{
++ByteBuffer[] elements = new ByteBuffer[size];
++for (int i = 0; i size; i++)
++elements[i] = ByteBufferUtil.EMPTY_BYTE_BUFFER;
++
++return new CompoundComposite(elements, size, true)
++{
++@Override
++public boolean isStatic()
++{
++return true;
++}
++
++@Override
++public long unsharedHeapSize()
++{
++// We'll share this for a given type.
++return 0;
++}
++
++@Override
++public Composite copy(AbstractAllocator allocator)
++{
++return this;
++}
++
++@Override
++public void free(PoolAllocator? allocator)
++{
++}
++};
+}
+
+protected static CompoundCType makeCType(CompoundCType clusteringType,
AbstractType? columnNameType, ColumnToCollectionType collectionType)
+{
+ListAbstractType? allSubtypes = new
ArrayListAbstractType?(clusteringType.size() + (collectionType == null ? 1
: 2));
+for (int i = 0; i clusteringType.size(); i++)
+
[jira] [Updated] (CASSANDRA-6695) Don't exchange schema between nodes with different versions (no pull, no push)
[ https://issues.apache.org/jira/browse/CASSANDRA-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-6695: --- Fix Version/s: 2.0.6 1.2.16 Don't exchange schema between nodes with different versions (no pull, no push) -- Key: CASSANDRA-6695 URL: https://issues.apache.org/jira/browse/CASSANDRA-6695 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Piotr Kołaczkowski Assignee: Aleksey Yeschenko Fix For: 1.2.16, 2.0.6, 2.1 beta1 Attachments: 6695-v2.txt, 6695.patch Subject. Don't push schema to unknown-, or differently major-versioned nodes, and don't pull schema from them, either. Since we don't support schema altering during upgrade, and adding nodes during cluster upgrades is also a non-recommended thing, this is what we are going to do. Until CASSANDRA-6038, that is. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6649) CQL: disallow counter update with USING TIMESTAMP and USING TTL
[ https://issues.apache.org/jira/browse/CASSANDRA-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-6649: --- Fix Version/s: 2.0.6 1.2.16 CQL: disallow counter update with USING TIMESTAMP and USING TTL --- Key: CASSANDRA-6649 URL: https://issues.apache.org/jira/browse/CASSANDRA-6649 Project: Cassandra Issue Type: Bug Reporter: Sylvain Lebresne Assignee: Aleksey Yeschenko Priority: Trivial Fix For: 1.2.16, 2.0.6, 2.1 beta1 Attachments: 6649-2.0.txt, 6649-2.1.txt Timestamps are not used by counters and TTL are not supported, but it appears we don't reject counter updates that have USING TIMESTAMP X or USING TTL X. We should since both are non-sensical (the value is completely ignored currently). Note: we should also refuse USING TIMESTAMP on DELETE statements on counters table: even though we kind of do use a timestamp internally, it's more of an implementation detail and in fact may go away with CASSANDRA-6506 (there is also nothing clever you can do with it by providing it client side). Note bis: strictly speaking doing that could break a few users that where setting those thinking it does something. I think that the lack of validation is more of a bug and that user that think it's doing something probably ought to know it's not sooner than later, but I could be fine with just warning in the log file for 1.2 and 2.0, and only rejecting in 2.1 if someone thinks it's safer. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6713) Snapshot based repair does not send snapshot command to itself
[ https://issues.apache.org/jira/browse/CASSANDRA-6713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-6713: --- Fix Version/s: 2.0.6 1.2.16 Snapshot based repair does not send snapshot command to itself -- Key: CASSANDRA-6713 URL: https://issues.apache.org/jira/browse/CASSANDRA-6713 Project: Cassandra Issue Type: Bug Reporter: sankalp kohli Assignee: Yuki Morishita Fix For: 1.2.16, 2.0.6, 2.1 beta1 Attachments: 6713-1.2.txt Due to this, the Merkle tree created will differ a lot causing lot of streaming to happen. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-5631) NPE when creating column family shortly after multinode startup
[
https://issues.apache.org/jira/browse/CASSANDRA-5631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremiah Jordan updated CASSANDRA-5631:
---
Fix Version/s: 2.0.6
1.2.16
NPE when creating column family shortly after multinode startup
---
Key: CASSANDRA-5631
URL: https://issues.apache.org/jira/browse/CASSANDRA-5631
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.2.0
Reporter: Martin Serrano
Assignee: Aleksey Yeschenko
Fix For: 1.2.16, 2.0.6, 2.1 beta1
Attachments: 5631.txt
I'm testing a 2-node cluster and creating a column family right after the
nodes startup. I am using the Astyanax client. Sometimes column family
creation fails and I see NPEs on the cassandra server:
{noformat}
2013-06-12 14:55:31,773 ERROR CassandraDaemon [MigrationStage:1] - Exception
in thread Thread[MigrationStage:1,5,main]
java.lang.NullPointerException
at org.apache.cassandra.db.DefsTable.addColumnFamily(DefsTable.java:510)
at
org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:444)
at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:354)
at
org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:55)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
{noformat}
{noformat}
2013-06-12 14:55:31,880 ERROR CassandraDaemon [MigrationStage:1] - Exception
in thread Thread[MigrationStage:1,5,main]
java.lang.NullPointerException
at
org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:475)
at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:354)
at
org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:55)
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6700) Use real node messaging versions for schema exchange decisions
[ https://issues.apache.org/jira/browse/CASSANDRA-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-6700: --- Fix Version/s: 2.0.6 1.2.16 Use real node messaging versions for schema exchange decisions -- Key: CASSANDRA-6700 URL: https://issues.apache.org/jira/browse/CASSANDRA-6700 Project: Cassandra Issue Type: Bug Components: Core Reporter: Piotr Kołaczkowski Assignee: Aleksey Yeschenko Fix For: 1.2.16, 2.0.6, 2.1 beta1 Attachments: 6700.txt IncomingTcpConnection#handleModernVersion sets version to min(my version, version of the peer). This messes up schema pull/push. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6742) Make it safe to concurrently access ABSC after its construction
[
https://issues.apache.org/jira/browse/CASSANDRA-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-6742:
-
Attachment: 6742-v3.txt
We add in sorted order a lot, especially on reads, thus if making size volatile
is an issue, so is making unsortedSize volatile.
So v3 introduces a separate boolean volatile isSorted flag that gets updated
only when we break the sorting order or sort the cells (in other words, almost
never), and reverts all the complexity added by the recent revisions.
Make it safe to concurrently access ABSC after its construction
---
Key: CASSANDRA-6742
URL: https://issues.apache.org/jira/browse/CASSANDRA-6742
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Aleksey Yeschenko
Priority: Critical
Fix For: 2.1 beta2
Attachments: 6742-v2.txt, 6742-v2.txt, 6742-v3.txt,
bdplab0.alternate.log, bdplab0.log, bdplab0_cassandra.yaml
This is a physical four node cluster. Configuration is attached.
Create a keyspace and table from the first node:
{code}
CREATE KEYSPACE Keyspace1 WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '1'
};
USE Keyspace1;
CREATE TABLE Counter1 (
key blob,
column1 ascii,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={};
{code}
And the find the following in the logs:
{code}
INFO [Thrift:1] 2014-02-19 14:04:35,828 MigrationManager.java:210 - Create
new ColumnFamily:
org.apache.cassandra.config.CFMetaData@d824292[cfId=d1bc0c30-99b1-11e3-a5f9-c187ff8103e2,ksName=Keyspace1,cfName=Counter1,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.CounterColumnType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.CounterColumnType, kind=COMPACT_VALUE,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.AsciiType, kind=CLUSTERING_COLUMN,
componentIndex=null, indexName=null,
indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching=KEYS_ONLY,defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=NONE,populateIoCacheOnFlush=false,droppedColumns={},triggers={},rowsPerPartitionToCache=100]
ERROR [WRITE-/172.16.1.211] 2014-02-19 14:04:35,838
OutboundTcpConnection.java:256 - error writing to /172.16.1.211
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.cassandra.db.ArrayBackedSortedColumns.internalAppendOrReconcile(ArrayBackedSortedColumns.java:231)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:143)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:103)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.getColumnCount(ArrayBackedSortedColumns.java:313)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.contentSerializedSize(ColumnFamilySerializer.java:117)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.serializedSize(ColumnFamilySerializer.java:132)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:337)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:397)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:371)
~[main/:na]
at
[jira] [Updated] (CASSANDRA-6714) Fix replaying old (1.2) commitlog in Cassandra 2.0
[ https://issues.apache.org/jira/browse/CASSANDRA-6714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-6714: --- Fix Version/s: 2.0.6 Fix replaying old (1.2) commitlog in Cassandra 2.0 -- Key: CASSANDRA-6714 URL: https://issues.apache.org/jira/browse/CASSANDRA-6714 Project: Cassandra Issue Type: Bug Components: Core Environment: Old node: Cassandra 1.2.15 (DSE) New node: Cassandra 2.0.5.1 (DSE) Reporter: Piotr Kołaczkowski Assignee: Aleksey Yeschenko Fix For: 2.0.6, 2.1 beta1 Attachments: 6714.txt Our docs, and code, both explicitly say that you should drain a node before upgrading to a new major release. If you don't do what the docs explicitly tell you to do, however, Cassandra won't scream at you. Also, we *do* currently have logic to replay 1.2 commitlog in 2.0, but it seems to be slightly broken, unfortunately. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6711) SecondaryIndexManager#deleteFromIndexes() doesn't correctly retrieve column indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-6711: --- Fix Version/s: 2.0.6 1.2.16 SecondaryIndexManager#deleteFromIndexes() doesn't correctly retrieve column indexes --- Key: CASSANDRA-6711 URL: https://issues.apache.org/jira/browse/CASSANDRA-6711 Project: Cassandra Issue Type: Bug Components: Core Reporter: Sergio Bossa Assignee: Sergio Bossa Fix For: 1.2.16, 2.0.6, 2.1 beta1 Attachments: CASSANDRA-6711-1.2.patch, CASSANDRA-6711-2.0.patch SecondaryIndexManager#deleteFromIndexes() tries to retrieve the index class from indexesByColumn by using the raw column name, but indexes are inserted in indexesByColumn by using the ColumnDefinition name. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6742) Make it safe to concurrently access ABSC after its construction
[
https://issues.apache.org/jira/browse/CASSANDRA-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907208#comment-13907208
]
Benedict commented on CASSANDRA-6742:
-
Like it.
Only question is fastAddAll: you've limited it to only support array backed,
but we sometimes build directly from BTree, and the second path works for any
source CF type (skipping N comparisons, since we know there will be no
reconcile necessary if building from a CF)
Make it safe to concurrently access ABSC after its construction
---
Key: CASSANDRA-6742
URL: https://issues.apache.org/jira/browse/CASSANDRA-6742
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Aleksey Yeschenko
Priority: Critical
Fix For: 2.1 beta2
Attachments: 6742-v2.txt, 6742-v2.txt, 6742-v3.txt,
bdplab0.alternate.log, bdplab0.log, bdplab0_cassandra.yaml
This is a physical four node cluster. Configuration is attached.
Create a keyspace and table from the first node:
{code}
CREATE KEYSPACE Keyspace1 WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '1'
};
USE Keyspace1;
CREATE TABLE Counter1 (
key blob,
column1 ascii,
value counter,
PRIMARY KEY (key, column1)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.10 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='NONE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={};
{code}
And the find the following in the logs:
{code}
INFO [Thrift:1] 2014-02-19 14:04:35,828 MigrationManager.java:210 - Create
new ColumnFamily:
org.apache.cassandra.config.CFMetaData@d824292[cfId=d1bc0c30-99b1-11e3-a5f9-c187ff8103e2,ksName=Keyspace1,cfName=Counter1,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.CounterColumnType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.CounterColumnType, kind=COMPACT_VALUE,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.AsciiType, kind=CLUSTERING_COLUMN,
componentIndex=null, indexName=null,
indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={},bloomFilterFpChance=0.01,memtableFlushPeriod=0,caching=KEYS_ONLY,defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=NONE,populateIoCacheOnFlush=false,droppedColumns={},triggers={},rowsPerPartitionToCache=100]
ERROR [WRITE-/172.16.1.211] 2014-02-19 14:04:35,838
OutboundTcpConnection.java:256 - error writing to /172.16.1.211
java.lang.ArrayIndexOutOfBoundsException: -1
at
org.apache.cassandra.db.ArrayBackedSortedColumns.internalAppendOrReconcile(ArrayBackedSortedColumns.java:231)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:143)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:103)
~[main/:na]
at
org.apache.cassandra.db.ArrayBackedSortedColumns.getColumnCount(ArrayBackedSortedColumns.java:313)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.contentSerializedSize(ColumnFamilySerializer.java:117)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.serializedSize(ColumnFamilySerializer.java:132)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.serializedSize(Mutation.java:337)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:397)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$MigrationsSerializer.serializedSize(MigrationManager.java:371)
~[main/:na]
at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:116)
~[main/:na]
[jira] [Updated] (CASSANDRA-6564) Gossiper failed with ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CASSANDRA-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremiah Jordan updated CASSANDRA-6564:
---
Fix Version/s: 1.2.14
Gossiper failed with ArrayIndexOutOfBoundsException
---
Key: CASSANDRA-6564
URL: https://issues.apache.org/jira/browse/CASSANDRA-6564
Project: Cassandra
Issue Type: Bug
Reporter: Shao-Chuan Wang
Assignee: Tyler Hobbs
Priority: Minor
Labels: gossip
Fix For: 1.2.14, 2.1 beta1
Attachments: 6564-v1.patch
We have 30 nodes in one DC, 25 nodes in another. We are running 2.0.1.
Two nodes are joining the ring, but one of them failed with
ArrayIndexOutOfBoundsException:
{noformat}
java.lang.ArrayIndexOutOfBoundsException: 2
at
org.apache.cassandra.service.StorageService.extractExpireTime(StorageService.java:1594)
at
org.apache.cassandra.service.StorageService.handleStateRemoving(StorageService.java:1550)
at
org.apache.cassandra.service.StorageService.onChange(StorageService.java:1174)
at
org.apache.cassandra.service.StorageService.onJoin(StorageService.java:1887)
at
org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:844)
at
org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:922)
at
org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6564) Gossiper failed with ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CASSANDRA-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremiah Jordan updated CASSANDRA-6564:
---
Fix Version/s: 2.0.5
Gossiper failed with ArrayIndexOutOfBoundsException
---
Key: CASSANDRA-6564
URL: https://issues.apache.org/jira/browse/CASSANDRA-6564
Project: Cassandra
Issue Type: Bug
Reporter: Shao-Chuan Wang
Assignee: Tyler Hobbs
Priority: Minor
Labels: gossip
Fix For: 1.2.14, 2.0.5, 2.1 beta1
Attachments: 6564-v1.patch
We have 30 nodes in one DC, 25 nodes in another. We are running 2.0.1.
Two nodes are joining the ring, but one of them failed with
ArrayIndexOutOfBoundsException:
{noformat}
java.lang.ArrayIndexOutOfBoundsException: 2
at
org.apache.cassandra.service.StorageService.extractExpireTime(StorageService.java:1594)
at
org.apache.cassandra.service.StorageService.handleStateRemoving(StorageService.java:1550)
at
org.apache.cassandra.service.StorageService.onChange(StorageService.java:1174)
at
org.apache.cassandra.service.StorageService.onJoin(StorageService.java:1887)
at
org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:844)
at
org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:922)
at
org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:49)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[1/3] git commit: Fix ABTC NPE patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6692
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 63b1ef4ee - 5223c4797
refs/heads/trunk 4d75ed206 - 7c730279e
Fix ABTC NPE
patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6692
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5223c479
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5223c479
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5223c479
Branch: refs/heads/cassandra-2.1
Commit: 5223c4797cbb46d5df427d72bb115f96af342d60
Parents: 63b1ef4
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Feb 20 11:36:34 2014 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Feb 20 11:36:48 2014 -0600
--
CHANGES.txt | 2 ++
.../apache/cassandra/db/AtomicBTreeColumns.java | 14
.../org/apache/cassandra/utils/btree/BTree.java | 24
3 files changed, 30 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5223c479/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index feece93..efc54f8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,7 +1,9 @@
2.1.0-beta2
+ * Fix ABTC NPE (CASSANDRA-6692)
* Allow nodetool to use a file or prompt for password (CASSANDRA-6660)
* Fix AIOOBE when concurrently accessing ABSC (CASSANDRA-6742)
+
2.1.0-beta1
* Add flush directory distinct from compaction directories (CASSANDRA-6357)
* Require JNA by default (CASSANDRA-6575)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5223c479/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
index 5f56326..f75efd2 100644
--- a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
@@ -230,25 +230,29 @@ public class AtomicBTreeColumns extends ColumnFamily
Holder current = ref;
delta.reset();
-DeletionInfo deletionInfo = cm.deletionInfo();
-if (deletionInfo.mayModify(current.deletionInfo))
+DeletionInfo deletionInfo;
+if (cm.deletionInfo().mayModify(current.deletionInfo))
{
-if (deletionInfo.hasRanges())
+if (cm.deletionInfo().hasRanges())
{
for (IteratorCell iter : new Iterator[] {
insert.iterator(), BTree.Cellslice(current.tree, true) })
{
while (iter.hasNext())
{
Cell col = iter.next();
-if (deletionInfo.isDeleted(col))
+if (cm.deletionInfo().isDeleted(col))
indexer.remove(col);
}
}
}
-deletionInfo = current.deletionInfo.copy().add(deletionInfo);
+deletionInfo =
current.deletionInfo.copy().add(cm.deletionInfo());
delta.addHeapSize(deletionInfo.unsharedHeapSize() -
current.deletionInfo.unsharedHeapSize());
}
+else
+{
+deletionInfo = current.deletionInfo;
+}
ColumnUpdater updater = new ColumnUpdater(this, current,
allocator, transformation, indexer, delta);
Object[] tree = BTree.update(current.tree,
metadata.comparator.columnComparator(), insert, true, updater);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5223c479/src/java/org/apache/cassandra/utils/btree/BTree.java
--
diff --git a/src/java/org/apache/cassandra/utils/btree/BTree.java
b/src/java/org/apache/cassandra/utils/btree/BTree.java
index 82f5574..69cf145 100644
--- a/src/java/org/apache/cassandra/utils/btree/BTree.java
+++ b/src/java/org/apache/cassandra/utils/btree/BTree.java
@@ -18,9 +18,11 @@
*/
package org.apache.cassandra.utils.btree;
+import java.util.ArrayDeque;
import java.util.Arrays;
import java.util.Collection;
import java.util.Comparator;
+import java.util.Queue;
import org.apache.cassandra.utils.ObjectSizes;
@@ -113,7 +115,13 @@ public class BTree
if (!sorted)
source = sorted(source, comparator, size);
-return modifier.get().build(source, size);
+QueueBuilder queue = modifier.get();
+Builder builder = queue.poll();
+if (builder == null)
+builder = new Builder();
+Object[] btree = builder.build(source, size);
+
[2/3] git commit: Fix ABTC NPE patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6692
Fix ABTC NPE
patch by Benedict Elliott Smith; reviewed by jbellis for CASSANDRA-6692
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5223c479
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5223c479
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5223c479
Branch: refs/heads/trunk
Commit: 5223c4797cbb46d5df427d72bb115f96af342d60
Parents: 63b1ef4
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu Feb 20 11:36:34 2014 -0600
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu Feb 20 11:36:48 2014 -0600
--
CHANGES.txt | 2 ++
.../apache/cassandra/db/AtomicBTreeColumns.java | 14
.../org/apache/cassandra/utils/btree/BTree.java | 24
3 files changed, 30 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5223c479/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index feece93..efc54f8 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,7 +1,9 @@
2.1.0-beta2
+ * Fix ABTC NPE (CASSANDRA-6692)
* Allow nodetool to use a file or prompt for password (CASSANDRA-6660)
* Fix AIOOBE when concurrently accessing ABSC (CASSANDRA-6742)
+
2.1.0-beta1
* Add flush directory distinct from compaction directories (CASSANDRA-6357)
* Require JNA by default (CASSANDRA-6575)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5223c479/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
index 5f56326..f75efd2 100644
--- a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
@@ -230,25 +230,29 @@ public class AtomicBTreeColumns extends ColumnFamily
Holder current = ref;
delta.reset();
-DeletionInfo deletionInfo = cm.deletionInfo();
-if (deletionInfo.mayModify(current.deletionInfo))
+DeletionInfo deletionInfo;
+if (cm.deletionInfo().mayModify(current.deletionInfo))
{
-if (deletionInfo.hasRanges())
+if (cm.deletionInfo().hasRanges())
{
for (IteratorCell iter : new Iterator[] {
insert.iterator(), BTree.Cellslice(current.tree, true) })
{
while (iter.hasNext())
{
Cell col = iter.next();
-if (deletionInfo.isDeleted(col))
+if (cm.deletionInfo().isDeleted(col))
indexer.remove(col);
}
}
}
-deletionInfo = current.deletionInfo.copy().add(deletionInfo);
+deletionInfo =
current.deletionInfo.copy().add(cm.deletionInfo());
delta.addHeapSize(deletionInfo.unsharedHeapSize() -
current.deletionInfo.unsharedHeapSize());
}
+else
+{
+deletionInfo = current.deletionInfo;
+}
ColumnUpdater updater = new ColumnUpdater(this, current,
allocator, transformation, indexer, delta);
Object[] tree = BTree.update(current.tree,
metadata.comparator.columnComparator(), insert, true, updater);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/5223c479/src/java/org/apache/cassandra/utils/btree/BTree.java
--
diff --git a/src/java/org/apache/cassandra/utils/btree/BTree.java
b/src/java/org/apache/cassandra/utils/btree/BTree.java
index 82f5574..69cf145 100644
--- a/src/java/org/apache/cassandra/utils/btree/BTree.java
+++ b/src/java/org/apache/cassandra/utils/btree/BTree.java
@@ -18,9 +18,11 @@
*/
package org.apache.cassandra.utils.btree;
+import java.util.ArrayDeque;
import java.util.Arrays;
import java.util.Collection;
import java.util.Comparator;
+import java.util.Queue;
import org.apache.cassandra.utils.ObjectSizes;
@@ -113,7 +115,13 @@ public class BTree
if (!sorted)
source = sorted(source, comparator, size);
-return modifier.get().build(source, size);
+QueueBuilder queue = modifier.get();
+Builder builder = queue.poll();
+if (builder == null)
+builder = new Builder();
+Object[] btree = builder.build(source, size);
+queue.add(builder);
+return btree;
}
/**
@@ -154,7 +162,13 @@ public class BTree
if (!updateWithIsSorted)
[3/3] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7c730279 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7c730279 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7c730279 Branch: refs/heads/trunk Commit: 7c730279e8d6d29d0ab7b902826f6fd40e0cc390 Parents: 4d75ed2 5223c47 Author: Jonathan Ellis jbel...@apache.org Authored: Thu Feb 20 11:36:55 2014 -0600 Committer: Jonathan Ellis jbel...@apache.org Committed: Thu Feb 20 11:36:55 2014 -0600 -- CHANGES.txt | 2 ++ .../apache/cassandra/db/AtomicBTreeColumns.java | 14 .../org/apache/cassandra/utils/btree/BTree.java | 24 3 files changed, 30 insertions(+), 10 deletions(-) --
[jira] [Updated] (CASSANDRA-5561) Compaction strategy that minimizes re-compaction of old/frozen data
[ https://issues.apache.org/jira/browse/CASSANDRA-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-5561: -- Fix Version/s: (was: 2.0.6) 2.1 Compaction strategy that minimizes re-compaction of old/frozen data --- Key: CASSANDRA-5561 URL: https://issues.apache.org/jira/browse/CASSANDRA-5561 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.3 Reporter: Tupshin Harper Fix For: 2.1 Neither LCS nor STCS are good for data that becomes immutable over time. The most obvious case is for time-series data where the application can guarantee that out-of-order delivery (to Cassandra) of events can't take place more than N minutes/seconds/hours/days have elapsed after the real (wall time). There are various approaches that could involve paying attention to the row keys (if they include a time component) and/or the column name (if they are TimeUUID or Integer based and are inherently time-ordered), but it might be sufficient to just look at the timestamp of the columns themselves. A possible approach: 1) Define an optional max-out-of-order window on a per-CF basis. 2) Use normal (LCS or STCS) compaction strategy for any SSTables that include any columns younger than max-out-of-order-delivery). 3) Use alternate compaction strategy (will call it TWCS time window compaction strategy for now) for any SSTables that only contain columns older than max-out-of-order-delivery. 4) TWCS will only compact sstables containing data older than max-out-of-order-delivery. 5) TWCS will only perform compaction to reduce row fragmentation (if there is any by the time it gets to TWCS or to reduce the number of small sstables. 6) To minimize re-compaction in TWCS, it should aggresively try to compact as many small sstables as possible into one large sstable that would never have to get recompacted. In the case of large datasets (e.g. 5TB per node) with LCS, there would be on the order of seven levels, and hence seven separate writes of the same data over time. With this approach, it should be possible to get about 3 compactions per column (2 in original compaction and one more once hitting TWCS) in most cases, cutting the write workload by a factor of two or more for high volume time-series applications. Note that the only workaround I can currently suggest to minimize compaction for these workloads is to programatically shard your data across time-window ranges (e.g. new CF per week), but that pushes unnecessary writing and querying logic out to the user and is not as convenient nor flexible. Also note that I am not convinced that the approach I've suggested above is the best/most general way to solve the problem, but it does appear to be a relatively easy one to implement. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6577) ConcurrentModificationException during nodetool netstats
[
https://issues.apache.org/jira/browse/CASSANDRA-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907238#comment-13907238
]
Joshua McKenzie commented on CASSANDRA-6577:
As for the StorageService.java exception - there's no protection around the
HashSet of InetAddress (leavingEndpoints) in TokenMetadata.
TokenMetadata.getLeavingEndpoints() returns an iterator to the underlying
collection that could be invalidated by either an addLeavingEndpoint(),
removeEndpoint(), or updateNormalTokens() call. The cleanest way around this
looks to be putting the data into a ConcurrentSkipListSet, similar to how we do
in Gossiper.java, to provide iterator thread-safety to external users.
ConcurrentModificationException during nodetool netstats
Key: CASSANDRA-6577
URL: https://issues.apache.org/jira/browse/CASSANDRA-6577
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: AWS EC2 machines,
java version 1.7.0_45
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Each node has 32 vnodes.
Reporter: Shao-Chuan Wang
Assignee: Joshua McKenzie
Priority: Minor
Labels: decommission, nodetool
Fix For: 2.0.6
The node is leaving and I wanted to check its netstats, but it raises
ConcurrentModificationException.
{code}
[ubuntu@ip-10-4-202-48 :~]# /mnt/cassandra_latest/bin/nodetool netstats
Mode: LEAVING
Exception in thread main java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
at java.util.HashMap$ValueIterator.next(HashMap.java:954)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toArrayOfCompositeData(SessionInfoCompositeData.java:161)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toCompositeData(SessionInfoCompositeData.java:98)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:82)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:79)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData.toCompositeData(StreamStateCompositeData.java:78)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:87)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:84)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Sets.newHashSet(Sets.java:238)
at com.google.common.collect.Sets.newHashSet(Sets.java:218)
at
org.apache.cassandra.streaming.StreamManager.getCurrentStreams(StreamManager.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at
[jira] [Comment Edited] (CASSANDRA-6577) ConcurrentModificationException during nodetool netstats
[
https://issues.apache.org/jira/browse/CASSANDRA-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13905861#comment-13905861
]
Joshua McKenzie edited comment on CASSANDRA-6577 at 2/20/14 5:49 PM:
-
Regarding the netstats exception: the SessionInfo objects being operated on
inside the creation of CompositeData aren't thread-safe, so the following use
presents an opportunity for the Exception you're seeing.
Declaration:
{code:title=SessionInfo.java|borderstyle=solid}
this.receivingFiles = new HashMap();
this.sendingFiles = new HashMap();
{code}
These are iterated across creating the various CompositeData types requested,
however updateProgress within SessionInfo would cause invalidation of any open
iterators on the collection if any are active.
Edit: ConcurrentHashMaps provide iterator thread-safety with weak guarantee on
the contents of the HashMap. I'll look into the code a bit further and see if
using this structure would satisfy the various users of these functions as that
should prevent this race condition.
was (Author: joshuamckenzie):
Regarding the netstats exception: the SessionInfo objects being operated on
inside the creation of CompositeData aren't thread-safe, so the following use
presents an opportunity for the Exception you're seeing.
Declaration:
{code:title=SessionInfo.java|borderstyle=solid}
this.receivingFiles = new HashMap();
this.sendingFiles = new HashMap();
{code}
These are iterated across creating the various CompositeData types requested,
however updateProgress within SessionInfo would cause invalidation of any open
iterators on the collection if any are active.
Changing these to ConcurrentHashMaps still won't prevent this race since it's
an iterator invalidation problem - we may need to consider locking on this
Collection externally on the SessionInfo level, or creating a copy of the
internal data for representation on netstats.
ConcurrentModificationException during nodetool netstats
Key: CASSANDRA-6577
URL: https://issues.apache.org/jira/browse/CASSANDRA-6577
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: AWS EC2 machines,
java version 1.7.0_45
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Each node has 32 vnodes.
Reporter: Shao-Chuan Wang
Assignee: Joshua McKenzie
Priority: Minor
Labels: decommission, nodetool
Fix For: 2.0.6
The node is leaving and I wanted to check its netstats, but it raises
ConcurrentModificationException.
{code}
[ubuntu@ip-10-4-202-48 :~]# /mnt/cassandra_latest/bin/nodetool netstats
Mode: LEAVING
Exception in thread main java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
at java.util.HashMap$ValueIterator.next(HashMap.java:954)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toArrayOfCompositeData(SessionInfoCompositeData.java:161)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toCompositeData(SessionInfoCompositeData.java:98)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:82)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:79)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData.toCompositeData(StreamStateCompositeData.java:78)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:87)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:84)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Sets.newHashSet(Sets.java:238)
at
[jira] [Updated] (CASSANDRA-6705) ALTER TYPE type RENAME field fails sometime with java.lang.AssertionError: null
[
https://issues.apache.org/jira/browse/CASSANDRA-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Stepura updated CASSANDRA-6705:
---
Reproduced In: 2.1 beta1
ALTER TYPE type RENAME field fails sometime with
java.lang.AssertionError: null
---
Key: CASSANDRA-6705
URL: https://issues.apache.org/jira/browse/CASSANDRA-6705
Project: Cassandra
Issue Type: Bug
Environment: trunk
Reporter: Mikhail Stepura
Assignee: Mikhail Stepura
Priority: Minor
Fix For: 2.1 beta2
Attachments: trunk-6705.patch
Here are the steps w
{code}
cqlsh create KEYSPACE bug WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 1};
cqlsh
cqlsh
cqlsh use bug;
cqlsh:bug create type first_type (first_field int);
cqlsh:bug create type second_type (second_field listfirst_type );
cqlsh:bug alter type first_type RENAME first_field TO first_fieldd;
TSocket read 0 bytes
{code}
And here is from the C* side:
{code}
NFO 05:11:54 Loading
org.apache.cassandra.db.marshal.UserType(bug,7365636f6e645f74797065,7365636f6e645f6669656c64:org.apache.cassandra.db.marshal.ListType(org.apache.cassandra.db.marshal.UserType(bug,66697273745f74797065,66697273745f6669656c6464:org.apache.cassandra.db.marshal.Int32Type)))
INFO 05:11:54 Compacted 4 sstables to
[/var/lib/cassandra/data/system/schema_usertypes-3aa752254f82350b8d5c430fa221fa0a/system-schema_usertypes-ka-5,].
908 bytes to 425 (~46% of original) in 12ms = 0.033776MB/s. 4 total
partitions merged to 1. Partition merge counts were {4:1, }
ERROR 05:11:54 Exception in thread Thread[MigrationStage:1,5,main]
java.lang.AssertionError: null
at org.apache.cassandra.config.UTMetaData.addType(UTMetaData.java:145)
~[main/:na]
at org.apache.cassandra.db.DefsTables.addType(DefsTables.java:412)
~[main/:na]
at org.apache.cassandra.db.DefsTables.mergeTypes(DefsTables.java:365)
~[main/:na]
at org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:182)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager$2.runMayThrow(MigrationManager.java:299)
~[main/:na]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
~[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
ERROR 05:11:54 Error occurred during processing of message.
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.AssertionError
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:411)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:281)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager.announceNewType(MigrationManager.java:216)
~[main/:na]
at
org.apache.cassandra.service.MigrationManager.announceTypeUpdate(MigrationManager.java:247)
~[main/:na]
at
org.apache.cassandra.cql3.statements.AlterTypeStatement.announceMigration(AlterTypeStatement.java:139)
~[main/:na]
at
org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:71)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:180)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:214)
~[main/:na]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:204)
~[main/:na]
at
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1973)
~[main/:na]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4486)
~[thrift/:na]
at
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4470)
~[thrift/:na]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.1.jar:0.9.1]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.1.jar:0.9.1]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:194)
~[main/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_51]
at
[jira] [Updated] (CASSANDRA-6737) A batch statements on a single partition should not create a new CF object for each update
[ https://issues.apache.org/jira/browse/CASSANDRA-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6737: -- Labels: performance (was: ) A batch statements on a single partition should not create a new CF object for each update -- Key: CASSANDRA-6737 URL: https://issues.apache.org/jira/browse/CASSANDRA-6737 Project: Cassandra Issue Type: Improvement Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Labels: performance Fix For: 2.0.6 Attachments: 6737.2.patch, 6737.txt BatchStatement creates a new ColumnFamily object (as well as a new RowMutation object) for every update in the batch, even if all those update are actually on the same partition. This is particularly inefficient when bulkloading data into a single partition (which is not all that uncommon). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6357) Flush memtables to separate directory
[ https://issues.apache.org/jira/browse/CASSANDRA-6357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6357: -- Tester: dan jatnieks (was: Quentin Conner) Labels: performance (was: ) Flush memtables to separate directory - Key: CASSANDRA-6357 URL: https://issues.apache.org/jira/browse/CASSANDRA-6357 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Patrick McFadin Assignee: Jonathan Ellis Priority: Minor Labels: performance Fix For: 2.1 beta1 Attachments: 6357-v2.txt, 6357.txt, c6357-stress-write-latency-99th-1.png Flush writers are a critical element for keeping a node healthy. When several compactions run on systems with low performing data directories, IO becomes a premium. Once the disk subsystem is saturated, write IO is blocked which will cause flush writer threads to backup. Since memtables are large blocks of memory in the JVM, too much blocking can cause excessive GC over time degrading performance. In the worst case causing an OOM. Since compaction is running on the data directories. My proposal is to create a separate directory for flushing memtables. Potentially we can use the same methodology of keeping the commit log separate and minimize disk contention against the critical function of the flushwriter. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[1/2] git commit: Fast ABSC.addAll() path + ASPCSI optimization
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 5223c4797 - 6ed241acb
Fast ABSC.addAll() path + ASPCSI optimization
patch by Benedict Elliott Smith; reviewed by Aleksey Yeschenko
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6ed241ac
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6ed241ac
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6ed241ac
Branch: refs/heads/cassandra-2.1
Commit: 6ed241acb33087e6cedf8f4637aca5dce0fe502c
Parents: e69e526
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Feb 20 21:33:28 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Feb 20 21:35:33 2014 +0300
--
.../cassandra/db/ArrayBackedSortedColumns.java | 44
.../apache/cassandra/db/AtomicBTreeColumns.java | 2 +-
.../org/apache/cassandra/db/ColumnFamily.java | 7 +++-
.../AbstractSimplePerColumnSecondaryIndex.java | 19 +
4 files changed, 46 insertions(+), 26 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6ed241ac/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index 0ce36e2..1fdebd8 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -52,18 +52,18 @@ public class ArrayBackedSortedColumns extends ColumnFamily
public static final ColumnFamily.FactoryArrayBackedSortedColumns factory
= new FactoryArrayBackedSortedColumns()
{
-public ArrayBackedSortedColumns create(CFMetaData metadata, boolean
insertReversed)
+public ArrayBackedSortedColumns create(CFMetaData metadata, boolean
insertReversed, int initialCapacity)
{
-return new ArrayBackedSortedColumns(metadata, insertReversed);
+return new ArrayBackedSortedColumns(metadata, insertReversed,
initialCapacity);
}
};
-private ArrayBackedSortedColumns(CFMetaData metadata, boolean reversed)
+private ArrayBackedSortedColumns(CFMetaData metadata, boolean reversed,
int initialCapacity)
{
super(metadata);
this.reversed = reversed;
this.deletionInfo = DeletionInfo.live();
-this.cells = EMPTY_ARRAY;
+this.cells = initialCapacity == 0 ? EMPTY_ARRAY : new
Cell[initialCapacity];
this.size = 0;
this.sortedSize = 0;
this.isSorted = true;
@@ -235,9 +235,39 @@ public class ArrayBackedSortedColumns extends ColumnFamily
if (other.getColumnCount() == 0)
return;
-IteratorCell iterator = reversed ? other.reverseIterator() :
other.iterator();
-while (iterator.hasNext())
-addColumn(iterator.next());
+// In reality, with ABSC being the only remaining container (aside
from ABTC), other will aways be ABSC.
+if (size == 0 other instanceof ArrayBackedSortedColumns)
+{
+fastAddAll((ArrayBackedSortedColumns) other);
+}
+else
+{
+IteratorCell iterator = reversed ? other.reverseIterator() :
other.iterator();
+while (iterator.hasNext())
+addColumn(iterator.next());
+}
+}
+
+// Fast path, when this ABSC is empty.
+private void fastAddAll(ArrayBackedSortedColumns other)
+{
+if (other.isInsertReversed() == isInsertReversed())
+{
+cells = Arrays.copyOf(other.cells, other.cells.length);
+size = other.size;
+sortedSize = other.sortedSize;
+isSorted = other.isSorted;
+}
+else
+{
+if (cells.length other.getColumnCount())
+cells = new Cell[Math.max(MINIMAL_CAPACITY,
other.getColumnCount())];
+IteratorCell iterator = reversed ? other.reverseIterator() :
other.iterator();
+while (iterator.hasNext())
+cells[size++] = iterator.next();
+sortedSize = size;
+isSorted = true;
+}
}
/**
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6ed241ac/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
index f75efd2..5cc43d9 100644
--- a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
@@ -66,7 +66,7 @@ public class AtomicBTreeColumns extends ColumnFamily
[2/2] git commit: Better solution for CASSANDRA-6742
Better solution for CASSANDRA-6742
patch by Aleksey Yeschenko; reviewed by Benedict Elliott Smith for
CASSANDRA-6742
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e69e526b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e69e526b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e69e526b
Branch: refs/heads/cassandra-2.1
Commit: e69e526b9053e3d3634a005cc94a8954c2d7d41f
Parents: 5223c47
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Feb 20 21:26:21 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Feb 20 21:35:33 2014 +0300
--
.../cassandra/db/ArrayBackedSortedColumns.java | 86 ++--
1 file changed, 42 insertions(+), 44 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e69e526b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index 24a0755..0ce36e2 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -43,12 +43,12 @@ public class ArrayBackedSortedColumns extends ColumnFamily
private static final int MINIMAL_CAPACITY = 10;
private final boolean reversed;
-private DeletionInfo deletionInfo;
+private DeletionInfo deletionInfo;
private Cell[] cells;
-
-private volatile int size;
-private volatile int sortedSize;
+private int size;
+private int sortedSize;
+private volatile boolean isSorted;
public static final ColumnFamily.FactoryArrayBackedSortedColumns factory
= new FactoryArrayBackedSortedColumns()
{
@@ -66,6 +66,7 @@ public class ArrayBackedSortedColumns extends ColumnFamily
this.cells = EMPTY_ARRAY;
this.size = 0;
this.sortedSize = 0;
+this.isSorted = true;
}
private ArrayBackedSortedColumns(ArrayBackedSortedColumns original)
@@ -76,6 +77,7 @@ public class ArrayBackedSortedColumns extends ColumnFamily
this.cells = Arrays.copyOf(original.cells, original.size);
this.size = original.size;
this.sortedSize = original.sortedSize;
+this.isSorted = original.isSorted;
}
public ColumnFamily.Factory getFactory()
@@ -100,7 +102,7 @@ public class ArrayBackedSortedColumns extends ColumnFamily
private void maybeSortCells()
{
-if (size != sortedSize)
+if (!isSorted)
sortCells();
}
@@ -109,8 +111,8 @@ public class ArrayBackedSortedColumns extends ColumnFamily
*/
private synchronized void sortCells()
{
-if (size == sortedSize)
-return; // Just sorted by a previous call.
+if (isSorted)
+return; // Just sorted by a previous call
ComparatorCell comparator = reversed
? getComparator().columnReverseComparator()
@@ -133,8 +135,8 @@ public class ArrayBackedSortedColumns extends ColumnFamily
int rightStart = sortedSize;
int rightEnd = size;
-sortedSize = -1; // Set to -1 to avoid the pos == sortedSize edge case
-size = pos; // 'Trim' the size to what's left without the leftCopy
+// 'Trim' the sizes to what's left without the leftCopy
+size = sortedSize = pos;
// Merge the cells from both segments. When adding from the left
segment we can rely on it not having any
// duplicate cells, and thus omit the comparison with the previously
entered cell - we'll never need to reconcile.
@@ -143,23 +145,38 @@ public class ArrayBackedSortedColumns extends ColumnFamily
{
int cmp = comparator.compare(leftCopy[l], cells[r]);
if (cmp 0)
-internalAppend(leftCopy[l++]);
+append(leftCopy[l++]);
else if (cmp == 0)
-internalAppend(leftCopy[l++].reconcile(cells[r++]));
+append(leftCopy[l++].reconcile(cells[r++]));
else
-internalAppendOrReconcile(cells[r++]);
+appendOrReconcile(cells[r++]);
}
while (l leftCopy.length)
-internalAppend(leftCopy[l++]);
+append(leftCopy[l++]);
while (r rightEnd)
-internalAppendOrReconcile(cells[r++]);
-
-// Fully sorted at this point
-sortedSize = size;
+appendOrReconcile(cells[r++]);
// Nullify the remainder of the array (in case we had duplicate cells
that got reconciled)
for (int i = size; i rightEnd; i++)
cells[i] = null;
+
+//
[1/3] git commit: Fast ABSC.addAll() path + ASPCSI optimization
Repository: cassandra
Updated Branches:
refs/heads/trunk 7c730279e - 3d7c5fdd3
Fast ABSC.addAll() path + ASPCSI optimization
patch by Benedict Elliott Smith; reviewed by Aleksey Yeschenko
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6ed241ac
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6ed241ac
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6ed241ac
Branch: refs/heads/trunk
Commit: 6ed241acb33087e6cedf8f4637aca5dce0fe502c
Parents: e69e526
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Feb 20 21:33:28 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Feb 20 21:35:33 2014 +0300
--
.../cassandra/db/ArrayBackedSortedColumns.java | 44
.../apache/cassandra/db/AtomicBTreeColumns.java | 2 +-
.../org/apache/cassandra/db/ColumnFamily.java | 7 +++-
.../AbstractSimplePerColumnSecondaryIndex.java | 19 +
4 files changed, 46 insertions(+), 26 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6ed241ac/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index 0ce36e2..1fdebd8 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -52,18 +52,18 @@ public class ArrayBackedSortedColumns extends ColumnFamily
public static final ColumnFamily.FactoryArrayBackedSortedColumns factory
= new FactoryArrayBackedSortedColumns()
{
-public ArrayBackedSortedColumns create(CFMetaData metadata, boolean
insertReversed)
+public ArrayBackedSortedColumns create(CFMetaData metadata, boolean
insertReversed, int initialCapacity)
{
-return new ArrayBackedSortedColumns(metadata, insertReversed);
+return new ArrayBackedSortedColumns(metadata, insertReversed,
initialCapacity);
}
};
-private ArrayBackedSortedColumns(CFMetaData metadata, boolean reversed)
+private ArrayBackedSortedColumns(CFMetaData metadata, boolean reversed,
int initialCapacity)
{
super(metadata);
this.reversed = reversed;
this.deletionInfo = DeletionInfo.live();
-this.cells = EMPTY_ARRAY;
+this.cells = initialCapacity == 0 ? EMPTY_ARRAY : new
Cell[initialCapacity];
this.size = 0;
this.sortedSize = 0;
this.isSorted = true;
@@ -235,9 +235,39 @@ public class ArrayBackedSortedColumns extends ColumnFamily
if (other.getColumnCount() == 0)
return;
-IteratorCell iterator = reversed ? other.reverseIterator() :
other.iterator();
-while (iterator.hasNext())
-addColumn(iterator.next());
+// In reality, with ABSC being the only remaining container (aside
from ABTC), other will aways be ABSC.
+if (size == 0 other instanceof ArrayBackedSortedColumns)
+{
+fastAddAll((ArrayBackedSortedColumns) other);
+}
+else
+{
+IteratorCell iterator = reversed ? other.reverseIterator() :
other.iterator();
+while (iterator.hasNext())
+addColumn(iterator.next());
+}
+}
+
+// Fast path, when this ABSC is empty.
+private void fastAddAll(ArrayBackedSortedColumns other)
+{
+if (other.isInsertReversed() == isInsertReversed())
+{
+cells = Arrays.copyOf(other.cells, other.cells.length);
+size = other.size;
+sortedSize = other.sortedSize;
+isSorted = other.isSorted;
+}
+else
+{
+if (cells.length other.getColumnCount())
+cells = new Cell[Math.max(MINIMAL_CAPACITY,
other.getColumnCount())];
+IteratorCell iterator = reversed ? other.reverseIterator() :
other.iterator();
+while (iterator.hasNext())
+cells[size++] = iterator.next();
+sortedSize = size;
+isSorted = true;
+}
}
/**
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6ed241ac/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
index f75efd2..5cc43d9 100644
--- a/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
+++ b/src/java/org/apache/cassandra/db/AtomicBTreeColumns.java
@@ -66,7 +66,7 @@ public class AtomicBTreeColumns extends ColumnFamily
public
[2/3] git commit: Better solution for CASSANDRA-6742
Better solution for CASSANDRA-6742
patch by Aleksey Yeschenko; reviewed by Benedict Elliott Smith for
CASSANDRA-6742
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e69e526b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e69e526b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e69e526b
Branch: refs/heads/trunk
Commit: e69e526b9053e3d3634a005cc94a8954c2d7d41f
Parents: 5223c47
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu Feb 20 21:26:21 2014 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu Feb 20 21:35:33 2014 +0300
--
.../cassandra/db/ArrayBackedSortedColumns.java | 86 ++--
1 file changed, 42 insertions(+), 44 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e69e526b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index 24a0755..0ce36e2 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -43,12 +43,12 @@ public class ArrayBackedSortedColumns extends ColumnFamily
private static final int MINIMAL_CAPACITY = 10;
private final boolean reversed;
-private DeletionInfo deletionInfo;
+private DeletionInfo deletionInfo;
private Cell[] cells;
-
-private volatile int size;
-private volatile int sortedSize;
+private int size;
+private int sortedSize;
+private volatile boolean isSorted;
public static final ColumnFamily.FactoryArrayBackedSortedColumns factory
= new FactoryArrayBackedSortedColumns()
{
@@ -66,6 +66,7 @@ public class ArrayBackedSortedColumns extends ColumnFamily
this.cells = EMPTY_ARRAY;
this.size = 0;
this.sortedSize = 0;
+this.isSorted = true;
}
private ArrayBackedSortedColumns(ArrayBackedSortedColumns original)
@@ -76,6 +77,7 @@ public class ArrayBackedSortedColumns extends ColumnFamily
this.cells = Arrays.copyOf(original.cells, original.size);
this.size = original.size;
this.sortedSize = original.sortedSize;
+this.isSorted = original.isSorted;
}
public ColumnFamily.Factory getFactory()
@@ -100,7 +102,7 @@ public class ArrayBackedSortedColumns extends ColumnFamily
private void maybeSortCells()
{
-if (size != sortedSize)
+if (!isSorted)
sortCells();
}
@@ -109,8 +111,8 @@ public class ArrayBackedSortedColumns extends ColumnFamily
*/
private synchronized void sortCells()
{
-if (size == sortedSize)
-return; // Just sorted by a previous call.
+if (isSorted)
+return; // Just sorted by a previous call
ComparatorCell comparator = reversed
? getComparator().columnReverseComparator()
@@ -133,8 +135,8 @@ public class ArrayBackedSortedColumns extends ColumnFamily
int rightStart = sortedSize;
int rightEnd = size;
-sortedSize = -1; // Set to -1 to avoid the pos == sortedSize edge case
-size = pos; // 'Trim' the size to what's left without the leftCopy
+// 'Trim' the sizes to what's left without the leftCopy
+size = sortedSize = pos;
// Merge the cells from both segments. When adding from the left
segment we can rely on it not having any
// duplicate cells, and thus omit the comparison with the previously
entered cell - we'll never need to reconcile.
@@ -143,23 +145,38 @@ public class ArrayBackedSortedColumns extends ColumnFamily
{
int cmp = comparator.compare(leftCopy[l], cells[r]);
if (cmp 0)
-internalAppend(leftCopy[l++]);
+append(leftCopy[l++]);
else if (cmp == 0)
-internalAppend(leftCopy[l++].reconcile(cells[r++]));
+append(leftCopy[l++].reconcile(cells[r++]));
else
-internalAppendOrReconcile(cells[r++]);
+appendOrReconcile(cells[r++]);
}
while (l leftCopy.length)
-internalAppend(leftCopy[l++]);
+append(leftCopy[l++]);
while (r rightEnd)
-internalAppendOrReconcile(cells[r++]);
-
-// Fully sorted at this point
-sortedSize = size;
+appendOrReconcile(cells[r++]);
// Nullify the remainder of the array (in case we had duplicate cells
that got reconciled)
for (int i = size; i rightEnd; i++)
cells[i] = null;
+
+// Fully
[3/3] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d7c5fdd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d7c5fdd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d7c5fdd Branch: refs/heads/trunk Commit: 3d7c5fdd30fd162b0eb3d2a85faa8b70075be109 Parents: 7c73027 6ed241a Author: Aleksey Yeschenko alek...@apache.org Authored: Thu Feb 20 21:36:08 2014 +0300 Committer: Aleksey Yeschenko alek...@apache.org Committed: Thu Feb 20 21:36:08 2014 +0300 -- .../cassandra/db/ArrayBackedSortedColumns.java | 126 +++ .../apache/cassandra/db/AtomicBTreeColumns.java | 2 +- .../org/apache/cassandra/db/ColumnFamily.java | 7 +- .../AbstractSimplePerColumnSecondaryIndex.java | 19 +-- 4 files changed, 86 insertions(+), 68 deletions(-) --
[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907318#comment-13907318 ] T Jake Luciani commented on CASSANDRA-2434: --- Pushed an update to https://github.com/tjake/cassandra/tree/2434 that addresses the comments. I'm going to work on support for move/relocate. range movements can violate consistency --- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: T Jake Luciani Fix For: 2.1 beta2 Attachments: 2434-3.patch.txt, 2434-testery.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5918) Remove CQL2 entirely from Cassandra 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907326#comment-13907326 ] Aleksey Yeschenko commented on CASSANDRA-5918: -- Left them alone (throwing exceptions instead of removing entirely). That, and regenerated thrift (tiny change). Pushed -f to the same branch. Remove CQL2 entirely from Cassandra 3.0 --- Key: CASSANDRA-5918 URL: https://issues.apache.org/jira/browse/CASSANDRA-5918 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Priority: Minor Labels: cql Fix For: 3.0 CQL2 is officially no longer worked on since 1.2. cqlsh no longer supports CQL2 as of Cassandra 2.0. It's probably the time to deprecate CQL2 in 2.0 and to remove it entirely in 2.2 - there is nothing in CQL2 now that can't be done via CQL3 and two versions advance warning is plenty of time for those few still using CQL2 to switch to CQL3. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-3534) Remove memory emergency pressure valve
[ https://issues.apache.org/jira/browse/CASSANDRA-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3534: -- Summary: Remove memory emergency pressure valve (was: Enhance or remove memory emergency pressure valve) Remove memory emergency pressure valve -- Key: CASSANDRA-3534 URL: https://issues.apache.org/jira/browse/CASSANDRA-3534 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1.0 Reporter: Vijay Assignee: Aleksey Yeschenko Priority: Minor Fix For: 2.0 beta 1 Attachments: 3534.txt Seems like when these thresholds are reached we are trying to reduce the keycache and Memtable sizes, but in the trunk memtable is moved off-heap hence reducing that will not actually help. Reducing the keycache might help but we might want to provide a way to bring the cache level back to the configured value when the pressure goes away. Conversation from IRC: we might want to remove these setting all together after global keycache. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (CASSANDRA-6745) Require specifying rows_per_partition_to_cache
Jonathan Ellis created CASSANDRA-6745: - Summary: Require specifying rows_per_partition_to_cache Key: CASSANDRA-6745 URL: https://issues.apache.org/jira/browse/CASSANDRA-6745 Project: Cassandra Issue Type: Bug Reporter: Jonathan Ellis Assignee: Marcus Eriksson Priority: Trivial Fix For: 2.1 beta2 We should require specifying rows_to_cache_per_partition for new tables or newly ALTERed when row caching is enabled. Pre-upgrade should be grandfathered in as ALL to match existing semantics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5918) Remove CQL2 entirely from Cassandra 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-5918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907364#comment-13907364 ] Jonathan Ellis commented on CASSANDRA-5918: --- Ship it! Remove CQL2 entirely from Cassandra 3.0 --- Key: CASSANDRA-5918 URL: https://issues.apache.org/jira/browse/CASSANDRA-5918 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Priority: Minor Labels: cql Fix For: 3.0 CQL2 is officially no longer worked on since 1.2. cqlsh no longer supports CQL2 as of Cassandra 2.0. It's probably the time to deprecate CQL2 in 2.0 and to remove it entirely in 2.2 - there is nothing in CQL2 now that can't be done via CQL3 and two versions advance warning is plenty of time for those few still using CQL2 to switch to CQL3. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6733) Upgrade of 1.2.11 to 2.0.5 make IllegalArgumentException in Buffer.limit on read of a super column family
[
https://issues.apache.org/jira/browse/CASSANDRA-6733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Lalevée updated CASSANDRA-6733:
---
Attachment: QaUser_user_view_node2.tgz
QaUser_user_view_node1.tgz
See attached the compressed tar of the snapshot on the failing column family
for both nodes.
For the creation of the column family, it was done via Hector:
{noformat}
ColumnFamilyDefinition cfViewDef =
HFactory.createColumnFamilyDefinition(keyspaceName, columnFamilyName);
cfViewDef.setColumnType(ColumnType.SUPER);
cfViewDef.setComparatorType(ComparatorType.INTEGERTYPE);
cfViewDef.setSubComparatorType(ComparatorType.UTF8TYPE);
cfViewDef.setDefaultValidationClass(ComparatorType.COUNTERTYPE.getClassName());
cluster.getCluster().addColumnFamily(cfViewDef, true);
{noformat}
Then along the life of this column family, some manual operations were done via
the cassandra-cli, like changing the compression, and setting a key validation
class.
Upgrade of 1.2.11 to 2.0.5 make IllegalArgumentException in Buffer.limit on
read of a super column family
-
Key: CASSANDRA-6733
URL: https://issues.apache.org/jira/browse/CASSANDRA-6733
Project: Cassandra
Issue Type: Bug
Reporter: Nicolas Lalevée
Assignee: Sylvain Lebresne
Attachments: QaUser_user_view_node1.tgz, QaUser_user_view_node2.tgz
We have a super column family which was first created with a 1.0.x. Then
upgraded to 1.1.x, then to 1.2.11, and now to 2.0.5.
{noformat}
cqlsh:QaUser desc table user_view;
CREATE TABLE user_view (
key bigint,
column1 varint,
column2 text,
value counter,
PRIMARY KEY (key, column1, column2)
) WITH COMPACT STORAGE AND
bloom_filter_fp_chance=0.01 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.00 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=1.00 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='99.0PERCENTILE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'SnappyCompressor'};
{noformat}
With cqlsh, the following query was doing a timeout:
{noformat}
select * from user_view where key = 3 and column1 = 1 and column2 =
'20130218';
{noformat}
In the log of cassandra, we could read:
{noformat}
ERROR [ReadStage:1385] 2014-02-19 14:45:19,549 CassandraDaemon.java (line
192) Exception in thread Thread[ReadStage:1385,5,main]
java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:267)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:55)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:64)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:82)
at
org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35)
at
org.apache.cassandra.db.marshal.AbstractType$1.compare(AbstractType.java:63)
at
org.apache.cassandra.db.marshal.AbstractType$1.compare(AbstractType.java:60)
at java.util.Collections.indexedBinarySearch(Collections.java:377)
at java.util.Collections.binarySearch(Collections.java:365)
at
org.apache.cassandra.io.sstable.IndexHelper.indexFor(IndexHelper.java:144)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.setNextSlice(IndexedSliceReader.java:262)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader$IndexedBlockFetcher.init(IndexedSliceReader.java:255)
at
org.apache.cassandra.db.columniterator.IndexedSliceReader.init(IndexedSliceReader.java:91)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:65)
at
org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:42)
at
org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:167)
at
org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
at
org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:250)
at
org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
at
org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1560)
at
org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1379)
at
[1/4] Remove CQL2
Repository: cassandra
Updated Branches:
refs/heads/trunk 3d7c5fdd3 - 262170712
http://git-wip-us.apache.org/repos/asf/cassandra/blob/26217071/src/java/org/apache/cassandra/cql/hooks/PreExecutionHook.java
--
diff --git a/src/java/org/apache/cassandra/cql/hooks/PreExecutionHook.java
b/src/java/org/apache/cassandra/cql/hooks/PreExecutionHook.java
deleted file mode 100644
index 29ed38e..000
--- a/src/java/org/apache/cassandra/cql/hooks/PreExecutionHook.java
+++ /dev/null
@@ -1,46 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * License); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an AS IS BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.cassandra.cql.hooks;
-
-import org.apache.cassandra.cql.CQLStatement;
-import org.apache.cassandra.exceptions.RequestExecutionException;
-import org.apache.cassandra.exceptions.RequestValidationException;
-
-/**
- * Run before the CQL Statement is executed in
- * {@link org.apache.cassandra.cql.QueryProcessor}. The CQLStatement
- * returned from the processStatement method is what is actually executed
- * by the QueryProcessor.
- */
-public interface PreExecutionHook
-{
-/**
- * Perform pre-processing on a CQL statement prior to it being
- * executed by the QueryProcessor. If required, implementations
- * may modify the statement as the returned instance is what
- * is actually executed.
- *
- * @param statement the statement to perform pre-processing on
- * @param context execution context containing additional info
- *about the operation and statement
- * @return the actual statement that will be executed, possibly
- * a modification of the initial statement
- * @throws RequestExecutionException, RequestValidationException
- */
-CQLStatement processStatement(CQLStatement statement, ExecutionContext
context) throws RequestExecutionException, RequestValidationException;
-}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/26217071/src/java/org/apache/cassandra/cql/hooks/PreparationContext.java
--
diff --git a/src/java/org/apache/cassandra/cql/hooks/PreparationContext.java
b/src/java/org/apache/cassandra/cql/hooks/PreparationContext.java
deleted file mode 100644
index 00cce78..000
--- a/src/java/org/apache/cassandra/cql/hooks/PreparationContext.java
+++ /dev/null
@@ -1,39 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * License); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an AS IS BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.cassandra.cql.hooks;
-
-import org.apache.cassandra.cql.CQLStatement;
-import org.apache.cassandra.thrift.ThriftClientState;
-
-/**
- * Contextual information about the preparation of a CQLStatement.
- * Used by {@link org.apache.cassandra.cql.hooks.PostPreparationHook}
- */
-public class PreparationContext
-{
-public final ThriftClientState clientState;
-public final String queryString;
-public final CQLStatement statement;
-
-public PreparationContext(ThriftClientState clientState, String
queryString, CQLStatement statement)
-{
-this.clientState = clientState;
-this.queryString = queryString;
-this.statement = statement;
-}
-}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/26217071/src/java/org/apache/cassandra/cql3/QueryProcessor.java
--
diff --git a/src/java/org/apache/cassandra/cql3/QueryProcessor.java
[3/4] Remove CQL2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/26217071/src/java/org/apache/cassandra/cql/Cql.g
--
diff --git a/src/java/org/apache/cassandra/cql/Cql.g
b/src/java/org/apache/cassandra/cql/Cql.g
deleted file mode 100644
index 3c41f44..000
--- a/src/java/org/apache/cassandra/cql/Cql.g
+++ /dev/null
@@ -1,656 +0,0 @@
-
-/*
- *
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * License); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- *
- */
-
-grammar Cql;
-
-options {
-language = Java;
-}
-
-@header {
-package org.apache.cassandra.cql;
-import java.util.Map;
-import java.util.HashMap;
-import java.util.Collections;
-import java.util.List;
-import java.util.ArrayList;
-import org.apache.cassandra.exceptions.SyntaxException;
-import org.apache.cassandra.utils.Pair;
-import org.apache.cassandra.db.ConsistencyLevel;
-
-import static org.apache.cassandra.cql.AlterTableStatement.OperationType;
-}
-
-@members {
-private ListString recognitionErrors = new ArrayListString();
-private int currentBindMarkerIdx = -1;
-
-public void displayRecognitionError(String[] tokenNames,
RecognitionException e)
-{
-String hdr = getErrorHeader(e);
-String msg = getErrorMessage(e, tokenNames);
-recognitionErrors.add(hdr + + msg);
-}
-
-public ListString getRecognitionErrors()
-{
-return recognitionErrors;
-}
-
-public void throwLastRecognitionError() throws SyntaxException
-{
-if (recognitionErrors.size() 0)
-throw new
SyntaxException(recognitionErrors.get((recognitionErrors.size()-1)));
-}
-
-// used by UPDATE of the counter columns to validate if '-' was supplied
by user
-public void validateMinusSupplied(Object op, final Term value, IntStream
stream) throws MissingTokenException
-{
-if (op == null Long.parseLong(value.getText()) 0)
-throw new MissingTokenException(102, stream, value);
-}
-}
-
-@lexer::header {
-package org.apache.cassandra.cql;
-import org.apache.cassandra.exceptions.SyntaxException;
-}
-
-@lexer::members {
-ListToken tokens = new ArrayListToken();
-
-public void emit(Token token) {
-state.token = token;
-tokens.add(token);
-}
-
-public Token nextToken() {
-super.nextToken();
-if (tokens.size() == 0)
-return Token.EOF_TOKEN;
-return tokens.remove(0);
-}
-
-private ListString recognitionErrors = new ArrayListString();
-
-public void displayRecognitionError(String[] tokenNames,
RecognitionException e)
-{
-String hdr = getErrorHeader(e);
-String msg = getErrorMessage(e, tokenNames);
-recognitionErrors.add(hdr + + msg);
-}
-
-public ListString getRecognitionErrors()
-{
-return recognitionErrors;
-}
-
-public void throwLastRecognitionError() throws SyntaxException
-{
-if (recognitionErrors.size() 0)
-throw new
SyntaxException(recognitionErrors.get((recognitionErrors.size()-1)));
-}
-}
-
-query returns [CQLStatement stmnt]
-: selectStatement { $stmnt = new CQLStatement(StatementType.SELECT,
$selectStatement.expr, currentBindMarkerIdx); }
-| insertStatement endStmnt { $stmnt = new
CQLStatement(StatementType.INSERT, $insertStatement.expr,
currentBindMarkerIdx); }
-| updateStatement endStmnt { $stmnt = new
CQLStatement(StatementType.UPDATE, $updateStatement.expr,
currentBindMarkerIdx); }
-| batchStatement { $stmnt = new CQLStatement(StatementType.BATCH,
$batchStatement.expr, currentBindMarkerIdx); }
-| useStatement { $stmnt = new CQLStatement(StatementType.USE,
$useStatement.keyspace, currentBindMarkerIdx); }
-| truncateStatement { $stmnt = new CQLStatement(StatementType.TRUNCATE,
$truncateStatement.cf, currentBindMarkerIdx); }
-| deleteStatement endStmnt { $stmnt = new
CQLStatement(StatementType.DELETE, $deleteStatement.expr,
currentBindMarkerIdx); }
-| createKeyspaceStatement { $stmnt = new
CQLStatement(StatementType.CREATE_KEYSPACE, $createKeyspaceStatement.expr,
currentBindMarkerIdx); }
-|
[jira] [Commented] (CASSANDRA-5202) CFs should have globally and temporally unique CF IDs to prevent reusing data from earlier incarnation of same CF name
[ https://issues.apache.org/jira/browse/CASSANDRA-5202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907414#comment-13907414 ] Jonathan Ellis commented on CASSANDRA-5202: --- Did we give up on this? bq. Could we clean up the redundancy a little by moving the ID into the directory name? e.g., ks/cf-uuid/version-generation-component.db I'm worried about path length, which is limited on Windows. CFs should have globally and temporally unique CF IDs to prevent reusing data from earlier incarnation of same CF name Key: CASSANDRA-5202 URL: https://issues.apache.org/jira/browse/CASSANDRA-5202 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.9 Environment: OS: Windows 7, Server: Cassandra 1.1.9 release drop Client: astyanax 1.56.21, JVM: Sun/Oracle JVM 64 bit (jdk1.6.0_27) Reporter: Marat Bedretdinov Assignee: Yuki Morishita Labels: test Fix For: 2.1 beta1 Attachments: 0001-make-2i-CFMetaData-have-parent-s-CF-ID.patch, 0002-Don-t-scrub-2i-CF-if-index-type-is-CUSTOM.patch, 0003-Fix-user-defined-compaction.patch, 0004-Fix-serialization-test.patch, 0005-Create-system_auth-tables-with-fixed-CFID.patch, 0005-auth-v2.txt, 5202.txt, astyanax-stress-driver.zip Attached is a driver that sequentially: 1. Drops keyspace 2. Creates keyspace 4. Creates 2 column families 5. Seeds 1M rows with 100 columns 6. Queries these 2 column families The above steps are repeated 1000 times. The following exception is observed at random (race - SEDA?): ERROR [ReadStage:55] 2013-01-29 19:24:52,676 AbstractCassandraDaemon.java (line 135) Exception in thread Thread[ReadStage:55,5,main] java.lang.AssertionError: DecoratedKey(-1, ) != DecoratedKey(62819832764241410631599989027761269388, 313a31) in C:\var\lib\cassandra\data\user_role_reverse_index\business_entity_role\user_role_reverse_index-business_entity_role-hf-1-Data.db at org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:60) at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:67) at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:79) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:256) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:64) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1367) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1229) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1164) at org.apache.cassandra.db.Table.getRow(Table.java:378) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:822) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1271) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) This exception appears in the server at the time of client submitting a query request (row slice) and not at the time data is seeded. The client times out and this data can no longer be queried as the same exception would always occur from there on. Also on iteration 201, it appears that dropping column families failed and as a result their recreation failed with unique column family name violation (see exception below). Note that the data files are actually gone, so it appears that the server runtime responsible for creating column family was out of sync with the piece that dropped them: Starting dropping column families Dropped column families Starting dropping keyspace Dropped keyspace Starting creating column families Created column families Starting seeding data Total rows inserted: 100 in 5105 ms Iteration: 200; Total running time for 1000 queries is 232; Average running time of 1000 queries is 0 ms Starting dropping column families Dropped column families Starting dropping keyspace Dropped keyspace Starting creating column families Created column families Starting seeding data Total rows inserted: 100 in 5361 ms Iteration: 201; Total running time for 1000 queries is 222; Average running time of 1000 queries is 0 ms Starting dropping column families Starting
[jira] [Updated] (CASSANDRA-6577) ConcurrentModificationException during nodetool netstats
[
https://issues.apache.org/jira/browse/CASSANDRA-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-6577:
---
Attachment: 6577_TokenMetadata.patch
6577_SessionInfo.patch
ConcurrentModificationException during nodetool netstats
Key: CASSANDRA-6577
URL: https://issues.apache.org/jira/browse/CASSANDRA-6577
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: AWS EC2 machines,
java version 1.7.0_45
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Each node has 32 vnodes.
Reporter: Shao-Chuan Wang
Assignee: Joshua McKenzie
Priority: Minor
Labels: decommission, nodetool
Fix For: 2.0.6
Attachments: 6577_SessionInfo.patch, 6577_TokenMetadata.patch
The node is leaving and I wanted to check its netstats, but it raises
ConcurrentModificationException.
{code}
[ubuntu@ip-10-4-202-48 :~]# /mnt/cassandra_latest/bin/nodetool netstats
Mode: LEAVING
Exception in thread main java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
at java.util.HashMap$ValueIterator.next(HashMap.java:954)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toArrayOfCompositeData(SessionInfoCompositeData.java:161)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toCompositeData(SessionInfoCompositeData.java:98)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:82)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:79)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData.toCompositeData(StreamStateCompositeData.java:78)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:87)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:84)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Sets.newHashSet(Sets.java:238)
at com.google.common.collect.Sets.newHashSet(Sets.java:218)
at
org.apache.cassandra.streaming.StreamManager.getCurrentStreams(StreamManager.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at
com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1464)
at
[jira] [Commented] (CASSANDRA-6577) ConcurrentModificationException during nodetool netstats
[
https://issues.apache.org/jira/browse/CASSANDRA-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907415#comment-13907415
]
Joshua McKenzie commented on CASSANDRA-6577:
Attaching patch for each of the issues since they're logically independent.
ConcurrentModificationException during nodetool netstats
Key: CASSANDRA-6577
URL: https://issues.apache.org/jira/browse/CASSANDRA-6577
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: AWS EC2 machines,
java version 1.7.0_45
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Each node has 32 vnodes.
Reporter: Shao-Chuan Wang
Assignee: Joshua McKenzie
Priority: Minor
Labels: decommission, nodetool
Fix For: 2.0.6
Attachments: 6577_SessionInfo.patch, 6577_TokenMetadata.patch
The node is leaving and I wanted to check its netstats, but it raises
ConcurrentModificationException.
{code}
[ubuntu@ip-10-4-202-48 :~]# /mnt/cassandra_latest/bin/nodetool netstats
Mode: LEAVING
Exception in thread main java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
at java.util.HashMap$ValueIterator.next(HashMap.java:954)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toArrayOfCompositeData(SessionInfoCompositeData.java:161)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toCompositeData(SessionInfoCompositeData.java:98)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:82)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:79)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData.toCompositeData(StreamStateCompositeData.java:78)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:87)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:84)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Sets.newHashSet(Sets.java:238)
at com.google.common.collect.Sets.newHashSet(Sets.java:218)
at
org.apache.cassandra.streaming.StreamManager.getCurrentStreams(StreamManager.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at
com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1464)
at
[jira] [Comment Edited] (CASSANDRA-5202) CFs should have globally and temporally unique CF IDs to prevent reusing data from earlier incarnation of same CF name
[ https://issues.apache.org/jira/browse/CASSANDRA-5202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907414#comment-13907414 ] Jonathan Ellis edited comment on CASSANDRA-5202 at 2/20/14 8:03 PM: Did we give up on this? bq. Could we clean up the redundancy a little by moving the ID into the directory name? e.g., ks/cf-uuid/version-generation-component.db I'm worried about path length, which is limited on Windows. Edit: to give a specific example, for KS foo Table bar we now have /var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/foo-bar-ka-1-Data.db I'm proposing /var/lib/cassandra/flush/foo/bar-2fbb89709a6911e3b7dc4d7d4e3ca4b4/ka-1-Data.db was (Author: jbellis): Did we give up on this? bq. Could we clean up the redundancy a little by moving the ID into the directory name? e.g., ks/cf-uuid/version-generation-component.db I'm worried about path length, which is limited on Windows. CFs should have globally and temporally unique CF IDs to prevent reusing data from earlier incarnation of same CF name Key: CASSANDRA-5202 URL: https://issues.apache.org/jira/browse/CASSANDRA-5202 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.9 Environment: OS: Windows 7, Server: Cassandra 1.1.9 release drop Client: astyanax 1.56.21, JVM: Sun/Oracle JVM 64 bit (jdk1.6.0_27) Reporter: Marat Bedretdinov Assignee: Yuki Morishita Labels: test Fix For: 2.1 beta1 Attachments: 0001-make-2i-CFMetaData-have-parent-s-CF-ID.patch, 0002-Don-t-scrub-2i-CF-if-index-type-is-CUSTOM.patch, 0003-Fix-user-defined-compaction.patch, 0004-Fix-serialization-test.patch, 0005-Create-system_auth-tables-with-fixed-CFID.patch, 0005-auth-v2.txt, 5202.txt, astyanax-stress-driver.zip Attached is a driver that sequentially: 1. Drops keyspace 2. Creates keyspace 4. Creates 2 column families 5. Seeds 1M rows with 100 columns 6. Queries these 2 column families The above steps are repeated 1000 times. The following exception is observed at random (race - SEDA?): ERROR [ReadStage:55] 2013-01-29 19:24:52,676 AbstractCassandraDaemon.java (line 135) Exception in thread Thread[ReadStage:55,5,main] java.lang.AssertionError: DecoratedKey(-1, ) != DecoratedKey(62819832764241410631599989027761269388, 313a31) in C:\var\lib\cassandra\data\user_role_reverse_index\business_entity_role\user_role_reverse_index-business_entity_role-hf-1-Data.db at org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:60) at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:67) at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:79) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:256) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:64) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1367) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1229) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1164) at org.apache.cassandra.db.Table.getRow(Table.java:378) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:69) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:822) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1271) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) This exception appears in the server at the time of client submitting a query request (row slice) and not at the time data is seeded. The client times out and this data can no longer be queried as the same exception would always occur from there on. Also on iteration 201, it appears that dropping column families failed and as a result their recreation failed with unique column family name violation (see exception below). Note that the data files are actually gone, so it appears that the server runtime responsible for creating column family was out of sync with the piece that dropped them: Starting dropping column families Dropped column families Starting dropping keyspace Dropped keyspace Starting creating column families Created column families Starting
[jira] [Commented] (CASSANDRA-6285) LCS compaction failing with Exception
[ https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907434#comment-13907434 ] Nikolai Grigoriev commented on CASSANDRA-6285: -- Can confirm on my side. I have switched to sync RPC server and after few scrubs/restarts I am running my load tests on a 6-node 2.0.5 cluster without a single exception in last ~8 hours. I tried to correlate the moment I started getting large number of FileNotFoundException's with other events in my clusterrealized that it was not exactly 2.0.5 upgrade. It seems to correlate mostly with a moment when my jmeter server went out of free space and a bunch of tests crashed. Obviously, these crashes have terminated a few hundreds of client connections to Cassandra. Not sure if it is related but it seems that from that moment it was some sort of snowball effect. LCS compaction failing with Exception - Key: CASSANDRA-6285 URL: https://issues.apache.org/jira/browse/CASSANDRA-6285 Project: Cassandra Issue Type: Bug Components: Core Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2 Reporter: David Sauer Assignee: Tyler Hobbs Fix For: 2.0.6 Attachments: compaction_test.py After altering everything to LCS the table OpsCenter.rollups60 amd one other none OpsCenter-Table got stuck with everything hanging around in L0. The compaction started and ran until the logs showed this: ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime] java.lang.RuntimeException: Last written key DecoratedKey(1326283851463420237, 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564) = current key DecoratedKey(954210699457429663, 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f) writing into /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db at org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141) at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164) at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160) at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) at org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Moving back to STC worked to keep the compactions running. Especialy my own Table i would like to move to LCS. After a major compaction with STC the move to LCS fails with the same Exception. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[1/2] git commit: Post-merge fixup
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 6ed241acb - 76e5c93fb
Post-merge fixup
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d7019cc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d7019cc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d7019cc
Branch: refs/heads/cassandra-2.1
Commit: 3d7019cc3c93acd0c4f3556f10497a2bd07c3a05
Parents: 63b1ef4
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Feb 20 21:07:08 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Thu Feb 20 21:07:08 2014 +0100
--
src/java/org/apache/cassandra/db/composites/AbstractCType.java | 2 ++
src/java/org/apache/cassandra/db/filter/ColumnSlice.java | 5 +
2 files changed, 7 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d7019cc/src/java/org/apache/cassandra/db/composites/AbstractCType.java
--
diff --git a/src/java/org/apache/cassandra/db/composites/AbstractCType.java
b/src/java/org/apache/cassandra/db/composites/AbstractCType.java
index d3b8940..054706d 100644
--- a/src/java/org/apache/cassandra/db/composites/AbstractCType.java
+++ b/src/java/org/apache/cassandra/db/composites/AbstractCType.java
@@ -90,6 +90,8 @@ public abstract class AbstractCType implements CType
{
if (c1 == null || c1.isEmpty())
return c2 == null || c2.isEmpty() ? 0 : -1;
+if (c2 == null || c2.isEmpty())
+return 1;
if (c1.isStatic() != c2.isStatic())
return c1.isStatic() ? -1 : 1;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d7019cc/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
index 02a5a0c..19d26c7 100644
--- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
@@ -215,6 +215,11 @@ public class ColumnSlice
return prefix.size();
}
+public boolean isStatic()
+{
+return prefix.isStatic();
+}
+
public ByteBuffer get(int i)
{
return prefix.get(i);
[2/2] git commit: Merge branch 'cassandra-2.1' of https://git-wip-us.apache.org/repos/asf/cassandra into cassandra-2.1
Merge branch 'cassandra-2.1' of https://git-wip-us.apache.org/repos/asf/cassandra into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/76e5c93f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/76e5c93f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/76e5c93f Branch: refs/heads/cassandra-2.1 Commit: 76e5c93fb9eb6c3554a1a04fd4b5daa6e7fdbd05 Parents: 3d7019c 6ed241a Author: Sylvain Lebresne sylv...@datastax.com Authored: Thu Feb 20 21:07:27 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Thu Feb 20 21:07:27 2014 +0100 -- CHANGES.txt | 2 + .../cassandra/db/ArrayBackedSortedColumns.java | 126 +++ .../apache/cassandra/db/AtomicBTreeColumns.java | 16 ++- .../org/apache/cassandra/db/ColumnFamily.java | 7 +- .../AbstractSimplePerColumnSecondaryIndex.java | 19 +-- .../org/apache/cassandra/utils/btree/BTree.java | 24 +++- 6 files changed, 116 insertions(+), 78 deletions(-) --
git commit: Fix typos
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 b09d87691 - edf16c950
Fix typos
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/edf16c95
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/edf16c95
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/edf16c95
Branch: refs/heads/cassandra-2.0
Commit: edf16c950dadcf2b75c8faed33269881b58b1de6
Parents: b09d876
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Feb 20 21:11:33 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Thu Feb 20 21:11:33 2014 +0100
--
.../org/apache/cassandra/db/marshal/AbstractCompositeType.java | 2 ++
src/java/org/apache/cassandra/db/marshal/LongType.java | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/edf16c95/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
--
diff --git
a/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
b/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
index 9250b0f..c789d9b 100644
--- a/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
@@ -76,6 +76,8 @@ public abstract class AbstractCompositeType extends
AbstractTypeByteBuffer
{
if (o1 == null || !o1.hasRemaining())
return o2 == null || !o2.hasRemaining() ? 0 : -1;
+if (o2 == null || !o2.hasRemaining())
+return 1;
ByteBuffer bb1 = o1.duplicate();
ByteBuffer bb2 = o2.duplicate();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/edf16c95/src/java/org/apache/cassandra/db/marshal/LongType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/LongType.java
b/src/java/org/apache/cassandra/db/marshal/LongType.java
index 142683d..9ad078e 100644
--- a/src/java/org/apache/cassandra/db/marshal/LongType.java
+++ b/src/java/org/apache/cassandra/db/marshal/LongType.java
@@ -47,7 +47,7 @@ public class LongType extends AbstractTypeLong
return 1;
}
-int diff = o1.get(o1.position() + o1.arrayOffset()) -
o2.get(o2.position() + o2.arrayOffset());
+int diff = o1.get(o1.position()) - o2.get(o2.position());
if (diff != 0)
return diff;
[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/13b753b0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/13b753b0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/13b753b0 Branch: refs/heads/cassandra-2.1 Commit: 13b753b0169aa668c8c6091d68705a4034f7cdd4 Parents: 76e5c93 edf16c9 Author: Sylvain Lebresne sylv...@datastax.com Authored: Thu Feb 20 21:11:55 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Thu Feb 20 21:11:55 2014 +0100 -- .../org/apache/cassandra/db/marshal/AbstractCompositeType.java | 2 ++ src/java/org/apache/cassandra/db/marshal/LongType.java | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/13b753b0/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java --
[1/2] git commit: Fix typos
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 76e5c93fb - 13b753b01
Fix typos
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/edf16c95
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/edf16c95
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/edf16c95
Branch: refs/heads/cassandra-2.1
Commit: edf16c950dadcf2b75c8faed33269881b58b1de6
Parents: b09d876
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Feb 20 21:11:33 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Thu Feb 20 21:11:33 2014 +0100
--
.../org/apache/cassandra/db/marshal/AbstractCompositeType.java | 2 ++
src/java/org/apache/cassandra/db/marshal/LongType.java | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/edf16c95/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
--
diff --git
a/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
b/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
index 9250b0f..c789d9b 100644
--- a/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
@@ -76,6 +76,8 @@ public abstract class AbstractCompositeType extends
AbstractTypeByteBuffer
{
if (o1 == null || !o1.hasRemaining())
return o2 == null || !o2.hasRemaining() ? 0 : -1;
+if (o2 == null || !o2.hasRemaining())
+return 1;
ByteBuffer bb1 = o1.duplicate();
ByteBuffer bb2 = o2.duplicate();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/edf16c95/src/java/org/apache/cassandra/db/marshal/LongType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/LongType.java
b/src/java/org/apache/cassandra/db/marshal/LongType.java
index 142683d..9ad078e 100644
--- a/src/java/org/apache/cassandra/db/marshal/LongType.java
+++ b/src/java/org/apache/cassandra/db/marshal/LongType.java
@@ -47,7 +47,7 @@ public class LongType extends AbstractTypeLong
return 1;
}
-int diff = o1.get(o1.position() + o1.arrayOffset()) -
o2.get(o2.position() + o2.arrayOffset());
+int diff = o1.get(o1.position()) - o2.get(o2.position());
if (diff != 0)
return diff;
[2/5] git commit: Merge branch 'cassandra-2.1' of https://git-wip-us.apache.org/repos/asf/cassandra into cassandra-2.1
Merge branch 'cassandra-2.1' of https://git-wip-us.apache.org/repos/asf/cassandra into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/76e5c93f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/76e5c93f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/76e5c93f Branch: refs/heads/trunk Commit: 76e5c93fb9eb6c3554a1a04fd4b5daa6e7fdbd05 Parents: 3d7019c 6ed241a Author: Sylvain Lebresne sylv...@datastax.com Authored: Thu Feb 20 21:07:27 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Thu Feb 20 21:07:27 2014 +0100 -- CHANGES.txt | 2 + .../cassandra/db/ArrayBackedSortedColumns.java | 126 +++ .../apache/cassandra/db/AtomicBTreeColumns.java | 16 ++- .../org/apache/cassandra/db/ColumnFamily.java | 7 +- .../AbstractSimplePerColumnSecondaryIndex.java | 19 +-- .../org/apache/cassandra/utils/btree/BTree.java | 24 +++- 6 files changed, 116 insertions(+), 78 deletions(-) --
[4/5] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/13b753b0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/13b753b0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/13b753b0 Branch: refs/heads/trunk Commit: 13b753b0169aa668c8c6091d68705a4034f7cdd4 Parents: 76e5c93 edf16c9 Author: Sylvain Lebresne sylv...@datastax.com Authored: Thu Feb 20 21:11:55 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Thu Feb 20 21:11:55 2014 +0100 -- .../org/apache/cassandra/db/marshal/AbstractCompositeType.java | 2 ++ src/java/org/apache/cassandra/db/marshal/LongType.java | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/13b753b0/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java --
[3/5] git commit: Fix typos
Fix typos
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/edf16c95
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/edf16c95
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/edf16c95
Branch: refs/heads/trunk
Commit: edf16c950dadcf2b75c8faed33269881b58b1de6
Parents: b09d876
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Feb 20 21:11:33 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Thu Feb 20 21:11:33 2014 +0100
--
.../org/apache/cassandra/db/marshal/AbstractCompositeType.java | 2 ++
src/java/org/apache/cassandra/db/marshal/LongType.java | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/edf16c95/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
--
diff --git
a/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
b/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
index 9250b0f..c789d9b 100644
--- a/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
+++ b/src/java/org/apache/cassandra/db/marshal/AbstractCompositeType.java
@@ -76,6 +76,8 @@ public abstract class AbstractCompositeType extends
AbstractTypeByteBuffer
{
if (o1 == null || !o1.hasRemaining())
return o2 == null || !o2.hasRemaining() ? 0 : -1;
+if (o2 == null || !o2.hasRemaining())
+return 1;
ByteBuffer bb1 = o1.duplicate();
ByteBuffer bb2 = o2.duplicate();
http://git-wip-us.apache.org/repos/asf/cassandra/blob/edf16c95/src/java/org/apache/cassandra/db/marshal/LongType.java
--
diff --git a/src/java/org/apache/cassandra/db/marshal/LongType.java
b/src/java/org/apache/cassandra/db/marshal/LongType.java
index 142683d..9ad078e 100644
--- a/src/java/org/apache/cassandra/db/marshal/LongType.java
+++ b/src/java/org/apache/cassandra/db/marshal/LongType.java
@@ -47,7 +47,7 @@ public class LongType extends AbstractTypeLong
return 1;
}
-int diff = o1.get(o1.position() + o1.arrayOffset()) -
o2.get(o2.position() + o2.arrayOffset());
+int diff = o1.get(o1.position()) - o2.get(o2.position());
if (diff != 0)
return diff;
[5/5] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/63e0eb4f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/63e0eb4f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/63e0eb4f Branch: refs/heads/trunk Commit: 63e0eb4faab4f06181cddd9911db9f7be93a3961 Parents: 2621707 13b753b Author: Sylvain Lebresne sylv...@datastax.com Authored: Thu Feb 20 21:13:10 2014 +0100 Committer: Sylvain Lebresne sylv...@datastax.com Committed: Thu Feb 20 21:13:10 2014 +0100 -- src/java/org/apache/cassandra/db/composites/AbstractCType.java | 2 ++ src/java/org/apache/cassandra/db/filter/ColumnSlice.java| 5 + .../org/apache/cassandra/db/marshal/AbstractCompositeType.java | 2 ++ src/java/org/apache/cassandra/db/marshal/LongType.java | 2 +- 4 files changed, 10 insertions(+), 1 deletion(-) --
[1/5] git commit: Post-merge fixup
Repository: cassandra
Updated Branches:
refs/heads/trunk 262170712 - 63e0eb4fa
Post-merge fixup
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d7019cc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d7019cc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d7019cc
Branch: refs/heads/trunk
Commit: 3d7019cc3c93acd0c4f3556f10497a2bd07c3a05
Parents: 63b1ef4
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Thu Feb 20 21:07:08 2014 +0100
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Thu Feb 20 21:07:08 2014 +0100
--
src/java/org/apache/cassandra/db/composites/AbstractCType.java | 2 ++
src/java/org/apache/cassandra/db/filter/ColumnSlice.java | 5 +
2 files changed, 7 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d7019cc/src/java/org/apache/cassandra/db/composites/AbstractCType.java
--
diff --git a/src/java/org/apache/cassandra/db/composites/AbstractCType.java
b/src/java/org/apache/cassandra/db/composites/AbstractCType.java
index d3b8940..054706d 100644
--- a/src/java/org/apache/cassandra/db/composites/AbstractCType.java
+++ b/src/java/org/apache/cassandra/db/composites/AbstractCType.java
@@ -90,6 +90,8 @@ public abstract class AbstractCType implements CType
{
if (c1 == null || c1.isEmpty())
return c2 == null || c2.isEmpty() ? 0 : -1;
+if (c2 == null || c2.isEmpty())
+return 1;
if (c1.isStatic() != c2.isStatic())
return c1.isStatic() ? -1 : 1;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d7019cc/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
--
diff --git a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
index 02a5a0c..19d26c7 100644
--- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java
@@ -215,6 +215,11 @@ public class ColumnSlice
return prefix.size();
}
+public boolean isStatic()
+{
+return prefix.isStatic();
+}
+
public ByteBuffer get(int i)
{
return prefix.get(i);
Git Push Summary
Repository: cassandra Updated Tags: refs/tags/2.1.0-beta1-tentative [deleted] 73dcdbdf2
Git Push Summary
Repository: cassandra Updated Tags: refs/tags/cassandra-2.1.0-beta1 [created] f4d5f40b1
[jira] [Updated] (CASSANDRA-6577) ConcurrentModificationException during nodetool netstats
[
https://issues.apache.org/jira/browse/CASSANDRA-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-6577:
---
Reviewer: Tyler Hobbs
ConcurrentModificationException during nodetool netstats
Key: CASSANDRA-6577
URL: https://issues.apache.org/jira/browse/CASSANDRA-6577
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: AWS EC2 machines,
java version 1.7.0_45
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Each node has 32 vnodes.
Reporter: Shao-Chuan Wang
Assignee: Joshua McKenzie
Priority: Minor
Labels: decommission, nodetool
Fix For: 2.0.6
Attachments: 6577_SessionInfo.patch, 6577_TokenMetadata.patch
The node is leaving and I wanted to check its netstats, but it raises
ConcurrentModificationException.
{code}
[ubuntu@ip-10-4-202-48 :~]# /mnt/cassandra_latest/bin/nodetool netstats
Mode: LEAVING
Exception in thread main java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
at java.util.HashMap$ValueIterator.next(HashMap.java:954)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toArrayOfCompositeData(SessionInfoCompositeData.java:161)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toCompositeData(SessionInfoCompositeData.java:98)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:82)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:79)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData.toCompositeData(StreamStateCompositeData.java:78)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:87)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:84)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Sets.newHashSet(Sets.java:238)
at com.google.common.collect.Sets.newHashSet(Sets.java:218)
at
org.apache.cassandra.streaming.StreamManager.getCurrentStreams(StreamManager.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at
com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1464)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
at
[jira] [Commented] (CASSANDRA-6561) Static columns in CQL3
[
https://issues.apache.org/jira/browse/CASSANDRA-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907450#comment-13907450
]
DOAN DuyHai commented on CASSANDRA-6561:
Really great change ! It brings lots of new possibilities for data modeling
with CQL3, big +1.
A very dumb question though: is it allowed to create a static column in a
non-clustered table ?
I mean, it clearly does not make sense to do so but the question is more on
validation checking side ? Does C* raise exception if I try to create a
static column in a plain non-clustered table ?
Static columns in CQL3
--
Key: CASSANDRA-6561
URL: https://issues.apache.org/jira/browse/CASSANDRA-6561
Project: Cassandra
Issue Type: New Feature
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0.6
I'd like to suggest the following idea for adding static columns to CQL3.
I'll note that the basic idea has been suggested by jhalliday on irc but the
rest of the details are mine and I should be blamed for anything stupid in
what follows.
Let me start with a rational: there is 2 main family of CF that have been
historically used in Thrift: static ones and dynamic ones. CQL3 handles both
family through the presence or not of clustering columns. There is however
some cases where mixing both behavior has its use. I like to think of those
use cases as 3 broad category:
# to denormalize small amounts of not-entirely-static data in otherwise
static entities. It's say tags for a product or custom properties in a
user profile. This is why we've added CQL3 collections. Importantly, this is
the *only* use case for which collections are meant (which doesn't diminishes
their usefulness imo, and I wouldn't disagree that we've maybe not
communicated this too well).
# to optimize fetching both a static entity and related dynamic ones. Say you
have blog posts, and each post has associated comments (chronologically
ordered). *And* say that a very common query is fetch a post and its 50 last
comments. In that case, it *might* be beneficial to store a blog post
(static entity) in the same underlying CF than it's comments for performance
reason. So that fetch a post and it's 50 last comments is just one slice
internally.
# you want to CAS rows of a dynamic partition based on some partition
condition. This is the same use case than why CASSANDRA-5633 exists for.
As said above, 1) is already covered by collections, but 2) and 3) are not
(and
I strongly believe collections are not the right fit, API wise, for those).
Also, note that I don't want to underestimate the usefulness of 2). In most
cases, using a separate table for the blog posts and the comments is The
Right Solution, and trying to do 2) is premature optimisation. Yet, when used
properly, that kind of optimisation can make a difference, so I think having
a relatively native solution for it in CQL3 could make sense.
Regarding 3), though CASSANDRA-5633 would provide one solution for it, I have
the feeling that static columns actually are a more natural approach (in term
of API). That's arguably more of a personal opinion/feeling though.
So long story short, CQL3 lacks a way to mix both some static and dynamic
rows in the same partition of the same CQL3 table, and I think such a tool
could have it's use.
The proposal is thus to allow static columns. Static columns would only
make sense in table with clustering columns (the dynamic ones). A static
column value would be static to the partition (all rows of the partition
would share the value for such column). The syntax would just be:
{noformat}
CREATE TABLE t (
k text,
s text static,
i int,
v text,
PRIMARY KEY (k, i)
)
{noformat}
then you'd get:
{noformat}
INSERT INTO t(k, s, i, v) VALUES (k0, I'm shared, 0, foo);
INSERT INTO t(k, s, i, v) VALUES (k0, I'm still shared, 1, bar);
SELECT * FROM t;
k | s | i |v
k0 | I'm still shared | 0 | bar
k0 | I'm still shared | 1 | foo
{noformat}
There would be a few semantic details to decide on regarding deletions, ttl,
etc. but let's see if we agree it's a good idea first before ironing those
out.
One last point is the implementation. Though I do think this idea has merits,
it's definitively not useful enough to justify rewriting the storage engine
for it. But I think we can support this relatively easily (emphasis on
relatively :)), which is probably the main reason why I like the approach.
Namely, internally, we can store static columns as cells whose clustering
column values are empty. So in terms of cells, the partition of my example
would look like:
{noformat}
k0 : [
(:s - I'm still shared), // the static column
[jira] [Commented] (CASSANDRA-6561) Static columns in CQL3
[
https://issues.apache.org/jira/browse/CASSANDRA-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907452#comment-13907452
]
Aleksey Yeschenko commented on CASSANDRA-6561:
--
bq. Does C* raise exception if I try to create a static column in a plain
non-clustered table ?
Yes.
Static columns in CQL3
--
Key: CASSANDRA-6561
URL: https://issues.apache.org/jira/browse/CASSANDRA-6561
Project: Cassandra
Issue Type: New Feature
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0.6
I'd like to suggest the following idea for adding static columns to CQL3.
I'll note that the basic idea has been suggested by jhalliday on irc but the
rest of the details are mine and I should be blamed for anything stupid in
what follows.
Let me start with a rational: there is 2 main family of CF that have been
historically used in Thrift: static ones and dynamic ones. CQL3 handles both
family through the presence or not of clustering columns. There is however
some cases where mixing both behavior has its use. I like to think of those
use cases as 3 broad category:
# to denormalize small amounts of not-entirely-static data in otherwise
static entities. It's say tags for a product or custom properties in a
user profile. This is why we've added CQL3 collections. Importantly, this is
the *only* use case for which collections are meant (which doesn't diminishes
their usefulness imo, and I wouldn't disagree that we've maybe not
communicated this too well).
# to optimize fetching both a static entity and related dynamic ones. Say you
have blog posts, and each post has associated comments (chronologically
ordered). *And* say that a very common query is fetch a post and its 50 last
comments. In that case, it *might* be beneficial to store a blog post
(static entity) in the same underlying CF than it's comments for performance
reason. So that fetch a post and it's 50 last comments is just one slice
internally.
# you want to CAS rows of a dynamic partition based on some partition
condition. This is the same use case than why CASSANDRA-5633 exists for.
As said above, 1) is already covered by collections, but 2) and 3) are not
(and
I strongly believe collections are not the right fit, API wise, for those).
Also, note that I don't want to underestimate the usefulness of 2). In most
cases, using a separate table for the blog posts and the comments is The
Right Solution, and trying to do 2) is premature optimisation. Yet, when used
properly, that kind of optimisation can make a difference, so I think having
a relatively native solution for it in CQL3 could make sense.
Regarding 3), though CASSANDRA-5633 would provide one solution for it, I have
the feeling that static columns actually are a more natural approach (in term
of API). That's arguably more of a personal opinion/feeling though.
So long story short, CQL3 lacks a way to mix both some static and dynamic
rows in the same partition of the same CQL3 table, and I think such a tool
could have it's use.
The proposal is thus to allow static columns. Static columns would only
make sense in table with clustering columns (the dynamic ones). A static
column value would be static to the partition (all rows of the partition
would share the value for such column). The syntax would just be:
{noformat}
CREATE TABLE t (
k text,
s text static,
i int,
v text,
PRIMARY KEY (k, i)
)
{noformat}
then you'd get:
{noformat}
INSERT INTO t(k, s, i, v) VALUES (k0, I'm shared, 0, foo);
INSERT INTO t(k, s, i, v) VALUES (k0, I'm still shared, 1, bar);
SELECT * FROM t;
k | s | i |v
k0 | I'm still shared | 0 | bar
k0 | I'm still shared | 1 | foo
{noformat}
There would be a few semantic details to decide on regarding deletions, ttl,
etc. but let's see if we agree it's a good idea first before ironing those
out.
One last point is the implementation. Though I do think this idea has merits,
it's definitively not useful enough to justify rewriting the storage engine
for it. But I think we can support this relatively easily (emphasis on
relatively :)), which is probably the main reason why I like the approach.
Namely, internally, we can store static columns as cells whose clustering
column values are empty. So in terms of cells, the partition of my example
would look like:
{noformat}
k0 : [
(:s - I'm still shared), // the static column
(0: - ) // row marker
(0:v - bar)
(1: - ) // row marker
(1:v - foo)
]
{noformat}
Of course, using empty values for the clustering columns doesn't quite work
because it could conflict with the user using empty clustering columns. But
[jira] [Commented] (CASSANDRA-6561) Static columns in CQL3
[
https://issues.apache.org/jira/browse/CASSANDRA-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907453#comment-13907453
]
Sylvain Lebresne commented on CASSANDRA-6561:
-
bq. Does C* raise exception if I try to create a static column in a plain
non-clustered table ?
It does.
Static columns in CQL3
--
Key: CASSANDRA-6561
URL: https://issues.apache.org/jira/browse/CASSANDRA-6561
Project: Cassandra
Issue Type: New Feature
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0.6
I'd like to suggest the following idea for adding static columns to CQL3.
I'll note that the basic idea has been suggested by jhalliday on irc but the
rest of the details are mine and I should be blamed for anything stupid in
what follows.
Let me start with a rational: there is 2 main family of CF that have been
historically used in Thrift: static ones and dynamic ones. CQL3 handles both
family through the presence or not of clustering columns. There is however
some cases where mixing both behavior has its use. I like to think of those
use cases as 3 broad category:
# to denormalize small amounts of not-entirely-static data in otherwise
static entities. It's say tags for a product or custom properties in a
user profile. This is why we've added CQL3 collections. Importantly, this is
the *only* use case for which collections are meant (which doesn't diminishes
their usefulness imo, and I wouldn't disagree that we've maybe not
communicated this too well).
# to optimize fetching both a static entity and related dynamic ones. Say you
have blog posts, and each post has associated comments (chronologically
ordered). *And* say that a very common query is fetch a post and its 50 last
comments. In that case, it *might* be beneficial to store a blog post
(static entity) in the same underlying CF than it's comments for performance
reason. So that fetch a post and it's 50 last comments is just one slice
internally.
# you want to CAS rows of a dynamic partition based on some partition
condition. This is the same use case than why CASSANDRA-5633 exists for.
As said above, 1) is already covered by collections, but 2) and 3) are not
(and
I strongly believe collections are not the right fit, API wise, for those).
Also, note that I don't want to underestimate the usefulness of 2). In most
cases, using a separate table for the blog posts and the comments is The
Right Solution, and trying to do 2) is premature optimisation. Yet, when used
properly, that kind of optimisation can make a difference, so I think having
a relatively native solution for it in CQL3 could make sense.
Regarding 3), though CASSANDRA-5633 would provide one solution for it, I have
the feeling that static columns actually are a more natural approach (in term
of API). That's arguably more of a personal opinion/feeling though.
So long story short, CQL3 lacks a way to mix both some static and dynamic
rows in the same partition of the same CQL3 table, and I think such a tool
could have it's use.
The proposal is thus to allow static columns. Static columns would only
make sense in table with clustering columns (the dynamic ones). A static
column value would be static to the partition (all rows of the partition
would share the value for such column). The syntax would just be:
{noformat}
CREATE TABLE t (
k text,
s text static,
i int,
v text,
PRIMARY KEY (k, i)
)
{noformat}
then you'd get:
{noformat}
INSERT INTO t(k, s, i, v) VALUES (k0, I'm shared, 0, foo);
INSERT INTO t(k, s, i, v) VALUES (k0, I'm still shared, 1, bar);
SELECT * FROM t;
k | s | i |v
k0 | I'm still shared | 0 | bar
k0 | I'm still shared | 1 | foo
{noformat}
There would be a few semantic details to decide on regarding deletions, ttl,
etc. but let's see if we agree it's a good idea first before ironing those
out.
One last point is the implementation. Though I do think this idea has merits,
it's definitively not useful enough to justify rewriting the storage engine
for it. But I think we can support this relatively easily (emphasis on
relatively :)), which is probably the main reason why I like the approach.
Namely, internally, we can store static columns as cells whose clustering
column values are empty. So in terms of cells, the partition of my example
would look like:
{noformat}
k0 : [
(:s - I'm still shared), // the static column
(0: - ) // row marker
(0:v - bar)
(1: - ) // row marker
(1:v - foo)
]
{noformat}
Of course, using empty values for the clustering columns doesn't quite work
because it could conflict with the user using empty clustering columns. But
[jira] [Commented] (CASSANDRA-6722) cross-partition ordering should have warning or be disallowed when paging
[
https://issues.apache.org/jira/browse/CASSANDRA-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907463#comment-13907463
]
Aleksey Yeschenko commented on CASSANDRA-6722:
--
LGTM, +1.
cross-partition ordering should have warning or be disallowed when paging
-
Key: CASSANDRA-6722
URL: https://issues.apache.org/jira/browse/CASSANDRA-6722
Project: Cassandra
Issue Type: Bug
Reporter: Russ Hatch
Assignee: Sylvain Lebresne
Priority: Minor
Fix For: 2.0.6
Attachments: 6722.txt
consider this schema/data/query:
{noformat}
CREATE TABLE paging_test (
id int,
value text,
PRIMARY KEY (id, value)
) WITH CLUSTERING ORDER BY (value ASC)
|id|value|
|1 |a|
|2 |b|
|1 |c|
|2 |d|
|1 |e|
|2 |f|
|1 |g|
|2 |h|
|1 |i|
|2 |j|
select * from paging_test where id in (1,2) order by value asc;
{noformat}
When paging the above query I get the sorted results from id=1 first, then
the sorted results from id=2 after that. I was testing this because I was
curious if the paging system could somehow globally sort the results but it
makes sense that we can't do that, since that would require all results to be
collated up front.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907468#comment-13907468 ] T Jake Luciani commented on CASSANDRA-2434: --- Do we care about decommissions? It seems when we push data to other nodes there isn't anything todo. Only when we pick the replica to stream from does this ticket apply range movements can violate consistency --- Key: CASSANDRA-2434 URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 Project: Cassandra Issue Type: Bug Reporter: Peter Schuller Assignee: T Jake Luciani Fix For: 2.1 beta2 Attachments: 2434-3.patch.txt, 2434-testery.patch.txt My reading (a while ago) of the code indicates that there is no logic involved during bootstrapping that avoids consistency level violations. If I recall correctly it just grabs neighbors that are currently up. There are at least two issues I have with this behavior: * If I have a cluster where I have applications relying on QUORUM with RF=3, and bootstrapping complete based on only one node, I have just violated the supposedly guaranteed consistency semantics of the cluster. * Nodes can flap up and down at any time, so even if a human takes care to look at which nodes are up and things about it carefully before bootstrapping, there's no guarantee. A complication is that not only does it depend on use-case where this is an issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster which is otherwise used for QUORUM operations you may wish to accept less-than-quorum nodes during bootstrap in various emergency situations. A potential easy fix is to have bootstrap take an argument which is the number of hosts to bootstrap from, or to assume QUORUM if none is given. (A related concern is bootstrapping across data centers. You may *want* to bootstrap to a local node and then do a repair to avoid sending loads of data across DC:s while still achieving consistency. Or even if you don't care about the consistency issues, I don't think there is currently a way to bootstrap from local nodes only.) Thoughts? -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6577) ConcurrentModificationException during nodetool netstats
[
https://issues.apache.org/jira/browse/CASSANDRA-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907477#comment-13907477
]
Tyler Hobbs commented on CASSANDRA-6577:
[~JoshuaMcKenzie] it looks like the {{TokenMetadata.getLeavingEndpoints()}} was
fixed by CASSANDRA-6103. Access to leavingEndpoints is now locked, and
getLeavingEndpoints returns a new immutable copy.
For the SessionInfo patch, your fix looks good. {{updateProgress()}} was
synchronized, but obviously that didn't protect readers. Now that you're using
a ConcurrentHashMap, we should be able to remove that.
ConcurrentModificationException during nodetool netstats
Key: CASSANDRA-6577
URL: https://issues.apache.org/jira/browse/CASSANDRA-6577
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: AWS EC2 machines,
java version 1.7.0_45
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Each node has 32 vnodes.
Reporter: Shao-Chuan Wang
Assignee: Joshua McKenzie
Priority: Minor
Labels: decommission, nodetool
Fix For: 2.0.6
Attachments: 6577_SessionInfo.patch, 6577_TokenMetadata.patch
The node is leaving and I wanted to check its netstats, but it raises
ConcurrentModificationException.
{code}
[ubuntu@ip-10-4-202-48 :~]# /mnt/cassandra_latest/bin/nodetool netstats
Mode: LEAVING
Exception in thread main java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
at java.util.HashMap$ValueIterator.next(HashMap.java:954)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toArrayOfCompositeData(SessionInfoCompositeData.java:161)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toCompositeData(SessionInfoCompositeData.java:98)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:82)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:79)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData.toCompositeData(StreamStateCompositeData.java:78)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:87)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:84)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Sets.newHashSet(Sets.java:238)
at com.google.common.collect.Sets.newHashSet(Sets.java:218)
at
org.apache.cassandra.streaming.StreamManager.getCurrentStreams(StreamManager.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at
com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at
svn commit: r1570349 - in /cassandra/site: publish/download/index.html src/settings.py
Author: slebresne Date: Thu Feb 20 20:50:29 2014 New Revision: 1570349 URL: http://svn.apache.org/r1570349 Log: Update website for 2.1.0-beta1 release Modified: cassandra/site/publish/download/index.html cassandra/site/src/settings.py Modified: cassandra/site/publish/download/index.html URL: http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1570349r1=1570348r2=1570349view=diff == --- cassandra/site/publish/download/index.html (original) +++ cassandra/site/publish/download/index.html Thu Feb 20 20:50:29 2014 @@ -96,6 +96,22 @@ + h2 class=hdrDevelopment Cassandra Server Releases (not production ready)/h2 + p + The latest development release is 2.1.0-beta1 (released on + 2014-02-20). + /p + + ul +li +a class=filename href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.1.0/apache-cassandra-2.1.0-beta1-bin.tar.gz;apache-cassandra-2.1.0-beta1-bin.tar.gz/a +[a href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-beta1-bin.tar.gz.asc;PGP/a] +[a href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-beta1-bin.tar.gz.md5;MD5/a] +[a href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-beta1-bin.tar.gz.sha1;SHA1/a] +/li + /ul + + h2 class=hdrPrevious and Archived Cassandra Server Releases/h2 @@ -169,6 +185,13 @@ /li +li +a class=filename href=http://www.apache.org/dyn/closer.cgi?path=/cassandra/2.1.0/apache-cassandra-2.1.0-beta1-src.tar.gz;apache-cassandra-2.1.0-beta1-src.tar.gz/a +[a href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-beta1-src.tar.gz.asc;PGP/a] +[a href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-beta1-src.tar.gz.md5;MD5/a] +[a href=http://www.apache.org/dist/cassandra/2.1.0/apache-cassandra-2.1.0-beta1-src.tar.gz.sha1;SHA1/a] +/li + /ul Development is done in the Apache Git repository. To check out a copy: Modified: cassandra/site/src/settings.py URL: http://svn.apache.org/viewvc/cassandra/site/src/settings.py?rev=1570349r1=1570348r2=1570349view=diff == --- cassandra/site/src/settings.py (original) +++ cassandra/site/src/settings.py Thu Feb 20 20:50:29 2014 @@ -100,9 +100,9 @@ class CassandraDef(object): veryoldstable_exists = True stable_version = '2.0.5' stable_release_date = '2014-02-07' -devel_version = '2.0.0-rc2' -devel_release_date = '2013-08-20' -devel_exists = False +devel_version = '2.1.0-beta1' +devel_release_date = '2014-02-20' +devel_exists = True _apache_base_url = 'http://www.apache.org' _svn_base_url = 'https://svn.apache.org/repos/asf' _git_url = 'http://git-wip-us.apache.org/repos/asf?p=cassandra.git'
[jira] [Commented] (CASSANDRA-6740) Non-empty flush directory causes NPE on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907487#comment-13907487
]
Jonathan Ellis commented on CASSANDRA-6740:
---
What do we do if you wipe everything but a data directory?
Non-empty flush directory causes NPE on startup
---
Key: CASSANDRA-6740
URL: https://issues.apache.org/jira/browse/CASSANDRA-6740
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Priority: Minor
With the changes in CASSANDRA-6357, there is now the case where someone may
want to start a fresh cluster on a machine that previously hosted one, clean
out the directories they are familiar with (eg, data_file_directories,
commitlog_directories) but fail to clean out the flush_directory. If they
don't clean that out, they will see this in the logs:
{code}
INFO [main] 2014-02-19 12:20:13,530 ColumnFamilyStore.java:281 -
Initializing system.IndexInfo
INFO [main] 2014-02-19 12:20:13,534 ColumnFamilyStore.java:281 -
Initializing system.peers
INFO [main] 2014-02-19 12:20:13,539 ColumnFamilyStore.java:281 -
Initializing system.local
ERROR [main] 2014-02-19 12:20:13,703 CassandraDaemon.java:458 - Exception
encountered during startup
java.lang.NullPointerException: null
at
org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:167)
~[main/:na]
at
org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:39)
~[main/:na]
at
org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:26)
~[main/:na]
at
org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:66)
~[main/:na]
at
org.apache.cassandra.cql3.UntypedResultSet$Row.getString(UntypedResultSet.java:150)
~[main/:na]
at
org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1761)
~[main/:na]
at
org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1883)
~[main/:na]
at
org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:320)
~[main/:na]
at
org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:301)
~[main/:na]
at
org.apache.cassandra.db.DefsTables.loadFromKeyspace(DefsTables.java:131)
~[main/:na]
at
org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:539)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:230)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:441)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:530)
[main/:na]
{code}
I suggest a better warning message telling them it's due to the unclean flush
dir, instead of an NPE.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6740) Non-empty flush directory causes NPE on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907490#comment-13907490
]
Brandon Williams commented on CASSANDRA-6740:
-
Actually that could be the problem here, schema mismatch.
Non-empty flush directory causes NPE on startup
---
Key: CASSANDRA-6740
URL: https://issues.apache.org/jira/browse/CASSANDRA-6740
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Priority: Minor
With the changes in CASSANDRA-6357, there is now the case where someone may
want to start a fresh cluster on a machine that previously hosted one, clean
out the directories they are familiar with (eg, data_file_directories,
commitlog_directories) but fail to clean out the flush_directory. If they
don't clean that out, they will see this in the logs:
{code}
INFO [main] 2014-02-19 12:20:13,530 ColumnFamilyStore.java:281 -
Initializing system.IndexInfo
INFO [main] 2014-02-19 12:20:13,534 ColumnFamilyStore.java:281 -
Initializing system.peers
INFO [main] 2014-02-19 12:20:13,539 ColumnFamilyStore.java:281 -
Initializing system.local
ERROR [main] 2014-02-19 12:20:13,703 CassandraDaemon.java:458 - Exception
encountered during startup
java.lang.NullPointerException: null
at
org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:167)
~[main/:na]
at
org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:39)
~[main/:na]
at
org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:26)
~[main/:na]
at
org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:66)
~[main/:na]
at
org.apache.cassandra.cql3.UntypedResultSet$Row.getString(UntypedResultSet.java:150)
~[main/:na]
at
org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1761)
~[main/:na]
at
org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1883)
~[main/:na]
at
org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:320)
~[main/:na]
at
org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:301)
~[main/:na]
at
org.apache.cassandra.db.DefsTables.loadFromKeyspace(DefsTables.java:131)
~[main/:na]
at
org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:539)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:230)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:441)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:530)
[main/:na]
{code}
I suggest a better warning message telling them it's due to the unclean flush
dir, instead of an NPE.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (CASSANDRA-6746) Reads have a slow ramp up in speed
Ryan McGuire created CASSANDRA-6746: --- Summary: Reads have a slow ramp up in speed Key: CASSANDRA-6746 URL: https://issues.apache.org/jira/browse/CASSANDRA-6746 Project: Cassandra Issue Type: Bug Reporter: Ryan McGuire Attachments: 2.1_vs_2.0_read.png On a physical four node cluister I am doing a big write and then a big read. The read takes a long time to ramp up to respectable speeds. !2.1_vs_2.0_read.png! [See data here|http://ryanmcguire.info/ds/graph/graph.html?stats=stats.2.1_vs_2.0_vs_1.2.retry1.jsonmetric=interval_op_rateoperation=stress-readsmoothing=1] -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6740) Non-empty flush directory causes NPE on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907489#comment-13907489
]
Brandon Williams commented on CASSANDRA-6740:
-
We do nothing, until there's schema for whatever you left in the data dir, and
then we load it.
Non-empty flush directory causes NPE on startup
---
Key: CASSANDRA-6740
URL: https://issues.apache.org/jira/browse/CASSANDRA-6740
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Assignee: Jonathan Ellis
Priority: Minor
With the changes in CASSANDRA-6357, there is now the case where someone may
want to start a fresh cluster on a machine that previously hosted one, clean
out the directories they are familiar with (eg, data_file_directories,
commitlog_directories) but fail to clean out the flush_directory. If they
don't clean that out, they will see this in the logs:
{code}
INFO [main] 2014-02-19 12:20:13,530 ColumnFamilyStore.java:281 -
Initializing system.IndexInfo
INFO [main] 2014-02-19 12:20:13,534 ColumnFamilyStore.java:281 -
Initializing system.peers
INFO [main] 2014-02-19 12:20:13,539 ColumnFamilyStore.java:281 -
Initializing system.local
ERROR [main] 2014-02-19 12:20:13,703 CassandraDaemon.java:458 - Exception
encountered during startup
java.lang.NullPointerException: null
at
org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:167)
~[main/:na]
at
org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:39)
~[main/:na]
at
org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:26)
~[main/:na]
at
org.apache.cassandra.db.marshal.AbstractType.compose(AbstractType.java:66)
~[main/:na]
at
org.apache.cassandra.cql3.UntypedResultSet$Row.getString(UntypedResultSet.java:150)
~[main/:na]
at
org.apache.cassandra.config.CFMetaData.fromSchemaNoTriggers(CFMetaData.java:1761)
~[main/:na]
at
org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1883)
~[main/:na]
at
org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:320)
~[main/:na]
at
org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:301)
~[main/:na]
at
org.apache.cassandra.db.DefsTables.loadFromKeyspace(DefsTables.java:131)
~[main/:na]
at
org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:539)
~[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:230)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:441)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:530)
[main/:na]
{code}
I suggest a better warning message telling them it's due to the unclean flush
dir, instead of an NPE.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6746) Reads have a slow ramp up in speed
[ https://issues.apache.org/jira/browse/CASSANDRA-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McGuire updated CASSANDRA-6746: Since Version: 2.1 beta1 Reads have a slow ramp up in speed -- Key: CASSANDRA-6746 URL: https://issues.apache.org/jira/browse/CASSANDRA-6746 Project: Cassandra Issue Type: Bug Reporter: Ryan McGuire Attachments: 2.1_vs_2.0_read.png On a physical four node cluister I am doing a big write and then a big read. The read takes a long time to ramp up to respectable speeds. !2.1_vs_2.0_read.png! [See data here|http://ryanmcguire.info/ds/graph/graph.html?stats=stats.2.1_vs_2.0_vs_1.2.retry1.jsonmetric=interval_op_rateoperation=stress-readsmoothing=1] -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6577) ConcurrentModificationException during nodetool netstats
[
https://issues.apache.org/jira/browse/CASSANDRA-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907507#comment-13907507
]
Joshua McKenzie commented on CASSANDRA-6577:
Good catch on 6103 - I was working off 2.0.1 where the stacks were produced and
didn't think to check that the issue might have been duplicated - that's my
mistake. Attaching revised SessionInfo patch w/synchronized removed.
ConcurrentModificationException during nodetool netstats
Key: CASSANDRA-6577
URL: https://issues.apache.org/jira/browse/CASSANDRA-6577
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: AWS EC2 machines,
java version 1.7.0_45
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Each node has 32 vnodes.
Reporter: Shao-Chuan Wang
Assignee: Joshua McKenzie
Priority: Minor
Labels: decommission, nodetool
Fix For: 2.0.6
Attachments: 6577_SessionInfo.patch, 6577_SessionInfo_v2.patch,
6577_TokenMetadata.patch
The node is leaving and I wanted to check its netstats, but it raises
ConcurrentModificationException.
{code}
[ubuntu@ip-10-4-202-48 :~]# /mnt/cassandra_latest/bin/nodetool netstats
Mode: LEAVING
Exception in thread main java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
at java.util.HashMap$ValueIterator.next(HashMap.java:954)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toArrayOfCompositeData(SessionInfoCompositeData.java:161)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toCompositeData(SessionInfoCompositeData.java:98)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:82)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:79)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData.toCompositeData(StreamStateCompositeData.java:78)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:87)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:84)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Sets.newHashSet(Sets.java:238)
at com.google.common.collect.Sets.newHashSet(Sets.java:218)
at
org.apache.cassandra.streaming.StreamManager.getCurrentStreams(StreamManager.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at
com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
at
[jira] [Assigned] (CASSANDRA-6746) Reads have a slow ramp up in speed
[ https://issues.apache.org/jira/browse/CASSANDRA-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-6746: - Assignee: Benedict Reads have a slow ramp up in speed -- Key: CASSANDRA-6746 URL: https://issues.apache.org/jira/browse/CASSANDRA-6746 Project: Cassandra Issue Type: Bug Components: Core Reporter: Ryan McGuire Assignee: Benedict Labels: performance Fix For: 2.1 beta2 Attachments: 2.1_vs_2.0_read.png On a physical four node cluister I am doing a big write and then a big read. The read takes a long time to ramp up to respectable speeds. !2.1_vs_2.0_read.png! [See data here|http://ryanmcguire.info/ds/graph/graph.html?stats=stats.2.1_vs_2.0_vs_1.2.retry1.jsonmetric=interval_op_rateoperation=stress-readsmoothing=1] -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (CASSANDRA-6577) ConcurrentModificationException during nodetool netstats
[
https://issues.apache.org/jira/browse/CASSANDRA-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-6577:
---
Attachment: 6577_SessionInfo_v2.patch
ConcurrentModificationException during nodetool netstats
Key: CASSANDRA-6577
URL: https://issues.apache.org/jira/browse/CASSANDRA-6577
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: AWS EC2 machines,
java version 1.7.0_45
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Each node has 32 vnodes.
Reporter: Shao-Chuan Wang
Assignee: Joshua McKenzie
Priority: Minor
Labels: decommission, nodetool
Fix For: 2.0.6
Attachments: 6577_SessionInfo.patch, 6577_SessionInfo_v2.patch,
6577_TokenMetadata.patch
The node is leaving and I wanted to check its netstats, but it raises
ConcurrentModificationException.
{code}
[ubuntu@ip-10-4-202-48 :~]# /mnt/cassandra_latest/bin/nodetool netstats
Mode: LEAVING
Exception in thread main java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
at java.util.HashMap$ValueIterator.next(HashMap.java:954)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toArrayOfCompositeData(SessionInfoCompositeData.java:161)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toCompositeData(SessionInfoCompositeData.java:98)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:82)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:79)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData.toCompositeData(StreamStateCompositeData.java:78)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:87)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:84)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Sets.newHashSet(Sets.java:238)
at com.google.common.collect.Sets.newHashSet(Sets.java:218)
at
org.apache.cassandra.streaming.StreamManager.getCurrentStreams(StreamManager.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at
com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1464)
at
javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
[jira] [Updated] (CASSANDRA-6746) Reads have a slow ramp up in speed
[ https://issues.apache.org/jira/browse/CASSANDRA-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6746: -- Component/s: Core Fix Version/s: 2.1 beta2 Labels: performance (was: ) Reads have a slow ramp up in speed -- Key: CASSANDRA-6746 URL: https://issues.apache.org/jira/browse/CASSANDRA-6746 Project: Cassandra Issue Type: Bug Components: Core Reporter: Ryan McGuire Assignee: Benedict Labels: performance Fix For: 2.1 beta2 Attachments: 2.1_vs_2.0_read.png On a physical four node cluister I am doing a big write and then a big read. The read takes a long time to ramp up to respectable speeds. !2.1_vs_2.0_read.png! [See data here|http://ryanmcguire.info/ds/graph/graph.html?stats=stats.2.1_vs_2.0_vs_1.2.retry1.jsonmetric=interval_op_rateoperation=stress-readsmoothing=1] -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907528#comment-13907528
]
Tyler Hobbs commented on CASSANDRA-2434:
bq. Do we care about decommissions? It seems when we push data to other nodes
there isn't anything todo. Only when we pick the replica to stream from does
this ticket apply
I checked that {{StorageService.getChangedRangesForLeaving()}} pushes to the
correct nodes (those that are gaining a range), so you're right, we don't need
to do anything new for decom.
range movements can violate consistency
---
Key: CASSANDRA-2434
URL: https://issues.apache.org/jira/browse/CASSANDRA-2434
Project: Cassandra
Issue Type: Bug
Reporter: Peter Schuller
Assignee: T Jake Luciani
Fix For: 2.1 beta2
Attachments: 2434-3.patch.txt, 2434-testery.patch.txt
My reading (a while ago) of the code indicates that there is no logic
involved during bootstrapping that avoids consistency level violations. If I
recall correctly it just grabs neighbors that are currently up.
There are at least two issues I have with this behavior:
* If I have a cluster where I have applications relying on QUORUM with RF=3,
and bootstrapping complete based on only one node, I have just violated the
supposedly guaranteed consistency semantics of the cluster.
* Nodes can flap up and down at any time, so even if a human takes care to
look at which nodes are up and things about it carefully before
bootstrapping, there's no guarantee.
A complication is that not only does it depend on use-case where this is an
issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster
which is otherwise used for QUORUM operations you may wish to accept
less-than-quorum nodes during bootstrap in various emergency situations.
A potential easy fix is to have bootstrap take an argument which is the
number of hosts to bootstrap from, or to assume QUORUM if none is given.
(A related concern is bootstrapping across data centers. You may *want* to
bootstrap to a local node and then do a repair to avoid sending loads of data
across DC:s while still achieving consistency. Or even if you don't care
about the consistency issues, I don't think there is currently a way to
bootstrap from local nodes only.)
Thoughts?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5590) User defined types for CQL3
[
https://issues.apache.org/jira/browse/CASSANDRA-5590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907534#comment-13907534
]
DOAN DuyHai commented on CASSANDRA-5590:
Awesome feature again.
It is possible to have *nested* custom types ? If yes, is there a depth limit
other than the column max size to store the nested path (type1.type2.long) ?
User defined types for CQL3
---
Key: CASSANDRA-5590
URL: https://issues.apache.org/jira/browse/CASSANDRA-5590
Project: Cassandra
Issue Type: New Feature
Components: API, Core
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.1 beta1
Attachments: ocd-and-corrections-patch.txt
A typical use case for a collection could be to store a bunch of addresses in
a user profile. An address could typically be composed of a few properties:
say a street, a city, a postal code and maybe a few phone numbers associated
to it.
To model that currently with collections, you might use a {{mapstring,
blob}}, where the map key could be a string identifying the address, and the
value would be all the infos of an address serialized manually (you can use
{{text}} instead of {{blob}} and shove everything in a string if you prefer
but the principle is the same).
This ticket suggests to make this more user friendly by allowing:
{noformat}
CREATE TYPE address (
street text,
city text,
zip_code int,
phones settext
)
CREATE TABLE users (
id uuid PRIMARY KEY,
name text,
addresses mapstring, address
)
{noformat}
Under the hood, that type declaration would just be metadata on top of
CompositeType (which does mean a limitation would be that we wouldn't allow
re-ordering or removal of fields in a custom TYPE). Namely, the {{address}}
type would be in practice a {{CompositeType(UTF8Type, UTF8Type, Int32Type,
SetType(UTF8Type))}} + some metadata that records the name of each component.
In other words, this would mostly be user-friendly syntactic sugar to create
composite blobs.
I'll note that this would also be useful outside collections, as it might
sometimes be more efficient/useful to have such simple composite blob. For
instance, you could imagine to have a:
{noformat}
CREATE TYPE fullname (
firstname text,
lastname text
)
{noformat}
and to rewrite the {{users}} table above as
{noformat}
CREATE TABLE users (
id uuid PRIMARY KEY,
name fullname,
addresses mapstring, address
)
{noformat}
In terms of inserts we'd need a syntax for those new struct. Could be:
{noformat}
INSERT INTO users (id, name)
VALUES (2ad..., { firstname: 'Paul', lastname: 'smith'});
UPDATE users
SET addresses = address + { 'home': { street: '...', city: 'SF', zip_code:
94102, phones: {} } }
WHERE id=2ad...;
{noformat}
where the difference with a map is that the key would be a column name (in
the CQL3 sense), not a value/literal. Though we might find that a bit
confusing and find some other syntax.
On the query side, we could optionally allow things like:
{noformat}
SELECT name.firstname, name.lastname FROM users WHERE id=2ad...;
{noformat}
One open question however is what type do we send back in the result set
for a query like:
{noformat}
SELECT name FROM users WHERE id=2ad...;
{noformat}
We could:
# return just that it's the user defined type named {{address}}, but that
imply the client has to query the cluster metadata to find out the definition
of the type.
# return the full definition of the type every time.
I also note that client side, it might be a tad harder to support such types
cleanly in statically type languages than in dynamically typed ones, but
that's not the end of the world either.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (CASSANDRA-6577) ConcurrentModificationException during nodetool netstats
[
https://issues.apache.org/jira/browse/CASSANDRA-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907535#comment-13907535
]
Tyler Hobbs commented on CASSANDRA-6577:
+1 with the exception that your patch accidentally changes the mode on a file
in the test/ dir
ConcurrentModificationException during nodetool netstats
Key: CASSANDRA-6577
URL: https://issues.apache.org/jira/browse/CASSANDRA-6577
Project: Cassandra
Issue Type: Bug
Components: Tools
Environment: AWS EC2 machines,
java version 1.7.0_45
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Each node has 32 vnodes.
Reporter: Shao-Chuan Wang
Assignee: Joshua McKenzie
Priority: Minor
Labels: decommission, nodetool
Fix For: 2.0.6
Attachments: 6577_SessionInfo.patch, 6577_SessionInfo_v2.patch,
6577_TokenMetadata.patch
The node is leaving and I wanted to check its netstats, but it raises
ConcurrentModificationException.
{code}
[ubuntu@ip-10-4-202-48 :~]# /mnt/cassandra_latest/bin/nodetool netstats
Mode: LEAVING
Exception in thread main java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:926)
at java.util.HashMap$ValueIterator.next(HashMap.java:954)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toArrayOfCompositeData(SessionInfoCompositeData.java:161)
at
org.apache.cassandra.streaming.management.SessionInfoCompositeData.toCompositeData(SessionInfoCompositeData.java:98)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:82)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData$1.apply(StreamStateCompositeData.java:79)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Lists.newArrayList(Lists.java:146)
at com.google.common.collect.Lists.newArrayList(Lists.java:128)
at
org.apache.cassandra.streaming.management.StreamStateCompositeData.toCompositeData(StreamStateCompositeData.java:78)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:87)
at
org.apache.cassandra.streaming.StreamManager$1.apply(StreamManager.java:84)
at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators.addAll(Iterators.java:357)
at com.google.common.collect.Sets.newHashSet(Sets.java:238)
at com.google.common.collect.Sets.newHashSet(Sets.java:218)
at
org.apache.cassandra.streaming.StreamManager.getCurrentStreams(StreamManager.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at
com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at
com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at
[jira] [Assigned] (CASSANDRA-6326) Snapshot should create manifest file
[ https://issues.apache.org/jira/browse/CASSANDRA-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli reassigned CASSANDRA-6326: Assignee: sankalp kohli Snapshot should create manifest file - Key: CASSANDRA-6326 URL: https://issues.apache.org/jira/browse/CASSANDRA-6326 Project: Cassandra Issue Type: Improvement Components: Core Reporter: sankalp kohli Assignee: sankalp kohli Priority: Minor We should create a manifest file as part of the snapshot which should contain all the stables included in the snapshot. This will be very important for systems consuming this snapshot as they can validate the fact that they got the complete snapshot. If Cassandra crashes mid way creating a snapshot, I think it will create an incomplete snapshot. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (CASSANDRA-5590) User defined types for CQL3
[
https://issues.apache.org/jira/browse/CASSANDRA-5590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907539#comment-13907539
]
Aleksey Yeschenko commented on CASSANDRA-5590:
--
bq. It is possible to have nested custom types ? If yes, is there a depth limit
other than the column max size to store the nested path (type1.type2.long) ?
Yes, it's possible. No, there is no limit.
User defined types for CQL3
---
Key: CASSANDRA-5590
URL: https://issues.apache.org/jira/browse/CASSANDRA-5590
Project: Cassandra
Issue Type: New Feature
Components: API, Core
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.1 beta1
Attachments: ocd-and-corrections-patch.txt
A typical use case for a collection could be to store a bunch of addresses in
a user profile. An address could typically be composed of a few properties:
say a street, a city, a postal code and maybe a few phone numbers associated
to it.
To model that currently with collections, you might use a {{mapstring,
blob}}, where the map key could be a string identifying the address, and the
value would be all the infos of an address serialized manually (you can use
{{text}} instead of {{blob}} and shove everything in a string if you prefer
but the principle is the same).
This ticket suggests to make this more user friendly by allowing:
{noformat}
CREATE TYPE address (
street text,
city text,
zip_code int,
phones settext
)
CREATE TABLE users (
id uuid PRIMARY KEY,
name text,
addresses mapstring, address
)
{noformat}
Under the hood, that type declaration would just be metadata on top of
CompositeType (which does mean a limitation would be that we wouldn't allow
re-ordering or removal of fields in a custom TYPE). Namely, the {{address}}
type would be in practice a {{CompositeType(UTF8Type, UTF8Type, Int32Type,
SetType(UTF8Type))}} + some metadata that records the name of each component.
In other words, this would mostly be user-friendly syntactic sugar to create
composite blobs.
I'll note that this would also be useful outside collections, as it might
sometimes be more efficient/useful to have such simple composite blob. For
instance, you could imagine to have a:
{noformat}
CREATE TYPE fullname (
firstname text,
lastname text
)
{noformat}
and to rewrite the {{users}} table above as
{noformat}
CREATE TABLE users (
id uuid PRIMARY KEY,
name fullname,
addresses mapstring, address
)
{noformat}
In terms of inserts we'd need a syntax for those new struct. Could be:
{noformat}
INSERT INTO users (id, name)
VALUES (2ad..., { firstname: 'Paul', lastname: 'smith'});
UPDATE users
SET addresses = address + { 'home': { street: '...', city: 'SF', zip_code:
94102, phones: {} } }
WHERE id=2ad...;
{noformat}
where the difference with a map is that the key would be a column name (in
the CQL3 sense), not a value/literal. Though we might find that a bit
confusing and find some other syntax.
On the query side, we could optionally allow things like:
{noformat}
SELECT name.firstname, name.lastname FROM users WHERE id=2ad...;
{noformat}
One open question however is what type do we send back in the result set
for a query like:
{noformat}
SELECT name FROM users WHERE id=2ad...;
{noformat}
We could:
# return just that it's the user defined type named {{address}}, but that
imply the client has to query the cluster metadata to find out the definition
of the type.
# return the full definition of the type every time.
I also note that client side, it might be a tad harder to support such types
cleanly in statically type languages than in dynamically typed ones, but
that's not the end of the world either.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
