[jira] [Commented] (CASSANDRA-3292) creating column family sets durable_writes to true
[ https://issues.apache.org/jira/browse/CASSANDRA-3292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130415#comment-13130415 ] Hudson commented on CASSANDRA-3292: --- Integrated in Cassandra-0.8 #380 (See [https://builds.apache.org/job/Cassandra-0.8/380/]) finish fixing changing durable_writes keyspace option during CF creation patch by jbellis; reviewed by pyaskevich for CASSANDRA-3292 add KSM.systemKeyspace and cloneWith methods patch by jbellis; reviewed by pyaskevich for CASSANDRA-3292 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1185963 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/KSMetaData.java * /cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/SchemaLoader.java * /cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/config/DatabaseDescriptorTest.java * /cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/db/DefsTest.java jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1185961 Files : * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/DatabaseDescriptor.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/config/KSMetaData.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/migration/AddColumnFamily.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/migration/DropColumnFamily.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/migration/UpdateKeyspace.java creating column family sets durable_writes to true -- Key: CASSANDRA-3292 URL: https://issues.apache.org/jira/browse/CASSANDRA-3292 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.5 Reporter: Radim Kolar Assignee: Jonathan Ellis Priority: Minor Labels: schema Fix For: 0.8.8 Attachments: 0001-r-m-rename-migrations.patch, 0002-clean-up-KSM.durableWrites.patch, 0003-cloneWith.patch, 0004-update-tests-to-use-KSM.testMetadata.patch [default@rapidshare] describe keyspace rapidshare; Keyspace: rapidshare: Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy Durable Writes: *false* Options: [datacenter1:1] Column Families: [default@rapidshare] create column family t1; 1ba19300-ebfa-11e0--34912694d0bf Waiting for schema agreement... ... schemas agree across the cluster [default@rapidshare] describe keyspace rapidshare; Keyspace: rapidshare: Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy Durable Writes: *true* Options: [datacenter1:1] Column Families: ColumnFamily: t1 Key Validation Class: org.apache.cassandra.db.marshal.BytesType Default column value validator: org.apache.cassandra.db.marshal.BytesType Columns sorted by: org.apache.cassandra.db.marshal.BytesType Row cache size / save period in seconds: 0.0/0 Key cache size / save period in seconds: 20.0/14400 Memtable thresholds: 0.0281249997/1440/6 (millions of ops/minutes/MB) GC grace seconds: 864000 Compaction min/max thresholds: 4/32 Read repair chance: 1.0 Replicate on write: true Built indexes: [] -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1186069 - /cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/thrift/ThriftValidation.java
Author: slebresne Date: Wed Oct 19 10:19:57 2011 New Revision: 1186069 URL: http://svn.apache.org/viewvc?rev=1186069view=rev Log: Fix build Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/thrift/ThriftValidation.java Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/thrift/ThriftValidation.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/thrift/ThriftValidation.java?rev=1186069r1=1186068r2=1186069view=diff == --- cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/thrift/ThriftValidation.java (original) +++ cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/thrift/ThriftValidation.java Wed Oct 19 10:19:57 2011 @@ -720,16 +720,6 @@ public class ThriftValidation max_compaction_threshold)); } -public static void validateMemtableSettings(org.apache.cassandra.thrift.CfDef cf_def) throws ConfigurationException -{ -if (cf_def.isSetMemtable_flush_after_mins()) - DatabaseDescriptor.validateMemtableFlushPeriod(cf_def.memtable_flush_after_mins); -if (cf_def.isSetMemtable_throughput_in_mb()) - DatabaseDescriptor.validateMemtableThroughput(cf_def.memtable_throughput_in_mb); -if (cf_def.isSetMemtable_operations_in_millions()) - DatabaseDescriptor.validateMemtableOperations(cf_def.memtable_operations_in_millions); -} - public static void validateKeyspaceNotYetExisting(String newKsName) throws InvalidRequestException { // keyspace names must be unique case-insensitively because the keyspace name becomes the directory
svn commit: r1186072 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/
Author: slebresne Date: Wed Oct 19 10:22:28 2011 New Revision: 1186072 URL: http://svn.apache.org/viewvc?rev=1186072view=rev Log: merge from 1.0 Modified: cassandra/trunk/ (props changed) cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Oct 19 10:22:28 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1183002,1183241 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 -/cassandra/branches/cassandra-1.0:1167085-1183245,1185665,1185675 +/cassandra/branches/cassandra-1.0:1167085-1183245,1185665,1185675,1186069 /cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Oct 19 10:22:28 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1183002,1183241 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 -/cassandra/branches/cassandra-1.0/contrib:1167085-1183245,1185665,1185675 +/cassandra/branches/cassandra-1.0/contrib:1167085-1183245,1185665,1185675,1186069 /cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1/contrib:1102511-1125020 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Oct 19 10:22:28 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1183002,1183241 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018 -/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167085-1183245,1185665,1185675 +/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167085-1183245,1185665,1185675,1186069 /cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1102511-1125020 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Wed Oct 19 10:22:28 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1090934-1125013,1125019-1183002,1183241 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1101014-1125018 -/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1167085-1183245,1185665,1185675 +/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1167085-1183245,1185665,1185675,1186069 /cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1167104-1167229,1167232-1181093,1181741,1181816,1181820,1182951,1183243 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689
[jira] [Commented] (CASSANDRA-3314) Fail to delete -Index files if index is currently building
[ https://issues.apache.org/jira/browse/CASSANDRA-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130494#comment-13130494 ] Sylvain Lebresne commented on CASSANDRA-3314: - The first patch removes the import of Lock in CompactionManager which prevents compilation. But otherwise +1. Fail to delete -Index files if index is currently building -- Key: CASSANDRA-3314 URL: https://issues.apache.org/jira/browse/CASSANDRA-3314 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: Radim Kolar Assignee: Jonathan Ellis Priority: Minor Labels: compaction, indexing Fix For: 1.0.1 Attachments: 0001-cleanup.patch, 0002-acquire-references.patch If there is index building in progress, following errors are thrown if cassandra is trying to delete *-Index.db files. There is no problem with deleting -Data or -Filter.. files. CF is using leveled compaction but it is probably not related. ERROR [NonPeriodicTasks:1] 2011-10-05 09:13:03,702 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[NonPeriodicTasks:1,5,main] java.lang.RuntimeException: java.io.IOException: Failed to delete C:\var\lib\cas sandra\data\test\sipdb-h-772-Index.db at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:3 4) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:44 1) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask. access$301(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask. run(ScheduledThreadPoolExecutor.java:206) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec utor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor .java:908) at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3370) Deflate Compression corrupts SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130553#comment-13130553 ] Sylvain Lebresne commented on CASSANDRA-3370: - Still cannot reproduce. I've tried multiple times inserting 5000 keys using the stress tool using values of 10KB, 20KB, 100KB and 200KB (using 'stress -I DeflateCompressor -S 20 -n 5000'). I then try to reading both with 'stress -o RANGE_SLICE -n 5000' and by simply fetching the 100 first keys using the CLI (with a simple 'list Standard1;') and got no exceptions (the actual listing in the CLI took a while to be printed on screen because the columns are big but outside of that, no errors). Would you mind trying the same experiment (with the same tools) or providing the test script you're using so we can check if it has to do with the specific insertions or with something in your environment. Deflate Compression corrupts SSTables - Key: CASSANDRA-3370 URL: https://issues.apache.org/jira/browse/CASSANDRA-3370 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: Ubuntu Linux, amd64, Cassandra 1.0.0-rc2 Reporter: Christian Spriegel Assignee: Sylvain Lebresne Attachments: system.log Hi, it seems that the Deflate Compressor corrupts the SSTables. 3 out of 3 Installations were corrupt. Snappy works fine. Here is what I did: 1. Start a single cassandra node (I was using ByteOrderedPartitioner) 2. Write data into cf that uses deflate compression - I think it has to be enough data so that the data folder contains some files. 3. When I now try to read (I did a range scan) from my application, it fails and the logs show corruptions: Caused by: org.apache.cassandra.io.compress.CorruptedBlockException: (/home/cspriegel/Development/cassandra1/data/Test/Response-h-2-Data.db): corruption detected, chunk at 0 of length 65536. regards, Christian -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1186204 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/net/MessagingService.java
Author: jbellis Date: Wed Oct 19 13:28:01 2011 New Revision: 1186204 URL: http://svn.apache.org/viewvc?rev=1186204view=rev Log: correct dropped messages logging patch by jbellis; reviewed by brandonwilliams for CASSANDRA-3377 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1186204r1=1186203r2=1186204view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Wed Oct 19 13:28:01 2011 @@ -23,6 +23,7 @@ (CASSANDRA-3170) * fix changing durable_writes keyspace option during CF creation (CASSANDRA-3292) + * correct dropped messages logging (CASSANDRA-3377) 0.8.7 Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java?rev=1186204r1=1186203r2=1186204view=diff == --- cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java (original) +++ cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java Wed Oct 19 13:28:01 2011 @@ -609,7 +609,8 @@ public final class MessagingService impl if (recent 0) { logTpstats = true; -logger_.info({} {} messages dropped in server lifetime, recent, verb); +logger_.info({} {} messages dropped in last {}ms, + new Object[] {recent, verb, LOG_DROPPED_INTERVAL_IN_MS}); lastDroppedInternal.put(verb, dropped.get()); } }
svn commit: r1186207 - in /cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra: db/ db/compaction/ db/index/ io/sstable/ streaming/
Author: jbellis Date: Wed Oct 19 13:31:25 2011 New Revision: 1186207 URL: http://svn.apache.org/viewvc?rev=1186207view=rev Log: cleanup for #3314 Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/compaction/CompactionManager.java cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexBuilder.java cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/io/sstable/ReducingKeyIterator.java cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/streaming/StreamInSession.java Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1186207r1=1186206r2=1186207view=diff == --- cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java (original) +++ cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java Wed Oct 19 13:31:25 2011 @@ -886,11 +886,6 @@ public class ColumnFamilyStore implement return false; } -public boolean isKeyExistenceExpensive(Set? extends SSTable sstablesToIgnore) -{ -return compactionStrategy.isKeyExistenceExpensive(sstablesToIgnore); -} - /* * Called after a BinaryMemtable flushes its in-memory data, or we add a file * via bootstrap. This information is cached in the ColumnFamilyStore. Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/compaction/CompactionManager.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/compaction/CompactionManager.java?rev=1186207r1=1186206r2=1186207view=diff == --- cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/compaction/CompactionManager.java (original) +++ cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/compaction/CompactionManager.java Wed Oct 19 13:31:25 2011 @@ -772,7 +772,7 @@ public class CompactionManager implement } // flush to ensure we don't lose the tombstones on a restart, since they are not commitlog'd -cfs.indexManager.flushIndexes(); +cfs.indexManager.flushIndexesBlocking(); cfs.replaceCompactedSSTables(Arrays.asList(sstable), results); Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexBuilder.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexBuilder.java?rev=1186207r1=1186206r2=1186207view=diff == --- cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexBuilder.java (original) +++ cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexBuilder.java Wed Oct 19 13:31:25 2011 @@ -29,8 +29,7 @@ import org.apache.cassandra.db.compactio import org.apache.cassandra.io.sstable.ReducingKeyIterator; /** - * Manages building an entire index from column family data - * Sent to compaction manager + * Manages building an entire index from column family data. Runs on to compaction manager. */ public class SecondaryIndexBuilder implements CompactionInfo.Holder { Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java?rev=1186207r1=1186206r2=1186207view=diff == --- cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java (original) +++ cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndexManager.java Wed Oct 19 13:31:25 2011 @@ -97,8 +97,11 @@ public class SecondaryIndexManager /** - * Does a full rebuild of the indexes specified by columns from the sstables. + * Does a full, blocking rebuild of the indexes specified by columns from the sstables. * Does nothing if columns is empty. + * + * Caller must acquire and release references to the sstables used here. + * * @param sstables the data to build from * @param columns the list of columns to index * @throws IOException @@ -116,7 +119,7 @@ public class SecondaryIndexManager try {
svn commit: r1186208 - in /cassandra/branches/cassandra-1.0: CHANGES.txt src/java/org/apache/cassandra/db/ColumnFamilyStore.java src/java/org/apache/cassandra/db/index/SecondaryIndex.java
Author: jbellis Date: Wed Oct 19 13:32:53 2011 New Revision: 1186208 URL: http://svn.apache.org/viewvc?rev=1186208view=rev Log: acquire references during index build patch by jbellis; reviewed by slebresne for CASSANDRA-3314 Modified: cassandra/branches/cassandra-1.0/CHANGES.txt cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndex.java Modified: cassandra/branches/cassandra-1.0/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/CHANGES.txt?rev=1186208r1=1186207r2=1186208view=diff == --- cassandra/branches/cassandra-1.0/CHANGES.txt (original) +++ cassandra/branches/cassandra-1.0/CHANGES.txt Wed Oct 19 13:32:53 2011 @@ -1,4 +1,6 @@ 1.0.1 + * acquire references during index build to prevent delete problems + on Windows (CASSANDRA-3314) * describe_ring should include datacenter/topology information (CASSANDRA-2882) * Thrift sockets are not properly buffered (CASSANDRA-3261) * performance improvement for bytebufferutil compare function (CASSANDRA-3286) Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1186208r1=1186207r2=1186208view=diff == --- cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java (original) +++ cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java Wed Oct 19 13:32:53 2011 @@ -562,8 +562,8 @@ public class ColumnFamilyStore implement } logger.info(Loading new SSTable Set for + table.name + / + columnFamily + : + sstables); +SSTableReader.acquireReferences(sstables); data.addSSTables(sstables); // this will call updateCacheSizes() for us - logger.info(Requesting a full secondary index re-build for + table.name + / + columnFamily); try { @@ -573,6 +573,10 @@ public class ColumnFamilyStore implement { throw new IOError(e); } +finally +{ +SSTableReader.releaseReferences(sstables); +} logger.info(Setting up new generation: + generation); fileIndexGenerator.set(generation); Modified: cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndex.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndex.java?rev=1186208r1=1186207r2=1186208view=diff == --- cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndex.java (original) +++ cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/index/SecondaryIndex.java Wed Oct 19 13:32:53 2011 @@ -30,6 +30,7 @@ import org.apache.cassandra.db.SystemTab import org.apache.cassandra.db.compaction.CompactionManager; import org.apache.cassandra.db.index.keys.KeysIndex; import org.apache.cassandra.io.sstable.ReducingKeyIterator; +import org.apache.cassandra.io.sstable.SSTableReader; import org.apache.cassandra.utils.ByteBufferUtil; import org.apache.commons.lang.StringUtils; import org.slf4j.Logger; @@ -149,11 +150,11 @@ public abstract class SecondaryIndex for (ColumnDefinition cdef : columnDefs) columnNames.add(cdef.name); - + +CollectionSSTableReader sstables = baseCfs.markCurrentSSTablesReferenced(); SecondaryIndexBuilder builder = new SecondaryIndexBuilder(baseCfs, columnNames, - new ReducingKeyIterator(baseCfs.getSSTables())); - + new ReducingKeyIterator(sstables)); Future? future = CompactionManager.instance.submitIndexBuild(builder); try { @@ -179,6 +180,10 @@ public abstract class SecondaryIndex { throw new IOException(e); } +finally +{ +SSTableReader.releaseReferences(sstables); +} logger.info(Index build of + getIndexName() + complete); }
svn commit: r1186209 [2/2] - in /cassandra/trunk: ./ contrib/ contrib/pig/src/java/org/apache/cassandra/hadoop/pig/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/avro/ src/java/org/apache
Modified: cassandra/trunk/src/java/org/apache/cassandra/hadoop/ConfigHelper.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/hadoop/ConfigHelper.java?rev=1186209r1=1186208r2=1186209view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/hadoop/ConfigHelper.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/hadoop/ConfigHelper.java Wed Oct 19 13:35:57 2011 @@ -19,9 +19,13 @@ package org.apache.cassandra.hadoop; * under the License. * */ +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; import org.apache.cassandra.config.ConfigurationException; import org.apache.cassandra.dht.IPartitioner; +import org.apache.cassandra.thrift.Cassandra; import org.apache.cassandra.thrift.KeyRange; import org.apache.cassandra.thrift.SlicePredicate; import org.apache.cassandra.thrift.TBinaryProtocol; @@ -31,6 +35,13 @@ import org.apache.hadoop.conf.Configurat import org.apache.thrift.TDeserializer; import org.apache.thrift.TException; import org.apache.thrift.TSerializer; +import org.apache.thrift.transport.TFramedTransport; +import org.apache.thrift.transport.TSocket; +import org.apache.thrift.transport.TTransport; +import org.apache.thrift.transport.TTransportException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + public class ConfigHelper { @@ -54,6 +65,9 @@ public class ConfigHelper private static final String INITIAL_THRIFT_ADDRESS = cassandra.thrift.address; private static final String READ_CONSISTENCY_LEVEL = cassandra.consistencylevel.read; private static final String WRITE_CONSISTENCY_LEVEL = cassandra.consistencylevel.write; + +private static final Logger logger = LoggerFactory.getLogger(ColumnFamilyInputFormat.class); + /** * Set the keyspace and column family for the input of this job. @@ -331,4 +345,50 @@ public class ConfigHelper throw new RuntimeException(e); } } + + +public static Cassandra.Client getClientFromAddressList(Configuration conf) throws IOException +{ +String[] addresses = ConfigHelper.getInitialAddress(conf).split(,); +Cassandra.Client client = null; +ListIOException exceptions = new ArrayListIOException(); +for (String address : addresses) +{ +try +{ +client = createConnection(address, ConfigHelper.getRpcPort(conf), true); +break; +} +catch (IOException ioe) +{ +exceptions.add(ioe); +} +} +if (client == null) +{ +logger.error(failed to connect to any initial addresses); +for (IOException ioe : exceptions) +{ +logger.error(, ioe); +} +throw exceptions.get(exceptions.size() - 1); +} +return client; +} + +public static Cassandra.Client createConnection(String host, Integer port, boolean framed) +throws IOException +{ +TSocket socket = new TSocket(host, port); +TTransport trans = framed ? new TFramedTransport(socket) : socket; +try +{ +trans.open(); +} +catch (TTransportException e) +{ +throw new IOException(unable to connect to server, e); +} +return new Cassandra.Client(new TBinaryProtocol(trans)); +} } Modified: cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java?rev=1186209r1=1186208r2=1186209view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java Wed Oct 19 13:35:57 2011 @@ -199,7 +199,7 @@ public class CompressionMetadata seek(dataLengthOffset + 8 // size reserved for uncompressed data length + 4 // size reserved for chunk count - + (chunkIndex * 8)); + + (chunkIndex * 8L)); try { @@ -221,7 +221,7 @@ public class CompressionMetadata seek(dataLengthOffset + 8 // size reserved for uncompressed data length + 4 // size reserved for chunk count - + (chunkIndex * 8)); + + (chunkIndex * 8L)); getChannel().truncate(getFilePointer()); } } Modified: cassandra/trunk/src/java/org/apache/cassandra/io/sstable/ReducingKeyIterator.java URL:
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure on builder cassandra-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1732 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra/trunk] 1186209 Blamelist: jbellis BUILD FAILED: failed compile sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-3377) correct dropped messages logging
[ https://issues.apache.org/jira/browse/CASSANDRA-3377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130640#comment-13130640 ] Hudson commented on CASSANDRA-3377: --- Integrated in Cassandra-0.8 #381 (See [https://builds.apache.org/job/Cassandra-0.8/381/]) correct dropped messages logging patch by jbellis; reviewed by brandonwilliams for CASSANDRA-3377 jbellis : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1186204 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/net/MessagingService.java correct dropped messages logging Key: CASSANDRA-3377 URL: https://issues.apache.org/jira/browse/CASSANDRA-3377 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.4 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Labels: logging Fix For: 0.8.8, 1.0.1 Attachments: 3377.txt CASSANDRA-3004 switched MessagingService back to logging only recent dropped messages instead of server lifetime totals, but the log message was not updated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3370) Deflate Compression corrupts SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130685#comment-13130685 ] Christian Spriegel commented on CASSANDRA-3370: --- Ok, I will try the stress tool. Just to be sure: You are talking about stress.py and not the java-based stress? Because I was trying the java stress and it did not accept the -I parameter. Deflate Compression corrupts SSTables - Key: CASSANDRA-3370 URL: https://issues.apache.org/jira/browse/CASSANDRA-3370 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: Ubuntu Linux, amd64, Cassandra 1.0.0-rc2 Reporter: Christian Spriegel Assignee: Sylvain Lebresne Attachments: system.log Hi, it seems that the Deflate Compressor corrupts the SSTables. 3 out of 3 Installations were corrupt. Snappy works fine. Here is what I did: 1. Start a single cassandra node (I was using ByteOrderedPartitioner) 2. Write data into cf that uses deflate compression - I think it has to be enough data so that the data folder contains some files. 3. When I now try to read (I did a range scan) from my application, it fails and the logs show corruptions: Caused by: org.apache.cassandra.io.compress.CorruptedBlockException: (/home/cspriegel/Development/cassandra1/data/Test/Response-h-2-Data.db): corruption detected, chunk at 0 of length 65536. regards, Christian -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3116) Compactions can (seriously) delay schema migrations
[ https://issues.apache.org/jira/browse/CASSANDRA-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130688#comment-13130688 ] Sylvain Lebresne commented on CASSANDRA-3116: - Not sure this work correctly. I believe we first have a problem with DT.removeAllSSTables(), because this is during the drop and really do remove *all* sstables, including the ones that are being compacted (and thus it will unreference those while they are being compacted, which is bad). So we should first change that to only remove the one that are not compacting. Then we must make sure that anything that was not removed by that gets removed later. Which involves removing any flushed memtable (though that doesn't really matter since a dropped CF is flushed before being invalidated) and we must make sure that compacted sstable do are marked compacted but also that replacements are directly marked as compacted too (which mainly involve that we call removeOldSSTablesSize() on them). And I suppose we could make sure no new compaction is automatically triggered on an invalidated CF so we don't have a race or something. bq. compactionLock is still used but only for major compaction. should we get rid of that too and say if you want to be absolutely sure you're compacting everything, disable minor compactions before invoking major? I think there is really no much cost to keeping the lock in there if the write lock is only acquired by events triggered by a user and I would prefer having major compaction do what it pretend by default rather that having a complicated procedure. That being said, I would be for replacing the global compactionLock by one lock per CF (which should be easy). Compactions can (seriously) delay schema migrations --- Key: CASSANDRA-3116 URL: https://issues.apache.org/jira/browse/CASSANDRA-3116 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: Eric Evans Assignee: Jonathan Ellis Labels: compaction Fix For: 1.1 Attachments: 3116.txt A compaction lock is acquired when dropping keyspaces or column families which will cause the schema migration to block if a compaction is in progress. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3370) Deflate Compression corrupts SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130691#comment-13130691 ] Sylvain Lebresne commented on CASSANDRA-3370: - No, I'm talking of the java one. The python one is old and won't support compression for instance. The java one in the 1.0.0 source does support compression through the -I parameter. Are you sure you're looking at the right version ? Deflate Compression corrupts SSTables - Key: CASSANDRA-3370 URL: https://issues.apache.org/jira/browse/CASSANDRA-3370 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: Ubuntu Linux, amd64, Cassandra 1.0.0-rc2 Reporter: Christian Spriegel Assignee: Sylvain Lebresne Attachments: system.log Hi, it seems that the Deflate Compressor corrupts the SSTables. 3 out of 3 Installations were corrupt. Snappy works fine. Here is what I did: 1. Start a single cassandra node (I was using ByteOrderedPartitioner) 2. Write data into cf that uses deflate compression - I think it has to be enough data so that the data folder contains some files. 3. When I now try to read (I did a range scan) from my application, it fails and the logs show corruptions: Caused by: org.apache.cassandra.io.compress.CorruptedBlockException: (/home/cspriegel/Development/cassandra1/data/Test/Response-h-2-Data.db): corruption detected, chunk at 0 of length 65536. regards, Christian -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3370) Deflate Compression corrupts SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130695#comment-13130695 ] Christian Spriegel commented on CASSANDRA-3370: --- I see! I got the wrong version. Sorry, I did not know that it was included in the cassandra source. I thought I had to download it some place else. I will try with that and let you know about the results... Deflate Compression corrupts SSTables - Key: CASSANDRA-3370 URL: https://issues.apache.org/jira/browse/CASSANDRA-3370 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: Ubuntu Linux, amd64, Cassandra 1.0.0-rc2 Reporter: Christian Spriegel Assignee: Sylvain Lebresne Attachments: system.log Hi, it seems that the Deflate Compressor corrupts the SSTables. 3 out of 3 Installations were corrupt. Snappy works fine. Here is what I did: 1. Start a single cassandra node (I was using ByteOrderedPartitioner) 2. Write data into cf that uses deflate compression - I think it has to be enough data so that the data folder contains some files. 3. When I now try to read (I did a range scan) from my application, it fails and the logs show corruptions: Caused by: org.apache.cassandra.io.compress.CorruptedBlockException: (/home/cspriegel/Development/cassandra1/data/Test/Response-h-2-Data.db): corruption detected, chunk at 0 of length 65536. regards, Christian -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3100) Secondary index still does minor compacting after deleting index
[ https://issues.apache.org/jira/browse/CASSANDRA-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130704#comment-13130704 ] Sylvain Lebresne commented on CASSANDRA-3100: - I agree with the cure being worse than the disease. I think it will piss off people when they have to wait 2 hours for a simple column family update because a long compaction is running. We should at least only grab the locks when the update drops an index. Even then are we sure this is solving the problem at end? In 0.7, I'm not sure why not holding the flush/compaction locks during the index drop would trigger what is described here. In particular, as long as the schema change has succeed, a bootstrapping node shouldn't be trying to build a secondary index that isn't there. All this being said, I think we may have a legit bug in 1.0.0, since an update of column family can drop an index which will call removeAllSSTables which doesn't work as expected if the compaction lock is not hold. But this should be fixed by CASSANDRA-3116. Secondary index still does minor compacting after deleting index Key: CASSANDRA-3100 URL: https://issues.apache.org/jira/browse/CASSANDRA-3100 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.7.0 Reporter: Jeremy Hanna Assignee: Jonathan Ellis Priority: Minor Labels: compaction Fix For: 0.7.10 Attachments: 3100.txt We deleted all of our secondary indexes. A couple of days later I was watching compactionstats on one of the nodes and it was in the process of minor compacting one of the deleted secondary indexes. I double checked the keyspace definitions on the CLI and there were no secondary indexes defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3383) Make Row cache reject wide rows which are more than x columns
Make Row cache reject wide rows which are more than x columns - Key: CASSANDRA-3383 URL: https://issues.apache.org/jira/browse/CASSANDRA-3383 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Environment: Linux with Sun JDK Reporter: Vijay Assignee: Vijay Priority: Minor The problem with the row cache is it has to cache everything in memory, this works great for small rows but not for the wide rows. Even of off-heap cache has to read everything into the heap eventually causing GC overhead. It will be nice to reject the rows which are bigger than x columns (This can be a KS setting) this will help cases where we have mixed load of small and wide column datasets in the same cf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3363) Allow one leveled compaction task to kick off another
[ https://issues.apache.org/jira/browse/CASSANDRA-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130747#comment-13130747 ] Sylvain Lebresne commented on CASSANDRA-3363: - An alternative to waiting the latch before triggering the next task would be to make sure that we've released the latch before triggering that next task. I see two easy ways to do this: # move the submit of a next task outside of the compactionTask completely and move it in CompactionManager after the task execution. I kind of like that because I don't think triggering the next compaction should be the job of the compactionTask itself (but I'm more talking of a feeling here than anything technical). # have compactionTask call a protected finalize() method at the end of execute (the exact same than the submitNextTask of this patch but I'd prefer a less specific name). The leveledCompaction would only override that (i.e, it wouldn't override execute anymore) and countDown the latch there. I think those would be less ugly, though the patch looks ok technically speaking. Allow one leveled compaction task to kick off another - Key: CASSANDRA-3363 URL: https://issues.apache.org/jira/browse/CASSANDRA-3363 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Labels: compaction Fix For: 1.0.1 Attachments: 3363.txt Leveled compaction wants to prevent multiple tasks from running at once, but this check also defeats the kick off another compaction if there is more work to do code in CompactionTask. So currently LCS relies completely on the every-five-minutes compaction check, which is not enough to keep up with heavy insert load. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3313) Cancelling index build throws assert error
[ https://issues.apache.org/jira/browse/CASSANDRA-3313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130750#comment-13130750 ] Sylvain Lebresne commented on CASSANDRA-3313: - +1 nitpick: to be uber pedantic, we could use the row key validator to print the row key in the log messages. Cancelling index build throws assert error -- Key: CASSANDRA-3313 URL: https://issues.apache.org/jira/browse/CASSANDRA-3313 Project: Cassandra Issue Type: Bug Affects Versions: 0.7.0 Reporter: Radim Kolar Assignee: Jonathan Ellis Priority: Minor Labels: indexing Fix For: 1.0.1 Attachments: 3313.txt Canceling index build throws this, but checking log there was no compaction running in background. INFO 08:46:41,343 Writing Memtable-IndexInfo@9480253(34/42 serialized/live byte s, 1 ops) ERROR 08:46:41,343 Fatal exception in thread Thread[CompactionExecutor:3,5,main] java.lang.AssertionError at org.apache.cassandra.db.index.SecondaryIndexManager.applyIndexUpdates (SecondaryIndexManager.java:397) at org.apache.cassandra.db.Table.indexRow(Table.java:534) at org.apache.cassandra.db.index.SecondaryIndexBuilder.build(SecondaryIn dexBuilder.java:64) at org.apache.cassandra.db.compaction.CompactionManager$7.run(Compaction Manager.java:856) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:44 1) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec utor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor .java:908) at java.lang.Thread.run(Thread.java:662) INFO 08:46:41,531 Completed flushing \var\lib\cassandra\data\system\IndexInfo-h -1-Data.db (88 bytes) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3291) 1.0 needs to clean out old-style hints
[ https://issues.apache.org/jira/browse/CASSANDRA-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130753#comment-13130753 ] Yang Yang commented on CASSANDRA-3291: -- I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more 1.0 needs to clean out old-style hints -- Key: CASSANDRA-3291 URL: https://issues.apache.org/jira/browse/CASSANDRA-3291 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 1.0.0 Attachments: 3291.txt (Only marking this Minor because the manual workaround of deleting hint files is trivial.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3334) dropping index causes some inflight mutations to fail
[ https://issues.apache.org/jira/browse/CASSANDRA-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130758#comment-13130758 ] Sylvain Lebresne commented on CASSANDRA-3334: - +1 dropping index causes some inflight mutations to fail - Key: CASSANDRA-3334 URL: https://issues.apache.org/jira/browse/CASSANDRA-3334 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: windows Reporter: Radim Kolar Assignee: Jonathan Ellis Priority: Minor Labels: indexing Fix For: 1.0.1 Attachments: 0001-cleanup.patch, 0002-fix.patch dropping index causes some inflight mutations to fail. hector on client side didnt throw any exception INFO [MigrationStage:1] 2011-10-07 23:11:53,742 Migration.java (line 119) Applying migration fb1a8540-f128-11e0--23b38323f4da Update column family to org.apache.cassandra.config.CFMetaData@786669[cfId=1000,ksName=test,cfName=sipdb,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,subcolumncomparator=null,comment=phone calls routing information,rowCacheSize=0.0,keyCacheSize=0.0,readRepairChance=0.0,replicateOnWrite=false,gcGraceSeconds=0,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.Int32Type,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=0,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@8bb33c,mergeShardsChance=0.1,keyAlias=java.nio.HeapByteBuffer[pos=461 lim=464 cap=466],column_metadata={java.nio.HeapByteBuffer[pos=0 lim=3 cap=3]=ColumnDefinition{name=6b616d, validator=org.apache.cassandra.db.marshal.AsciiType, index_type=null, index_name='null'}},compactionStrategyClass=class org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}] INFO [MigrationStage:1] 2011-10-07 23:11:53,805 ColumnFamilyStore.java (line 664) Enqueuing flush of Memtable-Migrations@27537043(7860/9825 serialized/live bytes, 1 ops) INFO [MigrationStage:1] 2011-10-07 23:11:53,820 ColumnFamilyStore.java (line 664) Enqueuing flush of Memtable-Schema@8340427(3320/4150 serialized/live bytes, 3 ops) INFO [FlushWriter:3] 2011-10-07 23:11:53,820 Memtable.java (line 237) Writing Memtable-Migrations@27537043(7860/9825 serialized/live bytes, 1 ops) INFO [FlushWriter:3] 2011-10-07 23:11:55,008 Memtable.java (line 273) Completed flushing \var\lib\cassandra\data\system\Migrations-h-14-Data.db (7924 bytes) INFO [FlushWriter:3] 2011-10-07 23:11:55,008 Memtable.java (line 237) Writing Memtable-Schema@8340427(3320/4150 serialized/live bytes, 3 ops) INFO [CompactionExecutor:4] 2011-10-07 23:11:55,008 CompactionTask.java (line 119) Compacting [SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-13-Data.db'), SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-14-Data.db'), SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-11-Data.db'), SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-12-Data.db')] INFO [FlushWriter:3] 2011-10-07 23:11:56,430 Memtable.java (line 273) Completed flushing \var\lib\cassandra\data\system\Schema-h-14-Data.db (3470 bytes) INFO [CompactionExecutor:3] 2011-10-07 23:11:56,446 CompactionTask.java (line 119) Compacting [SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-13-Data.db'), SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-14-Data.db'), SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-12-Data.db'), SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-11-Data.db')] ERROR [MutationStage:56] 2011-10-07 23:11:56,508 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[MutationStage:56,5,main] java.lang.AssertionError at org.apache.cassandra.db.index.SecondaryIndexManager.applyIndexUpdates(SecondaryIndexManager.java:369) at org.apache.cassandra.db.Table.apply(Table.java:457) at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:253) at org.apache.cassandra.service.StorageProxy$5.runMayThrow(StorageProxy.java:436) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1263) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) ERROR [MutationStage:51] 2011-10-07 23:11:56,539 AbstractCassandraDaemon.java (line 133) Fatal exception in thread
[jira] [Commented] (CASSANDRA-3383) Make Row cache reject wide rows which are more than x columns
[ https://issues.apache.org/jira/browse/CASSANDRA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130768#comment-13130768 ] Jonathan Ellis commented on CASSANDRA-3383: --- My reaction would be that 1) if you really have different data sets involved, the right solution would be to split those into different CFs 2) but if you have the same data set where some outlier rows are much larger than others then a query cache like CASSANDRA-1956 might be a good general purpose solution Make Row cache reject wide rows which are more than x columns - Key: CASSANDRA-3383 URL: https://issues.apache.org/jira/browse/CASSANDRA-3383 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Environment: Linux with Sun JDK Reporter: Vijay Assignee: Vijay Priority: Minor The problem with the row cache is it has to cache the whole row into the memory, this works great for small rows but not for the wide rows. Even off-heap cache has to read everything into the heap eventually causing GC overhead (If your query is to get few columns). It will be nice to reject the rows which are bigger than x columns (This can be a KS setting) this will help cases where we have mixed load of small and wide column datasets in the same cf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3065) Major file corruption after running nodetool cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130770#comment-13130770 ] Sylvain Lebresne commented on CASSANDRA-3065: - Realizing this one has been sitting over there for some time now. If you still have that corrupted sstable around, I can offer to take a look. Otherwise, there isn't much to chew on on those stacktraces (I mean, other than there is something wrong with those sstables). Major file corruption after running nodetool cleanup Key: CASSANDRA-3065 URL: https://issues.apache.org/jira/browse/CASSANDRA-3065 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.3 Reporter: Benjamin Schrauwen Assignee: Sylvain Lebresne After running nodetool cleanup on two of the nodes in my 4 node cluster, almost all SSTables on those those machine got corrupted. I am not able to read them anymore with sstable2json, and the cassandra daemon is repetitively throwing: ERROR [ReadStage:11] 2011-08-20 04:44:46,846 AbstractCassandraDaemon.java (line 139) Fatal exception in thread Thread[ReadStage:11,5,main] java.lang.RuntimeException: java.lang.IndexOutOfBoundsException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Buffer.java:514) at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:209) at org.apache.cassandra.io.util.MappedFileDataInput.read(MappedFileDataInput.java:104) at java.io.InputStream.read(InputStream.java:154) at org.apache.cassandra.io.util.AbstractDataInput.readInt(AbstractDataInput.java:196) at org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:61) at org.apache.cassandra.db.columniterator.SimpleSliceReader.init(SimpleSliceReader.java:58) at org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:91) at org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:67) at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:66) at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1314) at org.apache.cassandra.db.ColumnFamilyStore.cacheRow(ColumnFamilyStore.java:1181) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1221) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1168) at org.apache.cassandra.db.Table.getRow(Table.java:385) at org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:58) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:641) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3384) it would be nice if show schema; in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3384: -- Reviewer: vijay2...@yahoo.com Affects Version/s: (was: 1.1) 0.8.4 Fix Version/s: 1.0.1 0.8.8 Assignee: Pavel Yaskevich Issue Type: Bug (was: Improvement) it would be nice if show schema; in cli shows Cache Provider Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Show schema in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3291) 1.0 needs to clean out old-style hints
[ https://issues.apache.org/jira/browse/CASSANDRA-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130780#comment-13130780 ] Sylvain Lebresne commented on CASSANDRA-3291: - Note sure what is the current HEAD of 1.0.0 given that branch as been removed in favor of the 1.0 one. But if it still points to the line {noformat} assert mutationColumn != null; {noformat} then I'm willing to bet it's another bug, since I believe the preceding assertions should have fired before this one if this was old-style hints. In which case, do you mind opening a new ticket? 1.0 needs to clean out old-style hints -- Key: CASSANDRA-3291 URL: https://issues.apache.org/jira/browse/CASSANDRA-3291 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 1.0.0 Attachments: 3291.txt (Only marking this Minor because the manual workaround of deleting hint files is trivial.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3384) it would be nice if show schema; in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130789#comment-13130789 ] Pavel Yaskevich commented on CASSANDRA-3384: In the latest 0.8 branch I see row_cache_provider showing, can you clarify what else do you want to see there? it would be nice if show schema; in cli shows Cache Provider Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Show schema in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3385) NPE in hinted handoff
NPE in hinted handoff - Key: CASSANDRA-3385 URL: https://issues.apache.org/jira/browse/CASSANDRA-3385 Project: Cassandra Issue Type: Bug Reporter: Yang Yang I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more this could possibly be related to #3291 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3291) 1.0 needs to clean out old-style hints
[ https://issues.apache.org/jira/browse/CASSANDRA-3291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130810#comment-13130810 ] Yang Yang commented on CASSANDRA-3291: -- created #3385, thanks 1.0 needs to clean out old-style hints -- Key: CASSANDRA-3291 URL: https://issues.apache.org/jira/browse/CASSANDRA-3291 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 1.0.0 Attachments: 3291.txt (Only marking this Minor because the manual workaround of deleting hint files is trivial.) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3385) NPE in hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130814#comment-13130814 ] Yang Yang commented on CASSANDRA-3385: -- enabled assertion INFO [HintedHandoff:1] 2011-10-19 13:44:08,346 HintedHandOffManager.java (line 263) Started hinted handoff for token: 0 with IP: /10.196.37.187 ERROR [HintedHandoff:1] 2011-10-19 13:44:08,513 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.AssertionError at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:285) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) line 285 is: assert versionColumn != null; well the other following fields could also be null too, assert versionColumn != null; assert tableColumn != null; assert keyColumn != null; assert mutationColumn != null; NPE in hinted handoff - Key: CASSANDRA-3385 URL: https://issues.apache.org/jira/browse/CASSANDRA-3385 Project: Cassandra Issue Type: Bug Reporter: Yang Yang I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more this could possibly be related to #3291 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3384) it would be nice if show schema; in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams resolved CASSANDRA-3384. - Resolution: Not A Problem I too see this in 0.8. it would be nice if show schema; in cli shows Cache Provider Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Show schema in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3384) it would be nice if show schema; in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130840#comment-13130840 ] Vijay commented on CASSANDRA-3384: -- I dont see it in 0.8.6 may be in the later version? [default@unknown] use Keyspace2; Authenticated to keyspace: Keyspace2 [default@Keyspace2] update column family Standard2 with rows_cached=1000 and row_cache_provider=SerializingCacheProvider; bf5faf90-fa7d-11e0--90ba79db4aff Waiting for schema agreement... [default@Keyspace2] describe keyspace Keyspace2; WARNING: Could not connect to the JMX on 0.0.0.0:7199, information won't be shown. Keyspace: Keyspace2: Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy Durable Writes: true Options: [us-east:2] Column Families: ColumnFamily: Standard2 Key Validation Class: org.apache.cassandra.db.marshal.BytesType Default column value validator: org.apache.cassandra.db.marshal.BytesType Columns sorted by: org.apache.cassandra.db.marshal.BytesType Row cache size / save period in seconds: 1000.0/0 Key cache size / save period in seconds: 1.0/14400 Memtable thresholds: 3.5390625/120/128 (millions of ops/minutes/MB) GC grace seconds: 864000 Compaction min/max thresholds: 4/32 Read repair chance: 1.0 Replicate on write: true [default@Keyspace2] it would be nice if show schema; in cli shows Cache Provider Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Show schema in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3384) it would be nice if show schema; in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130843#comment-13130843 ] Pavel Yaskevich commented on CASSANDRA-3384: cassandra-0.8 branch has it. it would be nice if show schema; in cli shows Cache Provider Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Show schema in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3251) CassandraStorage uses comparator for both super column names and sub column names.
[ https://issues.apache.org/jira/browse/CASSANDRA-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130850#comment-13130850 ] Brandon Williams commented on CASSANDRA-3251: - Dana, can you rebase? I can't get this to apply. CassandraStorage uses comparator for both super column names and sub column names. -- Key: CASSANDRA-3251 URL: https://issues.apache.org/jira/browse/CASSANDRA-3251 Project: Cassandra Issue Type: Bug Components: Contrib, Hadoop Affects Versions: 0.8.6 Reporter: Dana H. P'Simer, Jr. Labels: cassandra, hadoop, pig Fix For: 0.8.8 Attachments: CASSANDRA-3251.patch The CassandraStorage class uses the same comparator for super and sub column names. This is because it calls columnsToTuple recursively without any indication that the subsequent call is for sub columns. Also, the getDefaultMarshallers method does not return the sub column name comparator. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3327) Support TimeUUID in CassandraStorage
[ https://issues.apache.org/jira/browse/CASSANDRA-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3327: Attachment: 3327-v2.txt v2 uses UUIDGen to decompose to a byte array. Support TimeUUID in CassandraStorage Key: CASSANDRA-3327 URL: https://issues.apache.org/jira/browse/CASSANDRA-3327 Project: Cassandra Issue Type: Bug Components: Contrib Affects Versions: 0.8.7 Environment: Cassandra 0.8.6 Build #348 (CASSANDRA-2777 + CASSANDRA-2810) Reporter: Manuel Kreutz Assignee: Brandon Williams Labels: pig Fix For: 0.8.8 Attachments: 3327-v2.txt, 3327.txt Cassandra CLI: {code} grunt raw = LOAD 'cassandra://TEST/CF' USING CassandraStorage() AS ( key:chararray, columns:bag { column:tuple( name, value ) }); grunt describe raw; raw: {key: chararray,columns: {(name: bytearray,value: bytearray)}} log_test = FOREACH raw GENERATE (CHARARRAY) key, flatten(columns); grunt DUMP log_test; {code} Returns: {code} org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias log_test. Backend error : Unexpected data type java.util.UUID found in stream. Note only standard Pig type is supported when you output from UDF/LoadFunc at org.apache.pig.PigServer.openIterator(PigServer.java:890) at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:655) at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164) at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:67) at org.apache.pig.Main.run(Main.java:487) at org.apache.pig.Main.main(Main.java:108) Caused by: java.lang.RuntimeException: Unexpected data type java.util.UUID found in stream. Note only standard Pig type is supported when you output from UDF/LoadFunc at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:478) at org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:542) at org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357) at org.apache.pig.impl.io.InterRecordWriter.write(InterRecordWriter.java:73) at org.apache.pig.impl.io.InterStorage.putNext(InterStorage.java:87) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:138) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:97) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:498) at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:263) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:256) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) {code} According to driftx on IRC the setTupleValue function in CassandraStorage needs to handle the uuid case and cast it to a DataByteArray. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3384) it would be nice if show schema; in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130903#comment-13130903 ] Vijay commented on CASSANDRA-3384: -- Pavel, I dont see it... are you seeing it by doing show keyspaces/describe keyspace name? i dont see in 1.0 or the latest 0.8 build. [default@vj] update column family Standard1 with rows_cached=1000 and row_cache_provider=SerializingCacheProvider; 7ed92090-fa88-11e0--242d50cf1ffe Waiting for schema agreement... ... schemas agree across the cluster [default@vj] describe keyspace vj; Keyspace: vj: Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy Durable Writes: true Options: [datacenter1:1] Column Families: ColumnFamily: Standard1 Key Validation Class: org.apache.cassandra.db.marshal.BytesType Default column value validator: org.apache.cassandra.db.marshal.BytesType Columns sorted by: org.apache.cassandra.db.marshal.BytesType Row cache size / save period in seconds: 1000.0/0 Key cache size / save period in seconds: 20.0/14400 Memtable thresholds: 0.290624997/1440/62 (millions of ops/minutes/MB) GC grace seconds: 864000 Compaction min/max thresholds: 4/32 Read repair chance: 1.0 Replicate on write: true Built indexes: [] [default@vj] it would be nice if show schema; in cli shows Cache Provider Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Show schema in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3384) it would be nice if show schema; in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130908#comment-13130908 ] Pavel Yaskevich commented on CASSANDRA-3384: Take a look at the title, it states that command is show schema; and open CliClient.java:1673 under cassandra-0.8, let me check if describe keyspace shows it. it would be nice if show schema; in cli shows Cache Provider Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Show schema in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130907#comment-13130907 ] Eric Evans commented on CASSANDRA-3380: --- Hi Brian, I've only briefly glanced over the code, but what I see looks good. It's great that you've already taken care of things like build and docs. However, I'm not sure where (or even if )this should live within Cassandra. contrib/ would be the right place I think, except that we're in (have been) in the process of trying to eliminate that. And, the impetus for removing contrib/ was that it was a place of second-class citizenship, with mixed expectations that didn't reflect well on Cassandra, or the authors of the contributed code. In other words, it should either by fully supported, or maintaining it out of tree is probably in everyone's best interest. Have you considered maintaining this as a separate project? It seems as though it would be pretty easy, logistically speaking. REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3384) it would be nice if show schema; in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-3384: --- Attachment: CASSANDRA-3384.patch Here is one-line patch to show it in describe keyspace keyspace; and show keyspaces it would be nice if show schema; in cli shows Cache Provider Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Attachments: CASSANDRA-3384.patch Show schema in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (CASSANDRA-3384) it would be nice if show schema; in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reopened CASSANDRA-3384: - +1 it would be nice if show schema; in cli shows Cache Provider Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Attachments: CASSANDRA-3384.patch Show schema in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3384) it would be nice if describe keyspace in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3384: Summary: it would be nice if describe keyspace in cli shows Cache Provider (was: it would be nice if show schema; in cli shows Cache Provider) it would be nice if describe keyspace in cli shows Cache Provider - Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Attachments: CASSANDRA-3384.patch Show schema in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3384) it would be nice if describe keyspace in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3384: Reviewer: brandon.williams (was: vijay2...@yahoo.com) it would be nice if describe keyspace in cli shows Cache Provider - Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Attachments: CASSANDRA-3384.patch Describe keyspace in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3384) it would be nice if describe keyspace in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3384: Description: Describe keyspace in the cli doesn't show the cache provider it would be nice to show it to verify the settings. (was: Show schema in the cli doesn't show the cache provider it would be nice to show it to verify the settings.) it would be nice if describe keyspace in cli shows Cache Provider - Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Attachments: CASSANDRA-3384.patch Describe keyspace in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1186429 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/cli/CliClient.java
Author: xedin Date: Wed Oct 19 19:44:24 2011 New Revision: 1186429 URL: http://svn.apache.org/viewvc?rev=1186429view=rev Log: CLI `describe keyspace ks` to show Row Cache Provider patch by Pavel Yaskevich; reviewed by Brandon Williams for CASSANDRA-3384 Modified: cassandra/branches/cassandra-0.8/CHANGES.txt cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cli/CliClient.java Modified: cassandra/branches/cassandra-0.8/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1186429r1=1186428r2=1186429view=diff == --- cassandra/branches/cassandra-0.8/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.8/CHANGES.txt Wed Oct 19 19:44:24 2011 @@ -24,7 +24,7 @@ * fix changing durable_writes keyspace option during CF creation (CASSANDRA-3292) * correct dropped messages logging (CASSANDRA-3377) - + * CLI `describe keyspace ks` to show Row Cache Provider (CASSANDRA-3384) 0.8.7 * Kill server on wrapped OOME such as from FileChannel.map (CASSANDRA-3201) Modified: cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cli/CliClient.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cli/CliClient.java?rev=1186429r1=1186428r2=1186429view=diff == --- cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cli/CliClient.java (original) +++ cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cli/CliClient.java Wed Oct 19 19:44:24 2011 @@ -1905,6 +1905,7 @@ public class CliClient sessionState.out.printf( Default column value validator: %s%n, cf_def.default_validation_class); sessionState.out.printf( Columns sorted by: %s%s%n, cf_def.comparator_type, cf_def.column_type.equals(Super) ? / + cf_def.subcomparator_type : ); sessionState.out.printf( Row cache size / save period in seconds: %s/%s%n, cf_def.row_cache_size, cf_def.row_cache_save_period_in_seconds); +sessionState.out.printf( Row Cache Provider: %s%n, cf_def.getRow_cache_provider()); sessionState.out.printf( Key cache size / save period in seconds: %s/%s%n, cf_def.key_cache_size, cf_def.key_cache_save_period_in_seconds); sessionState.out.printf( Memtable thresholds: %s/%s/%s (millions of ops/minutes/MB)%n, cf_def.memtable_operations_in_millions, cf_def.memtable_flush_after_mins, cf_def.memtable_throughput_in_mb);
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130913#comment-13130913 ] Jonathan Ellis commented on CASSANDRA-3380: --- My criterion for maintaining a REST API in-tree would be, I'd need to have a clear understanding of what problem the REST API solves that the CQL one does not. E.g. if I'm writing Python code why would I use the REST API instead of the Python dbapi driver? REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3384) it would be nice if describe keyspace in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich resolved CASSANDRA-3384. Resolution: Fixed Committed. it would be nice if describe keyspace in cli shows Cache Provider - Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Attachments: CASSANDRA-3384.patch Describe keyspace in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3384) it would be nice if describe keyspace in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130919#comment-13130919 ] Vijay commented on CASSANDRA-3384: -- Sorry for the confusion... Thanks +1 it would be nice if describe keyspace in cli shows Cache Provider - Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Attachments: CASSANDRA-3384.patch Describe keyspace in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130922#comment-13130922 ] Jeremy Hanna commented on CASSANDRA-3380: - if not in the tree, could this be something that would fit the apache-extras category? REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2405) should expose 'time since last successful repair' for easier aes monitoring
[ https://issues.apache.org/jira/browse/CASSANDRA-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich updated CASSANDRA-2405: --- Assignee: Sylvain Lebresne (was: Pavel Yaskevich) should expose 'time since last successful repair' for easier aes monitoring --- Key: CASSANDRA-2405 URL: https://issues.apache.org/jira/browse/CASSANDRA-2405 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Peter Schuller Assignee: Sylvain Lebresne Priority: Minor Fix For: 1.0.1 Attachments: CASSANDRA-2405-v2.patch, CASSANDRA-2405-v3.patch, CASSANDRA-2405-v4.patch, CASSANDRA-2405.patch The practical implementation issues of actually ensuring repair runs is somewhat of an undocumented/untreated issue. One hopefully low hanging fruit would be to at least expose the time since last successful repair for a particular column family, to make it easier to write a correct script to monitor for lack of repair in a non-buggy fashion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3385) NPE in hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yang updated CASSANDRA-3385: - Priority: Critical (was: Major) I just found that this contributes to another symptom I'm seeing: for RF=3, and a ring of 3 nodes, if I bring down 1 box, the remaining 2 still work fine for Quorum access, but the latency is 20x high. I can see from debugging that a lot of time is spent on storing hints into local system table on the coordinator. but this Table.apply is slow because a lot of time is spent on the lock, while it really should not happen since the lock is sharded into 4096 ones. it turns out that all the keys used in the hints writing are the same key, at least in the examples I looked at in the debugger, if I'm correct in this observation, this is a serious bug NPE in hinted handoff - Key: CASSANDRA-3385 URL: https://issues.apache.org/jira/browse/CASSANDRA-3385 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Critical I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more this could possibly be related to #3291 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3383) Make Row cache reject wide rows which are more than x columns
[ https://issues.apache.org/jira/browse/CASSANDRA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130926#comment-13130926 ] Vijay commented on CASSANDRA-3383: -- Hi Jonathan, I was actually talking about 2. even with the query cache will it make sense to add the upper bound? so that we dont waste resources doing copies? Make Row cache reject wide rows which are more than x columns - Key: CASSANDRA-3383 URL: https://issues.apache.org/jira/browse/CASSANDRA-3383 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Environment: Linux with Sun JDK Reporter: Vijay Assignee: Vijay Priority: Minor The problem with the row cache is it has to cache the whole row into the memory, this works great for small rows but not for the wide rows. Even off-heap cache has to read everything into the heap eventually causing GC overhead (If your query is to get few columns). It will be nice to reject the rows which are bigger than x columns (This can be a KS setting) this will help cases where we have mixed load of small and wide column datasets in the same cf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1740) Nodetool commands to query and stop compaction, repair, cleanup and scrub
[ https://issues.apache.org/jira/browse/CASSANDRA-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130930#comment-13130930 ] Pavel Yaskevich commented on CASSANDRA-1740: Stu, are you still planning to work on this? Nodetool commands to query and stop compaction, repair, cleanup and scrub - Key: CASSANDRA-1740 URL: https://issues.apache.org/jira/browse/CASSANDRA-1740 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Chip Salzenberg Assignee: Pavel Yaskevich Priority: Minor Fix For: 1.0.1 Attachments: CASSANDRA-1740.patch Original Estimate: 24h Remaining Estimate: 24h The only way to stop compaction, repair, cleanup, or scrub in progress is to stop and restart the entire Cassandra server. Please provide nodetool commands to query whether such things are running, and stop them if they are. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3385) NPE in hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130928#comment-13130928 ] Jonathan Ellis commented on CASSANDRA-3385: --- So either new-style hints are being written without versionColumn by RowMutation.hintFor, or old style hints did not get cleaned out properly by SystemTable.purgeIncompatibleHints. But both of those look fine to me. NPE in hinted handoff - Key: CASSANDRA-3385 URL: https://issues.apache.org/jira/browse/CASSANDRA-3385 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Critical I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more this could possibly be related to #3291 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3383) Make Row cache reject wide rows which are more than x columns
[ https://issues.apache.org/jira/browse/CASSANDRA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130935#comment-13130935 ] Jonathan Ellis commented on CASSANDRA-3383: --- The idea with the query cache is that you'd be saying here's a query that I do a lot, that does a cacheable subset of the row. So... no, I don't see where a separate upper bound concept really makes sense (since slice queries already have a count limit, and name queries are obviously limited by nature). Make Row cache reject wide rows which are more than x columns - Key: CASSANDRA-3383 URL: https://issues.apache.org/jira/browse/CASSANDRA-3383 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Environment: Linux with Sun JDK Reporter: Vijay Assignee: Vijay Priority: Minor The problem with the row cache is it has to cache the whole row into the memory, this works great for small rows but not for the wide rows. Even off-heap cache has to read everything into the heap eventually causing GC overhead (If your query is to get few columns). It will be nice to reject the rows which are bigger than x columns (This can be a KS setting) this will help cases where we have mixed load of small and wide column datasets in the same cf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3383) Make Row cache reject wide rows which are more than x columns
[ https://issues.apache.org/jira/browse/CASSANDRA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay resolved CASSANDRA-3383. -- Resolution: Not A Problem Cool, Makes sense... thanks! Make Row cache reject wide rows which are more than x columns - Key: CASSANDRA-3383 URL: https://issues.apache.org/jira/browse/CASSANDRA-3383 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Environment: Linux with Sun JDK Reporter: Vijay Assignee: Vijay Priority: Minor The problem with the row cache is it has to cache the whole row into the memory, this works great for small rows but not for the wide rows. Even off-heap cache has to read everything into the heap eventually causing GC overhead (If your query is to get few columns). It will be nice to reject the rows which are bigger than x columns (This can be a KS setting) this will help cases where we have mixed load of small and wide column datasets in the same cf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3385) NPE in hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130936#comment-13130936 ] Yang Yang commented on CASSANDRA-3385: -- the hints code was from: https://github.com/apache/cassandra/commit/3893f24098c3d82dc31571f0b6841e2d5821ea74#diff-12 #CASSANDRA-2034 maybe it should be better to NOT let the writer wait for hints to finish? right now the local hints write make the entire write slower in probably 2 ways : 1) the main write has to wait for hint write to finish, which is slow due to lock 2) hints writes are slow, which create a lot of jobs on MUTATION stage, so even if main write does not wait for them, the MUTATION stage could possibly be bogged down with hints writes, and not able to handle normal writes fast enough NPE in hinted handoff - Key: CASSANDRA-3385 URL: https://issues.apache.org/jira/browse/CASSANDRA-3385 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Critical I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more this could possibly be related to #3291 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (CASSANDRA-3383) Make Row cache reject wide rows which are more than x columns
[ https://issues.apache.org/jira/browse/CASSANDRA-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay closed CASSANDRA-3383. Make Row cache reject wide rows which are more than x columns - Key: CASSANDRA-3383 URL: https://issues.apache.org/jira/browse/CASSANDRA-3383 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1 Environment: Linux with Sun JDK Reporter: Vijay Assignee: Vijay Priority: Minor The problem with the row cache is it has to cache the whole row into the memory, this works great for small rows but not for the wide rows. Even off-heap cache has to read everything into the heap eventually causing GC overhead (If your query is to get few columns). It will be nice to reject the rows which are bigger than x columns (This can be a KS setting) this will help cases where we have mixed load of small and wide column datasets in the same cf. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3385) NPE in hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130941#comment-13130941 ] Jonathan Ellis commented on CASSANDRA-3385: --- IMO the right fix to the lock contention is to simply not synchronize when there are no indexes present. But that's unrelated to the assertion failure here. NPE in hinted handoff - Key: CASSANDRA-3385 URL: https://issues.apache.org/jira/browse/CASSANDRA-3385 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Critical I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more this could possibly be related to #3291 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130942#comment-13130942 ] Brian ONeill commented on CASSANDRA-3380: - Thanks for the comments guys. The reason REST/JSON support is important is articulated well here: http://nosql.mypopescu.com/post/411195754/nosql-protocols-are-important That article boils down to three points: ease of use, standardization across interfaces, and decreased number of application dependencies. Enterprises typically have technology eco-systems that have many services/capabilities. (e.g. ours includes Neo4j and SOLR) That ecosystem includes third-party services as well as internal. It helps if you can standardize on interfaces across the entire ecosystem. May people are standardizing on JSON/HTTP (as evidenced by support in CouchDB, MongoDO, Neo4j, SOLR, Elastic Search, etc.) That standardization decreases the integration/adoption cost. Since many languages have native support for REST calls, typically an application can consume the capability without adding any additional application dependencies. (e.g. drivers) Also, JSON/HTTP is especially nice if the data is making it out to the web. Many javascript frameworks can natively consume the data. Even if the browser/javascript isn't hitting the database directly (via HTTP), there services layer in between often can just become a proxy of sorts. (this has happened with us with SOLR) REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3385) NPE in hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130944#comment-13130944 ] Yang Yang commented on CASSANDRA-3385: -- we should probably change the schema in https://issues.apache.org/jira/browse/CASSANDRA-2045?focusedCommentId=13057419page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13057419 and NOT use ip as the key, as it leads to lock contention, and many other problems NPE in hinted handoff - Key: CASSANDRA-3385 URL: https://issues.apache.org/jira/browse/CASSANDRA-3385 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Critical I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more this could possibly be related to #3291 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130951#comment-13130951 ] Brian ONeill commented on CASSANDRA-3380: - I was originally going to maintain it as a separate project, before Gary Dubasbek chimed in and pointed out that it is inefficient to deploy a separate server (and use something like Hector) because you would end up marshaling the document twice with an added hop. That is when I decided to write this as a patch for the tree since it would need to stay in lock-step with the CassandraServer object. (since the REST calls would come into the same VM and make the calls in-thread/natively against the java objects) Ideally, this wouldn't remain in the contrib package. I think it would be better to get it integrated into the main tree (as not to have mismatched expectations). From the users perspective, I would expect to add an optional configuration parameter in the yaml file that would start the REST server on the specified port. when the parameter is defined. REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3385) NPE in hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yang updated CASSANDRA-3385: - Comment: was deleted (was: we should probably change the schema in https://issues.apache.org/jira/browse/CASSANDRA-2045?focusedCommentId=13057419page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13057419 and NOT use ip as the key, as it leads to lock contention, and many other problems) NPE in hinted handoff - Key: CASSANDRA-3385 URL: https://issues.apache.org/jira/browse/CASSANDRA-3385 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Critical I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more this could possibly be related to #3291 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3385) NPE in hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130952#comment-13130952 ] Yang Yang commented on CASSANDRA-3385: -- that works for me too. but I guess you will finally have to handle cases where indexes are needed. in those cases, probably we need to change the hints format away from using IP as key NPE in hinted handoff - Key: CASSANDRA-3385 URL: https://issues.apache.org/jira/browse/CASSANDRA-3385 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Critical I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more this could possibly be related to #3291 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130955#comment-13130955 ] Jonathan Ellis commented on CASSANDRA-3380: --- Saying Everything is REST is about as useful as saying Everything is TCP. It's fine for a driver to be REST-based but that doesn't make, say, Cassandra and Neo4j interchangeable. So I don't see the advantage from a development point of view. Nor do I see import cql as a deal breaker over import urllib2 (especially since the former gives you a much better experience.) Javascript frameworks is a valid point, although I hope everyone agrees that the browser hitting the db directly is a colossally bad idea. I can see though that it would simplify the proxy if it just has to decide accept/reject on a REST query vs translate REST into another API. REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3385) NPE in hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130958#comment-13130958 ] Jonathan Ellis commented on CASSANDRA-3385: --- If you're not going to use IPs as keys how are you going to replay hints efficiently? You need to consider the read path as well as the write when modeling something. :) NPE in hinted handoff - Key: CASSANDRA-3385 URL: https://issues.apache.org/jira/browse/CASSANDRA-3385 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Critical I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more this could possibly be related to #3291 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130961#comment-13130961 ] Brian ONeill commented on CASSANDRA-3380: - One last point (then I'll shut up ;) We are literally trying to decide between CouchDB and Cassandra by Monday. Because CouchDB has a REST interface, we had it integrated into our application in hours. But we *love* Cassandra, and the hadoop/pig integration potential. But for obvious reasons, I'd like to avoid deploying/supporting/maintaining both if I can. If Cassandra had a REST layer, we'd have one database to rule them all and we'd be able to pick our interface mechanism depending on who/what needed to operate on and access the data. The web front-end could come in through REST, while the backend services use Pig/Hadoop to do the heavy lifting. REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3370) Deflate Compression corrupts SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130962#comment-13130962 ] Christian Spriegel commented on CASSANDRA-3370: --- I tried using stress from 1.0.0 and I got the same results as you. Stress for some reason works fine. One thing is strange about stress: I let stress run for quite some time, but there is only 12 MB in the datafolder. I let my tool run for 10 seconds, but there are 97MB in the data folder. Is stress maybe not generating random data, so that it compresses really well? Might that be the difference? Can I maybe share my application with you? Its a single source file with a pom.xml. If you have any idea what I can do, please let me know. Deflate Compression corrupts SSTables - Key: CASSANDRA-3370 URL: https://issues.apache.org/jira/browse/CASSANDRA-3370 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: Ubuntu Linux, amd64, Cassandra 1.0.0-rc2 Reporter: Christian Spriegel Assignee: Sylvain Lebresne Attachments: system.log Hi, it seems that the Deflate Compressor corrupts the SSTables. 3 out of 3 Installations were corrupt. Snappy works fine. Here is what I did: 1. Start a single cassandra node (I was using ByteOrderedPartitioner) 2. Write data into cf that uses deflate compression - I think it has to be enough data so that the data folder contains some files. 3. When I now try to read (I did a range scan) from my application, it fails and the logs show corruptions: Caused by: org.apache.cassandra.io.compress.CorruptedBlockException: (/home/cspriegel/Development/cassandra1/data/Test/Response-h-2-Data.db): corruption detected, chunk at 0 of length 65536. regards, Christian -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3386) Avoid lock contention in hint rows
Avoid lock contention in hint rows -- Key: CASSANDRA-3386 URL: https://issues.apache.org/jira/browse/CASSANDRA-3386 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 0.8.8, 1.0.1 As pointed out by Yang in CASSANDRA-3385, hint writes are keyed by target IP, to make replay efficient. However, this means that we'll hit a lot of lock contention in table.apply where we synchronize for potential index updates. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3386) Avoid lock contention in hint rows
[ https://issues.apache.org/jira/browse/CASSANDRA-3386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3386: -- Attachment: 3386.txt Patch skips the synchronization when there are no mutated index columns. (I'm 99% sure that's the only reason we need to lock there...) Avoid lock contention in hint rows -- Key: CASSANDRA-3386 URL: https://issues.apache.org/jira/browse/CASSANDRA-3386 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 0.7.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Fix For: 0.8.8, 1.0.1 Attachments: 3386.txt As pointed out by Yang in CASSANDRA-3385, hint writes are keyed by target IP, to make replay efficient. However, this means that we'll hit a lot of lock contention in table.apply where we synchronize for potential index updates. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3385) NPE in hinted handoff
[ https://issues.apache.org/jira/browse/CASSANDRA-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130967#comment-13130967 ] Jonathan Ellis commented on CASSANDRA-3385: --- Created CASSANDRA-3386 for the contention problem. NPE in hinted handoff - Key: CASSANDRA-3385 URL: https://issues.apache.org/jira/browse/CASSANDRA-3385 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Critical I'm using the current HEAD of 1.0.0 github branch, and I'm still seeing this error, not sure if it's this bug or another one. INFO [HintedHandoff:1] 2011-10-19 12:43:17,674 HintedHandOffManager.java (line 263) Started hinted handoff for token: 11342745564 0312821154458202477256070484 with IP: /10.39.85.140 ERROR [HintedHandoff:1] 2011-10-19 12:43:17,885 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHan doff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more ERROR [HintedHandoff:1] 2011-10-19 12:43:17,886 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[HintedHandoff:1,1,main] java.lang.RuntimeException: java.lang.NullPointerException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.NullPointerException at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:289) at org.apache.cassandra.db.HintedHandOffManager.access$100(HintedHandOffManager.java:81) at org.apache.cassandra.db.HintedHandOffManager$2.runMayThrow(HintedHandOffManager.java:337) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more this could possibly be related to #3291 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130971#comment-13130971 ] Jonathan Ellis commented on CASSANDRA-3380: --- Are you developing in a niche environment that doesn't have a decent client already? Because I don't see Because X has a REST interface, we can integrate it easily as cause and effect at all, or rather, the same would be true for any decent client ant not just a REST one. I'm really trying to understand here if I'm missing something important or if your team just happens to really have a thing for REST. :) REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2268) CQL-enabled stress.java
[ https://issues.apache.org/jira/browse/CASSANDRA-2268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Evans updated CASSANDRA-2268: -- Assignee: Eric Evans (was: paul cannon) CQL-enabled stress.java --- Key: CASSANDRA-2268 URL: https://issues.apache.org/jira/browse/CASSANDRA-2268 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Eric Evans Assignee: Eric Evans Priority: Minor Labels: cql Fix For: 1.0.1 Attachments: 0001-2268-wip.patch It would be great if stress.java had a CQL mode. For making the inevitable RPC-CQL comparisons, but also as a basis for measuring optimizations, and spotting performance regressions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1740) Nodetool commands to query and stop compaction, repair, cleanup and scrub
[ https://issues.apache.org/jira/browse/CASSANDRA-1740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130976#comment-13130976 ] Stu Hood commented on CASSANDRA-1740: - Hey Pavel, sorry: I should have commented ages ago: I won't be working on this. Nodetool commands to query and stop compaction, repair, cleanup and scrub - Key: CASSANDRA-1740 URL: https://issues.apache.org/jira/browse/CASSANDRA-1740 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Chip Salzenberg Assignee: Pavel Yaskevich Priority: Minor Fix For: 1.0.1 Attachments: CASSANDRA-1740.patch Original Estimate: 24h Remaining Estimate: 24h The only way to stop compaction, repair, cleanup, or scrub in progress is to stop and restart the entire Cassandra server. Please provide nodetool commands to query whether such things are running, and stop them if they are. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3371) Cassandra inferred schema and actual data don't match
[ https://issues.apache.org/jira/browse/CASSANDRA-3371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-3371: Attachment: 3371-v2.txt This approach has many problems, such as column name and key collisions. Instead, what we need to is wrap the column/value pairs in their own tuple and then insert these into an outer tuple that goes in the bag, causing the schema the match our actual output. v2 does this. Cassandra inferred schema and actual data don't match - Key: CASSANDRA-3371 URL: https://issues.apache.org/jira/browse/CASSANDRA-3371 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 0.8.7 Reporter: Pete Warden Assignee: Brandon Williams Attachments: 3371-v2.txt, pig.diff It's looking like there may be a mismatch between the schema that's being reported by the latest CassandraStorage.java, and the data that's actually returned. Here's an example: rows = LOAD 'cassandra://Frap/PhotoVotes' USING CassandraStorage(); DESCRIBE rows; rows: {key: chararray,columns: {(name: chararray,value: bytearray,photo_owner: chararray,value_photo_owner: bytearray,pid: chararray,value_pid: bytearray,matched_string: chararray,value_matched_string: bytearray,src_big: chararray,value_src_big: bytearray,time: chararray,value_time: bytearray,vote_type: chararray,value_vote_type: bytearray,voter: chararray,value_voter: bytearray)}} DUMP rows; (691831038_1317937188.48955,{(photo_owner,1596090180),(pid,6855155124568798560),(matched_string,),(src_big,),(time,Thu Oct 06 14:39:48 -0700 2011),(vote_type,album_dislike),(voter,691831038)}) getSchema() is reporting the columns as an inner bag of tuples, each of which contains 16 values. In fact, getNext() seems to return an inner bag containing 7 tuples, each of which contains two values. It appears that things got out of sync with this change: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?r1=1177083r2=1177082pathrev=1177083 See more discussion at: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/pig-cassandra-problem-quot-Incompatible-field-schema-quot-error-tc6882703.html Here's a patch I ended up creating for my own use, which gives the results I need (though it doesn't handle super-columns): DESCRIBE rows; rows: {cassandra_key: chararray,photo_owner: bytearray,pid: bytearray,place_matched_string: bytearray,src_big: bytearray,time: bytearray,vote_type: bytearray,voter: bytearray} Index: contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java === --- contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java (revision 1185044) +++ contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java (working copy) @@ -26,7 +26,7 @@ import org.apache.cassandra.db.marshal.IntegerType; import org.apache.cassandra.db.marshal.TypeParser; import org.apache.cassandra.thrift.*; -import org.apache.cassandra.utils.Hex; +import org.apache.cassandra.utils.FBUtilities; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; @@ -122,15 +122,15 @@ assert key != null cf != null; // and wrap it in a tuple - Tuple tuple = TupleFactory.getInstance().newTuple(2); + Tuple tuple = TupleFactory.getInstance().newTuple(cf.size()+1); ArrayListTuple columns = new ArrayListTuple(); -tuple.set(0, new DataByteArray(key.array(), key.position()+key.arrayOffset(), key.limit()+key.arrayOffset())); +int tupleIndex = 0; +tuple.set(tupleIndex++, new DataByteArray(key.array(), key.position()+key.arrayOffset(), key.limit()+key.arrayOffset())); for (Map.EntryByteBuffer, IColumn entry : cf.entrySet()) { -columns.add(columnToTuple(entry.getKey(), entry.getValue(), cfDef)); +tuple.set(tupleIndex++, columnToTuple(entry.getKey(), entry.getValue(), cfDef)); } -tuple.set(1, new DefaultDataBag(columns)); return tuple; } catch (InterruptedException e) @@ -139,30 +139,22 @@ } } -private Tuple columnToTuple(ByteBuffer name, IColumn col, CfDef cfDef) throws IOException +private Object columnToTuple(ByteBuffer name, IColumn col, CfDef cfDef) throws IOException { -Tuple pair = TupleFactory.getInstance().newTuple(2); ListAbstractType marshallers = getDefaultMarshallers(cfDef);
[jira] [Commented] (CASSANDRA-3371) Cassandra inferred schema and actual data don't match
[ https://issues.apache.org/jira/browse/CASSANDRA-3371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130990#comment-13130990 ] Pete Warden commented on CASSANDRA-3371: Is there a reason the columns can't at least go into a map? As things stand, it's painfully hard to do the natural row.column.value lookup in a script. Or to put it as a concrete example, I can currently do this: all_votes = LOAD 'cassandra://Frap/PhotoVotes' USING CassandraStorage(); album_votes = FILTER votes BY ((vote_type EQ 'album_like') OR (vote_type EQ 'album_dislike')); What does this example look like with your approach? Cassandra inferred schema and actual data don't match - Key: CASSANDRA-3371 URL: https://issues.apache.org/jira/browse/CASSANDRA-3371 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 0.8.7 Reporter: Pete Warden Assignee: Brandon Williams Attachments: 3371-v2.txt, pig.diff It's looking like there may be a mismatch between the schema that's being reported by the latest CassandraStorage.java, and the data that's actually returned. Here's an example: rows = LOAD 'cassandra://Frap/PhotoVotes' USING CassandraStorage(); DESCRIBE rows; rows: {key: chararray,columns: {(name: chararray,value: bytearray,photo_owner: chararray,value_photo_owner: bytearray,pid: chararray,value_pid: bytearray,matched_string: chararray,value_matched_string: bytearray,src_big: chararray,value_src_big: bytearray,time: chararray,value_time: bytearray,vote_type: chararray,value_vote_type: bytearray,voter: chararray,value_voter: bytearray)}} DUMP rows; (691831038_1317937188.48955,{(photo_owner,1596090180),(pid,6855155124568798560),(matched_string,),(src_big,),(time,Thu Oct 06 14:39:48 -0700 2011),(vote_type,album_dislike),(voter,691831038)}) getSchema() is reporting the columns as an inner bag of tuples, each of which contains 16 values. In fact, getNext() seems to return an inner bag containing 7 tuples, each of which contains two values. It appears that things got out of sync with this change: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?r1=1177083r2=1177082pathrev=1177083 See more discussion at: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/pig-cassandra-problem-quot-Incompatible-field-schema-quot-error-tc6882703.html Here's a patch I ended up creating for my own use, which gives the results I need (though it doesn't handle super-columns): DESCRIBE rows; rows: {cassandra_key: chararray,photo_owner: bytearray,pid: bytearray,place_matched_string: bytearray,src_big: bytearray,time: bytearray,vote_type: bytearray,voter: bytearray} Index: contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java === --- contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java (revision 1185044) +++ contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java (working copy) @@ -26,7 +26,7 @@ import org.apache.cassandra.db.marshal.IntegerType; import org.apache.cassandra.db.marshal.TypeParser; import org.apache.cassandra.thrift.*; -import org.apache.cassandra.utils.Hex; +import org.apache.cassandra.utils.FBUtilities; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; @@ -122,15 +122,15 @@ assert key != null cf != null; // and wrap it in a tuple - Tuple tuple = TupleFactory.getInstance().newTuple(2); + Tuple tuple = TupleFactory.getInstance().newTuple(cf.size()+1); ArrayListTuple columns = new ArrayListTuple(); -tuple.set(0, new DataByteArray(key.array(), key.position()+key.arrayOffset(), key.limit()+key.arrayOffset())); +int tupleIndex = 0; +tuple.set(tupleIndex++, new DataByteArray(key.array(), key.position()+key.arrayOffset(), key.limit()+key.arrayOffset())); for (Map.EntryByteBuffer, IColumn entry : cf.entrySet()) { -columns.add(columnToTuple(entry.getKey(), entry.getValue(), cfDef)); +tuple.set(tupleIndex++, columnToTuple(entry.getKey(), entry.getValue(), cfDef)); } -tuple.set(1, new DefaultDataBag(columns)); return tuple; } catch (InterruptedException e) @@ -139,30 +139,22 @@ } } -private Tuple columnToTuple(ByteBuffer name, IColumn col, CfDef cfDef) throws IOException +private Object columnToTuple(ByteBuffer name, IColumn col, CfDef
[jira] [Created] (CASSANDRA-3387) unnecessary locking in hint path due to InetAddress.getLocalhost()
unnecessary locking in hint path due to InetAddress.getLocalhost() -- Key: CASSANDRA-3387 URL: https://issues.apache.org/jira/browse/CASSANDRA-3387 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Minor in tests we found locking contention also due to the following Stacks at 01:49:19 PM (uptime 10:54) MutationStage:62 [BLOCKED] CPU time: 0:06 java.net.InetAddress.getLocalHost() org.apache.cassandra.utils.UUIDGen.getClockSeqAndNode() org.apache.cassandra.utils.UUIDGen.createTimeUUIDBytes(long) org.apache.cassandra.utils.UUIDGen.getTimeUUIDBytes() org.apache.cassandra.db.RowMutation.hintFor(RowMutation, ByteBuffer) org.apache.cassandra.service.StorageProxy$4.run() java.lang.Thread.run() we can easily change every getLocalHost() call to use a cached value -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3387) unnecessary locking in hint path due to InetAddress.getLocalhost()
[ https://issues.apache.org/jira/browse/CASSANDRA-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yang updated CASSANDRA-3387: - Attachment: 3387.diff use cached value , do not call getLocalhost() every time when we try to construct a hint unnecessary locking in hint path due to InetAddress.getLocalhost() -- Key: CASSANDRA-3387 URL: https://issues.apache.org/jira/browse/CASSANDRA-3387 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Minor Attachments: 3387.diff in tests we found locking contention also due to the following Stacks at 01:49:19 PM (uptime 10:54) MutationStage:62 [BLOCKED] CPU time: 0:06 java.net.InetAddress.getLocalHost() org.apache.cassandra.utils.UUIDGen.getClockSeqAndNode() org.apache.cassandra.utils.UUIDGen.createTimeUUIDBytes(long) org.apache.cassandra.utils.UUIDGen.getTimeUUIDBytes() org.apache.cassandra.db.RowMutation.hintFor(RowMutation, ByteBuffer) org.apache.cassandra.service.StorageProxy$4.run() java.lang.Thread.run() we can easily change every getLocalHost() call to use a cached value -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3387) unnecessary locking in hint path due to InetAddress.getLocalhost()
[ https://issues.apache.org/jira/browse/CASSANDRA-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130999#comment-13130999 ] Jonathan Ellis commented on CASSANDRA-3387: --- Why not just use FBUtilities.getLocalAddress? unnecessary locking in hint path due to InetAddress.getLocalhost() -- Key: CASSANDRA-3387 URL: https://issues.apache.org/jira/browse/CASSANDRA-3387 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Minor Attachments: 3387.diff in tests we found locking contention also due to the following Stacks at 01:49:19 PM (uptime 10:54) MutationStage:62 [BLOCKED] CPU time: 0:06 java.net.InetAddress.getLocalHost() org.apache.cassandra.utils.UUIDGen.getClockSeqAndNode() org.apache.cassandra.utils.UUIDGen.createTimeUUIDBytes(long) org.apache.cassandra.utils.UUIDGen.getTimeUUIDBytes() org.apache.cassandra.db.RowMutation.hintFor(RowMutation, ByteBuffer) org.apache.cassandra.service.StorageProxy$4.run() java.lang.Thread.run() we can easily change every getLocalHost() call to use a cached value -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of ArchitectureCommitLog by RickBranson
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The ArchitectureCommitLog page has been changed by RickBranson: http://wiki.apache.org/cassandra/ArchitectureCommitLog?action=diffrev1=5rev2=6 Comment: Updated to reflect current CommitLog implementation as of 1.0. - The !CommitLog class manages the !CommitLogSegments, each of which corresponds to a file on disk containing a fixed-size !CommitLogHeader followed by serialized !RowMutation objects. + The !CommitLog class manages the !CommitLogSegments, each of which corresponds to a file on disk containing serialized !RowMutation objects. - A !CommitLogHeader has one entry per !ColumnFamily, consisting of a dirty bit and a replay offset, indicating the position in the !CommitLog file to start replaying the log for a particular !ColumnFamily. + The !CommitLogSegment keeps track of which column families have been modified in memory using a hash map called cfLastWrite. cfLastWrite has one entry per !ColumnFamily, consisting of an offset, indicating the position in the !CommitLog file where the last write took place for a particular !ColumnFamily. - Each insertion (deletion) has to first write a log entry to the !CommitLog. + Each mutation has to first write a log entry to the !CommitLog. - * The writing of all log entries is handled by a single thread in !CommitLogExecutorService. - * For the first insert to a given !ColumnFamily CF in each !CommitLogSegment, the !CommitLogHeader is updated: the CF's dirty bit is turned on and the replay offset for CF in the !CommitLogHeader is updated with the current position (represented by a !CommitLogContext object) in the !CommitLog file. + * All log entries are written by a single thread in one of the !CommitLogExecutorService classes. + * For the first mutation to a given !ColumnFamily CF in each !CommitLogSegment, an entry is set in cfLastWrite map keyed by the CF's id containing the offset in the mutation was written at. * A !RowMutation entry is then appended to the !CommitLogSegment - * If !CommitLogSync is set to batch, the insertion further waits until the !CommitLogSegment is sync-ed to disk before the insert is allowed to proceed + * If the configuration directive !commitlog_sync is set to batch, the mutation further waits until the !CommitLogSegment is sync-ed to disk before the mutation is allowed to proceed * Once a !CommitLogSegment becomes too large, a new segment is created and new operations are appended there instead. On the completion of a flush for a !ColumnFamily CF, + * The !ReplayPosition for CF is written to the !SSTable metadata. * For each !CommitLogSegment F generated when or before the flush is initiated, - * If F is not the one being used when the flush was initated, the dirty bit for CF in the !CommitLogHeader of F is turned off -* If all dirty bits in the !CommitLogHeader are off, F is deleted. - * Otherwise, the dirty bit for CF in the !CommitLogHeader is turned on and the replay offset for CF is updated with the position in the log file when the flush was initiated. + * If F is not the one being used when the flush was initiated, the CF's entry in cfLastWrite is removed. +* If the cfLastWrite map is empty, the segment is no longer needed and is deleted. + * Otherwise, for the CF, the value is set in cfLastWrite map with the replay position when the flush was initiated (as long as no writes have taken place). Recovery during a restart, * Each !CommitLogSegment is iterated in ascending time order. - * The segment is read from the lowest replay offset among all entries in the !CommitLogHeader. + * The segment is read from the lowest replay offset among the !ReplayPositions read from the SSTable metadata. - * For each log entry read, the log is replayed for a !ColumnFamily CF if the position of the log entry is no less than the replay offset for CF in the !CommitLogHeader. + * For each log entry read, the log is replayed for a !ColumnFamily CF if the position of the log entry is no less than the !ReplayPosition for CF in the most recent !SSTable metadata. * When log replay is done, all Memtables are force flushed to disk and all commitlog segments are deleted.
[jira] [Commented] (CASSANDRA-3387) unnecessary locking in hint path due to InetAddress.getLocalhost()
[ https://issues.apache.org/jira/browse/CASSANDRA-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131011#comment-13131011 ] Yang Yang commented on CASSANDRA-3387: -- oh, I didn't know that one. yes, FBUtilities.getLocalAddress() is better unnecessary locking in hint path due to InetAddress.getLocalhost() -- Key: CASSANDRA-3387 URL: https://issues.apache.org/jira/browse/CASSANDRA-3387 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Minor Attachments: 3387.diff in tests we found locking contention also due to the following Stacks at 01:49:19 PM (uptime 10:54) MutationStage:62 [BLOCKED] CPU time: 0:06 java.net.InetAddress.getLocalHost() org.apache.cassandra.utils.UUIDGen.getClockSeqAndNode() org.apache.cassandra.utils.UUIDGen.createTimeUUIDBytes(long) org.apache.cassandra.utils.UUIDGen.getTimeUUIDBytes() org.apache.cassandra.db.RowMutation.hintFor(RowMutation, ByteBuffer) org.apache.cassandra.service.StorageProxy$4.run() java.lang.Thread.run() we can easily change every getLocalHost() call to use a cached value -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3387) unnecessary locking in hint path due to InetAddress.getLocalhost()
[ https://issues.apache.org/jira/browse/CASSANDRA-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Yang updated CASSANDRA-3387: - Attachment: 3387_v2.diff version 2 , using fb utils unnecessary locking in hint path due to InetAddress.getLocalhost() -- Key: CASSANDRA-3387 URL: https://issues.apache.org/jira/browse/CASSANDRA-3387 Project: Cassandra Issue Type: Bug Reporter: Yang Yang Priority: Minor Attachments: 3387.diff, 3387_v2.diff in tests we found locking contention also due to the following Stacks at 01:49:19 PM (uptime 10:54) MutationStage:62 [BLOCKED] CPU time: 0:06 java.net.InetAddress.getLocalHost() org.apache.cassandra.utils.UUIDGen.getClockSeqAndNode() org.apache.cassandra.utils.UUIDGen.createTimeUUIDBytes(long) org.apache.cassandra.utils.UUIDGen.getTimeUUIDBytes() org.apache.cassandra.db.RowMutation.hintFor(RowMutation, ByteBuffer) org.apache.cassandra.service.StorageProxy$4.run() java.lang.Thread.run() we can easily change every getLocalHost() call to use a cached value -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3370) Deflate Compression corrupts SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Spriegel updated CASSANDRA-3370: -- Attachment: Test.zip Attached my client that causes the crash. Deflate Compression corrupts SSTables - Key: CASSANDRA-3370 URL: https://issues.apache.org/jira/browse/CASSANDRA-3370 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Environment: Ubuntu Linux, amd64, Cassandra 1.0.0-rc2 Reporter: Christian Spriegel Assignee: Sylvain Lebresne Attachments: Test.zip, system.log Hi, it seems that the Deflate Compressor corrupts the SSTables. 3 out of 3 Installations were corrupt. Snappy works fine. Here is what I did: 1. Start a single cassandra node (I was using ByteOrderedPartitioner) 2. Write data into cf that uses deflate compression - I think it has to be enough data so that the data folder contains some files. 3. When I now try to read (I did a range scan) from my application, it fails and the logs show corruptions: Caused by: org.apache.cassandra.io.compress.CorruptedBlockException: (/home/cspriegel/Development/cassandra1/data/Test/Response-h-2-Data.db): corruption detected, chunk at 0 of length 65536. regards, Christian -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131031#comment-13131031 ] Brian ONeill commented on CASSANDRA-3380: - I'll admit. My team does have a thing for REST. ;) We process hundreds of data feeds, all with different schemas. We then cleanse, standardize, de-dupe, transform, match, and consolidate that information using systems and processes across languages and platforms. (which is why JSON is important to us) http://www.healthmarketscience.com/insights/mdm-paradoxical-problem Looking at our end-to-end processing, here are three other examples that may be more universal. (increasing in priority/value) The general theme of these is that there are many off-the-shelf tools out there that can integrate with JSON/XML over HTTP out of the box. Not all of them are easily extended to accommodate different drivers/data sources. TESTING: We use SOAP UI to test. We use it to smoke test and validate environments. It can easily hit REST interfaces on all our systems (and third party tools) to verify everything is in a good state and has the proper data populated. ETL TOOLS: Many GUI-based ETL Tools (e.g. Talend) can extract and load via XML/JSON over HTTP. They grab/push the data over HTTP natively. If Cassandra had a native REST interface, we could tap into it directly. Without it, we'll be looking at some additional process / plugins to make this happen. (PITA) The users of these systems are not developers, they are data stewards that just want to be able to point to a url and get access to the data from their GUI. ESB / Workflow Tools: Similarly, many of the ESB tools (e.g. Sonic ESB, ServiceMix and Mule) can also grab data via XML/JSON over HTTP. The workflow/process can then operate on that data, (mapping elements to inputs of services) without having to write any code. Again, to integrate Cassandra into some of those service orchestrations / mashups, we'll need to develop a pluginĀ if we can't get to it via REST. (Any more compelling? =) REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3005) OutboundTcpConnection's sending queue grows unboundedly without any backpressure logic
[ https://issues.apache.org/jira/browse/CASSANDRA-3005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Melvin Wang updated CASSANDRA-3005: --- Attachment: c3005-v2 Sorry for the slackness. I got my hands really full these days. Incorporated Stu's feedback. OutboundTcpConnection's sending queue grows unboundedly without any backpressure logic -- Key: CASSANDRA-3005 URL: https://issues.apache.org/jira/browse/CASSANDRA-3005 Project: Cassandra Issue Type: Improvement Reporter: Melvin Wang Assignee: Melvin Wang Attachments: c3005-v2, c3005.patch OutboundTcpConnection's sending queue unconditionally queues up the request and process them in sequence. Thinking about tagging the message coming in with timestamp and drop them before actually sending it if the message stays in the queue for too long, which is defined by the message's own time out value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131050#comment-13131050 ] Brian ONeill commented on CASSANDRA-3380: - As for Does it belong integrated?... I agree. As it stands, the elementary interface that is implemented thus far hides some of the rich types, but I would expect that we would continue to enhance the layer to provide access to those rich types and it would end up much like how SOLR exposes the underlying features and functions of Lucene via its REST interface. As for demand, I think there is significant interest; enough to spawn up projects: http://code.google.com/p/restish/ https://github.com/stinkymatt/Helena http://www.onemanclapping.org/2010/09/restful-cassandra.html https://github.com/gdusbabek/cassandra I think these are good, but the development is fragmented and is not native. (the first two use Hector I believe) By pulling the effort into the main tree, we can increase the size of the community and consolidate the effort behind it. REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3388) StorageService.setMode() is used inconsistently
StorageService.setMode() is used inconsistently --- Key: CASSANDRA-3388 URL: https://issues.apache.org/jira/browse/CASSANDRA-3388 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Tyler Hobbs Priority: Trivial {{StorageService.setMode()}}, which ends up setting the OperationMode attribute of the related mbean, is used inconsistently. In most places, it's used like {{setMode(MODE: details)}}, but in a few places, it's used more like a normal log message. To make this attribute more usable through JMX, {{setMode()}} should have a signature like {{setMode(mode, details)}}, where the mode parameter could be an enum (or even just a string, the main thing is just being consistent). The OperationMode JMX attribute should definitely remain a string, though. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3388) StorageService.setMode() is used inconsistently
[ https://issues.apache.org/jira/browse/CASSANDRA-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131099#comment-13131099 ] Jonathan Ellis commented on CASSANDRA-3388: --- We're talking about this, right? {code} private void setMode(String m, boolean log) {code} Why would we even expose this over JMX? It's not intended to be public. StorageService.setMode() is used inconsistently --- Key: CASSANDRA-3388 URL: https://issues.apache.org/jira/browse/CASSANDRA-3388 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Tyler Hobbs Priority: Trivial {{StorageService.setMode()}}, which ends up setting the OperationMode attribute of the related mbean, is used inconsistently. In most places, it's used like {{setMode(MODE: details)}}, but in a few places, it's used more like a normal log message. To make this attribute more usable through JMX, {{setMode()}} should have a signature like {{setMode(mode, details)}}, where the mode parameter could be an enum (or even just a string, the main thing is just being consistent). The OperationMode JMX attribute should definitely remain a string, though. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3065) Major file corruption after running nodetool cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131100#comment-13131100 ] Benjamin Schrauwen commented on CASSANDRA-3065: --- I lost the index file, but a corrupt datafile can be found here: http://dl.dropbox.com/u/3030404/corrupt_sstable-g-6296-Data.db.bz2 Major file corruption after running nodetool cleanup Key: CASSANDRA-3065 URL: https://issues.apache.org/jira/browse/CASSANDRA-3065 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 0.8.3 Reporter: Benjamin Schrauwen Assignee: Sylvain Lebresne After running nodetool cleanup on two of the nodes in my 4 node cluster, almost all SSTables on those those machine got corrupted. I am not able to read them anymore with sstable2json, and the cassandra daemon is repetitively throwing: ERROR [ReadStage:11] 2011-08-20 04:44:46,846 AbstractCassandraDaemon.java (line 139) Fatal exception in thread Thread[ReadStage:11,5,main] java.lang.RuntimeException: java.lang.IndexOutOfBoundsException at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.lang.IndexOutOfBoundsException at java.nio.Buffer.checkIndex(Buffer.java:514) at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:209) at org.apache.cassandra.io.util.MappedFileDataInput.read(MappedFileDataInput.java:104) at java.io.InputStream.read(InputStream.java:154) at org.apache.cassandra.io.util.AbstractDataInput.readInt(AbstractDataInput.java:196) at org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:61) at org.apache.cassandra.db.columniterator.SimpleSliceReader.init(SimpleSliceReader.java:58) at org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:91) at org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:67) at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:66) at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1314) at org.apache.cassandra.db.ColumnFamilyStore.cacheRow(ColumnFamilyStore.java:1181) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1221) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1168) at org.apache.cassandra.db.Table.getRow(Table.java:385) at org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:58) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:641) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131105#comment-13131105 ] Jonathan Ellis commented on CASSANDRA-3380: --- I also remember that Mozilla cited REST support as a reason for using Riak (http://blog.mozilla.com/data/2010/05/18/riak-and-cassandra-and-hbase-oh-my/). Not sure how valid that is but it wouldn't be surprising if they were dealing with a lot of JSON data too. REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2819) Split rpc timeout for read and write ops
[ https://issues.apache.org/jira/browse/CASSANDRA-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131145#comment-13131145 ] Melvin Wang commented on CASSANDRA-2819: bq. - currently, expiring map checks callbacks for expiration every 1/2 of default timeout. This means short timeouts could go much longer than desired, before being cancelled. I see two options: go back to direct await(timeout) in the callback/response handlers, or update EM to scan much more frequently (which could burn a lot of CPU for a system with many outstanding callbacks) yeah, i guess, calling getReadRpcTimeout/getWriteRpcTimeout explicitly in ReadCallback/RepairCallback would be simpler and more correct. Split rpc timeout for read and write ops Key: CASSANDRA-2819 URL: https://issues.apache.org/jira/browse/CASSANDRA-2819 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Stu Hood Assignee: Melvin Wang Fix For: 1.0.1 Attachments: 2819-v4.txt, 2819-v5-rebased.txt, c2819.patch, rpc-jira.patch Given the vastly different latency characteristics of reads and writes, it makes sense for them to have independent rpc timeouts internally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3388) StorageService.setMode() is used inconsistently
[ https://issues.apache.org/jira/browse/CASSANDRA-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131153#comment-13131153 ] Tyler Hobbs commented on CASSANDRA-3388: Yes, that's the method. Sorry, I forgot that there was also a log parameter. It's a fairly reliable way to figure out if a node is joining, moving, decommissioning, or has finished decommissioning. I'm open to alternative ways of discovering this reliably. I assumed that's why this was exposed through JMX in the first place. StorageService.setMode() is used inconsistently --- Key: CASSANDRA-3388 URL: https://issues.apache.org/jira/browse/CASSANDRA-3388 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Tyler Hobbs Priority: Trivial {{StorageService.setMode()}}, which ends up setting the OperationMode attribute of the related mbean, is used inconsistently. In most places, it's used like {{setMode(MODE: details)}}, but in a few places, it's used more like a normal log message. To make this attribute more usable through JMX, {{setMode()}} should have a signature like {{setMode(mode, details)}}, where the mode parameter could be an enum (or even just a string, the main thing is just being consistent). The OperationMode JMX attribute should definitely remain a string, though. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3384) it would be nice if describe keyspace in cli shows Cache Provider
[ https://issues.apache.org/jira/browse/CASSANDRA-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131157#comment-13131157 ] Hudson commented on CASSANDRA-3384: --- Integrated in Cassandra-0.8 #382 (See [https://builds.apache.org/job/Cassandra-0.8/382/]) CLI `describe keyspace ks` to show Row Cache Provider patch by Pavel Yaskevich; reviewed by Brandon Williams for CASSANDRA-3384 xedin : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1186429 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/cli/CliClient.java it would be nice if describe keyspace in cli shows Cache Provider - Key: CASSANDRA-3384 URL: https://issues.apache.org/jira/browse/CASSANDRA-3384 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 0.8.4 Environment: JVM on Linux Reporter: Vijay Assignee: Pavel Yaskevich Priority: Minor Labels: lhf Fix For: 0.8.8, 1.0.1 Attachments: CASSANDRA-3384.patch Describe keyspace in the cli doesn't show the cache provider it would be nice to show it to verify the settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131166#comment-13131166 ] Eric Evans commented on CASSANDRA-3380: --- {quote} I agree. As it stands, the elementary interface that is implemented thus far hides some of the rich types, but I would expect that we would continue to enhance the layer to provide access to those rich types and it would end up much like how SOLR exposes the underlying features and functions of Lucene via its REST interface {quote} How are you going to implement Cassandra's type system without implementing schema? Once you drag schema into the mix, how will you justify this approach when it's no longer possible to plug-and-play existing systems, or drive queries with curl? {quote} As for demand, I think there is significant interest; enough to spawn up projects: http://code.google.com/p/restish/ https://github.com/stinkymatt/Helena http://www.onemanclapping.org/2010/09/restful-cassandra.html https://github.com/gdusbabek/cassandra {quote} I believe the project in that first link is from Courtney Robinson, and I believe that he now advocates CQL (he started work on a CQL driver, and stopped working on that). I'm not sure what the story is behind the second link, other than it doesn't appear to have generated much interest (based on forks and watchers). The last two links are both from Gary Dusbabek (a member of the Cassandra PMC). This was a proof-of-concept that he only spent a few hours on, and one that he decided not to continue with. It might be worth asking him why. Honestly, I think this does more to prove why a REST interface _does not_ belong integrated in Cassandra. It is not enough to simply have code. That code needs to be maintained, and it needs more than one person who cares enough to make sure that happens. It also needs to be documented, and supported on all the usual forums, again, something that will requires a little more critical mass And, introducing choice can be a Good Thing, but it can also be a Bad Thing. We need to know that this is going to be useful enough, to a large enough audience, to offset the confusion it will almost certainly generate. Think of the people who are going to be compelled to ask which interface they should use, and who are going to have to weigh the pros and cons and then make a choice (and remember that this would bring us from 2, to 3 choices of interface). Think of the users who are going to make assumptions about semantics or performance characteristics based on one interface or the other, only to find it's not applicable to their choice. There is a cost associated with this, and it's fair to ask the hard questions to make ensure it's worth it. You've also repeatedly alluded that not having this accepted as part of the project is something of a deal breaker. Why? Now that you have this code, doesn't it solve your particular needs? I won't veto this if consensus is that it should be added, but I'm still not convinced that this will succeed where the other attempts have failed. Nor am I convinced that the only way to establish success is by committing it first. What would convince me is a moderately successful externally maintained project, with a modicum of users. REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131166#comment-13131166 ] Eric Evans edited comment on CASSANDRA-3380 at 10/19/11 11:47 PM: -- {quote} I agree. As it stands, the elementary interface that is implemented thus far hides some of the rich types, but I would expect that we would continue to enhance the layer to provide access to those rich types and it would end up much like how SOLR exposes the underlying features and functions of Lucene via its REST interface {quote} How are you going to implement Cassandra's type system without implementing schema? Once you drag schema into the mix, how will you justify this approach when it's no longer possible to plug-and-play existing systems, or drive queries with curl? {quote} As for demand, I think there is significant interest; enough to spawn up projects: http://code.google.com/p/restish/ https://github.com/stinkymatt/Helena http://www.onemanclapping.org/2010/09/restful-cassandra.html https://github.com/gdusbabek/cassandra {quote} I believe the project in that first link is from Courtney Robinson, and I believe that he now advocates CQL (he started work on a CQL driver, and stopped working on that). I'm not sure what the story is behind the second link, other than it doesn't appear to have generated much interest (based on forks and watchers). The last two links are both from Gary Dusbabek (a member of the Cassandra PMC). This was a proof-of-concept that he only spent a few hours on, and one that he decided not to continue with. It might be worth asking him why. Honestly, I think this does more to prove why a REST interface _does not_ belong integrated in Cassandra. It is not enough to simply have code. That code needs to be maintained, and it needs more than one person who cares enough to make sure that happens. It also needs to be documented, and supported on all the usual forums, again, something that will require a little more critical mass. And, introducing choice can be a Good Thing, but it can also be a Bad Thing. We need to know that this is going to be useful enough, to a large enough audience, to offset the confusion it will almost certainly generate. Think of the people who are going to be compelled to ask which interface they should use, and who are going to have to weigh the pros and cons and then make a choice (and remember that this would bring us from 2, to 3 choices of interface). Think of the users who are going to make assumptions about semantics or performance characteristics based on one interface or the other, only to find it's not applicable to their choice. There is a cost associated with this, and it's fair to ask the hard questions to make ensure it's worth it. You've also repeatedly alluded that not having this accepted as part of the project is something of a deal breaker. Why? Now that you have this code, doesn't it solve your particular needs? I won't veto this if consensus is that it should be added, but I'm still not convinced that this will succeed where the other attempts have failed. Nor am I convinced that the only way to establish success is by committing it first. What would convince me is a moderately successful externally maintained project, with a modicum of users. was (Author: urandom): {quote} I agree. As it stands, the elementary interface that is implemented thus far hides some of the rich types, but I would expect that we would continue to enhance the layer to provide access to those rich types and it would end up much like how SOLR exposes the underlying features and functions of Lucene via its REST interface {quote} How are you going to implement Cassandra's type system without implementing schema? Once you drag schema into the mix, how will you justify this approach when it's no longer possible to plug-and-play existing systems, or drive queries with curl? {quote} As for demand, I think there is significant interest; enough to spawn up projects: http://code.google.com/p/restish/ https://github.com/stinkymatt/Helena http://www.onemanclapping.org/2010/09/restful-cassandra.html https://github.com/gdusbabek/cassandra {quote} I believe the project in that first link is from Courtney Robinson, and I believe that he now advocates CQL (he started work on a CQL driver, and stopped working on that). I'm not sure what the story is behind the second link, other than it doesn't appear to have generated much interest (based on forks and watchers). The last two links are both from Gary Dusbabek (a member of the Cassandra PMC). This was a proof-of-concept that he only spent a few hours on, and one that he decided not to continue with. It might be worth asking him why. Honestly, I think this does more to prove why a REST interface _does
svn commit: r1186591 - /cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java
Author: jbellis Date: Thu Oct 20 01:08:23 2011 New Revision: 1186591 URL: http://svn.apache.org/viewvc?rev=1186591view=rev Log: fix merge Modified: cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java Modified: cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java URL: http://svn.apache.org/viewvc/cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java?rev=1186591r1=1186590r2=1186591view=diff == --- cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java (original) +++ cassandra/trunk/src/java/org/apache/cassandra/thrift/ThriftValidation.java Thu Oct 20 01:08:23 2011 @@ -731,16 +731,6 @@ public class ThriftValidation max_compaction_threshold)); } -public static void validateMemtableSettings(org.apache.cassandra.thrift.CfDef cf_def) throws ConfigurationException -{ -if (cf_def.isSetMemtable_flush_after_mins()) - DatabaseDescriptor.validateMemtableFlushPeriod(cf_def.memtable_flush_after_mins); -if (cf_def.isSetMemtable_throughput_in_mb()) - DatabaseDescriptor.validateMemtableThroughput(cf_def.memtable_throughput_in_mb); -if (cf_def.isSetMemtable_operations_in_millions()) - DatabaseDescriptor.validateMemtableOperations(cf_def.memtable_operations_in_millions); -} - public static void validateKeyspaceNotYetExisting(String newKsName) throws InvalidRequestException { // keyspace names must be unique case-insensitively because the keyspace name becomes the directory
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build on builder cassandra-trunk while building ASF Buildbot. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/1733 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: isis_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra/trunk] 1186591 Blamelist: jbellis Build succeeded! sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131247#comment-13131247 ] Brian ONeill commented on CASSANDRA-3380: - I can appreciate that perspective. When I first posted the thought of a REST layer to the dev list, Gary immediately responded with his thoughts and Jeremy asked I respond to the list when I had something. Based on their responses and Matt's link to his Helena project, I may have overestimated the demand/need. I'll reach out to Gary to get his input, but I don't mind letting this JIRA issue brew for a while to see if there is interest. Acceptance is certainly not a deal breaker. Like you said, this code solves our needs and we'll continue to extend it. I can throw it out into an open source project to see if it sticks. Any preferred forum for that project? (One final note that I posted today to the user list; we could potentially use this REST layer as an integration point for Elastic Search, much the way CouchDB integrated as a river, http://www.elasticsearch.org/tutorials/2010/08/01/couchb-integration.html. We may try to head that way depending on what is available in DataStax Enterprise. I'll let you guys know if that manifests itself.) REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2819) Split rpc timeout for read and write ops
[ https://issues.apache.org/jira/browse/CASSANDRA-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Melvin Wang updated CASSANDRA-2819: --- Attachment: c2819-v6 Split rpc timeout for read and write ops Key: CASSANDRA-2819 URL: https://issues.apache.org/jira/browse/CASSANDRA-2819 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Stu Hood Assignee: Melvin Wang Fix For: 1.0.1 Attachments: 2819-v4.txt, 2819-v5-rebased.txt, c2819-v6, c2819.patch, rpc-jira.patch Given the vastly different latency characteristics of reads and writes, it makes sense for them to have independent rpc timeouts internally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2819) Split rpc timeout for read and write ops
[ https://issues.apache.org/jira/browse/CASSANDRA-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131256#comment-13131256 ] Melvin Wang commented on CASSANDRA-2819: bq. Writes still use the old rpc_timeout Changed to getWriteRpcTimeout() bq. looks like most remaining uses of old rpc_timeout should really be max(different timeouts) wanna modify the getRpcTimeout in DD so that it returns the max of the other get*RpcTimeout()? bq. currently, expiring map checks callbacks for expiration every 1/2 of default timeout. This means short timeouts could go much longer than desired, before being cancelled. I see two options: go back to direct await(timeout) in the callback/response handlers, or update EM to scan much more frequently (which could burn a lot of CPU for a system with many outstanding callbacks) Not a so elegant solution in this patch. It uses the corresponding get*RpcTimeout in different callbacks. Split rpc timeout for read and write ops Key: CASSANDRA-2819 URL: https://issues.apache.org/jira/browse/CASSANDRA-2819 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Stu Hood Assignee: Melvin Wang Fix For: 1.0.1 Attachments: 2819-v4.txt, 2819-v5-rebased.txt, c2819-v6, c2819.patch, rpc-jira.patch Given the vastly different latency characteristics of reads and writes, it makes sense for them to have independent rpc timeouts internally. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3388) StorageService.setMode() is used inconsistently
[ https://issues.apache.org/jira/browse/CASSANDRA-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131257#comment-13131257 ] Jonathan Ellis commented on CASSANDRA-3388: --- I don't see setMode or getMode in StorageServiceMBean. I'm lost, how is it exposed through JMX? StorageService.setMode() is used inconsistently --- Key: CASSANDRA-3388 URL: https://issues.apache.org/jira/browse/CASSANDRA-3388 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.0.0 Reporter: Tyler Hobbs Priority: Trivial {{StorageService.setMode()}}, which ends up setting the OperationMode attribute of the related mbean, is used inconsistently. In most places, it's used like {{setMode(MODE: details)}}, but in a few places, it's used more like a normal log message. To make this attribute more usable through JMX, {{setMode()}} should have a signature like {{setMode(mode, details)}}, where the mode parameter could be an enum (or even just a string, the main thing is just being consistent). The OperationMode JMX attribute should definitely remain a string, though. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3380) REST Layer
[ https://issues.apache.org/jira/browse/CASSANDRA-3380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13131263#comment-13131263 ] Jonathan Ellis commented on CASSANDRA-3380: --- bq. How are you going to implement Cassandra's type system without implementing schema? If I were going to do it, I'd do it the way CQL does, which is to use the to/fromString methods to attempt to parse the json string as what the schema says it should be. REST Layer --- Key: CASSANDRA-3380 URL: https://issues.apache.org/jira/browse/CASSANDRA-3380 Project: Cassandra Issue Type: New Feature Environment: Unix / Max OS X Reporter: Brian ONeill Attachments: trunk-3380.txt This is a native rest layer for Cassandra implementing AbstractCassandraDaemon. It uses JAX-RS fueled by Apache CXF. Presently it supports the following operations JSON over HTTP: - Create keyspace - Drop keyspace - Create column family - Drop column family - Insert row - Fetch row - Delete row - Insert column - Delete column - Fetch column The patch creates a new project in contrib/rest. You can compile the project using ant, which uses ivy to pull in dependencies. To get setup, you can also use the pom.xml file and m2eclipse to get it into Eclipse. Once compiled, simpy run bin/rest_cassandra and follow along in the README.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira