[jira] [Commented] (CASSANDRA-3317) SStable corruption possible caused by Leveldb compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128084#comment-13128084 ] Radim Kolar commented on CASSANDRA-3317: this one is difficult to reproduce, probably timing dependant or something. i am using my own tool, it is doing batch writes and then mixed read/write workload. > SStable corruption possible caused by Leveldb compaction > > > Key: CASSANDRA-3317 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3317 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.0.0 >Reporter: Radim Kolar > > During stress testing of leveldb compaction i found that sometimes it create > corrupted ssfile and > cassandra starts with error. I am not sure what caused that error, but i cant > reproduce it if i switch to standard > compaction strategy. > ERROR [SSTableBatchOpen:1] 2011-10-05 15:31:31,781 > AbstractCassandraDaemon.java > (line 133) Fatal exception in thread Thread[SSTableBatchOpen:1,5,main] > java.lang.AssertionError > at > org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java > :129) > at > org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.jav > a:196) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:44 > 1) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec > utor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor > .java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3126) unable to remove column metadata via CLI
[
https://issues.apache.org/jira/browse/CASSANDRA-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reassigned CASSANDRA-3126:
-
Assignee: Pavel Yaskevich
> unable to remove column metadata via CLI
>
>
> Key: CASSANDRA-3126
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3126
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Affects Versions: 0.7.0
>Reporter: Radim Kolar
>Assignee: Pavel Yaskevich
>Priority: Minor
> Labels: cassandra-cli
> Fix For: 0.8.8
>
>
> I cant find way how to remove all columns definitions without CF
> import/export.
> [default@int4] update column family sipdb with column_metadata = [];
> Syntax error at position 51: required (...)+ loop did not match anything at
> input ']'
> [default@int4] update column family sipdb with column_metadata = [{}];
> Command not found: `update column family sipdb with column_metadata = [{}];`.
> Type 'help;' or '?' for help.
> [default@int4]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2453) node.js cql driver
[ https://issues.apache.org/jira/browse/CASSANDRA-2453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2453. --- Resolution: Not A Problem Fix Version/s: (was: 0.8.8) Assignee: (was: Gary Dusbabek) Closing in favor of moving to apache extras. > node.js cql driver > -- > > Key: CASSANDRA-2453 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2453 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Gary Dusbabek >Priority: Minor > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3186) nodetool should not NPE when rack/dc info is not yet available
[ https://issues.apache.org/jira/browse/CASSANDRA-3186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-3186: - Assignee: Brandon Williams > nodetool should not NPE when rack/dc info is not yet available > -- > > Key: CASSANDRA-3186 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3186 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.8.1 >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Labels: lhf > Fix For: 0.8.8 > > > As the title says. What happens is the persisted ring is loaded, but if > those nodes are down and you're using a snitch like ec2 that gets rack/dc > info from gossip, nodetool NPEs instead of showing that the node is down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3088) Expose compaction_throughput_mb_per_sec for JMX tweaking
[ https://issues.apache.org/jira/browse/CASSANDRA-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-3088: - Assignee: Brandon Williams > Expose compaction_throughput_mb_per_sec for JMX tweaking > > > Key: CASSANDRA-3088 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3088 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Affects Versions: 0.8.0 >Reporter: Jonathan Ellis >Assignee: Brandon Williams >Priority: Minor > Labels: lhf > Fix For: 0.8.8 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2268) CQL-enabled stress.java
[ https://issues.apache.org/jira/browse/CASSANDRA-2268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-2268: - Assignee: paul cannon (was: Aaron Morton) > CQL-enabled stress.java > --- > > Key: CASSANDRA-2268 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2268 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Eric Evans >Assignee: paul cannon >Priority: Minor > Labels: cql > Fix For: 0.8.8 > > Attachments: 0001-2268-wip.patch > > > It would be great if stress.java had a CQL mode. For making the inevitable > RPC->CQL comparisons, but also as a basis for measuring optimizations, and > spotting performance regressions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2963) Add a convenient way to reset a node's schema
[ https://issues.apache.org/jira/browse/CASSANDRA-2963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128061#comment-13128061 ] Jonathan Ellis commented on CASSANDRA-2963: --- Pierre-Yves, are you still planning a v2 here? > Add a convenient way to reset a node's schema > - > > Key: CASSANDRA-2963 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2963 > Project: Cassandra > Issue Type: New Feature > Components: Tools >Reporter: Brandon Williams >Priority: Minor > Labels: lhf > Fix For: 0.8.8 > > Attachments: system_reset_schema.txt > > > People often encounter a schema disagreement where just one node is out of > sync. To get it back in sync, they shutdown the node, move the Schema* and > Migration* files out of the system ks, and then start it back up. Rather > than go through this process, it would be nice if you could just tell the > node to reset its schema. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3292) creating column family sets durable_writes to true
[ https://issues.apache.org/jira/browse/CASSANDRA-3292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3292: -- Reviewer: xedin (was: tjake) > creating column family sets durable_writes to true > -- > > Key: CASSANDRA-3292 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3292 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.8.5 >Reporter: Radim Kolar >Assignee: Jonathan Ellis >Priority: Minor > Labels: schema > Fix For: 0.8.8 > > Attachments: 0001-r-m-rename-migrations.patch, > 0002-add-KSM.cloneWith.patch > > > [default@rapidshare] describe keyspace rapidshare; > Keyspace: rapidshare: > Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy > Durable Writes: *false* > Options: [datacenter1:1] > Column Families: > [default@rapidshare] create column family t1; > 1ba19300-ebfa-11e0--34912694d0bf > Waiting for schema agreement... > ... schemas agree across the cluster > [default@rapidshare] describe keyspace rapidshare; > Keyspace: rapidshare: > Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy > Durable Writes: *true* > Options: [datacenter1:1] > Column Families: > ColumnFamily: t1 > Key Validation Class: org.apache.cassandra.db.marshal.BytesType > Default column value validator: > org.apache.cassandra.db.marshal.BytesType > Columns sorted by: org.apache.cassandra.db.marshal.BytesType > Row cache size / save period in seconds: 0.0/0 > Key cache size / save period in seconds: 20.0/14400 > Memtable thresholds: 0.0281249997/1440/6 (millions of > ops/minutes/MB) > GC grace seconds: 864000 > Compaction min/max thresholds: 4/32 > Read repair chance: 1.0 > Replicate on write: true > Built indexes: [] -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2910) expose cluster availability
[ https://issues.apache.org/jira/browse/CASSANDRA-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2910. --- Resolution: Won't Fix ISTM these are more the responsibility of monitoring systems than Cassandra per se. > expose cluster availability > --- > > Key: CASSANDRA-2910 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2910 > Project: Cassandra > Issue Type: New Feature >Reporter: Wojciech Meler > > Cluster should be able to: > - return % of tokenspace available for given ConsistencyLevel > - return highest available ConsistencyLevel for given token -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2944) Show the nodes' schemas in the nodetool ring result.
[ https://issues.apache.org/jira/browse/CASSANDRA-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2944. --- Resolution: Not A Problem Use the cli "describe cluster" command instead. > Show the nodes' schemas in the nodetool ring result. > > > Key: CASSANDRA-2944 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2944 > Project: Cassandra > Issue Type: New Feature > Components: Tools >Affects Versions: 0.8.1 >Reporter: Dikang Gu > Labels: cassandra, nodetool, schema > > Sometimes, we get the "schema does not agree" error from the cassandra > cluster, it will be great if we can view the schemas directly on the nodetool > ring output. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2932) Implement "assume" in cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-2932. --- Resolution: Duplicate being done in CASSANDRA-3188 > Implement "assume" in cqlsh > --- > > Key: CASSANDRA-2932 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2932 > Project: Cassandra > Issue Type: Improvement >Reporter: Jeremy Hanna >Priority: Minor > Labels: lhf > > In the CLI there is a handy way to assume CF comparators/validators > (CASSANDRA-1693). It would be very nice to have the "assume" command in cqlsh > as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2997) Enhance human-readability of snapshot names created by drop column family
[ https://issues.apache.org/jira/browse/CASSANDRA-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-2997: - Assignee: Yuki Morishita Eric's suggestion to include the CF name in single-CF snapshots is a good one. > Enhance human-readability of snapshot names created by drop column family > - > > Key: CASSANDRA-2997 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2997 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Affects Versions: 0.8.2 >Reporter: Eric Gilmore >Assignee: Yuki Morishita >Priority: Minor > > Currently when you drop a column family, a snapshot is automatically created > in a directory named with the timestamp of the drop. > Clever folk will find a way to understand the timestamps and locate > particular snapshots, but it is not as effortless as it could be if part or > all of the CF name was included in the snapshot name. > Any strategy to make the snapshot name more user-friendly and easy to find > would be helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3060) Boostrapping node stuck after receiving file with version mismatch
[ https://issues.apache.org/jira/browse/CASSANDRA-3060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-3060. --- Resolution: Not A Problem There's not much the bootstrapping node can do in that situation, other than wait for you to scrub and then retry the bootstrap. > Boostrapping node stuck after receiving file with version mismatch > -- > > Key: CASSANDRA-3060 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3060 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.8.3 >Reporter: Benjamin Schrauwen >Priority: Minor > > When adding a node to the cluster, the boostrapping process get's stuck when > it receives a file which is apparently from an older version. This is the > Exception > java.lang.RuntimeException: Cannot recover SSTable > /var/lib/cassandra/data/Mollom/recent_sessions-tmp-f-1 due to version > mismatch. (current version is g). > at > org.apache.cassandra.io.sstable.SSTableWriter.createBuilder(SSTableWriter.java:240) > at > org.apache.cassandra.db.compaction.CompactionManager.submitSSTableBuild(CompactionManager.java:1090) > at > org.apache.cassandra.streaming.StreamInSession.finished(StreamInSession.java:110) > at > org.apache.cassandra.streaming.IncomingStreamReader.readFile(IncomingStreamReader.java:104) > at > org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:61) > at > org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:177) > at > org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:114) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3065) Major file corruption after running nodetool cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-3065: - Assignee: Sylvain Lebresne > Major file corruption after running nodetool cleanup > > > Key: CASSANDRA-3065 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3065 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.8.3 >Reporter: Benjamin Schrauwen >Assignee: Sylvain Lebresne > > After running nodetool cleanup on two of the nodes in my 4 node cluster, > almost all SSTables on those those machine got corrupted. I am not able to > read them anymore with sstable2json, and the cassandra daemon is repetitively > throwing: > ERROR [ReadStage:11] 2011-08-20 04:44:46,846 AbstractCassandraDaemon.java > (line 139) Fatal exception in thread Thread[ReadStage:11,5,main] > java.lang.RuntimeException: java.lang.IndexOutOfBoundsException > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) > Caused by: java.lang.IndexOutOfBoundsException > at java.nio.Buffer.checkIndex(Buffer.java:514) > at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:209) > at > org.apache.cassandra.io.util.MappedFileDataInput.read(MappedFileDataInput.java:104) > at java.io.InputStream.read(InputStream.java:154) > at > org.apache.cassandra.io.util.AbstractDataInput.readInt(AbstractDataInput.java:196) > at > org.apache.cassandra.io.sstable.IndexHelper.skipIndex(IndexHelper.java:61) > at > org.apache.cassandra.db.columniterator.SimpleSliceReader.(SimpleSliceReader.java:58) > at > org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:91) > at > org.apache.cassandra.db.columniterator.SSTableSliceIterator.(SSTableSliceIterator.java:67) > at > org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:66) > at > org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:80) > at > org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1314) > at > org.apache.cassandra.db.ColumnFamilyStore.cacheRow(ColumnFamilyStore.java:1181) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1221) > at > org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1168) > at org.apache.cassandra.db.Table.getRow(Table.java:385) > at > org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:58) > at > org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:641) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3105) MarshallException no longer showing for incompatible key_validation_class
[ https://issues.apache.org/jira/browse/CASSANDRA-3105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-3105: - Assignee: Pavel Yaskevich > MarshallException no longer showing for incompatible key_validation_class > - > > Key: CASSANDRA-3105 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3105 > Project: Cassandra > Issue Type: Bug > Components: Tools >Affects Versions: 0.8.4 > Environment: Mac OS X 10.6.8 > Cassandra-cli 0.8.4 > Cassandra 0.8.1 >Reporter: Anthony Ikeda >Assignee: Pavel Yaskevich >Priority: Minor > > Recently I tried querying a column family which had the default bytes > key_validation_class using a string. Rather than the usual MarshallException > error displayed, it just displays 'null' > [default@unknown] connect localhost/9160; > Connected to: "Test Cluster" on localhost/9160 > [default@unknown] use RegistryFoundation; > Authenticated to keyspace: RegistryFoundation > [default@RegistryFoundation] list Registry limit 1; > > --- > RowKey: REG:e38e47af-da8d-4cea-917b-44b93198643d > => (column=c-createInfo, > value=7b2274696d657374616d70223a22323031312d30382d31315431333a33343a32362e3136312d30373a3030222c2270726f66696c654964223a2265636f6d6d3a70742f7072663a61696b65646140777367632e636f6d227d, > timestamp=1313094866191002) > => (column=c-externalId, value=15cf9a223f8ad92768499dbdd497c524, > timestamp=1313094866191001) > => (column=c-name, value=416e74686f6e79277320776973686c697374, > timestamp=1313094866191000) > => (column=l-pt-bfbb2342-c85e-43c9-bb69-b3bb54c723fb, value=7074, > timestamp=1313094866388000) > => (column=p-ecomm:pt/prf:aik...@wsgc.com, value=4f574e4552, > timestamp=1313094866191003) > 1 Row Returned. > [default@RegistryFoundation] assume Registry keys as bytes; > > Assumption for column family 'Registry' added successfully. > [default@RegistryFoundation] get > Registry['REG:e38e47af-da8d-4cea-917b-44b93198643d']; > null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3130) CQL queries should alow talbe names to be qualified by keyspace
[
https://issues.apache.org/jira/browse/CASSANDRA-3130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reassigned CASSANDRA-3130:
-
Assignee: Pavel Yaskevich
> CQL queries should alow talbe names to be qualified by keyspace
> ---
>
> Key: CASSANDRA-3130
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3130
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Pavel Yaskevich
>Priority: Minor
> Fix For: 1.0.1
>
>
> While the 0.6.X api was "ugly" in terms of method signatures, it did allow
> you to use the same client to query multiple keyspaces without having to call
> set_keyspace(String). I totally dislike set_keyspace but I know the thrift
> API is definitely not changing.
> The following command sequence is three RPC operations.
> {noformat}
> select * from cf;
> use otherkeyspace;
> select * from othercf;
> {noformat}
> CQL should allow us to do:
> {noformat}
> select * from keyspace1.cf;
> select * from keyspace2.cf;
> {noformat}
> This will make the connection pool management on the client much easier.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3189) redhat init script can report success (exit 0) for startup failures
[ https://issues.apache.org/jira/browse/CASSANDRA-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-3189: - Assignee: paul cannon > redhat init script can report success (exit 0) for startup failures > --- > > Key: CASSANDRA-3189 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3189 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Affects Versions: 0.8.5 >Reporter: Eric Evans >Assignee: paul cannon >Priority: Minor > Fix For: 1.0.1 > > > The Redhat init script's start target merely returns the status code of the > JVM. In a perfect world that would be adequate, but the JVM does not always > return a non-zero status when it should. > The most obvious fix is to test that the process is live before exiting. See > the Debian init script for an example of this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3217) javax.management.InstanceAlreadyExistsException: during startup
[ https://issues.apache.org/jira/browse/CASSANDRA-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-3217: - Assignee: Pavel Yaskevich The second report here is confusing to me, I'm not sure how it could fail once but then succeed with no schema changes in between. > javax.management.InstanceAlreadyExistsException: during startup > --- > > Key: CASSANDRA-3217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3217 > Project: Cassandra > Issue Type: Bug >Affects Versions: 0.8.5 >Reporter: Thibaut >Assignee: Pavel Yaskevich >Priority: Minor > > Hi, > I got this error on one machines after restarting (the node could have been > killed due to an outofmemory error). It would crash on each restart on this > error. > I copied over the system table from another machine and deleted the > locationinfo files and everything worked fine. > I also had to delete 2 db files which coulnd't be loaded due to UTF-8 > encoding errors. These rows were from the same table_feedfetch. The node > might have memory errors, I'm going to check this. So it might be related to > that. > ERROR [main] 2011-09-16 16:13:18,468 AbstractCassandraDaemon.java (line 358) > Exception encountered during startup. > java.lang.RuntimeException: javax.management.InstanceAlreadyExistsException: > org.apache.cassandra.db:type=ColumnFamilies,keyspace=table_feedfetch,columnfamily=table_feedfetch > at > org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:303) > at > org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:465) > at > org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:435) > at org.apache.cassandra.db.Table.initCf(Table.java:369) > at org.apache.cassandra.db.Table.(Table.java:306) > at org.apache.cassandra.db.Table.open(Table.java:111) > at > org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:187) > at > org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:341) > at > org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:97) > Caused by: javax.management.InstanceAlreadyExistsException: > org.apache.cassandra.db:type=ColumnFamilies,keyspace=table_feedfetch,columnfamily=table_feedfetch > at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453) > at > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484) > at > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963) > at > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917) > at > com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312) > at > com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482) > at > org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:299) > ... 8 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3170) Schema versions output should be on separate lines for separate versions
[ https://issues.apache.org/jira/browse/CASSANDRA-3170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-3170: - Assignee: Pavel Yaskevich > Schema versions output should be on separate lines for separate versions > > > Key: CASSANDRA-3170 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3170 > Project: Cassandra > Issue Type: Improvement >Reporter: Jeremy Hanna >Assignee: Pavel Yaskevich >Priority: Minor > Labels: cli, lhf > > On the CLI if you do a 'describe cluster;' it would be really nice to have > different versions on different lines or some way to call out multiple > versions more. Right now it's a UUID with a list of nodes for one, but with > multiple versions, it's on the same line - another UUID and another list of > nodes. That's hard to distinguish between one version and multiple versions > at a glance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3250) fsync the directory after new sstable or commit log segment are created
[ https://issues.apache.org/jira/browse/CASSANDRA-3250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3250: -- Reviewer: jbellis Component/s: Core Priority: Minor (was: Major) Fix Version/s: 1.0.1 Assignee: Pavel Yaskevich > fsync the directory after new sstable or commit log segment are created > --- > > Key: CASSANDRA-3250 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3250 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Zhu Han >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.0.1 > > > The mannual of fsync said: > bq. Calling fsync() does not necessarily ensure that the entry in > the directory containing the file has also reached disk. For that an > explicit fsync() on a file descriptor for the directory is also needed. > At least on ext4, syncing the directory is a must to have step, as described > by [1]. Otherwise, the new sstables or commit logs could be missed after > crash even if itself is synced. > Unfortunately, JVM does not provide an approach to sync the directory... > [1] > http://www.linuxfoundation.org/news-media/blogs/browse/2009/03/don%E2%80%99t-fear-fsync -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3267) Failed to change replication factor on 3 node cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-3267.
---
Resolution: Not A Problem
See wiki.apache.org/cassandra/FAQ#schema_disagreement for how to solve schema
disagreement problems. (And CASSANDRA-1391 for how we plan to fix that
permanently.)
> Failed to change replication factor on 3 node cluster
> -
>
> Key: CASSANDRA-3267
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3267
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 0.8.5
> Environment: Linux, openjdk16, cassandra 0.8.5
>Reporter: Timur Tulyaganov
>
> 3-node cluster config all up and running failed to change replication factor
> /usr/local/cassandra/bin/nodetool -h 10.0.1.3 ring Address Status State Load
> Owns Token
> Address DC RackStatus State LoadOwns
> Token
>
> 57715317233367107622067286720208938865
> 10.0.1.1 datacenter1 rack1 Up Normal 1.49 GB 78.15%
> 20543402371996174596346065790779111550
> 10.0.1.2 datacenter1 rack1 Up Normal 1011.54 MB 7.59%
> 33454860067234500516210522518260948578
> 10.0.1.3 datacenter1 rack1 Up Normal 232.25 MB 14.26%
> 57715317233367107622067286720208938865
> [default@API] describe keyspace API;
> Keyspace: API:
> Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
> Durable Writes: true
> Options: [replication_factor:1]
> Column Families:
> ColumnFamily: AdCampaign
> Key Validation Class: org.apache.cassandra.db.marshal.BytesType
> Default column value validator:
> org.apache.cassandra.db.marshal.BytesType
> [default@API] update keyspace API with
> strategy_options=[{replication_factor:2}];
> null
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3307) Java JDBC driver fails with DELETE statements
[ https://issues.apache.org/jira/browse/CASSANDRA-3307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-3307. --- Resolution: Invalid Moved to http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/issues/detail?id=1 > Java JDBC driver fails with DELETE statements > - > > Key: CASSANDRA-3307 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3307 > Project: Cassandra > Issue Type: Bug > Components: Drivers >Reporter: Corey Hulen > > The following 2 statements are valid, but fail in the java jdbc driver. > "DELETE FROM users WHERE KEY = 'user1'" // missing column should be valid to > delete entire row > "DELETE test FROM users USING CONSISTENCY QUORUM WHERE KEY = 'user1'" // > Should be valid to pass consistency level to a delete statment -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3317) SStable corruption possible caused by Leveldb compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128050#comment-13128050 ] Jonathan Ellis commented on CASSANDRA-3317: --- I can't reproduce. Are you using the Cassandra stress tool or something else? > SStable corruption possible caused by Leveldb compaction > > > Key: CASSANDRA-3317 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3317 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.0.0 >Reporter: Radim Kolar > > During stress testing of leveldb compaction i found that sometimes it create > corrupted ssfile and > cassandra starts with error. I am not sure what caused that error, but i cant > reproduce it if i switch to standard > compaction strategy. > ERROR [SSTableBatchOpen:1] 2011-10-05 15:31:31,781 > AbstractCassandraDaemon.java > (line 133) Fatal exception in thread Thread[SSTableBatchOpen:1,5,main] > java.lang.AssertionError > at > org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java > :129) > at > org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.jav > a:196) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:44 > 1) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec > utor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor > .java:908) > at java.lang.Thread.run(Thread.java:662) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3143) Global caches (key/row)
[ https://issues.apache.org/jira/browse/CASSANDRA-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3143: -- Reviewer: slebresne (was: jbellis) > Global caches (key/row) > --- > > Key: CASSANDRA-3143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3143 > Project: Cassandra > Issue Type: Improvement >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Labels: Core > Fix For: 1.1 > > Attachments: 0001-global-key-cache.patch, > 0002-global-row-cache-and-ASC.readSaved-changed-to-abstra.patch, > 0003-CacheServiceMBean-and-correct-key-cache-loading.patch, > 0004-key-row-cache-tests-and-tweaks.patch, > 0005-cleanup-of-the-CFMetaData-and-thrift-avro-CfDef-and-.patch > > > Caches are difficult to configure well as ColumnFamilies are added, similar > to how memtables were difficult pre-CASSANDRA-2006. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3069) multiget support in CQL (UNION / OR)
[ https://issues.apache.org/jira/browse/CASSANDRA-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13128048#comment-13128048 ] Jonathan Ellis commented on CASSANDRA-3069: --- Pavel, do you remember which ticket it was where we decided that full OR support was too messy? > multiget support in CQL (UNION / OR) > > > Key: CASSANDRA-3069 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3069 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis >Assignee: Pavel Yaskevich >Priority: Minor > Fix For: 1.0.1 > > Attachments: CASSANDRA-3069.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3334) dropping index causes some inflight mutations to fail
[
https://issues.apache.org/jira/browse/CASSANDRA-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-3334:
--
Attachment: 0002-fix.patch
0001-cleanup.patch
Patch to fix attached, as well as patch to clean up the SecondaryIndexManager a
bit.
> dropping index causes some inflight mutations to fail
> -
>
> Key: CASSANDRA-3334
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3334
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 1.0.0
> Environment: windows
>Reporter: Radim Kolar
>Priority: Minor
> Labels: indexing
> Fix For: 1.0.1
>
> Attachments: 0001-cleanup.patch, 0002-fix.patch
>
>
> dropping index causes some inflight mutations to fail. hector on client side
> didnt throw any exception
> INFO [MigrationStage:1] 2011-10-07 23:11:53,742 Migration.java (line 119)
> Applying migration fb1a8540-f128-11e0--23b38323f4da Update column family
> to
> org.apache.cassandra.config.CFMetaData@786669[cfId=1000,ksName=test,cfName=sipdb,cfType=Standard,comparator=org.apache.cassandra.db.marshal.AsciiType,subcolumncomparator=,comment=phone
> calls routing
> information,rowCacheSize=0.0,keyCacheSize=0.0,readRepairChance=0.0,replicateOnWrite=false,gcGraceSeconds=0,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.Int32Type,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=0,rowCacheKeysToSave=2147483647,rowCacheProvider=org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider@8bb33c,mergeShardsChance=0.1,keyAlias=java.nio.HeapByteBuffer[pos=461
> lim=464 cap=466],column_metadata={java.nio.HeapByteBuffer[pos=0 lim=3
> cap=3]=ColumnDefinition{name=6b616d,
> validator=org.apache.cassandra.db.marshal.AsciiType, index_type=null,
> index_name='null'}},compactionStrategyClass=class
> org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionOptions={}]
> INFO [MigrationStage:1] 2011-10-07 23:11:53,805 ColumnFamilyStore.java (line
> 664) Enqueuing flush of Memtable-Migrations@27537043(7860/9825
> serialized/live bytes, 1 ops)
> INFO [MigrationStage:1] 2011-10-07 23:11:53,820 ColumnFamilyStore.java (line
> 664) Enqueuing flush of Memtable-Schema@8340427(3320/4150 serialized/live
> bytes, 3 ops)
> INFO [FlushWriter:3] 2011-10-07 23:11:53,820 Memtable.java (line 237)
> Writing Memtable-Migrations@27537043(7860/9825 serialized/live bytes, 1 ops)
> INFO [FlushWriter:3] 2011-10-07 23:11:55,008 Memtable.java (line 273)
> Completed flushing \var\lib\cassandra\data\system\Migrations-h-14-Data.db
> (7924 bytes)
> INFO [FlushWriter:3] 2011-10-07 23:11:55,008 Memtable.java (line 237)
> Writing Memtable-Schema@8340427(3320/4150 serialized/live bytes, 3 ops)
> INFO [CompactionExecutor:4] 2011-10-07 23:11:55,008 CompactionTask.java
> (line 119) Compacting
> [SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-13-Data.db'),
>
> SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-14-Data.db'),
> SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-11-Data.db'),
> SSTableReader(path='\var\lib\cassandra\data\system\Migrations-h-12-Data.db')]
> INFO [FlushWriter:3] 2011-10-07 23:11:56,430 Memtable.java (line 273)
> Completed flushing \var\lib\cassandra\data\system\Schema-h-14-Data.db (3470
> bytes)
> INFO [CompactionExecutor:3] 2011-10-07 23:11:56,446 CompactionTask.java
> (line 119) Compacting
> [SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-13-Data.db'),
> SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-14-Data.db'),
> SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-12-Data.db'),
> SSTableReader(path='\var\lib\cassandra\data\system\Schema-h-11-Data.db')]
> ERROR [MutationStage:56] 2011-10-07 23:11:56,508 AbstractCassandraDaemon.java
> (line 133) Fatal exception in thread Thread[MutationStage:56,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.db.index.SecondaryIndexManager.applyIndexUpdates(SecondaryIndexManager.java:369)
> at org.apache.cassandra.db.Table.apply(Table.java:457)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:253)
> at
> org.apache.cassandra.service.StorageProxy$5.runMayThrow(StorageProxy.java:436)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1263)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> ERROR [MutationS
[jira] [Updated] (CASSANDRA-3363) Allow one leveled compaction task to kick off another
[ https://issues.apache.org/jira/browse/CASSANDRA-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3363: -- Attachment: 3363.txt Patch attached that encapsulates CT.submitNextTask and CM.getBackgroundSubmitter so LCT can override and wrap them. Kind of ugly, open to better ideas. > Allow one leveled compaction task to kick off another > - > > Key: CASSANDRA-3363 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3363 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 1.0.0 >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Labels: compaction > Fix For: 1.0.1 > > Attachments: 3363.txt > > > Leveled compaction wants to prevent multiple tasks from running at once, but > this check also defeats the "kick off another compaction if there is more > work to do" code in CompactionTask. So currently LCS relies completely on > the every-five-minutes compaction check, which is not enough to keep up with > heavy insert load. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3363) Allow one leveled compaction task to kick off another
[ https://issues.apache.org/jira/browse/CASSANDRA-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3363: -- Reviewer: slebresne (was: brandon.williams) > Allow one leveled compaction task to kick off another > - > > Key: CASSANDRA-3363 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3363 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 1.0.0 >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Labels: compaction > Fix For: 1.0.1 > > Attachments: 3363.txt > > > Leveled compaction wants to prevent multiple tasks from running at once, but > this check also defeats the "kick off another compaction if there is more > work to do" code in CompactionTask. So currently LCS relies completely on > the every-five-minutes compaction check, which is not enough to keep up with > heavy insert load. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3363) Allow one leveled compaction task to kick off another
[ https://issues.apache.org/jira/browse/CASSANDRA-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3363: -- Reviewer: brandon.williams (was: slebresne) > Allow one leveled compaction task to kick off another > - > > Key: CASSANDRA-3363 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3363 > Project: Cassandra > Issue Type: Improvement > Components: Core >Affects Versions: 1.0.0 >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Labels: compaction > Fix For: 1.0.1 > > > Leveled compaction wants to prevent multiple tasks from running at once, but > this check also defeats the "kick off another compaction if there is more > work to do" code in CompactionTask. So currently LCS relies completely on > the every-five-minutes compaction check, which is not enough to keep up with > heavy insert load. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3363) Allow one leveled compaction task to kick off another
Allow one leveled compaction task to kick off another - Key: CASSANDRA-3363 URL: https://issues.apache.org/jira/browse/CASSANDRA-3363 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.0.0 Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 1.0.1 Leveled compaction wants to prevent multiple tasks from running at once, but this check also defeats the "kick off another compaction if there is more work to do" code in CompactionTask. So currently LCS relies completely on the every-five-minutes compaction check, which is not enough to keep up with heavy insert load. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3327) Support TimeUUID in CassandraStorage
[
https://issues.apache.org/jira/browse/CASSANDRA-3327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-3327:
Attachment: 3327.txt
> Support TimeUUID in CassandraStorage
>
>
> Key: CASSANDRA-3327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3327
> Project: Cassandra
> Issue Type: Bug
> Components: Contrib
>Affects Versions: 0.8.7
> Environment: Cassandra 0.8.6 Build #348 (CASSANDRA-2777 +
> CASSANDRA-2810)
>Reporter: Manuel Kreutz
>Assignee: Brandon Williams
> Labels: pig
> Fix For: 0.8.8
>
> Attachments: 3327.txt
>
>
> Cassandra CLI:
> {code}
> grunt> raw = LOAD 'cassandra://TEST/CF'
> >> USING CassandraStorage()
> >> AS (
> >> key:chararray,
> >> columns:bag {
> >> column:tuple(
> >> name,
> >> value
> >> )
> >> });
> grunt> describe raw;
> raw: {key: chararray,columns: {(name: bytearray,value: bytearray)}}
> log_test =
> FOREACH raw
> GENERATE
> (CHARARRAY) key,
> flatten(columns);
> grunt> DUMP log_test;
> {code}
> Returns:
> {code}
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
> open iterator for alias log_test. Backend error : Unexpected data type
> java.util.UUID found in stream. Note only standard Pig type is supported when
> you output from UDF/LoadFunc
> at org.apache.pig.PigServer.openIterator(PigServer.java:890)
> at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:655)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:67)
> at org.apache.pig.Main.run(Main.java:487)
> at org.apache.pig.Main.main(Main.java:108)
> Caused by: java.lang.RuntimeException: Unexpected data type java.util.UUID
> found in stream. Note only standard Pig type is supported when you output
> from UDF/LoadFunc
> at
> org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:478)
> at
> org.apache.pig.data.BinInterSedes.writeTuple(BinInterSedes.java:542)
> at
> org.apache.pig.data.BinInterSedes.writeDatum(BinInterSedes.java:357)
> at
> org.apache.pig.impl.io.InterRecordWriter.write(InterRecordWriter.java:73)
> at org.apache.pig.impl.io.InterStorage.putNext(InterStorage.java:87)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:138)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:97)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:498)
> at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:263)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:256)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:58)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> {code}
> According to driftx on IRC the setTupleValue function in CassandraStorage
> needs to handle the uuid case and cast it to a DataByteArray.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3362) allow sub-row repair
allow sub-row repair Key: CASSANDRA-3362 URL: https://issues.apache.org/jira/browse/CASSANDRA-3362 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Jonathan Ellis Assignee: Sylvain Lebresne Priority: Minor With large rows, it would be nice to not have to send an entire row if a small part is out of sync. Could we use the row index blocks as repair atoms instead of the full row? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3335) ThreadPoolExecutor creates threads as non-daemon and will block on shutdown by default
[ https://issues.apache.org/jira/browse/CASSANDRA-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127926#comment-13127926 ] Jonathan Ellis commented on CASSANDRA-3335: --- (all the task logging is done w/ the StorageService logger so it can be enabled separately from the reset of the package involved) > ThreadPoolExecutor creates threads as non-daemon and will block on shutdown > by default > -- > > Key: CASSANDRA-3335 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3335 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Brandon Williams >Assignee: Jonathan Ellis >Priority: Minor > Fix For: 1.0.1 > > Attachments: 3335-v2.txt, 3335.txt > > > This is most obviously visible in OptionalTasks which should not block > shutdown, but often does. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3335) ThreadPoolExecutor creates threads as non-daemon and will block on shutdown by default
[ https://issues.apache.org/jira/browse/CASSANDRA-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-3335: -- Attachment: 3335-v2.txt v2 adds a log line before running any scheduled task. Maybe that will help pinpoint the culprit. > ThreadPoolExecutor creates threads as non-daemon and will block on shutdown > by default > -- > > Key: CASSANDRA-3335 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3335 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Brandon Williams >Assignee: Jonathan Ellis >Priority: Minor > Fix For: 1.0.1 > > Attachments: 3335-v2.txt, 3335.txt > > > This is most obviously visible in OptionalTasks which should not block > shutdown, but often does. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3280) Pig Storage Handler: Add >=0.8.1 types, Guess right type for Key in Schema
[
https://issues.apache.org/jira/browse/CASSANDRA-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127915#comment-13127915
]
Hudson commented on CASSANDRA-3280:
---
Integrated in Cassandra-0.8 #374 (See
[https://builds.apache.org/job/Cassandra-0.8/374/])
Add 0.8+ types and key validation type to pig schema.
Patch by Steeve Morin, reviewed by brandonwilliams for CASSANDRA-3280
brandonwilliams :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183518
Files :
* /cassandra/branches/cassandra-0.8/CHANGES.txt
*
/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
> Pig Storage Handler: Add >=0.8.1 types, Guess right type for Key in Schema
> --
>
> Key: CASSANDRA-3280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3280
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
>Affects Versions: 0.8.1
>Reporter: Steeve Morin
> Labels: patch
> Fix For: 0.8.8
>
> Attachments: new_types_and_key_type.txt, new_types_and_key_type_v2.txt
>
>
> Various patches in the Pig Storage Handler:
> - correctly guess the right Pig type for the Row Key
> - add support for FloatType, DoubleType, -UUIDType (as String) and DateType
> (as time epoch)- (removed in v2 because it would break when storing data back
> in Cassandra)
> - add support to specify correct type comparator in SlicePredicate
> *SlicePredicate comparator:*
> For instance:
> {quote}
> {{raw = LOAD
> 'cassandra://ks/cf?slice_start=2.5&slice_end=5.3&comparator=DoubleType' USING
> CassandraStorage() AS ();}}
> {quote}
> It's an optional parameter. If it's not present, it will default to
> BytesType. Which mean you must use hex strings.
> Hence {{slice_start=00&slice_end=FF}} isn't the same as
> {{slice_start=00&slice_end=FF&comparator=AsciiType}} !
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3185) Unify support across different map/reduce related classes for comma seperated list of hosts for initial thrift port connection.
[ https://issues.apache.org/jira/browse/CASSANDRA-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127916#comment-13127916 ] Hudson commented on CASSANDRA-3185: --- Integrated in Cassandra-0.8 #374 (See [https://builds.apache.org/job/Cassandra-0.8/374/]) Unify hadoop support for accept CDL for initial thrift address Patch by Eldon Stegall, reviewed by brandonwilliams for CASSANDRA-3185 brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183506 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ConfigHelper.java * /cassandra/branches/cassandra-0.8/test/distributed/org/apache/cassandra/TestBase.java * /cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/client/TestRingCache.java > Unify support across different map/reduce related classes for comma seperated > list of hosts for initial thrift port connection. > --- > > Key: CASSANDRA-3185 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3185 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Affects Versions: 0.8.0 > Environment: Hadoop-0.20+pig-0.8 >Reporter: Eldon Stegall >Assignee: Eldon Stegall >Priority: Minor > Fix For: 0.8.8 > > Attachments: > unify-comma-seperated-hadoop-initial-thrift-connection-v4.patch > > > Unify support across different map/reduce related classes for comma seperated > list of hosts for initial thrift port connection. Column family input format > already had support for this since CASSANDRA-2807, and RingCache did it the > same way, but it was coded seperately. This JIRA should add pig support, and > abstract a common method to iterate over these seeds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3342) cassandra-cli allows setting min_compaction_threshold to 1
[
https://issues.apache.org/jira/browse/CASSANDRA-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127914#comment-13127914
]
Hudson commented on CASSANDRA-3342:
---
Integrated in Cassandra-0.8 #374 (See
[https://builds.apache.org/job/Cassandra-0.8/374/])
ColumnFamily min_compaction_threshold should be >= 2
patch by Pavel Yaskevich; reviewed by Brandon Williams for CASSANDRA-3342
xedin : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183510
Files :
* /cassandra/branches/cassandra-0.8/CHANGES.txt
*
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java
> cassandra-cli allows setting min_compaction_threshold to 1
> --
>
> Key: CASSANDRA-3342
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3342
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Linux 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Jun 20
> 14:15:38 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux (RHEL 6)
>Reporter: Jason Baker
>Assignee: Pavel Yaskevich
> Fix For: 0.8.8, 1.0.1
>
> Attachments: CASSANDRA-3342.patch
>
>
> {{
> [root@Apture] update column family MagicLinks with min_compaction_threshold=1
> and max_compaction_threshold=20;
> b98e3b80-f3a3-11e0--76abb4a6dbbf
> Waiting for schema agreement...
> ... schemas agree across the cluster
> }}
> I'm told that a min_compaction_threshold of 1 is nonsensical. I had a spell
> where my servers stopped doing compactions. Once I upped the
> min_compaction_threshold, they started compacting again. I'm unable to
> confirm for sure that this was the case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3272) READ Operation with CL=EACH_QUORUM succeed when a DC is down (RF=3)
[
https://issues.apache.org/jira/browse/CASSANDRA-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-3272:
--
Attachment: 3272-v2.txt
v2 attached
> READ Operation with CL=EACH_QUORUM succeed when a DC is down (RF=3)
> ---
>
> Key: CASSANDRA-3272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3272
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Jackson Chung
>Assignee: Jonathan Ellis
>Priority: Minor
> Fix For: 1.1
>
> Attachments: 3272-v2.txt, 3272.txt
>
>
> "READ EACH_QUORUM:Returns the record with the most recent timestamp once
> a quorum of replicas in each data center of the cluster has responded."
> In other words, if a DC is down and the QUORUM could not be reached on that
> DC, read should fail.
> test case:
> - Cassandra version 0.8.6:
> INFO [main] 2011-09-28 22:26:24,297 StorageService.java (line 371) Cassandra
> version: 0.8.6
> - 6-node cluster with 2 DC and 3 node each. RF=3 in each DC:
> [default@Keyspace3] describe keyspace;
> Keyspace: Keyspace3:
> Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy
> Durable Writes: true
> Options: [DC2:3, DC1:3]
> Column Families:
> ColumnFamily: test
> Key Validation Class: org.apache.cassandra.db.marshal.BytesType
> Default column value validator: org.apache.cassandra.db.marshal.BytesType
> Columns sorted by: org.apache.cassandra.db.marshal.BytesType
> Row cache size / save period in seconds: 0.0/0
> Key cache size / save period in seconds: 20.0/14400
> Memtable thresholds: 1.0875/1440/232 (millions of ops/minutes/MB)
> GC grace seconds: 864000
> Compaction min/max thresholds: 4/32
> Read repair chance: 1.0
> Replicate on write: true
> Built indexes: []
> all nodes are up, insert a row:
> $ nodetool -h localhost ring
> Address DC Rack Status State Load Owns Token
> 141784319550391026443072753096570088106
> 10.34.79.179 DC1 RAC1 Up Normal 11.13 KB 16.67% 0
> 10.34.70.163 DC2 RAC1 Up Normal 11.14 KB 16.67%
> 28356863910078205288614550619314017621
> 10.35.81.147 DC1 RAC1 Up Normal 11.14 KB 16.67%
> 56713727820156410577229101238628035242
> 10.84.233.170 DC2 RAC1 Up Normal 11.14 KB 16.67%
> 85070591730234615865843651857942052864
> 10.195.201.236 DC1 RAC1 Up Normal 11.14 KB 16.67%
> 113427455640312821154458202477256070485
> 10.118.147.73 DC2 RAC1 Up Normal 11.14 KB 16.67%
> 141784319550391026443072753096570088106
> - insert a value
> [default@Keyspace3] set
> test[utf8('test-key-1')][utf8('test-col')]=utf8('test-value');
> Value inserted.
> sanity check (cli connects to a node in DC1) :
> [default@Keyspace3] consistencylevel as EACH_QUORUM;
>
> Consistency level is set to 'EACH_QUORUM'.
> [default@Keyspace3] get test[utf8('test-key-1')];
> => (column=746573742d636f6c, value=test-value, timestamp=1317249361722000)
> Returned 1 results
> shut down DC2:
> $ nodetool -h localhost ring
> Address DC RackStatus State LoadOwns
> Token
>
> 141784319550391026443072753096570088106
> 10.34.79.179DC1 RAC1Up Normal 51.86 KB16.67%
> 0
> 10.34.70.163DC2 RAC1Down Normal 51.88 KB16.67%
> 28356863910078205288614550619314017621
> 10.35.81.147DC1 RAC1Up Normal 47.5 KB 16.67%
> 56713727820156410577229101238628035242
> 10.84.233.170 DC2 RAC1Down Normal 51.88 KB16.67%
> 85070591730234615865843651857942052864
> 10.195.201.236 DC1 RAC1Up Normal 47.5 KB 16.67%
> 113427455640312821154458202477256070485
> 10.118.147.73 DC2 RAC1Down Normal 51.88 KB16.67%
> 141784319550391026443072753096570088106
> [default@Keyspace3] get test[utf8('test-key-1')];
> => (column=746573742d636f6c, value=746573742d76616c7565,
> timestamp=1317249361722000)
> Returned 1 results.
> tried with pycassaShell:
> >>> col_fam.get('test-key-1',read_consistency_level=pycassa.ConsistencyLevel.EACH_QUORUM)
> OrderedDict([('test-col', 'test-value')])
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3280) Pig Storage Handler: Add >=0.8.1 types, Guess right type for Key in Schema
[
https://issues.apache.org/jira/browse/CASSANDRA-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127904#comment-13127904
]
Brandon Williams commented on CASSANDRA-3280:
-
Committed, thanks!
> Pig Storage Handler: Add >=0.8.1 types, Guess right type for Key in Schema
> --
>
> Key: CASSANDRA-3280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3280
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
>Affects Versions: 0.8.1
>Reporter: Steeve Morin
> Labels: patch
> Fix For: 0.8.8
>
> Attachments: new_types_and_key_type.txt, new_types_and_key_type_v2.txt
>
>
> Various patches in the Pig Storage Handler:
> - correctly guess the right Pig type for the Row Key
> - add support for FloatType, DoubleType, -UUIDType (as String) and DateType
> (as time epoch)- (removed in v2 because it would break when storing data back
> in Cassandra)
> - add support to specify correct type comparator in SlicePredicate
> *SlicePredicate comparator:*
> For instance:
> {quote}
> {{raw = LOAD
> 'cassandra://ks/cf?slice_start=2.5&slice_end=5.3&comparator=DoubleType' USING
> CassandraStorage() AS ();}}
> {quote}
> It's an optional parameter. If it's not present, it will default to
> BytesType. Which mean you must use hex strings.
> Hence {{slice_start=00&slice_end=FF}} isn't the same as
> {{slice_start=00&slice_end=FF&comparator=AsciiType}} !
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1183518 - in /cassandra/branches/cassandra-0.8: CHANGES.txt contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
Author: brandonwilliams
Date: Fri Oct 14 21:57:29 2011
New Revision: 1183518
URL: http://svn.apache.org/viewvc?rev=1183518&view=rev
Log:
Add 0.8+ types and key validation type to pig schema.
Patch by Steeve Morin, reviewed by brandonwilliams for CASSANDRA-3280
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1183518&r1=1183517&r2=1183518&view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Oct 14 21:57:29 2011
@@ -15,6 +15,8 @@
* (Hadoop) accept comma delimited lists of initial thrift connections
(CASSANDRA-3185)
* ColumnFamily min_compaction_threshold should be >= 2 (CASSANDRA-3342)
+ * (Pig) add 0.8+ types and key validation type in schema (CASSANDRA-3280)
+
0.8.7
* Kill server on wrapped OOME such as from FileChannel.map (CASSANDRA-3201)
Modified:
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?rev=1183518&r1=1183517&r2=1183518&view=diff
==
---
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
(original)
+++
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
Fri Oct 14 21:57:29 2011
@@ -107,7 +107,7 @@ public class CassandraStorage extends Lo
return limit;
}
- @Override
+@Override
public Tuple getNext() throws IOException
{
try
@@ -122,7 +122,7 @@ public class CassandraStorage extends Lo
assert key != null && cf != null;
// and wrap it in a tuple
- Tuple tuple = TupleFactory.getInstance().newTuple(2);
+Tuple tuple = TupleFactory.getInstance().newTuple(2);
ArrayList columns = new ArrayList();
tuple.set(0, new DataByteArray(key.array(),
key.position()+key.arrayOffset(), key.limit()+key.arrayOffset()));
for (Map.Entry entry : cf.entrySet())
@@ -187,10 +187,12 @@ public class CassandraStorage extends Lo
ArrayList marshallers = new ArrayList();
AbstractType comparator = null;
AbstractType default_validator = null;
+AbstractType key_validator = null;
try
{
-comparator = TypeParser.parse(cfDef.comparator_type);
-default_validator =
TypeParser.parse(cfDef.default_validation_class);
+comparator = TypeParser.parse(cfDef.getComparator_type());
+default_validator =
TypeParser.parse(cfDef.getDefault_validation_class());
+key_validator = TypeParser.parse(cfDef.getKey_validation_class());
}
catch (ConfigurationException e)
{
@@ -199,13 +201,14 @@ public class CassandraStorage extends Lo
marshallers.add(comparator);
marshallers.add(default_validator);
+marshallers.add(key_validator);
return marshallers;
}
-private Map getValidatorMap(CfDef cfDef) throws
IOException
+private Map getValidatorMap(CfDef cfDef) throws
IOException
{
Map validators = new HashMap();
-for (ColumnDef cd : cfDef.column_metadata)
+for (ColumnDef cd : cfDef.getColumn_metadata())
{
if (cd.getValidation_class() != null &&
!cd.getValidation_class().isEmpty())
{
@@ -236,6 +239,18 @@ public class CassandraStorage extends Lo
this.reader = reader;
}
+public static Map getQueryMap(String query)
+{
+String[] params = query.split("&");
+Map map = new HashMap();
+for (String param : params)
+{
+String[] keyValue = param.split("=");
+map.put(keyValue[0], keyValue[1]);
+}
+return map;
+}
+
private void setLocationFromUri(String location) throws IOException
{
// parse uri into keyspace and columnfamily
@@ -247,18 +262,18 @@ public class CassandraStorage extends Lo
String[] urlParts = location.split("\\?");
if (urlParts.length > 1)
{
-for (String param : urlParts[1].split("&"))
-{
-String[] pair = param.split("=");
-if (pair[0].equals("slice_start"))
-slice_start = ByteBufferUtil.bytes(pair[1]);
-else if (pair[0].equals("slice_end"))
-slice_end = ByteBuf
svn commit: r1183517 - in /cassandra/trunk: ./ bin/ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/cql/
Author: jbellis Date: Fri Oct 14 21:56:38 2011 New Revision: 1183517 URL: http://svn.apache.org/viewvc?rev=1183517&view=rev Log: merge from 1.0 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/bin/nodetool cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/cql/Cql.g Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 21:56:38 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1183002,1183241 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 -/cassandra/branches/cassandra-1.0:1167085-1183003,1183245 +/cassandra/branches/cassandra-1.0:1167085-1183245 /cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1181093,1181741,1182951,1183243 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1183517&r1=1183516&r2=1183517&view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Fri Oct 14 21:56:38 2011 @@ -12,11 +12,12 @@ * (CQL) allow numeric keyspace names in USE statement (CASSANDRA-3350) * reduce network copies (CASSANDRA-) * (Hadoop) skip empty rows when slicing the entire row (CASSANDRA-2855) - * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357) * fix ColumnIndexer to use long offsets (CASSANDRA-3358) * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357) * Only count compaction as active (for throttling) when they have successfully acquired the compaction lock (CASSANDRA-3344) + * limit nodetool to 32MB of heap (CASSANDRA-3124) + * (CQL) update parser to accept "timestamp" instead of "date" (CASSANDRA-3149) 1.0.0-final Modified: cassandra/trunk/bin/nodetool URL: http://svn.apache.org/viewvc/cassandra/trunk/bin/nodetool?rev=1183517&r1=1183516&r2=1183517&view=diff == --- cassandra/trunk/bin/nodetool (original) +++ cassandra/trunk/bin/nodetool Fri Oct 14 21:56:38 2011 @@ -55,8 +55,10 @@ case "`uname`" in ;; esac -$JAVA -cp $CLASSPATH -Dstorage-config=$CASSANDRA_CONF \ --Dlog4j.configuration=log4j-tools.properties \ -org.apache.cassandra.tools.NodeCmd $@ +$JAVA -cp $CLASSPATH \ + -Xmx32m \ + -Dlog4j.configuration=log4j-tools.properties \ + -Dstorage-config=$CASSANDRA_CONF \ + org.apache.cassandra.tools.NodeCmd $@ # vi:ai sw=4 ts=4 tw=0 et Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 21:56:38 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1183002,1183241 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 -/cassandra/branches/cassandra-1.0/contrib:1167085-1183003,1183245 +/cassandra/branches/cassandra-1.0/contrib:1167085-1183245 /cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1181093,1181741,1182951,1183243 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1/contrib:1102511-1125020 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 21:56:38 2011 @@ -4,7 +4,7 @@ /cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1183002,1183241 /cassandra/branches/cassandra-0.8.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1125021-1130369 /cassandra/branches/cassandra-0.8.1/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1101014-1125018 -/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1167085-1183003,1183245 +/cassandra/branches/cassandra-1.0/interface/thrift/gen-java/o
svn commit: r1183516 - /cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java
Author: jbellis
Date: Fri Oct 14 21:54:12 2011
New Revision: 1183516
URL: http://svn.apache.org/viewvc?rev=1183516&view=rev
Log:
r/m redundant validateColumnFamily calls
patch by jbellis
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java?rev=1183516&r1=1183515&r2=1183516&view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/cql/QueryProcessor.java
Fri Oct 14 21:54:12 2011
@@ -73,11 +73,10 @@ public class QueryProcessor
public static final String DEFAULT_KEY_NAME =
bufferToString(CFMetaData.DEFAULT_KEY_NAME);
-private static List getSlice(String keyspace,
SelectStatement select)
+private static List getSlice(CFMetaData
metadata, SelectStatement select)
throws InvalidRequestException, TimedOutException, UnavailableException
{
QueryPath queryPath = new QueryPath(select.getColumnFamily());
-CFMetaData metadata = validateColumnFamily(keyspace,
select.getColumnFamily());
List commands = new ArrayList();
// ...of a list of column names
@@ -91,13 +90,13 @@ public class QueryProcessor
ByteBuffer key =
rawKey.getByteBuffer(metadata.getKeyValidator());
validateKey(key);
-commands.add(new SliceByNamesReadCommand(keyspace, key,
queryPath, columnNames));
+commands.add(new SliceByNamesReadCommand(metadata.ksName, key,
queryPath, columnNames));
}
}
// ...a range (slice) of column names
else
{
-AbstractType comparator = select.getComparator(keyspace);
+AbstractType comparator = select.getComparator(metadata.ksName);
ByteBuffer start =
select.getColumnStart().getByteBuffer(comparator);
ByteBuffer finish =
select.getColumnFinish().getByteBuffer(comparator);
@@ -107,7 +106,7 @@ public class QueryProcessor
validateKey(key);
validateSliceRange(metadata, start, finish,
select.isColumnsReversed());
-commands.add(new SliceFromReadCommand(keyspace,
+commands.add(new SliceFromReadCommand(metadata.ksName,
key,
queryPath,
start,
@@ -144,13 +143,13 @@ public class QueryProcessor
return columnNames;
}
-private static List multiRangeSlice(String
keyspace, SelectStatement select)
+private static List
multiRangeSlice(CFMetaData metadata, SelectStatement select)
throws TimedOutException, UnavailableException, InvalidRequestException
{
List rows;
IPartitioner p = StorageService.getPartitioner();
-AbstractType keyType = Schema.instance.getCFMetaData(keyspace,
select.getColumnFamily()).getKeyValidator();
+AbstractType keyType =
Schema.instance.getCFMetaData(metadata.ksName,
select.getColumnFamily()).getKeyValidator();
ByteBuffer startKey = (select.getKeyStart() != null)
? select.getKeyStart().getByteBuffer(keyType)
@@ -170,7 +169,6 @@ public class QueryProcessor
}
AbstractBounds bounds = new Bounds(startToken, finishToken);
-CFMetaData metadata = validateColumnFamily(keyspace,
select.getColumnFamily());
// XXX: Our use of Thrift structs internally makes me Sad. :(
SlicePredicate thriftSlicePredicate = slicePredicateFromSelect(select,
metadata);
validateSlicePredicate(metadata, thriftSlicePredicate);
@@ -181,7 +179,7 @@ public class QueryProcessor
try
{
-rows = StorageProxy.getRangeSlice(new RangeSliceCommand(keyspace,
+rows = StorageProxy.getRangeSlice(new
RangeSliceCommand(metadata.ksName,
select.getColumnFamily(),
null,
thriftSlicePredicate,
@@ -220,10 +218,9 @@ public class QueryProcessor
return rows.subList(0, select.getNumRecords() < rows.size() ?
select.getNumRecords() : rows.size());
}
-private static List getIndexedSlices(String
keyspace, SelectStatement select)
+private static List
getIndexedSlices(CFMetaData metadata, SelectStatement select)
throws TimedOutException, UnavailableException, InvalidRequestException
{
-CF
[jira] [Resolved] (CASSANDRA-2999) cassandra-cli doesn't accept floats for memtable_thoughput
[ https://issues.apache.org/jira/browse/CASSANDRA-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Yaskevich resolved CASSANDRA-2999. Resolution: Won't Fix > cassandra-cli doesn't accept floats for memtable_thoughput > -- > > Key: CASSANDRA-2999 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2999 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Tyler Hobbs >Assignee: Pavel Yaskevich >Priority: Trivial > Fix For: 1.0.1 > > > The cassandra-cli does not accept floats for the value of memtable_throughput > (memtable_throughput_in_mb). This means that you can't specify a value with > sub-MB resolution, such as 0.5. > This probably doesn't matter much for production, but it's useful for testing > really low memtable thresholds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2999) cassandra-cli doesn't accept floats for memtable_thoughput
[ https://issues.apache.org/jira/browse/CASSANDRA-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127893#comment-13127893 ] Brandon Williams commented on CASSANDRA-2999: - I'm -1 on changing from an int to a float in what will be an oldstable release very shortly. > cassandra-cli doesn't accept floats for memtable_thoughput > -- > > Key: CASSANDRA-2999 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2999 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Tyler Hobbs >Assignee: Pavel Yaskevich >Priority: Trivial > Fix For: 1.0.1 > > > The cassandra-cli does not accept floats for the value of memtable_throughput > (memtable_throughput_in_mb). This means that you can't specify a value with > sub-MB resolution, such as 0.5. > This probably doesn't matter much for production, but it's useful for testing > really low memtable thresholds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2999) cassandra-cli doesn't accept floats for memtable_thoughput
[ https://issues.apache.org/jira/browse/CASSANDRA-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127890#comment-13127890 ] Pavel Yaskevich commented on CASSANDRA-2999: I think that this should be moved to 0.8.8 instead 1.0.1 because memtable_throughput_in_mb was removed in 1.0.0. Also CLI does not accept float because that parameter is int, do you think that this is a good idea to change it to float even if it would be used only for testing proposes? > cassandra-cli doesn't accept floats for memtable_thoughput > -- > > Key: CASSANDRA-2999 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2999 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Tyler Hobbs >Assignee: Pavel Yaskevich >Priority: Trivial > Fix For: 1.0.1 > > > The cassandra-cli does not accept floats for the value of memtable_throughput > (memtable_throughput_in_mb). This means that you can't specify a value with > sub-MB resolution, such as 0.5. > This probably doesn't matter much for production, but it's useful for testing > really low memtable thresholds. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3361) add logging to StorageProxy for mutations and CL
[
https://issues.apache.org/jira/browse/CASSANDRA-3361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-3361.
---
Resolution: Fixed
Fix Version/s: 1.0.1
Reviewer: jbellis
Assignee: Jackson Chung
committed, thanks!
> add logging to StorageProxy for mutations and CL
>
>
> Key: CASSANDRA-3361
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3361
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jackson Chung
>Assignee: Jackson Chung
>Priority: Minor
> Fix For: 1.0.1
>
> Attachments: 3361.patch
>
>
> Through
> https://github.com/apache/cassandra/commit/dc9992a391f01ef79b74b5d9fc69fb7390184ecf#L20R322
> the read command and CL is logged (under DEBUG) and it is very helpful. But
> I don't think I have seen the same for mutation...
> It'd be nice if we could also log this for the mutation (and others?), please
> see the attached patch (just done on mutation) . Note also this prints the
> IMutation which I did wonder if it is a good idea (could the list be so large
> that it floods the log instead of being useful?)
> sample output:
> {noformat}
> DEBUG [RPC-Thread:24] 2011-10-14 12:09:10,949 StorageProxy.java (line 171)
> Mutations/ConsistencyLevel is [RowMutation(keyspace='testks', key='70',
> modifications=[ColumnFamily(testindx
> [testcol:false:1@1318619350892000,])])]/ONE
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1183510 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/thrift/ThriftValidation.java
Author: xedin
Date: Fri Oct 14 21:35:54 2011
New Revision: 1183510
URL: http://svn.apache.org/viewvc?rev=1183510&view=rev
Log:
ColumnFamily min_compaction_threshold should be >= 2
patch by Pavel Yaskevich; reviewed by Brandon Williams for CASSANDRA-3342
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1183510&r1=1183509&r2=1183510&view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Oct 14 21:35:54 2011
@@ -14,7 +14,7 @@
(CASSANDRA-3214)
* (Hadoop) accept comma delimited lists of initial thrift connections
(CASSANDRA-3185)
-
+ * ColumnFamily min_compaction_threshold should be >= 2 (CASSANDRA-3342)
0.8.7
* Kill server on wrapped OOME such as from FileChannel.map (CASSANDRA-3201)
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java?rev=1183510&r1=1183509&r2=1183510&view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/thrift/ThriftValidation.java
Fri Oct 14 21:35:54 2011
@@ -679,19 +679,11 @@ public class ThriftValidation
{
if (cf_def.isSetMin_compaction_threshold() &&
cf_def.isSetMax_compaction_threshold())
{
-if ((cf_def.min_compaction_threshold >
cf_def.max_compaction_threshold)
-&& cf_def.max_compaction_threshold != 0)
-{
-throw new ConfigurationException("min_compaction_threshold
cannot be greater than max_compaction_threshold");
-}
+validateMinCompactionThreshold(cf_def.min_compaction_threshold,
cf_def.max_compaction_threshold);
}
else if (cf_def.isSetMin_compaction_threshold())
{
-if (cf_def.min_compaction_threshold >
CFMetaData.DEFAULT_MAX_COMPACTION_THRESHOLD)
-{
-throw new
ConfigurationException(String.format("min_compaction_threshold cannot be
greather than max_compaction_threshold (default %d)",
-
CFMetaData.DEFAULT_MAX_COMPACTION_THRESHOLD));
-}
+validateMinCompactionThreshold(cf_def.min_compaction_threshold,
CFMetaData.DEFAULT_MAX_COMPACTION_THRESHOLD);
}
else if (cf_def.isSetMax_compaction_threshold())
{
@@ -706,6 +698,16 @@ public class ThriftValidation
}
}
+public static void validateMinCompactionThreshold(int
min_compaction_threshold, int max_compaction_threshold) throws
ConfigurationException
+{
+if (min_compaction_threshold <= 1)
+throw new ConfigurationException("min_compaction_threshold cannot
be less than 2.");
+
+if (min_compaction_threshold > max_compaction_threshold &&
max_compaction_threshold != 0)
+throw new
ConfigurationException(String.format("min_compaction_threshold cannot be
greater than max_compaction_threshold %d",
+
max_compaction_threshold));
+}
+
public static void
validateMemtableSettings(org.apache.cassandra.thrift.CfDef cf_def) throws
ConfigurationException
{
if (cf_def.isSetMemtable_flush_after_mins())
svn commit: r1183509 - /cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java
Author: jbellis
Date: Fri Oct 14 21:35:42 2011
New Revision: 1183509
URL: http://svn.apache.org/viewvc?rev=1183509&view=rev
Log:
add debug log of mutations + consistencylevel in SP.mutate
patch by Jackson Chung; reviewed by jbellis for CASSANDRA-3361
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java
Modified:
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java?rev=1183509&r1=1183508&r2=1183509&view=diff
==
---
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java
(original)
+++
cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/service/StorageProxy.java
Fri Oct 14 21:35:42 2011
@@ -169,6 +169,7 @@ public class StorageProxy implements Sto
*/
public static void mutate(List mutations,
ConsistencyLevel consistency_level) throws UnavailableException,
TimeoutException
{
+logger.debug("Mutations/ConsistencyLevel are {}/{}", mutations,
consistency_level);
final String localDataCenter =
DatabaseDescriptor.getEndpointSnitch().getDatacenter(FBUtilities.getBroadcastAddress());
long startTime = System.nanoTime();
[jira] [Commented] (CASSANDRA-3342) cassandra-cli allows setting min_compaction_threshold to 1
[
https://issues.apache.org/jira/browse/CASSANDRA-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127870#comment-13127870
]
Brandon Williams commented on CASSANDRA-3342:
-
+1 with minor typo fixes: s/then/than/ and s/greather/greater/
> cassandra-cli allows setting min_compaction_threshold to 1
> --
>
> Key: CASSANDRA-3342
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3342
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Linux 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Jun 20
> 14:15:38 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux (RHEL 6)
>Reporter: Jason Baker
>Assignee: Pavel Yaskevich
> Fix For: 0.8.8, 1.0.1
>
> Attachments: CASSANDRA-3342.patch
>
>
> {{
> [root@Apture] update column family MagicLinks with min_compaction_threshold=1
> and max_compaction_threshold=20;
> b98e3b80-f3a3-11e0--76abb4a6dbbf
> Waiting for schema agreement...
> ... schemas agree across the cluster
> }}
> I'm told that a min_compaction_threshold of 1 is nonsensical. I had a spell
> where my servers stopped doing compactions. Once I upped the
> min_compaction_threshold, they started compacting again. I'm unable to
> confirm for sure that this was the case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns
[ https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127871#comment-13127871 ] Jonathan Ellis commented on CASSANDRA-2466: --- Great! > bloom filters should avoid huge array allocations to avoid fragmentation > concerns > - > > Key: CASSANDRA-2466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2466 > Project: Cassandra > Issue Type: Bug >Reporter: Peter Schuller >Priority: Minor > Attachments: BloomFilterSerializer.java, OpenBitSet.java > > > The fact that bloom filters are backed by single large arrays of longs is > expected to interact badly with promotion of objects into old gen with CMS, > due to fragmentation concerns (as discussed in CASSANDRA-2463). > It should be less of an issue than CASSANDRA-2463 in the sense that you need > to have a lot of rows before the array sizes become truly huge. For > comparison, the ~ 143 million row key limit implied by the use of 'int' in > BitSet prior to the switch to OpenBitSet translates roughly to 238 MB > (assuming the limitation factor there was the addressability of the bits with > a 32 bit int, which is my understanding). > Having a preliminary look at OpenBitSet with an eye towards replacing the > single long[] with multiple arrays, it seems that if we're willing to drop > some of the functionality that is not used for bloom filter purposes, the > bits[i] indexing should be pretty easy to augment with modulo to address an > appropriate smaller array. Locality is not an issue since the bloom filter > case is the worst possible case for locality anyway, and it doesn't matter > whether it's one huge array or a number of ~ 64k arrays. > Callers may be affected like BloomFilterSerializer which cares about the > underlying bit array. > If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) > some additional acrobatics would be necessary and presumably at a noticable > performance cost if such operations were to be used in performance critical > places. > An argument against touching OpenBitSet is that it seems to be pretty > carefully written and tested and has some non-trivial details and people have > seemingly benchmarked it quite carefully. On the other hand, the improvement > would then apply to other things as well, such as the bitsets used to keep > track of in-core pages (off the cuff for scale, a 64 gig sstable should imply > a 2 mb bit set, with one bit per 4k page). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3342) cassandra-cli allows setting min_compaction_threshold to 1
[
https://issues.apache.org/jira/browse/CASSANDRA-3342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Yaskevich updated CASSANDRA-3342:
---
Attachment: CASSANDRA-3342.patch
> cassandra-cli allows setting min_compaction_threshold to 1
> --
>
> Key: CASSANDRA-3342
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3342
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: Linux 2.6.32-131.6.1.el6.x86_64 #1 SMP Mon Jun 20
> 14:15:38 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux (RHEL 6)
>Reporter: Jason Baker
>Assignee: Pavel Yaskevich
> Fix For: 0.8.8, 1.0.1
>
> Attachments: CASSANDRA-3342.patch
>
>
> {{
> [root@Apture] update column family MagicLinks with min_compaction_threshold=1
> and max_compaction_threshold=20;
> b98e3b80-f3a3-11e0--76abb4a6dbbf
> Waiting for schema agreement...
> ... schemas agree across the cluster
> }}
> I'm told that a min_compaction_threshold of 1 is nonsensical. I had a spell
> where my servers stopped doing compactions. Once I upped the
> min_compaction_threshold, they started compacting again. I'm unable to
> confirm for sure that this was the case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1183506 - in /cassandra/branches/cassandra-0.8: ./ contrib/pig/src/java/org/apache/cassandra/hadoop/pig/ src/java/org/apache/cassandra/client/ src/java/org/apache/cassandra/hadoop/ test/d
Author: brandonwilliams
Date: Fri Oct 14 21:22:14 2011
New Revision: 1183506
URL: http://svn.apache.org/viewvc?rev=1183506&view=rev
Log:
Unify hadoop support for accept CDL for initial thrift address
Patch by Eldon Stegall, reviewed by brandonwilliams for CASSANDRA-3185
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyRecordWriter.java
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ConfigHelper.java
cassandra/branches/cassandra-0.8/test/distributed/org/apache/cassandra/TestBase.java
cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/client/TestRingCache.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1183506&r1=1183505&r2=1183506&view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Oct 14 21:22:14 2011
@@ -12,6 +12,8 @@
successfully acquired the compaction lock (CASSANDRA-3344)
* (Hadoop) make CFIF try rpc_address or fallback to listen_address
(CASSANDRA-3214)
+ * (Hadoop) accept comma delimited lists of initial thrift connections
+ (CASSANDRA-3185)
0.8.7
Modified:
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?rev=1183506&r1=1183505&r2=1183506&view=diff
==
---
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
(original)
+++
cassandra/branches/cassandra-0.8/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java
Fri Oct 14 21:22:14 2011
@@ -545,7 +545,7 @@ public class CassandraStorage extends Lo
Cassandra.Client client = null;
try
{
-client =
createConnection(ConfigHelper.getInitialAddress(conf),
ConfigHelper.getRpcPort(conf), true);
+client = ConfigHelper.getClientFromAddressList(conf);
CfDef cfDef = null;
client.set_keyspace(keyspace);
KsDef ksDef = client.describe_keyspace(keyspace);
@@ -579,21 +579,6 @@ public class CassandraStorage extends Lo
}
}
-private static Cassandra.Client createConnection(String host, Integer
port, boolean framed) throws IOException
-{
-TSocket socket = new TSocket(host, port);
-TTransport trans = framed ? new TFramedTransport(socket) : socket;
-try
-{
-trans.open();
-}
-catch (TTransportException e)
-{
-throw new IOException("unable to connect to server", e);
-}
-return new Cassandra.Client(new TBinaryProtocol(trans));
-}
-
private static String cfdefToString(CfDef cfDef)
{
assert cfDef != null;
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java?rev=1183506&r1=1183505&r2=1183506&view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/client/RingCache.java
Fri Oct 14 21:22:14 2011
@@ -21,25 +21,22 @@ import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.nio.ByteBuffer;
-import java.util.HashSet;
import java.util.List;
-import java.util.Set;
-
-import com.google.common.collect.ArrayListMultimap;
-import com.google.common.collect.Multimap;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
import org.apache.cassandra.dht.IPartitioner;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.hadoop.ConfigHelper;
import org.apache.cassandra.thrift.Cassandra;
import org.apache.cassandra.thrift.InvalidRequestException;
import org.apache.cassandra.thrift.TokenRange;
+import org.apache.hadoop.conf.Configuration;
import org.apache.thrift.TException;
-import org.apache.cassandra.thrift.TBinaryProtocol;
-import org.apache.thrift.transport.TFramedTransport;
-import org.apache.thr
[jira] [Commented] (CASSANDRA-3214) Make CFIF use rpc_endpoint prior to trying endpoint
[ https://issues.apache.org/jira/browse/CASSANDRA-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127868#comment-13127868 ] Hudson commented on CASSANDRA-3214: --- Integrated in Cassandra-0.8 #373 (See [https://builds.apache.org/job/Cassandra-0.8/373/]) Make CFIF use rpc_endpoint prior to trying endpoint. Patch by Eldon Stegall and Scott Fines, reviewed by brandonwilliams for CASSANDRA-3214 brandonwilliams : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183489 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java > Make CFIF use rpc_endpoint prior to trying endpoint > --- > > Key: CASSANDRA-3214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3214 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Affects Versions: 0.8.6 > Environment: Hadoop 0.20 / Cassandra 0.8.5 / Ubuntu 10.04 / >Reporter: Eldon Stegall >Assignee: Eldon Stegall >Priority: Minor > Fix For: 0.8.8 > > Attachments: CASSANDRA-3214-make-cfif-use-rpc-endpoints-v1.txt, > CASSANDRA-3214.patch > > > Following up on CASSANDRA-3187 , we probably need to attempt to use the > rpc_endpoint address first, and then fall back to the gossip endpoint if we > don't get what we want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3214) Make CFIF use rpc_endpoint prior to trying endpoint
[ https://issues.apache.org/jira/browse/CASSANDRA-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams resolved CASSANDRA-3214. - Resolution: Fixed Committed with formatting fixes. Thanks! > Make CFIF use rpc_endpoint prior to trying endpoint > --- > > Key: CASSANDRA-3214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3214 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Affects Versions: 0.8.6 > Environment: Hadoop 0.20 / Cassandra 0.8.5 / Ubuntu 10.04 / >Reporter: Eldon Stegall >Assignee: Eldon Stegall >Priority: Minor > Fix For: 0.8.8 > > Attachments: CASSANDRA-3214-make-cfif-use-rpc-endpoints-v1.txt, > CASSANDRA-3214.patch > > > Following up on CASSANDRA-3187 , we probably need to attempt to use the > rpc_endpoint address first, and then fall back to the gossip endpoint if we > don't get what we want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1183489 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
Author: brandonwilliams
Date: Fri Oct 14 20:27:43 2011
New Revision: 1183489
URL: http://svn.apache.org/viewvc?rev=1183489&view=rev
Log:
Make CFIF use rpc_endpoint prior to trying endpoint.
Patch by Eldon Stegall and Scott Fines, reviewed by brandonwilliams for
CASSANDRA-3214
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1183489&r1=1183488&r2=1183489&view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Oct 14 20:27:43 2011
@@ -10,6 +10,8 @@
* Display CLI version string on startup (CASSANDRA-3196)
* Only count compaction as active (for throttling) when they have
successfully acquired the compaction lock (CASSANDRA-3344)
+ * (Hadoop) make CFIF try rpc_address or fallback to listen_address
+ (CASSANDRA-3214)
0.8.7
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java?rev=1183489&r1=1183488&r2=1183489&view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/hadoop/ColumnFamilyInputFormat.java
Fri Oct 14 20:27:43 2011
@@ -188,13 +188,17 @@ public class ColumnFamilyInputFormat ext
{
ArrayList splits = new ArrayList();
List tokens = getSubSplits(keyspace, cfName, range, conf);
-
+assert range.rpc_endpoints.size() == range.endpoints.size() :
"rpc_endpoints size must match endpoints size";
// turn the sub-ranges into InputSplits
String[] endpoints = range.endpoints.toArray(new
String[range.endpoints.size()]);
// hadoop needs hostname, not ip
-for (int i = 0; i < endpoints.length; i++)
+int endpointIndex = 0;
+for (String endpoint: range.rpc_endpoints)
{
-endpoints[i] =
InetAddress.getByName(endpoints[i]).getHostName();
+String endpoint_address = endpoint;
+ if(endpoint_address == null || endpoint_address ==
"0.0.0.0")
+ endpoint_address =
range.endpoints.get(endpointIndex);
+ endpoints[endpointIndex++] =
InetAddress.getByName(endpoint_address).getHostName();
}
for (int i = 1; i < tokens.size(); i++)
@@ -210,7 +214,7 @@ public class ColumnFamilyInputFormat ext
private List getSubSplits(String keyspace, String cfName,
TokenRange range, Configuration conf) throws IOException
{
int splitsize = ConfigHelper.getInputSplitSize(conf);
-for (String host : range.endpoints)
+for (String host : range.rpc_endpoints)
{
try
{
[jira] [Updated] (CASSANDRA-3361) add logging to StorageProxy for mutations and CL
[
https://issues.apache.org/jira/browse/CASSANDRA-3361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jackson Chung updated CASSANDRA-3361:
-
Attachment: 3361.patch
> add logging to StorageProxy for mutations and CL
>
>
> Key: CASSANDRA-3361
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3361
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jackson Chung
>Priority: Minor
> Attachments: 3361.patch
>
>
> Through
> https://github.com/apache/cassandra/commit/dc9992a391f01ef79b74b5d9fc69fb7390184ecf#L20R322
> the read command and CL is logged (under DEBUG) and it is very helpful. But
> I don't think I have seen the same for mutation...
> It'd be nice if we could also log this for the mutation (and others?), please
> see the attached patch (just done on mutation) . Note also this prints the
> IMutation which I did wonder if it is a good idea (could the list be so large
> that it floods the log instead of being useful?)
> sample output:
> {noformat}
> DEBUG [RPC-Thread:24] 2011-10-14 12:09:10,949 StorageProxy.java (line 171)
> Mutations/ConsistencyLevel is [RowMutation(keyspace='testks', key='70',
> modifications=[ColumnFamily(testindx
> [testcol:false:1@1318619350892000,])])]/ONE
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3361) add logging to StorageProxy for mutations and CL
add logging to StorageProxy for mutations and CL
Key: CASSANDRA-3361
URL: https://issues.apache.org/jira/browse/CASSANDRA-3361
Project: Cassandra
Issue Type: Improvement
Reporter: Jackson Chung
Priority: Minor
Attachments: 3361.patch
Through
https://github.com/apache/cassandra/commit/dc9992a391f01ef79b74b5d9fc69fb7390184ecf#L20R322
the read command and CL is logged (under DEBUG) and it is very helpful. But I
don't think I have seen the same for mutation...
It'd be nice if we could also log this for the mutation (and others?), please
see the attached patch (just done on mutation) . Note also this prints the
IMutation which I did wonder if it is a good idea (could the list be so large
that it floods the log instead of being useful?)
sample output:
{noformat}
DEBUG [RPC-Thread:24] 2011-10-14 12:09:10,949 StorageProxy.java (line 171)
Mutations/ConsistencyLevel is [RowMutation(keyspace='testks', key='70',
modifications=[ColumnFamily(testindx [testcol:false:1@1318619350892000,])])]/ONE
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns
[ https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127749#comment-13127749 ] Oleg Anastasyev commented on CASSANDRA-2466: Of course. I'll try to prepare it till monday. > bloom filters should avoid huge array allocations to avoid fragmentation > concerns > - > > Key: CASSANDRA-2466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2466 > Project: Cassandra > Issue Type: Bug >Reporter: Peter Schuller >Priority: Minor > Attachments: BloomFilterSerializer.java, OpenBitSet.java > > > The fact that bloom filters are backed by single large arrays of longs is > expected to interact badly with promotion of objects into old gen with CMS, > due to fragmentation concerns (as discussed in CASSANDRA-2463). > It should be less of an issue than CASSANDRA-2463 in the sense that you need > to have a lot of rows before the array sizes become truly huge. For > comparison, the ~ 143 million row key limit implied by the use of 'int' in > BitSet prior to the switch to OpenBitSet translates roughly to 238 MB > (assuming the limitation factor there was the addressability of the bits with > a 32 bit int, which is my understanding). > Having a preliminary look at OpenBitSet with an eye towards replacing the > single long[] with multiple arrays, it seems that if we're willing to drop > some of the functionality that is not used for bloom filter purposes, the > bits[i] indexing should be pretty easy to augment with modulo to address an > appropriate smaller array. Locality is not an issue since the bloom filter > case is the worst possible case for locality anyway, and it doesn't matter > whether it's one huge array or a number of ~ 64k arrays. > Callers may be affected like BloomFilterSerializer which cares about the > underlying bit array. > If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) > some additional acrobatics would be necessary and presumably at a noticable > performance cost if such operations were to be used in performance critical > places. > An argument against touching OpenBitSet is that it seems to be pretty > carefully written and tested and has some non-trivial details and people have > seemingly benchmarked it quite carefully. On the other hand, the improvement > would then apply to other things as well, such as the bitsets used to keep > track of in-core pages (off the cuff for scale, a 64 gig sstable should imply > a 2 mb bit set, with one bit per 4k page). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3360) Read data inconsistancy in Cassandra 1.0.0-rc2
[ https://issues.apache.org/jira/browse/CASSANDRA-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127743#comment-13127743 ] Gopalakrishnan Rajagopal commented on CASSANDRA-3360: - I am not too sure... the same hector version and code works with 1.0.0-beta1 version though. So this just is a compatibilty issue with 1.0.0-rc2? > Read data inconsistancy in Cassandra 1.0.0-rc2 > -- > > Key: CASSANDRA-3360 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3360 > Project: Cassandra > Issue Type: Bug > Components: API >Affects Versions: 1.0.0 >Reporter: Gopalakrishnan Rajagopal > > When qsuper column for a particular key is being queried > using hector-core-0.8.0-2, > the data retrieved is inconsistent. I mean, for the key that I use to fetch > data, there are 7 sub columns actually. But the query returns 1 or 3 sub > columns depending on which nodes respond to it. (I tested by bringing down > each one of the three nodes in turn). > When I tried to fetch the data for the same key using cassandra-cli tool, I > get all the 7 sub columns for both the consistancy levels ONE and QUORUM. > Below is the code that I used to fetch data > superColumnQuery = HFactory.createSuperColumnQuery > (keyspaceOperator, > stringSerializer, > stringSerializer, stringSerializer, stringSerializer); > superColumnQuery.setColumnFamily(cfName).setKey > (key).setSuperName(scName); > result=superColumnQuery.execute(); > superColumn=result.get(); > columnList=superColumn.getColumns(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns
[ https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127735#comment-13127735 ] Jonathan Ellis commented on CASSANDRA-2466: --- Are there any other moving parts to this? Could we ask you to submit a patch against trunk? > bloom filters should avoid huge array allocations to avoid fragmentation > concerns > - > > Key: CASSANDRA-2466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2466 > Project: Cassandra > Issue Type: Bug >Reporter: Peter Schuller >Priority: Minor > Attachments: BloomFilterSerializer.java, OpenBitSet.java > > > The fact that bloom filters are backed by single large arrays of longs is > expected to interact badly with promotion of objects into old gen with CMS, > due to fragmentation concerns (as discussed in CASSANDRA-2463). > It should be less of an issue than CASSANDRA-2463 in the sense that you need > to have a lot of rows before the array sizes become truly huge. For > comparison, the ~ 143 million row key limit implied by the use of 'int' in > BitSet prior to the switch to OpenBitSet translates roughly to 238 MB > (assuming the limitation factor there was the addressability of the bits with > a 32 bit int, which is my understanding). > Having a preliminary look at OpenBitSet with an eye towards replacing the > single long[] with multiple arrays, it seems that if we're willing to drop > some of the functionality that is not used for bloom filter purposes, the > bits[i] indexing should be pretty easy to augment with modulo to address an > appropriate smaller array. Locality is not an issue since the bloom filter > case is the worst possible case for locality anyway, and it doesn't matter > whether it's one huge array or a number of ~ 64k arrays. > Callers may be affected like BloomFilterSerializer which cares about the > underlying bit array. > If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) > some additional acrobatics would be necessary and presumably at a noticable > performance cost if such operations were to be used in performance critical > places. > An argument against touching OpenBitSet is that it seems to be pretty > carefully written and tested and has some non-trivial details and people have > seemingly benchmarked it quite carefully. On the other hand, the improvement > would then apply to other things as well, such as the bitsets used to keep > track of in-core pages (off the cuff for scale, a 64 gig sstable should imply > a 2 mb bit set, with one bit per 4k page). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns
[ https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127728#comment-13127728 ] Oleg Anastasyev edited comment on CASSANDRA-2466 at 10/14/11 5:46 PM: -- You mean on-disk format ? Yes. Specifics are handled by BloomFilterSerializer. Attached serializer as implemented for 0.6. (as far as i see no changes are required in BloomFilter itself) was (Author: m0nstermind): You mean on-disk format ? Yes. Specifics are handled by BloomFilterSerializer. Attached serializer as implemented for 0.6. > bloom filters should avoid huge array allocations to avoid fragmentation > concerns > - > > Key: CASSANDRA-2466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2466 > Project: Cassandra > Issue Type: Bug >Reporter: Peter Schuller >Priority: Minor > Attachments: BloomFilterSerializer.java, OpenBitSet.java > > > The fact that bloom filters are backed by single large arrays of longs is > expected to interact badly with promotion of objects into old gen with CMS, > due to fragmentation concerns (as discussed in CASSANDRA-2463). > It should be less of an issue than CASSANDRA-2463 in the sense that you need > to have a lot of rows before the array sizes become truly huge. For > comparison, the ~ 143 million row key limit implied by the use of 'int' in > BitSet prior to the switch to OpenBitSet translates roughly to 238 MB > (assuming the limitation factor there was the addressability of the bits with > a 32 bit int, which is my understanding). > Having a preliminary look at OpenBitSet with an eye towards replacing the > single long[] with multiple arrays, it seems that if we're willing to drop > some of the functionality that is not used for bloom filter purposes, the > bits[i] indexing should be pretty easy to augment with modulo to address an > appropriate smaller array. Locality is not an issue since the bloom filter > case is the worst possible case for locality anyway, and it doesn't matter > whether it's one huge array or a number of ~ 64k arrays. > Callers may be affected like BloomFilterSerializer which cares about the > underlying bit array. > If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) > some additional acrobatics would be necessary and presumably at a noticable > performance cost if such operations were to be used in performance critical > places. > An argument against touching OpenBitSet is that it seems to be pretty > carefully written and tested and has some non-trivial details and people have > seemingly benchmarked it quite carefully. On the other hand, the improvement > would then apply to other things as well, such as the bitsets used to keep > track of in-core pages (off the cuff for scale, a 64 gig sstable should imply > a 2 mb bit set, with one bit per 4k page). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns
[ https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Anastasyev updated CASSANDRA-2466: --- Attachment: BloomFilterSerializer.java You mean on-disk format ? Yes. Specifics are handled by BloomFilterSerializer. Attached serializer as implemented for 0.6. > bloom filters should avoid huge array allocations to avoid fragmentation > concerns > - > > Key: CASSANDRA-2466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2466 > Project: Cassandra > Issue Type: Bug >Reporter: Peter Schuller >Priority: Minor > Attachments: BloomFilterSerializer.java, OpenBitSet.java > > > The fact that bloom filters are backed by single large arrays of longs is > expected to interact badly with promotion of objects into old gen with CMS, > due to fragmentation concerns (as discussed in CASSANDRA-2463). > It should be less of an issue than CASSANDRA-2463 in the sense that you need > to have a lot of rows before the array sizes become truly huge. For > comparison, the ~ 143 million row key limit implied by the use of 'int' in > BitSet prior to the switch to OpenBitSet translates roughly to 238 MB > (assuming the limitation factor there was the addressability of the bits with > a 32 bit int, which is my understanding). > Having a preliminary look at OpenBitSet with an eye towards replacing the > single long[] with multiple arrays, it seems that if we're willing to drop > some of the functionality that is not used for bloom filter purposes, the > bits[i] indexing should be pretty easy to augment with modulo to address an > appropriate smaller array. Locality is not an issue since the bloom filter > case is the worst possible case for locality anyway, and it doesn't matter > whether it's one huge array or a number of ~ 64k arrays. > Callers may be affected like BloomFilterSerializer which cares about the > underlying bit array. > If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) > some additional acrobatics would be necessary and presumably at a noticable > performance cost if such operations were to be used in performance critical > places. > An argument against touching OpenBitSet is that it seems to be pretty > carefully written and tested and has some non-trivial details and people have > seemingly benchmarked it quite carefully. On the other hand, the improvement > would then apply to other things as well, such as the bitsets used to keep > track of in-core pages (off the cuff for scale, a 64 gig sstable should imply > a 2 mb bit set, with one bit per 4k page). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3360) Read data inconsistancy in Cassandra 1.0.0-rc2
[ https://issues.apache.org/jira/browse/CASSANDRA-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127721#comment-13127721 ] Jonathan Ellis commented on CASSANDRA-3360: --- If the CLI is behaving correctly, it sounds like a Hector bug? > Read data inconsistancy in Cassandra 1.0.0-rc2 > -- > > Key: CASSANDRA-3360 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3360 > Project: Cassandra > Issue Type: Bug > Components: API >Affects Versions: 1.0.0 >Reporter: Gopalakrishnan Rajagopal > > When qsuper column for a particular key is being queried > using hector-core-0.8.0-2, > the data retrieved is inconsistent. I mean, for the key that I use to fetch > data, there are 7 sub columns actually. But the query returns 1 or 3 sub > columns depending on which nodes respond to it. (I tested by bringing down > each one of the three nodes in turn). > When I tried to fetch the data for the same key using cassandra-cli tool, I > get all the 7 sub columns for both the consistancy levels ONE and QUORUM. > Below is the code that I used to fetch data > superColumnQuery = HFactory.createSuperColumnQuery > (keyspaceOperator, > stringSerializer, > stringSerializer, stringSerializer, stringSerializer); > superColumnQuery.setColumnFamily(cfName).setKey > (key).setSuperName(scName); > result=superColumnQuery.execute(); > superColumn=result.get(); > columnList=superColumn.getColumns(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns
[ https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127720#comment-13127720 ] Jonathan Ellis commented on CASSANDRA-2466: --- Is serialization the same, or at least backwards-compatible w/ existing OBS BFs? > bloom filters should avoid huge array allocations to avoid fragmentation > concerns > - > > Key: CASSANDRA-2466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2466 > Project: Cassandra > Issue Type: Bug >Reporter: Peter Schuller >Priority: Minor > Attachments: OpenBitSet.java > > > The fact that bloom filters are backed by single large arrays of longs is > expected to interact badly with promotion of objects into old gen with CMS, > due to fragmentation concerns (as discussed in CASSANDRA-2463). > It should be less of an issue than CASSANDRA-2463 in the sense that you need > to have a lot of rows before the array sizes become truly huge. For > comparison, the ~ 143 million row key limit implied by the use of 'int' in > BitSet prior to the switch to OpenBitSet translates roughly to 238 MB > (assuming the limitation factor there was the addressability of the bits with > a 32 bit int, which is my understanding). > Having a preliminary look at OpenBitSet with an eye towards replacing the > single long[] with multiple arrays, it seems that if we're willing to drop > some of the functionality that is not used for bloom filter purposes, the > bits[i] indexing should be pretty easy to augment with modulo to address an > appropriate smaller array. Locality is not an issue since the bloom filter > case is the worst possible case for locality anyway, and it doesn't matter > whether it's one huge array or a number of ~ 64k arrays. > Callers may be affected like BloomFilterSerializer which cares about the > underlying bit array. > If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) > some additional acrobatics would be necessary and presumably at a noticable > performance cost if such operations were to be used in performance critical > places. > An argument against touching OpenBitSet is that it seems to be pretty > carefully written and tested and has some non-trivial details and people have > seemingly benchmarked it quite carefully. On the other hand, the improvement > would then apply to other things as well, such as the bitsets used to keep > track of in-core pages (off the cuff for scale, a 64 gig sstable should imply > a 2 mb bit set, with one bit per 4k page). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1183417 - in /cassandra/site: publish/download/index.html src/content/download/index.html
Author: slebresne Date: Fri Oct 14 16:45:18 2011 New Revision: 1183417 URL: http://svn.apache.org/viewvc?rev=1183417&view=rev Log: Update link to java CQL driver on website Modified: cassandra/site/publish/download/index.html cassandra/site/src/content/download/index.html Modified: cassandra/site/publish/download/index.html URL: http://svn.apache.org/viewvc/cassandra/site/publish/download/index.html?rev=1183417&r1=1183416&r2=1183417&view=diff == --- cassandra/site/publish/download/index.html (original) +++ cassandra/site/publish/download/index.html Fri Oct 14 16:45:18 2011 @@ -131,11 +131,11 @@ - http://www.apache.org/dist/cassandra/drivers/java";> + http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/";> Java (JDBC) - http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/downloads/list";> + http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/";> Python (DBAPI2) Modified: cassandra/site/src/content/download/index.html URL: http://svn.apache.org/viewvc/cassandra/site/src/content/download/index.html?rev=1183417&r1=1183416&r2=1183417&view=diff == --- cassandra/site/src/content/download/index.html (original) +++ cassandra/site/src/content/download/index.html Fri Oct 14 16:45:18 2011 @@ -90,11 +90,11 @@ - http://www.apache.org/dist/cassandra/drivers/java";> + http://code.google.com/a/apache-extras.org/p/cassandra-jdbc/";> Java (JDBC) - http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/downloads/list";> + http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/";> Python (DBAPI2)
[jira] [Updated] (CASSANDRA-3214) Make CFIF use rpc_endpoint prior to trying endpoint
[ https://issues.apache.org/jira/browse/CASSANDRA-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Fines updated CASSANDRA-3214: --- Attachment: CASSANDRA-3214.patch The patch file isn't exactly correct. In getSubSplits, it still refers to range.endpoints instead of range.rpc_endpoints I'm attaching a patch which fixes the typ0; please let me know if it is improperly formatted. I tested this patch against a 0.8.7 cluster with a different listen_address than rpc_address, and it appears to work correctly. Not that that is exactly an automated test, but... > Make CFIF use rpc_endpoint prior to trying endpoint > --- > > Key: CASSANDRA-3214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3214 > Project: Cassandra > Issue Type: Improvement > Components: Hadoop >Affects Versions: 0.8.6 > Environment: Hadoop 0.20 / Cassandra 0.8.5 / Ubuntu 10.04 / >Reporter: Eldon Stegall >Assignee: Eldon Stegall >Priority: Minor > Fix For: 0.8.8 > > Attachments: CASSANDRA-3214-make-cfif-use-rpc-endpoints-v1.txt, > CASSANDRA-3214.patch > > > Following up on CASSANDRA-3187 , we probably need to attempt to use the > rpc_endpoint address first, and then fall back to the gossip endpoint if we > don't get what we want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-3360) Read data inconsistancy in Cassandra 1.0.0-rc2
Read data inconsistancy in Cassandra 1.0.0-rc2 -- Key: CASSANDRA-3360 URL: https://issues.apache.org/jira/browse/CASSANDRA-3360 Project: Cassandra Issue Type: Bug Components: API Affects Versions: 1.0.0 Reporter: Gopalakrishnan Rajagopal When qsuper column for a particular key is being queried using hector-core-0.8.0-2, the data retrieved is inconsistent. I mean, for the key that I use to fetch data, there are 7 sub columns actually. But the query returns 1 or 3 sub columns depending on which nodes respond to it. (I tested by bringing down each one of the three nodes in turn). When I tried to fetch the data for the same key using cassandra-cli tool, I get all the 7 sub columns for both the consistancy levels ONE and QUORUM. Below is the code that I used to fetch data superColumnQuery = HFactory.createSuperColumnQuery (keyspaceOperator, stringSerializer, stringSerializer, stringSerializer, stringSerializer); superColumnQuery.setColumnFamily(cfName).setKey (key).setSuperName(scName); result=superColumnQuery.execute(); superColumn=result.get(); columnList=superColumn.getColumns(); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3067) Simple SSTable Pluggability
[
https://issues.apache.org/jira/browse/CASSANDRA-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127618#comment-13127618
]
Sylvain Lebresne commented on CASSANDRA-3067:
-
Had an initial look at the patches. I haven't try applying the patches yet
(which likely need rebase anyway), so following comments are fairly high level.
For the abstract interfaces to SSTable{Reader, Writer, *Iterator}, I would
prefer calling them AbstractSSTable* and keep the concrete implementation with
their current name. It should make the diffs much more digestible and much more
easy to check. Same thing for future merges. And we kind of use Abstract* for
abstract place around the place so it would rather more consistent than less.
For patch 2, I'm not too sure about what this refactoring buys us, in
particular as long as files are versioned by one character. I understand that
to support other kind of sstables, we would need to somehow distinguish between
"kind" of sstables, but this doesn't seem to fully solve that problem. I think
we already had kind of the same problem with compression, where we went with
using the presence of a sstable component to decide if it's compressed or not.
I'm happy to have a discussion on how to best evolve the sstable versioning but
not sure about the solution here. Follows the same holds for patch 6.
Small nitpick: In patch1, I'm not too fond of introducing a compareTo for
SSTableIdentityIterator (given a SSTableIdentityIterator is not really equal to
it's key), is there an underlying reason ?
> Simple SSTable Pluggability
> ---
>
> Key: CASSANDRA-3067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3067
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: Stu Hood
>Assignee: Stu Hood
> Fix For: 1.1
>
> Attachments:
> 0001-CASSANDRA-3067-Create-an-ABC-for-SSTableIdentityIterat.txt,
> 0002-CASSANDRA-3067-Move-from-linear-SSTable-versions-to-fe.txt,
> 0003-CASSANDRA-3067-Create-an-ABC-for-SSTableWriter.txt,
> 0004-CASSANDRA-3067-Rename-SSTable-Names-Slice-Iterator-to-.txt,
> 0005-CASSANDRA-3067-Create-ABCs-for-SSTableReader-and-KeyIt.txt,
> 0006-CASSANDRA-3067-Allow-overriding-the-current-sstable-ve.txt
>
>
> CASSANDRA-2995 proposes full storage engine pluggability, which is probably
> unavoidable in the long run. For now though, I'd like to propose an
> incremental alternative that preserves the sstable model, but allows it to
> evolve non-linearly.
> The sstable "version" field could allow for simple switching between writable
> sstable types, without moving all the way to differentiating between engines
> as CASSANDRA-2995 requires. This can be accomplished by moving towards a
> "feature flags" model (with a mapping between versions and feature sets),
> rather than a linear versions model (where versions can be strictly ordered
> and all versions above X have a feature).
> There are restrictions on this approach:
> * It's sufficient for an alternate SSTable(Writer|Reader|*) set to require a
> patch to enable (rather than a JAR)
> * Filenames/descriptors/components must conform to the existing conventions
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of "HadoopSupport" by jeremyhanna
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for
change notification.
The "HadoopSupport" page has been changed by jeremyhanna:
http://wiki.apache.org/cassandra/HadoopSupport?action=diff&rev1=44&rev2=45
Comment:
Fixed typo.
* '''cassandra.range.batch.size''' - the default is 4096, but you may need
to lower this depending on your data. This is either specified in your hadoop
configuration or using
`org.apache.cassandra.hadoop.ConfigHelper.setRangeBatchSize`.
* '''rpc_timeout_in_ms''' - this is set in your `cassandra.yaml` (in 0.6
it's `RpcTimeoutInMillis` in `storage-conf.xml`). The rpc timeout is not for
timing out from the client but between nodes. This can be increased to reduce
chances of timing out.
- If you still see timeout exceptions with resultant failed jobs and/or
blacklisted tasktrackers, there are settings that can give Cassandra more
latitude before failing the jobs. An example of usage (in either the job
configuration or taskracker mapred-site.xml):
+ If you still see timeout exceptions with resultant failed jobs and/or
blacklisted tasktrackers, there are settings that can give Cassandra more
latitude before failing the jobs. An example of usage (in either the job
configuration or tasktracker mapred-site.xml):
{{{
[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns
[ https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127586#comment-13127586 ] Oleg Anastasyev commented on CASSANDRA-2466: Promotion failures caused by bloom filters gone away completely. I did not measured performance overhead specifically on bloom filters, so all I can tell that it was not introduced measureable performance degradation on production cluster operation from client perspective. (currently we have ~3300 thrift single column read reqs/sec per node with ~1ms avg call duration measured from client; 98% reads are bounced by bloom filters; size of single bloom filter is up to 250Mb) > bloom filters should avoid huge array allocations to avoid fragmentation > concerns > - > > Key: CASSANDRA-2466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2466 > Project: Cassandra > Issue Type: Bug >Reporter: Peter Schuller >Priority: Minor > Attachments: OpenBitSet.java > > > The fact that bloom filters are backed by single large arrays of longs is > expected to interact badly with promotion of objects into old gen with CMS, > due to fragmentation concerns (as discussed in CASSANDRA-2463). > It should be less of an issue than CASSANDRA-2463 in the sense that you need > to have a lot of rows before the array sizes become truly huge. For > comparison, the ~ 143 million row key limit implied by the use of 'int' in > BitSet prior to the switch to OpenBitSet translates roughly to 238 MB > (assuming the limitation factor there was the addressability of the bits with > a 32 bit int, which is my understanding). > Having a preliminary look at OpenBitSet with an eye towards replacing the > single long[] with multiple arrays, it seems that if we're willing to drop > some of the functionality that is not used for bloom filter purposes, the > bits[i] indexing should be pretty easy to augment with modulo to address an > appropriate smaller array. Locality is not an issue since the bloom filter > case is the worst possible case for locality anyway, and it doesn't matter > whether it's one huge array or a number of ~ 64k arrays. > Callers may be affected like BloomFilterSerializer which cares about the > underlying bit array. > If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) > some additional acrobatics would be necessary and presumably at a noticable > performance cost if such operations were to be used in performance critical > places. > An argument against touching OpenBitSet is that it seems to be pretty > carefully written and tested and has some non-trivial details and people have > seemingly benchmarked it quite carefully. On the other hand, the improvement > would then apply to other things as well, such as the bitsets used to keep > track of in-core pages (off the cuff for scale, a 64 gig sstable should imply > a 2 mb bit set, with one bit per 4k page). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3331) Apache Daemon missing from the binary tarball
[ https://issues.apache.org/jira/browse/CASSANDRA-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127487#comment-13127487 ] Jonathan Ellis commented on CASSANDRA-3331: --- bq. Ideally we would just expect it to be installed in whatever is supposed to be standard for windows Is there a standard? It just comes as a .zip, not a proper installer. bq. That would mean we ship procrun for 1.0.0 but then never after that The alternative is to keep it for all of 1.0 which will set the expectation that we'll keep doing it... I'd rather bite the bullet early. Everyone waits for 1.0.1 anyway right? :) > Apache Daemon missing from the binary tarball > - > > Key: CASSANDRA-3331 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3331 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Benjamin Coverston >Assignee: Benjamin Coverston > Fix For: 1.0.0 > > Attachments: 0001-add-files-to-src-and-bin-add-uninstall.patch, > 0002-adding-daemon-directory.patch > > > Apparently the tools used to run the binary release are missing from the > binary tarball. > I will verify that they are in the 1.0 branch, then update the ticket so we > can ensure that they are included. > Ben -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2466) bloom filters should avoid huge array allocations to avoid fragmentation concerns
[ https://issues.apache.org/jira/browse/CASSANDRA-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127481#comment-13127481 ] Jonathan Ellis commented on CASSANDRA-2466: --- What effect did you observe when deploying this, Oleg? > bloom filters should avoid huge array allocations to avoid fragmentation > concerns > - > > Key: CASSANDRA-2466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2466 > Project: Cassandra > Issue Type: Bug >Reporter: Peter Schuller >Priority: Minor > Attachments: OpenBitSet.java > > > The fact that bloom filters are backed by single large arrays of longs is > expected to interact badly with promotion of objects into old gen with CMS, > due to fragmentation concerns (as discussed in CASSANDRA-2463). > It should be less of an issue than CASSANDRA-2463 in the sense that you need > to have a lot of rows before the array sizes become truly huge. For > comparison, the ~ 143 million row key limit implied by the use of 'int' in > BitSet prior to the switch to OpenBitSet translates roughly to 238 MB > (assuming the limitation factor there was the addressability of the bits with > a 32 bit int, which is my understanding). > Having a preliminary look at OpenBitSet with an eye towards replacing the > single long[] with multiple arrays, it seems that if we're willing to drop > some of the functionality that is not used for bloom filter purposes, the > bits[i] indexing should be pretty easy to augment with modulo to address an > appropriate smaller array. Locality is not an issue since the bloom filter > case is the worst possible case for locality anyway, and it doesn't matter > whether it's one huge array or a number of ~ 64k arrays. > Callers may be affected like BloomFilterSerializer which cares about the > underlying bit array. > If the full functionality of OpenBitSet is to be maintained (e.g., xorCount) > some additional acrobatics would be necessary and presumably at a noticable > performance cost if such operations were to be used in performance critical > places. > An argument against touching OpenBitSet is that it seems to be pretty > carefully written and tested and has some non-trivial details and people have > seemingly benchmarked it quite carefully. On the other hand, the improvement > would then apply to other things as well, such as the bitsets used to keep > track of in-core pages (off the cuff for scale, a 64 gig sstable should imply > a 2 mb bit set, with one bit per 4k page). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2348) Allow periods in secondary index column names
[
https://issues.apache.org/jira/browse/CASSANDRA-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis resolved CASSANDRA-2348.
---
Resolution: Not A Problem
WFM.
> Allow periods in secondary index column names
> -
>
> Key: CASSANDRA-2348
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2348
> Project: Cassandra
> Issue Type: Wish
> Components: API
>Reporter: David Semeria
>Priority: Minor
> Labels: index, json
>
> Premise: it is natural to map JSON objects to column names using periods to
> denote the object hierarchy.
> For example {food:{fruit;{a:apples,b:bananas,c:cherries}}}
> Would map to the following CF names: food.fruit.a, food.fruit.b, food.fruit.c
> However, secondary index names cannot contain periods which means workarounds
> (ie denormalization) must be used to index such columns.
> It would be nice if this restriction could be removed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2524) Use SSTableBoundedScanner for cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-2524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127403#comment-13127403 ] Sylvain Lebresne commented on CASSANDRA-2524: - This looks good to me, though one thing that goes away with that is that for cf that use the BoundedScanner, we don't cleanup the row cache. This is not a really big deal, but it's perfectly possible that following the cleanup, rows that are still on the node get evicted from cache before rows that are not in the ranges of the node, so it's a slight regression. Maybe we could simply add a cache cleanup that iterate through the cache and only keeps keys in range? Though one may argue it's on the fringe of this ticket, I'd prefer we add this here so we commit something that is an improvement, rather than an improvement with a slight regression to be fixed later. > Use SSTableBoundedScanner for cleanup > - > > Key: CASSANDRA-2524 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2524 > Project: Cassandra > Issue Type: Improvement >Reporter: Stu Hood >Assignee: Stu Hood >Priority: Minor > Labels: lhf > Attachments: > 0001-Use-a-SSTableBoundedScanner-for-cleanup-and-improve-cl.txt, > 0002-Oops.-When-indexes-or-counters-are-in-use-must-continu.txt > > > SSTableBoundedScanner seeks rather than scanning through rows, so it would be > significantly more efficient than the existing per-key filtering that cleanup > does. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3344) Compaction throttling can be too slow
[ https://issues.apache.org/jira/browse/CASSANDRA-3344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127388#comment-13127388 ] Hudson commented on CASSANDRA-3344: --- Integrated in Cassandra-0.8 #372 (See [https://builds.apache.org/job/Cassandra-0.8/372/]) Only count compaction as active (for throttling) once the compaction lock has been acquired. patch by Fabien Rousseau and slebresne; reviewed by jbellis for CASSANDRA-3344 slebresne : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1183241 Files : * /cassandra/branches/cassandra-0.8/CHANGES.txt * /cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java > Compaction throttling can be too slow > - > > Key: CASSANDRA-3344 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3344 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.8.0 >Reporter: Fabien Rousseau >Assignee: Sylvain Lebresne >Priority: Minor > Fix For: 0.8.8, 1.0.1 > > Attachments: 001-CASSANDRA-3344.patch, 3344.patch, 3344_v2.patch > > > Compaction throttling needs to know how many active compactions are running > (to divide bandwith for each active compaction). > The way active compaction is counted can be broken because it counts the > number of active threads in the executor BUT the thread starts by acquiring a > lock. > If the lock can't be acquired immediately : the thread is seen as "active" > but does not participate in IO operations. > The case can happen when major compaction are triggered (major compaction > acquire a write lock, while minor compactions acquire a read lock). > Having compaction througput to 16Mb/s, we observed is the following (two > times) : > - only 1 active compaction (a long one for a few hours) starting at 16Mb/s, > then after some time running at 2Mb/s, thus taking a very long time to > complete > - many pending compactions > Using JMX and monitoring the stack trace of the compaction threads showed > that : > - 1 thread was effectively compacting > - 1 thread was waiting to acquire the write lock (due to a major compaction) > - 6 threads were waiting to acquire the read lock (probably due to the > thread above trying to acquire the write lock) > Attached is a proposed patch (very simple, not yet tested) which counts only > active compactions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1183247 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/compaction/
Author: slebresne Date: Fri Oct 14 08:55:08 2011 New Revision: 1183247 URL: http://svn.apache.org/viewvc?rev=1183247&view=rev Log: merge from 1.0 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/db/compaction/CompactionManager.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 08:55:08 2011 @@ -1,11 +1,11 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7:1026516-1183000 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1183002 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1183002,1183241 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 -/cassandra/branches/cassandra-1.0:1167085-1183003 -/cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1181093,1181741,1182951 +/cassandra/branches/cassandra-1.0:1167085-1183003,1183245 +/cassandra/branches/cassandra-1.0.0:1167104-1167229,1167232-1181093,1181741,1182951,1183243 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020 /incubator/cassandra/branches/cassandra-0.3:774578-796573 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1183247&r1=1183246&r2=1183247&view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Fri Oct 14 08:55:08 2011 @@ -14,6 +14,9 @@ * (Hadoop) skip empty rows when slicing the entire row (CASSANDRA-2855) * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357) * fix ColumnIndexer to use long offsets (CASSANDRA-3358) + * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357) + * Only count compaction as active (for throttling) when they have + successfully acquired the compaction lock (CASSANDRA-3344) 1.0.0-final @@ -51,7 +54,6 @@ Fixes merged from 0.8 below: * Force flush of system table after updating/removing a token (CASSANDRA-3243) * Make Pig storage handle implements LoadMetadata (CASSANDRA-2777) * Improved CLI exceptions (CASSANDRA-3312) - * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357) 1.0.0-rc2 Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 08:55:08 2011 @@ -1,11 +1,11 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1183000 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1183002 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1183002,1183241 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 -/cassandra/branches/cassandra-1.0/contrib:1167085-1183003 -/cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1181093,1181741,1182951 +/cassandra/branches/cassandra-1.0/contrib:1167085-1183003,1183245 +/cassandra/branches/cassandra-1.0.0/contrib:1167104-1167229,1167232-1181093,1181741,1182951,1183243 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1/contrib:1102511-1125020 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 08:55:08 2011 @@ -1,11 +1,11 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1183000 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1
svn commit: r1183245 - in /cassandra/branches/cassandra-1.0: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/compaction/
Author: slebresne Date: Fri Oct 14 08:54:00 2011 New Revision: 1183245 URL: http://svn.apache.org/viewvc?rev=1183245&view=rev Log: merge from 1.0.0 Modified: cassandra/branches/cassandra-1.0/ (props changed) cassandra/branches/cassandra-1.0/CHANGES.txt cassandra/branches/cassandra-1.0/contrib/ (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-1.0/src/java/org/apache/cassandra/db/compaction/CompactionManager.java Propchange: cassandra/branches/cassandra-1.0/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 08:54:00 2011 @@ -1,11 +1,11 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7:1026516-1183000 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1183002 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1183002,1183241 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/branches/cassandra-1.0:1167106,1167185 -/cassandra/branches/cassandra-1.0.0:1167104-1181093,1181741,1182951 +/cassandra/branches/cassandra-1.0.0:1167104-1181093,1181741,1182951,1183243 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1:1102511-1125020 /cassandra/trunk:1167085-1167102,1169870 Modified: cassandra/branches/cassandra-1.0/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0/CHANGES.txt?rev=1183245&r1=1183244&r2=1183245&view=diff == --- cassandra/branches/cassandra-1.0/CHANGES.txt (original) +++ cassandra/branches/cassandra-1.0/CHANGES.txt Fri Oct 14 08:54:00 2011 @@ -12,6 +12,9 @@ * fix ColumnIndexer to use long offsets (CASSANDRA-3358) * limit nodetool to 32MB of heap (CASSANDRA-3124) * (CQL) update parser to accept "timestamp" instead of "date" (CASSANDRA-3149) + * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357) + * Only count compaction as active (for throttling) when they have + successfully acquired the compaction lock (CASSANDRA-3344) 1.0.0-final @@ -49,7 +52,6 @@ Fixes merged from 0.8 below: * Force flush of system table after updating/removing a token (CASSANDRA-3243) * Make Pig storage handle implements LoadMetadata (CASSANDRA-2777) * Improved CLI exceptions (CASSANDRA-3312) - * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357) 1.0.0-rc2 Propchange: cassandra/branches/cassandra-1.0/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 08:54:00 2011 @@ -1,11 +1,11 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1183000 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1183002 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1183002,1183241 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/branches/cassandra-1.0/contrib:1167106,1167185 -/cassandra/branches/cassandra-1.0.0/contrib:1167104-1181093,1181741,1182951 +/cassandra/branches/cassandra-1.0.0/contrib:1167104-1181093,1181741,1182951,1183243 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /cassandra/tags/cassandra-0.8.0-rc1/contrib:1102511-1125020 /cassandra/trunk/contrib:1167085-1167102,1169870 Propchange: cassandra/branches/cassandra-1.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 08:54:00 2011 @@ -1,11 +1,11 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1183000 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cas
svn commit: r1183243 - in /cassandra/branches/cassandra-1.0.0: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/compaction/
Author: slebresne Date: Fri Oct 14 08:52:05 2011 New Revision: 1183243 URL: http://svn.apache.org/viewvc?rev=1183243&view=rev Log: merge from 0.8 Modified: cassandra/branches/cassandra-1.0.0/ (props changed) cassandra/branches/cassandra-1.0.0/CHANGES.txt cassandra/branches/cassandra-1.0.0/contrib/ (props changed) cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/branches/cassandra-1.0.0/src/java/org/apache/cassandra/db/compaction/CompactionManager.java Propchange: cassandra/branches/cassandra-1.0.0/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 08:52:05 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7:1026516-1170333,1172024 /cassandra/branches/cassandra-0.7.0:1053690-1055654 -/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1178554,1178785,1178957,1179359,1179364,1180958,1182950 +/cassandra/branches/cassandra-0.8:1090934-1125013,1125019-1178554,1178785,1178957,1179359,1179364,1180958,1182950,1183241 /cassandra/branches/cassandra-0.8.0:1125021-1130369 /cassandra/branches/cassandra-0.8.1:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Modified: cassandra/branches/cassandra-1.0.0/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-1.0.0/CHANGES.txt?rev=1183243&r1=1183242&r2=1183243&view=diff == --- cassandra/branches/cassandra-1.0.0/CHANGES.txt (original) +++ cassandra/branches/cassandra-1.0.0/CHANGES.txt Fri Oct 14 08:52:05 2011 @@ -1,3 +1,9 @@ +1.0.1 + * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357) + * Only count compaction as active (for throttling) when they have + successfully acquired the compaction lock (CASSANDRA-3344) + + 1.0.0-final * close scrubbed sstable fd before deleting it (CASSANDRA-3318) * fix bug preventing obsolete commitlog segments from being removed @@ -30,7 +36,6 @@ Fixes merged from 0.8 below: * Nodetool no longer leaks threads and closes JMX connections (CASSANDRA-3309) * fix truncate allowing data to be replayed post-restart (CASSANDRA-3297) * Move SimpleAuthority and SimpleAuthenticator to examples (CASSANDRA-2922) - * Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357) 1.0.0-rc2 Propchange: cassandra/branches/cassandra-1.0.0/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 08:52:05 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 /cassandra/branches/cassandra-0.7/contrib:1026516-1170333,1172024 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 -/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1178554,1178785,1178957,1179359,1179364,1180958,1182950 +/cassandra/branches/cassandra-0.8/contrib:1090934-1125013,1125019-1178554,1178785,1178957,1179359,1179364,1180958,1182950,1183241 /cassandra/branches/cassandra-0.8.0/contrib:1125021-1130369 /cassandra/branches/cassandra-0.8.1/contrib:1101014-1125018 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 Propchange: cassandra/branches/cassandra-1.0.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 14 08:52:05 2011 @@ -1,7 +1,7 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1131291 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1170333,1172024 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 -/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1178554,1178785,1178957,1179359,1179364,1180958,1182950 +/cassandra/branches/cassandra-0.8/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1090934-1125013,1125019-1178554,1178785,1178957,1179359,117936
svn commit: r1183241 - in /cassandra/branches/cassandra-0.8: CHANGES.txt src/java/org/apache/cassandra/db/compaction/CompactionManager.java
Author: slebresne
Date: Fri Oct 14 08:50:27 2011
New Revision: 1183241
URL: http://svn.apache.org/viewvc?rev=1183241&view=rev
Log:
Only count compaction as active (for throttling) once the compaction lock has
been acquired.
patch by Fabien Rousseau and slebresne; reviewed by jbellis for CASSANDRA-3344
Modified:
cassandra/branches/cassandra-0.8/CHANGES.txt
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
Modified: cassandra/branches/cassandra-0.8/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/CHANGES.txt?rev=1183241&r1=1183240&r2=1183241&view=diff
==
--- cassandra/branches/cassandra-0.8/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.8/CHANGES.txt Fri Oct 14 08:50:27 2011
@@ -8,6 +8,8 @@
* Fix handling of tombstone by SSTableExport/Import (CASSANDRA-3357)
* fix ColumnIndexer to use long offsets (CASSANDRA-3358)
* Display CLI version string on startup (CASSANDRA-3196)
+ * Only count compaction as active (for throttling) when they have
+ successfully acquired the compaction lock (CASSANDRA-3344)
0.8.7
Modified:
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java?rev=1183241&r1=1183240&r2=1183241&view=diff
==
---
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
(original)
+++
cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
Fri Oct 14 08:50:27 2011
@@ -1216,7 +1216,7 @@ public class CompactionManager implement
public int getActiveCompactions()
{
-return executor.getActiveCount() + validationExecutor.getActiveCount();
+return CompactionExecutor.compactions.size();
}
private static class CompactionExecutor extends
DebuggableThreadPoolExecutor
[jira] [Commented] (CASSANDRA-2854) Java Build Path is broken in Eclipse after running generate-eclipse-files Ant target
[ https://issues.apache.org/jira/browse/CASSANDRA-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127366#comment-13127366 ] David Allsopp commented on CASSANDRA-2854: -- Will try to take a look. Probably needs to detect the OS in Ant to modify the regex passed to split(), e.g.: No path.sep set for this OS! Or (possibly) use unix-style paths in the first place if Ant handles them OK on Windows (but not sure where they originate from; this may not be possible)... > Java Build Path is broken in Eclipse after running generate-eclipse-files Ant > target > > > Key: CASSANDRA-2854 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2854 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Affects Versions: 0.8.1 > Environment: Windows 7 64-bit, Eclipse Helios SR1, Subclipse >Reporter: David Allsopp >Priority: Minor > > Following the instructions in > http://wiki.apache.org/cassandra/RunningCassandraInEclipse, but checking out > v0.8.1: > After running the 'generate-eclipse-files' Ant target, the build path is set > up to include all the libraries in build/lib/jars, but each library has ;C > (semicolon, letter C) appended to it, so Eclipse can't find the libraries - > there are about 55 errors like: > Project 'cassandra-0.8.1' is missing required library: > '\Users\David\eclipse_workspace\cassandra-0.8.1\build\lib\jars\commons-cli-1.2.jar;C' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3272) READ Operation with CL=EACH_QUORUM succeed when a DC is down (RF=3)
[
https://issues.apache.org/jira/browse/CASSANDRA-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127361#comment-13127361
]
Sylvain Lebresne commented on CASSANDRA-3272:
-
Shouldn't we move the validation up. We already have a validateConsistencyLevel
in ThriftValidation for instance, we could just add a flag to if it's a write
or read operation and throw the right exception there. If only for consistency,
but I think it's nice to keep validating the queries upfront so we don't have
to later anyway.
Which btw make me see that this validateConsistencyLevel is not called by CQL
(which thus don't do the right check this function already does I suppose, or
duplicate it). It would be worth checking if that's the only place where CQL is
more loose than thrift in validating the query (and maybe we could refactor a
bit the validation code so it's easier to apply it to both thrift and CQL).
> READ Operation with CL=EACH_QUORUM succeed when a DC is down (RF=3)
> ---
>
> Key: CASSANDRA-3272
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3272
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 0.7.0
>Reporter: Jackson Chung
>Assignee: Jonathan Ellis
>Priority: Minor
> Fix For: 1.1
>
> Attachments: 3272.txt
>
>
> "READ EACH_QUORUM:Returns the record with the most recent timestamp once
> a quorum of replicas in each data center of the cluster has responded."
> In other words, if a DC is down and the QUORUM could not be reached on that
> DC, read should fail.
> test case:
> - Cassandra version 0.8.6:
> INFO [main] 2011-09-28 22:26:24,297 StorageService.java (line 371) Cassandra
> version: 0.8.6
> - 6-node cluster with 2 DC and 3 node each. RF=3 in each DC:
> [default@Keyspace3] describe keyspace;
> Keyspace: Keyspace3:
> Replication Strategy: org.apache.cassandra.locator.NetworkTopologyStrategy
> Durable Writes: true
> Options: [DC2:3, DC1:3]
> Column Families:
> ColumnFamily: test
> Key Validation Class: org.apache.cassandra.db.marshal.BytesType
> Default column value validator: org.apache.cassandra.db.marshal.BytesType
> Columns sorted by: org.apache.cassandra.db.marshal.BytesType
> Row cache size / save period in seconds: 0.0/0
> Key cache size / save period in seconds: 20.0/14400
> Memtable thresholds: 1.0875/1440/232 (millions of ops/minutes/MB)
> GC grace seconds: 864000
> Compaction min/max thresholds: 4/32
> Read repair chance: 1.0
> Replicate on write: true
> Built indexes: []
> all nodes are up, insert a row:
> $ nodetool -h localhost ring
> Address DC Rack Status State Load Owns Token
> 141784319550391026443072753096570088106
> 10.34.79.179 DC1 RAC1 Up Normal 11.13 KB 16.67% 0
> 10.34.70.163 DC2 RAC1 Up Normal 11.14 KB 16.67%
> 28356863910078205288614550619314017621
> 10.35.81.147 DC1 RAC1 Up Normal 11.14 KB 16.67%
> 56713727820156410577229101238628035242
> 10.84.233.170 DC2 RAC1 Up Normal 11.14 KB 16.67%
> 85070591730234615865843651857942052864
> 10.195.201.236 DC1 RAC1 Up Normal 11.14 KB 16.67%
> 113427455640312821154458202477256070485
> 10.118.147.73 DC2 RAC1 Up Normal 11.14 KB 16.67%
> 141784319550391026443072753096570088106
> - insert a value
> [default@Keyspace3] set
> test[utf8('test-key-1')][utf8('test-col')]=utf8('test-value');
> Value inserted.
> sanity check (cli connects to a node in DC1) :
> [default@Keyspace3] consistencylevel as EACH_QUORUM;
>
> Consistency level is set to 'EACH_QUORUM'.
> [default@Keyspace3] get test[utf8('test-key-1')];
> => (column=746573742d636f6c, value=test-value, timestamp=1317249361722000)
> Returned 1 results
> shut down DC2:
> $ nodetool -h localhost ring
> Address DC RackStatus State LoadOwns
> Token
>
> 141784319550391026443072753096570088106
> 10.34.79.179DC1 RAC1Up Normal 51.86 KB16.67%
> 0
> 10.34.70.163DC2 RAC1Down Normal 51.88 KB16.67%
> 28356863910078205288614550619314017621
> 10.35.81.147DC1 RAC1Up Normal 47.5 KB 16.67%
> 56713727820156410577229101238628035242
> 10.84.233.170 DC2 RAC1Down Normal 51.88 KB16.67%
> 85070591730234615865843651857942052864
> 10.195.201.236 DC1 RAC1Up Normal 47.5 KB 16.67%
> 113427455640312821154458202477256070485
> 10.118.147.73 DC2 RAC1Down Normal 51.88 KB16.67%
> 141784319550391026443072753096570088106
> [default@Keyspace3] get test[utf8('te
[jira] [Commented] (CASSANDRA-3127) Message (inter-node) compression
[ https://issues.apache.org/jira/browse/CASSANDRA-3127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127341#comment-13127341 ] Sylvain Lebresne commented on CASSANDRA-3127: - bq. Let's go with this: intranode_message_compression_threshold > 0 means compress messages larger than it. <= 0 means off Agreed with the idea. Though to nitpick, I would find it more natural to have == 0 means 'compress all messages'. And maybe < 0 means off. bq. Let's leave off the cross-dc complexity for now. Totally agree. > Message (inter-node) compression > > > Key: CASSANDRA-3127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3127 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Sylvain Lebresne >Priority: Minor > > CASSANDRA-3015 adds compression of streams. But it could be useful to also > compress some messages. > Compressing messages is easy, but what may be little bit trickier is when and > what messages to compress to get the best performances. > The simple solution would be to just have it either always on or always off. > But for very small messages (gossip?) that may be counter-productive. On the > other side of the spectrum, this is likely always a good choice to compress > for say the exchange of merkle trees across data-centers. We could maybe > define a size of messages after which we start to compress. Maybe the option > to only compress for cross data-center messages would be useful too (but I > may also just be getting carried away). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3331) Apache Daemon missing from the binary tarball
[ https://issues.apache.org/jira/browse/CASSANDRA-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127335#comment-13127335 ] Sylvain Lebresne commented on CASSANDRA-3331: - I'm also fine with reverting this, but I'd add a few remarks: # if we do revert, it would be nice to "fix" the .bat file so that it don't look for it in our specific repository that we would not ship (i.e, do a little bit more than reverting because the .bat before this patch was clearly suggesting we intended to ship it but did not). Ideally we would just expect it to be installed in whatever is supposed to be standard for windows and offer an helpful message if it is not found (since we can't do what we do for the debian package, i.e make it a dependency of the package) # it's been committed to 1.0.0 and I don't really want to re-roll the vote for that. That would mean we ship procrun for 1.0.0 but then never after that. A little weird imho. > Apache Daemon missing from the binary tarball > - > > Key: CASSANDRA-3331 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3331 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Benjamin Coverston >Assignee: Benjamin Coverston > Fix For: 1.0.0 > > Attachments: 0001-add-files-to-src-and-bin-add-uninstall.patch, > 0002-adding-daemon-directory.patch > > > Apparently the tools used to run the binary release are missing from the > binary tarball. > I will verify that they are in the 1.0 branch, then update the ticket so we > can ensure that they are included. > Ben -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2749) fine-grained control over data directories
[ https://issues.apache.org/jira/browse/CASSANDRA-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127329#comment-13127329 ] Marcus Eriksson commented on CASSANDRA-2749: Biggest part left is figuring out how to estimate disk space to know where to write an sstable, ill work on that this weekend Other than that small cleanups (codestyle), unit tests and adding the config parameter > fine-grained control over data directories > -- > > Key: CASSANDRA-2749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2749 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Priority: Minor > Fix For: 1.1 > > Attachments: > 0001-Make-it-possible-to-put-column-families-in-subdirect.patch, > 2749_backwards_compatible_v1.patch, 2749_backwards_compatible_v2.patch > > > Currently Cassandra supports multiple data directories but no way to control > what sstables are placed where. Particularly for systems with mixed SSDs and > rotational disks, it would be nice to pin frequently accessed columnfamilies > to the SSDs. > Postgresql does this with tablespaces > (http://www.postgresql.org/docs/9.0/static/manage-ag-tablespaces.html) but we > should probably avoid using that name because of confusing similarity to > "keyspaces." -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
