svn commit: r1077830 - in /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra: io/sstable/SSTableReader.java utils/FBUtilities.java
Author: jbellis
Date: Fri Mar 4 06:45:10 2011
New Revision: 1077830
URL: http://svn.apache.org/viewvc?rev=1077830&view=rev
Log:
r/m unnecessary BBU.clone calls
patch by jbellis; reviewed by tjake for CASSANDRA-2271
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/FBUtilities.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableReader.java?rev=1077830&r1=1077829&r2=1077830&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
Fri Mar 4 06:45:10 2011
@@ -301,7 +301,7 @@ public class SSTableReader extends SSTab
long dataPosition = input.readLong();
if (key != null)
{
-DecoratedKey decoratedKey = decodeKey(partitioner,
descriptor, ByteBufferUtil.clone(key));
+DecoratedKey decoratedKey = decodeKey(partitioner,
descriptor, key);
if (recreatebloom)
bf.add(decoratedKey.key);
if (shouldAddEntry)
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/FBUtilities.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/FBUtilities.java?rev=1077830&r1=1077829&r2=1077830&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/FBUtilities.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/FBUtilities.java
Fri Mar 4 06:45:10 2011
@@ -235,7 +235,7 @@ public class FBUtilities
MessageDigest messageDigest = localMD5Digest.get();
for(ByteBuffer block : data)
{
-messageDigest.update(ByteBufferUtil.clone(block));
+messageDigest.update(block.duplicate());
}
return messageDigest.digest();
[jira] Commented: (CASSANDRA-2027) term definitions
[
https://issues.apache.org/jira/browse/CASSANDRA-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002472#comment-13002472
]
Jonathan Ellis commented on CASSANDRA-2027:
---
bq. That seems corner-case enough for me to warrant leaving it out entirely
No strong feelings here either way.
bq. I'm also continuing to have a hard time accepting that different rules
should exist (syntax and semantics) for column names and values
I feel strongly about this b/c forcing quoting of "normal" column names is
going to needlessly give people the impression that "this is a low-quality
implementation."
> term definitions
>
>
> Key: CASSANDRA-2027
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2027
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Eric Evans
>Priority: Minor
> Labels: cql
> Fix For: 0.8
>
> Attachments: v1-0001-CASSANDRA-2027-utf8-and-integer-term-types.txt,
> v1-0002-column-name-validation.txt,
> v1-0003-system-tests-for-integer-and-utf8-term-types.txt,
> v1-0004-uuid-term-definitions.txt,
> v1-0005-missed-doc-update-for-utf8-term-type.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> h3. String
> Anything between double-quotes. Node-side this is just converted to bytes,
> so it could really be used to represent *any* type so long as it is
> appropriately encoded.
> Examples:
> {code:style=SQL}
> SELECT "name" FROM cf;
> UPDATE cf SET "name" = "value" WHERE KEY = "key";
> {code}
> h3. UTF-8
> A double-quoted string literal that is prefixed with a "u" to indicated that
> it should be encoded to bytes using the utf-8 charset node-side.
> Examples:
> {code:style=SQL}
> SELECT u"name" FROM cf;
> UPDATE cf SET u"name" = u"value" WHERE KEY = "key";
> {code}
> h3. Integer
> An undecorated numeric literal, converted to a 4-byte int node-side.
> Examples:
> {code:style=SQL}
> SELECT 10..100 FROM cf WHERE KEY = "key";
> UPDATE cf SET 1000 = "thousand", 100 = "hundred" WHERE KEY = "key";
> {code}
> h3. Long
> A numeric literal suffixed with an "L", converted to an 8-byte long node-side.
> Examples:
> {code:style=SQL}
> SELECT 10L..100L FROM cf WHERE KEY = "key";
> UPDATE cf SET 1000L = "thousand", 100L = "hundred" WHERE KEY = "key";
> {code}
> h3. UUID
> A string-formatted UUID supplied as an "argument" to a ctor/function formated
> string ({{uuid()}}). Node-side this is converted back to the
> corresponding UUID.
> Examples:
> {code:style=SQL}
> SELECT uuid(5f989e95-ae07-4425-b84a-6876ba106c66) FROM cf WHERE KEY = "key";
> UPDATE cf SET uuid(5621b93d-d3a2-4d22-8a59-bdb93202b6cb) = "username" WHERE
> KEY = "key";
> {code}
> h3. TimeUUID (UUID Type 1)
> A string-formatted time-based UUID (type 1) supplied as an "argument" to a
> ctor/function formated string ({{timeuuid()}}). Node-side this
> is converted back to the corresponding UUID. In addition to a
> string-formatted UUID, it should also be possible to supply dates in a
> variety of formats which will result in a new UUID being created node-side.
> Examples:
> {code:style=SQL}
> SELECT timeuuid(2011-01-01)..timeuuid(2010-01-21) FROM cf WHERE KEY = "key";
> UPDATE cf SET timeuuid(now) = 1000L WHERE KEY = "key";
> {code}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2027) term definitions
[
https://issues.apache.org/jira/browse/CASSANDRA-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002468#comment-13002468
]
Jonathan Ellis commented on CASSANDRA-2027:
---
Why are we using double quotes instead of standard single?
> term definitions
>
>
> Key: CASSANDRA-2027
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2027
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Eric Evans
>Priority: Minor
> Labels: cql
> Fix For: 0.8
>
> Attachments: v1-0001-CASSANDRA-2027-utf8-and-integer-term-types.txt,
> v1-0002-column-name-validation.txt,
> v1-0003-system-tests-for-integer-and-utf8-term-types.txt,
> v1-0004-uuid-term-definitions.txt,
> v1-0005-missed-doc-update-for-utf8-term-type.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> h3. String
> Anything between double-quotes. Node-side this is just converted to bytes,
> so it could really be used to represent *any* type so long as it is
> appropriately encoded.
> Examples:
> {code:style=SQL}
> SELECT "name" FROM cf;
> UPDATE cf SET "name" = "value" WHERE KEY = "key";
> {code}
> h3. UTF-8
> A double-quoted string literal that is prefixed with a "u" to indicated that
> it should be encoded to bytes using the utf-8 charset node-side.
> Examples:
> {code:style=SQL}
> SELECT u"name" FROM cf;
> UPDATE cf SET u"name" = u"value" WHERE KEY = "key";
> {code}
> h3. Integer
> An undecorated numeric literal, converted to a 4-byte int node-side.
> Examples:
> {code:style=SQL}
> SELECT 10..100 FROM cf WHERE KEY = "key";
> UPDATE cf SET 1000 = "thousand", 100 = "hundred" WHERE KEY = "key";
> {code}
> h3. Long
> A numeric literal suffixed with an "L", converted to an 8-byte long node-side.
> Examples:
> {code:style=SQL}
> SELECT 10L..100L FROM cf WHERE KEY = "key";
> UPDATE cf SET 1000L = "thousand", 100L = "hundred" WHERE KEY = "key";
> {code}
> h3. UUID
> A string-formatted UUID supplied as an "argument" to a ctor/function formated
> string ({{uuid()}}). Node-side this is converted back to the
> corresponding UUID.
> Examples:
> {code:style=SQL}
> SELECT uuid(5f989e95-ae07-4425-b84a-6876ba106c66) FROM cf WHERE KEY = "key";
> UPDATE cf SET uuid(5621b93d-d3a2-4d22-8a59-bdb93202b6cb) = "username" WHERE
> KEY = "key";
> {code}
> h3. TimeUUID (UUID Type 1)
> A string-formatted time-based UUID (type 1) supplied as an "argument" to a
> ctor/function formated string ({{timeuuid()}}). Node-side this
> is converted back to the corresponding UUID. In addition to a
> string-formatted UUID, it should also be possible to supply dates in a
> variety of formats which will result in a new UUID being created node-side.
> Examples:
> {code:style=SQL}
> SELECT timeuuid(2011-01-01)..timeuuid(2010-01-21) FROM cf WHERE KEY = "key";
> UPDATE cf SET timeuuid(now) = 1000L WHERE KEY = "key";
> {code}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2271) Audit uses of BBU.clone
[ https://issues.apache.org/jira/browse/CASSANDRA-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002448#comment-13002448 ] T Jake Luciani commented on CASSANDRA-2271: --- +1 > Audit uses of BBU.clone > --- > > Key: CASSANDRA-2271 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2271 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Fix For: 0.7.4 > > Attachments: 2271.txt > > -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[ https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002445#comment-13002445 ] Stu Hood commented on CASSANDRA-2231: - > Yes, IntegerType would be a better choice because it uses BigInteger that > packs > stuffs as much as it can (aka, 1 byte if it fits). Speaking of which: should we use variable length integers for the composite field lengths here? 99% of the time the length will fit in a single byte. > Add CompositeType comparer to the comparers provided in > org.apache.cassandra.db.marshal > --- > > Key: CASSANDRA-2231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2231 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Affects Versions: 0.7.3 >Reporter: Ed Anuff >Priority: Minor > Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, > 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip > > > CompositeType is a custom comparer that makes it possible to create > comparable composite values out of the basic types that Cassandra currently > supports, such as Long, UUID, etc. This is very useful in both the creation > of custom inverted indexes using columns in a skinny row, where each column > name is a composite value, and also when using Cassandra's built-in secondary > index support, where it can be used to encode the values in the columns that > Cassandra indexes. One scenario for the usage of these is documented here: > http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for > contribution is attached and has been previously maintained on github here: > https://github.com/edanuff/CassandraCompositeType -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2269) OOM in the Thrift thread doesn't shut down server
[
https://issues.apache.org/jira/browse/CASSANDRA-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002443#comment-13002443
]
Jonathan Ellis commented on CASSANDRA-2269:
---
Adding ByteBuffer.allocateDirect(256 * 1024 * 1024) to any of the
CassandraServer methods should do it
> OOM in the Thrift thread doesn't shut down server
> -
>
> Key: CASSANDRA-2269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2269
> Project: Cassandra
> Issue Type: Bug
> Components: API
>Affects Versions: 0.6
>Reporter: Jonathan Ellis
>Assignee: Jonathan Ellis
>Priority: Minor
> Fix For: 0.6.13, 0.7.4
>
> Attachments: 2269-0.6.txt
>
>
> Example:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.resize(NonBlockingHashMap.java:849)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.access$200(NonBlockingHashMap.java:699)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:634)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:339)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.put(NonBlockingHashMap.java:302)
> at org.apache.cassandra.utils.ExpiringMap.put(ExpiringMap.java:112)
> at
> org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:237)
> at
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:305)
> at
> org.apache.cassandra.service.StorageProxy.weakRead(StorageProxy.java:386)
> at
> org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:347)
> at
> org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:92)
> at
> org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:175)
> at
> org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:254)
> at
> org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:215)
> at
> org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:1272)
> at
> org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:1166)
> at
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {noformat}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2271) Audit uses of BBU.clone
[ https://issues.apache.org/jira/browse/CASSANDRA-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2271: -- Attachment: 2271.txt Fixes a couple places where we appear to be cloning unnecessarily. > Audit uses of BBU.clone > --- > > Key: CASSANDRA-2271 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2271 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis >Priority: Minor > Fix For: 0.7.4 > > Attachments: 2271.txt > > -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2269) OOM in the Thrift thread doesn't shut down server
[
https://issues.apache.org/jira/browse/CASSANDRA-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002441#comment-13002441
]
T Jake Luciani commented on CASSANDRA-2269:
---
Makes sense... Is there any known way to reproduce it to verify the fix infact
works?
> OOM in the Thrift thread doesn't shut down server
> -
>
> Key: CASSANDRA-2269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2269
> Project: Cassandra
> Issue Type: Bug
> Components: API
>Affects Versions: 0.6
>Reporter: Jonathan Ellis
>Assignee: Jonathan Ellis
>Priority: Minor
> Fix For: 0.6.13, 0.7.4
>
> Attachments: 2269-0.6.txt
>
>
> Example:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.resize(NonBlockingHashMap.java:849)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.access$200(NonBlockingHashMap.java:699)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:634)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:339)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.put(NonBlockingHashMap.java:302)
> at org.apache.cassandra.utils.ExpiringMap.put(ExpiringMap.java:112)
> at
> org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:237)
> at
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:305)
> at
> org.apache.cassandra.service.StorageProxy.weakRead(StorageProxy.java:386)
> at
> org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:347)
> at
> org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:92)
> at
> org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:175)
> at
> org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:254)
> at
> org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:215)
> at
> org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:1272)
> at
> org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:1166)
> at
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {noformat}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (CASSANDRA-2271) Audit uses of BBU.clone
Audit uses of BBU.clone --- Key: CASSANDRA-2271 URL: https://issues.apache.org/jira/browse/CASSANDRA-2271 Project: Cassandra Issue Type: Improvement Reporter: Jonathan Ellis Assignee: Jonathan Ellis Priority: Minor Fix For: 0.7.4 Attachments: 2271.txt -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1077486 - /cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
Author: jbellis
Date: Fri Mar 4 04:19:58 2011
New Revision: 1077486
URL: http://svn.apache.org/viewvc?rev=1077486&view=rev
Log:
javadoc for BBU.clone
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java?rev=1077486&r1=1077485&r2=1077486&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/utils/ByteBufferUtil.java
Fri Mar 4 04:19:58 2011
@@ -188,23 +188,28 @@ public class ByteBufferUtil
throw new RuntimeException(e);
}
}
-
-public static ByteBuffer clone(ByteBuffer o)
+
+/**
+ * @return a new copy of the data in @param buffer
+ * USUALLY YOU SHOULD USE ByteBuffer.duplicate() INSTEAD, which creates a
new Buffer
+ * (so you can mutate its position without affecting the original) without
copying the underlying array.
+ */
+public static ByteBuffer clone(ByteBuffer buffer)
{
-assert o != null;
+assert buffer != null;
-if (o.remaining() == 0)
+if (buffer.remaining() == 0)
return EMPTY_BYTE_BUFFER;
-ByteBuffer clone = ByteBuffer.allocate(o.remaining());
+ByteBuffer clone = ByteBuffer.allocate(buffer.remaining());
-if (o.hasArray())
+if (buffer.hasArray())
{
-System.arraycopy(o.array(), o.arrayOffset() + o.position(),
clone.array(), 0, o.remaining());
+System.arraycopy(buffer.array(), buffer.arrayOffset() +
buffer.position(), clone.array(), 0, buffer.remaining());
}
else
{
-clone.put(o.duplicate());
+clone.put(buffer.duplicate());
clone.flip();
}
[jira] Commented: (CASSANDRA-2252) off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002440#comment-13002440 ] Jonathan Ellis commented on CASSANDRA-2252: --- sounds reasonable > off-heap memtables > -- > > Key: CASSANDRA-2252 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2252 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis > Fix For: 0.8 > > Attachments: 0001-add-MemtableAllocator.txt, > 0002-add-off-heap-MemtableAllocator-support.txt, 2252-alternate-v2.tgz > > > The memtable design practically actively fights Java's GC design. Todd > Lipcon gave a good explanation over on HBASE-3455. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (CASSANDRA-2252) off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002414#comment-13002414 ] Stu Hood edited comment on CASSANDRA-2252 at 3/4/11 3:18 AM: - I'd like to go ahead and merge these patches together onto jbellis's version. Things that I will cherry pick from the alternate patch: * A generic Allocator interface, rather than referring to Memtables. There are various other places that we need to apply slabbing * The additional counter tests we added * Using the slab allocator's allocation count to determine throughput for memtables * Slabbing of keys The other realization we've had about slab allocation is that unless _all_ sources of fragmentation are eliminated, slabbing actually causes promotion failures to happen earlier, since it is harder to promote a slab into a fragmented oldgen. The other sources of fragmentation we suspect are: * IndexSummaries (easily slabbed) * the key and row caches (row cache tackled in CASSANDRA-1969) These can probably be defragged in separate tickets, as long as we commit to fixing them. was (Author: stuhood): I'd like to go ahead and merge these patches together onto jbellis's version. Things that I will cherry pick from the alternate patch: * A generic Allocator interface, rather than referring to Memtables. There are various other places that we need to apply slabbing * The additional counter tests we added * Using the slab allocator's allocation count to determine throughput for memtables The other realization we've had about slab allocation is that unless _all_ sources of fragmentation are eliminated, slabbing actually causes promotion failures to happen earlier, since it is harder to promote a slab into a fragmented oldgen. The other sources of fragmentation we suspect are: * IndexSummaries (easily slabbed) * the key and row caches (row cache tackled in CASSANDRA-1969) These can probably be defragged in separate tickets, as long as we commit to fixing them. > off-heap memtables > -- > > Key: CASSANDRA-2252 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2252 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis > Fix For: 0.8 > > Attachments: 0001-add-MemtableAllocator.txt, > 0002-add-off-heap-MemtableAllocator-support.txt, 2252-alternate-v2.tgz > > > The memtable design practically actively fights Java's GC design. Todd > Lipcon gave a good explanation over on HBASE-3455. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2252) off-heap memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002414#comment-13002414 ] Stu Hood commented on CASSANDRA-2252: - I'd like to go ahead and merge these patches together onto jbellis's version. Things that I will cherry pick from the alternate patch: * A generic Allocator interface, rather than referring to Memtables. There are various other places that we need to apply slabbing * The additional counter tests we added * Using the slab allocator's allocation count to determine throughput for memtables The other realization we've had about slab allocation is that unless _all_ sources of fragmentation are eliminated, slabbing actually causes promotion failures to happen earlier, since it is harder to promote a slab into a fragmented oldgen. The other sources of fragmentation we suspect are: * IndexSummaries (easily slabbed) * the key and row caches (row cache tackled in CASSANDRA-1969) These can probably be defragged in separate tickets, as long as we commit to fixing them. > off-heap memtables > -- > > Key: CASSANDRA-2252 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2252 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jonathan Ellis > Fix For: 0.8 > > Attachments: 0001-add-MemtableAllocator.txt, > 0002-add-off-heap-MemtableAllocator-support.txt, 2252-alternate-v2.tgz > > > The memtable design practically actively fights Java's GC design. Todd > Lipcon gave a good explanation over on HBASE-3455. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2027) term definitions
[
https://issues.apache.org/jira/browse/CASSANDRA-2027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002396#comment-13002396
]
Eric Evans commented on CASSANDRA-2027:
---
The following reboot is the result of a discussion between Gary Dusbabek,
Jonathan Ellis, and myself (any errors or misunderstandings are my fault).
h2. Revised definitions (+ semantics)
h3. String
Anything between quotes, _or_ any unquoted alnum values that begins with a
letter.
Examples:
{code:style=SQL}
SELECT "0day" FROM cf;
SELECT B_day FROM cf;
UPDATE cf SET "value-low" = "14%" WHERE KEY = "@skinny";
UPDATE cf SET foo = bar WHERE KEY = baz;
{code}
h3. Integer
An undecorated numeric literal. How the term is converted node-side is
determined by the comparator/validator in use. For example, {{100}} could be
converted to a 4-byte integer or an 8-byte long depending on whether the
comparator/validator was an {{IntegerType}} or {{LongType}} respectively.
Examples:
{code:style=SQL}
SELECT 10..100 FROM cf WHERE KEY = "key";
UPDATE cf SET 1000 = "thousand", 100 = "hundred" WHERE KEY = "key";
{code}
h3. UUID
A UUID formated as a hexidecimal-hyphenated string (i.e.
{{b137dd10-45b6-11e0-8955-00247ee1f924}}).
Examples:
{code:style=SQL}
SELECT f1fa6c22-45b7-11e0-8955-00247ee1f924 FROM cf WHERE KEY = key;
UPDATE cf SET 0ceb632e-45b8-11e0-8955-00247ee1f924 = 9 WHERE KEY = key;
{code}
As a special-case, when the comparator/validator is TimeUUIDType, a quoted
string literal can be used to supply a parse-able timestamp (currently most
ISO8601 variants).
{code:style=SQL}
SELECT "2011-01-01".."2011-02-01" FROM cf WHERE KEY = key;
{code}
_Note: it doesn't make sense to try to query by-column using a timestamp like
this, because date-time is only one component of a type 1 UUID. The docs will
need to be clear about this._
h3. UTF-8
A double-quoted string literal that is prefixed with a "u" to indicate that it
should be encoded to bytes using the utf-8 charset node-side.
Examples:
{code:style=SQL}
SELECT u"name" FROM cf;
UPDATE cf SET u"name" = u"value" WHERE KEY = "key";
{code}
_This one is iffy. Consensus seems to be that the UTF8 charset should
implicitly be used in the conversion to bytes when comparator/validator is
UTF8Type. If that's the case, then the only time where this term would do
anything useful would be for storing UTF8 where comparator/validator is
BytesType. That seems corner-case enough for me to warrant leaving it out
entirely._
One point of contention during the discussion that spawned this reboot was type
inference. What's proposed above adds some inference, (namely for unicode,
decimal values, and some UUID cases), but I'm going to make one more attempt at
stopping it there. I'm nothing if not persistent, right? :)
For example, Least Surprise says that {{"10"}} and {{10}} differ in that one is
explicitly a string, so converting it to a numeric type with a decimal value of
10 (still) seems wrong to me. I'd prefer to raise an exception for such
mismatches, which also seems like a good way of protecting users from a whole
class of bugs.
I'm also continuing to have a hard time accepting that different rules should
exist (syntax and semantics) for column names and values. The general argument
for SQL parity is a strong one, and I'm trying to be convinced on this issue,
(honest), but I keep coming back to the notion that SQL column names are not
typed, and that forcing that distinction on Cassandra seems contrived.
> term definitions
>
>
> Key: CASSANDRA-2027
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2027
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Eric Evans
>Priority: Minor
> Labels: cql
> Fix For: 0.8
>
> Attachments: v1-0001-CASSANDRA-2027-utf8-and-integer-term-types.txt,
> v1-0002-column-name-validation.txt,
> v1-0003-system-tests-for-integer-and-utf8-term-types.txt,
> v1-0004-uuid-term-definitions.txt,
> v1-0005-missed-doc-update-for-utf8-term-type.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> h3. String
> Anything between double-quotes. Node-side this is just converted to bytes,
> so it could really be used to represent *any* type so long as it is
> appropriately encoded.
> Examples:
> {code:style=SQL}
> SELECT "name" FROM cf;
> UPDATE cf SET "name" = "value" WHERE KEY = "key";
> {code}
> h3. UTF-8
> A double-quoted string literal that is prefixed with a "u" to indicated that
> it should be encoded to bytes using the utf-8 charset node-side.
> Examples:
> {code:style=SQL}
> SELECT u"name" FROM cf;
> UPDATE cf SET u"name" = u"value" WHERE KEY = "key";
> {code}
> h3. Integer
> An undecorated numeric literal, converted to a 4-by
[jira] Created: (CASSANDRA-2270) nodetool info NPE when node isn't fully booted
nodetool info NPE when node isn't fully booted -- Key: CASSANDRA-2270 URL: https://issues.apache.org/jira/browse/CASSANDRA-2270 Project: Cassandra Issue Type: Bug Reporter: Sébastien Giroux Priority: Minor Fix For: 0.7.4 Running "nodetool -h 127.0.0.1 info" when the node is not yet ready throw a NPE. Exception in thread "main" java.lang.NullPointerException at org.apache.cassandra.gms.Gossiper.getCurrentGenerationNumber(Gossiper.java:313) at org.apache.cassandra.service.StorageService.getCurrentGenerationNumber(StorageService.java:1239) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:111) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:45) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:226) at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83) at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:205) -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1828) Create a pig storefunc
[ https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002382#comment-13002382 ] Hudson commented on CASSANDRA-1828: --- Integrated in Cassandra-0.7 #345 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/345/]) Pig storefunc. Patch by brandonwilliams, reviewed by Jeremy Hanna for CASSANDRA-1828. > Create a pig storefunc > -- > > Key: CASSANDRA-1828 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1828 > Project: Cassandra > Issue Type: New Feature > Components: Contrib, Hadoop >Affects Versions: 0.7 beta 1 >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 0.7.4 > > Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, > 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Now that we have a ColumnFamilyOutputFormat, we can write data back to > cassandra in mapreduce jobs, however we can only do this in java. It would > be nice if pig could also output to cassandra. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (CASSANDRA-1828) Create a pig storefunc
[ https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams resolved CASSANDRA-1828. - Resolution: Fixed Committed with the second point change (also in the input format) and updated README > Create a pig storefunc > -- > > Key: CASSANDRA-1828 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1828 > Project: Cassandra > Issue Type: New Feature > Components: Contrib, Hadoop >Affects Versions: 0.7 beta 1 >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 0.7.4 > > Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, > 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Now that we have a ColumnFamilyOutputFormat, we can write data back to > cassandra in mapreduce jobs, however we can only do this in java. It would > be nice if pig could also output to cassandra. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1076905 - in /cassandra/branches/cassandra-0.7: CHANGES.txt contrib/pig/README.txt contrib/pig/bin/pig_cassandra contrib/pig/build.xml contrib/pig/src/java/org/apache/cassandra/hadoop/pig
Author: brandonwilliams Date: Fri Mar 4 00:41:55 2011 New Revision: 1076905 URL: http://svn.apache.org/viewvc?rev=1076905&view=rev Log: Pig storefunc. Patch by brandonwilliams, reviewed by Jeremy Hanna for CASSANDRA-1828. Modified: cassandra/branches/cassandra-0.7/CHANGES.txt cassandra/branches/cassandra-0.7/contrib/pig/README.txt cassandra/branches/cassandra-0.7/contrib/pig/bin/pig_cassandra cassandra/branches/cassandra-0.7/contrib/pig/build.xml cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java Modified: cassandra/branches/cassandra-0.7/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1076905&r1=1076904&r2=1076905&view=diff == --- cassandra/branches/cassandra-0.7/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.7/CHANGES.txt Fri Mar 4 00:41:55 2011 @@ -2,6 +2,7 @@ * add nodetool join command (CASSANDRA-2160) * fix secondary indexes on pre-existing or streamed data (CASSANDRA-2244) * initialize endpoing in gossiper earlier (CASSANDRA-2228) + * add ability to write to Cassandra from Pig (CASSANDRA-1828) 0.7.3 * Keep endpoint state until aVeryLongTime (CASSANDRA-2115) Modified: cassandra/branches/cassandra-0.7/contrib/pig/README.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/pig/README.txt?rev=1076905&r1=1076904&r2=1076905&view=diff == --- cassandra/branches/cassandra-0.7/contrib/pig/README.txt (original) +++ cassandra/branches/cassandra-0.7/contrib/pig/README.txt Fri Mar 4 00:41:55 2011 @@ -1,4 +1,5 @@ -A Pig LoadFunc that reads all columns from a given ColumnFamily. +A Pig storage class that reads all columns from a given ColumnFamily, or writes +properly formatted results into a ColumnFamily. Setup: @@ -7,10 +8,15 @@ configuration and set the PIG_HOME and J variables to the location of a Pig >= 0.7.0 install and your Java install. +NOTE: if you intend to _output_ to Cassandra, until there is a Pig release that +uses jackson > 1.0.1 (see https://issues.apache.org/jira/browse/PIG-1863) you +will need to build Pig yourself with jackson 1.4. To do this, edit Pig's +ivy/libraries.properties, and run ant. + If you would like to run using the Hadoop backend, you should also set PIG_CONF_DIR to the location of your Hadoop config. -FInally, set the following as environment variables (uppercase, +Finally, set the following as environment variables (uppercase, underscored), or as Hadoop configuration variables (lowercase, dotted): * PIG_RPC_PORT or cassandra.thrift.port : the port thrift is listening on * PIG_INITIAL_ADDRESS or cassandra.thrift.address : initial address to connect to @@ -40,3 +46,11 @@ grunt> namecounts = FOREACH namegroups G grunt> orderednames = ORDER namecounts BY $0; grunt> topnames = LIMIT orderednames 50; grunt> dump topnames; + +Outputting to Cassandra requires the same format from input, so the simplest example is: + +grunt> rows = LOAD 'cassandra://Keyspace1/Standard1' USING CassandraStorage(); +grunt> STORE rows into 'cassandra://Keyspace1/Standard2' USING CassandraStorage(); + +Which will copy the ColumnFamily. Note that the destination ColumnFamily must +already exist for this to work. Modified: cassandra/branches/cassandra-0.7/contrib/pig/bin/pig_cassandra URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/pig/bin/pig_cassandra?rev=1076905&r1=1076904&r2=1076905&view=diff == --- cassandra/branches/cassandra-0.7/contrib/pig/bin/pig_cassandra (original) +++ cassandra/branches/cassandra-0.7/contrib/pig/bin/pig_cassandra Fri Mar 4 00:41:55 2011 @@ -38,7 +38,7 @@ if [ "x$PIG_HOME" = "x" ]; then fi # pig jar. -PIG_JAR=$PIG_HOME/pig*core.jar +PIG_JAR=$PIG_HOME/pig*.jar if [ ! -e $PIG_JAR ]; then echo "Unable to locate Pig jar" >&2 exit 1 Modified: cassandra/branches/cassandra-0.7/contrib/pig/build.xml URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/pig/build.xml?rev=1076905&r1=1076904&r2=1076905&view=diff == --- cassandra/branches/cassandra-0.7/contrib/pig/build.xml (original) +++ cassandra/branches/cassandra-0.7/contrib/pig/build.xml Fri Mar 4 00:41:55 2011 @@ -21,7 +21,7 @@ - + @@ -31,8 +31,9 @@ - + + Modified: cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/contrib/pig/src/java/org/apache/cassandra/hadoop/pig/CassandraStorage.java?rev=1076905&r1=107
[jira] Commented: (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.
[ https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002365#comment-13002365 ] Vijay commented on CASSANDRA-1969: -- Again coming back to the original POC code... where we will reuse the BB's which are not GC'ed (using weak references) shouldn't that approach work? (if we are not going to use JNA) i think that way we can guarantee to some extent that we dont fragment the Direct Memory... Assuming most of our queries results will be approx of the same size (where chunk size can be defined by the user). > Use BB for row cache - To Improve GC performance. > - > > Key: CASSANDRA-1969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1969 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Linux and Mac >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Attachments: 0001-Config-1969.txt, > 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, > 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, > 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, > 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, > 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, BB_Cache-1945.png, > JMX-Cache-1945.png, Old_Cahce-1945.png, POC-0001-Config-1945.txt, > POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt > > > Java BB.allocateDirect() will allocate native memory out of the JVM and will > help reducing the GC pressure in the JVM with a large Cache. > From some of the basic tests it shows around 50% improvement than doing a > normal Object cache. > In addition this patch provide the users an option to choose > BB.allocateDirect or store everything in the heap. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.
[ https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002350#comment-13002350 ] Jonathan Ellis commented on CASSANDRA-1969: --- Those are never promoted to old gen, though, so the JVM can run the cleanup in the frequent and cheap new-gen collections. Once you promote to old gen it gets much messier (because old gen collections can be days apart and take multiple seconds to finish). > Use BB for row cache - To Improve GC performance. > - > > Key: CASSANDRA-1969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1969 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Linux and Mac >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Attachments: 0001-Config-1969.txt, > 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, > 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, > 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, > 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, > 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, BB_Cache-1945.png, > JMX-Cache-1945.png, Old_Cahce-1945.png, POC-0001-Config-1945.txt, > POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt > > > Java BB.allocateDirect() will allocate native memory out of the JVM and will > help reducing the GC pressure in the JVM with a large Cache. > From some of the basic tests it shows around 50% improvement than doing a > normal Object cache. > In addition this patch provide the users an option to choose > BB.allocateDirect or store everything in the heap. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1076899 - in /cassandra/branches/cassandra-0.7: ./ interface/thrift/gen-java/org/apache/cassandra/thrift/
Author: jbellis Date: Fri Mar 4 00:16:57 2011 New Revision: 1076899 URL: http://svn.apache.org/viewvc?rev=1076899&view=rev Log: merge from 0.6 Modified: cassandra/branches/cassandra-0.7/ (props changed) cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) Propchange: cassandra/branches/cassandra-0.7/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Mar 4 00:16:57 2011 @@ -1,4 +1,4 @@ -/cassandra/branches/cassandra-0.6:922689-1071777 +/cassandra/branches/cassandra-0.6:922689-1071777,1076891 /cassandra/branches/cassandra-0.7:1026516,1035666,1050269 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 Propchange: cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Mar 4 00:16:57 2011 @@ -1,4 +1,4 @@ -/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1071777 +/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1071777,1076891 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516,1035666,1050269 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 Propchange: cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Mar 4 00:16:57 2011 @@ -1,4 +1,4 @@ -/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1071777 +/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1071777,1076891 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516,1035666,1050269 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689 Propchange: cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Mar 4 00:16:57 2011 @@ -1,4 +1,4 @@ -/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1071777 +/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:922689-1071777,1076891 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1026516,1035666,1050269 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java:1051699-1053689 Propchange: cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Mar 4 00:16:57 2011 @@ -1,4 +1,4 @@ -/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:922689-1071777 +/cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:922689-1071777,1076891 /cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1026516,1035666,1050269 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/th
[jira] Updated: (CASSANDRA-2269) OOM in the Thrift thread doesn't shut down server
[
https://issues.apache.org/jira/browse/CASSANDRA-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-2269:
--
Attachment: 2269-0.6.txt
patch to extract exception logging from DTPE and call from the Thrift executor.
(the actual shutdown is done by the default exception hook we set up -- it's
not normally called on executor threads because both Future and executor
afterExecute machinery swallow exceptions.)
> OOM in the Thrift thread doesn't shut down server
> -
>
> Key: CASSANDRA-2269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2269
> Project: Cassandra
> Issue Type: Bug
> Components: API
>Affects Versions: 0.6
>Reporter: Jonathan Ellis
>Assignee: Jonathan Ellis
>Priority: Minor
> Fix For: 0.6.13, 0.7.4
>
> Attachments: 2269-0.6.txt
>
>
> Example:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.resize(NonBlockingHashMap.java:849)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.access$200(NonBlockingHashMap.java:699)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:634)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:339)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.put(NonBlockingHashMap.java:302)
> at org.apache.cassandra.utils.ExpiringMap.put(ExpiringMap.java:112)
> at
> org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:237)
> at
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:305)
> at
> org.apache.cassandra.service.StorageProxy.weakRead(StorageProxy.java:386)
> at
> org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:347)
> at
> org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:92)
> at
> org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:175)
> at
> org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:254)
> at
> org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:215)
> at
> org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:1272)
> at
> org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:1166)
> at
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {noformat}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Trivial Update of "HadoopSupport" by jeremyhanna
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "HadoopSupport" page has been changed by jeremyhanna. http://wiki.apache.org/cassandra/HadoopSupport?action=diff&rev1=27&rev2=28 -- Hadoop Streaming As of 0.7, there is support for [[http://hadoop.apache.org/common/docs/r0.20.0/streaming.html|Hadoop Streaming]]. For examples on how to use Streaming with Cassandra, see the contrib section of the Cassandra source. The relevant tickets are [[https://issues.apache.org/jira/browse/CASSANDRA-1368|CASSANDRA-1368]] and [[https://issues.apache.org/jira/browse/CASSANDRA-1497|CASSANDRA-1497]]. + [[#Top|Top]] <>
[Cassandra Wiki] Update of "HadoopSupport" by jeremyhanna
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for
change notification.
The "HadoopSupport" page has been changed by jeremyhanna.
The comment on this change is: Adding some more troubleshooting info in a
separate section..
http://wiki.apache.org/cassandra/HadoopSupport?action=diff&rev1=26&rev2=27
--
* [[#Pig|Pig]]
* [[#Hive|Hive]]
* [[#ClusterConfig|Cluster Configuration]]
+ * [[#Troubleshooting|Troubleshooting]]
* [[#Support|Support]]
<>
@@ -37, +38 @@
Hadoop Streaming
As of 0.7, there is support for
[[http://hadoop.apache.org/common/docs/r0.20.0/streaming.html|Hadoop
Streaming]]. For examples on how to use Streaming with Cassandra, see the
contrib section of the Cassandra source. The relevant tickets are
[[https://issues.apache.org/jira/browse/CASSANDRA-1368|CASSANDRA-1368]] and
[[https://issues.apache.org/jira/browse/CASSANDRA-1497|CASSANDRA-1497]].
-
- Some troubleshooting
- Releases before 0.6.2/0.7 are affected by a small resource leak that may
cause jobs to fail (connections are not released properly, causing a resource
leak). Depending on your local setup you may hit this issue, and workaround it
by raising the limit of open file descriptors for the process (e.g. in
linux/bash using `ulimit -n 32000`). The error will be reported on the hadoop
job side as a thrift !TimedOutException.
-
- If you are testing the integration against a single node and you obtain some
failures, this may be normal: you are probably overloading the single machine,
which may again result in timeout errors. You can workaround it by reducing
the number of concurrent tasks
-
- {{{
- Configuration conf = job.getConfiguration();
- conf.setInt("mapred.tasktracker.map.tasks.maximum",1);
- }}}
- Also, you may reduce the size in rows of the batch you are reading from
cassandra
-
- {{{
- ConfigHelper.setRangeBatchSize(job.getConfiguration(), 1000);
- }}}
[[#Top|Top]]
<>
@@ -93, +79 @@
[[#Top|Top]]
+ <>
+
+ == Troubleshooting ==
+ If you are running into timeout exceptions, you might need to tweak one or
both of these settings:
+ * '''cassandra.range.batch.size''' - the default is 4096, but you may need
to lower this depending on your data. This is either specified in your hadoop
configuration or using
`org.apache.cassandra.hadoop.ConfigHelper.setRangeBatchSize`.
+ * '''rpc_timeout_in_ms''' - this is set in your `cassandra.yaml` (in 0.6
it's `RpcTimeoutInMillis` in `storage-conf.xml`). The rpc timeout is not for
timing out from the client but between nodes. This can be increased to reduce
chances of timing out.
+
+ Releases before 0.6.2/0.7 are affected by a small resource leak that may
cause jobs to fail (connections are not released properly, causing a resource
leak). Depending on your local setup you may hit this issue, and workaround it
by raising the limit of open file descriptors for the process (e.g. in
linux/bash using `ulimit -n 32000`). The error will be reported on the hadoop
job side as a thrift !TimedOutException.
+
+ If you are testing the integration against a single node and you obtain some
failures, this may be normal: you are probably overloading the single machine,
which may again result in timeout errors. You can workaround it by reducing the
number of concurrent tasks
+
+ {{{
+ Configuration conf = job.getConfiguration();
+ conf.setInt("mapred.tasktracker.map.tasks.maximum",1);
+ }}}
+ Also, you may reduce the size in rows of the batch you are reading from
cassandra
+
+ {{{
+ ConfigHelper.setRangeBatchSize(job.getConfiguration(), 1000);
+ }}}
+
+ [[#Top|Top]]
+
<>
== Support ==
svn commit: r1076892 - /cassandra/branches/cassandra-0.6/interface/cassandra.thrift
Author: jbellis
Date: Thu Mar 3 23:50:27 2011
New Revision: 1076892
URL: http://svn.apache.org/viewvc?rev=1076892&view=rev
Log:
revert changes to cassandra.thrift for newer compiler version
Modified:
cassandra/branches/cassandra-0.6/interface/cassandra.thrift
Modified: cassandra/branches/cassandra-0.6/interface/cassandra.thrift
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.6/interface/cassandra.thrift?rev=1076892&r1=1076891&r2=1076892&view=diff
==
--- cassandra/branches/cassandra-0.6/interface/cassandra.thrift (original)
+++ cassandra/branches/cassandra-0.6/interface/cassandra.thrift Thu Mar 3
23:50:27 2011
@@ -300,7 +300,7 @@ service Cassandra {
ColumnOrSuperColumn get(1:required string keyspace,
2:required string key,
3:required ColumnPath column_path,
- 4:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
+ 4:required ConsistencyLevel consistency_level=ONE)
throws (1:InvalidRequestException ire,
2:NotFoundException nfe, 3:UnavailableException ue, 4:TimedOutException te),
/**
@@ -311,7 +311,7 @@ service Cassandra {
2:required string key,
3:required ColumnParent column_parent,
4:required SlicePredicate predicate,
- 5:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
+ 5:required ConsistencyLevel
consistency_level=ONE)
throws (1:InvalidRequestException ire,
2:UnavailableException ue, 3:TimedOutException te),
/**
@@ -323,7 +323,7 @@ service Cassandra {
map multiget(1:required string keyspace,
2:required list keys,
3:required ColumnPath column_path,
- 4:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
+ 4:required ConsistencyLevel
consistency_level=ONE)
throws (1:InvalidRequestException ire,
2:UnavailableException ue, 3:TimedOutException te),
/**
@@ -333,7 +333,7 @@ service Cassandra {
2:required list
keys,
3:required ColumnParent
column_parent,
4:required
SlicePredicate predicate,
- 5:required
ConsistencyLevel consistency_level=ConsistencyLevel.ONE)
+ 5:required
ConsistencyLevel consistency_level=ONE)
throws (1:InvalidRequestException ire,
2:UnavailableException ue, 3:TimedOutException te),
/**
@@ -342,7 +342,7 @@ service Cassandra {
i32 get_count(1:required string keyspace,
2:required string key,
3:required ColumnParent column_parent,
-4:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
+4:required ConsistencyLevel consistency_level=ONE)
throws (1:InvalidRequestException ire, 2:UnavailableException ue,
3:TimedOutException te),
/**
@@ -355,7 +355,7 @@ service Cassandra {
4:required string start_key="",
5:required string finish_key="",
6:required i32 row_count=100,
- 7:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
+ 7:required ConsistencyLevel
consistency_level=ONE)
throws (1:InvalidRequestException ire, 2:UnavailableException
ue, 3:TimedOutException te),
/**
@@ -365,7 +365,7 @@ service Cassandra {
2:required ColumnParent column_parent,
3:required SlicePredicate predicate,
4:required KeyRange range,
- 5:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
+ 5:required ConsistencyLevel
consistency_level=ONE)
throws (1:InvalidRequestException ire, 2:UnavailableException
ue, 3:TimedOutException te),
# modification methods
@@ -380,7 +380,7 @@ service Cassandra {
3:required ColumnPath column_path,
4:required binary value,
5:required i64 timestamp,
- 6:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
+ 6:required ConsistencyLevel consistency_level=ONE)
svn commit: r1076891 - in /cassandra/branches/cassandra-0.6: interface/cassandra.thrift src/java/org/apache/cassandra/thrift/CustomTThreadPoolServer.java
Author: jbellis
Date: Thu Mar 3 23:49:44 2011
New Revision: 1076891
URL: http://svn.apache.org/viewvc?rev=1076891&view=rev
Log:
reformat
Modified:
cassandra/branches/cassandra-0.6/interface/cassandra.thrift
cassandra/branches/cassandra-0.6/src/java/org/apache/cassandra/thrift/CustomTThreadPoolServer.java
Modified: cassandra/branches/cassandra-0.6/interface/cassandra.thrift
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.6/interface/cassandra.thrift?rev=1076891&r1=1076890&r2=1076891&view=diff
==
--- cassandra/branches/cassandra-0.6/interface/cassandra.thrift (original)
+++ cassandra/branches/cassandra-0.6/interface/cassandra.thrift Thu Mar 3
23:49:44 2011
@@ -300,7 +300,7 @@ service Cassandra {
ColumnOrSuperColumn get(1:required string keyspace,
2:required string key,
3:required ColumnPath column_path,
- 4:required ConsistencyLevel consistency_level=ONE)
+ 4:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
throws (1:InvalidRequestException ire,
2:NotFoundException nfe, 3:UnavailableException ue, 4:TimedOutException te),
/**
@@ -311,7 +311,7 @@ service Cassandra {
2:required string key,
3:required ColumnParent column_parent,
4:required SlicePredicate predicate,
- 5:required ConsistencyLevel
consistency_level=ONE)
+ 5:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
throws (1:InvalidRequestException ire,
2:UnavailableException ue, 3:TimedOutException te),
/**
@@ -323,7 +323,7 @@ service Cassandra {
map multiget(1:required string keyspace,
2:required list keys,
3:required ColumnPath column_path,
- 4:required ConsistencyLevel
consistency_level=ONE)
+ 4:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
throws (1:InvalidRequestException ire,
2:UnavailableException ue, 3:TimedOutException te),
/**
@@ -333,7 +333,7 @@ service Cassandra {
2:required list
keys,
3:required ColumnParent
column_parent,
4:required
SlicePredicate predicate,
- 5:required
ConsistencyLevel consistency_level=ONE)
+ 5:required
ConsistencyLevel consistency_level=ConsistencyLevel.ONE)
throws (1:InvalidRequestException ire,
2:UnavailableException ue, 3:TimedOutException te),
/**
@@ -342,7 +342,7 @@ service Cassandra {
i32 get_count(1:required string keyspace,
2:required string key,
3:required ColumnParent column_parent,
-4:required ConsistencyLevel consistency_level=ONE)
+4:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
throws (1:InvalidRequestException ire, 2:UnavailableException ue,
3:TimedOutException te),
/**
@@ -355,7 +355,7 @@ service Cassandra {
4:required string start_key="",
5:required string finish_key="",
6:required i32 row_count=100,
- 7:required ConsistencyLevel
consistency_level=ONE)
+ 7:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
throws (1:InvalidRequestException ire, 2:UnavailableException
ue, 3:TimedOutException te),
/**
@@ -365,7 +365,7 @@ service Cassandra {
2:required ColumnParent column_parent,
3:required SlicePredicate predicate,
4:required KeyRange range,
- 5:required ConsistencyLevel
consistency_level=ONE)
+ 5:required ConsistencyLevel
consistency_level=ConsistencyLevel.ONE)
throws (1:InvalidRequestException ire, 2:UnavailableException
ue, 3:TimedOutException te),
# modification methods
@@ -380,7 +380,7 @@ service Cassandra {
3:required ColumnPath column_path,
4:required binary value,
5:required i64 timestamp,
- 6:required ConsistencyLevel consistency_level=ONE)
+ 6:required Consist
[jira] Updated: (CASSANDRA-2269) OOM in the Thrift thread doesn't shut down server
[
https://issues.apache.org/jira/browse/CASSANDRA-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-2269:
--
Fix Version/s: 0.6.13
> OOM in the Thrift thread doesn't shut down server
> -
>
> Key: CASSANDRA-2269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2269
> Project: Cassandra
> Issue Type: Bug
> Components: API
>Affects Versions: 0.6
>Reporter: Jonathan Ellis
>Assignee: Jonathan Ellis
>Priority: Minor
> Fix For: 0.6.13, 0.7.4
>
>
> Example:
> {noformat}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.resize(NonBlockingHashMap.java:849)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.access$200(NonBlockingHashMap.java:699)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:634)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:339)
> at
> org.cliffc.high_scale_lib.NonBlockingHashMap.put(NonBlockingHashMap.java:302)
> at org.apache.cassandra.utils.ExpiringMap.put(ExpiringMap.java:112)
> at
> org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:237)
> at
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:305)
> at
> org.apache.cassandra.service.StorageProxy.weakRead(StorageProxy.java:386)
> at
> org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:347)
> at
> org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:92)
> at
> org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:175)
> at
> org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:254)
> at
> org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:215)
> at
> org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:1272)
> at
> org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:1166)
> at
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {noformat}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2170) Load spikes
[ https://issues.apache.org/jira/browse/CASSANDRA-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2170: -- Fix Version/s: (was: 0.6.12) 0.6.13 > Load spikes > --- > > Key: CASSANDRA-2170 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2170 > Project: Cassandra > Issue Type: Bug >Affects Versions: 0.6.11 >Reporter: Jonathan Ellis > Fix For: 0.6.13 > > > as reported on CASSANDRA-2058, some users are still seeing load spikes on > 0.6.11, even with fairly low-volume read workloads. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (CASSANDRA-2269) OOM in the Thrift thread doesn't shut down server
OOM in the Thrift thread doesn't shut down server
-
Key: CASSANDRA-2269
URL: https://issues.apache.org/jira/browse/CASSANDRA-2269
Project: Cassandra
Issue Type: Bug
Components: API
Affects Versions: 0.6
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
Fix For: 0.7.4
Example:
{noformat}
java.lang.OutOfMemoryError: Java heap space
at
org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.resize(NonBlockingHashMap.java:849)
at
org.cliffc.high_scale_lib.NonBlockingHashMap$CHM.access$200(NonBlockingHashMap.java:699)
at
org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:634)
at
org.cliffc.high_scale_lib.NonBlockingHashMap.putIfMatch(NonBlockingHashMap.java:339)
at
org.cliffc.high_scale_lib.NonBlockingHashMap.put(NonBlockingHashMap.java:302)
at org.apache.cassandra.utils.ExpiringMap.put(ExpiringMap.java:112)
at
org.apache.cassandra.net.MessagingService.addCallback(MessagingService.java:237)
at
org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:305)
at
org.apache.cassandra.service.StorageProxy.weakRead(StorageProxy.java:386)
at
org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:347)
at
org.apache.cassandra.thrift.CassandraServer.readColumnFamily(CassandraServer.java:92)
at
org.apache.cassandra.thrift.CassandraServer.getSlice(CassandraServer.java:175)
at
org.apache.cassandra.thrift.CassandraServer.multigetSliceInternal(CassandraServer.java:254)
at
org.apache.cassandra.thrift.CassandraServer.get_slice(CassandraServer.java:215)
at
org.apache.cassandra.thrift.Cassandra$Processor$get_slice.process(Cassandra.java:1272)
at
org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:1166)
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:167)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{noformat}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.
[
https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002330#comment-13002330
]
Vijay commented on CASSANDRA-1969:
--
For single allocation thats true the following code works!... but wont our
byte[] within a column less than that? is this a real concern?
final int SIZE = 1024;
final int KBS = 1024 * 1024;
for (int i = 0; i < KBS; i++) {
ByteBuffer.allocateDirect(SIZE );
System.out.println("allocating: "+ i);
}
System.out.println("ok");
Thread.sleep(1);
}
> Use BB for row cache - To Improve GC performance.
> -
>
> Key: CASSANDRA-1969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1969
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: Linux and Mac
>Reporter: Vijay
>Assignee: Vijay
>Priority: Minor
> Attachments: 0001-Config-1969.txt,
> 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt,
> 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt,
> 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt,
> 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt,
> 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, BB_Cache-1945.png,
> JMX-Cache-1945.png, Old_Cahce-1945.png, POC-0001-Config-1945.txt,
> POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt
>
>
> Java BB.allocateDirect() will allocate native memory out of the JVM and will
> help reducing the GC pressure in the JVM with a large Cache.
> From some of the basic tests it shows around 50% improvement than doing a
> normal Object cache.
> In addition this patch provide the users an option to choose
> BB.allocateDirect or store everything in the heap.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1828) Create a pig storefunc
[ https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002328#comment-13002328 ] Jeremy Hanna commented on CASSANDRA-1828: - +1 2 comments: - I really like exceptions for invalid configuration - e.g. "PIG_INITIAL_ADDRESS environment variable not set" - why not just have getOutputFormat just return new ColumnFamilyOutputFormat(); > Create a pig storefunc > -- > > Key: CASSANDRA-1828 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1828 > Project: Cassandra > Issue Type: New Feature > Components: Contrib, Hadoop >Affects Versions: 0.7 beta 1 >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 0.7.4 > > Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, > 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Now that we have a ColumnFamilyOutputFormat, we can write data back to > cassandra in mapreduce jobs, however we can only do this in java. It would > be nice if pig could also output to cassandra. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1828) Create a pig storefunc
[ https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna updated CASSANDRA-1828: Reviewer: jeromatron > Create a pig storefunc > -- > > Key: CASSANDRA-1828 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1828 > Project: Cassandra > Issue Type: New Feature > Components: Contrib, Hadoop >Affects Versions: 0.7 beta 1 >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 0.7.4 > > Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, > 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Now that we have a ColumnFamilyOutputFormat, we can write data back to > cassandra in mapreduce jobs, however we can only do this in java. It would > be nice if pig could also output to cassandra. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)
[
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002323#comment-13002323
]
Hudson commented on CASSANDRA-2228:
---
Integrated in Cassandra-0.7 #344 (See
[https://hudson.apache.org/hudson/job/Cassandra-0.7/344/])
initialize localendpoint in gossiper earlier. patch by gdusbabek, reviewed
by jbellis. CASSANDRA-2228
> Race conditions when reinitialisating nodes (OOM + Nullpointer)
> ---
>
> Key: CASSANDRA-2228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.7.2
>Reporter: Thibaut
>Assignee: Gary Dusbabek
> Fix For: 0.7.4
>
> Attachments: v1-0001-initialize-localEndpoint_.txt
>
>
> I had a corrupt system table which wouldn't compact anymore and I deleted the
> files and restarted cassandra and let it take the same token/ip address.
> I experienced the same errors when I'm adding a newly installed node under
> the same token/ip address before calling repair.
> 1)
> After a few seconds/minutes, I get a OOM error:
> INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164)
> Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133)
> Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace:
> table_userentriesrep factor:3rep
> strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
>
> org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Migrations at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1
> operations)
> INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157)
> Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Schema at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
> INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89)
> table_sourcedetection.table_sourcedetection 0,0
> 0/00/20
> ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java
> (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:39)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
> at
> org.apache.cassandra.io.util.BufferedRandomAccessFile.(BufferedRandomAccessFile.java:126)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.(SSTableWriter.java:75)
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
> at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
> at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.
[jira] Commented: (CASSANDRA-1969) Use BB for row cache - To Improve GC performance.
[ https://issues.apache.org/jira/browse/CASSANDRA-1969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002303#comment-13002303 ] Jonathan Ellis commented on CASSANDRA-1969: --- Interesting, it's exactly 2x the 64M you pasted: default sun.misc.VM.maxDirectMemory()) == 129957888. So allocating 90MB works fine but 256M fails with OOM. > Use BB for row cache - To Improve GC performance. > - > > Key: CASSANDRA-1969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1969 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Linux and Mac >Reporter: Vijay >Assignee: Vijay >Priority: Minor > Attachments: 0001-Config-1969.txt, > 0001-introduce-ICache-InstrumentingCache-IRowCacheProvider.txt, > 0002-Update_existing-1965.txt, 0002-implement-SerializingCache.txt, > 0002-implement-SerializingCacheV2.txt, 0003-New_Cache_Providers-1969.txt, > 0003-add-ICache.isCopying-method.txt, 0004-Null-Check-and-duplicate-bb.txt, > 0004-TestCase-1969.txt, 1969-rollup-and-config.txt, BB_Cache-1945.png, > JMX-Cache-1945.png, Old_Cahce-1945.png, POC-0001-Config-1945.txt, > POC-0002-Update_existing-1945.txt, POC-0003-New_Cache_Providers-1945.txt > > > Java BB.allocateDirect() will allocate native memory out of the JVM and will > help reducing the GC pressure in the JVM with a large Cache. > From some of the basic tests it shows around 50% improvement than doing a > normal Object cache. > In addition this patch provide the users an option to choose > BB.allocateDirect or store everything in the heap. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1076868 - in /cassandra/trunk: ./ contrib/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/gms/ test/unit/org/apache/cassandra/cli/
Author: gdusbabek Date: Thu Mar 3 22:59:36 2011 New Revision: 1076868 URL: http://svn.apache.org/viewvc?rev=1076868&view=rev Log: merge from 0.7 Modified: cassandra/trunk/ (props changed) cassandra/trunk/CHANGES.txt cassandra/trunk/contrib/ (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/NotFoundException.java (props changed) cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/SuperColumn.java (props changed) cassandra/trunk/src/java/org/apache/cassandra/gms/Gossiper.java cassandra/trunk/test/unit/org/apache/cassandra/cli/CliTest.java Propchange: cassandra/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Mar 3 22:59:36 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6:922689-1052356,1052358-1053452,1053454,1053456-1071777 -/cassandra/branches/cassandra-0.7:1026516-1076706 +/cassandra/branches/cassandra-0.7:1026516-1076866 /cassandra/branches/cassandra-0.7.0:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3:774578-796573 Modified: cassandra/trunk/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/trunk/CHANGES.txt?rev=1076868&r1=1076867&r2=1076868&view=diff == --- cassandra/trunk/CHANGES.txt (original) +++ cassandra/trunk/CHANGES.txt Thu Mar 3 22:59:36 2011 @@ -14,7 +14,7 @@ 0.7.4 * add nodetool join command (CASSANDRA-2160) * fix secondary indexes on pre-existing or streamed data (CASSANDRA-2244) - + * initialize endpoing in gossiper earlier (CASSANDRA-2228) 0.7.3 * Keep endpoint state until aVeryLongTime (CASSANDRA-2115) Propchange: cassandra/trunk/contrib/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Mar 3 22:59:36 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/contrib:922689-1052356,1052358-1053452,1053454,1053456-1068009 -/cassandra/branches/cassandra-0.7/contrib:1026516-1076706 +/cassandra/branches/cassandra-0.7/contrib:1026516-1076866 /cassandra/branches/cassandra-0.7.0/contrib:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/contrib:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/contrib:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Mar 3 22:59:36 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:922689-1052356,1052358-1053452,1053454,1053456-1071777 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1076706 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1026516-1076866 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Cassandra.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/Cassandra.java:774578-796573 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java -- --- svn:mergeinfo (original) +++ svn:mergeinfo Thu Mar 3 22:59:36 2011 @@ -1,5 +1,5 @@ /cassandra/branches/cassandra-0.6/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:922689-1052356,1052358-1053452,1053454,1053456-1071777 -/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1076706 +/cassandra/branches/cassandra-0.7/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1026516-1076866 /cassandra/branches/cassandra-0.7.0/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1053690-1055654 /cassandra/tags/cassandra-0.7.0-rc3/interface/thrift/gen-java/org/apache/cassandra/thrift/Column.java:1051699-1053689 /incubator/cassandra/branches/cassandra-0.3/interface/gen-java/org/apache/cassandra/service/column_t.java:774578-792198 Propchange: cassandra/trunk/interface/thrift/gen-java/org/apache/cassandra/thrift/InvalidRequestException.java -- --- sv
svn commit: r1076866 - in /cassandra/branches/cassandra-0.7: ./ src/java/org/apache/cassandra/gms/ src/java/org/apache/cassandra/service/ test/unit/org/apache/cassandra/cli/ test/unit/org/apache/cassa
Author: gdusbabek
Date: Thu Mar 3 22:54:56 2011
New Revision: 1076866
URL: http://svn.apache.org/viewvc?rev=1076866&view=rev
Log:
initialize localendpoint in gossiper earlier. patch by gdusbabek, reviewed by
jbellis. CASSANDRA-2228
Modified:
cassandra/branches/cassandra-0.7/CHANGES.txt
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/gms/Gossiper.java
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/cli/CliTest.java
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/service/RemoveTest.java
Modified: cassandra/branches/cassandra-0.7/CHANGES.txt
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1076866&r1=1076865&r2=1076866&view=diff
==
--- cassandra/branches/cassandra-0.7/CHANGES.txt (original)
+++ cassandra/branches/cassandra-0.7/CHANGES.txt Thu Mar 3 22:54:56 2011
@@ -1,7 +1,7 @@
0.7.4
* add nodetool join command (CASSANDRA-2160)
* fix secondary indexes on pre-existing or streamed data (CASSANDRA-2244)
-
+ * initialize endpoing in gossiper earlier (CASSANDRA-2228)
0.7.3
* Keep endpoint state until aVeryLongTime (CASSANDRA-2115)
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/gms/Gossiper.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/gms/Gossiper.java?rev=1076866&r1=1076865&r2=1076866&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/gms/Gossiper.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/gms/Gossiper.java
Thu Mar 3 22:54:56 2011
@@ -26,6 +26,7 @@ import java.util.*;
import java.util.Map.Entry;
import java.util.concurrent.*;
+import org.apache.cassandra.utils.FBUtilities;
import org.cliffc.high_scale_lib.NonBlockingHashMap;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -149,6 +150,7 @@ public class Gossiper implements IFailur
private Gossiper()
{
+localEndpoint_ = FBUtilities.getLocalAddress();
// 3 days
aVeryLongTime_ = 259200 * 1000;
// half of QUARATINE_DELAY, to ensure justRemovedEndpoints has enough
leeway to prevent re-gossip
@@ -870,14 +872,13 @@ public class Gossiper implements IFailur
* Start the gossiper with the generation # retrieved from the System
* table
*/
-public void start(InetAddress localEndpoint, int generationNbr)
+public void start(int generationNbr)
{
-localEndpoint_ = localEndpoint;
/* Get the seeds from the config and initialize them. */
Set seedHosts = DatabaseDescriptor.getSeeds();
for (InetAddress seed : seedHosts)
{
-if (seed.equals(localEndpoint))
+if (seed.equals(localEndpoint_))
continue;
seeds_.add(seed);
}
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java?rev=1076866&r1=1076865&r2=1076866&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/service/StorageService.java
Thu Mar 3 22:54:56 2011
@@ -282,7 +282,7 @@ public class StorageService implements I
if (!initialized)
{
logger_.warn("Starting gossip by operator request");
-Gossiper.instance.start(FBUtilities.getLocalAddress(),
(int)(System.currentTimeMillis() / 1000));
+Gossiper.instance.start((int)(System.currentTimeMillis() / 1000));
initialized = true;
}
}
@@ -343,7 +343,7 @@ public class StorageService implements I
logger_.info("Starting up client gossip");
setMode("Client", false);
Gossiper.instance.register(this);
-Gossiper.instance.start(FBUtilities.getLocalAddress(),
(int)(System.currentTimeMillis() / 1000)); // needed for node-ring gathering.
+Gossiper.instance.start((int)(System.currentTimeMillis() / 1000)); //
needed for node-ring gathering.
MessagingService.instance().listen(FBUtilities.getLocalAddress());
// sleep a while to allow gossip to warm up (the other nodes need to
know about this one before they can reply).
@@ -418,7 +418,7 @@ public class StorageService implements I
// (we won't be part of the storage ring though until we add a nodeId
to our state, below.)
Gossiper.instance.register(this);
Gossiper.instance.register(migr
[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Dusbabek updated CASSANDRA-2262:
-
Attachment: (was: v1-0004-assume-utf8-in-CliTest-keys-dammit.txt)
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v2-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v2-0002-BytesType.fromString-expects-a-hex-string.txt,
> v2-0003-compose-method-for-AbstractTypes.txt,
> v2-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Dusbabek updated CASSANDRA-2262:
-
Attachment: (was: v1-0002-BytesType.fromString-expects-a-hex-string.txt)
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v2-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v2-0002-BytesType.fromString-expects-a-hex-string.txt,
> v2-0003-compose-method-for-AbstractTypes.txt,
> v2-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Dusbabek updated CASSANDRA-2262:
-
Attachment: (was: v1-0001-test-shows-no-roundtrip-in-BytesType.txt)
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v2-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v2-0002-BytesType.fromString-expects-a-hex-string.txt,
> v2-0003-compose-method-for-AbstractTypes.txt,
> v2-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Dusbabek updated CASSANDRA-2262:
-
Attachment: (was: v1-0003-compose-method-for-AbstractTypes.txt)
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v2-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v2-0002-BytesType.fromString-expects-a-hex-string.txt,
> v2-0003-compose-method-for-AbstractTypes.txt,
> v2-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Dusbabek updated CASSANDRA-2262:
-
Attachment: v2-0004-assume-utf8-in-CliTest-keys-dammit.txt
v2-0003-compose-method-for-AbstractTypes.txt
v2-0002-BytesType.fromString-expects-a-hex-string.txt
v2-0001-test-shows-no-roundtrip-in-BytesType.txt
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v1-0002-BytesType.fromString-expects-a-hex-string.txt,
> v1-0003-compose-method-for-AbstractTypes.txt,
> v1-0004-assume-utf8-in-CliTest-keys-dammit.txt,
> v2-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v2-0002-BytesType.fromString-expects-a-hex-string.txt,
> v2-0003-compose-method-for-AbstractTypes.txt,
> v2-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002275#comment-13002275
]
Gary Dusbabek commented on CASSANDRA-2262:
--
Yes. I'm reworking those tests to do just that.
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v1-0002-BytesType.fromString-expects-a-hex-string.txt,
> v1-0003-compose-method-for-AbstractTypes.txt,
> v1-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002271#comment-13002271
]
Jonathan Ellis commented on CASSANDRA-2262:
---
bq. the ticket description said that we should have get(from(s)) == s, but some
of the test in RoundTripTest actually test from(get(b)) == b
don't we want to test both?
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v1-0002-BytesType.fromString-expects-a-hex-string.txt,
> v1-0003-compose-method-for-AbstractTypes.txt,
> v1-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)
[
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002268#comment-13002268
]
Jonathan Ellis commented on CASSANDRA-2228:
---
+1
> Race conditions when reinitialisating nodes (OOM + Nullpointer)
> ---
>
> Key: CASSANDRA-2228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.7.2
>Reporter: Thibaut
>Assignee: Gary Dusbabek
> Fix For: 0.7.4
>
> Attachments: v1-0001-initialize-localEndpoint_.txt
>
>
> I had a corrupt system table which wouldn't compact anymore and I deleted the
> files and restarted cassandra and let it take the same token/ip address.
> I experienced the same errors when I'm adding a newly installed node under
> the same token/ip address before calling repair.
> 1)
> After a few seconds/minutes, I get a OOM error:
> INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164)
> Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133)
> Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace:
> table_userentriesrep factor:3rep
> strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
>
> org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Migrations at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1
> operations)
> INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157)
> Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Schema at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
> INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89)
> table_sourcedetection.table_sourcedetection 0,0
> 0/00/20
> ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java
> (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:39)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
> at
> org.apache.cassandra.io.util.BufferedRandomAccessFile.(BufferedRandomAccessFile.java:126)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.(SSTableWriter.java:75)
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
> at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
> at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2) If I restart then, I'm getting an Nullpointer exception. The OOM error
> will only appear once.
>
[jira] Commented: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002258#comment-13002258
]
Sylvain Lebresne commented on CASSANDRA-2262:
-
bq. Can you elaborate?
I mean that the ticket description said that we should have get(from(s)) == s,
but some of the test in RoundTripTest actually test from(get(b)) == b (all of
them except for the two UUID tests). So this is not really testing what it
should. And this is not the same since for instance, get(from(s)) always work
for IntegerType while from(get(b)) can throw an exception (and thus not work)
depending on what is b as shown by testInteger2.
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v1-0002-BytesType.fromString-expects-a-hex-string.txt,
> v1-0003-compose-method-for-AbstractTypes.txt,
> v1-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2124) JDBC driver for CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-2124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002253#comment-13002253 ] Gary Dusbabek commented on CASSANDRA-2124: -- bq. For this, either we need to introduce a wrapper or Ascii, BytesType, UUID etc. classes need to provide a way for toString() calls. for AsciiType use getString(), for BytesType use getBytes(), for UUID use getObject and cast to a java.util.UUID. I think this patch set is ready to commit and we can move on to the next iteration (ResultSetMetadata or whatever). > JDBC driver for CQL > --- > > Key: CASSANDRA-2124 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2124 > Project: Cassandra > Issue Type: New Feature > Components: API >Reporter: Eric Evans >Assignee: Vivek Mishra >Priority: Minor > Labels: cql > Attachments: Cassandra-2124_v1.0, Cassandra_2124_decoder.patch, > cassandra-0.7.1-2124_v2.0, cassandra-0.7.1-2124_v2.1, > cassandra_generic_decoder.patch, cassandra_generic_decoder_v1.1.patch, > v1-0001-first-pass-at-column-decoding.txt, > v1-0002-clean-up-JdbcDriverTest.txt, > v1-0003-implements-getXXX-methods-to-return-values-of-the-corr.txt, > v1-0004-more-comments-in-ColumnDecoder.txt > > > A simple connection class and corresponding pool was created for CQL as a > part of CASSANDRA-1710, but a JDBC driver (either in addition to, or as a > replacement for) would also be interesting. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[ https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002254#comment-13002254 ] Sylvain Lebresne commented on CASSANDRA-2231: - Yes, IntegerType would be a better choice because it uses BigInteger that packs stuffs as much as it can (aka, 1 byte if it fits). I know that because it bitted me while writing the unit test for this :)) > Add CompositeType comparer to the comparers provided in > org.apache.cassandra.db.marshal > --- > > Key: CASSANDRA-2231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2231 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Affects Versions: 0.7.3 >Reporter: Ed Anuff >Priority: Minor > Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, > 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip > > > CompositeType is a custom comparer that makes it possible to create > comparable composite values out of the basic types that Cassandra currently > supports, such as Long, UUID, etc. This is very useful in both the creation > of custom inverted indexes using columns in a skinny row, where each column > name is a composite value, and also when using Cassandra's built-in secondary > index support, where it can be used to encode the values in the columns that > Cassandra indexes. One scenario for the usage of these is documented here: > http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for > contribution is attached and has been previously maintained on github here: > https://github.com/edanuff/CassandraCompositeType -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)
[
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Dusbabek updated CASSANDRA-2228:
-
Attachment: v1-0001-initialize-localEndpoint_.txt
> Race conditions when reinitialisating nodes (OOM + Nullpointer)
> ---
>
> Key: CASSANDRA-2228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.7.2
>Reporter: Thibaut
>Assignee: Gary Dusbabek
> Fix For: 0.7.4
>
> Attachments: v1-0001-initialize-localEndpoint_.txt
>
>
> I had a corrupt system table which wouldn't compact anymore and I deleted the
> files and restarted cassandra and let it take the same token/ip address.
> I experienced the same errors when I'm adding a newly installed node under
> the same token/ip address before calling repair.
> 1)
> After a few seconds/minutes, I get a OOM error:
> INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164)
> Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133)
> Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace:
> table_userentriesrep factor:3rep
> strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
>
> org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Migrations at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1
> operations)
> INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157)
> Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Schema at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
> INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89)
> table_sourcedetection.table_sourcedetection 0,0
> 0/00/20
> ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java
> (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:39)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
> at
> org.apache.cassandra.io.util.BufferedRandomAccessFile.(BufferedRandomAccessFile.java:126)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.(SSTableWriter.java:75)
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
> at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
> at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2) If I restart then, I'm getting an Nullpointer exception. The OOM error
> will only appear once.
> ERROR [
[jira] Updated: (CASSANDRA-2248) ant javadoc fails on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-2248: - Attachment: v1-0001-initialize-localEndpoint_.txt > ant javadoc fails on windows > > > Key: CASSANDRA-2248 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2248 > Project: Cassandra > Issue Type: Bug > Components: Packaging > Environment: windows 7, ant 1.8.2 >Reporter: Norman Maurer >Assignee: Norman Maurer > Fix For: 0.7.3 > > Attachments: CASSANDRA-2248.diff > > > When try to run "ant javadoc" (or any task that include javadoc) on windows > it fails with the error: > Javadoc failed: java.io.IOException: Cannot run program "c:\Program > Files\Java\jdk1.6.0_17\bin\javadoc.exe": CreateProcess error=87, The > parameter is incorrect -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2248) ant javadoc fails on windows
[ https://issues.apache.org/jira/browse/CASSANDRA-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Dusbabek updated CASSANDRA-2248: - Attachment: (was: v1-0001-initialize-localEndpoint_.txt) > ant javadoc fails on windows > > > Key: CASSANDRA-2248 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2248 > Project: Cassandra > Issue Type: Bug > Components: Packaging > Environment: windows 7, ant 1.8.2 >Reporter: Norman Maurer >Assignee: Norman Maurer > Fix For: 0.7.3 > > Attachments: CASSANDRA-2248.diff > > > When try to run "ant javadoc" (or any task that include javadoc) on windows > it fails with the error: > Javadoc failed: java.io.IOException: Cannot run program "c:\Program > Files\Java\jdk1.6.0_17\bin\javadoc.exe": CreateProcess error=87, The > parameter is incorrect -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[ https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002226#comment-13002226 ] Ed Anuff commented on CASSANDRA-2231: - Hmm, that would work, although you certainly wouldn't want to use the LongType as your integer. I guess the minimum overhead for a component is 6 bytes - 2 header, 2 length, 1 value, 1 inclusion flag. I'm not seeing anything else that wouldn't let me use this as a functional replacement for the original CompositeType, so I'm +1 on it. Thanks! > Add CompositeType comparer to the comparers provided in > org.apache.cassandra.db.marshal > --- > > Key: CASSANDRA-2231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2231 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Affects Versions: 0.7.3 >Reporter: Ed Anuff >Priority: Minor > Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, > 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip > > > CompositeType is a custom comparer that makes it possible to create > comparable composite values out of the basic types that Cassandra currently > supports, such as Long, UUID, etc. This is very useful in both the creation > of custom inverted indexes using columns in a skinny row, where each column > name is a composite value, and also when using Cassandra's built-in secondary > index support, where it can be used to encode the values in the columns that > Cassandra indexes. One scenario for the usage of these is documented here: > http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for > contribution is attached and has been previously maintained on github here: > https://github.com/edanuff/CassandraCompositeType -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (CASSANDRA-2268) CQL-enabled stress.java
[ https://issues.apache.org/jira/browse/CASSANDRA-2268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-2268: - Assignee: Pavel Yaskevich > CQL-enabled stress.java > --- > > Key: CASSANDRA-2268 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2268 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Eric Evans >Assignee: Pavel Yaskevich >Priority: Minor > Labels: cql > Fix For: 0.8 > > > It would be great if stress.java had a CQL mode. For making the inevitable > RPC->CQL comparisons, but also as a basis for measuring optimizations, and > spotting performance regressions. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2267) stress.java daemon mode
[ https://issues.apache.org/jira/browse/CASSANDRA-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002204#comment-13002204 ] Brandon Williams commented on CASSANDRA-2267: - To clarify, this isn't so much about warming the JVM, but that is a side effect gained from having a daemon mode and the operations exposed via an API: the operator can warm it sufficiently themselves. > stress.java daemon mode > --- > > Key: CASSANDRA-2267 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2267 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Affects Versions: 0.7.1 >Reporter: Brandon Williams >Assignee: Pavel Yaskevich > > stress.java uses a JVM, but there is no way to warm it up, which skews > results. It would be nice if there was some sort of daemon mode so the JVM > could stay hot, and even better if there was a way to control the daemon > programmatically, perhaps via JMX. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (CASSANDRA-2268) CQL-enabled stress.java
CQL-enabled stress.java --- Key: CASSANDRA-2268 URL: https://issues.apache.org/jira/browse/CASSANDRA-2268 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Eric Evans Priority: Minor Fix For: 0.8 It would be great if stress.java had a CQL mode. For making the inevitable RPC->CQL comparisons, but also as a basis for measuring optimizations, and spotting performance regressions. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2267) stress.java daemon mode
[ https://issues.apache.org/jira/browse/CASSANDRA-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002206#comment-13002206 ] Brandon Williams commented on CASSANDRA-2267: - It doesn't *have* to be JMX, but that seems like the easiest path initially. > stress.java daemon mode > --- > > Key: CASSANDRA-2267 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2267 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Affects Versions: 0.7.1 >Reporter: Brandon Williams >Assignee: Pavel Yaskevich > > stress.java uses a JVM, but there is no way to warm it up, which skews > results. It would be nice if there was some sort of daemon mode so the JVM > could stay hot, and even better if there was a way to control the daemon > programmatically, perhaps via JMX. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2266) Add standard deviation to column names/values in stress.java
[ https://issues.apache.org/jira/browse/CASSANDRA-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002203#comment-13002203 ] Brandon Williams commented on CASSANDRA-2266: - That sounds reasonable to me. > Add standard deviation to column names/values in stress.java > > > Key: CASSANDRA-2266 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2266 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Reporter: Brandon Williams >Assignee: Pavel Yaskevich >Priority: Minor > > stress currently produce fixed-size columns and values across rows, leading > to unnatural problems such as the memtables turning over at the same time on > all nodes. Instead these could take standard deviation flags. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2267) stress.java daemon mode
[ https://issues.apache.org/jira/browse/CASSANDRA-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002202#comment-13002202 ] Eric Evans commented on CASSANDRA-2267: --- bq. ...a way to control the daemon programmatically, perhaps via JMX. I love JMX, but everytime you use it a Ruby dev kills a kitten. > stress.java daemon mode > --- > > Key: CASSANDRA-2267 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2267 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Affects Versions: 0.7.1 >Reporter: Brandon Williams >Assignee: Pavel Yaskevich > > stress.java uses a JVM, but there is no way to warm it up, which skews > results. It would be nice if there was some sort of daemon mode so the JVM > could stay hot, and even better if there was a way to control the daemon > programmatically, perhaps via JMX. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2266) Add standard deviation to column names/values in stress.java
[ https://issues.apache.org/jira/browse/CASSANDRA-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002199#comment-13002199 ] Jonathan Ellis commented on CASSANDRA-2266: --- ISTM that we could get 99% of the benefit here w/o too much complexity by decreeing that column value sizes go from 0..2*size, i.e., average stays the same but you do get all kinds of sizes in there. I.e., I don't think I'm in favor of adding more options. > Add standard deviation to column names/values in stress.java > > > Key: CASSANDRA-2266 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2266 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Reporter: Brandon Williams >Assignee: Pavel Yaskevich >Priority: Minor > > stress currently produce fixed-size columns and values across rows, leading > to unnatural problems such as the memtables turning over at the same time on > all nodes. Instead these could take standard deviation flags. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)
[
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002197#comment-13002197
]
Gary Dusbabek commented on CASSANDRA-2228:
--
aha. I was looking in the trunk code (no chance of null). I see the problem in
the 0.7 branch.
> Race conditions when reinitialisating nodes (OOM + Nullpointer)
> ---
>
> Key: CASSANDRA-2228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.7.2
>Reporter: Thibaut
>Assignee: Gary Dusbabek
> Fix For: 0.7.4
>
>
> I had a corrupt system table which wouldn't compact anymore and I deleted the
> files and restarted cassandra and let it take the same token/ip address.
> I experienced the same errors when I'm adding a newly installed node under
> the same token/ip address before calling repair.
> 1)
> After a few seconds/minutes, I get a OOM error:
> INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164)
> Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133)
> Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace:
> table_userentriesrep factor:3rep
> strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
>
> org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Migrations at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1
> operations)
> INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157)
> Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Schema at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
> INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89)
> table_sourcedetection.table_sourcedetection 0,0
> 0/00/20
> ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java
> (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:39)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
> at
> org.apache.cassandra.io.util.BufferedRandomAccessFile.(BufferedRandomAccessFile.java:126)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.(SSTableWriter.java:75)
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
> at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
> at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2) If I restart then, I'm getting an Nullpointer exception. The OOM erro
[jira] Created: (CASSANDRA-2267) stress.java daemon mode
stress.java daemon mode --- Key: CASSANDRA-2267 URL: https://issues.apache.org/jira/browse/CASSANDRA-2267 Project: Cassandra Issue Type: Improvement Components: Contrib Affects Versions: 0.7.1 Reporter: Brandon Williams Assignee: Pavel Yaskevich stress.java uses a JVM, but there is no way to warm it up, which skews results. It would be nice if there was some sort of daemon mode so the JVM could stay hot, and even better if there was a way to control the daemon programmatically, perhaps via JMX. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (CASSANDRA-2266) Add standard deviation to column names/values in stress.java
Add standard deviation to column names/values in stress.java Key: CASSANDRA-2266 URL: https://issues.apache.org/jira/browse/CASSANDRA-2266 Project: Cassandra Issue Type: Improvement Components: Contrib Reporter: Brandon Williams Assignee: Pavel Yaskevich Priority: Minor stress currently produce fixed-size columns and values across rows, leading to unnatural problems such as the memtables turning over at the same time on all nodes. Instead these could take standard deviation flags. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002196#comment-13002196
]
Gary Dusbabek commented on CASSANDRA-2262:
--
bq. Last remark, the tests of the first patch are testing the wrong equation.
Can you elaborate?
as for testInteger2(), that should produce the buffer error as you later
expected.
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v1-0002-BytesType.fromString-expects-a-hex-string.txt,
> v1-0003-compose-method-for-AbstractTypes.txt,
> v1-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[ https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002193#comment-13002193 ] Sylvain Lebresne commented on CASSANDRA-2231: - We could do that. But I have to admit that I'm usually not very fond of making things work when it's not clear they should. I think that could hide problems more than it helps. Better fail fast if there is a mess up. And if you want to store in the same row column names that have intrinsically different structures, you can prefix it manually to distinguish thinks. That is, in you example, you'll replace (type1) by and (type2) by and assign the same integer for all the column name of type1 and the same integer (but a different from first one) to all those of type2. And this has actually a bunch of advantages: because you use a true component, you can query for the whole range of column name having the same 'type' (aka all type1 or all type2). Moreover you do control the prefix so you have more control on how each separate set of columns sort between each other. Yes you have to manage those prefix client side, but honestly I think it's a small price for the type safety it provides. > Add CompositeType comparer to the comparers provided in > org.apache.cassandra.db.marshal > --- > > Key: CASSANDRA-2231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2231 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Affects Versions: 0.7.3 >Reporter: Ed Anuff >Priority: Minor > Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, > 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip > > > CompositeType is a custom comparer that makes it possible to create > comparable composite values out of the basic types that Cassandra currently > supports, such as Long, UUID, etc. This is very useful in both the creation > of custom inverted indexes using columns in a skinny row, where each column > name is a composite value, and also when using Cassandra's built-in secondary > index support, where it can be used to encode the values in the columns that > Cassandra indexes. One scenario for the usage of these is documented here: > http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for > contribution is attached and has been previously maintained on github here: > https://github.com/edanuff/CassandraCompositeType -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2265) Fix distributed tests after updating Guava to 08
[
https://issues.apache.org/jira/browse/CASSANDRA-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Yaskevich updated CASSANDRA-2265:
---
Priority: Blocker (was: Major)
> Fix distributed tests after updating Guava to 08
>
>
> Key: CASSANDRA-2265
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2265
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
>Reporter: Pavel Yaskevich
>Assignee: Pavel Yaskevich
>Priority: Blocker
> Fix For: 0.8
>
>
> Distributed tests fail to start cluster:
> {code}
> [junit] java.lang.NoSuchMethodError:
> com.google.common.net.InternetDomainName.isValid(Ljava/lang/String;)Z
> [junit] at
> org.jclouds.rest.binders.BindAsHostPrefix.bindToRequest(BindAsHostPrefix.java:52)
> [junit] at
> org.jclouds.aws.s3.binders.BindAsHostPrefixIfConfigured.bindToRequest(BindAsHostPrefixIfConfigured.java:67)
> [junit] at
> org.jclouds.rest.internal.RestAnnotationProcessor.decorateRequest(RestAnnotationProcessor.java:860)
> [junit] at
> org.jclouds.rest.internal.RestAnnotationProcessor.createRequest(RestAnnotationProcessor.java:477)
> [junit] at
> org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFuture(AsyncRestClientProxy.java:117)
> [junit] at
> org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:97)
> [junit] at $Proxy58.objectExists(Unknown Source)
> [junit] at
> org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
> [junit] at $Proxy59.objectExists(Unknown Source)
> [junit] at
> org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
> [junit] at
> org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
> [junit] at
> org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
> [junit] at
> org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
> [junit] at
> org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
> [junit] at
> org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
> [junit] at
> org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
> [junit] at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
> [junit] at
> org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
> [junit] at $Proxy59.objectExists(Unknown Source)
> [junit] at
> org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
> [junit] at
> org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
> [junit] at
> org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
> [junit] at
> org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
> [junit] at
> org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
> [junit] at
> org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
> [junit] at
> org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
> [junit] at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
> {code}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2265) Fix distributed tests after updating Guava to 08
[
https://issues.apache.org/jira/browse/CASSANDRA-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002174#comment-13002174
]
Pavel Yaskevich commented on CASSANDRA-2265:
Created an issue in jclouds tracker
http://code.google.com/p/jclouds/issues/detail?id=492
> Fix distributed tests after updating Guava to 08
>
>
> Key: CASSANDRA-2265
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2265
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
>Reporter: Pavel Yaskevich
>Assignee: Pavel Yaskevich
> Fix For: 0.8
>
>
> Distributed tests fail to start cluster:
> {code}
> [junit] java.lang.NoSuchMethodError:
> com.google.common.net.InternetDomainName.isValid(Ljava/lang/String;)Z
> [junit] at
> org.jclouds.rest.binders.BindAsHostPrefix.bindToRequest(BindAsHostPrefix.java:52)
> [junit] at
> org.jclouds.aws.s3.binders.BindAsHostPrefixIfConfigured.bindToRequest(BindAsHostPrefixIfConfigured.java:67)
> [junit] at
> org.jclouds.rest.internal.RestAnnotationProcessor.decorateRequest(RestAnnotationProcessor.java:860)
> [junit] at
> org.jclouds.rest.internal.RestAnnotationProcessor.createRequest(RestAnnotationProcessor.java:477)
> [junit] at
> org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFuture(AsyncRestClientProxy.java:117)
> [junit] at
> org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:97)
> [junit] at $Proxy58.objectExists(Unknown Source)
> [junit] at
> org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
> [junit] at $Proxy59.objectExists(Unknown Source)
> [junit] at
> org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
> [junit] at
> org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
> [junit] at
> org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
> [junit] at
> org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
> [junit] at
> org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
> [junit] at
> org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
> [junit] at
> org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
> [junit] at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
> [junit] at
> org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
> [junit] at $Proxy59.objectExists(Unknown Source)
> [junit] at
> org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
> [junit] at
> org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
> [junit] at
> org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
> [junit] at
> org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
> [junit] at
> org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
> [junit] at
> org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
> [junit] at
> org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
> [junit] at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
> {code}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[ https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002170#comment-13002170 ] Ed Anuff commented on CASSANDRA-2231: - If two dynamic composite types are compared, the first and the second , this results in an exception being thrown in line 659, correct? In the original CompositeType, the component types each had an ordinal type value and the comparison was done on those type values if the components were of different types. I might suggest that in your code using the alias character byte or the hashCode() of the classname as the type value and doing a similar comparison, rather than throwing an exception. > Add CompositeType comparer to the comparers provided in > org.apache.cassandra.db.marshal > --- > > Key: CASSANDRA-2231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2231 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Affects Versions: 0.7.3 >Reporter: Ed Anuff >Priority: Minor > Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, > 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip > > > CompositeType is a custom comparer that makes it possible to create > comparable composite values out of the basic types that Cassandra currently > supports, such as Long, UUID, etc. This is very useful in both the creation > of custom inverted indexes using columns in a skinny row, where each column > name is a composite value, and also when using Cassandra's built-in secondary > index support, where it can be used to encode the values in the columns that > Cassandra indexes. One scenario for the usage of these is documented here: > http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for > contribution is attached and has been previously maintained on github here: > https://github.com/edanuff/CassandraCompositeType -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (CASSANDRA-1828) Create a pig storefunc
[ https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002162#comment-13002162 ] Brandon Williams edited comment on CASSANDRA-1828 at 3/3/11 7:35 PM: - Couldn't get Eldon's patch to apply, but updated 0001 with his changes to add deletions and explicitly cast String, as well as other cleanups. Only 0001 and 0002 are part of the patchset, 0003 is an outdated conglomeration of the two now. was (Author: brandon.williams): Couldn't get Eldon's patch to apply, but update 0001 with his changes to add deletions and explicitly cast String, as well as other cleanups. Only 0001 and 0002 are part of the patchset, 0003 is an outdated conglomeration of the two now. > Create a pig storefunc > -- > > Key: CASSANDRA-1828 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1828 > Project: Cassandra > Issue Type: New Feature > Components: Contrib, Hadoop >Affects Versions: 0.7 beta 1 >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 0.7.4 > > Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, > 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Now that we have a ColumnFamilyOutputFormat, we can write data back to > cassandra in mapreduce jobs, however we can only do this in java. It would > be nice if pig could also output to cassandra. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1828) Create a pig storefunc
[ https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-1828: Attachment: 0001-add-storage-ability-to-pig-CassandraStorage.txt Couldn't get Eldon's patch to apply, but update 0001 with his changes to add deletions and explicitly cast String, as well as other cleanups. Only 0001 and 0002 are part of the patchset, 0003 is an outdated conglomeration of the two now. > Create a pig storefunc > -- > > Key: CASSANDRA-1828 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1828 > Project: Cassandra > Issue Type: New Feature > Components: Contrib, Hadoop >Affects Versions: 0.7 beta 1 >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 0.7.4 > > Attachments: 0001-add-storage-ability-to-pig-CassandraStorage.txt, > 0002-Fix-build-bin-script.txt, 0003-StoreFunc_with_deletion.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Now that we have a ColumnFamilyOutputFormat, we can write data back to > cassandra in mapreduce jobs, however we can only do this in java. It would > be nice if pig could also output to cassandra. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1828) Create a pig storefunc
[ https://issues.apache.org/jira/browse/CASSANDRA-1828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-1828: Attachment: (was: 0001-add-storage-ability-to-pig-CassandraStorage.txt) > Create a pig storefunc > -- > > Key: CASSANDRA-1828 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1828 > Project: Cassandra > Issue Type: New Feature > Components: Contrib, Hadoop >Affects Versions: 0.7 beta 1 >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Minor > Fix For: 0.7.4 > > Attachments: 0002-Fix-build-bin-script.txt, > 0003-StoreFunc_with_deletion.txt > > Original Estimate: 32h > Remaining Estimate: 32h > > Now that we have a ColumnFamilyOutputFormat, we can write data back to > cassandra in mapreduce jobs, however we can only do this in java. It would > be nice if pig could also output to cassandra. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1684) Entity groups
[ https://issues.apache.org/jira/browse/CASSANDRA-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002155#comment-13002155 ] Sylvain Lebresne commented on CASSANDRA-1684: - bq. Do tokens have to be one-to-one unique with keys, or could you have multiple keys share the same token? (apparently that's currently possible, although an extreme edge case, with the RandomPartitioner) Right now, they do have to be one-to-one. That's the 'raison d'être' of CASSANDRA-1034 (and I won't hide that my interest for the latter is motivated by this ticket, even though we should fix it because of RandomPartioner anyway). As for this ticket, I think using parts of the key for the token is only the first step (but an important one). The main thing we want here is to apply mutation on an entity group consistently, that is in one commit log transaction. That in turn is not very complicated in theory, but will be much more work in practice I believe. As a side note, I think it would also be nice to find "a trick" to make this work with the existing partitioners. Otherwise, since we can't change partitioners, this would make this useful for only new clusters, which would be sad. > Entity groups > - > > Key: CASSANDRA-1684 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1684 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: Sylvain Lebresne > Fix For: 0.8 > > Original Estimate: 80h > Remaining Estimate: 80h > > Supporting entity groups similar to App Engine's (that is, allow rows to be > part of a parent "entity group," whose key is used for routing instead of the > row itself) allows several improvements: > - batches within an EG can be atomic across multiple rows > - order-by-value queries within an EG only have to touch a single replica > even with RandomPartitioner -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[ https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002147#comment-13002147 ] Sylvain Lebresne commented on CASSANDRA-2231: - Yes that is also correct. > Add CompositeType comparer to the comparers provided in > org.apache.cassandra.db.marshal > --- > > Key: CASSANDRA-2231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2231 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Affects Versions: 0.7.3 >Reporter: Ed Anuff >Priority: Minor > Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, > 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip > > > CompositeType is a custom comparer that makes it possible to create > comparable composite values out of the basic types that Cassandra currently > supports, such as Long, UUID, etc. This is very useful in both the creation > of custom inverted indexes using columns in a skinny row, where each column > name is a composite value, and also when using Cassandra's built-in secondary > index support, where it can be used to encode the values in the columns that > Cassandra indexes. One scenario for the usage of these is documented here: > http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for > contribution is attached and has been previously maintained on github here: > https://github.com/edanuff/CassandraCompositeType -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[ https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002144#comment-13002144 ] Ed Anuff commented on CASSANDRA-2231: - bq.Greater-than is already doable in my previous patch (up to the bug in validation). For the less-than part, I agree that it is nice to be able to do it easily. In my new patch, I add a leading byte to each component, whose purpose is to always be 0, except for lesser-than query. That way, you can do the query above easily. The price is a slightly more complicated encoding but I think it's totally worth it. Just to be clear, the original idea was to make it possible to construct a key for the purposes of doing a range slice that would compare inclusive either or both at the start and finish of the range. This appears to be possible with the "inclusion byte" that you're using in lines 179 through 184 of your patch. Is that correct? > Add CompositeType comparer to the comparers provided in > org.apache.cassandra.db.marshal > --- > > Key: CASSANDRA-2231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2231 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Affects Versions: 0.7.3 >Reporter: Ed Anuff >Priority: Minor > Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, > 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip > > > CompositeType is a custom comparer that makes it possible to create > comparable composite values out of the basic types that Cassandra currently > supports, such as Long, UUID, etc. This is very useful in both the creation > of custom inverted indexes using columns in a skinny row, where each column > name is a composite value, and also when using Cassandra's built-in secondary > index support, where it can be used to encode the values in the columns that > Cassandra indexes. One scenario for the usage of these is documented here: > http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for > contribution is attached and has been previously maintained on github here: > https://github.com/edanuff/CassandraCompositeType -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-1034) Remove assumption that Key to Token is one-to-one
[ https://issues.apache.org/jira/browse/CASSANDRA-1034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-1034: Attachment: 0002-LengthPartitioner.patch 0001-Make-range-accept-both-Token-and-DecoratedKey.patch Attaching a patch that I believe solves this. It makes Range accept both Token and DecoratedKey and makes those two compare together correctly. It introduces a new marker interface (RingPosition) instead of making DecoratedKey extends Token (for reason explained in the comments of RingPosition but to sum up: I think it's cleaner). The second patch attached is just a stupid partitioner that use for token the length of the key. It's just for testing and not meant for inclusion. But this shows that with the first patch, you can do correct range query that go from 'the middle of a token' to the 'middle of another one'. An important note is that this breaks the serialization unit tests, because now an AbstractBounds can use decoratedKeys, and thus serialized AbstractBounds are incompatible with previous version. Not sure how to deal with that though, I though we had a plan for dealing with that but I'll admit I don't remember the details. > Remove assumption that Key to Token is one-to-one > - > > Key: CASSANDRA-1034 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1034 > Project: Cassandra > Issue Type: Bug >Reporter: Stu Hood >Assignee: Sylvain Lebresne >Priority: Minor > Attachments: > 0001-Make-range-accept-both-Token-and-DecoratedKey.patch, > 0002-LengthPartitioner.patch, 1034_v1.txt > > > get_range_slices assumes that Tokens do not collide and converts a KeyRange > to an AbstractBounds. For RandomPartitioner, this assumption isn't safe, and > would lead to a very weird heisenberg. > Converting AbstractBounds to use a DecoratedKey would solve this, because the > byte[] key portion of the DecoratedKey can act as a tiebreaker. > Alternatively, we could make DecoratedKey extend Token, and then use > DecoratedKeys in places where collisions are unacceptable. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1076729 [2/2] - in /cassandra/trunk: ./ contrib/ debian/ interface/thrift/gen-java/org/apache/cassandra/thrift/ src/java/org/apache/cassandra/db/ src/java/org/apache/cassandra/db/columnit
Modified:
cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java?rev=1076729&r1=1076728&r2=1076729&view=diff
==
---
cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
(original)
+++
cassandra/trunk/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
Thu Mar 3 19:11:17 2011
@@ -307,7 +307,7 @@ public class ColumnFamilyStoreTest exten
}
@Test
-public void testIndexCreate() throws IOException, ConfigurationException,
InterruptedException
+public void testIndexCreate() throws IOException, ConfigurationException,
InterruptedException, ExecutionException
{
Table table = Table.open("Keyspace1");
@@ -324,6 +324,9 @@ public class ColumnFamilyStoreTest exten
while (!SystemTable.isIndexBuilt("Keyspace1",
cfs.getIndexedColumnFamilyStore(ByteBufferUtil.bytes("birthdate")).columnFamily))
TimeUnit.MILLISECONDS.sleep(100);
+// we had a bug (CASSANDRA-2244) where index would get created but not
flushed -- check for that
+assert cfs.getIndexedColumnFamilyStore(cd.name).getSSTables().size() >
0;
+
IndexExpression expr = new
IndexExpression(ByteBufferUtil.bytes("birthdate"), IndexOperator.EQ,
ByteBufferUtil.bytes(1L));
IndexClause clause = new IndexClause(Arrays.asList(expr),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 100);
IFilter filter = new IdentityQueryFilter();
@@ -331,8 +334,7 @@ public class ColumnFamilyStoreTest exten
Range range = new Range(p.getMinimumToken(), p.getMinimumToken());
List rows = table.getColumnFamilyStore("Indexed2").scan(clause,
range, filter);
assert rows.size() == 1 : StringUtils.join(rows, ",");
-String key = new
String(rows.get(0).key.key.array(),rows.get(0).key.key.position(),rows.get(0).key.key.remaining());
-assert "k1".equals( key );
+assertEquals("k1", ByteBufferUtil.string(rows.get(0).key.key));
}
@Test
Modified: cassandra/trunk/test/unit/org/apache/cassandra/db/KeyCacheTest.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/db/KeyCacheTest.java?rev=1076729&r1=1076728&r2=1076729&view=diff
==
--- cassandra/trunk/test/unit/org/apache/cassandra/db/KeyCacheTest.java
(original)
+++ cassandra/trunk/test/unit/org/apache/cassandra/db/KeyCacheTest.java Thu Mar
3 19:11:17 2011
@@ -1,4 +1,25 @@
package org.apache.cassandra.db;
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
import java.io.IOException;
import java.util.concurrent.ExecutionException;
Modified: cassandra/trunk/test/unit/org/apache/cassandra/db/ScrubTest.java
URL:
http://svn.apache.org/viewvc/cassandra/trunk/test/unit/org/apache/cassandra/db/ScrubTest.java?rev=1076729&r1=1076728&r2=1076729&view=diff
==
--- cassandra/trunk/test/unit/org/apache/cassandra/db/ScrubTest.java (original)
+++ cassandra/trunk/test/unit/org/apache/cassandra/db/ScrubTest.java Thu Mar 3
19:11:17 2011
@@ -1,4 +1,25 @@
package org.apache.cassandra.db;
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the Licen
[jira] Created: (CASSANDRA-2265) Fix distributed tests after updating Guava to 08
Fix distributed tests after updating Guava to 08
Key: CASSANDRA-2265
URL: https://issues.apache.org/jira/browse/CASSANDRA-2265
Project: Cassandra
Issue Type: Sub-task
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
Distributed tests fail to start cluster:
{code}
[junit] java.lang.NoSuchMethodError:
com.google.common.net.InternetDomainName.isValid(Ljava/lang/String;)Z
[junit] at
org.jclouds.rest.binders.BindAsHostPrefix.bindToRequest(BindAsHostPrefix.java:52)
[junit] at
org.jclouds.aws.s3.binders.BindAsHostPrefixIfConfigured.bindToRequest(BindAsHostPrefixIfConfigured.java:67)
[junit] at
org.jclouds.rest.internal.RestAnnotationProcessor.decorateRequest(RestAnnotationProcessor.java:860)
[junit] at
org.jclouds.rest.internal.RestAnnotationProcessor.createRequest(RestAnnotationProcessor.java:477)
[junit] at
org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFuture(AsyncRestClientProxy.java:117)
[junit] at
org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:97)
[junit] at $Proxy58.objectExists(Unknown Source)
[junit] at
org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
[junit] at $Proxy59.objectExists(Unknown Source)
[junit] at
org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
[junit] at
org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
[junit] at
org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
[junit] at
org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
[junit] at
org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
[junit] at
org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
[junit] at
org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
[junit] at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
[junit] at
org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:134)
[junit] at $Proxy59.objectExists(Unknown Source)
[junit] at
org.jclouds.aws.s3.blobstore.S3BlobStore.blobExists(S3BlobStore.java:184)
[junit] at
org.jclouds.blobstore.internal.BaseBlobMap.containsKey(BaseBlobMap.java:201)
[junit] at
org.jclouds.blobstore.internal.InputStreamMapImpl.putInternal(InputStreamMapImpl.java:187)
[junit] at
org.jclouds.blobstore.internal.InputStreamMapImpl.putFile(InputStreamMapImpl.java:166)
[junit] at
org.apache.cassandra.utils.BlobUtils.storeBlob(BlobUtils.java:85)
[junit] at
org.apache.cassandra.CassandraServiceController.startup(CassandraServiceController.java:169)
[junit] at
org.apache.cassandra.CassandraServiceController.ensureClusterRunning(CassandraServiceController.java:236)
[junit] at org.apache.cassandra.TestBase.setUp(TestBase.java:140)
{code}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (CASSANDRA-1170) java.lang.NumberFormatException: For input string: "02473253("
[
https://issues.apache.org/jira/browse/CASSANDRA-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002108#comment-13002108
]
Joaquin Casares edited comment on CASSANDRA-1170 at 3/3/11 6:27 PM:
This was seen on a 0.6.8 cluster which is fairly fresh.
ERROR [ROW-READ-STAGE:45] 2011-03-02 10:45:39,634 CassandraDaemon.java (line
87) Uncaught exception in thread Thread[ROW-READ-STAGE:45,5,main]
java.lang.RuntimeException: java.lang.NumberFormatException: For input
string: "981834x22"
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:53)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NumberFormatException: For input string: "981834x22"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:458)
at java.math.BigInteger.(BigInteger.java:325)
at java.math.BigInteger.(BigInteger.java:451)
at
org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
at
org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
at
org.apache.cassandra.io.SSTableReader.getNearestPosition(SSTableReader.java:482)
at org.apache.cassandra.io.SSTableScanner.seekTo(SSTableScanner.java:62)
at
org.apache.cassandra.db.ColumnFamilyStore.getKeyRange(ColumnFamilyStore.java:1061)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1155)
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:41)
The Hector code that was used to run this was:
private void buildBatchMutation(BatchMutation batch,
String columnFamily,
String key, byte [] superColumnKey, byte
[] columnKey, byte [] columnVal)
{
List columnFamilies = new ArrayList();
columnFamilies.add(columnFamily);
if (columnVal == null)
{
Deletion deletion = new Deletion();
SlicePredicate slicePredicate = new SlicePredicate();
List columnList = new ArrayList();
columnList.add(columnKey);
slicePredicate.setColumn_names(columnList);
if (superColumnKey != null)
deletion.setSuper_column(superColumnKey);
deletion.predicate = slicePredicate;
deletion.setTimestamp(System.currentTimeMillis() * 1000);
batch.addDeletion(key, columnFamilies, deletion);
}
else
{
Column column = new Column();
column.setName(columnKey);
column.setValue(columnVal);
column.setTimestamp(System.currentTimeMillis() * 1000);
if (superColumnKey == null)
batch.addInsertion(key, columnFamilies, column);
else
{
SuperColumn superColumn = new SuperColumn();
List columns = new ArrayList();
columns.add(column);
superColumn.setColumns(columns);
superColumn.setName(superColumnKey);
batch.addSuperInsertion(key, columnFamilies, superColumn);
}
}
}
If this helps any. Properly formatted: http://pastebin.com/SGu9AuDV
was (Author: j.casares):
ERROR [ROW-READ-STAGE:45] 2011-03-02 10:45:39,634 CassandraDaemon.java
(line
87) Uncaught exception in thread Thread[ROW-READ-STAGE:45,5,main]
java.lang.RuntimeException: java.lang.NumberFormatException: For input
string: "981834x22"
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:53)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NumberFormatException: For input string: "981834x22"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:458)
at java.math.BigInteger.(BigInteger.java:325)
at java.math.BigInteger.(BigInteger.java:451)
at
org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
at
org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
at
org.apache.cassandra.io.SSTableReader.getNearestPosition(SSTableReader.java:482)
at org.apache.cassandra.io.SSTableScanner.seekTo(SSTableScanner.java:62)
at
org.apache.cassandra.db.ColumnFamilyStore.getKeyRange(ColumnFamilyStore.java:1061)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1155)
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:41)
This was seen on a 0.6.8 cluster which is fairly fresh.
The Hector code that was used to run thi
[jira] Updated: (CASSANDRA-1170) java.lang.NumberFormatException: For input string: "02473253("
[
https://issues.apache.org/jira/browse/CASSANDRA-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joaquin Casares updated CASSANDRA-1170:
---
Affects Version/s: (was: 0.6.8)
> java.lang.NumberFormatException: For input string: "02473253("
> --
>
> Key: CASSANDRA-1170
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1170
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.6.2
> Environment: Cassandra 0.6.2
> Ubuntu 9.10 on EC2
> java version "1.6.0_0"
> IcedTea6 1.3.1 (6b12-0ubuntu6.6) Runtime Environment (build 1.6.0_0-b12)
> OpenJDK 64-Bit Server VM (build 1.6.0_0-b12, mixed mode)
>Reporter: David King
>
> When trying to boot a node:
> INFO 11:05:16,117 Sampling index for
> /cassandra/data/permacache/permacache-13662-Data.db
> ERROR 11:05:17,389 Exception encountered during startup.
> java.lang.NumberFormatException: For input string: "72392391 "
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:481)
> at java.math.BigInteger.(BigInteger.java:343)
> at java.math.BigInteger.(BigInteger.java:467)
> at
> org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
> at
> org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
> at
> org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:178)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
> at org.apache.cassandra.db.Table.(Table.java:338)
> at org.apache.cassandra.db.Table.open(Table.java:199)
> at
> org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
> Exception encountered during startup.
> java.lang.NumberFormatException: For input string: "72392391 "
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:481)
> at java.math.BigInteger.(BigInteger.java:343)
> at java.math.BigInteger.(BigInteger.java:467)
> at
> org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
> at
> org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
> at
> org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:178)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
> at org.apache.cassandra.db.Table.(Table.java:338)
> at org.apache.cassandra.db.Table.open(Table.java:199)
> at
> org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
> I deleted the sstable with the broken key (RF==3, so I figured I could just
> repair when it came back up), but now instead I get:
> ERROR 11:16:02,045 Exception encountered during startup.
> java.lang.NumberFormatException: For input string: "02473253("
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:481)
> at java.math.BigInteger.(BigInteger.java:343)
> at java.math.BigInteger.(BigInteger.java:467)
> at
> org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
> at
> org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
> at
> org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:178)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
> at org.apache.cassandra.db.Table.(Table.java:338)
> at org.apache.cassandra.db.Table.open(Table.java:199)
> at
> org.apache.cassandra.thrift.CassandraDaemon.setup(C
[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002126#comment-13002126
]
Sylvain Lebresne commented on CASSANDRA-2231:
-
{quote}
Does this mean that I can have any number of components in my dynamic composite
key and that "b" and "t" are aliases to BytesType and TimeUUIDType for the
purpose of space efficienct? Using both or neither of those aliases is valid
and the order in which I use them isn't mandated, correct?
{quote}
That is correct. The aliases are here only for space efficiency. No obligation
of using them of any sort.
> Add CompositeType comparer to the comparers provided in
> org.apache.cassandra.db.marshal
> ---
>
> Key: CASSANDRA-2231
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
> Project: Cassandra
> Issue Type: Improvement
> Components: Contrib
>Affects Versions: 0.7.3
>Reporter: Ed Anuff
>Priority: Minor
> Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch,
> 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip
>
>
> CompositeType is a custom comparer that makes it possible to create
> comparable composite values out of the basic types that Cassandra currently
> supports, such as Long, UUID, etc. This is very useful in both the creation
> of custom inverted indexes using columns in a skinny row, where each column
> name is a composite value, and also when using Cassandra's built-in secondary
> index support, where it can be used to encode the values in the columns that
> Cassandra indexes. One scenario for the usage of these is documented here:
> http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for
> contribution is attached and has been previously maintained on github here:
> https://github.com/edanuff/CassandraCompositeType
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[ https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002118#comment-13002118 ] Ed Anuff commented on CASSANDRA-2231: - Sylvain, this looks like it could do the trick. Just to clarify, in your example, if I declare something like this: DynamicCompositeType(b => BytesType, t => TimeUUIDType) Does this mean that I can have any number of components in my dynamic composite key and that "b" and "t" are aliases to BytesType and TimeUUIDType for the purpose of space efficienct? Using both or neither of those aliases is valid and the order in which I use them isn't mandated, correct? > Add CompositeType comparer to the comparers provided in > org.apache.cassandra.db.marshal > --- > > Key: CASSANDRA-2231 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2231 > Project: Cassandra > Issue Type: Improvement > Components: Contrib >Affects Versions: 0.7.3 >Reporter: Ed Anuff >Priority: Minor > Attachments: 0001-Add-compositeType-and-DynamicCompositeType.patch, > 0001-Add-compositeType.patch, edanuff-CassandraCompositeType-1e253c4.zip > > > CompositeType is a custom comparer that makes it possible to create > comparable composite values out of the basic types that Cassandra currently > supports, such as Long, UUID, etc. This is very useful in both the creation > of custom inverted indexes using columns in a skinny row, where each column > name is a composite value, and also when using Cassandra's built-in secondary > index support, where it can be used to encode the values in the columns that > Cassandra indexes. One scenario for the usage of these is documented here: > http://www.anuff.com/2010/07/secondary-indexes-in-cassandra.html. Source for > contribution is attached and has been previously maintained on github here: > https://github.com/edanuff/CassandraCompositeType -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (CASSANDRA-1170) java.lang.NumberFormatException: For input string: "02473253("
[
https://issues.apache.org/jira/browse/CASSANDRA-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002108#comment-13002108
]
Joaquin Casares edited comment on CASSANDRA-1170 at 3/3/11 6:01 PM:
ERROR [ROW-READ-STAGE:45] 2011-03-02 10:45:39,634 CassandraDaemon.java (line
87) Uncaught exception in thread Thread[ROW-READ-STAGE:45,5,main]
java.lang.RuntimeException: java.lang.NumberFormatException: For input
string: "981834x22"
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:53)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NumberFormatException: For input string: "981834x22"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:458)
at java.math.BigInteger.(BigInteger.java:325)
at java.math.BigInteger.(BigInteger.java:451)
at
org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
at
org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
at
org.apache.cassandra.io.SSTableReader.getNearestPosition(SSTableReader.java:482)
at org.apache.cassandra.io.SSTableScanner.seekTo(SSTableScanner.java:62)
at
org.apache.cassandra.db.ColumnFamilyStore.getKeyRange(ColumnFamilyStore.java:1061)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1155)
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:41)
This was seen on a 0.6.8 cluster which is fairly fresh.
The Hector code that was used to run this was:
private void buildBatchMutation(BatchMutation batch,
String columnFamily,
String key, byte [] superColumnKey, byte
[] columnKey, byte [] columnVal)
{
List columnFamilies = new ArrayList();
columnFamilies.add(columnFamily);
if (columnVal == null)
{
Deletion deletion = new Deletion();
SlicePredicate slicePredicate = new SlicePredicate();
List columnList = new ArrayList();
columnList.add(columnKey);
slicePredicate.setColumn_names(columnList);
if (superColumnKey != null)
deletion.setSuper_column(superColumnKey);
deletion.predicate = slicePredicate;
deletion.setTimestamp(System.currentTimeMillis() * 1000);
batch.addDeletion(key, columnFamilies, deletion);
}
else
{
Column column = new Column();
column.setName(columnKey);
column.setValue(columnVal);
column.setTimestamp(System.currentTimeMillis() * 1000);
if (superColumnKey == null)
batch.addInsertion(key, columnFamilies, column);
else
{
SuperColumn superColumn = new SuperColumn();
List columns = new ArrayList();
columns.add(column);
superColumn.setColumns(columns);
superColumn.setName(superColumnKey);
batch.addSuperInsertion(key, columnFamilies, superColumn);
}
}
}
If this helps any. Properly formatted: http://pastebin.com/SGu9AuDV
was (Author: j.casares):
ERROR [ROW-READ-STAGE:45] 2011-03-02 10:45:39,634 CassandraDaemon.java
(line
87) Uncaught exception in thread Thread[ROW-READ-STAGE:45,5,main]
java.lang.RuntimeException: java.lang.NumberFormatException: For input
string: "981834x22"
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:53)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NumberFormatException: For input string: "981834x22"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:458)
at java.math.BigInteger.(BigInteger.java:325)
at java.math.BigInteger.(BigInteger.java:451)
at
org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
at
org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
at
org.apache.cassandra.io.SSTableReader.getNearestPosition(SSTableReader.java:482)
at org.apache.cassandra.io.SSTableScanner.seekTo(SSTableScanner.java:62)
at
org.apache.cassandra.db.ColumnFamilyStore.getKeyRange(ColumnFamilyStore.java:1061)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1155)
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:41)
This was seen on a 0.6.8 cluster which is fairly fresh.
The Hector code that was used to run thi
[jira] Commented: (CASSANDRA-1170) java.lang.NumberFormatException: For input string: "02473253("
[
https://issues.apache.org/jira/browse/CASSANDRA-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002108#comment-13002108
]
Joaquin Casares commented on CASSANDRA-1170:
ERROR [ROW-READ-STAGE:45] 2011-03-02 10:45:39,634 CassandraDaemon.java (line
87) Uncaught exception in thread Thread[ROW-READ-STAGE:45,5,main]
java.lang.RuntimeException: java.lang.NumberFormatException: For input
string: "981834x22"
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:53)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NumberFormatException: For input string: "981834x22"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:458)
at java.math.BigInteger.(BigInteger.java:325)
at java.math.BigInteger.(BigInteger.java:451)
at
org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
at
org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
at
org.apache.cassandra.io.SSTableReader.getNearestPosition(SSTableReader.java:482)
at org.apache.cassandra.io.SSTableScanner.seekTo(SSTableScanner.java:62)
at
org.apache.cassandra.db.ColumnFamilyStore.getKeyRange(ColumnFamilyStore.java:1061)
at
org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1155)
at
org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:41)
This was seen on a 0.6.8 cluster which is fairly fresh.
The Hector code that was used to run this was:
private void buildBatchMutation(BatchMutation batch,
String columnFamily,
String key, byte [] superColumnKey, byte
[] columnKey, byte [] columnVal)
{
List columnFamilies = new ArrayList();
columnFamilies.add(columnFamily);
if (columnVal == null)
{
Deletion deletion = new Deletion();
SlicePredicate slicePredicate = new SlicePredicate();
List columnList = new ArrayList();
columnList.add(columnKey);
slicePredicate.setColumn_names(columnList);
if (superColumnKey != null)
deletion.setSuper_column(superColumnKey);
deletion.predicate = slicePredicate;
deletion.setTimestamp(System.currentTimeMillis() * 1000);
batch.addDeletion(key, columnFamilies, deletion);
}
else
{
Column column = new Column();
column.setName(columnKey);
column.setValue(columnVal);
column.setTimestamp(System.currentTimeMillis() * 1000);
if (superColumnKey == null)
batch.addInsertion(key, columnFamilies, column);
else
{
SuperColumn superColumn = new SuperColumn();
List columns = new ArrayList();
columns.add(column);
superColumn.setColumns(columns);
superColumn.setName(superColumnKey);
batch.addSuperInsertion(key, columnFamilies, superColumn);
}
}
}
If this helps any.
> java.lang.NumberFormatException: For input string: "02473253("
> --
>
> Key: CASSANDRA-1170
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1170
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.6.2, 0.6.8
> Environment: Cassandra 0.6.2
> Ubuntu 9.10 on EC2
> java version "1.6.0_0"
> IcedTea6 1.3.1 (6b12-0ubuntu6.6) Runtime Environment (build 1.6.0_0-b12)
> OpenJDK 64-Bit Server VM (build 1.6.0_0-b12, mixed mode)
>Reporter: David King
>
> When trying to boot a node:
> INFO 11:05:16,117 Sampling index for
> /cassandra/data/permacache/permacache-13662-Data.db
> ERROR 11:05:17,389 Exception encountered during startup.
> java.lang.NumberFormatException: For input string: "72392391 "
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:481)
> at java.math.BigInteger.(BigInteger.java:343)
> at java.math.BigInteger.(BigInteger.java:467)
> at
> org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
> at
> org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
> at
> org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:178)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.ja
[jira] Updated: (CASSANDRA-1170) java.lang.NumberFormatException: For input string: "02473253("
[
https://issues.apache.org/jira/browse/CASSANDRA-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joaquin Casares updated CASSANDRA-1170:
---
Affects Version/s: 0.6.8
> java.lang.NumberFormatException: For input string: "02473253("
> --
>
> Key: CASSANDRA-1170
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1170
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.6.2, 0.6.8
> Environment: Cassandra 0.6.2
> Ubuntu 9.10 on EC2
> java version "1.6.0_0"
> IcedTea6 1.3.1 (6b12-0ubuntu6.6) Runtime Environment (build 1.6.0_0-b12)
> OpenJDK 64-Bit Server VM (build 1.6.0_0-b12, mixed mode)
>Reporter: David King
>
> When trying to boot a node:
> INFO 11:05:16,117 Sampling index for
> /cassandra/data/permacache/permacache-13662-Data.db
> ERROR 11:05:17,389 Exception encountered during startup.
> java.lang.NumberFormatException: For input string: "72392391 "
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:481)
> at java.math.BigInteger.(BigInteger.java:343)
> at java.math.BigInteger.(BigInteger.java:467)
> at
> org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
> at
> org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
> at
> org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:178)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
> at org.apache.cassandra.db.Table.(Table.java:338)
> at org.apache.cassandra.db.Table.open(Table.java:199)
> at
> org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
> Exception encountered during startup.
> java.lang.NumberFormatException: For input string: "72392391 "
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:481)
> at java.math.BigInteger.(BigInteger.java:343)
> at java.math.BigInteger.(BigInteger.java:467)
> at
> org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
> at
> org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
> at
> org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:178)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
> at org.apache.cassandra.db.Table.(Table.java:338)
> at org.apache.cassandra.db.Table.open(Table.java:199)
> at
> org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:91)
> at
> org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:177)
> I deleted the sstable with the broken key (RF==3, so I figured I could just
> repair when it came back up), but now instead I get:
> ERROR 11:16:02,045 Exception encountered during startup.
> java.lang.NumberFormatException: For input string: "02473253("
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:481)
> at java.math.BigInteger.(BigInteger.java:343)
> at java.math.BigInteger.(BigInteger.java:467)
> at
> org.apache.cassandra.dht.BigIntegerToken.(BigIntegerToken.java:32)
> at
> org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:53)
> at
> org.apache.cassandra.io.SSTableReader.loadIndexFile(SSTableReader.java:261)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:125)
> at org.apache.cassandra.io.SSTableReader.open(SSTableReader.java:114)
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:178)
> at
> org.apache.cassandra.db.ColumnFamilyStore.createColumnFamilyStore(ColumnFamilyStore.java:248)
> at org.apache.cassandra.db.Table.(Table.java:338)
> at org.apache.cassandra.db.Table.open(Table.java:199)
> at
> org.apache.cassandra.thrift.CassandraDaemon.setup(Cassa
[jira] Commented: (CASSANDRA-2244) secondary indexes aren't created on pre-existing or streamed data
[ https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002103#comment-13002103 ] Hudson commented on CASSANDRA-2244: --- Integrated in Cassandra-0.7 #342 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/342/]) CFS correctly flushes index CFs. Patch by jbellis, reviewed by brandonwilliams for CASSANDRA-2244 > secondary indexes aren't created on pre-existing or streamed data > - > > Key: CASSANDRA-2244 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2244 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.7.0 >Reporter: Brandon Williams >Assignee: Jonathan Ellis > Fix For: 0.7.4 > > Attachments: 2244-v2.txt > > > The repaired node neither receives indexes from the replicas, nor does it > generate them afterwards. The same bug prevents generation of new indexes > against existing data. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
svn commit: r1076700 - /cassandra/branches/cassandra-0.7/CHANGES.txt
Author: brandonwilliams Date: Thu Mar 3 17:42:16 2011 New Revision: 1076700 URL: http://svn.apache.org/viewvc?rev=1076700&view=rev Log: update CHANGES Modified: cassandra/branches/cassandra-0.7/CHANGES.txt Modified: cassandra/branches/cassandra-0.7/CHANGES.txt URL: http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/CHANGES.txt?rev=1076700&r1=1076699&r2=1076700&view=diff == --- cassandra/branches/cassandra-0.7/CHANGES.txt (original) +++ cassandra/branches/cassandra-0.7/CHANGES.txt Thu Mar 3 17:42:16 2011 @@ -1,5 +1,6 @@ 0.7.4 * add nodetool join command (CASSANDRA-2160) + * fix secondary indexes on pre-existing or streamed data (CASSANDRA-2244) 0.7.3
svn commit: r1076699 - in /cassandra/branches/cassandra-0.7: src/java/org/apache/cassandra/db/ColumnFamilyStore.java test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
Author: brandonwilliams
Date: Thu Mar 3 17:40:59 2011
New Revision: 1076699
URL: http://svn.apache.org/viewvc?rev=1076699&view=rev
Log:
CFS correctly flushes index CFs.
Patch by jbellis, reviewed by brandonwilliams for CASSANDRA-2244
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
Modified:
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java?rev=1076699&r1=1076698&r2=1076699&view=diff
==
---
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
(original)
+++
cassandra/branches/cassandra-0.7/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
Thu Mar 3 17:40:59 2011
@@ -685,8 +685,11 @@ public class ColumnFamilyStore implement
{
if (oldMemtable.isFrozen())
return null;
-
-if (DatabaseDescriptor.getCFMetaData(metadata.cfId) == null)
+
+boolean isDropped = isIndex()
+ ? DatabaseDescriptor.getCFMetaData(table.name,
getParentColumnfamily()) == null
+ :
DatabaseDescriptor.getCFMetaData(metadata.cfId) == null;
+if (isDropped)
return null; // column family was dropped. no point in
flushing.
assert memtable == oldMemtable;
@@ -2113,6 +2116,12 @@ public class ColumnFamilyStore implement
return partitioner instanceof LocalPartitioner;
}
+private String getParentColumnfamily()
+{
+assert isIndex();
+return columnFamily.split("\\.")[0];
+}
+
/**
* sets each cache's maximum capacity to 75% of its current size
*/
Modified:
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
URL:
http://svn.apache.org/viewvc/cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java?rev=1076699&r1=1076698&r2=1076699&view=diff
==
---
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
(original)
+++
cassandra/branches/cassandra-0.7/test/unit/org/apache/cassandra/db/ColumnFamilyStoreTest.java
Thu Mar 3 17:40:59 2011
@@ -307,7 +307,7 @@ public class ColumnFamilyStoreTest exten
}
@Test
-public void testIndexCreate() throws IOException, ConfigurationException,
InterruptedException
+public void testIndexCreate() throws IOException, ConfigurationException,
InterruptedException, ExecutionException
{
Table table = Table.open("Keyspace1");
@@ -324,6 +324,9 @@ public class ColumnFamilyStoreTest exten
while (!SystemTable.isIndexBuilt("Keyspace1",
cfs.getIndexedColumnFamilyStore(ByteBufferUtil.bytes("birthdate")).columnFamily))
TimeUnit.MILLISECONDS.sleep(100);
+// we had a bug (CASSANDRA-2244) where index would get created but not
flushed -- check for that
+assert cfs.getIndexedColumnFamilyStore(cd.name).getSSTables().size() >
0;
+
IndexExpression expr = new
IndexExpression(ByteBufferUtil.bytes("birthdate"), IndexOperator.EQ,
ByteBufferUtil.bytes(1L));
IndexClause clause = new IndexClause(Arrays.asList(expr),
ByteBufferUtil.EMPTY_BYTE_BUFFER, 100);
IFilter filter = new IdentityQueryFilter();
@@ -331,8 +334,7 @@ public class ColumnFamilyStoreTest exten
Range range = new Range(p.getMinimumToken(), p.getMinimumToken());
List rows = table.getColumnFamilyStore("Indexed2").scan(clause,
range, filter);
assert rows.size() == 1 : StringUtils.join(rows, ",");
-String key = new
String(rows.get(0).key.key.array(),rows.get(0).key.key.position(),rows.get(0).key.key.remaining());
-assert "k1".equals( key );
+assertEquals("k1", ByteBufferUtil.string(rows.get(0).key.key));
}
@Test
[jira] Commented: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002089#comment-13002089
]
Sylvain Lebresne commented on CASSANDRA-2262:
-
I don't think that matter for the purpose of this ticket, but as a remark,
IntegerType doesn't always roundtrip correctly. For example
{noformat}
@Test
public void testInteger2()
{
long v = 1;
ByteBuffer bb = ByteBuffer.allocate(8);
bb.putLong(v);
bb.flip();
String s = IntegerType.instance.getString(bb);
ByteBuffer internal = IntegerType.instance.fromString(s);
assert internal.getLong() == v;
}
{noformat}
fails (while it works on LongType) because of the internal use of BigInteger
that pack the bytes as tightly as possible.
This is probably fine since
{noformat}
IntegerType.getString(IntegerType.fromString(s)) == s
{noformat}
always holds. It's
{noformat}
IntegerType.fromString(IntegerType.getString(b)) == b
{noformat}
that doesn't always hold.
Last remark, the tests of the first patch are testing the wrong equation.
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v1-0002-BytesType.fromString-expects-a-hex-string.txt,
> v1-0003-compose-method-for-AbstractTypes.txt,
> v1-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)
[
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002084#comment-13002084
]
Jonathan Ellis commented on CASSANDRA-2228:
---
My point was that we can call put before initializing localEndpoint_ to
FBU.gLA, so it's naturally going to be null.
> Race conditions when reinitialisating nodes (OOM + Nullpointer)
> ---
>
> Key: CASSANDRA-2228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.7.2
>Reporter: Thibaut
>Assignee: Gary Dusbabek
> Fix For: 0.7.4
>
>
> I had a corrupt system table which wouldn't compact anymore and I deleted the
> files and restarted cassandra and let it take the same token/ip address.
> I experienced the same errors when I'm adding a newly installed node under
> the same token/ip address before calling repair.
> 1)
> After a few seconds/minutes, I get a OOM error:
> INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164)
> Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133)
> Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace:
> table_userentriesrep factor:3rep
> strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
>
> org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Migrations at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1
> operations)
> INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157)
> Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Schema at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
> INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89)
> table_sourcedetection.table_sourcedetection 0,0
> 0/00/20
> ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java
> (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:39)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
> at
> org.apache.cassandra.io.util.BufferedRandomAccessFile.(BufferedRandomAccessFile.java:126)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.(SSTableWriter.java:75)
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
> at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
> at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2) If I restart then, I'm getting an Nullpointer
[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-2262:
--
Reviewer: slebresne
I take it compose() is "give me a Java object that this value represents?"
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v1-0002-BytesType.fromString-expects-a-hex-string.txt,
> v1-0003-compose-method-for-AbstractTypes.txt,
> v1-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)
[
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002080#comment-13002080
]
Gary Dusbabek commented on CASSANDRA-2228:
--
CHM throws NPE when the key is null. To me, then, the bug is that
FBU.getLocalAddress() is somehow returning null.
I checked yesterday: even if local_address is unspecified, it ends up returning
localhost.
> Race conditions when reinitialisating nodes (OOM + Nullpointer)
> ---
>
> Key: CASSANDRA-2228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.7.2
>Reporter: Thibaut
>Assignee: Gary Dusbabek
> Fix For: 0.7.4
>
>
> I had a corrupt system table which wouldn't compact anymore and I deleted the
> files and restarted cassandra and let it take the same token/ip address.
> I experienced the same errors when I'm adding a newly installed node under
> the same token/ip address before calling repair.
> 1)
> After a few seconds/minutes, I get a OOM error:
> INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164)
> Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133)
> Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace:
> table_userentriesrep factor:3rep
> strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
>
> org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Migrations at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1
> operations)
> INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157)
> Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Schema at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
> INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89)
> table_sourcedetection.table_sourcedetection 0,0
> 0/00/20
> ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java
> (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:39)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
> at
> org.apache.cassandra.io.util.BufferedRandomAccessFile.(BufferedRandomAccessFile.java:126)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.(SSTableWriter.java:75)
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
> at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
> at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Dusbabek updated CASSANDRA-2262:
-
Attachment: v1-0004-assume-utf8-in-CliTest-keys-dammit.txt
v1-0003-compose-method-for-AbstractTypes.txt
v1-0002-BytesType.fromString-expects-a-hex-string.txt
v1-0001-test-shows-no-roundtrip-in-BytesType.txt
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v1-0002-BytesType.fromString-expects-a-hex-string.txt,
> v1-0003-compose-method-for-AbstractTypes.txt,
> v1-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2244) secondary indexes aren't created on pre-existing or streamed data
[ https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002073#comment-13002073 ] Jonathan Ellis commented on CASSANDRA-2244: --- Gary points out that migration does acquire the flushlock. > secondary indexes aren't created on pre-existing or streamed data > - > > Key: CASSANDRA-2244 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2244 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.7.0 >Reporter: Brandon Williams >Assignee: Jonathan Ellis > Fix For: 0.7.4 > > Attachments: 2244-v2.txt > > > The repaired node neither receives indexes from the replicas, nor does it > generate them afterwards. The same bug prevents generation of new indexes > against existing data. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2244) secondary indexes aren't created on pre-existing or streamed data
[ https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2244: -- Attachment: 2244-v2.txt patch adds test that catches the problem, and adds fix to CFS. > secondary indexes aren't created on pre-existing or streamed data > - > > Key: CASSANDRA-2244 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2244 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.7.0 >Reporter: Brandon Williams >Assignee: Jonathan Ellis > Fix For: 0.7.4 > > Attachments: 2244-v2.txt > > > The repaired node neither receives indexes from the replicas, nor does it > generate them afterwards. The same bug prevents generation of new indexes > against existing data. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2262) use o.a.c.marshal.*Type for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-2262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Dusbabek updated CASSANDRA-2262:
-
Attachment: (was: v0-0001-test-shows-no-roundtrip-in-BytesType.txt)
> use o.a.c.marshal.*Type for CQL
>
>
> Key: CASSANDRA-2262
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2262
> Project: Cassandra
> Issue Type: Sub-task
> Components: API
>Affects Versions: 0.8
>Reporter: Eric Evans
>Assignee: Gary Dusbabek
>Priority: Minor
> Fix For: 0.8
>
> Attachments: v1-0001-test-shows-no-roundtrip-in-BytesType.txt,
> v1-0002-BytesType.fromString-expects-a-hex-string.txt,
> v1-0003-compose-method-for-AbstractTypes.txt,
> v1-0004-assume-utf8-in-CliTest-keys-dammit.txt
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> Presently, {{o.a.c.cql.Term.getByteBuffer}} manage's the conversion from a
> parsed term to a {{ByteBuffer}} of the right contents, this should be moved
> into the individual {{AbstractType}} subclasses (aka
> {{AbstractType.fromString}}).
> Additionally, a method that does the inverse (returning the string
> equivalent), needs to exists such that
> {{AbstractType.getString(AbstractType.fromString(s)) == s}}
> Finally, for use in results decoding a method should exist that given a
> {{ByteBuffer}} will return the appropriate object for that type. For
> example, {{IntegerType.compose(ByteBuffer) -> java.math.BigInteger}}, or
> {{LexicalUUIDType.compose(ByteBuffer) -> java.util.UUID}}.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2244) secondary indexes aren't created on pre-existing or streamed data
[ https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2244: -- Attachment: (was: 2244.txt) > secondary indexes aren't created on pre-existing or streamed data > - > > Key: CASSANDRA-2244 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2244 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.7.0 >Reporter: Brandon Williams >Assignee: Jonathan Ellis > Fix For: 0.7.4 > > > The repaired node neither receives indexes from the replicas, nor does it > generate them afterwards. The same bug prevents generation of new indexes > against existing data. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2244) secondary indexes aren't created on pre-existing or streamed data
[ https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2244: -- Description: The repaired node neither receives indexes from the replicas, nor does it generate them afterwards. The same bug prevents generation of new indexes against existing data. (was: The repaired node neither receives indexes from the replicas, nor does it generate them afterwards.) Summary: secondary indexes aren't created on pre-existing or streamed data (was: repair doesn't handle secondary indexes) > secondary indexes aren't created on pre-existing or streamed data > - > > Key: CASSANDRA-2244 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2244 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.7.0 >Reporter: Brandon Williams >Assignee: Jonathan Ellis > Fix For: 0.7.4 > > Attachments: 2244.txt > > > The repaired node neither receives indexes from the replicas, nor does it > generate them afterwards. The same bug prevents generation of new indexes > against existing data. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2231) Add CompositeType comparer to the comparers provided in org.apache.cassandra.db.marshal
[
https://issues.apache.org/jira/browse/CASSANDRA-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-2231:
Attachment: 0001-Add-compositeType-and-DynamicCompositeType.patch
bq. (1) having validation only require that the components, if they're
provided, are of the correct type (looking at Sylvain's code, the compare
function should work, but the validation wouldn't)
That's a bug. The last test of validate() should be negated. The only goal was
forbid having leading bytes after all the comparator have been used (because
compare makes the assumption that there will not have bytes without a proper
comparator for them).
However it is clear that we should allow a key having less component than
comparator (btw, the unit test in my patch was actually doing it).
This is fixed in the patch I'm attaching.
bq. My other concern is that one of the things that got stripped out was the
MATCH_MINIMUM, MATCH_MAXIMUM feature, which made implementing
greater-than-equals, less-than-equals, etc. in ranges much easier
I'm not familiar enough with your code to be sure what those were doing
exactly, but I suppose that the goal is that, if for instance you have the
following column names:
{noformat}
test:bar:42
test:foo:24
test:foo:42
test:foobar
{noformat}
then you want to be able to query anything that starts with the two component
{{test}} and {{foo}} (that is {{test:foo:24}} and {{test:foo:42}} in my
example).
Greater-than is already doable in my previous patch (up to the bug in
validation). For the less-than part, I agree that it is nice to be able to do
it easily. In my new patch, I add a leading byte to each component, whose
purpose is to always be 0, except for lesser-than query. That way, you can do
the query above easily. The price is a slightly more complicated encoding but I
think it's totally worth it.
bq. While i agree in theory, this is why thrift exists right?
I don't know thrift as well as you do, but I would think that changing the type
of column name would be a massive compatibility breaker. And I really don't see
a not crappy way to do it. Well, actually I do, but it consists in revamping
the API completely, ditching the super columns in favor of composite column
name. Don't get me wrong, I'm 100% in, but it's well beyond this particular
ticket and I think that as long as we have a generic and simple enough encoding
for those composite type, I believe it's ok to include this as is and just have
a decent description of the encoding for client to use. We can still branch
that later on on thrift anyway.
bq. (2) creating a DynamicType that could be used for dynamic types
I do agree with you that it could be useful and I understand the use case. I
also agree that whatever we do, we should make it clear that it is not beginner
stuff and that you shouldn't use a dynamic composite type unless you really
need it (since the possibility of shooting yourself in the foot if you don't
know what you do is fairly high). All this being said, I took a stab at a
DynamicCompositeType and I think the result really makes sense.
The main idea is really just to modify my static CompositeType so that each
component starts with the name of the comparator class (that way, this is again
completely generic, custom comparator can be used without problems). There is
then 2 optimisations to make it a reasonable solution:
# FBUtilities.getComparator() uses a cache, to avoid the cost of
introspection.
# There is a notion of alias. They are user defined and are local to each
DynamicCompositeType (cf. test/conf/cassandra.yaml in the patch). An alias is a
simple (printable) ascii character (there is around 100 possible alias which
should be aplenty). You can then use those alias in the encoded column name, in
which case you only need 2 bytes to encode the comparator, making it fairly
compact.
The encoded format is very close to the one of the static CompositeType, even
though it is a tiny bit more complex due to the aliases handling. But given
that dynamic composite type is not meant for the faint of heart, that should be
fine.
I'm attaching a new patch (against 0.7 branch) with both the static and dynamic
composite type. The bulk of the code is shared between them. This also include
a fair amount of unit tests.
> Add CompositeType comparer to the comparers provided in
> org.apache.cassandra.db.marshal
> ---
>
> Key: CASSANDRA-2231
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2231
> Project: Cassandra
> Issue Type: Improvement
> Components: Contrib
>Affects Versions: 0.7.3
>Reporter: Ed Anuff
>Priority: Minor
> Attachments: 0001-Add-compositeType-and-DynamicCompositeT
[jira] Updated: (CASSANDRA-2244) repair doesn't handle secondary indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-2244: -- Attachment: 2244.txt The "don't bother flushing dropped CFs" check was unaware that index CFs are never part of the global CF metadata. (I think that check is race-prone to begin with, so we should probably drop it and do something more robust, but for now it's better than nothing -- it will *usually* prevent creating new sstables for dropped CFs.) > repair doesn't handle secondary indexes > --- > > Key: CASSANDRA-2244 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2244 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 0.7.0 >Reporter: Brandon Williams >Assignee: Jonathan Ellis > Fix For: 0.7.4 > > Attachments: 2244.txt > > > The repaired node neither receives indexes from the replicas, nor does it > generate them afterwards. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2256) BRAF assertion error
[
https://issues.apache.org/jira/browse/CASSANDRA-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002049#comment-13002049
]
Hudson commented on CASSANDRA-2256:
---
Integrated in Cassandra-0.7 #341 (See
[https://hudson.apache.org/hudson/job/Cassandra-0.7/341/])
throw EOFException when seeking past EOF in read-only mode
patch by Pavel Yaskevich; reviewed by tjake and jbellis for CASSANDRA-2256
> BRAF assertion error
>
>
> Key: CASSANDRA-2256
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2256
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.7.3
>Reporter: Jonathan Ellis
>Assignee: Pavel Yaskevich
> Fix For: 0.7.4
>
> Attachments: CASSANDRA-2256-v2.patch, CASSANDRA-2256.patch
>
>
> While investigating CASSANDRA-2240 I ran into this:
> {noformat}
> java.lang.AssertionError
> at
> org.apache.cassandra.io.util.BufferedRandomAccessFile.read(BufferedRandomAccessFile.java\
> :230)
> at java.io.RandomAccessFile.readByte(RandomAccessFile.java:589)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readShortLength(ByteBufferUtil.java:273)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readWithShortLength(ByteBufferUtil.java:284)
> at
> org.apache.cassandra.db.CompactionManager.doScrub(CompactionManager.java:539)
> {noformat}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2176) Add configuration setting to cap the number of Thrift connections
[ https://issues.apache.org/jira/browse/CASSANDRA-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002042#comment-13002042 ] T Jake Luciani commented on CASSANDRA-2176: --- LGTM +1 > Add configuration setting to cap the number of Thrift connections > - > > Key: CASSANDRA-2176 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2176 > Project: Cassandra > Issue Type: Improvement > Components: API >Reporter: Jonathan Ellis >Assignee: T Jake Luciani >Priority: Minor > Fix For: 0.7.4 > > Attachments: > 0001-CASSANDRA-2176-limit-connected-clients-and-config.txt, 2176-v2.txt > > Original Estimate: 4h > Remaining Estimate: 4h > > At least until CASSANDRA-1405 is done, it's useful to have a connection cap > to prevent misbehaving clients from DOSing the server. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2228) Race conditions when reinitialisating nodes (OOM + Nullpointer)
[
https://issues.apache.org/jira/browse/CASSANDRA-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13002037#comment-13002037
]
Jonathan Ellis commented on CASSANDRA-2228:
---
It looks like we're doing Migration stuff before Gossiper.start is called.
One possible solution: initialize Gossiper.localEndpoint in constructor instead
of in start.
> Race conditions when reinitialisating nodes (OOM + Nullpointer)
> ---
>
> Key: CASSANDRA-2228
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2228
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 0.7.2
>Reporter: Thibaut
>Assignee: Gary Dusbabek
> Fix For: 0.7.4
>
>
> I had a corrupt system table which wouldn't compact anymore and I deleted the
> files and restarted cassandra and let it take the same token/ip address.
> I experienced the same errors when I'm adding a newly installed node under
> the same token/ip address before calling repair.
> 1)
> After a few seconds/minutes, I get a OOM error:
> INFO [FlushWriter:1] 2011-02-23 16:40:28,958 Memtable.java (line 164)
> Completed flushing /cassandra/data/system/Schema-f-15-Data.db (8037 bytes)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 Migration.java (line 133)
> Applying migration 3e30e76b-1e3f-11e0-8369-5a9c1faed4ae Add keyspace:
> table_userentriesrep factor:3rep
> strategy:SimpleStrategy{org.apache.cassandra.config.CFMetaData@58925d9[cfId=1024,tableName=table_userentries,cfName=table_userentries,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}],
>
> org.apache.cassandra.config.CFMetaData@11ab7246[cfId=1025,tableName=table_userentries,cfName=table_userentries_meta,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType@b44dff0,subcolumncomparator=,comment=,rowCacheSize=0.0,keyCacheSize=20.0,readRepairChance=0.0,gcGraceSeconds=86400,defaultValidator=org.apache.cassandra.db.marshal.BytesType@b44dff0,minCompactionThreshold=4,maxCompactionThreshold=32,rowCacheSavePeriodInSeconds=0,keyCacheSavePeriodInSeconds=3600,memtableFlushAfterMins=60,memtableThroughputInMb=64,memtableOperationsInMillions=10.0,column_metadata={}]}
> INFO [MigrationStage:1] 2011-02-23 16:40:28,965 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Migrations at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Migrations@2121008793(12529 bytes, 1
> operations)
> INFO [FlushWriter:1] 2011-02-23 16:40:28,966 Memtable.java (line 157)
> Writing Memtable-Migrations@2121008793(12529 bytes, 1 operations)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,966 ColumnFamilyStore.java (line
> 666) switching in a fresh Memtable for Schema at
> CommitLogContext(file='/cassandra/commitlog/CommitLog-1298475572022.log',
> position=226075)
> INFO [MigrationStage:1] 2011-02-23 16:40:28,967 ColumnFamilyStore.java (line
> 977) Enqueuing flush of Memtable-Schema@139610466(8370 bytes, 15 operations)
> INFO [ScheduledTasks:1] 2011-02-23 16:40:28,972 StatusLogger.java (line 89)
> table_sourcedetection.table_sourcedetection 0,0
> 0/00/20
> ERROR [FlushWriter:1] 2011-02-23 16:41:01,240 AbstractCassandraDaemon.java
> (line 114) Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.(HeapByteBuffer.java:39)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
> at
> org.apache.cassandra.io.util.BufferedRandomAccessFile.(BufferedRandomAccessFile.java:126)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.(SSTableWriter.java:75)
> at
> org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:158)
> at org.apache.cassandra.db.Memtable.access$000(Memtable.java:51)
> at org.apache.cassandra.db.Memtable$1.runMayThrow(Memtable.java:176)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
