[jira] [Commented] (CASSANDRA-5374) Errors while opening SSTables on Cassandra start after an outage
[
https://issues.apache.org/jira/browse/CASSANDRA-5374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611722#comment-13611722
]
Ivan Sobolev commented on CASSANDRA-5374:
-
I've checked now - can see no tmp files in that CF's folder. Think it's
impossible to have some for that SSTable, because tmp files are cleaned up on
startup(just a guess).
I've captured -CompressionInfo.db, -Data.db, -Filter.db, -Index.db,
-Statistics.db for that SSTable. Unfortunately, they contain clients data and
thus I can't share them, but will be glad to run any checks you'll suggest.
Errors while opening SSTables on Cassandra start after an outage
Key: CASSANDRA-5374
URL: https://issues.apache.org/jira/browse/CASSANDRA-5374
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.1.9
Reporter: Ivan Sobolev
Getting plenty of those - is that normal? I thought *tmp files Cassandra
generates should guarantee all SSTables will be the ones written in proper
state.
{code}ERROR [SSTableBatchOpen:3] 2013-03-21 19:28:34,030
AbstractCassandraDaemon.java (line 135) Exception in thread
Thread[SSTableBatchOpen:3,5,main]
java.io.IOError: java.io.EOFException
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:63)
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.complete(CompressedSegmentedFile.java:55)
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:429)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:200)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:153)
at
org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:242)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.EOFException
at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
at java.io.DataInputStream.readUTF(DataInputStream.java:572)
at java.io.DataInputStream.readUTF(DataInputStream.java:547)
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:74)
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:59)
... 11 more{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5374) Errors while opening SSTables on Cassandra start after an outage
[
https://issues.apache.org/jira/browse/CASSANDRA-5374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611723#comment-13611723
]
Ivan Sobolev commented on CASSANDRA-5374:
-
I thought this could be somehow related to our migration 1.1.6--1.1.9 3 weeks
ago and some SSTables in old format, but the logging message concerning this
SSTable says it was created after the migration:
{code}INFO [CompactionExecutor:18551] 2013-03-20 20:38:46,657
CompactionTask.java (line 230) Compacted to
[/keyspace/cf/keyspace-cf-hf-9591-Data.db,/keyspace/cf/keyspace-cf-hf-9593-Data.db,/keyspace/cf/keyspace-cf-hf-9594-Data.db,/keyspace/cf/keyspace-cf-hf-9595-Data.db,/keyspace/cf/keyspace-cf-hf-9596-Data.db,/keyspace/cf/keyspace-cf-hf-9597-Data.db,/keyspace/cf/keyspace-cf-hf-9598-Data.db,].
44,032,116 to 40,625,649 (~92% of original) bytes for 24,625 keys at
0.995290MB/s. Time: 38,927ms.
INFO [CompactionExecutor:18551] 2013-03-20 20:38:46,658 CompactionTask.java
(line 107) Compacting
[SSTableReader(path='/keyspace/cf/keyspace-cf-hf-9060-Data.db'),
SSTableReader(path='/keyspace/cf/keyspace-cf-hf-9058-Data.db'),
SSTableReader(path='/keyspace/cf/keyspace-cf-hf-9063-Data.db'),
SSTableReader(path='/keyspace/cf/keyspace-cf-hf-9554-Data.db'),
SSTableReader(path='/keyspace/cf/keyspace-cf-hf-9059-Data.db'),
SSTableReader(path='/keyspace/cf/keyspace-cf-hf-9061-Data.db'),
SSTableReader(path='/keyspace/cf/keyspace-cf-hf-9062-Data.db')]{code}
Errors while opening SSTables on Cassandra start after an outage
Key: CASSANDRA-5374
URL: https://issues.apache.org/jira/browse/CASSANDRA-5374
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.1.9
Reporter: Ivan Sobolev
Getting plenty of those - is that normal? I thought *tmp files Cassandra
generates should guarantee all SSTables will be the ones written in proper
state.
{code}ERROR [SSTableBatchOpen:3] 2013-03-21 19:28:34,030
AbstractCassandraDaemon.java (line 135) Exception in thread
Thread[SSTableBatchOpen:3,5,main]
java.io.IOError: java.io.EOFException
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:63)
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.complete(CompressedSegmentedFile.java:55)
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:429)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:200)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:153)
at
org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:242)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.EOFException
at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
at java.io.DataInputStream.readUTF(DataInputStream.java:572)
at java.io.DataInputStream.readUTF(DataInputStream.java:547)
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:74)
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:59)
... 11 more{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5374) Errors while opening SSTables on Cassandra start after an outage
Ivan Sobolev created CASSANDRA-5374:
---
Summary: Errors while opening SSTables on Cassandra start after an
outage
Key: CASSANDRA-5374
URL: https://issues.apache.org/jira/browse/CASSANDRA-5374
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.1.9
Reporter: Ivan Sobolev
Getting plenty of those - is that normal? I thought *tmp files Cassandra
generates should guarantee all SSTables will be the ones written in proper
state.
{code}ERROR [SSTableBatchOpen:3] 2013-03-21 19:28:34,030
AbstractCassandraDaemon.java (line 135) Exception in thread
Thread[SSTableBatchOpen:3,5,main]
java.io.IOError: java.io.EOFException
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:63)
at
org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.complete(CompressedSegmentedFile.java:55)
at
org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:429)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:200)
at
org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:153)
at
org.apache.cassandra.io.sstable.SSTableReader$1.run(SSTableReader.java:242)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.EOFException
at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
at java.io.DataInputStream.readUTF(DataInputStream.java:572)
at java.io.DataInputStream.readUTF(DataInputStream.java:547)
at
org.apache.cassandra.io.compress.CompressionMetadata.init(CompressionMetadata.java:74)
at
org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:59)
... 11 more{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2698) Instrument repair to be able to assess it's efficiency (precision)
[ https://issues.apache.org/jira/browse/CASSANDRA-2698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13575928#comment-13575928 ] Ivan Sobolev commented on CASSANDRA-2698: - Hi, guys, it appears, the thing would be useful for me too. I'm going to give it a try. I'll follow the plan summarized by [~jbellis]. If you feel that I should know something you know - please shout :) Instrument repair to be able to assess it's efficiency (precision) -- Key: CASSANDRA-2698 URL: https://issues.apache.org/jira/browse/CASSANDRA-2698 Project: Cassandra Issue Type: Improvement Reporter: Sylvain Lebresne Priority: Minor Labels: lhf Attachments: nodetool_repair_and_cfhistogram.tar.gz Some reports indicate that repair sometime transfer huge amounts of data. One hypothesis is that the merkle tree precision may deteriorate too much at some data size. To check this hypothesis, it would be reasonably to gather statistic during the merkle tree building of how many rows each merkle tree range account for (and the size that this represent). It is probably an interesting statistic to have anyway. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-2698) Instrument repair to be able to assess it's efficiency (precision)
[ https://issues.apache.org/jira/browse/CASSANDRA-2698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sobolev updated CASSANDRA-2698: Attachment: patch_2698_v1.txt Instrument repair to be able to assess it's efficiency (precision) -- Key: CASSANDRA-2698 URL: https://issues.apache.org/jira/browse/CASSANDRA-2698 Project: Cassandra Issue Type: Improvement Reporter: Sylvain Lebresne Priority: Minor Labels: lhf Attachments: nodetool_repair_and_cfhistogram.tar.gz, patch_2698_v1.txt Some reports indicate that repair sometime transfer huge amounts of data. One hypothesis is that the merkle tree precision may deteriorate too much at some data size. To check this hypothesis, it would be reasonably to gather statistic during the merkle tree building of how many rows each merkle tree range account for (and the size that this represent). It is probably an interesting statistic to have anyway. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5222) OOM Exception during repair session with LeveledCompactionStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sobolev updated CASSANDRA-5222: Attachment: sstablescanner.png OOM Exception during repair session with LeveledCompactionStrategy -- Key: CASSANDRA-5222 URL: https://issues.apache.org/jira/browse/CASSANDRA-5222 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.6 Environment: 3Gb Heap(12Gb per node RAM) 36 nodes, 0.9 Tb of data per node, Leveled compaction strategy, SSTable size =100Mb Reporter: Ivan Sobolev Priority: Critical Attachments: sstablescanner.png 1.8 Gb of heap is consumed with 12k SSTableBoundedScanner * 140kbytes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5222) OOM Exception during repair session with LeveledCompactionStrategy
Ivan Sobolev created CASSANDRA-5222: --- Summary: OOM Exception during repair session with LeveledCompactionStrategy Key: CASSANDRA-5222 URL: https://issues.apache.org/jira/browse/CASSANDRA-5222 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.6 Environment: 3Gb Heap(12Gb per node RAM) 36 nodes, 0.9 Tb of data per node, Leveled compaction strategy, SSTable size =100Mb Reporter: Ivan Sobolev Priority: Critical Attachments: sstablescanner.png 1.8 Gb of heap is consumed with 12k SSTableBoundedScanner * 140kbytes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5222) OOM Exception during repair session with LeveledCompactionStrategy
[
https://issues.apache.org/jira/browse/CASSANDRA-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571299#comment-13571299
]
Ivan Sobolev commented on CASSANDRA-5222:
-
Seems, all sstables appear there:
{quote}ls | grep Data | wc -l
10760{quote}
OOM Exception during repair session with LeveledCompactionStrategy
--
Key: CASSANDRA-5222
URL: https://issues.apache.org/jira/browse/CASSANDRA-5222
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.1.6
Environment: 3Gb Heap(12Gb per node RAM)
36 nodes, 0.9 Tb of data per node, Leveled compaction strategy, SSTable size
=100Mb
Reporter: Ivan Sobolev
Priority: Critical
Attachments: sstablescanner.png
1.8 Gb of heap is consumed with 12k SSTableBoundedScanner * 140kbytes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5222) OOM Exception during repair session with LeveledCompactionStrategy
[
https://issues.apache.org/jira/browse/CASSANDRA-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571301#comment-13571301
]
Ivan Sobolev commented on CASSANDRA-5222:
-
{quote}Sounds to me like you're not actually running 1.1.6.
[root@da1-node30 chunks]# nodetool -h `hostname` version
ReleaseVersion: 1.1.6
That's DataStax version and it may differ a bit - not sure how do the versions
correlate.
OOM Exception during repair session with LeveledCompactionStrategy
--
Key: CASSANDRA-5222
URL: https://issues.apache.org/jira/browse/CASSANDRA-5222
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.1.6
Environment: 3Gb Heap(12Gb per node RAM)
36 nodes, 0.9 Tb of data per node, Leveled compaction strategy, SSTable size
=100Mb
Reporter: Ivan Sobolev
Priority: Critical
Attachments: sstablescanner.png
1.8 Gb of heap is consumed with 12k SSTableBoundedScanner * 140kbytes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5222) OOM Exception during repair session with LeveledCompactionStrategy
[
https://issues.apache.org/jira/browse/CASSANDRA-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571301#comment-13571301
]
Ivan Sobolev edited comment on CASSANDRA-5222 at 2/5/13 1:48 PM:
-
{quote}Sounds to me like you're not actually running 1.1.6.
[root@da1-node30 chunks]# nodetool -h `hostname` version
ReleaseVersion: 1.1.6{quote}
That's DataStax version and it may differ a bit - not sure how do the versions
correlate.
was (Author: soboleiv):
{quote}Sounds to me like you're not actually running 1.1.6.
[root@da1-node30 chunks]# nodetool -h `hostname` version
ReleaseVersion: 1.1.6
That's DataStax version and it may differ a bit - not sure how do the versions
correlate.
OOM Exception during repair session with LeveledCompactionStrategy
--
Key: CASSANDRA-5222
URL: https://issues.apache.org/jira/browse/CASSANDRA-5222
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.1.6
Environment: 3Gb Heap(12Gb per node RAM)
36 nodes, 0.9 Tb of data per node, Leveled compaction strategy, SSTable size
=100Mb
Reporter: Ivan Sobolev
Priority: Critical
Attachments: sstablescanner.png
1.8 Gb of heap is consumed with 12k SSTableBoundedScanner * 140kbytes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-5222) OOM Exception during repair session with LeveledCompactionStrategy
[
https://issues.apache.org/jira/browse/CASSANDRA-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571301#comment-13571301
]
Ivan Sobolev edited comment on CASSANDRA-5222 at 2/5/13 1:50 PM:
-
{quote}Sounds to me like you're not actually running 1.1.6.
[root@da1-node30 chunks]# nodetool -h `hostname` version
ReleaseVersion: 1.1.6{quote}
That's DataStax version and it may differ a bit - not sure how do the versions
correlate.
Upd: should be the same:
http://www.datastax.com/dev/blog/datastax-community-edition-1-1-6-now-available
was (Author: soboleiv):
{quote}Sounds to me like you're not actually running 1.1.6.
[root@da1-node30 chunks]# nodetool -h `hostname` version
ReleaseVersion: 1.1.6{quote}
That's DataStax version and it may differ a bit - not sure how do the versions
correlate.
OOM Exception during repair session with LeveledCompactionStrategy
--
Key: CASSANDRA-5222
URL: https://issues.apache.org/jira/browse/CASSANDRA-5222
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.1.6
Environment: 3Gb Heap(12Gb per node RAM)
36 nodes, 0.9 Tb of data per node, Leveled compaction strategy, SSTable size
=100Mb
Reporter: Ivan Sobolev
Priority: Critical
Attachments: sstablescanner.png
1.8 Gb of heap is consumed with 12k SSTableBoundedScanner * 140kbytes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5222) OOM Exception during repair session with LeveledCompactionStrategy
[
https://issues.apache.org/jira/browse/CASSANDRA-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571306#comment-13571306
]
Ivan Sobolev commented on CASSANDRA-5222:
-
{quote}So it's also possible this was set back to STCS by mistake.{quote}
You mean SizeTieredCompactionStrategy?
Sstable's *-Data files are up to 100Mb
{quote}[default@chunks] describe chunks;
ColumnFamily: chunks
Key Validation Class: org.apache.cassandra.db.marshal.UTF8Type
Default column value validator: org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.BytesType
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 0.1
DC Local Read repair chance: 0.0
Replicate on write: true
Caching: KEYS_ONLY
Bloom Filter FP chance: default
Built indexes: []
Compaction Strategy:
org.apache.cassandra.db.compaction.LeveledCompactionStrategy
Compaction Strategy Options:
sstable_size_in_mb: 100
Compression Options:
sstable_compression:
org.apache.cassandra.io.compress.SnappyCompressor{quote}
Anything else I could check?
OOM Exception during repair session with LeveledCompactionStrategy
--
Key: CASSANDRA-5222
URL: https://issues.apache.org/jira/browse/CASSANDRA-5222
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.1.6
Environment: 3Gb Heap(12Gb per node RAM)
36 nodes, 0.9 Tb of data per node, Leveled compaction strategy, SSTable size
=100Mb
Reporter: Ivan Sobolev
Priority: Critical
Attachments: sstablescanner.png
1.8 Gb of heap is consumed with 12k SSTableBoundedScanner * 140kbytes
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5222) OOM Exception during repair session with LeveledCompactionStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sobolev updated CASSANDRA-5222: Attachment: chunks.json OOM Exception during repair session with LeveledCompactionStrategy -- Key: CASSANDRA-5222 URL: https://issues.apache.org/jira/browse/CASSANDRA-5222 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.6 Environment: 3Gb Heap(12Gb per node RAM) 36 nodes, 0.9 Tb of data per node, Leveled compaction strategy, SSTable size =100Mb Reporter: Ivan Sobolev Priority: Critical Attachments: chunks.json, sstablescanner.png 1.8 Gb of heap is consumed with 12k SSTableBoundedScanner * 140kbytes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5222) OOM Exception during repair session with LeveledCompactionStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571326#comment-13571326 ] Ivan Sobolev commented on CASSANDRA-5222: - Do you think backlogged L0 may lead to such effects? Currently I can see 3k tables there, not sure if it went from 12k to 3k that quick. OOM Exception during repair session with LeveledCompactionStrategy -- Key: CASSANDRA-5222 URL: https://issues.apache.org/jira/browse/CASSANDRA-5222 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.6 Environment: 3Gb Heap(12Gb per node RAM) 36 nodes, 0.9 Tb of data per node, Leveled compaction strategy, SSTable size =100Mb Reporter: Ivan Sobolev Priority: Critical Attachments: chunks.json, sstablescanner.png 1.8 Gb of heap is consumed with 12k SSTableBoundedScanner * 140kbytes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5222) OOM Exception during repair session with LeveledCompactionStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13571331#comment-13571331 ] Ivan Sobolev commented on CASSANDRA-5222: - Another node died. Can see exactly 12k L0 tables there, seems we're just under capacity with current setup. OOM Exception during repair session with LeveledCompactionStrategy -- Key: CASSANDRA-5222 URL: https://issues.apache.org/jira/browse/CASSANDRA-5222 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.6 Environment: 3Gb Heap(12Gb per node RAM) 36 nodes, 0.9 Tb of data per node, Leveled compaction strategy, SSTable size =100Mb Reporter: Ivan Sobolev Priority: Critical Attachments: chunks.json, sstablescanner.png 1.8 Gb of heap is consumed with 12k SSTableBoundedScanner * 140kbytes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4651) RangeStreamer fails the bootstrap process if any node streaming data fails.
[ https://issues.apache.org/jira/browse/CASSANDRA-4651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13505434#comment-13505434 ] Ivan Sobolev commented on CASSANDRA-4651: - Also it seems that as a result newjoiner node may accept responsibility for token range it has no/partial data for. RangeStreamer fails the bootstrap process if any node streaming data fails. --- Key: CASSANDRA-4651 URL: https://issues.apache.org/jira/browse/CASSANDRA-4651 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.1.5 Reporter: sankalp kohli Priority: Minor If any node streaming the data fails, RangeStreamer throws a Runtime exception and aborts the bootstrap. If the N = 3, it can stream data from some other node and need not fail the bootstrap process. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4417) invalid counter shard detected
[ https://issues.apache.org/jira/browse/CASSANDRA-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sobolev updated CASSANDRA-4417: Attachment: err.txt We have examples where increment is not always 1. invalid counter shard detected --- Key: CASSANDRA-4417 URL: https://issues.apache.org/jira/browse/CASSANDRA-4417 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.1 Environment: Amazon Linux Reporter: Senthilvel Rangaswamy Attachments: err.txt Seeing errors like these: 2012-07-06_07:00:27.22662 ERROR 07:00:27,226 invalid counter shard detected; (17bfd850-ac52-11e1--6ecd0b5b61e7, 1, 13) and (17bfd850-ac52-11e1--6ecd0b5b61e7, 1, 1) differ only in count; will pick highest to self-heal; this indicates a bug or corruption generated a bad counter shard What does it mean ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4417) invalid counter shard detected
[
https://issues.apache.org/jira/browse/CASSANDRA-4417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486872#comment-13486872
]
Ivan Sobolev commented on CASSANDRA-4417:
-
{quote}
[~slebresne]
Quick question: do you always increment by the same value by any chance? {quote}
Attached a log has not only +1 increments(though, think not any log would help
you there :) )
We run 1.1.5, no upgradesstables, most probably unclean shutdown too.
invalid counter shard detected
---
Key: CASSANDRA-4417
URL: https://issues.apache.org/jira/browse/CASSANDRA-4417
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.1
Environment: Amazon Linux
Reporter: Senthilvel Rangaswamy
Attachments: err.txt
Seeing errors like these:
2012-07-06_07:00:27.22662 ERROR 07:00:27,226 invalid counter shard detected;
(17bfd850-ac52-11e1--6ecd0b5b61e7, 1, 13) and
(17bfd850-ac52-11e1--6ecd0b5b61e7, 1, 1) differ only in count; will pick
highest to self-heal; this indicates a bug or corruption generated a bad
counter shard
What does it mean ?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4780) Not informative error reporting when calling getNaturalEndpoints for a CF without key_validation_class
Ivan Sobolev created CASSANDRA-4780:
---
Summary: Not informative error reporting when calling
getNaturalEndpoints for a CF without key_validation_class
Key: CASSANDRA-4780
URL: https://issues.apache.org/jira/browse/CASSANDRA-4780
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 1.1.5
Reporter: Ivan Sobolev
Priority: Minor
While trying to call getNaturalEndpoints(String,String,String) via jmx, got the
following:
{code}Problem invoking getNaturalEndpoints: java.rmi.UnmarshalException: Error
unmarshaling return; nested exception is: java.lang.ClassNotFoundException:
org.apache.cassandra.db.marshal.MarshalException (no security manager: RMI
class loader disabled){code}
Googling suggested to try out different classpath mysteries[1]
Doing that from the nodetool was a bit more informative though:
{code}Exception in thread main
org.apache.cassandra.db.marshal.MarshalException: cannot parse
'28589689-0bf0-9ebf-e405-d8f1d798cd7d' as hex bytes
at
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:74)
at
org.apache.cassandra.service.StorageService.getNaturalEndpoints(StorageService.java:2131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
at
javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:303)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NumberFormatException: Non-hex characters in
28589689-0bf0-9ebf-e405-d8f1d798cd7d
at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:60)
at
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:70)
{code}
{code}
update column family xxx with key_validation_class=UTF8Type;{code}
Goals of the ticket:
* Ensure that message from exception will properly pop up to the JMX client
* Guys googling the same problem will have a direction
___
[1] forum.springsource.org/showthread.php?25937-RMI-class-loader-disabled
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4780) Not informative error reporting when calling getNaturalEndpoints for a CF without key_validation_class
[
https://issues.apache.org/jira/browse/CASSANDRA-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Sobolev updated CASSANDRA-4780:
Description:
While trying to call getNaturalEndpoints(String,String,String) via jmx, got the
following:
{code}Problem invoking getNaturalEndpoints: java.rmi.UnmarshalException: Error
unmarshaling return; nested exception is: java.lang.ClassNotFoundException:
org.apache.cassandra.db.marshal.MarshalException (no security manager: RMI
class loader disabled){code}
Googling suggested to try out different classpath mysteries[1]
Doing that from the nodetool was a bit more informative though:
{code}Exception in thread main
org.apache.cassandra.db.marshal.MarshalException: cannot parse
'28589689-0bf0-9ebf-e405-d8f1d798cd7d' as hex bytes
at
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:74)
at
org.apache.cassandra.service.StorageService.getNaturalEndpoints(StorageService.java:2131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
at
javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:303)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NumberFormatException: Non-hex characters in
28589689-0bf0-9ebf-e405-d8f1d798cd7d
at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:60)
at
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:70)
{code}
The missing bit:
{code}
update column family xxx with key_validation_class=UTF8Type;{code}
Goals of the ticket:
* Ensure that message from exception will properly pop up to the JMX client
* Guys googling the same problem will have a direction
___
[1] forum.springsource.org/showthread.php?25937-RMI-class-loader-disabled
was:
While trying to call getNaturalEndpoints(String,String,String) via jmx, got the
following:
{code}Problem invoking getNaturalEndpoints: java.rmi.UnmarshalException: Error
unmarshaling return; nested exception is: java.lang.ClassNotFoundException:
org.apache.cassandra.db.marshal.MarshalException (no security manager: RMI
class loader disabled){code}
Googling suggested to try out different classpath mysteries[1]
Doing that from the nodetool was a bit more informative though:
{code}Exception in thread main
org.apache.cassandra.db.marshal.MarshalException: cannot parse
'28589689-0bf0-9ebf-e405-d8f1d798cd7d' as hex bytes
at
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:74)
at
[jira] [Updated] (CASSANDRA-4780) Not informative error reporting when calling getNaturalEndpoints for a CF without key_validation_class
[
https://issues.apache.org/jira/browse/CASSANDRA-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Sobolev updated CASSANDRA-4780:
Description:
While trying to call getNaturalEndpoints(String,String,String) via jmx, got the
following:
{code}Problem invoking getNaturalEndpoints: java.rmi.UnmarshalException: Error
unmarshaling return;
nested exception is: java.lang.ClassNotFoundException:
org.apache.cassandra.db.marshal.MarshalException (no security manager: RMI
class loader disabled){code}
Googling suggested to try out different classpath mysteries[1]
Doing that from the nodetool was a bit more informative though:
{code}Exception in thread main
org.apache.cassandra.db.marshal.MarshalException: cannot parse
'28589689-0bf0-9ebf-e405-d8f1d798cd7d' as hex bytes
at
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:74)
at
org.apache.cassandra.service.StorageService.getNaturalEndpoints(StorageService.java:2131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
at
javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:303)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NumberFormatException: Non-hex characters in
28589689-0bf0-9ebf-e405-d8f1d798cd7d
at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:60)
at
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:70)
{code}
The missing bit:
{code}
update column family xxx with key_validation_class=UTF8Type;{code}
Goals of the ticket:
* Ensure that message from exception will properly pop up to the JMX client
* Guys googling the same problem will have a direction
___
[1] forum.springsource.org/showthread.php?25937-RMI-class-loader-disabled
was:
While trying to call getNaturalEndpoints(String,String,String) via jmx, got the
following:
{code}Problem invoking getNaturalEndpoints: java.rmi.UnmarshalException: Error
unmarshaling return; nested exception is: java.lang.ClassNotFoundException:
org.apache.cassandra.db.marshal.MarshalException (no security manager: RMI
class loader disabled){code}
Googling suggested to try out different classpath mysteries[1]
Doing that from the nodetool was a bit more informative though:
{code}Exception in thread main
org.apache.cassandra.db.marshal.MarshalException: cannot parse
'28589689-0bf0-9ebf-e405-d8f1d798cd7d' as hex bytes
at
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:74)
at
[jira] [Updated] (CASSANDRA-4780) Not informative error reporting when calling getNaturalEndpoints for a CF without key_validation_class
[
https://issues.apache.org/jira/browse/CASSANDRA-4780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Sobolev updated CASSANDRA-4780:
Description:
While trying to call getNaturalEndpoints(String,String,String) via
jmx(jconsole), got the following:
{code}Problem invoking getNaturalEndpoints: java.rmi.UnmarshalException: Error
unmarshaling return;
nested exception is: java.lang.ClassNotFoundException:
org.apache.cassandra.db.marshal.MarshalException (no security manager: RMI
class loader disabled){code}
Googling suggested to try out different classpath mysteries[1]
Doing that from the nodetool was a bit more informative though:
{code}Exception in thread main
org.apache.cassandra.db.marshal.MarshalException: cannot parse
'28589689-0bf0-9ebf-e405-d8f1d798cd7d' as hex bytes
at
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:74)
at
org.apache.cassandra.service.StorageService.getNaturalEndpoints(StorageService.java:2131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
at
com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
at
com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
at
javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1427)
at
javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
at
javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1265)
at
javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1360)
at
javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:303)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.NumberFormatException: Non-hex characters in
28589689-0bf0-9ebf-e405-d8f1d798cd7d
at org.apache.cassandra.utils.Hex.hexToBytes(Hex.java:60)
at
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:70)
{code}
The missing bit:
{code}
update column family xxx with key_validation_class=UTF8Type;{code}
Goals of the ticket:
* Ensure that message from exception will properly pop up to the JMX client
* Guys googling the same problem will have a direction
___
[1] forum.springsource.org/showthread.php?25937-RMI-class-loader-disabled
was:
While trying to call getNaturalEndpoints(String,String,String) via jmx, got the
following:
{code}Problem invoking getNaturalEndpoints: java.rmi.UnmarshalException: Error
unmarshaling return;
nested exception is: java.lang.ClassNotFoundException:
org.apache.cassandra.db.marshal.MarshalException (no security manager: RMI
class loader disabled){code}
Googling suggested to try out different classpath mysteries[1]
Doing that from the nodetool was a bit more informative though:
{code}Exception in thread main
org.apache.cassandra.db.marshal.MarshalException: cannot parse
'28589689-0bf0-9ebf-e405-d8f1d798cd7d' as hex bytes
at
org.apache.cassandra.db.marshal.BytesType.fromString(BytesType.java:74)
at
[jira] [Created] (CASSANDRA-4676) Can't delete/create a keyspace
Ivan Sobolev created CASSANDRA-4676: --- Summary: Can't delete/create a keyspace Key: CASSANDRA-4676 URL: https://issues.apache.org/jira/browse/CASSANDRA-4676 Project: Cassandra Issue Type: Bug Environment: CentOs 5.5. Cassandra 1.0.6--1.1.5 Reporter: Ivan Sobolev The main question for me is what to try. Deletion/recreation of the keyspace is not possible for some reason and this prevents me from reinstalling OpsCenter(which I'd like to have). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4676) Can't delete/create a keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sobolev updated CASSANDRA-4676: Attachment: cassandra-ex.txt cassandra-cli.txt Can't delete/create a keyspace -- Key: CASSANDRA-4676 URL: https://issues.apache.org/jira/browse/CASSANDRA-4676 Project: Cassandra Issue Type: Bug Environment: CentOs 5.5. Cassandra 1.0.6--1.1.5 Reporter: Ivan Sobolev Attachments: cassandra-cli.txt, cassandra-ex.txt The main question for me is what to try. Deletion/recreation of the keyspace is not possible for some reason and this prevents me from reinstalling OpsCenter(which I'd like to have). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4676) Can't delete/create a keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13457727#comment-13457727 ] Ivan Sobolev commented on CASSANDRA-4676: - Cassandra-ex is what I can find on Cassandra logs when try to create a keyspace. Deletion is silent. Can't delete/create a keyspace -- Key: CASSANDRA-4676 URL: https://issues.apache.org/jira/browse/CASSANDRA-4676 Project: Cassandra Issue Type: Bug Environment: CentOs 5.5. Cassandra 1.0.6--1.1.5 Reporter: Ivan Sobolev Attachments: cassandra-cli.txt, cassandra-ex.txt The main question for me is what to try. Deletion/recreation of the keyspace is not possible for some reason and this prevents me from reinstalling OpsCenter(which I'd like to have). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4676) Can't delete/create a keyspace
[
https://issues.apache.org/jira/browse/CASSANDRA-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Sobolev updated CASSANDRA-4676:
Description:
Deletion/recreation of the keyspace was not possible.
*Workaround:*
use system;
set schema_keyspaces['OpsCenter']['durable_writes']=true;
set schema_keyspaces['OpsCenter']['strategy_options']='{datacenter1:1}';
set schema_keyspaces['OpsCenter']['name']='OpsCenter';
set
schema_keyspaces['OpsCenter']['strategy_class']='org.apache.cassandra.locator.NetworkTopologyStrategy';
drop keyspace OpsCenter;
was:
The main question for me is what to try. Deletion/recreation of the keyspace
was not possible.
*Workaround:*
use system;
set schema_keyspaces['OpsCenter']['durable_writes']=true;
set schema_keyspaces['OpsCenter']['strategy_options']='{datacenter1:1}';
set schema_keyspaces['OpsCenter']['name']='OpsCenter';
set
schema_keyspaces['OpsCenter']['strategy_class']='org.apache.cassandra.locator.NetworkTopologyStrategy';
drop keyspace OpsCenter;
Can't delete/create a keyspace
--
Key: CASSANDRA-4676
URL: https://issues.apache.org/jira/browse/CASSANDRA-4676
Project: Cassandra
Issue Type: Bug
Environment: CentOs 5.5.
Cassandra 1.0.6--1.1.5
Reporter: Ivan Sobolev
Attachments: cassandra-cli.txt, cassandra-ex.txt
Deletion/recreation of the keyspace was not possible.
*Workaround:*
use system;
set schema_keyspaces['OpsCenter']['durable_writes']=true;
set schema_keyspaces['OpsCenter']['strategy_options']='{datacenter1:1}';
set schema_keyspaces['OpsCenter']['name']='OpsCenter';
set
schema_keyspaces['OpsCenter']['strategy_class']='org.apache.cassandra.locator.NetworkTopologyStrategy';
drop keyspace OpsCenter;
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4676) Can't delete/create a keyspace
[
https://issues.apache.org/jira/browse/CASSANDRA-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Sobolev updated CASSANDRA-4676:
Description:
The main question for me is what to try. Deletion/recreation of the keyspace
was not possible.
*Workaround:*
use system;
set schema_keyspaces['OpsCenter']['durable_writes']=true;
set schema_keyspaces['OpsCenter']['strategy_options']='{datacenter1:1}';
set schema_keyspaces['OpsCenter']['name']='OpsCenter';
set
schema_keyspaces['OpsCenter']['strategy_class']='org.apache.cassandra.locator.NetworkTopologyStrategy';
drop keyspace OpsCenter;
was:The main question for me is what to try. Deletion/recreation of the
keyspace is not possible for some reason and this prevents me from reinstalling
OpsCenter(which I'd like to have).
Can't delete/create a keyspace
--
Key: CASSANDRA-4676
URL: https://issues.apache.org/jira/browse/CASSANDRA-4676
Project: Cassandra
Issue Type: Bug
Environment: CentOs 5.5.
Cassandra 1.0.6--1.1.5
Reporter: Ivan Sobolev
Attachments: cassandra-cli.txt, cassandra-ex.txt
The main question for me is what to try. Deletion/recreation of the keyspace
was not possible.
*Workaround:*
use system;
set schema_keyspaces['OpsCenter']['durable_writes']=true;
set schema_keyspaces['OpsCenter']['strategy_options']='{datacenter1:1}';
set schema_keyspaces['OpsCenter']['name']='OpsCenter';
set
schema_keyspaces['OpsCenter']['strategy_class']='org.apache.cassandra.locator.NetworkTopologyStrategy';
drop keyspace OpsCenter;
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4676) Can't delete/create a keyspace
[
https://issues.apache.org/jira/browse/CASSANDRA-4676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Sobolev updated CASSANDRA-4676:
Priority: Trivial (was: Major)
Can't delete/create a keyspace
--
Key: CASSANDRA-4676
URL: https://issues.apache.org/jira/browse/CASSANDRA-4676
Project: Cassandra
Issue Type: Bug
Environment: CentOs 5.5.
Cassandra 1.0.6--1.1.5
Reporter: Ivan Sobolev
Priority: Trivial
Attachments: cassandra-cli.txt, cassandra-ex.txt
Deletion/recreation of the keyspace was not possible.
*Workaround:*
use system;
set schema_keyspaces['OpsCenter']['durable_writes']=true;
set schema_keyspaces['OpsCenter']['strategy_options']='{datacenter1:1}';
set schema_keyspaces['OpsCenter']['name']='OpsCenter';
set
schema_keyspaces['OpsCenter']['strategy_class']='org.apache.cassandra.locator.NetworkTopologyStrategy';
drop keyspace OpsCenter;
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
