[jira] [Created] (CASSANDRA-4420) Possible schema corruption with cql 3.0
bert Passek created CASSANDRA-4420: -- Summary: Possible schema corruption with cql 3.0 Key: CASSANDRA-4420 URL: https://issues.apache.org/jira/browse/CASSANDRA-4420 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.2 Environment: Lenny Squeeze Reporter: bert Passek Hi, i got some problems while creating schemas with cql 3.0. Af that i can't even start cassandra anymore. Following steps for reproduction were done on a new installation of cassandra: 1. simply create a keyspace test via cqlsh -3 create keyspace test with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor = 1; 2. add cf with composite columns via cqlsh -3 create table test1 ( a int, b int, c int, d int, primary key (a, b, c) ); 3. drop column family drop columnfamily test1; So until now everything went fine. Now i'm trying to insert a slightly modified column family with the same name above. 4. create new cf via cqlsh -3 create table test1 ( a int, b int, c int, primary key (a, b) ); This creation fails with following exception: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106) Not at this point you can't do anything anymore via sql or cli. Shutting down and starting cassandra again throws same exceptions: ERROR 14:48:41,705 Exception encountered during startup java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106) java.lang.IndexOutOfBoundsException: Index: 2, Size: 2Exception encountered during startup: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at
[jira] [Updated] (CASSANDRA-4420) Possible schema corruption with cql 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bert Passek updated CASSANDRA-4420: --- Description: Hi, i've got some problems while creating schemas with cql 3.0. After that i can't even start cassandra anymore. Following steps for reproduction were done on a new installation of cassandra: 1. simply create a keyspace test via cqlsh -3 create keyspace test with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor = 1; 2. add cf with composite columns via cqlsh -3 create table test1 ( a int, b int, c int, d int, primary key (a, b, c) ); 3. drop column family drop columnfamily test1; So until now everything went fine. Now i'm trying to insert a slightly modified column family with the same name above. 4. create new cf via cqlsh -3 create table test1 ( a int, b int, c int, primary key (a, b) ); This creation fails with following exception: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106) Now at this point you can't do anything anymore via cql or cli. Shutting down and starting cassandra again throws same exceptions: ERROR 14:48:41,705 Exception encountered during startup java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106) java.lang.IndexOutOfBoundsException: Index: 2, Size: 2Exception encountered during startup: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106) Actually it's the result of a slightly different problem in
[jira] [Created] (CASSANDRA-4421) Invalid validator used for column values of composite columns - writing fails
bert Passek created CASSANDRA-4421:
--
Summary: Invalid validator used for column values of composite
columns - writing fails
Key: CASSANDRA-4421
URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
Project: Cassandra
Issue Type: Bug
Components: API
Affects Versions: 1.1.2
Environment: Debian Squeeze
Reporter: bert Passek
Hello,
i faced a bug while writing composite column values and following validation on
server side.
This is the setup for reproduction:
1. create a keyspace
create keyspace test with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor = 1;
2. create a cf via cql (3.0)
create table test1 (
a int,
b int,
c int,
primary key (a, b)
);
If i have a look at the schema in cli i noticed that there is no column
metadata for columns not part of primary key.
create column family test1
with column_type = 'Standard'
and comparator =
'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'Int32Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
Please notice the default validation class: UTF8Type
Now i would like to insert value 127 via cassandra client (no cql, part of
mr-jobs). Have a look at the attachement.
Batch mutate fails:
InvalidRequestException(why:(String didn't validate.) [test][test1][1:c] failed
validation)
A validator for column value is fetched in ThriftValidation::validateColumnData
which returns always the default validator which is UTF8Type as described above
(The ColumnDefinition for given column name c is always null)
In UTF8Type there is a check for
if (b 127)
return false;
Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table creation.
I assigned data types to all columns, but i can not set values for a composite
column because the default validation class is used.
I think the schema should know the correct validator even for composite
columns. The usage of the default validation class does not make sense.
Best Regards
Bert Passek
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4420) Possible schema corruption with cql 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bert Passek updated CASSANDRA-4420: --- Attachment: TestCompositeColumn.java Simple test class for inserting data via batch_mutate. Possible schema corruption with cql 3.0 --- Key: CASSANDRA-4420 URL: https://issues.apache.org/jira/browse/CASSANDRA-4420 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.2 Environment: Lenny Squeeze Reporter: bert Passek Attachments: TestCompositeColumn.java Hi, i've got some problems while creating schemas with cql 3.0. After that i can't even start cassandra anymore. Following steps for reproduction were done on a new installation of cassandra: 1. simply create a keyspace test via cqlsh -3 create keyspace test with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor = 1; 2. add cf with composite columns via cqlsh -3 create table test1 ( a int, b int, c int, d int, primary key (a, b, c) ); 3. drop column family drop columnfamily test1; So until now everything went fine. Now i'm trying to insert a slightly modified column family with the same name above. 4. create new cf via cqlsh -3 create table test1 ( a int, b int, c int, primary key (a, b) ); This creation fails with following exception: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106) Now at this point you can't do anything anymore via cql or cli. Shutting down and starting cassandra again throws same exceptions: ERROR 14:48:41,705 Exception encountered during startup java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106) java.lang.IndexOutOfBoundsException: Index: 2, Size: 2Exception encountered during startup: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at
[jira] [Updated] (CASSANDRA-4420) Possible schema corruption with cql 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bert Passek updated CASSANDRA-4420: --- Comment: was deleted (was: Simple test class for inserting data via batch_mutate.) Possible schema corruption with cql 3.0 --- Key: CASSANDRA-4420 URL: https://issues.apache.org/jira/browse/CASSANDRA-4420 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.2 Environment: Lenny Squeeze Reporter: bert Passek Hi, i've got some problems while creating schemas with cql 3.0. After that i can't even start cassandra anymore. Following steps for reproduction were done on a new installation of cassandra: 1. simply create a keyspace test via cqlsh -3 create keyspace test with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor = 1; 2. add cf with composite columns via cqlsh -3 create table test1 ( a int, b int, c int, d int, primary key (a, b, c) ); 3. drop column family drop columnfamily test1; So until now everything went fine. Now i'm trying to insert a slightly modified column family with the same name above. 4. create new cf via cqlsh -3 create table test1 ( a int, b int, c int, primary key (a, b) ); This creation fails with following exception: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106) Now at this point you can't do anything anymore via cql or cli. Shutting down and starting cassandra again throws same exceptions: ERROR 14:48:41,705 Exception encountered during startup java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106) java.lang.IndexOutOfBoundsException: Index: 2, Size: 2Exception encountered during startup: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at
[jira] [Updated] (CASSANDRA-4420) Possible schema corruption with cql 3.0
[ https://issues.apache.org/jira/browse/CASSANDRA-4420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bert Passek updated CASSANDRA-4420: --- Attachment: (was: TestCompositeColumn.java) Possible schema corruption with cql 3.0 --- Key: CASSANDRA-4420 URL: https://issues.apache.org/jira/browse/CASSANDRA-4420 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.1.2 Environment: Lenny Squeeze Reporter: bert Passek Hi, i've got some problems while creating schemas with cql 3.0. After that i can't even start cassandra anymore. Following steps for reproduction were done on a new installation of cassandra: 1. simply create a keyspace test via cqlsh -3 create keyspace test with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor = 1; 2. add cf with composite columns via cqlsh -3 create table test1 ( a int, b int, c int, d int, primary key (a, b, c) ); 3. drop column family drop columnfamily test1; So until now everything went fine. Now i'm trying to insert a slightly modified column family with the same name above. 4. create new cf via cqlsh -3 create table test1 ( a int, b int, c int, primary key (a, b) ); This creation fails with following exception: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106) Now at this point you can't do anything anymore via cql or cli. Shutting down and starting cassandra again throws same exceptions: ERROR 14:48:41,705 Exception encountered during startup java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:535) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:182) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:353) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:106) java.lang.IndexOutOfBoundsException: Index: 2, Size: 2Exception encountered during startup: Index: 2, Size: 2 at java.util.ArrayList.RangeCheck(ArrayList.java:547) at java.util.ArrayList.get(ArrayList.java:322) at org.apache.cassandra.config.CFMetaData.getColumnDefinitionComparator(CFMetaData.java:1280) at org.apache.cassandra.config.ColumnDefinition.fromSchema(ColumnDefinition.java:256) at org.apache.cassandra.config.CFMetaData.addColumnDefinitionSchema(CFMetaData.java:1293) at org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1225) at org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:294) at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) at org.apache.cassandra.db.DefsTable.loadFromTable(DefsTable.java:158) at
[jira] [Commented] (CASSANDRA-4421) Invalid validator used for column values of composite columns - writing fails
[
https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13408007#comment-13408007
]
bert Passek commented on CASSANDRA-4421:
Yes, as i can remember i started cqlsh via cqlsh -3 which is supposed to use
cql 3.0, isn't it. But let me check again.
Invalid validator used for column values of composite columns - writing fails
--
Key: CASSANDRA-4421
URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
Project: Cassandra
Issue Type: Bug
Components: API
Affects Versions: 1.1.2
Environment: Debian Squeeze
Reporter: bert Passek
Hello,
i faced a bug while writing composite column values and following validation
on server side.
This is the setup for reproduction:
1. create a keyspace
create keyspace test with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor = 1;
2. create a cf via cql (3.0)
create table test1 (
a int,
b int,
c int,
primary key (a, b)
);
If i have a look at the schema in cli i noticed that there is no column
metadata for columns not part of primary key.
create column family test1
with column_type = 'Standard'
and comparator =
'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'Int32Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
Please notice the default validation class: UTF8Type
Now i would like to insert value 127 via cassandra client (no cql, part of
mr-jobs). Have a look at the attachement.
Batch mutate fails:
InvalidRequestException(why:(String didn't validate.) [test][test1][1:c]
failed validation)
A validator for column value is fetched in
ThriftValidation::validateColumnData which returns always the default
validator which is UTF8Type as described above (The ColumnDefinition for
given column name c is always null)
In UTF8Type there is a check for
if (b 127)
return false;
Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table
creation. I assigned data types to all columns, but i can not set values for
a composite column because the default validation class is used.
I think the schema should know the correct validator even for composite
columns. The usage of the default validation class does not make sense.
Best Regards
Bert Passek
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4421) Invalid validator used for column values of composite columns - writing fails
[
https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bert Passek updated CASSANDRA-4421:
---
Comment: was deleted
(was: Yes, as i can remember i started cqlsh via cqlsh -3 which is supposed
to use cql 3.0, isn't it. But let me check again.)
Invalid validator used for column values of composite columns - writing fails
--
Key: CASSANDRA-4421
URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
Project: Cassandra
Issue Type: Bug
Components: API
Affects Versions: 1.1.2
Environment: Debian Squeeze
Reporter: bert Passek
Hello,
i faced a bug while writing composite column values and following validation
on server side.
This is the setup for reproduction:
1. create a keyspace
create keyspace test with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor = 1;
2. create a cf via cql (3.0)
create table test1 (
a int,
b int,
c int,
primary key (a, b)
);
If i have a look at the schema in cli i noticed that there is no column
metadata for columns not part of primary key.
create column family test1
with column_type = 'Standard'
and comparator =
'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'Int32Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
Please notice the default validation class: UTF8Type
Now i would like to insert value 127 via cassandra client (no cql, part of
mr-jobs). Have a look at the attachement.
Batch mutate fails:
InvalidRequestException(why:(String didn't validate.) [test][test1][1:c]
failed validation)
A validator for column value is fetched in
ThriftValidation::validateColumnData which returns always the default
validator which is UTF8Type as described above (The ColumnDefinition for
given column name c is always null)
In UTF8Type there is a check for
if (b 127)
return false;
Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table
creation. I assigned data types to all columns, but i can not set values for
a composite column because the default validation class is used.
I think the schema should know the correct validator even for composite
columns. The usage of the default validation class does not make sense.
Best Regards
Bert Passek
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4421) Invalid validator used for column values of composite columns - writing fails
[
https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13408012#comment-13408012
]
bert Passek commented on CASSANDRA-4421:
Oh, i got it, so it means we can not use MR-Jobs using ColumnFamilyOutputFormat
or BulkOutputFormat?
Invalid validator used for column values of composite columns - writing fails
--
Key: CASSANDRA-4421
URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
Project: Cassandra
Issue Type: Bug
Components: API
Affects Versions: 1.1.2
Environment: Debian Squeeze
Reporter: bert Passek
Hello,
i faced a bug while writing composite column values and following validation
on server side.
This is the setup for reproduction:
1. create a keyspace
create keyspace test with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor = 1;
2. create a cf via cql (3.0)
create table test1 (
a int,
b int,
c int,
primary key (a, b)
);
If i have a look at the schema in cli i noticed that there is no column
metadata for columns not part of primary key.
create column family test1
with column_type = 'Standard'
and comparator =
'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'Int32Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
Please notice the default validation class: UTF8Type
Now i would like to insert value 127 via cassandra client (no cql, part of
mr-jobs). Have a look at the attachement.
Batch mutate fails:
InvalidRequestException(why:(String didn't validate.) [test][test1][1:c]
failed validation)
A validator for column value is fetched in
ThriftValidation::validateColumnData which returns always the default
validator which is UTF8Type as described above (The ColumnDefinition for
given column name c is always null)
In UTF8Type there is a check for
if (b 127)
return false;
Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table
creation. I assigned data types to all columns, but i can not set values for
a composite column because the default validation class is used.
I think the schema should know the correct validator even for composite
columns. The usage of the default validation class does not make sense.
Best Regards
Bert Passek
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4421) Invalid validator used for column values of composite columns - writing fails
[
https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bert Passek updated CASSANDRA-4421:
---
Comment: was deleted
(was: A simple test case.)
Invalid validator used for column values of composite columns - writing fails
--
Key: CASSANDRA-4421
URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
Project: Cassandra
Issue Type: Bug
Components: API
Affects Versions: 1.1.2
Environment: Debian Squeeze
Reporter: bert Passek
Hello,
i faced a bug while writing composite column values and following validation
on server side.
This is the setup for reproduction:
1. create a keyspace
create keyspace test with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor = 1;
2. create a cf via cql (3.0)
create table test1 (
a int,
b int,
c int,
primary key (a, b)
);
If i have a look at the schema in cli i noticed that there is no column
metadata for columns not part of primary key.
create column family test1
with column_type = 'Standard'
and comparator =
'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'Int32Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
Please notice the default validation class: UTF8Type
Now i would like to insert value 127 via cassandra client (no cql, part of
mr-jobs). Have a look at the attachement.
Batch mutate fails:
InvalidRequestException(why:(String didn't validate.) [test][test1][1:c]
failed validation)
A validator for column value is fetched in
ThriftValidation::validateColumnData which returns always the default
validator which is UTF8Type as described above (The ColumnDefinition for
given column name c is always null)
In UTF8Type there is a check for
if (b 127)
return false;
Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table
creation. I assigned data types to all columns, but i can not set values for
a composite column because the default validation class is used.
I think the schema should know the correct validator even for composite
columns. The usage of the default validation class does not make sense.
Best Regards
Bert Passek
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4421) Invalid validator used for column values of composite columns - writing fails
[
https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bert Passek updated CASSANDRA-4421:
---
Attachment: TestCompositeColumn.java
A simple test case.
Invalid validator used for column values of composite columns - writing fails
--
Key: CASSANDRA-4421
URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
Project: Cassandra
Issue Type: Bug
Components: API
Affects Versions: 1.1.2
Environment: Debian Squeeze
Reporter: bert Passek
Hello,
i faced a bug while writing composite column values and following validation
on server side.
This is the setup for reproduction:
1. create a keyspace
create keyspace test with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor = 1;
2. create a cf via cql (3.0)
create table test1 (
a int,
b int,
c int,
primary key (a, b)
);
If i have a look at the schema in cli i noticed that there is no column
metadata for columns not part of primary key.
create column family test1
with column_type = 'Standard'
and comparator =
'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'Int32Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
Please notice the default validation class: UTF8Type
Now i would like to insert value 127 via cassandra client (no cql, part of
mr-jobs). Have a look at the attachement.
Batch mutate fails:
InvalidRequestException(why:(String didn't validate.) [test][test1][1:c]
failed validation)
A validator for column value is fetched in
ThriftValidation::validateColumnData which returns always the default
validator which is UTF8Type as described above (The ColumnDefinition for
given column name c is always null)
In UTF8Type there is a check for
if (b 127)
return false;
Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table
creation. I assigned data types to all columns, but i can not set values for
a composite column because the default validation class is used.
I think the schema should know the correct validator even for composite
columns. The usage of the default validation class does not make sense.
Best Regards
Bert Passek
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4421) Invalid validator used for column values of composite columns - writing fails
[
https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bert Passek updated CASSANDRA-4421:
---
Attachment: (was: TestCompositeColumn.java)
Invalid validator used for column values of composite columns - writing fails
--
Key: CASSANDRA-4421
URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
Project: Cassandra
Issue Type: Bug
Components: API
Affects Versions: 1.1.2
Environment: Debian Squeeze
Reporter: bert Passek
Hello,
i faced a bug while writing composite column values and following validation
on server side.
This is the setup for reproduction:
1. create a keyspace
create keyspace test with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor = 1;
2. create a cf via cql (3.0)
create table test1 (
a int,
b int,
c int,
primary key (a, b)
);
If i have a look at the schema in cli i noticed that there is no column
metadata for columns not part of primary key.
create column family test1
with column_type = 'Standard'
and comparator =
'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'Int32Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
Please notice the default validation class: UTF8Type
Now i would like to insert value 127 via cassandra client (no cql, part of
mr-jobs). Have a look at the attachement.
Batch mutate fails:
InvalidRequestException(why:(String didn't validate.) [test][test1][1:c]
failed validation)
A validator for column value is fetched in
ThriftValidation::validateColumnData which returns always the default
validator which is UTF8Type as described above (The ColumnDefinition for
given column name c is always null)
In UTF8Type there is a check for
if (b 127)
return false;
Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table
creation. I assigned data types to all columns, but i can not set values for
a composite column because the default validation class is used.
I think the schema should know the correct validator even for composite
columns. The usage of the default validation class does not make sense.
Best Regards
Bert Passek
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4421) Invalid validator used for column values of composite columns - writing fails
[
https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13408182#comment-13408182
]
bert Passek commented on CASSANDRA-4421:
Ah, well the issue was already closed so i didn't care the attachement. But you
are right, i had problems inserting data from thrift (via client.batch_mutate)
to a cql3 cf. I just mentioned in the debugger, that ThriftValidation doesn't
know about metadata, so default validator is always used which results in
rejecting data on server side.
Sometimes it can be confusing when talking about cql 2, 3, cli,thrift etc. :)
Thanks.
Invalid validator used for column values of composite columns - writing fails
--
Key: CASSANDRA-4421
URL: https://issues.apache.org/jira/browse/CASSANDRA-4421
Project: Cassandra
Issue Type: Bug
Components: API
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
Assignee: Sylvain Lebresne
Labels: cql3
Fix For: 1.1.3
Hello,
i faced a bug while writing composite column values and following validation
on server side.
This is the setup for reproduction:
1. create a keyspace
create keyspace test with strategy_class = 'SimpleStrategy' and
strategy_options:replication_factor = 1;
2. create a cf via cql (3.0)
create table test1 (
a int,
b int,
c int,
primary key (a, b)
);
If i have a look at the schema in cli i noticed that there is no column
metadata for columns not part of primary key.
create column family test1
with column_type = 'Standard'
and comparator =
'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)'
and default_validation_class = 'UTF8Type'
and key_validation_class = 'Int32Type'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
Please notice the default validation class: UTF8Type
Now i would like to insert value 127 via cassandra client (no cql, part of
mr-jobs). Have a look at the attachement.
Batch mutate fails:
InvalidRequestException(why:(String didn't validate.) [test][test1][1:c]
failed validation)
A validator for column value is fetched in
ThriftValidation::validateColumnData which returns always the default
validator which is UTF8Type as described above (The ColumnDefinition for
given column name c is always null)
In UTF8Type there is a check for
if (b 127)
return false;
Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table
creation. I assigned data types to all columns, but i can not set values for
a composite column because the default validation class is used.
I think the schema should know the correct validator even for composite
columns. The usage of the default validation class does not make sense.
Best Regards
Bert Passek
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4229) Infinite MapReduce Task while reading via ColumnFamilyInputFormat
[
https://issues.apache.org/jira/browse/CASSANDRA-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403834#comment-13403834
]
bert Passek commented on CASSANDRA-4229:
Hello,
i spend some time for debugging and i could find the reason for the problem
described above.
This is the map function for reading from cassandra:
protected void map(ByteBuffer key, SortedMapByteBuffer, IColumn values,
Context context);
If you are using the row-key, you must duplicate the bytebuffer, otherwise the
RowIterator in ColumnFamilyRecordReader does not finish correctly.
This is one of the exit-condition in StaticRowIterator:
startToken =
partitioner.getTokenFactory().toString(partitioner.getToken(Iterables.getLast(rows).key));
if (startToken.equals(split.getEndToken()))
{
// reached end of the split
rows = null;
return;
}
Without duplicating the row-key you will never run into it. This kind of
behaviour was different to former versions of Cassandra und should be clearly
documented.
Best Regards
bert Passek
Infinite MapReduce Task while reading via ColumnFamilyInputFormat
-
Key: CASSANDRA-4229
URL: https://issues.apache.org/jira/browse/CASSANDRA-4229
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
Attachments: screenshot.jpg
Hi,
we recently upgraded cassandra from version 1.0.9 to 1.1.0. After that we can
not execute any hadoop jobs which reads data from cassandra via
ColumnFamilyInputFormat.
A map task is created which is running infinitely. We are trying to read from
a super column family with more or less 1000 row keys.
This is the output from job interface where we already have 17 million map
input records !!!
Map input records 17.273.127 0 17.273.127
Reduce shuffle bytes 0 391 391
Spilled Records 3.288 0 3.288
Map output bytes 639.849.351 0 639.849.351
CPU time spent (ms) 792.750 7.600 800.350
Total committed heap usage (bytes)354.680.832 48.955.392
403.636.224
Combine input records 17.039.783 0 17.039.783
SPLIT_RAW_BYTES 212 0 212
Reduce input records 0 0 0
Reduce input groups 0 0 0
Combine output records3.288 0 3.288
Physical memory (bytes) snapshot 510.275.584 96.370.688
606.646.272
Reduce output records 0 0 0
Virtual memory (bytes) snapshot 1.826.496.512 934.473.728
2.760.970.240
Map output records17.273.126 0 17.273.126
We must kill the job and we have to go back to version 1.0.9 because 1.1.0 is
not usable for reading from cassandra.
Best regards
Bert Passek
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4228) Exception while reading from cassandra via ColumnFamilyInputFormat and OrderPreservingPartitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290965#comment-13290965
]
bert Passek commented on CASSANDRA-4228:
Well, i just checked the cluster configuration and i had a misconfiguration
with one of our test clusters. We fixed this and it works like a charm :)
Exception while reading from cassandra via ColumnFamilyInputFormat and
OrderPreservingPartitioner
-
Key: CASSANDRA-4228
URL: https://issues.apache.org/jira/browse/CASSANDRA-4228
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
Attachments: CassandraTest.java
We recently updated cassandra from verison 1.0.8 to 1.1.0 on a debian squeeze
system. After that we can not use ColumnFamilyInputFormat anymore due to
exceptions in cassandra. A simple unit test is provided via attachement.
Here are some details about our simple setup:
Ring:
Address DC RackStatus State LoadOwns
Token
127.0.0.1 datacenter1 rack1 Up Normal 859.36 KB
100,00% 55894951196891831822413178196787984716
Schema Definition:
create column family TestSuper
with column_type = 'Super'
and comparator = 'BytesType'
and subcomparator = 'BytesType'
and default_validation_class = 'BytesType'
and key_validation_class = 'BytesType'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
While running the test we face following exception on client side:
12/05/09 10:18:22 INFO junit.TestRunner:
testColumnFamilyInputFormat(de.unister.cpc.tests.CassandraTest):
org.apache.thrift.transport.TTransportException
12/05/09 10:18:22 INFO junit.TestRunner: java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:391)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:397)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:323)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:188)
at
de.unister.cpc.tests.CassandraTest.testColumnFamilyInputFormat(CassandraTest.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
at org.junit.runners.Suite.runChild(Suite.java:115)
at org.junit.runners.Suite.runChild(Suite.java:23)
[jira] [Commented] (CASSANDRA-4229) Infinite MapReduce Task while reading via ColumnFamilyInputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13287346#comment-13287346 ] bert Passek commented on CASSANDRA-4229: I'm trying to make a setup for reproduction, will let you know. Infinite MapReduce Task while reading via ColumnFamilyInputFormat - Key: CASSANDRA-4229 URL: https://issues.apache.org/jira/browse/CASSANDRA-4229 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.0 Environment: Debian Squeeze Reporter: bert Passek Attachments: screenshot.jpg Hi, we recently upgraded cassandra from version 1.0.9 to 1.1.0. After that we can not execute any hadoop jobs which reads data from cassandra via ColumnFamilyInputFormat. A map task is created which is running infinitely. We are trying to read from a super column family with more or less 1000 row keys. This is the output from job interface where we already have 17 million map input records !!! Map input records 17.273.127 0 17.273.127 Reduce shuffle bytes 0 391 391 Spilled Records 3.288 0 3.288 Map output bytes 639.849.351 0 639.849.351 CPU time spent (ms) 792.750 7.600 800.350 Total committed heap usage (bytes)354.680.832 48.955.392 403.636.224 Combine input records 17.039.783 0 17.039.783 SPLIT_RAW_BYTES 212 0 212 Reduce input records 0 0 0 Reduce input groups 0 0 0 Combine output records3.288 0 3.288 Physical memory (bytes) snapshot 510.275.584 96.370.688 606.646.272 Reduce output records 0 0 0 Virtual memory (bytes) snapshot 1.826.496.512 934.473.728 2.760.970.240 Map output records17.273.126 0 17.273.126 We must kill the job and we have to go back to version 1.0.9 because 1.1.0 is not usable for reading from cassandra. Best regards Bert Passek -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4228) Exception while reading from cassandra via ColumnFamilyInputFormat and OrderPreservingPartitioner
bert Passek created CASSANDRA-4228:
--
Summary: Exception while reading from cassandra via
ColumnFamilyInputFormat and OrderPreservingPartitioner
Key: CASSANDRA-4228
URL: https://issues.apache.org/jira/browse/CASSANDRA-4228
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
We recently updated cassandra from verison 1.0.8 to 1.1.0 on a debian squeeze
system. After that we can not use ColumnFamilyInputFormat anymore due to
exceptions in cassandra. A simple unit test is provided via attachement.
Here are some details about our simple setup:
Ring:
Address DC RackStatus State LoadOwns
Token
127.0.0.1 datacenter1 rack1 Up Normal 859.36 KB 100,00%
55894951196891831822413178196787984716
Schema Definition:
create column family TestSuper
with column_type = 'Super'
and comparator = 'BytesType'
and subcomparator = 'BytesType'
and default_validation_class = 'BytesType'
and key_validation_class = 'BytesType'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
While running the test we face following exception on client side:
12/05/09 10:18:22 INFO junit.TestRunner:
testColumnFamilyInputFormat(de.unister.cpc.tests.CassandraTest):
org.apache.thrift.transport.TTransportException
12/05/09 10:18:22 INFO junit.TestRunner: java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:391)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:397)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:323)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:188)
at
de.unister.cpc.tests.CassandraTest.testColumnFamilyInputFormat(CassandraTest.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
at org.junit.runners.Suite.runChild(Suite.java:115)
at org.junit.runners.Suite.runChild(Suite.java:23)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
at
[jira] [Updated] (CASSANDRA-4228) Exception while reading from cassandra via ColumnFamilyInputFormat and OrderPreservingPartitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bert Passek updated CASSANDRA-4228:
---
Attachment: CassandraTest.java
Unit-Test ti reproduce described exception.
Exception while reading from cassandra via ColumnFamilyInputFormat and
OrderPreservingPartitioner
-
Key: CASSANDRA-4228
URL: https://issues.apache.org/jira/browse/CASSANDRA-4228
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
Attachments: CassandraTest.java
We recently updated cassandra from verison 1.0.8 to 1.1.0 on a debian squeeze
system. After that we can not use ColumnFamilyInputFormat anymore due to
exceptions in cassandra. A simple unit test is provided via attachement.
Here are some details about our simple setup:
Ring:
Address DC RackStatus State LoadOwns
Token
127.0.0.1 datacenter1 rack1 Up Normal 859.36 KB
100,00% 55894951196891831822413178196787984716
Schema Definition:
create column family TestSuper
with column_type = 'Super'
and comparator = 'BytesType'
and subcomparator = 'BytesType'
and default_validation_class = 'BytesType'
and key_validation_class = 'BytesType'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
While running the test we face following exception on client side:
12/05/09 10:18:22 INFO junit.TestRunner:
testColumnFamilyInputFormat(de.unister.cpc.tests.CassandraTest):
org.apache.thrift.transport.TTransportException
12/05/09 10:18:22 INFO junit.TestRunner: java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:391)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:397)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:323)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:188)
at
de.unister.cpc.tests.CassandraTest.testColumnFamilyInputFormat(CassandraTest.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
at org.junit.runners.Suite.runChild(Suite.java:115)
at org.junit.runners.Suite.runChild(Suite.java:23)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at
[jira] [Issue Comment Edited] (CASSANDRA-4228) Exception while reading from cassandra via ColumnFamilyInputFormat and OrderPreservingPartitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271209#comment-13271209
]
bert Passek edited comment on CASSANDRA-4228 at 5/9/12 8:29 AM:
Unit-Test to reproduce described exception.
was (Author: bertpassek):
Unit-Test ti reproduce described exception.
Exception while reading from cassandra via ColumnFamilyInputFormat and
OrderPreservingPartitioner
-
Key: CASSANDRA-4228
URL: https://issues.apache.org/jira/browse/CASSANDRA-4228
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
Attachments: CassandraTest.java
We recently updated cassandra from verison 1.0.8 to 1.1.0 on a debian squeeze
system. After that we can not use ColumnFamilyInputFormat anymore due to
exceptions in cassandra. A simple unit test is provided via attachement.
Here are some details about our simple setup:
Ring:
Address DC RackStatus State LoadOwns
Token
127.0.0.1 datacenter1 rack1 Up Normal 859.36 KB
100,00% 55894951196891831822413178196787984716
Schema Definition:
create column family TestSuper
with column_type = 'Super'
and comparator = 'BytesType'
and subcomparator = 'BytesType'
and default_validation_class = 'BytesType'
and key_validation_class = 'BytesType'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
While running the test we face following exception on client side:
12/05/09 10:18:22 INFO junit.TestRunner:
testColumnFamilyInputFormat(de.unister.cpc.tests.CassandraTest):
org.apache.thrift.transport.TTransportException
12/05/09 10:18:22 INFO junit.TestRunner: java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:391)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:397)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:323)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:188)
at
de.unister.cpc.tests.CassandraTest.testColumnFamilyInputFormat(CassandraTest.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
at org.junit.runners.Suite.runChild(Suite.java:115)
at
[jira] [Updated] (CASSANDRA-4228) Exception while reading from cassandra via ColumnFamilyInputFormat and OrderPreservingPartitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bert Passek updated CASSANDRA-4228:
---
Attachment: (was: CassandraTest.java)
Exception while reading from cassandra via ColumnFamilyInputFormat and
OrderPreservingPartitioner
-
Key: CASSANDRA-4228
URL: https://issues.apache.org/jira/browse/CASSANDRA-4228
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
We recently updated cassandra from verison 1.0.8 to 1.1.0 on a debian squeeze
system. After that we can not use ColumnFamilyInputFormat anymore due to
exceptions in cassandra. A simple unit test is provided via attachement.
Here are some details about our simple setup:
Ring:
Address DC RackStatus State LoadOwns
Token
127.0.0.1 datacenter1 rack1 Up Normal 859.36 KB
100,00% 55894951196891831822413178196787984716
Schema Definition:
create column family TestSuper
with column_type = 'Super'
and comparator = 'BytesType'
and subcomparator = 'BytesType'
and default_validation_class = 'BytesType'
and key_validation_class = 'BytesType'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
While running the test we face following exception on client side:
12/05/09 10:18:22 INFO junit.TestRunner:
testColumnFamilyInputFormat(de.unister.cpc.tests.CassandraTest):
org.apache.thrift.transport.TTransportException
12/05/09 10:18:22 INFO junit.TestRunner: java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:391)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:397)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:323)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:188)
at
de.unister.cpc.tests.CassandraTest.testColumnFamilyInputFormat(CassandraTest.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
at org.junit.runners.Suite.runChild(Suite.java:115)
at org.junit.runners.Suite.runChild(Suite.java:23)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
at
[jira] [Updated] (CASSANDRA-4228) Exception while reading from cassandra via ColumnFamilyInputFormat and OrderPreservingPartitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bert Passek updated CASSANDRA-4228:
---
Attachment: CassandraTest.java
Unit-Test for reproduction.
Exception while reading from cassandra via ColumnFamilyInputFormat and
OrderPreservingPartitioner
-
Key: CASSANDRA-4228
URL: https://issues.apache.org/jira/browse/CASSANDRA-4228
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
Attachments: CassandraTest.java
We recently updated cassandra from verison 1.0.8 to 1.1.0 on a debian squeeze
system. After that we can not use ColumnFamilyInputFormat anymore due to
exceptions in cassandra. A simple unit test is provided via attachement.
Here are some details about our simple setup:
Ring:
Address DC RackStatus State LoadOwns
Token
127.0.0.1 datacenter1 rack1 Up Normal 859.36 KB
100,00% 55894951196891831822413178196787984716
Schema Definition:
create column family TestSuper
with column_type = 'Super'
and comparator = 'BytesType'
and subcomparator = 'BytesType'
and default_validation_class = 'BytesType'
and key_validation_class = 'BytesType'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
While running the test we face following exception on client side:
12/05/09 10:18:22 INFO junit.TestRunner:
testColumnFamilyInputFormat(de.unister.cpc.tests.CassandraTest):
org.apache.thrift.transport.TTransportException
12/05/09 10:18:22 INFO junit.TestRunner: java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:391)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:397)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:323)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:188)
at
de.unister.cpc.tests.CassandraTest.testColumnFamilyInputFormat(CassandraTest.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
at org.junit.runners.Suite.runChild(Suite.java:115)
at org.junit.runners.Suite.runChild(Suite.java:23)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at
[jira] [Updated] (CASSANDRA-4228) Exception while reading from cassandra via ColumnFamilyInputFormat and OrderPreservingPartitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bert Passek updated CASSANDRA-4228:
---
Comment: was deleted
(was: Unit-Test to reproduce described exception.)
Exception while reading from cassandra via ColumnFamilyInputFormat and
OrderPreservingPartitioner
-
Key: CASSANDRA-4228
URL: https://issues.apache.org/jira/browse/CASSANDRA-4228
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
Attachments: CassandraTest.java
We recently updated cassandra from verison 1.0.8 to 1.1.0 on a debian squeeze
system. After that we can not use ColumnFamilyInputFormat anymore due to
exceptions in cassandra. A simple unit test is provided via attachement.
Here are some details about our simple setup:
Ring:
Address DC RackStatus State LoadOwns
Token
127.0.0.1 datacenter1 rack1 Up Normal 859.36 KB
100,00% 55894951196891831822413178196787984716
Schema Definition:
create column family TestSuper
with column_type = 'Super'
and comparator = 'BytesType'
and subcomparator = 'BytesType'
and default_validation_class = 'BytesType'
and key_validation_class = 'BytesType'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
While running the test we face following exception on client side:
12/05/09 10:18:22 INFO junit.TestRunner:
testColumnFamilyInputFormat(de.unister.cpc.tests.CassandraTest):
org.apache.thrift.transport.TTransportException
12/05/09 10:18:22 INFO junit.TestRunner: java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:391)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:397)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:323)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:188)
at
de.unister.cpc.tests.CassandraTest.testColumnFamilyInputFormat(CassandraTest.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
at org.junit.runners.Suite.runChild(Suite.java:115)
at org.junit.runners.Suite.runChild(Suite.java:23)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
at
[jira] [Created] (CASSANDRA-4229) Infinite MapReduce Task while reading via ColumnFamilyInputFormat
bert Passek created CASSANDRA-4229: -- Summary: Infinite MapReduce Task while reading via ColumnFamilyInputFormat Key: CASSANDRA-4229 URL: https://issues.apache.org/jira/browse/CASSANDRA-4229 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.0 Environment: Debian Squeeze Reporter: bert Passek Attachments: screenshot.jpg Hi, we recently upgraded cassandra from version 1.0.9 to 1.1.0. After that we can not execute any hadoop jobs which reads data from cassandra via ColumnFamilyInputFormat. A map task is created which is running infinitely. We are trying to read from a super column family with more or less 1000 row keys. This is the output from job interface where we already have 17 million map input records !!! Map input records 17.273.127 0 17.273.127 Reduce shuffle bytes0 391 391 Spilled Records 3.288 0 3.288 Map output bytes639.849.351 0 639.849.351 CPU time spent (ms) 792.750 7.600 800.350 Total committed heap usage (bytes) 354.680.832 48.955.392 403.636.224 Combine input records 17.039.783 0 17.039.783 SPLIT_RAW_BYTES 212 0 212 Reduce input records0 0 0 Reduce input groups 0 0 0 Combine output records 3.288 0 3.288 Physical memory (bytes) snapshot510.275.584 96.370.688 606.646.272 Reduce output records 0 0 0 Virtual memory (bytes) snapshot 1.826.496.512 934.473.728 2.760.970.240 Map output records 17.273.126 0 17.273.126 We must kill the job and we have to go back to version 1.0.9 because 1.1.0 is not usable for reading from cassandra. Best regards Bert Passek -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4229) Infinite MapReduce Task while reading via ColumnFamilyInputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bert Passek updated CASSANDRA-4229: --- Attachment: screenshot.jpg Screenshot from map task with almost 30.000% progress. Infinite MapReduce Task while reading via ColumnFamilyInputFormat - Key: CASSANDRA-4229 URL: https://issues.apache.org/jira/browse/CASSANDRA-4229 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.0 Environment: Debian Squeeze Reporter: bert Passek Attachments: screenshot.jpg Hi, we recently upgraded cassandra from version 1.0.9 to 1.1.0. After that we can not execute any hadoop jobs which reads data from cassandra via ColumnFamilyInputFormat. A map task is created which is running infinitely. We are trying to read from a super column family with more or less 1000 row keys. This is the output from job interface where we already have 17 million map input records !!! Map input records 17.273.127 0 17.273.127 Reduce shuffle bytes 0 391 391 Spilled Records 3.288 0 3.288 Map output bytes 639.849.351 0 639.849.351 CPU time spent (ms) 792.750 7.600 800.350 Total committed heap usage (bytes)354.680.832 48.955.392 403.636.224 Combine input records 17.039.783 0 17.039.783 SPLIT_RAW_BYTES 212 0 212 Reduce input records 0 0 0 Reduce input groups 0 0 0 Combine output records3.288 0 3.288 Physical memory (bytes) snapshot 510.275.584 96.370.688 606.646.272 Reduce output records 0 0 0 Virtual memory (bytes) snapshot 1.826.496.512 934.473.728 2.760.970.240 Map output records17.273.126 0 17.273.126 We must kill the job and we have to go back to version 1.0.9 because 1.1.0 is not usable for reading from cassandra. Best regards Bert Passek -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4231) ConcurrentModificationException while writing mutations with ColumnFamilyRecordWrite
bert Passek created CASSANDRA-4231: -- Summary: ConcurrentModificationException while writing mutations with ColumnFamilyRecordWrite Key: CASSANDRA-4231 URL: https://issues.apache.org/jira/browse/CASSANDRA-4231 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.0.10 Environment: Debian Squeeze, one local cassandra node Reporter: bert Passek Hello, we are using mapreduce jobs for writing data into cassandra. Sometimes the job fails because of a concurrent modification exception. java.io.IOException: java.util.ConcurrentModificationException at org.apache.cassandra.hadoop.ColumnFamilyRecordWriter$RangeClient.run(ColumnFamilyRecordWriter.java:307) Caused by: java.util.ConcurrentModificationException at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372) at java.util.AbstractList$Itr.next(AbstractList.java:343) at org.apache.cassandra.thrift.SuperColumn.write(SuperColumn.java:440) at org.apache.cassandra.thrift.ColumnOrSuperColumn.write(ColumnOrSuperColumn.java:561) at org.apache.cassandra.thrift.Mutation.write(Mutation.java:384) at org.apache.cassandra.thrift.Cassandra$batch_mutate_args.write(Cassandra.java:19021) at org.apache.cassandra.thrift.Cassandra$Client.send_batch_mutate(Cassandra.java:1018) at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:1008) at org.apache.cassandra.hadoop.ColumnFamilyRecordWriter$RangeClient.run(ColumnFamilyRecordWriter.java:299) We were using Cassandra 1.0.8 for quite a long time without such problems. Regards Bert Passek -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4228) Exception while reading from cassandra via ColumnFamilyInputFormat and OrderPreservingPartitioner
[
https://issues.apache.org/jira/browse/CASSANDRA-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271636#comment-13271636
]
bert Passek commented on CASSANDRA-4228:
I already noticed the RandomPartitioner in the stack trace. Data was written to
Cassandra by a hadoop job with configured OrderPreservingPartitioner. A
different job reads from Cassandra where the partitioner in the job
configuration was also set to OrderPreservingPartitioner.
We haven't actually changed any hadoop jobs, we just updated cassandra from
1.0.8 to 1.1.0. And then we ran into this exception. The test case was written
to track down the problem. It's strange because the exception is thrown even if
we are trying to read from empty column families.
I'm gonna check the cluster and job configuration again, i might have setup
something wrong.
Exception while reading from cassandra via ColumnFamilyInputFormat and
OrderPreservingPartitioner
-
Key: CASSANDRA-4228
URL: https://issues.apache.org/jira/browse/CASSANDRA-4228
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.1.0
Environment: Debian Squeeze
Reporter: bert Passek
Attachments: CassandraTest.java
We recently updated cassandra from verison 1.0.8 to 1.1.0 on a debian squeeze
system. After that we can not use ColumnFamilyInputFormat anymore due to
exceptions in cassandra. A simple unit test is provided via attachement.
Here are some details about our simple setup:
Ring:
Address DC RackStatus State LoadOwns
Token
127.0.0.1 datacenter1 rack1 Up Normal 859.36 KB
100,00% 55894951196891831822413178196787984716
Schema Definition:
create column family TestSuper
with column_type = 'Super'
and comparator = 'BytesType'
and subcomparator = 'BytesType'
and default_validation_class = 'BytesType'
and key_validation_class = 'BytesType'
and read_repair_chance = 0.1
and dclocal_read_repair_chance = 0.0
and gc_grace = 864000
and min_compaction_threshold = 4
and max_compaction_threshold = 32
and replicate_on_write = true
and compaction_strategy =
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy'
and caching = 'KEYS_ONLY'
and compression_options = {'sstable_compression' :
'org.apache.cassandra.io.compress.SnappyCompressor'};
While running the test we face following exception on client side:
12/05/09 10:18:22 INFO junit.TestRunner:
testColumnFamilyInputFormat(de.unister.cpc.tests.CassandraTest):
org.apache.thrift.transport.TTransportException
12/05/09 10:18:22 INFO junit.TestRunner: java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:391)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:397)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:323)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:188)
at
de.unister.cpc.tests.CassandraTest.testColumnFamilyInputFormat(CassandraTest.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:73)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:46)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
[jira] [Commented] (CASSANDRA-4229) Infinite MapReduce Task while reading via ColumnFamilyInputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-4229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13271643#comment-13271643 ] bert Passek commented on CASSANDRA-4229: Yes, i can reproduce it. Actually our developing environment simply consists of a single node. The hadoop job is very simple, just reading data from Cassandra and writing back to Cassandra. Infinite MapReduce Task while reading via ColumnFamilyInputFormat - Key: CASSANDRA-4229 URL: https://issues.apache.org/jira/browse/CASSANDRA-4229 Project: Cassandra Issue Type: Bug Components: Hadoop Affects Versions: 1.1.0 Environment: Debian Squeeze Reporter: bert Passek Attachments: screenshot.jpg Hi, we recently upgraded cassandra from version 1.0.9 to 1.1.0. After that we can not execute any hadoop jobs which reads data from cassandra via ColumnFamilyInputFormat. A map task is created which is running infinitely. We are trying to read from a super column family with more or less 1000 row keys. This is the output from job interface where we already have 17 million map input records !!! Map input records 17.273.127 0 17.273.127 Reduce shuffle bytes 0 391 391 Spilled Records 3.288 0 3.288 Map output bytes 639.849.351 0 639.849.351 CPU time spent (ms) 792.750 7.600 800.350 Total committed heap usage (bytes)354.680.832 48.955.392 403.636.224 Combine input records 17.039.783 0 17.039.783 SPLIT_RAW_BYTES 212 0 212 Reduce input records 0 0 0 Reduce input groups 0 0 0 Combine output records3.288 0 3.288 Physical memory (bytes) snapshot 510.275.584 96.370.688 606.646.272 Reduce output records 0 0 0 Virtual memory (bytes) snapshot 1.826.496.512 934.473.728 2.760.970.240 Map output records17.273.126 0 17.273.126 We must kill the job and we have to go back to version 1.0.9 because 1.1.0 is not usable for reading from cassandra. Best regards Bert Passek -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
