Looking at this closer. I noticed the following in the SSTableImport Class:
if (col.isDeleted) { cfamily.addColumn(path, hexToBytes(col.value), new TimestampClock(col.timestamp)); } else { cfamily.addTombstone(path, hexToBytes(col.value), new TimestampClock(col.timestamp)); } This appears to be backwards. On Tue, Nov 16, 2010 at 4:03 PM, CassUser CassUser <cassu...@gmail.com>wrote: > Looked at how DatabaseDescriptor is loading the yaml file. Using that > approach solves the problem with the column_families mapping exception. > > The problems we are running into currently is regarding a known dataset not > being loaded into our test instance correctly. > > Steps: > 1. Create temp directory to host cassandra test instance > 2. Create keyspace directories > 3. Update yaml file, and copy to test location > 4. Add schema from yaml using DatabaseDescriptor. > readTablesFromYaml() & DatabaseDescriptor. > setTableDefinition > 4. Use SSTableImport.importJson to setup a known dataset. We have JSON > files converted to 0.7 compliant (byte [] rows) > 5. Start Cassandra instance using the EmbeddedCassandraService class > > Everything appears to work from the log messages. I get the message " > Sampling index for ..", and finally "Listening for thrift clients...". When > i use a client to query the data in test instance, I notice keys with no > columns/values stored (via keyrange slice). Using a open ended column slice > for a known key I'm getting zero results. > > I'm currently running cassandra 0.7. beta2. Is the steps I've outlined > above suppose to work in 0.7? We did something similar in 0.6.4 without any > problems. Is there known bugs I can look into? > > Thanks. > > > > On Tue, Nov 16, 2010 at 12:24 PM, Aaron Morton <aa...@thelastpickle.com>wrote: > >> I've not used the embedded service. >> >> The code in o.a.c.service.EmbeddedCassandraService says it will read the >> yaml file. If the cluster does not have a schema stored I think it will load >> the one from yaml. >> >> Have you tried starting it up with an empty system data dir ? Does it >> pickup the schema from the yaml? >> >> Aaron >> >> On 17 Nov, 2010,at 09:17 AM, CassUser CassUser <cassu...@gmail.com> >> wrote: >> >> Loading yaml file like so: >> FileInputStream yamlInputStream = new FileInputStream( >> configTemplateFile); >> Constructor constructor = new Constructor(Config.class); >> Yaml yaml = new Yaml(new Loader(constructor)); >> Config conf = (Config) yaml.load(yamlInputStream); >> >> Fails on the last line. >> >> Although if i have the CFs defined like this: >> keyspaces: >> - column_families: >> - !!org.apache.cassandra.config.RawColumnFamily >> column_metadata: [] >> column_type: null >> comment: null >> compare_subcolumns_with: null >> compare_with: BytesType >> default_validation_class: null >> gc_grace_seconds: 864000 >> keys_cached: 200000.0 >> max_compaction_threshold: 32 >> min_compaction_threshold: 4 >> name: Similarity >> preload_row_cache: false >> read_repair_chance: 1.0 >> rows_cached: 0.0 >> >> >> it appears to startup. Any idea whats going on here? >> >> >> On Tue, Nov 16, 2010 at 11:58 AM, CassUser CassUser >> <cassu...@gmail.com>wrote: >> >>> This is embedded for testing cassandra 0.7 beta2. using >>> EmbeddedCassandraService. >>> >>> and manually adding schema programmatically using: >>> for (KSMetaData table : DatabaseDescriptor.readTablesFromYaml()) >>> { >>> for (CFMetaData cfm : table.cfMetaData().values()) { >>> CFMetaData.map(cfm); >>> } >>> DatabaseDescriptor.setTableDefinition(table, >>> DatabaseDescriptor.getDefsVersion()); >>> } >>> >>> Is this the correct way to start up a test server, with the schema >>> loaded? >>> >>> >>> >>> >>> On Tue, Nov 16, 2010 at 11:41 AM, Aaron Morton >>> <aa...@thelastpickle.com>wrote: >>> >>>> AFAIK the ArrayStoreException is similar to a type mismatch. Is it >>>> possible you have something mixed up in your class path or source code if >>>> you built from source? >>>> >>>> It looks like the column family info was deserialised into a >>>> o.a.c.config.RawColumnFamily but when that object was added to the >>>> RawColumnFamily[] array on o.a.c.config.RawKeyspace if was the wrong type. >>>> >>>> Have you tried a clean build ? There are some things in the call stack >>>> which look custom, are you starting from the command line or is this >>>> embedded for testing? >>>> >>>> Hope that helps. >>>> Aaron >>>> >>>> >>>> >>>> >>>> >>>> On 17 Nov, 2010,at 08:09 AM, CassUser CassUser <cassu...@gmail.com> >>>> wrote: >>>> >>>> >>>> Here is the yaml: >>>> >>>> # Cassandra YAML generated from previous config >>>> # Configuration wiki: >>>> http://wiki.apache.org/cassandra/StorageConfiguration >>>> authenticator: org.apache.cassandraauth.AllowAllAuthenticator >>>> >>>> auto_bootstrap: false >>>> binary_memtable_throughput_in_mb: 256 >>>> cluster_name: Test Cluster >>>> column_index_size_in_kb: 64 >>>> commitlog_rotation_threshold_in_mb: 128 >>>> commitlog_sync: periodic >>>> commitlog_sync_period_in_ms: 10000 >>>> compaction_thread_priority: 1 >>>> concurrent_reads: 8 >>>> concurrent_writes: 32 >>>> disk_access_mode: auto >>>> dynamic_snitch: false >>>> endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch >>>> request_scheduler: org.apache.cassandra.scheduler.RoundRobinScheduler >>>> request_scheduler_id: keyspace >>>> hinted_handoff_enabled: true >>>> in_memory_compaction_limit_in_mb: 128 >>>> index_interval: 128 >>>> keyspaces: >>>> - name: myKeyspace >>>> replica_placement_strategy: >>>> orgapache.cassandra.locator.SimpleStrategy >>>> >>>> >>>> replication_factor: 1 >>>> column_families: >>>> - name: Standard1 >>>> rows_cached: 100 >>>> keys_cached: 1 >>>> compare_with: UTF8Type >>>> memtable_flush_after_mins: 60 >>>> memtable_operations_in_millions: 0.3 >>>> memtable_throughput_in_mb: 64 >>>> partitioner: org.apache.cassandra.dht.RandomPartitioner >>>> phi_convict_threshold: 8 >>>> rpc_keepalive: true >>>> rpc_port: 9160 >>>> rpc_timeout_in_ms: 10000 >>>> sliced_buffer_size_in_kb: 64 >>>> snapshot_before_compaction: false >>>> storage_port: 7000 >>>> thrift_framed_transport_size_in_mb: 15 >>>> thrift_max_message_length_in_mb: 16 >>>> >>>> and here is the exception i'm receiving: >>>> >>>> Caused by: Can't construct a java object for >>>> tag:yaml.org,2002:org.apache.cassandra.config.Config; >>>> exception=Cannot create property=keyspaces for Java >>>> bean=org.apache.cassandra.config.con...@12e43f1; Cannot create >>>> property=column_families for >>>> javabean=org.apache.cassandra.config.rawkeysp...@1a8bd74; >>>> null >>>> in "<reader>", line 3, column 1: >>>> authenticator: org.apache.cassan ... >>>> ^ >>>> >>>> at >>>> org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:372) >>>> at >>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177) >>>> at >>>> org.yaml.snakeyamlconstructor.BaseConstructorconstructDocument(BaseConstructor.java:136) >>>> >>>> at >>>> org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:122) >>>> at org.yaml.snakeyaml.Loader.load(Loader.java:52) >>>> at org.yaml.snakeyaml.Yaml.load(Yaml.java:166) >>>> at >>>> com.atsid.cassandra.testutils.CassandraTestRunner.updateYamlConfig(CassandraTestRunner.java:134) >>>> at >>>> com.atsid.cassandra.testutils.CassandraTestRunner.init(CassandraTestRunner.java:119) >>>> at >>>> com.atsid.cassandra.testutils.CassandraTestRunner.main(CassandraTestRunner.java:272) >>>> ... 6 more >>>> Caused by: org.yaml.snakeyaml.error.YAMLException: Cannot create >>>> property=keyspaces for javabean=org.apache.cassandra.config.con...@12e43f1; >>>> Cannot cr >>>> eate property=column_families for >>>> javabean=org.apache.cassandra.config.rawkeysp...@1a8bd74; null >>>> at >>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:305) >>>> at >>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:184) >>>> at >>>> org.yaml.snakeyaml.constructor.Constructor$ConstructYamlObject.construct(Constructor.java:370) >>>> ... 14 more >>>> Caused by: org.yaml.snakeyaml.error.YAMLException: Cannot create >>>> property=column_families for >>>> javabean=org.apache.cassandra.config.rawkeysp...@1a8bd74 >>>> ; null >>>> at >>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:305) >>>> at >>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.construct(Constructor.java:184) >>>> at >>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177) >>>> at >>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructSequenceStep2(BaseConstructor.java:239) >>>> at >>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructSequence(BaseConstructor.java:233) >>>> at >>>> org.yaml.snakeyaml.constructorConstructor$ConstructSequence.construct(Constructor.java:543) >>>> >>>> at >>>> org.yaml.snakeyaml.constructor.BaseConstructor.constructObject(BaseConstructor.java:177) >>>> at >>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:298) >>>> ... 16 more >>>> Caused by: java.lang.ArrayStoreException >>>> at java.lang.System.arraycopy(Native Method) >>>> at java.util.Arrays.copyOf(Arrays.java:2763) >>>> at java.util.ArrayList.toArray(ArrayList.java:305) >>>> at >>>> org.yaml.snakeyaml.constructor.Constructor$ConstructMapping.constructJavaBean2ndStep(Constructor.java:301) >>>> ... 23 more >>>> >>>> >>> >> >