[jira] [Commented] (CASSANDRA-4421) Support cql3 table definitions in Hadoop InputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13651767#comment-13651767 ] Cyril Scetbon commented on CASSANDRA-4421: -- [~alexliu68] Oh, I was thinking that only thrift was affected by the CQL3 issue and that it was why pig was not working. If Pig needs to be updated too (as you say), I'll wait for that with hope that it will come soon… Can you change the status of [CASSANDRA-5234|https://issues.apache.org/jira/browse/CASSANDRA-5234] otherwise it won't be assigned unless there is another JIRA for that ? [~jbellis] I thought it was ONE whole block Support cql3 table definitions in Hadoop InputFormat Key: CASSANDRA-4421 URL: https://issues.apache.org/jira/browse/CASSANDRA-4421 Project: Cassandra Issue Type: Improvement Components: API Affects Versions: 1.1.0 Environment: Debian Squeeze Reporter: bert Passek Labels: cql3 Fix For: 1.2.5 Attachments: 4421-1.txt, 4421-2.txt, 4421.txt Hello, i faced a bug while writing composite column values and following validation on server side. This is the setup for reproduction: 1. create a keyspace create keyspace test with strategy_class = 'SimpleStrategy' and strategy_options:replication_factor = 1; 2. create a cf via cql (3.0) create table test1 ( a int, b int, c int, primary key (a, b) ); If i have a look at the schema in cli i noticed that there is no column metadata for columns not part of primary key. create column family test1 with column_type = 'Standard' and comparator = 'CompositeType(org.apache.cassandra.db.marshal.Int32Type,org.apache.cassandra.db.marshal.UTF8Type)' and default_validation_class = 'UTF8Type' and key_validation_class = 'Int32Type' and read_repair_chance = 0.1 and dclocal_read_repair_chance = 0.0 and gc_grace = 864000 and min_compaction_threshold = 4 and max_compaction_threshold = 32 and replicate_on_write = true and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy' and caching = 'KEYS_ONLY' and compression_options = {'sstable_compression' : 'org.apache.cassandra.io.compress.SnappyCompressor'}; Please notice the default validation class: UTF8Type Now i would like to insert value 127 via cassandra client (no cql, part of mr-jobs). Have a look at the attachement. Batch mutate fails: InvalidRequestException(why:(String didn't validate.) [test][test1][1:c] failed validation) A validator for column value is fetched in ThriftValidation::validateColumnData which returns always the default validator which is UTF8Type as described above (The ColumnDefinition for given column name c is always null) In UTF8Type there is a check for if (b 127) return false; Anyway, maybe i'm doing something wrong, but i used cql 3.0 for table creation. I assigned data types to all columns, but i can not set values for a composite column because the default validation class is used. I think the schema should know the correct validator even for composite columns. The usage of the default validation class does not make sense. Best Regards Bert Passek -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5550) nodetool repair hanging on multi sites config
Kévin LOVATO created CASSANDRA-5550: --- Summary: nodetool repair hanging on multi sites config Key: CASSANDRA-5550 URL: https://issues.apache.org/jira/browse/CASSANDRA-5550 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 1.2.4 Reporter: Kévin LOVATO We recently added a new datacenter to our cluster, and we want some keyspaces to be replicated to it, but not all. So some of our Keyspaces have a replication factor of {code}{ dcMain: 3, dcNew: 1 }{code}and some have {code}{ dcMain: 3 }{code}We then ran a nodetool repair on one of the new servers so the data would be replicated correctly, but it just hung. After some investigation, we saw that if a Keyspace didn't have replication info for dcNew, or a replication factor of 0, a nodetool repair from the new servers would hang (nothing concerning repair in the logs, not even in DEBUG level). On the other hand, it works properly if the Keyspace has a replica in dcNew or if the repair is called from dcMain. We might be doing something wrong, but I don't think nodetool should hang silently even if something is going wrong. Nodetool repair -pr also works in all situations, but it seems that it is not copying the data to the new datacenter at all, it runs and logs that the keyspace is up to date after receiving the merkle trees, even though nothing is actually copied on the disk, even after flushing (maybe an unrelated issue, should I open another ticket?). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-5550) nodetool repair hanging on multi sites config
[ https://issues.apache.org/jira/browse/CASSANDRA-5550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-5550. --- Resolution: Duplicate nodetool repair hanging on multi sites config - Key: CASSANDRA-5550 URL: https://issues.apache.org/jira/browse/CASSANDRA-5550 Project: Cassandra Issue Type: Bug Components: Tools Affects Versions: 1.2.4 Reporter: Kévin LOVATO We recently added a new datacenter to our cluster, and we want some keyspaces to be replicated to it, but not all. So some of our Keyspaces have a replication factor of {code}{ dcMain: 3, dcNew: 1 }{code}and some have {code}{ dcMain: 3 }{code}We then ran a nodetool repair on one of the new servers so the data would be replicated correctly, but it just hung. After some investigation, we saw that if a Keyspace didn't have replication info for dcNew, or a replication factor of 0, a nodetool repair from the new servers would hang (nothing concerning repair in the logs, not even in DEBUG level). On the other hand, it works properly if the Keyspace has a replica in dcNew or if the repair is called from dcMain. We might be doing something wrong, but I don't think nodetool should hang silently even if something is going wrong. Nodetool repair -pr also works in all situations, but it seems that it is not copying the data to the new datacenter at all, it runs and logs that the keyspace is up to date after receiving the merkle trees, even though nothing is actually copied on the disk, even after flushing (maybe an unrelated issue, should I open another ticket?). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5540) Concurrent secondary index updates remove rows from the index
[ https://issues.apache.org/jira/browse/CASSANDRA-5540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13651991#comment-13651991 ] Jonathan Ellis commented on CASSANDRA-5540: --- How does this avoid leaking index entries that will never be cleaned up by compaction? Concurrent secondary index updates remove rows from the index - Key: CASSANDRA-5540 URL: https://issues.apache.org/jira/browse/CASSANDRA-5540 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.4 Reporter: Alexei Bakanov Assignee: Sam Tunnicliffe Attachments: 0001-Use-different-index-updater-for-live-updates-compact.patch Existing rows disappear from secondary index when doing simultaneous updates of a row with the same secondary index value. Here is a little pycassa script that reproduces a bug. The script inserts 4 rows with same secondary index value, reads those rows back and check that there are 4 of them. Please run two instances of the script simultaneously in two separate terminals in order to simulate concurrent updates: {code} -scrpit.py START- import pycassa from pycassa.index import * pool = pycassa.ConnectionPool('ks123') cf = pycassa.ColumnFamily(pool, 'cf1') while True: for rowKey in xrange(4): cf.insert(str(rowKey), {'indexedColumn': 'indexedValue'}) index_expression = create_index_expression('indexedColumn', 'indexedValue') index_clause = create_index_clause([index_expression]) rows = cf.get_indexed_slices(index_clause) length = len(list(rows)) if length == 4: pass else: print 'found just %d rows out of 4' % length pool.dispose() ---script.py FINISH--- ---schema cli start--- create keyspace ks123 with placement_strategy = 'NetworkTopologyStrategy' and strategy_options = {datacenter1 : 1} and durable_writes = true; use ks123; create column family cf1 with column_type = 'Standard' and comparator = 'AsciiType' and default_validation_class = 'AsciiType' and key_validation_class = 'AsciiType' and read_repair_chance = 0.1 and dclocal_read_repair_chance = 0.0 and populate_io_cache_on_flush = false and gc_grace = 864000 and min_compaction_threshold = 4 and max_compaction_threshold = 32 and replicate_on_write = true and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy' and caching = 'KEYS_ONLY' and column_metadata = [ {column_name : 'indexedColumn', validation_class : AsciiType, index_name : 'INDEX1', index_type : 0}] and compression_options = {'sstable_compression' : 'org.apache.cassandra.io.compress.SnappyCompressor'}; ---schema cli finish--- {code} Test cluster created with 'ccm create --cassandra-version 1.2.4 --nodes 1 --start testUpdate' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5543) Ant issues when building gen-cql2-grammar
[ https://issues.apache.org/jira/browse/CASSANDRA-5543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13651998#comment-13651998 ] Jonathan Ellis commented on CASSANDRA-5543: --- Maybe [~anto...@apache.org] can shed some light here as well. Ant issues when building gen-cql2-grammar - Key: CASSANDRA-5543 URL: https://issues.apache.org/jira/browse/CASSANDRA-5543 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.3 Reporter: Joaquin Casares Priority: Trivial Below are the commands and outputs that were returned. The first `ant` command fails on gen-cql2-grammar, but if I don't run `ant realclean` then it works fine after a second pass. {CODE} ubuntu@ip-10-196-153-29:~/.ccm/repository/1.2.3$ ant realclean Buildfile: /home/ubuntu/.ccm/repository/1.2.3/build.xml clean: [delete] Deleting directory /home/ubuntu/.ccm/repository/1.2.3/build/test [delete] Deleting directory /home/ubuntu/.ccm/repository/1.2.3/build/classes [delete] Deleting directory /home/ubuntu/.ccm/repository/1.2.3/src/gen-java [delete] Deleting: /home/ubuntu/.ccm/repository/1.2.3/build/internode.avpr realclean: [delete] Deleting directory /home/ubuntu/.ccm/repository/1.2.3/build BUILD SUCCESSFUL Total time: 0 seconds {CODE} {CODE} ubuntu@ip-10-196-153-29:~/.ccm/repository/1.2.3$ ant Buildfile: /home/ubuntu/.ccm/repository/1.2.3/build.xml maven-ant-tasks-localrepo: maven-ant-tasks-download: [echo] Downloading Maven ANT Tasks... [mkdir] Created dir: /home/ubuntu/.ccm/repository/1.2.3/build [get] Getting: http://repo2.maven.org/maven2/org/apache/maven/maven-ant-tasks/2.1.3/maven-ant-tasks-2.1.3.jar [get] To: /home/ubuntu/.ccm/repository/1.2.3/build/maven-ant-tasks-2.1.3.jar maven-ant-tasks-init: [mkdir] Created dir: /home/ubuntu/.ccm/repository/1.2.3/build/lib maven-declare-dependencies: maven-ant-tasks-retrieve-build: [artifact:dependencies] Downloading: asm/asm/3.2/asm-3.2-sources.jar from repository central at http://repo1.maven.org/maven2 [artifact:dependencies] [INFO] Unable to find resource 'hsqldb:hsqldb:java-source:sources:1.8.0.10' in repository java.net2 (http://download.java.net/maven/2) [artifact:dependencies] Building ant file: /home/ubuntu/.ccm/repository/1.2.3/build/build-dependencies.xml [copy] Copying 45 files to /home/ubuntu/.ccm/repository/1.2.3/build/lib/jars [copy] Copying 35 files to /home/ubuntu/.ccm/repository/1.2.3/build/lib/sources init: [mkdir] Created dir: /home/ubuntu/.ccm/repository/1.2.3/build/classes/main [mkdir] Created dir: /home/ubuntu/.ccm/repository/1.2.3/build/classes/thrift [mkdir] Created dir: /home/ubuntu/.ccm/repository/1.2.3/build/test/lib [mkdir] Created dir: /home/ubuntu/.ccm/repository/1.2.3/build/test/classes [mkdir] Created dir: /home/ubuntu/.ccm/repository/1.2.3/src/gen-java check-avro-generate: avro-interface-generate-internode: [echo] Generating Avro internode code... avro-generate: build-subprojects: check-gen-cli-grammar: gen-cli-grammar: [echo] Building Grammar /home/ubuntu/.ccm/repository/1.2.3/src/java/org/apache/cassandra/cli/Cli.g check-gen-cql2-grammar: gen-cql2-grammar: [echo] Building Grammar /home/ubuntu/.ccm/repository/1.2.3/src/java/org/apache/cassandra/cql/Cql.g ... [java] warning(200): /home/ubuntu/.ccm/repository/1.2.3/src/java/org/apache/cassandra/cql/Cql.g:479:20: Decision can match input such as IDENT using multiple alternatives: 1, 2 [java] As a result, alternative(s) 2 were disabled for that input [java] warning(200): /home/ubuntu/.ccm/repository/1.2.3/src/java/org/apache/cassandra/cql/Cql.g:479:20: Decision can match input such as K_KEY using multiple alternatives: 1, 2 [java] As a result, alternative(s) 2 were disabled for that input [java] warning(200): /home/ubuntu/.ccm/repository/1.2.3/src/java/org/apache/cassandra/cql/Cql.g:479:20: Decision can match input such as QMARK using multiple alternatives: 1, 2 [java] As a result, alternative(s) 2 were disabled for that input [java] warning(200): /home/ubuntu/.ccm/repository/1.2.3/src/java/org/apache/cassandra/cql/Cql.g:479:20: Decision can match input such as FLOAT using multiple alternatives: 1, 2 [java] As a result, alternative(s) 2 were disabled for that input [java] warning(200): /home/ubuntu/.ccm/repository/1.2.3/src/java/org/apache/cassandra/cql/Cql.g:479:20: Decision can match input such as STRING_LITERAL using multiple alternatives: 1, 2 [java] As a result, alternative(s) 2 were disabled for that input [java] warning(200): /home/ubuntu/.ccm/repository/1.2.3/src/java/org/apache/cassandra/cql/Cql.g:479:20: Decision can
[2/2] git commit: r/m redundant log line on startup
r/m redundant log line on startup Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ae0d6ebd Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ae0d6ebd Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ae0d6ebd Branch: refs/heads/trunk Commit: ae0d6ebd26389d9504456cf96582eb76c24c9ff3 Parents: 7825361 Author: Jonathan Ellis jbel...@apache.org Authored: Wed May 8 17:01:06 2013 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Wed May 8 17:01:06 2013 -0500 -- .../cassandra/config/DatabaseDescriptor.java |2 -- 1 files changed, 0 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ae0d6ebd/src/java/org/apache/cassandra/config/DatabaseDescriptor.java -- diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java index 89f4c77..8b1b223 100644 --- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java +++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java @@ -377,8 +377,6 @@ public class DatabaseDescriptor logger.info((conf.multithreaded_compaction ? : Not ) + using multi-threaded compaction); -logger.info((conf.multithreaded_compaction ? : Not ) + using multi-threaded compaction); - if (conf.in_memory_compaction_limit_in_mb != null conf.in_memory_compaction_limit_in_mb = 0) { throw new ConfigurationException(in_memory_compaction_limit_in_mb must be a positive integer);
[1/2] git commit: whitespace
Updated Branches: refs/heads/cassandra-1.2 29ad6bdca - 6db71b29d refs/heads/trunk 782536176 - ae0d6ebd2 whitespace Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6db71b29 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6db71b29 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6db71b29 Branch: refs/heads/cassandra-1.2 Commit: 6db71b29d754eebdb64a9eab802b96af1e01595d Parents: 29ad6bd Author: Jonathan Ellis jbel...@apache.org Authored: Wed May 8 16:37:14 2013 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Wed May 8 16:37:14 2013 -0500 -- CHANGES.txt |1 + 1 files changed, 1 insertions(+), 0 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6db71b29/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 1abfc65..ca3f130 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -29,6 +29,7 @@ Merged from 1.1 * Fix trying to load deleted row into row cache on startup (CASSANDRA-4463) * fsync leveled manifest to avoid corruption (CASSANDRA-5535) + 1.2.4 * Ensure that PerRowSecondaryIndex updates see the most recent values (CASSANDRA-5397)
[3/3] git commit: Merge branch 'cassandra-1.2' into trunk
Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ce6429dc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ce6429dc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ce6429dc Branch: refs/heads/trunk Commit: ce6429dc62cfdfdaed3c4d1c2bad6063792794dc Parents: a5a3a28 6db71b2 Author: Jonathan Ellis jbel...@apache.org Authored: Wed May 8 17:14:52 2013 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Wed May 8 17:14:52 2013 -0500 -- CHANGES.txt |1 + 1 files changed, 1 insertions(+), 0 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ce6429dc/CHANGES.txt --
[1/3] git commit: whitespace
Updated Branches: refs/heads/trunk ae0d6ebd2 - ce6429dc6 whitespace Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6db71b29 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6db71b29 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6db71b29 Branch: refs/heads/trunk Commit: 6db71b29d754eebdb64a9eab802b96af1e01595d Parents: 29ad6bd Author: Jonathan Ellis jbel...@apache.org Authored: Wed May 8 16:37:14 2013 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Wed May 8 16:37:14 2013 -0500 -- CHANGES.txt |1 + 1 files changed, 1 insertions(+), 0 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6db71b29/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 1abfc65..ca3f130 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -29,6 +29,7 @@ Merged from 1.1 * Fix trying to load deleted row into row cache on startup (CASSANDRA-4463) * fsync leveled manifest to avoid corruption (CASSANDRA-5535) + 1.2.4 * Ensure that PerRowSecondaryIndex updates see the most recent values (CASSANDRA-5397)
[2/3] git commit: MemoryMeter is no longer used by CacheService
MemoryMeter is no longer used by CacheService Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a5a3a28f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a5a3a28f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a5a3a28f Branch: refs/heads/trunk Commit: a5a3a28fc8cf69fa985b6c59fecb28578f12f4cf Parents: ae0d6eb Author: Jonathan Ellis jbel...@apache.org Authored: Wed May 8 17:14:44 2013 -0500 Committer: Jonathan Ellis jbel...@apache.org Committed: Wed May 8 17:14:44 2013 -0500 -- .../org/apache/cassandra/service/CacheService.java | 25 +- 1 files changed, 2 insertions(+), 23 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/a5a3a28f/src/java/org/apache/cassandra/service/CacheService.java -- diff --git a/src/java/org/apache/cassandra/service/CacheService.java b/src/java/org/apache/cassandra/service/CacheService.java index 8396d61..5eff28a 100644 --- a/src/java/org/apache/cassandra/service/CacheService.java +++ b/src/java/org/apache/cassandra/service/CacheService.java @@ -33,8 +33,6 @@ import javax.management.MBeanServer; import javax.management.ObjectName; import com.google.common.util.concurrent.Futures; -import com.googlecode.concurrentlinkedhashmap.EntryWeigher; -import org.github.jamm.MemoryMeter; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -102,7 +100,6 @@ public class CacheService implements CacheServiceMBean } /** - * We can use Weighers.singleton() because Long can't be leaking memory * @return auto saving cache object */ private AutoSavingCacheKeyCacheKey, RowIndexEntry initKeyCache() @@ -114,24 +111,7 @@ public class CacheService implements CacheServiceMBean // as values are constant size we can use singleton weigher // where 48 = 40 bytes (average size of the key) + 8 bytes (size of value) ICacheKeyCacheKey, RowIndexEntry kc; -if (MemoryMeter.isInitialized()) -{ -kc = ConcurrentLinkedHashCache.create(keyCacheInMemoryCapacity); -} -else -{ -logger.warn(MemoryMeter uninitialized (jamm not specified as java agent); KeyCache size in JVM Heap will not be calculated accurately. + -Usually this means cassandra-env.sh disabled jamm because you are using a buggy JRE; upgrade to the Sun JRE instead); -/* We don't know the overhead size because memory meter is not enabled. */ -EntryWeigherKeyCacheKey, RowIndexEntry weigher = new EntryWeigherKeyCacheKey, RowIndexEntry() -{ -public int weightOf(KeyCacheKey key, RowIndexEntry entry) -{ -return key.key.length + entry.serializedSize(); -} -}; -kc = ConcurrentLinkedHashCache.create(keyCacheInMemoryCapacity, weigher); -} +kc = ConcurrentLinkedHashCache.create(keyCacheInMemoryCapacity); AutoSavingCacheKeyCacheKey, RowIndexEntry keyCache = new AutoSavingCacheKeyCacheKey, RowIndexEntry(kc, CacheType.KEY_CACHE, new KeyCacheSerializer()); int keyCacheKeysToSave = DatabaseDescriptor.getKeyCacheKeysToSave(); @@ -272,8 +252,7 @@ public class CacheService implements CacheServiceMBean if (capacity 0) throw new RuntimeException(capacity should not be negative.); -long weightedCapacity = capacity * 1024 * 1024; -keyCache.setCapacity(MemoryMeter.isInitialized() ? weightedCapacity : (weightedCapacity / 48)); +keyCache.setCapacity(capacity * 1024 * 1024); } public long getRowCacheSize()