[jira] [Updated] (CASSANDRA-7193) rows_per_partition_to_cache is not reflected in table DESCRIBE
[
https://issues.apache.org/jira/browse/CASSANDRA-7193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-7193:
---
Attachment: 0001-remove-rows_per_partition_to_cache.patch
the correct syntax is "WITH caching = {'keys': 'NONE', 'rows_per_parti
tion': 'ALL'};"
attached patch removes leftover rows_per_partition_to_cache keyword
> rows_per_partition_to_cache is not reflected in table DESCRIBE
> --
>
> Key: CASSANDRA-7193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7193
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Marcus Eriksson
>Priority: Minor
> Attachments: 0001-remove-rows_per_partition_to_cache.patch
>
>
> I can create a table with the new query cache from CASSANDRA-5357:
> {code}
> CREATE TABLE status (user text, status_id timeuuid, status text, PRIMARY KEY
> (user, status_id)) WITH caching = '{"keys":"ALL", "rows_per_partition":"10"}';
> {code}
> This method appears to work fine.
> However, that is not the syntax mentioned in that ticket. It says to use a
> rows_per_partition_to_cache setting instead, which does appear to work in cql:
> {code}
> CREATE TABLE status2 (user text, status_id timeuuid, status text, PRIMARY KEY
> (user, status_id)) WITH rows_per_partition_to_cache = 200;
> {code}
> But that setting is not reflected in the table description, instead it still
> shows NONE:
> {code}
> cqlsh:test> DESCRIBE TABLE status2;
> CREATE TABLE test.status2 (
> user text,
> status_id timeuuid,
> status text,
> PRIMARY KEY (user, status_id)
> ) WITH CLUSTERING ORDER BY (status_id ASC)
> AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
> ...
> {code}
> Similarly, alter table with that syntax, does not produce an error, but also
> does not seem to affect the setting :
> {code}
> ALTER TABLE test.status WITH rows_per_partition_to_cache = 200;
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (CASSANDRA-7205) Node never know a table has been DROP or CREATE if its gossip is disabled while executing this query
[ https://issues.apache.org/jira/browse/CASSANDRA-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis reassigned CASSANDRA-7205: - Assignee: Michael Shuler Can you reproduce, [~mshuler]? > Node never know a table has been DROP or CREATE if its gossip is disabled > while executing this query > > > Key: CASSANDRA-7205 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7205 > Project: Cassandra > Issue Type: Bug > Environment: Cassandra 2.0.6 >Reporter: Zhe Yang >Assignee: Michael Shuler > > I'm using Cassandra 2.0.6 and I have 8 nodes. I'm doing some tests by using > operations below: > disable gossip of node A; > check the status by nodetool in other node, node A is Down now; > use cqlsh connecting an "Up" node and create a table; > enable gossip of node A; > check the status, all nodes are "Up" now. > Then I find that node A doesn't know this table has been created. Both its > own cql shell and nodetool cfstats tell me the table doesn't exist. Even > waiting for a few minutes to get the "eventual consistency" final status, > node A still doesn't know this table. And I find if each node knows there is > a table but I drop it when one node's gossip is disabled, this node will > never know the table has been dropped. > Is this a bug? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6974) Replaying archived commitlogs isn't working
[
https://issues.apache.org/jira/browse/CASSANDRA-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-6974:
--
Tester: Michael Shuler
> Replaying archived commitlogs isn't working
> ---
>
> Key: CASSANDRA-6974
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6974
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Benedict
> Labels: qa-resolved
> Fix For: 2.1 rc1
>
> Attachments: 2.0.system.log, 2.1.system.log, 6974.txt
>
>
> I have a test for restoring archived commitlogs, which is not working in 2.1
> HEAD. My commitlogs consist of 30,000 inserts, but system.log indicates
> there were only 2 mutations replayed:
> {code}
> INFO [main] 2014-04-02 11:49:54,173 CommitLog.java:115 - Log replay
> complete, 2 replayed mutations
> {code}
> There are several warnings in the logs about bad headers and invalid CRCs:
> {code}
> WARN [main] 2014-04-02 11:49:54,156 CommitLogReplayer.java:138 - Encountered
> bad header at position 0 of commit log /tmp/dtest
> -mZIlPE/test/node1/commitlogs/CommitLog-4-1396453793570.log, with invalid
> CRC. The end of segment marker should be zero.
> {code}
> compare that to the same test run on 2.0, where it replayed many more
> mutations:
> {code}
> INFO [main] 2014-04-02 11:49:04,673 CommitLog.java (line 132) Log replay
> complete, 35960 replayed mutations
> {code}
> I'll attach the system logs for reference.
> [Here is the dtest to reproduce
> this|https://github.com/riptano/cassandra-dtest/blob/master/snapshot_test.py#L75]
> - (This currently relies on the fix for snapshots available in
> CASSANDRA-6965.)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[4/4] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0cb1db68 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0cb1db68 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0cb1db68 Branch: refs/heads/cassandra-2.1 Commit: 0cb1db68fc3ec56379fa478544e452eea671b5d9 Parents: 411abc7 ea7d0c8 Author: Jake Luciani Authored: Wed May 7 15:17:58 2014 -0400 Committer: Jake Luciani Committed: Wed May 7 15:18:49 2014 -0400 -- CHANGES.txt | 1 + .../cql3/statements/BatchStatement.java | 2 + .../cql3/statements/ModificationStatement.java | 338 +++ .../apache/cassandra/net/MessagingService.java | 6 +- .../net/OutboundTcpConnectionPool.java | 41 ++- 5 files changed, 239 insertions(+), 149 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/0cb1db68/CHANGES.txt -- diff --cc CHANGES.txt index 684d003,20fd115..5ecd19d --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -13,73 -22,22 +13,74 @@@ Merged from 1.2 * Add Cloudstack snitch (CASSANDRA-7147) * Update system.peers correctly when relocating tokens (CASSANDRA-7126) * Add Google Compute Engine snitch (CASSANDRA-7132) - * Fix nodetool display with vnodes (CASSANDRA-7082) - * Fix schema concurrency exceptions (CASSANDRA-6841) - * Fix BatchlogManager#deleteBatch() use of millisecond timsestamps - (CASSANDRA-6822) - * Fix batchlog to account for CF truncation records (CASSANDRA-6999) - * Fix CQLSH parsing of functions and BLOB literals (CASSANDRA-7018) - * Require nodetool rebuild_index to specify index names (CASSANDRA-7038) - * Ensure that batchlog and hint timeouts do not produce hints (CASSANDRA-7058) - * Always clean up references in SerializingCache (CASSANDRA-6994) - * Don't shut MessagingService down when replacing a node (CASSANDRA-6476) - * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751) - * Preserves CQL metadata when updating table from thrift (CASSANDRA-6831) * remove duplicate query for local tokens (CASSANDRA-7182) - * raise streaming phi convict threshold level (CASSANDRA-7063) -2.0.7 + +2.1.0-beta2 + * Increase default CL space to 8GB (CASSANDRA-7031) + * Add range tombstones to read repair digests (CASSANDRA-6863) + * Fix BTree.clear for large updates (CASSANDRA-6943) + * Fail write instead of logging a warning when unable to append to CL + (CASSANDRA-6764) + * Eliminate possibility of CL segment appearing twice in active list + (CASSANDRA-6557) + * Apply DONTNEED fadvise to commitlog segments (CASSANDRA-6759) + * Switch CRC component to Adler and include it for compressed sstables + (CASSANDRA-4165) + * Allow cassandra-stress to set compaction strategy options (CASSANDRA-6451) + * Add broadcast_rpc_address option to cassandra.yaml (CASSANDRA-5899) + * Auto reload GossipingPropertyFileSnitch config (CASSANDRA-5897) + * Fix overflow of memtable_total_space_in_mb (CASSANDRA-6573) + * Fix ABTC NPE and apply update function correctly (CASSANDRA-6692) + * Allow nodetool to use a file or prompt for password (CASSANDRA-6660) + * Fix AIOOBE when concurrently accessing ABSC (CASSANDRA-6742) + * Fix assertion error in ALTER TYPE RENAME (CASSANDRA-6705) + * Scrub should not always clear out repaired status (CASSANDRA-5351) + * Improve handling of range tombstone for wide partitions (CASSANDRA-6446) + * Fix ClassCastException for compact table with composites (CASSANDRA-6738) + * Fix potentially repairing with wrong nodes (CASSANDRA-6808) + * Change caching option syntax (CASSANDRA-6745) + * Fix stress to do proper counter reads (CASSANDRA-6835) + * Fix help message for stress counter_write (CASSANDRA-6824) + * Fix stress smart Thrift client to pick servers correctly (CASSANDRA-6848) + * Add logging levels (minimal, normal or verbose) to stress tool (CASSANDRA-6849) + * Fix race condition in Batch CLE (CASSANDRA-6860) + * Improve cleanup/scrub/upgradesstables failure handling (CASSANDRA-6774) + * ByteBuffer write() methods for serializing sstables (CASSANDRA-6781) + * Proper compare function for CollectionType (CASSANDRA-6783) + * Update native server to Netty 4 (CASSANDRA-6236) + * Fix off-by-one error in stress (CASSANDRA-6883) + * Make OpOrder AutoCloseable (CASSANDRA-6901) + * Remove sync repair JMX interface (CASSANDRA-6900) + * Add multiple memory allocation options for memtables (CASSANDRA-6689, 6694) + * Remove adjusted op rate from stress output (CASSANDRA-6921) + * Add optimized CF.hasColumns() implementations (CASSA
[6/6] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6cecbf91 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6cecbf91 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6cecbf91 Branch: refs/heads/trunk Commit: 6cecbf914d7608a5111cafbb220b3a7e41b8fb5d Parents: 2629c30 9da58ed Author: Sylvain Lebresne Authored: Wed May 7 11:08:28 2014 +0200 Committer: Sylvain Lebresne Committed: Wed May 7 11:08:28 2014 +0200 -- CHANGES.txt| 4 .../cassandra/cql3/statements/BatchStatement.java | 13 - .../cql3/statements/ModificationStatement.java | 3 --- 3 files changed, 12 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6cecbf91/CHANGES.txt --
[jira] [Updated] (CASSANDRA-6973) timestamp data type does ISO 8601 formats with 'Z' as time zone.
[
https://issues.apache.org/jira/browse/CASSANDRA-6973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chander S Pechetty updated CASSANDRA-6973:
--
Attachment: trunk-6973_unittest_v2.txt
patch v2 has the updated unit test to print all un-parsed dates instead of one
date per a single run.
> timestamp data type does ISO 8601 formats with 'Z' as time zone.
>
>
> Key: CASSANDRA-6973
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6973
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Juho Mäkinen
>Assignee: Chander S Pechetty
>Priority: Trivial
> Attachments: trunk-6973.txt, trunk-6973_unittest.txt,
> trunk-6973_unittest_v2.txt, trunk-6973_v2.txt
>
>
> The timestamp data type does not support format where time zone is specified
> with 'Z' (as in zulu aka. UTC+0 aka + time zone). Example:
> create table foo(ts timestamp primary key);
> insert into foo(ts) values('2014-04-01T20:17:35+'); -- this works
> cqlsh:test> insert into foo(ts) values('2014-04-01T20:17:35Z');
> Bad Request: unable to coerce '2014-04-01T20:17:35Z' to a formatted date
> (long)
> The example date was copied directly from ISO 8601 Wikipedia page. The
> standard says that "If the time is in UTC, add a Z directly after the time
> without a space. Z is the zone designator for the zero UTC offset."
> Tested with cqlsh with 2.0.6 version.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-5635) ThriftServer.stop() hangs forever
[
https://issues.apache.org/jira/browse/CASSANDRA-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-5635:
Priority: Trivial (was: Major)
> ThriftServer.stop() hangs forever
> -
>
> Key: CASSANDRA-5635
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5635
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 1.2.4
>Reporter: Robert Stupp
>Assignee: Dave Brosius
>Priority: Trivial
> Fix For: 2.1 rc1
>
> Attachments: 5635.txt
>
>
> I've written a very small main() method just to start to test "how to embed
> Cassandra". But the code hangs while executing CassandraDaemon.stop()...
> I've used a default {{cassandra.yaml}} file.
> {noformat}
> cassandraDaemon = new CassandraDaemon();
> cassandraDaemon.init(null);
> cassandraDaemon.start();
> cassandraDaemon.stop();
> {noformat}
> {{CassandraDaemon.stop()}} calls {{ThriftServer.stop()}, which ends somehow
> in {{TCustomServerSocket.close()}}, which sets its field
> {{serverSocket=null}}. This causes {{CustomTThreadPoolServer.server()}} to
> loop forever, because it's {{stopped}} field is still {{false}} -
> {{TServerTransport.accept()}} immediatly throws a {{TTransportException}}
> because {{TCustomServerSocket}}'s {{serverSocket}} is {{null}}.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-5635) ThriftServer.stop() hangs forever
[
https://issues.apache.org/jira/browse/CASSANDRA-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Brosius updated CASSANDRA-5635:
Attachment: 5635.txt
make SlabPoolCleaner thread, a daemon.
> ThriftServer.stop() hangs forever
> -
>
> Key: CASSANDRA-5635
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5635
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 1.2.4
>Reporter: Robert Stupp
> Attachments: 5635.txt
>
>
> I've written a very small main() method just to start to test "how to embed
> Cassandra". But the code hangs while executing CassandraDaemon.stop()...
> I've used a default {{cassandra.yaml}} file.
> {noformat}
> cassandraDaemon = new CassandraDaemon();
> cassandraDaemon.init(null);
> cassandraDaemon.start();
> cassandraDaemon.stop();
> {noformat}
> {{CassandraDaemon.stop()}} calls {{ThriftServer.stop()}, which ends somehow
> in {{TCustomServerSocket.close()}}, which sets its field
> {{serverSocket=null}}. This causes {{CustomTThreadPoolServer.server()}} to
> loop forever, because it's {{stopped}} field is still {{false}} -
> {{TServerTransport.accept()}} immediatly throws a {{TTransportException}}
> because {{TCustomServerSocket}}'s {{serverSocket}} is {{null}}.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7171) Startup Cassandra and encountered java.lang.OutOfMemoryError: Java heap space
[ https://issues.apache.org/jira/browse/CASSANDRA-7171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994742#comment-13994742 ] liguosheng commented on CASSANDRA-7171: --- I have been got this error several times at 2.0.0. every time I set java heap size 3000m or delete key cache file to solve it.and that is the only way to solve that error. > Startup Cassandra and encountered java.lang.OutOfMemoryError: Java heap space > - > > Key: CASSANDRA-7171 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7171 > Project: Cassandra > Issue Type: Bug > Environment: windows 2008 > cassandra 2.0.0 > jre 1.7 >Reporter: liguosheng > > canssdra.log: > INFO 15:06:44,665 Data files directories: [../../../../cassandra_data] > INFO 15:06:44,665 Commit log directory: ../../../../cassandra_data/commitlog > INFO 15:06:44,665 DiskAccessMode 'auto' determined to be mmap, > indexAccessMode is mmap > INFO 15:06:44,665 disk_failure_policy is stop > INFO 15:06:44,758 Global memtable threshold is enabled at 247MB > INFO 15:06:45,039 Not using multi-threaded compaction > INFO 15:06:45,507 Initializing key cache with capacity of 37 MBs. > INFO 15:06:45,538 Scheduling key cache save to each 14400 seconds (going to > save all keys). > INFO 15:06:45,538 Initializing row cache with capacity of 0 MBs > INFO 15:06:45,554 Scheduling row cache save to each 0 seconds (going to save > all keys). > INFO 15:06:45,819 Initializing system.schema_triggers > INFO 15:06:45,913 Initializing system.batchlog > INFO 15:06:45,913 Initializing system.peer_events > INFO 15:06:45,928 Initializing system.compactions_in_progress > INFO 15:06:45,928 Opening > ..\..\..\..\cassandra_data\system\compactions_in_progress\system-compactions_in_progress-ja-1772 > (42 bytes) > INFO 15:06:45,928 Opening > ..\..\..\..\cassandra_data\system\compactions_in_progress\system-compactions_in_progress-ja-1773 > (42 bytes) > INFO 15:06:45,928 Opening > ..\..\..\..\cassandra_data\system\compactions_in_progress\system-compactions_in_progress-ja-1771 > (252 bytes) > INFO 15:06:46,022 Initializing system.hints > INFO 15:06:46,037 Initializing system.schema_keyspaces > INFO 15:06:46,037 Opening > ..\..\..\..\cassandra_data\system\schema_keyspaces\system-schema_keyspaces-ja-37 > (266 bytes) > INFO 15:06:46,069 Initializing system.range_xfers > INFO 15:06:46,069 Initializing system.schema_columnfamilies > INFO 15:06:46,069 Opening > ..\..\..\..\cassandra_data\system\schema_columnfamilies\system-schema_columnfamilies-ja-37 > (5979 bytes) > INFO 15:06:46,115 Initializing system.NodeIdInfo > INFO 15:06:46,115 Initializing system.paxos > INFO 15:06:46,115 Initializing system.schema_columns > INFO 15:06:46,115 Opening > ..\..\..\..\cassandra_data\system\schema_columns\system-schema_columns-ja-42 > (9423 bytes) > INFO 15:06:46,162 reading saved cache > ..\..\..\..\cassandra_data\saved_caches\system-schema_columns-KeyCache-b.db > INFO 15:06:46,162 Initializing system.IndexInfo > INFO 15:06:46,178 Initializing system.peers > INFO 15:06:46,178 Initializing system.local > INFO 15:06:46,178 Opening > ..\..\..\..\cassandra_data\system\local\system-local-ja-149 (5752 bytes) > INFO 15:06:46,225 reading saved cache > ..\..\..\..\cassandra_data\saved_caches\system-local-KeyCache-b.db > INFO 15:06:47,224 Enqueuing flush of Memtable-local@714233355(114/114 > serialized/live bytes, 4 ops) > INFO 15:06:47,224 Writing Memtable-local@714233355(114/114 serialized/live > bytes, 4 ops) > INFO 15:06:47,318 Completed flushing > ..\..\..\..\cassandra_data\system\local\system-local-ja-150-Data.db (148 > bytes) for commitlog position ReplayPosition(segmentId=1399273607006, > position=289) > INFO 15:06:47,333 Initializing kairosdb.row_key_index > INFO 15:06:47,333 Opening > ..\..\..\..\cassandra_data\kairosdb\row_key_index\kairosdb-row_key_index-ja-3663 > (1107026 bytes) > INFO 15:06:47,333 Opening > ..\..\..\..\cassandra_data\kairosdb\row_key_index\kairosdb-row_key_index-ja-3662 > (9834029 bytes) > INFO 15:06:47,411 reading saved cache > ..\..\..\..\cassandra_data\saved_caches\kairosdb-row_key_index-KeyCache-b.db > INFO 15:06:47,442 Initializing kairosdb.string_index > INFO 15:06:47,442 Opening > ..\..\..\..\cassandra_data\kairosdb\string_index\kairosdb-string_index-ja-3602 > (96700 bytes) > INFO 15:06:47,442 Opening > ..\..\..\..\cassandra_data\kairosdb\string_index\kairosdb-string_index-ja-3601 > (123957 bytes) > INFO 15:06:47,458 reading saved cache > ..\..\..\..\cassandra_data\saved_caches\kairosdb-string_index-KeyCache-b.db > INFO 15:06:47,458 Initializing kairosdb.data_points > INFO 15:06:47,474 Opening > ..\..\..\..\cassandra_data\kairosdb\data_points\kairosdb-data_points-ja-2608 > (813190770 bytes) > INFO 15:06:47,489 Opening > ..\..\..
[1/2] git commit: Fix broken build
Repository: cassandra
Updated Branches:
refs/heads/trunk fe2d7ddaf -> 56c76bebe
Fix broken build
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/60dbe8b7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/60dbe8b7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/60dbe8b7
Branch: refs/heads/trunk
Commit: 60dbe8b700ee0ee1a15fbcb94f9543d1477de7a3
Parents: 0cb1db6
Author: Jake Luciani
Authored: Wed May 7 16:55:53 2014 -0400
Committer: Jake Luciani
Committed: Wed May 7 16:55:53 2014 -0400
--
.../cql3/statements/ModificationStatement.java | 338 ---
1 file changed, 141 insertions(+), 197 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/60dbe8b7/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --git
a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index 23f7cfe..7f8b678 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@ -23,23 +23,18 @@ import java.util.*;
import com.google.common.base.Function;
import com.google.common.collect.Iterables;
import org.github.jamm.MemoryMeter;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
import org.apache.cassandra.auth.Permission;
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.config.ColumnDefinition;
import org.apache.cassandra.cql3.*;
import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.composites.CBuilder;
+import org.apache.cassandra.db.composites.Composite;
import org.apache.cassandra.db.filter.ColumnSlice;
-import org.apache.cassandra.db.filter.IDiskAtomFilter;
import org.apache.cassandra.db.filter.SliceQueryFilter;
-import org.apache.cassandra.db.marshal.CompositeType;
-import org.apache.cassandra.db.marshal.UTF8Type;
-import org.apache.cassandra.db.marshal.ListType;
import org.apache.cassandra.db.marshal.BooleanType;
import org.apache.cassandra.exceptions.*;
-import org.apache.cassandra.service.CASConditions;
import org.apache.cassandra.service.ClientState;
import org.apache.cassandra.service.QueryState;
import org.apache.cassandra.service.StorageProxy;
@@ -54,21 +49,16 @@ public abstract class ModificationStatement implements
CQLStatement, MeasurableF
{
private static final ColumnIdentifier CAS_RESULT_COLUMN = new
ColumnIdentifier("[applied]", false);
-private static final Logger logger =
LoggerFactory.getLogger(ModificationStatement.class);
-
-private static boolean loggedCounterTTL = false;
-private static boolean loggedCounterTimestamp = false;
-
public static enum StatementType { INSERT, UPDATE, DELETE }
public final StatementType type;
+private final int boundTerms;
public final CFMetaData cfm;
public final Attributes attrs;
private final Map processedKeys = new
HashMap();
private final List columnOperations = new
ArrayList();
-private int boundTerms;
// Separating normal and static conditions makes things somewhat easier
private List columnConditions;
private List staticConditions;
@@ -80,17 +70,18 @@ public abstract class ModificationStatement implements
CQLStatement, MeasurableF
private boolean setsStaticColumns;
private boolean setsRegularColumns;
-private final Function
getColumnForCondition = new Function()
+private final Function
getColumnForCondition = new Function()
{
-public ColumnIdentifier apply(ColumnCondition cond)
+public ColumnDefinition apply(ColumnCondition cond)
{
-return cond.column.name;
+return cond.column;
}
};
-public ModificationStatement(StatementType type, CFMetaData cfm,
Attributes attrs)
+public ModificationStatement(StatementType type, int boundTerms,
CFMetaData cfm, Attributes attrs)
{
this.type = type;
+this.boundTerms = boundTerms;
this.cfm = cfm;
this.attrs = attrs;
}
@@ -106,7 +97,7 @@ public abstract class ModificationStatement implements
CQLStatement, MeasurableF
}
public abstract boolean requireFullClusteringKey();
-public abstract void addUpdateForKey(ColumnFamily updates, ByteBuffer key,
ColumnNameBuilder builder, UpdateParameters params) throws
InvalidRequestException;
+public abstract void addUpdateForKey(ColumnFamily updates, ByteBuffer key,
Composite prefix, UpdateParameters params) throws InvalidRequestException;
public int getBoundTerms()
{
@@ -125,12 +116,12 @@ public abstract class Modificatio
[6/6] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/677df414 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/677df414 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/677df414 Branch: refs/heads/trunk Commit: 677df414dc1d1f776af1be604810258039287139 Parents: 310d6e4 355e69b Author: Mikhail Stepura Authored: Thu May 8 13:45:32 2014 -0700 Committer: Mikhail Stepura Committed: Thu May 8 13:45:32 2014 -0700 -- CHANGES.txt| 1 + pylib/cqlshlib/cql3handling.py | 6 +- 2 files changed, 6 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/677df414/CHANGES.txt --
[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8dab582c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8dab582c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8dab582c Branch: refs/heads/trunk Commit: 8dab582c2421b21f8d87a26f55941f1ee1f8b516 Parents: 3b299c4 d6f32e4 Author: Sylvain Lebresne Authored: Thu May 8 18:05:13 2014 +0200 Committer: Sylvain Lebresne Committed: Thu May 8 18:05:13 2014 +0200 -- CHANGES.txt| 1 + src/java/org/apache/cassandra/db/marshal/AbstractType.java | 3 +-- .../apache/cassandra/serializers/InetAddressSerializer.java| 2 +- .../org/apache/cassandra/serializers/IntegerSerializer.java| 6 +++--- src/java/org/apache/cassandra/serializers/LongSerializer.java | 2 +- .../org/apache/cassandra/serializers/TimestampSerializer.java | 2 +- src/java/org/apache/cassandra/serializers/UUIDSerializer.java | 2 +- 7 files changed, 9 insertions(+), 9 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/8dab582c/CHANGES.txt -- diff --cc CHANGES.txt index 5afe800,a6cbc18..057fd34 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,20 -1,27 +1,21 @@@ -2.0.9 - * Warn when 'USING TIMESTAMP' is used on a CAS BATCH (CASSANDRA-7067) - * Starting threads in OutboundTcpConnectionPool constructor causes race conditions (CASSANDRA-7177) - * return all cpu values from BackgroundActivityMonitor.readAndCompute (CASSANDRA-7183) - * fix c* launch issues on Russian os's due to output of linux 'free' cmd (CASSANDRA-6162) - * Fix disabling autocompaction (CASSANDRA-7187) - * Fix potential NumberFormatException when deserializing IntegerType (CASSANDRA-7088) - -2.0.8 +2.1.0-rc1 + * Fix marking commitlogsegments clean (CASSANDRA-6959) + * Add snapshot "manifest" describing files included (CASSANDRA-6326) + * Parallel streaming for sstableloader (CASSANDRA-3668) + * Fix bugs in supercolumns handling (CASSANDRA-7138) + * Fix ClassClassException on composite dense tables (CASSANDRA-7112) + * Cleanup and optimize collation and slice iterators (CASSANDRA-7107) + * Upgrade NBHM lib (CASSANDRA-7128) + * Optimize netty server (CASSANDRA-6861) +Merged from 2.0: * Correctly delete scheduled range xfers (CASSANDRA-7143) * Make batchlog replica selection rack-aware (CASSANDRA-6551) - * Allow overriding cassandra-rackdc.properties file (CASSANDRA-7072) - * Set JMX RMI port to 7199 (CASSANDRA-7087) - * Use LOCAL_QUORUM for data reads at LOCAL_SERIAL (CASSANDRA-6939) - * Log a warning for large batches (CASSANDRA-6487) - * Queries on compact tables can return more rows that requested (CASSANDRA-7052) - * USING TIMESTAMP for batches does not work (CASSANDRA-7053) - * Fix performance regression from CASSANDRA-5614 (CASSANDRA-6949) - * Merge groupable mutations in TriggerExecutor#execute() (CASSANDRA-7047) - * Fix CFMetaData#getColumnDefinitionFromColumnName() (CASSANDRA-7074) - * Plug holes in resource release when wiring up StreamSession (CASSANDRA-7073) - * Re-add parameter columns to tracing session (CASSANDRA-6942) - * Fix writetime/ttl functions for static columns (CASSANDRA-7081) * Suggest CTRL-C or semicolon after three blank lines in cqlsh (CASSANDRA-7142) + * return all cpu values from BackgroundActivityMonitor.readAndCompute (CASSANDRA-7183) + * reduce garbage creation in calculatePendingRanges (CASSANDRA-7191) + * fix c* launch issues on Russian os's due to output of linux 'free' cmd (CASSANDRA-6162) + * Fix disabling autocompaction (CASSANDRA-7187) ++ * Fix potential NumberFormatException when deserializing IntegerType (CASSANDRA-7088) Merged from 1.2: * Add Cloudstack snitch (CASSANDRA-7147) * Update system.peers correctly when relocating tokens (CASSANDRA-7126) http://git-wip-us.apache.org/repos/asf/cassandra/blob/8dab582c/src/java/org/apache/cassandra/db/marshal/AbstractType.java --
git commit: Ninja: add missing "enabled" option to compaction options docs
Repository: cassandra Updated Branches: refs/heads/cassandra-2.0 a626c6dcb -> 46f7f84ce Ninja: add missing "enabled" option to compaction options docs Done for CASSANDRA-7185 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/46f7f84c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/46f7f84c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/46f7f84c Branch: refs/heads/cassandra-2.0 Commit: 46f7f84cea0b608da22e1e315c8a05096ce494ac Parents: a626c6d Author: Tyler Hobbs Authored: Thu May 8 15:57:35 2014 -0500 Committer: Tyler Hobbs Committed: Thu May 8 15:57:35 2014 -0500 -- doc/cql3/CQL.textile | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/46f7f84c/doc/cql3/CQL.textile -- diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile index bedd189..66a4566 100644 --- a/doc/cql3/CQL.textile +++ b/doc/cql3/CQL.textile @@ -336,6 +336,7 @@ h4(#compactionOptions). @compaction@ options The @compaction@ property must at least define the @'class'@ sub-option, that defines the compaction strategy class to use. The default supported class are @'SizeTieredCompactionStrategy'@ and @'LeveledCompactionStrategy'@. Custom strategy can be provided by specifying the full class name as a "string constant":#constants. The rest of the sub-options depends on the chosen class. The sub-options supported by the default classes are: |_. option|_. supported compaction strategy |_. default |_. description | +| @enabled@ | _all_ | true | A boolean denoting whether compaction should be enabled or not.| | @tombstone_threshold@ | _all_ | 0.2 | A ratio such that if a sstable has more than this ratio of gcable tombstones over all contained columns, the sstable will be compacted (with no other sstables) for the purpose of purging those tombstones. | | @tombstone_compaction_interval@ | _all_ | 1 day | The minimum time to wait after an sstable creation time before considering it for "tombstone compaction", where "tombstone compaction" is the compaction triggered if the sstable has more gcable tombstones than @tombstone_threshold@. | | @min_sstable_size@ | SizeTieredCompactionStrategy| 50MB | The size tiered strategy groups SSTables to compact in buckets. A bucket groups SSTables that differs from less than 50% in size. However, for small sizes, this would result in a bucketing that is too fine grained. @min_sstable_size@ defines a size threshold (in bytes) below which all SSTables belong to one unique bucket|
[1/3] git commit: Fix potential SlabAllocator yield-starvation patch by bes; reviewed by jbellis for CASSANDRA-7133
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 46f7f84ce -> d13f78ab8
refs/heads/cassandra-2.1 1ac72f637 -> 750235931
Fix potential SlabAllocator yield-starvation
patch by bes; reviewed by jbellis for CASSANDRA-7133
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d13f78ab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d13f78ab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d13f78ab
Branch: refs/heads/cassandra-2.0
Commit: d13f78ab82c6fe4a6fbb5293126670fb7032203f
Parents: 46f7f84
Author: Jonathan Ellis
Authored: Thu May 8 17:04:17 2014 -0500
Committer: Jonathan Ellis
Committed: Thu May 8 17:04:17 2014 -0500
--
CHANGES.txt | 2 +
.../apache/cassandra/utils/SlabAllocator.java | 53 +---
2 files changed, 15 insertions(+), 40 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d13f78ab/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ce30239..2712398 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.9
+ * Fix potential SlabAllocator yield-starvation (CASSANDRA-7133)
* Warn when 'USING TIMESTAMP' is used on a CAS BATCH (CASSANDRA-7067)
* Starting threads in OutboundTcpConnectionPool constructor causes race
conditions (CASSANDRA-7177)
* return all cpu values from BackgroundActivityMonitor.readAndCompute
(CASSANDRA-7183)
@@ -7,6 +8,7 @@
* Fix potential NumberFormatException when deserializing IntegerType
(CASSANDRA-7088)
* cqlsh can't tab-complete disabling compaction (CASSANDRA-7185)
+
2.0.8
* Correctly delete scheduled range xfers (CASSANDRA-7143)
* Make batchlog replica selection rack-aware (CASSANDRA-6551)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d13f78ab/src/java/org/apache/cassandra/utils/SlabAllocator.java
--
diff --git a/src/java/org/apache/cassandra/utils/SlabAllocator.java
b/src/java/org/apache/cassandra/utils/SlabAllocator.java
index dedf869..20939fe 100644
--- a/src/java/org/apache/cassandra/utils/SlabAllocator.java
+++ b/src/java/org/apache/cassandra/utils/SlabAllocator.java
@@ -18,11 +18,11 @@
package org.apache.cassandra.utils;
import java.nio.ByteBuffer;
+import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
-import com.google.common.base.Preconditions;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -47,6 +47,9 @@ public class SlabAllocator extends Allocator
private final static int REGION_SIZE = 1024 * 1024;
private final static int MAX_CLONED_SIZE = 128 * 1024; // bigger than this
don't go in the region
+// globally stash any Regions we allocate but are beaten to using, and use
these up before allocating any more
+private static final ConcurrentLinkedQueue RACE_ALLOCATED = new
ConcurrentLinkedQueue<>();
+
private final AtomicReference currentRegion = new
AtomicReference();
private final AtomicInteger regionCount = new AtomicInteger(0);
private AtomicLong unslabbed = new AtomicLong(0);
@@ -92,19 +95,20 @@ public class SlabAllocator extends Allocator
return region;
// No current region, so we want to allocate one. We race
-// against other allocators to CAS in an uninitialized region
-// (which is cheap to allocate)
-region = new Region(REGION_SIZE);
+// against other allocators to CAS in a Region, and if we fail we
stash the region for re-use
+region = RACE_ALLOCATED.poll();
+if (region == null)
+region = new Region(REGION_SIZE);
if (currentRegion.compareAndSet(null, region))
{
-// we won race - now we need to actually do the expensive
allocation step
-region.init();
regionCount.incrementAndGet();
logger.trace("{} regions now allocated in {}", regionCount,
this);
return region;
}
+
// someone else won race - that's fine, we'll try to grab theirs
// in the next iteration of the loop.
+RACE_ALLOCATED.add(region);
}
}
@@ -129,24 +133,18 @@ public class SlabAllocator extends Allocator
/**
* Actual underlying data
*/
-private ByteBuffer data;
+private final ByteBuffer data;
-private static final int UNINITIALIZED = -1;
/**
* Offset for the next allocation, or the sentinel value -1
* whic
[jira] [Commented] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data
[ https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994677#comment-13994677 ] Pavel Yaskevich commented on CASSANDRA-6285: [~brandon.williams] I have released 0.3.5 just now, with reallocation and on-heap buffers turned on by default. +1 on the change so it's either we commit 0.3.5 or your patch. [~appodictic] Here is a definition of [unit testing|http://en.wikipedia.org/wiki/Unit_testing], in my tests I cover single/multi connection on-heap/off-heap +/- reallocation scenarios, basically everything to prove that server functions properly in all of the modes and returns correct results based on the operation being used. end-to-end tests are what systems which integrate project are supposed to do and that is done by stress and bpdlab testing. If you have been following discussion in this ticket you must have already realized that the problem is not caused by HsHa server directly but rather by the fact that Cassandra holds Thrift buffers even after blocking evaluation is finished. > 2.0 HSHA server introduces corrupt data > --- > > Key: CASSANDRA-6285 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6285 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2 >Reporter: David Sauer >Assignee: Pavel Yaskevich >Priority: Critical > Fix For: 2.0.8 > > Attachments: 6285_testnotes1.txt, > CASSANDRA-6285-disruptor-heap.patch, cassandra-attack-src.zip, > compaction_test.py, disruptor-high-cpu.patch, > disruptor-memory-corruption.patch, enable_reallocate_buffers.txt > > > After altering everything to LCS the table OpsCenter.rollups60 amd one other > none OpsCenter-Table got stuck with everything hanging around in L0. > The compaction started and ran until the logs showed this: > ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java > (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime] > java.lang.RuntimeException: Last written key > DecoratedKey(1326283851463420237, > 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564) > >= current key DecoratedKey(954210699457429663, > 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f) > writing into > /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db > at > org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141) > at > org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164) > at > org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160) > at > org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at > org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) > at > org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) > at > org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296) > at > org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > Moving back to STC worked to keep the compactions running. > Especialy my own Table i would like to move to LCS. > After a major compaction with STC the move to LCS fails with the same > Exception. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7084) o.a.c.db.RecoveryManagerTest.testNothingToRecover Unit Test Flaps in 2.0
[
https://issues.apache.org/jira/browse/CASSANDRA-7084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992821#comment-13992821
]
Jonathan Ellis commented on CASSANDRA-7084:
---
+1 on the patch.
nit: prefer Uninterruptibles.sleepUninterruptibly to hand-rolling try/catch.
> o.a.c.db.RecoveryManagerTest.testNothingToRecover Unit Test Flaps in 2.0
>
>
> Key: CASSANDRA-7084
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7084
> Project: Cassandra
> Issue Type: Test
> Components: Tests
>Reporter: Michael Shuler
>Assignee: Benedict
>Priority: Minor
> Labels: qa-resolved
> Fix For: 2.0.8
>
> Attachments: 7084.txt
>
>
> Example:
> http://cassci.datastax.com/job/cassandra-2.0_utest/326/
> (this test appears to pass consistently in 1.2 and 2.1 with a quick glance -
> will test out the other branches more thoroughly and bisect)
> {noformat}
> REGRESSION: org.apache.cassandra.db.RecoveryManagerTest.testNothingToRecover
> Error Message:
> java.io.FileNotFoundException:
> /var/lib/jenkins/jobs/cassandra-2.0_test/workspace/build/test/cassandra/commitlog/CommitLog-3-1398354429966.log
> (No such file or directory)
> Stack Trace:
> java.lang.RuntimeException: java.io.FileNotFoundException:
> /var/lib/jenkins/jobs/cassandra-2.0_test/workspace/build/test/cassandra/commitlog/CommitLog-3-1398354429966.log
> (No such file or directory)
> at
> org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:102)
> at
> org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:90)
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:186)
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:95)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:151)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:131)
> at
> org.apache.cassandra.db.RecoveryManagerTest.testNothingToRecover(RecoveryManagerTest.java:42)
> Caused by: java.io.FileNotFoundException:
> /var/lib/jenkins/jobs/cassandra-2.0_test/workspace/build/test/cassandra/commitlog/CommitLog-3-1398354429966.log
> (No such file or directory)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.(RandomAccessFile.java:241)
> at
> org.apache.cassandra.io.util.RandomAccessReader.(RandomAccessReader.java:58)
> at
> org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:98)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7196) "Select" query with "IN" restriction times out in CQLSH
[
https://issues.apache.org/jira/browse/CASSANDRA-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
T Jake Luciani updated CASSANDRA-7196:
--
Attachment: 7196-v3.txt
v3 better fix.
> "Select" query with "IN" restriction times out in CQLSH
> ---
>
> Key: CASSANDRA-7196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7196
> Project: Cassandra
> Issue Type: Bug
>Reporter: Mikhail Stepura
>Assignee: T Jake Luciani
> Labels: regression
> Fix For: 2.1 rc1
>
> Attachments: 7196-v2.txt, 7196-v3.txt, 7196.txt, init_bug.cql
>
>
> I've noticed that
> {{pylib.cqlshlib.test.test_cqlsh_output.TestCqlshOutput#test_numeric_output}}
> tests fails on the current 2.1 branch, which wasn't the case before.
> Here are the steps to reproduce. I'm attaching the script to populate schema.
> {code}
> mstepura-mac:cassandra mikhail$ bin/cqlsh -f /init_bug.cql
> mstepura-mac:cassandra mikhail$ bin/cqlsh
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.0 | Cassandra 2.1.0-beta2-SNAPSHOT | CQL spec 3.1.6 | Native
> protocol v2]
> Use HELP for help.
> cqlsh> use test;
> cqlsh:test> select intcol, bigintcol, varintcol from has_all_types where num
> in (0, 1, 2, 3, 4);
> errors={}, last_host=127.0.0.1
> cqlsh:test>
> {code}
> That works perfectly on 2.0 branch. And there are no errors in the logs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[1/3] git commit: Fix long where int needed in commitlog segment.
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 7f3d07ac0 -> 65a46265d
refs/heads/trunk ac1a9cd63 -> 05a54e0ce
Fix long where int needed in commitlog segment.
Patch by Benedict, reviewed by brandonwilliams for CASSANDRA-7199
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/65a46265
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/65a46265
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/65a46265
Branch: refs/heads/cassandra-2.1
Commit: 65a46265df79593c684efceba0ebb93e37339e0e
Parents: 7f3d07a
Author: Brandon Williams
Authored: Fri May 9 13:28:27 2014 -0500
Committer: Brandon Williams
Committed: Fri May 9 13:28:27 2014 -0500
--
src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/65a46265/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
index c87b328..78ba824 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
@@ -316,7 +316,7 @@ public class CommitLogSegment
if (nextMarker < buffer.capacity())
{
buffer.putInt(nextMarker, 0);
-buffer.putLong(nextMarker + 4, 0);
+buffer.putInt(nextMarker + 4, 0);
}
// actually perform the sync and signal those waiting for it
[jira] [Commented] (CASSANDRA-6815) Decided if we want to bring back thrift HSHA in 2.0.7
[ https://issues.apache.org/jira/browse/CASSANDRA-6815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993814#comment-13993814 ] Rick Branson commented on CASSANDRA-6815: - FYI I reopened CASSANDRA-6285 because the "fix" doesn't fix the issue at all. > Decided if we want to bring back thrift HSHA in 2.0.7 > - > > Key: CASSANDRA-6815 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6815 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne >Assignee: Pavel Yaskevich > > This is the followup of CASSANDRA-6285, to decide what we want to do > regarding thrift servers moving forward. My reading of CASSANDRA-6285 > suggests that the possible options includes: > # bring back the old HSHA implementation from 1.2 as "hsha" and make the > disruptor implementation be "disruptor_hsha". > # use the new TThreadedSelectorServer from thrift as "hsha", making the > disruptor implementation "disruptor_hsha" as above > # just wait for Pavel to fix the disruptor implementation for off-heap > buffers to switch back to that, keeping on-heap buffer until then. > # keep on-heap buffer for the disruptor implementation and do nothing > particular. > I could be missing some options and we can probably do some mix of those. I > don't have a particular opinion to offer on the matter. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7196) "Select" query with "IN" restriction times out in CQLSH
[
https://issues.apache.org/jira/browse/CASSANDRA-7196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
T Jake Luciani updated CASSANDRA-7196:
--
Reviewer: Benedict
> "Select" query with "IN" restriction times out in CQLSH
> ---
>
> Key: CASSANDRA-7196
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7196
> Project: Cassandra
> Issue Type: Bug
>Reporter: Mikhail Stepura
>Assignee: T Jake Luciani
> Labels: regression
> Fix For: 2.1 rc1
>
> Attachments: 7196-v2.txt, 7196-v3.txt, 7196.txt, init_bug.cql
>
>
> I've noticed that
> {{pylib.cqlshlib.test.test_cqlsh_output.TestCqlshOutput#test_numeric_output}}
> tests fails on the current 2.1 branch, which wasn't the case before.
> Here are the steps to reproduce. I'm attaching the script to populate schema.
> {code}
> mstepura-mac:cassandra mikhail$ bin/cqlsh -f /init_bug.cql
> mstepura-mac:cassandra mikhail$ bin/cqlsh
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.0 | Cassandra 2.1.0-beta2-SNAPSHOT | CQL spec 3.1.6 | Native
> protocol v2]
> Use HELP for help.
> cqlsh> use test;
> cqlsh:test> select intcol, bigintcol, varintcol from has_all_types where num
> in (0, 1, 2, 3, 4);
> errors={}, last_host=127.0.0.1
> cqlsh:test>
> {code}
> That works perfectly on 2.0 branch. And there are no errors in the logs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993059#comment-13993059 ] Benedict commented on CASSANDRA-4718: - I have a few branches to test out, and I want to test them out an a variety of hardware. [~enigmacurry] can you run them on our internal multi-cpu boxes, and an AWS c3.8xlarge 4node cluster to the following spec: For each branch run: 20M inserts over 1M unique keys with 30, 90, 270 and 810 threads, then wipe each cluster and perform a single 1M key insert, and then run 20M reads over 1M unique keys with the same thread counts. All told that should take around 3hrs for -mode cql3 native prepared; I'd then like to repeat the tests for -mode thrift smart. The branches are: [https://github.com/belliottsmith/cassandra/tree/4718-lse] [https://github.com/belliottsmith/cassandra/tree/4718-lse-batchnetty] [https://github.com/belliottsmith/cassandra/tree/4718-fjp] [https://github.com/belliottsmith/cassandra/tree/4718-lowsignal] [https://github.com/belliottsmith/cassandra/tree/cassandra-2.1] Make sure you use my cassandra-2.1 so we're testing like-to-like (they're all rebased to the same version). I'll elaborate on the contents of these branches later, but suffice it to say the 4718-lse branch contains a new executor which attempts to reduce signalling costs to near zero by scheduling the correct number of threads to deal with the level of throughput the executor has been dealing with over the previous (short) adjustment window. -batchnetty includes some simple batching of netty messages. 4718-lowsignal is an enhanced version of the patch I uploaded previously to this ticket, and 4718-fjp is largely unchanged. On my own box, and on our austin test cluster, I see -lse faster than both -fjp and -lowsignal, however on our austin cluster (which is a not super-modern 4-cpu no-hyperthreading setup) I see both of them slower than stock 2.1, however -lse is only slightly slower, whereas -fjp is around 30% slower. I'll post polished numbers a little later. > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, > backpressure-stress.out.txt, baq vs trunk.png, op costs of various > queues.ods, stress op rate with various queues.ods, v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7196) "Select" query with "IN" restriction times out in CQLSH
Mikhail Stepura created CASSANDRA-7196:
--
Summary: "Select" query with "IN" restriction times out in CQLSH
Key: CASSANDRA-7196
URL: https://issues.apache.org/jira/browse/CASSANDRA-7196
Project: Cassandra
Issue Type: Bug
Reporter: Mikhail Stepura
Attachments: init_bug.cql
I've noticed that
{{pylib.cqlshlib.test.test_cqlsh_output.TestCqlshOutput#test_numeric_output}}
tests fails on the current 2.1 branch, which wasn't the case before.
Here are the steps to reproduce. I'm attaching the script to populate schema.
{code}
mstepura-mac:cassandra mikhail$ bin/cqlsh -f /init_bug.cql
mstepura-mac:cassandra mikhail$ bin/cqlsh
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.0 | Cassandra 2.1.0-beta2-SNAPSHOT | CQL spec 3.1.6 | Native
protocol v2]
Use HELP for help.
cqlsh> use test;
cqlsh:test> select intcol, bigintcol, varintcol from has_all_types where num in
(0, 1, 2, 3, 4);
errors={}, last_host=127.0.0.1
cqlsh:test>
{code}
That works perfectly on 2.0 branch. And there are no errors in the logs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994679#comment-13994679 ] Ryan McGuire edited comment on CASSANDRA-4718 at 5/11/14 9:06 PM: -- [~benedict], fwiw here's EC2 benchmarks, c3.8xlarge - 4 C* nodes - 1 separate stress node hitting those 4 nodes: * [810 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-810.log] * [270 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-270.log] * [90 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-90.log] was (Author: enigmacurry): [~benedict], fwiw here's EC2 benchmarks: * [810 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-810.log] * [270 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-270.log] * [90 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-90.log] > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, aws.svg, > backpressure-stress.out.txt, baq vs trunk.png, > belliotsmith_branches-stress.out.txt, jason_read.svg, jason_read_latency.svg, > jason_write.svg, op costs of various queues.ods, stress op rate with various > queues.ods, v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994679#comment-13994679 ] Ryan McGuire commented on CASSANDRA-4718: - @benedict, fwiw here's EC2 benchmarks: * [810 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-810.log] * [270 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-270.log] * [90 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-90.log] > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, aws.svg, > backpressure-stress.out.txt, baq vs trunk.png, > belliotsmith_branches-stress.out.txt, jason_read.svg, jason_read_latency.svg, > jason_write.svg, op costs of various queues.ods, stress op rate with various > queues.ods, v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994679#comment-13994679 ] Ryan McGuire edited comment on CASSANDRA-4718 at 5/11/14 9:03 PM: -- [~benedict], fwiw here's EC2 benchmarks: * [810 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-810.log] * [270 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-270.log] * [90 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-90.log] was (Author: enigmacurry): @benedict, fwiw here's EC2 benchmarks: * [810 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-810.log] * [270 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-270.log] * [90 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.ec2-4node.threads-90.log] > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, aws.svg, > backpressure-stress.out.txt, baq vs trunk.png, > belliotsmith_branches-stress.out.txt, jason_read.svg, jason_read_latency.svg, > jason_write.svg, op costs of various queues.ods, stress op rate with various > queues.ods, v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[2/3] git commit: prefer MemoryUtil.getByteBuffer to JNA Native.getDirectByteBuffer; specify native endian on the former patch by bes; reviewed by jbellis for CASSANDRA-6575
prefer MemoryUtil.getByteBuffer to JNA Native.getDirectByteBuffer; specify
native endian on the former
patch by bes; reviewed by jbellis for CASSANDRA-6575
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1ac72f63
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1ac72f63
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1ac72f63
Branch: refs/heads/trunk
Commit: 1ac72f637cdfc9876d2d121302061e46ac104bf8
Parents: 0b26c77
Author: Jonathan Ellis
Authored: Thu May 8 16:44:35 2014 -0500
Committer: Jonathan Ellis
Committed: Thu May 8 16:44:35 2014 -0500
--
src/java/org/apache/cassandra/io/util/Memory.java | 5 +++--
src/java/org/apache/cassandra/utils/memory/MemoryUtil.java | 1 +
2 files changed, 4 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1ac72f63/src/java/org/apache/cassandra/io/util/Memory.java
--
diff --git a/src/java/org/apache/cassandra/io/util/Memory.java
b/src/java/org/apache/cassandra/io/util/Memory.java
index b8a46bc..67dee81 100644
--- a/src/java/org/apache/cassandra/io/util/Memory.java
+++ b/src/java/org/apache/cassandra/io/util/Memory.java
@@ -22,6 +22,7 @@ import java.nio.ByteOrder;
import com.sun.jna.Native;
import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.utils.memory.MemoryUtil;
import sun.misc.Unsafe;
import sun.nio.ch.DirectBuffer;
@@ -329,10 +330,10 @@ public class Memory
int size = (int) (size() / result.length);
for (int i = 0 ; i < result.length - 1 ; i++)
{
-result[i] = Native.getDirectByteBuffer(peer + offset, size);
+result[i] = MemoryUtil.getByteBuffer(peer + offset, size);
offset += size;
}
-result[result.length - 1] = Native.getDirectByteBuffer(peer + offset,
(int) (size() - offset));
+result[result.length - 1] = MemoryUtil.getByteBuffer(peer + offset,
(int) (size() - offset));
return result;
}
}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1ac72f63/src/java/org/apache/cassandra/utils/memory/MemoryUtil.java
--
diff --git a/src/java/org/apache/cassandra/utils/memory/MemoryUtil.java
b/src/java/org/apache/cassandra/utils/memory/MemoryUtil.java
index 5f7d410..532d071 100644
--- a/src/java/org/apache/cassandra/utils/memory/MemoryUtil.java
+++ b/src/java/org/apache/cassandra/utils/memory/MemoryUtil.java
@@ -125,6 +125,7 @@ public abstract class MemoryUtil
unsafe.putLong(instance, DIRECT_BYTE_BUFFER_ADDRESS_OFFSET, address);
unsafe.putInt(instance, DIRECT_BYTE_BUFFER_CAPACITY_OFFSET, length);
unsafe.putInt(instance, DIRECT_BYTE_BUFFER_LIMIT_OFFSET, length);
+instance.order(ByteOrder.nativeOrder());
return instance;
}
[1/8] git commit: Fix potential SlabAllocator yield-starvation patch by bes; reviewed by jbellis for CASSANDRA-7133
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 d13f78ab8 -> d48c797f7
refs/heads/cassandra-2.1 750235931 -> 56136946a
refs/heads/trunk cbacf8578 -> 211d81c24
Fix potential SlabAllocator yield-starvation
patch by bes; reviewed by jbellis for CASSANDRA-7133
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d13f78ab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d13f78ab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d13f78ab
Branch: refs/heads/trunk
Commit: d13f78ab82c6fe4a6fbb5293126670fb7032203f
Parents: 46f7f84
Author: Jonathan Ellis
Authored: Thu May 8 17:04:17 2014 -0500
Committer: Jonathan Ellis
Committed: Thu May 8 17:04:17 2014 -0500
--
CHANGES.txt | 2 +
.../apache/cassandra/utils/SlabAllocator.java | 53 +---
2 files changed, 15 insertions(+), 40 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d13f78ab/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ce30239..2712398 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.9
+ * Fix potential SlabAllocator yield-starvation (CASSANDRA-7133)
* Warn when 'USING TIMESTAMP' is used on a CAS BATCH (CASSANDRA-7067)
* Starting threads in OutboundTcpConnectionPool constructor causes race
conditions (CASSANDRA-7177)
* return all cpu values from BackgroundActivityMonitor.readAndCompute
(CASSANDRA-7183)
@@ -7,6 +8,7 @@
* Fix potential NumberFormatException when deserializing IntegerType
(CASSANDRA-7088)
* cqlsh can't tab-complete disabling compaction (CASSANDRA-7185)
+
2.0.8
* Correctly delete scheduled range xfers (CASSANDRA-7143)
* Make batchlog replica selection rack-aware (CASSANDRA-6551)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/d13f78ab/src/java/org/apache/cassandra/utils/SlabAllocator.java
--
diff --git a/src/java/org/apache/cassandra/utils/SlabAllocator.java
b/src/java/org/apache/cassandra/utils/SlabAllocator.java
index dedf869..20939fe 100644
--- a/src/java/org/apache/cassandra/utils/SlabAllocator.java
+++ b/src/java/org/apache/cassandra/utils/SlabAllocator.java
@@ -18,11 +18,11 @@
package org.apache.cassandra.utils;
import java.nio.ByteBuffer;
+import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
-import com.google.common.base.Preconditions;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -47,6 +47,9 @@ public class SlabAllocator extends Allocator
private final static int REGION_SIZE = 1024 * 1024;
private final static int MAX_CLONED_SIZE = 128 * 1024; // bigger than this
don't go in the region
+// globally stash any Regions we allocate but are beaten to using, and use
these up before allocating any more
+private static final ConcurrentLinkedQueue RACE_ALLOCATED = new
ConcurrentLinkedQueue<>();
+
private final AtomicReference currentRegion = new
AtomicReference();
private final AtomicInteger regionCount = new AtomicInteger(0);
private AtomicLong unslabbed = new AtomicLong(0);
@@ -92,19 +95,20 @@ public class SlabAllocator extends Allocator
return region;
// No current region, so we want to allocate one. We race
-// against other allocators to CAS in an uninitialized region
-// (which is cheap to allocate)
-region = new Region(REGION_SIZE);
+// against other allocators to CAS in a Region, and if we fail we
stash the region for re-use
+region = RACE_ALLOCATED.poll();
+if (region == null)
+region = new Region(REGION_SIZE);
if (currentRegion.compareAndSet(null, region))
{
-// we won race - now we need to actually do the expensive
allocation step
-region.init();
regionCount.incrementAndGet();
logger.trace("{} regions now allocated in {}", regionCount,
this);
return region;
}
+
// someone else won race - that's fine, we'll try to grab theirs
// in the next iteration of the loop.
+RACE_ALLOCATED.add(region);
}
}
@@ -129,24 +133,18 @@ public class SlabAllocator extends Allocator
/**
* Actual underlying data
*/
-private ByteBuffer data;
+private final ByteBuffer data;
-private static final int UNINITIALIZED = -1;
/**
* Offset for the next allocation, or the
[1/2] git commit: return all cpu values from BackgroundActivityMonitor.readAndCompute
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 39b0d0e3e -> e241319b2
return all cpu values from BackgroundActivityMonitor.readAndCompute
patch by dbrosius reviewed by jbellis for cassandra-7183
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c4fcb166
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c4fcb166
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c4fcb166
Branch: refs/heads/cassandra-2.1
Commit: c4fcb16699690b7ed66014cd3a8d38516b14b607
Parents: dceb739
Author: Dave Brosius
Authored: Wed May 7 19:20:13 2014 -0400
Committer: Dave Brosius
Committed: Wed May 7 19:20:13 2014 -0400
--
CHANGES.txt| 1 +
src/java/org/apache/cassandra/utils/BackgroundActivityMonitor.java | 2 +-
2 files changed, 2 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c4fcb166/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 20fd115..8df7d95 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,7 @@
2.0.9
* Warn when 'USING TIMESTAMP' is used on a CAS BATCH (CASSANDRA-7067)
* Starting threads in OutboundTcpConnectionPool constructor causes race
conditions (CASSANDRA-7177)
+ * return all cpu values from BackgroundActivityMonitor.readAndCompute
(CASSANDRA-7183)
2.0.8
* Correctly delete scheduled range xfers (CASSANDRA-7143)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c4fcb166/src/java/org/apache/cassandra/utils/BackgroundActivityMonitor.java
--
diff --git a/src/java/org/apache/cassandra/utils/BackgroundActivityMonitor.java
b/src/java/org/apache/cassandra/utils/BackgroundActivityMonitor.java
index 93906eb..43113fc 100644
--- a/src/java/org/apache/cassandra/utils/BackgroundActivityMonitor.java
+++ b/src/java/org/apache/cassandra/utils/BackgroundActivityMonitor.java
@@ -85,7 +85,7 @@ public class BackgroundActivityMonitor
String name = tokenizer.nextToken();
assert name.equalsIgnoreCase("cpu");
long[] returned = new long[tokenizer.countTokens()];
-for (int i = 0; i < tokenizer.countTokens(); i++)
+for (int i = 0; i < returned.length; i++)
returned[i] = Long.parseLong(tokenizer.nextToken());
return returned;
}
[3/3] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b7d5f5a1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b7d5f5a1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b7d5f5a1 Branch: refs/heads/trunk Commit: b7d5f5a16a656e36f11d26b69e423c29ea31a04e Parents: 70d18cd 259e17d Author: Brandon Williams Authored: Fri May 9 10:05:46 2014 -0500 Committer: Brandon Williams Committed: Fri May 9 10:05:46 2014 -0500 -- src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java | 2 +- test/unit/org/apache/cassandra/db/CommitLogTest.java | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) --
[4/6] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0
Merge branch 'cassandra-1.2' into cassandra-2.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b4a3b520 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b4a3b520 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b4a3b520 Branch: refs/heads/trunk Commit: b4a3b52076e221f3fa7c65a70c7c4ddec439689c Parents: 8d4dc6d 0132e54 Author: Dave Brosius Authored: Wed May 7 01:37:48 2014 -0400 Committer: Dave Brosius Committed: Wed May 7 01:37:48 2014 -0400 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/service/StorageService.java | 5 +++-- 2 files changed, 4 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b4a3b520/CHANGES.txt -- diff --cc CHANGES.txt index d65a694,d7b7f00..517f0ab --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -26,69 -15,15 +26,70 @@@ Merged from 1.2 * Fix CQLSH parsing of functions and BLOB literals (CASSANDRA-7018) * Require nodetool rebuild_index to specify index names (CASSANDRA-7038) * Ensure that batchlog and hint timeouts do not produce hints (CASSANDRA-7058) - * Don't shut MessagingService down when replacing a node (CASSANDRA-6476) * Always clean up references in SerializingCache (CASSANDRA-6994) + * Don't shut MessagingService down when replacing a node (CASSANDRA-6476) * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751) * Preserves CQL metadata when updating table from thrift (CASSANDRA-6831) - * fix time conversion to milliseconds in SimpleCondition.await (CASSANDRA-7149) + * remove duplicate query for local tokens (CASSANDRA-7182) -1.2.16 +2.0.7 + * Put nodes in hibernate when join_ring is false (CASSANDRA-6961) + * Continue assassinating even if the endpoint vanishes (CASSANDRA-6787) + * Non-droppable verbs shouldn't be dropped from OTC (CASSANDRA-6980) + * Shutdown batchlog executor in SS#drain() (CASSANDRA-7025) + * Schedule schema pulls on change (CASSANDRA-6971) + * Avoid early loading of non-system keyspaces before compaction-leftovers + cleanup at startup (CASSANDRA-6913) + * Restrict Windows to parallel repairs (CASSANDRA-6907) + * (Hadoop) Allow manually specifying start/end tokens in CFIF (CASSANDRA-6436) + * Fix NPE in MeteredFlusher (CASSANDRA-6820) + * Fix race processing range scan responses (CASSANDRA-6820) + * Allow deleting snapshots from dropped keyspaces (CASSANDRA-6821) + * Add uuid() function (CASSANDRA-6473) + * Omit tombstones from schema digests (CASSANDRA-6862) + * Include correct consistencyLevel in LWT timeout (CASSANDRA-6884) + * Lower chances for losing new SSTables during nodetool refresh and + ColumnFamilyStore.loadNewSSTables (CASSANDRA-6514) + * Add support for DELETE ... IF EXISTS to CQL3 (CASSANDRA-5708) + * Update hadoop_cql3_word_count example (CASSANDRA-6793) + * Fix handling of RejectedExecution in sync Thrift server (CASSANDRA-6788) + * Log more information when exceeding tombstone_warn_threshold (CASSANDRA-6865) + * Fix truncate to not abort due to unreachable fat clients (CASSANDRA-6864) + * Fix schema concurrency exceptions (CASSANDRA-6841) + * Fix leaking validator FH in StreamWriter (CASSANDRA-6832) + * Fix saving triggers to schema (CASSANDRA-6789) + * Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790) + * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838) + * Fix static counter columns (CASSANDRA-6827) + * Restore expiring->deleted (cell) compaction optimization (CASSANDRA-6844) + * Fix CompactionManager.needsCleanup (CASSANDRA-6845) + * Correctly compare BooleanType values other than 0 and 1 (CASSANDRA-6779) + * Read message id as string from earlier versions (CASSANDRA-6840) + * Properly use the Paxos consistency for (non-protocol) batch (CASSANDRA-6837) + * Add paranoid disk failure option (CASSANDRA-6646) + * Improve PerRowSecondaryIndex performance (CASSANDRA-6876) + * Extend triggers to support CAS updates (CASSANDRA-6882) + * Static columns with IF NOT EXISTS don't always work as expected (CASSANDRA-6873) + * Fix paging with SELECT DISTINCT (CASSANDRA-6857) + * Fix UnsupportedOperationException on CAS timeout (CASSANDRA-6923) + * Improve MeteredFlusher handling of MF-unaffected column families + (CASSANDRA-6867) + * Add CqlRecordReader using native pagination (CASSANDRA-6311) + * Add QueryHandler interface (CASSANDRA-6659) + * Track liveRatio per-memtable, not per-CF (CASSANDRA-6945) + * Make sure upgradesstables keeps sstable level (CASSANDRA-6958) + * Fix LIMIT with static columns (CASSANDRA-6956) + * Fix clash with CQL column name in thrift validation (CASSANDRA-6892) + * Fix error with supe
[jira] [Commented] (CASSANDRA-6563) TTL histogram compactions not triggered at high "Estimated droppable tombstones" rate
[
https://issues.apache.org/jira/browse/CASSANDRA-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992870#comment-13992870
]
Paulo Ricardo Motta Gomes commented on CASSANDRA-6563:
--
Related issue: CASSANDRA-6654
> TTL histogram compactions not triggered at high "Estimated droppable
> tombstones" rate
> -
>
> Key: CASSANDRA-6563
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6563
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: 1.2.12ish
>Reporter: Chris Burroughs
>Assignee: Paulo Ricardo Motta Gomes
> Fix For: 1.2.17, 2.0.8
>
> Attachments: 1.2.16-CASSANDRA-6563.txt, 2.0.7-CASSANDRA-6563.txt,
> patched-droppadble-ratio.png, patched-storage-load.png,
> unpatched-droppable-ratio.png, unpatched-storage-load.png
>
>
> I have several column families in a largish cluster where virtually all
> columns are written with a (usually the same) TTL. My understanding of
> CASSANDRA-3442 is that sstables that have a high ( > 20%) estimated
> percentage of droppable tombstones should be individually compacted. This
> does not appear to be occurring with size tired compaction.
> Example from one node:
> {noformat}
> $ ll /data/sstables/data/ks/Cf/*Data.db
> -rw-rw-r-- 31 cassandra cassandra 26651211757 Nov 26 22:59
> /data/sstables/data/ks/Cf/ks-Cf-ic-295562-Data.db
> -rw-rw-r-- 31 cassandra cassandra 6272641818 Nov 27 02:51
> /data/sstables/data/ks/Cf/ks-Cf-ic-296121-Data.db
> -rw-rw-r-- 31 cassandra cassandra 1814691996 Dec 4 21:50
> /data/sstables/data/ks/Cf/ks-Cf-ic-320449-Data.db
> -rw-rw-r-- 30 cassandra cassandra 10909061157 Dec 11 17:31
> /data/sstables/data/ks/Cf/ks-Cf-ic-340318-Data.db
> -rw-rw-r-- 29 cassandra cassandra 459508942 Dec 12 10:37
> /data/sstables/data/ks/Cf/ks-Cf-ic-342259-Data.db
> -rw-rw-r-- 1 cassandra cassandra 336908 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342307-Data.db
> -rw-rw-r-- 1 cassandra cassandra 2063935 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342309-Data.db
> -rw-rw-r-- 1 cassandra cassandra 409 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342314-Data.db
> -rw-rw-r-- 1 cassandra cassandra31180007 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342319-Data.db
> -rw-rw-r-- 1 cassandra cassandra 2398345 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342322-Data.db
> -rw-rw-r-- 1 cassandra cassandra 21095 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342331-Data.db
> -rw-rw-r-- 1 cassandra cassandra 81454 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342335-Data.db
> -rw-rw-r-- 1 cassandra cassandra 1063718 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342339-Data.db
> -rw-rw-r-- 1 cassandra cassandra 127004 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342344-Data.db
> -rw-rw-r-- 1 cassandra cassandra 146785 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342346-Data.db
> -rw-rw-r-- 1 cassandra cassandra 697338 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342351-Data.db
> -rw-rw-r-- 1 cassandra cassandra 3921428 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342367-Data.db
> -rw-rw-r-- 1 cassandra cassandra 240332 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342370-Data.db
> -rw-rw-r-- 1 cassandra cassandra 45669 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342374-Data.db
> -rw-rw-r-- 1 cassandra cassandra53127549 Dec 12 12:03
> /data/sstables/data/ks/Cf/ks-Cf-ic-342375-Data.db
> -rw-rw-r-- 16 cassandra cassandra 12466853166 Dec 25 22:40
> /data/sstables/data/ks/Cf/ks-Cf-ic-396473-Data.db
> -rw-rw-r-- 12 cassandra cassandra 3903237198 Dec 29 19:42
> /data/sstables/data/ks/Cf/ks-Cf-ic-408926-Data.db
> -rw-rw-r-- 7 cassandra cassandra 3692260987 Jan 3 08:25
> /data/sstables/data/ks/Cf/ks-Cf-ic-427733-Data.db
> -rw-rw-r-- 4 cassandra cassandra 3971403602 Jan 6 20:50
> /data/sstables/data/ks/Cf/ks-Cf-ic-437537-Data.db
> -rw-rw-r-- 3 cassandra cassandra 1007832224 Jan 7 15:19
> /data/sstables/data/ks/Cf/ks-Cf-ic-440331-Data.db
> -rw-rw-r-- 2 cassandra cassandra 896132537 Jan 8 11:05
> /data/sstables/data/ks/Cf/ks-Cf-ic-447740-Data.db
> -rw-rw-r-- 1 cassandra cassandra 963039096 Jan 9 04:59
> /data/sstables/data/ks/Cf/ks-Cf-ic-449425-Data.db
> -rw-rw-r-- 1 cassandra cassandra 232168351 Jan 9 10:14
> /data/sstables/data/ks/Cf/ks-Cf-ic-450287-Data.db
> -rw-rw-r-- 1 cassandra cassandra73126319 Jan 9 11:28
> /data/sstables/data/ks/Cf/ks-Cf-ic-450307-Data.db
> -rw-rw-r-- 1 cassandra cassandra40921916 Jan 9 12:08
> /data/sstables/data/ks/Cf/ks-Cf-ic-450336-Data.db
> -rw-rw-r-- 1 cassandra cassandra60881193 Jan 9 12:23
> /data/ssta
[jira] [Created] (CASSANDRA-7200) word count broken
Brandon Williams created CASSANDRA-7200:
---
Summary: word count broken
Key: CASSANDRA-7200
URL: https://issues.apache.org/jira/browse/CASSANDRA-7200
Project: Cassandra
Issue Type: Bug
Components: Examples
Reporter: Brandon Williams
Fix For: 2.0.9
word_count_setup hangs forever, and word_count loops forever with this
exception:
{noformat}
DEBUG 17:52:42,875 java.io.IOException: config(config)
at org.apache.hadoop.conf.Configuration.(Configuration.java:260)
at org.apache.hadoop.mapred.JobConf.(JobConf.java:341)
at org.apache.hadoop.mapreduce.JobContext.(JobContext.java:76)
at
org.apache.hadoop.mapreduce.TaskAttemptContext.(TaskAttemptContext.java:35)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.(TaskInputOutputContext.java:44)
at org.apache.hadoop.mapreduce.MapContext.(MapContext.java:43)
at org.apache.hadoop.mapreduce.Mapper$Context.(Mapper.java:105)
at sun.reflect.GeneratedConstructorAccessor14.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:759)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7001) Windows launch feature parity - augment launch process using PowerShell to match capabilities of *nix launching
[ https://issues.apache.org/jira/browse/CASSANDRA-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994122#comment-13994122 ] Philip Thompson commented on CASSANDRA-7001: Summary of manual testing (dtest integration coming. See CASSANDRA-7202): All functionality was tested by running the .bat files through cmd and powershell, on both Windows 7 and Windows Server 2008. All of the command line arguments were functional (foreground, background, -p, -H, -E, -verbose, -help). System still started up correctly if invalid paths for output files were specified, but warning messages were displayed. An issue was found with the -p command line argument, which was fixed in v3 of the patch. Heap and new-gen memory were correctly set based on system memory (according to rules defined in comments in cassandra.ps1). The .bat file correctly handles problems with powershell shell file execution policy. For stop-server.bat, all command line arguments were tested, in both cmd and powershell, on both Windows 7 and Windows Server 2008. Stop-server.bat correctly handles receving invalid pid files, as well as valid pid-files but invalid pid's. It should be noted that CASSANDRA-7202 is tracking improving ccm compatibility with this patch, at which point the bootstrap tests will be an effective test for this patch, with a few modifications. It will also ensure that Cassandra is fully functional using these startup scripts. > Windows launch feature parity - augment launch process using PowerShell to > match capabilities of *nix launching > --- > > Key: CASSANDRA-7001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7001 > Project: Cassandra > Issue Type: Improvement >Reporter: Joshua McKenzie >Assignee: Joshua McKenzie > Labels: qa-resolved > Fix For: 2.1 rc1 > > Attachments: 7001_v1.txt, 7001_v2.txt, 7001_v3.txt > > > The current .bat-based launching has neither the logic nor robustness of a > bash or PowerShell-based solution. In pursuit of making Windows a 1st-class > citizen for C*, we need to augment the launch-process using something like > PowerShell to get as close to feature-parity as possible with Linux. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6861) Optimise our Netty 4 integration
[ https://issues.apache.org/jira/browse/CASSANDRA-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993380#comment-13993380 ] Norman Maurer commented on CASSANDRA-6861: -- Opened a PR with some improvements: https://github.com/apache/cassandra/pull/35 > Optimise our Netty 4 integration > > > Key: CASSANDRA-6861 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6861 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Assignee: T Jake Luciani >Priority: Minor > Labels: performance > Fix For: 2.1 rc1 > > > Now we've upgraded to Netty 4, we're generating a lot of garbage that could > be avoided, so we should probably stop that. Should be reasonably easy to > hook into Netty's pooled buffers, returning them to the pool once a given > message is completed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6285) 2.0 HSHA server introduces corrupt data
[
https://issues.apache.org/jira/browse/CASSANDRA-6285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994597#comment-13994597
]
Edward Capriolo commented on CASSANDRA-6285:
I was poking around the dependency a bit
{quote}
Running com.thinkaurelius.thrift.OffHeapMultiRequestTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.515 sec
Running com.thinkaurelius.thrift.OnHeapMultiConnectionWithReallocateTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.481 sec
Running com.thinkaurelius.thrift.OffheapMultiConnectionWithRellocateTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.478 sec
Running com.thinkaurelius.thrift.OnHeapMultiConnectionTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.417 sec
Running com.thinkaurelius.thrift.OnHeapMultiRequestWithReallocateTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.735 sec
Running com.thinkaurelius.thrift.OffHeapMultiRequestWithReallocateTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.437 sec
Running com.thinkaurelius.thrift.OffHeapMultiConnectionTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.282 sec
Running com.thinkaurelius.thrift.OnHeapMultiRequestTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.491 sec
{quote}
Q. Are a few tests that run in roughly 20 seconds enough to prove that this
component fit for production? These highly concurrent off heap systems can have
very subtle bugs as this ticket shows.
If I understand correct HSHA is not the default, the artifact has good test
coverage, and only a handful of findbugs issues. Is there any piece that is
going to run end-to-end or attempt to load/concurrently test these classes and
be more rigorous then the previous system? Can that be made?
> 2.0 HSHA server introduces corrupt data
> ---
>
> Key: CASSANDRA-6285
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6285
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: 4 nodes, shortly updated from 1.2.11 to 2.0.2
>Reporter: David Sauer
>Assignee: Pavel Yaskevich
>Priority: Critical
> Fix For: 2.0.8
>
> Attachments: 6285_testnotes1.txt,
> CASSANDRA-6285-disruptor-heap.patch, cassandra-attack-src.zip,
> compaction_test.py, disruptor-high-cpu.patch,
> disruptor-memory-corruption.patch, enable_reallocate_buffers.txt
>
>
> After altering everything to LCS the table OpsCenter.rollups60 amd one other
> none OpsCenter-Table got stuck with everything hanging around in L0.
> The compaction started and ran until the logs showed this:
> ERROR [CompactionExecutor:111] 2013-11-01 19:14:53,865 CassandraDaemon.java
> (line 187) Exception in thread Thread[CompactionExecutor:111,1,RMI Runtime]
> java.lang.RuntimeException: Last written key
> DecoratedKey(1326283851463420237,
> 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574426c6f6f6d46696c746572537061636555736564)
> >= current key DecoratedKey(954210699457429663,
> 37382e34362e3132382e3139382d6a7576616c69735f6e6f72785f696e6465785f323031335f31305f30382d63616368655f646f63756d656e74736c6f6f6b75702d676574546f74616c4469736b5370616365557365640b0f)
> writing into
> /var/lib/cassandra/data/OpsCenter/rollups60/OpsCenter-rollups60-tmp-jb-58656-Data.db
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:141)
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:164)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:160)
> at
> org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
> at
> org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Moving back to STC worked to keep the compactions running.
> Especialy my own Table i would
[jira] [Commented] (CASSANDRA-7001) Windows launch feature parity - augment launch process using PowerShell to match capabilities of *nix launching
[ https://issues.apache.org/jira/browse/CASSANDRA-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993750#comment-13993750 ] Philip Thompson commented on CASSANDRA-7001: >From a testing perspective this all looks good Josh. The dtests will start >running soon when I get this ccm pull request to Sylvain. > Windows launch feature parity - augment launch process using PowerShell to > match capabilities of *nix launching > --- > > Key: CASSANDRA-7001 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7001 > Project: Cassandra > Issue Type: Improvement >Reporter: Joshua McKenzie >Assignee: Joshua McKenzie > Labels: qa-resolved > Fix For: 2.1 rc1 > > Attachments: 7001_v1.txt, 7001_v2.txt, 7001_v3.txt > > > The current .bat-based launching has neither the logic nor robustness of a > bash or PowerShell-based solution. In pursuit of making Windows a 1st-class > citizen for C*, we need to augment the launch-process using something like > PowerShell to get as close to feature-parity as possible with Linux. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7101) Unable to complete request: one or more nodes were unavailable.
[
https://issues.apache.org/jira/browse/CASSANDRA-7101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994572#comment-13994572
]
Michael Shuler commented on CASSANDRA-7101:
---
Hi [~sri7gster], were you able to provide some details or steps to reproduce
this? Thanks!
> Unable to complete request: one or more nodes were unavailable.
> ---
>
> Key: CASSANDRA-7101
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7101
> Project: Cassandra
> Issue Type: Bug
> Components: Config
>Reporter: srikanth
> Fix For: 2.0.6
>
>
> I setup multidata center cluster setup in cassandra. i created keysapce using
> network topology and created a table.but i am unable to insert data into
> tables. i am getting following error to insert data into cassandra
> "Unable to complete request: one or more nodes were unavailable.
> "
> Sample Code:
> For Creating Keyspace:
> CREATE KEYSPACE TestSample
>WITH replication = {'class': 'NetworkTopologyStrategy', 'DC1' : 1,
> 'DC2' : 1}
> AND durable_writes = false;
> For Creating Tables:
> CREATE TABLE users (
> user_id int PRIMARY KEY,
> fname text,
> lname text
> );
> Inserting Into Tables:
> INSERT INTO users (user_id, fname, lname)
> VALUES (1745, 'john', 'smith');
--
This message was sent by Atlassian JIRA
(v6.2#6252)
git commit: fix c* launch issues on Russian os's due to output of linux 'free' cmd
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.0 fb0a78a23 -> 16fd1a4a8
fix c* launch issues on Russian os's due to output of linux 'free' cmd
patch by dbrosius reviewed by bwilliams for cassandra-6162
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/16fd1a4a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/16fd1a4a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/16fd1a4a
Branch: refs/heads/cassandra-2.0
Commit: 16fd1a4a89958595ca2ae44fdac2eb7aa1ad6be2
Parents: fb0a78a
Author: Dave Brosius
Authored: Thu May 8 00:32:29 2014 -0400
Committer: Dave Brosius
Committed: Thu May 8 00:32:29 2014 -0400
--
CHANGES.txt | 3 ++-
conf/cassandra-env.sh | 2 +-
2 files changed, 3 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/16fd1a4a/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6c8f1fb..05cc193 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,7 +1,8 @@
2.0.9
* Warn when 'USING TIMESTAMP' is used on a CAS BATCH (CASSANDRA-7067)
* Starting threads in OutboundTcpConnectionPool constructor causes race
conditions (CASSANDRA-7177)
- * return all cpu values from BackgroundActivityMonitor.readAndCompute
(CASSANDRA-7183)
+ * return all cpu values from BackgroundActivityMonitor.readAndCompute
(CASSANDRA-7183)
+ * fix c* launch issues on Russian os's due to output of linux 'free' cmd
(CASSANDRA-6162)
2.0.8
* Correctly delete scheduled range xfers (CASSANDRA-7143)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/16fd1a4a/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index fc4fa3d..7604918 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -18,7 +18,7 @@ calculate_heap_sizes()
{
case "`uname`" in
Linux)
-system_memory_in_mb=`free -m | awk '/Mem:/ {print $2}'`
+system_memory_in_mb=`free -m | awk '/:/ {print $2;exit}'`
system_cpu_cores=`egrep -c 'processor([[:space:]]+):.*'
/proc/cpuinfo`
;;
FreeBSD)
[jira] [Updated] (CASSANDRA-6950) Secondary index query fails with tc range query when ordered by DESC
[
https://issues.apache.org/jira/browse/CASSANDRA-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-6950:
---
Attachment: 6950-pycassa-repro.py
bq. As far as CQL3 goes we could fix it in ExtendedFilter.isSatisfiedBy by
check for ReversedType and ignoring it since the IndexExpression are really
refering to the non-reverse order, but I guess that would break thrift.
Actually, I think that would *fix* thrift, because the current behavior could
be considered wrong. See the attached script to reproduce the issue, but to
summarize, if we create an index on two columns:
* indexed: LongType
* indexed2: ReversedType(LongType)
insert data, then run a query equivalent to {{... WHERE indexed = 1 AND
indexed2 < 4}}, you'll get results where indexed2 > 4.
I have to admit that it's a little confusing as to whether using ReversedType
on indexed2 should invert the comparison, but I feel like ReversedType should
only really affect the storage format and not value comparisons.
In any case, it doesn't really make sense to use ReversedType on a column
validator, so this is unlikely to affect anybody in practice. So, unless I'm
missing another Thrift case that this could affect, I would just make your
suggested fix in {{ExtendedFilter.isSatisfiedBy}}.
> Secondary index query fails with tc range query when ordered by DESC
>
>
> Key: CASSANDRA-6950
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6950
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: RHEL 6.3 virtual guest,
> apache-cassandra-2.0.6-SNAPSHOT-src.tar.gz from build #284 (also tried with
> 2.0.5 with CASSANDRA- patch custom-applied with same result).
>Reporter: Andre Campeau
>Assignee: Sylvain Lebresne
> Fix For: 2.0.8
>
> Attachments: 6950-pycassa-repro.py, 6950.txt
>
>
> create table test4 ( name text, lname text, tc bigint, record text,
> PRIMARY KEY ((name, lname), tc)) WITH CLUSTERING ORDER BY (tc DESC) AND
> compaction={'class': 'LeveledCompactionStrategy'};
> create index test4_index ON test4(lname);
> Populate it with some data and non-zero tc values, then try:
> select * from test4 where lname='blah' and tc>0 allow filtering;
> And, (0 rows) returned, even though there are rows which should be found.
> When I create the table using CLUSTERING ORDER BY (tc ASC), the above query
> works. Rows are correctly returned based on the range check.
> Tried various combinations but with descending order on tc nothing works.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[3/4] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/773b95ef Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/773b95ef Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/773b95ef Branch: refs/heads/trunk Commit: 773b95efb85a381f56cdd537c595f0f9c9b3065f Parents: d267cf8 fb0a78a Author: Dave Brosius Authored: Wed May 7 21:22:27 2014 -0400 Committer: Dave Brosius Committed: Wed May 7 21:22:27 2014 -0400 -- CHANGES.txt| 3 ++- .../cassandra/service/PendingRangeCalculatorService.java | 6 +++--- 2 files changed, 5 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/773b95ef/CHANGES.txt -- diff --cc CHANGES.txt index ea3a192,6c8f1fb..4a0548a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,16 -1,24 +1,17 @@@ -2.0.9 - * Warn when 'USING TIMESTAMP' is used on a CAS BATCH (CASSANDRA-7067) - * Starting threads in OutboundTcpConnectionPool constructor causes race conditions (CASSANDRA-7177) - * return all cpu values from BackgroundActivityMonitor.readAndCompute (CASSANDRA-7183) - -2.0.8 +2.1.0-rc1 + * Add snapshot "manifest" describing files included (CASSANDRA-6326) + * Parallel streaming for sstableloader (CASSANDRA-3668) + * Fix bugs in supercolumns handling (CASSANDRA-7138) + * Fix ClassClassException on composite dense tables (CASSANDRA-7112) + * Cleanup and optimize collation and slice iterators (CASSANDRA-7107) + * Upgrade NBHM lib (CASSANDRA-7128) + * Optimize netty server (CASSANDRA-6861) +Merged from 2.0: * Correctly delete scheduled range xfers (CASSANDRA-7143) * Make batchlog replica selection rack-aware (CASSANDRA-6551) - * Allow overriding cassandra-rackdc.properties file (CASSANDRA-7072) - * Set JMX RMI port to 7199 (CASSANDRA-7087) - * Use LOCAL_QUORUM for data reads at LOCAL_SERIAL (CASSANDRA-6939) - * Log a warning for large batches (CASSANDRA-6487) - * Queries on compact tables can return more rows that requested (CASSANDRA-7052) - * USING TIMESTAMP for batches does not work (CASSANDRA-7053) - * Fix performance regression from CASSANDRA-5614 (CASSANDRA-6949) - * Merge groupable mutations in TriggerExecutor#execute() (CASSANDRA-7047) - * Fix CFMetaData#getColumnDefinitionFromColumnName() (CASSANDRA-7074) - * Plug holes in resource release when wiring up StreamSession (CASSANDRA-7073) - * Re-add parameter columns to tracing session (CASSANDRA-6942) - * Fix writetime/ttl functions for static columns (CASSANDRA-7081) * Suggest CTRL-C or semicolon after three blank lines in cqlsh (CASSANDRA-7142) - * return all cpu values from BackgroundActivityMonitor.readAndCompute (CASSANDRA-7183) ++ * return all cpu values from BackgroundActivityMonitor.readAndCompute (CASSANDRA-7183) ++ * reduce garbage creation in calculatePendingRanges (CASSANDRA-7191) Merged from 1.2: * Add Cloudstack snitch (CASSANDRA-7147) * Update system.peers correctly when relocating tokens (CASSANDRA-7126)
[jira] [Updated] (CASSANDRA-5663) Add write batching for the native protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-5663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-5663: Fix Version/s: (was: 3.0) 2.1.0 > Add write batching for the native protocol > -- > > Key: CASSANDRA-5663 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5663 > Project: Cassandra > Issue Type: Improvement >Reporter: Sylvain Lebresne > Labels: performance > Fix For: 2.1.0 > > > As discussed in CASSANDRA-5422, adding write batching to the native protocol > implementation is likely to improve throughput in a number of cases. -- This message was sent by Atlassian JIRA (v6.2#6252)
[2/6] git commit: CQLSH: adjusted compaction_strategy_options with the latest docs
CQLSH: adjusted compaction_strategy_options with the latest docs
patch by Mikhail Stepura; reviewed by Brandon Williams for CASSANDRA-7185
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a626c6dc
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a626c6dc
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a626c6dc
Branch: refs/heads/cassandra-2.1
Commit: a626c6dcbcfa4692167e57e3217d46642e26ff53
Parents: 51f9e98
Author: Mikhail Stepura
Authored: Thu May 8 13:00:03 2014 -0700
Committer: Mikhail Stepura
Committed: Thu May 8 13:43:42 2014 -0700
--
CHANGES.txt| 1 +
pylib/cqlshlib/cql3handling.py | 6 +-
2 files changed, 6 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a626c6dc/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3f7d68e..ce30239 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -5,6 +5,7 @@
* fix c* launch issues on Russian os's due to output of linux 'free' cmd
(CASSANDRA-6162)
* Fix disabling autocompaction (CASSANDRA-7187)
* Fix potential NumberFormatException when deserializing IntegerType
(CASSANDRA-7088)
+ * cqlsh can't tab-complete disabling compaction (CASSANDRA-7185)
2.0.8
* Correctly delete scheduled range xfers (CASSANDRA-7143)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a626c6dc/pylib/cqlshlib/cql3handling.py
--
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index af3067a..9b78638 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -79,7 +79,7 @@ class Cql3ParsingRuleSet(CqlParsingRuleSet):
# (CQL3 option name, schema_columnfamilies column name (or None if
same),
# list of known map keys)
('compaction', 'compaction_strategy_options',
-('class', 'min_threshold', 'max_threshold')),
+('class', 'max_threshold', 'tombstone_compaction_interval',
'tombstone_threshold', 'enabled')),
('compression', 'compression_parameters',
('sstable_compression', 'chunk_length_kb', 'crc_check_chance')),
)
@@ -473,6 +473,10 @@ def cf_prop_val_mapkey_completer(ctxt, cass):
csc = csc.split('.')[-1]
if csc == 'SizeTieredCompactionStrategy':
opts.add('min_sstable_size')
+opts.add('min_threshold')
+opts.add('bucket_high')
+opts.add('bucket_low')
+opts.add('cold_reads_to_omit')
elif csc == 'LeveledCompactionStrategy':
opts.add('sstable_size_in_mb')
return map(escape_value, opts)
[10/10] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/310d6e4e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/310d6e4e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/310d6e4e Branch: refs/heads/trunk Commit: 310d6e4efba63a809fac6237660545aad4009a40 Parents: 28497fd a0d096b Author: Mikhail Stepura Authored: Thu May 8 13:33:00 2014 -0700 Committer: Mikhail Stepura Committed: Thu May 8 13:33:00 2014 -0700 -- CHANGES.txt | 2 +- bin/cqlsh | 4 2 files changed, 5 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/310d6e4e/CHANGES.txt --
[jira] [Updated] (CASSANDRA-6877) pig tests broken
[ https://issues.apache.org/jira/browse/CASSANDRA-6877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-6877: Summary: pig tests broken (was: pig tests broken on 2.1/trunk) > pig tests broken > > > Key: CASSANDRA-6877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6877 > Project: Cassandra > Issue Type: Bug >Reporter: Brandon Williams >Assignee: Brandon Williams > Fix For: 2.0.9, 2.1 rc1 > > > Not sure what happened here, but I get a smorgasbord of errors running the > pig tests now, from xml errors in xerces to NotFoundExceptions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993374#comment-13993374 ] Ryan McGuire commented on CASSANDRA-4718: - Above is with 4 nodes, one of which was the one hosting stress. Here's a 3 node variety, with stress on a separate host: * [30 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.3node.threads-30.log] * [90 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.3node.threads-90.log] * [270 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.3node.threads-270.log] * [810 threads|http://riptano.github.io/cassandra_performance/graph_v3/graph.html?stats=stats.4718.3node.threads-810.log] > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, aws.svg, > backpressure-stress.out.txt, baq vs trunk.png, > belliotsmith_branches-stress.out.txt, jason_read.svg, jason_read_latency.svg, > jason_write.svg, op costs of various queues.ods, stress op rate with various > queues.ods, v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6651) Repair hanging
[ https://issues.apache.org/jira/browse/CASSANDRA-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13994485#comment-13994485 ] David Huang commented on CASSANDRA-6651: We are also running 2.0.6 with multiple single node datacenters. In testing with almost empty keyspaces, some repairs return quickly, but there are also cases where repair never completes even after 1-2 hours. The logging for replication is also very confusing. > Repair hanging > -- > > Key: CASSANDRA-6651 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6651 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Eitan Eibschutz >Assignee: Yuki Morishita > > Hi, > We have a 12 node cluster in PROD environment and we've noticed that repairs > are never finishing. The behavior that we've observed is that a repair > process will run until at some point it hangs and no other processing is > happening. > For example, at the moment, I have a repair process that has been running for > two days and not finishing: > nodetool tpstats is showing 2 active and 2 pending AntiEntropySessions > nodetool compactionstats is showing: > pending tasks: 0 > Active compaction remaining time :n/a > nodetools netstats is showing: > Mode: NORMAL > Not sending any streams. > Read Repair Statistics: > Attempted: 0 > Mismatch (Blocking): 142110 > Mismatch (Background): 0 > Pool NameActive Pending Completed > Commandsn/a 0 107589657 > Responses n/a 0 116430785 > The last entry that I see in the log is: > INFO [AntiEntropySessions:18] 2014-02-03 04:01:39,145 RepairJob.java (line > 116) [repair #ae78c6c0-8c2b-11e3-b950-c3b81a36bc9b] requesting merkle trees > for MyCF (to [/x.x.x.x, /y.y.y.y, /z.z.z.z]) > The repair started at 4am so it stopped after 1:40 minute. > On node y.y.y.y I can see this in the log: > INFO [MiscStage:1] 2014-02-03 04:01:38,985 ColumnFamilyStore.java (line 740) > Enqueuing flush of Memtable-MyCF@1290890489(2176/5931 serialized/live bytes, > 32 ops) > INFO [FlushWriter:411] 2014-02-03 04:01:38,986 Memtable.java (line 333) > Writing Memtable-MyCF@1290890489(2176/5931 serialized/live bytes, 32 ops) > INFO [FlushWriter:411] 2014-02-03 04:01:39,048 Memtable.java (line 373) > Completed flushing > /var/lib/cassandra/main-db/data/MyKS/MyCF/MyKS-MyCF-jb-518-Data.db (1789 > bytes) for commitlog position ReplayPosition(segmentId=1390437013339, > position=21868792) > INFO [ScheduledTasks:1] 2014-02-03 05:00:04,794 ColumnFamilyStore.java (line > 740) Enqueuing flush of Memtable-compaction_history@1649414699(1635/17360 > serialized/live bytes, 42 ops) > So for some reason the merkle tree for this CF is never sent back to the node > being repaired and it's hanging. > I've also noticed that sometimes, restarting node y.y.y.y will cause the > repair to resume. > Another observation is that sometimes when restarting y.y.y.y it will not > start with these errors: > ERROR 16:34:18,485 Exception encountered during startup > java.lang.IllegalStateException: Unfinished compactions reference missing > sstables. This should never happen since compactions are marked finished > before we start removing the old sstables. > at > org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:495) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:264) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:461) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:504) > java.lang.IllegalStateException: Unfinished compactions reference missing > sstables. This should never happen since compactions are marked finished > before we start removing the old sstables. > at > org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:495) > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:264) > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:461) > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:504) > Exception encountered during startup: Unfinished compactions reference > missing sstables. This should never happen since compactions are marked > finished before we start removing the old sstables. > And it will only restart after manually cleaning the compactions_in-progress > folder. > I'm not sure if these two issues are related but we've seen both on all the > nodes in our cluster. > I'll be happy to provide more info if needed as we are not sure what could > cause this behavior. > Another thing in our environment is that some of the Cass
[jira] [Updated] (CASSANDRA-7199) [dtest] snapshot_test abort logs
[
https://issues.apache.org/jira/browse/CASSANDRA-7199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Shuler updated CASSANDRA-7199:
--
Attachment: jenkins-scratch-2.1_dtest-failed-snapshot_test-dtestdir.tar.gz
dtest DEBUG log (last line created when SIGKILL was sent at job abort):
{noformat}
11:57:44,461 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG cluster ccm directory: /tmp/dtest-HHwzRI
11:57:44,600 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG tmp_commitlog: /tmp/tmp4PKZDL
11:57:49,723 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Writing first 30,000 rows...
11:57:59,958 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Making snapshot
11:58:02,171 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Running snapshot cmd: snapshot ks -cf cf -t basic
11:58:03,559 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG snapshot_dir is :
/tmp/dtest-HHwzRI/test/node1/data/ks/cf-2494e1a0d77111e393f77ff88ef322c6/snapshots/basic
11:58:03,559 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG snapshot copy is : /tmp/tmpgTsloD
11:58:03,637 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Writing second 30,000 rows...
11:58:22,562 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Writing final 5,000 rows...
11:58:25,215 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG node1 commitlog dir: /tmp/dtest-HHwzRI/test/node1/commitlogs
11:58:34,394 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing ccm cluster test at: /tmp/dtest-HHwzRI
11:58:34,404 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG cluster ccm directory: /tmp/dtest-ElgmcU
11:58:39,747 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Restoring snapshot
11:58:45,70 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG Restarting node1..
11:58:59,783 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing snapshot_dir: /tmp/tmpgTsloD
11:58:59,785 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing tmp_commitlog: /tmp/tmp4PKZDL
11:58:59,787 dtest
snapshot_test.TestArchiveCommitlog.dont_test_archive_commitlogdont_test_archive_commitlog
DEBUG removing ccm cluster test at: /tmp/dtest-ElgmcU
11:58:59,899 dtest
snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG cluster ccm directory: /tmp/dtest-se6EyY
11:59:00,35 dtest
snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG tmp_commitlog: /tmp/tmp6c0qNr
11:59:05,146 dtest
snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG Writing first 30,000 rows...
11:59:15,186 dtest
snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG Making snapshot
12:14:15,224 dtest
snapshot_test.TestArchiveCommitlog.test_archive_commitlogtest_archive_commitlog
DEBUG Running snapshot cmd: snapshot ks -cf cf -t basic
{noformat}
ccm node1 log (full ccm node tar attached) shows:
{noformat}
INFO [MemtableFlushWriter:1] 2014-05-09 11:59:05,098 Memtable.java:365 -
Completed flushing
/tmp/dtest-se6EyY/test/node1/data/system/schema_columns-296e9c049bec3085827dc17d3df2122a/system-schema_columns-ka-2-Data.db
(306 bytes) for commitlog position ReplayPosition(segmentId=1399636743311,
position=114155)
INFO [MigrationStage:1] 2014-05-09 11:59:05,121 DefsTables.java:388 - Loading
org.apache.cassandra.config.CFMetaData@5341453d[cfId=5189aba0-d771-11e3-ba97-7ff88ef322c6,ksName=ks,cfName=cf,cfType=Standard,comparator=org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.UTF8Type),comment=,readRepairChance=0.1,dcLocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.LongType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.LongType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=3 cap=3]=ColumnDefinition{name=val,
type=org.apache.cassandra.db.
[jira] [Created] (CASSANDRA-7205) Node never know a table has been DROP or CREATE if its gossip is disabled while executing this query
Zhe Yang created CASSANDRA-7205: --- Summary: Node never know a table has been DROP or CREATE if its gossip is disabled while executing this query Key: CASSANDRA-7205 URL: https://issues.apache.org/jira/browse/CASSANDRA-7205 Project: Cassandra Issue Type: Bug Environment: Cassandra 2.0.6 Reporter: Zhe Yang I'm using Cassandra 2.0.6 and I have 8 nodes. I'm doing some tests by using operations below: disable gossip of node A; check the status by nodetool in other node, node A is Down now; use cqlsh connecting an "Up" node and create a table; enable gossip of node A; check the status, all nodes are "Up" now. Then I find that node A doesn't know this table has been created. Both its own cql shell and nodetool cfstats tell me the table doesn't exist. Even waiting for a few minutes to get the "eventual consistency" final status, node A still doesn't know this table. And I find if each node knows there is a table but I drop it when one node's gossip is disabled, this node will never know the table has been dropped. Is this a bug? -- This message was sent by Atlassian JIRA (v6.2#6252)
