[jira] [Commented] (CASSANDRA-7731) Get max values for live/tombstone cells per slice
[ https://issues.apache.org/jira/browse/CASSANDRA-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096681#comment-14096681 ] Robert Stupp commented on CASSANDRA-7731: - Yea - please don't nail me on the exact versions (it's actually deprecated a long time) - but it has been refactored :) > Get max values for live/tombstone cells per slice > - > > Key: CASSANDRA-7731 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7731 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Cyril Scetbon >Assignee: Robert Stupp >Priority: Minor > Fix For: 2.0.11, 2.1.1 > > Attachments: 7731-2.0.txt, 7731-2.1.txt > > > I think you should not say that slice statistics are valid for the [last five > minutes > |https://github.com/apache/cassandra/blob/cassandra-2.0/src/java/org/apache/cassandra/tools/NodeCmd.java#L955-L956] > in CFSTATS command of nodetool. I've read the documentation from yammer for > Histograms and there is no way to force values to expire after x minutes > except by > [clearing|http://grepcode.com/file/repo1.maven.org/maven2/com.yammer.metrics/metrics-core/2.1.2/com/yammer/metrics/core/Histogram.java#96] > it . The only thing I can see is that the last snapshot used to provide the > median (or whatever you'd used instead) value is based on 1028 values. > I think we should also be able to detect that some requests are accessing a > lot of live/tombstone cells per query and that's not possible for now without > activating DEBUG for SliceQueryFilter for example and by tweaking the > threshold. Currently as nodetool cfstats returns the median if a low part of > the queries are scanning a lot of live/tombstone cells we miss it ! -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7762) range query on secondary index times out (1 row, 1 node)
[
https://issues.apache.org/jira/browse/CASSANDRA-7762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7762:
Assignee: Tyler Hobbs
> range query on secondary index times out (1 row, 1 node)
>
>
> Key: CASSANDRA-7762
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7762
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Mac OSX 10.7.5
> Fresh install of Cassandra 2.0.9
>Reporter: Ian Rose
>Assignee: Tyler Hobbs
> Attachments: query-trace.txt
>
>
> Setup:
> - single cassandra node (on localhost)
> - in cassandra.yaml, listen_address: "127.0.0.1" and seeds: "127.0.0.1"
> Here is my test schema:
> CREATE TABLE foo (
> foo_name ascii,
> foo_shard bigint,
> int_val bigint,
> PRIMARY KEY ((foo_name, foo_shard))
> ) WITH read_repair_chance=0.1;
> CREATE INDEX ON foo (int_val);
> CREATE INDEX ON foo (foo_name);
> I have inserted just a single row into this table:
> insert into foo(foo_name, foo_shard, int_val) values('dave', 27, 100);
> This query works fine:
> > select * from foo where foo_name='dave';
> But when I run this query, I get an RPC timeout:
> > select * from foo where foo_name='dave' and int_val > 0 allow filtering;
> Request did not complete within rpc_timeout.
> With tracing enabled, here is the trace output (also attached):
> http://pastebin.com/raw.php?i=6XMEVUcQ
> (In short, everything looks fine to my untrained eye until 10s elapsed, at
> which time the following event is logged: "Timed out; received 0 of 1
> responses for range 257 of 257")
> Also note that if I *remove* the index on int_val (leaving just the index on
> foo_name), then the query works fine:
> > select * from foo where foo_name='dave' and int_val > 0 allow filtering;
> foo_name | foo_shard | int_val
> --+---+-
> dave |27 | 100
> (1 rows)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7762) range query on secondary index times out (1 row, 1 node)
[
https://issues.apache.org/jira/browse/CASSANDRA-7762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7762:
Assignee: (was: Tyler Hobbs)
> range query on secondary index times out (1 row, 1 node)
>
>
> Key: CASSANDRA-7762
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7762
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Mac OSX 10.7.5
> Fresh install of Cassandra 2.0.9
>Reporter: Ian Rose
> Attachments: query-trace.txt
>
>
> Setup:
> - single cassandra node (on localhost)
> - in cassandra.yaml, listen_address: "127.0.0.1" and seeds: "127.0.0.1"
> Here is my test schema:
> CREATE TABLE foo (
> foo_name ascii,
> foo_shard bigint,
> int_val bigint,
> PRIMARY KEY ((foo_name, foo_shard))
> ) WITH read_repair_chance=0.1;
> CREATE INDEX ON foo (int_val);
> CREATE INDEX ON foo (foo_name);
> I have inserted just a single row into this table:
> insert into foo(foo_name, foo_shard, int_val) values('dave', 27, 100);
> This query works fine:
> > select * from foo where foo_name='dave';
> But when I run this query, I get an RPC timeout:
> > select * from foo where foo_name='dave' and int_val > 0 allow filtering;
> Request did not complete within rpc_timeout.
> With tracing enabled, here is the trace output (also attached):
> http://pastebin.com/raw.php?i=6XMEVUcQ
> (In short, everything looks fine to my untrained eye until 10s elapsed, at
> which time the following event is logged: "Timed out; received 0 of 1
> responses for range 257 of 257")
> Also note that if I *remove* the index on int_val (leaving just the index on
> foo_name), then the query works fine:
> > select * from foo where foo_name='dave' and int_val > 0 allow filtering;
> foo_name | foo_shard | int_val
> --+---+-
> dave |27 | 100
> (1 rows)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (CASSANDRA-7762) range query on secondary index times out (1 row, 1 node)
[
https://issues.apache.org/jira/browse/CASSANDRA-7762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne resolved CASSANDRA-7762.
-
Resolution: Duplicate
This is a duplicate of CASSANDRA-6612. The same AssertionError shows in the
server log in particular (and the trace confirms that the no index is used
while it should have). I've confirmed that this does work properly on the
current cassandra-2.0 branch.
> range query on secondary index times out (1 row, 1 node)
>
>
> Key: CASSANDRA-7762
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7762
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Mac OSX 10.7.5
> Fresh install of Cassandra 2.0.9
>Reporter: Ian Rose
> Attachments: query-trace.txt
>
>
> Setup:
> - single cassandra node (on localhost)
> - in cassandra.yaml, listen_address: "127.0.0.1" and seeds: "127.0.0.1"
> Here is my test schema:
> CREATE TABLE foo (
> foo_name ascii,
> foo_shard bigint,
> int_val bigint,
> PRIMARY KEY ((foo_name, foo_shard))
> ) WITH read_repair_chance=0.1;
> CREATE INDEX ON foo (int_val);
> CREATE INDEX ON foo (foo_name);
> I have inserted just a single row into this table:
> insert into foo(foo_name, foo_shard, int_val) values('dave', 27, 100);
> This query works fine:
> > select * from foo where foo_name='dave';
> But when I run this query, I get an RPC timeout:
> > select * from foo where foo_name='dave' and int_val > 0 allow filtering;
> Request did not complete within rpc_timeout.
> With tracing enabled, here is the trace output (also attached):
> http://pastebin.com/raw.php?i=6XMEVUcQ
> (In short, everything looks fine to my untrained eye until 10s elapsed, at
> which time the following event is logged: "Timed out; received 0 of 1
> responses for range 257 of 257")
> Also note that if I *remove* the index on int_val (leaving just the index on
> foo_name), then the query works fine:
> > select * from foo where foo_name='dave' and int_val > 0 allow filtering;
> foo_name | foo_shard | int_val
> --+---+-
> dave |27 | 100
> (1 rows)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7499) Unable to update list element by index using CAS condition
[
https://issues.apache.org/jira/browse/CASSANDRA-7499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7499:
Attachment: 7499-2.0.txt
Attaching simple "throw IRE" patch for 2.0 too.
bq. The cas_and_list_index_test is failing on 2.1
Right, that's the test for this issue and this hasn't been committed yet. The
test shouldn't have been run, it's my fault for commenting out the
'@require(7499)' on the test. I've re-enabled it (I'll re-remove it once it's
committed).
> Unable to update list element by index using CAS condition
> --
>
> Key: CASSANDRA-7499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7499
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra *2.0.9*, Java Driver *2.0.2* & *2.0.3*
> Client: cqlsh *3.1.8*, CQL spec *3.1.0*, Thrift protocol *19.39.0*
>Reporter: DOAN DuyHai
>Assignee: Sylvain Lebresne
> Fix For: 2.0.10
>
> Attachments: 7499-2.0.txt, 7499-2.1.txt, 7499.txt
>
>
> While running IT tests for *Achilles*, I ran into a strange bug:
> *With CQLSH*
> {code:sql}
> cqlsh:test> CREATE TABLE cas_update(id int PRIMARY KEY,name text,friends
> list);
> cqlsh:test> INSERT INTO cas_update (id, name , friends ) VALUES (
> 10,'John',['Paul','George']);
> cqlsh:test> SELECT * FROM cas_update WHERE id=10;
> id | friends| name
> ++--
> 10 | ['Paul', 'George'] | John
> cqlsh:test> UPDATE cas_update SET friends[0]='Helen' WHERE id=10 IF
> name='John';
> Bad Request: List index 0 out of bound, list has size 0
> cqlsh:test> UPDATE cas_update SET friends[0]='Helen' WHERE id=10;
> cqlsh:test> SELECT * FROM cas_update WHERE id=10;
> id | friends | name
> +-+--
> 10 | ['Helen', 'George'] | John
> {code}
> It seems that we cannot update list element by index with a CAS condition.
> *With Java driver 2.0.2 or 2.0.3*
> {code:java}
> ACHILLES_DML_STATEMENT@:writeDMLStatementLog Prepared statement : [INSERT
> INTO CompleteBean(id,followers,friends,name,preferences) VALUES
> (:id,:followers,:friends,:name,:preferences) USING TTL :ttl;] with
> CONSISTENCY LEVEL [ONE]
> ACHILLES_DML_STATEMENT@:writeDMLStatementLogbound values :
> [621309709026375591, [], [Paul, Andrew], John, {}, 0]
> ACHILLES_DML_STATEMENT@:writeDMLStartBatch
> ACHILLES_DML_STATEMENT@:writeDMLStartBatch
> ACHILLES_DML_STATEMENT@:writeDMLStartBatch ** BATCH UNLOGGED START
> **
> ACHILLES_DML_STATEMENT@:writeDMLStartBatch
> ACHILLES_DML_STATEMENT@:writeDMLStatementLog Parameterized statement :
> [UPDATE CompleteBean USING TTL 100 SET friends[0]=? WHERE
> id=621309709026375591 IF name=?;] with CONSISTENCY LEVEL [ONE]
> ACHILLES_DML_STATEMENT@:writeDMLStatementLogbound values : [100, 0,
> Helen, 621309709026375591, John]
> ACHILLES_DML_STATEMENT@:writeDMLStatementLog Parameterized statement :
> [UPDATE CompleteBean USING TTL 100 SET friends[1]=null WHERE
> id=621309709026375591 IF name=?;] with CONSISTENCY LEVEL [ONE]
> ACHILLES_DML_STATEMENT@:writeDMLStatementLogbound values : [100, 1,
> null, 621309709026375591, John]
> ACHILLES_DML_STATEMENT@:writeDMLEndBatch
> ACHILLES_DML_STATEMENT@:writeDMLEndBatch ** BATCH UNLOGGED END with
> CONSISTENCY LEVEL [DEFAULT] **
> ACHILLES_DML_STATEMENT@:writeDMLEndBatch
> ACHILLES_DML_STATEMENT@:writeDMLEndBatch
> ACHILLES_DML_STATEMENT@:truncateTable Simple query : [TRUNCATE
> entity_with_enum] with CONSISTENCY LEVEL [ALL]
> ACHILLES_DML_STATEMENT@:truncateTable Simple query : [TRUNCATE
> CompleteBean] with CONSISTENCY LEVEL [ALL]
> com.datastax.driver.core.exceptions.InvalidQueryException: List index 0 out
> of bound, list has size 0
> at
> com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:35)
> at
> com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:256)
> at
> com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:172)
> at
> com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:52)
> {code}
> With Cassandra *2.0.8* and Java Driver 2.0.2 or 2.0.3, *the test passed* so
> it seems that there is a regression somewhere in the CAS update code
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7766) Secondary index not working after a while
Fabrice Larcher created CASSANDRA-7766: -- Summary: Secondary index not working after a while Key: CASSANDRA-7766 URL: https://issues.apache.org/jira/browse/CASSANDRA-7766 Project: Cassandra Issue Type: Bug Environment: C* 2.1.0-rc5 with small clusters (one or two nodes) Reporter: Fabrice Larcher Since 2.1.0-rc2, it appears that the secondary indexes are not always working. Immediately after the INSERT of a row, the index seems to be there. But after a while (I do not know when or why), SELECT statements based on any secondary index do not return the corresponding row(s) anymore. I noticed that a restart of C* may have an impact (the data inserted before the restart may be seen through the index, even if it was not returned before the restart). Here is a use-case example (in order to clarify my request) : CREATE TABLE IF NOT EXISTS ks.cf ( k int PRIMARY KEY, ind ascii, value text); CREATE INDEX IF NOT EXISTS ks_cf_index ON ks.cf(ind); INSERT INTO ks.cf (k, ind, value) VALUES (1, 'toto', 'Hello'); SELECT * FROM ks.cf WHERE ind = 'toto'; // Returns no result after a while The last SELECT statement may or may not return a row depending on the instant of the request. I experienced that with 2.1.0-rc5 through CQLSH with clusters of one and two nodes. Since it depends on the instant of the request, I am not able to deliver any way to reproduce that systematically (It appears to be linked with some scheduled job inside C*). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7766) Secondary index not working after a while
[
https://issues.apache.org/jira/browse/CASSANDRA-7766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabrice Larcher updated CASSANDRA-7766:
---
Description:
Since 2.1.0-rc2, it appears that the secondary indexes are not always working.
Immediately after the INSERT of a row, the index seems to be there. But after a
while (I do not know when or why), SELECT statements based on any secondary
index do not return the corresponding row(s) anymore. I noticed that a restart
of C* may have an impact (the data inserted before the restart may be seen
through the index, even if it was not returned before the restart).
Here is a use-case example (in order to clarify my request) :
{code}
CREATE TABLE IF NOT EXISTS ks.cf ( k int PRIMARY KEY, ind ascii, value text);
CREATE INDEX IF NOT EXISTS ks_cf_index ON ks.cf(ind);
INSERT INTO ks.cf (k, ind, value) VALUES (1, 'toto', 'Hello');
SELECT * FROM ks.cf WHERE ind = 'toto'; // Returns no result after a while
{code}
The last SELECT statement may or may not return a row depending on the instant
of the request. I experienced that with 2.1.0-rc5 through CQLSH with clusters
of one and two nodes. Since it depends on the instant of the request, I am not
able to deliver any way to reproduce that systematically (It appears to be
linked with some scheduled job inside C*).
was:
Since 2.1.0-rc2, it appears that the secondary indexes are not always working.
Immediately after the INSERT of a row, the index seems to be there. But after a
while (I do not know when or why), SELECT statements based on any secondary
index do not return the corresponding row(s) anymore. I noticed that a restart
of C* may have an impact (the data inserted before the restart may be seen
through the index, even if it was not returned before the restart).
Here is a use-case example (in order to clarify my request) :
CREATE TABLE IF NOT EXISTS ks.cf ( k int PRIMARY KEY, ind ascii, value text);
CREATE INDEX IF NOT EXISTS ks_cf_index ON ks.cf(ind);
INSERT INTO ks.cf (k, ind, value) VALUES (1, 'toto', 'Hello');
SELECT * FROM ks.cf WHERE ind = 'toto'; // Returns no result after a while
The last SELECT statement may or may not return a row depending on the instant
of the request. I experienced that with 2.1.0-rc5 through CQLSH with clusters
of one and two nodes. Since it depends on the instant of the request, I am not
able to deliver any way to reproduce that systematically (It appears to be
linked with some scheduled job inside C*).
Attachment: result-success.txt
result-failure.txt
You will find some traces attached ; one file the success case and one for the
failure case. Here are some information about them ;
*CQL structure :*
A table {{user}} with an index {{user_email}} on column {{email}} of type
{{ascii}}.
*CQLSH commands :*
{code}
use main;
tracing on;
consistency all;
select * from user where email = 'fabrice.larc...@level5.fr';
{code}
I hope it could help
> Secondary index not working after a while
> -
>
> Key: CASSANDRA-7766
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7766
> Project: Cassandra
> Issue Type: Bug
> Environment: C* 2.1.0-rc5 with small clusters (one or two nodes)
>Reporter: Fabrice Larcher
> Attachments: result-failure.txt, result-success.txt
>
>
> Since 2.1.0-rc2, it appears that the secondary indexes are not always
> working. Immediately after the INSERT of a row, the index seems to be there.
> But after a while (I do not know when or why), SELECT statements based on any
> secondary index do not return the corresponding row(s) anymore. I noticed
> that a restart of C* may have an impact (the data inserted before the restart
> may be seen through the index, even if it was not returned before the
> restart).
> Here is a use-case example (in order to clarify my request) :
> {code}
> CREATE TABLE IF NOT EXISTS ks.cf ( k int PRIMARY KEY, ind ascii, value text);
> CREATE INDEX IF NOT EXISTS ks_cf_index ON ks.cf(ind);
> INSERT INTO ks.cf (k, ind, value) VALUES (1, 'toto', 'Hello');
> SELECT * FROM ks.cf WHERE ind = 'toto'; // Returns no result after a while
> {code}
> The last SELECT statement may or may not return a row depending on the
> instant of the request. I experienced that with 2.1.0-rc5 through CQLSH with
> clusters of one and two nodes. Since it depends on the instant of the
> request, I am not able to deliver any way to reproduce that systematically
> (It appears to be linked with some scheduled job inside C*).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7704) FileNotFoundException during STREAM-OUT triggers 100% CPU usage
[ https://issues.apache.org/jira/browse/CASSANDRA-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096779#comment-14096779 ] Benedict commented on CASSANDRA-7704: - Not cleaning up resources is really not ideal in my book, however there is absolutely no reason we need to cancel with interruption - note this does *not* always result in a cancelled state if it ran, only if it was in the middle of running at the time (but still completed), and this can be fixed by not permitting it to be interrupted. However this is not the problem - in the test it will often be the case that the task was _genuinely_ successfully cancelled. In my opinion the test is broken, since previously there was _no_ guarantee that all cancellations would run (although the cancellation in the test case will); after the last task completes successfully the scheduled tasks were all removed from the queue (but *not* cancelled), so the behaviour of the future in this case would be to never return, which is much more surprising and inconsistent in my book. I'm not entirely sure what the offending line is intended to test, anyway? > FileNotFoundException during STREAM-OUT triggers 100% CPU usage > --- > > Key: CASSANDRA-7704 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7704 > Project: Cassandra > Issue Type: Bug >Reporter: Rick Branson >Assignee: Benedict > Attachments: 7704.20.v2.txt, 7704.txt, backtrace.txt, other-errors.txt > > > See attached backtrace which was what triggered this. This stream failed and > then ~12 seconds later it emitted that exception. At that point, all CPUs > went to 100%. A thread dump shows all the ReadStage threads stuck inside > IntervalTree.searchInternal inside of CFS.markReferenced(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6726) Recycle CompressedRandomAccessReader/RandomAccessReader buffers independently of their owners, and move them off-heap when possible
[ https://issues.apache.org/jira/browse/CASSANDRA-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6726: Assignee: Branimir Lambov > Recycle CompressedRandomAccessReader/RandomAccessReader buffers independently > of their owners, and move them off-heap when possible > --- > > Key: CASSANDRA-6726 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6726 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Branimir Lambov >Priority: Minor > Labels: performance > Fix For: 3.0 > > > Whilst CRAR and RAR are pooled, we could and probably should pool the buffers > independently, so that they are not tied to a specific sstable. It may be > possible to move the RAR buffer off-heap, and the CRAR sometimes (e.g. Snappy > may possibly support off-heap buffers) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-5902) Dealing with hints after a topology change
[ https://issues.apache.org/jira/browse/CASSANDRA-5902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-5902: Assignee: Branimir Lambov > Dealing with hints after a topology change > -- > > Key: CASSANDRA-5902 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5902 > Project: Cassandra > Issue Type: Bug >Reporter: Jonathan Ellis >Assignee: Branimir Lambov >Priority: Minor > > Hints are stored and delivered by destination node id. This allows them to > survive IP changes in the target, while making "scan all the hints for a > given destination" an efficient operation. However, we do not detect and > handle new node assuming responsibility for the hinted row via bootstrap > before it can be delivered. > I think we have to take a performance hit in this case -- we need to deliver > such a hint to *all* replicas, since we don't know which is the "new" one. > This happens infrequently enough, however -- requiring first the target node > to be down to create the hint, then the hint owner to be down long enough for > the target to both recover and stream to a new node -- that this should be > okay. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7039) DirectByteBuffer compatible LZ4 methods
[ https://issues.apache.org/jira/browse/CASSANDRA-7039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-7039: Assignee: Branimir Lambov (was: Lyuben Todorov) > DirectByteBuffer compatible LZ4 methods > --- > > Key: CASSANDRA-7039 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7039 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Assignee: Branimir Lambov >Priority: Minor > Labels: performance > Fix For: 3.0 > > > As we move more things off-heap, it's becoming more and more essential to be > able to use DirectByteBuffer (or native pointers) in various places. > Unfortunately LZ4 doesn't currently support this operation, despite being JNI > based - this means we both have to perform unnecessary copies to de/compress > data from DBB, but also we can stall GC as any JNI method operating over a > java array using the GetPrimitiveArrayCritical enters a critical section that > prevents GC for its duration. This means STWs will be at least as long any > running compression/decompression (and no GC will happen until they complete, > so it's additive). > We should temporarily fork (and then resubmit upstream) jpountz-lz4 to > support operating over a native pointer, so that we can pass a DBB or a raw > pointer we have allocated ourselves. This will help improve performance when > flushing the new offheap memtables, as well as enable us to implement > CASSANDRA-6726 and finish CASSANDRA-4338. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-7546: Fix Version/s: 2.1.1 2.0.11 > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Fix For: 2.0.11, 2.1.1 > > Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, > 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, > 7546.20_alt.txt, suggestion1.txt, suggestion1_21.txt > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7561) On DROP we should invalidate CounterKeyCache as well as Key/Row cache
[ https://issues.apache.org/jira/browse/CASSANDRA-7561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-7561: Fix Version/s: 2.1.0 > On DROP we should invalidate CounterKeyCache as well as Key/Row cache > - > > Key: CASSANDRA-7561 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7561 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 2.1.0 > > > We should also probably ensure we don't attempt to auto save _any_ of the > caches while they are in an inconsistent state (i.e. there are keys present > to be saved that should not be restored, or that would throw exceptions when > we save (e.g. CounterCacheKey)) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7704) FileNotFoundException during STREAM-OUT triggers 100% CPU usage
[ https://issues.apache.org/jira/browse/CASSANDRA-7704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-7704: Fix Version/s: 2.1.0 2.0.10 > FileNotFoundException during STREAM-OUT triggers 100% CPU usage > --- > > Key: CASSANDRA-7704 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7704 > Project: Cassandra > Issue Type: Bug >Reporter: Rick Branson >Assignee: Benedict > Fix For: 2.0.10, 2.1.0 > > Attachments: 7704.20.v2.txt, 7704.txt, backtrace.txt, other-errors.txt > > > See attached backtrace which was what triggered this. This stream failed and > then ~12 seconds later it emitted that exception. At that point, all CPUs > went to 100%. A thread dump shows all the ReadStage threads stuck inside > IntervalTree.searchInternal inside of CFS.markReferenced(). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-3852) use LIFO queueing policy when queue size exceeds thresholds
[
https://issues.apache.org/jira/browse/CASSANDRA-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict updated CASSANDRA-3852:
Fix Version/s: 3.0
> use LIFO queueing policy when queue size exceeds thresholds
> ---
>
> Key: CASSANDRA-3852
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3852
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Peter Schuller
>Assignee: Peter Schuller
> Labels: performance
> Fix For: 3.0
>
>
> A strict FIFO policy for queueing (between stages) is detrimental to latency
> and forward progress. Whenever a node is saturated beyond incoming request
> rate, *all* requests become slow. If it is consistently saturated, you start
> effectively timing out on *all* requests.
> A much better strategy from the point of view of latency is to serve a subset
> requests quickly, and letting some time out, rather than letting all either
> time out or be slow.
> Care must be taken such that:
> * We still guarantee that requests are processed reasonably timely (we
> couldn't go strict LIFO for example as that would result in requests getting
> stuck potentially forever on a loaded node).
> * Maybe, depending on the previous point's solution, ensure that some
> requests bypass the policy and get prioritized (e.g., schema migrations, or
> anything "internal" to a node).
> A possible implementation is to go LIFO whenever there are requests in the
> queue that are older than N milliseconds (or a certain queue size, etc).
> Benefits:
> * All cases where the client is directly, or is indirectly affecting through
> other layers, a system which has limited concurrency (e.g., thread pool size
> of X to serve some incoming request rate), it is *much* better for a few
> requests to time out while most are serviced quickly, than for all requests
> to become slow, as it doesn't explode concurrency. Think any random
> non-super-advanced php app, ruby web app, java servlet based app, etc.
> Essentially, it optimizes very heavily for improved average latencies.
> * Systems with strict p95/p99/p999 requirements on latencies should greatly
> benefit from such a policy. For example, suppose you have a system at 85% of
> capacity, and it takes a write spike (or has a hiccup like GC pause, blocking
> on a commit log write, etc). Suppose the hiccup racks up 500 ms worth of
> requests. At 15% margin at steady state, that takes 500ms * 100/15 = 3.2
> seconds to recover. Instead of *all* requests for an entire 3.2 second window
> being slow, we'd serve requests quickly for 2.7 of those seconds, with the
> incoming requests during that 500 ms interval being the ones primarily
> affected. The flip side though is that once you're at the point where more
> than N percent of requests end up having to wait for others to take LIFO
> priority, the p(100-N) latencies will actually be *worse* than without this
> change (but at this point you have to consider what the root reason for those
> pXX requirements are).
> * In the case of complete saturation, it allows forward progress. Suppose
> you're taking 25% more traffic than you are able to handle. Instead of
> getting backed up and ending up essentially timing out *every single
> request*, you will succeed in processing up to 75% of them (I say "up to"
> because it depends; for example on a {{QUORUM}} request you need at least two
> of the requests from the co-ordinator to succeed so the percentage is brought
> down) and allowing clients to make forward progress and get work done, rather
> than being stuck.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7542) Reduce CAS contention
[ https://issues.apache.org/jira/browse/CASSANDRA-7542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096805#comment-14096805 ] Benedict commented on CASSANDRA-7542: - [~kohlisankalp] any news? > Reduce CAS contention > - > > Key: CASSANDRA-7542 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7542 > Project: Cassandra > Issue Type: Improvement >Reporter: sankalp kohli >Assignee: Benedict > Fix For: 2.0.10 > > > CAS updates on same CQL partition can lead to heavy contention inside C*. I > am looking for simple ways(no algorithmic changes) to reduce contention as > the penalty of it is high in terms of latency, specially for reads. > We can put some sort of synchronization on CQL partition at StorageProxy > level. This will reduce contention at least for all requests landing on one > box for same partition. > Here is an example of why it will help: > 1) Say 1 write and 2 read CAS requests for the same partition key is send to > C* in parallel. > 2) Since client is token-aware, it sends these 3 request to the same C* > instance A. (Lets assume that all 3 requests goto same instance A) > 3) In this C* instance A, all 3 CAS requests will contend with each other in > Paxos. (This is bad) > To improve contention in 3), what I am proposing is to add a lock on > partition key similar to what we do in PaxosState.java to serialize these 3 > requests. This will remove the contention and improve performance as these 3 > requests will not collide with each other. > Another improvement we can do in client is to pick a deterministic live > replica for a given partition doing CAS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-6780) Memtable OffHeap GC Statistics
[ https://issues.apache.org/jira/browse/CASSANDRA-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict resolved CASSANDRA-6780. - Resolution: Later Since offheap GC has been postponed indefinitely, this ticket should also be closed to revisit later. > Memtable OffHeap GC Statistics > -- > > Key: CASSANDRA-6780 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6780 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Priority: Minor > > As mentioned in CASSANDRA-6689, it would be nice to expose via JMX some > statistics on GC behaviour, instead of just optionally debug logging it (and > maybe expand to cover some more things): > - Time spent in GC > - Amount of memory reclaimed > - Number of collections (per CFS?), and average reclaimed per collection -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp reassigned CASSANDRA-7577: --- Assignee: Robert Stupp > cqlsh: CTRL-R history search not working on OSX > --- > > Key: CASSANDRA-7577 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 > Project: Cassandra > Issue Type: Bug > Environment: OSX - plain Terminal program > C* 2.0.x, 2.1, trunk >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Minor > > _recursive-history-search_ via ctrl-R does not work in cqlsh. The history > itself works via cursor up/down. > It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-6709) Changes to KeyCache
[ https://issues.apache.org/jira/browse/CASSANDRA-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict resolved CASSANDRA-6709. - Resolution: Later Closing as the new sstable format most likely makes this unnecessary by eliminating the need for a separate key cache, although we *may* want to revisit this at some point afterwards since a separate cache could still be beneficial by improving memory occupancy rate, so closing as later instead of duplicate. > Changes to KeyCache > --- > > Key: CASSANDRA-6709 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6709 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Priority: Minor > > It seems to me that KeyCache can be improved in a number of ways, but first > let's state the basic goal of KeyCache: to reduce the average query response > time by providing an exact seek position in a file for a given key. > As it stands, KeyCache is both 100% accurate, but requires a lot of overhead > per entry. > I propose to make KeyCache *mostly* accurate (say 99.%), by which I means > it will always fail accurately, but may rarely return an incorrect address, > and code the end users of it to be able to retry to confirm they seeked to > the correct position in the file, and to retry without the cache if they did > not. > The advantage of this is that we can both take the cache off-heap easily, and > pack a lot more items into the cache. If we permit collisions across files > and simply use the (full 128-bit) murmur hash of the key + cfId + file > generation, we should get enough uniqueness to rarely have an erroneuous > collision, however we will be using only 20 bytes per key, instead of the > current 100 + bytes. This should allow us to answer far more > queries from the key cache than before, so the positive improvement to > performance should be greater than the negative drain. > For the structure I propose an associative cache, where a single contiguous > address space is broken up into regions of, say, 8 entries, plus one counter. > The counter tracks the recency of access of each of the entries, so that on > write the least recently accessed/written can be replaced. A linear probe > within the region is used to determine if the entry we're looking for is > present. This should be very quick, as the entire region should fit into one > or two lines of L1. > Advantage: we may see 5x bump in cache hit-rate, or even more for large keys. -- This message was sent by Atlassian JIRA (v6.2#6252)
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure on builder cassandra-trunk while building libcloud. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/646 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: lares_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] 23d4355a1c86cefdeb04936265a820723a3f5c15 Blamelist: Tomaz Muraus BUILD FAILED: failed sincerely, -The Buildbot
[jira] [Resolved] (CASSANDRA-6802) Row cache improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict resolved CASSANDRA-6802. - Resolution: Later Since offheap GC has been postponed indefinitely, this ticket should also be closed to revisit later. > Row cache improvements > -- > > Key: CASSANDRA-6802 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6802 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson > Labels: performance > Fix For: 3.0 > > > There are a few things we could do; > * Start using the native memory constructs from CASSANDRA-6694 to avoid > serialization/deserialization costs and to minimize the on-heap overhead > * Stop invalidating cached rows on writes (update on write instead). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7029) Investigate alternative transport protocols for both client and inter-server communications
[ https://issues.apache.org/jira/browse/CASSANDRA-7029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096814#comment-14096814 ] Robert Stupp commented on CASSANDRA-7029: - mTCP seems to be a nice thing - although not directly related to C* but to Java Runtime: http://shader.kaist.edu/mtcp/ https://github.com/eunyoung14/mtcp Give that a try? Means: original OpenJDK vs. OpenJDK with mTCP. Patching mTCP into OpenJDK seems to pretty straightforward ( s/foo/bar/g ). > Investigate alternative transport protocols for both client and inter-server > communications > --- > > Key: CASSANDRA-7029 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7029 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > There are a number of reasons to think we can do better than TCP for our > communications: > 1) We can actually tolerate sporadic small message losses, so guaranteed > delivery isn't essential (although for larger messages it probably is) > 2) As shown in \[1\] and \[2\], Linux can behave quite suboptimally with > regard to TCP message delivery when the system is under load. Judging from > the theoretical description, this is likely to apply even when the > system-load is not high, but the number of processes to schedule is high. > Cassandra generally has a lot of threads to schedule, so this is quite > pertinent for us. UDP performs substantially better here. > 3) Even when the system is not under load, UDP has a lower CPU burden, and > that burden is constant regardless of the number of connections it processes. > 4) On a simple benchmark on my local PC, using non-blocking IO for UDP and > busy spinning on IO I can actually push 20-40% more throughput through > loopback (where TCP should be optimal, as no latency), even for very small > messages. Since we can see networking taking multiple CPUs' worth of time > during a stress test, using a busy-spin for ~100micros after last message > receipt is almost certainly acceptable, especially as we can (ultimately) > process inter-server and client communications on the same thread/socket in > this model. > 5) We can optimise the threading model heavily: since we generally process > very small messages (200 bytes not at all implausible), the thread signalling > costs on the processing thread can actually dramatically impede throughput. > In general it costs ~10micros to signal (and passing the message to another > thread for processing in the current model requires signalling). For 200-byte > messages this caps our throughput at 20MB/s. > I propose to knock up a highly naive UDP-based connection protocol with > super-trivial congestion control over the course of a few days, with the only > initial goal being maximum possible performance (not fairness, reliability, > or anything else), and trial it in Netty (possibly making some changes to > Netty to mitigate thread signalling costs). The reason for knocking up our > own here is to get a ceiling on what the absolute limit of potential for this > approach is. Assuming this pans out with performance gains in C* proper, we > then look to contributing to/forking the udt-java project and see how easy it > is to bring performance in line with what we can get with our naive approach > (I don't suggest starting here, as the project is using blocking old-IO, and > modifying it with latency in mind may be challenging, and we won't know for > sure what the best case scenario is). > \[1\] > http://test-docdb.fnal.gov/0016/001648/002/Potential%20Performance%20Bottleneck%20in%20Linux%20TCP.PDF > \[2\] > http://cd-docdb.fnal.gov/cgi-bin/RetrieveFile?docid=1968;filename=Performance%20Analysis%20of%20Linux%20Networking%20-%20Packet%20Receiving%20(Official).pdf;version=2 > Further related reading: > http://public.dhe.ibm.com/software/commerce/doc/mft/cdunix/41/UDTWhitepaper.pdf > https://mospace.umsystem.edu/xmlui/bitstream/handle/10355/14482/ChoiUndPerTcp.pdf?sequence=1 > https://access.redhat.com/site/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.153.3762&rep=rep1&type=pdf -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-5019) Still too much object allocation on reads
[ https://issues.apache.org/jira/browse/CASSANDRA-5019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict resolved CASSANDRA-5019. - Resolution: Duplicate Since the read path will be rewritten as part of efforts to introduce CASSANDRA-7447 (both regards internal APIs, and the implementation details for the new format), this ticket should be addressed by "doing things right" here. This may mean the legacy format continues to be somewhat inefficient, but this may or may not eventually be retired entirely, so there is probably not much point spending a lot of time optimising it, esp. when the impact is unknown and probably not dramatic in relation to the other costs associated with this format. > Still too much object allocation on reads > - > > Key: CASSANDRA-5019 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5019 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Jonathan Ellis > Labels: performance > Fix For: 3.0 > > > ArrayBackedSortedColumns was a step in the right direction but it's still > relatively heavyweight thanks to allocating individual Columns. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7029) Investigate alternative transport protocols for both client and inter-server communications
[ https://issues.apache.org/jira/browse/CASSANDRA-7029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096817#comment-14096817 ] Benedict commented on CASSANDRA-7029: - mTCP is not stable enough, nor universal enough, to be useful to us. It requires very specific linux kernel versions, and very specific network interfaces, in order to work. If it matures it will be worth revisiting. > Investigate alternative transport protocols for both client and inter-server > communications > --- > > Key: CASSANDRA-7029 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7029 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > There are a number of reasons to think we can do better than TCP for our > communications: > 1) We can actually tolerate sporadic small message losses, so guaranteed > delivery isn't essential (although for larger messages it probably is) > 2) As shown in \[1\] and \[2\], Linux can behave quite suboptimally with > regard to TCP message delivery when the system is under load. Judging from > the theoretical description, this is likely to apply even when the > system-load is not high, but the number of processes to schedule is high. > Cassandra generally has a lot of threads to schedule, so this is quite > pertinent for us. UDP performs substantially better here. > 3) Even when the system is not under load, UDP has a lower CPU burden, and > that burden is constant regardless of the number of connections it processes. > 4) On a simple benchmark on my local PC, using non-blocking IO for UDP and > busy spinning on IO I can actually push 20-40% more throughput through > loopback (where TCP should be optimal, as no latency), even for very small > messages. Since we can see networking taking multiple CPUs' worth of time > during a stress test, using a busy-spin for ~100micros after last message > receipt is almost certainly acceptable, especially as we can (ultimately) > process inter-server and client communications on the same thread/socket in > this model. > 5) We can optimise the threading model heavily: since we generally process > very small messages (200 bytes not at all implausible), the thread signalling > costs on the processing thread can actually dramatically impede throughput. > In general it costs ~10micros to signal (and passing the message to another > thread for processing in the current model requires signalling). For 200-byte > messages this caps our throughput at 20MB/s. > I propose to knock up a highly naive UDP-based connection protocol with > super-trivial congestion control over the course of a few days, with the only > initial goal being maximum possible performance (not fairness, reliability, > or anything else), and trial it in Netty (possibly making some changes to > Netty to mitigate thread signalling costs). The reason for knocking up our > own here is to get a ceiling on what the absolute limit of potential for this > approach is. Assuming this pans out with performance gains in C* proper, we > then look to contributing to/forking the udt-java project and see how easy it > is to bring performance in line with what we can get with our naive approach > (I don't suggest starting here, as the project is using blocking old-IO, and > modifying it with latency in mind may be challenging, and we won't know for > sure what the best case scenario is). > \[1\] > http://test-docdb.fnal.gov/0016/001648/002/Potential%20Performance%20Bottleneck%20in%20Linux%20TCP.PDF > \[2\] > http://cd-docdb.fnal.gov/cgi-bin/RetrieveFile?docid=1968;filename=Performance%20Analysis%20of%20Linux%20Networking%20-%20Packet%20Receiving%20(Official).pdf;version=2 > Further related reading: > http://public.dhe.ibm.com/software/commerce/doc/mft/cdunix/41/UDTWhitepaper.pdf > https://mospace.umsystem.edu/xmlui/bitstream/handle/10355/14482/ChoiUndPerTcp.pdf?sequence=1 > https://access.redhat.com/site/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.153.3762&rep=rep1&type=pdf -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7029) Investigate alternative transport protocols for both client and inter-server communications
[ https://issues.apache.org/jira/browse/CASSANDRA-7029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096823#comment-14096823 ] Robert Stupp commented on CASSANDRA-7029: - Damn - didn't see that mTCP has so strict requirements/constraints. > Investigate alternative transport protocols for both client and inter-server > communications > --- > > Key: CASSANDRA-7029 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7029 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict > Labels: performance > Fix For: 3.0 > > > There are a number of reasons to think we can do better than TCP for our > communications: > 1) We can actually tolerate sporadic small message losses, so guaranteed > delivery isn't essential (although for larger messages it probably is) > 2) As shown in \[1\] and \[2\], Linux can behave quite suboptimally with > regard to TCP message delivery when the system is under load. Judging from > the theoretical description, this is likely to apply even when the > system-load is not high, but the number of processes to schedule is high. > Cassandra generally has a lot of threads to schedule, so this is quite > pertinent for us. UDP performs substantially better here. > 3) Even when the system is not under load, UDP has a lower CPU burden, and > that burden is constant regardless of the number of connections it processes. > 4) On a simple benchmark on my local PC, using non-blocking IO for UDP and > busy spinning on IO I can actually push 20-40% more throughput through > loopback (where TCP should be optimal, as no latency), even for very small > messages. Since we can see networking taking multiple CPUs' worth of time > during a stress test, using a busy-spin for ~100micros after last message > receipt is almost certainly acceptable, especially as we can (ultimately) > process inter-server and client communications on the same thread/socket in > this model. > 5) We can optimise the threading model heavily: since we generally process > very small messages (200 bytes not at all implausible), the thread signalling > costs on the processing thread can actually dramatically impede throughput. > In general it costs ~10micros to signal (and passing the message to another > thread for processing in the current model requires signalling). For 200-byte > messages this caps our throughput at 20MB/s. > I propose to knock up a highly naive UDP-based connection protocol with > super-trivial congestion control over the course of a few days, with the only > initial goal being maximum possible performance (not fairness, reliability, > or anything else), and trial it in Netty (possibly making some changes to > Netty to mitigate thread signalling costs). The reason for knocking up our > own here is to get a ceiling on what the absolute limit of potential for this > approach is. Assuming this pans out with performance gains in C* proper, we > then look to contributing to/forking the udt-java project and see how easy it > is to bring performance in line with what we can get with our naive approach > (I don't suggest starting here, as the project is using blocking old-IO, and > modifying it with latency in mind may be challenging, and we won't know for > sure what the best case scenario is). > \[1\] > http://test-docdb.fnal.gov/0016/001648/002/Potential%20Performance%20Bottleneck%20in%20Linux%20TCP.PDF > \[2\] > http://cd-docdb.fnal.gov/cgi-bin/RetrieveFile?docid=1968;filename=Performance%20Analysis%20of%20Linux%20Networking%20-%20Packet%20Receiving%20(Official).pdf;version=2 > Further related reading: > http://public.dhe.ibm.com/software/commerce/doc/mft/cdunix/41/UDTWhitepaper.pdf > https://mospace.umsystem.edu/xmlui/bitstream/handle/10355/14482/ChoiUndPerTcp.pdf?sequence=1 > https://access.redhat.com/site/documentation/en-US/JBoss_Enterprise_Web_Platform/5/html/Administration_And_Configuration_Guide/jgroups-perf-udpbuffer.html > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.153.3762&rep=rep1&type=pdf -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7710) nodetool ring throws exception if run on machine without Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-7710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096824#comment-14096824 ] Jimmy Mårdell commented on CASSANDRA-7710: -- Thanks, yes, that's the one I was looking for! > nodetool ring throws exception if run on machine without Cassandra > -- > > Key: CASSANDRA-7710 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7710 > Project: Cassandra > Issue Type: Bug > Components: Tools >Reporter: Jimmy Mårdell >Assignee: Michael Shuler >Priority: Minor > > DatabaseDescriptor.getNumTokens() is invoked in the nodetool ring command > which doesn't work so well when running on a machine where Cassandra doesnt' > exist. And it has all kind of side effects as well. > This seems fixed in 2.1 but would be nice if it was fixed in 2.0 as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7763) cql_tests static_with_empty_clustering test failure
[
https://issues.apache.org/jira/browse/CASSANDRA-7763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7763:
Attachment: 7763.txt
That's not related to CASSANDRA-7455, it's just that the test was introduced
there and exposed the bug. This is actually a regression from CASSANDRA-6934
which mistakenly removed the static case from {{AbstractCType.compare}}. This
was actually indicated in the comments from CASSANDRA-6934 so this is my bad
for not catching it. We do need to special case static since a static cell
(which has empty components for the clustering columns) should still sort
before a non static cell with empty components, which is exactly what this test
is testing.
> cql_tests static_with_empty_clustering test failure
> ---
>
> Key: CASSANDRA-7763
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7763
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Sylvain Lebresne
> Fix For: 2.1 rc6
>
> Attachments: 7763.txt
>
>
> {code}
> ==
> FAIL: static_with_empty_clustering_test (cql_tests.TestCQL)
> --
> Traceback (most recent call last):
> File "/home/ryan/git/datastax/cassandra-dtest/tools.py", line 213, in
> wrapped
> f(obj)
> File "/home/ryan/git/datastax/cassandra-dtest/cql_tests.py", line 4082, in
> static_with_empty_clustering_test
> assert_one(cursor, "SELECT * FROM test", ['partition1', '', 'static
> value', 'value'])
> File "/home/ryan/git/datastax/cassandra-dtest/assertions.py", line 40, in
> assert_one
> assert res == [expected], res
> AssertionError: [[u'partition1', u'', None, None], [u'partition1', u'', None,
> None], [u'partition1', u'', None, u'value']]
> >> begin captured logging <<
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-Ex54V7
> - >> end captured logging << -
> --
> Ran 1 test in 6.866s
> FAILED (failures=1)
> {code}
> regression from CASSANDRA-7455?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (CASSANDRA-7188) Wrong class type: class org.apache.cassandra.db.Column in CounterColumn.reconcile
[
https://issues.apache.org/jira/browse/CASSANDRA-7188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis reassigned CASSANDRA-7188:
-
Assignee: Michael Shuler (was: Albert P Tobey)
> Wrong class type: class org.apache.cassandra.db.Column in
> CounterColumn.reconcile
> -
>
> Key: CASSANDRA-7188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7188
> Project: Cassandra
> Issue Type: Bug
>Reporter: Nicolas Lalevée
>Assignee: Michael Shuler
>
> When migrating a cluster of 6 nodes from 1.2.11 to 2.0.7, we started to see
> on the first migrated node this error:
> {noformat}
> ERROR [ReplicateOnWriteStage:1] 2014-05-07 11:26:59,779 CassandraDaemon.java
> (line 198) Exception in thread Thread[ReplicateOnWriteStage:1,5,main]
> java.lang.AssertionError: Wrong class type: class
> org.apache.cassandra.db.Column
> at
> org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:159)
> at
> org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:109)
> at
> org.apache.cassandra.db.filter.QueryFilter$1.reduce(QueryFilter.java:103)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:112)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at
> org.apache.cassandra.db.filter.NamesQueryFilter.collectReducedColumns(NamesQueryFilter.java:98)
> at
> org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
> at
> org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
> at
> org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
> at
> org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
> at
> org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1540)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1369)
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:327)
> at
> org.apache.cassandra.db.SliceByNamesReadCommand.getRow(SliceByNamesReadCommand.java:55)
> at
> org.apache.cassandra.db.CounterMutation.makeReplicationMutation(CounterMutation.java:100)
> at
> org.apache.cassandra.service.StorageProxy$8$1.runMayThrow(StorageProxy.java:1085)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1916)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {noformat}
> We then saw on the other 5 nodes, still on 1.2.x, this error:
> {noformat}
> ERROR [MutationStage:2793] 2014-05-07 11:46:12,301 CassandraDaemon.java (line
> 191) Exception in thread Thread[MutationStage:2793,5,main]
> java.lang.AssertionError: Wrong class type: class
> org.apache.cassandra.db.Column
> at
> org.apache.cassandra.db.CounterColumn.reconcile(CounterColumn.java:165)
> at
> org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:378)
> at
> org.apache.cassandra.db.AtomicSortedColumns.addColumn(AtomicSortedColumns.java:166)
> at
> org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:119)
> at org.apache.cassandra.db.SuperColumn.addColumn(SuperColumn.java:218)
> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:229)
> at
> org.apache.cassandra.db.ThreadSafeSortedColumns.addColumnInternal(ThreadSafeSortedColumns.java:108)
> at
> org.apache.cassandra.db.ThreadSafeSortedColumns.addAllWithSizeDelta(ThreadSafeSortedColumns.java:138)
> at
> org.apache.cassandra.db.AbstractColumnContainer.addAllWithSizeDelta(AbstractColumnContainer.java:99)
> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:205)
> at org.apache.cassandra.db.Memtable.put(Memtable.java:168)
> at
> org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:742)
> at org.apache.cassandra.db.Table.apply(Table.java:388)
> at org.apache.cassandra.db.Table.apply(Table.java:353)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:280)
> at
> org.apache.cassandra.d
git commit: Skip strict endpoint selection for ranges if RF == nodes
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1.0 ed3694b86 -> e4e56c2fd
Skip strict endpoint selection for ranges if RF == nodes
Patch by Sam Tunnicliffe; reviewed by tjake for CASSANDRA-7765
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4e56c2f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4e56c2f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4e56c2f
Branch: refs/heads/cassandra-2.1.0
Commit: e4e56c2fd08037b4ddc7459f341e2c86c272e803
Parents: ed3694b
Author: beobal
Authored: Wed Aug 13 21:10:52 2014 +0100
Committer: Jake Luciani
Committed: Thu Aug 14 09:51:45 2014 -0400
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2913607..058681a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.0-rc6
+ * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765)
* Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741)
* Add tracing entries about concurrent range requests (CASSANDRA-7599)
* (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/src/java/org/apache/cassandra/dht/RangeStreamer.java
--
diff --git a/src/java/org/apache/cassandra/dht/RangeStreamer.java
b/src/java/org/apache/cassandra/dht/RangeStreamer.java
index c5d45f9..d84a951 100644
--- a/src/java/org/apache/cassandra/dht/RangeStreamer.java
+++ b/src/java/org/apache/cassandra/dht/RangeStreamer.java
@@ -128,7 +128,7 @@ public class RangeStreamer
public void addRanges(String keyspaceName, Collection> ranges)
{
-Multimap, InetAddress> rangesForKeyspace =
!DatabaseDescriptor.isReplacing() && useStrictConsistency && tokens != null
+Multimap, InetAddress> rangesForKeyspace =
useStrictSourcesForRanges(keyspaceName)
? getAllRangesWithStrictSourcesFor(keyspaceName, ranges) :
getAllRangesWithSourcesFor(keyspaceName, ranges);
if (logger.isDebugEnabled())
@@ -148,6 +148,15 @@ public class RangeStreamer
}
}
+private boolean useStrictSourcesForRanges(String keyspaceName)
+{
+AbstractReplicationStrategy strat =
Keyspace.open(keyspaceName).getReplicationStrategy();
+return !DatabaseDescriptor.isReplacing()
+&& useStrictConsistency
+&& tokens != null
+&& metadata.getAllEndpoints().size() !=
strat.getReplicationFactor();
+}
+
/**
* Get a map of all ranges and their respective sources that are
candidates for streaming the given ranges
* to us. For each range, the list of sources is sorted by proximity
relative to the given destAddress.
[2/3] git commit: Skip strict endpoint selection for ranges if RF == nodes
Skip strict endpoint selection for ranges if RF == nodes
Patch by Sam Tunnicliffe; reviewed by tjake for CASSANDRA-7765
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4e56c2f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4e56c2f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4e56c2f
Branch: refs/heads/cassandra-2.1
Commit: e4e56c2fd08037b4ddc7459f341e2c86c272e803
Parents: ed3694b
Author: beobal
Authored: Wed Aug 13 21:10:52 2014 +0100
Committer: Jake Luciani
Committed: Thu Aug 14 09:51:45 2014 -0400
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2913607..058681a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.0-rc6
+ * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765)
* Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741)
* Add tracing entries about concurrent range requests (CASSANDRA-7599)
* (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/src/java/org/apache/cassandra/dht/RangeStreamer.java
--
diff --git a/src/java/org/apache/cassandra/dht/RangeStreamer.java
b/src/java/org/apache/cassandra/dht/RangeStreamer.java
index c5d45f9..d84a951 100644
--- a/src/java/org/apache/cassandra/dht/RangeStreamer.java
+++ b/src/java/org/apache/cassandra/dht/RangeStreamer.java
@@ -128,7 +128,7 @@ public class RangeStreamer
public void addRanges(String keyspaceName, Collection> ranges)
{
-Multimap, InetAddress> rangesForKeyspace =
!DatabaseDescriptor.isReplacing() && useStrictConsistency && tokens != null
+Multimap, InetAddress> rangesForKeyspace =
useStrictSourcesForRanges(keyspaceName)
? getAllRangesWithStrictSourcesFor(keyspaceName, ranges) :
getAllRangesWithSourcesFor(keyspaceName, ranges);
if (logger.isDebugEnabled())
@@ -148,6 +148,15 @@ public class RangeStreamer
}
}
+private boolean useStrictSourcesForRanges(String keyspaceName)
+{
+AbstractReplicationStrategy strat =
Keyspace.open(keyspaceName).getReplicationStrategy();
+return !DatabaseDescriptor.isReplacing()
+&& useStrictConsistency
+&& tokens != null
+&& metadata.getAllEndpoints().size() !=
strat.getReplicationFactor();
+}
+
/**
* Get a map of all ranges and their respective sources that are
candidates for streaming the given ranges
* to us. For each range, the list of sources is sorted by proximity
relative to the given destAddress.
[1/3] git commit: export MALLOC_ARENA_MAX corrrectly
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 74e7f28f7 -> 1040b520d export MALLOC_ARENA_MAX corrrectly Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-6126 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed3694b8 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed3694b8 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed3694b8 Branch: refs/heads/cassandra-2.1 Commit: ed3694b868cdfbf674445f878477d9f7589ba86e Parents: 0623ea8 Author: Brandon Williams Authored: Wed Aug 13 13:28:28 2014 -0500 Committer: Brandon Williams Committed: Wed Aug 13 13:29:42 2014 -0500 -- conf/cassandra-env.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed3694b8/conf/cassandra-env.sh -- diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh index 211b81b..43a121c 100644 --- a/conf/cassandra-env.sh +++ b/conf/cassandra-env.sh @@ -144,7 +144,7 @@ esac #HEAP_NEWSIZE="800M" # Set this to control the amount of arenas per-thread in glibc -#MALLOC_ARENA_MAX=4 +#export MALLOC_ARENA_MAX=4 if [ "x$MAX_HEAP_SIZE" = "x" ] && [ "x$HEAP_NEWSIZE" = "x" ]; then calculate_heap_sizes @@ -157,7 +157,7 @@ fi if [ "x$MALLOC_ARENA_MAX" = "x" ] then -MALLOC_ARENA_MAX=4 +export MALLOC_ARENA_MAX=4 fi # Specifies the default port over which Cassandra will be available for
[3/3] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1
Merge branch 'cassandra-2.1.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1040b520 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1040b520 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1040b520 Branch: refs/heads/cassandra-2.1 Commit: 1040b520d116abb089acfa24d30aba8556f91f0e Parents: 74e7f28 e4e56c2 Author: Jake Luciani Authored: Thu Aug 14 09:53:38 2014 -0400 Committer: Jake Luciani Committed: Thu Aug 14 09:53:38 2014 -0400 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++- 2 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/1040b520/CHANGES.txt -- diff --cc CHANGES.txt index f34cec7,058681a..cf2f34a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,27 -1,5 +1,28 @@@ +2.1.1 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569) + * SSTableExport uses correct validator to create string representation of partition + keys (CASSANDRA-7498) + * Avoid NPEs when receiving type changes for an unknown keyspace (CASSANDRA-7689) + * Add support for custom 2i validation (CASSANDRA-7575) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) +Merged from 2.0: + * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Catch errors when the JVM pulls the rug out from GCInspector (CASSANDRA-5345) + * cqlsh fails when version number parts are not int (CASSANDRA-7524) + + 2.1.0-rc6 + * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765) * Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741) * Add tracing entries about concurrent range requests (CASSANDRA-7599) * (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729)
[1/4] git commit: export MALLOC_ARENA_MAX corrrectly
Repository: cassandra Updated Branches: refs/heads/trunk 048f72bd3 -> 4304b3a6d export MALLOC_ARENA_MAX corrrectly Patch by brandonwilliams, reviewed by jbellis for CASSANDRA-6126 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed3694b8 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed3694b8 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed3694b8 Branch: refs/heads/trunk Commit: ed3694b868cdfbf674445f878477d9f7589ba86e Parents: 0623ea8 Author: Brandon Williams Authored: Wed Aug 13 13:28:28 2014 -0500 Committer: Brandon Williams Committed: Wed Aug 13 13:29:42 2014 -0500 -- conf/cassandra-env.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed3694b8/conf/cassandra-env.sh -- diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh index 211b81b..43a121c 100644 --- a/conf/cassandra-env.sh +++ b/conf/cassandra-env.sh @@ -144,7 +144,7 @@ esac #HEAP_NEWSIZE="800M" # Set this to control the amount of arenas per-thread in glibc -#MALLOC_ARENA_MAX=4 +#export MALLOC_ARENA_MAX=4 if [ "x$MAX_HEAP_SIZE" = "x" ] && [ "x$HEAP_NEWSIZE" = "x" ]; then calculate_heap_sizes @@ -157,7 +157,7 @@ fi if [ "x$MALLOC_ARENA_MAX" = "x" ] then -MALLOC_ARENA_MAX=4 +export MALLOC_ARENA_MAX=4 fi # Specifies the default port over which Cassandra will be available for
[4/4] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4304b3a6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4304b3a6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4304b3a6 Branch: refs/heads/trunk Commit: 4304b3a6df8e10e7abd998cd79abe330607d38c2 Parents: 048f72b 1040b52 Author: Jake Luciani Authored: Thu Aug 14 09:54:22 2014 -0400 Committer: Jake Luciani Committed: Thu Aug 14 09:54:22 2014 -0400 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++- 2 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4304b3a6/CHANGES.txt --
[3/4] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1
Merge branch 'cassandra-2.1.0' into cassandra-2.1 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1040b520 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1040b520 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1040b520 Branch: refs/heads/trunk Commit: 1040b520d116abb089acfa24d30aba8556f91f0e Parents: 74e7f28 e4e56c2 Author: Jake Luciani Authored: Thu Aug 14 09:53:38 2014 -0400 Committer: Jake Luciani Committed: Thu Aug 14 09:53:38 2014 -0400 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++- 2 files changed, 11 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/1040b520/CHANGES.txt -- diff --cc CHANGES.txt index f34cec7,058681a..cf2f34a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,27 -1,5 +1,28 @@@ +2.1.1 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569) + * SSTableExport uses correct validator to create string representation of partition + keys (CASSANDRA-7498) + * Avoid NPEs when receiving type changes for an unknown keyspace (CASSANDRA-7689) + * Add support for custom 2i validation (CASSANDRA-7575) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) +Merged from 2.0: + * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Catch errors when the JVM pulls the rug out from GCInspector (CASSANDRA-5345) + * cqlsh fails when version number parts are not int (CASSANDRA-7524) + + 2.1.0-rc6 + * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765) * Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741) * Add tracing entries about concurrent range requests (CASSANDRA-7599) * (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729)
[2/4] git commit: Skip strict endpoint selection for ranges if RF == nodes
Skip strict endpoint selection for ranges if RF == nodes
Patch by Sam Tunnicliffe; reviewed by tjake for CASSANDRA-7765
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e4e56c2f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e4e56c2f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e4e56c2f
Branch: refs/heads/trunk
Commit: e4e56c2fd08037b4ddc7459f341e2c86c272e803
Parents: ed3694b
Author: beobal
Authored: Wed Aug 13 21:10:52 2014 +0100
Committer: Jake Luciani
Committed: Thu Aug 14 09:51:45 2014 -0400
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/dht/RangeStreamer.java | 11 ++-
2 files changed, 11 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2913607..058681a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.0-rc6
+ * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765)
* Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741)
* Add tracing entries about concurrent range requests (CASSANDRA-7599)
* (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e4e56c2f/src/java/org/apache/cassandra/dht/RangeStreamer.java
--
diff --git a/src/java/org/apache/cassandra/dht/RangeStreamer.java
b/src/java/org/apache/cassandra/dht/RangeStreamer.java
index c5d45f9..d84a951 100644
--- a/src/java/org/apache/cassandra/dht/RangeStreamer.java
+++ b/src/java/org/apache/cassandra/dht/RangeStreamer.java
@@ -128,7 +128,7 @@ public class RangeStreamer
public void addRanges(String keyspaceName, Collection> ranges)
{
-Multimap, InetAddress> rangesForKeyspace =
!DatabaseDescriptor.isReplacing() && useStrictConsistency && tokens != null
+Multimap, InetAddress> rangesForKeyspace =
useStrictSourcesForRanges(keyspaceName)
? getAllRangesWithStrictSourcesFor(keyspaceName, ranges) :
getAllRangesWithSourcesFor(keyspaceName, ranges);
if (logger.isDebugEnabled())
@@ -148,6 +148,15 @@ public class RangeStreamer
}
}
+private boolean useStrictSourcesForRanges(String keyspaceName)
+{
+AbstractReplicationStrategy strat =
Keyspace.open(keyspaceName).getReplicationStrategy();
+return !DatabaseDescriptor.isReplacing()
+&& useStrictConsistency
+&& tokens != null
+&& metadata.getAllEndpoints().size() !=
strat.getReplicationFactor();
+}
+
/**
* Get a map of all ranges and their respective sources that are
candidates for streaming the given ranges
* to us. For each range, the list of sources is sorted by proximity
relative to the given destAddress.
buildbot success in ASF Buildbot on cassandra-2.1
The Buildbot has detected a restored build on builder cassandra-2.1 while building cassandra. Full details are available at: http://ci.apache.org/builders/cassandra-2.1/builds/393 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: lares_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra-2.1] 1040b520d116abb089acfa24d30aba8556f91f0e Blamelist: Brandon Williams ,Jake Luciani ,beobal Build succeeded! sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-5091) Provide a way to sort read results on the coordinator node
[ https://issues.apache.org/jira/browse/CASSANDRA-5091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096988#comment-14096988 ] Sylvain Lebresne commented on CASSANDRA-5091: - I'll note that ordering on coordinator and query paging don't play very well at all together so I'm not sure how much we can do here. > Provide a way to sort read results on the coordinator node > -- > > Key: CASSANDRA-5091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5091 > Project: Cassandra > Issue Type: Wish >Reporter: Sergio Bossa > Labels: cql > > While Cassandra provides a way to scatter/gather queries with secondary > indexes, as well as custom secondary indexes implementations, there is > currently no way to sort gathered results on the coordinator node, which > would be very useful (actually, essential) if the index implementation > provides results in sorted order. > So, what I'm proposing here is a way to provide (custom) sort > implementations, which may either be as easy as just adding one more > interface and a bunch of method calls, or get a bit more complex by providing > an ORDER BY clause in CQL. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7767) Expose sizes of off-heap data structures via JMX and `nodetool info`
J.B. Langston created CASSANDRA-7767: Summary: Expose sizes of off-heap data structures via JMX and `nodetool info` Key: CASSANDRA-7767 URL: https://issues.apache.org/jira/browse/CASSANDRA-7767 Project: Cassandra Issue Type: New Feature Reporter: J.B. Langston It would be very helpful for troubleshooting memory consumption to know the individual sizes of off-heap data structures such as bloom filters, index summaries, compression metadata, etc. Can we expose this over JMX? Also, since `nodetool info` already shows size of heap, key cache, etc. it seems like a natural place to show this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7386) JBOD threshold to prevent unbalanced disk utilization
[ https://issues.apache.org/jira/browse/CASSANDRA-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lyuben Todorov updated CASSANDRA-7386: -- Attachment: sstable-count-second-run.png mean-writevalue-7disks.png After loading 750GB of data into a 7 disk cluster the test shows what we expected, the drives that had data written to them previously do indeed have lower Write values and these values increase as the rest of the drives begin filling up (view graph). I also tracked sstable creation and compaction. To sum it up, on the first run only 2 drives were used, on the second run another five were added. At the end of the first run d1 and d2 (disk1 and disk2 respectively) were saturated with sstables, d1 had 357 and d2 318. When the second run was started another ks, second_run, was created and an additional 5 disks were used in the node. A majority of sstables were sent to the five new directories as expected (view piechart). The second test to see if busy disks are used for new tables is coming up. > JBOD threshold to prevent unbalanced disk utilization > - > > Key: CASSANDRA-7386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7386 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Chris Lohfink >Assignee: Lyuben Todorov >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7386-v1.patch, 7386v2.diff, Mappe1.ods, > mean-writevalue-7disks.png, patch_2_1_branch_proto.diff, > sstable-count-second-run.png > > > Currently the pick the disks are picked first by number of current tasks, > then by free space. This helps with performance but can lead to large > differences in utilization in some (unlikely but possible) scenarios. Ive > seen 55% to 10% and heard reports of 90% to 10% on IRC. With both LCS and > STCS (although my suspicion is that STCS makes it worse since harder to be > balanced). > I purpose the algorithm change a little to have some maximum range of > utilization where it will pick by free space over load (acknowledging it can > be slower). So if a disk A is 30% full and disk B is 5% full it will never > pick A over B until it balances out. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7386) JBOD threshold to prevent unbalanced disk utilization
[ https://issues.apache.org/jira/browse/CASSANDRA-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097014#comment-14097014 ] Lyuben Todorov edited comment on CASSANDRA-7386 at 8/14/14 2:41 PM: After loading 750GB of data into a 7 disk cluster the test shows what we expected, the drives that had data written to them previously do indeed have lower Write values and these values increase as the rest of the drives begin filling up (view graph). !https://issues.apache.org/jira/secure/attachment/12661718/mean-writevalue-7disks.png! I also tracked sstable creation and compaction. To sum it up, on the first run only 2 drives were used, on the second run another five were added. At the end of the first run d1 and d2 (disk1 and disk2 respectively) were saturated with sstables, d1 had 357 and d2 318. When the second run was started another ks, second_run, was created and an additional 5 disks were used in the node. A majority of sstables were sent to the five new directories as expected (view piechart). !https://issues.apache.org/jira/secure/attachment/12661719/sstable-count-second-run.png! The second test to see if busy disks are used for new tables is coming up. was (Author: lyubent): After loading 750GB of data into a 7 disk cluster the test shows what we expected, the drives that had data written to them previously do indeed have lower Write values and these values increase as the rest of the drives begin filling up (view graph). I also tracked sstable creation and compaction. To sum it up, on the first run only 2 drives were used, on the second run another five were added. At the end of the first run d1 and d2 (disk1 and disk2 respectively) were saturated with sstables, d1 had 357 and d2 318. When the second run was started another ks, second_run, was created and an additional 5 disks were used in the node. A majority of sstables were sent to the five new directories as expected (view piechart). The second test to see if busy disks are used for new tables is coming up. > JBOD threshold to prevent unbalanced disk utilization > - > > Key: CASSANDRA-7386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7386 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Chris Lohfink >Assignee: Lyuben Todorov >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7386-v1.patch, 7386v2.diff, Mappe1.ods, > mean-writevalue-7disks.png, patch_2_1_branch_proto.diff, > sstable-count-second-run.png > > > Currently the pick the disks are picked first by number of current tasks, > then by free space. This helps with performance but can lead to large > differences in utilization in some (unlikely but possible) scenarios. Ive > seen 55% to 10% and heard reports of 90% to 10% on IRC. With both LCS and > STCS (although my suspicion is that STCS makes it worse since harder to be > balanced). > I purpose the algorithm change a little to have some maximum range of > utilization where it will pick by free space over load (acknowledging it can > be slower). So if a disk A is 30% full and disk B is 5% full it will never > pick A over B until it balances out. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-7304: Labels: cql protocolv4 (was: cql) > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Improvement >Reporter: Drew Kutcharian > Labels: cql, protocolv4 > Attachments: 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7759) DROP TABLE makes C* unreachable
[
https://issues.apache.org/jira/browse/CASSANDRA-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097049#comment-14097049
]
Fabrice Larcher commented on CASSANDRA-7759:
After a {{DROP TABLE my_old_table;}}, I experienced again the issue. Then I was
unable to kill C*2 (still running) using {{kill }}. I forced it with
{{kill -9 }}. Then, trying to start C* 2.1.0-rc5 again, I get the
following NPE in the console output :
{code}
ERROR 15:02:52 Exception encountered during startup
java.lang.NullPointerException: null
at org.apache.cassandra.db.Directories.(Directories.java:191)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:553)
~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:245)
[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:455)
[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:544)
[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
java.lang.NullPointerException
at org.apache.cassandra.db.Directories.(Directories.java:191)
at
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:553)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:245)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:455)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:544)
Exception encountered during startup: null
{code}
I do not know exactly how recover and I guest that it is related with this
issue.
> DROP TABLE makes C* unreachable
> ---
>
> Key: CASSANDRA-7759
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7759
> Project: Cassandra
> Issue Type: Bug
> Environment: CQLSH of C* 2.10.-rc5 with one or two node(s).
>Reporter: Fabrice Larcher
>
> After a {{DROP KEYSPACE}} or {{DROP TABLE}} command from _CQLSH_ I often get
> the output :
> {code}
> errors={}, last_host=127.0.0.1
> {code}
> Then, the application can not access the node anymore (it uses the Java
> Driver 2.1.0-rc1). And I get the following stacktraces from the
> {{system.log}} file ;
> {code}
> ERROR [MemtableFlushWriter:5] 2014-08-13 11:26:07,577
> CassandraDaemon.java:166 - Exception in thread
> Thread[MemtableFlushWriter:5,5,main]
> java.lang.RuntimeException: Last written key
> DecoratedKey(28149890686391545-8456361251720325,
> 30313034353666392d666464362d343539362d393537372d653539336430333138396437) >=
> current key DecoratedKey(108b5f3f-fc06-4a0d-99f1-3c6484b32e04,
> 31303862356633662d666330362d346130642d393966312d336336343834623332653034)
> writing into
> ./../data/data/metrics/run-a3c1fe80216911e49fd1ab9c3338a2ff/metrics-run.run_probe-tmp-ka-2-Data.db
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
> ~[guava-16.0.jar:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ~[na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]
> {code}
> I also get the following (perhaps not related)
> {code}
> ERROR [MemtableFlushWriter:6] 2014-08-13 11:26:08,827
> CassandraDaemon.java:166 - Exception in thread
> Thread[MemtableFlushWriter:6,5,main]
> org.apache.cassandra.serializers.MarshalException: Invalid byte for ascii: -50
> at
> org.apache.cassandra.serializers.AsciiSerializer.validate(AsciiSerializer.java:39)
>
[jira] [Created] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
Paul Pak created CASSANDRA-7768: --- Summary: Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace Key: CASSANDRA-7768 URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 Project: Cassandra Issue Type: Bug Components: Core Reporter: Paul Pak Assignee: Paul Pak Reason why this occurs is because Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Priority: Minor (was: Major) > Error when creating multiple CQLSSTableWriters for more than one column > family in the same keyspace > --- > > Key: CASSANDRA-7768 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 > Project: Cassandra > Issue Type: Bug >Reporter: Paul Pak >Assignee: Paul Pak >Priority: Minor > > Reason why this occurs is because Schema.load(CFMetaData) only adds to the > Schema.cfIdMap without making the proper addition to the CFMetaData map > belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Component/s: (was: Core) > Error when creating multiple CQLSSTableWriters for more than one column > family in the same keyspace > --- > > Key: CASSANDRA-7768 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 > Project: Cassandra > Issue Type: Bug >Reporter: Paul Pak >Assignee: Paul Pak > > Reason why this occurs is because Schema.load(CFMetaData) only adds to the > Schema.cfIdMap without making the proper addition to the CFMetaData map > belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Description: The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded), CQLSSTableWriter builder only loades the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. (was: Reason why this occurs is because Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map.) > Error when creating multiple CQLSSTableWriters for more than one column > family in the same keyspace > --- > > Key: CASSANDRA-7768 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 > Project: Cassandra > Issue Type: Bug >Reporter: Paul Pak >Assignee: Paul Pak >Priority: Minor > > The reason why this occurs is if the keyspace has already been loaded (due to > another column family being previously loaded), CQLSSTableWriter builder only > loades the column family via Schema.load(CFMetaData). However, > Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the > proper addition to the CFMetaData map belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Description: The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded in the same keyspace), CQLSSTableWriter builder only loads the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. (was: The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded in the same keyspace), CQLSSTableWriter builder only loades the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map.) > Error when creating multiple CQLSSTableWriters for more than one column > family in the same keyspace > --- > > Key: CASSANDRA-7768 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 > Project: Cassandra > Issue Type: Bug >Reporter: Paul Pak >Assignee: Paul Pak >Priority: Minor > > The reason why this occurs is if the keyspace has already been loaded (due to > another column family being previously loaded in the same keyspace), > CQLSSTableWriter builder only loads the column family via > Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the > Schema.cfIdMap without making the proper addition to the CFMetaData map > belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Description: The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded in the same keyspace), CQLSSTableWriter builder only loades the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map. (was: The reason why this occurs is if the keyspace has already been loaded (due to another column family being previously loaded), CQLSSTableWriter builder only loades the column family via Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the Schema.cfIdMap without making the proper addition to the CFMetaData map belonging to the KSMetaData map.) > Error when creating multiple CQLSSTableWriters for more than one column > family in the same keyspace > --- > > Key: CASSANDRA-7768 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 > Project: Cassandra > Issue Type: Bug >Reporter: Paul Pak >Assignee: Paul Pak >Priority: Minor > > The reason why this occurs is if the keyspace has already been loaded (due to > another column family being previously loaded in the same keyspace), > CQLSSTableWriter builder only loades the column family via > Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the > Schema.cfIdMap without making the proper addition to the CFMetaData map > belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
[2/3] git commit: Fix compactionstats layout for long ks/cf names
Fix compactionstats layout for long ks/cf names
Patch by Nicolas Lalevée, reviewed by brandonwilliams for CASSANDRA-7263
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4c510dc6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4c510dc6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4c510dc6
Branch: refs/heads/trunk
Commit: 4c510dc6cc1ea3fdb21180cfc74ab6c29b78c24f
Parents: 1040b52
Author: Brandon Williams
Authored: Thu Aug 14 10:51:57 2014 -0500
Committer: Brandon Williams
Committed: Thu Aug 14 10:51:57 2014 -0500
--
.../org/apache/cassandra/tools/NodeTool.java| 66 ++--
1 file changed, 48 insertions(+), 18 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4c510dc6/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java
b/src/java/org/apache/cassandra/tools/NodeTool.java
index b25edd9..6aca507 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1129,26 +1129,56 @@ public class NodeTool
int compactionThroughput = probe.getCompactionThroughput();
CompactionManagerMBean cm = probe.getCompactionManagerProxy();
System.out.println("pending tasks: " +
probe.getCompactionMetric("PendingTasks"));
-if (cm.getCompactions().size() > 0)
-System.out.printf("%25s%16s%16s%16s%16s%10s%10s%n",
"compaction type", "keyspace", "table", "completed", "total", "unit",
"progress");
long remainingBytes = 0;
-for (Map c : cm.getCompactions())
-{
-String percentComplete = new Long(c.get("total")) == 0
- ? "n/a"
- : new
DecimalFormat("0.00").format((double) new Long(c.get("completed")) / new
Long(c.get("total")) * 100) + "%";
-System.out.printf("%25s%16s%16s%16s%16s%10s%10s%n",
c.get("taskType"), c.get("keyspace"), c.get("columnfamily"),
c.get("completed"), c.get("total"), c.get("unit"), percentComplete);
-if
(c.get("taskType").equals(OperationType.COMPACTION.toString()))
-remainingBytes += (new Long(c.get("total")) - new
Long(c.get("completed")));
+if (cm.getCompactions().size() > 0)
+{
+List lines = new ArrayList<>();
+int[] columnSizes = new int[] { 0, 0, 0, 0, 0, 0, 0 };
+
+addLine(lines, columnSizes, "compaction type", "keyspace",
"table", "completed", "total", "unit", "progress");
+for (Map c : cm.getCompactions())
+{
+long total = Long.parseLong(c.get("total"));
+long completed = Long.parseLong(c.get("completed"));
+String taskType = c.get("taskType");
+String keyspace = c.get("keyspace");
+String columnFamily = c.get("columnfamily");
+String unit = c.get("unit");
+String percentComplete = total == 0 ? "n/a" : new
DecimalFormat("0.00").format((double) completed / total * 100) + "%";
+addLine(lines, columnSizes, taskType, keyspace,
columnFamily, Long.toString(completed), Long.toString(total), unit,
percentComplete);
+if (taskType.equals(OperationType.COMPACTION.toString()))
+remainingBytes += total - completed;
+}
+
+StringBuilder buffer = new StringBuilder();
+for (int columnSize : columnSizes) {
+buffer.append("%");
+buffer.append(columnSize + 3);
+buffer.append("s");
+}
+buffer.append("%n");
+String format = buffer.toString();
+
+for (String[] line : lines)
+{
+System.out.printf(format, line[0], line[1], line[2],
line[3], line[4], line[5], line[6]);
+}
+
+String remainingTime = "n/a";
+if (compactionThroughput != 0)
+{
+long remainingTimeInSecs = remainingBytes / (1024L * 1024L
* compactionThroughput);
+remainingTime = format("%dh%02dm%02ds",
remainingTimeInSecs / 3600, (remainingTimeInSecs % 3600) / 60,
(remainingTimeInSecs % 60));
+}
+System.out.printf("%25s%10s%n", "Active compaction remaining
time : ", remainingTime);
+}
+}
+
+private void addLine(List lines, int[] columnSizes,
String... columns) {
+
[3/3] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fd43c33c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fd43c33c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fd43c33c Branch: refs/heads/trunk Commit: fd43c33c66ac493bf7cf1f7b96c7c93346b407c3 Parents: 4304b3a 4c510dc Author: Brandon Williams Authored: Thu Aug 14 10:53:00 2014 -0500 Committer: Brandon Williams Committed: Thu Aug 14 10:53:00 2014 -0500 -- .../org/apache/cassandra/tools/NodeTool.java| 66 ++-- 1 file changed, 48 insertions(+), 18 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/fd43c33c/src/java/org/apache/cassandra/tools/NodeTool.java --
[1/3] git commit: Fix compactionstats layout for long ks/cf names
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 1040b520d -> 4c510dc6c
refs/heads/trunk 4304b3a6d -> fd43c33c6
Fix compactionstats layout for long ks/cf names
Patch by Nicolas Lalevée, reviewed by brandonwilliams for CASSANDRA-7263
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4c510dc6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4c510dc6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4c510dc6
Branch: refs/heads/cassandra-2.1
Commit: 4c510dc6cc1ea3fdb21180cfc74ab6c29b78c24f
Parents: 1040b52
Author: Brandon Williams
Authored: Thu Aug 14 10:51:57 2014 -0500
Committer: Brandon Williams
Committed: Thu Aug 14 10:51:57 2014 -0500
--
.../org/apache/cassandra/tools/NodeTool.java| 66 ++--
1 file changed, 48 insertions(+), 18 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4c510dc6/src/java/org/apache/cassandra/tools/NodeTool.java
--
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java
b/src/java/org/apache/cassandra/tools/NodeTool.java
index b25edd9..6aca507 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -1129,26 +1129,56 @@ public class NodeTool
int compactionThroughput = probe.getCompactionThroughput();
CompactionManagerMBean cm = probe.getCompactionManagerProxy();
System.out.println("pending tasks: " +
probe.getCompactionMetric("PendingTasks"));
-if (cm.getCompactions().size() > 0)
-System.out.printf("%25s%16s%16s%16s%16s%10s%10s%n",
"compaction type", "keyspace", "table", "completed", "total", "unit",
"progress");
long remainingBytes = 0;
-for (Map c : cm.getCompactions())
-{
-String percentComplete = new Long(c.get("total")) == 0
- ? "n/a"
- : new
DecimalFormat("0.00").format((double) new Long(c.get("completed")) / new
Long(c.get("total")) * 100) + "%";
-System.out.printf("%25s%16s%16s%16s%16s%10s%10s%n",
c.get("taskType"), c.get("keyspace"), c.get("columnfamily"),
c.get("completed"), c.get("total"), c.get("unit"), percentComplete);
-if
(c.get("taskType").equals(OperationType.COMPACTION.toString()))
-remainingBytes += (new Long(c.get("total")) - new
Long(c.get("completed")));
+if (cm.getCompactions().size() > 0)
+{
+List lines = new ArrayList<>();
+int[] columnSizes = new int[] { 0, 0, 0, 0, 0, 0, 0 };
+
+addLine(lines, columnSizes, "compaction type", "keyspace",
"table", "completed", "total", "unit", "progress");
+for (Map c : cm.getCompactions())
+{
+long total = Long.parseLong(c.get("total"));
+long completed = Long.parseLong(c.get("completed"));
+String taskType = c.get("taskType");
+String keyspace = c.get("keyspace");
+String columnFamily = c.get("columnfamily");
+String unit = c.get("unit");
+String percentComplete = total == 0 ? "n/a" : new
DecimalFormat("0.00").format((double) completed / total * 100) + "%";
+addLine(lines, columnSizes, taskType, keyspace,
columnFamily, Long.toString(completed), Long.toString(total), unit,
percentComplete);
+if (taskType.equals(OperationType.COMPACTION.toString()))
+remainingBytes += total - completed;
+}
+
+StringBuilder buffer = new StringBuilder();
+for (int columnSize : columnSizes) {
+buffer.append("%");
+buffer.append(columnSize + 3);
+buffer.append("s");
+}
+buffer.append("%n");
+String format = buffer.toString();
+
+for (String[] line : lines)
+{
+System.out.printf(format, line[0], line[1], line[2],
line[3], line[4], line[5], line[6]);
+}
+
+String remainingTime = "n/a";
+if (compactionThroughput != 0)
+{
+long remainingTimeInSecs = remainingBytes / (1024L * 1024L
* compactionThroughput);
+remainingTime = format("%dh%02dm%02ds",
remainingTimeInSecs / 3600, (remainingTimeInSecs % 3600) / 60,
(remainingTimeInSecs % 60));
+}
+System.out.printf("%25s%10s%n", "Active compaction remaining
time :
buildbot failure in ASF Buildbot on cassandra-2.1
The Buildbot has detected a new failure on builder cassandra-2.1 while building cassandra. Full details are available at: http://ci.apache.org/builders/cassandra-2.1/builds/394 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: lares_ubuntu Build Reason: scheduler Build Source Stamp: [branch cassandra-2.1] 4c510dc6cc1ea3fdb21180cfc74ab6c29b78c24f Blamelist: Brandon Williams BUILD FAILED: failed sincerely, -The Buildbot
[jira] [Updated] (CASSANDRA-7768) Error when creating multiple CQLSSTableWriters for more than one column family in the same keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-7768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Pak updated CASSANDRA-7768: Attachment: trunk-7768-v1.txt > Error when creating multiple CQLSSTableWriters for more than one column > family in the same keyspace > --- > > Key: CASSANDRA-7768 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7768 > Project: Cassandra > Issue Type: Bug >Reporter: Paul Pak >Assignee: Paul Pak >Priority: Minor > Attachments: trunk-7768-v1.txt > > > The reason why this occurs is if the keyspace has already been loaded (due to > another column family being previously loaded in the same keyspace), > CQLSSTableWriter builder only loads the column family via > Schema.load(CFMetaData). However, Schema.load(CFMetaData) only adds to the > Schema.cfIdMap without making the proper addition to the CFMetaData map > belonging to the KSMetaData map. -- This message was sent by Atlassian JIRA (v6.2#6252)
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build on builder cassandra-trunk while building cassandra. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/650 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: lares_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] fd43c33c66ac493bf7cf1f7b96c7c93346b407c3 Blamelist: Brandon Williams Build succeeded! sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-7585) cassandra sstableloader connection refused with inter_node_encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097146#comment-14097146 ] Yuki Morishita commented on CASSANDRA-7585: --- bq. Could we make it possible to state the server_encryption_options on the command line as well as via the config? The command will look horrible... That's why I added option to load yaml file. I could add 8 more command line options for server encryption but it seems redundant. I'd rather deprecate the current way of setting client encryption options. I think I can highlight this more in help string. Will update with other nits fixed. > cassandra sstableloader connection refused with inter_node_encryption > - > > Key: CASSANDRA-7585 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7585 > Project: Cassandra > Issue Type: Bug > Components: Core, Tools >Reporter: Samphel Norden >Assignee: Yuki Morishita > Fix For: 2.0.10, 2.1.1 > > Attachments: 7585-2.0.txt > > > cassandra sstableloader connection refused with inter_node_encryption > When using sstableloader to import tables (cassandra 2.0.5) with inter-node > encryption and client encryption enabled, I get a connection refused error > I am using > sstableloader -d $myhost -p 9160 -u cassandra -pw cassandra -ciphers > TLS_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA > -st JKS -tf org.apache.cassandra.thrift.SSLTransportFactory -ts > /path/to/truststore -tspw $fullpath/$table > Errors out with > Streaming session ID: 1bc395c0-fbb2-11e3-9812-73da15121373 > WARN 17:13:34,147 Failed attempt 1 to connect to > Similar problem reported in cassandra 2.0.8 by another user > http://stackoverflow.com/questions/24390604/cassandra-sstableloader-connection-refused-with-inter-node-encryption > == > Relevant cassandra.yaml snippet (with obfuscation) > server_encryption_options: > > internode_encryption: all > > keystore:/path/to/keystore > > keystore_password: > > truststore:/path/to/truststore > > truststore_password: > > # More advanced defaults below: > > protocol: TLS > > algorithm: SunX509 > > store_type: JKS > > cipher_suites: > [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA] > > require_client_auth: true > > > > # enable or disable client/server encryption. > > client_encryption_options: > > enabled: true > > keystore: /path/to/keystore > > keystore_password: > > #require_client_auth: true > > # Set trustore and truststore_password if require_client_auth is true > > truststore:/path/to/truststore > > truststore_password: > > # More advanced defaults below: > > protocol: TLS
[jira] [Commented] (CASSANDRA-7286) Exception: NPE
[ https://issues.apache.org/jira/browse/CASSANDRA-7286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097155#comment-14097155 ] Julien Anguenot commented on CASSANDRA-7286: [~mshuler] I actually have a trace with that NPE from one of our staging Cassandra instances. Possible to sent it to you offline? > Exception: NPE > --- > > Key: CASSANDRA-7286 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7286 > Project: Cassandra > Issue Type: Bug >Reporter: Julien Anguenot > Attachments: npe_cassandra_2_0_8.txt, readstage_npe.txt > > > Sometimes Cassandra nodes (in a multi datacenter deployment) are throwing > NPE (see attached stack trace) > Let me know what additional information I could provide. > Thank you. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7767) Expose sizes of off-heap data structures via JMX and `nodetool info`
[ https://issues.apache.org/jira/browse/CASSANDRA-7767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097166#comment-14097166 ] Jeremiah Jordan commented on CASSANDRA-7767: A total size could go in info. The individual sizes should go in cfstats since they are per cf, bloom filter size is already in there. > Expose sizes of off-heap data structures via JMX and `nodetool info` > > > Key: CASSANDRA-7767 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7767 > Project: Cassandra > Issue Type: New Feature >Reporter: J.B. Langston > > It would be very helpful for troubleshooting memory consumption to know the > individual sizes of off-heap data structures such as bloom filters, index > summaries, compression metadata, etc. Can we expose this over JMX? Also, > since `nodetool info` already shows size of heap, key cache, etc. it seems > like a natural place to show this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7748) Get Windows command-line flags in-line with linux
[ https://issues.apache.org/jira/browse/CASSANDRA-7748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-7748: --- Attachment: 7748_v1.txt I'm not in love with -s being 'show' for verbose command-line args but PowerShell isn't case-sensitive so I can't re-use upper-case V or a long version. Patch attached that gets -v in line w/linux and adds -s for show env. > Get Windows command-line flags in-line with linux > - > > Key: CASSANDRA-7748 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7748 > Project: Cassandra > Issue Type: Improvement >Reporter: Joshua McKenzie >Assignee: Joshua McKenzie >Priority: Trivial > Labels: Windows > Fix For: 2.1.1, 3.0 > > Attachments: 7748_v1.txt > > > linux was here first. In the PowerShell launch scripts, -v is verbose right > now as I missed that it's used for 'version' on linux. Add version print > functionality to Windows launch using -v and find another sane flag to use > for verbose env output printing. -- This message was sent by Atlassian JIRA (v6.2#6252)
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure on builder cassandra-trunk while building libcloud. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/651 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: lares_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] 988b1392a7aa061b220318f2fd85f727db2e9f61 Blamelist: Tomaz Muraus BUILD FAILED: failed sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-7759) DROP TABLE makes C* unreachable
[
https://issues.apache.org/jira/browse/CASSANDRA-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097267#comment-14097267
]
T Jake Luciani commented on CASSANDRA-7759:
---
The first error should be fixed by CASSANDRA-7735 which will go into rc6. Once
released please retest once that's out and get back to us
> DROP TABLE makes C* unreachable
> ---
>
> Key: CASSANDRA-7759
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7759
> Project: Cassandra
> Issue Type: Bug
> Environment: CQLSH of C* 2.10.-rc5 with one or two node(s).
>Reporter: Fabrice Larcher
>
> After a {{DROP KEYSPACE}} or {{DROP TABLE}} command from _CQLSH_ I often get
> the output :
> {code}
> errors={}, last_host=127.0.0.1
> {code}
> Then, the application can not access the node anymore (it uses the Java
> Driver 2.1.0-rc1). And I get the following stacktraces from the
> {{system.log}} file ;
> {code}
> ERROR [MemtableFlushWriter:5] 2014-08-13 11:26:07,577
> CassandraDaemon.java:166 - Exception in thread
> Thread[MemtableFlushWriter:5,5,main]
> java.lang.RuntimeException: Last written key
> DecoratedKey(28149890686391545-8456361251720325,
> 30313034353666392d666464362d343539362d393537372d653539336430333138396437) >=
> current key DecoratedKey(108b5f3f-fc06-4a0d-99f1-3c6484b32e04,
> 31303862356633662d666330362d346130642d393966312d336336343834623332653034)
> writing into
> ./../data/data/metrics/run-a3c1fe80216911e49fd1ab9c3338a2ff/metrics-run.run_probe-tmp-ka-2-Data.db
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
> ~[guava-16.0.jar:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ~[na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]
> {code}
> I also get the following (perhaps not related)
> {code}
> ERROR [MemtableFlushWriter:6] 2014-08-13 11:26:08,827
> CassandraDaemon.java:166 - Exception in thread
> Thread[MemtableFlushWriter:6,5,main]
> org.apache.cassandra.serializers.MarshalException: Invalid byte for ascii: -50
> at
> org.apache.cassandra.serializers.AsciiSerializer.validate(AsciiSerializer.java:39)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:78)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at org.apache.cassandra.dht.LocalToken.toString(LocalToken.java:39)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
> at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
> at org.apache.cassandra.db.DecoratedKey.toString(DecoratedKey.java:118)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
> at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunn
[jira] [Resolved] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[
https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp resolved CASSANDRA-7577.
-
Resolution: Not a Problem
According to http://pypi.python.org/pypi/readline:
"Mac OS X, do not ship with GNU readline installed. The readline extension
module in the standard library of Mac "system" Python uses NetBSD's editline
(libedit) library instead, which is a readline replacement with a less
restrictive software license."
So, you can install it with the command:
{noformat}
sudo easy_install readline
{noformat}
(Found on
https://stackoverflow.com/questions/8884001/python-2-7-with-mac-osx-lion-fn-delete-and-control-r)
> cqlsh: CTRL-R history search not working on OSX
> ---
>
> Key: CASSANDRA-7577
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7577
> Project: Cassandra
> Issue Type: Bug
> Environment: OSX - plain Terminal program
> C* 2.0.x, 2.1, trunk
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
>
> _recursive-history-search_ via ctrl-R does not work in cqlsh. The history
> itself works via cursor up/down.
> It works on Linux (and I guess on Windows with cygwin) but not on my Mac.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7286) Exception: NPE
[ https://issues.apache.org/jira/browse/CASSANDRA-7286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097395#comment-14097395 ] Mikhail Stepura commented on CASSANDRA-7286: Looks suspiciously similar to CASSANDRA-7756 or CASSANDRA-7278 > Exception: NPE > --- > > Key: CASSANDRA-7286 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7286 > Project: Cassandra > Issue Type: Bug >Reporter: Julien Anguenot > Attachments: npe_cassandra_2_0_8.txt, readstage_npe.txt > > > Sometimes Cassandra nodes (in a multi datacenter deployment) are throwing > NPE (see attached stack trace) > Let me know what additional information I could provide. > Thank you. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097494#comment-14097494 ] Joshua McKenzie commented on CASSANDRA-7577: I added a warning to cqlsh on Windows if you run without the pyreadline dependency - might be worth considering on OSX as well. > cqlsh: CTRL-R history search not working on OSX > --- > > Key: CASSANDRA-7577 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 > Project: Cassandra > Issue Type: Bug > Environment: OSX - plain Terminal program > C* 2.0.x, 2.1, trunk >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Minor > > _recursive-history-search_ via ctrl-R does not work in cqlsh. The history > itself works via cursor up/down. > It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp reopened CASSANDRA-7577: - > cqlsh: CTRL-R history search not working on OSX > --- > > Key: CASSANDRA-7577 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 > Project: Cassandra > Issue Type: Bug > Environment: OSX - plain Terminal program > C* 2.0.x, 2.1, trunk >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Minor > > _recursive-history-search_ via ctrl-R does not work in cqlsh. The history > itself works via cursor up/down. > It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7561) On DROP we should invalidate CounterKeyCache as well as Key/Row cache
[ https://issues.apache.org/jira/browse/CASSANDRA-7561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-7561: - Attachment: 7561.txt > On DROP we should invalidate CounterKeyCache as well as Key/Row cache > - > > Key: CASSANDRA-7561 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7561 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 2.1.0 > > Attachments: 7561.txt > > > We should also probably ensure we don't attempt to auto save _any_ of the > caches while they are in an inconsistent state (i.e. there are keys present > to be saved that should not be restored, or that would throw exceptions when > we save (e.g. CounterCacheKey)) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-7577: Fix Version/s: 2.1.1 2.0.11 > cqlsh: CTRL-R history search not working on OSX > --- > > Key: CASSANDRA-7577 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 > Project: Cassandra > Issue Type: Bug > Environment: OSX - plain Terminal program > C* 2.0.x, 2.1, trunk >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Minor > Fix For: 2.0.11, 2.1.1 > > Attachments: 7577-2.0.txt, 7577-2.1.txt > > > _recursive-history-search_ via ctrl-R does not work in cqlsh. The history > itself works via cursor up/down. > It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-7577: Attachment: 7577-2.1.txt 7577-2.0.txt [~joshuaMcKenzie] good point. Added patches for 2.0 and 2.1. 2.0 patch also backports the Windows test. > cqlsh: CTRL-R history search not working on OSX > --- > > Key: CASSANDRA-7577 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 > Project: Cassandra > Issue Type: Bug > Environment: OSX - plain Terminal program > C* 2.0.x, 2.1, trunk >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Minor > Fix For: 2.0.11, 2.1.1 > > Attachments: 7577-2.0.txt, 7577-2.1.txt > > > _recursive-history-search_ via ctrl-R does not work in cqlsh. The history > itself works via cursor up/down. > It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7641) cqlsh should automatically disable tracing when selecting from system_traces
[ https://issues.apache.org/jira/browse/CASSANDRA-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Stepura updated CASSANDRA-7641: --- Attachment: CASSANDRA-2.0-7641-v3.patch [~philipthompson] your patch disables the tracing if a current keyspace is "system_traces" even if user selects from a different keyspace. I'm attaching v3 of the patch which address that has slightly simplified logic, please take a look > cqlsh should automatically disable tracing when selecting from system_traces > > > Key: CASSANDRA-7641 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7641 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Brandon Williams >Assignee: Philip Thompson >Priority: Minor > Labels: lhf > Fix For: 2.0.10, 2.1.1 > > Attachments: CASSANDRA-2.0-7641-v3.patch, cassandra-2.0-7641-2.txt, > cassandra-2.0-7641.txt, cassandra-2.1-7641.txt > > > Nobody needs to trace their traces while they're tracing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7577) cqlsh: CTRL-R history search not working on OSX
[ https://issues.apache.org/jira/browse/CASSANDRA-7577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097570#comment-14097570 ] Joshua McKenzie commented on CASSANDRA-7577: Windows portion of the patches lgtm. Good call on backporting it to 2.0 - no reason not to while we're here. > cqlsh: CTRL-R history search not working on OSX > --- > > Key: CASSANDRA-7577 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7577 > Project: Cassandra > Issue Type: Bug > Environment: OSX - plain Terminal program > C* 2.0.x, 2.1, trunk >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Minor > Fix For: 2.0.11, 2.1.1 > > Attachments: 7577-2.0.txt, 7577-2.1.txt > > > _recursive-history-search_ via ctrl-R does not work in cqlsh. The history > itself works via cursor up/down. > It works on Linux (and I guess on Windows with cygwin) but not on my Mac. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7740) Parsing of UDF body is broken
[
https://issues.apache.org/jira/browse/CASSANDRA-7740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-7740:
Attachment: 7740.txt
Patch for first string-style variant.
Will open a separate ticket for pg-style variant.
> Parsing of UDF body is broken
> -
>
> Key: CASSANDRA-7740
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7740
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Robert Stupp
> Fix For: 3.0
>
> Attachments: 7740.txt
>
>
> The parsing of function body introduced by CASSANDRA-7395 is somewhat broken.
> It blindly parse everything up to {{END_BODY}}, which as 2 problems:
> # it parse function body as if it was part of the CQL syntax, so anything
> that don't happen to be a valid CQL token won't even parse.
> # something like
> {noformat}
> CREATE FUNCTION foo() RETURNS text LANGUAGE JAVA BODY return "END_BODY";
> END_BODY;
> {noformat}
> will not parse correctly.
> I don't think we can accept random syntax like that. A better solution (which
> is the one Postgresql uses) is to pass the function body as a normal string.
> And in fact I'd be in favor of reusing Postgresql syntax (because why not),
> that is to have:
> {noformat}
> CREATE FUNCTION foo() RETURNS text LANGUAGE JAVA AS 'return "END_BODY"';
> {noformat}
> One minor annoyance might be, for certain languages, the necessity to double
> every quote inside the string. But in a separate ticket we could introduce
> Postregsql solution of adding an [alternate syntax for string
> constants|http://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html#SQL-SYNTAX-DOLLAR-QUOTING].
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7641) cqlsh should automatically disable tracing when selecting from system_traces
[ https://issues.apache.org/jira/browse/CASSANDRA-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097574#comment-14097574 ] Philip Thompson commented on CASSANDRA-7641: [~mishail], I like your logic better yes. I'll make the same changes for 2.1 and attach. > cqlsh should automatically disable tracing when selecting from system_traces > > > Key: CASSANDRA-7641 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7641 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Brandon Williams >Assignee: Philip Thompson >Priority: Minor > Labels: lhf > Fix For: 2.0.10, 2.1.1 > > Attachments: CASSANDRA-2.0-7641-v3.patch, cassandra-2.0-7641-2.txt, > cassandra-2.0-7641.txt, cassandra-2.1-7641.txt > > > Nobody needs to trace their traces while they're tracing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7708) UDF schema change events/results
[
https://issues.apache.org/jira/browse/CASSANDRA-7708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7708:
Labels: protocolv4 (was: protocolV4)
> UDF schema change events/results
>
>
> Key: CASSANDRA-7708
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7708
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Labels: protocolv4
>
> Schema change notifications for UDF might be interesting for client.
> This covers both - the result of {{CREATE}} + {{DROP}} statements and events.
> Just adding {{FUNCTION}} as a new target for these events breaks previous
> native protocol contract.
> Proposal is to introduce a new target {{FUNCTION}} in native protocol v4.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7769) Implement pg-style UDF source declaration
Robert Stupp created CASSANDRA-7769:
---
Summary: Implement pg-style UDF source declaration
Key: CASSANDRA-7769
URL: https://issues.apache.org/jira/browse/CASSANDRA-7769
Project: Cassandra
Issue Type: Improvement
Reporter: Robert Stupp
Assignee: Robert Stupp
Fix For: 3.0
Follow-up of CASSANDRA-7740:
{{$function$...$function$}} in addition to string style variant.
See also
http://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html#SQL-SYNTAX-DOLLAR-QUOTING
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7641) cqlsh should automatically disable tracing when selecting from system_traces
[ https://issues.apache.org/jira/browse/CASSANDRA-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-7641: --- Attachment: cassandra-2.1-7641-v3.txt > cqlsh should automatically disable tracing when selecting from system_traces > > > Key: CASSANDRA-7641 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7641 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Brandon Williams >Assignee: Philip Thompson >Priority: Minor > Labels: lhf > Fix For: 2.0.10, 2.1.1 > > Attachments: CASSANDRA-2.0-7641-v3.patch, cassandra-2.0-7641-2.txt, > cassandra-2.0-7641.txt, cassandra-2.1-7641-v3.txt, cassandra-2.1-7641.txt > > > Nobody needs to trace their traces while they're tracing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7641) cqlsh should automatically disable tracing when selecting from system_traces
[ https://issues.apache.org/jira/browse/CASSANDRA-7641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097607#comment-14097607 ] Philip Thompson commented on CASSANDRA-7641: I've attached the 2.1 patch, and labeled it v3 to match the 2.0 patch for clarity's sake. I'm having trouble testing it because I'm seeing CASSANDRA-7613 even though it should be resolved. > cqlsh should automatically disable tracing when selecting from system_traces > > > Key: CASSANDRA-7641 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7641 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Brandon Williams >Assignee: Philip Thompson >Priority: Minor > Labels: lhf > Fix For: 2.0.10, 2.1.1 > > Attachments: CASSANDRA-2.0-7641-v3.patch, cassandra-2.0-7641-2.txt, > cassandra-2.0-7641.txt, cassandra-2.1-7641-v3.txt, cassandra-2.1-7641.txt > > > Nobody needs to trace their traces while they're tracing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (CASSANDRA-7613) Error when tracing query with 2.1 cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson reopened CASSANDRA-7613: Reproduced In: 2.1.1 > Error when tracing query with 2.1 cqlsh > --- > > Key: CASSANDRA-7613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7613 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani > Fix For: 2.1 rc5 > > > cqlsh isn't working for me in 2.1 > when executing a query with TRACING ON; I get the following error > myformat_colname() takes exactly 3 arguments (2 given) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7613) Error when tracing query with 2.1 cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097609#comment-14097609 ] Philip Thompson commented on CASSANDRA-7613: I am still able to reproduce on 2.1-HEAD this even though CASSANDRA-7603 has been fixed and the patch is committed. > Error when tracing query with 2.1 cqlsh > --- > > Key: CASSANDRA-7613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7613 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani > Fix For: 2.1 rc5 > > > cqlsh isn't working for me in 2.1 > when executing a query with TRACING ON; I get the following error > myformat_colname() takes exactly 3 arguments (2 given) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7613) Error when tracing query with 2.1 cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097613#comment-14097613 ] Robert Stupp commented on CASSANDRA-7613: - bq. still able to reproduce me too > Error when tracing query with 2.1 cqlsh > --- > > Key: CASSANDRA-7613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7613 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani > Fix For: 2.1 rc5 > > > cqlsh isn't working for me in 2.1 > when executing a query with TRACING ON; I get the following error > myformat_colname() takes exactly 3 arguments (2 given) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7770) NPE when clean up compaction leftover if table is already dropped
Yuki Morishita created CASSANDRA-7770: - Summary: NPE when clean up compaction leftover if table is already dropped Key: CASSANDRA-7770 URL: https://issues.apache.org/jira/browse/CASSANDRA-7770 Project: Cassandra Issue Type: Bug Reporter: Yuki Morishita Assignee: Yuki Morishita Priority: Minor Fix For: 2.1.0 Attachments: 0001-add-check-before-cleaning-up-compaction-leftovers.patch As reported in CASSANDRA-7759, Directories throws NPE when trying to remove compaction leftovers on already dropped table. Attaching patch to check if table exists in schema before clean up. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (CASSANDRA-7613) Error when tracing query with 2.1 cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Stepura reassigned CASSANDRA-7613: -- Assignee: Mikhail Stepura > Error when tracing query with 2.1 cqlsh > --- > > Key: CASSANDRA-7613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7613 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani >Assignee: Mikhail Stepura > Fix For: 2.1 rc5 > > > cqlsh isn't working for me in 2.1 > when executing a query with TRACING ON; I get the following error > myformat_colname() takes exactly 3 arguments (2 given) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7759) DROP TABLE makes C* unreachable
[
https://issues.apache.org/jira/browse/CASSANDRA-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097620#comment-14097620
]
Yuki Morishita commented on CASSANDRA-7759:
---
[~fabrice.larcher] I think NPE on the comment above is different problem from
description.
I filed CASSANDRA-7770 and attached fix there.
> DROP TABLE makes C* unreachable
> ---
>
> Key: CASSANDRA-7759
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7759
> Project: Cassandra
> Issue Type: Bug
> Environment: CQLSH of C* 2.10.-rc5 with one or two node(s).
>Reporter: Fabrice Larcher
>
> After a {{DROP KEYSPACE}} or {{DROP TABLE}} command from _CQLSH_ I often get
> the output :
> {code}
> errors={}, last_host=127.0.0.1
> {code}
> Then, the application can not access the node anymore (it uses the Java
> Driver 2.1.0-rc1). And I get the following stacktraces from the
> {{system.log}} file ;
> {code}
> ERROR [MemtableFlushWriter:5] 2014-08-13 11:26:07,577
> CassandraDaemon.java:166 - Exception in thread
> Thread[MemtableFlushWriter:5,5,main]
> java.lang.RuntimeException: Last written key
> DecoratedKey(28149890686391545-8456361251720325,
> 30313034353666392d666464362d343539362d393537372d653539336430333138396437) >=
> current key DecoratedKey(108b5f3f-fc06-4a0d-99f1-3c6484b32e04,
> 31303862356633662d666330362d346130642d393966312d336336343834623332653034)
> writing into
> ./../data/data/metrics/run-a3c1fe80216911e49fd1ab9c3338a2ff/metrics-run.run_probe-tmp-ka-2-Data.db
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
> ~[guava-16.0.jar:na]
> at
> org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ~[na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]
> {code}
> I also get the following (perhaps not related)
> {code}
> ERROR [MemtableFlushWriter:6] 2014-08-13 11:26:08,827
> CassandraDaemon.java:166 - Exception in thread
> Thread[MemtableFlushWriter:6,5,main]
> org.apache.cassandra.serializers.MarshalException: Invalid byte for ascii: -50
> at
> org.apache.cassandra.serializers.AsciiSerializer.validate(AsciiSerializer.java:39)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:78)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at org.apache.cassandra.dht.LocalToken.toString(LocalToken.java:39)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
> at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
> at org.apache.cassandra.db.DecoratedKey.toString(DecoratedKey.java:118)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
> at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
> ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunn
[jira] [Commented] (CASSANDRA-7770) NPE when clean up compaction leftover if table is already dropped
[ https://issues.apache.org/jira/browse/CASSANDRA-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097630#comment-14097630 ] Jonathan Ellis commented on CASSANDRA-7770: --- Shouldn't we also clean out the sstables involved, or do we already have code to clean out sstables that we don't have schema for? > NPE when clean up compaction leftover if table is already dropped > - > > Key: CASSANDRA-7770 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7770 > Project: Cassandra > Issue Type: Bug >Reporter: Yuki Morishita >Assignee: Yuki Morishita >Priority: Minor > Fix For: 2.1.0 > > Attachments: > 0001-add-check-before-cleaning-up-compaction-leftovers.patch > > > As reported in CASSANDRA-7759, Directories throws NPE when trying to remove > compaction leftovers on already dropped table. > Attaching patch to check if table exists in schema before clean up. -- This message was sent by Atlassian JIRA (v6.2#6252)
[4/5] git commit: merge
merge Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27cbf553 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27cbf553 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27cbf553 Branch: refs/heads/cassandra-2.1 Commit: 27cbf55343fe59e9c8e98fd8857e57adc672b5a5 Parents: 4c510dc 44764c0 Author: Brandon Williams Authored: Thu Aug 14 15:51:01 2014 -0500 Committer: Brandon Williams Committed: Thu Aug 14 15:51:01 2014 -0500 -- CHANGES.txt | 1 + .../hadoop/cql3/CqlBulkOutputFormat.java| 106 ++ .../hadoop/cql3/CqlBulkRecordWriter.java| 199 +++ 3 files changed, 306 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/27cbf553/CHANGES.txt -- diff --cc CHANGES.txt index cf2f34a,1ac22f9..4e1c2b0 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,42 -1,7 +1,43 @@@ -2.0.10 +2.1.1 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569) + * SSTableExport uses correct validator to create string representation of partition + keys (CASSANDRA-7498) + * Avoid NPEs when receiving type changes for an unknown keyspace (CASSANDRA-7689) + * Add support for custom 2i validation (CASSANDRA-7575) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) +Merged from 2.0: * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508) * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Catch errors when the JVM pulls the rug out from GCInspector (CASSANDRA-5345) + * cqlsh fails when version number parts are not int (CASSANDRA-7524) + + +2.1.0-rc6 + * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765) + * Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741) + * Add tracing entries about concurrent range requests (CASSANDRA-7599) + * (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729) + * Remove netty buffer ref-counting (CASSANDRA-7735) + * Pass mutated cf to index updater for use by PRSI (CASSANDRA-7742) + * Include stress yaml example in release and deb (CASSANDRA-7717) + * workaround for netty issue causing corrupted data off the wire (CASSANDRA-7695) + * cqlsh DESC CLUSTER fails retrieving ring information (CASSANDRA-7687) + * Fix binding null values inside UDT (CASSANDRA-7685) + * Fix UDT field selection with empty fields (CASSANDRA-7670) + * Bogus deserialization of static cells from sstable (CASSANDRA-7684) +Merged from 2.0: + * Remove duplicates from StorageService.getJoiningNodes (CASSANDRA-7478) + * Clone token map outside of hot gossip loops (CASSANDRA-7758) * Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752) * Do not flush on truncate if durable_writes is false (CASSANDRA-7750) * Give CRR a default input_cql Statement (CASSANDRA-7226)
[5/5] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c4077ad6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c4077ad6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c4077ad6 Branch: refs/heads/trunk Commit: c4077ad6615d6f96da1588a68c1805f160d6ad0d Parents: fd43c33 27cbf55 Author: Brandon Williams Authored: Thu Aug 14 15:51:09 2014 -0500 Committer: Brandon Williams Committed: Thu Aug 14 15:51:09 2014 -0500 -- CHANGES.txt | 1 + .../hadoop/cql3/CqlBulkOutputFormat.java| 106 ++ .../hadoop/cql3/CqlBulkRecordWriter.java| 199 +++ 3 files changed, 306 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c4077ad6/CHANGES.txt --
[1/5] git commit: backport CASSANDRA-6927 to 2.0
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 4c510dc6c -> 27cbf5534
refs/heads/trunk fd43c33c6 -> c4077ad66
backport CASSANDRA-6927 to 2.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44764c03
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44764c03
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44764c03
Branch: refs/heads/cassandra-2.1
Commit: 44764c03e892bfdb7294ad32d9ff703909186917
Parents: efde6ae
Author: Brandon Williams
Authored: Wed Aug 13 14:14:38 2014 -0500
Committer: Brandon Williams
Committed: Wed Aug 13 14:14:38 2014 -0500
--
CHANGES.txt | 1 +
.../hadoop/AbstractBulkOutputFormat.java| 73 ++
.../hadoop/AbstractBulkRecordWriter.java| 251 ++
.../cassandra/hadoop/BulkOutputFormat.java | 49 +---
.../cassandra/hadoop/BulkRecordWriter.java | 259 ++-
.../hadoop/cql3/CqlBulkOutputFormat.java| 106
.../hadoop/cql3/CqlBulkRecordWriter.java| 199 ++
.../cassandra/hadoop/cql3/CqlConfigHelper.java | 2 +-
.../cassandra/hadoop/cql3/CqlOutputFormat.java | 2 +-
.../io/sstable/AbstractSSTableSimpleWriter.java | 10 +-
.../cassandra/io/sstable/CQLSSTableWriter.java | 3 +-
11 files changed, 663 insertions(+), 292 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/44764c03/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3d31948..1ac22f9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.10
+ * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927)
* Don't depend on cassandra config for nodetool ring (CASSANDRA-7508)
* (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703)
* Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/44764c03/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
b/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
new file mode 100644
index 000..c0e91da
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop;
+
+
+import java.io.IOException;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapreduce.*;
+
+public abstract class AbstractBulkOutputFormat extends OutputFormat
+implements org.apache.hadoop.mapred.OutputFormat
+{
+@Override
+public void checkOutputSpecs(JobContext context)
+{
+checkOutputSpecs(HadoopCompat.getConfiguration(context));
+}
+
+private void checkOutputSpecs(Configuration conf)
+{
+if (ConfigHelper.getOutputKeyspace(conf) == null)
+{
+throw new UnsupportedOperationException("you must set the keyspace
with setColumnFamily()");
+}
+}
+
+@Override
+public OutputCommitter getOutputCommitter(TaskAttemptContext context)
throws IOException, InterruptedException
+{
+return new NullOutputCommitter();
+}
+
+/** Fills the deprecated OutputFormat interface for streaming. */
+@Deprecated
+public void checkOutputSpecs(org.apache.hadoop.fs.FileSystem filesystem,
org.apache.hadoop.mapred.JobConf job) throws IOException
+{
+checkOutputSpecs(job);
+}
+
+public static class NullOutputCommitter extends OutputCommitter
+{
+public void abortTask(TaskAttemptContext taskContext) { }
+
+public void cleanupJob(JobContext jobContext) { }
+
+public void commitTask(TaskAttemptContext taskContext) { }
+
+public boolean needsTaskCommit(TaskAttemptContext taskContext)
+{
+return false;
+}
+
+public
[3/5] git commit: merge
merge Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/27cbf553 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/27cbf553 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/27cbf553 Branch: refs/heads/trunk Commit: 27cbf55343fe59e9c8e98fd8857e57adc672b5a5 Parents: 4c510dc 44764c0 Author: Brandon Williams Authored: Thu Aug 14 15:51:01 2014 -0500 Committer: Brandon Williams Committed: Thu Aug 14 15:51:01 2014 -0500 -- CHANGES.txt | 1 + .../hadoop/cql3/CqlBulkOutputFormat.java| 106 ++ .../hadoop/cql3/CqlBulkRecordWriter.java| 199 +++ 3 files changed, 306 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/27cbf553/CHANGES.txt -- diff --cc CHANGES.txt index cf2f34a,1ac22f9..4e1c2b0 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,42 -1,7 +1,43 @@@ -2.0.10 +2.1.1 + * Avoid IOOBE when building SyntaxError message snippet (CASSANDRA-7569) + * SSTableExport uses correct validator to create string representation of partition + keys (CASSANDRA-7498) + * Avoid NPEs when receiving type changes for an unknown keyspace (CASSANDRA-7689) + * Add support for custom 2i validation (CASSANDRA-7575) + * Pig support for hadoop CqlInputFormat (CASSANDRA-6454) + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) +Merged from 2.0: * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927) + * Don't depend on cassandra config for nodetool ring (CASSANDRA-7508) * (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703) + * Fix IncompatibleClassChangeError from hadoop2 (CASSANDRA-7229) + * Add 'nodetool sethintedhandoffthrottlekb' (CASSANDRA-7635) + * (cqlsh) Add tab-completion for CREATE/DROP USER IF [NOT] EXISTS (CASSANDRA-7611) + * Catch errors when the JVM pulls the rug out from GCInspector (CASSANDRA-5345) + * cqlsh fails when version number parts are not int (CASSANDRA-7524) + + +2.1.0-rc6 + * Skip strict endpoint selection for ranges if RF == nodes (CASSANRA-7765) + * Fix Thrift range filtering without 2ary index lookups (CASSANDRA-7741) + * Add tracing entries about concurrent range requests (CASSANDRA-7599) + * (cqlsh) Fix DESCRIBE for NTS keyspaces (CASSANDRA-7729) + * Remove netty buffer ref-counting (CASSANDRA-7735) + * Pass mutated cf to index updater for use by PRSI (CASSANDRA-7742) + * Include stress yaml example in release and deb (CASSANDRA-7717) + * workaround for netty issue causing corrupted data off the wire (CASSANDRA-7695) + * cqlsh DESC CLUSTER fails retrieving ring information (CASSANDRA-7687) + * Fix binding null values inside UDT (CASSANDRA-7685) + * Fix UDT field selection with empty fields (CASSANDRA-7670) + * Bogus deserialization of static cells from sstable (CASSANDRA-7684) +Merged from 2.0: + * Remove duplicates from StorageService.getJoiningNodes (CASSANDRA-7478) + * Clone token map outside of hot gossip loops (CASSANDRA-7758) * Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752) * Do not flush on truncate if durable_writes is false (CASSANDRA-7750) * Give CRR a default input_cql Statement (CASSANDRA-7226)
[2/5] git commit: backport CASSANDRA-6927 to 2.0
backport CASSANDRA-6927 to 2.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44764c03
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44764c03
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44764c03
Branch: refs/heads/trunk
Commit: 44764c03e892bfdb7294ad32d9ff703909186917
Parents: efde6ae
Author: Brandon Williams
Authored: Wed Aug 13 14:14:38 2014 -0500
Committer: Brandon Williams
Committed: Wed Aug 13 14:14:38 2014 -0500
--
CHANGES.txt | 1 +
.../hadoop/AbstractBulkOutputFormat.java| 73 ++
.../hadoop/AbstractBulkRecordWriter.java| 251 ++
.../cassandra/hadoop/BulkOutputFormat.java | 49 +---
.../cassandra/hadoop/BulkRecordWriter.java | 259 ++-
.../hadoop/cql3/CqlBulkOutputFormat.java| 106
.../hadoop/cql3/CqlBulkRecordWriter.java| 199 ++
.../cassandra/hadoop/cql3/CqlConfigHelper.java | 2 +-
.../cassandra/hadoop/cql3/CqlOutputFormat.java | 2 +-
.../io/sstable/AbstractSSTableSimpleWriter.java | 10 +-
.../cassandra/io/sstable/CQLSSTableWriter.java | 3 +-
11 files changed, 663 insertions(+), 292 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/44764c03/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3d31948..1ac22f9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.10
+ * (Hadoop) Add CqlOutputFormat (CASSANDRA-6927)
* Don't depend on cassandra config for nodetool ring (CASSANDRA-7508)
* (cqlsh) Fix failing cqlsh formatting tests (CASSANDRA-7703)
* Fix MS expiring map timeout for Paxos messages (CASSANDRA-7752)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/44764c03/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
b/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
new file mode 100644
index 000..c0e91da
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/AbstractBulkOutputFormat.java
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.hadoop;
+
+
+import java.io.IOException;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapreduce.*;
+
+public abstract class AbstractBulkOutputFormat extends OutputFormat
+implements org.apache.hadoop.mapred.OutputFormat
+{
+@Override
+public void checkOutputSpecs(JobContext context)
+{
+checkOutputSpecs(HadoopCompat.getConfiguration(context));
+}
+
+private void checkOutputSpecs(Configuration conf)
+{
+if (ConfigHelper.getOutputKeyspace(conf) == null)
+{
+throw new UnsupportedOperationException("you must set the keyspace
with setColumnFamily()");
+}
+}
+
+@Override
+public OutputCommitter getOutputCommitter(TaskAttemptContext context)
throws IOException, InterruptedException
+{
+return new NullOutputCommitter();
+}
+
+/** Fills the deprecated OutputFormat interface for streaming. */
+@Deprecated
+public void checkOutputSpecs(org.apache.hadoop.fs.FileSystem filesystem,
org.apache.hadoop.mapred.JobConf job) throws IOException
+{
+checkOutputSpecs(job);
+}
+
+public static class NullOutputCommitter extends OutputCommitter
+{
+public void abortTask(TaskAttemptContext taskContext) { }
+
+public void cleanupJob(JobContext jobContext) { }
+
+public void commitTask(TaskAttemptContext taskContext) { }
+
+public boolean needsTaskCommit(TaskAttemptContext taskContext)
+{
+return false;
+}
+
+public void setupJob(JobContext jobContext) { }
+
+public void setupTask(TaskAttemptContext taskContext) { }
+}
+}
http://git-wip-us.ap
[jira] [Created] (CASSANDRA-7771) Allow multiple 2ndary index on the same column
Sylvain Lebresne created CASSANDRA-7771: --- Summary: Allow multiple 2ndary index on the same column Key: CASSANDRA-7771 URL: https://issues.apache.org/jira/browse/CASSANDRA-7771 Project: Cassandra Issue Type: Improvement Reporter: Sylvain Lebresne Priority: Minor Fix For: 3.0 Currently, the code assumes that we'll only have one 2ndary index per column. This has been reasonable so far but stop being it with CASSANDRA-6382 (you might want to index multiple fields of the same UDT column) and CASSANDRA-7458 (you may want to have one "normal" index an multiple functional index for the same column). So we should consider removing that assumption in the code, which is mainly 2 places: # in the schema: each ColumnDefinition only has infos for one index. This part should probably be tackled in CASSANDRA-6717 so I'm marking this issue as a follow-up of CASSANDRA-6717. # in the 2ndary index API: this is the part I'm suggesting we fix in this issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-5836) Seed nodes should be able to bootstrap without manual intervention
[ https://issues.apache.org/jira/browse/CASSANDRA-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097647#comment-14097647 ] Jon Travis commented on CASSANDRA-5836: --- I was just bitten by this as well. Our ops uses ZooKeeper to store a list of all our infrastructure, so I wrote a SeedProvider that peeked into Zk for the list of Cassandra nodes and returned that as the seed list ... big mistake.. Our push-button deployment launched the node (it thought it was a seed), so it essentially stopped doing anything, reported errors of missing keyspaces and column families, then simply sat there. All the while, it claims it has a portion of the ring, yet no data. There is no good documentation about this and no warnings in the logs -- this is certainly something that will bite more people. It would be nice if the process could warn about the error or refuse to start under this scenario. > Seed nodes should be able to bootstrap without manual intervention > -- > > Key: CASSANDRA-5836 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5836 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Bill Hathaway >Priority: Minor > > The current logic doesn't allow a seed node to be bootstrapped. If a user > wants to bootstrap a node configured as a seed (for example to replace a seed > node via replace_token), they first need to remove the node's own IP from the > seed list, and then start the bootstrap process. This seems like an > unnecessary step since a node never uses itself as a seed. > I think it would be a better experience if the logic was changed to allow a > seed node to bootstrap without manual intervention when there are other seed > nodes up in a ring. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7772) Windows - fsync-analog, flush data to disk
Joshua McKenzie created CASSANDRA-7772: -- Summary: Windows - fsync-analog, flush data to disk Key: CASSANDRA-7772 URL: https://issues.apache.org/jira/browse/CASSANDRA-7772 Project: Cassandra Issue Type: Improvement Reporter: Joshua McKenzie Assignee: Joshua McKenzie Fix For: 3.0 We currently use CLibrary fsync linux-native calls to flush to disk. Given the role this plays in our SequentialWriter and data integrity in general we need some analog to this function on Windows. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (CASSANDRA-7613) Error when tracing query with 2.1 cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp reassigned CASSANDRA-7613: --- Assignee: Robert Stupp (was: Mikhail Stepura) > Error when tracing query with 2.1 cqlsh > --- > > Key: CASSANDRA-7613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7613 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani >Assignee: Robert Stupp > Fix For: 2.1 rc5 > > Attachments: 7613.txt > > > cqlsh isn't working for me in 2.1 > when executing a query with TRACING ON; I get the following error > myformat_colname() takes exactly 3 arguments (2 given) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7770) NPE when clean up compaction leftover if table is already dropped
[ https://issues.apache.org/jira/browse/CASSANDRA-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097650#comment-14097650 ] Yuki Morishita commented on CASSANDRA-7770: --- Do you mean deleting SSTables that dropped table had? It is already done by releasing all references to SSTables when dropping. > NPE when clean up compaction leftover if table is already dropped > - > > Key: CASSANDRA-7770 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7770 > Project: Cassandra > Issue Type: Bug >Reporter: Yuki Morishita >Assignee: Yuki Morishita >Priority: Minor > Fix For: 2.1.0 > > Attachments: > 0001-add-check-before-cleaning-up-compaction-leftovers.patch > > > As reported in CASSANDRA-7759, Directories throws NPE when trying to remove > compaction leftovers on already dropped table. > Attaching patch to check if table exists in schema before clean up. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7613) Error when tracing query with 2.1 cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-7613: Attachment: 7613.txt > Error when tracing query with 2.1 cqlsh > --- > > Key: CASSANDRA-7613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7613 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani >Assignee: Robert Stupp > Fix For: 2.1 rc5 > > Attachments: 7613.txt > > > cqlsh isn't working for me in 2.1 > when executing a query with TRACING ON; I get the following error > myformat_colname() takes exactly 3 arguments (2 given) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7770) NPE when clean up compaction leftover if table is already dropped
[ https://issues.apache.org/jira/browse/CASSANDRA-7770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097655#comment-14097655 ] Jonathan Ellis commented on CASSANDRA-7770: --- +1 then > NPE when clean up compaction leftover if table is already dropped > - > > Key: CASSANDRA-7770 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7770 > Project: Cassandra > Issue Type: Bug >Reporter: Yuki Morishita >Assignee: Yuki Morishita >Priority: Minor > Fix For: 2.1.0 > > Attachments: > 0001-add-check-before-cleaning-up-compaction-leftovers.patch > > > As reported in CASSANDRA-7759, Directories throws NPE when trying to remove > compaction leftovers on already dropped table. > Attaching patch to check if table exists in schema before clean up. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7613) Error when tracing query with 2.1 cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097656#comment-14097656 ] Robert Stupp commented on CASSANDRA-7613: - colored-column-name patch lurked around :) > Error when tracing query with 2.1 cqlsh > --- > > Key: CASSANDRA-7613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7613 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani >Assignee: Robert Stupp > Fix For: 2.1 rc6 > > Attachments: 7613.txt > > > cqlsh isn't working for me in 2.1 > when executing a query with TRACING ON; I get the following error > myformat_colname() takes exactly 3 arguments (2 given) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7769) Implement pg-style dollar syntax for string constants
[
https://issues.apache.org/jira/browse/CASSANDRA-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-7769:
Summary: Implement pg-style dollar syntax for string constants (was:
Implement pg-style UDF source declaration)
> Implement pg-style dollar syntax for string constants
> -
>
> Key: CASSANDRA-7769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7769
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.0
>
>
> Follow-up of CASSANDRA-7740:
> {{$function$...$function$}} in addition to string style variant.
> See also
> http://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html#SQL-SYNTAX-DOLLAR-QUOTING
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7769) Implement pg-style dollar syntax for string constants
[
https://issues.apache.org/jira/browse/CASSANDRA-7769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097664#comment-14097664
]
Sylvain Lebresne commented on CASSANDRA-7769:
-
For the record, we should definitively not make this UDF specific (even if
that's a primary motivation).
> Implement pg-style dollar syntax for string constants
> -
>
> Key: CASSANDRA-7769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7769
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Robert Stupp
>Assignee: Robert Stupp
> Fix For: 3.0
>
>
> Follow-up of CASSANDRA-7740:
> {{$function$...$function$}} in addition to string style variant.
> See also
> http://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html#SQL-SYNTAX-DOLLAR-QUOTING
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7772) Windows - fsync-analog, flush data to disk
[ https://issues.apache.org/jira/browse/CASSANDRA-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097667#comment-14097667 ] Joshua McKenzie commented on CASSANDRA-7772: It looks like FileDescriptor in nio has a [method to sync to disk|http://docs.oracle.com/javase/7/docs/api/java/io/FileDescriptor.html#sync()]. There's the possibility for some [discrepancy in performance|http://stas-blogspot.blogspot.com/2011/11/java-file-flushing-performance.html] on various platforms so it might be worth checking if we're gaining anything with the native fsync call over a FileDescriptor.sync call on linux at this point. Another option would be to start on a windows-specific CLibrary and pursue using [FlushFileBuffers|http://msdn.microsoft.com/en-us/library/aa364439(v=vs.85).aspx]. The remainder of our CLibrary routines don't have many analogs on win7 (page cache hinting specifically). In my opinion syncing alone doesn't warrant adding an abstraction on top of CLibrary to branch to os-specific native code unless the FileDescriptor sync performance is terrible on Windows. > Windows - fsync-analog, flush data to disk > -- > > Key: CASSANDRA-7772 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7772 > Project: Cassandra > Issue Type: Improvement >Reporter: Joshua McKenzie >Assignee: Joshua McKenzie > Labels: Windows > Fix For: 3.0 > > > We currently use CLibrary fsync linux-native calls to flush to disk. Given > the role this plays in our SequentialWriter and data integrity in general we > need some analog to this function on Windows. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7772) Windows - fsync-analog, flush data to disk
[ https://issues.apache.org/jira/browse/CASSANDRA-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097679#comment-14097679 ] Jonathan Ellis commented on CASSANDRA-7772: --- We use CLibrary.trySync not because it performs better than FileDescriptor.sync, but because FD.sync doesn't sync the parent directory. (CASSANDRA-3250) > Windows - fsync-analog, flush data to disk > -- > > Key: CASSANDRA-7772 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7772 > Project: Cassandra > Issue Type: Improvement >Reporter: Joshua McKenzie >Assignee: Joshua McKenzie > Labels: Windows > Fix For: 3.0 > > > We currently use CLibrary fsync linux-native calls to flush to disk. Given > the role this plays in our SequentialWriter and data integrity in general we > need some analog to this function on Windows. -- This message was sent by Atlassian JIRA (v6.2#6252)
