[jira] [Updated] (CASSANDRA-7128) Switch NBHS for CHS on read path
[ https://issues.apache.org/jira/browse/CASSANDRA-7128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-7128: -- Attachment: 7128.txt > Switch NBHS for CHS on read path > - > > Key: CASSANDRA-7128 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7128 > Project: Cassandra > Issue Type: Improvement >Reporter: T Jake Luciani >Assignee: T Jake Luciani >Priority: Trivial > Labels: performance > Fix For: 2.1 beta2 > > Attachments: 7128.txt, Screen Shot 2014-04-30 at 11.07.22 PM.png > > > AbstractRowResolver uses a NBHM for each read request. > Profiler flagged this as a bottleneck since the init() call creates a > AtomicReferenceFieldUpdater which is stored in a synchronized collection. > A NBHS is most certainly overkill for such a short lived object. And turns > out switching it to a CHS in my tests yields a ~5-10% read boost. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7128) Switch NBHS for CHS on read path
T Jake Luciani created CASSANDRA-7128: - Summary: Switch NBHS for CHS on read path Key: CASSANDRA-7128 URL: https://issues.apache.org/jira/browse/CASSANDRA-7128 Project: Cassandra Issue Type: Improvement Reporter: T Jake Luciani Assignee: T Jake Luciani Priority: Trivial Fix For: 2.1 beta2 Attachments: 7128.txt, Screen Shot 2014-04-30 at 11.07.22 PM.png AbstractRowResolver uses a NBHM for each read request. Profiler flagged this as a bottleneck since the init() call creates a AtomicReferenceFieldUpdater which is stored in a synchronized collection. A NBHS is most certainly overkill for such a short lived object. And turns out switching it to a CHS in my tests yields a ~5-10% read boost. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6855) Native protocol V3
[ https://issues.apache.org/jira/browse/CASSANDRA-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Shuler updated CASSANDRA-6855: -- Attachment: auth_test_dtest-2.1~git9872b74.node1.log auth_test_dtest-2.1~git9872b74.txt This was committed as [9872b74|https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=9872b74ef20018e4e7645a8952fd7295e75764ad] and introduced a large number of errors in the [auth_test dtest|http://cassci.datastax.com/job/cassandra-2.1_dtest/130/]. I've attached an auth_test run and a sample node log file from one of the tests - they all look the same. Thanks! > Native protocol V3 > -- > > Key: CASSANDRA-6855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6855 > Project: Cassandra > Issue Type: New Feature >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 2.1 beta2 > > Attachments: auth_test_dtest-2.1~git9872b74.node1.log, > auth_test_dtest-2.1~git9872b74.txt > > > I think we need a V3 of the protocol for 2.1. The things that this > could/should includes are: > # Adding an optional Serial CL for protocol batches (like we have for QUERY > and EXECUTE). It was an oversight of V2 of not adding it, and now that we can > batch conditional updates, it's definitively missing. > # Proper type codes for UDT. This is not *strictly* needed to be able to > support UDT since currently a UDT will be sent as a "custom type" with his > fully class name + arguments. But parsing that is no fun nor convenient for > clients. It's also not particular space efficient (though that's probably not > a huge concern since with prepared statement you can avoid sending the > ResultSet metadata every time). > # Serialization format for collections. Currently the serialization format > only allow for 65K elements, each of 65K bytes size at most. While > collections are not meant to store large amount of data, having the > limitation in the protocol serialization format is the wrong way to deal with > that. Concretely, the current workaround for CASSANDRA-5428 is ugly. I'll > note that the current serialization format is also an obstacle to supporting > null inside collections (whether or not we want to support null there is a > good question, but here again I'm not sure being limited by the serialization > format is a good idea). > # CASSANDRA-6178: I continue to believe that in many case it makes somewhat > more sense to have the default timestamp provided by the client (this is a > necessary condition for true idempotent retries in particular). I'm > absolutely fine making that optional and leaving server-side generated > timestamps by default, but since client can already provide timestamp in > query string anyway, I don't see a big deal in making it easier for client > driver to control that without messing with the query string. > # Optional names for values in QUERY messages: it has been brought to my > attention that while V2 allows to send a query string with values for a > one-roundtrip bind-and-execute, a driver can't really support named bind > marker with that feature properly without parsing the query. The proposition > is thus to make it (optionally) possible to ship the name of the marker each > value is supposed to be bound to. > I think that 1) and 2) are enough reason to make a V3 (even if there is > disagreement on the rest that is). > 3) is a little bit more involved tbh but I do think having the current > limitations bolted in the protocol serialization format is wrong in the long > run, and it turns out that due to UDT we will start storing serialized > collections internally so if we want to lift said limitation in the > serialization format, we should do it now and everywhere, as doing it > afterwards will be a lot more painful. > 4) and 5) are probably somewhat more minor, but at the same time, both are > completely optional (a driver won't have to support those if he doesn't > want). They are really just about making things more flexible for client > drivers and they are not particularly hard to support so I don't see too many > reasons not to include them. > Last but not least, I know that some may find it wrong to do a new protocol > version with each major of C*, so let me state my view here: I fully agree > that we shouldn't make an habit of that in the long run and that's > definitively *not* my objective. However, it would be silly to expect that we > could get everything right and forget nothing in the very first version. It > shouldn't be surprising that we'll have to burn a few versions (and there > might be a few more yet) before getting something more stable and complete > and I think that delaying the addition of stuffs that are useful to create > some fake notion of stability would be
[jira] [Updated] (CASSANDRA-6855) Native protocol V3
[ https://issues.apache.org/jira/browse/CASSANDRA-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McGuire updated CASSANDRA-6855: Tester: Michael Shuler > Native protocol V3 > -- > > Key: CASSANDRA-6855 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6855 > Project: Cassandra > Issue Type: New Feature >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 2.1 beta2 > > Attachments: auth_test_dtest-2.1~git9872b74.node1.log, > auth_test_dtest-2.1~git9872b74.txt > > > I think we need a V3 of the protocol for 2.1. The things that this > could/should includes are: > # Adding an optional Serial CL for protocol batches (like we have for QUERY > and EXECUTE). It was an oversight of V2 of not adding it, and now that we can > batch conditional updates, it's definitively missing. > # Proper type codes for UDT. This is not *strictly* needed to be able to > support UDT since currently a UDT will be sent as a "custom type" with his > fully class name + arguments. But parsing that is no fun nor convenient for > clients. It's also not particular space efficient (though that's probably not > a huge concern since with prepared statement you can avoid sending the > ResultSet metadata every time). > # Serialization format for collections. Currently the serialization format > only allow for 65K elements, each of 65K bytes size at most. While > collections are not meant to store large amount of data, having the > limitation in the protocol serialization format is the wrong way to deal with > that. Concretely, the current workaround for CASSANDRA-5428 is ugly. I'll > note that the current serialization format is also an obstacle to supporting > null inside collections (whether or not we want to support null there is a > good question, but here again I'm not sure being limited by the serialization > format is a good idea). > # CASSANDRA-6178: I continue to believe that in many case it makes somewhat > more sense to have the default timestamp provided by the client (this is a > necessary condition for true idempotent retries in particular). I'm > absolutely fine making that optional and leaving server-side generated > timestamps by default, but since client can already provide timestamp in > query string anyway, I don't see a big deal in making it easier for client > driver to control that without messing with the query string. > # Optional names for values in QUERY messages: it has been brought to my > attention that while V2 allows to send a query string with values for a > one-roundtrip bind-and-execute, a driver can't really support named bind > marker with that feature properly without parsing the query. The proposition > is thus to make it (optionally) possible to ship the name of the marker each > value is supposed to be bound to. > I think that 1) and 2) are enough reason to make a V3 (even if there is > disagreement on the rest that is). > 3) is a little bit more involved tbh but I do think having the current > limitations bolted in the protocol serialization format is wrong in the long > run, and it turns out that due to UDT we will start storing serialized > collections internally so if we want to lift said limitation in the > serialization format, we should do it now and everywhere, as doing it > afterwards will be a lot more painful. > 4) and 5) are probably somewhat more minor, but at the same time, both are > completely optional (a driver won't have to support those if he doesn't > want). They are really just about making things more flexible for client > drivers and they are not particularly hard to support so I don't see too many > reasons not to include them. > Last but not least, I know that some may find it wrong to do a new protocol > version with each major of C*, so let me state my view here: I fully agree > that we shouldn't make an habit of that in the long run and that's > definitively *not* my objective. However, it would be silly to expect that we > could get everything right and forget nothing in the very first version. It > shouldn't be surprising that we'll have to burn a few versions (and there > might be a few more yet) before getting something more stable and complete > and I think that delaying the addition of stuffs that are useful to create > some fake notion of stability would be even more silly. On the bright side, > the additions of this V3 are comparatively much more simple to implement for > a client that those of V2 (in fact, for clients that want to support UDT, it > will probably require less effort to add the changes for this new version > than to try to support UDT without it), so I do think we make good progress > on getting the protocol stabilized -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7123) BATCH documentation should be explicit about ordering guarantees
[ https://issues.apache.org/jira/browse/CASSANDRA-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986264#comment-13986264 ] Alex P commented on CASSANDRA-7123: --- lgtm > BATCH documentation should be explicit about ordering guarantees > > > Key: CASSANDRA-7123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7123 > Project: Cassandra > Issue Type: Task > Components: Documentation & website >Reporter: Tyler Hobbs >Assignee: Tyler Hobbs >Priority: Minor > Attachments: 7123.txt > > > In the CQL3 [batch statement > documentation](http://cassandra.apache.org/doc/cql3/CQL.html#batchStmt) we > don't mention that there are no ordering guarantees, which can lead to > somewhat surprising behavior (CASSANDRA-6291). > We should also mention that you could specify timestamps in order to achieve > a particular ordering. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Issue Comment Deleted] (CASSANDRA-6831) Updates to COMPACT STORAGE tables via cli drop CQL information
[
https://issues.apache.org/jira/browse/CASSANDRA-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Stepura updated CASSANDRA-6831:
---
Comment: was deleted
(was: [~thobbs] I don't think so, the test works fine on a commit
2269adba6c28e24f60883700f657d36fff7b9d3f, which is 2.1 patch.
It's broken by some later commit)
> Updates to COMPACT STORAGE tables via cli drop CQL information
> --
>
> Key: CASSANDRA-6831
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6831
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Russell Bradberry
>Assignee: Mikhail Stepura
>Priority: Minor
> Fix For: 1.2.17, 2.0.8, 2.1 beta2
>
> Attachments: 6831-1.2.patch, 6831-2.0-v2.txt, 6831-2.1.patch
>
>
> If a COMPACT STORAGE table is altered using the CLI all information about the
> column names reverts to the initial "key, column1, column2" namings.
> Additionally, the changes in the columns name will not take effect until the
> Cassandra service is restarted. This means that the clients using CQL will
> continue to work properly until the service is restarted, at which time they
> will start getting errors about non-existant columns in the table.
> When attempting to rename the columns back using ALTER TABLE an error stating
> the column already exists will be raised. The only way to get it back is to
> ALTER TABLE and change the comment or something, which will bring back all
> the original column names.
> This seems to be related to CASSANDRA-6676 and CASSANDRA-6370
> In cqlsh
> {code}
> Connected to cluster1 at 127.0.0.3:9160.
> [cqlsh 3.1.8 | Cassandra 1.2.15-SNAPSHOT | CQL spec 3.0.0 | Thrift protocol
> 19.36.2]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH REPLICATION = { 'class' : 'SimpleStrategy',
> 'replication_factor' : 3 };
> cqlsh> USE test;
> cqlsh:test> CREATE TABLE foo (bar text, baz text, qux text, PRIMARY KEY(bar,
> baz) ) WITH COMPACT STORAGE;
> cqlsh:test> describe table foo;
> CREATE TABLE foo (
> bar text,
> baz text,
> qux text,
> PRIMARY KEY (bar, baz)
> ) WITH COMPACT STORAGE AND
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.10 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'SnappyCompressor'};
> {code}
> Now in cli:
> {code}
> Connected to: "cluster1" on 127.0.0.3/9160
> Welcome to Cassandra CLI version 1.2.15-SNAPSHOT
> Type 'help;' or '?' for help.
> Type 'quit;' or 'exit;' to quit.
> [default@unknown] use test;
> Authenticated to keyspace: test
> [default@test] UPDATE COLUMN FAMILY foo WITH comment='hey this is a comment';
> 3bf5fa49-5d03-34f0-b46c-6745f7740925
> {code}
> Now back in cqlsh:
> {code}
> cqlsh:test> describe table foo;
> CREATE TABLE foo (
> bar text,
> column1 text,
> value text,
> PRIMARY KEY (bar, column1)
> ) WITH COMPACT STORAGE AND
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='hey this is a comment' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.10 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'SnappyCompressor'};
> cqlsh:test> ALTER TABLE foo WITH comment='this is a new comment';
> cqlsh:test> describe table foo;
> CREATE TABLE foo (
> bar text,
> baz text,
> qux text,
> PRIMARY KEY (bar, baz)
> ) WITH COMPACT STORAGE AND
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='this is a new comment' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.10 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'SnappyCompressor'};
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6831) Updates to COMPACT STORAGE tables via cli drop CQL information
[
https://issues.apache.org/jira/browse/CASSANDRA-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986253#comment-13986253
]
Mikhail Stepura commented on CASSANDRA-6831:
[~thobbs] I don't think so, the test works fine on a commit
2269adba6c28e24f60883700f657d36fff7b9d3f, which is 2.1 patch.
It's broken by some later commit
> Updates to COMPACT STORAGE tables via cli drop CQL information
> --
>
> Key: CASSANDRA-6831
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6831
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Russell Bradberry
>Assignee: Mikhail Stepura
>Priority: Minor
> Fix For: 1.2.17, 2.0.8, 2.1 beta2
>
> Attachments: 6831-1.2.patch, 6831-2.0-v2.txt, 6831-2.1.patch
>
>
> If a COMPACT STORAGE table is altered using the CLI all information about the
> column names reverts to the initial "key, column1, column2" namings.
> Additionally, the changes in the columns name will not take effect until the
> Cassandra service is restarted. This means that the clients using CQL will
> continue to work properly until the service is restarted, at which time they
> will start getting errors about non-existant columns in the table.
> When attempting to rename the columns back using ALTER TABLE an error stating
> the column already exists will be raised. The only way to get it back is to
> ALTER TABLE and change the comment or something, which will bring back all
> the original column names.
> This seems to be related to CASSANDRA-6676 and CASSANDRA-6370
> In cqlsh
> {code}
> Connected to cluster1 at 127.0.0.3:9160.
> [cqlsh 3.1.8 | Cassandra 1.2.15-SNAPSHOT | CQL spec 3.0.0 | Thrift protocol
> 19.36.2]
> Use HELP for help.
> cqlsh> CREATE KEYSPACE test WITH REPLICATION = { 'class' : 'SimpleStrategy',
> 'replication_factor' : 3 };
> cqlsh> USE test;
> cqlsh:test> CREATE TABLE foo (bar text, baz text, qux text, PRIMARY KEY(bar,
> baz) ) WITH COMPACT STORAGE;
> cqlsh:test> describe table foo;
> CREATE TABLE foo (
> bar text,
> baz text,
> qux text,
> PRIMARY KEY (bar, baz)
> ) WITH COMPACT STORAGE AND
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.10 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'SnappyCompressor'};
> {code}
> Now in cli:
> {code}
> Connected to: "cluster1" on 127.0.0.3/9160
> Welcome to Cassandra CLI version 1.2.15-SNAPSHOT
> Type 'help;' or '?' for help.
> Type 'quit;' or 'exit;' to quit.
> [default@unknown] use test;
> Authenticated to keyspace: test
> [default@test] UPDATE COLUMN FAMILY foo WITH comment='hey this is a comment';
> 3bf5fa49-5d03-34f0-b46c-6745f7740925
> {code}
> Now back in cqlsh:
> {code}
> cqlsh:test> describe table foo;
> CREATE TABLE foo (
> bar text,
> column1 text,
> value text,
> PRIMARY KEY (bar, column1)
> ) WITH COMPACT STORAGE AND
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='hey this is a comment' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.10 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'SnappyCompressor'};
> cqlsh:test> ALTER TABLE foo WITH comment='this is a new comment';
> cqlsh:test> describe table foo;
> CREATE TABLE foo (
> bar text,
> baz text,
> qux text,
> PRIMARY KEY (bar, baz)
> ) WITH COMPACT STORAGE AND
> bloom_filter_fp_chance=0.01 AND
> caching='KEYS_ONLY' AND
> comment='this is a new comment' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.10 AND
> replicate_on_write='true' AND
> populate_io_cache_on_flush='false' AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'SnappyCompressor'};
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (CASSANDRA-6283) Windows 7 data files kept open / can't be deleted after compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie resolved CASSANDRA-6283.
Resolution: Fixed
Fix Version/s: (was: 2.0.8)
3.0
Reproduced In: 2.0.3, 2.0.2, 2.1.0 (was: 2.0.2, 2.0.3, 2.1.0)
Reference CASSANDRA-6993 and CASSANDRA-4050 for patches, CASSANDRA-6890 for
performance #'s.
> Windows 7 data files kept open / can't be deleted after compaction.
> ---
>
> Key: CASSANDRA-6283
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows 7 (32) / Java 1.7.0.45
>Reporter: Andreas Schnitzerling
>Assignee: Joshua McKenzie
> Labels: compaction
> Fix For: 3.0
>
> Attachments: 6283_StreamWriter_patch.txt, leakdetect.patch,
> neighbor-log.zip, root-log.zip, screenshot-1.jpg, system.log
>
>
> Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't
> help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause
> is: Opened file handles seem to be lost and not closed properly. Win 7
> blames, that another process is still using the file (but its obviously
> cassandra). Only restart of the server makes the files deleted. But after
> heavy using (changes) of tables, there are about 24K files in the data folder
> (instead of 35 after every restart) and Cassandra crashes. I experiminted and
> I found out, that a finalizer fixes the problem. So after GC the files will
> be deleted (not optimal, but working fine). It runs now 2 days continously
> without problem. Possible fix/test:
> I wrote the following finalizer at the end of class
> org.apache.cassandra.io.util.RandomAccessReader:
> {code:title=RandomAccessReader.java|borderStyle=solid}
> @Override
> protected void finalize() throws Throwable {
> deallocate();
> super.finalize();
> }
> {code}
> Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-3668) Parallel streaming for sstableloader
[ https://issues.apache.org/jira/browse/CASSANDRA-3668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986225#comment-13986225 ] Joshua McKenzie commented on CASSANDRA-3668: Looks good - I went back and forth between re-using those data structures in the StreamCoordinator vs. limiting its scope to the StreamResultFuture only during implementation. While this approach has a touch of duplication to it I think it's fine in order to keep the scope limited. Test of v3 looks good. I'm getting some NPE's from SSTableReader.scheduleTidy closing up IndexSummaries but I don't think that has anything to do with this patch. +1 > Parallel streaming for sstableloader > > > Key: CASSANDRA-3668 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3668 > Project: Cassandra > Issue Type: Improvement > Components: API >Reporter: Manish Zope >Assignee: Joshua McKenzie >Priority: Minor > Labels: streaming > Fix For: 2.1 rc1 > > Attachments: 3668-1.1-v2.txt, 3668-1.1.txt, 3668-v3.txt, 3668_v2.txt, > 3688-reply_before_closing_writer.txt, sstable-loader performance.txt > > Original Estimate: 48h > Remaining Estimate: 48h > > One of my colleague had reported the bug regarding the degraded performance > of the sstable generator and sstable loader. > ISSUE :- https://issues.apache.org/jira/browse/CASSANDRA-3589 > As stated in above issue generator performance is rectified but performance > of the sstableloader is still an issue. > 3589 is marked as duplicate of 3618.Both issues shows resolved status.But the > problem with sstableloader still exists. > So opening other issue so that sstbleloader problem should not go unnoticed. > FYI : We have tested the generator part with the patch given in 3589.Its > Working fine. > Please let us know if you guys require further inputs from our side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6546) disablethrift results in unclosed file descriptors
[ https://issues.apache.org/jira/browse/CASSANDRA-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Stepura updated CASSANDRA-6546: --- Attachment: CASSANDRA-6546-2.x.patch > disablethrift results in unclosed file descriptors > -- > > Key: CASSANDRA-6546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6546 > Project: Cassandra > Issue Type: Bug >Reporter: Jason Harvey >Assignee: Mikhail Stepura >Priority: Minor > Fix For: 1.2.17, 2.0.8, 2.1 beta2 > > Attachments: CASSANDRA-6546-1.2.patch, CASSANDRA-6546-2.x.patch > > > Disabling thrift results in unclosed thrift sockets being left around. > Steps to reproduce and observe: > 1. Have a handful of clients connect via thrift. > 2. Disable thrift. > 3. Enable thrift, have the clients reconnect. > 4. Observe netstat or lsof, and you'll find a lot of thrift sockets in > CLOSE_WAIT state, and they'll never go away. > * Also verifiable from > org.apache.cassandra.metrics:type=Client,name=connectedThriftClients MBean. > What's extra fun about this is the leaked sockets still count towards your > maximum RPC thread count. As a result, toggling thrift enough times will > result in an rpc_max_threads number of CLOSED_WAIT sockets, with no new > clients able to connect. > This was reproduced with HSHA. I haven't tried it in sync yet. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-3668) Parallel streaming for sstableloader
[ https://issues.apache.org/jira/browse/CASSANDRA-3668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-3668: -- Attachment: 3668-v3.txt Thanks Joshua, I attached v3 which is rebased and slightly modified version. I think it is better to not expose StreamCoordinator outside of StreamResultFuture just to track progress in bulkloader. What do you think? I haven't worked yet but I think we can split sessions based on the size we transfer rather than doing round robin (in separate ticket maybe). > Parallel streaming for sstableloader > > > Key: CASSANDRA-3668 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3668 > Project: Cassandra > Issue Type: Improvement > Components: API >Reporter: Manish Zope >Assignee: Joshua McKenzie >Priority: Minor > Labels: streaming > Fix For: 2.1 rc1 > > Attachments: 3668-1.1-v2.txt, 3668-1.1.txt, 3668-v3.txt, 3668_v2.txt, > 3688-reply_before_closing_writer.txt, sstable-loader performance.txt > > Original Estimate: 48h > Remaining Estimate: 48h > > One of my colleague had reported the bug regarding the degraded performance > of the sstable generator and sstable loader. > ISSUE :- https://issues.apache.org/jira/browse/CASSANDRA-3589 > As stated in above issue generator performance is rectified but performance > of the sstableloader is still an issue. > 3589 is marked as duplicate of 3618.Both issues shows resolved status.But the > problem with sstableloader still exists. > So opening other issue so that sstbleloader problem should not go unnoticed. > FYI : We have tested the generator part with the patch given in 3589.Its > Working fine. > Please let us know if you guys require further inputs from our side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7127) Add command to reclaim disk space for dropped column families
[ https://issues.apache.org/jira/browse/CASSANDRA-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986183#comment-13986183 ] Jonathan Ellis edited comment on CASSANDRA-7127 at 4/30/14 10:53 PM: - You can set auto_snapshot: false if you want to turn this behavior off entirely; otherwise, nodetool clearsnapshot will allow this post-CASSANDRA-6821 was (Author: jbellis): You can set auto_snapshot: false if you want to turn this behavior off entirely; otherwise, nodetool clearsnapshot will allow this post-CASSANDRA-6281 > Add command to reclaim disk space for dropped column families > - > > Key: CASSANDRA-7127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7127 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Bogdan Batranut >Priority: Trivial > Fix For: 2.1 beta2 > > > I found out that after a DROP a column family's data is not deleted. I looked > into the /data/keyspace dir and saw dirs corresponding to old columnfamilies > that I have dropped in the past. I do see the benefit of taking snapshots > when dropping but I can also see the need to reclaim disk space. During > developent on a cluster that will be in production, people might push a lot > of data into a columnfamily and change the definition for some reason. The > old unused data will eat a lot of space. > So can you please implement a feature that will reclaim badly used disk space? > Thank you. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-7127) Add command to reclaim disk space for dropped column families
[ https://issues.apache.org/jira/browse/CASSANDRA-7127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis resolved CASSANDRA-7127. --- Resolution: Not a Problem You can set auto_snapshot: false if you want to turn this behavior off entirely; otherwise, nodetool clearsnapshot will allow this post-CASSANDRA-6281 > Add command to reclaim disk space for dropped column families > - > > Key: CASSANDRA-7127 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7127 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Bogdan Batranut >Priority: Trivial > Fix For: 2.1 beta2 > > > I found out that after a DROP a column family's data is not deleted. I looked > into the /data/keyspace dir and saw dirs corresponding to old columnfamilies > that I have dropped in the past. I do see the benefit of taking snapshots > when dropping but I can also see the need to reclaim disk space. During > developent on a cluster that will be in production, people might push a lot > of data into a columnfamily and change the definition for some reason. The > old unused data will eat a lot of space. > So can you please implement a feature that will reclaim badly used disk space? > Thank you. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6546) disablethrift results in unclosed file descriptors
[
https://issues.apache.org/jira/browse/CASSANDRA-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986181#comment-13986181
]
Mikhail Stepura commented on CASSANDRA-6546:
To fix this for 2.x we have to use the latest build of {{thrift-server}} -
https://github.com/xedin/disruptor_thrift_server/pull/7
> disablethrift results in unclosed file descriptors
> --
>
> Key: CASSANDRA-6546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6546
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jason Harvey
>Assignee: Mikhail Stepura
>Priority: Minor
> Fix For: 1.2.17, 2.0.8, 2.1 beta2
>
> Attachments: CASSANDRA-6546-1.2.patch
>
>
> Disabling thrift results in unclosed thrift sockets being left around.
> Steps to reproduce and observe:
> 1. Have a handful of clients connect via thrift.
> 2. Disable thrift.
> 3. Enable thrift, have the clients reconnect.
> 4. Observe netstat or lsof, and you'll find a lot of thrift sockets in
> CLOSE_WAIT state, and they'll never go away.
> * Also verifiable from
> org.apache.cassandra.metrics:type=Client,name=connectedThriftClients MBean.
> What's extra fun about this is the leaked sockets still count towards your
> maximum RPC thread count. As a result, toggling thrift enough times will
> result in an rpc_max_threads number of CLOSED_WAIT sockets, with no new
> clients able to connect.
> This was reproduced with HSHA. I haven't tried it in sync yet.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7120) Bad paging state returned for prepared statements for last page
[
https://issues.apache.org/jira/browse/CASSANDRA-7120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-7120:
---
Fix Version/s: 2.1 beta2
> Bad paging state returned for prepared statements for last page
> ---
>
> Key: CASSANDRA-7120
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7120
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
> Fix For: 2.1 beta2
>
>
> When executing a paged query with a prepared statement, a non-null paging
> state is sometimes being returned for the final page, causing an endless
> paging loop.
> Specifically, this is the schema being used:
> {noformat}
> CREATE KEYSPACE test3rf WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '3'}';
> USE test3rf;
> CREATE TABLE test3rf.test (
> k int PRIMARY KEY,
> v int
> )
> {noformat}
> The inserts are like so:
> {noformat}
> INSERT INTO test3rf.test (k, v) VALUES (?, 0)
> {noformat}
> With values from [0, 99] used for k.
> The query is {{SELECT * FROM test3rf.test}} with a fetch size of 3.
> The final page returns the row with k=3, and the paging state is
> {{000400420004000176007fa2}}. This matches the paging state from
> three pages earlier. When executing this with a non-prepared statement, no
> paging state is returned for this page.
> This problem doesn't happen with the 2.0 branch.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7127) Add command to reclaim disk space for dropped column families
Bogdan Batranut created CASSANDRA-7127: -- Summary: Add command to reclaim disk space for dropped column families Key: CASSANDRA-7127 URL: https://issues.apache.org/jira/browse/CASSANDRA-7127 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Bogdan Batranut Priority: Trivial Fix For: 2.1 beta2 I found out that after a DROP a column family's data is not deleted. I looked into the /data/keyspace dir and saw dirs corresponding to old columnfamilies that I have dropped in the past. I do see the benefit of taking snapshots when dropping but I can also see the need to reclaim disk space. During developent on a cluster that will be in production, people might push a lot of data into a columnfamily and change the definition for some reason. The old unused data will eat a lot of space. So can you please implement a feature that will reclaim badly used disk space? Thank you. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6546) disablethrift results in unclosed file descriptors
[ https://issues.apache.org/jira/browse/CASSANDRA-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Stepura updated CASSANDRA-6546: --- Attachment: (was: 2014-04-30 12-22-58.png) > disablethrift results in unclosed file descriptors > -- > > Key: CASSANDRA-6546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6546 > Project: Cassandra > Issue Type: Bug >Reporter: Jason Harvey >Assignee: Mikhail Stepura >Priority: Minor > Fix For: 1.2.17, 2.0.8, 2.1 beta2 > > Attachments: CASSANDRA-6546-1.2.patch > > > Disabling thrift results in unclosed thrift sockets being left around. > Steps to reproduce and observe: > 1. Have a handful of clients connect via thrift. > 2. Disable thrift. > 3. Enable thrift, have the clients reconnect. > 4. Observe netstat or lsof, and you'll find a lot of thrift sockets in > CLOSE_WAIT state, and they'll never go away. > * Also verifiable from > org.apache.cassandra.metrics:type=Client,name=connectedThriftClients MBean. > What's extra fun about this is the leaked sockets still count towards your > maximum RPC thread count. As a result, toggling thrift enough times will > result in an rpc_max_threads number of CLOSED_WAIT sockets, with no new > clients able to connect. > This was reproduced with HSHA. I haven't tried it in sync yet. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6546) disablethrift results in unclosed file descriptors
[ https://issues.apache.org/jira/browse/CASSANDRA-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Stepura updated CASSANDRA-6546: --- Attachment: (was: 2014-04-30 12-22-17.png) > disablethrift results in unclosed file descriptors > -- > > Key: CASSANDRA-6546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6546 > Project: Cassandra > Issue Type: Bug >Reporter: Jason Harvey >Assignee: Mikhail Stepura >Priority: Minor > Fix For: 1.2.17, 2.0.8, 2.1 beta2 > > Attachments: CASSANDRA-6546-1.2.patch > > > Disabling thrift results in unclosed thrift sockets being left around. > Steps to reproduce and observe: > 1. Have a handful of clients connect via thrift. > 2. Disable thrift. > 3. Enable thrift, have the clients reconnect. > 4. Observe netstat or lsof, and you'll find a lot of thrift sockets in > CLOSE_WAIT state, and they'll never go away. > * Also verifiable from > org.apache.cassandra.metrics:type=Client,name=connectedThriftClients MBean. > What's extra fun about this is the leaked sockets still count towards your > maximum RPC thread count. As a result, toggling thrift enough times will > result in an rpc_max_threads number of CLOSED_WAIT sockets, with no new > clients able to connect. > This was reproduced with HSHA. I haven't tried it in sync yet. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6875) CQL3: select multiple CQL rows in a single partition using IN
[
https://issues.apache.org/jira/browse/CASSANDRA-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986178#comment-13986178
]
Tyler Hobbs commented on CASSANDRA-6875:
bq. It bothers me to start adding unit tests for CQL queries when all of our
CQL tests are currently in dtest. I'd much rather keep it all in the dtests to
avoid the confusion on where is what tested.
I went with unit tests for a couple of reasons. The main one is that we can't
use prepared statements with the dtests right now (as far as I can tell). The
other is that it's a lot faster to work with unit tests than dtests here, and a
distributed setup isn't really required for any of these tests. We should
definitely have a discussion about where and how to test in general (maybe in a
dev ML thread), but my feeling is that dtests are good for:
* Testing end-to-end
* Tests that require more than one node
Otherwise, if it's reasonable to do in a unit test, I don't see why we wouldn't
do it there. FWIW, if we could use prepared statements in the dtests, I would
also add some of the prepared statement test cases from the unit tests there.
All of your other suggestions sound good, thanks!
> CQL3: select multiple CQL rows in a single partition using IN
> -
>
> Key: CASSANDRA-6875
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6875
> Project: Cassandra
> Issue Type: Bug
> Components: API
>Reporter: Nicolas Favre-Felix
>Assignee: Tyler Hobbs
>Priority: Minor
> Fix For: 2.0.8
>
>
> In the spirit of CASSANDRA-4851 and to bring CQL to parity with Thrift, it is
> important to support reading several distinct CQL rows from a given partition
> using a distinct set of "coordinates" for these rows within the partition.
> CASSANDRA-4851 introduced a range scan over the multi-dimensional space of
> clustering keys. We also need to support a "multi-get" of CQL rows,
> potentially using the "IN" keyword to define a set of clustering keys to
> fetch at once.
> (reusing the same example\:)
> Consider the following table:
> {code}
> CREATE TABLE test (
> k int,
> c1 int,
> c2 int,
> PRIMARY KEY (k, c1, c2)
> );
> {code}
> with the following data:
> {code}
> k | c1 | c2
> ---++
> 0 | 0 | 0
> 0 | 0 | 1
> 0 | 1 | 0
> 0 | 1 | 1
> {code}
> We can fetch a single row or a range of rows, but not a set of them:
> {code}
> > SELECT * FROM test WHERE k = 0 AND (c1, c2) IN ((0, 0), (1,1)) ;
> Bad Request: line 1:54 missing EOF at ','
> {code}
> Supporting this syntax would return:
> {code}
> k | c1 | c2
> ---++
> 0 | 0 | 0
> 0 | 1 | 1
> {code}
> Being able to fetch these two CQL rows in a single read is important to
> maintain partition-level isolation.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7123) BATCH documentation should be explicit about ordering guarantees
[ https://issues.apache.org/jira/browse/CASSANDRA-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-7123: --- Attachment: 7123.txt The attached patch makes a change that's pretty close to the one suggested by [~alexyz]. > BATCH documentation should be explicit about ordering guarantees > > > Key: CASSANDRA-7123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7123 > Project: Cassandra > Issue Type: Task > Components: Documentation & website >Reporter: Tyler Hobbs >Assignee: Tyler Hobbs >Priority: Minor > Attachments: 7123.txt > > > In the CQL3 [batch statement > documentation](http://cassandra.apache.org/doc/cql3/CQL.html#batchStmt) we > don't mention that there are no ordering guarantees, which can lead to > somewhat surprising behavior (CASSANDRA-6291). > We should also mention that you could specify timestamps in order to achieve > a particular ordering. -- This message was sent by Atlassian JIRA (v6.2#6252)
git commit: use parameterized logging
Repository: cassandra
Updated Branches:
refs/heads/trunk 43f19eb57 -> e78b002b1
use parameterized logging
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e78b002b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e78b002b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e78b002b
Branch: refs/heads/trunk
Commit: e78b002b131bb34792dd7a04f26d9a3a57c531c1
Parents: 43f19eb
Author: Dave Brosius
Authored: Wed Apr 30 18:20:26 2014 -0400
Committer: Dave Brosius
Committed: Wed Apr 30 18:20:26 2014 -0400
--
src/java/org/apache/cassandra/service/StorageService.java | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/e78b002b/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 51be5e7..d94c0cd 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -822,7 +822,7 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
{
bootstrapTokens =
BootStrapper.getRandomTokens(tokenMetadata, DatabaseDescriptor.getNumTokens());
if (DatabaseDescriptor.getNumTokens() == 1)
-logger.warn("Generated random token " +
bootstrapTokens + ". Random tokens will result in an unbalanced ring; see
http://wiki.apache.org/cassandra/Operations";);
+logger.warn("Generated random token {}. Random tokens
will result in an unbalanced ring; see
http://wiki.apache.org/cassandra/Operations";, bootstrapTokens);
else
logger.info("Generated random tokens. tokens are {}",
bootstrapTokens);
}
@@ -877,10 +877,10 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
}
bootstrapTokens.add(midpoint);
}
-logger.info("Split previous range (" + left + ", " + right
+ "] into " + bootstrapTokens);
+logger.info("Split previous range ({}, {}] into {}", left,
right, bootstrapTokens);
}
else
-logger.info("Using saved tokens " + bootstrapTokens);
+logger.info("Using saved tokens {}", bootstrapTokens);
}
}
git commit: trivial StringBuilder usage fixes
Repository: cassandra
Updated Branches:
refs/heads/trunk 5a9ca7765 -> 43f19eb57
trivial StringBuilder usage fixes
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/43f19eb5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/43f19eb5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/43f19eb5
Branch: refs/heads/trunk
Commit: 43f19eb576004ac7694e76425d37cf876c01673c
Parents: 5a9ca77
Author: Dave Brosius
Authored: Wed Apr 30 18:15:43 2014 -0400
Committer: Dave Brosius
Committed: Wed Apr 30 18:15:43 2014 -0400
--
.../src/org/apache/cassandra/stress/settings/GroupedOptions.java | 2 +-
.../stress/src/org/apache/cassandra/stress/settings/Legacy.java | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/43f19eb5/tools/stress/src/org/apache/cassandra/stress/settings/GroupedOptions.java
--
diff --git
a/tools/stress/src/org/apache/cassandra/stress/settings/GroupedOptions.java
b/tools/stress/src/org/apache/cassandra/stress/settings/GroupedOptions.java
index d0e63b0..e351521 100644
--- a/tools/stress/src/org/apache/cassandra/stress/settings/GroupedOptions.java
+++ b/tools/stress/src/org/apache/cassandra/stress/settings/GroupedOptions.java
@@ -85,7 +85,7 @@ public abstract class GroupedOptions
}
firstRow = false;
-StringBuilder sb = new StringBuilder("Usage: " + command);
+StringBuilder sb = new StringBuilder("Usage: ").append(command);
for (Option option : grouping.options())
{
sb.append(" ");
http://git-wip-us.apache.org/repos/asf/cassandra/blob/43f19eb5/tools/stress/src/org/apache/cassandra/stress/settings/Legacy.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/settings/Legacy.java
b/tools/stress/src/org/apache/cassandra/stress/settings/Legacy.java
index 5ddb346..eb04290 100644
--- a/tools/stress/src/org/apache/cassandra/stress/settings/Legacy.java
+++ b/tools/stress/src/org/apache/cassandra/stress/settings/Legacy.java
@@ -216,12 +216,12 @@ public class Legacy implements Serializable
{
StringBuilder rep = new StringBuilder();
if (cmd.hasOption("R"))
-rep.append("strategy=" + cmd.getOptionValue("R"));
+rep.append("strategy=").append(cmd.getOptionValue("R"));
if (cmd.hasOption("l"))
{
if (rep.length() > 0)
rep.append(",");
-rep.append("factor=" + cmd.getOptionValue("l"));
+rep.append("factor=").append(cmd.getOptionValue("l"));
}
if (cmd.hasOption("O"))
{
[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986144#comment-13986144
]
Tyler Hobbs commented on CASSANDRA-2434:
+1 overall, with a few minor nitpicks on the dtest:
You can replace that string-building loop with:
{noformat}
tl = " ".join(str(t) for t in tokens[0][:8])
{noformat}
There's also a leftover line: {{#assert 1 == 0}}
Don't forget to mark the new tests with a {{@since('2.1')}} decorator.
> range movements can violate consistency
> ---
>
> Key: CASSANDRA-2434
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2434
> Project: Cassandra
> Issue Type: Bug
>Reporter: Peter Schuller
>Assignee: T Jake Luciani
> Fix For: 2.1 beta2
>
> Attachments: 2434-3.patch.txt, 2434-testery.patch.txt
>
>
> My reading (a while ago) of the code indicates that there is no logic
> involved during bootstrapping that avoids consistency level violations. If I
> recall correctly it just grabs neighbors that are currently up.
> There are at least two issues I have with this behavior:
> * If I have a cluster where I have applications relying on QUORUM with RF=3,
> and bootstrapping complete based on only one node, I have just violated the
> supposedly guaranteed consistency semantics of the cluster.
> * Nodes can flap up and down at any time, so even if a human takes care to
> look at which nodes are up and things about it carefully before
> bootstrapping, there's no guarantee.
> A complication is that not only does it depend on use-case where this is an
> issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster
> which is otherwise used for QUORUM operations you may wish to accept
> less-than-quorum nodes during bootstrap in various emergency situations.
> A potential easy fix is to have bootstrap take an argument which is the
> number of hosts to bootstrap from, or to assume QUORUM if none is given.
> (A related concern is bootstrapping across data centers. You may *want* to
> bootstrap to a local node and then do a repair to avoid sending loads of data
> across DC:s while still achieving consistency. Or even if you don't care
> about the consistency issues, I don't think there is currently a way to
> bootstrap from local nodes only.)
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7126) relocate doesn't remove tokens from system.local when done
[ https://issues.apache.org/jira/browse/CASSANDRA-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-7126: Summary: relocate doesn't remove tokens from system.local when done (was: relocate is broken) > relocate doesn't remove tokens from system.local when done > -- > > Key: CASSANDRA-7126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7126 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani >Assignee: Brandon Williams > Fix For: 2.0.8 > > > While testing CASSANDRA-2434 I noticed the tokens being relocated aren't > being removed from the source node. > Here is a dtest https://github.com/tjake/cassandra-dtest/tree/taketoken -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7122) Replacement nodes have null entries in system.peers
[
https://issues.apache.org/jira/browse/CASSANDRA-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986130#comment-13986130
]
Brandon Williams commented on CASSANDRA-7122:
-
Yep! Care to post the patch?
> Replacement nodes have null entries in system.peers
> ---
>
> Key: CASSANDRA-7122
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7122
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>
> If a node is replaced with -Dcassandra.replace_address, the new node has
> mostly null entries in system.peers:
> {code}
> > select * from system.peers;
> peer | data_center | host_id | rack | release_version | rpc_address |
> schema_version | tokens
> ---+-+-+--+-+-++--
> 127.0.0.3 |null |null | null |null |null |
> null | {'-3074457345618258602'}
> {code}
> To reproduce, simply kill a node and replace it. The entries are correctly
> populated if the replacement node is restarted but they are never populated
> if it isn't.
> I can think of at least two bad consequences of this:
> 1. Drivers like Datastax java-driver use the peers table to find the
> rpc_address and location info of a node. If the entires are null it assumes
> rpc_address=ip and the node is in the local DC.
> 2. When using GossipingPropertyFileSnitch and node won't persist the DC/rack
> of another node so may not be able to locate it during restarts.
> I reproduced in 1.2.15 but from inspection it looks to be present in 1.2.16
> and 2.0.7.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7122) Replacement nodes have null entries in system.peers
[
https://issues.apache.org/jira/browse/CASSANDRA-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986127#comment-13986127
]
Richard Low commented on CASSANDRA-7122:
Thanks Brandon. I tested adding a call to onChange for each non-STATUS state in
SS.handleStateNormal and it fixed the issue. Is that what you meant?
> Replacement nodes have null entries in system.peers
> ---
>
> Key: CASSANDRA-7122
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7122
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>
> If a node is replaced with -Dcassandra.replace_address, the new node has
> mostly null entries in system.peers:
> {code}
> > select * from system.peers;
> peer | data_center | host_id | rack | release_version | rpc_address |
> schema_version | tokens
> ---+-+-+--+-+-++--
> 127.0.0.3 |null |null | null |null |null |
> null | {'-3074457345618258602'}
> {code}
> To reproduce, simply kill a node and replace it. The entries are correctly
> populated if the replacement node is restarted but they are never populated
> if it isn't.
> I can think of at least two bad consequences of this:
> 1. Drivers like Datastax java-driver use the peers table to find the
> rpc_address and location info of a node. If the entires are null it assumes
> rpc_address=ip and the node is in the local DC.
> 2. When using GossipingPropertyFileSnitch and node won't persist the DC/rack
> of another node so may not be able to locate it during restarts.
> I reproduced in 1.2.15 but from inspection it looks to be present in 1.2.16
> and 2.0.7.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Reopened] (CASSANDRA-6831) Updates to COMPACT STORAGE tables via cli drop CQL information
[
https://issues.apache.org/jira/browse/CASSANDRA-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs reopened CASSANDRA-6831:
Reproduced In: 2.1 beta1, 2.0.6, 1.2.16 (was: 1.2.16, 2.0.6, 2.1 beta1)
It looks like the 2.1 patch broke CF secondary index creation through Thrift.
The following error shows up when when adding a column definition with an index
on it:
{noformat}
INFO [MigrationStage:1] 2014-04-30 16:34:11,766 DefsTables.java:388 - Loading
org.apache.cassandra.config.CFMetaData@1547508d[cfId=2b5c99b0-d0af-11e3-a938-6b09a6cc3d5a,ksName=PycassaTestKeyspace,cfName=Indexed1,cfType=Standard,comparator=org.apache.cassandra.db.marshal.BytesType,comment=,readRepairChance=0.1,dclocalReadRepairChance=0.0,gcGraceSeconds=864000,defaultValidator=org.apache.cassandra.db.marshal.BytesType,keyValidator=org.apache.cassandra.db.marshal.BytesType,minCompactionThreshold=4,maxCompactionThreshold=32,columnMetadata={java.nio.HeapByteBuffer[pos=0
lim=3 cap=3]=ColumnDefinition{name=key,
type=org.apache.cassandra.db.marshal.BytesType, kind=PARTITION_KEY,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=5 cap=5]=ColumnDefinition{name=value,
type=org.apache.cassandra.db.marshal.BytesType, kind=COMPACT_VALUE,
componentIndex=null, indexName=null, indexType=null},
java.nio.HeapByteBuffer[pos=0 lim=7 cap=7]=ColumnDefinition{name=column1,
type=org.apache.cassandra.db.marshal.BytesType, kind=CLUSTERING_COLUMN,
componentIndex=null, indexName=null,
indexType=null}},compactionStrategyClass=class
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,compactionStrategyOptions={},compressionParameters={sstable_compression=org.apache.cassandra.io.compress.LZ4Compressor},bloomFilterFpChance=,memtableFlushPeriod=0,caching={"keys":"ALL",
"rows_per_partition":"NONE"},defaultTimeToLive=0,minIndexInterval=128,maxIndexInterval=2048,speculativeRetry=NONE,droppedColumns={},triggers={}]
INFO [MigrationStage:1] 2014-04-30 16:34:11,771 ColumnFamilyStore.java:285 -
Initializing PycassaTestKeyspace.Indexed1
ERROR [Thrift:1] 2014-04-30 16:34:11,810 CustomTThreadPoolServer.java:219 -
Error occurred during processing of message.
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_40]
at java.util.ArrayList.set(ArrayList.java:426) ~[na:1.7.0_40]
at org.apache.cassandra.config.CFMetaData.rebuild(CFMetaData.java:2037)
~[main/:na]
at
org.apache.cassandra.config.CFMetaData.fromThriftForUpdate(CFMetaData.java:1036)
~[main/:na]
at
org.apache.cassandra.thrift.CassandraServer.system_update_column_family(CassandraServer.java:1668)
~[main/:na]
at
org.apache.cassandra.thrift.Cassandra$Processor$system_update_column_family.getResult(Cassandra.java:4430)
~[thrift/:na]
at
org.apache.cassandra.thrift.Cassandra$Processor$system_update_column_family.getResult(Cassandra.java:4414)
~[thrift/:na]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.1.jar:0.9.1]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.1.jar:0.9.1]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:201)
~[main/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_40]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_40]
at java.lang.Thread.run(Thread.java:724) [na:1.7.0_40]
{noformat}.
> Updates to COMPACT STORAGE tables via cli drop CQL information
> --
>
> Key: CASSANDRA-6831
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6831
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Russell Bradberry
>Assignee: Mikhail Stepura
>Priority: Minor
> Fix For: 1.2.17, 2.0.8, 2.1 beta2
>
> Attachments: 6831-1.2.patch, 6831-2.0-v2.txt, 6831-2.1.patch
>
>
> If a COMPACT STORAGE table is altered using the CLI all information about the
> column names reverts to the initial "key, column1, column2" namings.
> Additionally, the changes in the columns name will not take effect until the
> Cassandra service is restarted. This means that the clients using CQL will
> continue to work properly until the service is restarted, at which time they
> will start getting errors about non-existant columns in the table.
> When attempting to rename the columns back using ALTER TABLE an error stating
> the column already exists will be raised. The only way to get it back is to
> ALTER TABLE and change the comment or something, which will bring back all
> the original column names.
> This seems to be related to CASSANDRA-6676 and CAS
[jira] [Assigned] (CASSANDRA-7126) relocate is broken
[ https://issues.apache.org/jira/browse/CASSANDRA-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reassigned CASSANDRA-7126: --- Assignee: Brandon Williams > relocate is broken > -- > > Key: CASSANDRA-7126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7126 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani >Assignee: Brandon Williams > Fix For: 2.0.8 > > > While testing CASSANDRA-2434 I noticed the tokens being relocated aren't > being removed from the source node. > Here is a dtest https://github.com/tjake/cassandra-dtest/tree/taketoken -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6959) Reusing Keyspace and CF names raises assertion errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986117#comment-13986117
]
Tyler Hobbs commented on CASSANDRA-6959:
git bisect points to 2269adba6c28e24f60883700f657d36fff7b9d3f (CASSANDRA-6831).
> Reusing Keyspace and CF names raises assertion errors
> -
>
> Key: CASSANDRA-6959
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6959
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Benedict
> Fix For: 2.1 beta2
>
> Attachments: 6959.txt
>
>
> The [dtest I
> introduced|https://github.com/riptano/cassandra-dtest/commit/36960090d219ab8dbc7f108faa91c3ea5cea2bec]
> to test CASSANDRA-6924 introduces some log errors which I think may be
> related to CASSANDRA-5202.
> On 2.1 :
> {code}
> ERROR [MigrationStage:1] 2014-03-31 14:36:43,463
> CommitLogSegmentManager.java:306 - Failed waiting for a forced recycle of
> in-use commit log segments
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManager.forceRecycleAll(CommitLogSegmentManager.java:301)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.forceRecycleAllSegments(CommitLog.java:160)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.dropColumnFamily(DefsTables.java:497)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:296)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:181) [main/:na]
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49)
> [main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {code}
> On 2.0:
> {code}
> ERROR [ReadStage:3] 2014-03-31 13:28:11,014 CassandraDaemon.java (line 198)
> Exception in thread Thread[ReadStage:3,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.db.filter.ExtendedFilter$WithClauses.getExtraFilter(ExtendedFilter.java:258)
> at
> org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1744)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
> at
> org.apache.cassandra.db.PagedRangeCommand.executeLocally(PagedRangeCommand.java:119)
> at
> org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:39)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> To reproduce, you many need to comment out the assertion in that test, as it
> is not 100% reproducible on the first try.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7126) relocate is broken
[ https://issues.apache.org/jira/browse/CASSANDRA-7126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-7126: -- Fix Version/s: 2.0.8 > relocate is broken > -- > > Key: CASSANDRA-7126 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7126 > Project: Cassandra > Issue Type: Bug >Reporter: T Jake Luciani > Fix For: 2.0.8 > > > While testing CASSANDRA-2434 I noticed the tokens being relocated aren't > being removed from the source node. > Here is a dtest https://github.com/tjake/cassandra-dtest/tree/taketoken -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7126) relocate is broken
T Jake Luciani created CASSANDRA-7126: - Summary: relocate is broken Key: CASSANDRA-7126 URL: https://issues.apache.org/jira/browse/CASSANDRA-7126 Project: Cassandra Issue Type: Bug Reporter: T Jake Luciani While testing CASSANDRA-2434 I noticed the tokens being relocated aren't being removed from the source node. Here is a dtest https://github.com/tjake/cassandra-dtest/tree/taketoken -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-6694) Slightly More Off-Heap Memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko resolved CASSANDRA-6694. -- Resolution: Fixed Ninja-committed. > Slightly More Off-Heap Memtables > > > Key: CASSANDRA-6694 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6694 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Assignee: Benedict > Labels: performance > Fix For: 2.1 beta2 > > Attachments: 6694.fix1.txt > > > The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as > the on-heap overhead is still very large. It should not be tremendously > difficult to extend these changes so that we allocate entire Cells off-heap, > instead of multiple BBs per Cell (with all their associated overhead). > The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 > bytes per cell on average for the btree overhead, for a total overhead of > around 20-22 bytes). This translates to 8-byte object overhead, 4-byte > address (we will do alignment tricks like the VM to allow us to address a > reasonably large memory space, although this trick is unlikely to last us > forever, at which point we will have to bite the bullet and accept a 24-byte > per cell overhead), and 4-byte object reference for maintaining our internal > list of allocations, which is unfortunately necessary since we cannot safely > (and cheaply) walk the object graph we allocate otherwise, which is necessary > for (allocation-) compaction and pointer rewriting. > The ugliest thing here is going to be implementing the various CellName > instances so that they may be backed by native memory OR heap memory. -- This message was sent by Atlassian JIRA (v6.2#6252)
[2/2] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a9ca776 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a9ca776 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a9ca776 Branch: refs/heads/trunk Commit: 5a9ca7765fad0e9289891b190456c8b405721faf Parents: 3608129 68961a6 Author: Aleksey Yeschenko Authored: Wed Apr 30 23:29:23 2014 +0200 Committer: Aleksey Yeschenko Committed: Wed Apr 30 23:29:23 2014 +0200 -- src/java/org/apache/cassandra/utils/PureJavaCrc32.java | 1 + 1 file changed, 1 insertion(+) --
[1/2] git commit: Ninja-fix PureJavaCrc32
Repository: cassandra
Updated Branches:
refs/heads/trunk 3608129a3 -> 5a9ca7765
Ninja-fix PureJavaCrc32
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/68961a6a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/68961a6a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/68961a6a
Branch: refs/heads/trunk
Commit: 68961a6a9089a8cd374dc90577f2d48b515aa0a7
Parents: 70fd478
Author: Aleksey Yeschenko
Authored: Wed Apr 30 23:29:05 2014 +0200
Committer: Aleksey Yeschenko
Committed: Wed Apr 30 23:29:05 2014 +0200
--
src/java/org/apache/cassandra/utils/PureJavaCrc32.java | 1 +
1 file changed, 1 insertion(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/68961a6a/src/java/org/apache/cassandra/utils/PureJavaCrc32.java
--
diff --git a/src/java/org/apache/cassandra/utils/PureJavaCrc32.java
b/src/java/org/apache/cassandra/utils/PureJavaCrc32.java
index 15fe0c6..041652f 100644
--- a/src/java/org/apache/cassandra/utils/PureJavaCrc32.java
+++ b/src/java/org/apache/cassandra/utils/PureJavaCrc32.java
@@ -122,6 +122,7 @@ public class PureJavaCrc32 implements Checksum {
ByteBufferUtil.arrayCopy(b, off, buf, 0, l);
update(buf, 0, l);
len -= l;
+off += l;
}
}
}
git commit: Ninja-fix PureJavaCrc32
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 70fd47827 -> 68961a6a9
Ninja-fix PureJavaCrc32
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/68961a6a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/68961a6a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/68961a6a
Branch: refs/heads/cassandra-2.1
Commit: 68961a6a9089a8cd374dc90577f2d48b515aa0a7
Parents: 70fd478
Author: Aleksey Yeschenko
Authored: Wed Apr 30 23:29:05 2014 +0200
Committer: Aleksey Yeschenko
Committed: Wed Apr 30 23:29:05 2014 +0200
--
src/java/org/apache/cassandra/utils/PureJavaCrc32.java | 1 +
1 file changed, 1 insertion(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/68961a6a/src/java/org/apache/cassandra/utils/PureJavaCrc32.java
--
diff --git a/src/java/org/apache/cassandra/utils/PureJavaCrc32.java
b/src/java/org/apache/cassandra/utils/PureJavaCrc32.java
index 15fe0c6..041652f 100644
--- a/src/java/org/apache/cassandra/utils/PureJavaCrc32.java
+++ b/src/java/org/apache/cassandra/utils/PureJavaCrc32.java
@@ -122,6 +122,7 @@ public class PureJavaCrc32 implements Checksum {
ByteBufferUtil.arrayCopy(b, off, buf, 0, l);
update(buf, 0, l);
len -= l;
+off += l;
}
}
}
[jira] [Commented] (CASSANDRA-6959) Reusing Keyspace and CF names raises assertion errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986098#comment-13986098
]
Tyler Hobbs commented on CASSANDRA-6959:
Hmm, I can reproduce that, but it wasn't happening yesterday, so a recent
change must has broken something.
> Reusing Keyspace and CF names raises assertion errors
> -
>
> Key: CASSANDRA-6959
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6959
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Benedict
> Fix For: 2.1 beta2
>
> Attachments: 6959.txt
>
>
> The [dtest I
> introduced|https://github.com/riptano/cassandra-dtest/commit/36960090d219ab8dbc7f108faa91c3ea5cea2bec]
> to test CASSANDRA-6924 introduces some log errors which I think may be
> related to CASSANDRA-5202.
> On 2.1 :
> {code}
> ERROR [MigrationStage:1] 2014-03-31 14:36:43,463
> CommitLogSegmentManager.java:306 - Failed waiting for a forced recycle of
> in-use commit log segments
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManager.forceRecycleAll(CommitLogSegmentManager.java:301)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.forceRecycleAllSegments(CommitLog.java:160)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.dropColumnFamily(DefsTables.java:497)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:296)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:181) [main/:na]
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49)
> [main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {code}
> On 2.0:
> {code}
> ERROR [ReadStage:3] 2014-03-31 13:28:11,014 CassandraDaemon.java (line 198)
> Exception in thread Thread[ReadStage:3,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.db.filter.ExtendedFilter$WithClauses.getExtraFilter(ExtendedFilter.java:258)
> at
> org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1744)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
> at
> org.apache.cassandra.db.PagedRangeCommand.executeLocally(PagedRangeCommand.java:119)
> at
> org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:39)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> To reproduce, you many need to comment out the assertion in that test, as it
> is not 100% reproducible on the first try.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[1/2] git commit: Fix column name in invalid constant type error msg
Repository: cassandra
Updated Branches:
refs/heads/trunk b35486a19 -> 3608129a3
Fix column name in invalid constant type error msg
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/70fd4782
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/70fd4782
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/70fd4782
Branch: refs/heads/trunk
Commit: 70fd478271fe13afeb85f05b8897b91058967b96
Parents: 4485e6d
Author: Tyler Hobbs
Authored: Wed Apr 30 15:59:21 2014 -0500
Committer: Tyler Hobbs
Committed: Wed Apr 30 15:59:21 2014 -0500
--
src/java/org/apache/cassandra/cql3/Constants.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/70fd4782/src/java/org/apache/cassandra/cql3/Constants.java
--
diff --git a/src/java/org/apache/cassandra/cql3/Constants.java
b/src/java/org/apache/cassandra/cql3/Constants.java
index 5af84f0..abd039f 100644
--- a/src/java/org/apache/cassandra/cql3/Constants.java
+++ b/src/java/org/apache/cassandra/cql3/Constants.java
@@ -125,7 +125,7 @@ public abstract class Constants
public Value prepare(String keyspace, ColumnSpecification receiver)
throws InvalidRequestException
{
if (!isAssignableTo(keyspace, receiver))
-throw new InvalidRequestException(String.format("Invalid %s
constant (%s) for %s of type %s", type, text, receiver,
receiver.type.asCQL3Type()));
+throw new InvalidRequestException(String.format("Invalid %s
constant (%s) for \"%s\" of type %s", type, text, receiver.name,
receiver.type.asCQL3Type()));
return new Value(parsedValue(receiver.type));
}
[2/2] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3608129a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3608129a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3608129a Branch: refs/heads/trunk Commit: 3608129a3d668fda1dc00ccf5bc14eaae1cfdaf6 Parents: b35486a 70fd478 Author: Tyler Hobbs Authored: Wed Apr 30 16:06:48 2014 -0500 Committer: Tyler Hobbs Committed: Wed Apr 30 16:06:48 2014 -0500 -- src/java/org/apache/cassandra/cql3/Constants.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --
git commit: Fix column name in invalid constant type error msg
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 4485e6dbf -> 70fd47827
Fix column name in invalid constant type error msg
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/70fd4782
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/70fd4782
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/70fd4782
Branch: refs/heads/cassandra-2.1
Commit: 70fd478271fe13afeb85f05b8897b91058967b96
Parents: 4485e6d
Author: Tyler Hobbs
Authored: Wed Apr 30 15:59:21 2014 -0500
Committer: Tyler Hobbs
Committed: Wed Apr 30 15:59:21 2014 -0500
--
src/java/org/apache/cassandra/cql3/Constants.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/70fd4782/src/java/org/apache/cassandra/cql3/Constants.java
--
diff --git a/src/java/org/apache/cassandra/cql3/Constants.java
b/src/java/org/apache/cassandra/cql3/Constants.java
index 5af84f0..abd039f 100644
--- a/src/java/org/apache/cassandra/cql3/Constants.java
+++ b/src/java/org/apache/cassandra/cql3/Constants.java
@@ -125,7 +125,7 @@ public abstract class Constants
public Value prepare(String keyspace, ColumnSpecification receiver)
throws InvalidRequestException
{
if (!isAssignableTo(keyspace, receiver))
-throw new InvalidRequestException(String.format("Invalid %s
constant (%s) for %s of type %s", type, text, receiver,
receiver.type.asCQL3Type()));
+throw new InvalidRequestException(String.format("Invalid %s
constant (%s) for \"%s\" of type %s", type, text, receiver.name,
receiver.type.asCQL3Type()));
return new Value(parsedValue(receiver.type));
}
[jira] [Reopened] (CASSANDRA-6694) Slightly More Off-Heap Memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict reopened CASSANDRA-6694: - > Slightly More Off-Heap Memtables > > > Key: CASSANDRA-6694 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6694 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Assignee: Benedict > Labels: performance > Fix For: 2.1 beta2 > > Attachments: 6694.fix1.txt > > > The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as > the on-heap overhead is still very large. It should not be tremendously > difficult to extend these changes so that we allocate entire Cells off-heap, > instead of multiple BBs per Cell (with all their associated overhead). > The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 > bytes per cell on average for the btree overhead, for a total overhead of > around 20-22 bytes). This translates to 8-byte object overhead, 4-byte > address (we will do alignment tricks like the VM to allow us to address a > reasonably large memory space, although this trick is unlikely to last us > forever, at which point we will have to bite the bullet and accept a 24-byte > per cell overhead), and 4-byte object reference for maintaining our internal > list of allocations, which is unfortunately necessary since we cannot safely > (and cheaply) walk the object graph we allocate otherwise, which is necessary > for (allocation-) compaction and pointer rewriting. > The ugliest thing here is going to be implementing the various CellName > instances so that they may be backed by native memory OR heap memory. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7067) Refuse CAS batch that have a 'USING TIMESTAMP'
[
https://issues.apache.org/jira/browse/CASSANDRA-7067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Stepura updated CASSANDRA-7067:
---
Labels: lhf (was: )
> Refuse CAS batch that have a 'USING TIMESTAMP'
> --
>
> Key: CASSANDRA-7067
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7067
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Mikhail Stepura
>Priority: Minor
> Labels: lhf
> Fix For: 2.0.8, 2.1 beta2
>
>
> Cassandra must refuse BATCHes with {{TIMESTAMP}}, if they contain a CAS
> statement(s). Like this one:
> {code}
> BEGIN BATCH USING TIMESTAMP
> INSERT INTO users (id, firstname, lastname) VALUES (999, 'Jack', 'Sparrow')
> IF NOT EXISTS
> APPLY BATCH
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6546) disablethrift results in unclosed file descriptors
[
https://issues.apache.org/jira/browse/CASSANDRA-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986068#comment-13986068
]
Mikhail Stepura edited comment on CASSANDRA-6546 at 4/30/14 9:01 PM:
-
So, we have to cleanup slector keys in HSHA server before closing the selector
itself.
There is no such issue with the SYNC server, and behavior observed by
[~enigmacurry] in "Scenario 2" happens because his script didn't close the
connections from *the client side*, and all those ESTABLISHED connections are
actually held by the client. As soon as the client(s) close the connections
(using {{pool.dispose()}} in dtest for ex.) those ESTABLISHED will be gone. And
that is correct.
In case of HSHA those ESTABLISHED would become CLOSE_WAIT, and attached patch
cleans them up.
was (Author: mishail):
So, we have to cleanup slector keys in HSHA server before closing the selector
itself.
There is no such issue with the SYNC server, and behavior observed by
[~enigmacurry] in "Scenario 2" happens because his script didn't close the
connections from *the client side*, and all those ESTABLISHED connections are
actually held by the client. As soon as the client(s) close the connection
(using {{pool.dispose()}} in dtest for ex.}} those ESTABLISHED are gone. And
that is correct.
In case of HSHA those ESTABLISHED would become CLOSE_WAIT, and attached patch
cleans them up.
> disablethrift results in unclosed file descriptors
> --
>
> Key: CASSANDRA-6546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6546
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jason Harvey
>Assignee: Mikhail Stepura
>Priority: Minor
> Fix For: 1.2.17, 2.0.8, 2.1 beta2
>
> Attachments: 2014-04-30 12-22-17.png, 2014-04-30 12-22-58.png,
> CASSANDRA-6546-1.2.patch
>
>
> Disabling thrift results in unclosed thrift sockets being left around.
> Steps to reproduce and observe:
> 1. Have a handful of clients connect via thrift.
> 2. Disable thrift.
> 3. Enable thrift, have the clients reconnect.
> 4. Observe netstat or lsof, and you'll find a lot of thrift sockets in
> CLOSE_WAIT state, and they'll never go away.
> * Also verifiable from
> org.apache.cassandra.metrics:type=Client,name=connectedThriftClients MBean.
> What's extra fun about this is the leaked sockets still count towards your
> maximum RPC thread count. As a result, toggling thrift enough times will
> result in an rpc_max_threads number of CLOSED_WAIT sockets, with no new
> clients able to connect.
> This was reproduced with HSHA. I haven't tried it in sync yet.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-6694) Slightly More Off-Heap Memtables
[ https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6694: Attachment: 6694.fix1.txt A bug snuck through - one line missing from the changes to PureJavaCrc32 > Slightly More Off-Heap Memtables > > > Key: CASSANDRA-6694 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6694 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Assignee: Benedict > Labels: performance > Fix For: 2.1 beta2 > > Attachments: 6694.fix1.txt > > > The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as > the on-heap overhead is still very large. It should not be tremendously > difficult to extend these changes so that we allocate entire Cells off-heap, > instead of multiple BBs per Cell (with all their associated overhead). > The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 > bytes per cell on average for the btree overhead, for a total overhead of > around 20-22 bytes). This translates to 8-byte object overhead, 4-byte > address (we will do alignment tricks like the VM to allow us to address a > reasonably large memory space, although this trick is unlikely to last us > forever, at which point we will have to bite the bullet and accept a 24-byte > per cell overhead), and 4-byte object reference for maintaining our internal > list of allocations, which is unfortunately necessary since we cannot safely > (and cheaply) walk the object graph we allocate otherwise, which is necessary > for (allocation-) compaction and pointer rewriting. > The ugliest thing here is going to be implementing the various CellName > instances so that they may be backed by native memory OR heap memory. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7125) Fail to start by default if Commit Log fails to validate any messages
Benedict created CASSANDRA-7125: --- Summary: Fail to start by default if Commit Log fails to validate any messages Key: CASSANDRA-7125 URL: https://issues.apache.org/jira/browse/CASSANDRA-7125 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Benedict Assignee: Benedict Priority: Minor Fix For: 2.1.0 Current behaviour can be pretty dangerous, and also has a tendency to mask bugs during development. We should change the behaviour to default to failure if anything unexpected happens, and introduce a cassandra.yaml option that permits overriding the default behaviour. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7124) Use JMX Notifications to Indicate Success/Failure of Long-Running Operations
[
https://issues.apache.org/jira/browse/CASSANDRA-7124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-7124:
--
Labels: lhf (was: )
> Use JMX Notifications to Indicate Success/Failure of Long-Running Operations
>
>
> Key: CASSANDRA-7124
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7124
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Tyler Hobbs
>Priority: Minor
> Labels: lhf
>
> If {{nodetool cleanup}} or some other long-running operation takes too long
> to complete, you'll see an error like the one in CASSANDRA-2126, so you can't
> tell if the operation completed successfully or not. CASSANDRA-4767 fixed
> this for repairs with JMX notifications. We should do something similar for
> nodetool cleanup, compact, decommission, move, relocate, etc.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6959) Reusing Keyspace and CF names raises assertion errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986071#comment-13986071
]
Benedict commented on CASSANDRA-6959:
-
It dies before I get there:
{noformat}
21:36:05.851 [Thrift:1] ERROR o.a.c.thrift.CustomTThreadPoolServer - Error
occurred during processing of message.
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_45]
at java.util.ArrayList.set(ArrayList.java:426) ~[na:1.7.0_45]
at org.apache.cassandra.config.CFMetaData.rebuild(CFMetaData.java:2037)
~[cassandra/:na]
at
org.apache.cassandra.config.CFMetaData.fromThriftForUpdate(CFMetaData.java:1036)
~[cassandra/:na]
at
org.apache.cassandra.thrift.CassandraServer.system_update_column_family(CassandraServer.java:1668)
~[cassandra/:na]
at
org.apache.cassandra.thrift.Cassandra$Processor$system_update_column_family.getResult(Cassandra.java:4430)
~[cassandra/:na]
at
org.apache.cassandra.thrift.Cassandra$Processor$system_update_column_family.getResult(Cassandra.java:4414)
~[cassandra/:na]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
~[libthrift-0.9.1.jar:0.9.1]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
~[libthrift-0.9.1.jar:0.9.1]
at
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:201)
~[cassandra/:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
{noformat}
It looks sufficiently like it may be pycassa test related that I haven't
investigated further.
> Reusing Keyspace and CF names raises assertion errors
> -
>
> Key: CASSANDRA-6959
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6959
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Benedict
> Fix For: 2.1 beta2
>
> Attachments: 6959.txt
>
>
> The [dtest I
> introduced|https://github.com/riptano/cassandra-dtest/commit/36960090d219ab8dbc7f108faa91c3ea5cea2bec]
> to test CASSANDRA-6924 introduces some log errors which I think may be
> related to CASSANDRA-5202.
> On 2.1 :
> {code}
> ERROR [MigrationStage:1] 2014-03-31 14:36:43,463
> CommitLogSegmentManager.java:306 - Failed waiting for a forced recycle of
> in-use commit log segments
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManager.forceRecycleAll(CommitLogSegmentManager.java:301)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.forceRecycleAllSegments(CommitLog.java:160)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.dropColumnFamily(DefsTables.java:497)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:296)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:181) [main/:na]
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49)
> [main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {code}
> On 2.0:
> {code}
> ERROR [ReadStage:3] 2014-03-31 13:28:11,014 CassandraDaemon.java (line 198)
> Exception in thread Thread[ReadStage:3,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.db.filter.ExtendedFilter$WithClauses.getExtraFilter(ExtendedFilter.java:258)
> at
> org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1744)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
> at
> org.apache.cassandra.db.PagedRangeCommand.executeLocally(PagedRangeCommand.java:119)
> at
> org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:39)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>
[jira] [Created] (CASSANDRA-7124) Use JMX Notifications to Indicate Success/Failure of Long-Running Operations
Tyler Hobbs created CASSANDRA-7124:
--
Summary: Use JMX Notifications to Indicate Success/Failure of
Long-Running Operations
Key: CASSANDRA-7124
URL: https://issues.apache.org/jira/browse/CASSANDRA-7124
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Tyler Hobbs
Priority: Minor
If {{nodetool cleanup}} or some other long-running operation takes too long to
complete, you'll see an error like the one in CASSANDRA-2126, so you can't tell
if the operation completed successfully or not. CASSANDRA-4767 fixed this for
repairs with JMX notifications. We should do something similar for nodetool
cleanup, compact, decommission, move, relocate, etc.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-6546) disablethrift results in unclosed file descriptors
[
https://issues.apache.org/jira/browse/CASSANDRA-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Stepura updated CASSANDRA-6546:
---
Attachment: CASSANDRA-6546-1.2.patch
So, we have to cleanup slector keys in HSHA server before closing the selector
itself.
There is no such issue with the SYNC server, and behavior observed by
[~enigmacurry] in "Scenario 2" happens because his script didn't close the
connections from *the client side*, and all those ESTABLISHED connections are
actually held by the client. As soon as the client(s) close the connection
(using {{pool.dispose()}} in dtest for ex.}} those ESTABLISHED are gone. And
that is correct.
In case of HSHA those ESTABLISHED would become CLOSE_WAIT, and attached patch
cleans them up.
> disablethrift results in unclosed file descriptors
> --
>
> Key: CASSANDRA-6546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6546
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jason Harvey
>Assignee: Mikhail Stepura
>Priority: Minor
> Attachments: 2014-04-30 12-22-17.png, 2014-04-30 12-22-58.png,
> CASSANDRA-6546-1.2.patch
>
>
> Disabling thrift results in unclosed thrift sockets being left around.
> Steps to reproduce and observe:
> 1. Have a handful of clients connect via thrift.
> 2. Disable thrift.
> 3. Enable thrift, have the clients reconnect.
> 4. Observe netstat or lsof, and you'll find a lot of thrift sockets in
> CLOSE_WAIT state, and they'll never go away.
> * Also verifiable from
> org.apache.cassandra.metrics:type=Client,name=connectedThriftClients MBean.
> What's extra fun about this is the leaked sockets still count towards your
> maximum RPC thread count. As a result, toggling thrift enough times will
> result in an rpc_max_threads number of CLOSED_WAIT sockets, with no new
> clients able to connect.
> This was reproduced with HSHA. I haven't tried it in sync yet.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7122) Replacement nodes have null entries in system.peers
[
https://issues.apache.org/jira/browse/CASSANDRA-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986070#comment-13986070
]
Brandon Williams commented on CASSANDRA-7122:
-
I think the problem is that SS onChange does track and update changes to these
properties, but in a replace none of them ever change so they don't get updated
in the peers table. We should probably just update everything wholesale in
handleStateNormal.
> Replacement nodes have null entries in system.peers
> ---
>
> Key: CASSANDRA-7122
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7122
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>
> If a node is replaced with -Dcassandra.replace_address, the new node has
> mostly null entries in system.peers:
> {code}
> > select * from system.peers;
> peer | data_center | host_id | rack | release_version | rpc_address |
> schema_version | tokens
> ---+-+-+--+-+-++--
> 127.0.0.3 |null |null | null |null |null |
> null | {'-3074457345618258602'}
> {code}
> To reproduce, simply kill a node and replace it. The entries are correctly
> populated if the replacement node is restarted but they are never populated
> if it isn't.
> I can think of at least two bad consequences of this:
> 1. Drivers like Datastax java-driver use the peers table to find the
> rpc_address and location info of a node. If the entires are null it assumes
> rpc_address=ip and the node is in the local DC.
> 2. When using GossipingPropertyFileSnitch and node won't persist the DC/rack
> of another node so may not be able to locate it during restarts.
> I reproduced in 1.2.15 but from inspection it looks to be present in 1.2.16
> and 2.0.7.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986065#comment-13986065 ] Benedict commented on CASSANDRA-4718: - Yes, typo - corrected. Yeah, just straight up read or write requests (I can push the same for both, but they tank for writes when we start hitting compaction/flush etc). I'm being as conservative as possible and assuming every core is spending every moment working on one of the ops (in reality it's more like half of that time). I don't have CASSANDRA-7061 yet to have any really accurate numbers to play with. As regards costs for unpark(), I've timed them in the past and that's in the ball park of what you'd expect given the literature and general OS behaviour (10us is probably a bit heavier than they often clock in, but a good figure to work with) > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op > costs of various queues.ods, stress op rate with various queues.ods, > v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986048#comment-13986048 ] Benedict edited comment on CASSANDRA-4718 at 4/30/14 8:48 PM: -- Sure. My box can push around 60Kop/s - this translates to around 60us/op (core time), when unpark() clocks in around 10us, you want to avoid it, _especially_ when you have network streams blocked on the consumer that is handing off the work. was (Author: benedict): Sure. My box can push around 60Kop/s - this translates to around 60us/up (core time), when unpark() clocks in around 10us, you want to avoid it, _especially_ when you have network streams blocked on the consumer that is handing off the work. > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op > costs of various queues.ods, stress op rate with various queues.ods, > v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986060#comment-13986060 ] Jason Brown commented on CASSANDRA-4718: bq. translates to around 60us/up Did you mean 60us/*op* ? Also, are these just requests from (new) cassandra stress? You've described the time to process each 'tiny message' but not what a tiny message is :). How are you measuring the time for each request (more for my own curiosity)? > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op > costs of various queues.ods, stress op rate with various queues.ods, > v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7122) Replacement nodes have null entries in system.peers
[
https://issues.apache.org/jira/browse/CASSANDRA-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986057#comment-13986057
]
Brandon Williams commented on CASSANDRA-7122:
-
bq. Is it safe to special case hibernate and persist the peer info? Dead states
were ignored by CASSANDRA-6053, maybe that was overly restrictive?
No, CASSANDRA-6053 is why replace can work at all today. Without digging in
though, the node jumping to normal when the replace is complete *should* fire
the events for SS that eventually update this, so that's the place to look.
> Replacement nodes have null entries in system.peers
> ---
>
> Key: CASSANDRA-7122
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7122
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>
> If a node is replaced with -Dcassandra.replace_address, the new node has
> mostly null entries in system.peers:
> {code}
> > select * from system.peers;
> peer | data_center | host_id | rack | release_version | rpc_address |
> schema_version | tokens
> ---+-+-+--+-+-++--
> 127.0.0.3 |null |null | null |null |null |
> null | {'-3074457345618258602'}
> {code}
> To reproduce, simply kill a node and replace it. The entries are correctly
> populated if the replacement node is restarted but they are never populated
> if it isn't.
> I can think of at least two bad consequences of this:
> 1. Drivers like Datastax java-driver use the peers table to find the
> rpc_address and location info of a node. If the entires are null it assumes
> rpc_address=ip and the node is in the local DC.
> 2. When using GossipingPropertyFileSnitch and node won't persist the DC/rack
> of another node so may not be able to locate it during restarts.
> I reproduced in 1.2.15 but from inspection it looks to be present in 1.2.16
> and 2.0.7.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986048#comment-13986048 ] Benedict commented on CASSANDRA-4718: - Sure. My box can push around 60Kop/s - this translates to around 60us/up (core time), when unpark() clocks in around 10us, you want to avoid it. > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op > costs of various queues.ods, stress op rate with various queues.ods, > v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986048#comment-13986048 ] Benedict edited comment on CASSANDRA-4718 at 4/30/14 8:40 PM: -- Sure. My box can push around 60Kop/s - this translates to around 60us/up (core time), when unpark() clocks in around 10us, you want to avoid it, _especially_ when you have network streams blocked on the consumer that is handing off the work. was (Author: benedict): Sure. My box can push around 60Kop/s - this translates to around 60us/up (core time), when unpark() clocks in around 10us, you want to avoid it. > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op > costs of various queues.ods, stress op rate with various queues.ods, > v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986040#comment-13986040 ] Jason Brown commented on CASSANDRA-4718: [~benedict] Can you clarify what you mean by "when dealing with a flood of tiny messages"? > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op > costs of various queues.ods, stress op rate with various queues.ods, > v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986039#comment-13986039 ] Benedict commented on CASSANDRA-4718: - bq. Perhaps, but the overall improvement in performance (should we attribute that to the work stealing?) seems compelling enough. I'm not suggesting we forego the patch because of this concern, I'm raising it as something to bear in mind for the future. As I said, though, to some extent I have addressed this concern with the _lowsignal_ patch I uploaded, although it's debatable how elegant that approach is. bq. I didn't consider this a 'fork' as we're not mucking about with the internals of the FJP itself. Perhaps we're getting crossed wires and mixing up the patches I have uploaded (no fork), with the suggestion that we _may want to investigate forking in future_ in order to address these issues in a more elegant manner. > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op > costs of various queues.ods, stress op rate with various queues.ods, > v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7123) BATCH documentation should be explicit about ordering guarantees
[
https://issues.apache.org/jira/browse/CASSANDRA-7123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986036#comment-13986036
]
Alex P commented on CASSANDRA-7123:
---
Current:
bq. Note however that the BATCH statement only allows UPDATE, INSERT and DELETE
statements and is not a full analogue for SQL transactions.
Suggested edit:
{quote}
Note that:
1. the BATCH statement only allows UPDATE, INSERT and DELETE statements
2. it is *not* a full analogue for SQL transactions
3. there is *no* guaranteed order of applying the operations (even for the same
partition key). To achieve a particular ordering of operations you must specify
timestamps.
{quote}
> BATCH documentation should be explicit about ordering guarantees
>
>
> Key: CASSANDRA-7123
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7123
> Project: Cassandra
> Issue Type: Task
> Components: Documentation & website
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
>Priority: Minor
>
> In the CQL3 [batch statement
> documentation](http://cassandra.apache.org/doc/cql3/CQL.html#batchStmt) we
> don't mention that there are no ordering guarantees, which can lead to
> somewhat surprising behavior (CASSANDRA-6291).
> We should also mention that you could specify timestamps in order to achieve
> a particular ordering.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7122) Replacement nodes have null entries in system.peers
[
https://issues.apache.org/jira/browse/CASSANDRA-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986029#comment-13986029
]
Richard Low edited comment on CASSANDRA-7122 at 4/30/14 8:30 PM:
-
It looks to be caused by StorageService.onChange ignoring state changes when
%{code}Gossiper.instance.isDeadState(epState)%{/code}. When a replacement node
comes up, it is in the hibernate state, which is counted as dead. Since the
info like DC and rack never changes, it is never resent so is never persisted
to system.peers.
Is it safe to special case hibernate and persist the peer info? Dead states
were ignored by CASSANDRA-6053, maybe that was overly restrictive?
was (Author: rlow):
It looks to be caused by StorageService.onChange ignoring state changes when
%{code}Gossiper.instance.isDeadState(epState)%{code}. When a replacement node
comes up, it is in the hibernate state, which is counted as dead. Since the
info like DC and rack never changes, it is never resent so is never persisted
to system.peers.
Is it safe to special case hibernate and persist the peer info? Dead states
were ignored by CASSANDRA-6053, maybe that was overly restrictive?
> Replacement nodes have null entries in system.peers
> ---
>
> Key: CASSANDRA-7122
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7122
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>
> If a node is replaced with -Dcassandra.replace_address, the new node has
> mostly null entries in system.peers:
> {code}
> > select * from system.peers;
> peer | data_center | host_id | rack | release_version | rpc_address |
> schema_version | tokens
> ---+-+-+--+-+-++--
> 127.0.0.3 |null |null | null |null |null |
> null | {'-3074457345618258602'}
> {code}
> To reproduce, simply kill a node and replace it. The entries are correctly
> populated if the replacement node is restarted but they are never populated
> if it isn't.
> I can think of at least two bad consequences of this:
> 1. Drivers like Datastax java-driver use the peers table to find the
> rpc_address and location info of a node. If the entires are null it assumes
> rpc_address=ip and the node is in the local DC.
> 2. When using GossipingPropertyFileSnitch and node won't persist the DC/rack
> of another node so may not be able to locate it during restarts.
> I reproduced in 1.2.15 but from inspection it looks to be present in 1.2.16
> and 2.0.7.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6959) Reusing Keyspace and CF names raises assertion errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986030#comment-13986030
]
Tyler Hobbs commented on CASSANDRA-6959:
bq. I can run dtests just fine (using simple "nosetests"...) I am a python
packaging n00b, though, so maybe that is irrelevant, or the syntax means
there's something subtly different about the nosetest setup?
Oh, you should able to just use "nosetests" here, sorry. The setup.py version
will make a fresh build of the package and use it (instead of potentially using
a system-installed version), so I'm in the habit of using that, but it's not
needed here.
> Reusing Keyspace and CF names raises assertion errors
> -
>
> Key: CASSANDRA-6959
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6959
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Benedict
> Fix For: 2.1 beta2
>
> Attachments: 6959.txt
>
>
> The [dtest I
> introduced|https://github.com/riptano/cassandra-dtest/commit/36960090d219ab8dbc7f108faa91c3ea5cea2bec]
> to test CASSANDRA-6924 introduces some log errors which I think may be
> related to CASSANDRA-5202.
> On 2.1 :
> {code}
> ERROR [MigrationStage:1] 2014-03-31 14:36:43,463
> CommitLogSegmentManager.java:306 - Failed waiting for a forced recycle of
> in-use commit log segments
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManager.forceRecycleAll(CommitLogSegmentManager.java:301)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.forceRecycleAllSegments(CommitLog.java:160)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.dropColumnFamily(DefsTables.java:497)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:296)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:181) [main/:na]
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49)
> [main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {code}
> On 2.0:
> {code}
> ERROR [ReadStage:3] 2014-03-31 13:28:11,014 CassandraDaemon.java (line 198)
> Exception in thread Thread[ReadStage:3,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.db.filter.ExtendedFilter$WithClauses.getExtraFilter(ExtendedFilter.java:258)
> at
> org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1744)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
> at
> org.apache.cassandra.db.PagedRangeCommand.executeLocally(PagedRangeCommand.java:119)
> at
> org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:39)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> To reproduce, you many need to comment out the assertion in that test, as it
> is not 100% reproducible on the first try.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7122) Replacement nodes have null entries in system.peers
[
https://issues.apache.org/jira/browse/CASSANDRA-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986029#comment-13986029
]
Richard Low edited comment on CASSANDRA-7122 at 4/30/14 8:30 PM:
-
It looks to be caused by StorageService.onChange ignoring state changes when
{code}Gossiper.instance.isDeadState(epState){code}. When a replacement node
comes up, it is in the hibernate state, which is counted as dead. Since the
info like DC and rack never changes, it is never resent so is never persisted
to system.peers.
Is it safe to special case hibernate and persist the peer info? Dead states
were ignored by CASSANDRA-6053, maybe that was overly restrictive?
was (Author: rlow):
It looks to be caused by StorageService.onChange ignoring state changes when
%{code}Gossiper.instance.isDeadState(epState)%{/code}. When a replacement node
comes up, it is in the hibernate state, which is counted as dead. Since the
info like DC and rack never changes, it is never resent so is never persisted
to system.peers.
Is it safe to special case hibernate and persist the peer info? Dead states
were ignored by CASSANDRA-6053, maybe that was overly restrictive?
> Replacement nodes have null entries in system.peers
> ---
>
> Key: CASSANDRA-7122
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7122
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>
> If a node is replaced with -Dcassandra.replace_address, the new node has
> mostly null entries in system.peers:
> {code}
> > select * from system.peers;
> peer | data_center | host_id | rack | release_version | rpc_address |
> schema_version | tokens
> ---+-+-+--+-+-++--
> 127.0.0.3 |null |null | null |null |null |
> null | {'-3074457345618258602'}
> {code}
> To reproduce, simply kill a node and replace it. The entries are correctly
> populated if the replacement node is restarted but they are never populated
> if it isn't.
> I can think of at least two bad consequences of this:
> 1. Drivers like Datastax java-driver use the peers table to find the
> rpc_address and location info of a node. If the entires are null it assumes
> rpc_address=ip and the node is in the local DC.
> 2. When using GossipingPropertyFileSnitch and node won't persist the DC/rack
> of another node so may not be able to locate it during restarts.
> I reproduced in 1.2.15 but from inspection it looks to be present in 1.2.16
> and 2.0.7.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986032#comment-13986032 ] Jason Brown commented on CASSANDRA-4718: bq. The Semaphore is blocking After reading the correct jdk source class this time, I was mistaken about the blocking (i.e. park) aspect of Semaphore (got caught up in the rest of AbstractQueuedSynchronizer, which Semaphore subclasses and uses internally). Thus, I'll test out your branch as is this afternoon. bq. It isn't forked - this is all in the same extension class that you introduced I didn't consider this a 'fork' as we're not mucking about with the internals of the FJP itself. bq. FJP uses an exclusive lock for enqueueing work onto the pool, but does more whilst owning the lock, so is likely to take longer within the critical section Perhaps, but the overall improvement in performance (should we attribute that to the work stealing?) seems compelling enough. > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op > costs of various queues.ods, stress op rate with various queues.ods, > v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7122) Replacement nodes have null entries in system.peers
[
https://issues.apache.org/jira/browse/CASSANDRA-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986029#comment-13986029
]
Richard Low commented on CASSANDRA-7122:
It looks to be caused by StorageService.onChange ignoring state changes when
%{code}Gossiper.instance.isDeadState(epState)%{code}. When a replacement node
comes up, it is in the hibernate state, which is counted as dead. Since the
info like DC and rack never changes, it is never resent so is never persisted
to system.peers.
Is it safe to special case hibernate and persist the peer info? Dead states
were ignored by CASSANDRA-6053, maybe that was overly restrictive?
> Replacement nodes have null entries in system.peers
> ---
>
> Key: CASSANDRA-7122
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7122
> Project: Cassandra
> Issue Type: Bug
>Reporter: Richard Low
>
> If a node is replaced with -Dcassandra.replace_address, the new node has
> mostly null entries in system.peers:
> {code}
> > select * from system.peers;
> peer | data_center | host_id | rack | release_version | rpc_address |
> schema_version | tokens
> ---+-+-+--+-+-++--
> 127.0.0.3 |null |null | null |null |null |
> null | {'-3074457345618258602'}
> {code}
> To reproduce, simply kill a node and replace it. The entries are correctly
> populated if the replacement node is restarted but they are never populated
> if it isn't.
> I can think of at least two bad consequences of this:
> 1. Drivers like Datastax java-driver use the peers table to find the
> rpc_address and location info of a node. If the entires are null it assumes
> rpc_address=ip and the node is in the local DC.
> 2. When using GossipingPropertyFileSnitch and node won't persist the DC/rack
> of another node so may not be able to locate it during restarts.
> I reproduced in 1.2.15 but from inspection it looks to be present in 1.2.16
> and 2.0.7.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7123) BATCH documentation should be explicit about ordering guarantees
Tyler Hobbs created CASSANDRA-7123: -- Summary: BATCH documentation should be explicit about ordering guarantees Key: CASSANDRA-7123 URL: https://issues.apache.org/jira/browse/CASSANDRA-7123 Project: Cassandra Issue Type: Task Components: Documentation & website Reporter: Tyler Hobbs Assignee: Tyler Hobbs Priority: Minor In the CQL3 [batch statement documentation](http://cassandra.apache.org/doc/cql3/CQL.html#batchStmt) we don't mention that there are no ordering guarantees, which can lead to somewhat surprising behavior (CASSANDRA-6291). We should also mention that you could specify timestamps in order to achieve a particular ordering. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7122) Replacement nodes have null entries in system.peers
Richard Low created CASSANDRA-7122:
--
Summary: Replacement nodes have null entries in system.peers
Key: CASSANDRA-7122
URL: https://issues.apache.org/jira/browse/CASSANDRA-7122
Project: Cassandra
Issue Type: Bug
Reporter: Richard Low
If a node is replaced with -Dcassandra.replace_address, the new node has mostly
null entries in system.peers:
{code}
> select * from system.peers;
peer | data_center | host_id | rack | release_version | rpc_address |
schema_version | tokens
---+-+-+--+-+-++--
127.0.0.3 |null |null | null |null |null |
null | {'-3074457345618258602'}
{code}
To reproduce, simply kill a node and replace it. The entries are correctly
populated if the replacement node is restarted but they are never populated if
it isn't.
I can think of at least two bad consequences of this:
1. Drivers like Datastax java-driver use the peers table to find the
rpc_address and location info of a node. If the entires are null it assumes
rpc_address=ip and the node is in the local DC.
2. When using GossipingPropertyFileSnitch and node won't persist the DC/rack of
another node so may not be able to locate it during restarts.
I reproduced in 1.2.15 but from inspection it looks to be present in 1.2.16 and
2.0.7.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6959) Reusing Keyspace and CF names raises assertion errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986008#comment-13986008
]
Benedict commented on CASSANDRA-6959:
-
I can run dtests just fine (using simple "nosetests"...) I am a python
packaging n00b, though, so maybe that is irrelevant, or the syntax means
there's something subtly different about the nosetest setup?
> Reusing Keyspace and CF names raises assertion errors
> -
>
> Key: CASSANDRA-6959
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6959
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Benedict
> Fix For: 2.1 beta2
>
> Attachments: 6959.txt
>
>
> The [dtest I
> introduced|https://github.com/riptano/cassandra-dtest/commit/36960090d219ab8dbc7f108faa91c3ea5cea2bec]
> to test CASSANDRA-6924 introduces some log errors which I think may be
> related to CASSANDRA-5202.
> On 2.1 :
> {code}
> ERROR [MigrationStage:1] 2014-03-31 14:36:43,463
> CommitLogSegmentManager.java:306 - Failed waiting for a forced recycle of
> in-use commit log segments
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManager.forceRecycleAll(CommitLogSegmentManager.java:301)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.forceRecycleAllSegments(CommitLog.java:160)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.dropColumnFamily(DefsTables.java:497)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:296)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:181) [main/:na]
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49)
> [main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {code}
> On 2.0:
> {code}
> ERROR [ReadStage:3] 2014-03-31 13:28:11,014 CassandraDaemon.java (line 198)
> Exception in thread Thread[ReadStage:3,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.db.filter.ExtendedFilter$WithClauses.getExtraFilter(ExtendedFilter.java:258)
> at
> org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1744)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
> at
> org.apache.cassandra.db.PagedRangeCommand.executeLocally(PagedRangeCommand.java:119)
> at
> org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:39)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> To reproduce, you many need to comment out the assertion in that test, as it
> is not 100% reproducible on the first try.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6959) Reusing Keyspace and CF names raises assertion errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986005#comment-13986005
]
Brandon Williams commented on CASSANDRA-6959:
-
You need to install the nose module for python.
> Reusing Keyspace and CF names raises assertion errors
> -
>
> Key: CASSANDRA-6959
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6959
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Benedict
> Fix For: 2.1 beta2
>
> Attachments: 6959.txt
>
>
> The [dtest I
> introduced|https://github.com/riptano/cassandra-dtest/commit/36960090d219ab8dbc7f108faa91c3ea5cea2bec]
> to test CASSANDRA-6924 introduces some log errors which I think may be
> related to CASSANDRA-5202.
> On 2.1 :
> {code}
> ERROR [MigrationStage:1] 2014-03-31 14:36:43,463
> CommitLogSegmentManager.java:306 - Failed waiting for a forced recycle of
> in-use commit log segments
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManager.forceRecycleAll(CommitLogSegmentManager.java:301)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.forceRecycleAllSegments(CommitLog.java:160)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.dropColumnFamily(DefsTables.java:497)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:296)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:181) [main/:na]
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49)
> [main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {code}
> On 2.0:
> {code}
> ERROR [ReadStage:3] 2014-03-31 13:28:11,014 CassandraDaemon.java (line 198)
> Exception in thread Thread[ReadStage:3,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.db.filter.ExtendedFilter$WithClauses.getExtraFilter(ExtendedFilter.java:258)
> at
> org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1744)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
> at
> org.apache.cassandra.db.PagedRangeCommand.executeLocally(PagedRangeCommand.java:119)
> at
> org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:39)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> To reproduce, you many need to comment out the assertion in that test, as it
> is not 100% reproducible on the first try.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[
https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13986003#comment-13986003
]
Jeremiah Jordan commented on CASSANDRA-4718:
bq. I can argue from running cassandra in production for almost four years that
these metrics are not very helpful. At best they indicate that 'something' is
amiss ("hey, pending tasks are getting higher"), but cannot give you a real
clue as to what is wrong (gc, I/O contention, cpu throttling). As we got these
data points largely for free from TPE, I guess it made sense to expose them,
but if we have to go out of our way to fabricate a subset of them for FJP, I
propose we drop them going forward (for FJP, at least).
I would argue they are very useful because they give you that high level
"something is wrong", so if its easy to keep them, I am very +1 on that.
> More-efficient ExecutorService for improved throughput
> --
>
> Key: CASSANDRA-4718
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4718
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Jason Brown
>Priority: Minor
> Labels: performance
> Fix For: 2.1.0
>
> Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op
> costs of various queues.ods, stress op rate with various queues.ods,
> v1-stress.out
>
>
> Currently all our execution stages dequeue tasks one at a time. This can
> result in contention between producers and consumers (although we do our best
> to minimize this by using LinkedBlockingQueue).
> One approach to mitigating this would be to make consumer threads do more
> work in "bulk" instead of just one task per dequeue. (Producer threads tend
> to be single-task oriented by nature, so I don't see an equivalent
> opportunity there.)
> BlockingQueue has a drainTo(collection, int) method that would be perfect for
> this. However, no ExecutorService in the jdk supports using drainTo, nor
> could I google one.
> What I would like to do here is create just such a beast and wire it into (at
> least) the write and read stages. (Other possible candidates for such an
> optimization, such as the CommitLog and OutboundTCPConnection, are not
> ExecutorService-based and will need to be one-offs.)
> AbstractExecutorService may be useful. The implementations of
> ICommitLogExecutorService may also be useful. (Despite the name these are not
> actual ExecutorServices, although they share the most important properties of
> one.)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6959) Reusing Keyspace and CF names raises assertion errors
[
https://issues.apache.org/jira/browse/CASSANDRA-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985988#comment-13985988
]
Benedict commented on CASSANDRA-6959:
-
I get:
{noformat}
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: invalid command 'nosetests'
{noformat}
> Reusing Keyspace and CF names raises assertion errors
> -
>
> Key: CASSANDRA-6959
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6959
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Benedict
> Fix For: 2.1 beta2
>
> Attachments: 6959.txt
>
>
> The [dtest I
> introduced|https://github.com/riptano/cassandra-dtest/commit/36960090d219ab8dbc7f108faa91c3ea5cea2bec]
> to test CASSANDRA-6924 introduces some log errors which I think may be
> related to CASSANDRA-5202.
> On 2.1 :
> {code}
> ERROR [MigrationStage:1] 2014-03-31 14:36:43,463
> CommitLogSegmentManager.java:306 - Failed waiting for a forced recycle of
> in-use commit log segments
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManager.forceRecycleAll(CommitLogSegmentManager.java:301)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.forceRecycleAllSegments(CommitLog.java:160)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.dropColumnFamily(DefsTables.java:497)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:296)
> [main/:na]
> at
> org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:181) [main/:na]
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49)
> [main/:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> [main/:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_51]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
> {code}
> On 2.0:
> {code}
> ERROR [ReadStage:3] 2014-03-31 13:28:11,014 CassandraDaemon.java (line 198)
> Exception in thread Thread[ReadStage:3,5,main]
> java.lang.AssertionError
> at
> org.apache.cassandra.db.filter.ExtendedFilter$WithClauses.getExtraFilter(ExtendedFilter.java:258)
> at
> org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1744)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1699)
> at
> org.apache.cassandra.db.PagedRangeCommand.executeLocally(PagedRangeCommand.java:119)
> at
> org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:39)
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> To reproduce, you many need to comment out the assertion in that test, as it
> is not 100% reproducible on the first try.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985978#comment-13985978 ] Benedict commented on CASSANDRA-4718: - bq. . The Semaphore is blocking (by design) It's non-blocking until you run out of permits, at which point it must block. We have _many more_ shared counters than this semaphore, so I highly doubt it will be an issue (if doing nothing but spinning on updating it we could push probably several thousand times our current op-rate, and in reality we will be doing a lot inbetween, so contention is highly unlikely to be an issue, although it will incur a slight QPI penalty - nothing we don't incur all over the place though). bq. but any solution is better than forking FJP It isn't forked - this is all in the same extension class that you introduced...? bq. I literally have no idea what this means. FJP uses an exclusive lock for enqueueing work onto the pool, but does more whilst owning the lock, so is likely to take longer within the critical section. The second patch I uploaded attempts to mitigate this for native transport threads as those micros are actually a pretty big deal when dealing with a flood of tiny messages. bq. As we got these data points largely for free from TPE, I guess it made sense to expose them, but if we have to go out of our way to fabricate a subset of them for FJP, I propose we drop them going forward (for FJP, at least). I don't really mind, but I think you're overestimating the penalty for maintaining these counters. > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op > costs of various queues.ods, stress op rate with various queues.ods, > v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share the most important properties of > one.) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985978#comment-13985978 ] Benedict edited comment on CASSANDRA-4718 at 4/30/14 7:46 PM: -- bq. . The Semaphore is blocking (by design) It's non-blocking until you run out of permits, at which point it must block. We have _many more_ shared counters than this semaphore, so I highly doubt it will be an issue (if doing nothing but spinning on updating it we could push probably several thousand times our current op-rate, and in reality we will be doing a lot inbetween, so contention is highly unlikely to be an issue, although it will incur a slight QPI penalty - nothing we don't incur all over the place though). That said, I have nothing against only conditionally creating the Semaphore which would eliminate it as a cost anywhere it isn't necessary. bq. but any solution is better than forking FJP It isn't forked - this is all in the same extension class that you introduced...? bq. I literally have no idea what this means. FJP uses an exclusive lock for enqueueing work onto the pool, but does more whilst owning the lock, so is likely to take longer within the critical section. The second patch I uploaded attempts to mitigate this for native transport threads as those micros are actually a pretty big deal when dealing with a flood of tiny messages. bq. As we got these data points largely for free from TPE, I guess it made sense to expose them, but if we have to go out of our way to fabricate a subset of them for FJP, I propose we drop them going forward (for FJP, at least). I don't really mind, but I think you're overestimating the penalty for maintaining these counters. was (Author: benedict): bq. . The Semaphore is blocking (by design) It's non-blocking until you run out of permits, at which point it must block. We have _many more_ shared counters than this semaphore, so I highly doubt it will be an issue (if doing nothing but spinning on updating it we could push probably several thousand times our current op-rate, and in reality we will be doing a lot inbetween, so contention is highly unlikely to be an issue, although it will incur a slight QPI penalty - nothing we don't incur all over the place though). bq. but any solution is better than forking FJP It isn't forked - this is all in the same extension class that you introduced...? bq. I literally have no idea what this means. FJP uses an exclusive lock for enqueueing work onto the pool, but does more whilst owning the lock, so is likely to take longer within the critical section. The second patch I uploaded attempts to mitigate this for native transport threads as those micros are actually a pretty big deal when dealing with a flood of tiny messages. bq. As we got these data points largely for free from TPE, I guess it made sense to expose them, but if we have to go out of our way to fabricate a subset of them for FJP, I propose we drop them going forward (for FJP, at least). I don't really mind, but I think you're overestimating the penalty for maintaining these counters. > More-efficient ExecutorService for improved throughput > -- > > Key: CASSANDRA-4718 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4718 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: Jason Brown >Priority: Minor > Labels: performance > Fix For: 2.1.0 > > Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op > costs of various queues.ods, stress op rate with various queues.ods, > v1-stress.out > > > Currently all our execution stages dequeue tasks one at a time. This can > result in contention between producers and consumers (although we do our best > to minimize this by using LinkedBlockingQueue). > One approach to mitigating this would be to make consumer threads do more > work in "bulk" instead of just one task per dequeue. (Producer threads tend > to be single-task oriented by nature, so I don't see an equivalent > opportunity there.) > BlockingQueue has a drainTo(collection, int) method that would be perfect for > this. However, no ExecutorService in the jdk supports using drainTo, nor > could I google one. > What I would like to do here is create just such a beast and wire it into (at > least) the write and read stages. (Other possible candidates for such an > optimization, such as the CommitLog and OutboundTCPConnection, are not > ExecutorService-based and will need to be one-offs.) > AbstractExecutorService may be useful. The implementations of > ICommitLogExecutorService may also be useful. (Despite the name these are not > actual ExecutorServices, although they share
[jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
[
https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985962#comment-13985962
]
Jason Brown commented on CASSANDRA-4718:
[~benedict] I agree that FJP does not have a native enqueueing mechanism, but
I'm not sure adding a Semaphore right in the middle of the class is the right
solution. The Semaphore is blocking (by design) and will be contended for
across (numa) cores. As an alternative, at least for the native protocol, does
it make sense to move the back pressure point earlier in the processing chain?
I'm unfamiliar with EventLoopGroup, but any solution is better than forking
FJP. Also, I have an idea I'd like to try out .. give me a day or two.
bq. ... is no more efficient (probably slightly less) than a standard
executor. That's a future problem, however
I literally have no idea what this means.
bq. support the metrics that users may have gotten used to.
I can argue from running cassandra in production for almost four years that
these metrics are not very helpful. At best they indicate that 'something' is
amiss ("hey, pending tasks are getting higher"), but cannot give you a real
clue as to what is wrong (gc, I/O contention, cpu throttling). As we got these
data points largely for free from TPE, I guess it made sense to expose them,
but if we have to go out of our way to fabricate a subset of them for FJP, I
propose we drop them going forward (for FJP, at least).
> More-efficient ExecutorService for improved throughput
> --
>
> Key: CASSANDRA-4718
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4718
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jonathan Ellis
>Assignee: Jason Brown
>Priority: Minor
> Labels: performance
> Fix For: 2.1.0
>
> Attachments: 4718-v1.patch, PerThreadQueue.java, baq vs trunk.png, op
> costs of various queues.ods, stress op rate with various queues.ods,
> v1-stress.out
>
>
> Currently all our execution stages dequeue tasks one at a time. This can
> result in contention between producers and consumers (although we do our best
> to minimize this by using LinkedBlockingQueue).
> One approach to mitigating this would be to make consumer threads do more
> work in "bulk" instead of just one task per dequeue. (Producer threads tend
> to be single-task oriented by nature, so I don't see an equivalent
> opportunity there.)
> BlockingQueue has a drainTo(collection, int) method that would be perfect for
> this. However, no ExecutorService in the jdk supports using drainTo, nor
> could I google one.
> What I would like to do here is create just such a beast and wire it into (at
> least) the write and read stages. (Other possible candidates for such an
> optimization, such as the CommitLog and OutboundTCPConnection, are not
> ExecutorService-based and will need to be one-offs.)
> AbstractExecutorService may be useful. The implementations of
> ICommitLogExecutorService may also be useful. (Despite the name these are not
> actual ExecutorServices, although they share the most important properties of
> one.)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6546) disablethrift results in unclosed file descriptors
[
https://issues.apache.org/jira/browse/CASSANDRA-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985953#comment-13985953
]
Mikhail Stepura edited comment on CASSANDRA-6546 at 4/30/14 7:28 PM:
-
I've reproduced the problem on 1.2 with HSHA, for some reason
{{ThriftSessionManager.connectionComplete(SocketAddress)}} is never called in
this case.
{code}
mstepura-mac:logs mikhail$ lsof -p 84299 | grep CLOSE_WAIT | wc -l
150
{code}
And all those 150 connections are still sitting in ThriftSessionManager
!2014-04-30 12-22-17.png!
!2014-04-30 12-22-58.png!
was (Author: mishail):
For some reason {{ThriftSessionManager.connectionComplete(SocketAddress)}} is
never called in this case.
{code}
mstepura-mac:logs mikhail$ lsof -p 84299 | grep CLOSE_WAIT | wc -l
150
{code}
And all those 150 connections are still sitting in ThriftSessionManager
!2014-04-30 12-22-17.png!
!2014-04-30 12-22-58.png!
> disablethrift results in unclosed file descriptors
> --
>
> Key: CASSANDRA-6546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6546
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jason Harvey
>Assignee: Mikhail Stepura
>Priority: Minor
> Attachments: 2014-04-30 12-22-17.png, 2014-04-30 12-22-58.png
>
>
> Disabling thrift results in unclosed thrift sockets being left around.
> Steps to reproduce and observe:
> 1. Have a handful of clients connect via thrift.
> 2. Disable thrift.
> 3. Enable thrift, have the clients reconnect.
> 4. Observe netstat or lsof, and you'll find a lot of thrift sockets in
> CLOSE_WAIT state, and they'll never go away.
> * Also verifiable from
> org.apache.cassandra.metrics:type=Client,name=connectedThriftClients MBean.
> What's extra fun about this is the leaked sockets still count towards your
> maximum RPC thread count. As a result, toggling thrift enough times will
> result in an rpc_max_threads number of CLOSED_WAIT sockets, with no new
> clients able to connect.
> This was reproduced with HSHA. I haven't tried it in sync yet.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6546) disablethrift results in unclosed file descriptors
[
https://issues.apache.org/jira/browse/CASSANDRA-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985953#comment-13985953
]
Mikhail Stepura edited comment on CASSANDRA-6546 at 4/30/14 7:27 PM:
-
For some reason {{ThriftSessionManager.connectionComplete(SocketAddress)}} is
never called in this case.
{code}
mstepura-mac:logs mikhail$ lsof -p 84299 | grep CLOSE_WAIT | wc -l
150
{code}
And all those 150 connections are still sitting in ThriftSessionManager
!2014-04-30 12-22-17.png!
!2014-04-30 12-22-58.png!
was (Author: mishail):
For some reason {{ThriftSessionManager.connectionComplete(SocketAddress)}} is
never called in this case.
{code}
mstepura-mac:logs mikhail$ lsof -p 84299 | grep CLOSE_WAIT | wc -l
150
{code}
And all those 150 connections are still sitting in ThriftSessionManager
> disablethrift results in unclosed file descriptors
> --
>
> Key: CASSANDRA-6546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6546
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jason Harvey
>Assignee: Mikhail Stepura
>Priority: Minor
> Attachments: 2014-04-30 12-22-17.png, 2014-04-30 12-22-58.png
>
>
> Disabling thrift results in unclosed thrift sockets being left around.
> Steps to reproduce and observe:
> 1. Have a handful of clients connect via thrift.
> 2. Disable thrift.
> 3. Enable thrift, have the clients reconnect.
> 4. Observe netstat or lsof, and you'll find a lot of thrift sockets in
> CLOSE_WAIT state, and they'll never go away.
> * Also verifiable from
> org.apache.cassandra.metrics:type=Client,name=connectedThriftClients MBean.
> What's extra fun about this is the leaked sockets still count towards your
> maximum RPC thread count. As a result, toggling thrift enough times will
> result in an rpc_max_threads number of CLOSED_WAIT sockets, with no new
> clients able to connect.
> This was reproduced with HSHA. I haven't tried it in sync yet.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-6546) disablethrift results in unclosed file descriptors
[ https://issues.apache.org/jira/browse/CASSANDRA-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Stepura updated CASSANDRA-6546: --- Attachment: 2014-04-30 12-22-17.png 2014-04-30 12-22-58.png > disablethrift results in unclosed file descriptors > -- > > Key: CASSANDRA-6546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6546 > Project: Cassandra > Issue Type: Bug >Reporter: Jason Harvey >Assignee: Mikhail Stepura >Priority: Minor > Attachments: 2014-04-30 12-22-17.png, 2014-04-30 12-22-58.png > > > Disabling thrift results in unclosed thrift sockets being left around. > Steps to reproduce and observe: > 1. Have a handful of clients connect via thrift. > 2. Disable thrift. > 3. Enable thrift, have the clients reconnect. > 4. Observe netstat or lsof, and you'll find a lot of thrift sockets in > CLOSE_WAIT state, and they'll never go away. > * Also verifiable from > org.apache.cassandra.metrics:type=Client,name=connectedThriftClients MBean. > What's extra fun about this is the leaked sockets still count towards your > maximum RPC thread count. As a result, toggling thrift enough times will > result in an rpc_max_threads number of CLOSED_WAIT sockets, with no new > clients able to connect. > This was reproduced with HSHA. I haven't tried it in sync yet. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6546) disablethrift results in unclosed file descriptors
[
https://issues.apache.org/jira/browse/CASSANDRA-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985953#comment-13985953
]
Mikhail Stepura commented on CASSANDRA-6546:
For some reason {{ThriftSessionManager.connectionComplete(SocketAddress)}} is
never called in this case.
{code}
mstepura-mac:logs mikhail$ lsof -p 84299 | grep CLOSE_WAIT | wc -l
150
{code}
And all those 150 connections are still sitting in ThriftSessionManager
> disablethrift results in unclosed file descriptors
> --
>
> Key: CASSANDRA-6546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6546
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jason Harvey
>Assignee: Mikhail Stepura
>Priority: Minor
> Attachments: 2014-04-30 12-22-17.png, 2014-04-30 12-22-58.png
>
>
> Disabling thrift results in unclosed thrift sockets being left around.
> Steps to reproduce and observe:
> 1. Have a handful of clients connect via thrift.
> 2. Disable thrift.
> 3. Enable thrift, have the clients reconnect.
> 4. Observe netstat or lsof, and you'll find a lot of thrift sockets in
> CLOSE_WAIT state, and they'll never go away.
> * Also verifiable from
> org.apache.cassandra.metrics:type=Client,name=connectedThriftClients MBean.
> What's extra fun about this is the leaked sockets still count towards your
> maximum RPC thread count. As a result, toggling thrift enough times will
> result in an rpc_max_threads number of CLOSED_WAIT sockets, with no new
> clients able to connect.
> This was reproduced with HSHA. I haven't tried it in sync yet.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[1/2] git commit: Optimize Cell liveness checks and clean up Cell
Repository: cassandra
Updated Branches:
refs/heads/trunk cf3f5b6ef -> b35486a19
Optimize Cell liveness checks and clean up Cell
patch by Aleksey Yeschenko; reviewed by Benedict Elliott Smith for
CASSANDRA-7119
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4485e6db
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4485e6db
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4485e6db
Branch: refs/heads/trunk
Commit: 4485e6dbfed89c9137a58412210e56ae88cfe217
Parents: 9872b74
Author: Aleksey Yeschenko
Authored: Wed Apr 30 21:06:46 2014 +0200
Committer: Aleksey Yeschenko
Committed: Wed Apr 30 21:06:46 2014 +0200
--
CHANGES.txt | 1 +
.../apache/cassandra/cql/QueryProcessor.java| 4 +-
.../cassandra/cql3/statements/Selection.java| 2 +-
.../org/apache/cassandra/db/AbstractCell.java | 39 +---
.../cassandra/db/BufferCounterUpdateCell.java | 6 +--
.../apache/cassandra/db/BufferDeletedCell.java | 8 ++--
.../apache/cassandra/db/BufferExpiringCell.java | 8 ++--
src/java/org/apache/cassandra/db/Cell.java | 9 +
.../apache/cassandra/db/CounterMutation.java| 2 +-
.../cassandra/db/HintedHandOffManager.java | 2 +-
.../apache/cassandra/db/NativeDeletedCell.java | 8 ++--
.../apache/cassandra/db/NativeExpiringCell.java | 8 ++--
.../db/compaction/LazilyCompactedRow.java | 5 +--
.../db/composites/AbstractCellNameType.java | 4 +-
.../AbstractSimplePerColumnSecondaryIndex.java | 3 +-
.../db/index/SecondaryIndexManager.java | 8 ++--
.../CompositesIndexOnCollectionKey.java | 4 +-
.../CompositesIndexOnCollectionValue.java | 10 ++---
.../composites/CompositesIndexOnRegular.java| 8 +---
.../db/index/composites/CompositesSearcher.java | 8 +---
.../cassandra/db/index/keys/KeysIndex.java | 8 +---
.../cassandra/db/index/keys/KeysSearcher.java | 2 +-
.../apache/cassandra/service/CacheService.java | 2 +-
.../cassandra/thrift/CassandraServer.java | 10 ++---
.../apache/cassandra/db/ColumnFamilyTest.java | 5 ++-
.../apache/cassandra/db/CounterCellTest.java| 8 ++--
.../apache/cassandra/db/RangeTombstoneTest.java | 2 +-
.../org/apache/cassandra/db/RemoveCellTest.java | 14 +++
.../apache/cassandra/db/RemoveSubCellTest.java | 5 ++-
test/unit/org/apache/cassandra/db/RowTest.java | 5 ++-
.../db/compaction/CompactionsPurgeTest.java | 6 +--
.../db/index/PerRowSecondaryIndexTest.java | 15 +++-
.../cassandra/service/QueryPagerTest.java | 2 +-
33 files changed, 92 insertions(+), 139 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4485e6db/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a4811f6..34533cc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -55,6 +55,7 @@
* Multi-threaded scrub/cleanup/upgradesstables (CASSANDRA-5547)
* Optimize cellname comparison (CASSANDRA-6934)
* Native protocol v3 (CASSANDRA-6855)
+ * Optimize Cell liveness checks and clean up Cell (CASSANDRA-7119)
Merged from 2.0:
* Allow overriding cassandra-rackdc.properties file (CASSANDRA-7072)
* Set JMX RMI port to 7199 (CASSANDRA-7087)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4485e6db/src/java/org/apache/cassandra/cql/QueryProcessor.java
--
diff --git a/src/java/org/apache/cassandra/cql/QueryProcessor.java
b/src/java/org/apache/cassandra/cql/QueryProcessor.java
index 3b3..3c1d555 100644
--- a/src/java/org/apache/cassandra/cql/QueryProcessor.java
+++ b/src/java/org/apache/cassandra/cql/QueryProcessor.java
@@ -469,7 +469,7 @@ public class QueryProcessor
{
for (org.apache.cassandra.db.Cell c :
row.cf.getSortedColumns())
{
-if (c.isMarkedForDelete(now))
+if (!c.isLive(now))
continue;
ColumnDefinition cd =
metadata.getColumnDefinition(c.name());
@@ -515,7 +515,7 @@ public class QueryProcessor
if (cd != null)
result.schema.value_types.put(nameBytes,
TypeParser.getShortName(cd.type));
org.apache.cassandra.db.Cell c =
row.cf.getColumn(name);
-if (c == null ||
c.isMarkedForDelete(System.currentTimeMillis()))
+if (c == null || !c.isLive())
thriftColumns.add(new
Column().setName(nameBytes));
else
[2/2] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: src/java/org/apache/cassandra/cql/QueryProcessor.java test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b35486a1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b35486a1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b35486a1 Branch: refs/heads/trunk Commit: b35486a19b43eebd3be1d2fec6a827db0903c8cb Parents: cf3f5b6 4485e6d Author: Aleksey Yeschenko Authored: Wed Apr 30 21:14:09 2014 +0200 Committer: Aleksey Yeschenko Committed: Wed Apr 30 21:14:09 2014 +0200 -- CHANGES.txt | 1 + .../cassandra/cql3/statements/Selection.java| 2 +- .../org/apache/cassandra/db/AbstractCell.java | 39 +--- .../cassandra/db/BufferCounterUpdateCell.java | 6 +-- .../apache/cassandra/db/BufferDeletedCell.java | 8 ++-- .../apache/cassandra/db/BufferExpiringCell.java | 8 ++-- src/java/org/apache/cassandra/db/Cell.java | 9 + .../apache/cassandra/db/CounterMutation.java| 2 +- .../cassandra/db/HintedHandOffManager.java | 2 +- .../apache/cassandra/db/NativeDeletedCell.java | 8 ++-- .../apache/cassandra/db/NativeExpiringCell.java | 8 ++-- .../db/compaction/LazilyCompactedRow.java | 5 +-- .../db/composites/AbstractCellNameType.java | 4 +- .../AbstractSimplePerColumnSecondaryIndex.java | 3 +- .../db/index/SecondaryIndexManager.java | 8 ++-- .../CompositesIndexOnCollectionKey.java | 4 +- .../CompositesIndexOnCollectionValue.java | 10 ++--- .../composites/CompositesIndexOnRegular.java| 8 +--- .../db/index/composites/CompositesSearcher.java | 8 +--- .../cassandra/db/index/keys/KeysIndex.java | 8 +--- .../cassandra/db/index/keys/KeysSearcher.java | 2 +- .../apache/cassandra/service/CacheService.java | 2 +- .../cassandra/thrift/CassandraServer.java | 10 ++--- .../apache/cassandra/db/ColumnFamilyTest.java | 5 ++- .../apache/cassandra/db/CounterCellTest.java| 8 ++-- .../apache/cassandra/db/RangeTombstoneTest.java | 2 +- .../org/apache/cassandra/db/RemoveCellTest.java | 14 +++ .../apache/cassandra/db/RemoveSubCellTest.java | 5 ++- test/unit/org/apache/cassandra/db/RowTest.java | 5 ++- .../db/compaction/CompactionsPurgeTest.java | 6 +-- .../db/index/PerRowSecondaryIndexTest.java | 15 +++- .../cassandra/service/QueryPagerTest.java | 2 +- 32 files changed, 90 insertions(+), 137 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b35486a1/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b35486a1/src/java/org/apache/cassandra/thrift/CassandraServer.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b35486a1/test/unit/org/apache/cassandra/db/RangeTombstoneTest.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b35486a1/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java -- diff --cc test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java index 6a98873,751e7ae..5820312 --- a/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java +++ b/test/unit/org/apache/cassandra/db/compaction/CompactionsPurgeTest.java @@@ -275,7 -273,7 +275,7 @@@ public class CompactionsPurgeTest exten ColumnFamily cf = cfs.getColumnFamily(QueryFilter.getIdentityFilter(key, cfName, System.currentTimeMillis())); assertEquals(10, cf.getColumnCount()); for (Cell c : cf) - assertFalse(c.isMarkedForDelete(System.currentTimeMillis())); -assert c.isLive(); ++assertTrue(c.isLive()); } @Test @@@ -319,7 -317,7 +319,7 @@@ cf = cfs.getColumnFamily(filter); assertEquals(10, cf.getColumnCount()); for (Cell c : cf) - assertFalse(c.isMarkedForDelete(System.currentTimeMillis())); -assert c.isLive(); ++assertTrue(c.isLive()); } @Test
[jira] [Commented] (CASSANDRA-7121) Fix build error introduced by CASSANDRA-7116
[
https://issues.apache.org/jira/browse/CASSANDRA-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985943#comment-13985943
]
Benedict commented on CASSANDRA-7121:
-
That was fast. I probably would have just replaced replies.get(0) with
replies.peek(), but no biggy
> Fix build error introduced by CASSANDRA-7116
>
>
> Key: CASSANDRA-7121
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7121
> Project: Cassandra
> Issue Type: Bug
>Reporter: Doğan Çeçen
>Priority: Trivial
> Attachments: trunk-7121.txt
>
>
> RowDataResolver is calling {{replies.get}} method, but Collection interface
> doesn't contain {{get}} method (List does). Build reports some errors about
> this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
git commit: fix build
Repository: cassandra Updated Branches: refs/heads/trunk 6b4d98035 -> cf3f5b6ef fix build Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf3f5b6e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf3f5b6e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf3f5b6e Branch: refs/heads/trunk Commit: cf3f5b6efbe80a95bfc57e40e60ef5531eddc71e Parents: 6b4d980 Author: Jonathan Ellis Authored: Wed Apr 30 14:07:51 2014 -0500 Committer: Jonathan Ellis Committed: Wed Apr 30 14:08:30 2014 -0500 -- src/java/org/apache/cassandra/service/AbstractRowResolver.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf3f5b6e/src/java/org/apache/cassandra/service/AbstractRowResolver.java -- diff --git a/src/java/org/apache/cassandra/service/AbstractRowResolver.java b/src/java/org/apache/cassandra/service/AbstractRowResolver.java index a84a6b6..9e287cf 100644 --- a/src/java/org/apache/cassandra/service/AbstractRowResolver.java +++ b/src/java/org/apache/cassandra/service/AbstractRowResolver.java @@ -21,6 +21,7 @@ import java.nio.ByteBuffer; import java.util.ArrayList; import java.util.Collection; import java.util.Collections; +import java.util.List; import java.util.concurrent.ConcurrentLinkedQueue; import org.slf4j.Logger; @@ -37,7 +38,7 @@ public abstract class AbstractRowResolver implements IResponseResolver> replies = Collections.synchronizedList(new ArrayList>()); +protected final List> replies = Collections.synchronizedList(new ArrayList>()); protected final DecoratedKey key; public AbstractRowResolver(ByteBuffer key, String keyspaceName)
git commit: Optimize Cell liveness checks and clean up Cell
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 9872b74ef -> 4485e6dbf
Optimize Cell liveness checks and clean up Cell
patch by Aleksey Yeschenko; reviewed by Benedict Elliott Smith for
CASSANDRA-7119
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4485e6db
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4485e6db
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4485e6db
Branch: refs/heads/cassandra-2.1
Commit: 4485e6dbfed89c9137a58412210e56ae88cfe217
Parents: 9872b74
Author: Aleksey Yeschenko
Authored: Wed Apr 30 21:06:46 2014 +0200
Committer: Aleksey Yeschenko
Committed: Wed Apr 30 21:06:46 2014 +0200
--
CHANGES.txt | 1 +
.../apache/cassandra/cql/QueryProcessor.java| 4 +-
.../cassandra/cql3/statements/Selection.java| 2 +-
.../org/apache/cassandra/db/AbstractCell.java | 39 +---
.../cassandra/db/BufferCounterUpdateCell.java | 6 +--
.../apache/cassandra/db/BufferDeletedCell.java | 8 ++--
.../apache/cassandra/db/BufferExpiringCell.java | 8 ++--
src/java/org/apache/cassandra/db/Cell.java | 9 +
.../apache/cassandra/db/CounterMutation.java| 2 +-
.../cassandra/db/HintedHandOffManager.java | 2 +-
.../apache/cassandra/db/NativeDeletedCell.java | 8 ++--
.../apache/cassandra/db/NativeExpiringCell.java | 8 ++--
.../db/compaction/LazilyCompactedRow.java | 5 +--
.../db/composites/AbstractCellNameType.java | 4 +-
.../AbstractSimplePerColumnSecondaryIndex.java | 3 +-
.../db/index/SecondaryIndexManager.java | 8 ++--
.../CompositesIndexOnCollectionKey.java | 4 +-
.../CompositesIndexOnCollectionValue.java | 10 ++---
.../composites/CompositesIndexOnRegular.java| 8 +---
.../db/index/composites/CompositesSearcher.java | 8 +---
.../cassandra/db/index/keys/KeysIndex.java | 8 +---
.../cassandra/db/index/keys/KeysSearcher.java | 2 +-
.../apache/cassandra/service/CacheService.java | 2 +-
.../cassandra/thrift/CassandraServer.java | 10 ++---
.../apache/cassandra/db/ColumnFamilyTest.java | 5 ++-
.../apache/cassandra/db/CounterCellTest.java| 8 ++--
.../apache/cassandra/db/RangeTombstoneTest.java | 2 +-
.../org/apache/cassandra/db/RemoveCellTest.java | 14 +++
.../apache/cassandra/db/RemoveSubCellTest.java | 5 ++-
test/unit/org/apache/cassandra/db/RowTest.java | 5 ++-
.../db/compaction/CompactionsPurgeTest.java | 6 +--
.../db/index/PerRowSecondaryIndexTest.java | 15 +++-
.../cassandra/service/QueryPagerTest.java | 2 +-
33 files changed, 92 insertions(+), 139 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4485e6db/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a4811f6..34533cc 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -55,6 +55,7 @@
* Multi-threaded scrub/cleanup/upgradesstables (CASSANDRA-5547)
* Optimize cellname comparison (CASSANDRA-6934)
* Native protocol v3 (CASSANDRA-6855)
+ * Optimize Cell liveness checks and clean up Cell (CASSANDRA-7119)
Merged from 2.0:
* Allow overriding cassandra-rackdc.properties file (CASSANDRA-7072)
* Set JMX RMI port to 7199 (CASSANDRA-7087)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4485e6db/src/java/org/apache/cassandra/cql/QueryProcessor.java
--
diff --git a/src/java/org/apache/cassandra/cql/QueryProcessor.java
b/src/java/org/apache/cassandra/cql/QueryProcessor.java
index 3b3..3c1d555 100644
--- a/src/java/org/apache/cassandra/cql/QueryProcessor.java
+++ b/src/java/org/apache/cassandra/cql/QueryProcessor.java
@@ -469,7 +469,7 @@ public class QueryProcessor
{
for (org.apache.cassandra.db.Cell c :
row.cf.getSortedColumns())
{
-if (c.isMarkedForDelete(now))
+if (!c.isLive(now))
continue;
ColumnDefinition cd =
metadata.getColumnDefinition(c.name());
@@ -515,7 +515,7 @@ public class QueryProcessor
if (cd != null)
result.schema.value_types.put(nameBytes,
TypeParser.getShortName(cd.type));
org.apache.cassandra.db.Cell c =
row.cf.getColumn(name);
-if (c == null ||
c.isMarkedForDelete(System.currentTimeMillis()))
+if (c == null || !c.isLive())
thriftColumns.add(new
Column().setName(nameBytes));
else
[jira] [Updated] (CASSANDRA-7121) Fix build error introduced by CASSANDRA-7116
[
https://issues.apache.org/jira/browse/CASSANDRA-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doğan Çeçen updated CASSANDRA-7121:
---
Attachment: trunk-7121.txt
> Fix build error introduced by CASSANDRA-7116
>
>
> Key: CASSANDRA-7121
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7121
> Project: Cassandra
> Issue Type: Bug
>Reporter: Doğan Çeçen
>Priority: Trivial
> Attachments: trunk-7121.txt
>
>
> RowDataResolver is calling {{replies.get}} method, but Collection interface
> doesn't contain {{get}} method (List does). Build reports some errors about
> this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7121) Fix build error introduced by CASSANDRA-7116
Doğan Çeçen created CASSANDRA-7121:
--
Summary: Fix build error introduced by CASSANDRA-7116
Key: CASSANDRA-7121
URL: https://issues.apache.org/jira/browse/CASSANDRA-7121
Project: Cassandra
Issue Type: Bug
Reporter: Doğan Çeçen
Priority: Trivial
RowDataResolver is calling {{replies.get}} method, but Collection interface
doesn't contain {{get}} method (List does). Build reports some errors about
this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7081) select writetime(colname) returns 0 for static columns
[
https://issues.apache.org/jira/browse/CASSANDRA-7081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985927#comment-13985927
]
Tyler Hobbs commented on CASSANDRA-7081:
+1
> select writetime(colname) returns 0 for static columns
> --
>
> Key: CASSANDRA-7081
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7081
> Project: Cassandra
> Issue Type: Bug
>Reporter: Nicolas Favre-Felix
>Assignee: Sylvain Lebresne
> Fix For: 2.0.8
>
> Attachments: 7081.txt
>
>
> Selecting the write time for a static column returns 0 in Cassandra 2.0
> (c3550fe) and an expected timestamp in 2.1 (trunk, acdbbb9). Would it be
> possible to include this timestamp in a 2.0 release too?
> {code}
> > CREATE TABLE test (partition_key text, cluster_key text, data text, st text
> > static, PRIMARY KEY(partition_key, cluster_key));
> > INSERT INTO test (partition_key, cluster_key, data, st) VALUES ( 'PK',
> > 'CK', 'DATA', 'ST');
> > SELECT writetime(st), writetime(data) FROM test where partition_key='PK';
> writetime(st) | writetime(data)
> ---+--
> 0 | 1398314681729000
> (1 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7107) General minor tidying of CollationController path
[ https://issues.apache.org/jira/browse/CASSANDRA-7107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985919#comment-13985919 ] Benedict commented on CASSANDRA-7107: - Weird. I was sure I'd run all of the unit tests before uploading. Two line patch uploaded that fixes those two tests. > General minor tidying of CollationController path > - > > Key: CASSANDRA-7107 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7107 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Benedict >Assignee: Benedict >Priority: Minor > Fix For: 2.1.0 > > > There is a lot of unnecessary boiler plate when grabbing an iterator from an > in-memory column family. This patch: > * Removes FakeCellName > * Avoids wrapping a non-OnDiskAtomIterator as an OnDiskAtomIterator except > when the wrapping is useful > * Removes ColumnSlice.NavigableSetIterator and creates a simpler more direct > equivalent in ABTC > * Does not construct a SliceIterator in either ABSC or ABTC if only one slice > is requested (just returns that slice as an Iterator) > * Does not construct multiple list indirections in ABSC when constructing a > slice > * Shares forward/reverse iterators in ABSC between slices and full-iteration > * Avoids O(N) comparisons during collation of results into an ABSC, by using > the knowledge that all columns are provided in insertion order from a merge > iterator -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6890) Standardize on a single read path
[ https://issues.apache.org/jira/browse/CASSANDRA-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985911#comment-13985911 ] Benedict commented on CASSANDRA-6890: - Sure there are - I want to push them through the same cache, so we can get our other knock on benefits of reduced message passing for all file types. > Standardize on a single read path > - > > Key: CASSANDRA-6890 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6890 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Joshua McKenzie >Assignee: Joshua McKenzie > Labels: performance > Fix For: 3.0 > > Attachments: mmap_gc.jpg, mmap_jstat.txt, mmap_perf.txt, > nommap_gc.jpg, nommap_jstat.txt > > > Since we actively unmap unreferenced SSTR's and also copy data out of those > readers on the read path, the current memory mapped i/o is a lot of > complexity for very little payoff. Clean out the mmapp'ed i/o on the read > path. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6890) Standardize on a single read path
[ https://issues.apache.org/jira/browse/CASSANDRA-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985905#comment-13985905 ] Jonathan Ellis commented on CASSANDRA-6890: --- bq. I'd be comfortable normalizing the system on buffered I/O leading up to 3.0 to give us a single read path to migrate to an in-process page cache All right. What do we do for uncompressed sstables? Currently there are no blocks/pages to cache. > Standardize on a single read path > - > > Key: CASSANDRA-6890 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6890 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Joshua McKenzie >Assignee: Joshua McKenzie > Labels: performance > Fix For: 3.0 > > Attachments: mmap_gc.jpg, mmap_jstat.txt, mmap_perf.txt, > nommap_gc.jpg, nommap_jstat.txt > > > Since we actively unmap unreferenced SSTR's and also copy data out of those > readers on the read path, the current memory mapped i/o is a lot of > complexity for very little payoff. Clean out the mmapp'ed i/o on the read > path. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6131) JAVA_HOME on cassandra-env.sh is ignored on Debian packages
[ https://issues.apache.org/jira/browse/CASSANDRA-6131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985874#comment-13985874 ] Sebastián Lacuesta commented on CASSANDRA-6131: --- Works for me. Thanks! > JAVA_HOME on cassandra-env.sh is ignored on Debian packages > --- > > Key: CASSANDRA-6131 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6131 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Sebastián Lacuesta >Assignee: Eric Evans > Labels: qa-resolved > Fix For: 2.0.5 > > Attachments: 6131-2.patch, 6131.patch > > > I've just got upgraded to 2.0.1 package from the apache repositories using > apt. I had the JAVA_HOME environment variable set in > /etc/cassandra/cassandra-env.sh but after the upgrade it only worked by > setting it on /usr/sbin/cassandra script. I can't configure java 7 system > wide, only for cassandra. > Off-toppic: Thanks for getting rid of the jsvc mess. -- This message was sent by Atlassian JIRA (v6.2#6252)
[5/5] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6b4d9803 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6b4d9803 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6b4d9803 Branch: refs/heads/trunk Commit: 6b4d980357de573f9128c3a065a6d11c54a3b571 Parents: ad34247 9872b74 Author: Sylvain Lebresne Authored: Wed Apr 30 20:22:53 2014 +0200 Committer: Sylvain Lebresne Committed: Wed Apr 30 20:22:53 2014 +0200 -- CHANGES.txt | 1 + build.xml | 2 +- doc/native_protocol_v3.spec | 911 +++ src/java/org/apache/cassandra/auth/Auth.java| 16 +- .../cassandra/auth/CassandraAuthorizer.java | 6 +- .../cassandra/auth/PasswordAuthenticator.java | 4 +- .../org/apache/cassandra/cql3/Attributes.java | 8 +- .../cassandra/cql3/BatchQueryOptions.java | 81 +- .../apache/cassandra/cql3/ColumnCondition.java | 32 +- .../org/apache/cassandra/cql3/Constants.java| 20 +- src/java/org/apache/cassandra/cql3/Lists.java | 34 +- src/java/org/apache/cassandra/cql3/Maps.java| 32 +- .../org/apache/cassandra/cql3/QueryHandler.java | 3 +- .../org/apache/cassandra/cql3/QueryOptions.java | 283 -- .../apache/cassandra/cql3/QueryProcessor.java | 29 +- .../org/apache/cassandra/cql3/ResultSet.java| 6 +- src/java/org/apache/cassandra/cql3/Sets.java| 26 +- src/java/org/apache/cassandra/cql3/Term.java| 18 +- .../apache/cassandra/cql3/UpdateParameters.java | 6 +- .../org/apache/cassandra/cql3/UserTypes.java| 18 +- .../cassandra/cql3/functions/FunctionCall.java | 20 +- .../cql3/statements/BatchStatement.java | 95 +- .../cql3/statements/CQL3CasConditions.java | 14 +- .../cql3/statements/ModificationStatement.java | 63 +- .../cassandra/cql3/statements/Restriction.java | 28 +- .../cql3/statements/SelectStatement.java| 150 +-- .../org/apache/cassandra/db/DefsTables.java | 19 +- .../cassandra/db/marshal/CollectionType.java| 29 +- .../apache/cassandra/db/marshal/ListType.java | 12 +- .../apache/cassandra/db/marshal/MapType.java| 21 +- .../apache/cassandra/db/marshal/SetType.java| 15 +- .../apache/cassandra/db/marshal/UserType.java | 5 + .../hadoop/pig/AbstractCassandraStorage.java| 11 +- .../cassandra/io/sstable/CQLSSTableWriter.java | 11 +- .../serializers/CollectionSerializer.java | 106 ++- .../cassandra/serializers/ListSerializer.java | 39 +- .../cassandra/serializers/MapSerializer.java| 48 +- .../cassandra/serializers/SetSerializer.java| 39 +- .../cassandra/service/IMigrationListener.java | 3 + .../cassandra/service/MigrationManager.java | 18 + .../cassandra/thrift/CassandraServer.java | 4 +- .../org/apache/cassandra/transport/CBUtil.java | 17 + .../org/apache/cassandra/transport/Client.java | 4 +- .../apache/cassandra/transport/DataType.java| 79 +- .../org/apache/cassandra/transport/Event.java | 158 +++- .../apache/cassandra/transport/OptionCodec.java | 28 +- .../org/apache/cassandra/transport/Server.java | 21 +- .../cassandra/transport/SimpleClient.java | 4 +- .../transport/messages/BatchMessage.java| 53 +- .../transport/messages/EventMessage.java| 6 +- .../transport/messages/ExecuteMessage.java | 5 +- .../cassandra/transport/SerDeserTest.java | 217 + 52 files changed, 2232 insertions(+), 646 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b4d9803/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b4d9803/build.xml -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6b4d9803/src/java/org/apache/cassandra/cql3/QueryProcessor.java -- diff --cc src/java/org/apache/cassandra/cql3/QueryProcessor.java index db99060,40c45af..9a5ac92 --- a/src/java/org/apache/cassandra/cql3/QueryProcessor.java +++ b/src/java/org/apache/cassandra/cql3/QueryProcessor.java @@@ -51,13 -51,14 +51,13 @@@ public class QueryProcessor implements private static final Logger logger = LoggerFactory.getLogger(QueryProcessor.class); private static final MemoryMeter meter = new MemoryMeter().withGuessing(MemoryMeter.Guess.FALLBACK_BEST); private static final long MAX_CACHE_PREPARED_MEMORY = Runtime.getRuntime().maxMemory() / 256; -private static final int MAX_CACHE_PREPARED_COUNT = 1; - private static EntryWeigher cqlMemoryUsageWeigher = new E
[1/5] Native protocol v3
Repository: cassandra
Updated Branches:
refs/heads/trunk ad3424707 -> 6b4d98035
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9872b74e/src/java/org/apache/cassandra/transport/messages/BatchMessage.java
--
diff --git a/src/java/org/apache/cassandra/transport/messages/BatchMessage.java
b/src/java/org/apache/cassandra/transport/messages/BatchMessage.java
index ef30a22..ec96ed1 100644
--- a/src/java/org/apache/cassandra/transport/messages/BatchMessage.java
+++ b/src/java/org/apache/cassandra/transport/messages/BatchMessage.java
@@ -28,6 +28,7 @@ import io.netty.buffer.ByteBuf;
import org.apache.cassandra.cql3.*;
import org.apache.cassandra.cql3.statements.BatchStatement;
import org.apache.cassandra.cql3.statements.ModificationStatement;
+import org.apache.cassandra.cql3.statements.ParsedStatement;
import org.apache.cassandra.db.ConsistencyLevel;
import org.apache.cassandra.exceptions.InvalidRequestException;
import org.apache.cassandra.exceptions.PreparedQueryNotFoundException;
@@ -61,8 +62,11 @@ public class BatchMessage extends Message.Request
throw new ProtocolException("Invalid query kind in BATCH
messages. Must be 0 or 1 but got " + kind);
variables.add(CBUtil.readValueList(body));
}
-ConsistencyLevel consistency = CBUtil.readConsistencyLevel(body);
-return new BatchMessage(toType(type), queryOrIds, variables,
consistency);
+QueryOptions options = version < 3
+ ?
QueryOptions.fromPreV3Batch(CBUtil.readConsistencyLevel(body))
+ : QueryOptions.codec.decode(body, version);
+
+return new BatchMessage(toType(type), queryOrIds, variables,
options);
}
public void encode(BatchMessage msg, ByteBuf dest, int version)
@@ -84,7 +88,10 @@ public class BatchMessage extends Message.Request
CBUtil.writeValueList(msg.values.get(i), dest);
}
-CBUtil.writeConsistencyLevel(msg.consistency, dest);
+if (version < 3)
+CBUtil.writeConsistencyLevel(msg.options.getConsistency(),
dest);
+else
+QueryOptions.codec.encode(msg.options, dest, version);
}
public int encodedSize(BatchMessage msg, int version)
@@ -99,7 +106,9 @@ public class BatchMessage extends Message.Request
size += CBUtil.sizeOfValueList(msg.values.get(i));
}
-size += CBUtil.sizeOfConsistencyLevel(msg.consistency);
+size += version < 3
+ ? CBUtil.sizeOfConsistencyLevel(msg.options.getConsistency())
+ : QueryOptions.codec.encodedSize(msg.options, version);
return size;
}
@@ -131,15 +140,15 @@ public class BatchMessage extends Message.Request
public final BatchStatement.Type type;
public final List queryOrIdList;
public final List> values;
-public final ConsistencyLevel consistency;
+public final QueryOptions options;
-public BatchMessage(BatchStatement.Type type, List queryOrIdList,
List> values, ConsistencyLevel consistency)
+public BatchMessage(BatchStatement.Type type, List queryOrIdList,
List> values, QueryOptions options)
{
super(Message.Type.BATCH);
this.type = type;
this.queryOrIdList = queryOrIdList;
this.values = values;
-this.consistency = consistency;
+this.options = options;
}
public Message.Response execute(QueryState state)
@@ -161,27 +170,39 @@ public class BatchMessage extends Message.Request
}
QueryHandler handler = state.getClientState().getCQLQueryHandler();
-List statements = new
ArrayList(queryOrIdList.size());
+List prepared = new
ArrayList<>(queryOrIdList.size());
for (int i = 0; i < queryOrIdList.size(); i++)
{
Object query = queryOrIdList.get(i);
-CQLStatement statement;
+ParsedStatement.Prepared p;
if (query instanceof String)
{
-statement = QueryProcessor.parseStatement((String)query,
state);
+p = QueryProcessor.parseStatement((String)query, state);
}
else
{
-statement = handler.getPrepared((MD5Digest)query);
-if (statement == null)
+p = handler.getPrepared((MD5Digest)query);
+if (p == null)
throw new
PreparedQueryNotFoundException((MD5Digest)query);
}
List queryValues = values.get(i);
-if (queryValues.size() != statement.getBoundTerms())
+if (queryValues.size() != p.statement.getBoundTerms())
throw n
[1/4] Native protocol v3
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 ece386439 -> 9872b74ef
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9872b74e/src/java/org/apache/cassandra/transport/messages/BatchMessage.java
--
diff --git a/src/java/org/apache/cassandra/transport/messages/BatchMessage.java
b/src/java/org/apache/cassandra/transport/messages/BatchMessage.java
index ef30a22..ec96ed1 100644
--- a/src/java/org/apache/cassandra/transport/messages/BatchMessage.java
+++ b/src/java/org/apache/cassandra/transport/messages/BatchMessage.java
@@ -28,6 +28,7 @@ import io.netty.buffer.ByteBuf;
import org.apache.cassandra.cql3.*;
import org.apache.cassandra.cql3.statements.BatchStatement;
import org.apache.cassandra.cql3.statements.ModificationStatement;
+import org.apache.cassandra.cql3.statements.ParsedStatement;
import org.apache.cassandra.db.ConsistencyLevel;
import org.apache.cassandra.exceptions.InvalidRequestException;
import org.apache.cassandra.exceptions.PreparedQueryNotFoundException;
@@ -61,8 +62,11 @@ public class BatchMessage extends Message.Request
throw new ProtocolException("Invalid query kind in BATCH
messages. Must be 0 or 1 but got " + kind);
variables.add(CBUtil.readValueList(body));
}
-ConsistencyLevel consistency = CBUtil.readConsistencyLevel(body);
-return new BatchMessage(toType(type), queryOrIds, variables,
consistency);
+QueryOptions options = version < 3
+ ?
QueryOptions.fromPreV3Batch(CBUtil.readConsistencyLevel(body))
+ : QueryOptions.codec.decode(body, version);
+
+return new BatchMessage(toType(type), queryOrIds, variables,
options);
}
public void encode(BatchMessage msg, ByteBuf dest, int version)
@@ -84,7 +88,10 @@ public class BatchMessage extends Message.Request
CBUtil.writeValueList(msg.values.get(i), dest);
}
-CBUtil.writeConsistencyLevel(msg.consistency, dest);
+if (version < 3)
+CBUtil.writeConsistencyLevel(msg.options.getConsistency(),
dest);
+else
+QueryOptions.codec.encode(msg.options, dest, version);
}
public int encodedSize(BatchMessage msg, int version)
@@ -99,7 +106,9 @@ public class BatchMessage extends Message.Request
size += CBUtil.sizeOfValueList(msg.values.get(i));
}
-size += CBUtil.sizeOfConsistencyLevel(msg.consistency);
+size += version < 3
+ ? CBUtil.sizeOfConsistencyLevel(msg.options.getConsistency())
+ : QueryOptions.codec.encodedSize(msg.options, version);
return size;
}
@@ -131,15 +140,15 @@ public class BatchMessage extends Message.Request
public final BatchStatement.Type type;
public final List queryOrIdList;
public final List> values;
-public final ConsistencyLevel consistency;
+public final QueryOptions options;
-public BatchMessage(BatchStatement.Type type, List queryOrIdList,
List> values, ConsistencyLevel consistency)
+public BatchMessage(BatchStatement.Type type, List queryOrIdList,
List> values, QueryOptions options)
{
super(Message.Type.BATCH);
this.type = type;
this.queryOrIdList = queryOrIdList;
this.values = values;
-this.consistency = consistency;
+this.options = options;
}
public Message.Response execute(QueryState state)
@@ -161,27 +170,39 @@ public class BatchMessage extends Message.Request
}
QueryHandler handler = state.getClientState().getCQLQueryHandler();
-List statements = new
ArrayList(queryOrIdList.size());
+List prepared = new
ArrayList<>(queryOrIdList.size());
for (int i = 0; i < queryOrIdList.size(); i++)
{
Object query = queryOrIdList.get(i);
-CQLStatement statement;
+ParsedStatement.Prepared p;
if (query instanceof String)
{
-statement = QueryProcessor.parseStatement((String)query,
state);
+p = QueryProcessor.parseStatement((String)query, state);
}
else
{
-statement = handler.getPrepared((MD5Digest)query);
-if (statement == null)
+p = handler.getPrepared((MD5Digest)query);
+if (p == null)
throw new
PreparedQueryNotFoundException((MD5Digest)query);
}
List queryValues = values.get(i);
-if (queryValues.size() != statement.getBoundTerms())
+if (queryValues.size() != p.statement.getBoundTerms())
[jira] [Commented] (CASSANDRA-7119) Optimise isLive(System.currentTimeMillis()) + minor Cell cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-7119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13985869#comment-13985869 ] Benedict commented on CASSANDRA-7119: - LGTM * Could use a test case for the timestamp summation (and I don't understand it, so have to trust you're right on that one) * Could call !isLive(Long.MIN_VALUE) in cases where we called isMarkedForDelete(Long.MIN_VALUE) to avoid further System.currentTimeMillis() - or could simply call instanceof DeletedCell. Or leave it as is. * I was under the impression we defaulted to explicit imports, not package.* (though I don't see a spec either way in the codestyle) > Optimise isLive(System.currentTimeMillis()) + minor Cell cleanup > > > Key: CASSANDRA-7119 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7119 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Fix For: 2.1 beta2 > > > There are lots of Cell#isMakredForDeleteAt(System.currentTimeMillis()) and > Cell#isLive(System.currentTimeMillis()) calls in the codebase - including in > Cell#reconcile(), while we only need to pass the current time to the > ExpiringCell. System.currentTimeMillis() is cheap, but not calling it at all > is cheaper (and it's not *that* cheap when virtualised under certain configs). > There is also another form - calling isMarkedForDelete() with > Long.MIN_VALUE/Long.MAX_VALUE, when we know we aren't expecting an > ExpiringCell, ever. > So the patch adds an argument-less isLive() method that only calls > System.currentTimeMillis() for ExpiringCell. > To reduce duplication between Native* and Buffer*, isMarkedForDelete() has > been removed entirely in favor of the shorter-to-type isLive() (plus I never > really liked the name anyway). > Also performs minor clean up and fixes one minor bug: > - removes the unused Cell#getMarkedForDeleteAt() method > - removes redundant Override-s in AbstractCell > - corrects BufferCounterUpdateCell#reconcile() to sum the timestamps and not > pick the max (must have slipped through 6694) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6851) Improve anticompaction after incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-6851: --- Labels: compaction lhf (was: ) > Improve anticompaction after incremental repair > --- > > Key: CASSANDRA-6851 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6851 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Priority: Minor > Labels: compaction, lhf > Fix For: 2.1 beta2 > > > After an incremental repair we iterate over all sstables and split them in > two parts, one containing the repaired data and one the unrepaired. We could > in theory double the number of sstables on a node. > To avoid this we could make anticompaction also do a compaction, for example, > if we are to anticompact 10 sstables, we could anticompact those to 2. > Note that we need to avoid creating too big sstables though, if we > anticompact all sstables on a node it would essentially be a major compaction. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7019) Major tombstone compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-7019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-7019: --- Labels: compaction (was: ) > Major tombstone compaction > -- > > Key: CASSANDRA-7019 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7019 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson > Labels: compaction > > It should be possible to do a "major" tombstone compaction by including all > sstables, but writing them out 1:1, meaning that if you have 10 sstables > before, you will have 10 sstables after the compaction with the same data, > minus all the expired tombstones. > We could do this in two ways: > # a nodetool command that includes _all_ sstables > # once we detect that an sstable has more than x% (20%?) expired tombstones, > we start one of these compactions, and include all overlapping sstables that > contain older data. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-5936) Improve the way we pick L0 compaction candidates
[ https://issues.apache.org/jira/browse/CASSANDRA-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-5936: --- Labels: compaction (was: ) > Improve the way we pick L0 compaction candidates > > > Key: CASSANDRA-5936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5936 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Labels: compaction > Fix For: 2.1 beta2 > > > We could improve the way we pick compaction candidates in level 0 in LCS. > The most common way for us to get behind on compaction is after repairs, we > should exploit the fact that the streamed sstables are most often very narrow > in range since the other nodes in the ring will have a similar > sstable-range-distribution. We should in theory be able to do 10 concurrent > compactions involving L1 - ie, partition L0 in buckets defined by the > sstables in L1 to only keep one L1 SSTable busy for every compaction (be it > L1 to L2 or L0 to L1). > we will need some heuristics on when to select candidates from the buckets > and when to do it the old way (since L0 sstables can span several L1 sstables) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6615) Changing the IP of a node on a live cluster leaves gossip infos and throws Exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-6615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McGuire updated CASSANDRA-6615:
Labels: qa-resolved (was: )
> Changing the IP of a node on a live cluster leaves gossip infos and throws
> Exceptions
> -
>
> Key: CASSANDRA-6615
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6615
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Fabien Rousseau
>Assignee: Brandon Williams
> Labels: qa-resolved
> Fix For: 1.2.14, 2.0.5
>
> Attachments: 6615.txt
>
>
> Following this procedure :
> https://engineering.eventbrite.com/changing-the-ip-address-of-a-cassandra-node-with-auto_bootstrapfalse/
> to change the IP of a node, we encountered an issue :
> - logs contains: "java.lang.RuntimeException: Host ID collision between
> active endpoint /127.0.0.5 and /127.0.0.3"
> - logs also indicate that the old IP is being removed of the cluster
> (FatClient timeout), then added again...
> - nodetool gossipinfo still list old IP (even a few hours after...)
> - the old IP is still seen as "UP" in the cluster... (according to the
> logs...)
> Below is a small shell script which allows to reproduce the scenario...
> {noformat}
> #! /bin/bash
> CLUSTER=$1
> ccm create $CLUSTER --cassandra-dir=.
> ccm populate -n 2
> ccm start
> ccm add node3 -i 127.0.0.3 -j 7300 -b
> ccm node3 start
> ccm node3 ring
> ccm node3 stop
> sed -i 's/127.0.0.3/127.0.0.5/g' ~/.ccm/$CLUSTER/node3/node.conf
> sed -i 's/127.0.0.3/127.0.0.5/g' ~/.ccm/$CLUSTER/node3/conf/cassandra.yaml
> ccm node3 start
> sleep 3
> nodetool --host 127.0.0.5 --port 7300 gossipinfo
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-5383) Windows 7 deleting/renaming files problem
[
https://issues.apache.org/jira/browse/CASSANDRA-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McGuire updated CASSANDRA-5383:
Labels: qa-resolved (was: )
> Windows 7 deleting/renaming files problem
> -
>
> Key: CASSANDRA-5383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5383
> Project: Cassandra
> Issue Type: Bug
> Components: Tests
>Affects Versions: 2.0 beta 1
>Reporter: Ryan McGuire
>Assignee: Marcus Eriksson
> Labels: qa-resolved
> Fix For: 2.0.2
>
> Attachments:
> 0001-CASSANDRA-5383-cant-move-a-file-on-top-of-another-fi.patch,
> 0001-CASSANDRA-5383-v2.patch,
> 0001-use-Java7-apis-for-deleting-and-moving-files-and-cre.patch,
> 5383-v3.patch, 5383_patch_v2_system.log, cant_move_file_patch.log,
> test_log.5383.patch_v2.log.txt, v2+cant_move_file_patch.log
>
>
> Two unit tests are failing on Windows 7 due to errors in renaming/deleting
> files:
> org.apache.cassandra.db.ColumnFamilyStoreTest:
> {code}
> [junit] Testsuite: org.apache.cassandra.db.ColumnFamilyStoreTest
> [junit] Tests run: 27, Failures: 0, Errors: 2, Skipped: 0, Time elapsed:
> 13.904 sec
> [junit]
> [junit] - Standard Error -
> [junit] ERROR 13:06:46,058 Unable to delete
> build\test\cassandra\data\Keyspace1\Indexed2\Keyspace1-Indexed2.birthdate_index-ja-1-Data.db
> (it will be removed on server restart; we'll also retry after GC)
> [junit] ERROR 13:06:48,508 Fatal exception in thread
> Thread[NonPeriodicTasks:1,5,main]
> [junit] java.lang.RuntimeException: Tried to hard link to file that does
> not exist
> build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-7-Statistics.db
> [junit] at
> org.apache.cassandra.io.util.FileUtils.createHardLink(FileUtils.java:72)
> [junit] at
> org.apache.cassandra.io.sstable.SSTableReader.createLinks(SSTableReader.java:1057)
> [junit] at
> org.apache.cassandra.db.DataTracker$1.run(DataTracker.java:168)
> [junit] at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> [junit] at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> [junit] at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> [junit] at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
> [junit] at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
> [junit] at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> [junit] at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> [junit] at java.lang.Thread.run(Thread.java:662)
> [junit] - ---
> [junit] Testcase:
> testSliceByNamesCommandOldMetatada(org.apache.cassandra.db.ColumnFamilyStoreTest):
> Caused an ERROR
> [junit] Failed to rename
> build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
> to
> build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
> [junit] java.lang.RuntimeException: Failed to rename
> build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db-tmp
> to
> build\test\cassandra\data\Keyspace1\Standard1\Keyspace1-Standard1-ja-6-Statistics.db
> [junit] at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
> [junit] at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:122)
> [junit] at
> org.apache.cassandra.db.compaction.LeveledManifest.mutateLevel(LeveledManifest.java:575)
> [junit] at
> org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:589)
> [junit] at
> org.apache.cassandra.db.ColumnFamilyStoreTest.testSliceByNamesCommandOldMetatada(ColumnFamilyStoreTest.java:885)
> [junit]
> [junit]
> [junit] Testcase:
> testRemoveUnifinishedCompactionLeftovers(org.apache.cassandra.db.ColumnFamilyStoreTest):
> Caused an ERROR
> [junit] java.io.IOException: Failed to delete
> c:\Users\Ryan\git\cassandra\build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
> [junit] FSWriteError in
> build\test\cassandra\data\Keyspace1\Standard3\Keyspace1-Standard3-ja-2-Data.db
> [junit] at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:112)
> [junit] at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:103)
> [junit] at
> org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:139)
> [junit] at
> org.apache.cassandra.db.ColumnFamilyStor
[jira] [Updated] (CASSANDRA-6893) Unintended update with conditional statement
[ https://issues.apache.org/jira/browse/CASSANDRA-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McGuire updated CASSANDRA-6893: Labels: qa-resolved (was: ) > Unintended update with conditional statement > > > Key: CASSANDRA-6893 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6893 > Project: Cassandra > Issue Type: Bug > Environment: Ubuntu Precise 64bit / Cassandra 2.0.6 >Reporter: Suguru Namura >Assignee: Sylvain Lebresne > Labels: qa-resolved > Fix For: 2.0.7 > > Attachments: 6893.txt, ConcurrentCASUpdate.java > > > After updated to 2.0.6, I have encountered the strange behavior of > conditional updates. > When I executed CQL like UPDATE test SET value = ? WHERE id = ? IF value = ? > in concurrent, sometimes cassandra returns true even if value is not > satisfied the condition. > I have attached the program which reproduce this issue. The program works > fine in cassandra 2.0.5. But it seems that resets values while execution in > 2.0.6. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-4899) add gitignore
[ https://issues.apache.org/jira/browse/CASSANDRA-4899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McGuire updated CASSANDRA-4899: Labels: qa-resolved (was: ) > add gitignore > - > > Key: CASSANDRA-4899 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4899 > Project: Cassandra > Issue Type: Task >Reporter: Radim Kolar >Assignee: Michael Shuler >Priority: Trivial > Labels: qa-resolved > Fix For: 1.2.14, 2.0.4 > > Attachments: .gitignore, cass-gitignore.txt > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-5337) vnode-aware replacenode command
[ https://issues.apache.org/jira/browse/CASSANDRA-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McGuire updated CASSANDRA-5337: Labels: qa-resolved (was: vnodes) > vnode-aware replacenode command > --- > > Key: CASSANDRA-5337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5337 > Project: Cassandra > Issue Type: New Feature > Components: Core >Affects Versions: 1.2.0 >Reporter: Jonathan Ellis >Assignee: Brandon Williams > Labels: qa-resolved > Fix For: 1.2.7, 2.0.0 > > Attachments: 5337-v2.txt, 5337.txt > > > Currently you have the following options to replace a dead, unrecoverable > node: > - replacetoken. this requires specifying all 256 or so vnode tokens as a CSL > - bootstrap new node, decommission old one. this is inefficient since the > new node's vnodes will probably not overlap much with the old one's, so we > replicate stream about 2x as much as if we were just replacing the old with > the new > We should add an analogue to replacetoken that takes the address or node ID > of the dead node instead. -- This message was sent by Atlassian JIRA (v6.2#6252)
