[jira] [Updated] (CASSANDRA-8776) nodetool status reports success for missing keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-8776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sachin Janani updated CASSANDRA-8776: - Attachment: 8776.patch Attaching patch for Cassandra # 8776 .Added System.exit(1) when keyspace does not exist error occurs > nodetool status reports success for missing keyspace > > > Key: CASSANDRA-8776 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8776 > Project: Cassandra > Issue Type: Bug >Reporter: Stuart Bishop >Priority: Minor > Labels: lhf > Attachments: 8776.patch > > > 'nodetool status somethinginvalid' will correctly output an error message > that the keyspace does not exist, but still returns a 'success' code of 0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8791) 'nodetool status' reports nodes in undocumented '?' state
Stuart Bishop created CASSANDRA-8791: Summary: 'nodetool status' reports nodes in undocumented '?' state Key: CASSANDRA-8791 URL: https://issues.apache.org/jira/browse/CASSANDRA-8791 Project: Cassandra Issue Type: Bug Reporter: Stuart Bishop Priority: Minor 'nodetool status' self documents the node status with the statement 'Status=Up/Down' which corresponds to the 'U' and 'D' statuses in the first column of the node listing. There is also a very important '?' status that is not documented, which seems to appear when a node is bootstrapping. Automation and deployment tools need to cope with nodes in this transition state, and the first step is to inform them about it in the documentation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8784) -bash: bin/cassandra: No such file or directory
[
https://issues.apache.org/jira/browse/CASSANDRA-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317254#comment-14317254
]
Eric Evans commented on CASSANDRA-8784:
---
{quote}
-bash: bin/cassandra: No such file or directory
{quote}
This is {{bash}} indicating that {{bin/cassandra}}, is neither a file, nor a
directory.
Hope this helps.
> -bash: bin/cassandra: No such file or directory
> ---
>
> Key: CASSANDRA-8784
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8784
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: ubuntu 14.04
>Reporter: R Scott
> Fix For: 2.1.2
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8779) Able to unintentionally nest tuples during insert
[
https://issues.apache.org/jira/browse/CASSANDRA-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317215#comment-14317215
]
Kishan Karunaratne edited comment on CASSANDRA-8779 at 2/11/15 11:45 PM:
-
This is for the insert statement:
{noformat}
["request buffer", #]
{noformat}
was (Author: kishkaru):
This is for the insert statement:
{noformat}
["request buffer", #]
{noformat}
> Able to unintentionally nest tuples during insert
> -
>
> Key: CASSANDRA-8779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8779
> Project: Cassandra
> Issue Type: Bug
> Environment: Linux Mint 64-bit | ruby-driver 2.1 | java-driver 2.1 |
> C* 2.1.2
>Reporter: Kishan Karunaratne
>Assignee: Tyler Hobbs
>
> If I insert a tuple using an extra pair of ()'s, C* will let me do the
> insert, but (incorrectly) creates a nested tuple as the first tuple value.
> Upon doing a select statement, the result is jumbled and has weird binary in
> it (which I wasn't able to copy into here).
> Example using ruby-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)")
> complete = Cassandra::Tuple.new('foo', 123, true)
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", arguments:
> [complete])# extra ()'s here
> result = session.execute("SELECT b FROM mytable WHERE a=0").first
> p result['b']
> {noformat}
> Output:
> {noformat}
> #
> {noformat}
> Bug also confirmed using java-driver.
> Example using java-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)");
> TupleType t = TupleType.of(DataType.ascii(), DataType.cint(),
> DataType.cboolean());
> TupleValue complete = t.newValue("foo", 123, true);
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", complete); //
> extra ()'s here
> TupleValue r = session.execute("SELECT b FROM mytable WHERE
> a=0").one().getTupleValue("b");
> System.out.println(r);
> {noformat}
> Output:
> {noformat}
> ('foo{', null, null)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317223#comment-14317223 ] Alex Liu edited comment on CASSANDRA-6091 at 2/11/15 11:44 PM: --- One more improvement is combining adjacent token ranges into one range. But it will create some small corner ranges, so the trade off is not that good as "as the number of adjacent token ranges is going to drop very quickly with the size of the cluster." quoted from Piotr was (Author: alexliu68): One more improvement is combining adjacent token ranges into one range. But it will create some small corner ranges, so the trade off is not that good as "as the number of adjacent token ranges is going to drop very quickly with the size of the cluster. Reply" > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317223#comment-14317223 ] Alex Liu commented on CASSANDRA-6091: - One more improvement is combining adjacent token ranges into one range. But it will create some small corner ranges, so the trade off is not that good as "as the number of adjacent token ranges is going to drop very quickly with the size of the cluster. Reply" > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8779) Able to unintentionally nest tuples during insert
[
https://issues.apache.org/jira/browse/CASSANDRA-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317215#comment-14317215
]
Kishan Karunaratne commented on CASSANDRA-8779:
---
This is for the insert statement:
{noformat}
["request buffer", #]
{noformat}
> Able to unintentionally nest tuples during insert
> -
>
> Key: CASSANDRA-8779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8779
> Project: Cassandra
> Issue Type: Bug
> Environment: Linux Mint 64-bit | ruby-driver 2.1 | java-driver 2.1 |
> C* 2.1.2
>Reporter: Kishan Karunaratne
>Assignee: Tyler Hobbs
>
> If I insert a tuple using an extra pair of ()'s, C* will let me do the
> insert, but (incorrectly) creates a nested tuple as the first tuple value.
> Upon doing a select statement, the result is jumbled and has weird binary in
> it (which I wasn't able to copy into here).
> Example using ruby-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)")
> complete = Cassandra::Tuple.new('foo', 123, true)
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", arguments:
> [complete])# extra ()'s here
> result = session.execute("SELECT b FROM mytable WHERE a=0").first
> p result['b']
> {noformat}
> Output:
> {noformat}
> #
> {noformat}
> Bug also confirmed using java-driver.
> Example using java-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)");
> TupleType t = TupleType.of(DataType.ascii(), DataType.cint(),
> DataType.cboolean());
> TupleValue complete = t.newValue("foo", 123, true);
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", complete); //
> extra ()'s here
> TupleValue r = session.execute("SELECT b FROM mytable WHERE
> a=0").one().getTupleValue("b");
> System.out.println(r);
> {noformat}
> Output:
> {noformat}
> ('foo{', null, null)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8784) -bash: bin/cassandra: No such file or directory
[ https://issues.apache.org/jira/browse/CASSANDRA-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317205#comment-14317205 ] Brandon Williams commented on CASSANDRA-8784: - Furthermore, if we want to examine communication skills, let's turn an eye toward the person who can't add a *description*. > -bash: bin/cassandra: No such file or directory > --- > > Key: CASSANDRA-8784 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8784 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: ubuntu 14.04 >Reporter: R Scott > Fix For: 2.1.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8784) -bash: bin/cassandra: No such file or directory
[ https://issues.apache.org/jira/browse/CASSANDRA-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317192#comment-14317192 ] Brandon Williams commented on CASSANDRA-8784: - Not only are you poor at reporting errors we can work with, but you excel at self-entitlement. Give us something to work with and we will work it, but all you've done is basically said ERROR. Great, we know there's an error, somewhere, caused by something. I'm not sure how you expect us to tackle that without more information, which you seem unwilling to give. > -bash: bin/cassandra: No such file or directory > --- > > Key: CASSANDRA-8784 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8784 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: ubuntu 14.04 >Reporter: R Scott > Fix For: 2.1.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8783) /etc/init.d/cassandra: 12: /etc/default/cassandra: -Dcassandra.compaction.priority=1: not found
[ https://issues.apache.org/jira/browse/CASSANDRA-8783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317177#comment-14317177 ] R Scott commented on CASSANDRA-8783: I am sure someone else will be willing to work with you on this error in future. > /etc/init.d/cassandra: 12: /etc/default/cassandra: > -Dcassandra.compaction.priority=1: not found > --- > > Key: CASSANDRA-8783 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8783 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: ubuntu >Reporter: R Scott > Fix For: 2.1.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8784) -bash: bin/cassandra: No such file or directory
[ https://issues.apache.org/jira/browse/CASSANDRA-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317173#comment-14317173 ] R Scott commented on CASSANDRA-8784: Not only are you a poor communicator, but rude as well > -bash: bin/cassandra: No such file or directory > --- > > Key: CASSANDRA-8784 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8784 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: ubuntu 14.04 >Reporter: R Scott > Fix For: 2.1.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8790) Improve handling of non-printable unicode characters in text fields and CQLSH
[
https://issues.apache.org/jira/browse/CASSANDRA-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317155#comment-14317155
]
Brandon Williams commented on CASSANDRA-8790:
-
Most operating systems have a way to enter unicode (as Tyler did in his
example) and unearthing the can of unicode worms that would be required to
handle escaping doesn't seem like much gain for the amount of effort.
> Improve handling of non-printable unicode characters in text fields and CQLSH
> -
>
> Key: CASSANDRA-8790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8790
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Johnny Miller
>Priority: Minor
>
> Currently to store a string/text value that contains a non-printable unicode
> character and than subsequently be able to query it CQLSH I need to store the
> field as a blob via the blobAsText and textAsBlob functions.
> This is not really optimal - it would be better if CQLSH handled this rather
> than having to model data around this limitation.
> For example:
> {code:title=Example Code|borderStyle=solid}
> String createTableCql = "CREATE TABLE IF NOT EXISTS test_ks.testunicode (id
> blob PRIMARY KEY, inserted_on timestamp, lorem text)";
> session.execute(createTableCql);
> System.out.println("Table created.");
>
> String dimension1 = "state";
> String dimension2 = "card";
> String key = dimension1 + '\u001C' + dimension2;
> Date now = new Date();
> String lorem = "Lorem ipsum dolor sit amet.";
>
> String insertcql = "INSERT INTO testunicode (id, inserted_on, lorem) VALUES
> (textAsBlob(?), ?, ?)";
> PreparedStatement ps = session.prepare(insertcql);
> BoundStatement bs = new BoundStatement(ps);
> bs.bind(key, now, lorem);
> session.execute(bs);
> System.out.println("Row inserted with key "+key);
>
> String selectcql = "SELECT blobAsText(id) AS id, inserted_on, lorem FROM
> testunicode WHERE id = textAsBlob(?)";
> PreparedStatement ps2 = session.prepare(selectcql);
> BoundStatement bs2 = new BoundStatement(ps2);
> bs2.bind(key);
> ResultSet results = session.execute(bs2);
>
> System.out.println("Got results...");
>
> for (Row row : results) {
> System.out.println(String.format("%-30s\t%-20s\t%-20s",
> row.getString("id"), row.getDate("inserted_on"), row.getString("lorem")));
> }
> {code}
> And to query via CQLSH:
> {code}
> select * from testunicode where id = 0x73746174651c63617264 ;
> id | inserted_on | lorem
> +--+-
> 0x73746174651c63617264 | 2015-02-11 20:32:20+ | Lorem ipsum dolor sit
> amet.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8784) -bash: bin/cassandra: No such file or directory
[ https://issues.apache.org/jira/browse/CASSANDRA-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317134#comment-14317134 ] Brandon Williams commented on CASSANDRA-8784: - Says the person who can't use jira correctly and add a description. > -bash: bin/cassandra: No such file or directory > --- > > Key: CASSANDRA-8784 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8784 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: ubuntu 14.04 >Reporter: R Scott > Fix For: 2.1.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8783) /etc/init.d/cassandra: 12: /etc/default/cassandra: -Dcassandra.compaction.priority=1: not found
[ https://issues.apache.org/jira/browse/CASSANDRA-8783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317131#comment-14317131 ] Brandon Williams commented on CASSANDRA-8783: - Then add a description so we have something to work on. > /etc/init.d/cassandra: 12: /etc/default/cassandra: > -Dcassandra.compaction.priority=1: not found > --- > > Key: CASSANDRA-8783 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8783 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: ubuntu >Reporter: R Scott > Fix For: 2.1.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8783) /etc/init.d/cassandra: 12: /etc/default/cassandra: -Dcassandra.compaction.priority=1: not found
[ https://issues.apache.org/jira/browse/CASSANDRA-8783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317117#comment-14317117 ] R Scott commented on CASSANDRA-8783: It has not been resolved!!! > /etc/init.d/cassandra: 12: /etc/default/cassandra: > -Dcassandra.compaction.priority=1: not found > --- > > Key: CASSANDRA-8783 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8783 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: ubuntu >Reporter: R Scott > Fix For: 2.1.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8784) -bash: bin/cassandra: No such file or directory
[ https://issues.apache.org/jira/browse/CASSANDRA-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317115#comment-14317115 ] R Scott commented on CASSANDRA-8784: Says the person who does not explain the error message > -bash: bin/cassandra: No such file or directory > --- > > Key: CASSANDRA-8784 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8784 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: ubuntu 14.04 >Reporter: R Scott > Fix For: 2.1.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317105#comment-14317105 ] mck edited comment on CASSANDRA-6091 at 2/11/15 10:28 PM: -- With handling of multiple token rages, we loose a little accuracy in the resulting splitSize because token ranges are so much smaller (eg from the corner splits), but apart from such errors evening out, isn't the more important goal here to have splits sized more consistently splitSize-ish so that users can tune and achieve a reasonably steady throughput of tasks? was (Author: michaelsembwever): With handling of multiple token rages, we loose a little accuracy in the resulting splitSize because token ranges are so much smaller (eg from the corner splits), but apart from such errors evening out, isn't the more important goal here is to have splits sized more consistently from splitSize so that users can tune and achieve a reasonably steady throughput of tasks? > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317105#comment-14317105 ] mck commented on CASSANDRA-6091: With handling of multiple token rages, we loose a little accuracy in the resulting splitSize because token ranges are so much smaller (eg from the corner splits), but apart from such errors evening out, isn't the more important goal here is to have splits sized more consistently from splitSize so that users can tune and achieve a reasonably steady throughput of tasks? > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317090#comment-14317090 ] mck edited comment on CASSANDRA-6091 at 2/11/15 10:20 PM: -- The approach in the patch is to do allow multiple token ranges per split. We do this with our custom input formats, and it is (very) effective in that it means splitSize is honoured. Handling multiple token ranges per split requires for example the code change found in CqlRecordReader whereby the reader must iterate over both rows and tokenRanges. The grouping of token ranges by common location sets, so that splits again honour the splitSize, happens in AbstractColumnFamilyInputForma.collectSplits(..) Token ranges do not need to be adjacent. Everything in this patch is done client-side. was (Author: michaelsembwever): The approach in the patch is to do allow multiple token ranges per split. We do this with our custom input formats, and it is (very) effective in that it means splitSize is honoured. Handling multiple token ranges per split requires for example the code change found in CqlRecordReader whereby the reader must iterate over both rows and tokenRanges. The grouping of token rages by common location sets, so that splits again honour the splitSize, happens in AbstractColumnFamilyInputForma.collectSplits(..) Token ranges do not need to be adjacent. Everything in this patch is done client-side. > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317090#comment-14317090 ] mck commented on CASSANDRA-6091: The approach in the patch is to do allow multiple token ranges per split. We do with our custom input formats, and it is (very) effective in that it means splitSize is honoured. Handling multiple token ranges per split requires for example the code change found in CqlRecordReader whereby the reader must iterate over both rows and tokenRanges. The grouping of token rages by common location sets, so that splits again honour the splitSize, happens in AbstractColumnFamilyInputForma.collectSplits(..) Token ranges do not need to be adjacent. Everything in this patch is done client-side. > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317090#comment-14317090 ] mck edited comment on CASSANDRA-6091 at 2/11/15 10:18 PM: -- The approach in the patch is to do allow multiple token ranges per split. We do this with our custom input formats, and it is (very) effective in that it means splitSize is honoured. Handling multiple token ranges per split requires for example the code change found in CqlRecordReader whereby the reader must iterate over both rows and tokenRanges. The grouping of token rages by common location sets, so that splits again honour the splitSize, happens in AbstractColumnFamilyInputForma.collectSplits(..) Token ranges do not need to be adjacent. Everything in this patch is done client-side. was (Author: michaelsembwever): The approach in the patch is to do allow multiple token ranges per split. We do with our custom input formats, and it is (very) effective in that it means splitSize is honoured. Handling multiple token ranges per split requires for example the code change found in CqlRecordReader whereby the reader must iterate over both rows and tokenRanges. The grouping of token rages by common location sets, so that splits again honour the splitSize, happens in AbstractColumnFamilyInputForma.collectSplits(..) Token ranges do not need to be adjacent. Everything in this patch is done client-side. > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8790) Improve handling of non-printable unicode characters in text fields and CQLSH
[
https://issues.apache.org/jira/browse/CASSANDRA-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Johnny Miller updated CASSANDRA-8790:
-
Description:
Currently to store a string/text value that contains a non-printable unicode
character and than subsequently be able to query it CQLSH I need to store the
field as a blob via the blobAsText and textAsBlob functions.
This is not really optimal - it would be better if CQLSH handled this rather
than having to model data around this limitation.
For example:
{code:title=Example Code|borderStyle=solid}
String createTableCql = "CREATE TABLE IF NOT EXISTS test_ks.testunicode (id
blob PRIMARY KEY, inserted_on timestamp, lorem text)";
session.execute(createTableCql);
System.out.println("Table created.");
String dimension1 = "state";
String dimension2 = "card";
String key = dimension1 + '\u001C' + dimension2;
Date now = new Date();
String lorem = "Lorem ipsum dolor sit amet.";
String insertcql = "INSERT INTO testunicode (id, inserted_on, lorem) VALUES
(textAsBlob(?), ?, ?)";
PreparedStatement ps = session.prepare(insertcql);
BoundStatement bs = new BoundStatement(ps);
bs.bind(key, now, lorem);
session.execute(bs);
System.out.println("Row inserted with key "+key);
String selectcql = "SELECT blobAsText(id) AS id, inserted_on, lorem FROM
testunicode WHERE id = textAsBlob(?)";
PreparedStatement ps2 = session.prepare(selectcql);
BoundStatement bs2 = new BoundStatement(ps2);
bs2.bind(key);
ResultSet results = session.execute(bs2);
System.out.println("Got results...");
for (Row row : results) {
System.out.println(String.format("%-30s\t%-20s\t%-20s",
row.getString("id"), row.getDate("inserted_on"), row.getString("lorem")));
}
{code}
And to query via CQLSH:
{code}
select * from testunicode where id = 0x73746174651c63617264 ;
id | inserted_on | lorem
+--+-
0x73746174651c63617264 | 2015-02-11 20:32:20+ | Lorem ipsum dolor sit amet.
{code}
was:
Currently to store a string/text value that contains a unicode character and
than subsequently be able to query it CQLSH I need to store the field as a blob
via the blobAsText and textAsBlob functions.
This is not really optimal - it would be better if CQLSH handled this rather
than having to model data around this limitation.
For example:
{code:title=Example Code|borderStyle=solid}
String createTableCql = "CREATE TABLE IF NOT EXISTS test_ks.testunicode (id
blob PRIMARY KEY, inserted_on timestamp, lorem text)";
session.execute(createTableCql);
System.out.println("Table created.");
String dimension1 = "state";
String dimension2 = "card";
String key = dimension1 + '\u001C' + dimension2;
Date now = new Date();
String lorem = "Lorem ipsum dolor sit amet.";
String insertcql = "INSERT INTO testunicode (id, inserted_on, lorem) VALUES
(textAsBlob(?), ?, ?)";
PreparedStatement ps = session.prepare(insertcql);
BoundStatement bs = new BoundStatement(ps);
bs.bind(key, now, lorem);
session.execute(bs);
System.out.println("Row inserted with key "+key);
String selectcql = "SELECT blobAsText(id) AS id, inserted_on, lorem FROM
testunicode WHERE id = textAsBlob(?)";
PreparedStatement ps2 = session.prepare(selectcql);
BoundStatement bs2 = new BoundStatement(ps2);
bs2.bind(key);
ResultSet results = session.execute(bs2);
System.out.println("Got results...");
for (Row row : results) {
System.out.println(String.format("%-30s\t%-20s\t%-20s",
row.getString("id"), row.getDate("inserted_on"), row.getString("lorem")));
}
{code}
And to query via CQLSH:
select * from testunicode where id = 0x73746174651c63617264 ;
id | inserted_on | lorem
+--+-
0x73746174651c63617264 | 2015-02-11 20:32:20+ | Lorem ipsum dolor sit amet.
> Improve handling of non-printable unicode characters in text fields and CQLSH
> -
>
> Key: CASSANDRA-8790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8790
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Johnny Miller
>Priority: Minor
>
> Currently to store a string/text value that contains a non-printable unicode
> character and than subsequently be able to query it CQLSH I need to store the
> field as a blob via the blobAsText and textAsBlob functions.
> This is not really optimal - it would be better if CQLSH handled this rather
> than having to model data around this limitation.
>
[jira] [Assigned] (CASSANDRA-7660) Indicate PK columns in "prepared" native protocol responses
[ https://issues.apache.org/jira/browse/CASSANDRA-7660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs reassigned CASSANDRA-7660: -- Assignee: Tyler Hobbs > Indicate PK columns in "prepared" native protocol responses > --- > > Key: CASSANDRA-7660 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7660 > Project: Cassandra > Issue Type: Sub-task > Components: Core >Reporter: Tyler Hobbs >Assignee: Tyler Hobbs > Labels: protocolv4 > Fix For: 3.0 > > > Currently, when native protocol drivers prepare statements, they need to > check their internal representation of the schema to determine which, if any, > of the columns are part of the partition key. If the native protocol > response indicated this, it would remove the drivers' dependency on having an > internal representation of the schema when preparing statements. > This would need to go in the v4 protocol. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8790) Improve handling of non-printable unicode characters in text fields and CQLSH
[
https://issues.apache.org/jira/browse/CASSANDRA-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317036#comment-14317036
]
Johnny Miller commented on CASSANDRA-8790:
--
The cause of this would just appear to be how CQLSH displays and interprets
characters like this. The actual data is stored correctly.
This works via CQLSH:
{code}
select * from testunicode1 where id = 'ק ר ש';
{code}
This doesn't:
{code}
SELECT * from testunicode where id = 'state\u001Ccard';
{code}
A potential solution is to introduce a function in CQLSH that will enable users
to escape and unescape fields such as this.
For example
{code}
SELECT * from testunicode where id = escape('state\u001Ccard');
{code}
> Improve handling of non-printable unicode characters in text fields and CQLSH
> -
>
> Key: CASSANDRA-8790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8790
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Johnny Miller
>Priority: Minor
>
> Currently to store a string/text value that contains a unicode character and
> than subsequently be able to query it CQLSH I need to store the field as a
> blob via the blobAsText and textAsBlob functions.
> This is not really optimal - it would be better if CQLSH handled this rather
> than having to model data around this limitation.
> For example:
> {code:title=Example Code|borderStyle=solid}
> String createTableCql = "CREATE TABLE IF NOT EXISTS test_ks.testunicode (id
> blob PRIMARY KEY, inserted_on timestamp, lorem text)";
> session.execute(createTableCql);
> System.out.println("Table created.");
>
> String dimension1 = "state";
> String dimension2 = "card";
> String key = dimension1 + '\u001C' + dimension2;
> Date now = new Date();
> String lorem = "Lorem ipsum dolor sit amet.";
>
> String insertcql = "INSERT INTO testunicode (id, inserted_on, lorem) VALUES
> (textAsBlob(?), ?, ?)";
> PreparedStatement ps = session.prepare(insertcql);
> BoundStatement bs = new BoundStatement(ps);
> bs.bind(key, now, lorem);
> session.execute(bs);
> System.out.println("Row inserted with key "+key);
>
> String selectcql = "SELECT blobAsText(id) AS id, inserted_on, lorem FROM
> testunicode WHERE id = textAsBlob(?)";
> PreparedStatement ps2 = session.prepare(selectcql);
> BoundStatement bs2 = new BoundStatement(ps2);
> bs2.bind(key);
> ResultSet results = session.execute(bs2);
>
> System.out.println("Got results...");
>
> for (Row row : results) {
> System.out.println(String.format("%-30s\t%-20s\t%-20s",
> row.getString("id"), row.getDate("inserted_on"), row.getString("lorem")));
> }
> {code}
> And to query via CQLSH:
> select * from testunicode where id = 0x73746174651c63617264 ;
> id | inserted_on | lorem
> +--+-
> 0x73746174651c63617264 | 2015-02-11 20:32:20+ | Lorem ipsum dolor sit
> amet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8790) Improve handling of non-printable unicode characters in text fields and CQLSH
[
https://issues.apache.org/jira/browse/CASSANDRA-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Johnny Miller updated CASSANDRA-8790:
-
Description:
Currently to store a string/text value that contains a unicode character and
than subsequently be able to query it CQLSH I need to store the field as a blob
via the blobAsText and textAsBlob functions.
This is not really optimal - it would be better if CQLSH handled this rather
than having to model data around this limitation.
For example:
{code:title=Example Code|borderStyle=solid}
String createTableCql = "CREATE TABLE IF NOT EXISTS test_ks.testunicode (id
blob PRIMARY KEY, inserted_on timestamp, lorem text)";
session.execute(createTableCql);
System.out.println("Table created.");
String dimension1 = "state";
String dimension2 = "card";
String key = dimension1 + '\u001C' + dimension2;
Date now = new Date();
String lorem = "Lorem ipsum dolor sit amet.";
String insertcql = "INSERT INTO testunicode (id, inserted_on, lorem) VALUES
(textAsBlob(?), ?, ?)";
PreparedStatement ps = session.prepare(insertcql);
BoundStatement bs = new BoundStatement(ps);
bs.bind(key, now, lorem);
session.execute(bs);
System.out.println("Row inserted with key "+key);
String selectcql = "SELECT blobAsText(id) AS id, inserted_on, lorem FROM
testunicode WHERE id = textAsBlob(?)";
PreparedStatement ps2 = session.prepare(selectcql);
BoundStatement bs2 = new BoundStatement(ps2);
bs2.bind(key);
ResultSet results = session.execute(bs2);
System.out.println("Got results...");
for (Row row : results) {
System.out.println(String.format("%-30s\t%-20s\t%-20s",
row.getString("id"), row.getDate("inserted_on"), row.getString("lorem")));
}
{code}
And to query via CQLSH:
select * from testunicode where id = 0x73746174651c63617264 ;
id | inserted_on | lorem
+--+-
0x73746174651c63617264 | 2015-02-11 20:32:20+ | Lorem ipsum dolor sit amet.
was:
Currently to store a string/text value that contains a unicode character and
than subsequently be able to query it CQLSH I need to store the field as a blob
via the blobAsText and textAsBlob functions.
This is not really optimal - it would be better if CQLSH handled this rather
than having to model data around this limitation.
For example:
{code:title=Example Code|borderStyle=solid}
String createTableCql = "CREATE TABLE IF NOT EXISTS test_ks.testunicode (id
blob PRIMARY KEY, inserted_on timestamp, lorem text)";
session.execute(createTableCql);
System.out.println("Table created.");
String dimension1 = "state";
String dimension2 = "card";
String key = dimension1 + '\u001C' + dimension2;
Date now = new Date();
String lorem = "Lorem ipsum dolor sit amet.";
String insertcql = "INSERT INTO testunicode (id, inserted_on, lorem) VALUES
(textAsBlob(?), ?, ?)";
PreparedStatement ps = session.prepare(insertcql);
BoundStatement bs = new BoundStatement(ps);
bs.bind(key, now, lorem);
session.execute(bs);
System.out.println("Row inserted with key "+key);
String selectcql = "SELECT blobAsText(id) AS id, inserted_on, lorem FROM
testunicode WHERE id = textAsBlob(?)";
PreparedStatement ps2 = session.prepare(selectcql);
BoundStatement bs2 = new BoundStatement(ps2);
bs2.bind(key);
ResultSet results = session.execute(bs2);
System.out.println("Got results...");
for (Row row : results) {
System.out.println(String.format("%-30s\t%-20s\t%-20s",
row.getString("id"), row.getDate("inserted_on"), row.getString("lorem")));
}
{code}
And to query via CQLSH:
select * from testunicode where id = 0x73746174651c63617264 ;
id | inserted_on | lorem
+--+-
0x73746174651c63617264 | 2015-02-11 20:32:20+ | Lorem ipsum dolor sit amet.
Summary: Improve handling of non-printable unicode characters in text
fields and CQLSH (was: Improve handling of unicode characters in text fields)
> Improve handling of non-printable unicode characters in text fields and CQLSH
> -
>
> Key: CASSANDRA-8790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8790
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Johnny Miller
>Priority: Minor
>
> Currently to store a string/text value that contains a unicode character and
> than subsequently be able to query it CQLSH I need to store the field as a
> blob via the blobAsText and textAsBlob functions.
> This is not rea
[jira] [Commented] (CASSANDRA-8790) Improve handling of unicode characters in text fields
[
https://issues.apache.org/jira/browse/CASSANDRA-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317006#comment-14317006
]
Johnny Miller commented on CASSANDRA-8790:
--
I think it is specifically around non-printable unicode characters e.g. \u001C
> Improve handling of unicode characters in text fields
> -
>
> Key: CASSANDRA-8790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8790
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Johnny Miller
>Priority: Minor
>
> Currently to store a string/text value that contains a unicode character and
> than subsequently be able to query it CQLSH I need to store the field as a
> blob via the blobAsText and textAsBlob functions.
> This is not really optimal - it would be better if CQLSH handled this rather
> than having to model data around this limitation.
> For example:
> {code:title=Example Code|borderStyle=solid}
> String createTableCql = "CREATE TABLE IF NOT EXISTS test_ks.testunicode (id
> blob PRIMARY KEY, inserted_on timestamp, lorem text)";
> session.execute(createTableCql);
> System.out.println("Table created.");
>
> String dimension1 = "state";
> String dimension2 = "card";
> String key = dimension1 + '\u001C' + dimension2;
> Date now = new Date();
> String lorem = "Lorem ipsum dolor sit amet.";
>
> String insertcql = "INSERT INTO testunicode (id, inserted_on, lorem) VALUES
> (textAsBlob(?), ?, ?)";
> PreparedStatement ps = session.prepare(insertcql);
> BoundStatement bs = new BoundStatement(ps);
> bs.bind(key, now, lorem);
> session.execute(bs);
> System.out.println("Row inserted with key "+key);
>
> String selectcql = "SELECT blobAsText(id) AS id, inserted_on, lorem FROM
> testunicode WHERE id = textAsBlob(?)";
> PreparedStatement ps2 = session.prepare(selectcql);
> BoundStatement bs2 = new BoundStatement(ps2);
> bs2.bind(key);
> ResultSet results = session.execute(bs2);
>
> System.out.println("Got results...");
>
> for (Row row : results) {
> System.out.println(String.format("%-30s\t%-20s\t%-20s",
> row.getString("id"), row.getDate("inserted_on"), row.getString("lorem")));
> }
> {code}
> And to query via CQLSH:
> select * from testunicode where id = 0x73746174651c63617264 ;
> id | inserted_on | lorem
> +--+-
> 0x73746174651c63617264 | 2015-02-11 20:32:20+ | Lorem ipsum dolor sit
> amet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8789) Revisit how OutboundTcpConnection pools two connections for different message types
[ https://issues.apache.org/jira/browse/CASSANDRA-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317003#comment-14317003 ] Benedict commented on CASSANDRA-8789: - bq. relaxing ordering guarantees is one of those things you want to opt into. We don't really have any ordering guarantees here. Currently we have single request-response rounds, with the entirety of each being encoded in a single message. Even if we didn't, the way we accumulate message results on the coordinator is to wait until they all arrive then combine the results, so it would still be fine. In the medium term we want to introduce streaming reads/writes, at which point we won't need any separate connection at all, since we can split each message up into bite size chunks below our maximum, and interleave them with the other messages. In the meantime I don't see any significant downside to this approach. It does potentially mean that a higher bandwidth allocation will be assigned to huge messages, but that's probably for the best anyway. They'll utilise bandwidth better, and we want them out of the VMs ASAP to reduce the temporary burden they impose. Besides, as [~jbellis] says, we're improving well behaved users with these considerations only affecting the naughty ones. > Revisit how OutboundTcpConnection pools two connections for different message > types > --- > > Key: CASSANDRA-8789 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8789 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.0 > > > I was looking at this trying to understand what messages flow over which > connection. > For reads the request goes out over the command connection and the response > comes back over the ack connection. > For writes the request goes out over the command connection and the response > comes back over the command connection. > Reads get a dedicated socket for responses. Mutation commands and responses > both travel over the same socket along with read requests. > Sockets are used uni-directional so there are actually four sockets in play > and four threads at each node (2 inbounded, 2 outbound). > CASSANDRA-488 doesn't leave a record what the impact of this change was. If > someone remembers what situations were made better it would be good to know. > I am not clear on when/how this is helpful. The consumer side shouldn't be > blocking so the only head of line blocking issue is the time it takes to > transfer data over the wire. > If message size is the cause of blocking issues then the current design mixes > small messages and large messages on the same connection retaining the head > of line blocking. > Read requests share the same connection as write requests (which are large), > and write acknowledgments (which are small) share the same connections as > write requests. The only winner is read acknowledgements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8786) NullPointerException in ColumnDefinition.hasIndexOption
[
https://issues.apache.org/jira/browse/CASSANDRA-8786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs updated CASSANDRA-8786:
---
Description:
We have a Cassandra cluster that we've been using through many upgrades, and
thus most of our column families have originally been created by Thrift. We are
on Cassandra 2.1.2 now.
We've now ported most of our code to use CQL, and our code occasionally tries
to recreate tables with "IF NOT EXISTS" to work properly on development /
testing environments.
When we issue the CQL statement "CREATE INDEX IF NOT EXISTS index ON
"tableName" (accountId)" (this index does exist on that table already), we get
a {{DriverInternalError: An unexpected error occurred server side on
cass_host/xx.xxx.xxx.xxx:9042: java.lang.NullPointerException}}
The error on the server is:
{noformat}
java.lang.NullPointerException: null
at
org.apache.cassandra.config.ColumnDefinition.hasIndexOption(ColumnDefinition.java:489)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.cql3.statements.CreateIndexStatement.validate(CreateIndexStatement.java:87)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:224)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:248)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
~[apache-cassandra-2.1.2.jar:2.1.2]
{noformat}
This happens every time we run this CQL statement. We've tried to reproduce it
in a test cassandra cluster by creating the table according to the exact
"DESCRIBE TABLE" specification, but then this NullPointerException doesn't
happon upon the CREATE INDEX one. So it seems that the tables on our production
cluster (that were originally created through thrift) are still subtly
different schema-wise then a freshly created table according to the same
creation statement.
was:
We have a Cassandra cluster that we've been using through many upgrades, and
thus most of our column families have originally been created by Thrift. We are
on Cassandra 2.1.2 now.
We've now ported most of our code to use CQL, and our code occasionally tries
to recreate tables with "IF NOT EXISTS" to work properly on development /
testing environments.
When we issue the CQL statement "CREATE INDEX IF NOT EXISTS index ON
"tableName" (accountId)" (this index does exist on that table already), we get
a DriverInternalError: An unexpected error occurred server side on
cass_host/xx.xxx.xxx.xxx:9042: java.lang.NullPointerException
The error on the server is java.lang.NullPointerException: null
at
org.apache.cassandra.config.ColumnDefinition.hasIndexOption(ColumnDefinition.java:489)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.cql3.statements.CreateIndexStatement.validate(CreateIndexStatement.java:87)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:224)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:248)
~[apache-cassandra-2.1.2.jar:2.1.2]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
~[apache-cassandra-2.1.2.jar:2.1.2]
This happens every time we run this CQL statement. We've tried to reproduce it
in a test cassandra cluster by creating the table according to the exact
"DESCRIBE TABLE" specification, but then this NullPointerException doesn't
happon upon the CREATE INDEX one. So it seems that the tables on our production
cluster (that were originally created through thrift) are still subtly
different schema-wise then a freshly created table according to the same
creation statement.
> NullPointerException in ColumnDefinition.hasIndexOption
> ---
>
> Key: CASSANDRA-8786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8786
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 2.1.2
>Reporter: Mathijs Vogelzang
>
> We have a Cassandra cluster that we've been using through many upgrades, and
> thus most of our column families have originally been created by Thrift. We
> are on Cassandra 2.1.2 now.
> We've now ported most of our code to use CQL, and our code occasionally tries
> to recreate tables with "IF NOT EXISTS" to work properly on development /
> testing environments.
> When we issue the CQL statement "CREATE INDEX IF NOT EXISTS index ON
> "tableName" (accountId)" (this index does exist on that table already), we
> get a {{DriverInternalError: An unexpected error occurred server side on
> cass_host/xx.xxx.xxx.xxx:9042:
[jira] [Commented] (CASSANDRA-8790) Improve handling of unicode characters in text fields
[
https://issues.apache.org/jira/browse/CASSANDRA-8790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316991#comment-14316991
]
Tyler Hobbs commented on CASSANDRA-8790:
You can work directly with unicode in cqlsh:
{noformat}
cqlsh:ks1> CREATE TABLE foo (a int PRIMARY KEY, b text);
cqlsh:ks1> INSERT INTO foo (a, b) VALUES (0, 'ק ר ש');
cqlsh:ks1> SELECT * FROM foo;
a | b
---+---
0 | ק ר ש
(1 rows)
{noformat}
What am I missing?
> Improve handling of unicode characters in text fields
> -
>
> Key: CASSANDRA-8790
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8790
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Johnny Miller
>Priority: Minor
>
> Currently to store a string/text value that contains a unicode character and
> than subsequently be able to query it CQLSH I need to store the field as a
> blob via the blobAsText and textAsBlob functions.
> This is not really optimal - it would be better if CQLSH handled this rather
> than having to model data around this limitation.
> For example:
> {code:title=Example Code|borderStyle=solid}
> String createTableCql = "CREATE TABLE IF NOT EXISTS test_ks.testunicode (id
> blob PRIMARY KEY, inserted_on timestamp, lorem text)";
> session.execute(createTableCql);
> System.out.println("Table created.");
>
> String dimension1 = "state";
> String dimension2 = "card";
> String key = dimension1 + '\u001C' + dimension2;
> Date now = new Date();
> String lorem = "Lorem ipsum dolor sit amet.";
>
> String insertcql = "INSERT INTO testunicode (id, inserted_on, lorem) VALUES
> (textAsBlob(?), ?, ?)";
> PreparedStatement ps = session.prepare(insertcql);
> BoundStatement bs = new BoundStatement(ps);
> bs.bind(key, now, lorem);
> session.execute(bs);
> System.out.println("Row inserted with key "+key);
>
> String selectcql = "SELECT blobAsText(id) AS id, inserted_on, lorem FROM
> testunicode WHERE id = textAsBlob(?)";
> PreparedStatement ps2 = session.prepare(selectcql);
> BoundStatement bs2 = new BoundStatement(ps2);
> bs2.bind(key);
> ResultSet results = session.execute(bs2);
>
> System.out.println("Got results...");
>
> for (Row row : results) {
> System.out.println(String.format("%-30s\t%-20s\t%-20s",
> row.getString("id"), row.getDate("inserted_on"), row.getString("lorem")));
> }
> {code}
> And to query via CQLSH:
> select * from testunicode where id = 0x73746174651c63617264 ;
> id | inserted_on | lorem
> +--+-
> 0x73746174651c63617264 | 2015-02-11 20:32:20+ | Lorem ipsum dolor sit
> amet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8689) Assertion error in 2.1.2: ERROR [IndexSummaryManager:1]
[
https://issues.apache.org/jira/browse/CASSANDRA-8689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316974#comment-14316974
]
T Jake Luciani commented on CASSANDRA-8689:
---
Apologies for the delay, could you rebase this?
It would be good to put your comment above as a code comment explaining any why
we need sstable map.
I've been trying to trigger this issue with stress by re summarizing every 1
minute. I agree with your guess that this is the problem causing this error
but it would be great to repro.
> Assertion error in 2.1.2: ERROR [IndexSummaryManager:1]
> ---
>
> Key: CASSANDRA-8689
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8689
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeff Liu
>Assignee: Benedict
> Fix For: 2.1.3
>
>
> After upgrading a 6 nodes cassandra from 2.1.0 to 2.1.2, start getting the
> following assertion error.
> {noformat}
> ERROR [IndexSummaryManager:1] 2015-01-26 20:55:40,451
> CassandraDaemon.java:153 - Exception in thread
> Thread[IndexSummaryManager:1,1,main]
> java.lang.AssertionError: null
> at org.apache.cassandra.io.util.Memory.size(Memory.java:307)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.io.sstable.IndexSummary.getOffHeapSize(IndexSummary.java:192)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.io.sstable.SSTableReader.getIndexSummaryOffHeapSize(SSTableReader.java:1070)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:292)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:77)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_45]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> [na:1.7.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> [na:1.7.0_45]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> {noformat}
> cassandra service is still running despite the issue. Node has total 8G
> memory with 2G allocated to heap. We are basically running read queries to
> retrieve data out of cassandra.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-8790) Improve handling of unicode characters in text fields
Johnny Miller created CASSANDRA-8790:
Summary: Improve handling of unicode characters in text fields
Key: CASSANDRA-8790
URL: https://issues.apache.org/jira/browse/CASSANDRA-8790
Project: Cassandra
Issue Type: Improvement
Reporter: Johnny Miller
Priority: Minor
Currently to store a string/text value that contains a unicode character and
than subsequently be able to query it CQLSH I need to store the field as a blob
via the blobAsText and textAsBlob functions.
This is not really optimal - it would be better if CQLSH handled this rather
than having to model data around this limitation.
For example:
{code:title=Example Code|borderStyle=solid}
String createTableCql = "CREATE TABLE IF NOT EXISTS test_ks.testunicode (id
blob PRIMARY KEY, inserted_on timestamp, lorem text)";
session.execute(createTableCql);
System.out.println("Table created.");
String dimension1 = "state";
String dimension2 = "card";
String key = dimension1 + '\u001C' + dimension2;
Date now = new Date();
String lorem = "Lorem ipsum dolor sit amet.";
String insertcql = "INSERT INTO testunicode (id, inserted_on, lorem) VALUES
(textAsBlob(?), ?, ?)";
PreparedStatement ps = session.prepare(insertcql);
BoundStatement bs = new BoundStatement(ps);
bs.bind(key, now, lorem);
session.execute(bs);
System.out.println("Row inserted with key "+key);
String selectcql = "SELECT blobAsText(id) AS id, inserted_on, lorem FROM
testunicode WHERE id = textAsBlob(?)";
PreparedStatement ps2 = session.prepare(selectcql);
BoundStatement bs2 = new BoundStatement(ps2);
bs2.bind(key);
ResultSet results = session.execute(bs2);
System.out.println("Got results...");
for (Row row : results) {
System.out.println(String.format("%-30s\t%-20s\t%-20s",
row.getString("id"), row.getDate("inserted_on"), row.getString("lorem")));
}
{code}
And to query via CQLSH:
select * from testunicode where id = 0x73746174651c63617264 ;
id | inserted_on | lorem
+--+-
0x73746174651c63617264 | 2015-02-11 20:32:20+ | Lorem ipsum dolor sit amet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8779) Able to unintentionally nest tuples during insert
[
https://issues.apache.org/jira/browse/CASSANDRA-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316964#comment-14316964
]
Tyler Hobbs commented on CASSANDRA-8779:
bq. TypeError: 'tuple' does not have the buffer interface
That's a bit of an obtuse error message, but basically the driver is expecting
AsciiType (since it's the first item in the tuple, from Cassandra's
perspective) and the driver encounters a TypeError when trying to write it as
an AsciiType value.
My guess is that the Ruby and Java drivers aren't catching this type error and
are writing out some bad binary. Somehow that binary is passing validation in
Cassandra. Can you get a dump of the binary Query message that's being sent by
the driver?
> Able to unintentionally nest tuples during insert
> -
>
> Key: CASSANDRA-8779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8779
> Project: Cassandra
> Issue Type: Bug
> Environment: Linux Mint 64-bit | ruby-driver 2.1 | java-driver 2.1 |
> C* 2.1.2
>Reporter: Kishan Karunaratne
>Assignee: Tyler Hobbs
>
> If I insert a tuple using an extra pair of ()'s, C* will let me do the
> insert, but (incorrectly) creates a nested tuple as the first tuple value.
> Upon doing a select statement, the result is jumbled and has weird binary in
> it (which I wasn't able to copy into here).
> Example using ruby-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)")
> complete = Cassandra::Tuple.new('foo', 123, true)
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", arguments:
> [complete])# extra ()'s here
> result = session.execute("SELECT b FROM mytable WHERE a=0").first
> p result['b']
> {noformat}
> Output:
> {noformat}
> #
> {noformat}
> Bug also confirmed using java-driver.
> Example using java-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)");
> TupleType t = TupleType.of(DataType.ascii(), DataType.cint(),
> DataType.cboolean());
> TupleValue complete = t.newValue("foo", 123, true);
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", complete); //
> extra ()'s here
> TupleValue r = session.execute("SELECT b FROM mytable WHERE
> a=0").one().getTupleValue("b");
> System.out.println(r);
> {noformat}
> Output:
> {noformat}
> ('foo{', null, null)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7970) JSON support for CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316950#comment-14316950
]
Tyler Hobbs commented on CASSANDRA-7970:
The branch has been updated to address your comments and merge in the latest
trunk.
bq. not sure, if you already have, but a unit test to test all combinations in
Json#handleCaseSensitivity would be great
There is a unit test: {{JsonTest.testCaseSensitivity()}}. I believe that
covers all combinations; let me know if I've missed something.
> JSON support for CQL
>
>
> Key: CASSANDRA-7970
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7970
> Project: Cassandra
> Issue Type: New Feature
> Components: API
>Reporter: Jonathan Ellis
>Assignee: Tyler Hobbs
> Fix For: 3.0
>
> Attachments: 7970-trunk-v1.txt
>
>
> JSON is popular enough that not supporting it is becoming a competitive
> weakness. We can add JSON support in a way that is compatible with our
> performance goals by *mapping* JSON to an existing schema: one JSON documents
> maps to one CQL row.
> Thus, it is NOT a goal to support schemaless documents, which is a misfeature
> [1] [2] [3]. Rather, it is to allow a convenient way to easily turn a JSON
> document from a service or a user into a CQL row, with all the validation
> that entails.
> Since we are not looking to support schemaless documents, we will not be
> adding a JSON data type (CASSANDRA-6833) a la postgresql. Rather, we will
> map the JSON to UDT, collections, and primitive CQL types.
> Here's how this might look:
> {code}
> CREATE TYPE address (
> street text,
> city text,
> zip_code int,
> phones set
> );
> CREATE TABLE users (
> id uuid PRIMARY KEY,
> name text,
> addresses map
> );
> INSERT INTO users JSON
> {‘id’: 4b856557-7153,
>‘name’: ‘jbellis’,
>‘address’: {“home”: {“street”: “123 Cassandra Dr”,
> “city”: “Austin”,
> “zip_code”: 78747,
> “phones”: [2101234567]}}};
> SELECT JSON id, address FROM users;
> {code}
> (We would also want to_json and from_json functions to allow mapping a single
> column's worth of data. These would not require extra syntax.)
> [1] http://rustyrazorblade.com/2014/07/the-myth-of-schema-less/
> [2] https://blog.compose.io/schema-less-is-usually-a-lie/
> [3] http://dl.acm.org/citation.cfm?id=2481247
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8789) Revisit how OutboundTcpConnection pools two connections for different message types
[ https://issues.apache.org/jira/browse/CASSANDRA-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316905#comment-14316905 ] Ariel Weisberg commented on CASSANDRA-8789: --- Doing it based on size rather then flow means you are changing order guarantees based on something that is relatively arbitrary. For instance if some code were steaming data as several large messages you might have the last message be smaller because it is the tail of the data, and that could end up delivered before the prior messages. I think that relaxing ordering guarantees is one of those things you want to opt into. > Revisit how OutboundTcpConnection pools two connections for different message > types > --- > > Key: CASSANDRA-8789 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8789 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.0 > > > I was looking at this trying to understand what messages flow over which > connection. > For reads the request goes out over the command connection and the response > comes back over the ack connection. > For writes the request goes out over the command connection and the response > comes back over the command connection. > Reads get a dedicated socket for responses. Mutation commands and responses > both travel over the same socket along with read requests. > Sockets are used uni-directional so there are actually four sockets in play > and four threads at each node (2 inbounded, 2 outbound). > CASSANDRA-488 doesn't leave a record what the impact of this change was. If > someone remembers what situations were made better it would be good to know. > I am not clear on when/how this is helpful. The consumer side shouldn't be > blocking so the only head of line blocking issue is the time it takes to > transfer data over the wire. > If message size is the cause of blocking issues then the current design mixes > small messages and large messages on the same connection retaining the head > of line blocking. > Read requests share the same connection as write requests (which are large), > and write acknowledgments (which are small) share the same connections as > write requests. The only winner is read acknowledgements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8789) Revisit how OutboundTcpConnection pools two connections for different message types
[ https://issues.apache.org/jira/browse/CASSANDRA-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316882#comment-14316882 ] Jonathan Ellis commented on CASSANDRA-8789: --- bq. Or possibly open up our second connection, lazily, only to service messages > ~64Kb in size I like this idea. Simple to implement and only people doing something suboptimal in the first place run into it. > Revisit how OutboundTcpConnection pools two connections for different message > types > --- > > Key: CASSANDRA-8789 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8789 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.0 > > > I was looking at this trying to understand what messages flow over which > connection. > For reads the request goes out over the command connection and the response > comes back over the ack connection. > For writes the request goes out over the command connection and the response > comes back over the command connection. > Reads get a dedicated socket for responses. Mutation commands and responses > both travel over the same socket along with read requests. > Sockets are used uni-directional so there are actually four sockets in play > and four threads at each node (2 inbounded, 2 outbound). > CASSANDRA-488 doesn't leave a record what the impact of this change was. If > someone remembers what situations were made better it would be good to know. > I am not clear on when/how this is helpful. The consumer side shouldn't be > blocking so the only head of line blocking issue is the time it takes to > transfer data over the wire. > If message size is the cause of blocking issues then the current design mixes > small messages and large messages on the same connection retaining the head > of line blocking. > Read requests share the same connection as write requests (which are large), > and write acknowledgments (which are small) share the same connections as > write requests. The only winner is read acknowledgements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8789) Revisit how OutboundTcpConnection pools two connections for different message types
[ https://issues.apache.org/jira/browse/CASSANDRA-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316874#comment-14316874 ] Benedict commented on CASSANDRA-8789: - We do have one risk, which is single gigantic messages (say hundreds of megabytes, or gigabytes). Seems we should perhaps first change the message protocol to support chunked message delivery. Or possibly open up our second connection, lazily, only to service messages > ~64Kb in size. > Revisit how OutboundTcpConnection pools two connections for different message > types > --- > > Key: CASSANDRA-8789 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8789 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.0 > > > I was looking at this trying to understand what messages flow over which > connection. > For reads the request goes out over the command connection and the response > comes back over the ack connection. > For writes the request goes out over the command connection and the response > comes back over the command connection. > Reads get a dedicated socket for responses. Mutation commands and responses > both travel over the same socket along with read requests. > Sockets are used uni-directional so there are actually four sockets in play > and four threads at each node (2 inbounded, 2 outbound). > CASSANDRA-488 doesn't leave a record what the impact of this change was. If > someone remembers what situations were made better it would be good to know. > I am not clear on when/how this is helpful. The consumer side shouldn't be > blocking so the only head of line blocking issue is the time it takes to > transfer data over the wire. > If message size is the cause of blocking issues then the current design mixes > small messages and large messages on the same connection retaining the head > of line blocking. > Read requests share the same connection as write requests (which are large), > and write acknowledgments (which are small) share the same connections as > write requests. The only winner is read acknowledgements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8789) Revisit how OutboundTcpConnection pools two connections for different message types
[ https://issues.apache.org/jira/browse/CASSANDRA-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316866#comment-14316866 ] Ariel Weisberg commented on CASSANDRA-8789: --- You don't sound like your concerned about single message head of line blocking. The kind of thing you would solve with something SCTPish. I'm fine with that if it's not an issue. This sounds promising so I will try this as a follow on to coalescing. > Revisit how OutboundTcpConnection pools two connections for different message > types > --- > > Key: CASSANDRA-8789 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8789 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.0 > > > I was looking at this trying to understand what messages flow over which > connection. > For reads the request goes out over the command connection and the response > comes back over the ack connection. > For writes the request goes out over the command connection and the response > comes back over the command connection. > Reads get a dedicated socket for responses. Mutation commands and responses > both travel over the same socket along with read requests. > Sockets are used uni-directional so there are actually four sockets in play > and four threads at each node (2 inbounded, 2 outbound). > CASSANDRA-488 doesn't leave a record what the impact of this change was. If > someone remembers what situations were made better it would be good to know. > I am not clear on when/how this is helpful. The consumer side shouldn't be > blocking so the only head of line blocking issue is the time it takes to > transfer data over the wire. > If message size is the cause of blocking issues then the current design mixes > small messages and large messages on the same connection retaining the head > of line blocking. > Read requests share the same connection as write requests (which are large), > and write acknowledgments (which are small) share the same connections as > write requests. The only winner is read acknowledgements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8606) sstablesplit does not remove original sstable
[ https://issues.apache.org/jira/browse/CASSANDRA-8606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316847#comment-14316847 ] Jeremy Hanna commented on CASSANDRA-8606: - [~jblangs...@datastax.com] told me that he only tested with 2.1. > sstablesplit does not remove original sstable > - > > Key: CASSANDRA-8606 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8606 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Branimir Lambov > Fix For: 2.1.3 > > > sstablesplit leaves the original file on disk, it should not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8783) /etc/init.d/cassandra: 12: /etc/default/cassandra: -Dcassandra.compaction.priority=1: not found
[ https://issues.apache.org/jira/browse/CASSANDRA-8783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams resolved CASSANDRA-8783. - Resolution: Invalid > /etc/init.d/cassandra: 12: /etc/default/cassandra: > -Dcassandra.compaction.priority=1: not found > --- > > Key: CASSANDRA-8783 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8783 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: ubuntu >Reporter: R Scott > Fix For: 2.1.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8784) -bash: bin/cassandra: No such file or directory
[ https://issues.apache.org/jira/browse/CASSANDRA-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams resolved CASSANDRA-8784. - Resolution: Invalid Invalid regardless, but especially without a description. > -bash: bin/cassandra: No such file or directory > --- > > Key: CASSANDRA-8784 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8784 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: ubuntu 14.04 >Reporter: R Scott > Fix For: 2.1.2 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8771) Remove commit log segment recycling
[ https://issues.apache.org/jira/browse/CASSANDRA-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316754#comment-14316754 ] Ariel Weisberg edited comment on CASSANDRA-8771 at 2/11/15 6:49 PM: There is also the option of calling fallocate via JNA. What that does is an open question for me as it is file system specific. Red Hat in their performance guide claim that it will encourage better allocation. With compression, at least the current proposed implementation it is a little undesirable because it doesn't get very close to the target size of 32 megabytes after compression shrinks the data in the segment. was (Author: aweisberg): There is also the option of calling fallocate via JNA. What that does is an open question for me as it is file system specific. Red Hat in their performance guide claim that it will encourage better allocation. > Remove commit log segment recycling > --- > > Key: CASSANDRA-8771 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8771 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg > Labels: commitlog > > For discussion > Commit log segment recycling introduces a lot of complexity in the existing > code. > CASSANDRA-8729 is a side effect of commit log segment recycling and > addressing it will require memory management code and thread coordination for > memory that the filesystem will no longer handle for us. > There is some discussion about what storage configurations actually benefit > from preallocated files. Fast random access devices like SSDs, or > non-volatile write caches etc. make the distinction not that great. > I haven't measured any difference in throughput for bulk appending vs > overwriting although it was pointed out that I didn't test with concurrent IO > streams. > What would it take to make removing commit log segment recycling acceptable? > Maybe a benchmark on a spinning disk that measures the performance impact of > preallocation when there are other IO streams? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8771) Remove commit log segment recycling
[ https://issues.apache.org/jira/browse/CASSANDRA-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316754#comment-14316754 ] Ariel Weisberg commented on CASSANDRA-8771: --- There is also the option of calling fallocate via JNA. What that does is an open question for me as it is file system specific. Red Hat in their performance guide claim that it will encourage better allocation. > Remove commit log segment recycling > --- > > Key: CASSANDRA-8771 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8771 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg > Labels: commitlog > > For discussion > Commit log segment recycling introduces a lot of complexity in the existing > code. > CASSANDRA-8729 is a side effect of commit log segment recycling and > addressing it will require memory management code and thread coordination for > memory that the filesystem will no longer handle for us. > There is some discussion about what storage configurations actually benefit > from preallocated files. Fast random access devices like SSDs, or > non-volatile write caches etc. make the distinction not that great. > I haven't measured any difference in throughput for bulk appending vs > overwriting although it was pointed out that I didn't test with concurrent IO > streams. > What would it take to make removing commit log segment recycling acceptable? > Maybe a benchmark on a spinning disk that measures the performance impact of > preallocation when there are other IO streams? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8789) Revisit how OutboundTcpConnection pools two connections for different message types
[ https://issues.apache.org/jira/browse/CASSANDRA-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-8789: Fix Version/s: 3.0 > Revisit how OutboundTcpConnection pools two connections for different message > types > --- > > Key: CASSANDRA-8789 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8789 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Fix For: 3.0 > > > I was looking at this trying to understand what messages flow over which > connection. > For reads the request goes out over the command connection and the response > comes back over the ack connection. > For writes the request goes out over the command connection and the response > comes back over the command connection. > Reads get a dedicated socket for responses. Mutation commands and responses > both travel over the same socket along with read requests. > Sockets are used uni-directional so there are actually four sockets in play > and four threads at each node (2 inbounded, 2 outbound). > CASSANDRA-488 doesn't leave a record what the impact of this change was. If > someone remembers what situations were made better it would be good to know. > I am not clear on when/how this is helpful. The consumer side shouldn't be > blocking so the only head of line blocking issue is the time it takes to > transfer data over the wire. > If message size is the cause of blocking issues then the current design mixes > small messages and large messages on the same connection retaining the head > of line blocking. > Read requests share the same connection as write requests (which are large), > and write acknowledgments (which are small) share the same connections as > write requests. The only winner is read acknowledgements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8789) Revisit how OutboundTcpConnection pools two connections for different message types
[ https://issues.apache.org/jira/browse/CASSANDRA-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316731#comment-14316731 ] Benedict commented on CASSANDRA-8789: - I think it makes more sense to have separate queues for each message type, that we round-robin over, so that a lot of large messages of one type can never unduly delay small messages of another. This permits us to coalesce more messages (with or without the coalescing patch of CASSANDRA-8692) > Revisit how OutboundTcpConnection pools two connections for different message > types > --- > > Key: CASSANDRA-8789 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8789 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > > I was looking at this trying to understand what messages flow over which > connection. > For reads the request goes out over the command connection and the response > comes back over the ack connection. > For writes the request goes out over the command connection and the response > comes back over the command connection. > Reads get a dedicated socket for responses. Mutation commands and responses > both travel over the same socket along with read requests. > Sockets are used uni-directional so there are actually four sockets in play > and four threads at each node (2 inbounded, 2 outbound). > CASSANDRA-488 doesn't leave a record what the impact of this change was. If > someone remembers what situations were made better it would be good to know. > I am not clear on when/how this is helpful. The consumer side shouldn't be > blocking so the only head of line blocking issue is the time it takes to > transfer data over the wire. > If message size is the cause of blocking issues then the current design mixes > small messages and large messages on the same connection retaining the head > of line blocking. > Read requests share the same connection as write requests (which are large), > and write acknowledgments (which are small) share the same connections as > write requests. The only winner is read acknowledgements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8779) Able to unintentionally nest tuples during insert
[
https://issues.apache.org/jira/browse/CASSANDRA-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316704#comment-14316704
]
Kishan Karunaratne commented on CASSANDRA-8779:
---
Simple statements won't work in the python-driver as it uses string
interpolation to create the raw CQL, as you have done. You need to serialize
the arguments using the native protocol (which is true for simple statements in
Java/Ruby). Try a prepared statement instead? I get the following output:
{noformat}
cassandra.cluster: DEBUG: Error querying host 127.0.0.1
Traceback (most recent call last):
File "/home/kishan/git/cstar/python-driver/cassandra/cluster.py", line 2630,
in _query
connection.send_msg(message, request_id, cb=cb)
File "/home/kishan/git/cstar/python-driver/cassandra/connection.py", line
300, in send_msg
self.push(msg.to_binary(request_id, self.protocol_version,
compression=self.compressor))
File "/home/kishan/git/cstar/python-driver/cassandra/protocol.py", line 75,
in to_binary
self.send_body(body, protocol_version)
File "/home/kishan/git/cstar/python-driver/cassandra/protocol.py", line 715,
in send_body
write_value(f, param)
File "/home/kishan/git/cstar/python-driver/cassandra/protocol.py", line 970,
in write_value
f.write(v)
TypeError: 'tuple' does not have the buffer interface
{noformat}
> Able to unintentionally nest tuples during insert
> -
>
> Key: CASSANDRA-8779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8779
> Project: Cassandra
> Issue Type: Bug
> Environment: Linux Mint 64-bit | ruby-driver 2.1 | java-driver 2.1 |
> C* 2.1.2
>Reporter: Kishan Karunaratne
>Assignee: Tyler Hobbs
>
> If I insert a tuple using an extra pair of ()'s, C* will let me do the
> insert, but (incorrectly) creates a nested tuple as the first tuple value.
> Upon doing a select statement, the result is jumbled and has weird binary in
> it (which I wasn't able to copy into here).
> Example using ruby-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)")
> complete = Cassandra::Tuple.new('foo', 123, true)
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", arguments:
> [complete])# extra ()'s here
> result = session.execute("SELECT b FROM mytable WHERE a=0").first
> p result['b']
> {noformat}
> Output:
> {noformat}
> #
> {noformat}
> Bug also confirmed using java-driver.
> Example using java-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)");
> TupleType t = TupleType.of(DataType.ascii(), DataType.cint(),
> DataType.cboolean());
> TupleValue complete = t.newValue("foo", 123, true);
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", complete); //
> extra ()'s here
> TupleValue r = session.execute("SELECT b FROM mytable WHERE
> a=0").one().getTupleValue("b");
> System.out.println(r);
> {noformat}
> Output:
> {noformat}
> ('foo{', null, null)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316667#comment-14316667 ] Alex Liu commented on CASSANDRA-6091: - We need at least check the total estimated rows of multiple token ranges per split, instead of multiple taken ranges per node. > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8789) Revisit how OutboundTcpConnection pools two connections for different message types
Ariel Weisberg created CASSANDRA-8789: - Summary: Revisit how OutboundTcpConnection pools two connections for different message types Key: CASSANDRA-8789 URL: https://issues.apache.org/jira/browse/CASSANDRA-8789 Project: Cassandra Issue Type: Improvement Components: Core Reporter: Ariel Weisberg Assignee: Ariel Weisberg I was looking at this trying to understand what messages flow over which connection. For reads the request goes out over the command connection and the response comes back over the ack connection. For writes the request goes out over the command connection and the response comes back over the command connection. Reads get a dedicated socket for responses. Mutation commands and responses both travel over the same socket along with read requests. Sockets are used uni-directional so there are actually four sockets in play and four threads at each node (2 inbounded, 2 outbound). CASSANDRA-488 doesn't leave a record what the impact of this change was. If someone remembers what situations were made better it would be good to know. I am not clear on when/how this is helpful. The consumer side shouldn't be blocking so the only head of line blocking issue is the time it takes to transfer data over the wire. If message size is the cause of blocking issues then the current design mixes small messages and large messages on the same connection retaining the head of line blocking. Read requests share the same connection as write requests (which are large), and write acknowledgments (which are small) share the same connections as write requests. The only winner is read acknowledgements. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8788) longer duration testing of heterogeneous clusters
Russ Hatch created CASSANDRA-8788: - Summary: longer duration testing of heterogeneous clusters Key: CASSANDRA-8788 URL: https://issues.apache.org/jira/browse/CASSANDRA-8788 Project: Cassandra Issue Type: Test Reporter: Russ Hatch We should consider more testing of cassandra in mixed version clusters (for longer timespans). The current upgrade tests from cassandra-dtest do a brief upgrade where the cluster is in a mixed version state, but to my knowledge there's no other testing of this kind happening for extended time periods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316635#comment-14316635 ] Piotr Kołaczkowski commented on CASSANDRA-6091: --- The way we dealt with this problem in Spark connector was to allow multiple token ranges per split. I don't think there is any other way, as the number of adjacent token ranges is going to drop very quickly with the size of the cluster. > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8779) Able to unintentionally nest tuples during insert
[
https://issues.apache.org/jira/browse/CASSANDRA-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316625#comment-14316625
]
Tyler Hobbs commented on CASSANDRA-8779:
I'm not having any luck reproducing (with the python driver) against 2.1.2 or
cassandra-2.1 head. This should be equivalent to what the Ruby/Java drivers
are producing, correct?
{noformat}
session.execute("INSERT INTO ks1.test (a, b) VALUES (0, (('foo', 123, true)))")
{noformat}
I get the correct error when executing that:
{noformat}
cassandra.InvalidRequest: code=2200 [Invalid query] message="Invalid tuple
literal for b: component 0 is not of type ascii"
{noformat}
> Able to unintentionally nest tuples during insert
> -
>
> Key: CASSANDRA-8779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8779
> Project: Cassandra
> Issue Type: Bug
> Environment: Linux Mint 64-bit | ruby-driver 2.1 | java-driver 2.1 |
> C* 2.1.2
>Reporter: Kishan Karunaratne
>Assignee: Tyler Hobbs
>
> If I insert a tuple using an extra pair of ()'s, C* will let me do the
> insert, but (incorrectly) creates a nested tuple as the first tuple value.
> Upon doing a select statement, the result is jumbled and has weird binary in
> it (which I wasn't able to copy into here).
> Example using ruby-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)")
> complete = Cassandra::Tuple.new('foo', 123, true)
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", arguments:
> [complete])# extra ()'s here
> result = session.execute("SELECT b FROM mytable WHERE a=0").first
> p result['b']
> {noformat}
> Output:
> {noformat}
> #
> {noformat}
> Bug also confirmed using java-driver.
> Example using java-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)");
> TupleType t = TupleType.of(DataType.ascii(), DataType.cint(),
> DataType.cboolean());
> TupleValue complete = t.newValue("foo", 123, true);
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", complete); //
> extra ()'s here
> TupleValue r = session.execute("SELECT b FROM mytable WHERE
> a=0").one().getTupleValue("b");
> System.out.println(r);
> {noformat}
> Output:
> {noformat}
> ('foo{', null, null)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Issue Comment Deleted] (CASSANDRA-8787) Decrease Default concurrent_compactors from number of CPU
[ https://issues.apache.org/jira/browse/CASSANDRA-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Estevez updated CASSANDRA-8787: - Comment: was deleted (was: Any thoughts on the 2.0 branch?) > Decrease Default concurrent_compactors from number of CPU > - > > Key: CASSANDRA-8787 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8787 > Project: Cassandra > Issue Type: Improvement >Reporter: Sebastian Estevez >Priority: Minor > Fix For: 3.0 > > > Currently we default concurrent_compactors to the number of avaliable CPUs > which means compactions can theoretically take over all threads and prevent > reads and writes from happening. > I'm currently seeing an extreme case of this that is caused by > [6405|https://issues.apache.org/jira/browse/CASSANDRA-6405]. But, in general, > this is a bad default. We should probably default concurrent_compactors to > num_cpu to cores/2 or even cores/4. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8779) Able to unintentionally nest tuples during insert
[
https://issues.apache.org/jira/browse/CASSANDRA-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tyler Hobbs reassigned CASSANDRA-8779:
--
Assignee: Tyler Hobbs
> Able to unintentionally nest tuples during insert
> -
>
> Key: CASSANDRA-8779
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8779
> Project: Cassandra
> Issue Type: Bug
> Environment: Linux Mint 64-bit | ruby-driver 2.1 | java-driver 2.1 |
> C* 2.1.2
>Reporter: Kishan Karunaratne
>Assignee: Tyler Hobbs
>
> If I insert a tuple using an extra pair of ()'s, C* will let me do the
> insert, but (incorrectly) creates a nested tuple as the first tuple value.
> Upon doing a select statement, the result is jumbled and has weird binary in
> it (which I wasn't able to copy into here).
> Example using ruby-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)")
> complete = Cassandra::Tuple.new('foo', 123, true)
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", arguments:
> [complete])# extra ()'s here
> result = session.execute("SELECT b FROM mytable WHERE a=0").first
> p result['b']
> {noformat}
> Output:
> {noformat}
> #
> {noformat}
> Bug also confirmed using java-driver.
> Example using java-driver:
> {noformat}
> session.execute("CREATE TABLE mytable (a int PRIMARY KEY, b
> frozen>)");
> TupleType t = TupleType.of(DataType.ascii(), DataType.cint(),
> DataType.cboolean());
> TupleValue complete = t.newValue("foo", 123, true);
> session.execute("INSERT INTO mytable (a, b) VALUES (0, (?))", complete); //
> extra ()'s here
> TupleValue r = session.execute("SELECT b FROM mytable WHERE
> a=0").one().getTupleValue("b");
> System.out.println(r);
> {noformat}
> Output:
> {noformat}
> ('foo{', null, null)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8787) Decrease Default concurrent_compactors from number of CPU
[ https://issues.apache.org/jira/browse/CASSANDRA-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316586#comment-14316586 ] Sebastian Estevez commented on CASSANDRA-8787: -- Any thoughts on the 2.0 branch? > Decrease Default concurrent_compactors from number of CPU > - > > Key: CASSANDRA-8787 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8787 > Project: Cassandra > Issue Type: Improvement >Reporter: Sebastian Estevez >Priority: Minor > Fix For: 3.0 > > > Currently we default concurrent_compactors to the number of avaliable CPUs > which means compactions can theoretically take over all threads and prevent > reads and writes from happening. > I'm currently seeing an extreme case of this that is caused by > [6405|https://issues.apache.org/jira/browse/CASSANDRA-6405]. But, in general, > this is a bad default. We should probably default concurrent_compactors to > num_cpu to cores/2 or even cores/4. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-6091) Better Vnode support in hadoop/pig
[ https://issues.apache.org/jira/browse/CASSANDRA-6091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316544#comment-14316544 ] Jeremy Hanna commented on CASSANDRA-6091: - [~michaelsembwever] can you give a high level description of the approach you're taking? > Better Vnode support in hadoop/pig > -- > > Key: CASSANDRA-6091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6091 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > > CASSANDRA-6084 shows there are some issues during running hadoop/pig job if > vnodes are enable. Also the hadoop performance of vnode enabled nodes are > bad for there are so many splits. > The idea is to combine vnode splits into a big sudo splits so it work like > vnode is disable for hadoop/pig job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8787) Decrease Default concurrent_compactors from number of CPU
[ https://issues.apache.org/jira/browse/CASSANDRA-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani resolved CASSANDRA-8787. --- Resolution: Duplicate Ok good > Decrease Default concurrent_compactors from number of CPU > - > > Key: CASSANDRA-8787 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8787 > Project: Cassandra > Issue Type: Improvement >Reporter: Sebastian Estevez >Priority: Minor > Fix For: 3.0 > > > Currently we default concurrent_compactors to the number of avaliable CPUs > which means compactions can theoretically take over all threads and prevent > reads and writes from happening. > I'm currently seeing an extreme case of this that is caused by > [6405|https://issues.apache.org/jira/browse/CASSANDRA-6405]. But, in general, > this is a bad default. We should probably default concurrent_compactors to > num_cpu to cores/2 or even cores/4. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8787) Decrease Default concurrent_compactors from number of CPU
[ https://issues.apache.org/jira/browse/CASSANDRA-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316543#comment-14316543 ] Yuki Morishita commented on CASSANDRA-8787: --- For 2.1+, it is capped by the number of data directories (CASSANDRA-7139). > Decrease Default concurrent_compactors from number of CPU > - > > Key: CASSANDRA-8787 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8787 > Project: Cassandra > Issue Type: Improvement >Reporter: Sebastian Estevez >Priority: Minor > Fix For: 3.0 > > > Currently we default concurrent_compactors to the number of avaliable CPUs > which means compactions can theoretically take over all threads and prevent > reads and writes from happening. > I'm currently seeing an extreme case of this that is caused by > [6405|https://issues.apache.org/jira/browse/CASSANDRA-6405]. But, in general, > this is a bad default. We should probably default concurrent_compactors to > num_cpu to cores/2 or even cores/4. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8787) Decrease Default concurrent_compactors from number of CPU
[ https://issues.apache.org/jira/browse/CASSANDRA-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316529#comment-14316529 ] T Jake Luciani commented on CASSANDRA-8787: --- I agree this is a bad default. For partitions with very small cells you can end up doing a lot more work per cell and throttling doesn't help. I suggest we make the default 1/2 cores > Decrease Default concurrent_compactors from number of CPU > - > > Key: CASSANDRA-8787 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8787 > Project: Cassandra > Issue Type: Improvement >Reporter: Sebastian Estevez >Priority: Minor > Fix For: 3.0 > > > Currently we default concurrent_compactors to the number of avaliable CPUs > which means compactions can theoretically take over all threads and prevent > reads and writes from happening. > I'm currently seeing an extreme case of this that is caused by > [6405|https://issues.apache.org/jira/browse/CASSANDRA-6405]. But, in general, > this is a bad default. We should probably default concurrent_compactors to > num_cpu to cores/2 or even cores/4. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8787) Decrease Default concurrent_compactors from number of CPU
[ https://issues.apache.org/jira/browse/CASSANDRA-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-8787: -- Priority: Minor (was: Major) > Decrease Default concurrent_compactors from number of CPU > - > > Key: CASSANDRA-8787 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8787 > Project: Cassandra > Issue Type: Improvement >Reporter: Sebastian Estevez >Priority: Minor > Fix For: 3.0 > > > Currently we default concurrent_compactors to the number of avaliable CPUs > which means compactions can theoretically take over all threads and prevent > reads and writes from happening. > I'm currently seeing an extreme case of this that is caused by > [6405|https://issues.apache.org/jira/browse/CASSANDRA-6405]. But, in general, > this is a bad default. We should probably default concurrent_compactors to > num_cpu to cores/2 or even cores/4. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8787) Decrease Default concurrent_compactors from number of CPU
[ https://issues.apache.org/jira/browse/CASSANDRA-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-8787: -- Fix Version/s: 3.0 > Decrease Default concurrent_compactors from number of CPU > - > > Key: CASSANDRA-8787 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8787 > Project: Cassandra > Issue Type: Improvement >Reporter: Sebastian Estevez > Fix For: 3.0 > > > Currently we default concurrent_compactors to the number of avaliable CPUs > which means compactions can theoretically take over all threads and prevent > reads and writes from happening. > I'm currently seeing an extreme case of this that is caused by > [6405|https://issues.apache.org/jira/browse/CASSANDRA-6405]. But, in general, > this is a bad default. We should probably default concurrent_compactors to > num_cpu to cores/2 or even cores/4. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/3] cassandra git commit: ninja-fix literal escape string
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 c643bbc8d -> f57ec8c46
refs/heads/trunk 827ad2438 -> 62ee1473b
ninja-fix literal escape string
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f57ec8c4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f57ec8c4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f57ec8c4
Branch: refs/heads/cassandra-2.1
Commit: f57ec8c464cc3941510cfb996e0335f7a5330b49
Parents: c643bbc
Author: Benedict Elliott Smith
Authored: Wed Feb 11 16:39:06 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 16:39:06 2015 +
--
.../src/org/apache/cassandra/stress/settings/StressSettings.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f57ec8c4/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
--
diff --git
a/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
b/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
index de0f6f6..335ca92 100644
--- a/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
+++ b/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
@@ -239,8 +239,8 @@ public class StressSettings implements Serializable
first = false;
}
return sb.toString()
- .replaceAll("\s+([,=()])", "$1")
- .replaceAll("([,=(])\s+", "$1")
+ .replaceAll("\\s+([,=()])", "$1")
+ .replaceAll("([,=(])\\s+", "$1")
.split(" +");
}
[2/3] cassandra git commit: ninja-fix literal escape string
ninja-fix literal escape string
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f57ec8c4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f57ec8c4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f57ec8c4
Branch: refs/heads/trunk
Commit: f57ec8c464cc3941510cfb996e0335f7a5330b49
Parents: c643bbc
Author: Benedict Elliott Smith
Authored: Wed Feb 11 16:39:06 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 16:39:06 2015 +
--
.../src/org/apache/cassandra/stress/settings/StressSettings.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f57ec8c4/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
--
diff --git
a/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
b/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
index de0f6f6..335ca92 100644
--- a/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
+++ b/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
@@ -239,8 +239,8 @@ public class StressSettings implements Serializable
first = false;
}
return sb.toString()
- .replaceAll("\s+([,=()])", "$1")
- .replaceAll("([,=(])\s+", "$1")
+ .replaceAll("\\s+([,=()])", "$1")
+ .replaceAll("([,=(])\\s+", "$1")
.split(" +");
}
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/62ee1473 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/62ee1473 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/62ee1473 Branch: refs/heads/trunk Commit: 62ee1473bea0bf82ffb7fa888375643556922505 Parents: 827ad24 f57ec8c Author: Benedict Elliott Smith Authored: Wed Feb 11 16:39:14 2015 + Committer: Benedict Elliott Smith Committed: Wed Feb 11 16:39:14 2015 + -- .../src/org/apache/cassandra/stress/settings/StressSettings.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --
[jira] [Commented] (CASSANDRA-8683) Ensure early reopening has no overlap with replaced files
[ https://issues.apache.org/jira/browse/CASSANDRA-8683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316487#comment-14316487 ] Alan Boudreault commented on CASSANDRA-8683: [~benedict] Great, I confirm the change you pushed fix the issue I was seeing. Thanks! > Ensure early reopening has no overlap with replaced files > - > > Key: CASSANDRA-8683 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8683 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Benedict >Priority: Critical > Fix For: 2.1.4 > > Attachments: 0001-avoid-NPE-in-getPositionsForRanges.patch, > system.log, test.sh > > > When introducing CASSANDRA-6916 we permitted the early opened files to > overlap with the files they were replacing by one DecoratedKey, as this > permitted a few minor simplifications. Unfortunately this breaks assumptions > in LeveledCompactionScanner, that are causing the intermittent unit test > failures: > http://cassci.datastax.com/job/trunk_utest/1330/testReport/junit/org.apache.cassandra.db.compaction/LeveledCompactionStrategyTest/testValidationMultipleSSTablePerLevel/ > This patch by itself does not fix the bug, but fixes the described aspect of > it, by ensuring the replaced and replacing files never overlap. This is > achieved first by always selecting the replaced file start as the next key > present in the file greater than the last key in the new file(s). If there > is no such key, however, there is no data to return for the reader, but to > permit abort and atomic replacement at the end of a macro compaction action, > we must keep the file in the DataTracker for replacement purposes, but not > return it to consumers (esp. as many assume a non-empty range). For this I > have introduced a new OpenReason called SHADOWED, and a > DataTracker.View.shadowed collection of sstables, that tracks those we still > consider to be in the live set, but from which we no longer answer any > queries. > CASSANDRA-8744 (and then CASSANDRA-8750) then ensures that these bounds are > honoured, so that we never break the assumption that files in LCS never > overlap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8787) Decrease Default concurrent_compactors from number of CPU
[ https://issues.apache.org/jira/browse/CASSANDRA-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sebastian Estevez updated CASSANDRA-8787: - Description: Currently we default concurrent_compactors to the number of avaliable CPUs which means compactions can theoretically take over all threads and prevent reads and writes from happening. I'm currently seeing an extreme case of this that is caused by [6405|https://issues.apache.org/jira/browse/CASSANDRA-6405]. But, in general, this is a bad default. We should probably default concurrent_compactors to num_cpu to cores/2 or even cores/4. was: Currently we default concurrent_compactors to the number of avaliable CPUs which means compactions can theoretically take over all threads and prevent reads and writes from happening. I'm currently seeing an extreme case of this that is caused by 6405. But, in general, this is a bad default. We should probably default concurrent_compactors to num_cpu to cores/2 or even cores/4. > Decrease Default concurrent_compactors from number of CPU > - > > Key: CASSANDRA-8787 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8787 > Project: Cassandra > Issue Type: Improvement >Reporter: Sebastian Estevez > > Currently we default concurrent_compactors to the number of avaliable CPUs > which means compactions can theoretically take over all threads and prevent > reads and writes from happening. > I'm currently seeing an extreme case of this that is caused by > [6405|https://issues.apache.org/jira/browse/CASSANDRA-6405]. But, in general, > this is a bad default. We should probably default concurrent_compactors to > num_cpu to cores/2 or even cores/4. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8787) Decrease Default concurrent_compactors from number of CPU
Sebastian Estevez created CASSANDRA-8787: Summary: Decrease Default concurrent_compactors from number of CPU Key: CASSANDRA-8787 URL: https://issues.apache.org/jira/browse/CASSANDRA-8787 Project: Cassandra Issue Type: Improvement Reporter: Sebastian Estevez Currently we default concurrent_compactors to the number of avaliable CPUs which means compactions can theoretically take over all threads and prevent reads and writes from happening. I'm currently seeing an extreme case of this that is caused by 6405. But, in general, this is a bad default. We should probably default concurrent_compactors to num_cpu to cores/2 or even cores/4. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8614) Select optimal CRC32 implementation at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-8614: -- Attachment: CRC32.class > Select optimal CRC32 implementation at runtime > -- > > Key: CASSANDRA-8614 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8614 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Labels: performance > Attachments: 8614.patch, CRC32.class, Sample.java > > > JDK 8 has support for an intrinsic for CRC32 that runs at 12-13 gigabytes/sec > per core in my quick and dirty test. PureJavaCRC32 is < 800 megabytes/sec if > I recall and it has a lookup table that evicts random cache lines every time > it runs. > In order to capture the benefit of that when it is available we can select a > CRC32 implementation at startup in a static block. > If JDK 8 is not what is running we can fall back to the existing > PureJavaCRC32 implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8747) Make SSTableWriter.openEarly behaviour more robust
[ https://issues.apache.org/jira/browse/CASSANDRA-8747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-8747: Reviewer: Marcus Eriksson > Make SSTableWriter.openEarly behaviour more robust > -- > > Key: CASSANDRA-8747 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8747 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Benedict >Assignee: Benedict > Fix For: 2.1.4 > > > Currently openEarly does some fairly ugly looping back over the summary data > we've collected looking for one we think should be fully covered in the Index > and Data files, and that should have a safe boundary between it and the end > of an IndexSummary entry so that when scanning across it we should not > accidentally read an incomplete key. The approach taken is a little difficult > to reason about though, and be confident of, and I now realise is also very > subtly broken. Since we're cleaning up the behaviour around this code, it > seemed worthwhile to improve its clarity and make its behaviour easier to > reason about. The current behaviour can be characterised as: > # Take the current Index file length > # Find the IndexSummary boundary key (first key in an interval) that starts > past this position > # Take the IndexSummary boundary key (first key) for the preceding interval > as our initial boundary > # Construct a reader with this boundary > # Lookup our last key in the reader, and if its end position is past the end > of the data file, take the prior summary boundary. Repeat until we find one > starting before the end. > The bug may well be very hard to exhibit, or even impossible, but is that if > we have a single very large partition followed by 127 very tiny partitions > (or whatever the IndexSummary interval is configured as), our IndexSummary > interval buffer may not guarantee the record we have selected as our end is > fully readable. > The new approach is to track in the IndexSummary the safe and optimal > boundary point (i.e. the last record in each summary interval) and its bounds > in the index and data files. On flushing either file, we notify the summary > builder to the new flush points, and it consults its map of these and selects > the last such boundary that can safely be read in both. This is much easier > to understand, and has no such subtle risk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/3] cassandra git commit: ninja fix whitespace parsing in cassandra-stress
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 bf43534dc -> c643bbc8d
refs/heads/trunk f41bac7bd -> 827ad2438
ninja fix whitespace parsing in cassandra-stress
patch by benedict; reviewed by jake for CASSANDRA-8648
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c643bbc8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c643bbc8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c643bbc8
Branch: refs/heads/cassandra-2.1
Commit: c643bbc8d11626d2609d622c9d0893c3ea43d9de
Parents: bf43534
Author: Benedict Elliott Smith
Authored: Wed Feb 11 16:11:38 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 16:11:38 2015 +
--
.../cassandra/stress/settings/StressSettings.java | 18 ++
1 file changed, 18 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c643bbc8/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
--
diff --git
a/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
b/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
index 6d2f7ea..de0f6f6 100644
--- a/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
+++ b/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
@@ -211,6 +211,7 @@ public class StressSettings implements Serializable
{
try
{
+args = repairParams(args);
final Map clArgs = parseMap(args);
if (clArgs.containsKey("legacy"))
return Legacy.build(Arrays.copyOfRange(args, 1, args.length));
@@ -226,6 +227,23 @@ public class StressSettings implements Serializable
}
}
+private static String[] repairParams(String[] args)
+{
+StringBuilder sb = new StringBuilder();
+boolean first = true;
+for (String arg : args)
+{
+if (!first)
+sb.append(" ");
+sb.append(arg);
+first = false;
+}
+return sb.toString()
+ .replaceAll("\s+([,=()])", "$1")
+ .replaceAll("([,=(])\s+", "$1")
+ .split(" +");
+}
+
public static StressSettings get(Map clArgs)
{
SettingsCommand command = SettingsCommand.get(clArgs);
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/827ad243 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/827ad243 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/827ad243 Branch: refs/heads/trunk Commit: 827ad24389ad9a750e553abff089690f2a0ab45d Parents: f41bac7 c643bbc Author: Benedict Elliott Smith Authored: Wed Feb 11 16:12:14 2015 + Committer: Benedict Elliott Smith Committed: Wed Feb 11 16:12:14 2015 + -- .../cassandra/stress/settings/StressSettings.java | 18 ++ 1 file changed, 18 insertions(+) --
[2/3] cassandra git commit: ninja fix whitespace parsing in cassandra-stress
ninja fix whitespace parsing in cassandra-stress
patch by benedict; reviewed by jake for CASSANDRA-8648
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c643bbc8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c643bbc8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c643bbc8
Branch: refs/heads/trunk
Commit: c643bbc8d11626d2609d622c9d0893c3ea43d9de
Parents: bf43534
Author: Benedict Elliott Smith
Authored: Wed Feb 11 16:11:38 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 16:11:38 2015 +
--
.../cassandra/stress/settings/StressSettings.java | 18 ++
1 file changed, 18 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/c643bbc8/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
--
diff --git
a/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
b/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
index 6d2f7ea..de0f6f6 100644
--- a/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
+++ b/tools/stress/src/org/apache/cassandra/stress/settings/StressSettings.java
@@ -211,6 +211,7 @@ public class StressSettings implements Serializable
{
try
{
+args = repairParams(args);
final Map clArgs = parseMap(args);
if (clArgs.containsKey("legacy"))
return Legacy.build(Arrays.copyOfRange(args, 1, args.length));
@@ -226,6 +227,23 @@ public class StressSettings implements Serializable
}
}
+private static String[] repairParams(String[] args)
+{
+StringBuilder sb = new StringBuilder();
+boolean first = true;
+for (String arg : args)
+{
+if (!first)
+sb.append(" ");
+sb.append(arg);
+first = false;
+}
+return sb.toString()
+ .replaceAll("\s+([,=()])", "$1")
+ .replaceAll("([,=(])\s+", "$1")
+ .split(" +");
+}
+
public static StressSettings get(Map clArgs)
{
SettingsCommand command = SettingsCommand.get(clArgs);
[2/5] cassandra git commit: Throw OOM if allocating memory fails to return a valid pointer
Throw OOM if allocating memory fails to return a valid pointer
patch by benedict; reviewed by jbellis for CASSANDRA-8726
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a72869c7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a72869c7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a72869c7
Branch: refs/heads/trunk
Commit: a72869c7f01d7be7b3535a6f4dd673b085a4d71e
Parents: 6ae8ada
Author: Benedict Elliott Smith
Authored: Wed Feb 11 16:06:11 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 16:06:11 2015 +
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/io/util/Memory.java | 2 ++
2 files changed, 3 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a72869c7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7fa5f63..d2e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.13:
+ * Throw OOM if allocating memory fails to return a valid pointer
(CASSANDRA-8726)
* Fix SSTableSimpleUnsortedWriter ConcurrentModificationException
(CASSANDRA-8619)
* Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
* Add batch remove iterator to ABSC (CASSANDRA-8414, 8666)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a72869c7/src/java/org/apache/cassandra/io/util/Memory.java
--
diff --git a/src/java/org/apache/cassandra/io/util/Memory.java
b/src/java/org/apache/cassandra/io/util/Memory.java
index 263205b..fb70b99 100644
--- a/src/java/org/apache/cassandra/io/util/Memory.java
+++ b/src/java/org/apache/cassandra/io/util/Memory.java
@@ -49,6 +49,8 @@ public class Memory
{
size = bytes;
peer = allocator.allocate(size);
+if (size != 0 && peer == 0)
+throw new OutOfMemoryError();
}
public static Memory allocate(long bytes)
[1/5] cassandra git commit: Throw OOM if allocating memory fails to return a valid pointer
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 4b05b204a -> bf43534dc
refs/heads/trunk 2d3743795 -> f41bac7bd
Throw OOM if allocating memory fails to return a valid pointer
patch by benedict; reviewed by jbellis for CASSANDRA-8726
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a72869c7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a72869c7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a72869c7
Branch: refs/heads/cassandra-2.1
Commit: a72869c7f01d7be7b3535a6f4dd673b085a4d71e
Parents: 6ae8ada
Author: Benedict Elliott Smith
Authored: Wed Feb 11 16:06:11 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 16:06:11 2015 +
--
CHANGES.txt | 1 +
src/java/org/apache/cassandra/io/util/Memory.java | 2 ++
2 files changed, 3 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a72869c7/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7fa5f63..d2e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.0.13:
+ * Throw OOM if allocating memory fails to return a valid pointer
(CASSANDRA-8726)
* Fix SSTableSimpleUnsortedWriter ConcurrentModificationException
(CASSANDRA-8619)
* Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645)
* Add batch remove iterator to ABSC (CASSANDRA-8414, 8666)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/a72869c7/src/java/org/apache/cassandra/io/util/Memory.java
--
diff --git a/src/java/org/apache/cassandra/io/util/Memory.java
b/src/java/org/apache/cassandra/io/util/Memory.java
index 263205b..fb70b99 100644
--- a/src/java/org/apache/cassandra/io/util/Memory.java
+++ b/src/java/org/apache/cassandra/io/util/Memory.java
@@ -49,6 +49,8 @@ public class Memory
{
size = bytes;
peer = allocator.allocate(size);
+if (size != 0 && peer == 0)
+throw new OutOfMemoryError();
}
public static Memory allocate(long bytes)
[4/5] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bf43534d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bf43534d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bf43534d Branch: refs/heads/cassandra-2.1 Commit: bf43534dcc77f923debf2af5086f5ca34ef1b34b Parents: 4b05b20 a72869c Author: Benedict Elliott Smith Authored: Wed Feb 11 16:07:31 2015 + Committer: Benedict Elliott Smith Committed: Wed Feb 11 16:07:31 2015 + -- CHANGES.txt | 2 ++ src/java/org/apache/cassandra/io/util/Memory.java | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/bf43534d/CHANGES.txt -- diff --cc CHANGES.txt index b2f894e,d2e..2466b19 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,114 -1,11 +1,116 @@@ -2.0.13: +2.1.4 + * Enforce SSTableReader.first/last (CASSANDRA-8744) + * Cleanup SegmentedFile API (CASSANDRA-8749) + * Avoid overlap with early compaction replacement (CASSANDRA-8683) + * Safer Resource Management++ (CASSANDRA-8707) + * Write partition size estimates into a system table (CASSANDRA-7688) + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output + (CASSANDRA-8154) +Merged from 2.0: + * Fix some multi-column relations with indexes on some clustering + columns (CASSANDRA-8275) + * Throw OOM if allocating memory fails to return a valid pointer (CASSANDRA-8726) + * Fix SSTableSimpleUnsortedWriter ConcurrentModificationException (CASSANDRA-8619) - * Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645) - * Add batch remove iterator to ABSC (CASSANDRA-8414, 8666) -2.0.12: +2.1.3 + * Fix HSHA/offheap_objects corruption (CASSANDRA-8719) + * Upgrade libthrift to 0.9.2 (CASSANDRA-8685) + * Don't use the shared ref in sstableloader (CASSANDRA-8704) + * Purge internal prepared statements if related tables or + keyspaces are dropped (CASSANDRA-8693) + * (cqlsh) Handle unicode BOM at start of files (CASSANDRA-8638) + * Stop compactions before exiting offline tools (CASSANDRA-8623) + * Update tools/stress/README.txt to match current behaviour (CASSANDRA-7933) + * Fix schema from Thrift conversion with empty metadata (CASSANDRA-8695) + * Safer Resource Management (CASSANDRA-7705) + * Make sure we compact highly overlapping cold sstables with + STCS (CASSANDRA-8635) + * rpc_interface and listen_interface generate NPE on startup when specified + interface doesn't exist (CASSANDRA-8677) + * Fix ArrayIndexOutOfBoundsException in nodetool cfhistograms (CASSANDRA-8514) + * Switch from yammer metrics for nodetool cf/proxy histograms (CASSANDRA-8662) + * Make sure we don't add tmplink files to the compaction + strategy (CASSANDRA-8580) + * (cqlsh) Handle maps with blob keys (CASSANDRA-8372) + * (cqlsh) Handle DynamicCompositeType schemas correctly (CASSANDRA-8563) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6706) + * Add tooling to detect hot partitions (CASSANDRA-7974) + * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) + * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) + * Don't allow starting multiple inc repairs on the same sstables (CASSANDRA-8316) + * Invalidate prepared BATCH statements when related tables + or keyspaces are dropped (CASSANDRA-8652) + * Fix missing results in secondary index queries on collections + with ALLOW FILTERING (CASSANDRA-8421) + * Expose EstimatedHistogram metrics for range slices (CASSANDRA-8627) + * (cqlsh) Escape clqshrc passwords properly (CASSANDRA-8618) + * Fix NPE when passing wrong argument in ALTER TABLE statement (CASSANDRA-8355) + * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599) + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unr
[5/5] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f41bac7b Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f41bac7b Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f41bac7b Branch: refs/heads/trunk Commit: f41bac7bdc4c67bcd9d82e044ec47dcf59df17f0 Parents: 2d37437 bf43534 Author: Benedict Elliott Smith Authored: Wed Feb 11 16:07:37 2015 + Committer: Benedict Elliott Smith Committed: Wed Feb 11 16:07:37 2015 + -- CHANGES.txt | 2 ++ src/java/org/apache/cassandra/io/util/Memory.java | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f41bac7b/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f41bac7b/src/java/org/apache/cassandra/io/util/Memory.java --
[3/5] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/bf43534d Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/bf43534d Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/bf43534d Branch: refs/heads/trunk Commit: bf43534dcc77f923debf2af5086f5ca34ef1b34b Parents: 4b05b20 a72869c Author: Benedict Elliott Smith Authored: Wed Feb 11 16:07:31 2015 + Committer: Benedict Elliott Smith Committed: Wed Feb 11 16:07:31 2015 + -- CHANGES.txt | 2 ++ src/java/org/apache/cassandra/io/util/Memory.java | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/bf43534d/CHANGES.txt -- diff --cc CHANGES.txt index b2f894e,d2e..2466b19 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,114 -1,11 +1,116 @@@ -2.0.13: +2.1.4 + * Enforce SSTableReader.first/last (CASSANDRA-8744) + * Cleanup SegmentedFile API (CASSANDRA-8749) + * Avoid overlap with early compaction replacement (CASSANDRA-8683) + * Safer Resource Management++ (CASSANDRA-8707) + * Write partition size estimates into a system table (CASSANDRA-7688) + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output + (CASSANDRA-8154) +Merged from 2.0: + * Fix some multi-column relations with indexes on some clustering + columns (CASSANDRA-8275) + * Throw OOM if allocating memory fails to return a valid pointer (CASSANDRA-8726) + * Fix SSTableSimpleUnsortedWriter ConcurrentModificationException (CASSANDRA-8619) - * Round up time deltas lower than 1ms in BulkLoader (CASSANDRA-8645) - * Add batch remove iterator to ABSC (CASSANDRA-8414, 8666) -2.0.12: +2.1.3 + * Fix HSHA/offheap_objects corruption (CASSANDRA-8719) + * Upgrade libthrift to 0.9.2 (CASSANDRA-8685) + * Don't use the shared ref in sstableloader (CASSANDRA-8704) + * Purge internal prepared statements if related tables or + keyspaces are dropped (CASSANDRA-8693) + * (cqlsh) Handle unicode BOM at start of files (CASSANDRA-8638) + * Stop compactions before exiting offline tools (CASSANDRA-8623) + * Update tools/stress/README.txt to match current behaviour (CASSANDRA-7933) + * Fix schema from Thrift conversion with empty metadata (CASSANDRA-8695) + * Safer Resource Management (CASSANDRA-7705) + * Make sure we compact highly overlapping cold sstables with + STCS (CASSANDRA-8635) + * rpc_interface and listen_interface generate NPE on startup when specified + interface doesn't exist (CASSANDRA-8677) + * Fix ArrayIndexOutOfBoundsException in nodetool cfhistograms (CASSANDRA-8514) + * Switch from yammer metrics for nodetool cf/proxy histograms (CASSANDRA-8662) + * Make sure we don't add tmplink files to the compaction + strategy (CASSANDRA-8580) + * (cqlsh) Handle maps with blob keys (CASSANDRA-8372) + * (cqlsh) Handle DynamicCompositeType schemas correctly (CASSANDRA-8563) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6706) + * Add tooling to detect hot partitions (CASSANDRA-7974) + * Fix cassandra-stress user-mode truncation of partition generation (CASSANDRA-8608) + * Only stream from unrepaired sstables during inc repair (CASSANDRA-8267) + * Don't allow starting multiple inc repairs on the same sstables (CASSANDRA-8316) + * Invalidate prepared BATCH statements when related tables + or keyspaces are dropped (CASSANDRA-8652) + * Fix missing results in secondary index queries on collections + with ALLOW FILTERING (CASSANDRA-8421) + * Expose EstimatedHistogram metrics for range slices (CASSANDRA-8627) + * (cqlsh) Escape clqshrc passwords properly (CASSANDRA-8618) + * Fix NPE when passing wrong argument in ALTER TABLE statement (CASSANDRA-8355) + * Pig: Refactor and deprecate CqlStorage (CASSANDRA-8599) + * Don't reuse the same cleanup strategy for all sstables (CASSANDRA-8537) + * Fix case-sensitivity of index name on CREATE and DROP INDEX + statements (CASSANDRA-8365) + * Better detection/logging for corruption in compressed sstables (CASSANDRA-8192) + * Use the correct repairedAt value when closing writer (CASSANDRA-8570) + * (cqlsh) Handle a schema mismatch being detected on startup (CASSANDRA-8512) + * Properly calculate expected write size during compaction (CASSANDRA-8532) + * Invalidate affected prepared statements when a table's columns + are altered (CASSANDRA-7910) + * Stress - user defined writes should populate sequentally (CASSANDRA-8524) + * Fix regression in SSTableRewriter causing some rows to become unreadable + during compaction (CASSANDRA-8429) + * Run major compactions for repaired/unrepaired
[jira] [Commented] (CASSANDRA-8614) Select optimal CRC32 implementation at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316419#comment-14316419 ] Benedict commented on CASSANDRA-8614: - Looks like you may have missed your JDK source file override to permit compilation with JDK8... > Select optimal CRC32 implementation at runtime > -- > > Key: CASSANDRA-8614 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8614 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg > Labels: performance > Attachments: 8614.patch, Sample.java > > > JDK 8 has support for an intrinsic for CRC32 that runs at 12-13 gigabytes/sec > per core in my quick and dirty test. PureJavaCRC32 is < 800 megabytes/sec if > I recall and it has a lookup table that evicts random cache lines every time > it runs. > In order to capture the benefit of that when it is available we can select a > CRC32 implementation at startup in a static block. > If JDK 8 is not what is running we can fall back to the existing > PureJavaCRC32 implementation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[3/5] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d374379/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
--
diff --cc
src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
index 4fcbc31,000..127a60c
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/big/BigTableReader.java
@@@ -1,250 -1,0 +1,260 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.io.sstable.format.big;
+
+import com.google.common.util.concurrent.RateLimiter;
+import org.apache.cassandra.cache.KeyCacheKey;
+import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.db.DataRange;
+import org.apache.cassandra.db.DecoratedKey;
+import org.apache.cassandra.db.RowIndexEntry;
+import org.apache.cassandra.db.RowPosition;
+import org.apache.cassandra.db.columniterator.OnDiskAtomIterator;
+import org.apache.cassandra.db.composites.CellName;
+import org.apache.cassandra.db.filter.ColumnSlice;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.dht.Range;
+import org.apache.cassandra.dht.Token;
+import org.apache.cassandra.io.sstable.Component;
+import org.apache.cassandra.io.sstable.CorruptSSTableException;
+import org.apache.cassandra.io.sstable.Descriptor;
+import org.apache.cassandra.io.sstable.ISSTableScanner;
+import org.apache.cassandra.io.sstable.format.SSTableReader;
+import org.apache.cassandra.io.sstable.metadata.StatsMetadata;
+import org.apache.cassandra.io.util.FileDataInput;
+import org.apache.cassandra.io.util.FileUtils;
+import org.apache.cassandra.tracing.Tracing;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.util.*;
+
+/**
+ * SSTableReaders are open()ed by Keyspace.onStart; after that they are
created by SSTableWriter.renameAndOpen.
+ * Do not re-call open() on existing SSTable files; use the references kept
by ColumnFamilyStore post-start instead.
+ */
+public class BigTableReader extends SSTableReader
+{
+private static final Logger logger =
LoggerFactory.getLogger(BigTableReader.class);
+
+BigTableReader(Descriptor desc, Set components, CFMetaData
metadata, IPartitioner partitioner, Long maxDataAge, StatsMetadata
sstableMetadata, OpenReason openReason)
+{
+super(desc, components, metadata, partitioner, maxDataAge,
sstableMetadata, openReason);
+}
+
+public OnDiskAtomIterator iterator(DecoratedKey key, SortedSet
columns)
+{
+return new SSTableNamesIterator(this, key, columns);
+}
+
+public OnDiskAtomIterator iterator(FileDataInput input, DecoratedKey key,
SortedSet columns, RowIndexEntry indexEntry )
+{
+return new SSTableNamesIterator(this, input, key, columns,
indexEntry);
+}
+
+public OnDiskAtomIterator iterator(DecoratedKey key, ColumnSlice[]
slices, boolean reverse)
+{
+return new SSTableSliceIterator(this, key, slices, reverse);
+}
+
+public OnDiskAtomIterator iterator(FileDataInput input, DecoratedKey key,
ColumnSlice[] slices, boolean reverse, RowIndexEntry indexEntry)
+{
+return new SSTableSliceIterator(this, input, key, slices, reverse,
indexEntry);
+}
+/**
+ *
+ * @param dataRange filter to use when reading the columns
+ * @return A Scanner for seeking over the rows of the SSTable.
+ */
+public ISSTableScanner getScanner(DataRange dataRange, RateLimiter
limiter)
+{
+return BigTableScanner.getScanner(this, dataRange, limiter);
+}
+
+
+/**
+ * Direct I/O SSTableScanner over a defined collection of ranges of
tokens.
+ *
+ * @param ranges the range of keys to cover
+ * @return A Scanner for seeking over the rows of the SSTable.
+ */
+public ISSTableScanner getScanner(Collection> ranges,
RateLimiter limiter)
+{
+return BigTableScanner.getSca
[1/5] cassandra git commit: Enforce SSTableReader.first/last
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 9d4909ee3 -> 4b05b204a
refs/heads/trunk 6d12ab8fd -> 2d3743795
Enforce SSTableReader.first/last
patch by benedict; reviewed by yukim for CASSANDRA-8744
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b05b204
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b05b204
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b05b204
Branch: refs/heads/cassandra-2.1
Commit: 4b05b204acfa60ecad5672c7e6068eb47b21397a
Parents: 9d4909e
Author: Benedict Elliott Smith
Authored: Wed Feb 11 15:49:43 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 15:49:43 2015 +
--
CHANGES.txt | 1 +
.../cassandra/io/sstable/SSTableReader.java | 49 ++-
.../cassandra/io/sstable/SSTableScanner.java| 63
3 files changed, 72 insertions(+), 41 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b05b204/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3f04637..b2f894e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.4
+ * Enforce SSTableReader.first/last (CASSANDRA-8744)
* Cleanup SegmentedFile API (CASSANDRA-8749)
* Avoid overlap with early compaction replacement (CASSANDRA-8683)
* Safer Resource Management++ (CASSANDRA-8707)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b05b204/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index f59e632..5abd1b7 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -1062,17 +1062,20 @@ public class SSTableReader extends SSTable implements
RefCounted
/**
* Gets the position in the index file to start scanning to find the given
key (at most indexInterval keys away,
- * modulo downsampling of the index summary).
+ * modulo downsampling of the index summary). Always returns a value >= 0
*/
public long getIndexScanPosition(RowPosition key)
{
+if (openReason == OpenReason.MOVED_START && key.compareTo(first) < 0)
+key = first;
+
return
getIndexScanPositionFromBinarySearchResult(indexSummary.binarySearch(key),
indexSummary);
}
private static long getIndexScanPositionFromBinarySearchResult(int
binarySearchResult, IndexSummary referencedIndexSummary)
{
if (binarySearchResult == -1)
-return -1;
+return 0;
else
return
referencedIndexSummary.getPosition(getIndexSummaryIndexFromBinarySearchResult(binarySearchResult));
}
@@ -1314,7 +1317,7 @@ public class SSTableReader extends SSTable implements
RefCounted
? (openReason == OpenReason.EARLY
// if opened early, we overlap with the old
sstables by one key, so we know that the last
// (and further) key(s) will be streamed from
these if necessary
-? getPosition(last.getToken().maxKeyBound(),
Operator.GT).position
+? getPosition(last, Operator.GT, false,
true).position
: uncompressedLength())
: getPosition(rightBound, Operator.GT).position;
@@ -1395,6 +1398,10 @@ public class SSTableReader extends SSTable implements
RefCounted
*/
public RowIndexEntry getPosition(RowPosition key, Operator op, boolean
updateCacheAndStats)
{
+return getPosition(key, op, updateCacheAndStats, false);
+}
+private RowIndexEntry getPosition(RowPosition key, Operator op, boolean
updateCacheAndStats, boolean permitMatchPastLast)
+{
// first, check bloom filter
if (op == Operator.EQ)
{
@@ -1420,25 +1427,36 @@ public class SSTableReader extends SSTable implements
RefCounted
}
// check the smallest and greatest keys in the sstable to see if it
can't be present
-if (first.compareTo(key) > 0 || last.compareTo(key) < 0)
+boolean skip = false;
+if (key.compareTo(first) < 0)
+{
+if (op == Operator.EQ)
+skip = true;
+else
+key = first;
+
+op = Operator.EQ;
+}
+else
+{
+int l = last.compareTo(key);
+// l <= 0 => we may be looking past the end of the file; we then
narrow our behaviour to:
+
[4/5] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d374379/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java -- diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java index 0ead6c0,000..35bb109 mode 100644,00..100644 --- a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java +++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java @@@ -1,2054 -1,0 +1,2061 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.io.sstable.format; + +import java.io.*; +import java.nio.ByteBuffer; +import java.util.*; +import java.util.concurrent.*; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicLong; + +import com.google.common.annotations.VisibleForTesting; +import com.google.common.base.Predicate; +import com.google.common.collect.Iterators; +import com.google.common.collect.Ordering; +import com.google.common.primitives.Longs; +import com.google.common.util.concurrent.RateLimiter; + +import com.clearspring.analytics.stream.cardinality.CardinalityMergeException; +import com.clearspring.analytics.stream.cardinality.HyperLogLogPlus; +import com.clearspring.analytics.stream.cardinality.ICardinality; +import org.apache.cassandra.cache.CachingOptions; +import org.apache.cassandra.cache.InstrumentingCache; +import org.apache.cassandra.cache.KeyCacheKey; +import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor; +import org.apache.cassandra.concurrent.ScheduledExecutors; +import org.apache.cassandra.config.*; +import org.apache.cassandra.db.*; +import org.apache.cassandra.db.columniterator.OnDiskAtomIterator; +import org.apache.cassandra.db.commitlog.ReplayPosition; +import org.apache.cassandra.db.composites.CellName; +import org.apache.cassandra.db.filter.ColumnSlice; +import org.apache.cassandra.db.index.SecondaryIndex; +import org.apache.cassandra.dht.*; - import org.apache.cassandra.io.compress.CompressedThrottledReader; +import org.apache.cassandra.io.compress.CompressionMetadata; +import org.apache.cassandra.io.sstable.*; +import org.apache.cassandra.io.sstable.metadata.*; +import org.apache.cassandra.io.util.*; +import org.apache.cassandra.metrics.RestorableMeter; +import org.apache.cassandra.metrics.StorageMetrics; +import org.apache.cassandra.service.ActiveRepairService; +import org.apache.cassandra.service.CacheService; +import org.apache.cassandra.service.StorageService; +import org.apache.cassandra.utils.*; +import org.apache.cassandra.utils.concurrent.OpOrder; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.apache.cassandra.utils.concurrent.Ref; +import org.apache.cassandra.utils.concurrent.RefCounted; + +import static org.apache.cassandra.db.Directories.SECONDARY_INDEX_NAME_SEPARATOR; + +/** + * An SSTableReader can be constructed in a number of places, but typically is either + * read from disk at startup, or constructed from a flushed memtable, or after compaction + * to replace some existing sstables. However once created, an sstablereader may also be modified. + * + * A reader's OpenReason describes its current stage in its lifecycle, as follows: + * + * NORMAL + * From: None=> Reader has been read from disk, either at startup or from a flushed memtable + * EARLY => Reader is the final result of a compaction + * MOVED_START => Reader WAS being compacted, but this failed and it has been restored to NORMAL status + * + * EARLY + * From: None=> Reader is a compaction replacement that is either incomplete and has been opened + *to represent its partial result status, or has been finished but the compaction + *it is a part of has not yet completed fully + * EARLY => Same as from None, only it is not the first time it has been + * + * MOVED_START + * From: NORMAL => Reader is being compacted. This compaction has not finished, but the compaction result + *
[2/5] cassandra git commit: Enforce SSTableReader.first/last
Enforce SSTableReader.first/last
patch by benedict; reviewed by yukim for CASSANDRA-8744
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4b05b204
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4b05b204
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4b05b204
Branch: refs/heads/trunk
Commit: 4b05b204acfa60ecad5672c7e6068eb47b21397a
Parents: 9d4909e
Author: Benedict Elliott Smith
Authored: Wed Feb 11 15:49:43 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 15:49:43 2015 +
--
CHANGES.txt | 1 +
.../cassandra/io/sstable/SSTableReader.java | 49 ++-
.../cassandra/io/sstable/SSTableScanner.java| 63
3 files changed, 72 insertions(+), 41 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b05b204/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 3f04637..b2f894e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.4
+ * Enforce SSTableReader.first/last (CASSANDRA-8744)
* Cleanup SegmentedFile API (CASSANDRA-8749)
* Avoid overlap with early compaction replacement (CASSANDRA-8683)
* Safer Resource Management++ (CASSANDRA-8707)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4b05b204/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index f59e632..5abd1b7 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -1062,17 +1062,20 @@ public class SSTableReader extends SSTable implements
RefCounted
/**
* Gets the position in the index file to start scanning to find the given
key (at most indexInterval keys away,
- * modulo downsampling of the index summary).
+ * modulo downsampling of the index summary). Always returns a value >= 0
*/
public long getIndexScanPosition(RowPosition key)
{
+if (openReason == OpenReason.MOVED_START && key.compareTo(first) < 0)
+key = first;
+
return
getIndexScanPositionFromBinarySearchResult(indexSummary.binarySearch(key),
indexSummary);
}
private static long getIndexScanPositionFromBinarySearchResult(int
binarySearchResult, IndexSummary referencedIndexSummary)
{
if (binarySearchResult == -1)
-return -1;
+return 0;
else
return
referencedIndexSummary.getPosition(getIndexSummaryIndexFromBinarySearchResult(binarySearchResult));
}
@@ -1314,7 +1317,7 @@ public class SSTableReader extends SSTable implements
RefCounted
? (openReason == OpenReason.EARLY
// if opened early, we overlap with the old
sstables by one key, so we know that the last
// (and further) key(s) will be streamed from
these if necessary
-? getPosition(last.getToken().maxKeyBound(),
Operator.GT).position
+? getPosition(last, Operator.GT, false,
true).position
: uncompressedLength())
: getPosition(rightBound, Operator.GT).position;
@@ -1395,6 +1398,10 @@ public class SSTableReader extends SSTable implements
RefCounted
*/
public RowIndexEntry getPosition(RowPosition key, Operator op, boolean
updateCacheAndStats)
{
+return getPosition(key, op, updateCacheAndStats, false);
+}
+private RowIndexEntry getPosition(RowPosition key, Operator op, boolean
updateCacheAndStats, boolean permitMatchPastLast)
+{
// first, check bloom filter
if (op == Operator.EQ)
{
@@ -1420,25 +1427,36 @@ public class SSTableReader extends SSTable implements
RefCounted
}
// check the smallest and greatest keys in the sstable to see if it
can't be present
-if (first.compareTo(key) > 0 || last.compareTo(key) < 0)
+boolean skip = false;
+if (key.compareTo(first) < 0)
+{
+if (op == Operator.EQ)
+skip = true;
+else
+key = first;
+
+op = Operator.EQ;
+}
+else
+{
+int l = last.compareTo(key);
+// l <= 0 => we may be looking past the end of the file; we then
narrow our behaviour to:
+// 1) skipping if strictly greater for GE and EQ;
+// 2) skipping if equal and searching GT, and we
aren'
[5/5] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java src/java/org/apache/cassandra/io/sstable/format/big/BigTableScanner.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2d374379 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2d374379 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2d374379 Branch: refs/heads/trunk Commit: 2d3743795eddcb45913310dae08091326d95a387 Parents: 6d12ab8 4b05b20 Author: Benedict Elliott Smith Authored: Wed Feb 11 15:58:10 2015 + Committer: Benedict Elliott Smith Committed: Wed Feb 11 15:58:10 2015 + -- CHANGES.txt | 1 + .../io/sstable/format/SSTableReader.java| 23 --- .../io/sstable/format/big/BigTableReader.java | 34 +++ .../io/sstable/format/big/BigTableScanner.java | 63 4 files changed, 77 insertions(+), 44 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d374379/CHANGES.txt -- diff --cc CHANGES.txt index 107c923,b2f894e..cc1dcec --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,63 -1,5 +1,64 @@@ +3.0 + * Add role based access control (CASSANDRA-7653, 8650) + * Avoid accessing partitioner through StorageProxy (CASSANDRA-8244, 8268) + * Upgrade Metrics library and remove depricated metrics (CASSANDRA-5657) + * Serializing Row cache alternative, fully off heap (CASSANDRA-7438) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560) + * Support direct buffer decompression for reads (CASSANDRA-8464) + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039) + * Group sstables for anticompaction correctly (CASSANDRA-8578) + * Add ReadFailureException to native protocol, respond + immediately when replicas encounter errors while handling + a read request (CASSANDRA-7886) + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308) + * Allow mixing token and partition key restrictions (CASSANDRA-7016) + * Support index key/value entries on map collections (CASSANDRA-8473) + * Modernize schema tables (CASSANDRA-8261) + * Support for user-defined aggregation functions (CASSANDRA-8053) + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419) + * Refactor SelectStatement, return IN results in natural order instead + of IN value list order and ignore duplicate values in partition key IN restrictions (CASSANDRA-7981) + * Support UDTs, tuples, and collections in user-defined + functions (CASSANDRA-7563) + * Fix aggregate fn results on empty selection, result column name, + and cqlsh parsing (CASSANDRA-8229) + * Mark sstables as repaired after full repair (CASSANDRA-7586) + * Extend Descriptor to include a format value and refactor reader/writer + APIs (CASSANDRA-7443) + * Integrate JMH for microbenchmarks (CASSANDRA-8151) + * Keep sstable levels when bootstrapping (CASSANDRA-7460) + * Add Sigar library and perform basic OS settings check on startup (CASSANDRA-7838) + * Support for aggregation functions (CASSANDRA-4914) + * Remove cassandra-cli (CASSANDRA-7920) + * Accept dollar quoted strings in CQL (CASSANDRA-7769) + * Make assassinate a first class command (CASSANDRA-7935) + * Support IN clause on any partition key column (CASSANDRA-7855) + * Support IN clause on any clustering column (CASSANDRA-4762) + * Improve compaction logging (CASSANDRA-7818) + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917) + * Do anticompaction in groups (CASSANDRA-6851) + * Support user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 7929, + 7924, 7812, 8063, 7813, 7708) + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416) + * Move sstable RandomAccessReader to nio2, which allows using the + FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050) + * Remove CQL2 (CASSANDRA-5918) + * Add Thrift get_multi_slice call (CASSANDRA-6757) + * Optimize fetching multiple cells by name (CASSANDRA-6933) + * Allow compilation in java 8 (CASSANDRA-7028) + * Make incremental repair default (CASSANDRA-7250) + * Enable code coverage thru JaCoCo (CASSANDRA-7226) + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) + * Shorten SSTable path (CASSANDRA-6962) + * Use unsafe mutations for most unit tests (CASSANDRA-6969) + * Fix race condition during calculation of pending ranges (CASSANDRA-7390) + * Fail on very large batch sizes (CASSANDRA-8011) + * Improve concurrency of repair (CASSANDRA-6455, 8208) + + 2.1.4 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
[jira] [Created] (CASSANDRA-8786) NullPointerException in ColumnDefinition.hasIndexOption
Mathijs Vogelzang created CASSANDRA-8786: Summary: NullPointerException in ColumnDefinition.hasIndexOption Key: CASSANDRA-8786 URL: https://issues.apache.org/jira/browse/CASSANDRA-8786 Project: Cassandra Issue Type: Bug Components: Core Environment: Cassandra 2.1.2 Reporter: Mathijs Vogelzang We have a Cassandra cluster that we've been using through many upgrades, and thus most of our column families have originally been created by Thrift. We are on Cassandra 2.1.2 now. We've now ported most of our code to use CQL, and our code occasionally tries to recreate tables with "IF NOT EXISTS" to work properly on development / testing environments. When we issue the CQL statement "CREATE INDEX IF NOT EXISTS index ON "tableName" (accountId)" (this index does exist on that table already), we get a DriverInternalError: An unexpected error occurred server side on cass_host/xx.xxx.xxx.xxx:9042: java.lang.NullPointerException The error on the server is java.lang.NullPointerException: null at org.apache.cassandra.config.ColumnDefinition.hasIndexOption(ColumnDefinition.java:489) ~[apache-cassandra-2.1.2.jar:2.1.2] at org.apache.cassandra.cql3.statements.CreateIndexStatement.validate(CreateIndexStatement.java:87) ~[apache-cassandra-2.1.2.jar:2.1.2] at org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:224) ~[apache-cassandra-2.1.2.jar:2.1.2] at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:248) ~[apache-cassandra-2.1.2.jar:2.1.2] at org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119) ~[apache-cassandra-2.1.2.jar:2.1.2] This happens every time we run this CQL statement. We've tried to reproduce it in a test cassandra cluster by creating the table according to the exact "DESCRIBE TABLE" specification, but then this NullPointerException doesn't happon upon the CREATE INDEX one. So it seems that the tables on our production cluster (that were originally created through thrift) are still subtly different schema-wise then a freshly created table according to the same creation statement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[4/4] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6d12ab8f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6d12ab8f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6d12ab8f Branch: refs/heads/trunk Commit: 6d12ab8fd78834c48fda8fa75473fe08f96279ed Parents: d73e7db 9d4909e Author: Benedict Elliott Smith Authored: Wed Feb 11 15:46:56 2015 + Committer: Benedict Elliott Smith Committed: Wed Feb 11 15:46:56 2015 + -- CHANGES.txt | 1 + .../compress/CompressedRandomAccessReader.java | 15 - .../io/sstable/format/SSTableReader.java| 11 +++--- .../io/util/BufferedPoolingSegmentedFile.java | 5 - .../io/util/CompressedPoolingSegmentedFile.java | 16 +- .../io/util/CompressedSegmentedFile.java| 14 + .../cassandra/io/util/MappedFileDataInput.java | 9 .../cassandra/io/util/MmappedSegmentedFile.java | 5 +++-- .../cassandra/io/util/PoolingSegmentedFile.java | 9 ++-- .../apache/cassandra/io/util/SegmentedFile.java | 22 ++-- 10 files changed, 63 insertions(+), 44 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/6d12ab8f/CHANGES.txt -- diff --cc CHANGES.txt index 1a4c87c,3f04637..107c923 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,63 -1,5 +1,64 @@@ +3.0 + * Add role based access control (CASSANDRA-7653, 8650) + * Avoid accessing partitioner through StorageProxy (CASSANDRA-8244, 8268) + * Upgrade Metrics library and remove depricated metrics (CASSANDRA-5657) + * Serializing Row cache alternative, fully off heap (CASSANDRA-7438) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560) + * Support direct buffer decompression for reads (CASSANDRA-8464) + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039) + * Group sstables for anticompaction correctly (CASSANDRA-8578) + * Add ReadFailureException to native protocol, respond + immediately when replicas encounter errors while handling + a read request (CASSANDRA-7886) + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308) + * Allow mixing token and partition key restrictions (CASSANDRA-7016) + * Support index key/value entries on map collections (CASSANDRA-8473) + * Modernize schema tables (CASSANDRA-8261) + * Support for user-defined aggregation functions (CASSANDRA-8053) + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419) + * Refactor SelectStatement, return IN results in natural order instead + of IN value list order and ignore duplicate values in partition key IN restrictions (CASSANDRA-7981) + * Support UDTs, tuples, and collections in user-defined + functions (CASSANDRA-7563) + * Fix aggregate fn results on empty selection, result column name, + and cqlsh parsing (CASSANDRA-8229) + * Mark sstables as repaired after full repair (CASSANDRA-7586) + * Extend Descriptor to include a format value and refactor reader/writer + APIs (CASSANDRA-7443) + * Integrate JMH for microbenchmarks (CASSANDRA-8151) + * Keep sstable levels when bootstrapping (CASSANDRA-7460) + * Add Sigar library and perform basic OS settings check on startup (CASSANDRA-7838) + * Support for aggregation functions (CASSANDRA-4914) + * Remove cassandra-cli (CASSANDRA-7920) + * Accept dollar quoted strings in CQL (CASSANDRA-7769) + * Make assassinate a first class command (CASSANDRA-7935) + * Support IN clause on any partition key column (CASSANDRA-7855) + * Support IN clause on any clustering column (CASSANDRA-4762) + * Improve compaction logging (CASSANDRA-7818) + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917) + * Do anticompaction in groups (CASSANDRA-6851) + * Support user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 7929, + 7924, 7812, 8063, 7813, 7708) + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416) + * Move sstable RandomAccessReader to nio2, which allows using the + FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050) + * Remove CQL2 (CASSANDRA-5918) + * Add Thrift get_multi_slice call (CASSANDRA-6757) + * Optimize fetching multiple cells by name (CASSANDRA-6933) + * Allow compilation in java 8 (CASSANDRA-7028) + * Make incremental repair default (CASSANDRA-7250) + * Enable code coverage thru JaCoCo (CASSANDRA-7226) + * Switch external naming of 'column families' to
[3/4] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
http://git-wip-us.apache.org/repos/asf/cassandra/blob/6d12ab8f/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java -- diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java index be783a1,000..0ead6c0 mode 100644,00..100644 --- a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java +++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java @@@ -1,2059 -1,0 +1,2054 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.io.sstable.format; + +import java.io.*; +import java.nio.ByteBuffer; +import java.util.*; +import java.util.concurrent.*; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicLong; + +import com.google.common.annotations.VisibleForTesting; +import com.google.common.base.Predicate; +import com.google.common.collect.Iterators; +import com.google.common.collect.Ordering; +import com.google.common.primitives.Longs; +import com.google.common.util.concurrent.RateLimiter; + +import com.clearspring.analytics.stream.cardinality.CardinalityMergeException; +import com.clearspring.analytics.stream.cardinality.HyperLogLogPlus; +import com.clearspring.analytics.stream.cardinality.ICardinality; +import org.apache.cassandra.cache.CachingOptions; +import org.apache.cassandra.cache.InstrumentingCache; +import org.apache.cassandra.cache.KeyCacheKey; +import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor; +import org.apache.cassandra.concurrent.ScheduledExecutors; +import org.apache.cassandra.config.*; +import org.apache.cassandra.db.*; +import org.apache.cassandra.db.columniterator.OnDiskAtomIterator; +import org.apache.cassandra.db.commitlog.ReplayPosition; +import org.apache.cassandra.db.composites.CellName; +import org.apache.cassandra.db.filter.ColumnSlice; +import org.apache.cassandra.db.index.SecondaryIndex; +import org.apache.cassandra.dht.*; - import org.apache.cassandra.io.compress.CompressedRandomAccessReader; +import org.apache.cassandra.io.compress.CompressedThrottledReader; +import org.apache.cassandra.io.compress.CompressionMetadata; +import org.apache.cassandra.io.sstable.*; +import org.apache.cassandra.io.sstable.metadata.*; +import org.apache.cassandra.io.util.*; +import org.apache.cassandra.metrics.RestorableMeter; +import org.apache.cassandra.metrics.StorageMetrics; +import org.apache.cassandra.service.ActiveRepairService; +import org.apache.cassandra.service.CacheService; +import org.apache.cassandra.service.StorageService; +import org.apache.cassandra.utils.*; +import org.apache.cassandra.utils.concurrent.OpOrder; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.apache.cassandra.utils.concurrent.Ref; +import org.apache.cassandra.utils.concurrent.RefCounted; + +import static org.apache.cassandra.db.Directories.SECONDARY_INDEX_NAME_SEPARATOR; + +/** + * An SSTableReader can be constructed in a number of places, but typically is either + * read from disk at startup, or constructed from a flushed memtable, or after compaction + * to replace some existing sstables. However once created, an sstablereader may also be modified. + * + * A reader's OpenReason describes its current stage in its lifecycle, as follows: + * + * NORMAL + * From: None=> Reader has been read from disk, either at startup or from a flushed memtable + * EARLY => Reader is the final result of a compaction + * MOVED_START => Reader WAS being compacted, but this failed and it has been restored to NORMAL status + * + * EARLY + * From: None=> Reader is a compaction replacement that is either incomplete and has been opened + *to represent its partial result status, or has been finished but the compaction + *it is a part of has not yet completed fully + * EARLY => Same as from None, only it is not the first time it has been + * + * MOVED_START + * From: NORMAL => Reader is being compacted. This comp
[1/4] cassandra git commit: Cleanup SegmentedFile API
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 207751c8a -> 9d4909ee3
refs/heads/trunk d73e7db80 -> 6d12ab8fd
Cleanup SegmentedFile API
patch by benedict; reviewed by marcus for CASSANDRA-8749
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9d4909ee
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9d4909ee
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9d4909ee
Branch: refs/heads/cassandra-2.1
Commit: 9d4909ee362be5082a6677bdbfba9d2b29848bfd
Parents: 207751c
Author: Benedict Elliott Smith
Authored: Wed Feb 11 15:45:18 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 15:45:18 2015 +
--
CHANGES.txt | 1 +
.../compress/CompressedRandomAccessReader.java | 15 -
.../cassandra/io/sstable/SSTableReader.java | 13 +++-
.../io/util/BufferedPoolingSegmentedFile.java | 5 -
.../io/util/CompressedPoolingSegmentedFile.java | 16 +-
.../io/util/CompressedSegmentedFile.java| 14 +
.../cassandra/io/util/MappedFileDataInput.java | 9
.../cassandra/io/util/MmappedSegmentedFile.java | 5 +++--
.../cassandra/io/util/PoolingSegmentedFile.java | 9 ++--
.../apache/cassandra/io/util/SegmentedFile.java | 22 ++--
10 files changed, 63 insertions(+), 46 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9d4909ee/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7352068..3f04637 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.4
+ * Cleanup SegmentedFile API (CASSANDRA-8749)
* Avoid overlap with early compaction replacement (CASSANDRA-8683)
* Safer Resource Management++ (CASSANDRA-8707)
* Write partition size estimates into a system table (CASSANDRA-7688)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9d4909ee/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
index 4521c19..49dcd3d 100644
---
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
+++
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
@@ -37,6 +37,10 @@ import org.apache.cassandra.utils.FBUtilities;
*/
public class CompressedRandomAccessReader extends RandomAccessReader
{
+public static CompressedRandomAccessReader open(String dataFilePath,
CompressionMetadata metadata)
+{
+return open(dataFilePath, metadata, null);
+}
public static CompressedRandomAccessReader open(String path,
CompressionMetadata metadata, CompressedPoolingSegmentedFile owner)
{
try
@@ -49,17 +53,6 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
}
}
-public static CompressedRandomAccessReader open(String dataFilePath,
CompressionMetadata metadata)
-{
-try
-{
-return new CompressedRandomAccessReader(dataFilePath, metadata,
null);
-}
-catch (FileNotFoundException e)
-{
-throw new RuntimeException(e);
-}
-}
private final CompressionMetadata metadata;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9d4909ee/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index a588bff..f59e632 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -77,8 +77,6 @@ import org.apache.cassandra.dht.IPartitioner;
import org.apache.cassandra.dht.LocalPartitioner;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.dht.Token;
-import org.apache.cassandra.io.compress.CompressedRandomAccessReader;
-import org.apache.cassandra.io.compress.CompressedThrottledReader;
import org.apache.cassandra.io.compress.CompressionMetadata;
import org.apache.cassandra.io.sstable.metadata.CompactionMetadata;
import org.apache.cassandra.io.sstable.metadata.MetadataComponent;
@@ -93,7 +91,6 @@ import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.io.util.ICompressedFile;
import org.apache.cassandra.io.util.RandomAccessReader;
import org.apache.cassandra.io.util.SegmentedFile;
-import org.apache.cassandra.io.util.ThrottledReader;
import org.apache.cassandra.metric
[2/4] cassandra git commit: Cleanup SegmentedFile API
Cleanup SegmentedFile API
patch by benedict; reviewed by marcus for CASSANDRA-8749
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9d4909ee
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9d4909ee
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9d4909ee
Branch: refs/heads/trunk
Commit: 9d4909ee362be5082a6677bdbfba9d2b29848bfd
Parents: 207751c
Author: Benedict Elliott Smith
Authored: Wed Feb 11 15:45:18 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 15:45:18 2015 +
--
CHANGES.txt | 1 +
.../compress/CompressedRandomAccessReader.java | 15 -
.../cassandra/io/sstable/SSTableReader.java | 13 +++-
.../io/util/BufferedPoolingSegmentedFile.java | 5 -
.../io/util/CompressedPoolingSegmentedFile.java | 16 +-
.../io/util/CompressedSegmentedFile.java| 14 +
.../cassandra/io/util/MappedFileDataInput.java | 9
.../cassandra/io/util/MmappedSegmentedFile.java | 5 +++--
.../cassandra/io/util/PoolingSegmentedFile.java | 9 ++--
.../apache/cassandra/io/util/SegmentedFile.java | 22 ++--
10 files changed, 63 insertions(+), 46 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9d4909ee/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7352068..3f04637 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.4
+ * Cleanup SegmentedFile API (CASSANDRA-8749)
* Avoid overlap with early compaction replacement (CASSANDRA-8683)
* Safer Resource Management++ (CASSANDRA-8707)
* Write partition size estimates into a system table (CASSANDRA-7688)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9d4909ee/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
--
diff --git
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
index 4521c19..49dcd3d 100644
---
a/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
+++
b/src/java/org/apache/cassandra/io/compress/CompressedRandomAccessReader.java
@@ -37,6 +37,10 @@ import org.apache.cassandra.utils.FBUtilities;
*/
public class CompressedRandomAccessReader extends RandomAccessReader
{
+public static CompressedRandomAccessReader open(String dataFilePath,
CompressionMetadata metadata)
+{
+return open(dataFilePath, metadata, null);
+}
public static CompressedRandomAccessReader open(String path,
CompressionMetadata metadata, CompressedPoolingSegmentedFile owner)
{
try
@@ -49,17 +53,6 @@ public class CompressedRandomAccessReader extends
RandomAccessReader
}
}
-public static CompressedRandomAccessReader open(String dataFilePath,
CompressionMetadata metadata)
-{
-try
-{
-return new CompressedRandomAccessReader(dataFilePath, metadata,
null);
-}
-catch (FileNotFoundException e)
-{
-throw new RuntimeException(e);
-}
-}
private final CompressionMetadata metadata;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9d4909ee/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index a588bff..f59e632 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -77,8 +77,6 @@ import org.apache.cassandra.dht.IPartitioner;
import org.apache.cassandra.dht.LocalPartitioner;
import org.apache.cassandra.dht.Range;
import org.apache.cassandra.dht.Token;
-import org.apache.cassandra.io.compress.CompressedRandomAccessReader;
-import org.apache.cassandra.io.compress.CompressedThrottledReader;
import org.apache.cassandra.io.compress.CompressionMetadata;
import org.apache.cassandra.io.sstable.metadata.CompactionMetadata;
import org.apache.cassandra.io.sstable.metadata.MetadataComponent;
@@ -93,7 +91,6 @@ import org.apache.cassandra.io.util.FileUtils;
import org.apache.cassandra.io.util.ICompressedFile;
import org.apache.cassandra.io.util.RandomAccessReader;
import org.apache.cassandra.io.util.SegmentedFile;
-import org.apache.cassandra.io.util.ThrottledReader;
import org.apache.cassandra.metrics.RestorableMeter;
import org.apache.cassandra.metrics.StorageMetrics;
import org.apache.cassandra.service.ActiveRepairService;
@@ -1852,21
[2/3] cassandra git commit: Safer Resource Management++
Safer Resource Management++
patch by benedict; reviewed by marcus for CASSANDRA-8707
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/207751c8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/207751c8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/207751c8
Branch: refs/heads/trunk
Commit: 207751c8ac9800b7e004ec72096c478a30aafebc
Parents: 857ee0a
Author: Benedict Elliott Smith
Authored: Wed Feb 11 15:35:50 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 15:35:50 2015 +
--
.../utils/concurrent/SharedCloseable.java | 35 ++
.../utils/concurrent/SharedCloseableImpl.java | 47 ++
.../concurrent/WrappedSharedCloseable.java | 51
3 files changed, 133 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/207751c8/src/java/org/apache/cassandra/utils/concurrent/SharedCloseable.java
--
diff --git
a/src/java/org/apache/cassandra/utils/concurrent/SharedCloseable.java
b/src/java/org/apache/cassandra/utils/concurrent/SharedCloseable.java
new file mode 100644
index 000..1e5a026
--- /dev/null
+++ b/src/java/org/apache/cassandra/utils/concurrent/SharedCloseable.java
@@ -0,0 +1,35 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+*http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*/
+package org.apache.cassandra.utils.concurrent;
+
+/**
+ * A simple extension of AutoCloseable, that represents a resource that can be
utilised in multiple locations,
+ * each managing their own closure of the resource, so that when the last such
instance is closed all are.
+ *
+ */
+public interface SharedCloseable extends AutoCloseable
+{
+/**
+ * @return a new instance of the object representing the same state and
backed by the same underlying resources.
+ * Coordinates with the original (and other instances) when the underlying
resource should be closed.
+ * Throws an exception if the shared resource has already been closed.
+ */
+public SharedCloseable sharedCopy();
+
+}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/207751c8/src/java/org/apache/cassandra/utils/concurrent/SharedCloseableImpl.java
--
diff --git
a/src/java/org/apache/cassandra/utils/concurrent/SharedCloseableImpl.java
b/src/java/org/apache/cassandra/utils/concurrent/SharedCloseableImpl.java
new file mode 100644
index 000..0d3a843
--- /dev/null
+++ b/src/java/org/apache/cassandra/utils/concurrent/SharedCloseableImpl.java
@@ -0,0 +1,47 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+*http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*/
+package org.apache.cassandra.utils.concurrent;
+
+/**
+ * A simple abstract implementation of SharedCloseable
+ */
+public abstract class SharedCloseableImpl implements SharedCloseable
+{
+final Ref ref;
+
+public SharedCloseableImpl(RefCounted.Tidy tidy)
+{
+ref = new Ref(null, tidy);
+}
+
+protected SharedCloseableImpl(SharedCloseableImpl copy)
+{
+this.ref = copy.ref.ref();
+}
+
+public boolean isCleanedUp()
+{
+return ref.globalCount() == 0;
+}
+
+public void close()
+{
+ref.ensureReleased();
+}
+}
http://git-wip-us.apache.org/repos/asf/cassan
[1/3] cassandra git commit: Safer Resource Management++
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 857ee0ac3 -> 207751c8a
refs/heads/trunk 96866ce53 -> d73e7db80
Safer Resource Management++
patch by benedict; reviewed by marcus for CASSANDRA-8707
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/207751c8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/207751c8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/207751c8
Branch: refs/heads/cassandra-2.1
Commit: 207751c8ac9800b7e004ec72096c478a30aafebc
Parents: 857ee0a
Author: Benedict Elliott Smith
Authored: Wed Feb 11 15:35:50 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 15:35:50 2015 +
--
.../utils/concurrent/SharedCloseable.java | 35 ++
.../utils/concurrent/SharedCloseableImpl.java | 47 ++
.../concurrent/WrappedSharedCloseable.java | 51
3 files changed, 133 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/207751c8/src/java/org/apache/cassandra/utils/concurrent/SharedCloseable.java
--
diff --git
a/src/java/org/apache/cassandra/utils/concurrent/SharedCloseable.java
b/src/java/org/apache/cassandra/utils/concurrent/SharedCloseable.java
new file mode 100644
index 000..1e5a026
--- /dev/null
+++ b/src/java/org/apache/cassandra/utils/concurrent/SharedCloseable.java
@@ -0,0 +1,35 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+*http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*/
+package org.apache.cassandra.utils.concurrent;
+
+/**
+ * A simple extension of AutoCloseable, that represents a resource that can be
utilised in multiple locations,
+ * each managing their own closure of the resource, so that when the last such
instance is closed all are.
+ *
+ */
+public interface SharedCloseable extends AutoCloseable
+{
+/**
+ * @return a new instance of the object representing the same state and
backed by the same underlying resources.
+ * Coordinates with the original (and other instances) when the underlying
resource should be closed.
+ * Throws an exception if the shared resource has already been closed.
+ */
+public SharedCloseable sharedCopy();
+
+}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/207751c8/src/java/org/apache/cassandra/utils/concurrent/SharedCloseableImpl.java
--
diff --git
a/src/java/org/apache/cassandra/utils/concurrent/SharedCloseableImpl.java
b/src/java/org/apache/cassandra/utils/concurrent/SharedCloseableImpl.java
new file mode 100644
index 000..0d3a843
--- /dev/null
+++ b/src/java/org/apache/cassandra/utils/concurrent/SharedCloseableImpl.java
@@ -0,0 +1,47 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+*http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*/
+package org.apache.cassandra.utils.concurrent;
+
+/**
+ * A simple abstract implementation of SharedCloseable
+ */
+public abstract class SharedCloseableImpl implements SharedCloseable
+{
+final Ref ref;
+
+public SharedCloseableImpl(RefCounted.Tidy tidy)
+{
+ref = new Ref(null, tidy);
+}
+
+protected SharedCloseableImpl(SharedCloseableImpl copy)
+{
+this.ref = copy.ref.ref();
+}
+
+public boolean isCleanedUp()
+{
+return ref.global
[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d73e7db8 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d73e7db8 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d73e7db8 Branch: refs/heads/trunk Commit: d73e7db80b8c3cf4d982eedb456de5816e9958d2 Parents: 96866ce 207751c Author: Benedict Elliott Smith Authored: Wed Feb 11 15:36:22 2015 + Committer: Benedict Elliott Smith Committed: Wed Feb 11 15:36:22 2015 + -- .../utils/concurrent/SharedCloseable.java | 35 ++ .../utils/concurrent/SharedCloseableImpl.java | 47 ++ .../concurrent/WrappedSharedCloseable.java | 51 3 files changed, 133 insertions(+) --
[jira] [Updated] (CASSANDRA-8683) Ensure early reopening has no overlap with replaced files
[ https://issues.apache.org/jira/browse/CASSANDRA-8683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-8683: Fix Version/s: (was: 2.1.3) 2.1.4 > Ensure early reopening has no overlap with replaced files > - > > Key: CASSANDRA-8683 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8683 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Benedict >Priority: Critical > Fix For: 2.1.4 > > Attachments: 0001-avoid-NPE-in-getPositionsForRanges.patch, > system.log, test.sh > > > When introducing CASSANDRA-6916 we permitted the early opened files to > overlap with the files they were replacing by one DecoratedKey, as this > permitted a few minor simplifications. Unfortunately this breaks assumptions > in LeveledCompactionScanner, that are causing the intermittent unit test > failures: > http://cassci.datastax.com/job/trunk_utest/1330/testReport/junit/org.apache.cassandra.db.compaction/LeveledCompactionStrategyTest/testValidationMultipleSSTablePerLevel/ > This patch by itself does not fix the bug, but fixes the described aspect of > it, by ensuring the replaced and replacing files never overlap. This is > achieved first by always selecting the replaced file start as the next key > present in the file greater than the last key in the new file(s). If there > is no such key, however, there is no data to return for the reader, but to > permit abort and atomic replacement at the end of a macro compaction action, > we must keep the file in the DataTracker for replacement purposes, but not > return it to consumers (esp. as many assume a non-empty range). For this I > have introduced a new OpenReason called SHADOWED, and a > DataTracker.View.shadowed collection of sstables, that tracks those we still > consider to be in the live set, but from which we no longer answer any > queries. > CASSANDRA-8744 (and then CASSANDRA-8750) then ensures that these bounds are > honoured, so that we never break the assumption that files in LCS never > overlap. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[3/4] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
http://git-wip-us.apache.org/repos/asf/cassandra/blob/96866ce5/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java -- diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java index 889ade3,000..be783a1 mode 100644,00..100644 --- a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java +++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java @@@ -1,2046 -1,0 +1,2059 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.io.sstable.format; + +import java.io.*; +import java.nio.ByteBuffer; +import java.util.*; +import java.util.concurrent.*; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicLong; + +import com.google.common.annotations.VisibleForTesting; +import com.google.common.base.Predicate; +import com.google.common.collect.Iterators; +import com.google.common.collect.Ordering; +import com.google.common.primitives.Longs; +import com.google.common.util.concurrent.RateLimiter; + +import com.clearspring.analytics.stream.cardinality.CardinalityMergeException; +import com.clearspring.analytics.stream.cardinality.HyperLogLogPlus; +import com.clearspring.analytics.stream.cardinality.ICardinality; +import org.apache.cassandra.cache.CachingOptions; +import org.apache.cassandra.cache.InstrumentingCache; +import org.apache.cassandra.cache.KeyCacheKey; +import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor; +import org.apache.cassandra.concurrent.ScheduledExecutors; +import org.apache.cassandra.config.*; +import org.apache.cassandra.db.*; +import org.apache.cassandra.db.columniterator.OnDiskAtomIterator; +import org.apache.cassandra.db.commitlog.ReplayPosition; +import org.apache.cassandra.db.composites.CellName; +import org.apache.cassandra.db.filter.ColumnSlice; +import org.apache.cassandra.db.index.SecondaryIndex; +import org.apache.cassandra.dht.*; +import org.apache.cassandra.io.compress.CompressedRandomAccessReader; +import org.apache.cassandra.io.compress.CompressedThrottledReader; +import org.apache.cassandra.io.compress.CompressionMetadata; +import org.apache.cassandra.io.sstable.*; +import org.apache.cassandra.io.sstable.metadata.*; +import org.apache.cassandra.io.util.*; +import org.apache.cassandra.metrics.RestorableMeter; +import org.apache.cassandra.metrics.StorageMetrics; +import org.apache.cassandra.service.ActiveRepairService; +import org.apache.cassandra.service.CacheService; +import org.apache.cassandra.service.StorageService; +import org.apache.cassandra.utils.*; +import org.apache.cassandra.utils.concurrent.OpOrder; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.apache.cassandra.utils.concurrent.Ref; +import org.apache.cassandra.utils.concurrent.RefCounted; + +import static org.apache.cassandra.db.Directories.SECONDARY_INDEX_NAME_SEPARATOR; + +/** + * An SSTableReader can be constructed in a number of places, but typically is either + * read from disk at startup, or constructed from a flushed memtable, or after compaction + * to replace some existing sstables. However once created, an sstablereader may also be modified. + * + * A reader's OpenReason describes its current stage in its lifecycle, as follows: + * + * NORMAL + * From: None=> Reader has been read from disk, either at startup or from a flushed memtable + * EARLY => Reader is the final result of a compaction + * MOVED_START => Reader WAS being compacted, but this failed and it has been restored to NORMAL status + * + * EARLY + * From: None=> Reader is a compaction replacement that is either incomplete and has been opened + *to represent its partial result status, or has been finished but the compaction + *it is a part of has not yet completed fully + * EARLY => Same as from None, only it is not the first time it has been + * + * MOVED_START + * From: NORMAL => Reader is being compacted. This comp
[2/4] cassandra git commit: Avoid overlap with early compaction replacement
Avoid overlap with early compaction replacement
patch by benedict; reviewed by marcus for CASSANDRA-8683
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/857ee0ac
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/857ee0ac
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/857ee0ac
Branch: refs/heads/trunk
Commit: 857ee0ac3765e12140a0e072cee3fd32b1ad1b3d
Parents: 61384c5
Author: Benedict Elliott Smith
Authored: Wed Feb 11 15:29:46 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 15:29:46 2015 +
--
CHANGES.txt | 1 +
.../org/apache/cassandra/db/DataTracker.java| 67 +--
.../cassandra/io/sstable/SSTableReader.java | 69
.../cassandra/io/sstable/SSTableRewriter.java | 61 -
.../io/sstable/SSTableRewriterTest.java | 9 +--
5 files changed, 138 insertions(+), 69 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/857ee0ac/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b323f18..7352068 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.4
+ * Avoid overlap with early compaction replacement (CASSANDRA-8683)
* Safer Resource Management++ (CASSANDRA-8707)
* Write partition size estimates into a system table (CASSANDRA-7688)
* cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
http://git-wip-us.apache.org/repos/asf/cassandra/blob/857ee0ac/src/java/org/apache/cassandra/db/DataTracker.java
--
diff --git a/src/java/org/apache/cassandra/db/DataTracker.java
b/src/java/org/apache/cassandra/db/DataTracker.java
index acf9f92..8224311 100644
--- a/src/java/org/apache/cassandra/db/DataTracker.java
+++ b/src/java/org/apache/cassandra/db/DataTracker.java
@@ -379,6 +379,7 @@ public class DataTracker
ImmutableList.of(),
Collections.emptySet(),
Collections.emptySet(),
+Collections.emptySet(),
SSTableIntervalTree.empty()));
}
@@ -612,10 +613,14 @@ public class DataTracker
private final List flushingMemtables;
public final Set compacting;
public final Set sstables;
+
+// all sstables that are still in the live set, but have been
completely shadowed by a replacement sstable
+public final Set shadowed;
public final SSTableIntervalTree intervalTree;
-View(List liveMemtables, List flushingMemtables,
Set sstables, Set compacting, SSTableIntervalTree
intervalTree)
+View(List liveMemtables, List flushingMemtables,
Set sstables, Set compacting, Set
shadowed, SSTableIntervalTree intervalTree)
{
+this.shadowed = shadowed;
assert liveMemtables != null;
assert flushingMemtables != null;
assert sstables != null;
@@ -664,7 +669,7 @@ public class DataTracker
View switchMemtable(Memtable newMemtable)
{
List newLiveMemtables =
ImmutableList.builder().addAll(liveMemtables).add(newMemtable).build();
-return new View(newLiveMemtables, flushingMemtables, sstables,
compacting, intervalTree);
+return new View(newLiveMemtables, flushingMemtables, sstables,
compacting, shadowed, intervalTree);
}
View markFlushing(Memtable toFlushMemtable)
@@ -691,7 +696,7 @@ public class DataTracker
.addAll(flushing.subList(i, flushing.size()))
.build();
-return new View(newLive, newFlushing, sstables, compacting,
intervalTree);
+return new View(newLive, newFlushing, sstables, compacting,
shadowed, intervalTree);
}
View replaceFlushed(Memtable flushedMemtable, SSTableReader newSSTable)
@@ -701,37 +706,61 @@ public class DataTracker
.addAll(flushingMemtables.subList(0, index))
.addAll(flushingMemtables.subList(index + 1, flushingMemtables.size()))
.build();
-Set newSSTables = newSSTable == null
- ? sstables
- : newSSTables(newSSTable);
-SSTableIntervalTree intervalTree = buildIntervalTree(newSSTables);
-return new View(liveMemtables, newQueuedMemtables, newSSTables,
compacting, intervalTree);
+Set newSSTables = sstables;
+SST
[4/4] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/96866ce5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/96866ce5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/96866ce5 Branch: refs/heads/trunk Commit: 96866ce53fb2750217305e881b284851958f4728 Parents: 02c3489 857ee0a Author: Benedict Elliott Smith Authored: Wed Feb 11 15:31:01 2015 + Committer: Benedict Elliott Smith Committed: Wed Feb 11 15:31:01 2015 + -- CHANGES.txt | 1 + .../org/apache/cassandra/db/DataTracker.java| 67 -- .../cassandra/io/sstable/SSTableRewriter.java | 61 - .../io/sstable/format/SSTableReader.java| 71 .../io/sstable/SSTableRewriterTest.java | 9 +-- 5 files changed, 139 insertions(+), 70 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/96866ce5/CHANGES.txt -- diff --cc CHANGES.txt index c32a2c4,7352068..1a4c87c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,63 -1,5 +1,64 @@@ +3.0 + * Add role based access control (CASSANDRA-7653, 8650) + * Avoid accessing partitioner through StorageProxy (CASSANDRA-8244, 8268) + * Upgrade Metrics library and remove depricated metrics (CASSANDRA-5657) + * Serializing Row cache alternative, fully off heap (CASSANDRA-7438) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560) + * Support direct buffer decompression for reads (CASSANDRA-8464) + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039) + * Group sstables for anticompaction correctly (CASSANDRA-8578) + * Add ReadFailureException to native protocol, respond + immediately when replicas encounter errors while handling + a read request (CASSANDRA-7886) + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308) + * Allow mixing token and partition key restrictions (CASSANDRA-7016) + * Support index key/value entries on map collections (CASSANDRA-8473) + * Modernize schema tables (CASSANDRA-8261) + * Support for user-defined aggregation functions (CASSANDRA-8053) + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419) + * Refactor SelectStatement, return IN results in natural order instead + of IN value list order and ignore duplicate values in partition key IN restrictions (CASSANDRA-7981) + * Support UDTs, tuples, and collections in user-defined + functions (CASSANDRA-7563) + * Fix aggregate fn results on empty selection, result column name, + and cqlsh parsing (CASSANDRA-8229) + * Mark sstables as repaired after full repair (CASSANDRA-7586) + * Extend Descriptor to include a format value and refactor reader/writer + APIs (CASSANDRA-7443) + * Integrate JMH for microbenchmarks (CASSANDRA-8151) + * Keep sstable levels when bootstrapping (CASSANDRA-7460) + * Add Sigar library and perform basic OS settings check on startup (CASSANDRA-7838) + * Support for aggregation functions (CASSANDRA-4914) + * Remove cassandra-cli (CASSANDRA-7920) + * Accept dollar quoted strings in CQL (CASSANDRA-7769) + * Make assassinate a first class command (CASSANDRA-7935) + * Support IN clause on any partition key column (CASSANDRA-7855) + * Support IN clause on any clustering column (CASSANDRA-4762) + * Improve compaction logging (CASSANDRA-7818) + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917) + * Do anticompaction in groups (CASSANDRA-6851) + * Support user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 7929, + 7924, 7812, 8063, 7813, 7708) + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416) + * Move sstable RandomAccessReader to nio2, which allows using the + FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050) + * Remove CQL2 (CASSANDRA-5918) + * Add Thrift get_multi_slice call (CASSANDRA-6757) + * Optimize fetching multiple cells by name (CASSANDRA-6933) + * Allow compilation in java 8 (CASSANDRA-7028) + * Make incremental repair default (CASSANDRA-7250) + * Enable code coverage thru JaCoCo (CASSANDRA-7226) + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) + * Shorten SSTable path (CASSANDRA-6962) + * Use unsafe mutations for most unit tests (CASSANDRA-6969) + * Fix race condition during calculation of pending ranges (CASSANDRA-7390) + * Fail on very large batch sizes (CASSANDRA-8011) + * Improve concurrency of repair (CASSANDRA-6455, 8208) + + 2.1.4 + * Avoid overlap with early compaction replacement (CASSANDR
[1/4] cassandra git commit: Avoid overlap with early compaction replacement
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 61384c575 -> 857ee0ac3
refs/heads/trunk 02c348939 -> 96866ce53
Avoid overlap with early compaction replacement
patch by benedict; reviewed by marcus for CASSANDRA-8683
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/857ee0ac
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/857ee0ac
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/857ee0ac
Branch: refs/heads/cassandra-2.1
Commit: 857ee0ac3765e12140a0e072cee3fd32b1ad1b3d
Parents: 61384c5
Author: Benedict Elliott Smith
Authored: Wed Feb 11 15:29:46 2015 +
Committer: Benedict Elliott Smith
Committed: Wed Feb 11 15:29:46 2015 +
--
CHANGES.txt | 1 +
.../org/apache/cassandra/db/DataTracker.java| 67 +--
.../cassandra/io/sstable/SSTableReader.java | 69
.../cassandra/io/sstable/SSTableRewriter.java | 61 -
.../io/sstable/SSTableRewriterTest.java | 9 +--
5 files changed, 138 insertions(+), 69 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/857ee0ac/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b323f18..7352068 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.4
+ * Avoid overlap with early compaction replacement (CASSANDRA-8683)
* Safer Resource Management++ (CASSANDRA-8707)
* Write partition size estimates into a system table (CASSANDRA-7688)
* cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
http://git-wip-us.apache.org/repos/asf/cassandra/blob/857ee0ac/src/java/org/apache/cassandra/db/DataTracker.java
--
diff --git a/src/java/org/apache/cassandra/db/DataTracker.java
b/src/java/org/apache/cassandra/db/DataTracker.java
index acf9f92..8224311 100644
--- a/src/java/org/apache/cassandra/db/DataTracker.java
+++ b/src/java/org/apache/cassandra/db/DataTracker.java
@@ -379,6 +379,7 @@ public class DataTracker
ImmutableList.of(),
Collections.emptySet(),
Collections.emptySet(),
+Collections.emptySet(),
SSTableIntervalTree.empty()));
}
@@ -612,10 +613,14 @@ public class DataTracker
private final List flushingMemtables;
public final Set compacting;
public final Set sstables;
+
+// all sstables that are still in the live set, but have been
completely shadowed by a replacement sstable
+public final Set shadowed;
public final SSTableIntervalTree intervalTree;
-View(List liveMemtables, List flushingMemtables,
Set sstables, Set compacting, SSTableIntervalTree
intervalTree)
+View(List liveMemtables, List flushingMemtables,
Set sstables, Set compacting, Set
shadowed, SSTableIntervalTree intervalTree)
{
+this.shadowed = shadowed;
assert liveMemtables != null;
assert flushingMemtables != null;
assert sstables != null;
@@ -664,7 +669,7 @@ public class DataTracker
View switchMemtable(Memtable newMemtable)
{
List newLiveMemtables =
ImmutableList.builder().addAll(liveMemtables).add(newMemtable).build();
-return new View(newLiveMemtables, flushingMemtables, sstables,
compacting, intervalTree);
+return new View(newLiveMemtables, flushingMemtables, sstables,
compacting, shadowed, intervalTree);
}
View markFlushing(Memtable toFlushMemtable)
@@ -691,7 +696,7 @@ public class DataTracker
.addAll(flushing.subList(i, flushing.size()))
.build();
-return new View(newLive, newFlushing, sstables, compacting,
intervalTree);
+return new View(newLive, newFlushing, sstables, compacting,
shadowed, intervalTree);
}
View replaceFlushed(Memtable flushedMemtable, SSTableReader newSSTable)
@@ -701,37 +706,61 @@ public class DataTracker
.addAll(flushingMemtables.subList(0, index))
.addAll(flushingMemtables.subList(index + 1, flushingMemtables.size()))
.build();
-Set newSSTables = newSSTable == null
- ? sstables
- : newSSTables(newSSTable);
-SSTableIntervalTree intervalTree = buildIntervalTree(newSSTables);
-return
[1/7] cassandra git commit: Safer Resource Management++
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 708b0ce6d -> 61384c575
refs/heads/trunk 5a17ec875 -> 02c348939
http://git-wip-us.apache.org/repos/asf/cassandra/blob/61384c57/src/java/org/apache/cassandra/utils/BloomFilter.java
--
diff --git a/src/java/org/apache/cassandra/utils/BloomFilter.java
b/src/java/org/apache/cassandra/utils/BloomFilter.java
index ceba89b..77b2d44 100644
--- a/src/java/org/apache/cassandra/utils/BloomFilter.java
+++ b/src/java/org/apache/cassandra/utils/BloomFilter.java
@@ -21,9 +21,10 @@ import java.nio.ByteBuffer;
import com.google.common.annotations.VisibleForTesting;
+import org.apache.cassandra.utils.concurrent.WrappedSharedCloseable;
import org.apache.cassandra.utils.obs.IBitSet;
-public abstract class BloomFilter implements IFilter
+public abstract class BloomFilter extends WrappedSharedCloseable implements
IFilter
{
private static final ThreadLocal reusableIndexes = new
ThreadLocal()
{
@@ -36,12 +37,20 @@ public abstract class BloomFilter implements IFilter
public final IBitSet bitset;
public final int hashCount;
-BloomFilter(int hashes, IBitSet bitset)
+BloomFilter(int hashCount, IBitSet bitset)
{
-this.hashCount = hashes;
+super(bitset);
+this.hashCount = hashCount;
this.bitset = bitset;
}
+BloomFilter(BloomFilter copy)
+{
+super(copy);
+this.hashCount = copy.hashCount;
+this.bitset = copy.bitset;
+}
+
// Murmur is faster than an SHA-based approach and provides as-good
collision
// resistance. The combinatorial generation approach described in
// http://www.eecs.harvard.edu/~kirsch/pubs/bbbf/esa06.pdf
@@ -110,9 +119,4 @@ public abstract class BloomFilter implements IFilter
{
bitset.clear();
}
-
-public void close()
-{
-bitset.close();
-}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/61384c57/src/java/org/apache/cassandra/utils/IFilter.java
--
diff --git a/src/java/org/apache/cassandra/utils/IFilter.java
b/src/java/org/apache/cassandra/utils/IFilter.java
index 60c0590..bde6333 100644
--- a/src/java/org/apache/cassandra/utils/IFilter.java
+++ b/src/java/org/apache/cassandra/utils/IFilter.java
@@ -17,10 +17,11 @@
*/
package org.apache.cassandra.utils;
-import java.io.Closeable;
import java.nio.ByteBuffer;
-public interface IFilter extends Closeable
+import org.apache.cassandra.utils.concurrent.SharedCloseable;
+
+public interface IFilter extends SharedCloseable
{
void add(ByteBuffer key);
@@ -32,6 +33,8 @@ public interface IFilter extends Closeable
void close();
+IFilter sharedCopy();
+
/**
* Returns the amount of memory in bytes used off heap.
* @return the amount of memory in bytes used off heap
http://git-wip-us.apache.org/repos/asf/cassandra/blob/61384c57/src/java/org/apache/cassandra/utils/Murmur3BloomFilter.java
--
diff --git a/src/java/org/apache/cassandra/utils/Murmur3BloomFilter.java
b/src/java/org/apache/cassandra/utils/Murmur3BloomFilter.java
index f7c7632..431ca5b 100644
--- a/src/java/org/apache/cassandra/utils/Murmur3BloomFilter.java
+++ b/src/java/org/apache/cassandra/utils/Murmur3BloomFilter.java
@@ -26,9 +26,14 @@ public class Murmur3BloomFilter extends BloomFilter
{
public static final Murmur3BloomFilterSerializer serializer = new
Murmur3BloomFilterSerializer();
-public Murmur3BloomFilter(int hashes, IBitSet bs)
+public Murmur3BloomFilter(int hashCount, IBitSet bs)
{
-super(hashes, bs);
+super(hashCount, bs);
+}
+
+protected Murmur3BloomFilter(Murmur3BloomFilter copy)
+{
+super(copy);
}
public long serializedSize()
@@ -36,6 +41,11 @@ public class Murmur3BloomFilter extends BloomFilter
return serializer.serializedSize(this, TypeSizes.NATIVE);
}
+public IFilter sharedCopy()
+{
+return new Murmur3BloomFilter(this);
+}
+
@Override
public long offHeapSize()
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/61384c57/src/java/org/apache/cassandra/utils/concurrent/Ref.java
--
diff --git a/src/java/org/apache/cassandra/utils/concurrent/Ref.java
b/src/java/org/apache/cassandra/utils/concurrent/Ref.java
index 4afceb0..ad1293b 100644
--- a/src/java/org/apache/cassandra/utils/concurrent/Ref.java
+++ b/src/java/org/apache/cassandra/utils/concurrent/Ref.java
@@ -2,24 +2,69 @@ package org.apache.cassandra.utils.concurrent;
import java.lang.ref.PhantomReference;
import java.lang.ref.ReferenceQueue;
+import java.util.Collections;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.con
[5/7] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
http://git-wip-us.apache.org/repos/asf/cassandra/blob/02c34893/src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java
--
diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java
index 32b5520,000..33f6bd1
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java
@@@ -1,215 -1,0 +1,213 @@@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.io.sstable.format;
+
+import com.google.common.collect.Sets;
+import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.config.Schema;
+import org.apache.cassandra.db.ColumnFamily;
+import org.apache.cassandra.db.DecoratedKey;
+import org.apache.cassandra.db.RowIndexEntry;
+import org.apache.cassandra.db.compaction.AbstractCompactedRow;
+import org.apache.cassandra.dht.IPartitioner;
+import org.apache.cassandra.io.sstable.Component;
+import org.apache.cassandra.io.sstable.Descriptor;
+import org.apache.cassandra.io.sstable.SSTable;
+import org.apache.cassandra.io.sstable.metadata.MetadataCollector;
+import org.apache.cassandra.io.sstable.metadata.StatsMetadata;
+import org.apache.cassandra.io.util.FileUtils;
+import org.apache.cassandra.utils.Pair;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.DataInput;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.Set;
+
+/**
+ * This is the API all table writers must implement.
+ *
+ * TableWriter.create() is the primary way to create a writer for a
particular format.
+ * The format information is part of the Descriptor.
+ */
+public abstract class SSTableWriter extends SSTable
+{
+private static final Logger logger =
LoggerFactory.getLogger(SSTableWriter.class);
+
+public static enum FinishType
+{
- NORMAL(SSTableReader.OpenReason.NORMAL),
- EARLY(SSTableReader.OpenReason.EARLY), // no renaming
- FINISH_EARLY(SSTableReader.OpenReason.NORMAL); // tidy up an EARLY
finish
++CLOSE(null, true),
++NORMAL(SSTableReader.OpenReason.NORMAL, true),
++EARLY(SSTableReader.OpenReason.EARLY, false), // no renaming
++FINISH_EARLY(SSTableReader.OpenReason.NORMAL, true); // tidy up an
EARLY finish
+public final SSTableReader.OpenReason openReason;
+
- FinishType(SSTableReader.OpenReason openReason)
++public final boolean isFinal;
++FinishType(SSTableReader.OpenReason openReason, boolean isFinal)
+{
+this.openReason = openReason;
++this.isFinal = isFinal;
+}
+}
+
+protected final long repairedAt;
+protected final long keyCount;
+protected final MetadataCollector metadataCollector;
+protected final RowIndexEntry.IndexSerializer rowIndexEntrySerializer;
+
+protected SSTableWriter(Descriptor descriptor, long keyCount, long
repairedAt, CFMetaData metadata, IPartitioner partitioner, MetadataCollector
metadataCollector)
+{
+super(descriptor, components(metadata), metadata, partitioner);
+this.keyCount = keyCount;
+this.repairedAt = repairedAt;
+this.metadataCollector = metadataCollector;
+this.rowIndexEntrySerializer =
descriptor.version.getSSTableFormat().getIndexSerializer(metadata);
+}
+
+public static SSTableWriter create(Descriptor descriptor, Long keyCount,
Long repairedAt, CFMetaData metadata, IPartitioner partitioner,
MetadataCollector metadataCollector)
+{
+Factory writerFactory = descriptor.getFormat().getWriterFactory();
+return writerFactory.open(descriptor, keyCount, repairedAt, metadata,
partitioner, metadataCollector);
+}
+
+public static SSTableWriter create(Descriptor descriptor, long keyCount,
long repairedAt)
+{
+return create(descriptor, keyCount, repairedAt, 0);
+}
+
+public static SSTableWriter create(Descriptor descrip
[6/7] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
http://git-wip-us.apache.org/repos/asf/cassandra/blob/02c34893/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java -- diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java index de2bbc6,000..889ade3 mode 100644,00..100644 --- a/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java +++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java @@@ -1,1908 -1,0 +1,2046 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.io.sstable.format; + +import java.io.*; +import java.nio.ByteBuffer; +import java.util.*; +import java.util.concurrent.*; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicLong; + +import com.google.common.annotations.VisibleForTesting; +import com.google.common.base.Predicate; +import com.google.common.collect.Iterators; +import com.google.common.collect.Ordering; +import com.google.common.primitives.Longs; +import com.google.common.util.concurrent.RateLimiter; + +import com.clearspring.analytics.stream.cardinality.CardinalityMergeException; +import com.clearspring.analytics.stream.cardinality.HyperLogLogPlus; +import com.clearspring.analytics.stream.cardinality.ICardinality; +import org.apache.cassandra.cache.CachingOptions; +import org.apache.cassandra.cache.InstrumentingCache; +import org.apache.cassandra.cache.KeyCacheKey; +import org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor; +import org.apache.cassandra.concurrent.ScheduledExecutors; +import org.apache.cassandra.config.*; +import org.apache.cassandra.db.*; +import org.apache.cassandra.db.columniterator.OnDiskAtomIterator; +import org.apache.cassandra.db.commitlog.ReplayPosition; +import org.apache.cassandra.db.composites.CellName; +import org.apache.cassandra.db.filter.ColumnSlice; +import org.apache.cassandra.db.index.SecondaryIndex; +import org.apache.cassandra.dht.*; +import org.apache.cassandra.io.compress.CompressedRandomAccessReader; +import org.apache.cassandra.io.compress.CompressedThrottledReader; +import org.apache.cassandra.io.compress.CompressionMetadata; +import org.apache.cassandra.io.sstable.*; +import org.apache.cassandra.io.sstable.metadata.*; +import org.apache.cassandra.io.util.*; +import org.apache.cassandra.metrics.RestorableMeter; +import org.apache.cassandra.metrics.StorageMetrics; +import org.apache.cassandra.service.ActiveRepairService; +import org.apache.cassandra.service.CacheService; +import org.apache.cassandra.service.StorageService; +import org.apache.cassandra.utils.*; +import org.apache.cassandra.utils.concurrent.OpOrder; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.apache.cassandra.utils.concurrent.Ref; +import org.apache.cassandra.utils.concurrent.RefCounted; + +import static org.apache.cassandra.db.Directories.SECONDARY_INDEX_NAME_SEPARATOR; + +/** - * SSTableReaders are open()ed by Keyspace.onStart; after that they are created by SSTableWriter.renameAndOpen. - * Do not re-call open() on existing SSTable files; use the references kept by ColumnFamilyStore post-start instead. ++ * An SSTableReader can be constructed in a number of places, but typically is either ++ * read from disk at startup, or constructed from a flushed memtable, or after compaction ++ * to replace some existing sstables. However once created, an sstablereader may also be modified. ++ * ++ * A reader's OpenReason describes its current stage in its lifecycle, as follows: ++ * ++ * NORMAL ++ * From: None=> Reader has been read from disk, either at startup or from a flushed memtable ++ * EARLY => Reader is the final result of a compaction ++ * MOVED_START => Reader WAS being compacted, but this failed and it has been restored to NORMAL status ++ * ++ * EARLY ++ * From: None=> Reader is a compaction replacement that is either incomplete and has been opened ++ *to represent its partial result status, or has been finished but the compaction ++ *
[2/7] cassandra git commit: Safer Resource Management++
Safer Resource Management++ patch by benedict; reviewed by marcus for CASSANDRA-8707 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/61384c57 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/61384c57 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/61384c57 Branch: refs/heads/cassandra-2.1 Commit: 61384c57546da3d411630c64c4aa89d90cac98f7 Parents: 708b0ce Author: Benedict Elliott Smith Authored: Wed Feb 11 14:56:23 2015 + Committer: Benedict Elliott Smith Committed: Wed Feb 11 14:56:23 2015 + -- CHANGES.txt | 1 + .../org/apache/cassandra/db/DataTracker.java| 6 +- .../cassandra/db/compaction/CompactionTask.java | 4 - .../SizeTieredCompactionStrategy.java | 10 +- .../io/compress/CompressionMetadata.java| 19 +- .../cassandra/io/sstable/IndexSummary.java | 47 +- .../io/sstable/IndexSummaryBuilder.java | 3 +- .../io/sstable/IndexSummaryManager.java | 6 +- .../io/sstable/SSTableDeletingTask.java | 36 +- .../cassandra/io/sstable/SSTableLoader.java | 4 +- .../cassandra/io/sstable/SSTableReader.java | 590 --- .../cassandra/io/sstable/SSTableRewriter.java | 10 +- .../cassandra/io/sstable/SSTableWriter.java | 45 +- .../io/util/BufferedPoolingSegmentedFile.java | 12 +- .../io/util/BufferedSegmentedFile.java | 23 +- .../io/util/CompressedPoolingSegmentedFile.java | 29 +- .../io/util/CompressedSegmentedFile.java| 32 +- .../org/apache/cassandra/io/util/Memory.java| 8 +- .../cassandra/io/util/MmappedSegmentedFile.java | 55 +- .../cassandra/io/util/PoolingSegmentedFile.java | 35 +- .../apache/cassandra/io/util/SegmentedFile.java | 40 +- .../cassandra/service/ActiveRepairService.java | 4 +- .../cassandra/streaming/StreamSession.java | 2 +- .../streaming/messages/OutgoingFileMessage.java | 2 +- .../cassandra/tools/StandaloneScrubber.java | 2 +- .../cassandra/utils/AlwaysPresentFilter.java| 5 + .../org/apache/cassandra/utils/BloomFilter.java | 20 +- .../org/apache/cassandra/utils/IFilter.java | 7 +- .../cassandra/utils/Murmur3BloomFilter.java | 14 +- .../apache/cassandra/utils/concurrent/Ref.java | 208 ++- .../cassandra/utils/concurrent/RefCounted.java | 52 +- .../utils/concurrent/RefCountedImpl.java| 132 - .../apache/cassandra/utils/concurrent/Refs.java | 26 +- .../cassandra/utils/obs/OffHeapBitSet.java | 5 + .../cassandra/db/ColumnFamilyStoreTest.java | 5 +- .../db/compaction/AntiCompactionTest.java | 6 +- .../SizeTieredCompactionStrategyTest.java | 30 +- .../io/sstable/IndexSummaryManagerTest.java | 37 +- .../cassandra/io/sstable/LegacySSTableTest.java | 3 +- .../cassandra/io/sstable/SSTableReaderTest.java | 11 +- .../io/sstable/SSTableRewriterTest.java | 13 +- .../streaming/StreamTransferTaskTest.java | 2 +- .../cassandra/tools/SSTableExportTest.java | 1 + .../cassandra/tools/SSTableImportTest.java | 7 + .../apache/cassandra/utils/BloomFilterTest.java | 22 +- .../cassandra/utils/SerializationsTest.java | 5 +- .../utils/concurrent/RefCountedTest.java| 12 +- 47 files changed, 991 insertions(+), 657 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/61384c57/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 248139f..b323f18 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.1.4 + * Safer Resource Management++ (CASSANDRA-8707) * Write partition size estimates into a system table (CASSANDRA-7688) * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output (CASSANDRA-8154) http://git-wip-us.apache.org/repos/asf/cassandra/blob/61384c57/src/java/org/apache/cassandra/db/DataTracker.java -- diff --git a/src/java/org/apache/cassandra/db/DataTracker.java b/src/java/org/apache/cassandra/db/DataTracker.java index 5ec06bc..acf9f92 100644 --- a/src/java/org/apache/cassandra/db/DataTracker.java +++ b/src/java/org/apache/cassandra/db/DataTracker.java @@ -367,7 +367,7 @@ public class DataTracker while (!view.compareAndSet(currentView, newView)); for (SSTableReader sstable : currentView.sstables) if (!remaining.contains(sstable)) -sstable.sharedRef().release(); +sstable.selfRef().release(); notifySSTablesChanged(remaining, Collections.emptySet(), OperationType.UNKNOWN); } @@ -406,7 +406,7 @@ public class DataTracker sstable.setTrackedBy(this); for (SSTa
[3/7] cassandra git commit: Safer Resource Management++
http://git-wip-us.apache.org/repos/asf/cassandra/blob/61384c57/src/java/org/apache/cassandra/utils/BloomFilter.java
--
diff --git a/src/java/org/apache/cassandra/utils/BloomFilter.java
b/src/java/org/apache/cassandra/utils/BloomFilter.java
index ceba89b..77b2d44 100644
--- a/src/java/org/apache/cassandra/utils/BloomFilter.java
+++ b/src/java/org/apache/cassandra/utils/BloomFilter.java
@@ -21,9 +21,10 @@ import java.nio.ByteBuffer;
import com.google.common.annotations.VisibleForTesting;
+import org.apache.cassandra.utils.concurrent.WrappedSharedCloseable;
import org.apache.cassandra.utils.obs.IBitSet;
-public abstract class BloomFilter implements IFilter
+public abstract class BloomFilter extends WrappedSharedCloseable implements
IFilter
{
private static final ThreadLocal reusableIndexes = new
ThreadLocal()
{
@@ -36,12 +37,20 @@ public abstract class BloomFilter implements IFilter
public final IBitSet bitset;
public final int hashCount;
-BloomFilter(int hashes, IBitSet bitset)
+BloomFilter(int hashCount, IBitSet bitset)
{
-this.hashCount = hashes;
+super(bitset);
+this.hashCount = hashCount;
this.bitset = bitset;
}
+BloomFilter(BloomFilter copy)
+{
+super(copy);
+this.hashCount = copy.hashCount;
+this.bitset = copy.bitset;
+}
+
// Murmur is faster than an SHA-based approach and provides as-good
collision
// resistance. The combinatorial generation approach described in
// http://www.eecs.harvard.edu/~kirsch/pubs/bbbf/esa06.pdf
@@ -110,9 +119,4 @@ public abstract class BloomFilter implements IFilter
{
bitset.clear();
}
-
-public void close()
-{
-bitset.close();
-}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/61384c57/src/java/org/apache/cassandra/utils/IFilter.java
--
diff --git a/src/java/org/apache/cassandra/utils/IFilter.java
b/src/java/org/apache/cassandra/utils/IFilter.java
index 60c0590..bde6333 100644
--- a/src/java/org/apache/cassandra/utils/IFilter.java
+++ b/src/java/org/apache/cassandra/utils/IFilter.java
@@ -17,10 +17,11 @@
*/
package org.apache.cassandra.utils;
-import java.io.Closeable;
import java.nio.ByteBuffer;
-public interface IFilter extends Closeable
+import org.apache.cassandra.utils.concurrent.SharedCloseable;
+
+public interface IFilter extends SharedCloseable
{
void add(ByteBuffer key);
@@ -32,6 +33,8 @@ public interface IFilter extends Closeable
void close();
+IFilter sharedCopy();
+
/**
* Returns the amount of memory in bytes used off heap.
* @return the amount of memory in bytes used off heap
http://git-wip-us.apache.org/repos/asf/cassandra/blob/61384c57/src/java/org/apache/cassandra/utils/Murmur3BloomFilter.java
--
diff --git a/src/java/org/apache/cassandra/utils/Murmur3BloomFilter.java
b/src/java/org/apache/cassandra/utils/Murmur3BloomFilter.java
index f7c7632..431ca5b 100644
--- a/src/java/org/apache/cassandra/utils/Murmur3BloomFilter.java
+++ b/src/java/org/apache/cassandra/utils/Murmur3BloomFilter.java
@@ -26,9 +26,14 @@ public class Murmur3BloomFilter extends BloomFilter
{
public static final Murmur3BloomFilterSerializer serializer = new
Murmur3BloomFilterSerializer();
-public Murmur3BloomFilter(int hashes, IBitSet bs)
+public Murmur3BloomFilter(int hashCount, IBitSet bs)
{
-super(hashes, bs);
+super(hashCount, bs);
+}
+
+protected Murmur3BloomFilter(Murmur3BloomFilter copy)
+{
+super(copy);
}
public long serializedSize()
@@ -36,6 +41,11 @@ public class Murmur3BloomFilter extends BloomFilter
return serializer.serializedSize(this, TypeSizes.NATIVE);
}
+public IFilter sharedCopy()
+{
+return new Murmur3BloomFilter(this);
+}
+
@Override
public long offHeapSize()
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/61384c57/src/java/org/apache/cassandra/utils/concurrent/Ref.java
--
diff --git a/src/java/org/apache/cassandra/utils/concurrent/Ref.java
b/src/java/org/apache/cassandra/utils/concurrent/Ref.java
index 4afceb0..ad1293b 100644
--- a/src/java/org/apache/cassandra/utils/concurrent/Ref.java
+++ b/src/java/org/apache/cassandra/utils/concurrent/Ref.java
@@ -2,24 +2,69 @@ package org.apache.cassandra.utils.concurrent;
import java.lang.ref.PhantomReference;
import java.lang.ref.ReferenceQueue;
+import java.util.Collections;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentLinkedQueue;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util
[7/7] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Conflicts: src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java src/java/org/apache/cassandra/io/util/BufferedSegmentedFile.java test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java test/unit/org/apache/cassandra/io/sstable/SSTableRewriterTest.java test/unit/org/apache/cassandra/utils/SerializationsTest.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/02c34893 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/02c34893 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/02c34893 Branch: refs/heads/trunk Commit: 02c348939a6c2ba391af5eb52d290b0838643cb4 Parents: 5a17ec8 61384c5 Author: Benedict Elliott Smith Authored: Wed Feb 11 15:23:17 2015 + Committer: Benedict Elliott Smith Committed: Wed Feb 11 15:23:17 2015 + -- CHANGES.txt | 1 + .../org/apache/cassandra/db/DataTracker.java| 6 +- .../SizeTieredCompactionStrategy.java | 10 +- .../io/compress/CompressionMetadata.java| 19 +- .../cassandra/io/sstable/IndexSummary.java | 47 +- .../io/sstable/IndexSummaryBuilder.java | 3 +- .../io/sstable/IndexSummaryManager.java | 6 +- .../io/sstable/SSTableDeletingTask.java | 36 +- .../cassandra/io/sstable/SSTableLoader.java | 4 +- .../cassandra/io/sstable/SSTableRewriter.java | 10 +- .../io/sstable/format/SSTableReader.java| 576 --- .../io/sstable/format/SSTableWriter.java| 18 +- .../io/sstable/format/big/BigTableWriter.java | 31 +- .../io/util/BufferedPoolingSegmentedFile.java | 12 +- .../io/util/BufferedSegmentedFile.java | 22 +- .../io/util/CompressedPoolingSegmentedFile.java | 29 +- .../io/util/CompressedSegmentedFile.java| 32 +- .../org/apache/cassandra/io/util/Memory.java| 8 +- .../cassandra/io/util/MmappedSegmentedFile.java | 55 +- .../cassandra/io/util/PoolingSegmentedFile.java | 35 +- .../apache/cassandra/io/util/SegmentedFile.java | 40 +- .../cassandra/service/ActiveRepairService.java | 4 +- .../cassandra/streaming/StreamSession.java | 2 +- .../streaming/messages/OutgoingFileMessage.java | 2 +- .../cassandra/tools/StandaloneScrubber.java | 2 +- .../cassandra/utils/AlwaysPresentFilter.java| 5 + .../org/apache/cassandra/utils/BloomFilter.java | 20 +- .../org/apache/cassandra/utils/IFilter.java | 7 +- .../cassandra/utils/Murmur3BloomFilter.java | 14 +- .../apache/cassandra/utils/concurrent/Ref.java | 208 ++- .../cassandra/utils/concurrent/RefCounted.java | 52 +- .../utils/concurrent/RefCountedImpl.java| 132 - .../apache/cassandra/utils/concurrent/Refs.java | 26 +- .../cassandra/utils/obs/OffHeapBitSet.java | 5 + .../cassandra/db/ColumnFamilyStoreTest.java | 5 +- .../db/compaction/AntiCompactionTest.java | 4 +- .../SizeTieredCompactionStrategyTest.java | 30 +- .../io/sstable/IndexSummaryManagerTest.java | 37 +- .../cassandra/io/sstable/LegacySSTableTest.java | 3 +- .../cassandra/io/sstable/SSTableReaderTest.java | 11 +- .../io/sstable/SSTableRewriterTest.java | 21 +- .../streaming/StreamTransferTaskTest.java | 2 +- .../cassandra/tools/SSTableExportTest.java | 1 + .../cassandra/tools/SSTableImportTest.java | 7 + .../apache/cassandra/utils/BloomFilterTest.java | 22 +- .../utils/concurrent/RefCountedTest.java| 12 +- 46 files changed, 986 insertions(+), 648 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/02c34893/CHANGES.txt -- diff --cc CHANGES.txt index f9faa6b,b323f18..c32a2c4 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,63 -1,5 +1,64 @@@ +3.0 + * Add role based access control (CASSANDRA-7653, 8650) + * Avoid accessing partitioner through StorageProxy (CASSANDRA-8244, 8268) + * Upgrade Metrics library and remove depricated metrics (CASSANDRA-5657) + * Serializing Row cache alternative, fully off heap (CASSANDRA-7438) + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707) + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560) + * Support direct buffer decompression for reads (CASSANDRA-8464) + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039) + * Group sstables for anticompaction correctly (CASSANDRA-8578) + * Add ReadFailureException to native protocol, respond + immediately when replicas encounter errors while handling + a read request (CASSANDRA-7886) + * Switch CommitL
