[jira] [Commented] (CASSANDRA-10216) Remove target type from internal index metadata
[
https://issues.apache.org/jira/browse/CASSANDRA-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14743413#comment-14743413
]
Sam Tunnicliffe commented on CASSANDRA-10216:
-
bq. My main remark is that for the new {{target}} option, I'd prefer keeping it
as close to the {{CREATE INDEX}} definition as possible. So if it's not a
collection index, I'd prefer the "target" to just be the column name instead of
defaulting to {{"values(name)"}}. My main reasoning is that it gives an ever
simpler match between the schema table and the corresponding statement. But it
also feels somewhat cleaner more future-proof to me (and "values" is plural,
which sounds weird for a non-collection column). And updating the regexp to
match that doesn't seem too hard.
Fair enough, though FTR it seemed cleaner & possibly more future-proof to be
explicit about the target type in all cases, rather than having a special case
for non-collection columns. Leaving aside the alignment with CQL this is also
what happens in the code, a statement like {{CREATE INDEX ON ks.t1(col)}}
produces an {{IndexTarget}} with type {{VALUES}}. Also, the plurality of
"values" doesn't seem particularly weird on a non-collection column to me, the
index is on the values of that column, of which there is only 1 per-row but
still many per-index.
I don't feel particularly strongly about it though, so I'll happily change it.
Just before I do though, you only mentioned non-collection columns. How do you
want to handle indexes on collection values? Do you want to make the value of
the {{target}} string match the CQL in those cases too, by omitting the
{{values}} for indexes on lists, sets & map values?
> Remove target type from internal index metadata
> ---
>
> Key: CASSANDRA-10216
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10216
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Labels: client-impacting
> Fix For: 3.0.0 rc1
>
>
> As part of CASSANDRA-6716 & in anticipation of CASSANDRA-10124, a distinction
> was introduced between secondary indexes which target a fixed set of 1 or
> more columns in the base data, and those which are agnostic to the structure
> of the underlying rows. This distinction is manifested in
> {{IndexMetadata.targetType}} and {{system_schema.indexes}}, in the
> {{target_type}} column. It could be argued that this distinction complicates
> the codebase without providing any tangible benefit, given that the target
> type is not actually used anywhere.
> It's only the impact on {{system_schema.indexes}} that makes puts this on the
> critical path for 3.0, any code changes are just implementation details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10216) Remove target type from internal index metadata
[
https://issues.apache.org/jira/browse/CASSANDRA-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14743792#comment-14743792
]
Sam Tunnicliffe commented on CASSANDRA-10216:
-
tl;dr the {{target}} stuff is changed in {{74a234d}}, (there's also a further
update to the python driver
[here|https://github.com/beobal/python-driver/commit/3e507497386f689440ebc0202b5cd95a11597dad]).
The other nits are addressed in {{cb8d43c}}.
bq.Hence why I want to match the user statement.
I totally get that, I was just thinking about the fact that the syntax itself
is somewhat inconsistent. The reason for that is completely understandable, I
was just wary of blindly reproducing it in the metadata. Also, I wasn't really
putting any weight on the current implementation of {{IndexTarget}}, just using
it to make the point that the specific syntax for 'regular' indexes is missing
some information which we have to infer. Anyway, I don't mean to labour the
point, I'm cool with going with your approach b/c of being able to recreate the
index without having to parse {{target}}.
bq.Well, since you mention it, I would have a slight preference for actually
using another "type" for that (REGULAR, NONE, SIMPLE, whatever).
wfm, I've added {{IndexTarget.Type.SIMPLE}}
bq.The fact that "values" is the default for collection is an historical
accident...but my preference would be to add support for CREATE INDEX ON
t(values(myCollection))
Sure, I'm aware that that's the history of the thing. Adding an explicit
version to the syntax is my preference too.
> Remove target type from internal index metadata
> ---
>
> Key: CASSANDRA-10216
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10216
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Labels: client-impacting
> Fix For: 3.0.0 rc1
>
>
> As part of CASSANDRA-6716 & in anticipation of CASSANDRA-10124, a distinction
> was introduced between secondary indexes which target a fixed set of 1 or
> more columns in the base data, and those which are agnostic to the structure
> of the underlying rows. This distinction is manifested in
> {{IndexMetadata.targetType}} and {{system_schema.indexes}}, in the
> {{target_type}} column. It could be argued that this distinction complicates
> the codebase without providing any tangible benefit, given that the target
> type is not actually used anywhere.
> It's only the impact on {{system_schema.indexes}} that makes puts this on the
> critical path for 3.0, any code changes are just implementation details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10338) Secondary index large values dtest failing on 3.0
Sam Tunnicliffe created CASSANDRA-10338:
---
Summary: Secondary index large values dtest failing on 3.0
Key: CASSANDRA-10338
URL: https://issues.apache.org/jira/browse/CASSANDRA-10338
Project: Cassandra
Issue Type: Sub-task
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Fix For: 3.0.0 rc1
{{secondary_index_test:TestSecondaryIndex.test_8280_validate_indexed_values}}
has been failing since build
[#147|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/147/].
The most likely cause is CASSANDRA-6237 (though I haven't confirmed that).
The problem is that if an oversized value is provided, the validation for LWT
statements (both regular and in batches) is not performed up front, but when
the {{Cql3CasRequest}} is executed via {{StorageProxy}}. This causes a timeout,
rather than an immediate validation error & hence the test fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10216) Remove target type from internal index metadata
[
https://issues.apache.org/jira/browse/CASSANDRA-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745380#comment-14745380
]
Sam Tunnicliffe commented on CASSANDRA-10216:
-
bq.We have a small problem with case-sensitive columns
Good point, I've pushed a further commit to deal with that by quoting such
column names in the {{target}} string.
> Remove target type from internal index metadata
> ---
>
> Key: CASSANDRA-10216
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10216
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Labels: client-impacting
> Fix For: 3.0.0 rc1
>
>
> As part of CASSANDRA-6716 & in anticipation of CASSANDRA-10124, a distinction
> was introduced between secondary indexes which target a fixed set of 1 or
> more columns in the base data, and those which are agnostic to the structure
> of the underlying rows. This distinction is manifested in
> {{IndexMetadata.targetType}} and {{system_schema.indexes}}, in the
> {{target_type}} column. It could be argued that this distinction complicates
> the codebase without providing any tangible benefit, given that the target
> type is not actually used anywhere.
> It's only the impact on {{system_schema.indexes}} that makes puts this on the
> critical path for 3.0, any code changes are just implementation details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10343) Erroneous partition deletion events are delivered to indexers
Sam Tunnicliffe created CASSANDRA-10343:
---
Summary: Erroneous partition deletion events are delivered to
indexers
Key: CASSANDRA-10343
URL: https://issues.apache.org/jira/browse/CASSANDRA-10343
Project: Cassandra
Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Fix For: 3.0.0 rc1
In {{AtomicBTreePartition#addAllWithSizeDelta}} we check
{{update.deletionInfo().isLive()}} to determine whether the
{{PartitionDeletion}} should be passed to the supplied {{IndexTransaction}}.
This is incorrect and it results in the {{PartitionDeletion}} being passed on
if *either* a partition delete or any range tombstones are present in the
update.
We need to change:
{code}
if (!update.deletionInfo().isLive())
{code}
to
{code}
if (!update.deletionInfo().getPartitionDeletion().isLive())
{code}
Note that the built-in indexes don't currently do anything with the
{{deletionTime}} and so aren't affected, but any custom implementation which
didn't examine it properly might remove data from its index where it shouldn't
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10343) Erroneous partition deletion events are delivered to indexers
[
https://issues.apache.org/jira/browse/CASSANDRA-10343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-10343:
Reviewer: Sylvain Lebresne
> Erroneous partition deletion events are delivered to indexers
> -
>
> Key: CASSANDRA-10343
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10343
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Fix For: 3.0.0 rc1
>
>
> In {{AtomicBTreePartition#addAllWithSizeDelta}} we check
> {{update.deletionInfo().isLive()}} to determine whether the
> {{PartitionDeletion}} should be passed to the supplied {{IndexTransaction}}.
> This is incorrect and it results in the {{PartitionDeletion}} being passed on
> if *either* a partition delete or any range tombstones are present in the
> update.
> We need to change:
> {code}
> if (!update.deletionInfo().isLive())
> {code}
> to
> {code}
> if (!update.deletionInfo().getPartitionDeletion().isLive())
> {code}
> Note that the built-in indexes don't currently do anything with the
> {{deletionTime}} and so aren't affected, but any custom implementation which
> didn't examine it properly might remove data from its index where it shouldn't
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10216) Remove target type from internal index metadata
[
https://issues.apache.org/jira/browse/CASSANDRA-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745721#comment-14745721
]
Sam Tunnicliffe commented on CASSANDRA-10216:
-
Thanks [~slebresne], I've pushed a commit which (hopefully) handles it properly.
> Remove target type from internal index metadata
> ---
>
> Key: CASSANDRA-10216
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10216
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Labels: client-impacting
> Fix For: 3.0.0 rc1
>
>
> As part of CASSANDRA-6716 & in anticipation of CASSANDRA-10124, a distinction
> was introduced between secondary indexes which target a fixed set of 1 or
> more columns in the base data, and those which are agnostic to the structure
> of the underlying rows. This distinction is manifested in
> {{IndexMetadata.targetType}} and {{system_schema.indexes}}, in the
> {{target_type}} column. It could be argued that this distinction complicates
> the codebase without providing any tangible benefit, given that the target
> type is not actually used anywhere.
> It's only the impact on {{system_schema.indexes}} that makes puts this on the
> critical path for 3.0, any code changes are just implementation details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9632) Preserve the Names of Query Parameters in QueryOptions
[
https://issues.apache.org/jira/browse/CASSANDRA-9632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745773#comment-14745773
]
Sam Tunnicliffe commented on CASSANDRA-9632:
+1 the new patch LGTM (sorry it took so long to review).
The only comment I'd make would be that you should probably rebase & push
2.2/3.0/trunk versions for CI before committing.
> Preserve the Names of Query Parameters in QueryOptions
> --
>
> Key: CASSANDRA-9632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9632
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: stephen mallette
>Assignee: Benjamin Lerer
>Priority: Minor
> Fix For: 2.2.x
>
> Attachments: 9632-2.2-V2.txt, 9632-2.2.txt
>
>
> When writing a custom {{QueryHandler}} that processes named parameters, the
> {{QueryOptions}} arrive to the various processing methods with the values
> converted to positional arguments and the names unavailable. A custom
> {{QueryHandler}} might want to make use of the names themselves so it would
> be helpful if they were preserved and exposed with their respective
> {{ByteBuffer}} values.
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/cql3/QueryOptions.java#L205
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10050) Secondary Index Performance Dependent on TokenRange Searched in Analytics
[
https://issues.apache.org/jira/browse/CASSANDRA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14747104#comment-14747104
]
Sam Tunnicliffe commented on CASSANDRA-10050:
-
bq.any thoughts on this?
yes, but no bandwidth to do anything with them before 3.0
tl;dr index entries are stored in key order, which is different from token
order so when paging using tokens, the index lookup can't efficiently slice
into the index table but has to scan and filter. I suspect this is related
> Secondary Index Performance Dependent on TokenRange Searched in Analytics
> -
>
> Key: CASSANDRA-10050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10050
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Single node, macbook, 2.1.8
>Reporter: Russell Alexander Spitzer
> Fix For: 3.x
>
>
> In doing some test work on the Spark Cassandra Connector I saw some odd
> performance when pushing down range queries with Secondary Index filters.
> When running the queries we see huge amount of time when the C* server is not
> doing any work and the query seem to be hanging. This investigation led to
> the work in this document
> https://docs.google.com/spreadsheets/d/1aJg3KX7nPnY77RJ9ZT-IfaYADgJh0A--nAxItvC6hb4/edit#gid=0
> The Spark Cassandra Connector builds up token range specific queries and
> allows the user to pushdown relevant fields to C*. Here we have two indexed
> fields (size) and (color) being pushed down to C*.
> {code}
> SELECT count(*) FROM ks.tab WHERE token("store") > $min AND token("store") <=
> $max AND color = 'red' AND size = 'P' ALLOW FILTERING;{code}
> These queries will have different token ranges inserted and executed as
> separate spark tasks. Spark tasks with token ranges near the Min(token) end
> up executing much faster than those near Max(token) which also happen to
> through errors.
> {code}
> Coordinator node timed out waiting for replica nodes' responses]
> message="Operation timed out - received only 0 responses."
> info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
> {code}
> I took the queries and ran them through CQLSH to see the difference in time.
> A linear relationship is seen based on where the tokenRange being queried is
> starting with only 2 second for queries near the beginning of the full token
> spectrum and over 12 seconds at the end of the spectrum.
> The question is, can this behavior be improved? or should we not recommend
> using secondary indexes with Analytics workloads?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10217) Support custom query expressions in SELECT
[
https://issues.apache.org/jira/browse/CASSANDRA-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14790958#comment-14790958
]
Sam Tunnicliffe commented on CASSANDRA-10217:
-
I've pushed a rebased version for review. There's some overlap in
{{CustomIndexTest}} with the branch up for review in CASSANDRA-10216. There
I've moved the test class from the {{org.apache.cassandra.index.internal}} to
{{org.apache.cassandra.index}}, which I haven't here (just FTR if anyone looks
at both).
Patches:
* [3.0 branch|https://github.com/beobal/cassandra/tree/10217-3.0]
* [trunk branch|https://github.com/beobal/cassandra/tree/10217-trunk]
CI Tests:
* [3.0
testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10217-3.0-testall/]
* [3.0
dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10217-trunk-dtest/]
* [trunk
testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10217-trunk-testall/]
* [trunk
dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10217-trunk-dtest/]
> Support custom query expressions in SELECT
> --
>
> Key: CASSANDRA-10217
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10217
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Fix For: 3.0.0 rc1
>
>
> (Broken out of CASSANDRA-10124)
> Custom index implementations often support query expressions which do not fit
> the structure of CQL. To support these, it has been necessary to add a fake
> column to the base table and query that using the custom syntax. Taking an
> example from the [Stratio
> docs|https://github.com/Stratio/cassandra-lucene-index]:
> {code}
> SELECT * FROM tweets WHERE lucene='{
> filter : {type:"range", field:"time", lower:"2014/04/25",
> upper:"2014/05/1"},
> query : {type:"phrase", field:"body", value:"big data gives
> organizations", slop:1}
> }'
> {code}
> The {{lucene}} field is a dummy column that has to be added to the table in
> order to associate the pre-3.0 row-based index with the {{tweets}} table. We
> could rewrite this query as:
> {code}
> SELECT * FROM tweets
> WHERE expr(lucene, '{filter : {type:"range", field:"time",
> lower:"2014/04/25", upper:"2014/05/1"},
> query : {type:"phrase", field:"body", value:"big data gives
> organizations", slop:1}}');
> {code}
> In this version the {{expr}} function takes 2 arguments: the first is the
> name of the index being targetted, {{lucene}} and the second is the query
> string itself.
> Parsing and validation of those expressions would be delegated to the custom
> index implementations which support them.
> One thing to consider is index selection. If a query contains custom
> expressions, but the target index is not selected, C* has no way to use the
> custom expressions as a post-query filter, like it does with standard
> expressions & {{ALLOW FILTERING}}. To compensate for that, index selection
> should be weighted in favour of indexes targetted by custom expressions. At
> least in the first instance, we should also restrict queries to targetting a
> single index via custom expressions, i.e. disallow queries like {{SELECT *
> FROM t WHERE expr(index1, 'foo') AND expr(index2, 'bar')}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9839) Move crc_check_chance out of compressions options
[ https://issues.apache.org/jira/browse/CASSANDRA-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-9839: --- Reviewer: Sam Tunnicliffe (was: Aleksey Yeschenko) > Move crc_check_chance out of compressions options > - > > Key: CASSANDRA-9839 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9839 > Project: Cassandra > Issue Type: Bug >Reporter: Aleksey Yeschenko >Assignee: Paulo Motta >Priority: Minor > Labels: client-impacting, docs-impacting > Fix For: 3.0.0 rc1 > > > Follow up to CASSANDRA-8384. The option doesn't belong to compression params > - it doesn't affect compression, itself, and isn't passed to compressors upon > initialization. > While it's true that it is (currently) only being honored when reading > compressed sstables, it still doesn't belong to compression params (and is > causing CASSANDRA-7978 -like issues). > [~tjake] suggested we should make it an option of its own, and I think we > should. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7313) SecondaryIndex not deleting column definition
[
https://issues.apache.org/jira/browse/CASSANDRA-7313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14089152#comment-14089152
]
Sam Tunnicliffe commented on CASSANDRA-7313:
I don't see a problem with the {{remove()}} method of the iterator obtained
from columnDefs.iterator, a quick test shows that it does in fact remove from
the underlying Set when invoked. My theory would be that the issue is actually
with the equality check between ByteBuffers.
Adding a unit test would let us verify whether or not the patch actually has
some effect (I suspect that it doesn't). If that isn't the case I would dig
into the name comparison.
(n.b. I'm assuming the affects version should be 2.1 here as the patch doesn't
apply to 2.0)
> SecondaryIndex not deleting column definition
> -
>
> Key: CASSANDRA-7313
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7313
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Andrés de la Peña
>Priority: Minor
> Labels: 2i, columnDefinition, secondaryIndex
> Attachments: dSecondaryIndex-removeColumnDef.diff.txt
>
>
> The method _SecondaryIndex#removeColumnDef_ does not delete the requested
> column definition. I think this may be due to the iterator's remove method:
> {code:java}
> void removeColumnDef(ByteBuffer name)
> {
> Iterator it = columnDefs.iterator();
> while (it.hasNext())
> {
> if (it.next().name.bytes.equals(name))
> it.remove();
> }
> }
> {code}
> For me it has been problematic when deleting a per row index in a column
> family and then creating a new one.
> I attach a patch that may solve the problem.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6612) Query failing due to AssertionError
[
https://issues.apache.org/jira/browse/CASSANDRA-6612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14089324#comment-14089324
]
Sam Tunnicliffe commented on CASSANDRA-6612:
I agree that SIM is in dire need of an overhaul and because of that this is a
slightly hackish solution. That aside, the patch looks good to me. Just one
thing to note, the bogus logic in SIM.hasIndexFor is already fixed in 2.1 by
CASSANDRA-7525
> Query failing due to AssertionError
> ---
>
> Key: CASSANDRA-6612
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6612
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra-2.0.4, CQL3, datastax driver 2.0.0-rc2
>Reporter: A Verma
>Assignee: Sylvain Lebresne
> Fix For: 2.0.10
>
> Attachments: 6612.txt
>
>
> I am trying out Cassandra for the first time and running it locally for
> simple session management db. [Cassandra-2.0.4, CQL3, datastax driver
> 2.0.0-rc2]
> The following count query works fine when there is no data in the table:
> {code}
> select count(*) from session_data where app_name=? and account=? and
> last_access > ?
> {code}
> But after even a single row is inserted into the table, the query fails with
> the following error:
> {code}
> java.lang.AssertionError
> at
> org.apache.cassandra.db.filter.ExtendedFilter$WithClauses.getExtraFilter(ExtendedFilter.java:258)
> at
> org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1719)
> at
> org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1674)
> at
> org.apache.cassandra.db.PagedRangeCommand.executeLocally(PagedRangeCommand.java:111)
> at
> org.apache.cassandra.service.StorageProxy$LocalRangeSliceRunnable.runMayThrow(StorageProxy.java:1418)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1931)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> Here is the schema I am using:
> {code}
> CREATE KEYSPACE session WITH replication= {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> CREATE TABLE session_data (
> username text,
> session_id text,
> app_name text,
> account text,
> last_access timestamp,
> created_on timestamp,
> PRIMARY KEY (username, session_id, app_name, account)
> );
> create index sessionIndex ON session_data (session_id);
> create index sessionAppName ON session_data (app_name);
> create index lastAccessIndex ON session_data (last_access);
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7742) Mutated ColumnFamily is not available to per-row indexes during update
Sam Tunnicliffe created CASSANDRA-7742: -- Summary: Mutated ColumnFamily is not available to per-row indexes during update Key: CASSANDRA-7742 URL: https://issues.apache.org/jira/browse/CASSANDRA-7742 Project: Cassandra Issue Type: Bug Reporter: Sam Tunnicliffe Assignee: Sam Tunnicliffe PerRowSecondaryIndex implementations are updated when SecondaryIndexManager.Updater.updateRowLevelIndexes is called from AtomicBTreeColumns.addAllWithSizeDelta. The StandardUpdater implementation does this by passing the key and cf from the incoming Mutation to the PRSI.index() method. However, the StandardUpdater instance created in Keyspace.apply() does not wrap the Mutation's cf as it is obtained using the wrong factory method. This results in the cf passed to the PRSI always being null. This isn't caught by PerRowSecondaryIndexTest because the stub index implementation there only verifies that the latest version of the cf can be obtained by the index reading from the base cf. Attaching a trivial patch to obtain an Updater which does wrap the mutated cf and also fix the test so it verifies both of the actual arguments passed to PRSI.index -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6696) Partition sstables by token range
[ https://issues.apache.org/jira/browse/CASSANDRA-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-6696: --- Attachment: 7742.txt > Partition sstables by token range > - > > Key: CASSANDRA-6696 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6696 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > In JBOD, when someone gets a bad drive, the bad drive is replaced with a new > empty one and repair is run. > This can cause deleted data to come back in some cases. Also this is true for > corrupt stables in which we delete the corrupt stable and run repair. > Here is an example: > Say we have 3 nodes A,B and C and RF=3 and GC grace=10days. > row=sankalp col=sankalp is written 20 days back and successfully went to all > three nodes. > Then a delete/tombstone was written successfully for the same row column 15 > days back. > Since this tombstone is more than gc grace, it got compacted in Nodes A and B > since it got compacted with the actual data. So there is no trace of this row > column in node A and B. > Now in node C, say the original data is in drive1 and tombstone is in drive2. > Compaction has not yet reclaimed the data and tombstone. > Drive2 becomes corrupt and was replaced with new empty drive. > Due to the replacement, the tombstone in now gone and row=sankalp col=sankalp > has come back to life. > Now after replacing the drive we run repair. This data will be propagated to > all nodes. > Note: This is still a problem even if we run repair every gc grace. > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6696) Partition sstables by token range
[ https://issues.apache.org/jira/browse/CASSANDRA-6696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-6696: --- Attachment: (was: 7742.txt) > Partition sstables by token range > - > > Key: CASSANDRA-6696 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6696 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > In JBOD, when someone gets a bad drive, the bad drive is replaced with a new > empty one and repair is run. > This can cause deleted data to come back in some cases. Also this is true for > corrupt stables in which we delete the corrupt stable and run repair. > Here is an example: > Say we have 3 nodes A,B and C and RF=3 and GC grace=10days. > row=sankalp col=sankalp is written 20 days back and successfully went to all > three nodes. > Then a delete/tombstone was written successfully for the same row column 15 > days back. > Since this tombstone is more than gc grace, it got compacted in Nodes A and B > since it got compacted with the actual data. So there is no trace of this row > column in node A and B. > Now in node C, say the original data is in drive1 and tombstone is in drive2. > Compaction has not yet reclaimed the data and tombstone. > Drive2 becomes corrupt and was replaced with new empty drive. > Due to the replacement, the tombstone in now gone and row=sankalp col=sankalp > has come back to life. > Now after replacing the drive we run repair. This data will be propagated to > all nodes. > Note: This is still a problem even if we run repair every gc grace. > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7742) Mutated ColumnFamily is not available to per-row indexes during update
[ https://issues.apache.org/jira/browse/CASSANDRA-7742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-7742: --- Attachment: 7742.txt > Mutated ColumnFamily is not available to per-row indexes during update > -- > > Key: CASSANDRA-7742 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7742 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe > Attachments: 7742.txt > > > PerRowSecondaryIndex implementations are updated when > SecondaryIndexManager.Updater.updateRowLevelIndexes is called from > AtomicBTreeColumns.addAllWithSizeDelta. The StandardUpdater implementation > does this by passing the key and cf from the incoming Mutation to the > PRSI.index() method. However, the StandardUpdater instance created in > Keyspace.apply() does not wrap the Mutation's cf as it is obtained using the > wrong factory method. This results in the cf passed to the PRSI always being > null. > This isn't caught by PerRowSecondaryIndexTest because the stub index > implementation there only verifies that the latest version of the cf can be > obtained by the index reading from the base cf. > Attaching a trivial patch to obtain an Updater which does wrap the mutated cf > and also fix the test so it verifies both of the actual arguments passed to > PRSI.index -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7742) Mutated ColumnFamily is not available to per-row indexes during update
[ https://issues.apache.org/jira/browse/CASSANDRA-7742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-7742: --- Fix Version/s: 2.1.0 > Mutated ColumnFamily is not available to per-row indexes during update > -- > > Key: CASSANDRA-7742 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7742 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe > Fix For: 2.1.0 > > Attachments: 7742.txt > > > PerRowSecondaryIndex implementations are updated when > SecondaryIndexManager.Updater.updateRowLevelIndexes is called from > AtomicBTreeColumns.addAllWithSizeDelta. The StandardUpdater implementation > does this by passing the key and cf from the incoming Mutation to the > PRSI.index() method. However, the StandardUpdater instance created in > Keyspace.apply() does not wrap the Mutation's cf as it is obtained using the > wrong factory method. This results in the cf passed to the PRSI always being > null. > This isn't caught by PerRowSecondaryIndexTest because the stub index > implementation there only verifies that the latest version of the cf can be > obtained by the index reading from the base cf. > Attaching a trivial patch to obtain an Updater which does wrap the mutated cf > and also fix the test so it verifies both of the actual arguments passed to > PRSI.index -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Reopened] (CASSANDRA-7726) Give CRR a default input_cql Statement
[
https://issues.apache.org/jira/browse/CASSANDRA-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe reopened CASSANDRA-7726:
Reproduced In: 2.1 rc5, 2.0.10 (was: 2.0.10, 2.1 rc5)
Reopening as we found further issues during testing
> Give CRR a default input_cql Statement
> --
>
> Key: CASSANDRA-7726
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7726
> Project: Cassandra
> Issue Type: Improvement
> Components: Hadoop
>Reporter: Russell Alexander Spitzer
>Assignee: Mike Adamson
> Fix For: 2.0.10, 2.1.0
>
> Attachments: 7726.txt
>
>
> Inorder to ease migration from CqlPagingRecordReader to CqlRecordReader, it
> would be helpful if CRR input_cql defaulted to a select statement that would
> mirror the behavior of CPRR.
> For example for a give table with partition key `((x,y,z),c1,c2)`
> It would automatically generate
> {code}
> input_cql = SELECT * FROM ks.tab WHERE token(x,y,z) > ? AND token (x,y,z) <=
> ?
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7741) Handle zero index searchers in StorageProxy#estimateResultRowsPerRange()
[ https://issues.apache.org/jira/browse/CASSANDRA-7741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-7741: --- Attachment: 7741.txt Attached patch falls back to estimating result rows per range based on CFS.estimateKeys() for reads with filters on with non-indexed columns > Handle zero index searchers in StorageProxy#estimateResultRowsPerRange() > > > Key: CASSANDRA-7741 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7741 > Project: Cassandra > Issue Type: Bug >Reporter: Aleksey Yeschenko >Assignee: Sam Tunnicliffe > Fix For: 2.1 rc6 > > Attachments: 7741.txt > > > CASSANDRA-7525 has broken Thrift's ability to filter based on arbitrary > columns, even those without a secondary index defined. > Two of the thrift tests are broken because of this: > - > https://github.com/apache/cassandra/blob/cassandra-2.1.0/test/system/test_thrift_server.py#L1982 > - > https://github.com/apache/cassandra/blob/cassandra-2.1.0/test/system/test_thrift_server.py#L1605 > Both trigger this assert: > https://github.com/apache/cassandra/blob/cassandra-2.1.0/src/java/org/apache/cassandra/service/StorageProxy.java#L1457 > This is not a path reachable via CQL3 yet (until/if we further extend ALLOW > FILTERING power), but it can be legally reached via Thrift, and should handle > the possibility. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (CASSANDRA-7765) Don't use strict endpoint selection for range movements where RF == node count
[
https://issues.apache.org/jira/browse/CASSANDRA-7765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe reassigned CASSANDRA-7765:
--
Assignee: Sam Tunnicliffe
> Don't use strict endpoint selection for range movements where RF == node count
> --
>
> Key: CASSANDRA-7765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7765
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Attachments: 7765.txt
>
>
> Post CASSANDRA-2434, we should skip strict endpoint selection for ranges
> where RF == the number of nodes in the cluster otherwise we hit the following
> assertion in {{RangeStreamer.getAllRangesWithStrictSourcesFor}}
> {code}
> if (oldEndpoints.size() == strat.getReplicationFactor())
> {
> oldEndpoints.removeAll(newEndpoints);
> assert oldEndpoints.size() == 1 : "Expected 1 endpoint but found " +
> oldEndpoints.size();
> }
> {code}
> I think this is a fairly niche case, plus there's a workaround in the
> {{-Dcassandra.consistent.rangemovement=false}} option.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7765) Don't use strict endpoint selection for range movements where RF == node count
[
https://issues.apache.org/jira/browse/CASSANDRA-7765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-7765:
---
Attachment: 7765.txt
> Don't use strict endpoint selection for range movements where RF == node count
> --
>
> Key: CASSANDRA-7765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7765
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Priority: Minor
> Attachments: 7765.txt
>
>
> Post CASSANDRA-2434, we should skip strict endpoint selection for ranges
> where RF == the number of nodes in the cluster otherwise we hit the following
> assertion in {{RangeStreamer.getAllRangesWithStrictSourcesFor}}
> {code}
> if (oldEndpoints.size() == strat.getReplicationFactor())
> {
> oldEndpoints.removeAll(newEndpoints);
> assert oldEndpoints.size() == 1 : "Expected 1 endpoint but found " +
> oldEndpoints.size();
> }
> {code}
> I think this is a fairly niche case, plus there's a workaround in the
> {{-Dcassandra.consistent.rangemovement=false}} option.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7765) Don't use strict endpoint selection for range movements where RF == node count
[
https://issues.apache.org/jira/browse/CASSANDRA-7765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-7765:
---
Reviewer: T Jake Luciani
> Don't use strict endpoint selection for range movements where RF == node count
> --
>
> Key: CASSANDRA-7765
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7765
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Attachments: 7765.txt
>
>
> Post CASSANDRA-2434, we should skip strict endpoint selection for ranges
> where RF == the number of nodes in the cluster otherwise we hit the following
> assertion in {{RangeStreamer.getAllRangesWithStrictSourcesFor}}
> {code}
> if (oldEndpoints.size() == strat.getReplicationFactor())
> {
> oldEndpoints.removeAll(newEndpoints);
> assert oldEndpoints.size() == 1 : "Expected 1 endpoint but found " +
> oldEndpoints.size();
> }
> {code}
> I think this is a fairly niche case, plus there's a workaround in the
> {{-Dcassandra.consistent.rangemovement=false}} option.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7765) Don't use strict endpoint selection for range movements where RF == node count
Sam Tunnicliffe created CASSANDRA-7765:
--
Summary: Don't use strict endpoint selection for range movements
where RF == node count
Key: CASSANDRA-7765
URL: https://issues.apache.org/jira/browse/CASSANDRA-7765
Project: Cassandra
Issue Type: Bug
Reporter: Sam Tunnicliffe
Priority: Minor
Attachments: 7765.txt
Post CASSANDRA-2434, we should skip strict endpoint selection for ranges where
RF == the number of nodes in the cluster otherwise we hit the following
assertion in {{RangeStreamer.getAllRangesWithStrictSourcesFor}}
{code}
if (oldEndpoints.size() == strat.getReplicationFactor())
{
oldEndpoints.removeAll(newEndpoints);
assert oldEndpoints.size() == 1 : "Expected 1 endpoint but found " +
oldEndpoints.size();
}
{code}
I think this is a fairly niche case, plus there's a workaround in the
{{-Dcassandra.consistent.rangemovement=false}} option.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-8653) Upgrading to trunk with auth throws exception
[
https://issues.apache.org/jira/browse/CASSANDRA-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971286#comment-14971286
]
Sam Tunnicliffe commented on CASSANDRA-8653:
b.q This ticket mentions trunk, but any reason to think 3.0 is immune to that?
It's actually the 3.0 that the test upgrades to, or it would be except it's
currently skipped (waiting on CASSANDRA-9704, so should be re-enabled). When I
run locally I see no auth problems and the test completes as expected. It fails
though because of an unexpected ERROR in the log of node1, which is thrown just
after the last node is upgraded:
{noformat}
RROR [HintsDispatcher:2] 2015-10-23 17:18:53,942 CassandraDaemon.java:195 -
Exception in thread Thread[HintsDispatcher:2,1,main]
java.lang.RuntimeException: java.nio.file.NoSuchFileException:
/home/sam/.ccm/repository/gitCOLONcassandra-3.0/data/hints/ac459445-1f7f-45f2-b9a8-2b185df34845-1445617063586-1.hints
at
org.apache.cassandra.io.util.ChannelProxy.openChannel(ChannelProxy.java:55)
~[main/:na]
at org.apache.cassandra.io.util.ChannelProxy.(ChannelProxy.java:66)
~[main/:na]
at
org.apache.cassandra.hints.ChecksummedDataInput.open(ChecksummedDataInput.java:63)
~[main/:na]
at org.apache.cassandra.hints.HintsReader.open(HintsReader.java:77)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatcher.create(HintsDispatcher.java:71)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:219)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:198)
~[main/:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_60]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
Caused by: java.nio.file.NoSuchFileException:
/home/sam/.ccm/repository/gitCOLONcassandra-3.0/data/hints/ac459445-1f7f-45f2-b9a8-2b185df34845-1445617063586-1.hints
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
~[na:1.8.0_60]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
~[na:1.8.0_60]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
~[na:1.8.0_60]
at
sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
~[na:1.8.0_60]
at java.nio.channels.FileChannel.open(FileChannel.java:287) ~[na:1.8.0_60]
at java.nio.channels.FileChannel.open(FileChannel.java:335) ~[na:1.8.0_60]
at
org.apache.cassandra.io.util.ChannelProxy.openChannel(ChannelProxy.java:51)
~[main/:na]
... 12 common frames omitted
{noformat}
> Upgrading to trunk with auth throws exception
> -
>
> Key: CASSANDRA-8653
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8653
> Project: Cassandra
> Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Sam Tunnicliffe
> Fix For: 3.0.0
>
> Attachments: node1.log, node2.log, node3.log
>
>
> When running Sam's upgrade_internal_auth_dtest, I am seeing the following
> exception (amongst others) in the log file of the second node to be upgraded
> to trunk from 2.1:
> {code}
> ERROR [GossipStage:1] 2015-01-20 13:46:21,679 CassandraDaemon.java:170 -
> Exception in thread Thread[GossipStage:1,5,main]
> java.lang.NoClassDefFoundError:
> org/apache/cassandra/transport/Event$TopologyChange$Change
> at
> org.apache.cassandra.transport.Server$EventNotifier.onJoinCluster(Server.java:374)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1668)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1384)
> ~[main/:na]
> at
> org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1094)
> ~[main/:na]
> at
> org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:1076)
> ~[main/:na]
> at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1034)
> ~[main/:na]
> at
> org.apache.cassandra.gms.GossipDigestAckVerbHandler.doVerb(GossipDigestAckVerbHandler.java:58)
> ~[main/:na]
> 1554 - Node /127.0.0.1 state jump to normal
> ERROR [GossipStage:1] 2015-01-20 13:46:21,679 CassandraDaemon.java
> :170 - Exception in thread Thread[GossipStage:1,5,main]
> java.lang.NoClassDefFoundErr
[jira] [Comment Edited] (CASSANDRA-8653) Upgrading to trunk with auth throws exception
[
https://issues.apache.org/jira/browse/CASSANDRA-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971286#comment-14971286
]
Sam Tunnicliffe edited comment on CASSANDRA-8653 at 10/23/15 4:23 PM:
--
bq. This ticket mentions trunk, but any reason to think 3.0 is immune to that?
It's actually the 3.0 that the test upgrades to, or it would be except it's
currently skipped (waiting on CASSANDRA-9704, so should be re-enabled). When I
run locally I see no auth problems and the test completes as expected. It fails
though because of an unexpected ERROR in the log of node1, which is thrown just
after the last node is upgraded:
{noformat}
ERROR [HintsDispatcher:2] 2015-10-23 17:18:53,942 CassandraDaemon.java:195 -
Exception in thread Thread[HintsDispatcher:2,1,main]
java.lang.RuntimeException: java.nio.file.NoSuchFileException:
/home/sam/.ccm/repository/gitCOLONcassandra-3.0/data/hints/ac459445-1f7f-45f2-b9a8-2b185df34845-1445617063586-1.hints
at
org.apache.cassandra.io.util.ChannelProxy.openChannel(ChannelProxy.java:55)
~[main/:na]
at org.apache.cassandra.io.util.ChannelProxy.(ChannelProxy.java:66)
~[main/:na]
at
org.apache.cassandra.hints.ChecksummedDataInput.open(ChecksummedDataInput.java:63)
~[main/:na]
at org.apache.cassandra.hints.HintsReader.open(HintsReader.java:77)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatcher.create(HintsDispatcher.java:71)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:219)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:198)
~[main/:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_60]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
Caused by: java.nio.file.NoSuchFileException:
/home/sam/.ccm/repository/gitCOLONcassandra-3.0/data/hints/ac459445-1f7f-45f2-b9a8-2b185df34845-1445617063586-1.hints
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
~[na:1.8.0_60]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
~[na:1.8.0_60]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
~[na:1.8.0_60]
at
sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
~[na:1.8.0_60]
at java.nio.channels.FileChannel.open(FileChannel.java:287) ~[na:1.8.0_60]
at java.nio.channels.FileChannel.open(FileChannel.java:335) ~[na:1.8.0_60]
at
org.apache.cassandra.io.util.ChannelProxy.openChannel(ChannelProxy.java:51)
~[main/:na]
... 12 common frames omitted
{noformat}
was (Author: beobal):
b.q This ticket mentions trunk, but any reason to think 3.0 is immune to that?
It's actually the 3.0 that the test upgrades to, or it would be except it's
currently skipped (waiting on CASSANDRA-9704, so should be re-enabled). When I
run locally I see no auth problems and the test completes as expected. It fails
though because of an unexpected ERROR in the log of node1, which is thrown just
after the last node is upgraded:
{noformat}
RROR [HintsDispatcher:2] 2015-10-23 17:18:53,942 CassandraDaemon.java:195 -
Exception in thread Thread[HintsDispatcher:2,1,main]
java.lang.RuntimeException: java.nio.file.NoSuchFileException:
/home/sam/.ccm/repository/gitCOLONcassandra-3.0/data/hints/ac459445-1f7f-45f2-b9a8-2b185df34845-1445617063586-1.hints
at
org.apache.cassandra.io.util.ChannelProxy.openChannel(ChannelProxy.java:55)
~[main/:na]
at org.apache.cassandra.io.util.ChannelProxy.(ChannelProxy.java:66)
~[main/:na]
at
org.apache.cassandra.hints.ChecksummedDataInput.open(ChecksummedDataInput.java:63)
~[main/:na]
at org.apache.cassandra.hints.HintsReader.open(HintsReader.java:77)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatcher.create(HintsDispatcher.java:71)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:242)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.dispatch(HintsDispatchExecutor.java:219)
~[main/:na]
at
org.apache.cassandra.hints.HintsDispatchExecutor$DispatchHintsTask.run(HintsDispatchExecutor.java:198)
~[main/:na]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_60]
at java.util.concurrent.FutureTask.run(FutureTask.java:2
[jira] [Commented] (CASSANDRA-10092) Generalize PerRowSecondaryIndex validation
[
https://issues.apache.org/jira/browse/CASSANDRA-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14974081#comment-14974081
]
Sam Tunnicliffe commented on CASSANDRA-10092:
-
Thanks for the updated patch, though I'm afraid there are a couple of issues
with it.
The validation in {{CassandraServer.createMutationList}} isn't really doing
what it's supposed to. It simply creates an empty columnfamily and passes that
to the validation method. This case isn't actually covered by the new tests
(but it would be good to do so), but even if it were, the implementation of the
new method in the test index impl would let it pass as the validation is based
solely on the key. So if you could change that validation & extend the test to
exercise each of the places where it's called (i.e. regular, batch and CAS
mutations), that'd be good.
The patch also doesn't merge cleanly to 2.2; there are some trivial merge
conflicts, but once those are resolved the several of the tests in
{{PerRowSecondaryIndexTest}} error. I suspect this is something to do with the
way {{testInvalidRowInsertThrift}} sets up {{Embedded CassandraService}}, you
might want to check out the other tests which use that without errors & verify
that they're equivalent.
> Generalize PerRowSecondaryIndex validation
> --
>
> Key: CASSANDRA-10092
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10092
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Andrés de la Peña
>Assignee: Andrés de la Peña
>Priority: Minor
> Labels: 2i, secondary_index, validation
> Fix For: 2.1.x, 2.2.x
>
> Attachments: CASSANDRA-10092_v2.patch, improve_2i_validation.patch
>
>
> Index validation is currently done in a per-cell basis. However, per-row
> secondary index developers can be interested in validating all the written
> columns at once, because some implementations need to check the validity of a
> row write by comparing some column values against others. For example, a per
> row 2i implementation indexing time ranges (composed by a start date column
> and an end date column) should check that the start date is before the stop
> date.
> I'm attaching a patch adding a new method to {{PerRowSecondaryIndex}}:
> {code:java}
> public void validate(ByteBuffer key, ColumnFamily cf) throws
> InvalidRequestException {}
> {code}
> and a new method to {{SecondaryIndexManager}}:
> {code:java}
> public void validateRowLevelIndexes(ByteBuffer key, ColumnFamily cf) throws
> InvalidRequestException
> {
> for (SecondaryIndex index : rowLevelIndexMap.values())
> {
> ((PerRowSecondaryIndex) index).validate(key, cf);
> }
> }
> {code}
> This method is invoked in CQL {{UpdateStatement#validateIndexedColumns}}.
> This way, {{PerRowSecondaryIndex}} could perform complex write validation.
> I have tried to do the patch in the least invasive way possible. Maybe the
> current method {{SecondaryIndex#validate(ByteBuffer, Cell)}} should be moved
> to {{PerColumnSecondaryIndex}}, and the {{InvalidRequestException}} that
> informs about the particular 64k limitation should be thrown by
> {{AbstractSimplePerColumnSecondaryIndex}}. However, given the incoming
> [CASSANDRA-9459|https://issues.apache.org/jira/browse/CASSANDRA-9459], I
> think that the proposed patch is more than enough to provide rich validation
> features to 2i implementations based on 2.1.x and 2.2.x.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10595) Don't initialize un-registered indexes
Sam Tunnicliffe created CASSANDRA-10595:
---
Summary: Don't initialize un-registered indexes
Key: CASSANDRA-10595
URL: https://issues.apache.org/jira/browse/CASSANDRA-10595
Project: Cassandra
Issue Type: Improvement
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Priority: Minor
Fix For: 3.0.0
If a secondary index implementation chooses not to register with
{{SecondaryIndexManager}}on a particular node, it won't be required to provide
either {{Indexer}} or {{Searcher}} instances. In this case, initialization is
unnecessary so we should avoid doing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10595) Don't initialize un-registered indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-10595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-10595:
Description: If a secondary index implementation chooses not to register
with {{SecondaryIndexManager}} on a particular node, it won't be required to
provide either {{Indexer}} or {{Searcher}} instances. In this case,
initialization is unnecessary so we should avoid doing it. (was: If a
secondary index implementation chooses not to register with
{{SecondaryIndexManager}}on a particular node, it won't be required to provide
either {{Indexer}} or {{Searcher}} instances. In this case, initialization is
unnecessary so we should avoid doing.)
> Don't initialize un-registered indexes
> --
>
> Key: CASSANDRA-10595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10595
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 3.0.0
>
>
> If a secondary index implementation chooses not to register with
> {{SecondaryIndexManager}} on a particular node, it won't be required to
> provide either {{Indexer}} or {{Searcher}} instances. In this case,
> initialization is unnecessary so we should avoid doing it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10595) Don't initialize un-registered indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-10595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14974868#comment-14974868
]
Sam Tunnicliffe commented on CASSANDRA-10595:
-
You're right, and actually {{Index::getIndexName}} was only supposed to be a
transitional thing while we fixed some inconsistencies in how we referred to
instances internally. That was done in CASSANDRA-10127, so I've pushed a second
commit to remove {{Index::getIndexName}} completely.
> Don't initialize un-registered indexes
> --
>
> Key: CASSANDRA-10595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10595
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 3.0.0
>
>
> If a secondary index implementation chooses not to register with
> {{SecondaryIndexManager}} on a particular node, it won't be required to
> provide either {{Indexer}} or {{Searcher}} instances. In this case,
> initialization is unnecessary so we should avoid doing it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10601) SecondaryIndexManager incorrectly updates index build status system table
Sam Tunnicliffe created CASSANDRA-10601:
---
Summary: SecondaryIndexManager incorrectly updates index build
status system table
Key: CASSANDRA-10601
URL: https://issues.apache.org/jira/browse/CASSANDRA-10601
Project: Cassandra
Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Fix For: 3.0.0
The {{markIndexBuilt}} and {{markIndexRemoved}} methods on
{{SecondaryIndexManager}} incorrectly supply the table name to the methods on
{{SystemKeyspace}} which update the underlying system table. They should pass
the keyspace name instead, which results in incorrect rows being written
to/removed from the system table when a rebuild of the index is performed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8068) Allow to create authenticator which is aware of the client connection
[
https://issues.apache.org/jira/browse/CASSANDRA-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976655#comment-14976655
]
Sam Tunnicliffe commented on CASSANDRA-8068:
Would the {{InetAddress}} of the client attempting authentication be sufficient
for the negotiator? QueryState & ClientState both also provide access to a
bunch of things that an IAuthenticator probably has no business with: the
tracing-related methods, login, authz functions etc.
> Allow to create authenticator which is aware of the client connection
> -
>
> Key: CASSANDRA-8068
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8068
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
>Reporter: Jacek Lewandowski
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Labels: security
> Fix For: 3.0.0
>
>
> Currently, the authenticator interface doesn't allow to make a decision
> according to the client connection properties (especially the client host
> name or address).
> The idea is to add the interface which extends the current SASL aware
> authenticator interface with additional method to set the client connection.
> ServerConnection then could supply the connection to the authenticator if the
> authenticator implements that interface.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8955) Allow modifying quasi-system tables (system_auth.*, system_traces.*)
[ https://issues.apache.org/jira/browse/CASSANDRA-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-8955: --- Assignee: (was: Sam Tunnicliffe) > Allow modifying quasi-system tables (system_auth.*, system_traces.*) > > > Key: CASSANDRA-8955 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8955 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko > Fix For: 3.x > > > Currently there is no way to add a column to any of those tables, because we > do not announce the tables if they already exist. > For tables in those keyspaces we should also announce if the newly complied > metadata differs from the one currently loaded, if it exists. > See CASSANDRA-8162. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9161) Add random interleaving for flush/compaction when running CQL unit tests
[
https://issues.apache.org/jira/browse/CASSANDRA-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-9161:
---
Assignee: (was: Sam Tunnicliffe)
> Add random interleaving for flush/compaction when running CQL unit tests
>
>
> Key: CASSANDRA-9161
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9161
> Project: Cassandra
> Issue Type: Test
>Reporter: Sylvain Lebresne
> Labels: retrospective_generated
>
> Most CQL tests don't bother flushing, which means that they overwhelmingly
> test the memtable path and not the sstables one. A simple way to improve on
> that would be to make {{CQLTester}} issue flushes and compactions randomly
> between statements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7232) Enable live replay of commit logs
[ https://issues.apache.org/jira/browse/CASSANDRA-7232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-7232: --- Assignee: (was: Sam Tunnicliffe) > Enable live replay of commit logs > - > > Key: CASSANDRA-7232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7232 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Patrick McFadin >Priority: Minor > Fix For: 2.1.x > > Attachments: > 0001-Expose-CommitLog-recover-to-JMX-add-nodetool-cmd-for.patch, > 0001-TRUNK-JMX-and-nodetool-cmd-for-commitlog-replay.patch > > > Replaying commit logs takes a restart but restoring sstables can be an online > operation with refresh. In order to restore a point-in-time without a > restart, the node needs to live replay the commit logs from JMX and a > nodetool command. > nodetool refreshcommitlogs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7666) Range-segmented sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-7666: --- Assignee: (was: Sam Tunnicliffe) > Range-segmented sstables > > > Key: CASSANDRA-7666 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7666 > Project: Cassandra > Issue Type: New Feature > Components: API, Core >Reporter: Jonathan Ellis > Labels: dense-storage > Fix For: 3.x > > > It would be useful to segment sstables by data range (not just token range as > envisioned by CASSANDRA-6696). > The primary use case is to allow deleting those data ranges for "free" by > dropping the sstables involved. We should also (possibly as a separate > ticket) be able to leverage this information in query planning to avoid > unnecessary sstable reads. > Relational databases typically call this "partitioning" the table, but > obviously we use that term already for something else: > http://www.postgresql.org/docs/9.1/static/ddl-partitioning.html > Tokutek's take for mongodb: > http://docs.tokutek.com/tokumx/tokumx-partitioned-collections.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10179) Duplicate index should throw AlreadyExistsException
[ https://issues.apache.org/jira/browse/CASSANDRA-10179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-10179: Assignee: (was: Sam Tunnicliffe) > Duplicate index should throw AlreadyExistsException > --- > > Key: CASSANDRA-10179 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10179 > Project: Cassandra > Issue Type: Sub-task >Reporter: T Jake Luciani >Priority: Minor > Fix For: 3.x > > > If a 2i already exists we currently throw a InvalidQueryException. This > should be a AlreadyExistsException for consistency like trying to create the > same CQL Table twice. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10604) Secondary index metadata is not reloaded when table is altered
[
https://issues.apache.org/jira/browse/CASSANDRA-10604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-10604:
Reviewer: Tyler Hobbs
> Secondary index metadata is not reloaded when table is altered
> --
>
> Key: CASSANDRA-10604
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10604
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Sam Tunnicliffe
> Fix For: 3.x
>
>
> The javadocs for {{Index.getMetadataReloadTask()}} state the following:
> {quote}
> Returns a task to reload the internal metadata of an index.
> Called when the base table metadata is modified or when the configuration of
> the Index is updated.
> {quote}
> However, altering a table does not result in the reload task being executed.
> I think the root of the problem is that in
> {{SecondaryIndexManager.reloadIndex()}}, we only execute the reload task when
> the old {{IndexMetadata}} does not equal the current {{IndexMetadata}}.
> Altering the table does not change the index metadata, so this check always
> fails.
> This especially affects per-row secondary indexes, where the index may need
> to handle columns being added or dropped.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10092) Generalize PerRowSecondaryIndex validation
[
https://issues.apache.org/jira/browse/CASSANDRA-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14978759#comment-14978759
]
Sam Tunnicliffe commented on CASSANDRA-10092:
-
[~adelapena] thanks for the updated patch. I appreciate the thrift API is a bit
esoteric, so thanks for persevering with it. I've backported the v3 patch to
2.1 and pushed both 2.1 & 2.2 branches to my local repo for CI. Once those test
results are clean I'll commit. Thanks again.
||2.1||2.2||
|[branch|https://github.com/beobal/cassandra/tree/10092-2.1]|[branch|https://github.com/beobal/cassandra/tree/10092-2.2]|
|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10092-2.1-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10092-2.2-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10092-2.1-dtest/]|[dtests|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10092-2.2-dtest/]|
> Generalize PerRowSecondaryIndex validation
> --
>
> Key: CASSANDRA-10092
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10092
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Andrés de la Peña
>Assignee: Andrés de la Peña
>Priority: Minor
> Labels: 2i, secondary_index, validation
> Fix For: 2.1.x, 2.2.x
>
> Attachments: CASSANDRA-10092_v2.patch, CASSANDRA-10092_v3.patch,
> improve_2i_validation.patch
>
>
> Index validation is currently done in a per-cell basis. However, per-row
> secondary index developers can be interested in validating all the written
> columns at once, because some implementations need to check the validity of a
> row write by comparing some column values against others. For example, a per
> row 2i implementation indexing time ranges (composed by a start date column
> and an end date column) should check that the start date is before the stop
> date.
> I'm attaching a patch adding a new method to {{PerRowSecondaryIndex}}:
> {code:java}
> public void validate(ByteBuffer key, ColumnFamily cf) throws
> InvalidRequestException {}
> {code}
> and a new method to {{SecondaryIndexManager}}:
> {code:java}
> public void validateRowLevelIndexes(ByteBuffer key, ColumnFamily cf) throws
> InvalidRequestException
> {
> for (SecondaryIndex index : rowLevelIndexMap.values())
> {
> ((PerRowSecondaryIndex) index).validate(key, cf);
> }
> }
> {code}
> This method is invoked in CQL {{UpdateStatement#validateIndexedColumns}}.
> This way, {{PerRowSecondaryIndex}} could perform complex write validation.
> I have tried to do the patch in the least invasive way possible. Maybe the
> current method {{SecondaryIndex#validate(ByteBuffer, Cell)}} should be moved
> to {{PerColumnSecondaryIndex}}, and the {{InvalidRequestException}} that
> informs about the particular 64k limitation should be thrown by
> {{AbstractSimplePerColumnSecondaryIndex}}. However, given the incoming
> [CASSANDRA-9459|https://issues.apache.org/jira/browse/CASSANDRA-9459], I
> think that the proposed patch is more than enough to provide rich validation
> features to 2i implementations based on 2.1.x and 2.2.x.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10513) Update cqlsh for new driver execution API
[ https://issues.apache.org/jira/browse/CASSANDRA-10513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14980145#comment-14980145 ] Sam Tunnicliffe commented on CASSANDRA-10513: - My understanding is that we also need the latest driver version for CASSANDRA-10390 (to include the fix for [PYTHON-413|https://datastax-oss.atlassian.net/browse/PYTHON-413]). See [this comment|https://datastax-oss.atlassian.net/browse/PYTHON-413?focusedCommentId=24609&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-24609 ] in particular. > Update cqlsh for new driver execution API > - > > Key: CASSANDRA-10513 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10513 > Project: Cassandra > Issue Type: New Feature > Components: Tools >Reporter: Adam Holmberg >Assignee: Paulo Motta >Priority: Minor > Labels: cqlsh > Fix For: 2.2.x, 3.0.x > > Attachments: 10513-2.1.txt, 10513-2.2.txt, 10513.txt > > > The 3.0 Python driver will have a few tweaks to the execution API. We'll need > to update cqlsh in a couple of minor ways. > [Results are always returned as an iterable > ResultSet|https://datastax-oss.atlassian.net/browse/PYTHON-368] > [Trace data is always attached to the > ResultSet|https://datastax-oss.atlassian.net/browse/PYTHON-318] (instead of > being attached to a statement) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10608) Adding a dynamic column to a compact storage table with the same name as the partition key causes a memtable flush deadlock
[
https://issues.apache.org/jira/browse/CASSANDRA-10608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14980486#comment-14980486
]
Sam Tunnicliffe commented on CASSANDRA-10608:
-
while [~mike_tr_adamson] gets set on with cassci I've pushed his branches to my
fork to get a CI run going:
||3.0||trunk||
|[branch|https://github.com/beobal/cassandra/tree/10608-3.0]|[branch|https://github.com/beobal/cassandra/tree/10608-trunk]|
|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10608-3.0-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10608-trunk-testall/]|
|[dtests|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10608-3.0-dtest/]|[dtests|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10608-trunk-dtest/]|
> Adding a dynamic column to a compact storage table with the same name as the
> partition key causes a memtable flush deadlock
> ---
>
> Key: CASSANDRA-10608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10608
> Project: Cassandra
> Issue Type: Bug
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: 10608.txt
>
>
> The reproduction steps for this are as follows:
> # Create the following schema:
> {noformat}
> CREATE KEYSPACE ks WITH replication = { 'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE ks.cf (k int, v int, PRIMARY KEY(k)) WITH COMPACT STORAGE;
> {noformat}
> # Using the thrift client execute the following:
> {noformat}
> Column column = new Column(ByteBufferUtil.bytes("k"));
> column.setValue(ByteBufferUtil.bytes(1));
> column.setTimestamp(1);
> client.insert(key, new ColumnParent(compactCf), column,
> ConsistencyLevel.ONE);
> {noformat}
> This causes an invalid {{PartitionUpdate}} to be added to {{Memtable}}
> containing a {{PartitionColumns}} containing the partition key
> {{ColumnDefinition}}.
> This happens because {{LegacyLayout.decodeCellName}} does not check whether
> the {{ColumnDefinition}} is a partition key
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-8653) Upgrading to trunk with auth throws exception
[
https://issues.apache.org/jira/browse/CASSANDRA-8653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe resolved CASSANDRA-8653.
Resolution: Not A Problem
Tracked down the hints file issue to a problem with ccm, whereby the
{{hints_directory}} isn't properly set during the upgrade, causing the cluster
nodes to share a single location, so I've opened a [ccm pull
request|https://github.com/pcmanus/ccm/pull/404] to fix that.
Running the {{upgrade_to_22_test}} also fails, but for a different reason. Once
the first node has been upgraded, cql connections to it start timing out. This
seems to be very similar to the problem described in
[PYTHON-303|https://datastax-oss.atlassian.net/browse/PYTHON-303] and raising
the timeout in the dtest gets it passing again. Opened a [dtest pull
request|https://github.com/riptano/cassandra-dtest/pull/638] for that, which
also removes the {{\@requires(9704)}} on {{upgrade_to_30_test}}.
> Upgrading to trunk with auth throws exception
> -
>
> Key: CASSANDRA-8653
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8653
> Project: Cassandra
> Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Sam Tunnicliffe
> Fix For: 3.0.0
>
> Attachments: node1.log, node2.log, node3.log
>
>
> When running Sam's upgrade_internal_auth_dtest, I am seeing the following
> exception (amongst others) in the log file of the second node to be upgraded
> to trunk from 2.1:
> {code}
> ERROR [GossipStage:1] 2015-01-20 13:46:21,679 CassandraDaemon.java:170 -
> Exception in thread Thread[GossipStage:1,5,main]
> java.lang.NoClassDefFoundError:
> org/apache/cassandra/transport/Event$TopologyChange$Change
> at
> org.apache.cassandra.transport.Server$EventNotifier.onJoinCluster(Server.java:374)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1668)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:1384)
> ~[main/:na]
> at
> org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1094)
> ~[main/:na]
> at
> org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:1076)
> ~[main/:na]
> at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1034)
> ~[main/:na]
> at
> org.apache.cassandra.gms.GossipDigestAckVerbHandler.doVerb(GossipDigestAckVerbHandler.java:58)
> ~[main/:na]
> 1554 - Node /127.0.0.1 state jump to normal
> ERROR [GossipStage:1] 2015-01-20 13:46:21,679 CassandraDaemon.java
> :170 - Exception in thread Thread[GossipStage:1,5,main]
> java.lang.NoClassDefFoundError: org/apache/cassandra/transport/Eve
> nt$TopologyChange$Change
> at org.apache.cassandra.transport.Server$EventNotifier.onJ
> oinCluster(Server.java:374) ~[main/:na]
> at org.apache.cassandra.service.StorageService.handleState
> Normal(StorageService.java:1668) ~[main/:na]
> at org.apache.cassandra.service.StorageService.onChange(St
> orageService.java:1384) ~[main/:na]
> at org.apache.cassandra.gms.Gossiper.doOnChangeNotificatio
> ns(Gossiper.java:1094) ~[main/:na]
> at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossip
> er.java:1076) ~[main/:na]
> at org.apache.cassandra.gms.Gossiper.applyStateLocally(Gos
> siper.java:1034) ~[main/:na]
> at org.apache.cassandra.gms.GossipDigestAckVerbHandler.doV
> erb(GossipDigestAckVerbHandler.java:58) ~[main/:na]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
> ~[main/:na]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_67]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ~[na:1.7.0_67]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_67]
> Caused by: java.lang.ClassNotFoundException:
> org.apache.cassandra.transport.Event$TopologyChange$Change
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> ~[na:1.7.0_67]
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> ~[na:1.7.0_67]
> at java.security.AccessController.doPrivileged(Native Method)
> ~[na:1.7.0_67]
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> ~[na:1.7.0_67]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> ~[na:1.7.0_67]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> ~[na:1.7.0_67]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ~[na:1.7.0_67]
> ... 11 common frames omitted
> WARN [Thread-10] 2015-01-20 13:46:34,500 IncomingTcpConnection.java:91 -
> UnknownColumnFamilyException reading from socket; closing
> org.a
[jira] [Updated] (CASSANDRA-10598) Unable to use same index name on column families which have different names in different keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-10598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-10598:
Reproduced In: 2.1.11, 1.2.x (was: 1.2.x, 2.1.11)
Reviewer: Sam Tunnicliffe
> Unable to use same index name on column families which have different names
> in different keyspaces
> --
>
> Key: CASSANDRA-10598
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10598
> Project: Cassandra
> Issue Type: Bug
>Reporter: Yeh Sheng Hsiung
> Fix For: 2.1.x, 2.2.x
>
> Attachments: cassandra-1.2-10598.patch, cassandra-2.1-10598.patch,
> cassandra-2.2-10598.patch
>
>
> Expected:
> The same index name can be used in different keyspaces without conflict
> Actual:
> Unable to use same index name on column families which have different names
> in different keyspaces
> Steps to Reproduce:
> {code}
> cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class':
> 'NetworkTopologyStrategy', 'DC1' : 1};
> cqlsh> CREATE KEYSPACE ks2 WITH replication = {'class':
> 'NetworkTopologyStrategy', 'DC1' : 1};
> cqlsh> use ks1;
> cqlsh:ks1> CREATE TABLE t1 (c1 int PRIMARY KEY, c2 int);
> cqlsh:ks1> create index c2_idx on t1 (c2);
> cqlsh:ks1> use ks2;
> cqlsh:ks2> CREATE TABLE t2 (c1 int PRIMARY KEY, c2 int);
> cqlsh:ks2> create index c2_idx on t2 (c2);
> ConfigurationException: configuration issue] message="Duplicate index name c2_idx">
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10598) Unable to use same index name on column families which have different names in different keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-10598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-10598:
Fix Version/s: (was: 2.2.x)
(was: 2.1.x)
> Unable to use same index name on column families which have different names
> in different keyspaces
> --
>
> Key: CASSANDRA-10598
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10598
> Project: Cassandra
> Issue Type: Bug
>Reporter: Yeh Sheng Hsiung
> Attachments: cassandra-1.2-10598.patch, cassandra-2.1-10598.patch,
> cassandra-2.2-10598.patch
>
>
> Expected:
> The same index name can be used in different keyspaces without conflict
> Actual:
> Unable to use same index name on column families which have different names
> in different keyspaces
> Steps to Reproduce:
> {code}
> cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class':
> 'NetworkTopologyStrategy', 'DC1' : 1};
> cqlsh> CREATE KEYSPACE ks2 WITH replication = {'class':
> 'NetworkTopologyStrategy', 'DC1' : 1};
> cqlsh> use ks1;
> cqlsh:ks1> CREATE TABLE t1 (c1 int PRIMARY KEY, c2 int);
> cqlsh:ks1> create index c2_idx on t1 (c2);
> cqlsh:ks1> use ks2;
> cqlsh:ks2> CREATE TABLE t2 (c1 int PRIMARY KEY, c2 int);
> cqlsh:ks2> create index c2_idx on t2 (c2);
> ConfigurationException: configuration issue] message="Duplicate index name c2_idx">
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10655) Skipping values during reads may cause incorrect read repairs
Sam Tunnicliffe created CASSANDRA-10655: --- Summary: Skipping values during reads may cause incorrect read repairs Key: CASSANDRA-10655 URL: https://issues.apache.org/jira/browse/CASSANDRA-10655 Project: Cassandra Issue Type: Bug Reporter: Sam Tunnicliffe Assignee: Sylvain Lebresne Priority: Critical Fix For: 3.0.0 Data responses may skip values for columns not selected by the column filter. This can lead to empty values being erroneously included in repair mutations sent out by the coordinator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10655) Skipping values during reads may cause incorrect read repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-10655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991759#comment-14991759 ] Sam Tunnicliffe commented on CASSANDRA-10655: - [This pull request|https://github.com/riptano/cassandra-dtest/pull/643] adds a dtest to expose this. > Skipping values during reads may cause incorrect read repairs > - > > Key: CASSANDRA-10655 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10655 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Sylvain Lebresne >Priority: Critical > Fix For: 3.0.0 > > > Data responses may skip values for columns not selected by the column filter. > This can lead to empty values being erroneously included in repair mutations > sent out by the coordinator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10655) Skipping values during reads may cause incorrect read repairs
[ https://issues.apache.org/jira/browse/CASSANDRA-10655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991776#comment-14991776 ] Sam Tunnicliffe commented on CASSANDRA-10655: - +1 new test passes with the patch applied. > Skipping values during reads may cause incorrect read repairs > - > > Key: CASSANDRA-10655 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10655 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Sylvain Lebresne >Priority: Critical > Fix For: 3.0.0 > > > Data responses may skip values for columns not selected by the column filter. > This can lead to empty values being erroneously included in repair mutations > sent out by the coordinator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10491) Inconsistent "position" numbering for keys in "system_schema.columns"
[
https://issues.apache.org/jira/browse/CASSANDRA-10491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14993591#comment-14993591
]
Sam Tunnicliffe commented on CASSANDRA-10491:
-
+1 LGTM.
{{LegacySSTableTest}} was failing on the branch, and not on 3.0 but rebasing
fixes that. I've pushed that rebased version
[here|https://github.com/beobal/cassandra/tree/10491], where I've also made a
trivial change to a comment (but that's very much a nit & I won't complain if
you disregard it).
> Inconsistent "position" numbering for keys in "system_schema.columns"
> -
>
> Key: CASSANDRA-10491
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10491
> Project: Cassandra
> Issue Type: Bug
>Reporter: Michael Penick
>Assignee: Sylvain Lebresne
>Priority: Minor
> Labels: client-impacting
> Fix For: 3.0.0
>
>
> A single component partition key starts off with a -1 position.
> {code}
> cqlsh> CREATE TABLE test.table1 (key1 text, value1 text, value2 text, PRIMARY
> KEY(key1));
> cqlsh> SELECT keyspace_name, table_name, column_name, kind, position FROM
> system_schema.columns WHERE keyspace_name = 'test' and table_name = 'table1'
> ;
> keyspace_name | table_name | column_name | kind | position
> ---++-+---+--
> test | table1 |key1 | partition_key | -1
> test | table1 | value1 | regular | -1
> test | table1 | value2 | regular | -1
> {code}
> A single component clustering key starts off with a 0 position.
> {code}
> cqlsh> CREATE TABLE test.table2 (key1 text, value1 text, value2 text, PRIMARY
> KEY(key1, value1));
> cqlsh> SELECT keyspace_name, table_name, column_name, kind, position FROM
> system_schema.columns WHERE keyspace_name = 'test' and table_name = 'table2'
> ;
> keyspace_name | table_name | column_name | kind | position
> ---++-+---+--
> test | table2 |key1 | partition_key | -1
> test | table2 | value1 |clustering |0
> test | table2 | value2 | regular | -1
> {code}
> When another component is added to the partition key it starts at 0.
> {code}
> cqlsh> CREATE TABLE test.table3 (key1 text, value1 text, value2 text, PRIMARY
> KEY((key1, value1)));
> cqlsh> SELECT keyspace_name, table_name, column_name, kind, position FROM
> system_schema.columns WHERE keyspace_name = 'test' and table_name = 'table3'
> ;
> keyspace_name | table_name | column_name | kind | position
> ---++-+---+--
> test | table3 |key1 | partition_key |0
> test | table3 | value1 | partition_key |1
> test | table3 | value2 | regular | -1
> {code}
> which is the same as a multiple component clustering key.
> {code}
> cqlsh> CREATE TABLE test.table4 (key1 text, value1 text, value2 text, PRIMARY
> KEY(key1, value1, value2));
> cqlsh> SELECT keyspace_name, table_name, column_name, kind, position FROM
> system_schema.columns WHERE keyspace_name = 'test' and table_name = 'table4'
> ;
> keyspace_name | table_name | column_name | kind | position
> ---++-+---+--
> test | table4 |key1 | partition_key | -1
> test | table4 | value1 |clustering |0
> test | table4 | value2 |clustering |1
> {code}
> Shouldn't a single component partition key start off with a position of 0?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-10594) Inconsistent permissions results return
[
https://issues.apache.org/jira/browse/CASSANDRA-10594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe reassigned CASSANDRA-10594:
---
Assignee: Sam Tunnicliffe
> Inconsistent permissions results return
> ---
>
> Key: CASSANDRA-10594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Adam Holmberg
>Assignee: Sam Tunnicliffe
>Priority: Minor
>
> The server returns inconsistent results when listing permissions, depending
> on whether a user is configured.
> *Observed with Cassandra 3.0:*
> Only super user configured:
> {code}
> cassandra@cqlsh> list all;
> role | resource | permissions
> --+--+-
> (0 rows)
> {code}
> VOID result type is returned (meaning no result meta is returned and cqlsh
> must use the table meta to determine columns)
> With one user configured, no grants:
> {code}
> cassandra@cqlsh> create user holmberg with password 'tmp';
> cassandra@cqlsh> list all;
> results meta: system_auth permissions 4
> role | username | resource| permission
> ---+---+-+
> cassandra | cassandra | | ALTER
> cassandra | cassandra | | DROP
> cassandra | cassandra | | AUTHORIZE
> (3 rows)
> {code}
> Now a ROWS result message is returned with the cassandra super user grants.
> Dropping the regular user causes the VOID message to be returned again.
> *Slightly different behavior on 2.2 branch:* VOID message with no result meta
> is returned, even if regular user is configured, until permissions are added
> to that user.
> *Expected:*
> It would be nice if the query always resulted in a ROWS result, even if there
> are no explicit permissions defined. This would provide the correct result
> metadata even if there are no rows.
> Additionally, it is strange that the 'cassandra' super user only appears in
> the results when another user is configured. I would expect it to always
> appear, or never.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-10390) inconsistent quoted identifier handling in UDTs
[
https://issues.apache.org/jira/browse/CASSANDRA-10390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe resolved CASSANDRA-10390.
-
Resolution: Fixed
[~jhalliday] with CASSANDRA-10513, the bundled driver has been updated to a
version including
[PYTHON-413|https://datastax-oss.atlassian.net/browse/PYTHON-413] so unless you
have any objections, I'll close this. I've opened a PR to add a dtest that
verifies the behaviour
[here|https://github.com/riptano/cassandra-dtest/pull/653].
> inconsistent quoted identifier handling in UDTs
> ---
>
> Key: CASSANDRA-10390
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10390
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
> Environment: 2.2.1
>Reporter: Jonathan Halliday
>Assignee: Sam Tunnicliffe
> Fix For: 2.2.x
>
>
> > create keyspace test with replication = {'class': 'SimpleStrategy',
> > 'replication_factor': 1 } ;
> > create type if not exists mytype ("my.field" text);
> > desc keyspace; -- observe that mytype is listed
> > create table mytable (pk int primary key, myfield frozen);
> > desc keyspace; -- observe that mytype is listed, but mytable is not.
> > select * from mytable;
> ValueError: Type names and field names can only contain alphanumeric
> characters and underscores: 'my.field'
> create table myothertable (pk int primary key, "my.field" text);
> select * from myothertable; -- valid
> huh? It's valid to create a field of a table, or a field of a type, with a
> quoted name containing non-alpha chars, but it's not valid to use a such a
> type in a table? I can just about live with that though it seems
> unnecessarily restrictive, but allowing creation of such a table and then
> making it invisible/unusable definitely seems wrong.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-10647) manual 2i rebuilding dtest failure on Windows
[
https://issues.apache.org/jira/browse/CASSANDRA-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe reassigned CASSANDRA-10647:
---
Assignee: Sam Tunnicliffe
> manual 2i rebuilding dtest failure on Windows
> -
>
> Key: CASSANDRA-10647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10647
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
>Assignee: Sam Tunnicliffe
> Fix For: 3.1
>
>
> {{secondary_indexes_test.py:TestSecondaryIndexes.test_manual_rebuild_index}}
> failed once on CassCI running C* 3.0 under Windows:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/101/testReport/secondary_indexes_test/TestSecondaryIndexes/test_manual_rebuild_index/history/
> It fails here:
> https://github.com/riptano/cassandra-dtest/blob/master/secondary_indexes_test.py#L294
> with the error {{1 != 0}}. [~beobal] IIRC you understand these tests. Any
> idea why it'd flap like this? or could this be a genuine regression?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-10647) manual 2i rebuilding dtest failure on Windows
[
https://issues.apache.org/jira/browse/CASSANDRA-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe resolved CASSANDRA-10647.
-
Resolution: Not A Problem
[~mambocab] looks like
[{{1e8007b}}|https://github.com/apache/cassandra/commit/1e8007b] broke it, the
[cassandra-3.0
dtest|http://cassci.datastax.com/view/cassandra-3.0/job/cassandra-3.0_dtest/313/testReport/secondary_indexes_test/TestSecondaryIndexes/test_manual_rebuild_index/]
run for the same commit also failed in the same way. That commit was swiftly
reverted & the test immediately started passing again. I've double checked and
running against the reverted commit, it fails consistently.
> manual 2i rebuilding dtest failure on Windows
> -
>
> Key: CASSANDRA-10647
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10647
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Jim Witschey
> Fix For: 3.1
>
>
> {{secondary_indexes_test.py:TestSecondaryIndexes.test_manual_rebuild_index}}
> failed once on CassCI running C* 3.0 under Windows:
> http://cassci.datastax.com/view/win32/job/cassandra-3.0_dtest_win32/101/testReport/secondary_indexes_test/TestSecondaryIndexes/test_manual_rebuild_index/history/
> It fails here:
> https://github.com/riptano/cassandra-dtest/blob/master/secondary_indexes_test.py#L294
> with the error {{1 != 0}}. [~beobal] IIRC you understand these tests. Any
> idea why it'd flap like this? or could this be a genuine regression?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10594) Inconsistent permissions results return
[
https://issues.apache.org/jira/browse/CASSANDRA-10594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14998550#comment-14998550
]
Sam Tunnicliffe commented on CASSANDRA-10594:
-
There are 2 things here; the VOID vs ROWS result message, and the
(non-)appearance of rows in the results.
The first is simply a historical artifact. I can't say for sure why we decided
to return a VOID result when there are no rows to display, but {{LIST
PERMISSIONS}} has always done that, as has {{LIST USERS}}, and so {{LIST
ROLES}} followed the precedent when introduced in 2.2. We could change such
that a ROWS result is always returned, but whilst this wouldn't be a protocol
change, it would be a change to expected, if undocumented, behaviour so we
can't just go ahead and do it whenever. Under the tick-tock model, this would
need to be targetted for 4.0, but you could even question whether it's worth it
at all, even though it's a pain for cqlsh.
The reason for what seems like inconsistent permissions being listed is that
the {{cassandra}} role has superuser status which is effectively a role-level
attribute, rather than a set of permissions. So before the additional user is
created, there are indeed no permissions which have been granted to
{{cassandra}}, and so the result of {{LIST ALL}} at that point is empty. When a
user (or any other resource) is created though, we automatically grant
'ownership' permissions to the role which created it, hence you then see the
permissions on {{}} granted to {{cassandra}} (see
CASSANDRA-7216). This came in in 2.2. so I'm a bit puzzled by
bq. Slightly different behavior on 2.2 branch: VOID message with no result meta
is returned, even if regular user is configured, until permissions are added to
that user.
and I can't reproduce this myself. Would you be able to detail the steps to
repro this on 2.2?
Your example actually highlights another problem in that the grammar is way too
permissive here, as {{LIST ALL}} shouldn't be valid without at least appending
{{PERMISSIONS}}. IMO, the valid forms should be (assuming superuser status):
{code}
LIST ALL PERMISSIONS;
LIST ALL PERMISSIONS OF ;
LIST ALL PERMISSIONS ON ;
LIST ALL PERMISSIONS ON OF ;
{code}
plus of course the variants which specify a single permission, rather than
{{ALL}}. As with the first issue above, it's regrettable that this is the case,
but realistically, I don't think we can fix it outside of a major
(super-major?) release.
> Inconsistent permissions results return
> ---
>
> Key: CASSANDRA-10594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Adam Holmberg
>Assignee: Sam Tunnicliffe
>Priority: Minor
>
> The server returns inconsistent results when listing permissions, depending
> on whether a user is configured.
> *Observed with Cassandra 3.0:*
> Only super user configured:
> {code}
> cassandra@cqlsh> list all;
> role | resource | permissions
> --+--+-
> (0 rows)
> {code}
> VOID result type is returned (meaning no result meta is returned and cqlsh
> must use the table meta to determine columns)
> With one user configured, no grants:
> {code}
> cassandra@cqlsh> create user holmberg with password 'tmp';
> cassandra@cqlsh> list all;
> results meta: system_auth permissions 4
> role | username | resource| permission
> ---+---+-+
> cassandra | cassandra | | ALTER
> cassandra | cassandra | | DROP
> cassandra | cassandra | | AUTHORIZE
> (3 rows)
> {code}
> Now a ROWS result message is returned with the cassandra super user grants.
> Dropping the regular user causes the VOID message to be returned again.
> *Slightly different behavior on 2.2 branch:* VOID message with no result meta
> is returned, even if regular user is configured, until permissions are added
> to that user.
> *Expected:*
> It would be nice if the query always resulted in a ROWS result, even if there
> are no explicit permissions defined. This would provide the correct result
> metadata even if there are no rows.
> Additionally, it is strange that the 'cassandra' super user only appears in
> the results when another user is configured. I would expect it to always
> appear, or never.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10683) Internal pagination of CQL index queries with compact storage is suboptimal
Sam Tunnicliffe created CASSANDRA-10683:
---
Summary: Internal pagination of CQL index queries with compact
storage is suboptimal
Key: CASSANDRA-10683
URL: https://issues.apache.org/jira/browse/CASSANDRA-10683
Project: Cassandra
Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Fix For: 2.1.12, 2.2.4, 3.0.1
The bug in page sizing logic for {{CompositesSearcher}} described in
CASSANDRA-8550 also affects {{KeysSearcher}}.
For any index with non-trivial cardinality, the page size tends to always be
set suboptimally to 2, resulting in many more (smaller) slices being read the
index table than necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10683) Internal pagination of CQL index queries with compact storage is suboptimal
[
https://issues.apache.org/jira/browse/CASSANDRA-10683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-10683:
Fix Version/s: (was: 3.0.1)
> Internal pagination of CQL index queries with compact storage is suboptimal
> ---
>
> Key: CASSANDRA-10683
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10683
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
> Fix For: 2.1.12, 2.2.4
>
>
> The bug in page sizing logic for {{CompositesSearcher}} described in
> CASSANDRA-8550 also affects {{KeysSearcher}}.
> For any index with non-trivial cardinality, the page size tends to always be
> set suboptimally to 2, resulting in many more (smaller) slices being read the
> index table than necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-10685) Index transaction dealing with cleanup doesn't notify indexes of partition deletion
Sam Tunnicliffe created CASSANDRA-10685:
---
Summary: Index transaction dealing with cleanup doesn't notify
indexes of partition deletion
Key: CASSANDRA-10685
URL: https://issues.apache.org/jira/browse/CASSANDRA-10685
Project: Cassandra
Issue Type: Bug
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Priority: Minor
Fix For: 3.0.1, 3.1
{SecondaryIndexManager.CleanupGCTransaction}} doesn't notify registered
indexers of the partition level deletion on commit. Indexers *are* notified of
each individual row removal caused by the partition delete, but also informing
them of the top level delete would enable them to make potential optimizations
during cleanup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10685) Index transaction dealing with cleanup doesn't notify indexes of partition deletion
[
https://issues.apache.org/jira/browse/CASSANDRA-10685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-10685:
Description: {{SecondaryIndexManager.CleanupGCTransaction}} doesn't notify
registered indexers of the partition level deletion on commit. Indexers *are*
notified of each individual row removal caused by the partition delete, but
also informing them of the top level delete would enable them to make potential
optimizations during cleanup. (was:
{SecondaryIndexManager.CleanupGCTransaction}} doesn't notify registered
indexers of the partition level deletion on commit. Indexers *are* notified of
each individual row removal caused by the partition delete, but also informing
them of the top level delete would enable them to make potential optimizations
during cleanup. )
> Index transaction dealing with cleanup doesn't notify indexes of partition
> deletion
> ---
>
> Key: CASSANDRA-10685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10685
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 3.0.1, 3.1
>
>
> {{SecondaryIndexManager.CleanupGCTransaction}} doesn't notify registered
> indexers of the partition level deletion on commit. Indexers *are* notified
> of each individual row removal caused by the partition delete, but also
> informing them of the top level delete would enable them to make potential
> optimizations during cleanup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8505) Invalid results are returned while secondary index are being build
[
https://issues.apache.org/jira/browse/CASSANDRA-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15000767#comment-15000767
]
Sam Tunnicliffe commented on CASSANDRA-8505:
I think the various index states can be reduced to a simple ready/not ready
check. What's more unless we intend to change the established behaviour fairly
significantly, once an index moves to a ready state it never moves back to
being not ready. The only times when we modify the status in the system table
are when the index is removed (in which case we have no problem with being able
to query using it) or during a rebuild. In the latter case though, we probably
shouldn't reject queries (and we don't currently), as an index rebuild is
incremental. That is, we don't scrap the existing index tables and rebuild
everything from scratch, just write new index SSTables to supercede the old
ones. So although it's certainly possible to get incorrect results during a
rebuild (because of missing/stale entries), the results only get more correct
as the rebuild progresses. Changing this so that all queries against that index
return errors until all rebuilds complete seems like a step backwards. It seems
more reasonable to reject queries until the initial build has been performed,
as per the example in the description, but this only requires a simple boolean
to track state between instantiating/registering the index and its initial
build task completing (if one is required).
It would be good to have some test coverage of this, although the best I could
come up with is a dtest which inserts many rows, then adds the index and
queries immediately expecting ReadFailureException, which is fairly lame and
fragile.
A couple of points specific to the 3.0 patch:
* The fix for CASSANDRA-10595 has been lost. If an index doesn't register
itself in {{createIndex}}, don't ask it for an initalization task, just set
{{initialBuildTask == null}}.
* {{SIM::reloadIndex}} has changed since the patch was created (due to
CASSANDRA-10604) - I think that no changes to this method are now required. I
did notice though that the current implementation actually makes a redundant
call to {{getMetadataReloadTask}}, so if you could fix that while you're here,
that'd be great.
bq. Secondary index and their build/not build status are node-local. By
consequence it is not possible to know on a coordinator node if the index is
fully build. It can be built on the coordinator but still building on other
nodes
For future reference on this point, we also have CASSANDRA-9967 which has a
very similar intent.
> Invalid results are returned while secondary index are being build
> --
>
> Key: CASSANDRA-8505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8505
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 2.2.x, 3.0.x
>
>
> If you request an index creation and then execute a query that use the index
> the results returned might be invalid until the index is fully build. This is
> caused by the fact that the table column will be marked as indexed before the
> index is ready.
> The following unit tests can be use to reproduce the problem:
> {code}
> @Test
> public void testIndexCreatedAfterInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
>
> createIndex("CREATE INDEX ON %s(b)");
>
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
>
> @Test
> public void testIndexCreatedBeforeInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> createIndex("CREATE INDEX ON %s(b)");
>
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
> {code}
> The first test will fail while the second will work.
> In my o
[jira] [Commented] (CASSANDRA-8505) Invalid results are returned while secondary index are being build
[
https://issues.apache.org/jira/browse/CASSANDRA-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15001055#comment-15001055
]
Sam Tunnicliffe commented on CASSANDRA-8505:
bq. What about accepting either ReadFailureException or the complete, correct
result? If index building got faster we might stop hitting the
ReadFailureException case, but at least the test wouldn't flap.
Yes absolutely, even then though we're not going to be certain exactly what's
being tested (if at all) - e.g. the index building gets faster but we also
introduce a regression with the {{ReadFailureException}}, we'd never know. But
like I say, I can't think of anything better.
> Invalid results are returned while secondary index are being build
> --
>
> Key: CASSANDRA-8505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8505
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 2.2.x, 3.0.x
>
>
> If you request an index creation and then execute a query that use the index
> the results returned might be invalid until the index is fully build. This is
> caused by the fact that the table column will be marked as indexed before the
> index is ready.
> The following unit tests can be use to reproduce the problem:
> {code}
> @Test
> public void testIndexCreatedAfterInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
>
> createIndex("CREATE INDEX ON %s(b)");
>
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
>
> @Test
> public void testIndexCreatedBeforeInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> createIndex("CREATE INDEX ON %s(b)");
>
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
> {code}
> The first test will fail while the second will work.
> In my opinion the first test should reject the request as invalid (as if the
> index was not existing) until the index is fully build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8505) Invalid results are returned while secondary index are being build
[
https://issues.apache.org/jira/browse/CASSANDRA-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15001195#comment-15001195
]
Sam Tunnicliffe commented on CASSANDRA-8505:
We could certainly do that in a utest (and we have plenty of tests with such
custom indexes), but it not a dtest as it would require the custom index to be
on the classpath. Naturally, a utest won't exercise the distributed side of
things, but it's still better than no testing, so +1
> Invalid results are returned while secondary index are being build
> --
>
> Key: CASSANDRA-8505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8505
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 2.2.x, 3.0.x
>
>
> If you request an index creation and then execute a query that use the index
> the results returned might be invalid until the index is fully build. This is
> caused by the fact that the table column will be marked as indexed before the
> index is ready.
> The following unit tests can be use to reproduce the problem:
> {code}
> @Test
> public void testIndexCreatedAfterInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
>
> createIndex("CREATE INDEX ON %s(b)");
>
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
>
> @Test
> public void testIndexCreatedBeforeInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> createIndex("CREATE INDEX ON %s(b)");
>
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
> {code}
> The first test will fail while the second will work.
> In my opinion the first test should reject the request as invalid (as if the
> index was not existing) until the index is fully build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10694) Deletion info is dropped on updated rows when notifying secondary index
[
https://issues.apache.org/jira/browse/CASSANDRA-10694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004043#comment-15004043
]
Sam Tunnicliffe commented on CASSANDRA-10694:
-
+1 (pending CI)
> Deletion info is dropped on updated rows when notifying secondary index
> ---
>
> Key: CASSANDRA-10694
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10694
> Project: Cassandra
> Issue Type: Bug
> Components: index
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
> Fix For: 3.0.1, 3.1
>
> Attachments: index-deletion.patch
>
>
> In {{SecondaryIndexManager.onUpdated()}}, we fail to copy the
> {{DeletionInfo}} from the existing and new rows before notifying the index of
> the update. This leads the index to believe a new, live row has been
> inserted instead of a single-row deletion. It looks like this has been a
> problem since 3.0.0-beta1.
> I've attached a simple patch that fixes the issue. I'm working on a full
> patch with tests, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8786) NullPointerException in ColumnDefinition.hasIndexOption
[
https://issues.apache.org/jira/browse/CASSANDRA-8786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe reassigned CASSANDRA-8786:
--
Assignee: Sam Tunnicliffe (was: Aleksey Yeschenko)
> NullPointerException in ColumnDefinition.hasIndexOption
> ---
>
> Key: CASSANDRA-8786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8786
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 2.1.2
>Reporter: Mathijs Vogelzang
>Assignee: Sam Tunnicliffe
> Fix For: 2.1.5
>
> Attachments: 8786.txt
>
>
> We have a Cassandra cluster that we've been using through many upgrades, and
> thus most of our column families have originally been created by Thrift. We
> are on Cassandra 2.1.2 now.
> We've now ported most of our code to use CQL, and our code occasionally tries
> to recreate tables with "IF NOT EXISTS" to work properly on development /
> testing environments.
> When we issue the CQL statement "CREATE INDEX IF NOT EXISTS index ON
> "tableName" (accountId)" (this index does exist on that table already), we
> get a {{DriverInternalError: An unexpected error occurred server side on
> cass_host/xx.xxx.xxx.xxx:9042: java.lang.NullPointerException}}
> The error on the server is:
> {noformat}
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.config.ColumnDefinition.hasIndexOption(ColumnDefinition.java:489)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.cql3.statements.CreateIndexStatement.validate(CreateIndexStatement.java:87)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:224)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:248)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> {noformat}
> This happens every time we run this CQL statement. We've tried to reproduce
> it in a test cassandra cluster by creating the table according to the exact
> "DESCRIBE TABLE" specification, but then this NullPointerException doesn't
> happon upon the CREATE INDEX one. So it seems that the tables on our
> production cluster (that were originally created through thrift) are still
> subtly different schema-wise then a freshly created table according to the
> same creation statement.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8786) NullPointerException in ColumnDefinition.hasIndexOption
[
https://issues.apache.org/jira/browse/CASSANDRA-8786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-8786:
---
Assignee: Aleksey Yeschenko (was: Sam Tunnicliffe)
> NullPointerException in ColumnDefinition.hasIndexOption
> ---
>
> Key: CASSANDRA-8786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8786
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 2.1.2
>Reporter: Mathijs Vogelzang
>Assignee: Aleksey Yeschenko
> Fix For: 2.1.5
>
> Attachments: 8786.txt
>
>
> We have a Cassandra cluster that we've been using through many upgrades, and
> thus most of our column families have originally been created by Thrift. We
> are on Cassandra 2.1.2 now.
> We've now ported most of our code to use CQL, and our code occasionally tries
> to recreate tables with "IF NOT EXISTS" to work properly on development /
> testing environments.
> When we issue the CQL statement "CREATE INDEX IF NOT EXISTS index ON
> "tableName" (accountId)" (this index does exist on that table already), we
> get a {{DriverInternalError: An unexpected error occurred server side on
> cass_host/xx.xxx.xxx.xxx:9042: java.lang.NullPointerException}}
> The error on the server is:
> {noformat}
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.config.ColumnDefinition.hasIndexOption(ColumnDefinition.java:489)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.cql3.statements.CreateIndexStatement.validate(CreateIndexStatement.java:87)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:224)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:248)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:119)
> ~[apache-cassandra-2.1.2.jar:2.1.2]
> {noformat}
> This happens every time we run this CQL statement. We've tried to reproduce
> it in a test cassandra cluster by creating the table according to the exact
> "DESCRIBE TABLE" specification, but then this NullPointerException doesn't
> happon upon the CREATE INDEX one. So it seems that the tables on our
> production cluster (that were originally created through thrift) are still
> subtly different schema-wise then a freshly created table according to the
> same creation statement.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-8941) Test Coverage for CASSANDRA-8786
[ https://issues.apache.org/jira/browse/CASSANDRA-8941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe reassigned CASSANDRA-8941: -- Assignee: Sam Tunnicliffe (was: Philip Thompson) > Test Coverage for CASSANDRA-8786 > > > Key: CASSANDRA-8941 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8941 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Tyler Hobbs >Assignee: Sam Tunnicliffe >Priority: Minor > Fix For: 2.1.x > > > We don't currently have a test to reproduce the issue from CASSANDRA-8786. > It would be good to track down exactly what circustances cause this and add > some test coverage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10690) Secondary index does not process deletes unless columns are specified
[
https://issues.apache.org/jira/browse/CASSANDRA-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010476#comment-15010476
]
Sam Tunnicliffe commented on CASSANDRA-10690:
-
I'm +1 with [~thobbs]. It's an improvement to the API, but it sets a bad
precedent if we make breaking changes to the interface in a bugfix version.
Even though in this particular case the odds of it actually impacting anybody
likely to be minimal.
> Secondary index does not process deletes unless columns are specified
> -
>
> Key: CASSANDRA-10690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10690
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Tyler Hobbs
> Fix For: 3.0.1, 3.1
>
>
> The new secondary index API does not notify indexes of single-row or slice
> deletions unless specific columns are deleted. I believe the problem is that
> in {{SecondaryIndexManager.newUpdateTransaction()}}, we skip indexes unless
> {{index.indexes(update.columns())}}. When no columns are specified in the
> the deletion, {{update.columns()}} is empty, which causes all indexes to be
> skipped.
> I think the correct fix is to do something like this in the
> {{ModificationStatement}} constructor:
> {code}
> if (type == StatementType.DELETE && modifiedColumns.isEmpty())
> modifiedColumns = cfm.partitionColumns();
> {code}
> However, I'm not sure if that may have unintended side-effects. What do you
> think, [~slebresne]?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10681) make index building pluggable via IndexBuildTask
[ https://issues.apache.org/jira/browse/CASSANDRA-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010652#comment-15010652 ] Sam Tunnicliffe commented on CASSANDRA-10681: - The issue with this is that it serializes the building of all indexes, regardless of whether they're SASI indexes or not. So whereas previously all indexes for a table were built in a single pass over the data, a separate iteration is required for each. This could be a problem wherever multiple indexes are defined for a table & indexes need to be rebuilt, so when new SSTables are added via streaming or sideloading, or during a user-requested rebuild from nodetool. > make index building pluggable via IndexBuildTask > > > Key: CASSANDRA-10681 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10681 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Labels: sasi > Fix For: 3.x > > > Currently index building assumes one and only way to build all of the indexes > - through SecondaryIndexBuilder - which merges all of the sstables together, > collates columns etc. Such works fine for built-in indexes but not for SASI > since it's attaches to every SSTable individually. We need a "IndexBuildTask" > interface (based on CompactionInfo.Holder) to be returned from Index on > demand to give power to SI interface implementers to decide how build should > work. This might be less effective for CassandraIndex, since this effectively > means that collation will have to be done multiple times on the same data, > but nevertheless is a good compromise for clean interface to outside world. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10678) add SSTable flush observer
[
https://issues.apache.org/jira/browse/CASSANDRA-10678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15010925#comment-15010925
]
Sam Tunnicliffe commented on CASSANDRA-10678:
-
I guess the following is a prerequisite of
https://github.com/xedin/sasi/issues/3, but I'll mention it here anyway:
{{SSTableFlushObserver::startRow}} is misnamed, being called when before a
partition, not a row, is written to the SSTable. In fact, rows are not really
tracked at all during the flush, only partitions and cells. The corresponding
tests also highlight this. So at least one additional method is required on the
interface to capture events at this level. Is it possible that RTs may be of
interest to some observers too? If so, perhaps we also need another method for
those.
Nit: missing javadoc for {{Index::getFlushObserver}}
The {{SSTableFlushObserver.Source}} enum is unused at the moment. Is this in
preparation for an upcoming patch, or a leftover from the pre-3.0 version?
> add SSTable flush observer
> --
>
> Key: CASSANDRA-10678
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10678
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Pavel Yaskevich
>Assignee: Pavel Yaskevich
> Fix For: 3.x
>
>
> Add general interface which can intercept per SSTable flush events e.g. -
> start of the key, columns etc. Make Index interface return such observer on
> request, which couples index with corresponding SSTable file if needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10678) add SSTable flush observer
[
https://issues.apache.org/jira/browse/CASSANDRA-10678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013330#comment-15013330
]
Sam Tunnicliffe commented on CASSANDRA-10678:
-
The v2 patch looks good to me, modulo a couple of nits that can be addressed on
commit.
* The entry in CHANGES.txt is appended to the list for 3.2, whereas lately the
convention has been to preprend (just to make subsequent merges easier).
* The javadoc for {{SSTableFlushObserver::nextCell}} refers to the now-renamed
{{startRow}}.
Related to the latter point: as I mentioned before, I suspect that when you
come to add SASI support for non-compact storage you're going to need some
method on the observer to track clusterings within a partition, but I'm fine
with waiting until that's actually required before adding that.
> add SSTable flush observer
> --
>
> Key: CASSANDRA-10678
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10678
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Pavel Yaskevich
>Assignee: Pavel Yaskevich
> Fix For: 3.x
>
> Attachments: 0001-Add-sstable-flush-observer.patch
>
>
> Add general interface which can intercept per SSTable flush events e.g. -
> start of the key, columns etc. Make Index interface return such observer on
> request, which couples index with corresponding SSTable file if needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10678) add SSTable flush observer
[ https://issues.apache.org/jira/browse/CASSANDRA-10678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013358#comment-15013358 ] Sam Tunnicliffe commented on CASSANDRA-10678: - [~xedin] seeing as you're not set up for dev builds on http://cassci.datastax.com I've pushed a branch to get a CI run (unfortunately the dtest results are not overly useful at the moment). It may be worth pinging exlt or ptnapoleon in \#cassandra-dev to set you up on the CI server. ||branch||testall||dtest|| |[10678-trunk|https://github.com/beobal/cassandra/tree/10678-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10678-trunk-testall/]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10678-trunk-dtest/] > add SSTable flush observer > -- > > Key: CASSANDRA-10678 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10678 > Project: Cassandra > Issue Type: Sub-task >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich > Fix For: 3.x > > Attachments: 0001-Add-sstable-flush-observer.patch > > > Add general interface which can intercept per SSTable flush events e.g. - > start of the key, columns etc. Make Index interface return such observer on > request, which couples index with corresponding SSTable file if needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10681) make index building pluggable via IndexBuildTask
[
https://issues.apache.org/jira/browse/CASSANDRA-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15016841#comment-15016841
]
Sam Tunnicliffe commented on CASSANDRA-10681:
-
My solution is almost identical to your first one, the only real difference
being to group the tasks by the class of the task itself, rather than by the
index class. That way, all existing index implementations can continue to share
a single task without any modifications. To avoid constructing some task
instances which are immediately discarded, I did consider having the new method
on {{Index}} return the class of the build task instead of an instance and
handling construction via reflection, but the added complexity doesn't seem
worth it right now. Patch on top of your initial commit
[here|https://github.com/beobal/cassandra/commits/10681-trunk]
> make index building pluggable via IndexBuildTask
>
>
> Key: CASSANDRA-10681
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10681
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local Write-Read Paths
>Reporter: Pavel Yaskevich
>Assignee: Pavel Yaskevich
>Priority: Minor
> Labels: sasi
> Fix For: 3.x
>
> Attachments: 0001-add-table-support-for-multi-table-builds.patch,
> 0001-make-index-building-pluggable-via-IndexBuildTask.patch
>
>
> Currently index building assumes one and only way to build all of the indexes
> - through SecondaryIndexBuilder - which merges all of the sstables together,
> collates columns etc. Such works fine for built-in indexes but not for SASI
> since it's attaches to every SSTable individually. We need a "IndexBuildTask"
> interface (based on CompactionInfo.Holder) to be returned from Index on
> demand to give power to SI interface implementers to decide how build should
> work. This might be less effective for CassandraIndex, since this effectively
> means that collation will have to be done multiple times on the same data,
> but nevertheless is a good compromise for clean interface to outside world.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10690) Remove unclear indexes() method from 2ndary index API
[
https://issues.apache.org/jira/browse/CASSANDRA-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-10690:
Reviewer: Sam Tunnicliffe
> Remove unclear indexes() method from 2ndary index API
> -
>
> Key: CASSANDRA-10690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10690
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Tyler Hobbs
> Fix For: 3.0.1, 3.1
>
>
> The new secondary index API does not notify indexes of single-row or slice
> deletions unless specific columns are deleted. I believe the problem is that
> in {{SecondaryIndexManager.newUpdateTransaction()}}, we skip indexes unless
> {{index.indexes(update.columns())}}. When no columns are specified in the
> the deletion, {{update.columns()}} is empty, which causes all indexes to be
> skipped.
> I think the correct fix is to do something like this in the
> {{ModificationStatement}} constructor:
> {code}
> if (type == StatementType.DELETE && modifiedColumns.isEmpty())
> modifiedColumns = cfm.partitionColumns();
> {code}
> However, I'm not sure if that may have unintended side-effects. What do you
> think, [~slebresne]?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10690) Remove unclear indexes() method from 2ndary index API
[
https://issues.apache.org/jira/browse/CASSANDRA-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15018254#comment-15018254
]
Sam Tunnicliffe commented on CASSANDRA-10690:
-
+1 on the patch, with 2 minuscule nits:
* Typo in the javadoc of {{Index::indexerFor}} - s/partiticular/particular
* I would also amend slightly the new comment in {{CassandraIndex::indexerFor}}
to make it clear that stale entries are actually removed from the index on read
as well as during compaction.
> Remove unclear indexes() method from 2ndary index API
> -
>
> Key: CASSANDRA-10690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10690
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Tyler Hobbs
>Assignee: Sylvain Lebresne
> Fix For: 3.0.1, 3.1
>
>
> The new secondary index API does not notify indexes of single-row or slice
> deletions unless specific columns are deleted. I believe the problem is that
> in {{SecondaryIndexManager.newUpdateTransaction()}}, we skip indexes unless
> {{index.indexes(update.columns())}}. When no columns are specified in the
> the deletion, {{update.columns()}} is empty, which causes all indexes to be
> skipped.
> I think the correct fix is to do something like this in the
> {{ModificationStatement}} constructor:
> {code}
> if (type == StatementType.DELETE && modifiedColumns.isEmpty())
> modifiedColumns = cfm.partitionColumns();
> {code}
> However, I'm not sure if that may have unintended side-effects. What do you
> think, [~slebresne]?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10681) make index building pluggable via IndexBuildTask
[
https://issues.apache.org/jira/browse/CASSANDRA-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022125#comment-15022125
]
Sam Tunnicliffe commented on CASSANDRA-10681:
-
bq. allocation of the build task would require careful closing of them (which
your patch doesn't do
good catch, thanks.
bq. we make a contract that each index type has to have it's own build task to
be grouped
I still think it's overly restrictive to only group by index type, but the
tablewide support patch doesn't necessarily entail this. If we introduce a
publicly accessible singleton and use that rather the private
{{CassandraIndexSupport}} instance on {{CassandraIndex}} we should be able to
share the build task among all indexes which are built in this way. I'd also
probably rename {{CassandraIndexSupport}} and {{CassandraIndexBuilder}} seeing
as they could both be reused by any impl. Also, the naming of
{{TableWideSupport}} is a bit obscure for me, can we just use
{{IndexBuildingSupport}}? I guess the iface could be extended later with other
stuff, but we should probably cross that bridge if/when we come to it.
Finally, the patch as is has a bug: {{SIM::buildIndexBlocking}} always throws
an NPE as it should be using {{Map::computeIfAbsent}} instead of
{{Map::putIfAbsent}}. I've pushed an updated version with those suggested
changes [here|https://github.com/beobal/cassandra/tree/10681-v2-trunk], wdyt?
> make index building pluggable via IndexBuildTask
>
>
> Key: CASSANDRA-10681
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10681
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local Write-Read Paths
>Reporter: Pavel Yaskevich
>Assignee: Pavel Yaskevich
>Priority: Minor
> Labels: sasi
> Fix For: 3.2
>
> Attachments: 0001-add-table-support-for-multi-table-builds.patch,
> 0001-make-index-building-pluggable-via-IndexBuildTask.patch
>
>
> Currently index building assumes one and only way to build all of the indexes
> - through SecondaryIndexBuilder - which merges all of the sstables together,
> collates columns etc. Such works fine for built-in indexes but not for SASI
> since it's attaches to every SSTable individually. We need a "IndexBuildTask"
> interface (based on CompactionInfo.Holder) to be returned from Index on
> demand to give power to SI interface implementers to decide how build should
> work. This might be less effective for CassandraIndex, since this effectively
> means that collation will have to be done multiple times on the same data,
> but nevertheless is a good compromise for clean interface to outside world.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8505) Invalid results are returned while secondary index are being build
[
https://issues.apache.org/jira/browse/CASSANDRA-8505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022173#comment-15022173
]
Sam Tunnicliffe commented on CASSANDRA-8505:
Looks pretty good to me, I just have a small nit & one question/suggestion
about naming:
Firstly, the catch blocks for {{TombstoneOverwhelmingException}} and
{{IndexNotAvailableException}} in {{MessageDeliveryTask::run}} can be combined
into one multi catch. Also, that catch for {{IndexNotAvailableException}} was
only added in 3.0, and it seems it could also be done for the 2.2 branch.
On the question of naming, I wonder if perhaps the names of the new methods
which check the state of the indexes ought to reflect the fact that the
readiness is in regard to querying (i.e. an index will process updates as soon
as it's created, the new flag just guards against it's use in queries). So, on
the 2.2 branch, we could rename {{SI::isReady}} to {{SI::isQueryable}} and on
3.0 {{SIM::isIndexReady}} -> {{SIM::isIndexQueryable}}.
Do you have any thoughts on that?
> Invalid results are returned while secondary index are being build
> --
>
> Key: CASSANDRA-8505
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8505
> Project: Cassandra
> Issue Type: Bug
> Components: Coordination
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 2.2.x, 3.0.x
>
>
> If you request an index creation and then execute a query that use the index
> the results returned might be invalid until the index is fully build. This is
> caused by the fact that the table column will be marked as indexed before the
> index is ready.
> The following unit tests can be use to reproduce the problem:
> {code}
> @Test
> public void testIndexCreatedAfterInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
>
> createIndex("CREATE INDEX ON %s(b)");
>
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
>
> @Test
> public void testIndexCreatedBeforeInsert() throws Throwable
> {
> createTable("CREATE TABLE %s (a int, b int, c int, primary key((a,
> b)))");
> createIndex("CREATE INDEX ON %s(b)");
>
> execute("INSERT INTO %s (a, b, c) VALUES (0, 0, 0);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 1, 1);");
> execute("INSERT INTO %s (a, b, c) VALUES (0, 2, 2);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 0, 3);");
> execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 4);");
> assertRows(execute("SELECT * FROM %s WHERE b = ?;", 1),
>row(0, 1, 1),
>row(1, 1, 4));
> }
> {code}
> The first test will fail while the second will work.
> In my opinion the first test should reject the request as invalid (as if the
> index was not existing) until the index is fully build.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10681) make index building pluggable via IndexBuildTask
[ https://issues.apache.org/jira/browse/CASSANDRA-10681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022637#comment-15022637 ] Sam Tunnicliffe commented on CASSANDRA-10681: - Alright then, if you're good with the approach of letting index impls share the support object, then lets go with that. If you want to squash the 2 commits from my branch, I'm +1 on that. > make index building pluggable via IndexBuildTask > > > Key: CASSANDRA-10681 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10681 > Project: Cassandra > Issue Type: Sub-task > Components: Local Write-Read Paths >Reporter: Pavel Yaskevich >Assignee: Pavel Yaskevich >Priority: Minor > Labels: sasi > Fix For: 3.2 > > Attachments: 0001-add-table-support-for-multi-table-builds.patch, > 0001-make-index-building-pluggable-via-IndexBuildTask.patch > > > Currently index building assumes one and only way to build all of the indexes > - through SecondaryIndexBuilder - which merges all of the sstables together, > collates columns etc. Such works fine for built-in indexes but not for SASI > since it's attaches to every SSTable individually. We need a "IndexBuildTask" > interface (based on CompactionInfo.Holder) to be returned from Index on > demand to give power to SI interface implementers to decide how build should > work. This might be less effective for CassandraIndex, since this effectively > means that collation will have to be done multiple times on the same data, > but nevertheless is a good compromise for clean interface to outside world. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10738) upgrading to v3.0.0 nodes crashes with "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous bytes after map value"
[ https://issues.apache.org/jira/browse/CASSANDRA-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024567#comment-15024567 ] Sam Tunnicliffe commented on CASSANDRA-10738: - [~iamaleksey] sure, will do that later today > upgrading to v3.0.0 nodes crashes with > "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous > bytes after map value" > --- > > Key: CASSANDRA-10738 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10738 > Project: Cassandra > Issue Type: Bug > Environment: ubuntu14, java8 >Reporter: cs >Assignee: Aleksey Yeschenko > Attachments: cassv3.txt > > > upgrading from v.2.2.3 to v3.0.0 > restarting after upgrade crashes with either > ERROR [main] 2015-11-19 20:21:26,511 CassandraDaemon.java:702 - Exception > encountered during startup > org.apache.cassandra.serializers.MarshalException: Unexpected extraneous > bytes after map value > OR > ERROR [main] 2015-11-19 20:01:13,945 CassandraDaemon.java:702 - Exception > encountered during startup > java.lang.IllegalArgumentException: null > see attached debug and schema -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-10551) Investigate JMX auth using JMXMP & SASL
[ https://issues.apache.org/jira/browse/CASSANDRA-10551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe resolved CASSANDRA-10551. - Resolution: Done Fix Version/s: (was: 3.x) I've spent a fair bit of time looking at this and I'm afraid I've come to the same conclusion. Although it's somewhat painful to use JMXMP because the Oracle JRE doesn't provide an implementation for the (optional) JMX Remote part of the JMX spec, it isn't too tricky to implement a generic SASL server to interface with IAuthenticator.SaslNegotiator. As you say though, support for JMXMP in tooling is pretty poor. I'm not so concerned with nodetool as we have control over that, plus we only ever guarantee that a given version of nodetool works with the corresponding C*. Lack of support in jconsole is a more of a concern, especially as the same is true of Java Mission Control, which would leave users who enabled JMXMP no means of interacting with the published MBeans. I'm going to close this issue, if we come across something new we can always reopen it. > Investigate JMX auth using JMXMP & SASL > --- > > Key: CASSANDRA-10551 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10551 > Project: Cassandra > Issue Type: Improvement >Reporter: Sam Tunnicliffe >Assignee: Jan Karlsson > > (broken out from CASSANDRA-10091) > We should look into whether using > [JMXMP|https://meteatamel.wordpress.com/2012/02/13/jmx-rmi-vs-jmxmp/] would > enable JMX authentication using SASL. If so, could we then define a custom > SaslServer which wraps a SaslNegotiator instance provided by the configured > IAuthenticator. > An intial look at the > [JMXMP|http://docs.oracle.com/cd/E19698-01/816-7609/6mdjrf873/] docs, > particularly section *11.4.2 SASL Provider*, suggests this might be feasible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10091) Align JMX authentication with internal authentication
[
https://issues.apache.org/jira/browse/CASSANDRA-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024697#comment-15024697
]
Sam Tunnicliffe commented on CASSANDRA-10091:
-
CASSANDRA-10551 seems to suggest that JMXMP & SASL isn't a viable option,
mostly due to lack of support in standard tooling.
I'm still not convinced that the {{IAuthorizer}} changes in the proposed patch
are the right way to go, but in the interest of making some progress here I
suggest we split out the authentication part of the patch and look at getting
something useful committed. In the meantime, we can continue to explore ways to
integrate JMX authz with the resources & permissions subsystems. [~Jan
Karlsson] how do you feel about that?
On the authentication parts of the patch, I have a couple of remarks:
* What does {{CassandraLoginModule}} give us? I appreciate that it's the
standard-ish java way to do things, but it seems to me that we could just
perform the call to {{legacyAuthenticate}} directly from
{{JMXPasswordAuthenticator::authenticate}}. The authenticator impl is already
pretty specific, so using the more generic APIs just seems to add bloat (but I
could be missing something useful here).
* The same thing goes for {{CassandraPrincipal}}, could we just create a
{{javax.management.remote.JMXPrincipal}} in the name of the
{{AuthenticatedUser}} obtained from the {{IAuthenticator}}?
* I think we probably should add an assertion, or at least a suitably
descriptive error message, that triggers when {{JMXPasswordAuthenticator}} is
used in conjunction with anything other than {{PasswordAuthenticator}} (or
possibly a subclass).
* Will MX4J work with {{JMXPasswordAuthenticator}}?
[~nickmbailey] do you have any thoughts on this?
> Align JMX authentication with internal authentication
> -
>
> Key: CASSANDRA-10091
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10091
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Jan Karlsson
>Assignee: Jan Karlsson
>Priority: Minor
> Fix For: 3.x
>
>
> It would be useful to authenticate with JMX through Cassandra's internal
> authentication. This would reduce the overhead of keeping passwords in files
> on the machine and would consolidate passwords to one location. It would also
> allow the possibility to handle JMX permissions in Cassandra.
> It could be done by creating our own JMX server and setting custom classes
> for the authenticator and authorizer. We could then add some parameters where
> the user could specify what authenticator and authorizer to use in case they
> want to make their own.
> This could also be done by creating a premain method which creates a jmx
> server. This would give us the feature without changing the Cassandra code
> itself. However I believe this would be a good feature to have in Cassandra.
> I am currently working on a solution which creates a JMX server and uses a
> custom authenticator and authorizer. It is currently build as a premain,
> however it would be great if we could put this in Cassandra instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10738) upgrading to v3.0.0 nodes crashes with "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous bytes after map value"
[
https://issues.apache.org/jira/browse/CASSANDRA-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024933#comment-15024933
]
Sam Tunnicliffe commented on CASSANDRA-10738:
-
The indexes table is a red herring. It appears in the log just before the
stacktrace because it's opened as part of the system keyspace to service the
query for keyspace names in {{SystemKeyspace::fetchKeyspacesWithout}}. The
stacktrace is definitely pointing to a problem with reading some replication
options from system_schema.keyspaces. Is the {{media}} keyspace the only one in
your schema, aside from the various system* keyspaces?
> upgrading to v3.0.0 nodes crashes with
> "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
> bytes after map value"
> ---
>
> Key: CASSANDRA-10738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10738
> Project: Cassandra
> Issue Type: Bug
> Environment: ubuntu14, java8
>Reporter: cs
>Assignee: Sam Tunnicliffe
> Attachments: cassv3.txt
>
>
> upgrading from v.2.2.3 to v3.0.0
> restarting after upgrade crashes with either
> ERROR [main] 2015-11-19 20:21:26,511 CassandraDaemon.java:702 - Exception
> encountered during startup
> org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
> bytes after map value
> OR
> ERROR [main] 2015-11-19 20:01:13,945 CassandraDaemon.java:702 - Exception
> encountered during startup
> java.lang.IllegalArgumentException: null
> see attached debug and schema
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10738) upgrading to v3.0.0 nodes crashes with "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous bytes after map value"
[ https://issues.apache.org/jira/browse/CASSANDRA-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025034#comment-15025034 ] Sam Tunnicliffe commented on CASSANDRA-10738: - bq. I can attach either/both? If you can attach both sets of sstables it would be really helpful, thanks! > upgrading to v3.0.0 nodes crashes with > "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous > bytes after map value" > --- > > Key: CASSANDRA-10738 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10738 > Project: Cassandra > Issue Type: Bug > Environment: ubuntu14, java8 >Reporter: cs >Assignee: Sam Tunnicliffe > Attachments: cassv3.txt, system_schemas.txt > > > upgrading from v.2.2.3 to v3.0.0 > restarting after upgrade crashes with either > ERROR [main] 2015-11-19 20:21:26,511 CassandraDaemon.java:702 - Exception > encountered during startup > org.apache.cassandra.serializers.MarshalException: Unexpected extraneous > bytes after map value > OR > ERROR [main] 2015-11-19 20:01:13,945 CassandraDaemon.java:702 - Exception > encountered during startup > java.lang.IllegalArgumentException: null > see attached debug and schema -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10738) upgrading to v3.0.0 nodes crashes with "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous bytes after map value"
[
https://issues.apache.org/jira/browse/CASSANDRA-10738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026819#comment-15026819
]
Sam Tunnicliffe commented on CASSANDRA-10738:
-
The post-upgrade sstables for {{system_schema.keyspaces}} you have are
definitely corrupted, I added some additional debug logging to the code that
tries to load the schema at startup and there's a load of junk in there. The
good(ish) news is I've run the same upgrade several times using your
pre-upgrade {{system.schema_keyspaces}} tables and have been unable to
reproduce the corruption. My suggestion then is to force the migration code
that converts from the old to new tables at startup to run again & see if you
get usable tables this time around.
To do that, remove all the corrupt tables from
{{data/system_schema/keyspaces-abac5682dea631c5b535b3d6cffd0fb6/}} and restore
the pre-upgrade tables (the ones in the zip you attached) to
{{data/system/schema_keyspaces-b0f2235744583cdb9631c43e59ce3676/}}. When you
restart the node, it should redo the conversion of the keyspaces table, leaving
in place the other system tables that were already converted. It's possible
there may be other corrupted sstables in the system keyspace, in which case you
should see a similar error as before. If that happens, can you attach the
debug.log, and we'll see where to go from there. It the node does start up
cleanly, you can verify the schema is correct, data accessible and so forth.
Of course, the bad news is that this tells us nothing about how those sstables
came to be corrupted in the first place.
> upgrading to v3.0.0 nodes crashes with
> "org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
> bytes after map value"
> ---
>
> Key: CASSANDRA-10738
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10738
> Project: Cassandra
> Issue Type: Bug
> Environment: ubuntu14, java8
>Reporter: Christian Sherwood
>Assignee: Sam Tunnicliffe
> Attachments: 1447894738468-upgrade-2.2.3-3.0.0.zip, cassv3.txt,
> keyspaces-abac5682dea631c5b535b3d6cffd0fb6.zip, system_schemas.txt
>
>
> upgrading from v.2.2.3 to v3.0.0
> restarting after upgrade crashes with either
> ERROR [main] 2015-11-19 20:21:26,511 CassandraDaemon.java:702 - Exception
> encountered during startup
> org.apache.cassandra.serializers.MarshalException: Unexpected extraneous
> bytes after map value
> OR
> ERROR [main] 2015-11-19 20:01:13,945 CassandraDaemon.java:702 - Exception
> encountered during startup
> java.lang.IllegalArgumentException: null
> see attached debug and schema
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-10761) Possible regression of CASSANDRA-9201
[
https://issues.apache.org/jira/browse/CASSANDRA-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe reassigned CASSANDRA-10761:
---
Assignee: Sam Tunnicliffe
> Possible regression of CASSANDRA-9201
> -
>
> Key: CASSANDRA-10761
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10761
> Project: Cassandra
> Issue Type: Sub-task
>Reporter: Philip Thompson
>Assignee: Sam Tunnicliffe
> Fix For: 3.0.1, 3.1, 2.2.x
>
> Attachments: 10761-logs.tar.gz
>
>
> Some dtests like
> {{consistency_test.TestAccuracy.test_network_topology_strategy_each_quorum_counters}}
> are failing with the follow auth related assertion exception
> {code}
> [node6 ERROR] java.lang.AssertionError:
> org.apache.cassandra.exceptions.InvalidRequestException: unconfigured table
> roles
> at
> org.apache.cassandra.auth.CassandraRoleManager.prepare(CassandraRoleManager.java:450)
> at
> org.apache.cassandra.auth.CassandraRoleManager.setup(CassandraRoleManager.java:144)
> at
> org.apache.cassandra.service.StorageService.doAuthSetup(StorageService.java:1036)
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:984)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:708)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:579)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:345)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:561)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:689)
> Caused by: org.apache.cassandra.exceptions.InvalidRequestException:
> unconfigured table roles
> at
> org.apache.cassandra.thrift.ThriftValidation.validateColumnFamily(ThriftValidation.java:114)
> at
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:757)
> at
> org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:752)
> at
> org.apache.cassandra.auth.CassandraRoleManager.prepare(CassandraRoleManager.java:446)
> ... 8 more
> {code}
> This looks very similar to CASSANDRA-9201.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10091) Integrated JMX authn & authz
[
https://issues.apache.org/jira/browse/CASSANDRA-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-10091:
Resolution: Fixed
Fix Version/s: (was: 3.x)
3.6
Status: Resolved (was: Patch Available)
Committed to trunk in {{7b0c7164aa22c156811a5d1a001c43d099aad8e4}}.In the final
CI runs, the only failures are dtests with known issues.
||branch||testall||dtest||
|[10091-trunk|https://github.com/beobal/cassandra/tree/10091-trunk]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10091-trunk-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-10091-trunk-dtest]|
> Integrated JMX authn & authz
>
>
> Key: CASSANDRA-10091
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10091
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Jan Karlsson
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 3.6
>
>
> It would be useful to authenticate with JMX through Cassandra's internal
> authentication. This would reduce the overhead of keeping passwords in files
> on the machine and would consolidate passwords to one location. It would also
> allow the possibility to handle JMX permissions in Cassandra.
> It could be done by creating our own JMX server and setting custom classes
> for the authenticator and authorizer. We could then add some parameters where
> the user could specify what authenticator and authorizer to use in case they
> want to make their own.
> This could also be done by creating a premain method which creates a jmx
> server. This would give us the feature without changing the Cassandra code
> itself. However I believe this would be a good feature to have in Cassandra.
> I am currently working on a solution which creates a JMX server and uses a
> custom authenticator and authorizer. It is currently build as a premain,
> however it would be great if we could put this in Cassandra instead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-11695) Move JMX connection config to cassandra.yaml
Sam Tunnicliffe created CASSANDRA-11695: --- Summary: Move JMX connection config to cassandra.yaml Key: CASSANDRA-11695 URL: https://issues.apache.org/jira/browse/CASSANDRA-11695 Project: Cassandra Issue Type: Improvement Components: Configuration Reporter: Sam Tunnicliffe Priority: Minor Fix For: 3.x Since CASSANDRA-10091, we always construct the JMX connector server programatically, so we could move its configuration from cassandra-env to yaml. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10091) Integrated JMX authn & authz
[ https://issues.apache.org/jira/browse/CASSANDRA-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-10091: Labels: doc-impacting (was: ) Component/s: Observability Lifecycle Configuration I've opened CASSANDRA-11695 as a followup to move JMX configuration settings to yaml, as really there's no need to manage them through system properties now we're always constructing the connector server programatically. > Integrated JMX authn & authz > > > Key: CASSANDRA-10091 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10091 > Project: Cassandra > Issue Type: New Feature > Components: Configuration, Lifecycle, Observability >Reporter: Jan Karlsson >Assignee: Sam Tunnicliffe >Priority: Minor > Labels: doc-impacting > Fix For: 3.6 > > > It would be useful to authenticate with JMX through Cassandra's internal > authentication. This would reduce the overhead of keeping passwords in files > on the machine and would consolidate passwords to one location. It would also > allow the possibility to handle JMX permissions in Cassandra. > It could be done by creating our own JMX server and setting custom classes > for the authenticator and authorizer. We could then add some parameters where > the user could specify what authenticator and authorizer to use in case they > want to make their own. > This could also be done by creating a premain method which creates a jmx > server. This would give us the feature without changing the Cassandra code > itself. However I believe this would be a good feature to have in Cassandra. > I am currently working on a solution which creates a JMX server and uses a > custom authenticator and authorizer. It is currently build as a premain, > however it would be great if we could put this in Cassandra instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8094) Heavy writes in RangeSlice read requests
[
https://issues.apache.org/jira/browse/CASSANDRA-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269018#comment-15269018
]
Sam Tunnicliffe commented on CASSANDRA-8094:
Coming back to look at this now, I think that using {{extensions}} to persist
the param is just too hacky to be considered a reasonable approach. Given that,
we should either defer this ticket until 4.0 or prioritize CASSANDRA-11382,
using this as a driver towards an implementation.
> Heavy writes in RangeSlice read requests
> --
>
> Key: CASSANDRA-8094
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8094
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Minh Do
>Assignee: Sam Tunnicliffe
> Labels: lhf
> Fix For: 3.x
>
>
> RangeSlice requests always do a scheduled read repair when coordinators try
> to resolve replicas' responses no matter read_repair_chance is set or not.
> Because of this, in low writes and high reads clusters, there are very high
> write requests going on between nodes.
> We should have an option to turn this off and this can be different than the
> read_repair_chance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8094) Heavy writes in RangeSlice read requests
[ https://issues.apache.org/jira/browse/CASSANDRA-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-8094: --- Status: Open (was: Patch Available) > Heavy writes in RangeSlice read requests > -- > > Key: CASSANDRA-8094 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8094 > Project: Cassandra > Issue Type: Improvement >Reporter: Minh Do >Assignee: Sam Tunnicliffe > Labels: lhf > Fix For: 3.x > > > RangeSlice requests always do a scheduled read repair when coordinators try > to resolve replicas' responses no matter read_repair_chance is set or not. > Because of this, in low writes and high reads clusters, there are very high > write requests going on between nodes. > We should have an option to turn this off and this can be different than the > read_repair_chance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10635) Add metrics for authentication failures
[ https://issues.apache.org/jira/browse/CASSANDRA-10635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269221#comment-15269221 ] Sam Tunnicliffe commented on CASSANDRA-10635: - [~soumava], did you get a chance to revisit this? > Add metrics for authentication failures > --- > > Key: CASSANDRA-10635 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10635 > Project: Cassandra > Issue Type: Improvement >Reporter: Soumava Ghosh >Assignee: Soumava Ghosh >Priority: Minor > Fix For: 3.x > > Attachments: 10635-2.1.txt, 10635-2.2.txt, 10635-3.0.txt > > > There should be no auth failures on a cluster in general. > Having metrics around the authentication code would help detect clients > that are connecting to the wrong cluster or have auth incorrectly configured. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11540) The JVM should exit if jmx fails to bind
[
https://issues.apache.org/jira/browse/CASSANDRA-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-11540:
Reviewer: Sam Tunnicliffe (was: Joshua McKenzie)
> The JVM should exit if jmx fails to bind
>
>
> Key: CASSANDRA-11540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11540
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Brandon Williams
>Assignee: Alex Petrov
> Labels: lhf
> Fix For: 2.2.x, 3.0.x, 3.x
>
>
> If you are already running a cassandra instance, but for some reason try to
> start another one, this happens:
> {noformat}
> INFO 20:57:09 JNA mlockall successful
> WARN 20:57:09 JMX is not enabled to receive remote connections. Please see
> cassandra-env.sh for more info.
> ERROR 20:57:10 Error starting local jmx server:
> java.rmi.server.ExportException: Port already in use: 7199; nested exception
> is:
> java.net.BindException: Address already in use
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:340)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:248)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
> ~[na:1.7.0_76]
> at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
> ~[na:1.7.0_76]
> at
> sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207)
> ~[na:1.7.0_76]
> at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:122)
> ~[na:1.7.0_76]
> at sun.rmi.registry.RegistryImpl.(RegistryImpl.java:98)
> ~[na:1.7.0_76]
> at
> java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239)
> ~[na:1.7.0_76]
> at
> org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:100)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:222)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653)
> [main/:na]
> Caused by: java.net.BindException: Address already in use
> at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.7.0_76]
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
> ~[na:1.7.0_76]
> at java.net.ServerSocket.bind(ServerSocket.java:376) ~[na:1.7.0_76]
> at java.net.ServerSocket.(ServerSocket.java:237) ~[na:1.7.0_76]
> at
> javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231)
> ~[na:1.7.0_76]
> at
> org.apache.cassandra.utils.RMIServerSocketFactoryImpl.createServerSocket(RMIServerSocketFactoryImpl.java:13)
> ~[main/:na]
> at
> sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
> ~[na:1.7.0_76]
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:329)
> ~[na:1.7.0_76]
> ... 11 common frames omitted
> {noformat}
> However the startup continues, and ends up replaying commitlogs, which is
> probably not a good thing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11540) The JVM should exit if jmx fails to bind
[
https://issues.apache.org/jira/browse/CASSANDRA-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-11540:
Resolution: Fixed
Fix Version/s: (was: 3.0.x)
(was: 2.2.x)
(was: 3.x)
3.0.7
3.7
2.2.7
Status: Resolved (was: Patch Available)
CI looks good, so committed to 2.2 in
{{93c5bc616e21ffa7f31266ad095ca374f2ba73a4}} and merged to 3.0/3.7/trunk.
> The JVM should exit if jmx fails to bind
>
>
> Key: CASSANDRA-11540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11540
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Brandon Williams
>Assignee: Alex Petrov
> Labels: lhf
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> If you are already running a cassandra instance, but for some reason try to
> start another one, this happens:
> {noformat}
> INFO 20:57:09 JNA mlockall successful
> WARN 20:57:09 JMX is not enabled to receive remote connections. Please see
> cassandra-env.sh for more info.
> ERROR 20:57:10 Error starting local jmx server:
> java.rmi.server.ExportException: Port already in use: 7199; nested exception
> is:
> java.net.BindException: Address already in use
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:340)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:248)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
> ~[na:1.7.0_76]
> at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
> ~[na:1.7.0_76]
> at
> sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207)
> ~[na:1.7.0_76]
> at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:122)
> ~[na:1.7.0_76]
> at sun.rmi.registry.RegistryImpl.(RegistryImpl.java:98)
> ~[na:1.7.0_76]
> at
> java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239)
> ~[na:1.7.0_76]
> at
> org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:100)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:222)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653)
> [main/:na]
> Caused by: java.net.BindException: Address already in use
> at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.7.0_76]
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
> ~[na:1.7.0_76]
> at java.net.ServerSocket.bind(ServerSocket.java:376) ~[na:1.7.0_76]
> at java.net.ServerSocket.(ServerSocket.java:237) ~[na:1.7.0_76]
> at
> javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231)
> ~[na:1.7.0_76]
> at
> org.apache.cassandra.utils.RMIServerSocketFactoryImpl.createServerSocket(RMIServerSocketFactoryImpl.java:13)
> ~[main/:na]
> at
> sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
> ~[na:1.7.0_76]
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:329)
> ~[na:1.7.0_76]
> ... 11 common frames omitted
> {noformat}
> However the startup continues, and ends up replaying commitlogs, which is
> probably not a good thing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-11716) cassandra 2.2 fails to start on jdk7u101
[
https://issues.apache.org/jira/browse/CASSANDRA-11716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe resolved CASSANDRA-11716.
-
Resolution: Duplicate
> cassandra 2.2 fails to start on jdk7u101
>
>
> Key: CASSANDRA-11716
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11716
> Project: Cassandra
> Issue Type: Bug
> Environment: ubuntu 12.04 fully updated
>Reporter: Tiago Batista
> Attachments: cassandra-env.patch
>
>
> Today I updated one of my clusters to 2.2.6, and was greeted with the message
> complaining about the jdk version:
> {code}
> $ nodetool status
> Cassandra 2.0 and later require Java 7u25 or later.
> {code}
> After digging into it, on cassandra-env.sh, i found that you are comparing
> the patch levels as strings, meaning that "101" is before "25":
> {code}
> if [ "$JVM_VERSION" \< "1.8" ] && [ "$JVM_PATCH_VERSION" \< "25" ] ; then
> echo "Cassandra 2.0 and later require Java 7u25 or later."
> exit 1;
> fi
> {code}
> I patched this on my system to
> {code}
> if [ "$JVM_VERSION" \< "1.8" ] && [ "$JVM_PATCH_VERSION" -lt "25" ] ; then
> echo "Cassandra 2.0 and later require Java 7u25 or later."
> exit 1;
> fi
> {code}
> this seems to work on bash. I can now start cassandra successfully on that
> cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-11725) Check for unnecessary JMX port setting in env vars at startup
Sam Tunnicliffe created CASSANDRA-11725:
---
Summary: Check for unnecessary JMX port setting in env vars at
startup
Key: CASSANDRA-11725
URL: https://issues.apache.org/jira/browse/CASSANDRA-11725
Project: Cassandra
Issue Type: Improvement
Components: Lifecycle
Reporter: Sam Tunnicliffe
Priority: Minor
Fix For: 3.x
Since CASSANDRA-10091, C* expects to always be in control of initializing its
JMX connector server. However, if {{com.sun.management.jmxremote.port}} is set
when the JVM is started, the bootstrap agent takes over and sets up the server
before any C* code runs. Because C* is then unable to bind the server it
creates to the specified port, startup is halted and the root cause is somewhat
unclear.
We should add a check at startup so a more informative message can be provided.
This would test for the presence of the system property which would
differentiate from the case where some other process is already bound to the
port.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11540) The JVM should exit if jmx fails to bind
[
https://issues.apache.org/jira/browse/CASSANDRA-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273755#comment-15273755
]
Sam Tunnicliffe commented on CASSANDRA-11540:
-
If the {{com.sun.management.jmxremote.port}} property is set, the JVM
automatically uses an inbuilt agent to initialize the JMX connector server. The
'expected' way to configure the JMX server is to modify {{cassandra-env.sh}}
with the desired port/ssl/auth settings and in fact setting up for remote
connections actually just sets those same properties. As you see though,
passing them directly via {{JVM_ARGS}} also worked without error (bypassing the
setup in {{cassandra-env.sh}}) as C* would interpret the presence of that
property to indicate that the JMX server had already been setup and wouldn't
attempt to initialize it; I'm afraid I overlooked this in CASSANDRA-10091.
Since CASSANDRA-10091, C* expects to always be in control of JMX configuration.
{{cassandra-env.sh}} has been modified so that
{{com.sun.management.jmxremote.port}} is never set (it's replaced by
{{cassandra.jmx.remote.port}}). On top of that, CASSANDRA-11540 makes C* setup
error out in the face of JMX bind errors, whereas previously it would not.
bq. I just found the config change that helps out with my env situation
That will create slightly different behaviour, in that the JMX server will only
listen on the loopback address but your previous setup would bind it to
{{0.0.0.0}}. If you want to still support remote JMX, you should set
{{cassandra.jmx.remote.port}} instead.
I've added CASSANDRA-11725 to provide a clearer message when this kind of
misconfiguration occurs.
> The JVM should exit if jmx fails to bind
>
>
> Key: CASSANDRA-11540
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11540
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Brandon Williams
>Assignee: Alex Petrov
> Labels: lhf
> Fix For: 2.2.7, 3.7, 3.0.7
>
>
> If you are already running a cassandra instance, but for some reason try to
> start another one, this happens:
> {noformat}
> INFO 20:57:09 JNA mlockall successful
> WARN 20:57:09 JMX is not enabled to receive remote connections. Please see
> cassandra-env.sh for more info.
> ERROR 20:57:10 Error starting local jmx server:
> java.rmi.server.ExportException: Port already in use: 7199; nested exception
> is:
> java.net.BindException: Address already in use
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:340)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPTransport.exportObject(TCPTransport.java:248)
> ~[na:1.7.0_76]
> at
> sun.rmi.transport.tcp.TCPEndpoint.exportObject(TCPEndpoint.java:411)
> ~[na:1.7.0_76]
> at sun.rmi.transport.LiveRef.exportObject(LiveRef.java:147)
> ~[na:1.7.0_76]
> at
> sun.rmi.server.UnicastServerRef.exportObject(UnicastServerRef.java:207)
> ~[na:1.7.0_76]
> at sun.rmi.registry.RegistryImpl.setup(RegistryImpl.java:122)
> ~[na:1.7.0_76]
> at sun.rmi.registry.RegistryImpl.(RegistryImpl.java:98)
> ~[na:1.7.0_76]
> at
> java.rmi.registry.LocateRegistry.createRegistry(LocateRegistry.java:239)
> ~[na:1.7.0_76]
> at
> org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:100)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:222)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:564)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:653)
> [main/:na]
> Caused by: java.net.BindException: Address already in use
> at java.net.PlainSocketImpl.socketBind(Native Method) ~[na:1.7.0_76]
> at
> java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
> ~[na:1.7.0_76]
> at java.net.ServerSocket.bind(ServerSocket.java:376) ~[na:1.7.0_76]
> at java.net.ServerSocket.(ServerSocket.java:237) ~[na:1.7.0_76]
> at
> javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:231)
> ~[na:1.7.0_76]
> at
> org.apache.cassandra.utils.RMIServerSocketFactoryImpl.createServerSocket(RMIServerSocketFactoryImpl.java:13)
> ~[main/:na]
> at
> sun.rmi.transport.tcp.TCPEndpoint.newServerSocket(TCPEndpoint.java:666)
> ~[na:1.7.0_76]
> at sun.rmi.transport.tcp.TCPTransport.listen(TCPTransport.java:329)
> ~[na:1.7.0_76]
> ... 11 common frames omitted
> {noformat}
> However the startup continues, and ends up replaying commitlogs, which is
> probably not a good thing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-11735) cassandra-env.sh doesn't test the correct java version
[ https://issues.apache.org/jira/browse/CASSANDRA-11735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe resolved CASSANDRA-11735. - Resolution: Duplicate > cassandra-env.sh doesn't test the correct java version > -- > > Key: CASSANDRA-11735 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11735 > Project: Cassandra > Issue Type: Bug > Environment: Ubuntu 14.04 > openjdk 7 patch >=100 >Reporter: Maxime Bugeia >Priority: Minor > > With the latest patch of openjdk, all nodetool actions fail and display > "Cassandra 2.0 and later require Java 7u25 or later." because > cassandra-env.sh test of java version is broken. > Line 102: > if [ "$JVM_VERSION" \< "1.7" ] && [ "$JVM_PATCH_VERSION" \< "25" ] ; then > echo "Cassandra 2.0 and later require Java 7u25 or later." > exit 1; > fi > The second test cause all java patch >100 to be considered as inferior. One > correct syntax is "-lt" instead of "\<". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11725) Check for unnecessary JMX port setting in env vars at startup
[
https://issues.apache.org/jira/browse/CASSANDRA-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-11725:
Assignee: Sam Tunnicliffe
Reviewer: T Jake Luciani
Status: Patch Available (was: Open)
I've pushed a branch which adds a {{StartupCheck}} that warns if
{{com.sun.management.jmxremote.port}} is set.
As this issue also exposed the fact that a number of clients where relying on
passing the JMX config directly via system properties, rather than using
{{cassandra-env.sh}}, I've made startup more permissive to emulate previous
behaviour. So, if the property is present, when it C* comes to init the JMX
server it will log an additional warning and skip the setup.
The additional warning is because at some point, we should remove this
compatibility mode and go back to failing startup if the {{jmxremote.port}} is
set directly, but we'll the {{StartupCheck}} should remain.
||branch||testall||dtest||
|[11725-3.6|https://github.com/beobal/cassandra/tree/11725-3.6]|[testall|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-11725-3.6-testall]|[dtest|http://cassci.datastax.com/view/Dev/view/beobal/job/beobal-11725-3.6-dtest]|
> Check for unnecessary JMX port setting in env vars at startup
> -
>
> Key: CASSANDRA-11725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11725
> Project: Cassandra
> Issue Type: Improvement
> Components: Lifecycle
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Labels: lhf
> Fix For: 3.x
>
>
> Since CASSANDRA-10091, C* expects to always be in control of initializing its
> JMX connector server. However, if {{com.sun.management.jmxremote.port}} is
> set when the JVM is started, the bootstrap agent takes over and sets up the
> server before any C* code runs. Because C* is then unable to bind the server
> it creates to the specified port, startup is halted and the root cause is
> somewhat unclear.
> We should add a check at startup so a more informative message can be
> provided. This would test for the presence of the system property which would
> differentiate from the case where some other process is already bound to the
> port.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11725) Check for unnecessary JMX port setting in env vars at startup
[
https://issues.apache.org/jira/browse/CASSANDRA-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278122#comment-15278122
]
Sam Tunnicliffe commented on CASSANDRA-11725:
-
Overriding it *only in the local-only case* as the branch does now it the exact
behaviour we had before CASSANDRA-10091.
Trying to binding to that hostname is the wrong thing to do whatever, so I've
just removed that for non-local connections. Ultimately, we can bind to the
broadcast address (CASSANDRA-2967), but I've left it binding to {{0.0.0.0}} in
the non-local case for now, just in case there are other tools/clients
depending on that behaviour.
> Check for unnecessary JMX port setting in env vars at startup
> -
>
> Key: CASSANDRA-11725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11725
> Project: Cassandra
> Issue Type: Improvement
> Components: Lifecycle
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Labels: lhf
> Fix For: 3.x
>
>
> Since CASSANDRA-10091, C* expects to always be in control of initializing its
> JMX connector server. However, if {{com.sun.management.jmxremote.port}} is
> set when the JVM is started, the bootstrap agent takes over and sets up the
> server before any C* code runs. Because C* is then unable to bind the server
> it creates to the specified port, startup is halted and the root cause is
> somewhat unclear.
> We should add a check at startup so a more informative message can be
> provided. This would test for the presence of the system property which would
> differentiate from the case where some other process is already bound to the
> port.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11725) Check for unnecessary JMX port setting in env vars at startup
[
https://issues.apache.org/jira/browse/CASSANDRA-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-11725:
Resolution: Fixed
Fix Version/s: (was: 3.x)
3.6
Status: Resolved (was: Patch Available)
Thanks, committed to 3.6 in {{c17cbe1875a974a00822ffbfad716abde363c8da}} and
merged to 3.7 & trunk.
> Check for unnecessary JMX port setting in env vars at startup
> -
>
> Key: CASSANDRA-11725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11725
> Project: Cassandra
> Issue Type: Improvement
> Components: Lifecycle
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Labels: lhf
> Fix For: 3.6
>
>
> Since CASSANDRA-10091, C* expects to always be in control of initializing its
> JMX connector server. However, if {{com.sun.management.jmxremote.port}} is
> set when the JVM is started, the bootstrap agent takes over and sets up the
> server before any C* code runs. Because C* is then unable to bind the server
> it creates to the specified port, startup is halted and the root cause is
> somewhat unclear.
> We should add a check at startup so a more informative message can be
> provided. This would test for the presence of the system property which would
> differentiate from the case where some other process is already bound to the
> port.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11725) Check for unnecessary JMX port setting in env vars at startup
[
https://issues.apache.org/jira/browse/CASSANDRA-11725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-11725:
Issue Type: Bug (was: Improvement)
> Check for unnecessary JMX port setting in env vars at startup
> -
>
> Key: CASSANDRA-11725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11725
> Project: Cassandra
> Issue Type: Bug
> Components: Lifecycle
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Labels: lhf
> Fix For: 3.6
>
>
> Since CASSANDRA-10091, C* expects to always be in control of initializing its
> JMX connector server. However, if {{com.sun.management.jmxremote.port}} is
> set when the JVM is started, the bootstrap agent takes over and sets up the
> server before any C* code runs. Because C* is then unable to bind the server
> it creates to the specified port, startup is halted and the root cause is
> somewhat unclear.
> We should add a check at startup so a more informative message can be
> provided. This would test for the presence of the system property which would
> differentiate from the case where some other process is already bound to the
> port.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11749) CQLSH gets SSL exception following a COPY FROM
[
https://issues.apache.org/jira/browse/CASSANDRA-11749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280220#comment-15280220
]
Sam Tunnicliffe commented on CASSANDRA-11749:
-
I don't I'm afraid & as far as I am aware, {{bad_record_mac}} is basically a
catch-all for any error with the encryption handshake. I did find a [netty
issue|https://github.com/netty/netty/issues/4116] presenting similar symptoms,
but it was only affecting Android clients. Perhaps [~norman] has more of an
idea?
One other thing, in {{stdout.txt}} there do seem to be quite a few warnings
prior to that too which suggest the client is shutting down the connection
prematurely, but I don't know if that's at all relevant:
{code}
epollEventLoopGroup-2-4, fatal error: 80: Inbound closed before receiving
peer's close_notify: possible truncation attack?
javax.net.ssl.SSLException: Inbound closed before receiving peer's
close_notify: possible truncation attack?
{code}
> CQLSH gets SSL exception following a COPY FROM
> --
>
> Key: CASSANDRA-11749
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11749
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Stefania
>Assignee: Stefania
> Fix For: 2.1.x
>
> Attachments: stdout.txt.zip
>
>
> When running Cassandra and cqlsh with SSL, the following command occasionally
> results in the exception below:
> {code}
> cqlsh --ssl -f kv.cql
> {code}
> {code}
> ERROR [SharedPool-Worker-2] 2016-05-11 12:41:03,583 Message.java:538 -
> Unexpected exception during request; channel = [id: 0xeb75e05d,
> /127.0.0.1:51083 => /127.0.0.1:9042]
> io.netty.handler.codec.DecoderException: javax.net.ssl.SSLException: bad
> record MAC
> at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:280)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:149)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.epollInReady(EpollSocketChannel.java:722)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:326)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> Caused by: javax.net.ssl.SSLException: bad record MAC
> at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
> ~[na:1.8.0_91]
> at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728)
> ~[na:1.8.0_91]
> at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:981)
> ~[na:1.8.0_91]
> at
> sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907)
> ~[na:1.8.0_91]
> at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781)
> ~[na:1.8.0_91]
> at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[na:1.8.0_91]
> at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:982)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:908)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:854)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:249)
> ~[netty-all-4.0.23.Final.jar:4.0.23.Final]
> ... 10 common frames omitted
> Caused by: javax.crypto.BadPaddingException: bad record MAC
> at sun.security.ssl.InputRecord.decrypt(InputRecord.java:219)
> ~[na:1.8.0_91]
> at
> sun.security.ssl.EngineInputRecord.decrypt(EngineInputRecord.java:177)
> ~[na:1.8.0_91]
> at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineIm
[jira] [Updated] (CASSANDRA-11038) Is node being restarted treated as node joining?
[
https://issues.apache.org/jira/browse/CASSANDRA-11038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-11038:
Summary: Is node being restarted treated as node joining? (was: t)
> Is node being restarted treated as node joining?
>
>
> Key: CASSANDRA-11038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11038
> Project: Cassandra
> Issue Type: Bug
> Components: Distributed Metadata
>Reporter: cheng ren
>Assignee: Sam Tunnicliffe
>Priority: Minor
>
> Hi,
> What we found recently is that every time we restart a node, all other nodes
> in the cluster treat the restarted node as a new node joining and issue node
> joining notification to clients. We have traced the code path being hit when
> a peer node detected a restarted node:
> src/java/org/apache/cassandra/gms/Gossiper.java
> {code}
> private void handleMajorStateChange(InetAddress ep, EndpointState epState)
> {
> if (!isDeadState(epState))
> {
> if (endpointStateMap.get(ep) != null)
> logger.info("Node {} has restarted, now UP", ep);
> else
> logger.info("Node {} is now part of the cluster", ep);
> }
> if (logger.isTraceEnabled())
> logger.trace("Adding endpoint state for " + ep);
> endpointStateMap.put(ep, epState);
> // the node restarted: it is up to the subscriber to take whatever
> action is necessary
> for (IEndpointStateChangeSubscriber subscriber : subscribers)
> subscriber.onRestart(ep, epState);
> if (!isDeadState(epState))
> markAlive(ep, epState);
> else
> {
> logger.debug("Not marking " + ep + " alive due to dead state");
> markDead(ep, epState);
> }
> for (IEndpointStateChangeSubscriber subscriber : subscribers)
> subscriber.onJoin(ep, epState);
> }
> {code}
> subscriber.onJoin(ep, epState) ends up with calling onJoinCluster in
> Server.java
> {code}
> src/java/org/apache/cassandra/transport/Server.java
> public void onJoinCluster(InetAddress endpoint)
> {
> server.connectionTracker.send(Event.TopologyChange.newNode(getRpcAddress(endpoint),
> server.socket.getPort()));
> }
> {code}
> We have a full trace of code path and skip some intermedia function calls
> here for being brief.
> Upon receiving the node joining notification, clients would go and scan
> system peer table to fetch the latest topology information. Since we have
> tens of thousands of client connections, scans from all of them put an
> enormous load to our cluster.
> Although in the newer version of driver, client skips fetching peer table if
> the new node has already existed in local metadata, we are still curious why
> node being restarted is handled as node joining on server side? Did we hit a
> bug or this is the way supposed to be? Our old java driver version is 1.0.4
> and cassandra version is 2.0.12.
> Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
