[jira] [Updated] (CASSANDRA-15076) Align record header of FQL and audit binary log
[
https://issues.apache.org/jira/browse/CASSANDRA-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Per Otterström updated CASSANDRA-15076:
---
Issue Type: Improvement (was: Sub-task)
Parent: (was: CASSANDRA-14772)
> Align record header of FQL and audit binary log
> ---
>
> Key: CASSANDRA-15076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15076
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/fql
>Reporter: Per Otterström
>Priority: Normal
> Fix For: 4.0
>
>
> The new full query logger and the audit logger support logging into binary
> Chronicle logs. Both create records with a small header to indicate what
> follows, but the two features have adopted different header formats. Let's
> align the record header format with this ticket.
> * Both features should use the same header layout. This makes it possible to
> give more user friendly error messages in the {{fqltool}} and
> {{auditlogviewr}} commands.
> * The record header should have a distinct {{type}} to indicate the type of
> record.
> * The record header should have a {{version}} so that the record format can
> evolve.
> Current record header format of the FQL is:
> {noformat}
> version:0(int16)
> type:(text)
> {noformat}
> where {{}} can be either {{batch}} or {{single-query}}.
> Current record header format of the binary audit log is:
> {noformat}
> type:AuditLog(text)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14772) Fix issues in audit / full query log interactions
[
https://issues.apache.org/jira/browse/CASSANDRA-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809574#comment-16809574
]
Per Otterström commented on CASSANDRA-14772:
Created CASSANDRA-15076 to discuss record header format.
> Fix issues in audit / full query log interactions
> -
>
> Key: CASSANDRA-14772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14772
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL, Legacy/Tools
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0
>
>
> There are some problems with the audit + full query log code that need to be
> resolved before 4.0 is released:
> * Fix performance regression in FQL that makes it less usable than it should
> be.
> * move full query log specific code to a separate package
> * do some audit log class renames (I keep reading {{BinLogAuditLogger}} vs
> {{BinAuditLogger}} wrong for example)
> * avoid parsing the CQL queries twice in {{QueryMessage}} when audit log is
> enabled.
> * add a new tool to dump audit logs (ie, let fqltool be full query log
> specific). fqltool crashes when pointed to them.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14983) Local reads potentially blocking remote reads
[
https://issues.apache.org/jira/browse/CASSANDRA-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809691#comment-16809691
]
Benedict commented on CASSANDRA-14983:
--
[~molsson] very sorry for taking so long to look at this patch, or to respond.
Thank you for producing the patch - I am looking forward to reviewing it soon.
I'm afraid I still don't quite have time - I hope to get to it in the next
couple of weeks.
The best way I can think of to break a local request, without modifying the
underlying codebase, would be to either register a java-based
FileSystemProvider that we can break, but this would also require changes to
the codebase, to use URIs instead of paths for the filesystem interactions.
This might not be too onerous, or might be really awful to undertake. But it
would offer us really powerful control over testing of filesystem interactions.
An alternative might be to simply use a pre-existing outside-of-JVM approach.
ScyllaDB seem to have written a [fault-injecting filesystem for
Linux|https://www.scylladb.com/2016/05/02/fault-injection-filesystem-cookbook/].
This would work in CircleCI and on users developing under Linux, which is
better than nothing, and might be lower effort to introduce.
> Local reads potentially blocking remote reads
> -
>
> Key: CASSANDRA-14983
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14983
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Marcus Olsson
>Priority: Low
> Attachments: graph_local_read.html, graph_local_read_trunk.html,
> local_read_trace.log
>
>
> Since CASSANDRA-4718 there is a fast path allowing local requests to continue
> to [work in the same
> thread|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/reads/AbstractReadExecutor.java#L157]
> rather than being sent over to the read stage.
> Based on the comment
> {code:java}
> // We delay the local (potentially blocking) read till the end to avoid
> stalling remote requests.
> {code}
> it seems like this should be performed last in the chain to avoid blocking
> remote requests but that does not seem to be the case when the local request
> is a data request. The digest request(s) are sent after the data requests are
> sent (and now the transient replica requests as well). When the fast path is
> used for local data/transient data requests this will block the next type of
> request from being sent away until the local read is finished and add
> additional latency to the request.
> In addition to this it seems like local requests are *always* data requests
> (might not be a problem), but the log message can say either ["digest" or
> "data"|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/reads/AbstractReadExecutor.java#L156]
> as the type of request.
> I have tried to run performance measurements to see the impact of this in 3.0
> (by moving local requests to the end of ARE#executeAsync()) but I haven't
> seen any big difference yet. I'll continue to run some more tests to see if I
> can find a use case affected by this.
> Attaching a trace (3.0) where this happens. Reproduction:
> # Create a three node CCM cluster
> # Provision data with stress (rf=3)
> # In parallel:
> ## Start stress read run
> ## Run multiple manual read queries in cqlsh with tracing on and
> local_quorum (as this does not always happen)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14983) Local reads potentially blocking remote reads
[
https://issues.apache.org/jira/browse/CASSANDRA-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict reassigned CASSANDRA-14983:
Assignee: Marcus Olsson
Reviewers: Benedict
> Local reads potentially blocking remote reads
> -
>
> Key: CASSANDRA-14983
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14983
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Low
> Attachments: graph_local_read.html, graph_local_read_trunk.html,
> local_read_trace.log
>
>
> Since CASSANDRA-4718 there is a fast path allowing local requests to continue
> to [work in the same
> thread|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/reads/AbstractReadExecutor.java#L157]
> rather than being sent over to the read stage.
> Based on the comment
> {code:java}
> // We delay the local (potentially blocking) read till the end to avoid
> stalling remote requests.
> {code}
> it seems like this should be performed last in the chain to avoid blocking
> remote requests but that does not seem to be the case when the local request
> is a data request. The digest request(s) are sent after the data requests are
> sent (and now the transient replica requests as well). When the fast path is
> used for local data/transient data requests this will block the next type of
> request from being sent away until the local read is finished and add
> additional latency to the request.
> In addition to this it seems like local requests are *always* data requests
> (might not be a problem), but the log message can say either ["digest" or
> "data"|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/reads/AbstractReadExecutor.java#L156]
> as the type of request.
> I have tried to run performance measurements to see the impact of this in 3.0
> (by moving local requests to the end of ARE#executeAsync()) but I haven't
> seen any big difference yet. I'll continue to run some more tests to see if I
> can find a use case affected by this.
> Attaching a trace (3.0) where this happens. Reproduction:
> # Create a three node CCM cluster
> # Provision data with stress (rf=3)
> # In parallel:
> ## Start stress read run
> ## Run multiple manual read queries in cqlsh with tracing on and
> local_quorum (as this does not always happen)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15022) SEPExecutor local requests not triggered often
[
https://issues.apache.org/jira/browse/CASSANDRA-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benedict updated CASSANDRA-15022:
-
Reviewers: Benedict
> SEPExecutor local requests not triggered often
> --
>
> Key: CASSANDRA-15022
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15022
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Marcus Olsson
>Priority: Low
>
> Based on observations done in
> [CASSANDRA-14983|https://issues.apache.org/jira/browse/CASSANDRA-14983?focusedCommentId=16745238&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16745238]
> SEPExecutor#maybeExecuteImmediately() does not necessarily run tasks in the
> same thread often.
> The logic [as it is
> today|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/concurrent/SEPExecutor.java#L166]
> basically requires another task to be in the work queue for it to trigger.
> One option to make it trigger more often could be to change the logic to:
> {code:java}
> if (workPermits == 0 || (takeTaskPermit && taskPermits == 0))
> return false;
> {code}
> So that we only check the taskPermits if we want to take a taskPermit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14983) Local reads potentially blocking remote reads
[
https://issues.apache.org/jira/browse/CASSANDRA-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809691#comment-16809691
]
Benedict edited comment on CASSANDRA-14983 at 4/4/19 10:07 AM:
---
[~molsson] very sorry for taking so long to look at this patch, or to respond.
Thank you for producing the patch - I am looking forward to reviewing it soon.
I'm afraid I still don't quite have time - I hope to get to it in the next
couple of weeks.
The best way I can think of to break a local request, without modifying the
underlying codebase, would be one of two possibilities:
1) Register a java-based FileSystemProvider that we can break, but this would
also require changes to the codebase, to use URIs instead of paths for the
filesystem interactions. This might not be too onerous, or might be really
awful to undertake. But it would offer us really powerful control over testing
of filesystem interactions.
2) Use a pre-existing outside-of-JVM approach. ScyllaDB seem to have written a
[fault-injecting filesystem for
Linux|https://www.scylladb.com/2016/05/02/fault-injection-filesystem-cookbook/].
This would work in CircleCI and on users developing under Linux, which is
better than nothing, and might be lower effort to introduce.
was (Author: benedict):
[~molsson] very sorry for taking so long to look at this patch, or to respond.
Thank you for producing the patch - I am looking forward to reviewing it soon.
I'm afraid I still don't quite have time - I hope to get to it in the next
couple of weeks.
The best way I can think of to break a local request, without modifying the
underlying codebase, would be to either register a java-based
FileSystemProvider that we can break, but this would also require changes to
the codebase, to use URIs instead of paths for the filesystem interactions.
This might not be too onerous, or might be really awful to undertake. But it
would offer us really powerful control over testing of filesystem interactions.
An alternative might be to simply use a pre-existing outside-of-JVM approach.
ScyllaDB seem to have written a [fault-injecting filesystem for
Linux|https://www.scylladb.com/2016/05/02/fault-injection-filesystem-cookbook/].
This would work in CircleCI and on users developing under Linux, which is
better than nothing, and might be lower effort to introduce.
> Local reads potentially blocking remote reads
> -
>
> Key: CASSANDRA-14983
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14983
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Low
> Attachments: graph_local_read.html, graph_local_read_trunk.html,
> local_read_trace.log
>
>
> Since CASSANDRA-4718 there is a fast path allowing local requests to continue
> to [work in the same
> thread|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/reads/AbstractReadExecutor.java#L157]
> rather than being sent over to the read stage.
> Based on the comment
> {code:java}
> // We delay the local (potentially blocking) read till the end to avoid
> stalling remote requests.
> {code}
> it seems like this should be performed last in the chain to avoid blocking
> remote requests but that does not seem to be the case when the local request
> is a data request. The digest request(s) are sent after the data requests are
> sent (and now the transient replica requests as well). When the fast path is
> used for local data/transient data requests this will block the next type of
> request from being sent away until the local read is finished and add
> additional latency to the request.
> In addition to this it seems like local requests are *always* data requests
> (might not be a problem), but the log message can say either ["digest" or
> "data"|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/reads/AbstractReadExecutor.java#L156]
> as the type of request.
> I have tried to run performance measurements to see the impact of this in 3.0
> (by moving local requests to the end of ARE#executeAsync()) but I haven't
> seen any big difference yet. I'll continue to run some more tests to see if I
> can find a use case affected by this.
> Attaching a trace (3.0) where this happens. Reproduction:
> # Create a three node CCM cluster
> # Provision data with stress (rf=3)
> # In parallel:
> ## Start stress read run
> ## Run multiple manual read queries in cqlsh with tracing on and
> local_quorum (as this does not always happen)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: co
[jira] [Comment Edited] (CASSANDRA-14983) Local reads potentially blocking remote reads
[
https://issues.apache.org/jira/browse/CASSANDRA-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809691#comment-16809691
]
Benedict edited comment on CASSANDRA-14983 at 4/4/19 10:10 AM:
---
[~molsson] very sorry for taking so long to look at this patch, or to respond.
Thank you for producing the patch - I am looking forward to reviewing it soon.
I'm afraid I still don't quite have time - I hope to get to it in the next
couple of weeks.
The best way I can think of to break a local request, without modifying the
underlying codebase, would be one of two possibilities:
1) Register a java-based FileSystemProvider that we can break, but this would
also require changes to the codebase, to use URIs instead of paths for the
filesystem interactions. This might not be too onerous, or might be really
awful to undertake. But it would offer us really powerful control over testing
of filesystem interactions.
2) Use a pre-existing outside-of-JVM approach. ScyllaDB seem to have written a
[fault-injecting filesystem for
Linux|https://www.scylladb.com/2016/05/02/fault-injection-filesystem-cookbook/].
This would work in CircleCI and on users developing under Linux, which is
better than nothing, and might be lower effort to introduce.
3) Byteman is also an option for a single fault injection, but in my experience
it is really unpleasant to work with, and costly to maintain. But this _might_
be the lowest effort.
We do also have prior form for modifying the real code to produce errors, but I
have a preference to avoid this.
FWIW, I don't think any of these approaches are _strictly_ necessary for
merging this patch, since none are trivial undertakings. But one would
certainly be preferable, and some would have wider positive impacts for the
project. They are just suggestions given your question.
was (Author: benedict):
[~molsson] very sorry for taking so long to look at this patch, or to respond.
Thank you for producing the patch - I am looking forward to reviewing it soon.
I'm afraid I still don't quite have time - I hope to get to it in the next
couple of weeks.
The best way I can think of to break a local request, without modifying the
underlying codebase, would be one of two possibilities:
1) Register a java-based FileSystemProvider that we can break, but this would
also require changes to the codebase, to use URIs instead of paths for the
filesystem interactions. This might not be too onerous, or might be really
awful to undertake. But it would offer us really powerful control over testing
of filesystem interactions.
2) Use a pre-existing outside-of-JVM approach. ScyllaDB seem to have written a
[fault-injecting filesystem for
Linux|https://www.scylladb.com/2016/05/02/fault-injection-filesystem-cookbook/].
This would work in CircleCI and on users developing under Linux, which is
better than nothing, and might be lower effort to introduce.
> Local reads potentially blocking remote reads
> -
>
> Key: CASSANDRA-14983
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14983
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Low
> Attachments: graph_local_read.html, graph_local_read_trunk.html,
> local_read_trace.log
>
>
> Since CASSANDRA-4718 there is a fast path allowing local requests to continue
> to [work in the same
> thread|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/reads/AbstractReadExecutor.java#L157]
> rather than being sent over to the read stage.
> Based on the comment
> {code:java}
> // We delay the local (potentially blocking) read till the end to avoid
> stalling remote requests.
> {code}
> it seems like this should be performed last in the chain to avoid blocking
> remote requests but that does not seem to be the case when the local request
> is a data request. The digest request(s) are sent after the data requests are
> sent (and now the transient replica requests as well). When the fast path is
> used for local data/transient data requests this will block the next type of
> request from being sent away until the local read is finished and add
> additional latency to the request.
> In addition to this it seems like local requests are *always* data requests
> (might not be a problem), but the log message can say either ["digest" or
> "data"|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/reads/AbstractReadExecutor.java#L156]
> as the type of request.
> I have tried to run performance measurements to see the impact of this in 3.0
> (by moving local requests to the end of ARE#executeAsync()) but I haven't
> seen
[jira] [Comment Edited] (CASSANDRA-14983) Local reads potentially blocking remote reads
[
https://issues.apache.org/jira/browse/CASSANDRA-14983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809691#comment-16809691
]
Benedict edited comment on CASSANDRA-14983 at 4/4/19 10:14 AM:
---
[~molsson] very sorry for taking so long to look at this patch, or to respond.
Thank you for producing the patch - I am looking forward to reviewing it soon.
I'm afraid I still don't quite have time - I hope to get to it in the next
couple of weeks.
The best ways I can think of to break a local request, without modifying the
underlying codebase, would be one of two possibilities:
1) Register a java-based FileSystemProvider that we can break, but this would
also require changes to the codebase, to use URIs instead of paths for the
filesystem interactions. This might not be too onerous, or might be really
awful to undertake. But it would offer us really powerful control over testing
of filesystem interactions.
2) Use a pre-existing outside-of-JVM approach. ScyllaDB seem to have written a
[fault-injecting filesystem for
Linux|https://www.scylladb.com/2016/05/02/fault-injection-filesystem-cookbook/].
This would work in CircleCI and on users developing under Linux, which is
better than nothing, and might be lower effort to introduce, though I'm not
sure if it would support a delay - and an error might not suffice for this test.
3) Byteman is also an option for a single fault injection, but in my experience
it is really unpleasant to work with, and costly to maintain. But this _might_
be the lowest effort.
4) You could mess with the local node state, copying the sstable thousands or
tens-of-thousands of times (and registering the copies with the node's Tracker
directly, since you can invoke any method you like).
We do also have prior form for modifying the real code to produce errors, but I
have a preference to avoid this.
I think 4 should be the easiest, so that might be the best bet!
was (Author: benedict):
[~molsson] very sorry for taking so long to look at this patch, or to respond.
Thank you for producing the patch - I am looking forward to reviewing it soon.
I'm afraid I still don't quite have time - I hope to get to it in the next
couple of weeks.
The best way I can think of to break a local request, without modifying the
underlying codebase, would be one of two possibilities:
1) Register a java-based FileSystemProvider that we can break, but this would
also require changes to the codebase, to use URIs instead of paths for the
filesystem interactions. This might not be too onerous, or might be really
awful to undertake. But it would offer us really powerful control over testing
of filesystem interactions.
2) Use a pre-existing outside-of-JVM approach. ScyllaDB seem to have written a
[fault-injecting filesystem for
Linux|https://www.scylladb.com/2016/05/02/fault-injection-filesystem-cookbook/].
This would work in CircleCI and on users developing under Linux, which is
better than nothing, and might be lower effort to introduce.
3) Byteman is also an option for a single fault injection, but in my experience
it is really unpleasant to work with, and costly to maintain. But this _might_
be the lowest effort.
We do also have prior form for modifying the real code to produce errors, but I
have a preference to avoid this.
FWIW, I don't think any of these approaches are _strictly_ necessary for
merging this patch, since none are trivial undertakings. But one would
certainly be preferable, and some would have wider positive impacts for the
project. They are just suggestions given your question.
> Local reads potentially blocking remote reads
> -
>
> Key: CASSANDRA-14983
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14983
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Marcus Olsson
>Assignee: Marcus Olsson
>Priority: Low
> Attachments: graph_local_read.html, graph_local_read_trunk.html,
> local_read_trace.log
>
>
> Since CASSANDRA-4718 there is a fast path allowing local requests to continue
> to [work in the same
> thread|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/reads/AbstractReadExecutor.java#L157]
> rather than being sent over to the read stage.
> Based on the comment
> {code:java}
> // We delay the local (potentially blocking) read till the end to avoid
> stalling remote requests.
> {code}
> it seems like this should be performed last in the chain to avoid blocking
> remote requests but that does not seem to be the case when the local request
> is a data request. The digest request(s) are sent after the data requests are
> sent (and now the transient replica requests as well). When the fast pat
[jira] [Assigned] (CASSANDRA-14903) Nodetool cfstats prints index name twice
[
https://issues.apache.org/jira/browse/CASSANDRA-14903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ian Cleasby reassigned CASSANDRA-14903:
---
Assignee: Stefan Liebig (was: Ian Cleasby)
> Nodetool cfstats prints index name twice
>
>
> Key: CASSANDRA-14903
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14903
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Kurt Greaves
>Assignee: Stefan Liebig
>Priority: Low
> Fix For: 4.0
>
>
> {code:java}
> CREATE TABLE test.test (
> id int PRIMARY KEY,
> data text
> );
> CREATE INDEX test_data_idx ON test.test (data);
> ccm node1 nodetool cfstats test
> Total number of tables: 40
>
> Keyspace : test
> Read Count: 0
> Read Latency: NaN ms
> Write Count: 0
> Write Latency: NaN ms
> Pending Flushes: 0
> Table (index): test.test_data_idxtest.test_data_idx
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14903) Nodetool cfstats prints index name twice
[
https://issues.apache.org/jira/browse/CASSANDRA-14903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ian Cleasby reassigned CASSANDRA-14903:
---
Assignee: Ian Cleasby (was: Stefan Liebig)
> Nodetool cfstats prints index name twice
>
>
> Key: CASSANDRA-14903
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14903
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Kurt Greaves
>Assignee: Ian Cleasby
>Priority: Low
> Fix For: 4.0
>
>
> {code:java}
> CREATE TABLE test.test (
> id int PRIMARY KEY,
> data text
> );
> CREATE INDEX test_data_idx ON test.test (data);
> ccm node1 nodetool cfstats test
> Total number of tables: 40
>
> Keyspace : test
> Read Count: 0
> Read Latency: NaN ms
> Write Count: 0
> Write Latency: NaN ms
> Pending Flushes: 0
> Table (index): test.test_data_idxtest.test_data_idx
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15066) Improvements to Internode Messaging
[
https://issues.apache.org/jira/browse/CASSANDRA-15066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16810247#comment-16810247
]
Benedict commented on CASSANDRA-15066:
--
Thanks for the feedback.
I'm sorry we failed to get back to you as promptly as we had promised. We
began review of CASSANDRA-14503 with the expectation it would be a 2-3 week
project, with perhaps some minor improvements and bugfixes identified.
Unfortunately, as we dug in, we came to the conclusion that there were issues
requiring significant work.
For context, as part of our effort toward stability in Cassandra 4.0, we
committed to completing a source audit of the messaging system. Reviewing
14503 and trunk together made sense, given that 14503 included necessary fixes
and constituted significant changes to trunk. We were also under the
impression that you intended to reduce your involvement in the project in your
new role, and didn’t expect this change would allow time for the amount of
collaboration needed.
So, we began preparing a candidate patch to resolve issues that we found. As
we progressed, we found it difficult to produce a small patch in whose
correctness we were confident, so the scope of modifications expanded in order
to produce confidence in the result. As this progressed, it became apparent
that we could also, at little cost, better tailor the system's characteristics
to Cassandra's needs by integrating knowledge we had gained during the wider
review. This also permitted, in our view, an overall risk reduction to the
project by: 1) reducing the need for further major modifications in the next
release; and 2) reducing the maintenance risk through clarified semantics.
Producing work for the community is a complex dance that is impossible to
conduct perfectly. In our view, a patchwork approach wouldn’t have yielded
high confidence in the result, and would have reintroduced long standing
defects to our 4.0 release. We are also, as a project, clearly under
significant time pressure to produce a 4.0 release that we can endorse.
Given the sole purpose and scope of this work is fixing stability issues that
would land in 4.0, we feel that it is in the spirit of the feature freeze and
our collective focus on stability. As a project, we have promised not to
repeat the mistake of rushing out an unsafe .0 release, and we feel this work
is important for achieving that. This is of course all open for debate, and it
may be that through discussion together we revise our proposal for what’s to be
merged, versus held for a future release. We look forward to hearing the
community's thoughts on this, which we hope to solicit from a wider discussion
on the mailing list.
As always, we are eager to receive feedback on our work. Do you have any
specific technical concerns with the approach taken?
> Improvements to Internode Messaging
> ---
>
> Key: CASSANDRA-15066
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15066
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Internode
>Reporter: Benedict
>Assignee: Benedict
>Priority: Normal
> Fix For: 4.0
>
>
> CASSANDRA-8457 introduced asynchronous networking to internode messaging, but
> there have been several follow-up endeavours to improve some semantic issues.
> CASSANDRA-14503 and CASSANDRA-13630 are the latest such efforts, and were
> combined some months ago into a single overarching refactor of the original
> work, to address some of the issues that have been discovered. Given the
> criticality of this work to the project, we wanted to bring some more eyes to
> bear to ensure the release goes ahead smoothly. In doing so, we uncovered a
> number of issues with messaging, some of which long standing, that we felt
> needed to be addressed. This patch widens the scope of CASSANDRA-14503 and
> CASSANDRA-13630 in an effort to close the book on the messaging service, at
> least for the foreseeable future.
> The patch includes a number of clarifying refactors that touch outside of the
> {{net.async}} package, and a number of semantic changes to the {{net.async}}
> packages itself. We believe it clarifies the intent and behaviour of the
> code while improving system stability, which we will outline in comments
> below.
> https://github.com/belliottsmith/cassandra/tree/messaging-improvements
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15077) Dropping column via thrift renders cf unreadable via CQL, leads to missing data
Muir Manders created CASSANDRA-15077:
Summary: Dropping column via thrift renders cf unreadable via CQL,
leads to missing data
Key: CASSANDRA-15077
URL: https://issues.apache.org/jira/browse/CASSANDRA-15077
Project: Cassandra
Issue Type: Bug
Reporter: Muir Manders
Hello
We have a lot of thrift/compact storage column families in production. We
upgraded to 3.11.4 last week. This week we ran a (thrift) schema change to drop
a column from a column family. Our CQL clients immediately starting getting a
read error ("ReadFailure: Error from server: code=1300 ...") trying to read the
column family. Thrift clients were still able to read the column family.
We determined restarting the nodes "fixed" CQL reads, so we did that, but soon
discovered that we were missing data because cassandra was skipping sstables it
didn't like on startup. That exception looked like this:
{noformat}
INFO [main] 2019-04-04 20:06:35,676 ColumnFamilyStore.java:430 - Initializing
test.test
ERROR [SSTableBatchOpen:1] 2019-04-04 20:06:35,689 CassandraDaemon.java:228 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.RuntimeException: Unknown column foo during deserialization
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:326)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:522)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:385)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.io.sstable.format.SSTableReader$3.run(SSTableReader.java:570)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_121]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_121]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_121]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_121]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
[apache-cassandra-3.11.4.jar:3.11.4]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
{noformat}
Below is a list of steps to reproduce the issue. Note that in production our
column families were all created via thrift, but I thought it was simpler to
create them using CQL for the reproduction script.
{code}
ccm create test -v 3.11.4 -n 1
ccm updateconf 'start_rpc: true'
ccm start
sleep 10
ccm node1 cqlsh <
[jira] [Updated] (CASSANDRA-15077) Dropping column via thrift renders cf unreadable via CQL, leads to missing data
[
https://issues.apache.org/jira/browse/CASSANDRA-15077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muir Manders updated CASSANDRA-15077:
-
Description:
Hello
We have a lot of thrift/compact storage column families in production. We
upgraded to 3.11.4 last week. This week we ran a (thrift) schema change to drop
a column from a column family. Our CQL clients immediately starting getting a
read error ("ReadFailure: Error from server: code=1300 ...") trying to read the
column family. Thrift clients were still able to read the column family.
We determined restarting the nodes "fixed" CQL reads, so we did that, but soon
discovered that we were missing data because cassandra was skipping sstables it
didn't like on startup. That exception looked like this:
{noformat}
INFO [main] 2019-04-04 20:06:35,676 ColumnFamilyStore.java:430 - Initializing
test.test
ERROR [SSTableBatchOpen:1] 2019-04-04 20:06:35,689 CassandraDaemon.java:228 -
Exception in thread Thread[SSTableBatchOpen:1,5,main]
java.lang.RuntimeException: Unknown column foo during deserialization
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:326)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:522)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:385)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.io.sstable.format.SSTableReader$3.run(SSTableReader.java:570)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_121]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_121]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_121]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_121]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
[apache-cassandra-3.11.4.jar:3.11.4]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
{noformat}
Below is a list of steps to reproduce the issue. Note that in production our
column families were all created via thrift, but I thought it was simpler to
create them using CQL for the reproduction script.
{code}
ccm create test -v 3.11.4 -n 1
ccm updateconf 'start_rpc: true'
ccm start
sleep 10
ccm node1 cqlsh < Dropping column via thrift renders cf unreadable via CQL, leads to missing
> data
> ---
>
> Key: CASSANDRA-15077
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15077
> Project: Cassandra
> Issue Type: Bug
>Reporter: Muir Manders
>Priority: Normal
>
> Hello
> We have a lot of thrift/compact storage column families in production. We
> upgraded to 3.11.4 last week. This week we ran a (thrift) schema change to
> drop a column from a column family. Our CQL clients immediately starting
> getting a read error ("ReadFailure: Error from server: code=1300 ...") trying
> to read the column family. Thrift clients were still able to read the column
> family.
> We determined restarting the nodes "fixed" CQL reads, so we did that, but
> soon discovered that we were missing data because cassandra was skipping
> sstables it didn't like on startup. That exception looked like this:
> {noformat}
> INFO [main] 2019-04-04 20:06:35,676 ColumnFamilyStore.java:430 -
> Initializing test.test
> ERROR [SSTableBatchOpen:1] 2019-04-04 20:06:35,689 CassandraDaemon.java:228 -
> Exception in thread Thread[SSTableBatchOpen:1,5,main]
> java.lang.RuntimeException: Unknown column foo during deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:326)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:522)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:385)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.io.sstable.format.SSTableReader$3.run(SSTableReader.java:570)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0
[jira] [Updated] (CASSANDRA-14629) Abstract Virtual Table for very large result sets
[ https://issues.apache.org/jira/browse/CASSANDRA-14629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-14629: - Reviewers: Dinesh Joshi > Abstract Virtual Table for very large result sets > - > > Key: CASSANDRA-14629 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14629 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/CQL, Legacy/Observability >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Low > Labels: pull-request-available, virtual-tables > Time Spent: 10m > Remaining Estimate: 0h > > For virtual tables that are very large we cannot use existing > abstractvirtualtable since it would OOM the node possibly. An example would > be a table to view the internal cache contents or to view contents of > sstables. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14903) Nodetool cfstats prints index name twice
[
https://issues.apache.org/jira/browse/CASSANDRA-14903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16810329#comment-16810329
]
mck commented on CASSANDRA-14903:
-
bq. Hang on the output is including the keyspace `test` and table name `test`
with idx of `test_data_idx`.
Yes, it's a bit messy :-(
It's expanding the scope of the ticket, but it would be nice to get the display
text correct in all modes in one commit.
I tested the yml and json output formats and they worked fine.
> Nodetool cfstats prints index name twice
>
>
> Key: CASSANDRA-14903
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14903
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Kurt Greaves
>Assignee: Ian Cleasby
>Priority: Low
> Fix For: 4.0
>
>
> {code:java}
> CREATE TABLE test.test (
> id int PRIMARY KEY,
> data text
> );
> CREATE INDEX test_data_idx ON test.test (data);
> ccm node1 nodetool cfstats test
> Total number of tables: 40
>
> Keyspace : test
> Read Count: 0
> Read Latency: NaN ms
> Write Count: 0
> Write Latency: NaN ms
> Pending Flushes: 0
> Table (index): test.test_data_idxtest.test_data_idx
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15078) Support cross version messaging in in-jvm upgrade dtests
Blake Eggleston created CASSANDRA-15078: --- Summary: Support cross version messaging in in-jvm upgrade dtests Key: CASSANDRA-15078 URL: https://issues.apache.org/jira/browse/CASSANDRA-15078 Project: Cassandra Issue Type: Improvement Reporter: Blake Eggleston -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15078) Support cross version messaging in in-jvm upgrade dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-15078: Component/s: Test/dtest > Support cross version messaging in in-jvm upgrade dtests > > > Key: CASSANDRA-15078 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15078 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Blake Eggleston >Priority: Normal > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15078) Support cross version messaging in in-jvm upgrade dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-15078: Fix Version/s: 4.0 3.11.5 3.0.19 2.2.15 > Support cross version messaging in in-jvm upgrade dtests > > > Key: CASSANDRA-15078 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15078 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Blake Eggleston >Priority: Normal > Fix For: 2.2.15, 3.0.19, 3.11.5, 4.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14903) Nodetool cfstats prints index name twice
[
https://issues.apache.org/jira/browse/CASSANDRA-14903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16810354#comment-16810354
]
Stefan Miklosovic edited comment on CASSANDRA-14903 at 4/4/19 10:38 PM:
Hi [~michaelsembwever] no worries, I ll cover this (I work with [~icleasby]
closely)
was (Author: stefan.miklosovic):
Hi [~michaelsembwever] no worries, I ll cover this.
> Nodetool cfstats prints index name twice
>
>
> Key: CASSANDRA-14903
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14903
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Kurt Greaves
>Assignee: Stefan Miklosovic
>Priority: Low
> Fix For: 4.0
>
>
> {code:java}
> CREATE TABLE test.test (
> id int PRIMARY KEY,
> data text
> );
> CREATE INDEX test_data_idx ON test.test (data);
> ccm node1 nodetool cfstats test
> Total number of tables: 40
>
> Keyspace : test
> Read Count: 0
> Read Latency: NaN ms
> Write Count: 0
> Write Latency: NaN ms
> Pending Flushes: 0
> Table (index): test.test_data_idxtest.test_data_idx
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14903) Nodetool cfstats prints index name twice
[
https://issues.apache.org/jira/browse/CASSANDRA-14903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16810354#comment-16810354
]
Stefan Miklosovic commented on CASSANDRA-14903:
---
Hi [~michaelsembwever] no worries, I ll cover this.
> Nodetool cfstats prints index name twice
>
>
> Key: CASSANDRA-14903
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14903
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Kurt Greaves
>Assignee: Ian Cleasby
>Priority: Low
> Fix For: 4.0
>
>
> {code:java}
> CREATE TABLE test.test (
> id int PRIMARY KEY,
> data text
> );
> CREATE INDEX test_data_idx ON test.test (data);
> ccm node1 nodetool cfstats test
> Total number of tables: 40
>
> Keyspace : test
> Read Count: 0
> Read Latency: NaN ms
> Write Count: 0
> Write Latency: NaN ms
> Pending Flushes: 0
> Table (index): test.test_data_idxtest.test_data_idx
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14903) Nodetool cfstats prints index name twice
[
https://issues.apache.org/jira/browse/CASSANDRA-14903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic reassigned CASSANDRA-14903:
-
Assignee: Stefan Miklosovic (was: Ian Cleasby)
> Nodetool cfstats prints index name twice
>
>
> Key: CASSANDRA-14903
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14903
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Kurt Greaves
>Assignee: Stefan Miklosovic
>Priority: Low
> Fix For: 4.0
>
>
> {code:java}
> CREATE TABLE test.test (
> id int PRIMARY KEY,
> data text
> );
> CREATE INDEX test_data_idx ON test.test (data);
> ccm node1 nodetool cfstats test
> Total number of tables: 40
>
> Keyspace : test
> Read Count: 0
> Read Latency: NaN ms
> Write Count: 0
> Write Latency: NaN ms
> Pending Flushes: 0
> Table (index): test.test_data_idxtest.test_data_idx
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15078) Support cross version messaging in in-jvm upgrade dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston reassigned CASSANDRA-15078: --- Assignee: Blake Eggleston > Support cross version messaging in in-jvm upgrade dtests > > > Key: CASSANDRA-15078 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15078 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 2.2.15, 3.0.19, 3.11.5, 4.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15072) Incomplete range results during 2.X -> 3.11.4 upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Blake Eggleston updated CASSANDRA-15072:
Severity: Normal
Complexity: Normal
Component/s: Legacy/Coordination
Status: Open (was: Triage Needed)
> Incomplete range results during 2.X -> 3.11.4 upgrade
> -
>
> Key: CASSANDRA-15072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15072
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination
>Reporter: Muir Manders
>Assignee: Blake Eggleston
>Priority: High
> Attachments: eriksw-repro.sh
>
>
> Hello
> During an upgrade from 2.1.17 to 3.11.4, our application starting getting
> back incomplete results for range queries. When all nodes were upgraded
> (before upgrading sstables), we stopped getting incomplete results. I was
> able to reproduce it and listed steps below. It seems to require the random
> partitioner and compact storage to reproduce reliably. It also reproduces
> coming from 2.1.21 and 2.2.14. You seem to get the bad behavior when an old
> node is your coordinator and it has to talk to an upgraded replica.
> {noformat}
> ccm create test -v 2.1.17 -n 3
> ccm updateconf 'partitioner: org.apache.cassandra.dht.RandomPartitioner'
> ccm node1 updateconf 'initial_token: 0'
> ccm node2 updateconf 'initial_token: 56713727820156410577229101238628035242'
> ccm node3 updateconf 'initial_token: 113427455640312821154458202477256070484'
> ccm start
> ccm node1 cqlsh < CREATE KEYSPACE test WITH REPLICATION = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> CREATE COLUMNFAMILY test.test (
> id text,
> foo text,
> bar text,
> PRIMARY KEY (id)
> ) WITH COMPACT STORAGE;
> CONSISTENCY QUORUM;
> INSERT INTO test.test (id, foo, bar) values ('1', 'hi', 'there');
> INSERT INTO test.test (id, foo, bar) values ('2', 'hi', 'there');
> SCHEMA
> ccm node1 stop
> ccm node1 setdir -v 3.11.4
> ccm node1 start
> ccm node2 stop
> ccm node2 setdir -v 3.11.4
> ccm node2 start
> # here I use 3.X cqlsh to connect to 2.X node so I can lower the page size (to
> # allow for simpler test setup)
> cqlsh 127.0.0.3 < CONSISTENCY QUORUM;
> PAGING 2;
> select * from test.test;
> QUERY
> {noformat}
> This results in:
> {noformat}
> Page size: 2
> id | bar | foo
> +---+-
> 2 | there | hi
> (1 rows)
> {noformat}
> Running it against the upgraded node (node1):
> {noformat}
> Page size: 2
> id | bar | foo
> +---+-
> 2 | there | hi
> 1 | there | hi
> (2 rows)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15078) Support cross version messaging in in-jvm upgrade dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-15078: Complexity: Normal Change Category: Quality Assurance Status: Open (was: Triage Needed) > Support cross version messaging in in-jvm upgrade dtests > > > Key: CASSANDRA-15078 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15078 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 2.2.15, 3.0.19, 3.11.5, 4.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15078) Support cross version messaging in in-jvm upgrade dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Blake Eggleston updated CASSANDRA-15078:
Test and Documentation Plan: circle ci once available
Status: Patch Available (was: Open)
|[2.2|https://github.com/bdeggleston/cassandra/tree/15078-2.2]|
|[3.0|https://github.com/bdeggleston/cassandra/tree/15078-3.0]|
|[3.11|https://github.com/bdeggleston/cassandra/tree/15078-3.11]|
|[trunk|https://github.com/bdeggleston/cassandra/tree/15078-trunk]|
[~ifesdjeen] / [~benedict] would one of you mind taking a look at this? It's
just adding and using hooks for getting/setting messaging versions. It also
fixes a minor issue where the {{setup}} method wasn't getting called. These
changes are in support of a test case for CASSANDRA-15072, feel free to take a
look and give feedback on that as well. Circle is behind picking up new
branches, but I'll update with the workflow links once they're available.
> Support cross version messaging in in-jvm upgrade dtests
>
>
> Key: CASSANDRA-15078
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15078
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 2.2.15, 3.0.19, 3.11.5, 4.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15072) Incomplete range results during 2.X -> 3.11.4 upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Blake Eggleston updated CASSANDRA-15072:
Test and Documentation Plan: circleci / in jvm upgrade dtests
Status: Patch Available (was: Open)
|[3.0|https://github.com/bdeggleston/cassandra/tree/15072-3.0]|
|[3.11|https://github.com/bdeggleston/cassandra/tree/15072-3.11]|
[~beobal] do you have time to review this? It seems to be related to
CASSANDRA-11087.
A few notes:
* From what I can tell, returning a row per cell is the right thing to do in
this case, so I'm using a modified result counter to only going entire
partitions in this specific case. However, I'm not familiar enough with all the
dark corners of the 2.1 storage engine to be sure that's appropriate, or won't
break something else.
* Doing a point read with the partition key also returns a row per cell, but
works correctly because the 2.2 coordinator seems to just discard the limit in
that case.
* If you're not familiar with the in-jvm dtests yet, and want to run the one
in this patch, you'll want to run {{ant dtest-jar}} on this branch and [this
2.2 branch|https://github.com/bdeggleston/cassandra/tree/15078-2.2], and put
the 2.2 dtest jar in the 3.0 build directory.
* CircleCI seems to be behind picking up new branches to test, but I'll update
this with links to the workflows once it catches up.
> Incomplete range results during 2.X -> 3.11.4 upgrade
> -
>
> Key: CASSANDRA-15072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15072
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination
>Reporter: Muir Manders
>Assignee: Blake Eggleston
>Priority: High
> Attachments: eriksw-repro.sh
>
>
> Hello
> During an upgrade from 2.1.17 to 3.11.4, our application starting getting
> back incomplete results for range queries. When all nodes were upgraded
> (before upgrading sstables), we stopped getting incomplete results. I was
> able to reproduce it and listed steps below. It seems to require the random
> partitioner and compact storage to reproduce reliably. It also reproduces
> coming from 2.1.21 and 2.2.14. You seem to get the bad behavior when an old
> node is your coordinator and it has to talk to an upgraded replica.
> {noformat}
> ccm create test -v 2.1.17 -n 3
> ccm updateconf 'partitioner: org.apache.cassandra.dht.RandomPartitioner'
> ccm node1 updateconf 'initial_token: 0'
> ccm node2 updateconf 'initial_token: 56713727820156410577229101238628035242'
> ccm node3 updateconf 'initial_token: 113427455640312821154458202477256070484'
> ccm start
> ccm node1 cqlsh < CREATE KEYSPACE test WITH REPLICATION = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> CREATE COLUMNFAMILY test.test (
> id text,
> foo text,
> bar text,
> PRIMARY KEY (id)
> ) WITH COMPACT STORAGE;
> CONSISTENCY QUORUM;
> INSERT INTO test.test (id, foo, bar) values ('1', 'hi', 'there');
> INSERT INTO test.test (id, foo, bar) values ('2', 'hi', 'there');
> SCHEMA
> ccm node1 stop
> ccm node1 setdir -v 3.11.4
> ccm node1 start
> ccm node2 stop
> ccm node2 setdir -v 3.11.4
> ccm node2 start
> # here I use 3.X cqlsh to connect to 2.X node so I can lower the page size (to
> # allow for simpler test setup)
> cqlsh 127.0.0.3 < CONSISTENCY QUORUM;
> PAGING 2;
> select * from test.test;
> QUERY
> {noformat}
> This results in:
> {noformat}
> Page size: 2
> id | bar | foo
> +---+-
> 2 | there | hi
> (1 rows)
> {noformat}
> Running it against the upgraded node (node1):
> {noformat}
> Page size: 2
> id | bar | foo
> +---+-
> 2 | there | hi
> 1 | there | hi
> (2 rows)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
