[jira] [Assigned] (CASSANDRA-15158) Wait for schema agreement rather then in flight schema requests when bootstrapping
[ https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cleasby reassigned CASSANDRA-15158: --- Assignee: (was: Ian Cleasby) > Wait for schema agreement rather then in flight schema requests when > bootstrapping > -- > > Key: CASSANDRA-15158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15158 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Vincent White >Priority: Normal > > Currently when a node is bootstrapping we use a set of latches > (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of > in-flight schema pull requests, and we don't proceed with > bootstrapping/stream until all the latches are released (or we timeout > waiting for each one). One issue with this is that if we have a large schema, > or the retrieval of the schema from the other nodes was unexpectedly slow > then we have no explicit check in place to ensure we have actually received a > schema before we proceed. > While it's possible to increase "migration_task_wait_in_seconds" to force the > node to wait on each latche longer, there are cases where this doesn't help > because the callbacks for the schema pull requests have expired off the > messaging service's callback map > (org.apache.cassandra.net.MessagingService#callbacks) after > request_timeout_in_ms (default 10 seconds) before the other nodes were able > to respond to the new node. > This patch checks for schema agreement between the bootstrapping node and the > rest of the live nodes before proceeding with bootstrapping. It also adds a > check to prevent the new node from flooding existing nodes with simultaneous > schema pull requests as can happen in large clusters. > Removing the latch system should also prevent new nodes in large clusters > getting stuck for extended amounts of time as they wait > `migration_task_wait_in_seconds` on each of the latches left orphaned by the > timed out callbacks. > > ||3.11|| > |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]| > |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]| > -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Issue Comment Deleted] (CASSANDRA-15158) Wait for schema agreement rather then in flight schema requests when bootstrapping
[ https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ian Cleasby updated CASSANDRA-15158: Comment: was deleted (was: HI [~michaelsembwever] just hoping to get a bit of love for this ticket. Patches can be found in the description above initially done by Vince. Any feedback would be great.) > Wait for schema agreement rather then in flight schema requests when > bootstrapping > -- > > Key: CASSANDRA-15158 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15158 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Vincent White >Assignee: Ian Cleasby >Priority: Normal > > Currently when a node is bootstrapping we use a set of latches > (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of > in-flight schema pull requests, and we don't proceed with > bootstrapping/stream until all the latches are released (or we timeout > waiting for each one). One issue with this is that if we have a large schema, > or the retrieval of the schema from the other nodes was unexpectedly slow > then we have no explicit check in place to ensure we have actually received a > schema before we proceed. > While it's possible to increase "migration_task_wait_in_seconds" to force the > node to wait on each latche longer, there are cases where this doesn't help > because the callbacks for the schema pull requests have expired off the > messaging service's callback map > (org.apache.cassandra.net.MessagingService#callbacks) after > request_timeout_in_ms (default 10 seconds) before the other nodes were able > to respond to the new node. > This patch checks for schema agreement between the bootstrapping node and the > rest of the live nodes before proceeding with bootstrapping. It also adds a > check to prevent the new node from flooding existing nodes with simultaneous > schema pull requests as can happen in large clusters. > Removing the latch system should also prevent new nodes in large clusters > getting stuck for extended amounts of time as they wait > `migration_task_wait_in_seconds` on each of the latches left orphaned by the > timed out callbacks. > > ||3.11|| > |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]| > |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]| > -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15235) Have Verifier check whether cells are out-of-order
[
https://issues.apache.org/jira/browse/CASSANDRA-15235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giacomo Lo Giusto updated CASSANDRA-15235:
--
Description:
Hello,
This change was tested only for version {{*2.2.13*}}.
We noticed the {{nodetool verify -e}} command was not able to detect corrupt
{{SSTables}} that exhibited out-of-order cells within a row.
This is in contrast to the {{nodetool scrub}} command, which was able to detect
and scrub such corrupted data files.
The proposed changes (see attached patch) include:
* Reusing Scrub's {{OrderCheckerIterator}} in the Verifier (for its _extended_
use).
* Some added logging to better debug what was the cause of the verification
failure and which key first showed the issue.
* Added unit tests for the Verifier ({{VerifyTest.java}}).
(Some other unrelated test where sometimes failing on our end and were
therefore changed to enhance their deterministic behavior).
Please let me know if the change has value and is correct and safe for all
possible configurations. Should we introduce an extra flag to enable the extra
cell ordering check?
In the {{Verifier}} code there was this line (n. 189) that seemed to suggest
that the newly introduced check was in fact an intended behavior all along,
although we could not replicate this behavior neither in unit test nor with our
production data:
{code:java}
//mimic the scrub read path
new SSTableIdentityIterator(sstable, dataFile, key, true);
{code}
Thanks in advance for your feedback and consideration,
Giacomo
was:
Hello,
This change was tested only for version {{*2.2.13*}}.
We noticed the {{nodetool verify -e}} command was not able to detect corrupt
{{SSTables}} that exhibited out-of-order cells within a row.
This is in contrast to the {{nodetool scrub}} command, which was able to
detect and scrub such corrupted data files.
The proposed changes (see attached patch) include:
* Reusing Scrub's {{OrderCheckerIterator}} in the Verifier (for its _extended_
use).
* Some added logging to better debug what was the cause of the verification
failure and which key first showed the issue.
* Added unit tests for the Verifier ({{VerifyTest.java}}).
(Some other unrelated test where sometimes failing on our end and where
therefore changed to enhance their deterministic behavior).
Please let me know if the change has value and is correct and safe for all
possible configurations. Should we introduce an extra flag to enable the extra
cell ordering check?
In the {{Verifier}} code there was this line (n. 189) that seemed to suggest
that the newly introduced check was in fact an intended behavior all along,
although we could not replicate this behavior neither in unit test nor with our
production data:
{code:java}
//mimic the scrub read path
new SSTableIdentityIterator(sstable, dataFile, key, true);
{code}
Thanks in advance for your feedback and consideration,
Giacomo
> Have Verifier check whether cells are out-of-order
> --
>
> Key: CASSANDRA-15235
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15235
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Giacomo Lo Giusto
>Priority: Normal
> Attachments: verifier.patch
>
>
> Hello,
> This change was tested only for version {{*2.2.13*}}.
> We noticed the {{nodetool verify -e}} command was not able to detect corrupt
> {{SSTables}} that exhibited out-of-order cells within a row.
> This is in contrast to the {{nodetool scrub}} command, which was able to
> detect and scrub such corrupted data files.
> The proposed changes (see attached patch) include:
> * Reusing Scrub's {{OrderCheckerIterator}} in the Verifier (for its
> _extended_ use).
> * Some added logging to better debug what was the cause of the verification
> failure and which key first showed the issue.
> * Added unit tests for the Verifier ({{VerifyTest.java}}).
> (Some other unrelated test where sometimes failing on our end and were
> therefore changed to enhance their deterministic behavior).
> Please let me know if the change has value and is correct and safe for all
> possible configurations. Should we introduce an extra flag to enable the
> extra cell ordering check?
> In the {{Verifier}} code there was this line (n. 189) that seemed to suggest
> that the newly introduced check was in fact an intended behavior all along,
> although we could not replicate this behavior neither in unit test nor with
> our production data:
> {code:java}
> //mimic the scrub read path
> new SSTableIdentityIterator(sstable, dataFile, key, true);
> {code}
> Thanks in advance for your feedback and consideration,
> Giacomo
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail:
[jira] [Commented] (CASSANDRA-8557) Default install from tarball is a bit puzzling
[
https://issues.apache.org/jira/browse/CASSANDRA-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887371#comment-16887371
]
Tanaka commented on CASSANDRA-8557:
---
[~crertel] Hi, I'm relatively new to Apache Cassandra and thus feel as though
this would be a great task for me to work through and document. Would you mind
assigning this task to me?
> Default install from tarball is a bit puzzling
> --
>
> Key: CASSANDRA-8557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8557
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools, Local/Config
> Environment: Tested with Crunchbang Waldorf built on Debian 7 Wheezy.
> Java version:
> {noformat}
> $ java -version
> java version "1.8.0_25"
> Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)
> {noformat}
> Python version:
> {noformat}
> $ python --version
> Python 2.7.3
> {noformat}
> I suspect this applies to all *nix tarball installs.
>Reporter: Chris E
>Priority: Low
> Labels: documentation, easyfix
>
> The default tarball installation of Apache Cassandra is kind of confusing for
> a newcomer.
> There are several different points of confusion:
> 1. The default installation doesn't allow remote connections.
> 2. The config file (cassandra.yaml) specifies the use of the
> AllowAllAuthenticator. This is almost immmediately replaced during setup by
> the PasswordAuthenticator. Why not start there?
> 3. The config file (cassandra.yaml) specifies the use of the
> AllowAllAuthorizer. This is almost immediately replaced during setup by the
> CassandraAuthorizer. Why not start there?
> 4. Why does cassandra-cli exist? It even tells the user "This is being
> deprecated." It's confusing figuring out whether to use cqlsh or
> cassandra-cli.
> 5. Running the cassandra script creates a background job that keeps
> running--if you control-c the script, the process continues running.
> 6. The config file (cassandra.yaml) has rpc_interface and rpc_address, and
> the docs there don't spell out that that address is *also* required for using
> remote logins from cqlsh.
> 7. On a freshly-created VM, the localhost flag to rpc_address doesn't appear
> to work (though this may be due to a misconfiguration of the VM). It seems to
> need the assigned IP address of the VM to get it accepting external
> connections.
> 8. The config file (cassandra.yaml) has the request_scheduler using
> NoScheduler--which is fine, but the docs are unclear as to whether or not
> this means that client requests won't be scheduled (at all).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13749) add documentation about upgrade process to docs
[ https://issues.apache.org/jira/browse/CASSANDRA-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887369#comment-16887369 ] Tanaka commented on CASSANDRA-13749: [~sumanth.pasupuleti] No worries, best of luck! > add documentation about upgrade process to docs > --- > > Key: CASSANDRA-13749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13749 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Documentation and Website >Reporter: Jon Haddad >Assignee: Sumanth Pasupuleti >Priority: Normal > Labels: documentation > > The docs don't have any information on how to upgrade. This question gets > asked constantly on the mailing list. > Seems like it belongs under the "Operating Cassandra" section. > https://cassandra.apache.org/doc/latest/operating/index.html -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15227) Remove StageManager
[ https://issues.apache.org/jira/browse/CASSANDRA-15227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Venkata Harikrishna Nukala reassigned CASSANDRA-15227: -- Assignee: Venkata Harikrishna Nukala > Remove StageManager > --- > > Key: CASSANDRA-15227 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15227 > Project: Cassandra > Issue Type: Task > Components: Local/Other >Reporter: Benedict >Assignee: Venkata Harikrishna Nukala >Priority: Normal > Fix For: 4.0 > > > his is a minor cleanup; this class should not exist, but should be embedded > in the Stage enum. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13749) add documentation about upgrade process to docs
[ https://issues.apache.org/jira/browse/CASSANDRA-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887302#comment-16887302 ] Sumanth Pasupuleti commented on CASSANDRA-13749: [~TanakaRM] this has been on my back burner, but I have been making steady progress on this. I should be having a patch up in a couple of weeks time. > add documentation about upgrade process to docs > --- > > Key: CASSANDRA-13749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13749 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Documentation and Website >Reporter: Jon Haddad >Assignee: Sumanth Pasupuleti >Priority: Normal > Labels: documentation > > The docs don't have any information on how to upgrade. This question gets > asked constantly on the mailing list. > Seems like it belongs under the "Operating Cassandra" section. > https://cassandra.apache.org/doc/latest/operating/index.html -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15236) date, smallint, time, and tinyint types should be fixed length
[
https://issues.apache.org/jira/browse/CASSANDRA-15236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15236:
--
Severity: Low
Complexity: Normal
Discovered By: Code Inspection
Bug Category: Parent values: Degradation(12984)Level 1 values: Performance
Bug/Regression(12997)
Status: Open (was: Triage Needed)
> date, smallint, time, and tinyint types should be fixed length
> --
>
> Key: CASSANDRA-15236
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15236
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Aleksey Yeschenko
>Priority: Normal
>
> When adding the 4 types above ({{date}}, {{smallint}}, {{time}}, and
> {{tinyint}}) we forgot to override their {{valueLengthIfFixed()}} method. As
> a result of this omission, all 4 types now require an extra byte when
> serialized - to encode length - even {{ByteType}} ({{tinyint}}).
> In the next major (4.0 or 5.0?) we should fix this inefficiency.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15236) date, smallint, time, and tinyint types should be fixed length
Aleksey Yeschenko created CASSANDRA-15236:
-
Summary: date, smallint, time, and tinyint types should be fixed
length
Key: CASSANDRA-15236
URL: https://issues.apache.org/jira/browse/CASSANDRA-15236
Project: Cassandra
Issue Type: Bug
Components: Local/SSTable
Reporter: Aleksey Yeschenko
When adding the 4 types above ({{date}}, {{smallint}}, {{time}}, and
{{tinyint}}) we forgot to override their {{valueLengthIfFixed()}} method. As a
result of this omission, all 4 types now require an extra byte when serialized
- to encode length - even {{ByteType}} ({{tinyint}}).
In the next major (4.0 or 5.0?) we should fix this inefficiency.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15223) OutboundTcpConnection leaks direct memory
[
https://issues.apache.org/jira/browse/CASSANDRA-15223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887287#comment-16887287
]
Venkata Harikrishna Nukala commented on CASSANDRA-15223:
Made changes for 3.0 and 3.11.
3.0 changes:
[https://github.com/nvharikrishna/cassandra/tree/15223-cassandra-3.0]
Circle ci:
[https://circleci.com/gh/nvharikrishna/cassandra/tree/15223-cassandra-3%2E0]
3.11 changes:
[https://github.com/nvharikrishna/cassandra/tree/15223-cassandra-3.11]
Circle ci:
[https://circleci.com/gh/nvharikrishna/cassandra/tree/15223-cassandra-3%2E11]
Closing the {{out}} closes the output stream passed to it
(socket.getOutputStreamI()). Closing socket's output stream closes the socket
too . So just closing {{out}}.
> OutboundTcpConnection leaks direct memory
> -
>
> Key: CASSANDRA-15223
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15223
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Benedict
>Assignee: Venkata Harikrishna Nukala
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> On disconnect we set {{out}} to null without first closing it
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13749) add documentation about upgrade process to docs
[ https://issues.apache.org/jira/browse/CASSANDRA-13749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887245#comment-16887245 ] Tanaka commented on CASSANDRA-13749: [~rustyrazorblade] Hi, is this ticket up for grabs? If so, I would like to work on the documentation for this! > add documentation about upgrade process to docs > --- > > Key: CASSANDRA-13749 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13749 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Documentation and Website >Reporter: Jon Haddad >Assignee: Sumanth Pasupuleti >Priority: Normal > Labels: documentation > > The docs don't have any information on how to upgrade. This question gets > asked constantly on the mailing list. > Seems like it belongs under the "Operating Cassandra" section. > https://cassandra.apache.org/doc/latest/operating/index.html -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15235) Have Verifier check whether cells are out-of-order
Giacomo Lo Giusto created CASSANDRA-15235:
-
Summary: Have Verifier check whether cells are out-of-order
Key: CASSANDRA-15235
URL: https://issues.apache.org/jira/browse/CASSANDRA-15235
Project: Cassandra
Issue Type: Improvement
Reporter: Giacomo Lo Giusto
Attachments: verifier.patch
Hello,
This change was tested only for version {{*2.2.13*}}.
We noticed the {{nodetool verify -e}} command was not able to detect corrupt
{{SSTables}} that exhibited out-of-order cells within a row.
This is in contrast to the {{nodetool scrub}} command, which was able to
detect and scrub such corrupted data files.
The proposed changes (see attached patch) include:
* Reusing Scrub's {{OrderCheckerIterator}} in the Verifier (for its _extended_
use).
* Some added logging to better debug what was the cause of the verification
failure and which key first showed the issue.
* Added unit tests for the Verifier ({{VerifyTest.java}}).
(Some other unrelated test where sometimes failing on our end and where
therefore changed to enhance their deterministic behavior).
Please let me know if the change has value and is correct and safe for all
possible configurations. Should we introduce an extra flag to enable the extra
cell ordering check?
In the {{Verifier}} code there was this line (n. 189) that seemed to suggest
that the newly introduced check was in fact an intended behavior all along,
although we could not replicate this behavior neither in unit test nor with our
production data:
{code:java}
//mimic the scrub read path
new SSTableIdentityIterator(sstable, dataFile, key, true);
{code}
Thanks in advance for your feedback and consideration,
Giacomo
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15097) Avoid updating unchanged gossip state
[ https://issues.apache.org/jira/browse/CASSANDRA-15097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-15097: Status: Ready to Commit (was: Review In Progress) +1 > Avoid updating unchanged gossip state > - > > Key: CASSANDRA-15097 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15097 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Jay Zhuang >Assignee: Jay Zhuang >Priority: Normal > > The node might get unchanged gossip states, the state might be just updated > after sending a GOSSIP_SYN, then it will get the state that is already up to > date. If the heartbeat in the GOSSIP_ACK message is updated, it will > unnecessary re-apply the same state again, which could be costly like > updating token change. > It's very likely to happen for large cluster when a node startup, as the > first gossip message will sync all endpoints tokens, it could take some time > (in our case about 200 seconds), during that time, it keeps gossip with other > node and get the full token states. Which causes lots of pending gossip tasks. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886938#comment-16886938
]
Benedict commented on CASSANDRA-15234:
--
Definitely.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
>Reporter: Benedict
>Priority: Normal
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886937#comment-16886937
]
Aleksey Yeschenko commented on CASSANDRA-15234:
---
Would be nice, while at it, to pull all JVM args into one class, a-la
{{Config}}, with strongly typed accessors and proper documentation for each arg.
Right now you have to look for usage of {{System.getProperty()}},
{{Boolean.getBoolean()}}, etc., to enumerate them all, which is a mess.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
>Reporter: Benedict
>Priority: Normal
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14722) New monotonic reads impl introduces risk during range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-14722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-14722: - Resolution: Duplicate Status: Resolved (was: Open) > New monotonic reads impl introduces risk during range movements > --- > > Key: CASSANDRA-14722 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14722 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Coordination >Reporter: Benedict >Priority: Urgent > Fix For: 4.0 > > > Previously on digest mismatch we upgraded to CL.ALL, which harmed > availability. However, because we do not have consistent range movements, > when they are underway it is possible for both the coordinator of a query and > one of its targets have stale ring information. > In this case, we would previously have detected a digest mismatch and ensured > ALL nodes responded before answering, so that any single stale node would > have been detected. In the new implementation, we try to get answers *only* > from the nodes whose digests mismatched, so if the node were stale, it can > respond with a stale data request also, reducing effective CL by one. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14740) BlockingReadRepair does not maintain monotonicity during range movements
[ https://issues.apache.org/jira/browse/CASSANDRA-14740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-14740: - Complexity: Normal Discovered By: Code Inspection Bug Category: Parent values: Correctness(12982)Level 1 values: Response Corruption / Loss(12987) > BlockingReadRepair does not maintain monotonicity during range movements > > > Key: CASSANDRA-14740 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14740 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Coordination >Reporter: Benedict >Priority: Urgent > Labels: correctness > Fix For: 4.0 > > > The BlockingReadRepair code introduced by CASSANDRA-10726 requires that each > of the queried nodes are written to, but pending nodes are not considered. > If there is a pending range movement, one of these writes may be ‘lost’ when > the range movement completes. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14846) Drop/Add Column Pre-existing Data Inconsistency
[ https://issues.apache.org/jira/browse/CASSANDRA-14846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886770#comment-16886770 ] Benedict edited comment on CASSANDRA-14846 at 7/17/19 8:15 AM: --- So, the real bug here is that we don't do this reliably, as we do apply a post-filter that _attempts_ to do this. There are race conditions with incompatible types in multiple locations, including compaction, internode messaging and client query execution. By fixing these inconsistencies, we should be able to remove the need for the type of any new column to be compatible with the type of any previous incarnation of the column. was (Author: benedict): So, the real bug here is that we don't do this reliably. There are race conditions with incompatible types in multiple locations, including compaction, internode messaging and client query execution. By fixing these inconsistencies, we should be able to remove the need for the type of any new column to be compatible with the type of any previous incarnation of the column. > Drop/Add Column Pre-existing Data Inconsistency > --- > > Key: CASSANDRA-14846 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14846 > Project: Cassandra > Issue Type: Bug > Components: CQL/Semantics, Local/SSTable >Reporter: Benedict >Priority: Normal > > If we drop a column, any data that is compacted before we add a column with > the same name (and compatible type) will be lost, but any data that was not > compacted will continue to be returned. This seems problematic - surely we > should consider all data prior to a Drop to be lost? Re-adding a column of > the same name should reset the column’s contents? > This would also permit us to not worry about the types being consistent in > the case of a dropped column (only alter column need worry about this) -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14846) Drop/Add Column Pre-existing Data Inconsistency
[ https://issues.apache.org/jira/browse/CASSANDRA-14846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886770#comment-16886770 ] Benedict commented on CASSANDRA-14846: -- So, the real bug here is that we don't do this reliably. There are race conditions with incompatible types in multiple locations, including compaction, internode messaging and client query execution. By fixing these inconsistencies, we should be able to remove the need for the type of any new column to be compatible with the type of any previous incarnation of the column. > Drop/Add Column Pre-existing Data Inconsistency > --- > > Key: CASSANDRA-14846 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14846 > Project: Cassandra > Issue Type: Bug > Components: CQL/Semantics, Local/SSTable >Reporter: Benedict >Priority: Normal > > If we drop a column, any data that is compacted before we add a column with > the same name (and compatible type) will be lost, but any data that was not > compacted will continue to be returned. This seems problematic - surely we > should consider all data prior to a Drop to be lost? Re-adding a column of > the same name should reset the column’s contents? > This would also permit us to not worry about the types being consistent in > the case of a dropped column (only alter column need worry about this) -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15234) Standardise config and JVM parameters
Benedict created CASSANDRA-15234:
Summary: Standardise config and JVM parameters
Key: CASSANDRA-15234
URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
Project: Cassandra
Issue Type: Bug
Reporter: Benedict
We have a bunch of inconsistent names and config patterns in the codebase, both
from the yams and JVM properties. It would be nice to standardise the naming
(such as otc_ vs internode_) as well as the provision of values with units -
while maintaining perpetual backwards compatibility with the old parameter
names, of course.
For temporal units, I would propose parsing strings with suffixes of:
{{code}}
u|micros(econds?)?
s(econds?)?
m(inutes?)?
h(ours?)?
d(ays?)?
mo(nths?)?
{{code}}
For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
MiB/s, GiB/s, TiB/s}}.
Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or powers
of 1000 such as {{KB/s}}, given these are regularly used for either their old
or new definition e.g. {{KiB/s}}, or we could support them and simply log the
value in bytes/s.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15233) Internode Messaging: Decouple large thresholds
[ https://issues.apache.org/jira/browse/CASSANDRA-15233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15233: - Fix Version/s: 4.x > Internode Messaging: Decouple large thresholds > -- > > Key: CASSANDRA-15233 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15233 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode >Reporter: Benedict >Priority: Normal > Fix For: 4.x > > > Internode messaging has a large message threshold, that conflates a number of > related switches: processing the serialisation on a separate thread, sending > the message on a separate connection for QoS, and the maximum number of bytes > to send in a single frame. These are all independent properties, that should > be untangled. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15232: - Fix Version/s: 4.0 > Arithmetic operators over decimal truncate results > -- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Benedict >Priority: Normal > Fix For: 4.0 > > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15233) Internode Messaging: Decouple large thresholds
[ https://issues.apache.org/jira/browse/CASSANDRA-15233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15233: - Complexity: Normal Change Category: Semantic Component/s: Messaging/Internode Status: Open (was: Triage Needed) > Internode Messaging: Decouple large thresholds > -- > > Key: CASSANDRA-15233 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15233 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode >Reporter: Benedict >Priority: Normal > > Internode messaging has a large message threshold, that conflates a number of > related switches: processing the serialisation on a separate thread, sending > the message on a separate connection for QoS, and the maximum number of bytes > to send in a single frame. These are all independent properties, that should > be untangled. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15233) Internode Messaging: Decouple large thresholds
[ https://issues.apache.org/jira/browse/CASSANDRA-15233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15233: - Workflow: Cassandra Default Workflow (was: Cassandra Bug Workflow) Issue Type: Improvement (was: Bug) > Internode Messaging: Decouple large thresholds > -- > > Key: CASSANDRA-15233 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15233 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Priority: Normal > > Internode messaging has a large message threshold, that conflates a number of > related switches: processing the serialisation on a separate thread, sending > the message on a separate connection for QoS, and the maximum number of bytes > to send in a single frame. These are all independent properties, that should > be untangled. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15232: - Complexity: Low Hanging Fruit Change Category: Semantic Component/s: CQL/Semantics Status: Open (was: Triage Needed) > Arithmetic operators over decimal truncate results > -- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Benedict >Priority: Normal > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-15232: - Workflow: Cassandra Default Workflow (was: Cassandra Bug Workflow) Issue Type: Improvement (was: Bug) > Arithmetic operators over decimal truncate results > -- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Priority: Normal > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15233) Internode Messaging: Decouple large thresholds
Benedict created CASSANDRA-15233: Summary: Internode Messaging: Decouple large thresholds Key: CASSANDRA-15233 URL: https://issues.apache.org/jira/browse/CASSANDRA-15233 Project: Cassandra Issue Type: Bug Reporter: Benedict Internode messaging has a large message threshold, that conflates a number of related switches: processing the serialisation on a separate thread, sending the message on a separate connection for QoS, and the maximum number of bytes to send in a single frame. These are all independent properties, that should be untangled. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
[ https://issues.apache.org/jira/browse/CASSANDRA-15232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886740#comment-16886740 ] Benedict commented on CASSANDRA-15232: -- I looked into this a while ago, and I believe Postgres uses the maximum precision of the two operands, which seems like a reasonable approach. > Arithmetic operators over decimal truncate results > -- > > Key: CASSANDRA-15232 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Priority: Normal > > The decimal operators hard-code a 128 bit precision for their computations. > Probably a precision needs to be configured or decided somehow, but it’s not > clear why 128bit was chosen. Particularly for multiplication and addition, > it’s very unclear why we truncate, which is different to our behaviour for > e.g. sum() aggregates. Probably for division we should also ensure that we > do not reduce the precision of the two operands. A minimum of decimal128 > seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15232) Arithmetic operators over decimal truncate results
Benedict created CASSANDRA-15232: Summary: Arithmetic operators over decimal truncate results Key: CASSANDRA-15232 URL: https://issues.apache.org/jira/browse/CASSANDRA-15232 Project: Cassandra Issue Type: Bug Reporter: Benedict The decimal operators hard-code a 128 bit precision for their computations. Probably a precision needs to be configured or decided somehow, but it’s not clear why 128bit was chosen. Particularly for multiplication and addition, it’s very unclear why we truncate, which is different to our behaviour for e.g. sum() aggregates. Probably for division we should also ensure that we do not reduce the precision of the two operands. A minimum of decimal128 seems reasonable, but a maximum does not. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
